Buckets of trouble by CyberWire

Jason Lamar, Cobalt’s Senior Vice President of Product recently joined David Bittner on the CyberWire Daily Podcast to share insights on pentesting, the growing need for offensive security and how companies can stay ahead of cyber threats.

Click the arrow above to listen and check out Cobalt’s listing in the GigaOm Radar Report for PTaaS 2024 to learn more.

CyberWire Episode Transcript

Dave Bittner: In our industry Voices segment, Jason Lamar, Cobalt's Senior Vice President of Product, joins us to share insights on Offensive Security and staying ahead of cyber threats.

Jason Lamar: It really matters. This whole area of Offensive Security matters. Why? Because being proactive is really critical to keeping your business safe from new and evolving cyberattacks. We have penetration testing, and that is changing more and more frequently as as part of the software development life cycle (SDLC). Customers increasingly are adopting a Pentesting as a Service model because that's an area where this whole model shines. There's also a lot of automation, where you have different capabilities being used. Offensive Security is all about being proactive. It's about getting in the mindset of an external attacker and looking for the weaknesses you have on your external attack surface.

Dave Bittner: Well, I want to dig into Pentesting as a Service, but before we do, what other things fall under the umbrella of Offensive Security measures? What sort of things would you categorize there?

Jason Lamar: I would say there are things like Red Teaming, where you're trying to understand your particular scenarios that are high risk for your particular kind of organization or threat stack, and how your defenses work in that is also really important. So having an adversarial mindset — looking at not just probing for vulnerabilities like a scanner, or even just basically trying to think and test like you're the adversary.

Dave Bittner: Let's dig into some of the details about Penetration Testing as a Service. Can we start off again, sort of at a high level here? I mean, where do you think we stand when it comes to the types of offerings that are out there for this?

Jason Lamar: We see a lot of snake oil, to be honest. Script kiddies that are running the same couple of tools, calling it a pentest, or even claiming that's offensive. There's automated DAST scans that people are are doing, and then they add some kind of human review calling that offensive. So there's a lot of snake oil. What we recommend is people look for the providers' methodology and how have they exercised that over years. Have they got experience with it? Offensive Security is about going beyond pentesting and delivering a breadth of engagements based on maturity of an organization. Some organizations are just starting out — they need to do scans to pick up the easy-to-find things. But as they mature, they want to do pentesting to bring a variety of testing and outside-in perspective. Then, as they get even more mature, they'll do things like Red Teaming, secure code reviews, and those kind of engagements. So, depends on the maturity of the customer, but everyone's on a journey, I would say, to up their game.

Dave Bittner: Do you have any suggestions or words of wisdom for the types of questions people should be asking out there when they want to align that provider with where they are on their own journey?

Jason Lamar: Well, I think every industry has kind of a standard — analysts or different folks that monitor the industry and give recommendations. For the Offensive Security arena, and especially the Penesting as a Service arena, GigaOm is a great resource. You have a thing called the GigaOm Radar. The GigaOm Radar really takes you through specific selection criteria areas, where you might evaluate different providers or different companies that you would get this from. They'll talk about how the company's doing with actionable reporting, or how good their integrations are, how scalable they are, how quickly they can do testing for you, because not everyone can plan ahead. This GigaOm Radar, I think, is super helpful in understanding different players and their strengths, and also who are the frontrunners. The terms they use are "outperformers". It just helps you understand what the field is of providers, and based upon your requirements, you can interpret what's most important there.

Dave Bittner: What about setting a cadence for this sort of thing? How often do you engage? How often do you have Penetration Testing happen? How do folks go about dialing that in?

Jason Lamar: I think it's always important to talk with the organization that you want to partner with and have them consult with you as part of the ongoing discussion you have about procuring tests or engaging them. Usually, there's an assessment to understand where you're at in your maturity of your testing program. That can help you understand what you need most. If you're not sure what you need to do, if you have very specific requirements and you know what you want, then it just depends on the kind of activity that you're looking at. You might say, "Well, I just need a very quick compliance test for a new product that I'm deploying". There's folks out there that can start a test within 24 hours and have you up and running. I know of one that can do that. Then there's whole programs where you've got a very mature application, or group of applications, and they change less frequently. So you don't do your annual test there, but maybe you spend your testing time on things that are changing a lot within applications, particularly around this area of LLM AI. Existing applications that have been stable, and kind of not changing a lot, are getting new experiences added to them. We've had folks come and say, "Hey, I want to test this AI stuff". They may want to do a smaller test where they're just doing prompt injection kind of tests, or they may want to have a new experience altogether, and they want to do a comprehensive test. It depends on what your application or asset is, the rates of change on it, and the risk that you want to try to mitigate by testing. At least having that visibility and then understanding how your controls will do and what compliance needs that you have to fulfill your organization's objectives.

Dave Bittner: What are the advantages of engaging with someone from outside of your own organization to do this, rather than handling it in house?

Jason Lamar: If you're blessed with a team that is able to do this kind of testing, that's really great. A lot of organizations don't have the ability to hire for this in-house. Even those that do often need what I would call surge resources. There's overflow work, and so that's where you want to look outside. If you don't know where to start, obviously, engage with someone, especially as part of the "meet us" kind of part of the relationship. You'd want to have a discussion about your maturity and what your needs are. If you have specific projects that you know you need to do, either they're on a timeline and scope is really clear, then it's a good opportunity to use resources that can go fast. You engage someone who's got a large network of testers available. That's going to be an easier process than going through a statement of work with each, with different vendors for each test. There's some economies that you can have by working with organization that does Pentesting as a Service, or Red Teaming as a Service. Other benefits would be to have the ability to have a relationship with a company, do some testing with them — they know you, they know your assets and your organization, and they kind of understand what your unique situation is and what you want to get out of things. As more projects come up, you can just add those into the work you're already doing ongoing. Some customers have annual testing, and that's kind of their thing. Other customers want to do agile testing, where, especially increasingly with software development, there's an area of an app that's been undergoing a significant amount of change. "Hey, we want to pentest that area outside of our annual cadence". It's very common for that to happen as well. So you want a relationship with organizations that have those flexible capabilities to meet you in the kind of testing that you need.

Dave Bittner: I can imagine that, particularly for companies who are just starting down this path, there could be a certain amount of intimidation here that you're asking someone to come and poke at all the soft, exposed parts of your company here. Is that part of that onboarding conversation, to put people at ease and let them know exactly what to expect?

Jason Lamar: Absolutely, and to get yourself comfortable. First of all, do you like the folks you're talking with? That's always important. But folks that do this a lot — like, we do over 4000 tests a year — there's a familiarity and understanding of folks that are entering the process, their care-abouts and concerns, that we are particularly tuned with and others like us. So don't be intimidated, but do engage and help the folks that you're talking with understand what your actual needs are. And if you don't know what your needs are, be willing to go through a conversation to uncover those because a lot of times, that can be very enriching, whether you buy anything or not.

Dave Bittner: That's Jason Lamar, Senior Vice President of Product at Cobalt. We'll have a link to their research in our show notes.