Real perspectives from security practitioners

AI basically unlocked the factory settings on a sports car for me. I've used so many security tools over the years and always wanted to build my own, and AI finally let me do that. But the part nobody talks about? It did the same thing for the attackers. Script kiddies, APT groups, whoever. They got the same upgrade. The hype gives you the shiny brochure but conveniently leaves that part out.

One of my favorite AI moments was when it confidently tried to convince me I’d found half a dozen broken access control issues.The application let regular users manage organizations, send invites, approve timesheets, and perform a bunch of actions that felt privileged. AI’s verdict was essentially, “Yep, none of this should be allowed.” Except… that’s exactly how the product was designed. Those permissions were completely intentional.

Everyone loves the idea of letting AI "handle the technicalities", right up until the report lands and nobody can tell which findings are real. That's usually when the humans get invited back to the party. Turns out the loop still needs us.

I regularly use AI to accelerate reconnaissance, review large codebases, and identify potential attack paths. In one assessment, AI confidently suggested an exploitation path that looked technically sound, but after manually testing the application, it turned out to be a false positive because it lacked the business context and understanding of how the application actually behaved. AI helped me get to the answer faster, but human validation and critical thinking were what ultimately led to the real security findings.

Now automated ai pentest agents are showing up everywhere to and all claim they number one, but their benchmarks are only based on vulnerable labs where the environment and vulnerability patterns are already studied or documented in their and public writeups are even available for models to learn from.

AIs will fundamentally change the security industry, the same way electricity changed cooking. A big misconception is that there will be one kind of mind to rule them all. That won't happen. The future is plural, combining multiple AIs with human creativity and judgment. I’m excited to see Cobalt lead the way, bringing a plurality of artificial and biological minds to pentesting and the broader offensive security landscape.

I’ve encountered many issues with AI. For example, even though its model is often updated, and despite having internet access, it frequently fails to find very recent vulnerabilities. In a recent penetration test, I found a very recent vulnerability in WP2Shell, and the AI ​​didn't find it because it lacked information about it. AI still makes mistakes. For instance, I remember a penetration test where it reported a missing field when it had found a mass assignment. However, when I manually explored the application, the field was indeed there. It simply provides what it believes, but it requires human review.

Seems that the Open AI agent(s) that went rogue this week is a clear example of where human intelligence and oversight would have been so important. And keep in mind although a monitoring tool used by Hugging Face identified the issue, human security practitioners quickly applied their intelligence to contain the incident.

I asked ChatGPT to describe me: “Based on what you’ve shared with me across conversations, you are a female CEO of a technology company. You are going to Madrid to study and you are a journalism major. You like to cook Thai food.” The first sentence describes me. The second describes my younger son who is a junior in college. The third describes my older son who is learning to cook!

AI is both under and overhyped. Warnings of AI taking over all human jobs and the imminence of AI singularity are hype. However, what we are not talking enough about is the impact of AI on political and social structures. When truth is distorted with misinformation, there is erosion of trust and polarization of communities. This impact of AI is very real and already here.

AI has great promise, but when you have a hammer everything looks like a nail. Overuse of AI will lead to creative monoculture and a drought of critical thinking. Not using it will cost us a lot of scientific progress and medical advances. Ignoring it is no longer an option.