A Berkeley Researcher Explains Why AI Agents Create a New Security Problem

Dawn Song, co-director of Berkeley's Center for Responsible, Decentralized Intelligence, explains why agentic AI creates a broader security attack surface than traditional chatbots, and what researchers are doing about it.

August 20, 2026
A Berkeley Researcher Explains Why AI Agents Create a New Security Problem Security

Summary: Dawn Song, co-director of UC Berkeley's Center for Responsible, Decentralized Intelligence, explains why agentic AI, systems that reason, plan and take real-world actions rather than just generating text, introduces security risks like prompt injection attacks that traditional chatbots don't face. The center, which hosted the 5,000-attendee Agentic AI Summit 2026 and has taught nearly 40,000 people through its free online course series, is building evaluation platforms like AgentBeats to measure agent reliability and security before deployment rather than after.

agenticaiaisecurityucberkeleydawnsongpromptinjectionaievaluation

A chatbot answers your question. An agent goes and does something about it.

That distinction is the whole story, according to Dawn Song, co-director of UC Berkeley's Center for Responsible, Decentralized Intelligence.

Traditional chatbots respond to prompts. Agentic systems reason, plan, use outside tools, and take multi-step actions with limited human supervision.

From Answering Questions to Taking Actions

Song frames the shift in plain terms.

"I believe we're moving from AI as a tool that people consult to AI as a collaborator and capable partner," Song said.

That's a meaningful upgrade in capability. It's also a meaningful upgrade in what can go wrong.

The New Attack Surface

A chatbot that gives a bad answer is an inconvenience. An agent that takes a bad action inside a real system is something else entirely.

"Agentic AI changes the security landscape because these systems are no longer limited to generating information, instead, they can take actions in the world," Song said.

That framing lines up with what security researchers have been warning about across the industry this year, as AI models with tool access have shown they can act on unintended goals when guardrails aren't sufficient.

Song's research describes mechanisms that can create risks like these: prompt injection attacks that trick an agent into ignoring its instructions, and errors that compound across long chains of autonomous decisions.

Song argues these risks need to be designed against from the start, not patched in after a system is already deployed.

That's not just a philosophical stance. Berkeley treats reliability, security and evaluation as core research problems that need to advance alongside raw capability, rather than bolted on once a system is already in production and something has already gone wrong.

That approach shapes how the center measures progress too, not just how it talks about risk.

How Berkeley Is Trying to Measure the Problem

Berkeley's response leans heavily on evaluation. The center built a platform called AgentBeats specifically to standardize how agent reliability and security get tested, since Song says the field still lacks rigorous, reproducible ways to measure whether an autonomous system can actually be trusted under real conditions.

The center is also betting on reach. Its Agentic AI MOOC series has taught nearly 40,000 people since 2024, spanning students, policymakers and industry practitioners, on the theory that the people deploying these systems need to understand the risks as well as the researchers building them.

Song is also watching a more technical frontier: combining AI-generated code with formal verification methods, so software agents write can eventually come with machine-checkable proof that it's actually correct and secure, rather than the current standard of code that looks reasonable but carries no such guarantee.

What This Means for Miami

For Miami's CISOs and security teams weighing whether to deploy agentic security tools, Song's research offers a useful counterweight to the industry's enthusiasm: agent deployment needs staged rollouts and rigorous evaluation, not blind trust in vendor claims about capability.

There's also a talent pipeline angle worth watching locally. Berkeley's free MOOC series is exactly the kind of accessible AI education Miami's own universities and workforce development programs could look to as a model, at a moment when local companies are racing to deploy agentic tools faster than most of their own security teams fully understand them.

The underlying lesson from Song's research applies just as directly to a Miami startup shipping its first agentic feature as it does to a frontier lab.

Capability and safety aren't separate workstreams to sequence one after the other. Treating them that way is how agentic systems end up causing the exact failures researchers like Song are trying to design against.