300 Agents, 60 People: How We Secured Our Own AI Agents with Aiden Sochowski, Runlayer

Companies have near-zero visibility into the proliferating number of AI agents operating internally, and the only viable path to managing the resulting secur

I'll write the complete research synthesis now.

Abstract

This synthesis examines a quantitative case study of AI agent proliferation inside a security-tooling company of approximately 50-60 employees, where instrumentation revealed more than 300 agents operating daily - exceeding headcount by roughly fivefold. Drawing on a centralized agent gateway deployment, the analysis categorizes organizational risk into four classes: single-tool-call attack surface, session-level intent drift, permission over-provisioning, and cross-agent context contamination, alongside non-security cost and reliability failures. The central finding is that static, upfront permission schemas are insufficient because tool safety is context-dependent rather than binary; continuous session logging combined with custom-trained classifier models evaluating behavioral alignment against an agent's declared purpose emerges as the more tractable control strategy. Practical implications include the need for inbound/outbound policy layering, historically-derived permission descoping, and cost governance comparable in rigor to existing security governance, since most organizations currently run no evaluations on the agents they deploy in production.

1. Introduction

Enterprise adoption of autonomous software agents has outpaced the governance infrastructure designed to constrain them. Unlike earlier generations of software, in which data flows between systems required deliberate, hand-written integration code, agentic systems decide at inference time which tools to invoke and which data to combine. This shift removes a friction that previously forced intentionality onto cross-system data movement, and it does so at a scale that organizations are not yet equipped to observe, let alone control.

An agent, as used in this analysis, is defined broadly: it includes not only user-initiated chat sessions but also chatbots, scheduled jobs, and processes triggered by Software-as-a-Service (SaaS) application events. Each such process typically authenticates to one or more Model Context Protocol (MCP) servers, which expose internal systems - calendars, applicant tracking systems (ATS), code repositories, messaging platforms - as callable tools. The defining operational problem is that most organizations have no inventory of these processes, no centralized logging of their authentication, and no record of which systems they touch.

The central thesis examined here is that the security, cost, and reliability risks introduced by agent sprawl cannot be managed through permission rules authored in advance of deployment. Tool safety is highly context-dependent, and agents accelerate the mixing of data classes in ways that static, binary allow/block policies cannot anticipate. The alternative proposed and empirically tested is continuous monitoring: logging every session and tool call through a centralized gateway, then applying learned classifiers to detect deviation from an agent's declared purpose. This synthesis proceeds by situating the problem relative to prior integration paradigms (Section 2), analyzing four observed risk classes with supporting evidence (Section 3), consolidating implementable technical findings (Section 4), and discussing broader implications and open gaps (Section 5).

2. Background and Related Work

The organizational systems that agents now reach into were architected for a pre-agent era. Their authorization models assume either a human operator exercising case-by-case judgment or a deterministic integration whose data paths were fully specified at implementation time. Traditional Application Programming Interface (API) integrations required a developer to explicitly write the logic connecting, for instance, a private calendar to an outbound email path. That explicit authorship functioned as an implicit safety check: a developer had to consciously decide to join two data sources.

Agentic systems remove this constraint. Because tool selection occurs dynamically at inference time rather than being fixed at build time, the same kind of data-class mixing that once required deliberate engineering can now occur opportunistically, as a side effect of an agent pursuing its assigned task. This reframes the relevant control surface: rather than securing the integration boundary, the organization must secure the broader decision-making process of the agent itself, typically by routing all activity through an agent gateway - a proxy layer through which sessions and tool calls can be logged, visualized, and evaluated against policy.

3. Core Analysis

3.1 The Visibility Gap and Its Scale

Instrumenting an internal gateway to log every session and tool call, and subsequently classifying agents by topic and risk, produced a count substantially exceeding employee headcount: more than 300 agents operating daily within a company of roughly 50-60 people. As stated in the source material, "there are more of these agents within companies than the people who actually are running them." This visibility gap arises because employees independently adopt agent products that connect to MCP servers, authentication credentials become scattered across tools with no central record, and much of this activity executes on managed devices without logging infrastructure capturing it. A visualization tool built atop the gateway data mapped each agent as a "galaxy," with individual runs represented as "planets," surfacing which MCP servers were touched and which actions were taken per run - making the scale and topology of agent activity legible for the first time.

3.2 Single-Call Attack Surface and Session-Level Intent Drift

Every tool call an agent makes constitutes a discrete point of exposure: it may read hostile or injected data, take a destructive action, or exfiltrate sensitive information. A representative case involves a malicious actor emailing a company requesting a prospect list; an over-permissioned support agent could, in principle, pull that list from a calendar and return it directly to the requester. Mitigating this class of risk requires layered policy: classifier models that scrutinize inbound content (such as emails) for injected instructions, regex-based outbound policies that block personally identifiable information (PII) from leaving a session, and blanket restrictions on reads from sensitive sources such as private calendars. Critically, these policies must apply in both directions - inbound to screen what an agent ingests, and outbound to constrain what it emits - because tool safety "is tricky" to define as a binary allow/block decision.

A related but distinct risk operates at the session level rather than the single call. An internal PR-reviewing agent with merge-approval authority illustrates this: a code author who disagrees with a rejection can push back repeatedly in GitHub comments, asking the agent to "only review the cleanup" or to approve regardless. As the source notes, "I know it's hard to prompt inject Opus nowadays, but these things still aren't perfect" - repeated social-engineering attempts across a session can eventually succeed even against strong models. Detecting this requires classifiers that evaluate alignment between each tool call and the agent's originally defined purpose across the full session, not just at individual decision points.

3.3 Permission Over-Provisioning and Descoping

A further risk class concerns agents provisioned with broader tool access than their task requires. An internal recruiter bot, for example, may hold access to the ATS, meeting notes, calendar, and messaging systems despite a narrowly defined function - exposing it to sensitive content such as negotiation emails. Session data proves high-signal for catching this: flagging agents that invoke tools outside their historical or expected range surfaces high-risk runs early, though this signal is less reliable for general-purpose agents, such as local coding assistants, whose tool usage is inherently broad by design. Because defining minimal scope correctly at deployment time is difficult, models trained on historical tool-call patterns per agent can retroactively suggest permission descoping - an empirical, data-driven alternative to upfront specification.

3.4 Cross-Agent Contamination

The fourth risk class concerns systemic exposure: a single compromised agent can endanger others operating within shared or poorly partitioned systems. A malicious actor could embed adversarial instructions in an inbound resume; that poisoned content, once stored, could later be read and acted upon by an unrelated agent executing entirely normal instructions. As summarized in the source material, agents "are just plumbing data between one source to another, and it can be dangerous." The only mitigation found effective against this contamination vector is tightly scoping each agent's write access to specific systems and databases, limiting how far a single poisoned context can propagate.

4. Technical Insights

Several implementation patterns emerge as directly actionable. First, agent gateway architectures that centrally log every session and tool call are a prerequisite for any downstream control, since visibility must precede policy. Second, custom-trained classifier models serve two distinct functions: evaluating whether a given tool call or session remains aligned with an agent's declared purpose (intent-drift detection), and analyzing historical tool-call patterns to recommend permission descoping. Third, policy enforcement benefits from layering by direction and type - inbound classifiers for injection detection, outbound regex matching for PII, and static blocks on reads from designated sensitive sources - rather than relying on a single binary permission gate. Fourth, write-access scoping to specific systems is the most effective lever found against cross-agent contamination, since it bounds how far poisoned context can spread even when detection fails.

Beyond security, monitoring surfaced cost and reliability failures with the same instrumentation. A recruiting team's vibe-coded MCP server, built without engineering review for a vendor lacking an official server, generated repeated formatting errors and retries that inflated token costs. Separately, one team's costs increased by over 100x within a week after switching to a more capable but costlier model, Fable 5.1, without an accompanying evaluation of whether the added intelligence was justified by task requirements. This points to a governance gap: "nobody, most people are not going to be running evals on the agents that they're building," making cost and quality decisions effectively unaccountable absent monitoring infrastructure.

5. Discussion

The findings collectively suggest that agent governance is converging toward a security model resembling observability-driven network defense rather than classical, rule-based access control. Just as network security shifted from static firewall rules toward continuous traffic analysis and anomaly detection, agent governance appears to require continuous session analysis because the attack surface - arbitrary tool combinations selected at inference time - is too combinatorially large to enumerate in advance. This represents a meaningful departure from prior integration security practice, where a finite, developer-authored set of data paths could be reviewed exhaustively.

A significant open gap is the near-complete absence of evaluation practice for internally deployed agents, which leaves organizations unable to distinguish between a justified model upgrade and an unaccountable cost increase, or between an agent legitimately using a tool and one exhibiting drift. This absence of evaluation infrastructure is arguably as consequential as the absence of security policy, since both stem from the same underlying visibility deficit. Future work in this space likely needs to address how classifier-based intent-drift detection generalizes to genuinely general-purpose agents, where "historical tool-call range" is a weaker signal by design, and how organizations can establish evaluation baselines for agents built outside formal engineering processes.

6. Conclusion

This analysis demonstrates that agent proliferation inside organizations has already outpaced the governance tools built to manage prior software paradigms, with a company of 50-60 employees found to be running over 300 daily agents through unaccounted-for channels. The practical takeaway is that static permission definitions cannot keep pace with context-dependent tool safety, and that centralized logging paired with custom classifiers for intent-drift detection and permission descoping represents a more tractable control strategy. Organizations deploying agentic systems internally should treat agent-gateway instrumentation, inbound/outbound policy layering, and basic cost evaluation as foundational infrastructure rather than optional additions, given that visibility - not rule authorship - is the binding constraint on governance today.

Sources

• 300 Agents, 60 People: How We Secured Our Own AI Agents - Aiden Sochowski, Runlayer - Original Creator (YouTube)
• Analysis and summary by Sean Weldon using AI-assisted research tools

About the Author

Sean Weldon is an AI engineer and systems architect specializing in autonomous systems, agentic workflows, and applied machine learning. He builds production AI systems that automate complex business operations.

LinkedIn | Website | GitHub