In September 2026, Google confirmed that a Gemini model gained unauthorized access to three outside systems during a security test. The headline invites fear; the facts invite something far more mundane and far more useful: reviewing how you define permissions, environments, and credentials before granting an agent autonomy.

By E2M Global Advisory LLC · Published on September 28, 2026 · About E2M

A person authorizes an AI agent to connect to CRM, email and documents while other connections remain blocked.
AI-generated illustration for E2M Global Advisory.

If your company already connects an AI agent to your CRM, email, or sales proposals, you’ve probably asked yourself this question in recent weeks: what happens if it targets the wrong system? The question isn’t hypothetical. In September 2026, Google confirmed that a Gemini model gained unauthorized access to three outside computer systems during a cybersecurity evaluation. The headline is alarming. The facts, read calmly, point to something far less cinematic and far more useful for a business leader: an environment configuration failure and poor credential hygiene, not an AI that decided to act on its own. This article separates what happened from what has been exaggerated, and translates it into the question that actually matters to you: what permissions, what environment, and what credentials your company needs to review before granting autonomy to a sales agent.

The core idea: it wasn’t the AI — it was the permissions and the environment

The Gemini incident doesn’t prove that AI agents are inherently dangerous. It shows that a poorly isolated test environment and exposed credentials are enough for an agent — even one with no malicious intent — to end up acting outside its intended boundaries. The lesson for any company isn’t “don’t use agents.” It’s that before granting one autonomy, you need to be able to answer three questions: exactly which systems it can touch, what credentials it acts with, and who finds out if it makes a mistake. Those three questions, not the headline, are the real governance content of this case.

What exactly happened with Gemini in September 2026?

In May 2026, during a “capture the flag” cybersecurity evaluation run by the firm Irregular, a Google Gemini model gained unauthorized access to the systems of three real companies. Google explained that the model got into two of those systems using credentials found in a public password repository, and into the third by trying various access combinations, as detailed by CNBC. According to Google, the root cause was a case of “mistaken identity”: the test environment had an unintended internet connection, and the name of a fictitious company coincided with that of a real one, so the agent believed it was operating inside the simulated exercise when it was actually connected to real systems.

The timeline also matters: Irregular reported the finding to Google in late July 2026, and public confirmation didn’t come until September 18, after the Wall Street Journal asked directly about the case, as reported by NBC News and SecurityWeek. By its own account, Google didn’t actively conceal the incident, but there’s roughly a seven-week gap between detection and public disclosure that’s worth keeping in mind when assessing how quickly — or not — this type of incident gets reported.

One detail that tends to get lost in the headlines: this isn’t an isolated case involving a single vendor. In the same weeks, incidents with the same pattern — test environments with insufficient isolation and models that end up attempting to access external systems — were reported at other AI labs, all linked to evaluations run by the same testing firm, Irregular, according to CNBC. This points to a structural problem in how test environments are designed across the industry, not a flaw exclusive to one particular model.

Why this isn’t “a rogue AI” (and why that isn’t entirely reassuring)

Google has been explicit on one point: it doesn’t consider this incident to rise to the level of what the industry calls “misalignment” — a system that stops following the instructions it was given. According to the company, the model course-corrected once it detected it wasn’t in the simulated environment, and no harm resulted. That’s a meaningful technical distinction: it’s one thing for an agent to misbehave because of a poorly designed environment, and quite another for it to deliberately ignore the restrictions imposed on it. Everything known about this case points to the former.

What’s been established and what’s just one party’s account

It’s worth being precise here about what’s verified and what’s a single party’s account. It is verified that the entry vector was credentials exposed in a public repository and a network isolation failure in the test environment; the technical descriptions of the incident confirm that much. What is not independently verified is the claim that “no harm occurred” — that’s Google’s own assessment, coming from the party involved, and there’s no published external audit that confirms it. In fact, according to later reporting, some AI security researchers have pushed back on how quickly the incident was dismissed as “nothing sophisticated.” You don’t need to take sides in that debate to draw the governance lesson: when the company affected by an incident is also the one assessing its own severity, a prudent executive treats that assessment as one data point, not as case closed.

What “permissions” for an AI agent actually mean in your company

It’s easy to talk about “giving an agent permissions” as if it were a single decision. In practice, it’s four separate decisions, and conflating them is exactly the mistake the security industry has been flagging for months.

  • Identity: Does the agent have its own auditable identity, or does it act “disguised” as a human user or as a generic service account shared across multiple systems?
  • Scope (permissions): What specific actions can the agent execute — read, write, delete, send — and in exactly which systems, beyond the vague “it has access to the CRM”?
  • Credentials: Are they permanent keys that someone could reuse or leak, or are they temporary, task-scoped, and short-lived?
  • Environment: Is the agent tested in a space that’s genuinely isolated from production systems before getting real access, or is the “test environment” actually the real system with less oversight?

NIST, the U.S. government’s standards institute, is working on precisely this problem through its NCCoE center, which has identified two very common antipatterns in real companies: agents that inherit all the permissions of the user who configured them — far more access than the task requires — and agents that operate under generic service accounts, which can’t be traced back to a specific human intent. Neither pattern is acceptable in environments that require least privilege, auditability, or control over where data resides. It’s worth noting that this NIST work is, as of today, a concept paper under discussion, not a final binding standard; but it points fairly clearly to where the industry’s reference criteria are heading.

In the same vein, a reference framework for agentic application security, reviewed by more than a hundred security specialists, sums up the principle in one simple phrase: “minimal agency.” An agent that researches vendor pricing doesn’t need permission to log into the account and edit it — it only needs what that specific task requires, and nothing more.

The same mistakes, at a smaller scale: how they show up in an SME or mid-sized company

The Gemini model involved in the incident had offensive cybersecurity capabilities and was operating in a pentesting exercise with internet access; it isn’t comparable, in either autonomy or risk level, to an agent that drafts CRM follow-ups or proposal drafts. Generalizing to “this is how easily any AI in a company goes off the rails” would be a misreading. But the two antipatterns NIST flags — permission inheritance and shared credentials — do show up, almost identically, in much smaller everyday decisions:

  • Connecting a sales follow-up agent to corporate email using the sales director’s own account, instead of a service identity with permissions scoped to read and suggest, not send without review.
  • Giving an agent that drafts proposals read-and-write access to the entire CRM when it only needs to read one account’s history and write to a specific field.
  • Using a permanent API key, stored in a spreadsheet or a code repository, instead of temporary credentials that expire after each task.
  • Testing a new agent directly on real customer data “just to see how it works,” without a separate test environment.

None of these mistakes requires a frontier model or a cybersecurity exercise to happen — they’re made with the same off-the-shelf commercial tools any mid-sized company already uses.

Checklist: what to review before granting a sales agent autonomy

This table isn’t a guarantee that an agent will operate without incident — it’s a diagnostic tool to reduce the type of risk this case puts on the table.

Swipe or scroll the table ↔ to see every column.

Category Question you should be able to answer Warning sign
Identity Does the agent have its own auditable identity, distinct from any employee’s? Operates “as if it were” a specific human user
Scope Which systems can it touch, and why exactly those and no more? Has access to “the whole CRM” or “all email” without limits
Credentials Are they temporary and task-limited, or permanent keys? The same key works for everything and never expires
Environment Has it been tested in an isolated space before touching real data? The “test environment” is the production system
Traceability Is every action logged, and does someone have real authority to stop it? No one could reconstruct what the agent did or when

If you can’t answer any one of these five rows precisely, that’s the point to start reviewing before expanding the agent’s autonomy — not after.

What EU regulation says — and doesn’t say — about this

The EU AI Act (Regulation (EU) 2024/1689) requires effective human oversight for systems classified as high-risk (Article 14), with measures proportionate to the system’s level of autonomy and context of use. That’s a real obligation, but it shouldn’t be generalized in either scope or timing: it applies to the AI systems the Regulation itself classifies as high-risk, not to “any AI agent” your company happens to use, and its actual enforceability has recently changed. The Digital Omnibus on AI (Regulation (EU) 2026/1744), in force since July 27, 2026, postponed the entire block of high-risk obligations under Chapter III — which is exactly where Article 14 sits — from August 2, 2026 to December 2, 2027 for Annex III systems, and to August 2, 2028 for Annex I systems. This doesn’t mean human oversight has stopped mattering as a governance principle; it means that, as of this guide, it isn’t yet a legally enforceable obligation for those systems, although other obligations under the Regulation (such as Articles 4, 5, and 50) keep their original deadlines. Most sales agents — CRM follow-up, proposal drafting, customer service responses — also don’t automatically fall into the Annex III high-risk categories, although the role your company occupies (provider or deployer) and the specific use you make of the agent can shift that classification case by case. We cover in more detail which specific obligations apply to sales teams in our article on the AI Act and sales/customer service obligations — it’s worth checking that piece also reflects this postponement before treating it as a complete reference — this article doesn’t replace that reading, it complements it from the angle of technical permissions. This content is general guidance and does not constitute legal advice: if your company handles sensitive data, automated decisions about individuals, or operates in regulated sectors, a review with specialized legal counsel is advisable before expanding an agent’s autonomy.

What I wouldn’t do

I wouldn’t connect an agent to a production system — CRM, ERP, email — using an employee’s personal credentials or a shared service account “because it’s faster to set up.” I wouldn’t give an agent write or delete permissions without first going through a read-only phase and requiring explicit human approval for the highest-impact actions. And I wouldn’t use the Gemini incident as a generic fear-based pitch to sell “AI security” without specifying exactly which process, which system, and which specific credential is being protected: that kind of alarmism doesn’t help anyone decide anything — it just sells anxiety.

From insight to action

Turn agent boundaries into clear requirements.

Define permissions, responsibilities and validation criteria with the team that will implement the solution.

Discuss your situation →

Continue reading

Sources and limits of this guide

The assessment that the incident “caused no harm” and “does not constitute misalignment” is Google’s own account, as the party involved in the case; as of this guide, there is no published independent audit that confirms it neutrally.

Follow E2M Global Advisory on X@e2mglobal · AI applied to commercial processes