The wrong kind of access. [Research Saturday]

Host Dave Bittner introduces Dan Lesichkin, an AI security researcher at Pillar Security, to discuss the research “A WIF Of Fresh Access: How a GitHub Issue on Gemini CLI Led to GCP Project Compromise.” Lesichkin had already found a prompt-injection vulnerability in Google’s Gemini CLI repository, but while reviewing the project again he suspected that a second, more serious weakness might exist. He noticed what he called a “weird configuration” in the way Gemini GitHub Actions connected to Google Cloud Platform: a broad configuration that appeared capable of creating credentials with which an attacker could take over the Gemini CLI’s GCP project. Lesichkin explains that Gemini CLI is Google’s harness for working with Gemini, comparable in purpose to tools such as Cloud Code or Codex. It gives a language model operating-system tools so it can write code and perform tasks on a computer. Google had connected it to GitHub Actions so that, whenever a user opened an issue on the Gemini CLI repository, an agent would read the issue and assign a label such as feature, duplicate, bug, or security issue. That made the workflow useful for a busy repository, but it also meant that text supplied by an untrusted external user was being ingested by an agent running inside a CI environment.

The original vulnerability involved an external user opening a GitHub issue containing a prompt injection. Gemini CLI processed the issue in GitHub Actions, and although Google had supposedly limited the agent to tools such as `echo` and reading files, it was running in “YOLO mode,” where the tool restrictions were not actually effective. Lesichkin had previously used that weakness to obtain a GitHub token by getting the agent to execute code inside the Actions runner. After Google appeared to fix that issue, Lesichkin looked for a way to reach the more valuable GCP credential. His first hypothesis was that Gemini could read the credential file present in the CI runner and print it into the workflow logs. That approach failed in practice. The credential was a roughly 3,000-character JSON-style authentication file, and asking the model to reproduce it verbatim was unreliable. GitHub also automatically redacted secrets detected in logs. Lesichkin tried splitting the token into pairs of characters—roughly 1,500 separate pieces—so that the secret would not be recognized as one continuous value. Gemini still could not reliably output it, although Lesichkin saw fragments of the credential appear, which itself represented an unwanted leak.

To investigate further, Lesichkin gave the task to his team’s custom cloud-code harness. He forked the Gemini CLI project and instructed the harness to keep working until it could make Gemini read the credential. About ten minutes later, the harness reported that Gemini CLI had executed code in the machine. Lesichkin initially assumed the result was a hallucination—he says he was in his living room, “banging on the table,” insisting that the system was lying—so he made the harness confirm exactly how code execution was possible when only `echo` and file-reading tools were supposed to be available. The problem was a configuration-field mismatch. Google had changed Gemini CLI’s tool-scoping scheme: the relevant field had become `core.tools`, with a dot. In the official repository, however, the configuration still used `core tools`, without the dot. As a result, Gemini CLI did not recognize the setting and did not scope the tools at all. The earlier vulnerability had therefore been fixed in the product’s expected configuration, but not in the repository’s actual configuration. Because the agent retained unrestricted tool access, Lesichkin no longer needed to force a 3,000-character credential through the model’s output. He could instead use the agent to run code directly in the CI runner.

Lesichkin reused the prompt injection from his earlier exploit. He instructed Gemini to act as though it were running a firewall test and to execute a command that sent the credential to his own server. Gemini complied, then reported that it had sent the token and that the sandbox had been breached. Lesichkin characterizes the prompt injection as the easy part; the difficult work was finding the chain of unintended interactions that made the injection consequential. The chain depended on several bridges: the agent had access to untrusted GitHub issue text, the intended tool restrictions were not active, a credential file was present in the CI runner, and the credential had excessive authority in GCP. Lesichkin says that code execution alone would not necessarily have produced a serious impact if the credential were inaccessible or worthless. Conversely, merely reading small portions of a credential would have had limited value. The significant result came from combining unrestricted execution with a Workload Identity Federation credential that allowed access to the GCP project and could be used to impersonate service accounts. He describes these combinations as “weird machines,” where individually familiar components create an unexpected capability when composed together.

Bittner asks how Google ultimately broke the attack chain. Lesichkin identifies three fixes. First, Google added the credential file to the Gemini ignore list so that the Gemini CLI process running in the GitHub Actions runner could no longer read it. Second, Google corrected the tool-configuration field so that the intended restrictions were actually applied. Third, Google fixed the credential’s excessive permissions by reducing the scope of the service account access it provided. The most important identity issue was that the agent held a credential capable of impersonating any service account in the relevant GCP project. Lesichkin notes that the project was not necessarily a highly critical project from Google’s perspective, but the agent was effectively given a human-level identity that could do anything within it. The episode uses that detail to show why an AI agent’s apparent instructions are not enough protection. A system prompt saying “do not do this,” or an allowlist that is misconfigured or incomplete, cannot substitute for enforced operating-system, application, and cloud-identity boundaries.

Lesichkin says system prompts and prompt injections should not be treated as security boundaries. Agents are nondeterministic: unlike ordinary programs, whose control flow can generally be modeled for a given input, an agent can produce different behavior from similar inputs. If an agent can ingest untrusted text and has powerful tools, defenders cannot safely assume it will use those tools only as intended. His preferred model is to give the agent the tools it genuinely needs, ensure those tools are themselves secure, and enforce the boundaries outside the model’s reasoning. He summarizes the advice in four rules. An allowlist or blocklist is not a sandbox: forbidding `curl` does not prevent an agent from trying raw sockets or another route to the Internet. Every tool is its own attack surface because application bugs may let an agent or attacker use it for unintended actions. An agent needs an identity, but that identity must have least privilege and only the permissions required for the task. Finally, an agent with no authority cannot cause the same kind of compromise as an agent with unrestricted authority; tool permissions must be modeled and scoped to the specific job. The episode closes by identifying this research as a warning about connecting AI agents to untrusted input and sensitive infrastructure.