GitHub Can Now Detect System-Prompt Injection: What I Learned from Researching CodeQL 2.26.0

I need to say this up front: I am not a prompt-injection expert, and I had not worked closely with this new CodeQL query before writing this article. This post is based on my own research after seeing GitHub's announcement. I wanted to understand what changed, why developers should care, and what the tool can and cannot do.

What I learned is worth sharing. GitHub's CodeQL 2.26.0 adds a JavaScript and TypeScript query called js/system-prompt-injection. Its job is to find cases where untrusted, user-controlled data flows into an AI model's system prompt. That sounds like a narrow implementation detail. It is not. A system prompt often defines what an AI application is allowed to do, which tools it can use, and how it should handle sensitive information.

We have spent years scanning code for SQL injection, command injection, and cross-site scripting. Now code-scanning tools are starting to treat unsafe prompt construction as a vulnerability worth finding in source code.

What GitHub actually released

According to GitHub's July 10 announcement and the CodeQL 2.26.0 changelog, the release adds prompt-injection sinks for more OpenAI, Anthropic, and Google GenAI SDK APIs. These include system instructions, cached content, OpenAI Realtime session instructions, Sora prompts, and Anthropic's legacy completion prompts.

In plain language, CodeQL follows the path data takes through an application. If data controlled by a user reaches a sensitive system-prompt field without a trustworthy boundary, the new query may raise an alert.

CodeQL is suited to this because it does more than search for suspicious words. It builds a database representing the program and lets security queries analyze control flow and data flow. A text search can tell you that a file contains the phrase systemPrompt. A data-flow query can try to determine whether an HTTP parameter eventually reaches that field.

Why system prompts deserve security review

A system prompt is usually treated as trusted instruction. It might tell a model to act as a support assistant, never reveal internal data, call only approved tools, or follow a particular workflow. The problem begins when an application builds that trusted instruction using untrusted input.

Imagine an application that creates its system message like this:

const systemPrompt = `You are a support assistant.
Customer context: ${req.body.customerContext}`;

The developer may think customerContext is only background information. The model still receives it inside the trusted instruction channel. An attacker could submit text that tries to replace the original rules, request hidden information, or influence a later tool call.

This does not mean every interpolated value produces a successful attack. Models, SDKs, application logic, tool permissions, and output handling all affect the outcome. It does mean the application has mixed two different trust levels inside one instruction. That is a reasonable place for a security scanner to ask questions.

Direct and indirect prompt injection

OWASP currently lists prompt injection as LLM01 in its Top 10 for LLM applications. The research literature generally separates attacks into direct and indirect forms.

With direct prompt injection, the attacker types instructions into an input the model will read. An example is a user telling a chatbot to ignore its previous rules.

Indirect prompt injection is more difficult to notice. The hostile instruction can be hidden inside a webpage, document, email, code comment, retrieved database record, or another external source. An AI agent reads that content while working and may interpret it as an instruction. The user does not have to type the malicious message into the chat directly.

This matters more as AI systems gain tools. A model that can only generate text has a limited blast radius. A model that can browse websites, read private files, query company data, send email, or execute commands can cause more damage when it follows the wrong instruction.

What CodeQL can contribute

The new CodeQL query gives teams a repeatable way to find one important class of design mistake: untrusted values flowing into system prompts in supported JavaScript and TypeScript code.

That can help during a pull-request review, especially in a large project where prompt construction is spread across route handlers, helper functions, SDK wrappers, and agent configuration. A reviewer may not notice that a value started as request data several functions earlier. Static data-flow analysis is designed for exactly that kind of path.

It also gives security teams a common location for findings. Prompt-safety problems do not have to live only in a separate checklist or an AI red-team report. They can appear alongside other code-scanning alerts and become part of the team's normal review process.

GitHub automatically deploys new CodeQL functionality to users of GitHub code scanning on GitHub.com. Organizations using older GitHub Enterprise Server versions may need to upgrade CodeQL manually, as explained in the release notes.

What it cannot prove

This is where I had to slow down during my research. A CodeQL alert is not proof that an attacker successfully controlled the model. It identifies a potentially unsafe data-flow path. A developer still needs to inspect the source, the destination, and the application's safeguards.

The reverse is also true. A clean scan does not prove that an AI application is safe from prompt injection.

Static analysis may not see a hostile instruction stored in a document and retrieved at runtime. It cannot fully predict how a model will interpret natural language. It may not understand every custom SDK wrapper or dynamically constructed object. It also cannot replace strict authorization around tools and sensitive data.

This distinction is important. CodeQL can find risky plumbing in code. Runtime testing, adversarial evaluation, access control, monitoring, and careful product design still have separate jobs.

A practical review process

Based on the guidance from GitHub, OWASP, NIST, OpenAI, Anthropic, Google, Microsoft, and the papers listed below, I would use the new query as one layer in a wider review.

1. Map every input source. Include HTTP parameters, uploaded files, retrieved webpages, database content, tool results, and messages from other agents.
2. Separate instructions from data. Do not insert user content into a system prompt simply because it is convenient. Keep untrusted content in a clearly identified data field whenever the API and design allow it.
3. Limit tool authority. The model should receive only the permissions required for the current task. A support bot usually does not need unrestricted access to a shell, every customer record, or an email account.
4. Enforce authorization outside the model. A sentence such as "only administrators may perform this action" is not an authorization system. The application must check the user's identity and permissions in code before executing a sensitive action.
5. Validate tool arguments and outputs. Treat model-generated parameters as untrusted. Check them against schemas and business rules before use.
6. Run adversarial tests. Test direct instructions as well as hostile content placed in documents, webpages, retrieved records, and tool responses.
7. Monitor decisions and tool calls. Logs should make it possible to reconstruct what input the system received, which tool it selected, and what action the application approved.
8. Review CodeQL findings in context. Trace the complete path, decide whether the boundary is real, and document intentional exceptions instead of automatically dismissing the alert.

An honest conclusion from a non-expert

I started this research knowing that prompt injection was a problem, but I had not considered how it could be expressed as a source-code data-flow query. That is the part I found most interesting.

The release does not mean GitHub has solved prompt injection. Nobody has. It means at least one recurring mistake can now be detected using a workflow developers already understand: scan the repository, inspect the path, and fix the trust boundary.

For me, the practical lesson is simple. If an AI feature can touch private data or perform actions, prompt construction deserves the same seriousness as any other security-sensitive code. CodeQL 2.26.0 gives JavaScript and TypeScript teams another way to start that review, but it should be treated as a warning system rather than a safety certificate.

References

1. GitHub Changelog: CodeQL 2.26.0 adds AI prompt-injection detection
2. CodeQL CLI 2.26.0 changelog
3. GitHub Docs: Code scanning with CodeQL
4. GitHub Docs: Configuring code scanning
5. GitHub CodeQL repository
6. CodeQL Docs: About CodeQL queries
7. OWASP Top 10 for Large Language Model Applications
8. OWASP LLM01:2025 Prompt Injection
9. NIST Generative AI Profile
10. Greshake et al.: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
11. Yi et al.: Benchmarking and Defending Against Indirect Prompt Injection Attacks
12. Liu et al.: Automatic and Universal Prompt Injection Attacks Against Large Language Models
13. Schulhoff et al.: An Early Categorization of Prompt Injection Attacks
14. Liu et al.: Formalizing and Benchmarking Prompt Injection Attacks and Defenses
15. Simon Willison: Prompt injection attacks against GPT-3
16. Simon Willison: Prompt injection, what's the worst that can happen?
17. OpenAI API: Safety best practices
18. Anthropic: Mitigate jailbreaks and prompt injections
19. Google Gemini API: Safety and factuality guidance
20. Microsoft Azure AI Content Safety: Prompt Shields
21. MITRE CWE-77: Improper Neutralization of Special Elements Used in a Command
22. GitHub Docs: Troubleshooting CodeQL analysis logs