Embrace The Red

Today we will cover a powerful, easy to use, autonomous agent called Manus. Manus is developed by the Chinese startup Butterfly Effect, headquartered in Singapore.

This post demonstrates an end-to-end indirect prompt injection attack leading to a compromise of Manus’ dev box.

This is achieved by tricking Manus to expose it’s internal VS Code Server to the Internet, and then sharing the URL and password with the atacker. Specifically, this post demonstrates that:

Recently, many of our favorite AI chatbots have gotten autonomous research capabilities. This allows the AI to go off for an extended period of time, while having access to tools, such as web search, integrations, connectors and also custom-built MCP servers.

This post will explore and explain in detail how there can be data spill between connected tools during Deep Research. The research is focused on ChatGPT but applies to other Deep Research agents as well.

Imagine a malicious instruction hidden in plain sight, invisible to you but not to the AI. This is a vulnerability discovered in Windsurf Cascade, it follows invisible instructions. This means there can be instructions in a file or result of a tool call that the developer cannot see, but the LLM does.

Some LLMs interpret invisible Unicode Tag characters as instructions, which can lead to hidden prompt injection.

In this second post about Windsurf Cascade we are exploring the SpAIware attack, which allows memory persistent data exfiltration. SpAIware is an attack we first successfully demonstrated with ChatGPT last year and OpenAI mitigated.

While inspecting the system prompt of Windsurf Cascade I noticed that it has a create_memory tool.

Creating Memories

The question that immediately popped into my head was if this tool will require human approval when Cascade creates a long-term memory, or if it is added automatically.

This is the first post in a series exploring security vulnerabilities in Windsurf. If you are unfamiliar with Windsurf, it is a fork of VS Code and the coding agent is called Windsurf Cascade.

The attack vectors we will explore today allow an adversary during an indirect prompt injection to exfiltrate data from the developer’s machine.

These vulnerabilities are a great example of Simon Willison’s lethal trifecta pattern.

Overall, the security vulnerability reporting experience with Windsurf has not been great. All findings were responsibly disclosed on May 30, 2025, and receipt was acknowledged a few days later. However, all further inquiries regarding bug status or fixes remain unanswered. The recent business disruptions and departure of CEO and core team members certainly put Windsurf in the news.

The Amazon Q Developer VS Code Extension (Amazon Q) is a very popular coding agent, with over 1 million downloads.

In previous posts we showed how prompt injection vulnerabilities in Amazon Q could lead to:

• Exfiltration of sensitive information from the user’s machine , and also to a
• System compromise by running arbitrary code

Today we will show how an attack can leverage invisible Unicode Tag characters that humans cannot see. However, the AI will interpret them as instructions, and this can be used to invoke tools and other nefarious actions.

The Amazon Q Developer VS Code Extension (Amazon Q) is a popular coding agent, with over 1 million downloads.

The extension is vulnerable to indirect prompt injection, and in this post we discuss a vulnerability that allowed an adversary (or also the AI for that matter) to run arbitrary commands on the host without the developer’s consent.

The resulting impact of the vulnerability is the same as CVE-2025-53773 that Microsoft fixed in GitHub Copilot, however AWS did not issue a CVE when patching the vulnerabiliy.

The next three posts will cover high severity vulnerabilities in the Amazon Q Developer VS Code Extension (Amazon Q Developer), which is a very popular coding agent, with over 1 million downloads.

It is vulnerable to prompt injection from untrusted data and its security depends heavily on model behavior.

At a high level Amazon Q Developer can leak sensitive information from a developer’s machine, e.g. API keys, to external servers via DNS requests. An adversary can also exploit this behavior during an indirect prompt injection attack.

In this post we discuss a vulnerability that was present in Amp Code from Sourcegraph by which an attacker could exploit markdown driven image rendering to exfiltrate sensitive information.

This vulnerability is common in AI applications and agents, and it’s actually similar to one we discussed last year in GitHub Copilot which Microsoft fixed.

Exploit Demonstration

For the proof-of-concept I use a pre-existing demo that created a longer time ago. It happened to just work with Amp as well. The prompt injection is hosted on a website which asks the AI to “backup” information to a third-party site by rendering an image and including previous chat data as a query parameter.

In this post we will look at Amp, a coding agent from Sourcegraph. The other day we discussed how invisible instructions impact Google Jules.

Turns out that many client applications are vulnerable to these kinds of attacks when they use models that support invisible instructions, like Claude.

Invisible Unicode Tag Characters Interpreted as Instructions

We have talked about hidden prompt injections quite a bit in the past, and so I’m keeping this short.

The latest Gemini models quite reliably interpret hidden Unicode Tag characters as instructions. This vulnerability, first reported to Google over a year ago, has not been mitigated at the model or API level, hence now affects all applications built on top of Gemini.

This includes Google’s own products and services, like Google Jules.

Hopefully, this post helps raise awareness of this emerging threat.

Invisible Prompt Injections in GitHub Issues

When Jules is asked to work on a task, such as a GitHub issue, it is possible to plant invisible instructions into a GitHub issue to add backdoor code, or have it run arbitrary commands and tools.

In the previous post, we explored two data exfiltration vectors that Jules is vulnerable to and that can be exploited via prompt injection. This post takes it further by demonstrating how Jules can be convinced to download malware and join a remote command & control server.

This research was performed in May 2025 and findings were shared with Google.

Remote Command & Control - Proof Of Concept

The basic attack chain follows the classic AI Kill Chain:

This post explores data exfiltration attacks in Google Jules, an asynchronous coding agent. This is the first of three posts that will highlight my research on Google Jules in May 2025. All information provided was also shared with Google at that time.

This first post will focus on data exfiltration, the lethal trifecta.

But let’s first talk about Jules’ system prompt.

Jules’ System Prompt and Multiple Agents

To grab the system prompt I just asked it to write it into a file.

This post is about an important, but also scary, prompt injection discovery that leads to full system compromise of the developer’s machine in GitHub Copilot and VS Code.

It is achieved by placing Copilot into YOLO mode by modifying the project’s settings.json file.

As described a few days ago with Amp, a vulnerability pattern in agents that might be overlooked is that if an agent can write to files and modify its own configuration or update security-relevant settings it can lead to remote code execution. This is not uncommon and is an area to always look for when performing a security review.

Today we cover Claude Code and a high severity vulnerability that Anthropic fixed in early June. The vulnerability allowed an attacker to hijack Claude Code via indirect prompt injection and leak sensitive information from the developer’s machine, e.g. API keys, to external servers by issuing DNS requests.

Prompt Injection Hijacks Claude

When reviewing or interacting with untrusted code or processing data from external systems, Claude Code can be hijacked to run bash commands that allow leaking of sensitive information without user approval.

Today we have another post about OpenHands from All Hands AI. It is a popular agent, initially named “OpenDevin”, and recently the company also provides a cloud-based service. Which is all pretty cool and exciting.

Prompt Injection to Full System Compromise

However, as you know, LLM powered apps and agents are vulnerable to prompt injection. That also applies to OpenHands and it can be hijacked by untrusted data, e.g. from a website. That impacts Confidentiality, Integrity, and Availability of the system.

Another day, another AI data exfiltration exploit. Today we talk about OpenHands, formerly referred to as OpenDevin. It’s created by All-Hands AI.

The OpenHands agent renders images in chat, which enables zero-click data exfiltration.

Simon Willison recently gave this data exfiltration attack pattern a great name: Lethal Trifecta.

We discuss this specific image based attack technique frequently. Sometimes a message must be repeated multiple times to raise awareness and become mainstream knowledge.

Today let’s explore Devin’s system prompt a bit more. Specifically, an interesing tool that I discovered when reading through it.

Hidden in Devin’s capabilities is a tool that can open any local port to the public Internet. That means, with the right indirect prompt injection nudge, Devin can be tricked into publishing sensitive files or services for anyone to access.

In this post we show how an attacker can make Devin send sensitive information to third-party servers, via multiple means. This post assumes that you read the first post about Devin as well.

But here is a quick recap: During an indirect prompt injection Devin can be tricked into download malware and extract sensitive information on the machine. But there is more…

Today we cover Devin AI from Cognition, the first AI Software Engineer.

We will cover Devin proof-of-concept exploits in multiple posts over the next few days. In this first post, we show how a prompt injection payload hosted on a website leads to a full compromise of Devin’s DevBox.

GitHub Issue To Remote Code Execution

By planting instructions on a website or GitHub issue that Devin processes, it can be tricked to download malware and launch it. This leads to full system compromise and turns Devin into a remote-controlled ZombAI. Any exposed secrets can then be leveraged to perform lateral movement, or other post-exploitation steps.