Anatomy of a Supply Chain Attack: LiteLLM on PyPI

ashwin@ashwingopalsamy.in (Ashwin Gopalsamy) — Wed, 25 Mar 2026 00:00:00 +0000

On March 24, 2026, Callum McMahon at FutureSearch was testing a Cursor MCP plugin that pulled in litellm as a transitive dependency. He never ran pip install litellm himself. The plugin resolved it automatically.

Shortly after, his machine became unresponsive. RAM exhausted. He traced it to a newly installed litellm package, decoded an obfuscated payload hidden inside it, and published the first disclosure . His team used Claude Code to help root-cause the crash.

The post spread to r/LocalLLaMA , r/Python , and the Hacker News front page within the hour. Andrej Karpathy tweeted about it, calling supply chain attacks “the most threatening issue in modern software.” That tweet crossed 24,000 likes in a day.

I use LiteLLM in a side project. When I saw the news, I checked my lockfile immediately. Here’s everything I found when I dug into what happened, stitched together from Datadog , Snyk , Armosec , ramimac’s incident timeline , Microsoft , Wiz , and the community threads on r/devops and r/cybersecurity .

What Is LiteLLM?

LiteLLM is a Python library that works like a universal remote for AI APIs. You write one function call, and it routes to OpenAI, Anthropic, Google, Cohere, Mistral, AWS Bedrock, Azure OpenAI, or any of 100+ providers. You give LiteLLM your API keys. It handles the rest.

95 million downloads per month on PyPI. ~40,000 GitHub stars. It’s also pulled in automatically by DSPy , MLflow , and a growing number of agent frameworks, MCP servers, and LLM tools. You can be using LiteLLM without knowing it.

This Started Three Weeks Earlier

The LiteLLM backdoor on March 24 was the last move in a campaign that began on March 1.

A threat actor called TeamPCP submitted a malicious pull request to Aqua Security’s Trivy repository. Trivy is a widely-used open-source vulnerability scanner - the kind of tool that runs in your CI pipeline to check for security issues. The PR exploited a flaw in Trivy’s CI workflow that let the attacker’s code run with elevated permissions (a technique called a Pwn Request - basically, a pull request that tricks CI into handing over secrets). This gave them a personal access token.

Aqua Security responded and rotated credentials, but the rotation wasn’t complete . Attackers may have captured the new tokens during the rotation. That gap is what enabled everything after.

sequenceDiagram participant A as Attacker
(TeamPCP) participant T as Trivy GitHub
Repo participant CI as LiteLLM CI
(GitHub Actions) participant P as PyPI
Registry Note over A,T: March 1 - Initial Compromise A->>T: Submit malicious PR (Pwn Request) T-->>A: CI leaks personal access token Note over A,T: Aqua rotates creds,
but rotation incomplete Note over A,T: March 19 - Pivot A->>T: Retag trivy-action versions
to point at malicious code Note over CI,P: March 24 - Package Takeover CI->>T: Pull trivy-action@latest
(not pinned to SHA) T-->>CI: Serve compromised action CI-->>A: Leak PyPI publisher token A->>P: Publish litellm v1.82.7
(inline payload) A->>P: Publish litellm v1.82.8
(.pth persistence) Note over P: PyPI quarantines
both versions

How They Got the LiteLLM PyPI Token

LiteLLM’s CI pipeline used Trivy to scan for vulnerabilities. Standard practice. But it pulled aquasecurity/trivy-action without pinning to a specific commit hash - it used a version tag like @latest instead of an exact SHA.

After March 19, every version tag pointed to malicious code. When LiteLLM’s CI ran, the compromised Trivy action scraped the GitHub Actions runner environment and found the PyPI publisher token. That token let TeamPCP publish packages as if they were the real LiteLLM maintainer.

graph LR A["Trivy Repo
(compromised)"]:::danger --> B["GitHub
Bot Army"]:::danger B --> C["PyPI Account
Takeover"]:::danger C --> D["LiteLLM
v1.82.7 / v1.82.8"]:::danger D --> E[".pth
Backdoor"]:::danger E --> F["Credential
Harvest"]:::danger F --> G["Exfil Server
(ICP Canister)"]:::danger H["Trivy Repo
(legitimate)"]:::safe -.->|compromised| A I["LiteLLM
(legitimate)"]:::safe -.->|hijacked| D J["PyPI
Registry"]:::safe -.->|abused| C classDef danger fill:#ef4444,color:#fff,stroke:none classDef safe fill:#10b981,color:#fff,stroke:none

What the Malware Does

The payload was triple-nested : a base64 blob decodes to an orchestrator script, which decodes a second base64 blob containing the actual harvester. Once running, it goes through six stages:

graph TD ROOT[".pth Backdoor"]:::danger ROOT --> ENV["ENV Variables
os.environ"]:::warm ROOT --> AWS["~/.aws/credentials
AWS keys"]:::warm ROOT --> DOTENV[".env Files
recursive scan"]:::warm ROOT --> PROC["/proc/self/environ
process secrets"]:::warm ROOT --> EXFIL["Outbound HTTP
Exfil to ICP canister"]:::warm ENV --> EXFIL AWS --> EXFIL DOTENV --> EXFIL PROC --> EXFIL classDef danger fill:#ef4444,color:#fff,stroke:none classDef warm fill:#f59e0b,color:#fff,stroke:none

Two Versions, Two Triggers

Version 1.82.7 injected the payload at line 128 of litellm/proxy/proxy_server.py, between two unrelated legitimate code blocks. It runs when your code imports the LiteLLM proxy module.

Version 1.82.8 did something more dangerous. It added a file called litellm_init.pth (34,628 bytes). In Python, .pth files are a little-known feature: any file with that extension in the packages directory gets executed every time Python starts. Not when you import something. Not when you run a script. Every single Python process.

★

Most developers don’t know .pth files can execute arbitrary code. Originally designed for adding directories to sys.path, any line in a .pth file starting with import is executed by CPython’s site.py at startup. This means a malicious .pth file in your site-packages directory runs code before your application even begins, on every Python invocation: pytest, your IDE’s language server, even pip install. CPython maintainers have acknowledged the risk. No patch exists.

graph TD A["Python Starts"]:::info --> B["Scans site-packages
for .pth files"]:::neutral B --> C["Finds litellm_init.pth"]:::danger C --> D{"Line starts
with import?"}:::neutral D -->|Yes| E["Executes code
via site.py"]:::danger E --> F["Decodes base64
orchestrator"]:::danger F --> G["Decodes base64
harvester"]:::danger G --> H["Scrapes credentials
from ENV, files, /proc"]:::danger H --> I["Exfiltrates to
ICP canister"]:::danger D -->|No| J["Adds path to
sys.path (normal)"]:::safe E --> K["Spawns child
Python process"]:::danger K -->|".pth fires again"| A classDef info fill:#6366f1,color:#fff,stroke:none classDef neutral fill:#64748b,color:#fff,stroke:none classDef danger fill:#ef4444,color:#fff,stroke:none classDef safe fill:#10b981,color:#fff,stroke:none

Running pytest starts Python - payload fires. Your IDE’s language server starts Python - same. Even pip install triggers it. In CI/CD, the payload runs during build steps, not just at application runtime. This maps to MITRE ATT&CK T1546.018 (Python Startup Hooks).

The .pth mechanism also caused an accidental fork bomb: the malware spawned a child Python process, which triggered .pth again, which spawned another child, and so on. Exponential process creation until the system ran out of memory. FutureSearch called it “a bug in the malware” - and it’s the bug that led to the discovery.

The Discovery and the Bot Army

McMahon published on FutureSearch’s blog . The disclosure spread to r/LocalLLaMA , r/Python , and the Hacker News front page within the hour. Then things got strange.

When the community opened GitHub issue #24512 to discuss the compromise, TeamPCP deployed 88 bot comments from 73 unique accounts in a 102-second window (12:44-12:46 UTC). These were previously compromised developer accounts, not fresh ones. Snyk found 76% overlap with the botnet used during the Trivy disclosure days earlier. The comments were a mix of generic praise and troll content (“sugma”, “ligma”), designed to bury the technical discussion.

Then, using the stolen LiteLLM maintainer account, they closed issue #24512 as “not planned.”

The community opened a parallel tracking issue. PyPI quarantined both versions. The real LiteLLM maintainer confirmed on HN that all GitHub, Docker, and PyPI keys had been rotated and accounts moved to new identities.

What Makes This Worse Than Usual

The target was a credential vault. LiteLLM holds more API keys per deployment than almost any other library. A typical setup has keys for OpenAI, Anthropic, Google, Azure, Hugging Face, Bedrock, plus cloud credentials, database passwords, and whatever MCP server access you’ve configured. As Armosec put it : “AI tooling is becoming the fattest, most credential-rich target in your entire infrastructure.”

Transitive dependency exposure. The FutureSearch developer never installed LiteLLM. It came in through a Cursor MCP plugin. DSPy pulls it in. MLflow pulls it in. You can be in the blast radius without choosing to use the library.

Unseizable infrastructure. TeamPCP’s command server includes an ICP canister replicated across 13 nodes in 10 countries. Datadog documented this as the first observed use of ICP as a command server in a supply chain campaign. By version 3.3 of their kamikaze.sh payload , they were hiding Python code inside WAV audio files using steganography to bypass detection filters.

Organized cover-up. Bot armies from a pre-existing botnet (76% account reuse), troll comments, and closing the disclosure issue using the stolen maintainer account. This is not a lone actor.

If You Installed 1.82.7 or 1.82.8

⚠

Check your environment immediately. If any of the following commands return results, the payload has already executed. Upgrading the package alone is not enough.

# Backdoor persistence
ls ~/.config/sysmon/sysmon.py 2>/dev/null && echo "BACKDOOR FOUND"
systemctl --user status sysmon.service 2>/dev/null

# .pth file (v1.82.8)
find $(python3 -c "import site; print(' '.join(site.getsitepackages()))") \
  -name "litellm_init.pth" 2>/dev/null

# Check uv caches too
find ~/.cache/uv -name "litellm_init.pth" 2>/dev/null

# Exfil artifacts
ls /tmp/tpcp.tar.gz /tmp/session.key /tmp/payload.enc /tmp/.pg_state 2>/dev/null

# Kubernetes spread
kubectl get pods --all-namespaces | grep node-setup

If anything shows up, upgrading the package is not enough. The payload already ran!

⚠

Rotate immediately:

All LLM provider API keys (OpenAI, Anthropic, Google, every key LiteLLM proxied)
Cloud credentials reachable from that runtime (AWS, GCP, Azure)
GitHub and PyPI publishing tokens
CI/CD secrets
SSH keys
Kubernetes service account tokens

Then rebuild. Known-good images, pinned dependencies. Audit transitive dependencies in every project that uses LiteLLM. The last known-clean version is 1.82.6.

What I’m Sitting With

The attack on GitHub issue #24512 spawned a Hacker News thread asking “What are you using to run dev environments safely?” That’s the right question to come out of this.

Consider the shape of a modern AI agent deployment: LLM provider keys for billing and access, tool credentials for SaaS integrations, MCP server access that can reach Slack, GitHub, and production infrastructure, vector databases with proprietary data, memory stores with conversation history. All of it in env vars, .env files, and Kubernetes Secrets. All of it accessible to any process in the runtime.

TeamPCP chose their targets in order: Trivy (security scanner), Checkmarx (code analysis), then LiteLLM (AI API proxy). Each one has elevated trust and broad credential access. The tools that check your code and route your AI requests have the widest blast radius when compromised, because we hand them the keys to everything.

Projects pin application dependencies. They rarely pin the tools that run in CI alongside them. trivy-action@v0.20.0 and trivy-action@latest pointed to different code on March 19. That distinction is what separates “compromised” from “unaffected.”

Anyways, it was fun running into the blogs and reading RCA of this incident. Just wanted to run through it for everyone here.

Thanks for your time and reading this piece.

Best, Ashwin.

Writing - Ashwin Gopalsamy