LiteLLM / TeamPCP - Poisoned Security Scanner → Stolen PyPI Token → .pth Autorun → Cloud Credential Sweep → Kubernetes Lateral Movement
On March 24, 2026 two malicious releases of litellm, versions 1.82.7 and 1.82.8, appeared on PyPI. LiteLLM is the open-source LLM gateway that sits in front of model providers for a very large number of AI applications, and it draws on the order of 95 million downloads a month, so the blast pattern was wide by construction. The releases were live for roughly forty minutes before PyPI quarantined the project, and they still collected tens of thousands of downloads. The interesting part is upstream of the package: the maintainers traced the compromise to Trivy, the vulnerability scanner running inside LiteLLM's own CI/CD pipeline. A threat actor group tracked as TeamPCP had poisoned that scanner, and the scanner step read the pipeline's secrets, which handed over the project's PyPI publishing token. Neither malicious version has a matching tag or release in the LiteLLM GitHub repository: the attacker uploaded straight to PyPI and skipped the release workflow entirely. Installed, the package planted a .pth file that runs on every Python interpreter start, harvested cloud and AI provider credentials, probed Kubernetes for lateral movement, and installed a systemd unit that beaconed for further payloads.
The entry point was not a typosquatted package or a phished maintainer. It was a security tool. Trivy is a scanner teams add to CI precisely to reduce supply chain risk, which means it is installed early in the job, runs with the pipeline's permissions, and is rarely pinned or reviewed with the suspicion applied to application dependencies. Researchers place the scanner compromise in the days immediately before the malicious publish, roughly March 19 to 23, 2026. LiteLLM's own advisory states that the incident appears linked to the broader Trivy compromise. Nothing about the LiteLLM codebase had to be touched for this to work: the attacker only had to be present in a step that the pipeline ran on the maintainers' behalf.
Approximate window: March 19 to 23, 2026, ahead of the March 24 publish
Why this target: Security tooling runs early, with pipeline credentials, and is trusted by definition
Structural problem: The tools that scan the supply chain are themselves part of the supply chain
A CI job is a shared credential space. Every step in it can read the environment, the mounted secret files, and in most configurations the tokens injected for later publish steps. One LiteLLM maintainer stated plainly that the breach originated from Trivy used in their CI/CD pipelines. The scanner step did not need to escalate anything: it simply read what the runner already held and shipped the PyPI publishing token out. This is the same lesson that tj-actions/changed-files taught a year earlier and that Mini Shai-Hulud taught two months later, arriving each time by a different door: a build step is a principal, and its blast radius is every secret in the job.
Mechanism: A trusted CI step reading the job's own environment and secret mounts
No escalation needed: The runner already held the token for the release job
Defense gap: Long-lived publishing tokens present in jobs that also run third-party tooling
At 10:39 UTC on March 24, 2026 version 1.82.7 went up carrying an obfuscated payload inside proxy_server.py. At 10:52 UTC version 1.82.8 followed with a cleaner delivery mechanism, the .pth file described in the next step. PyPI administrators quarantined the project the same day. Because the upload went straight to the index, the repository never showed a matching tag or release, which is the single cheapest detection available here and the one most consumers never check: the artifact and the source of truth disagreed for the entire window. The official LiteLLM Proxy Docker image was not affected, because its dependencies were pinned. Everyone who ran an unpinned pip install or built an image with a floating requirement during those minutes was.
proxy_server.py10:52 UTC: 1.82.8 published, adding the
litellm_init.pth autorun mechanismSame day: PyPI administrators quarantine the project
Tell: No matching git tag or GitHub release for either version
Not affected: The official LiteLLM Proxy Docker image, which pins its dependencies
litellm_init.pth in site-packages executed on every Python interpreter start, whether or not the application ever imported litellmThis is the detail worth carrying away from the whole incident. A .pth file in site-packages is processed by Python's site module at interpreter startup, and lines beginning with import are executed. So the payload ran before any application code, before any import guard, and in every Python process on the machine, including ones that had nothing to do with LLM traffic. A developer who installed the package once and never called it was still executing the payload on every python invocation. One accidental side effect became a useful behavioral indicator: some hosts saw a fork bomb pattern producing extreme lag or memory exhaustion, which is how a number of teams noticed anything at all.
litellm_init.pth in the environment's site-packagesTrigger: Python's
site module at interpreter startup, before application codeScope: Every Python process in that environment, imported or not
Accidental tell: A fork bomb pattern causing severe system lag or memory exhaustion
Hunt command:
find / -name 'litellm_init.pth' 2>/dev/null
An LLM gateway is an unusually rich place to land. By design it holds provider API keys for OpenAI, Anthropic, Azure OpenAI and others, and it usually runs somewhere that also carries cloud credentials so it can reach logging, caching and storage. The payload collected exactly that: environment variables, cloud provider credentials across all three major clouds, model provider keys, SSH private keys, Kubernetes configuration files, and database passwords. Harvested material was encrypted and posted to models.litellm.cloud, a domain chosen to read as project infrastructure in any egress log skimmed by a human.
Exfiltration: Encrypted
POST to models.litellm.cloudSecond C2 domain:
checkmarx.zone, also named after a security vendorWhy the gateway hurts: One host holds provider keys and cloud credentials together
Where the package landed in a container, the payload reached for the service account token that Kubernetes mounts into pods by default and used it to query the API server for what that identity could reach. That default mount is the point: a pod running an LLM proxy has no business enumerating cluster resources, but unless automountServiceAccountToken is disabled the token is sitting on the filesystem waiting to be used. Responders were told to look for pods named node-setup-* in the kube-system namespace, a name chosen to blend into cluster housekeeping.
Activity: Internal API probing to map reachable resources and escalation paths
Cluster IoC: Pods named
node-setup-* in the kube-system namespaceCheap control:
automountServiceAccountToken: false on workloads that never call the API
The last stage made removal of the package insufficient. A systemd unit was written alongside a payload at ~/.config/sysmon/sysmon.py, named to resemble a system monitoring agent, and it polled attacker infrastructure for additional code. Anyone who responded by downgrading litellm and moving on left a live C2 channel running on the host. Full recovery meant hunting the unit and the config directory, rotating everything the host could reach, and rebuilding self-hosted runner images and CI caches, because a poisoned cache reintroduces the artifact on the next build.
~/.config/sysmon/sysmon.py and a matching sysmon.service systemd unitBehavior: Beacon to C2 and poll for additional payloads
C2 domains:
models.litellm.cloud, checkmarx.zoneWhy downgrading is not enough: The unit survives package removal, and CI caches re-serve the artifact
🛡 How to Defend Against This Chain
pip install in a build as a control failure. The single reason the official LiteLLM Proxy image escaped is that it pinned its dependencies. Use a lockfile with hashes (pip install --require-hashes, uv.lock, or Poetry's lock) so a version published in a forty-minute window cannot enter your builds at all. Floating requirements turn every minute of every registry's uptime into your exposure window.litellm_init.pth in site-packages, for ~/.config/sysmon/sysmon.py and its systemd unit, and for pods named node-setup-* in kube-system. Downgrading to 1.82.6 removes the package; it does not remove the persistence the package installed.automountServiceAccountToken: false, scope the workload identity to least privilege, and put provider keys in a secrets manager with short-lived retrieval rather than in the process environment.models.litellm.cloud reads as project infrastructure, checkmarx.zone as a security vendor. Default-deny egress from build runners and AI workloads, allow-list the provider endpoints you actually call, and sweep EDR and DNS logs for both domains across the March 2026 window.