On this page · 12 sections
- What happened on 21 October 2026
- Affected artefacts and versions
- Indicators of compromise
- Triage: what to do with what you find
- Remediation, in order
- What to rotate
- How to verify LiteLLM images now
- AI gateway supply-chain controls that would have helped
- GPUStack CVE-2026-102895: the second AI-infrastructure case
- US and UK obligations, and the dates that matter
- How eCorpIT can help
- References
Summary. On 21 October 2026, LiteLLM's own release pipeline published a malicious litellm==1.107.0 to PyPI, with all seven wheels live from 11:39 to 13:23 UTC, and pointed the v1.107.0, latest, main-latest and main-stable Docker tags at a poisoned image until as late as 14:20 UTC. LiteLLM's root-cause report of 23 October says an administrative key committed to a public repository in May gave the attacker a GitHub organisation owner's token, which was used to rewrite the release workflow and build from a lookalike fork, Berri-AI/litellm. The wheels drop cook.pth, cook.py and sitecustomize.py into site-packages, so every Python start runs a credential stealer and a remote-code loader; on a proxy with DATABASE_URL set, it tries to dump every stored provider key. CERT-In advisory CIAD-2026-0053 of 23 October says to isolate hosts, preserve evidence, rebuild from clean images and rotate cloud, CI/CD, AI tool and LLM provider credentials. The poisoned images carry a valid signature from LiteLLM's previous cosign key, so a signature check alone passed them. The incident has no CVE; CVE-2026-40217, which CISA added to its exploited list on 18 November, is a separate flaw. Our verdict: hunt for the indicators today, rotate from a clean machine, then pin by digest, restrict gateway egress and delay new releases.
About this article. For AI platform, DevOps and security teams in the US and UK that self-host LiteLLM as an SDK or proxy, or run GPUStack for GPU inference. We checked LiteLLM's incident post, root-cause report and signing announcement, the Upwind report in GitHub issue #47826, PyPI and OSV records, CERT-In CIAD-2026-0053, the CVE, NVD and CISA KEV records, GPUStack's advisories and releases, and ICO guidance on 21 November 2026. Indicators are attributed to their source. This is defensive guidance, not legal advice and not a substitute for incident response on your own systems; no vendor paid for inclusion. Researched with AI assistance and checked against the primary sources listed in the references.
What happened on 21 October 2026
LiteLLM's canonical root-cause post traces the attack back five months. On 27 May, a configuration file holding an administrative key for an internal deployment of LiteLLM Agent Platform was committed to the public LiteLLM-Labs/lite-harness repository. A follow-up commit deleted the file about three hours later, but nobody revoked the key, so it still worked more than four months later. The timeline below is taken from that post.
| Time (UTC) | Event |
|---|---|
| 27 May, 18:53 | Agent Platform admin key committed to a public repository |
| 10 October, 21:27 | Attacker's API call returns a GitHub organisation owner's CLI token |
| 10 October, 22:11 to 22:12 | Throwaway CI branches post repository secrets, including two more maintainer tokens, to webhook.site |
| 21 October, 10:08 | Branch protection rejects a release-workflow change; the owner token overrides it |
| 21 October, 10:46 | Staging build dispatched from lookalike fork Berri-AI/litellm, commit 20535d8f |
| 21 October, 11:11 | A second stolen token approves the production gate |
| 21 October, 11:39 to 11:40 | Seven malicious 1.107.0 wheels published to PyPI |
| 21 October, 11:46 | Docker Hub and GHCR images published |
| 21 October, 12:06 | An Upwind researcher opens issue #47826 identifying the compromise |
| 21 October, by 13:23 | PyPI quarantines 1.107.0 |
| 21 October, 14:20 | All affected Docker Hub tags removed or repointed |
After an earlier incident in March, LiteLLM required releases to be approved by a special account locked to a device in its office. That rule had been relaxed earlier the same week so any two engineers could trigger and approve a release. The attacker held tokens for three engineers and used two of them to satisfy it. The official v1.107.0 source tag does not contain the malicious files; the artefacts were built from the attacker's fork.
Affected artefacts and versions
The artefacts, windows and image index digests in this table come from LiteLLM's incident post.
| Artefact | Affected tags or version | Window (UTC, 21 October) | Image index digest |
|---|---|---|---|
| PyPI litellm | 1.107.0, all seven wheels | 11:39:51 to 13:23 | Not applicable |
| Docker Hub litellm/litellm | v1.107.0, latest, main-latest, main-stable | 11:46:56 to 14:20 | sha256:1bbcb1443bc962e45f5aad53fe3d2c30600e0379715fb6c664cdd3ac126c339e |
| GHCR ghcr.io/berriai/litellm | Same four tags | 11:46:56 to 13:51 | sha256:1bbcb1443bc9... |
| GHCR ghcr.io/berriai/litellm-database | Same four tags | 11:46:56 to 13:51 | sha256:c00f9d92c427... |
| GHCR ghcr.io/berriai/litellm-non_root | Same four tags | 11:46:56 to 13:51 | sha256:7b00e91cd2fe... |
LiteLLM says all earlier versions are safe and that enterprise customers on its private registry were not affected. Upwind's report lists 1.106.2, 1.106.1, 1.106.0, 1.105.3 and both 1.107.0 release candidates among the versions it checked and found clean.
You did not have to choose 1.107.0 to get it. Any unpinned pip install litellm in the PyPI window, or any container that pulled one of the four floating tags in the Docker window, received the payload. CERT-In adds that the files run when Python initialises, so other Python applications in the same environment can be exposed even if they never import LiteLLM.
Indicators of compromise
The values below come from LiteLLM's two posts and from Upwind's report and its follow-up comments in issue #47826, which LiteLLM adopted as its status channel.
| Type | Value | Source |
|---|---|---|
| Package | litellm==1.107.0, litellm-1.107.0.dist-info | LiteLLM |
| Files in site-packages | cook.pth, cook.py, unexpected or modified sitecustomize.py | LiteLLM, CERT-In |
| Image index digest | sha256:1bbcb1443bc962e45f5aad53fe3d2c30600e0379715fb6c664cdd3ac126c339e | LiteLLM |
| Platform manifests | amd64 sha256:bd1aeee419ea36da9aa2b657288da4101887be468f020ac63d22ba6ed84a3ea5; arm64 sha256:56b74573... | Upwind |
| Lock file | /tmp/remote_code_loader.lock, held by a detached Python daemon | LiteLLM |
| Exfiltration marker | /tmp/.pc_once_72129118c59f5636c0bb, written only after a successful upload | Upwind |
| Environment and temp | PRIVCOLLECT_ACTIVE=1; /tmp/.pc_sq_ and /tmp/.pc_e_ directories | Upwind |
| Exfiltration endpoint | POST http://www.litellm.dev/healthcheck (not the project's domain, litellm.ai) | Upwind |
| HTTP header | X-Token: XWiA8gZNa7NcX2XRoAEb | Upwind |
| Dead-drop | GitHub user letters9813, repository letters9813/pythonclient-test, file domain.txt; follow-on requests to /codepy | Upwind, CERT-In |
| Release infrastructure | Lookalike organisation Berri-AI; fork commit 20535d8f; IPs 69.48.229.143 and 199.91.221.39 | LiteLLM |
Upwind's report also lists SHA-256 hashes for all seven wheels and the source distribution; take those from issue #47826. In its 21 October report, Upwind said the dead-drop file had been empty since 20 October, leaving the remote-code stage dormant but able to be re-armed with one commit.
CERT-In names the stealer privcollect and lists its targets: browser credentials and cookies, SSH and GPG keys, AWS, Azure and Google Cloud credentials, environment variables and CI/CD secrets, repository tokens, and tokens for AI development tools including Cursor and GitHub Copilot. Upwind adds that the payload checks stolen AWS role credentials with STS GetCallerIdentity.
Triage: what to do with what you find
| If you find | Treat it as | Next step |
|---|---|---|
| 1.107.0 in a lock file, pip cache, mirror or SBOM, with no record it was installed | Possible exposure | Purge the artefact everywhere, then check build and deploy logs for any install |
| A host or container that ran 1.107.0 or a listed digest | Compromised | Isolate, preserve evidence, rebuild, rotate everything the host could reach |
| The /tmp/.pc_once marker file | Confirmed exfiltration | Full credential rotation and cloud audit-log review |
| No marker file on a host that ran the package | Still compromised | A missing marker proves nothing; containers and temp cleanup remove it |
| A floating tag pulled on 21 October with no digest record | Unknown | Check registry pull and admission logs for the window; assume exposure if you cannot rule it out |
The marker rule comes from LiteLLM's incident post.
Remediation, in order
LiteLLM's first instruction is the one most people get wrong: do not start by running pip. Any Python start in an affected environment runs cook.pth and sitecustomize.py again, and that includes pip. Work from the shell, and image the host first if you need forensics.
# Run from a shell, not from Python. Report only; collect evidence before deleting.
find / -xdev \( -name 'cook.pth' -o -name 'cook.py' -o -name 'litellm-1.107.0.dist-info' \) 2>/dev/null
find / -xdev -name 'sitecustomize.py' 2>/dev/null # compare each hit with a known-good copy
ls -la /tmp/remote_code_loader.lock /tmp/.pc_once_72129118c59f5636c0bb 2>/dev/null
# Kubernetes: look for the poisoned digests in running pods
kubectl get pods -A -o 'custom-columns=NS:.metadata.namespace,POD:.metadata.name,IMAGE:.status.containerStatuses[*].imageID' \
| grep -E '1bbcb1443bc9|c00f9d92c427|7b00e91cd2fe|bd1aeee419ea|56b74573'
A sitecustomize.py is not proof on its own: Python's documentation says administrators typically create one in site-packages for site-specific setup, which is why CERT-In asks for unexpected or modified copies.
- Contain. Isolate suspect hosts and containers, preserving logs and forensic evidence before you change anything.
- Remove the payload. From the shell, delete
cook.pth,cook.py,sitecustomize.pyand matchingcook*.pycandsitecustomize*.pycfiles in__pycache__from every affected site-packages, then kill the detached daemon holding/tmp/remote_code_loader.lock.
- Rebuild, do not clean in place. Rebuild virtual environments and images from scratch on a verified release, and remove the bad artefacts from caches, internal mirrors and container registries.
- Rotate from a clean machine. Use the table in the next section.
- Review cloud audit logs. LiteLLM tells AWS users to check CloudTrail for
GetCallerIdentityfrom the instance role around install time, then for use of that role's credentials from outside their infrastructure. Run the equivalent sign-in and audit-log review in Azure and Google Cloud.
- Watch provider usage. CERT-In lists proxy database access, upstream provider usage and repository activity as places to look for misuse.
What to rotate
The rows below combine LiteLLM's rotation list, CERT-In's mandatory rotation list and Upwind's analysis of what the payload reads.
| Credential | Why it is exposed | Source |
|---|---|---|
| Every provider key stored in the proxy database | Payload dumps the credentials and model tables when DATABASE_URL is set | LiteLLM, Upwind |
| LiteLLM master key and virtual keys | Readable from the host and database | LiteLLM |
| LiteLLM salt key, following the relevant key-management procedure | Named in the mandatory rotation list | CERT-In |
| DATABASE_URL password | Present in the environment | LiteLLM |
| Instance, task or pod cloud role; any static cloud keys | Stealer reads instance metadata and validates AWS credentials | LiteLLM, Upwind |
| CI/CD secrets, repository and registry tokens, Vault tokens | Collected from files and process environments | CERT-In |
| SSH and GPG keys, browser sessions on developer machines | Collected from home directories and browser profiles | CERT-In |
| Cursor, GitHub Copilot and other AI tool tokens | Named targets of the stealer | CERT-In |
| Any secret in the environment of any process on the host | Payload reads every readable /proc/*/environ | LiteLLM, Upwind |
Worked example. Take a 300-person fintech that runs the litellm-database image from GHCR on Kubernetes, tracks main-stable, and stores 8 upstream provider keys, 1 master key and 60 virtual keys in the proxy database. If any pod pulled that tag between 11:46 and 13:51 UTC on 21 October (a scheduled redeploy, a node scale-up or a restart with an always-pull policy), it ran the payload at startup and kept running that image until replaced. The rotation list is 8 provider keys, the master key, 60 virtual keys, the salt key, the database password, the pod's cloud role and the Kubernetes service account token: 8 + 1 + 60 + 1 + 1 + 1 + 1 = 73 credentials, before every secret mounted into the namespace, CI and developer machines are counted. Rotate provider keys first, because they carry direct spend and data access.
How to verify LiteLLM images now
LiteLLM believes the private half of its previous cosign key was exposed, so it replaced the key on 23 October and signed v1.103.8, v1.104.6, v1.105.4 and v1.106.3 with the new one on 24 October. The new cosign.pub has the SHA-256 fingerprint e3fe3e08c44b16407fe0b37c43b4f8b603b3a0f4ee74c29e647bc022d762c2c0. Trust a signature from the old key only on an image published before the 1.107.0 release on 21 October, because the compromised 1.107.0 images also carry a valid signature from it.
LiteLLM's signing announcement sets out which method covers which release. Everything published from 15 November 2026, starting with v1.111.0-dev.3 and including patch releases on older lines, is signed keyless through Sigstore and recorded in the public Rekor log; the Helm chart is signed the same way. Releases from 24 October to 15 November verify with the new key, and releases up to v1.106.2 with the old key.
cosign verify \
--certificate-identity https://github.com/BerriAI/project-releaser/.github/workflows/release.yml@refs/heads/main \
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
ghcr.io/berriai/litellm:<release-tag>
Key-based verification with cosign verify --key does not work on keyless releases, so update any CI gate or admission policy that pins cosign.pub before you deploy one. LiteLLM's Docker image security guide still describes only the key-based method, so follow the announcement. The guide also says main-latest is no longer updated, main-stable is deprecated, and latest should be avoided in production. The highest stable version on PyPI on 21 November is 1.110.2.
AI gateway supply-chain controls that would have helped
A gateway concentrates every model credential you own in one process, so its supply chain deserves the controls you would put on an identity provider. Our enterprise AI gateway comparison weighs LiteLLM against the alternatives with this incident in view. The table draws on LiteLLM's two posts, AWS's IMDSv2 documentation and the incident evidence above.
| Control | What it would have changed on 21 October | Effort |
|---|---|---|
| Pin images by digest and Python packages by hash | Floating main-stable and latest pulls would not have picked up the new build | Low |
| Delay adoption of new releases by a few days | LiteLLM credits Upwind with catching the release within 20 minutes; a short cooldown keeps you out of that window | Low |
| Verify signatures and check digests against known-bad lists | A signature check alone passed 1.107.0; pairing it with a digest denylist would not | Low |
| Default-deny egress from gateway hosts | Blocks uploads to www.litellm.dev and dead-drop polling; a gateway needs model APIs, its database and telemetry | Medium |
| Keep the node's role away from containers | IMDSv2 with the default hop limit of 1, or workload identity, stops a container lifting the node role; scope task and pod roles tightly | Medium |
| Hold provider keys outside the gateway database where possible | Short-lived cloud identity or a secrets manager shrinks what one dump yields | Medium |
| Scan for and revoke committed secrets | The root cause was an admin key left valid for more than four months | Low |
| Redact request and response bodies in gateway logs | LiteLLM's first post said Claude Code can, in specific situations, put a GitHub CLI token in a response body that then lands in logs | Low |
Our npm supply-chain hardening guide makes the same point from the August keyv worm: a valid signature proves which pipeline built an artefact, not that its source was clean. For runtime controls in front of the model, see our AI agent guardrails comparison.
GPUStack CVE-2026-102895: the second AI-infrastructure case
GPUStack is an open-source GPU cluster manager for AI model serving. The CVE record, published on 4 November by GitHub as CNA, says that before 2.2.0 GPUStack treated a request from a loopback address as the built-in administrator when force_auth_localhost was false, which was the default. A same-host reverse proxy, or a client-supplied X-GPUStack-Real-IP header, could make a remote request look local. The result was unauthenticated admin access, persistent admin API keys, disclosure of worker and cloud-provider secrets, and code execution on workers through custom inference backends. Exploitation was observed against a 2.1.1 deployment. The table draws on the CVE and NVD records, GPUStack's advisories and releases, and the KEV entry.
| Item | Detail |
|---|---|
| CVE and score | CVE-2026-102895, CVSS 3.1 base 9.8 (Critical) |
| Affected | gpustack before 2.2.0 |
| Fixed release | 2.2.0, released 18 June 2026; advisory GHSA-gj9m-f8ph-pmhr published 31 July 2026 |
| Later security releases | 2.2.2 (24 July) fixed two flaws; 2.3.0 (23 October) fixed nine more; 2.3.1 (11 November) fixed ten, two rated critical |
| Latest stable | 2.3.1, 11 November 2026 |
| CISA KEV | Added 9 November 2026; federal due date 12 November 2026 |
Stopping at 2.2.0 is no longer enough. The 2.2.2 release fixed missing authorisation on cluster endpoints and a SAML callback bypass, and the 2.3.1 advisories include a critical flaw where model configuration evaluation runs repository Python on the server in every version from 2.0.0 up to 2.3.1. Upgrade to 2.3.1. If you cannot upgrade today, GPUStack's advisory gives three workarounds for CVE-2026-102895: set force_auth_localhost=true, strip any client-supplied X-GPUStack-Real-IP header at the edge, and keep the API off untrusted networks behind a VPN or edge authentication. After patching, review admin users, API keys and custom inference backends you did not create, since the reported attack used attacker-created backends.
US and UK obligations, and the dates that matter
US. KEV due dates bind federal civilian executive branch agencies under BOD 26-04, which CISA issued on 10 June 2026 and which replaced BOD 22-01. Both AI-infrastructure KEV entries point to BOD 26-04 and CISA's forensics triage requirements, and CISA marks forensic triage as required for the GPUStack entry. GPUStack's date, 12 November, has passed. Private companies are not bound by these dates, but they are a sound benchmark for exploited flaws.
CVE-2026-40217 is not the 1.107.0 compromise. It is a sandbox escape in the custom-code guardrail test endpoint, published in April 2026, rated 8.8 in its NVD record, which lists for versions 1.81.8 up to but not including 1.83.10. CISA added it on 18 November with a due date of 2 December 2026. Our CVE-2026-40217 patch guide covers that flaw.
The 1.107.0 malware has no CVE and no LiteLLM GitHub security advisory on 21 November. OSV tracks it as malicious-package record MAL-2026-2144, alias PYSEC-2026-2, which lists 1.82.7, 1.82.8 and 1.107.0. Scanners that match only CVE or GHSA identifiers may miss it, so hunt by version, file and digest.
UK. If stolen credentials exposed personal data, for example customer data reachable through a stolen cloud role or prompts stored in the proxy database, the ICO says you must assess the risk to people and, where a risk is likely, notify it as soon as possible and where feasible within 72 hours. It also suggests telling the NCSC when a malicious actor caused the breach.
Offshore teams. If an Indian subsidiary or delivery partner ran an affected host, CERT-In's Directions of 28 April 2022 require listed incidents to be reported within 6 hours of noticing them. Our guide to CERT-In's October 2026 AI agent advisory covers that rule.
Dates to work to:
- Today, 21 November. Run the hunt, purge 1.107.0 from mirrors and caches, and rotate anything a confirmed host touched.
- 2 December 2026. CISA's due date for CVE-2026-40217, 11 days from today.
- Already past. GPUStack's 12 November due date. Upgrade to 2.3.1 now if you have not.
How eCorpIT can help
eCorpIT is a Gurugram-based technology organisation, founded in 2021, assessed at CMMI Level 5 and MSME certified, with senior-led engineering teams working across AWS, Microsoft and Google platforms. Our cybersecurity services team can run the 1.107.0 and GPUStack hunts across hosts, images and mirrors, sequence credential rotation and review cloud audit logs for misuse. Talk to us at /contact-us/.
Last updated: 21 November 2026.