On this page
Treat every model weight, dataset, adapter, and CI action as a supply-chain artifact and enforce provenance, integrity checks, and runtime monitoring on all of them. That single discipline covers the majority of realistic attack paths. To get started now:
- Inventory your AIBOM/ML-BOM. List every model, dataset, tokenizer, adapter, serializer, and CI action your pipelines consume. If you can’t name it, you can’t sign it or monitor it.
- Enforce signing and verified publishing. Require cryptographic signatures on new model artifacts before they enter any registry. Unsigned artifacts from external sources get quarantined, not deployed.
- Add integrity gates to model and dataset loading. Verify checksums at load time in training and inference pipelines. A mismatch halts the job.
- Rotate and scope developer AI keys immediately. Coding assistants, MCP servers, and agent runtimes hold credentials that attackers actively target. Narrow their permissions and rotate on a short cycle.
Pro Tip: Developer-assistant credential rotation is the fastest risk reduction available. Add those config paths to your secret-scanning rules today — before any other control — because the CSA research note documents 2026 campaigns that specifically harvested credentials from AI developer tooling at scale.
Key Takeaways
Securing the AI supply chain requires treating models, datasets, adapters, and tooling as first-class supply-chain artifacts and enforcing provenance, integrity gates, and runtime monitoring across every lifecycle stage.
| Point | Details |
|---|---|
| Inventory first | Generate an AIBOM/ML-BOM covering weights, datasets, tokenizers, adapters, CI actions, and agent skills before applying any other control. |
| Sign and verify artifacts | Require cryptographic signatures on model artifacts and enforce SafeTensors format to eliminate serialization-based code execution. |
| Gate production deployments | Block promotion when any P1 scorecard row scores below 2; auto-fail on unsigned artifacts, pickle format, or missing dataset checksums. |
| Monitor at runtime | Collect inference logs, embedding drift, and canary label accuracy to detect poisoning and model substitution after deployment. |
| Require supplier ML-BOMs | Contractually require ML-BOMs and signing attestations from critical AI vendors; use structured questionnaires to standardize evidence collection. |
| Use Glitchive case studies | Map real failure cases from Glitchive to scorecard rows to prioritize controls based on observed incidents, not theoretical risk. |
Table of Contents
- How AI supply-chain risk differs from traditional software supply chains
- What to inventory: attack surfaces and components for an AIBOM/ML-BOM
- Provenance and integrity controls: what they cover and where they fall short
- How to assess and contractually control AI suppliers and third-party vendors
- Secure MLOps practices for training, packaging, and deployment pipelines
- Testing, verification, and runtime monitoring for AI supply-chain threats
- Incident response and recovery for AI supply-chain incidents
- Operational artifact: AI supply-chain security scorecard and checklist
- A prioritized 30/60/90-day roadmap for security teams
- Why verified failure cases sharpen these controls
- Glitchive’s verified failure case library
- Sources
- FAQ
How AI supply-chain risk differs from traditional software supply chains
Software supply-chain security has a decade of tooling behind it: SBOMs, reproducible builds, package signing, and dependency audits. AI supply chains share the same skeleton but carry three additional failure modes that change how you should think about inventory, verification, and threat modeling.
Data is code-equivalent. In a traditional pipeline, source code is the authoritative artifact. In an ML pipeline, training data shapes model behavior just as directly as code does. A poisoned dataset that passes schema validation can embed a backdoor that no code review will catch. OWASP GenAI documents this explicitly: data poisoning, compromised dependencies, and unsafe serialization are first-class AI supply-chain vectors, not edge cases.
Model opacity makes behavioral verification mandatory. You can diff two versions of source code. You cannot meaningfully diff two versions of a 7-billion-parameter model. A signed model artifact proves it hasn’t been tampered with after signing, but it says nothing about what was baked in during training. Provenance and signing are necessary; they are not sufficient without behavioral testing.
Auxiliary artifacts carry disproportionate risk. ReversingLabs notes that attackers frequently target configuration scripts, loaders, and serialized components rather than the model binary itself, because those files receive less scrutiny. A pickle file that executes arbitrary code on deserialization is a classic example. The Wiz academy frames the AI supply chain as spanning data, development, model artifacts, deployment, and third-party components — each layer a distinct attack surface.
| Dimension | Traditional software | AI/ML pipeline |
|---|---|---|
| Authoritative artifact | Source code | Source code + training data + model weights |
| Verification method | Code review, SAST, SBOM | SBOM + ML-BOM + behavioral tests + data validation |
| Opacity | Low (readable) | High (weights are opaque) |
| Serialization risk | Moderate | High (pickle, safetensors, ONNX loaders) |
| Provenance maturity | Established (Sigstore, SLSA) | Emerging; less tooling available |
Pro Tip: Don’t assume model signing alone covers unsafe serialization. A signed pickle file is a signed execution vector. Require SafeTensors or another format that cannot execute code on load, and treat format enforcement as a hard gate, not a recommendation.
What to inventory: attack surfaces and components for an AIBOM/ML-BOM
An AI Bill of Materials (AIBOM, also called an ML-BOM) extends the traditional SBOM to cover non-code assets. ReversingLabs recommends treating model weights, tokenizers, and serialized components as first-class entries requiring signing and verification, not footnotes.
Minimum provenance fields to capture per item
For every entry in your AIBOM, record: origin URI, cryptographic checksum (SHA-256 minimum), version or commit hash, publisher identity, retrieval method (direct download, registry pull, git submodule), and license.
| Item type | Why it matters | Minimum provenance fields |
|---|---|---|
| Training datasets | Poisoning entry point; shapes model behavior | Origin, checksum, version, curator, retrieval method |
| Model weights | Primary artifact; tamper target | Origin, checksum, signing key, version, publisher |
| Tokenizers | Affects input parsing; can embed malicious vocab | Origin, checksum, version, publisher |
| Adapters / LoRA weights | Fine-tune behavior; often less scrutinized | Origin, checksum, base model hash, publisher |
| Serialization loaders | Code execution risk on deserialization | Format, version, known-CVE status |
| Agent skills / plugins | Execution in agent context; credential access | Origin, checksum, declared permissions, sandbox policy |
| MCP server entries | Persistent agent tool registration | Registry source, checksum, declared capabilities |
| CI/CD actions | Pipeline code; can exfiltrate secrets | Pinned commit SHA, publisher, review status |
| Container images | Runtime environment; dependency bundling | Digest, base image, build manifest |
| External API schemas | Defines agent tool calls; can be poisoned | Schema hash, version, provider identity |
Detection and monitoring targets
Beyond the static inventory, instrument these paths for runtime anomaly detection:
- Developer assistant configuration files (
.continue/,.cursor/, Copilot workspace configs) - MCP server registration events and capability changes
- Package install logs from
pip,npm, andcondain training and inference environments - Model registry pull events, especially for unsigned or unverified artifacts
- CI runner network egress during training jobs
Pro Tip: MCP server registrations are a blind spot in most current inventories. Treat every new MCP entry as a new dependency install: require a checksum, a declared capability list, and a sandbox policy before it reaches a production agent runtime.
Provenance and integrity controls: what they cover and where they fall short
Provenance plus signing plus reproducibility form the foundation of supply-chain integrity. Pair them with behavioral verification and data validation, because a signed artifact proves only that it hasn’t changed since signing, not that it was clean when signed.
Google’s guidance recommends adapting SLSA-style provenance and tamper-evident metadata for AI artifacts, explicitly noting that dataset and model provenance ecosystems are less mature than code provenance. That gap is where most teams are currently exposed.
SBOM/ML-BOM scope and practical limits
CycloneDX and SPDX both support ML-BOM extensions. CycloneDX 1.5+ includes a components type for machine-learning-model, which lets you record training data references, hyperparameters, and evaluation metrics alongside the artifact hash. The practical limit: an ML-BOM records what was declared, not what was verified. A dataset entry with a valid checksum still passes if the checksum was computed against a poisoned version.
| Lifecycle stage | Attack vector | Mitigation | Verification | Priority/effort |
|---|---|---|---|---|
| Data collection | Poisoning, label manipulation | Provenance tracking, canaries | Schema validation, label-distribution checks | High / medium effort |
| Training | Backdoor injection via data | Reproducible training manifests, isolated environments | Behavioral tests, adversarial inputs | High / high effort |
| Model packaging | Artifact tampering, unsafe serialization | Signed artifacts, SafeTensors format | Checksum gates, binary analysis | High / low effort |
| Deployment | Registry substitution, dependency confusion | Verified registries, pinned digests | Digest verification at pull time | High / low effort |
| Runtime | Prompt injection, model extraction | Input/output monitoring, rate limits | Anomaly detection, embedding drift | Medium / medium effort |
Implementation notes
Use OIDC-based publisher tokens (GitHub Actions OIDC, for example) to generate short-lived signing credentials tied to the CI job identity rather than long-lived secrets. Sigstore’s cosign and the Rekor transparency log give you a verifiable, append-only record of signing events for model artifacts. For datasets, record a SHA-256 hash of the full dataset archive at ingestion time and store it in your ML-BOM. Item-level provenance (per-sample lineage) is expensive; start with dataset-level hashes.
Pro Tip: If item-level dataset provenance is cost-prohibitive, embed canary samples with known labels into your training set. A model that misclassifies those canaries at inference time is a signal that the dataset was modified after your baseline hash was recorded.
How to assess and contractually control AI suppliers and third-party vendors
Treat AI vendors as software and data suppliers simultaneously. NIST’s SCRM framework defines supply chain risk management as a systematic process for identifying, assessing, and mitigating supply chain vulnerabilities — that definition applies directly to model providers, dataset vendors, and platform operators.
Supplier evidence to request
Before onboarding any AI vendor or model provider, require:
- An ML-BOM covering model weights, training data provenance, and third-party components
- A training-data provenance statement (sources, curation process, known exclusions)
- CI signing attestations showing that published artifacts were built from declared source
- A documented vulnerability disclosure process with defined SLA timelines
- Audit logs for model registry access and artifact publication events
- Evidence of adversarial testing or red-team exercises against the model
Contractual guardrails
- Audit rights: the right to request artifact hashes and signing attestations on demand
- Data-use restrictions: explicit prohibition on using your data to train shared models
- Breach notification timelines: 72 hours for confirmed supply-chain compromise (align with CISA guidance)
- Scope-limited API keys: vendor-issued keys scoped to specific operations, not admin-level
- Incident remediation obligations: defined SLAs for patching or replacing compromised artifacts
Prioritize suppliers by two axes: how critical their artifact is to your pipeline, and how much access they have to your infrastructure. A model provider whose artifact runs in your production inference cluster ranks higher than a dataset vendor whose data is used only in offline experiments. Use a structured security questionnaire workflow to standardize evidence collection across vendors.
Pro Tip: Require that vendor-issued API keys be scoped to a single model or endpoint. A key that grants access to an entire model hub is a single credential that, if stolen, exposes your entire model inventory.
Secure MLOps practices for training, packaging, and deployment pipelines
Harden the pipeline by minimizing trust boundaries, enforcing least privilege for all identities, and treating model loading exactly like package installation: verify before execute.
Pipeline controls
- Immutable training environments. Use container images with pinned digests for training jobs. A mutable base image means your training environment can change between runs without a record.
- Signed datasets and artifacts. Sign dataset archives at ingestion and model artifacts at packaging. Require signature verification before any downstream job consumes them.
- Isolated build accounts. Training jobs run under service identities with no access to production systems. CI runners that build models have no credentials for production registries.
- Gated model registries. Models enter a staging registry first. Promotion to production requires a passing scorecard gate (see Section 9).
- Reproducible training manifests. Record the exact dataset hash, framework version, and hyperparameters used for each training run. This is your forensic baseline if a model behaves unexpectedly.
Dependency management
Pin all Python and npm dependencies to exact versions with hash verification (pip install --require-hashes, npm ci). Use a private mirror or artifact proxy (Artifactory, AWS CodeArtifact) rather than pulling directly from PyPI or npm at training time. The CSA research note documented campaigns in early 2026 that compromised over 1,000 packages across npm, PyPI, and GitHub, stealing credentials from multiple organizations. A private mirror with a verified snapshot breaks that attack path.
Scan dependencies for known-bad serialization formats. Pickle deserialization vulnerabilities are well-documented in academic research: a malicious pickle file executes arbitrary Python on load, making it a reliable code-execution vector inside your training environment.
Developer-tool controls
Inventory every coding assistant, notebook integration, and CI agent in use. Restrict their network access to declared endpoints only. Rotate provider keys on a 30-day cycle. Isolate CI runners that execute model training from runners that have production access.
Pro Tip: Treat Jupyter notebooks as CI artifacts, not scratch pads. Store them in version control, lint them with nbstripout before commit, and run them in isolated environments with no production credentials.
Testing, verification, and runtime monitoring for AI supply-chain threats
Layer pre-deployment artifact analysis, behavioral red-teaming, and runtime anomaly detection. No single layer catches everything; the combination does.
Pre-deployment test plan
- Static artifact checks. Verify checksums against ML-BOM entries. Scan serialized files for unsafe formats (pickle, joblib). Run binary analysis on model weights to detect appended payloads.
- Dataset validation. Check schema conformance, label-distribution stability versus a known-good baseline, and presence of canary samples. Flag any dataset where canary labels have shifted.
- Behavioral tests. Run a fixed adversarial input suite against every new model version. Compare outputs to the previous signed version. Regressions on safety-critical inputs block promotion.
- Dependency audit. Re-scan all pinned dependencies against current CVE feeds before each training run. Block installs of packages flagged for typosquatting or dependency confusion.
- Agent skill detonation. For any new agent skill or MCP entry, run it in a sandboxed environment instrumented to record API calls, file-system writes, and network connections. Use decoy credentials to detect exfiltration attempts.
Red-team practices for AI supply chains
- Prompt-injection red-team. Test whether injected instructions in training data, retrieval results, or tool outputs can override system behavior.
- Data-poisoning simulation. Introduce a small number of mislabeled samples into a test dataset and verify that your canary detection catches the shift before training completes.
- Skill and agent detonation. CSO Online documented trojanized agent skills reaching millions of installs. Detonation testing with decoy credentials is the only reliable way to detect environment-aware payloads that behave normally under static analysis.
Runtime monitoring signals
| Signal | What it detects | Collection method |
|---|---|---|
| Input distribution drift | Data poisoning, adversarial probing | Statistical tests on inference inputs |
| Embedding drift | Model behavior shift post-deployment | Cosine similarity vs. baseline embeddings |
| Anomaly scores | Out-of-distribution inputs | Isolation Forest or similar on inference logs |
| Canary label accuracy | Dataset tampering | Scheduled inference on held-out canary set |
| Registry pull events | Unauthorized artifact substitution | Registry audit logs, SIEM alerts |
Pro Tip: Automate re-scans of your dependency snapshot against updated CVE feeds on a weekly schedule. Threat intelligence on package-repo compromises moves fast; a scan that was clean at training time may flag a known-bad package within days.
Incident response and recovery for AI supply-chain incidents
When a supply-chain compromise is suspected: triage scope, preserve evidence, revoke and isolate, then decide between rollback and retrain based on whether the compromise is in the artifact or the training data.
Incident runbook
Immediate (0–4 hours):
- Identify the suspected artifact (model version, dataset slice, dependency, or CI action) and its ML-BOM entry.
- Preserve evidence: snapshot artifact hashes, CI build logs, registry pull logs, and signing attestation records before any remediation action.
- Revoke publisher tokens and API keys associated with the compromised artifact or pipeline.
- Block the artifact in your registry (quarantine, do not delete — deletion destroys forensic evidence).
- Notify your security team and, if the artifact was shared with downstream consumers, notify them immediately.
Containment (4–24 hours):
- Remove the compromised artifact from all active inference endpoints.
- Rotate all credentials that the compromised CI job or agent had access to, including developer assistant keys.
- Isolate affected training environments from production networks.
- Check whether the artifact was promoted to production and, if so, assess downstream data exposure.
Recovery:
- Rollback path: if the compromise is in the artifact (tampering after training), roll back to the last signed artifact whose provenance chain is intact.
- Retrain path: if the compromise is in the training data (poisoning), rolling back the artifact is insufficient. Identify the poisoned dataset slice using your canary records, remove it, and retrain from a clean baseline.
- Validate the replacement artifact against the full behavioral test suite before re-promotion.
- Conduct a post-incident review and update your ML-BOM and scorecard with findings.
Compliance notifications: align breach notification timelines with your contractual SLAs and applicable regulatory requirements. Document the artifact hash, the signing attestation, and the timeline of discovery for any regulatory filing.
Pro Tip: Never delete a compromised artifact from your registry before forensic analysis is complete. Quarantine it with an access-control block. The artifact hash and build provenance are your primary evidence for determining blast radius.
Operational artifact: AI supply-chain security scorecard and checklist
Use this scorecard in security reviews, vendor assessments, and pre-production gates. Score each row 0–3: 0 = not implemented, 1 = partial, 2 = implemented without automation, 3 = implemented and automated. A score below 2 on any Priority 1 row is an automatic gate failure.
Automated pre-approval gate: conditions
Auto-fail (block promotion):
- Model artifact has no valid cryptographic signature
- Serialization format is pickle or joblib without an explicit exemption
- Dataset checksum does not match ML-BOM entry
- Any CI action is not pinned to a commit SHA
- Agent skill has not passed detonation testing
Require before promotion:
- Behavioral test suite passes with no regression on adversarial inputs
- All P1 scorecard rows score 2 or higher
- Supplier ML-BOM on file for any externally sourced artifact
Pro Tip: Run the scorecard as a structured checklist in your pull-request template for model promotions. A checkbox that blocks merge is harder to skip than a Confluence page that nobody reads.
A prioritized 30/60/90-day roadmap for security teams
30-day goals: inventory and quick wins
- Generate an initial AIBOM/ML-BOM for all models and datasets currently in production. Use CycloneDX 1.5+ or SPDX 2.3 with ML extensions.
- Rotate all developer assistant and coding-agent API keys. Scope replacements to the minimum required permissions.
- Add developer assistant config paths and MCP registration files to your secret-scanning rules (GitHub Advanced Security, GitLeaks, or equivalent).
- Enforce SafeTensors format for all new model artifacts entering your staging registry. Quarantine existing pickle-format artifacts for review.
- Implement checksum verification for dataset loads in at least one production training pipeline.
Milestone: every production model has an ML-BOM entry; no new pickle artifacts enter staging; developer keys are rotated and scoped.
60-day goals: enforce and instrument
- Require OIDC-based publisher tokens for all CI jobs that sign or publish model artifacts.
- Deploy dataset validation checks (schema, label distribution, canary samples) on your highest-criticality training pipeline.
- Stand up a runtime telemetry pipeline: collect inference logs, input distributions, and embedding snapshots for at least one production model.
- Run a first detonation test on all agent skills and MCP entries currently in production.
Milestone: CI signing is enforced; canary detection is live on one pipeline; all agent skills have a detonation test result on file.
90-day goals: gate and require
- Require ML-BOMs from all critical suppliers as a contractual condition. Use a security questionnaire process to standardize evidence collection.
- Gate production model promotions with the scorecard: P1 rows must score 2 or higher to pass.
- Integrate red-team findings (prompt injection, data-poisoning simulation) into your release criteria. A model that fails adversarial behavioral tests does not ship.
- Conduct a tabletop exercise using the incident runbook from Section 8.
Milestone: scorecard gate is enforced in CI; supplier ML-BOM requirements are in at least two vendor contracts; one full red-team cycle is complete.
Pro Tip: Don’t wait for 90 days to run the scorecard. Score your current state in week one. The gap between your actual scores and the P1 thresholds is your prioritized backlog.
Why verified failure cases sharpen these controls
The controls in this guide were shaped by patterns that appear repeatedly in real AI system failures: unsafe serialization exploited in model-loading pipelines, developer assistant credentials harvested through compromised package installs, and MCP server registrations used for persistent access. Generic best-practice lists miss the specificity that comes from documented incidents.
Glitchive’s editorial approach is to start with the failure record, not the framework. When a control appears in this guide — detonation testing for agent skills, canary samples in training data, OIDC publisher tokens for CI — it is because the absence of that control shows up in real postmortems. The scorecard categories map directly to observed failure modes, not to a theoretical attack taxonomy.
That specificity matters for practitioners. A checklist derived from real incidents tells you which rows are genuinely high-priority because they have been exploited, not just theoretically possible. It also tells you what the fix looked like in practice, which is harder to find in standards documents.
Glitchive’s verified failure case library
Glitchive’s case library gives security teams and ML engineers something standards documents can’t: a searchable record of what actually went wrong, how it was discovered, and what fixed it. Each case documents the incident, contributing factors, technical analysis, and the specific remediation applied, with permanent citable URLs and fully sourced references.
For teams building or buying AI components, the case library serves three concrete purposes. First, vendor due-diligence: use cases like the coding agent that wiped a production database to stress-test a vendor’s claims about agent isolation and credential governance. Second, incident postmortem templates: cases provide a structured format for your own postmortems, including the failure mechanism, contributing factors, and fix. Third, training exercises: real failure scenarios make tabletop exercises specific and credible in a way that hypotheticals don’t.
The Glitchive case library is free to search. Start with the cases most relevant to your current pipeline — coding agents, model serving, or data pipelines — and map each failure mode to the corresponding scorecard row.
Sources
These primary sources back the controls and frameworks referenced throughout this guide:
- Supply chain risk management (NIST glossary)
- LLM03:2025 Supply Chain - OWASP Gen AI Security Project
- AI Supply Chain Security Solution | ReversingLabs
FAQ
What makes AI supply-chain security different from standard software supply-chain security?
AI supply chains include training data and model weights as authoritative artifacts alongside code, and model behavior can be manipulated through data poisoning without any code change. Auxiliary files like serializers and adapters also carry execution risk that traditional SBOM tooling doesn’t cover.
What should an ML-BOM include that a standard SBOM doesn’t?
An ML-BOM should include model weights, training dataset references with checksums, tokenizers, adapters, serialization formats, agent skills, and external API schemas — all with origin, checksum, version, and publisher fields. Standard SBOMs cover only code dependencies.
How do you detect data poisoning before a model reaches production?
Embed canary samples with known labels into your training dataset and verify that the model classifies them correctly after training. A canary failure signals that the dataset was modified after your baseline checksum was recorded.
What is the fastest risk reduction available for AI supply-chain threats?
Rotating and scoping developer assistant and coding-agent API keys is the highest-impact quick win. The CSA research note documents 2026 campaigns that specifically targeted these credentials through compromised packages and extensions.
Which serialization formats are safe to use for model artifacts?
SafeTensors is the recommended format because it cannot execute code on deserialization, unlike pickle or joblib. Treat any model artifact in pickle format as requiring an explicit security exemption and binary analysis before use.