Intranet CI keeps returning PASSED on an image while the same image on the public network shows a pile of CRITICAL findings—do not loosen --fail-on first. Check whether the Trivy vulnerability DB is months stale, or whether an offline import left you pointing at an empty or half-written cache.
Takeaway: SCA (software composition analysis) reads Trivy’s local vulnerability DB. On networked hosts run
update-trivy-db.sh. On air-gapped hosts import an offline bundle or land a manually pulled OCI DB as a whole directory. Always back up the cache directory before updating; on failure, roll the whole directory back, then rundoctor. Engine binary health belongs in the offline engines / doctor article (see Reference). Below focuses on DB update and rollback.
1. Where the DB lives
| Item | Guidance |
|---|---|
| Default cache root | $ANGUS_SECURITY_HOME/trivy-db (override with TRIVY_CACHE_DIR) |
| Who reads it | Trivy (sca: deps / artifacts / images) |
| If you skip updates | New CVEs never appear → false-green gates; the console may also warn on age |
ANGUS_SECURITY_HOME is the tools root (bin/, engines/, rules/). Paths follow the release README.
Typical layout (from tools/README; scans read db/—do not drop trivy.db under rules/):
trivy-db/
├── db/trivy.db # CVE store (BoltDB)
├── db/metadata.json # metadata; doctor checks schema
├── browse.sqlite # console CVE browser index; not required for scan path
├── java-db/ # bare-JAR → coordinate fingerprints; not a CVE table
├── policy/ # optional
└── fanal/ # scan cache; safe to delete
2. Binary health vs DB freshness
| Check | How | Notes |
|---|---|---|
| Engines not stub | "$ANGUS_SECURITY_HOME/bin/doctor.sh" or angus-security tools doctor |
Stub/missing → SCA untrustworthy |
| DB schema | doctor vs db/metadata.json |
Schema mismatch can yield empty results |
| Age | Console Rules → Vulnerability DB, or service age metrics | Stale DB can still “succeed with 0 vulns” |
Freshness-related service settings (exact keys follow your deploy docs; turn auto-refresh off on air-gapped hosts):
| Intent | Typical default | Ops meaning |
|---|---|---|
| Auto-refresh | Often on (e.g. daily update-trivy-db.sh) |
Must disable on air-gap or jobs keep failing |
| Min refresh interval | Hours | Avoid hammering pulls |
| Max age | Days | Banner / advisory Finding; fail-scan-on-stale-style flags may mark SCA PARTIAL |
Confirm key names and defaults against your current deploy docs—do not treat one YAML revision as a fixed contract.
3. Networked hosts: direct update
"$ANGUS_SECURITY_HOME/bin/doctor.sh"
# or
angus-security tools doctor
Backup (required before production gates):
TS=$(date +%Y%m%d%H%M%S)
CACHE="${TRIVY_CACHE_DIR:-$ANGUS_SECURITY_HOME/trivy-db}"
cp -a "$CACHE" "$CACHE.bak-$TS"
Update:
"$ANGUS_SECURITY_HOME/bin/update-trivy-db.sh"
# CVE only, skip Java index:
# "$ANGUS_SECURITY_HOME/bin/update-trivy-db.sh" --skip-java-db
The script needs a real trivy binary; a stub fails immediately—that is expected. If trivy is missing, run install-engines.sh first (see Reference: CLI first scan).
Acceptance:
- doctor no longer WARNs/FAILs on vulndb items (per script output).
- Rescan the same canary image known to contain an older CVE: Finding volume should match a healthy-DB baseline, not drop to zero.
- For JVM bare-JAR resolution, do not leave
--skip-java-dbon by mistake.
4. Air-gap: offline bundle
Prepare a bundle on a networked Linux host (export-offline-bundle.sh / prepare-linux-offline-bundle.sh—names follow the release), copy it in, then:
# Flags follow script --help (common shape)
"$ANGUS_SECURITY_HOME/bin/import-offline-bundle.sh" --bundle /path/to/bundle.tar.gz
"$ANGUS_SECURITY_HOME/bin/doctor.sh"
Alternatively: update-trivy-db.sh on the builder, export with Trivy DB options, or rsync -a trivy-db/ to production.
After import: disable auto-refresh → doctor → canary sca. Do not keep blocking production gates on a failed import.
5. Air-gap alternate: manual OCI / ORAS
When the host still has a stub, or ghcr.io is blocked, pull OCI artifacts (not docker pull images) on a jump box, then copy in. Upstream refs can change—confirm against Aqua Trivy docs / your release notes:
| DB | Common OCI ref | Unpack to |
|---|---|---|
| CVE | ghcr.io/aquasecurity/trivy-db:2 (or documented mirror) |
trivy-db/db/ (trivy.db + metadata.json) |
| Java index | ghcr.io/aquasecurity/trivy-java-db:1 |
trivy-db/java-db/ |
With a real trivy on the jump box:
CACHE=$(mktemp -d)
trivy image --download-db-only --cache-dir "$CACHE"
trivy image --download-java-db-only --cache-dir "$CACHE" || true
tar -czf trivy-db-manual.tgz -C "$CACHE" db java-db
Land on the intranet:
DEST="${TRIVY_CACHE_DIR:-$ANGUS_SECURITY_HOME/trivy-db}"
mkdir -p "$DEST"
tar -xzf trivy-db-manual.tgz -C "$DEST"
# expect $DEST/db/trivy.db and $DEST/db/metadata.json
"$ANGUS_SECURITY_HOME/bin/doctor.sh"
If the console CVE browser needs browse.sqlite, build the index per the runbook or POST /api/v1/vuln-db/reindex (permissions like rule packs—confirm current API). Gates care about db/, not only the browser index.
6. Rollback when it fails
| Symptom | Action |
|---|---|
update-trivy-db errors mid-run |
Stop relying on sca gates; restore .bak-* whole directory |
| Findings empty after import | Wrong TRIVY_CACHE_DIR; restore backup; confirm not stub; check schema |
java-db → no such table: indices |
Empty/corrupt Java index; re-pull without skip |
| Disk full / permissions | Clear old backups and fanal/; ensure writable cache volume |
| Proxy blocks OCI | Mirror / bridge, or switch to offline tarball |
CACHE="${TRIVY_CACHE_DIR:-$ANGUS_SECURITY_HOME/trivy-db}"
rm -rf "$CACHE"
mv "$CACHE.bak-20261019120000" "$CACHE" # your backup name
"$ANGUS_SECURITY_HOME/bin/doctor.sh"
Rescan the same image reference. If CRITICAL goes from present to absent with no image change, blame the DB/engine—not “we got safer.”
7. Multi-runner / multi-node
| Risk | Mitigation |
|---|---|
| Runner mounts another cache | Unify TRIVY_CACHE_DIR or the volume |
| Nodes drift on DB version | Ship one versioned trivy-db tarball + checksum |
| Scans read a half-written tree | Stage aside, verify, then atomic mv; or pause sca during cutover |
8. Relationship to gates
- A stale DB makes
PASSEDuntrustworthy. - Update off the release window; canary first, then restore blocking.
- CRITICAL grey rollout: see Reference (Trivy gate). Do not set fail-on to never to hide a bad DB.
- Treat “DB stale” advisory Findings as ops signals, not developer defects.
9. Failure table
| Symptom | Cause | Fix |
|---|---|---|
| Update OK, CI still old | Second cache on runner | Unify env / volume |
| doctor PASS, scan empty | Wrong target/capabilities | Follow CLI first-scan; not always the DB |
| Air-gap still schedules pulls | Auto-refresh left on | Disable; use bundle/manual |
Only synced browse.sqlite |
Confused index with CVE DB | Need db/trivy.db |
| JVM coordinate misses | Skipped Java DB | Restore java-db/ |
FAQ
How often should we update?
On your threat-response cadence; update immediately when a high-profile CVE hits your stack. Products often expose a max-age default (days), but there is no universal “must be N days” compliance constant—follow team policy and console banners.
Do we stop the Security process?
Usually no. In-flight sca jobs may see a mid-update tree. Safer: pause sca gates or read from a frozen copy, then cut over.
How does this relate to install-engines?
install-engines installs binaries; update-trivy-db refreshes the DB. Both must be healthy. Rules use install-rules.sh.
Is the console “Update vuln DB” the same as the script?
The tools page typically schedules the same class of update (and APIs exist). Air-gap still uses bundles / manual directories. Follow current console copy and RBAC.
Summary
- False green → check Trivy DB before loosening gates.
- Back up the whole directory; fail → whole-directory rollback.
- Air-gap: offline bundle or manual OCI + disable auto-refresh.
- Unify cache versions across runners; canary before restoring blocks.
Reference articles
- Offline engines and doctor: https://www.anguskit.com/en/blog/angussecurity-offline-engines-doctor-checklist
- CLI first scan: https://www.anguskit.com/en/blog/angussecurity-cli-first-scan-guide
- Private AppSec deploy checklist: https://www.anguskit.com/en/blog/private-appsec-platform-deployment-checklist
- Trivy CRITICAL gate rollout: https://www.anguskit.com/en/blog/container-image-vulnerability-scanning-trivy
- Container image scanning: https://www.anguskit.com/en/blog/container-image-vulnerability-scanning-trivy
- Replace Semgrep/Trivy/Gitleaks scripts: https://www.anguskit.com/en/blog/replace-semgrep-trivy-gitleaks-with-one-scan
- Ask about highs, findings, and gates via MCP: https://www.anguskit.com/en/blog/angussecurity-mcp-server-design
- Docs: https://www.anguskit.com/en/docs/security
About the author
- AngusKit - One AI-native platform for the software development lifecycle — from agent development, code collaboration, and artifact delivery to testing, security, and product analytics. Deploy privately and keep data and call paths inside your boundary.
Comments
Join the discussionNo comments yet. Be the first to comment.
