Container image scanning: layers, base images and real fixes

Container image scanning that separates base-image, OS package and application dependency findings, so each goes to the team that can fix it.

Examples in this article are illustrative, not customer case studies. Product availability and coverage are described on the linked capability pages.

Your image scan returns a long list of vulnerabilities for a single service, and most of them sit in packages your team never installed. This post shows how to read a container image scan layer by layer, so base-image debt, OS packages and application dependencies each reach the team that can actually fix them.

Container image scanning identifies supported packages in image content and matches them against vulnerability data. Coverage depends on the scanner, package format and available metadata. A useful scan also attributes every finding to the layer that introduced it (base image, OS package or application dependency), because that attribution decides who can fix it and how.

Our example is a notifications service: a Python application that sends email and push messages, built on a general-purpose Debian-based image chosen when the project started. Its Dockerfile has not changed much since. Its scan report has grown steadily anyway, and the team has stopped reading it.

Container image layers decide who can fix a finding

Every container image is a stack of layers, each a diff against the one below it, and every FROM instruction inherits the entire stack of the image it references. The notifications image therefore contains three kinds of content: the base image maintained upstream, OS packages added in its own RUN apt-get install steps, and the application layer holding its Python dependencies and code.

Most of the vulnerabilities in the report arrived with the base image. They will still be there after the team fixes every issue in its own code, because it never touched those layers. A base-layer finding needs a new base tag or distribution, while an application-layer finding needs a dependency bump or a code change.

Routing both to the same team at the same urgency wastes attention, because much of the list is not actionable by the people reading it. The first job of a useful scan is to say which layer introduced each package, and therefore who owns the fix.

Base-image debt accumulates even when the Dockerfile does not change

The base image is one of the most consequential choices in a container’s security posture, and it is usually made once and never revisited. The notifications service’s base tag was current when chosen. Since then, the upstream distribution has published fixes that the image never picked up, because nobody rebuilt it against a newer base. The vulnerable code was already present; new advisories and unconsumed fixes increased the known backlog.

Pinning a base image to a digest is good practice for reproducibility. A digest pinned once and never moved, however, is exactly how base-image debt builds up, because reproducibility and staleness are two sides of the same pin. Treat the base reference like a lockfile entry: update it on a regular cadence through automated pull requests, and rebuild even when application code has not changed.

Minimal and distroless images remove risk before it is scanned

The cheapest vulnerability to manage is one in a package you never shipped. Distroless and other minimal base images drop the package manager, the shell and most userland tools, leaving little beyond a language runtime and its direct system libraries.

The benefit is direct. Fewer packages in the base layer means fewer packages that can carry a CVE, and without a shell, an attacker who gains code execution loses an easy way to explore the container and chain commands.

The trade-off is operational. A distroless notifications image is harder to debug interactively, so teams need ephemeral debug containers and good structured logging, rather than adding tools back and eroding the benefit. Multi-stage builds help too: compile and test in a full image, then copy only the finished artefact into a clean final stage, leaving build tools behind.

OS packages and application dependencies need different fixes

A single container scan report mixes two categories. OS packages arrive through apt, apk or yum, either in the base image or in an explicit install step. Application dependencies arrive through pip, npm, Maven or Go modules as part of building your software.

OS package findings are usually fixed by moving to a newer base image or package version, which is decoupled from application code and often automatable. Application dependency findings are fixed by updating a lockfile entry, which may need code changes when the update crosses a major version.

Reachability also works differently for each. For the notifications service’s Python dependencies, static analysis can often show whether the code calls the vulnerable function at all, a technique covered in risk-based vulnerability management with reachability. For an OS library, call-graph reasoning is rarely conclusive, so exposure is better judged by runtime context: whether the container is privileged, mounts host paths or ships a shell.

Container scanning belongs at build time and in the registry

A software bill of materials generated for every image, listing each package and version in every layer, is what makes all of this manageable. When a new CVE is disclosed, an SBOM kept per scan turns “which images contain this package” from a full re-scan into a query. The difference between SBOM data and dependency scanning is explored further in SCA vs SBOM.

Build-time and registry scans answer different questions, and both are needed. A build-time scan in CI stops a problem before the image is pushed, the cheapest point to catch it. A registry scan catches what build time could not know: a CVE disclosed after the notifications image was built, sitting quietly until someone looks again.

Neither replaces the other. Registry scanning alone lets vulnerable images in before anyone looks, and build-time scanning cannot find tomorrow’s disclosure in yesterday’s image. What neither tells you is whether an image is running, which is the subject of the companion post on binding images to running workloads.

Key takeaways

  • Attribute every image finding to the layer that introduced it, and route base-layer findings to whoever owns the base image.
  • Update pinned base image digests on a regular cadence and rebuild even when application code is unchanged.
  • Use multi-stage builds and minimal base images so build tools and shells never reach the final image.
  • Keep an SBOM for every image scan so a new disclosure becomes a query rather than a re-scan.

How SecurityVault approaches this

SecurityVault combines image vulnerability results, available layer context and generated SBOMs with workload inventory. Digest-based associations help teams distinguish affected running images from registry inventory and route work to the relevant service or platform owner. Package coverage and layer attribution depend on scanner output. Review scan completion before treating absence of a finding as evidence.

Bring one image with a known base-image issue and one application dependency issue. Review the package identity, available layer attribution, scan completion state and associated workloads before deciding who should act. Rebuild the image with a controlled change and compare the retained inventories. Finally, inspect the running digest to separate a completed build from a completed rollout. This gives the platform team and application team a shared way to verify their respective parts of the remediation without assuming that one successful step proves the rest.

Explore Container security or talk to our team about an evaluation with your engineering, security and compliance stakeholders.

Further reading

See it on your stack

Bring Container security into one finding graph.

SecurityVault is the enterprise security operating system: one graph of findings, policy and evidence from code to cloud. Explore the Container security workflow with your engineering, security and compliance teams. No repository access is needed for the initial demonstration.