Examples in this article are illustrative, not customer case studies. Product availability and coverage are described on the linked capability pages.
To evaluate ASPM tools, test six things on your own code: whether findings from different scanners are correlated onto one model, whether ranking uses reachability and exposure, how duplicates collapse, whether gates fail closed, what evidence each decision leaves, and where the platform and its scanners run. Demos show the first; only a proof of concept shows the rest.
A polished demo alone cannot establish how a platform will handle your own services. The criteria below are the ones that separate a correlation platform from a dashboard over five scanners, written as tests you can run in a proof of concept rather than features to tick from a datasheet.
The worked example is an application security lead with a few dozen repositories, three scanners already in CI, and a shortlist of vendors. She has a month for a proof of concept and needs to defend the decision to an engineering director who has seen tools come and go.
Pick the proof-of-concept services before you pick the vendors
Choose two or three services that represent your real estate, not your cleanest one. A good set includes a service with known noisy findings, a service that is internet-facing, and a service that shares a base image with others. For our lead, that is the payments API, a public marketing backend and an internal batch job.
Write down what you already know about each one: which findings are real, which are noise, who owns them. That list is your answer key. An ASPM tool that ranks the known-noise finding above the known-real one has told you something no slide can.
ASPM evaluation criteria and the test for each
| Criterion | What good looks like | Test in the proof of concept |
|---|---|---|
| Correlation | One model from repository to running workload | Trace one CVE from commit to deployed service |
| Reachability | Ranking uses call paths and entry points | Compare a called and an uncalled vulnerable function |
| Exposure | Internet-facing services rank higher | Same CVE in public and internal services |
| Deduplication | Two scanners, one issue, one finding | Run two SCA tools on one repository |
| Ownership | Owner derived from the repository | Move a repository between teams |
| Gates | Explainable and fail-closed | Break the policy service mid-pipeline |
| Evidence | Every decision recorded with a reason | Ask why a finding was closed last week |
| Deployment | Scanners isolated from platform credentials | Read the scan job’s permissions |
The sections below expand the criteria that demos hide most often.
Correlation and deduplication decide whether ASPM saves any time
Correlation is the reason the category exists, so test it first. Pick one vulnerable dependency in the payments API and ask the tool to show the commit that introduced it, the image that ships it and the workload running that image. If the chain breaks at the image, ask whether producer data is missing or whether the product lacks that correlation capability. The graph this depends on is described in security finding correlation on one graph.
Then run two scanners that overlap on the same repository. A correlating platform shows one finding observed twice, with both sources attached. A dashboard shows two findings, and your backlog grows with every scanner you add.
Reachability and exposure should change the order, not hide findings
Ask the vendor to explain why one finding outranks another, and expect an answer in terms of your system: this function is called from a public route, that image runs behind an internet-facing load balancer. An answer in terms of scanner severity alone means the tool is re-sorting CVSS scores.
Also check what happens to the findings that rank low. Reachability should reorder the backlog, not delete part of it. A finding marked unreachable should remain visible and searchable, because the analysis can be wrong, a point made in detail in reducing SAST false positives without hiding real bugs.
Gates and evidence are where ASPM meets the auditor
A gate that passes when the policy engine is unreachable is not a gate. In the proof of concept, stop the policy service mid-pipeline and watch the build. Fail-closed behaviour should block the release and say why. Then look for the record: which policy version decided, on which inputs, and who overrode it with what reason.
The same applies to triage. Ask how a finding is marked false positive, whether a reason is mandatory, and whether an automated process can close findings without a person. Distinguish evidence-based lifecycle updates from risk acceptance. Automated closure should record a successful scan, matching scope and reason, and preserve human-held decisions.
Deployment model and scanner isolation are security questions
An ASPM platform holds read access to your source code and often to your cloud accounts, so treat its architecture as part of your threat model. Ask where scanners run, what credentials the scan job holds, and whether a compromised scanner image could reach the platform database. Scanners parse untrusted input by design and should be treated as hostile.
Ask too whether the platform can run in your own environment, and what that means today: a supported product, an early-access programme or a roadmap slide. Each is a legitimate answer as long as it is the honest one.
Key takeaways
- Run the proof of concept on services whose real and noisy findings you already know, and score tools against that answer key.
- Trace one vulnerability from commit to running workload, and require an explicit reason for each missing link.
- Stop the policy engine during a pipeline run to confirm that gates fail closed and record why.
- Read the scan job’s credentials yourself, because scanners process hostile input.
How SecurityVault approaches this
Evaluate SecurityVault with representative services and agreed success criteria: correlation accuracy, missing-link explanations, ownership quality, useful prioritisation and gate behavior under failure. The initial demonstration can use illustrative data without repository access. Agree scope, credentials and data handling before a proof of concept against your environment.
Include people who will operate the platform as well as those approving the purchase. Developers can assess whether evidence explains the fix; platform engineers can inspect integration effort; security can review ranking and exceptions; compliance can test evidence retrieval. Procurement should confirm scope, support and data-handling terms in writing. Record the baseline and the observed result for each criterion. This creates a decision the whole company can defend without relying on scanner counts, a vendor demonstration or an unverified promise of time saved.
Explore Platform walkthrough or talk to our team about an evaluation with your engineering, security and compliance stakeholders.