Tracing 20 SBOM scanner findings to the vulnerable code

An SBOM scanner answers one question: is a vulnerable component present? The question a team actually needs answered is a different one: does the application reach the vulnerable code? Those are related but they aren’t the same, and this small study is about the gap between them.

It started from Li Zhou, Marc Dacier and Charalambos Konstantinou’s 2025 paper, A Reality Check on SBOM-based Vulnerability Management. They worked with 2,414 repositories, showed that lock files from strong package managers give consistent SBOMs, and then found a very high false-positive rate in the vulnerability reports built on top of those SBOMs, with unreachable code as a main reason. I didn’t try to reproduce that scale. I took a small corpus, treated an accurate SBOM as the starting point, and followed findings into the source by hand.

Reachability

A package can be in the dependency graph without being imported. It can be imported without the vulnerable function being called. And the vulnerable function can be called without attacker-controlled input ever reaching the trigger. A package-level matcher sees none of that:

scanner       affected package + affected version = finding

application   affected package
                -> affected module
                -> affected function
                -> a call path from the application
                -> attacker-controlled input
                -> the trigger condition

Setup

The corpus was eight open-source repositories, mostly Python, with some JavaScript dependencies in SEED: CantusDB, CoPeP, cycode-cli, OpenHands, OpenPIV-python, pypi-browser, SEED and validator.

I didn’t just pick eight and scan them. Each candidate had to be a real project with usable dependency metadata and a uv.lock or poetry.lock that matched its manifest. Several candidates were dropped because the manifest and the lock file disagreed. That’s worth knowing on its own: a lock file existing doesn’t mean it’s trustworthy.

Each repository was frozen at an exact commit, and the tools were pinned too: Syft v1.51.0 producing CycloneDX JSON, Grype v0.117.0 with a vulnerability database built on 2026-08-22, Python 3.13.11. The database matters as much as the scanner version. The same Grype with a newer database gives different findings.

One small lesson from generating the SBOMs. jq empty file.json passes on an empty file, so a failed generation can slip through. This check catches it:

[ -s "$file" ] && jq -e . "$file"

246 findings

Grype reported 246 package-level findings:

That’s 246 findings reported by a scanner. It isn’t 246 exploitable vulnerabilities, and nothing here shows that it is.

Tracing all of them by hand would have made the study bigger without making the question sharper, so I took a stratified sample of 20: 2 critical, 3 high, 10 medium and 5 low, with a fixed seed of 42. That mix comes from the sampling design, not from the corpus.

Method

For every sampled finding:

1. Read the advisory and pin down exactly what the bug needs: the module, function or API, and the trigger condition.
2. Confirm the package and version are really installed.
3. Search the application for that functionality, with rg, the dependency metadata and plain reading.
4. Decide whether the application reaches it and, where it matters, whether attacker-controlled data can get there.

The classifications were deliberately conservative: not_reachable, conditionally_reachable, reachable and unclear.

Results

not reachable             19
conditionally reachable    1
reachable                  0
unclear                    0

This doesn’t mean 95% of Python vulnerabilities are false positives. It means that in this eight-repository corpus, 19 of the 20 sampled findings had no identified path to the vulnerable functionality. That’s the claim I can defend.

The reasons were more useful than the number.

Transitive and never used. napa, then download, then decompress. The vulnerable package was in the graph and no path from the application to it was found.

Development only. shell-quote, virtualenv, pytest. These belong to tooling, tests and linting, not the runtime, but a scanner still sees them because they’re part of the resolved environment.

Used, just not the vulnerable part. This was the most interesting group: hydra-core, requests, jinja2, angular, torch. The application really uses these packages. It doesn’t use the vulnerable functionality. cycode-cli leans on requests heavily, for example, but the extract_zipped_paths() path from the advisory isn’t used anywhere in the application.

Selected findings

Jinja2 in pypi-browser. The app genuinely uses Jinja2, through Jinja2Templates(...) and TemplateResponse, so calling the finding irrelevant because Jinja2 isn’t used would be wrong. The real question was whether attacker input ever becomes template source and reaches the vulnerable sandbox behavior. The templates are static and trusted, request and package data only go in as context, and there’s no SandboxedEnvironment anywhere. Not reachable.

setuptools 69.5.1. It came in through pre-commit, then nodeenv, then setuptools. Nothing in the application imports it. The repository has packaging metadata, but that doesn’t mean production runs the vulnerable source-distribution code. Not reachable.

Hydra in CoPeP. CoPeP does use hydra-core and calls @hydra.main(...). The vulnerable part is dynamic object instantiation, and I found no use of it. Not reachable.

F018, the conditional case

Finding F018 was pygments 2.18.0 in pypi-browser. The application really calls guess_lexer_for_filename() with an archive_path, so execution reaches Pygments’ lexer selection and the archive path decides which lexer gets picked:

pypi-browser -> archive_path -> guess_lexer_for_filename() -> Pygments -> AdlLexer

That still didn’t establish everything the advisory needs, so I classed it as conditionally reachable instead of reachable. Reaching the code and proving it’s exploitable are different results, and I’d rather report the one I have.

Dependency depth

I also noted how deep each finding sat in the dependency graph: 9 at depth 1, 6 at depth 2, 4 at depth 3 and 1 at depth 4. Every finding at depth 2 or deeper was not reachable, and the one conditional case was at depth 1. With 20 findings that’s an observation and nothing more. Depth also correlates with other things, like being a dev dependency, and graph depth isn’t call depth at runtime.

VEX

VEX, Vulnerability Exploitability eXchange, supported by CycloneDX and covered by CISA’s guidance, lets a supplier publish something like “not affected, vulnerable code not in execute path”. It’s a way to communicate a conclusion. It doesn’t produce one. Someone or something still has to trace the path that justifies the statement.

SBOM            what is present
SCA             what matches a known advisory
reachability    what the application actually uses
VEX             how that conclusion travels downstream

Limitations

• Eight repositories and 20 findings. The 95% doesn’t carry over to Python as a whole, and says nothing about Java, Go, Rust or anything else.
• The repositories were chosen with qualification criteria, not sampled at random.
• Classification was manual. A second analyst could call an ambiguous path differently, so “not reachable” means no path was found in the analyzed source and configuration. It doesn’t mean execution is impossible.
• Searching source misses dynamic imports, reflection, generated code, plugin loading and runtime dispatch, and I didn’t run every application in every configuration, feature flag and deployment mode.
• The 246 baseline depends on the Syft and Grype versions and the database snapshot, which is why all three are recorded.

Future work

The manual stage is the part worth automating: take the advisory, get the vulnerable symbols, build a call graph from the application’s entry points, decide reachability, emit VEX. Generating the SBOM isn’t the hard part. Getting reliable advisory-to-code metadata is. Go’s vulnerability data already lists affected symbols, which makes that link direct, and most other ecosystems don’t have that for every advisory yet.

CovSBOM (ISSRE 2024) goes after the same gap from another side, adding code coverage to the SBOM. On 23 applications and 145 alerts it found 105 false positives.

Everything from this study is in cuddest/sboms-study: the SBOMs, the scanner output, the normalized dataset, the sample, the final analysis and the environment versions.

References

• Li Zhou, Marc Dacier, Charalambos Konstantinou. A Reality Check on SBOM-based Vulnerability Management: An Empirical Study and A Path Forward. arXiv:2511.20313, 2025.
• Yunze Zhao, Yuchen Zhang, Dan Chacko, Justin Cappos. CovSBOM: Enhancing Software Bill of Materials with Integrated Code Coverage Analysis. IEEE ISSRE 2024, pp. 228–237.
• OWASP CycloneDX, VEX capabilities.
• CISA, SBOM and VEX resources.