Shared: Provenance-based filtering of flow summaries #21051
Merged
+15,876
−21,185
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
C#
C++
Java
Python
Go
Ruby
Rust
hvitved
force-pushed
the
shared/flow-summary-provenance-filtering
branch
3 times, most recently
from
a3e585d to
eb48820
Compare
hvitved
force-pushed
the
shared/flow-summary-provenance-filtering
branch
from
1e946f8 to
30a0791
Compare
This was referenced Jan 5, 2026
hvitved
force-pushed
the
shared/flow-summary-provenance-filtering
branch
3 times, most recently
from
0fbea88 to
5a2881d
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
hvitved
force-pushed
the
shared/flow-summary-provenance-filtering
branch
from
5a2881d to
a941f4a
Compare
hvitved
force-pushed
the
shared/flow-summary-provenance-filtering
branch
2 times, most recently
from
bf632b3 to
c6383ff
Compare
hvitved
force-pushed
the
shared/flow-summary-provenance-filtering
branch
2 times, most recently
from
9f81377 to
0057ae3
Compare
hvitved
force-pushed
the
shared/flow-summary-provenance-filtering
branch
2 times, most recently
from
1933d1c to
72dfe9c
Compare
hvitved
force-pushed
the
shared/flow-summary-provenance-filtering
branch
from
5d74edd to
27c102a
Compare
| c.fromSource() and | ||
| not c.getFile().isStub() and | ||
| not ( | ||
| c.getFile().extractedQlTest() and |
Contributor
There was a problem hiding this comment.
Maybe this deserves a comment (that ql test files where the body is just a throw are considered stub like and thus not a part of the source code).
Contributor
michaelnebel
left a comment
michaelnebel
left a comment
There was a problem hiding this comment.
Really nice work @hvitved !
Only a couple of minor questions/remarks.
Contributor
|
Offline feedback recap: Some of the added Java models look wrong. Tom and I identified several issues: the code snippet to identify and generate the missing models lacked the signature, and notably the signature can be different from the overridden method. Also, some existing manual exact models were missing signatures, which caused them to wrongly apply to inherited overloads. |
hvitved
force-pushed
the
shared/flow-summary-provenance-filtering
branch
from
27c102a to
eaaa4f2
Compare
hvitved
force-pushed
the
shared/flow-summary-provenance-filtering
branch
from
eaaa4f2 to
e486026
Compare
Missing manual models were added using the following code added to `FlowSummaryImpl.qll`:
```ql
private predicate testsummaryElement(
Input::SummarizedCallableBase c, string namespace, string type, boolean subtypes, string name,
string signature, string ext, string originalInput, string originalOutput, string kind,
string provenance, string model, boolean isExact
) {
exists(string input, string output, Callable baseCallable |
summaryModel(namespace, type, subtypes, name, signature, ext, originalInput, originalOutput,
kind, provenance, model) and
baseCallable = interpretElement(namespace, type, subtypes, name, signature, ext, isExact) and
(
c.asCallable() = baseCallable and input = originalInput and output = originalOutput
or
correspondingKotlinParameterDefaultsArgSpec(baseCallable, c.asCallable(), originalInput,
input) and
correspondingKotlinParameterDefaultsArgSpec(baseCallable, c.asCallable(), originalOutput,
output)
)
)
}
private predicate testsummaryElement2(
string namespace, string type, boolean subtypes, string name, string signature, string ext,
string originalInput, string originalOutput, string kind, string provenance, string model,
string namespace2, string type2
) {
exists(Input::SummarizedCallableBase c |
testsummaryElement(c, namespace2, type2, _, _, _, ext, originalInput, originalOutput, kind,
provenance, model, false) and
testsummaryElement(c, namespace, type, subtypes, name, _, _, _, _, _, provenance, _, true) and
signature = paramsString(c.asCallable()) and
not testsummaryElement(c, _, _, _, _, _, _, originalInput, originalOutput, kind, provenance,
_, true)
)
}
private string getAMissingManualModel(string namespace2, string type2) {
exists(
string namespace, string type, boolean subtypes, string name, string signature, string ext,
string originalInput, string originalOutput, string kind, string provenance, string model
|
testsummaryElement2(namespace, type, subtypes, name, signature, ext, originalInput,
originalOutput, kind, provenance, model, namespace2, type2) and
result =
"- [\"" + namespace + "\", \"" + type + "\", True, \"" + name + "\", \"" + signature +
"\", \"\", \"" + originalInput + "\", \"" + originalOutput + "\", \"" + kind + "\", \"" +
provenance + "\"]"
)
}
```
hvitved
force-pushed
the
shared/flow-summary-provenance-filtering
branch
from
e486026 to
df09f02
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR aligns the logic across languages for how flow summaries are prioritized based on provenance and exactness (that is, whether a model is defined directly for a function or for a function that is implemented/overridden).
A flow summary is considered relevant if:
Note that for dynamic languages we currently pretend that no source code is available for functions with flow summaries, so 3.(a) holds vacuously.
Points 2 and 3.c represent a change for e.g. Java, where we would previously union exact and inexact models, which meant that it was not possible to overrule inexact models. As a consequence, some inexact manual have been replicated. DCA for Java reports some lost
java/sensitive-logresults onapache_solr, but looking at those results, they all have flow paths of length > 150, so they are almost certainly false positives, and most likely a consequence of 3.c.In order for the logic to be defined in the shared flow summary library, I had to move provenance and exactness information into the
propagatesFlowpredicate, which is a breaking change.Lastly, I have applied the
::Rangepattern to theSummarizedCallableclass for all languages except C++, which currently does not expose this class. This means thatSummarizedCallable::Rangewill contain all flow summaries, whereasSummarizedCallablewill only contain relevant summaries.