Merge from master and resolve conflicts

Author: Dave Bartolomeo

change-notes/1.23/analysis-cpp.md

@@ -2,66 +2,65 @@
 
 The following changes in version 1.23 affect C/C++ analysis in all applications.
 
-## General improvements
-
 ## New queries
 
 | **Query**                   | **Tags**  | **Purpose**                                                        |
 |-----------------------------|-----------|--------------------------------------------------------------------|
-| Hard-coded Japanese era start date (`cpp/japanese-era/exact-era-date`) | reliability, japanese-era | This query is a combination of two old queries that were identical in purpose but separate as an implementation detail.  This new query replaces Hard-coded Japanese era start date in call (`cpp/japanese-era/constructor-or-method-with-exact-era-date`) and Hard-coded Japanese era start date in struct (`cpp/japanese-era/struct-with-exact-era-date`). |
-| Signed overflow check (`cpp/signed-overflow-check`) | correctness, security | Finds overflow checks that rely on signed integer addition to overflow, which has undefined behavior. Example: `a + b < a`. |
-| Pointer overflow check (`cpp/pointer-overflow-check`) | correctness, security | Finds overflow checks that rely on pointer addition to overflow, which has undefined behavior. Example: `ptr + a < ptr`. |
+| Hard-coded Japanese era start date (`cpp/japanese-era/exact-era-date`) | reliability, japanese-era | This query is a combination of two old queries that were identical in purpose but separate as an implementation detail.  This new query replaces Hard-coded Japanese era start date in call (`cpp/japanese-era/constructor-or-method-with-exact-era-date`) and Hard-coded Japanese era start date in struct (`cpp/japanese-era/struct-with-exact-era-date`). Results are not shown on LGTM by default. |
+| Pointer overflow check (`cpp/pointer-overflow-check`) | correctness, security | Finds overflow checks that rely on pointer addition to overflow, which has undefined behavior. Example: `ptr + a < ptr`. Results are shown on LGTM by default. |
+| Signed overflow check (`cpp/signed-overflow-check`) | correctness, security | Finds overflow checks that rely on signed integer addition to overflow, which has undefined behavior. Example: `a + b < a`. Results are shown on LGTM by default. |
+
 
 ## Changes to existing queries
 
 | **Query**                  | **Expected impact**    | **Change**                                                       |
 |----------------------------|------------------------|------------------------------------------------------------------|
-| Query name (`query id`) | Expected impact | Message. |
+| Comparison of narrow type with wide type in loop condition (`cpp/comparison-with-wider-type`) | Higher precision | The precision of this query has been increased to "high" as the alerts from this query have proved to be valuable on real-world projects. With this precision, results are now displayed by default in LGTM. |
 | Hard-coded Japanese era start date in call (`cpp/japanese-era/constructor-or-method-with-exact-era-date`) | Deprecated | This query has been deprecated.  Use the new combined query Hard-coded Japanese era start date (`cpp/japanese-era/exact-era-date`) instead. |
 | Hard-coded Japanese era start date in struct (`cpp/japanese-era/struct-with-exact-era-date`) | Deprecated | This query has been deprecated.  Use the new combined query Hard-coded Japanese era start date (`cpp/japanese-era/exact-era-date`) instead. |
 | Hard-coded Japanese era start date (`cpp/japanese-era/exact-era-date`) | More correct results | This query now checks for the beginning date of the Reiwa era (1st May 2019). |
+| Non-constant format string (`cpp/non-constant-format`) | Fewer false positive results | Fixed false positive results triggrered by mismatching declarations of a formatting function. |
 | Sign check of bitwise operation (`cpp/bitwise-sign-check`) | Fewer false positive results | Results involving `>=` or `<=` are no longer reported. |
-| Too few arguments to formatting function (`cpp/wrong-number-format-arguments`) | Fewer false positive results | Fixed false positives resulting from mistmatching declarations of a formatting function. |
-| Too many arguments to formatting function (`cpp/too-many-format-arguments`) | Fewer false positive results | Fixed false positives resulting from mistmatching declarations of a formatting function. |
-| Unclear comparison precedence (`cpp/comparison-precedence`) | Fewer false positive results | False positives involving template classes and functions have been fixed. |
-| Comparison of narrow type with wide type in loop condition (`cpp/comparison-with-wider-type`) | Higher precision | The precision of this query has been increased to "high" as the alerts from this query have proved to be valuable on real-world projects. With this precision, results are now displayed by default in LGTM. |
-| Non-constant format string (`cpp/non-constant-format`) | Fewer false positive results | Fixed false positives resulting from mistmatching declarations of a formatting function. |
-| Wrong type of arguments to formatting function (`cpp/wrong-type-format-argument`) | More correct results and fewer false positive results | This query now understands explicitly specified argument numbers in format strings, such as the `1$` in `%1$s`. |
+| Too few arguments to formatting function (`cpp/wrong-number-format-arguments`) | Fewer false positive results | Fixed false positive results triggered by mismatching declarations of a formatting function. |
+| Too many arguments to formatting function (`cpp/too-many-format-arguments`) | Fewer false positive results | Fixed false positive results triggered by mismatching declarations of a formatting function. |
+| Unclear comparison precedence (`cpp/comparison-precedence`) | Fewer false positive results | False positive results involving template classes and functions have been fixed. |
+| Wrong type of arguments to formatting function (`cpp/wrong-type-format-argument`) | More correct results and fewer false positive results | This query now understands explicitly-specified argument numbers in format strings, such as the `1$` in `%1$s`. |
 
 ## Changes to libraries
 
-* The data-flow library has been extended with a new feature to aid debugging.
-  Instead of specifying `isSink(Node n) { any() }` on a configuration to
-  explore the possible flow from a source, it is recommended to use the new
-  `Configuration::hasPartialFlow` predicate, as this gives a more complete
-  picture of the partial flow paths from a given source. The feature is
-  disabled by default and can be enabled for individual configurations by
-  overriding `int explorationLimit()`.
-* The data-flow library now supports flow out of C++ reference parameters.
-* The data-flow library now allows flow through the address-of operator (`&`).
-* The `DataFlow::DefinitionByReferenceNode` class now considers `f(x)` to be a
-  definition of `x` when `x` is a variable of pointer type. It no longer
-  considers deep paths such as `f(&x.myField)` to be definitions of `x`. These
-  changes are in line with the user expectations we've observed.
-* The data-flow library now makes it easier to specify barriers/sanitizers
-  arising from guards by overriding the predicate
-  `isBarrierGuard`/`isSanitizerGuard` on data-flow and taint-tracking
-  configurations respectively.
-* There is now a `DataFlow::localExprFlow` predicate and a
-  `TaintTracking::localExprTaint` predicate to make it easy to use the most
-  common case of local data flow and taint: from one `Expr` to another.
+* The data-flow library in `semmle.code.cpp.dataflow.DataFlow` and
+  `semmle.code.cpp.dataflow.TaintTracking` have had extensive changes:
+  * Data flow through fields is now more complete and reliable.
+  * The data-flow library has been extended with a new feature to aid debugging. 
+    Previously, to explore the possible flow from all sources you could specify `isSink(Node n) { any() }` on a configuration. 
+    Now you can use the new `Configuration::hasPartialFlow` predicate, 
+    which gives a more complete picture of the partial flow paths from a given source, including flow that doesn't reach any sink.
+    The feature is disabled by default and can be enabled for individual configurations by overriding `int explorationLimit()`.
+  * There is now flow out of C++ reference parameters.
+  * There is now flow through the address-of operator (`&`).
+  * The `DataFlow::DefinitionByReferenceNode` class now considers `f(x)` to be a
+    definition of `x` when `x` is a variable of pointer type. It no longer
+    considers deep paths such as `f(&x.myField)` to be definitions of `x`. These
+    changes are in line with the user expectations we've observed.
+  * It's now easier to specify barriers/sanitizers
+    arising from guards by overriding the predicate
+    `isBarrierGuard`/`isSanitizerGuard` on data-flow and taint-tracking
+    configurations respectively.
+  * There is now a `DataFlow::localExprFlow` predicate and a
+    `TaintTracking::localExprTaint` predicate to make it easy to use the most
+    common case of local data flow and taint: from one `Expr` to another.
 * The member predicates of the `FunctionInput` and `FunctionOutput` classes have been renamed for
-  clarity (e.g. `isOutReturnPointer()` to `isReturnValueDeref()`). The existing member predicates
+  clarity (for example, `isOutReturnPointer()` to `isReturnValueDeref()`). The existing member predicates
   have been deprecated, and will be removed in a future release. Code that uses the old member
   predicates should be updated to use the corresponding new member predicate.
-* The predicates `Declaration.hasStdName()` and `Declaration.hasGlobalOrStdName`
-  have been added, simplifying handling of C++ standard library functions.
+* The predicate `Declaration.hasGlobalOrStdName` has been added, making it
+  easier to recognize C library functions called from C++.
 * The control-flow graph is now computed in QL, not in the extractor. This can
-  lead to regressions (or improvements) in how queries are optimized because
+  lead to changes in how queries are optimized because
   optimization in QL relies on static size estimates, and the control-flow edge
   relations will now have different size estimates than before.
 * Support has been added for non-type template arguments.  This means that the
   return type of `Declaration::getTemplateArgument()` and
-  `Declaration::getATemplateArgument` have changed to `Locatable`.  See the
-  documentation for `Declaration::getTemplateArgument()` and
-  `Declaration::getTemplateArgumentKind()` for details.
+  `Declaration::getATemplateArgument` have changed to `Locatable`.  For details, see the
+  CodeQL library documentation for `Declaration::getTemplateArgument()` and
+  `Declaration::getTemplateArgumentKind()`.

change-notes/1.23/analysis-java.md

@@ -6,25 +6,23 @@ The following changes in version 1.23 affect Java analysis in all applications.
 
 | **Query**                   | **Tags**  | **Purpose**                                                        |
 |-----------------------------|-----------|--------------------------------------------------------------------|
-| Continue statement that does not continue (`java/continue-in-false-loop`) | correctness | Finds `continue` statements in `do { ... } while (false)` loops. |
+| Continue statement that does not continue (`java/continue-in-false-loop`) | correctness | Finds `continue` statements in `do { ... } while (false)` loops. Results are shown on LGTM by default. |
 
 ## Changes to existing queries
 
 | **Query**                    | **Expected impact**    | **Change**                        |
 |------------------------------|------------------------|-----------------------------------|
-| Dereferenced variable may be null (`java/dereferenced-value-may-be-null`) | Fewer false positives | Certain indirect null guards involving two auxiliary variables known to be equal can now be detected. |
-| Non-synchronized override of synchronized method (`java/non-sync-override`) | Fewer false positives | Results are now only reported if the immediately overridden method is synchronized. |
-| Query built from user-controlled sources (`java/sql-injection`) | More results | The query now identifies arguments to `Statement.executeLargeUpdate` and `Connection.prepareCall` as SQL expressions sinks. |
-| Query built from local-user-controlled sources (`java/sql-injection-local`) | More results | The query now identifies arguments to `Statement.executeLargeUpdate` and `Connection.prepareCall` as SQL expressions sinks. |
-| Query built without neutralizing special characters (`java/concatenated-sql-query`) | More results | The query now identifies arguments to `Statement.executeLargeUpdate` and `Connection.prepareCall` as SQL expressions sinks. |
-| Useless comparison test (`java/constant-comparison`) | Fewer false positives | Additional overflow check patterns are now recognized and no longer reported. |
+| Dereferenced variable may be null (`java/dereferenced-value-may-be-null`) | Fewer false positive results | Additional indirect null guards are detected, where two auxiliary variables are known to be equal. |
+| Non-synchronized override of synchronized method (`java/non-sync-override`) | Fewer false positive results | Results are now only reported if the immediately overridden method is synchronized. |
+| Query built from local-user-controlled sources (`java/sql-injection-local`) | More results | The query now identifies arguments to `Statement.executeLargeUpdate` and `Connection.prepareCall` as sinks for SQL expressions. |
+| Query built from user-controlled sources (`java/sql-injection`) | More results | The query now identifies arguments to `Statement.executeLargeUpdate` and `Connection.prepareCall` as sinks for SQL expressions. |
+| Query built without neutralizing special characters (`java/concatenated-sql-query`) | More results | The query now identifies arguments to `Statement.executeLargeUpdate` and `Connection.prepareCall` as sinks for SQL expressions. |
+| Useless comparison test (`java/constant-comparison`) | Fewer false positive results | Additional overflow check patterns are now recognized and no longer reported. Also, a few bug fixes in the range analysis for floating-point variables gives a further reduction in false positive results. |
 
 ## Changes to libraries
 
-* The data-flow library has been extended with a new feature to aid debugging.
-  Instead of specifying `isSink(Node n) { any() }` on a configuration to
-  explore the possible flow from a source, it is recommended to use the new
-  `Configuration::hasPartialFlow` predicate, as this gives a more complete
-  picture of the partial flow paths from a given source. The feature is
-  disabled by default and can be enabled for individual configurations by
-  overriding `int explorationLimit()`.
+The data-flow library has been extended with a new feature to aid debugging. 
+Previously, to explore the possible flow from all sources you could specify `isSink(Node n) { any() }` on a configuration. 
+Now you can use the new `Configuration::hasPartialFlow` predicate, 
+which gives a more complete picture of the partial flow paths from a given source, including flow that doesn't reach any sink.
+The feature is disabled by default and can be enabled for individual configurations by overriding `int explorationLimit()`.