Merge remote-tracking branch 'upstream/master' into EventEmitter

Author: erik-krogh

.github/labeler.yml

@@ -0,0 +1,24 @@
+"C++":
+  - cpp/**/*
+  - change-notes/**/*cpp*
+
+"C#":
+  - csharp/**/*
+  - change-notes/**/*csharp*
+
+Java:
+  - java/**/*
+  - change-notes/**/*java.*
+
+JS:
+  - javascript/**/*
+  - change-notes/**/*javascript*
+
+Python:
+  - python/**/*
+  - change-notes/**/*python*
+
+documentation:
+  - "**/*.qhelp"
+  - "**/*.md"
+  - docs/**/*

.gitignore

@@ -15,3 +15,5 @@
 
 # It's useful (though not required) to be able to unpack codeql in the ql checkout itself
 /codeql/
+.vscode/settings.json
+csharp/extractor/Semmle.Extraction.CSharp.Driver/Properties/launchSettings.json

CONTRIBUTING.md

@@ -15,7 +15,7 @@ Follow the steps below to help other users understand what your query does, and
 
 2. **Format your code correctly**
 
-   All of Semmle's standard queries and libraries are uniformly formatted for clarity and consistency, so we strongly recommend that all contributions follow the same formatting guidelines. If you use QL for Eclipse, you can auto-format your query in the [QL editor](https://help.semmle.com/ql-for-eclipse/Content/WebHelp/ql-editor.html). For more information, see the [CodeQL style guide](https://github.com/Semmle/ql/blob/master/docs/ql-style-guide.md).
+   All of Semmle's standard queries and libraries are uniformly formatted for clarity and consistency, so we strongly recommend that all contributions follow the same formatting guidelines. If you use CodeQL for VS Code, you can autoformat your query in the [Editor](https://help.semmle.com/codeql/codeql-for-vscode/reference/editor.html#autoformatting). For more information, see the [CodeQL style guide](https://github.com/Semmle/ql/blob/master/docs/ql-style-guide.md).
 
 3. **Make sure your query has the correct metadata**
 
@@ -26,7 +26,7 @@ Follow the steps below to help other users understand what your query does, and
 
 4. **Make sure the `select` statement is compatible with the query type**
 
-   The `select` statement of your query must be compatible with the query type (determined by the `@kind` metadata property) for alert or path results to be displayed correctly in LGTM and QL for Eclipse.
+   The `select` statement of your query must be compatible with the query type (determined by the `@kind` metadata property) for alert or path results to be displayed correctly in LGTM and CodeQL for VS Code.
    For more information on `select` statement format, see [Introduction to query files](https://help.semmle.com/QL/learn-ql/writing-queries/introduction-to-queries.html#select-clause) on help.semmle.com.
 
 5. **Save your query in a `.ql` file in the correct language directory in this repository**

change-notes/1.23/analysis-java.md

@@ -7,6 +7,7 @@ The following changes in version 1.23 affect Java analysis in all applications.
 | **Query**                   | **Tags**  | **Purpose**                                                        |
 |-----------------------------|-----------|--------------------------------------------------------------------|
 | Continue statement that does not continue (`java/continue-in-false-loop`) | correctness | Finds `continue` statements in `do { ... } while (false)` loops. Results are shown on LGTM by default. |
+| Disabled Netty HTTP header validation (`java/netty-http-response-splitting`) | security, external/cwe/cwe-113 | Finds response-splitting vulnerabilities due to Netty HTTP header validation being disabled. Results are shown on LGTM by default. |
 
 ## Changes to existing queries
 

change-notes/1.24/analysis-cpp.md

@@ -13,10 +13,20 @@ The following changes in version 1.24 affect C/C++ analysis in all applications.
 
 | **Query**                  | **Expected impact**    | **Change**                                                       |
 |----------------------------|------------------------|------------------------------------------------------------------|
+| Buffer not sufficient for string (`cpp/overflow-calculated`) | More true positive results | This query now identifies a wider variety of buffer allocations using the `semmle.code.cpp.models.interfaces.Allocation` library. |
+| No space for zero terminator (`cpp/no-space-for-terminator`) | More true positive results | This query now identifies a wider variety of buffer allocations using the `semmle.code.cpp.models.interfaces.Allocation` library. |
+| Memory is never freed (`cpp/memory-never-freed`) | More true positive results | This query now identifies a wider variety of buffer allocations using the `semmle.code.cpp.models.interfaces.Allocation` library. |
+| Memory may not be freed (`cpp/memory-may-not-be-freed`) | More true positive results | This query now identifies a wider variety of buffer allocations using the `semmle.code.cpp.models.interfaces.Allocation` library. |
+| Missing return statement (`cpp/missing-return`) | Fewer false positive results | Functions containing `asm` statements are no longer highlighted by this query. |
+| Hard-coded Japanese era start date (`cpp/japanese-era/exact-era-date`) |  | This query is no longer run on LGTM. |
 | No space for zero terminator (`cpp/no-space-for-terminator`) | Fewer false positive results | This query has been modified to be more conservative when identifying which pointers point to null-terminated strings.  This approach produces fewer, more accurate results. |
+| Overloaded assignment does not return 'this' (`cpp/assignment-does-not-return-this`) | Fewer false positive results | This query no longer reports incorrect results in template classes. |
+| Unsafe array for days of the year (`cpp/leap-year/unsafe-array-for-days-of-the-year`) |  | This query is no longer run on LGTM. |
 
 ## Changes to libraries
 
+* Created the `semmle.code.cpp.models.interfaces.Allocation` library to model allocation such as `new` expressions and calls to `malloc`. This in intended to replace the functionality in `semmle.code.cpp.commons.Alloc` with a more consistent and useful interface.
+* Created the `semmle.code.cpp.models.interfaces.Deallocation` library to model deallocation such as `delete` expressions and calls to `free`. This in intended to replace the functionality in `semmle.code.cpp.commons.Alloc` with a more consistent and useful interface.
 * The new class `StackVariable` should be used in place of `LocalScopeVariable`
   in most cases. The difference is that `StackVariable` does not include
   variables declared with `static` or `thread_local`.

change-notes/1.24/analysis-csharp.md

@@ -13,14 +13,20 @@ The following changes in version 1.24 affect C# analysis in all applications.
 
 | **Query**                    | **Expected impact**    | **Change**                        |
 |------------------------------|------------------------|-----------------------------------|
+| Useless assignment to local variable (`cs/useless-assignment-to-local`) | Fewer false positive results | Results have been removed when the variable is named `_` in a `foreach` statement. | 
 
 ## Removal of old queries
 
 ## Changes to code extraction
 
+* Tuple expressions, for example `(int,bool)` in `default((int,bool))` are now extracted correctly.
+* Expression nullability flow state is extracted. 
+
 ## Changes to libraries
 
 * The taint tracking library now tracks flow through (implicit or explicit) conversion operator calls.
+* [Code contracts](https://docs.microsoft.com/en-us/dotnet/framework/debug-trace-profile/code-contracts) are now recognized, and are treated like any other assertion methods.
+* Expression nullability flow state is given by the predicates `Expr.hasNotNullFlowState()` and `Expr.hasMaybeNullFlowState()`.
 
 ## Changes to autobuilder
 

change-notes/1.24/analysis-javascript.md

@@ -2,29 +2,35 @@
 
 ## General improvements
 
+* Alert suppression can now be done with single-line block comments (`/* ... */`) as well as line comments (`// ...`).
+
+* Imports with the `.js` extension can now be resolved to a TypeScript file,
+  when the import refers to a file generated by TypeScript.
+
+- The analysis of sanitizer guards has improved, leading to fewer false-positive results from the security queries.
+
 * Support for the following frameworks and libraries has been improved:
   - [react](https://www.npmjs.com/package/react)
   - [typeahead.js](https://www.npmjs.com/package/typeahead.js)
   - [Handlebars](https://www.npmjs.com/package/handlebars)
 
-- Imports with the `.js` extension can now be resolved to a TypeScript file,
-  when the import refers to a file generated by TypeScript.
-
 ## New queries
 
 | **Query**                                                                       | **Tags**                                                          | **Purpose**                                                                                                                                                                            |
 |---------------------------------------------------------------------------------|-------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | Cross-site scripting through exception (`js/xss-through-exception`) | security, external/cwe/cwe-079, external/cwe/cwe-116              | Highlights potential XSS vulnerabilities where an exception is written to the DOM. Results are not shown on LGTM by default. |
+| Regular expression always matches (`js/regex/always-matches`) | correctness, regular-expressions | Highlights regular expression checks that trivially succeed by matching an empty substring. Results are shown on LGTM by default. |
 
 ## Changes to existing queries
 
 | **Query**                      | **Expected impact**          | **Change**                                                                |
 |--------------------------------|------------------------------|---------------------------------------------------------------------------|
 | Clear-text logging of sensitive information (`js/clear-text-logging`) | More results | More results involving `process.env` and indirect calls to logging methods are recognized. |
+| Duplicate parameter names (`js/duplicate-parameter-name`) | Fewer results | This query now recognizes additional parameters that reasonably can have duplicated names. |
 | Incomplete string escaping or encoding (`js/incomplete-sanitization`) | Fewer false positive results | This query now recognizes additional cases where a single replacement is likely to be intentional. |
 | Unbound event handler receiver (`js/unbound-event-handler-receiver`) | Fewer false positive results | This query now recognizes additional ways event handler receivers can be bound. | 
+| Expression has no effect (`js/useless-expression`) | Fewer false positive results | The query now recognizes block-level flow type annotations. |
 
 ## Changes to libraries
 
 * The predicates `RegExpTerm.getSuccessor` and `RegExpTerm.getPredecessor` have been changed to reflect textual, not operational, matching order. This only makes a difference in lookbehind assertions, which are operationally matched backwards. Previously, `getSuccessor` would mimick this, so in an assertion `(?<=ab)` the term `b` would be considered the predecessor, not the successor, of `a`. Textually, however, `a` is still matched before `b`, and this is the order we now follow.
-

change-notes/1.24/extractor-javascript.md

@@ -0,0 +1,7 @@
+[[ condition: enterprise-only ]]
+
+# Improvements to JavaScript analysis
+
+## Changes to code extraction
+
+* `import.meta` expressions no longer result in a syntax error in JavaScript files.

config/identical-files.json

@@ -265,5 +265,12 @@
   "C# IR ValueNumberingImports": [
     "csharp/ql/src/semmle/code/csharp/ir/implementation/raw/gvn/internal/ValueNumberingImports.qll",
     "csharp/ql/src/semmle/code/csharp/ir/implementation/unaliased_ssa/gvn/internal/ValueNumberingImports.qll"
+  ],
+  "XML": [
+    "cpp/ql/src/semmle/code/cpp/XML.qll",
+    "csharp/ql/src/semmle/code/csharp/XML.qll",
+    "java/ql/src/semmle/code/xml/XML.qll",
+    "javascript/ql/src/semmle/javascript/XML.qll",
+    "python/ql/src/semmle/python/xml/XML.qll"
   ]
 }

cpp/config/suites/c/correctness

@@ -23,6 +23,8 @@
 + semmlecode-cpp-queries/Likely Bugs/Arithmetic/FloatComparison.ql: /Correctness/Common Errors
 + semmlecode-cpp-queries/Likely Bugs/Arithmetic/BitwiseSignCheck.ql: /Correctness/Common Errors
 + semmlecode-cpp-queries/Likely Bugs/Arithmetic/BadAdditionOverflowCheck.ql: /Correctness/Common Errors
++ semmlecode-cpp-queries/Likely Bugs/Arithmetic/SignedOverflowCheck.ql: /Correctness/Common Errors
++ semmlecode-cpp-queries/Likely Bugs/Memory Management/PointerOverflow.ql: /Correctness/Common Errors
 + semmlecode-cpp-queries/Likely Bugs/NestedLoopSameVar.ql: /Correctness/Common Errors
 + semmlecode-cpp-queries/Likely Bugs/UseInOwnInitializer.ql: /Correctness/Common Errors
 + semmlecode-cpp-queries/Critical/NewArrayDeleteMismatch.ql: /Correctness/Common Errors