Merge pull request #2496 from jf205/jf-mergeback-123

Merge rc/1.23 into master

Author: shati-patel

change-notes/1.23/analysis-csharp.md

@@ -6,20 +6,18 @@ The following changes in version 1.23 affect C# analysis in all applications.
 
 | **Query**                   | **Tags**  | **Purpose**                                                        |
 |-----------------------------|-----------|--------------------------------------------------------------------|
-| Deserialized delegate (`cs/deserialized-delegate`) | security, external/cwe/cwe-502 | Finds unsafe deserialization of delegate types. |
-| Deserialization of untrusted data (`cs/unsafe-deserialization-untrusted-input`) | security, external/cwe/cwe-502 | Finds flow of untrusted input to calls to unsafe deserializers. |
-| Unsafe year argument for 'DateTime' constructor (`cs/unsafe-year-construction`) | reliability, date-time | Finds incorrect manipulation of `DateTime` values, which could lead to invalid dates. |
-| Unsafe deserializer (`cs/unsafe-deserialization`) | security, external/cwe/cwe-502 | Finds calls to unsafe deserializers. |
-| Mishandling the Japanese era start date (`cs/mishandling-japanese-era`) | reliability, date-time | Finds hard-coded Japanese era start dates that could be invalid. |
+| Deserialized delegate (`cs/deserialized-delegate`) | security, external/cwe/cwe-502 | Finds unsafe deserialization of delegate types. Results are shown on LGTM by default. |
+| Deserialization of untrusted data (`cs/unsafe-deserialization-untrusted-input`) | security, external/cwe/cwe-502 | Finds flow of untrusted input to calls to unsafe deserializers. Results are shown on LGTM by default. |
+| Mishandling the Japanese era start date (`cs/mishandling-japanese-era`) | reliability, date-time | Finds hard-coded Japanese era start dates that could be invalid. Results are not shown on LGTM by default. |
+| Unsafe year argument for 'DateTime' constructor (`cs/unsafe-year-construction`) | reliability, date-time | Finds incorrect manipulation of `DateTime` values, which could lead to invalid dates. Results are not shown on LGTM by default. |
+| Unsafe deserializer (`cs/unsafe-deserialization`) | security, external/cwe/cwe-502 | Finds calls to unsafe deserializers. By default, the query is not run on LGTM. |
 
 ## Changes to existing queries
 
 | **Query**                    | **Expected impact**    | **Change**                        |
 |------------------------------|------------------------|-----------------------------------|
 | Dereferenced variable may be null (`cs/dereferenced-value-may-be-null`) | Fewer false positive results | More `null` checks are now taken into account, including `null` checks for `dynamic` expressions and `null` checks such as `object alwaysNull = null; if (x != alwaysNull) ...`. |
-| Missing Dispose call on local IDisposable (`cs/local-not-disposed`) | Fewer false positive results | The query has been rewritten in order to identify more dispose patterns. For example, a local `IDisposable` that is disposed of by passing through a fluent API is no longer reported. |
-
-## Removal of old queries
+| Missing Dispose call on local IDisposable (`cs/local-not-disposed`) | Fewer false positive results | The query has been rewritten in order to identify more dispose patterns. For example, a local `IDisposable` that is disposed of by passing through a fluent API is no longer reported as missing a dispose call. |
 
 ## Changes to code extraction
 
@@ -29,22 +27,19 @@ The following changes in version 1.23 affect C# analysis in all applications.
 
 * The new class `NamespaceAccess` models accesses to namespaces, for example in `nameof` expressions.
 * The data-flow library now makes it easier to specify barriers/sanitizers
-  arising from guards by overriding the predicate
+  arising from guards. You can override the predicate
   `isBarrierGuard`/`isSanitizerGuard` on data-flow and taint-tracking
   configurations respectively.
 * The data-flow library has been extended with a new feature to aid debugging.
-  Instead of specifying `isSink(Node n) { any() }` on a configuration to
-  explore the possible flow from a source, it is recommended to use the new
-  `Configuration::hasPartialFlow` predicate, as this gives a more complete
-  picture of the partial flow paths from a given source. The feature is
-  disabled by default and can be enabled for individual configurations by
+  Previously, to explore the possible flow from all sources you could specify `isSink(Node n) { any() }` on a configuration. 
+  Now you can use the new `Configuration::hasPartialFlow` predicate, 
+  which gives a more complete picture of the partial flow paths from a given source, including flow that doesn't reach any sink.
+  The feature is disabled by default and can be enabled for individual configurations by
   overriding `int explorationLimit()`.
-* `foreach` statements where the body is guaranteed to be executed at least once, such as `foreach (var x in new string[]{ "a", "b", "c" }) { ... }`, are now recognized by all analyses based on the control flow graph (such as SSA, data flow and taint tracking).
-* Fixed the control flow graph for `switch` statements where the `default` case was not the last case. This had caused the remaining cases to be unreachable. `SwitchStmt.getCase(int i)` now puts the `default` case last.
+* `foreach` statements where the body is guaranteed to be executed at least once, such as `foreach (var x in new string[]{ "a", "b", "c" }) { ... }`, are now recognized by all analyses based on the control-flow graph (such as SSA, data flow and taint tracking).
+* Fixed the control-flow graph for `switch` statements where the `default` case was not the last case. This had caused the remaining cases to be unreachable. `SwitchStmt.getCase(int i)` now puts the `default` case last.
 * There is now a `DataFlow::localExprFlow` predicate and a
   `TaintTracking::localExprTaint` predicate to make it easy to use the most
   common case of local data flow and taint: from one `Expr` to another.
 * Data is now tracked through null-coalescing expressions (`??`).
 * A new library `semmle.code.csharp.Unification` has been added. This library exposes two predicates `unifiable` and `subsumes` for calculating type unification and type subsumption, respectively.
-
-## Changes to autobuilder

csharp/ql/src/semmle/code/csharp/ir/implementation/unaliased_ssa/internal/reachability/ReachableBlockInternal.qll

@@ -1,2 +1,2 @@
-import semmle.code.cpp.ir.implementation.unaliased_ssa.IR as IR
-import semmle.code.cpp.ir.implementation.unaliased_ssa.constant.ConstantAnalysis as ConstantAnalysis
+import semmle.code.csharp.ir.implementation.unaliased_ssa.IR as IR
+import semmle.code.csharp.ir.implementation.unaliased_ssa.constant.ConstantAnalysis as ConstantAnalysis

csharp/ql/src/semmle/code/csharp/ir/internal/TIRVariable.qll

@@ -9,6 +9,8 @@ newtype TIRVariable =
     Construction::functionHasIR(callable) and
     var.getCallable() = callable
   } or
-  TIRTempVariable(Callable callable, Language::AST ast, TempVariableTag tag, Type type) {
+  TIRTempVariable(
+    Callable callable, Language::AST ast, TempVariableTag tag, Language::LanguageType type
+  ) {
     Construction::hasTempVariable(callable, ast, tag, type)
   }

docs/language/learn-ql/cobol/introduce-libraries-cobol.rst

@@ -149,4 +149,3 @@ What next?
 ----------
 
 -  Find out more about QL in the `QL language handbook <https://help.semmle.com/QL/ql-handbook/index.html>`__ and `QL language specification <https://help.semmle.com/QL/ql-spec/language.html>`__.
--  Learn more about the query console in `Using the query console <https://lgtm.com/help/lgtm/using-query-console>`__.

docs/language/learn-ql/cobol/ql-for-cobol.rst

@@ -6,15 +6,15 @@ CodeQL for COBOL
    :hidden:
 
    introduce-libraries-cobol
-   
+
+.. include:: ../../support/cobol-note.rst
+
 This page provides an overview of the CodeQL for COBOL documentation that is currently available.
 
--  `Basic COBOL query <https://lgtm.com/help/lgtm/console/ql-cobol-basic-example>`__ describes how to write and run queries using LGTM.
 -  :doc:`Introducing the CodeQL libraries for COBOL <introduce-libraries-cobol>` introduces the standard libraries used to write queries for COBOL code.
 
 
 Other resources
 ---------------
 
--  For the queries used in LGTM, display a `COBOL query <https://lgtm.com/search?q=language%3Acobol&t=rules>`__ and click **Open in query console** to see the code used to find alerts.
 -  For more information about the library for COBOL see the `CodeQL library for COBOL <https://help.semmle.com/qldoc/cobol/>`__.

docs/language/support/cobol-note.rst

@@ -0,0 +1,5 @@
+.. pull-quote:: Important
+
+   CodeQL for COBOL is being deprecated after the 1.23 release of CodeQL. 
+   Future releases, starting with 1.24, will no longer contain support for analyzing COBOL source code. 
+   We are not aware of any customers who will be affected by this change. If you do have any concerns, please contact your account manager.

docs/language/support/conf.py

@@ -80,4 +80,4 @@
 
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
-exclude_patterns = ['read-me-project.rst']
+exclude_patterns = ['read-me-project.rst', 'cobol-note.rst']

docs/language/support/language-support.rst

@@ -10,6 +10,8 @@ Customers with any questions should contact their usual Semmle contact with any
 If you're not a customer yet, contact us at info@semmle.com 
 with any questions you have about language and compiler support.
 
+.. include:: cobol-note.rst
+
 .. csv-table::
      :file: versions-compilers.csv
      :header-rows: 1