Merge pull request #2094 from rdmarsh2/rdmarsh/docs/cpp/advanced-library-guide

C++/Docs: Add guides to advanced AST libraries

Author: jbj

docs/language/learn-ql/cpp/guards.rst

@@ -0,0 +1,83 @@
+Using the guards library in C and C++
+=====================================
+
+Overview
+--------
+The guards library (defined in ``semmle.code.cpp.controlflow.Guards``) provides a class `GuardCondition <https://help.semmle.com/qldoc/cpp/semmle/code/cpp/controlflow/Guards.qll/type.Guards$GuardCondition.html>`__ representing Boolean values that are used to make control flow decisions.
+A ``GuardCondition`` is considered to guard a basic block if the block can only be reached if the ``GuardCondition`` is evaluated a certain way. For instance, in the following code, ``x < 10`` is a ``GuardCondition``, and it guards all the code before the return statement.
+
+.. code-block:: cpp
+
+    if(x < 10) {
+      f(x);
+    } else if (x < 20) {
+      g(x);
+    } else {
+      h(x);
+    }
+    return 0;
+
+
+The ``controls`` predicate
+------------------------------------------------
+The ``controls`` predicate helps determine which blocks are only run when the ``GuardCondition`` evaluates a certain way. ``guard.controls(block, testIsTrue)`` holds if ``block`` is only entered if the value of this condition is ``testIsTrue``.
+
+In the following code sample, the call to ``isValid`` controls the calls to ``performAction`` and ``logFailure`` but not the return statement.
+
+.. code-block:: cpp
+
+    if(isValid(accessToken)) {
+      performAction();
+      succeeded = 1;
+    } else {
+      logFailure();
+      succeeded = 0;
+    }
+    return succeeded;
+
+In the following code sample, the call to `isValid` controls the body of the if and also the code after the if.
+
+.. code-block:: cpp
+
+    if(!isValid(accessToken)) {
+      logFailure();
+      return 0;
+    }
+    performAction();
+    return succeeded;
+
+The ``ensuresEq`` and ``ensuresLt`` predicates
+----------------------------------------------
+The ``ensuresEq`` and ``ensuresLt`` predicates are the main way of determining what, if any, guarantees the ``GuardCondition`` provides for a given basic block.
+
+The ``ensuresEq`` predicate
+***************************
+When ``ensuresEq(left, right, k, block, true)`` holds, then ``block`` is only executed if ``left`` was equal to ``right + k`` at their last evaluation. When ``ensuresEq(left, right, k, block, false)`` holds, then ``block`` is only executed if ``left`` was not equal to ``right + k`` at their last evaluation.
+
+The ``ensuresLt`` predicate
+***************************
+When ``ensuresLt(left, right, k, block, true)`` holds, then ``block`` is only executed if ``left`` was strictly less than ``right + k`` at their last evaluation. When ``ensuresLt(left, right, k, block, false)`` holds, then ``block`` is only executed if ``left`` was greater than or equal to ``right + k`` at their last evaluation.
+
+In the following code sample, the comparison on the first line ensures that ``index`` is less than ``size`` in the "then" block, and that ``index`` is greater than or equal to ``size`` in the "else" block.
+
+.. code-block:: cpp
+
+    if(index < size) {
+      ret = array[index];
+    } else {
+      ret = nullptr
+    }
+    return ret;
+
+The ``comparesEq`` and ``comparesLt`` predicates
+------------------------------------------------
+The ``comparesEq`` and ``comparesLt`` predicates help determine if the ``GuardCondition`` evaluates to true.
+
+The ``comparesEq`` predicate
+****************************
+``comparesEq(left, right, k, true, testIsTrue)`` holds if ``left`` equals ``right + k`` when the expression evaluates to ``testIsTrue``.
+
+The ``comparesLt`` predicate
+****************************
+``comparesLt(left, right, k, isLessThan, testIsTrue)`` holds if ``left < right + k`` evaluates to ``isLessThan`` when the expression evaluates to ``testIsTrue``.
+

docs/language/learn-ql/cpp/ql-for-cpp.rst

@@ -31,6 +31,23 @@ These topics provide an overview of the QL C/C++ standard libraries and show exa
 
 -  :doc:`Example: Checking for allocations equal to strlen(string) without space for a null terminator <zero-space-terminator>` shows how a query to detect this particular buffer issue was developed.
 
+Advanced libraries
+----------------------------------
+
+.. toctree::
+   :hidden:
+
+   guards
+   range-analysis
+   value-numbering-hash-cons
+
+- :doc:`Using the guards library in C and C++ <guards>` demonstrates how to identify conditional expressions that control the execution of other code and what guarantees they provide.
+
+- :doc:`Using range analysis for C and C++ <range-analysis>` demonstrates how to determine constant upper and lower bounds and possible overflow or underflow of expressions.
+
+- :doc:`Using hash consing and value numbering for C and C++ <value-numbering-hash-cons>` demonstrates how to recognize expressions that are syntactically identical or compute the same value at runtime.
+
+
 Other resources
 ---------------
 

docs/language/learn-ql/cpp/range-analysis.rst

@@ -0,0 +1,39 @@
+Using range analysis for C and C++
+==================================
+
+Overview
+--------
+Range analysis determines upper and lower bounds for an expression.
+
+The range analysis library (defined in ``semmle.code.cpp.rangeanalysis.SimpleRangeAnalysis``) provides a set of predicates for determining constant upper and lower bounds on expressions, as well as recognizing integer overflows. For performance, the library performs automatic widening and therefore may not provide the tightest possible bounds.
+
+Bounds predicates
+-----------------
+The ``upperBound`` and ``lowerBound`` predicates provide constant bounds on expressions. No conversions of the argument are included in the bound. In the common case that your query needs to take conversions into account, call them on the converted form, such as ``upperBound(expr.getFullyConverted())``.
+
+Overflow predicates
+-------------------
+``exprMightOverflow`` and related predicates hold if the relevant expression might overflow, as determined by the range analysis library. The ``convertedExprMightOverflow`` family of predicates will take conversions into account.
+
+Example
+-------
+This query uses ``upperBound`` to determine whether the result of ``snprintf`` is checked when used in a loop.
+
+.. code-block:: ql
+
+    from FunctionCall call, DataFlow::Node source, DataFlow::Node sink, Expr convSink
+    where
+      // the call is an snprintf with a string format argument
+      call.getTarget().getName() = "snprintf" and
+      call.getArgument(2).getValue().regexpMatch(".*%s.*") and
+
+      // the result of the call influences its size argument in later iterations
+      TaintTracking::localTaint(source, sink) and
+      source.asExpr() = call and
+      sink.asExpr() = call.getArgument(1) and
+
+      // there is no fixed bound on the snprintf's size argument
+      upperBound(convSink) = typeUpperBound(convSink.getType().getUnspecifiedType()) and
+      convSink = call.getArgument(1).getFullyConverted()
+
+    select call, upperBound(call.getArgument(1).getFullyConverted())

docs/language/learn-ql/cpp/value-numbering-hash-cons.rst

@@ -0,0 +1,106 @@
+Hash consing and value numbering
+=================================================
+Overview
+--------
+In C and C++ QL databases, each node in the abstract syntax tree is represented by a separate object. This allows both analysis and results display to refer to specific appearances of a piece of syntax. However, it is frequently useful to determine whether two expressions are equivalent, either syntactically or semantically.
+
+The `hash consing <https://en.wikipedia.org/wiki/Hash_consing>`__ library (defined in ``semmle.code.cpp.valuenumbering.HashCons``) provides a mechanism for identifying expressions that have the same syntactic structure. The `global value numbering <https://en.wikipedia.org/wiki/Value_numbering>`__ library (defined in ``semmle.code.cpp.valuenumbering.GlobalValueNumbering``) provides a mechanism for identifying expressions that compute the same value at runtime.
+
+Both libraries partition the expressions in each function into equivalence classes represented by objects. Each ``HashCons`` object represents a set of expressions with identical parse trees, while ``GVN`` objects represent sets of expressions that will always compute the same value.
+
+
+Example C code
+--------------
+
+In the following C program, ``x + y`` and ``x + z`` will be assigned the same value number but different hash conses.
+
+.. code-block:: c
+
+    int x = 1;
+    int y = 2;
+    int z = y;
+    if(x + y == x + z) {
+      ...
+    }
+
+However, in the next example, the uses of ``x + y`` will have different value numbers but the same hash cons.
+
+.. code-block:: c
+
+    int x = 1;
+    int y = 2;
+    if(x + y) {
+      ...
+    }
+
+    x = 2;
+
+    if(x + y) {
+      ...
+    }
+
+Value numbering
+---------------
+The value numbering library (defined in ``semmle.code.cpp.valuenumbering.GlobalValueNumbering``) provides a mechanism for identifying expressions that compute the same value at runtime. Value numbering is useful when your primary concern is with the values being produced or the eventual machine code being run. For instance, value numbering might be used to determine whether a check is being done against the same value as the operation it is guarding.
+
+The value numbering API
+~~~~~~~~~~~~~~~~~~~~~~~
+The value numbering library exposes its interface primarily through the ``GVN`` class. Each instance of ``GVN`` represents a set of expressions that will always evaluate to the same value. To get an expression in the set represented by a particular ``GVN``, use the ``getAnExpr()`` member predicate.
+
+To get the ``GVN`` of an ``Expr``, use the ``globalValueNumber`` predicate.
+
+.. note::
+
+    While the ``GVN`` class has ``toString`` and ``getLocation`` methods, these are only provided as debugging aids. They give the ``toString`` and ``getLocation`` of an arbitrary ``Expr`` within the set.
+
+Why not a predicate?
+~~~~~~~~~~~~~~~~~~~~
+The obvious interface for this library would be a predicate ``equivalent(Expr e1, Expr e2)``. However, this predicate would be very large, with a quadratic number of rows for each set of equivalent expressions. By using a class as an intermediate step, the number of rows can be kept linear, and therefore can be cached.
+
+Example Queries
+~~~~~~~~~~~~~~~
+
+This query uses the ``GVN`` class to identify calls to ``strncpy`` where the size argument is derived from the source rather than the destination
+
+.. code-block:: ql
+
+    from FunctionCall strncpy, FunctionCall strlen
+    where
+      strncpy.getTarget().hasGlobalName("strncpy") and
+      strlen.getTarget().hasGlobalName("strlen") and
+      globalValueNumber(strncpy.getArgument(1)) = globalValueNumber(strlen.getArgument(0)) and
+      strlen = strncpy.getArgument(2)
+    select ci, "This call to strncpy is bounded by the size of the source rather than the destination"
+
+.. TODO: a second example
+
+Hash consing
+------------
+The hash consing library (defined in ``semmle.code.cpp.valuenumbering.HashCons``) provides a mechanism for identifying expressions that have the same syntactic structure. Hash consing is useful when your primary concern is with the text of the code. For instance, hash consing might be used to detect duplicate code within a function.
+
+The hash consing API
+~~~~~~~~~~~~~~~~~~~~
+The hash consing library exposes its interface primarily through the ``HashCons`` class. Each instance of ``HashCons`` represents a set of expressions within one function that have the same syntax (including referring to the same variables). To get an expression in the set represented by a particular ``HashCons``, use the ``getAnExpr()`` member predicate.
+
+.. note::
+
+    While the ``HashCons`` class has ``toString`` and ``getLocation`` methods, these are only provided as debugging aids. They give the ``toString`` and ``getLocation`` of an arbitrary ``Expr`` within the set.
+
+To get the ``HashCons`` of an ``Expr``, use the ``hashCons`` predicate.
+
+Examples
+~~~~~~~~
+
+.. TODO: prose explanations
+
+.. code-block:: ql
+
+    import cpp
+    import semmle.code.cpp.valuenumbering.HashCons
+
+    from IfStmt outer, IfStmt inner
+    where
+      outer.getElse+() = inner and
+      hashCons(outer.getCondition()) = hashCons(inner.getCondition())
+    select inner.getCondition(), "The condition of this if statement duplicates the condition of $@",
+      outer.getCondition(), "an enclosing if statement"