Update README and code comments (#391)

* Update README and code comments

* more

* PR comments

* Add cleanup docs

* Fix spelling

* pr comments

Author: Charles-Gagnon

README.md

@@ -58,6 +58,9 @@ Azure SQL bindings for Azure Functions are supported for:
       - [Primary Keys and Identity Columns](#primary-keys-and-identity-columns)
     - [Trigger Binding](#trigger-binding)
       - [Change Tracking](#change-tracking)
+      - [Internal State Tables](#internal-state-tables)
+        - [az_func.GlobalState](#az_funcglobalstate)
+        - [az_func.Leases_*](#az_funcleases_)
       - [Trigger Samples](#trigger-samples)
   - [Known Issues](#known-issues)
   - [Telemetry](#telemetry)
@@ -869,6 +872,34 @@ The trigger binding utilizes SQL [change tracking](https://docs.microsoft.com/sq
 
     > **NOTE:** The leases table contains all columns corresponding to the primary key from the user table and three additional columns named `_az_func_ChangeVersion`, `_az_func_AttemptCount` and `_az_func_LeaseExpirationTime`. If any of the primary key columns happen to have the same name, that will result in an error message listing any conflicts. In this case, the listed primary key columns must be renamed for the trigger to work.
 
+#### Internal State Tables
+
+The trigger functionality creates several tables to use for tracking the current state of the trigger. This allows state to be persisted across sessions and for multiple instances of a trigger binding to execute in parallel (for scaling purposes).
+
+In addition, a schema named `az_func` will be created that the tables will belong to.
+
+The login the trigger is configured to use must be given permissions to create these tables and schema. If not, then an error will be thrown and the trigger will fail to run.
+
+If the tables are deleted or modified, then unexpected behavior may occur. To reset the state of the triggers, first stop all currently running functions with trigger bindings and then either truncate or delete the tables. The next time a function with a trigger binding is started, it will recreate the tables as necessary.
+
+##### az_func.GlobalState
+
+This table stores information about each function being executed, what table that function is watching and what the [last sync state](https://learn.microsoft.com/sql/relational-databases/track-changes/work-with-change-tracking-sql-server) that has been processed.
+
+##### az_func.Leases_*
+
+A `Leases_*` table is created for every unique instance of a function and table. The full name will be in the format `Leases_<FunctionId>_<TableId>` where `<FunctionId>` is generated from the function ID and `<TableId>` is the object ID of the table being tracked. Such as `Leases_7d12c06c6ddff24c_1845581613`.
+
+This table is used to ensure that all changes are processed and that no change is processed more than once. This table consists of two groups of columns:
+
+   * A column for each column in the primary key of the target table - used to identify the row that it maps to in the target table
+   * A couple columns for tracking the state of each row. These are:
+     * `_az_func_ChangeVersion` for the change version of the row currently being processed
+     * `_az_func_AttemptCount` for tracking the number of times that a change has attempted to be processed to avoid getting stuck trying to process a change it's unable to handle
+     * `_az_func_LeaseExpirationTime` for tracking when the lease on this row for a particular instance is set to expire. This ensures that if an instance exits unexpectedly another instance will be able to pick up and process any changes it had leases for after the expiration time has passed.
+
+A row is created for every row in the target table that is modified. These are then cleaned up after the changes are processed for a set of changes corresponding to a change tracking sync version.
+
 #### Trigger Samples
 The trigger binding takes two [arguments](https://github.com/Azure/azure-functions-sql-extension/blob/main/src/TriggerBinding/SqlTriggerAttribute.cs)
 

src/TriggerBinding/SqlTableChangeMonitor.cs

@@ -331,14 +331,17 @@ private async Task ProcessTableChangesAsync(SqlConnection connection, Cancellati
 
                 try
                 {
-                    // What should we do if this fails? It doesn't make sense to retry since it's not a connection based
-                    // thing. We could still try to trigger on the correctly processed changes, but that adds additional
-                    // complication because we don't want to release the leases on the incorrectly processed changes.
-                    // For now, just give up I guess?
                     changes = this.ProcessChanges();
                 }
                 catch (Exception e)
                 {
+                    // Either there's a bug or we're in a bad state so not much we can do here. We'll try clearing
+                    //  our state and retry getting the changes from the top again in case something broke while
+                    // fetching the changes.
+                    // It doesn't make sense to retry processing the changes immediately since this isn't a connection-based issue.
+                    // We could probably send up the changes we were able to process and just skip the ones we couldn't, but given
+                    // that this is not a case we expect would happen during normal execution we'll err on the side of caution for
+                    // now and just retry getting the whole set of changes.
                     this._logger.LogError($"Failed to compose trigger parameter value for table: '{this._userTable.FullName} due to exception: {e.GetType()}. Exception message: {e.Message}");
                     TelemetryInstance.TrackException(TelemetryErrorName.ProcessChanges, e, this._telemetryProps);
                     await this.ClearRowsAsync();