stream updates for performance

[rnewson]

Overview

Nouveau switched from ibrowse to gun (and http/1.1 to http/2) in 3.5.0, in order to reduce the large number of connections made between couchdb and nouveau server. A user has found, on a larger test than I performed during code development, a significant indexing speed regression.

Before the ibrowse to gun transition the update requests to nouveau server used http pipelining (that is, multiple requests were made to the server, in order, without waiting for the responses). This was a significant optimization. With gun this was not possible (as http/2 uses multiplexing instead). The difference turns out much more significant that expected.

This PR adds a new endpoint on the nouveau server that handles a streamed request body in application/json-seq format. Each sequence of the request body is a document update or delete request. This reintroduces a concept like pipelining but within a single http request, and thus ordering is preserved. At each row upload we check to see if there has been an error response (the nouveau server will start the HTTP response only on error or at the end of the index updating process).

This has demonstrated a substantial performance improvement.

There is work still to do but am making a PR to make it easier for others to try this out. Remaining work includes ensuring a smoosh upgrade path and annotating the deprecated parts of nouveau server for the release after the one that introduces the streaming updates endpoint.

Testing recommendations

Will be covered by automated tests

Related Issues or Pull Requests

#5894

Checklist

• This is my own work, I did not use AI, LLM's or similar technology
• Code is written and works correctly
• Changes are covered by tests
• Any new configurable parameters are documented in rel/overlay/etc/default.ini
• Documentation changes were made in the src/docs folder
• Documentation changes were backported (separated PR) to affected branches