Keep the main codebase in container ephemeral storage

[mcronce]

This has several advantages

• Faster startup for initial container and during upgrades
  • On my system (shared storage for /var/www/html was cephfs on 2x10Gb network and consumer-grade NVMe drives), updates between 29, 30, 31, and 32 went from ~135 minutes to ~5 minutes. The main culprits for the difference were the rsync step and the code integrity check.
• Improved performance when volume storage is slower than container storage. This is common in cases where multiple replicas are running - especially on multiple machines, e.g. Kubernetes, because shared mutable network storage is necessarily slower than local. Even with the Linux filesystem cache, it needs to cache less optimistically than with a local filesystem to ensure consistency.
  • On my system, fully loading the login screen went from ~18 seconds to ~800ms
• More idiomatic container setup - the application is contained in the container. This means that tampering with application code isn't permanent, without an explicit code integrity check step.

It also may fix #2044; I don't want to speak for the person who originally filed it, but it sounded like they were asking for the same thing.

This is, obviously, a breaking change - anybody currently mounting in all of /var/www/html needs to now mount in several subdirectories instead. I'm not sure whether the breaking change is preferable over the maintenance burden of having a separate set of images; if a separate set of images is desired, I'll redo it that way. Notably, the helm chart already mounts them all in separately; the only change needed there is to no longer mount in /var/www and /var/www/html. I haven't looked for docker-compose files or anything like that.

Other than that, I don't believe I broke anything. The example theme and any changed "built-in" config/custom apps will still be copied over when the version changes.