PROOF OF CONCEPT: Fixing 10bit rendering support

[pbergeron-adsk]

Linked issues

SG-XXXXX

Summarize your change and reason for the change

NOTE: THIS CANNOT MAKE RV 2025.1 RELEASE

NOTE: THIS "FIX" IS A PROOF OF CONCEPT TO SHOW A BASELINE SOLUTION TO MAKE 10-BIT RENDERING WORK AGAIN. IT IS NOT PRODUCTION READY, ITS NOT PERFECT, ITS INCOMPLETE, AND IMPLEMENTS A REFACTOR/WORKAROUND THAT I DO NOT LIKE.

RV displayed visible banding artifacts when rendering 10-bit and 12-bit content, appearing quantized to 8-bit despite proper file loading, and despite enabling 10-bit support when loading DPX files. This is a regression when compared to RV 2024.1, where things worked ok.

The main difference, and the root cause, is that RV 2024.1 uses QGLWidget (which renders directly to the screen's pixels) and the Qt6 branch which uses QOpenGLWidget (which renders to an offscreen FBO which is then transferred to the Qt application compositor). The Qt6 approach is generally better code and allows better integration of heterogenous widgets regardless of their rendering method).

Indeed, the root cause appears to be the Qt6 compositor using the FBO content rendered into QOpenGLWidgets, which, even though might have a 10-10-10-2 OpenGL context and 10-10-10-2 backend FBO, appears to quantize everything back to 8-8-8-8 when transferring the 10-bit FBO to the compositor, which makes the entire use of using a 10-bit context, and 10-bit FBO, completely pointless and that's why we see 8-bit banding of 10-bit content.

After countless ways of trying to make this work with the compositor, the only way I've found to fix this (and this is as per Qt support's suggestion as well) is to actually to bypass the compositor entirely and make GLView a QOpenGLWindow instead of a QOpenGLWidget. Indeed, a QOpenGLWindow does not have a FBO backing surface at all, and renders to the screen surface directly; so if we've enabled a 10-10-10-2 context, and we use GL_RGB10_A2 textures and DPX content, then things actually draw with high precision depth color on the screen.

CHANGES

1. Change GLView to inherit from QOpenGLWindow vs QOpenGLWidget.

2. Use a special QWindowContainer to "host" a QOpenGLWindow in the RV document layout. The QWindowContainer is a special widgets that moves, resizes and constrains the dimensions of a "floating" window so that it always "appears" to be a child of the layout. But I dont think it's actually a child, It just "appears" this way.

3. Because the API of a QWidget is different than QWindow, we must redirect some calls to the WindowContainer, and some redirect to the view.

4. This introduces changes to things like QTTranslator, QTVideoDevice, and other things that we'd rather not touch. We have to do some more manipulations to handle window focus, keyboard focus, etc.

5. This introduces some changes to Mu code that assumes the Mu function "mainViewWidget()" is actually the GLView. That is no longer the case. The mainViewWidget is in fact the QWindowContainer. So we cannot call exposed functions in it. Instead we must redirect some calls to the window when we used to do it from the widget. We must also adjust other Mu functions that call into the GLView directly. Mu Code can still call normal functions like getting dimensions of the window container

CAVEATS

6. The changes proposed in this PR may not all be required / necessary. For example in this PR we have (maybe unnecessary) code that sets the default preferences values to 10-10-10-2 instead of 8-8-8-8, and PLANAR_RGB8 to PLANAR_RGB16. These may not be required. We may be able to leaver them to what they were previously.

7. On macOS, this requires a patched/recompiled libqcocoa.dylib to be placed in the Qt6 and RV folder, because it's not possible to create a 10-10-10-2 OpenGL context. Copy the file like this:

• Patched file: /Volumes/Qt6_5_3/qt-build/qtbase/plugins/platforms/libqcocoa.dylib
• Copied to: /Users/{username}/Qt6_5_3/6.5.3/macos/plugins/platforms
• Copied to: /Users/{username}/RV/_build/stage/app/RV.app/Contents/PlugIns/Qt/platforms
• Copied to: /Users/{username}/RV/_build_debug/stage/app/RV.app/Contents/PlugIns/Qt/platforms

Note: "/Volumes/Qt_6_5_3" is a mounted shared folder on the Qt 6.5.3 installation folder on the Qt build machine

8. Some files may be dirty and not optimal solution. For example, "QTGLVideoDevice.cpp" tries to be compatible with being attached to either a QWidget or QWindow. I'm not sure this is necessary or desirable. I

Describe what you have tested and on which operating system.

macOS.

Add a list of changes, and note any that might need special attention during the review.

If possible, provide screenshots.