Image/Video Background Refactor #831
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Make clicks working
+ Fix UI layout resized incorrectly + Fix UI state updated incorrectly + Cleaning-up constants and unused/duplicate methods + Add option to select pip by hovering instead of clicking + Add event handlers for ItemIndex and ItemsCount changes
+ By preloading it to an empty grid, this issue can be fixed. Yeah, for no reason, even when the element has already been added to the ContentPresenter, the first frame of the element will report the offset and size to 0x0. So by tricking it to load into an empty grid, the element will force itself to load and reports the correct offset and size.
The countdown is currently using Timer for counting down the slideshow. This approach, however, uses interval event on each ticks so the duration of the count might be inaccurate (but not significant). The tick resolution is 100ms for now.
+ CodeQA + Simplify content placement on XAML + Fade-in/out countdown bar on entering and exiting + Fix content not updating while adding/removing items from Items collection
+ Implements Parallax effect on mouse hover + The image is currently using a dummy one and not yet compositing. Will be fully implemented later (ETA: at the end of this week, hopefully)
+ Bug fix on object -> double conversion, causing throw + Add IMediaCacheHandler interface for handling cache on media loading
Now we are able to perform direct frame drawing without dependency on CanvasBitmap or any SoftwareBitmap which requires back-to-back GPU->CPU->GPU frame copy. The GPU usage difference is negligible, as close as mainstream video player GPU usage.
Member
|
Fatal crash on region change |
Member
Author
As mentioned above, the code for these changes haven't been applied to main Collapse Launcher code yet and still on |
Member
|
As per @neon-nyan statement above, copilot code check is removed until PR is actually finished in Collapse side Cc @Cryotechnic |
+ This also occurs during element ``Unloaded`` event + The loader will perform media seeking to the last position if the previous source was the same
+ Copying all controls to main Collapse Launcher code. Adjustment in the background mechanism is still required and not done yet
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Main Goal
As described by the title, this PR contains major changes to Collapse Launcher's Image/Video Background mechanism to match current behavior on HoYoPlay which have multiple sequence of background and layered images.
Here's the current list of what have been implemented:
Control-based element.PipsPager(Implemented as:NewPipsPager)FlipView(Implemented asPanelSlideshow)This implementation of a
Slideshowelement added some customization whichFlipViewcan't accommodate, including:Transitionanimation for navigating between elements.SlideshowDuration.LayeredBackgroundImage)This custom control has major improvements than its previous background loading mechanism, including:
Built-in external decoder support for unsupported WIC (Windows Imaging Component) formats, including
.webp,.avifand.jxrviaPhotoScalerlibrary.Using direct Bitmap copy for drawing video frames via native
ISurfaceImageSourceNativeWithD2D.Previously, Collapse Launcher has to rely on software bitmaps to draw each frames into the
UIElementto pulls of the XAML-effects and compositions, including XAML Blur.The reason why Collapse Launcher has to perform this multiple frame copy is because
MediaPlayer, by default, doesn't support XAML effects and Composition and instead, each frames were drawn directly into different pipeline ofIDXGISwapChainwhich decoupled from the main WinUI's, causing the element above it feels a bit off (including no acrylic effects for any elements above it)This causes few overhead, in-terms of memory allocation or CPU cycles as it requires to perform multiple buffer copy steps:
SoftwareBitmap(CPU)MediaPlayer's frame buffer intoSoftwareBitmap(GPU -> CPU)VirtualCanvasImageSourceas source forImageto copy fromSoftwareBitmap(GPU)SoftwareBitmapbuffer intoVirtualCanvasImageSource(CPU -> GPU)Image's surface (GPU)Now, thanks to huge changes on
Hi3Helper.Win32submodule (and documentations provided by Win2D and DirectNAot), we managed to copy the frames fromMediaPlayerdirectly into backed-VirtualSurfaceImageSource/ISurfaceImageSourceNativeWithD2Dpipeline, loaded asImage's source, with XAML-effects and Composition support and without additional buffer copy on CPU side. Now, the process only require these steps:ISurfaceImageSourceNativeWithD2D->BeginDrawto createIDirect3DSurfacedrawing session (GPU)MediaPlayer's frame buffer intoIDirect3DSurface(GPU -> GPU)Image's surface viaISurfaceImageSourceNativeWithD2D->EndDraw(GPU)TODO List:
MediaPlayer,ISurfaceImageSourceNativeWithD2Dand underlayID3D11Device + ID2D1Device[Optional] Use fully-asynchronous drawing session onCurrently not possible due to SurfaceImageSource thread model bound to the UI-Thread and not set to Multi-thread mode.ISurfaceImageSourceNativeWithD2D(Detaching draw session from UI-thread).PR Status :
Templates
Changelog Prefixes