[jotagalindo]

Hi!

This comes from tiltedphoques/TiltedEvolution#945.

I play on a 5120x1440 monitor and the F2 menu was almost impossible to use: it showed up in pieces and clicks didn't respond, especially outdoors. At 1920x1080 it worked perfectly and used AI to make this fix, Claude specifically. I tested it myself in game.

What was happening

• View upload: OverlayRenderHandlerD3D11 uploaded the whole view to the GPU on every CEF paint. At this resolution that is about 30 MB each time, and with low FPS several paints pile up per frame.
• CEF waiting on Skyrim: CEF draws the UI on the GPU and copies each frame back to memory. Outdoors Skyrim keeps the GPU maxed out, so CEF ends up waiting and drops frames: cef_debug.log shows Client rejects proposal to capture frame.

What changes

• OverlayRenderHandlerD3D11: uploads to the GPU only the area that changed, at most once per frame. It keeps its name and interface, so TiltedEvolution only needs the submodule bump.
• CEF starts with disable-gpu and disable-gpu-compositing (in OverlayApp): the UI is simple HTML and CSS, so drawing it on the CPU is cheap. This applies to anything that uses OverlayApp.
• View size: taken from the swap chain. Before, it came from the bound render target, which can be 512x512 while loading.
• Cursor: scaled by height instead of width; before, it was huge on ultrawide screens.

Testing: the same code, as part of tiltedphoques/TiltedEvolution#945, was built on windows-2022 and tested in game at 5120x1440 indoors and outdoors, on a 2080 Super. This TiltedUI version builds with TiltedEvolution on windows-2022 (build);

Before and after (5120x1440, outdoors in Whiterun)

Before (current dev): the menu shows up half drawn, parts of it come and go, and the cursor is huge.

f2-ultrawide-before-fix.mp4

After (this PR): the connect dialog and the settings show up complete right away, and the cursor has its normal size.

f2-ultrawide-after-fix.mp4

Thanks a lot for everything, and all feedback is welcome!

[Force67]

• Out-of-bounds read when the view shrinks: a dirty rect pending at the old size survives the resize (UploadDirtyPixels skips while pixel and texture sizes differ), then the size-change branch in OnPaint unions it with (0, 0, newW, newH). The box ends up larger than m_pixels and the texture, so UpdateSubresource reads past the vector. Set m_hasDirtyPixels = false before AddDirtyRect in that branch, and maybe clamp the box in UploadDirtyPixels too.
• disable-gpu / disable-gpu-compositing hit every OverlayApp user. Software raster of full repaints at 5120x1440 costs CPU, so please also check 1080p on a weaker CPU. Opt-in per app might be safer.
• The comment saying the back buffer matches the window client area isn't always true (the ini resolution can differ, mouse coords come from ScreenToClient). Using the swap chain is still better, just soften the comment.
• m_pixelLock is held during a full 30 MB UpdateSubresource, which stalls OnPaint. Fine for now.
• GetRenderTargetSize() no longer reads the render target, rename it.
• Comments are long, the ones in OverlayApp.cpp and on the class could be a lot shorter.

[miredirex]

@jotagalindo do you happen to know what exactly fixed the stutter (low fps)? If it's the "disable" flags, then yes it's worth keeping

Dirty rects are a good fix too (#3 can be closed once this is merged)

[jotagalindo]

@miredirex both helped, in different ways:

• Dirty rects: before, the whole view was uploaded on every paint, about 30 MB at 5120x1440. With only this change the F2 menu got better, but outdoors (in Whiterun) it still showed up half drawn.
• The disable-gpu / disable-gpu-compositing flags is what finished the fix. cef_debug.log showed Client rejects proposal to capture frame right when the UI changed, so CEF was dropping frames while the game kept the GPU maxed out. With the flags that message went away and the menu shows up complete right away.

I didn't measure FPS for each change separately; what I checked was that the menu showed up complete and responded to clicks. I can measure it with and without the flags if that helps.

[jotagalindo]

@Force67 Thanks for the review. Everything is addressed in 15210c2:

• Out-of-bounds read when the view shrinks: on a size change, the dirty area of the old size is dropped before the new view is marked dirty, and the box in UploadDirtyPixels is also always clamped to the view.
• Back buffer comment: softened; it no longer says it always matches the window's client area.
• GetRenderTargetSize() is now UpdateViewSize().
• Comments: shortened, including the one on the class. The one in OverlayApp.cpp is gone.
• m_pixelLock during UpdateSubresource: left as is, as you said.

I will write more in next comments.

[jotagalindo]

@miredirex About the measurements, I made another build in my TiltedEvolution fork to measure. It has this commit plus two things that are only there for testing: a counter of the UI frames CEF hands to the game each second, and a file that, when present, removes the flags. That way both modes use the same client and the same server (release).

• PC: i9-9900K (8 cores, 16 threads) and RTX 2080 Super.
• CPU sampled every 2 s from each process's CPU time, as % of the whole CPU. TPProcess are CEF's processes.
• In every session: Whiterun, Breezehome and outdoors, moving through the menu.

	5120x1440, with flags	5120x1440, without flags	1080p, 4 threads, with flags
TPProcess CPU while the UI changes (avg / max)	1.0% / 2.2%	2.6% / 6.7%	0.9% / 1.6% (3.6% / 6.4% of the 4 threads)
TPProcess CPU while the UI is idle	0.0%	0.9%	0.1%
Frames dropped by CEF (Client rejects proposal to capture frame)	0	191	0
UI frames per second, max	38	9.5	37

• I don't have a weaker CPU:, so I simulated it on my i9-9900K by limiting the game and all of CEF's processes to 4 threads (4 cores without HyperThreading). It isn't the same as a really slow CPU, since those cores still run at the 9900K's clock (up to 5 GHz). In the next few days I'll be playing with a friend who has an older PC, and I'll try to measure how it goes for him too.
• With 4 threads, the game's CPU didn't go up with the menu open (10.0% vs 9.5% with the UI idle).

With these numbers, drawing on the CPU costs little even with 4 threads, and less than on the GPU, probably (i think) because on the GPU CEF has to copy every frame back to memory.

So the flags are what fix the stutter. Dirty rects lower the cost of uploading the view, but on their own they aren't enough.

[jotagalindo]

@Force67 @miredirex with these numbers, how would you prefer the flags? I think we have two options

• A. Opt-in (what the PR does now): OverlayApp doesn't add them, and each app turns them on in its own OnBeforeCommandLineProcessing. TiltedEvolution would do it with an OverlayApp subclass, which is already written in Make the F2 UI usable on ultrawide screens TiltedEvolution#945. I could reopen that PR with just that change plus the submodule bump.
• B. On by default in TiltedUI (as this PR had it before this review): OverlayApp adds them for everyone again, and TiltedEvolution only needs the submodule bump.

The measurements answer the CPU cost question. Not forcing it on everyone who uses OverlayApp is more of a design decision, so I'll leave it up to you. The effect in game is the same either way it's the same two flags, only where they're added changes.

[miredirex]

I think we have two options

I don't think this matters too much, really. Whatever is simpler code wise

I could reopen that PR with just that change plus the submodule bump

That's totally fine too

[jotagalindo]

@miredirex thanks! Then I'll go with option A, in TiltedUI the flags are opt-in as @Force67 suggested and TiltedEvolution turns them on with its own subclass. I've already reopened tiltedphoques/TiltedEvolution#945 for that, and its submodule is already prepared.

[Force67]

LGTM