authorclover caruso <git@paperclover.net> 2026-10-06 22:08:37-07:00
committerclover caruso <git@paperclover.net> 2026-10-06 23:11:29-07:00
log2f1eb7876ce087844fe59c21ca41f1eb139de473
tree8fcadaab87180f3a3c8b9a5bbd47d0c6a7f0f1be
parent9e9eefa9e34d0067f716831ef397d23ecae5c895
signature Signed by SSH key SHA256:52mNGHRsVFBDED9IAX5pe+LRWUefqTbxEReunq21QvU (~clover)

Fix Windows surface capture and accelerate desktop streaming

Use cursor-free monitor capture on modern Windows, drain to the latest frames, and retain the GDI compatibility fallback. Align UWP window content extents, keep shell popup stacking correct, and close rejected capture sessions. Initialize shell surfaces on bounded workers, preserve covered background pixels, and verify caption hit regions before optimistic dragging. Validated fresh Calculator and Store launches, Start/taskbar stacking, resizing, dragging, native cursor reconnects, and idle capture on Windows 10/11. Animated browser video reached 37–44 FPS with no dropped frames; inference sustained 42.9 tokens/sec. x86/x64/ARM64 imports and subsystem checks plus cursor ASan/UBSan tests pass. Assisted-by: gpt-6.1-sol

4 files changed, 301 insertions(+), 67 deletions(-)

guest/DESIGN.md+4-4
...@@ -1,10 +1,10 @@...@@ -1,10 +1,10 @@
1# Guest display stream1# Guest display stream
22
3The executable is freestanding C99. `cursor.c` contains pixel conversion and dirty-rectangle detection; `windows.c` owns desktop capture and transport; `capture.c` implements optional window surfaces. Zig supplies the cross compiler and Windows headers, with no Zig or C runtime in the executable. x86 and x64 target Windows 7; ARM64 targets Windows 10. PE subsystem versions and imported DLLs are checked by `python3 guest/test.py`. Runtime validation uses Windows 10 and 11; Windows 7 runtime behavior is unverified.3The executable is freestanding C99. `cursor.c` contains pixel conversion and dirty-rectangle detection; `windows.c` owns desktop capture and transport; `capture.c` implements modern desktop capture and optional window surfaces. Zig supplies the cross compiler and Windows headers, with no Zig or C runtime in the executable. x86 and x64 target Windows 7; ARM64 targets Windows 10. PE subsystem versions and imported DLLs are checked by `python3 guest/test.py`. Runtime validation uses Windows 10 and 11; Windows 7 runtime behavior is unverified.
44
5Build with `python3 guest/build.py --arch x86_64` (also `x86` and `aarch64`). In an elevated PowerShell session inside the logged-in administrator's desktop, run `./install.ps1 -Binary ./build/snowglobe-guest-x86_64.exe`. The installer uses Task Scheduler COM APIs available to PowerShell 2, installs into protected Program Files, and starts an elevated task at that user's logon. `-Remove` removes that task and executable. Elevation is required by the VirtIO serial driver's device ACL; capture must run in the interactive session, not session 0. The guest needs a compatible VirtIO serial driver and the channel named in `protocol.json`. A standard-user launcher requires a privileged handle broker; it is not part of this first backend.5Build with `python3 guest/build.py --arch x86_64` (also `x86` and `aarch64`). In an elevated PowerShell session inside the logged-in administrator's desktop, run `./install.ps1 -Binary ./build/snowglobe-guest-x86_64.exe`. The installer uses Task Scheduler COM APIs available to PowerShell 2, installs into protected Program Files, and starts an elevated task at that user's logon. `-Remove` removes that task and executable. Elevation is required by the VirtIO serial driver's device ACL; capture must run in the interactive session, not session 0. The guest needs a compatible VirtIO serial driver and the channel named in `protocol.json`. A standard-user launcher requires a privileged handle broker; it is not part of this first backend.
66
7`BitBlt` copies the primary desktop into a CPU DIB without drawing the cursor. RGB comparisons ignore the unused fourth byte. Unchanged pixels produce no screen packets; changed pixels produce one bounding rectangle. Cursor polling runs every millisecond; desktop capture targets 60 Hz. The host's existing encoder supplies WebRTC video. Secure desktops, RDP sessions that differ from the physical console, and capture failures return the viewer to QEMU capture.7Windows 10 build 19041 and later use Windows Graphics Capture for the primary monitor with cursor capture disabled. Older or unsupported systems use `BitBlt` into a CPU DIB without drawing the cursor. Only the latest available capture frames are read back; an unchanged desktop needs no readback. RGB comparisons ignore the unused fourth byte. Unchanged pixels produce no screen packets; changed pixels produce one bounding rectangle. Cursor polling runs every millisecond; desktop capture targets 60 Hz. The host's existing encoder supplies WebRTC video. Secure desktops, RDP sessions that differ from the physical console, and capture failures return the viewer to QEMU capture.
88
9`GetCursorInfo` supplies the shape, visibility, and observed position. `GetIconInfo` and black/white `DrawIconEx` renders recover RGBA and binary XOR planes, including colored XOR. Nonbinary color XOR and unsupported cursor sizes explicitly fall back to the browser's default cursor, preserving video. Animation timing uses the optional `GetCursorFrameInfo` user32 export, resolved by name. Missing or unusable animation metadata falls back to a static frame. Animation starts a browser-local clock; Windows does not expose the current animation phase through this interface. Ordinary shapes use the browser cursor; XOR shapes use local layers with difference blending. Neither follows streamed positions.9`GetCursorInfo` supplies the shape, visibility, and observed position. `GetIconInfo` and black/white `DrawIconEx` renders recover RGBA and binary XOR planes, including colored XOR. Nonbinary color XOR and unsupported cursor sizes explicitly fall back to the browser's default cursor, preserving video. Animation timing uses the optional `GetCursorFrameInfo` user32 export, resolved by name. Missing or unusable animation metadata falls back to a static frame. Animation starts a browser-local clock; Windows does not expose the current animation phase through this interface. Ordinary shapes use the browser cursor; XOR shapes use local layers with difference blending. Neither follows streamed positions.
1010
...@@ -29,9 +29,9 @@ The host broker validates a running VM's libvirt-owned channel path. One guest r...@@ -29,9 +29,9 @@ The host broker validates a running VM's libvirt-owned channel path. One guest r
2929
30## Separate windows experiment30## Separate windows experiment
3131
32Windows 10 build 19041 and later can enable separate windows from the viewer’s View menu. Windows Graphics Capture supplies full unoccluded surfaces with cursor capture disabled. Its dependencies are loaded dynamically, preserving the Windows 7 import boundary. Desktop and taskbar surfaces use `PrintWindow` because Windows Graphics Capture rejects those shell HWNDs. Changed rectangles update a single host atlas and one existing video encoder. Metadata describes the atlas and stacking; the browser composes it and moves cached surfaces immediately during native caption drags. Atlas geometry is matched to the decoded RTP frame.32Windows 10 build 19041 and later can enable separate windows from the viewer’s View menu. Windows Graphics Capture supplies full unoccluded surfaces with cursor capture disabled. Its dependencies are loaded dynamically, preserving the Windows 7 import boundary. Desktop and taskbar surfaces initialize with a `PrintWindow` worker because Windows Graphics Capture rejects those shell HWNDs. Subsequent updates copy visible desktop spans, retaining covered pixels for local window dragging. Shell surfaces initialize before window capture sessions start, keeping capture indicators out of their initial pixels. Changed rectangles update a single host atlas and one existing video encoder. Metadata describes the atlas and stacking; the browser composes it and moves cached surfaces immediately during native caption drags. Atlas geometry is matched to the decoded RTP frame.
3333
34The experiment is off by default. Unsupported, layered, or oversized window sets fall back to desktop video. Secure desktops and session changes suspend guest capture. `PrintWindow` is synchronous; an unresponsive shell can interrupt the guest stream and trigger the host timeout. Protected content and custom caption behavior remain subject to the application’s capture support.34The experiment is off by default. Unsupported, layered, translucent, shell-popup, or oversized window sets close all window sessions and fall back to desktop video. Frame-pool resizing follows captured content dimensions; a window whose first or resized surface never arrives also falls back. Secure desktops and session changes suspend guest capture. Shell painting runs off the capture thread and falls back after 500 ms; its bounded drawing resources remain owned until the worker exits. Windows capture indicators remain subject to the OS; fallback disposes the sessions instead of leaving indicators attached to inactive windows. Protected content and custom caption behavior remain subject to the application’s capture support.
3535
36## Later backends36## Later backends
3737
guest/capture.c+275-51
...@@ -37,22 +37,35 @@ static const GUID dxgi_iid = {0x54ec77fa,0x1377,0x44e6,{0x8c,0x32,0x88,0xfd,0x5f...@@ -37,22 +37,35 @@ static const GUID dxgi_iid = {0x54ec77fa,0x1377,0x44e6,{0x8c,0x32,0x88,0xfd,0x5f
37static const GUID access_iid = {0xa9b3d012,0x3df2,0x4ee3,{0xb8,0xd1,0x86,0x95,0xf4,0x57,0xd3,0xc1}};37static const GUID access_iid = {0xa9b3d012,0x3df2,0x4ee3,{0xb8,0xd1,0x86,0x95,0xf4,0x57,0xd3,0xc1}};
38static const GUID texture_iid = {0x6f15aaf2,0xd208,0x4e89,{0x9a,0xb4,0x48,0x95,0x35,0xd3,0x4f,0x9c}};38static const GUID texture_iid = {0x6f15aaf2,0xd208,0x4e89,{0x9a,0xb4,0x48,0x95,0x35,0xd3,0x4f,0x9c}};
3939
40struct shell_paint {
41 HWND hwnd;
42 HANDLE thread;
43 struct shell_paint *next;
44 HDC dc;
45 HBITMAP dib, selected;
46 uint8_t *bits;
47 size_t pixels;
48 DWORD started;
49};
50
40struct window {51struct window {
41 HWND hwnd;52 HWND hwnd;
42 uint32_t id, width, height;53 uint32_t id, width, height;
43 struct size pool_size;54 struct size pool_size;
44 void *pool, *session;55 void *pool, *session;
45 ID3D11Texture2D *staging;56 ID3D11Texture2D *staging;
46 uint8_t *before, *bits;57 uint8_t *before;
47 HDC dc;58 struct shell_paint *paint;
48 HBITMAP dib, selected;
49 int initial, seen, shell;59 int initial, seen, shell;
60 DWORD waiting;
50};61};
51static struct window windows[SG_MAX_WINDOWS];62static struct window windows[SG_MAX_WINDOWS], desktop;
63static RECT bounds[SG_MAX_WINDOWS], capture_bounds[SG_MAX_WINDOWS];
52static uint32_t layout[SG_WINDOWS_WORDS + SG_MAX_WINDOWS * SG_WINDOW_WORDS];64static uint32_t layout[SG_WINDOWS_WORDS + SG_MAX_WINDOWS * SG_WINDOW_WORDS];
53static uint32_t prior[SG_WINDOWS_WORDS + SG_MAX_WINDOWS * SG_WINDOW_WORDS];65static uint32_t prior[SG_WINDOWS_WORDS + SG_MAX_WINDOWS * SG_WINDOW_WORDS];
54static uint32_t count, next_id;66static uint32_t count, next_id;
55static size_t allocated;67static size_t allocated, painted;
68static struct shell_paint *retired_paints;
56static int invalid;69static int invalid;
57static HANDLE heap;70static HANDLE heap;
58/* Capture workers outlive session disposal; retain their DLLs until process exit. */71/* Capture workers outlive session disposal; retain their DLLs until process exit. */
...@@ -65,7 +78,7 @@ static delete_string_fn delete_string;...@@ -65,7 +78,7 @@ static delete_string_fn delete_string;
65static factory_fn get_factory;78static factory_fn get_factory;
66static uninitialize_fn uninitialize;79static uninitialize_fn uninitialize;
67static attribute_fn attribute;80static attribute_fn attribute;
68static DWORD retry;81static DWORD retry, desktop_retry;
6982
70static void release(void *object) {83static void release(void *object) {
71 if (object) METHOD(object, 2, release_fn)(object);84 if (object) METHOD(object, 2, release_fn)(object);
...@@ -78,13 +91,41 @@ static void close_object(void *object) {...@@ -78,13 +91,41 @@ static void close_object(void *object) {
78 }91 }
79 release(object);92 release(object);
80}93}
94static void free_paint(struct shell_paint *paint) {
95 if (paint->thread) CloseHandle(paint->thread);
96 if (paint->selected) SelectObject(paint->dc, paint->selected);
97 if (paint->dib) DeleteObject(paint->dib);
98 if (paint->dc) DeleteDC(paint->dc);
99 painted -= paint->pixels;
100 HeapFree(heap, 0, paint);
101}
102static DWORD WINAPI paint_shell(void *data) {
103 struct shell_paint *paint = data;
104 BOOL success = PrintWindow(paint->hwnd, paint->dc, 2);
105 GdiFlush();
106 return success;
107}
108static void reap_paints(void) {
109 struct shell_paint **link = &retired_paints;
110 while (*link) {
111 struct shell_paint *paint = *link;
112 if (WaitForSingleObject(paint->thread, 0) == WAIT_OBJECT_0) {
113 *link = paint->next; free_paint(paint);
114 } else link = &paint->next;
115 }
116}
117static void detach_paint(struct window *window) {
118 struct shell_paint *paint = window->paint;
119 if (!paint) return;
120 window->paint = NULL;
121 if (!paint->thread || WaitForSingleObject(paint->thread, 0) == WAIT_OBJECT_0) free_paint(paint);
122 else { paint->next = retired_paints; retired_paints = paint; }
123}
124
81static void drop(struct window *window) {125static void drop(struct window *window) {
82 close_object(window->session); close_object(window->pool);126 close_object(window->session); close_object(window->pool);
83 if (window->staging) ID3D11Texture2D_Release(window->staging);127 if (window->staging) ID3D11Texture2D_Release(window->staging);
84 if (window->selected) SelectObject(window->dc, window->selected);128 detach_paint(window);
85 if (window->dib) DeleteObject(window->dib);
86 if (window->dc) DeleteDC(window->dc);
87 window->dc = NULL; window->dib = window->selected = NULL; window->bits = NULL;
88 if (window->before) { allocated -= (size_t)window->width * window->height; HeapFree(heap, 0, window->before); }129 if (window->before) { allocated -= (size_t)window->width * window->height; HeapFree(heap, 0, window->before); }
89 window->hwnd = NULL; window->session = window->pool = NULL;130 window->hwnd = NULL; window->session = window->pool = NULL;
90 window->staging = NULL; window->before = NULL;131 window->staging = NULL; window->before = NULL;
...@@ -103,26 +144,33 @@ static int factory(PCWSTR name, const GUID *iid, void **result) {...@@ -103,26 +144,33 @@ static int factory(PCWSTR name, const GUID *iid, void **result) {
103}144}
104145
105void sg_windows_stop(void) {146void sg_windows_stop(void) {
147 reap_paints();
106 uint32_t i;148 uint32_t i;
107 for (i = 0; i < SG_MAX_WINDOWS; ++i) drop(windows + i);149 for (i = 0; i < SG_MAX_WINDOWS; ++i) drop(windows + i);
150 count = 0;
151 retry = 0;
152 for (i = 0; i < sizeof(prior) / sizeof(prior[0]); ++i) prior[i] = 0;
153}
154
155void sg_capture_stop(void) {
156 sg_windows_stop();
157 drop(&desktop);
158 desktop_retry = 0;
108 release(pool_factory); release(interop); release(capture_device);159 release(pool_factory); release(interop); release(capture_device);
109 if (context) ID3D11DeviceContext_Release(context);160 if (context) ID3D11DeviceContext_Release(context);
110 if (device) ID3D11Device_Release(device);161 if (device) ID3D11Device_Release(device);
111 pool_factory = interop = capture_device = NULL; context = NULL; device = NULL;162 pool_factory = interop = capture_device = NULL; context = NULL; device = NULL;
112 if (uninitialize) uninitialize();163 if (uninitialize) uninitialize();
113 uninitialize = NULL;164 uninitialize = NULL;
114 count = 0;
115 retry = 0;
116 for (i = 0; i < sizeof(prior) / sizeof(prior[0]); ++i) prior[i] = 0;
117}165}
118166
119int sg_windows_start(void) {167int sg_capture_start(void) {
120 initialize_fn initialize;168 initialize_fn initialize;
121 typedef HRESULT (WINAPI *create_fn)(IDXGIAdapter *, D3D_DRIVER_TYPE, HMODULE, UINT, const D3D_FEATURE_LEVEL *, UINT, UINT, ID3D11Device **, D3D_FEATURE_LEVEL *, ID3D11DeviceContext **);169 typedef HRESULT (WINAPI *create_fn)(IDXGIAdapter *, D3D_DRIVER_TYPE, HMODULE, UINT, const D3D_FEATURE_LEVEL *, UINT, UINT, ID3D11Device **, D3D_FEATURE_LEVEL *, ID3D11DeviceContext **);
122 create_fn create;170 create_fn create;
123 device_fn wrap;171 device_fn wrap;
124 IDXGIDevice *dxgi = NULL;172 IDXGIDevice *dxgi = NULL;
125 sg_windows_stop();173 if (device) return 1;
126 heap = GetProcessHeap();174 heap = GetProcessHeap();
127 if (!runtime) runtime = LoadLibraryW(L"combase.dll");175 if (!runtime) runtime = LoadLibraryW(L"combase.dll");
128 if (!d3d) d3d = LoadLibraryW(L"d3d11.dll");176 if (!d3d) d3d = LoadLibraryW(L"d3d11.dll");
...@@ -148,7 +196,7 @@ int sg_windows_start(void) {...@@ -148,7 +196,7 @@ int sg_windows_start(void) {
148 if (FAILED(status)) goto failed;196 if (FAILED(status)) goto failed;
149 return 1;197 return 1;
150failed:198failed:
151 sg_windows_stop();199 sg_capture_stop();
152 return 0;200 return 0;
153}201}
154202
...@@ -173,11 +221,20 @@ static BOOL CALLBACK enumerate(HWND hwnd, LPARAM unused) {...@@ -173,11 +221,20 @@ static BOOL CALLBACK enumerate(HWND hwnd, LPARAM unused) {
173 GetClassNameW(hwnd, class_name, 32);221 GetClassNameW(hwnd, class_name, 32);
174 windows[i].shell = hwnd == GetShellWindow() ||222 windows[i].shell = hwnd == GetShellWindow() ||
175 !lstrcmpW(class_name, L"Shell_TrayWnd") || !lstrcmpW(class_name, L"Shell_SecondaryTrayWnd") || !lstrcmpW(class_name, L"WorkerW");223 !lstrcmpW(class_name, L"Shell_TrayWnd") || !lstrcmpW(class_name, L"Shell_SecondaryTrayWnd") || !lstrcmpW(class_name, L"WorkerW");
176 if (FAILED(attribute(hwnd, 9, &rect, sizeof(rect)))) GetWindowRect(hwnd, &rect);224 if ((!lstrcmpW(class_name, L"ApplicationFrameWindow") && !GetWindowTextLengthW(hwnd)) || !lstrcmpW(class_name, L"Windows.UI.Core.CoreWindow") || !lstrcmpW(class_name, L"Xaml_WindowedPopupClass") || !lstrcmpW(class_name, L"XamlExplorerHostIslandWindow")) { invalid = 1; return FALSE; }
225 if (!windows[i].shell && (GetWindowLongW(hwnd, GWL_STYLE) & WS_CAPTION) != WS_CAPTION) { invalid = 1; return FALSE; }
226 capture_bounds[count] = rect;
227 if (!windows[i].shell) attribute(hwnd, 9, &rect, sizeof(rect));
228 if (windows[i].before && !windows[i].initial &&
229 (windows[i].width != (uint32_t)(rect.right - rect.left) || windows[i].height != (uint32_t)(rect.bottom - rect.top))) {
230 windows[i].initial = 1; windows[i].waiting = GetTickCount();
231 }
232 bounds[count] = rect;
177 uint32_t *record = layout + SG_WINDOWS_WORDS + count++ * SG_WINDOW_WORDS;233 uint32_t *record = layout + SG_WINDOWS_WORDS + count++ * SG_WINDOW_WORDS;
178 record[SG_WINDOW_ID] = windows[i].id;234 record[SG_WINDOW_ID] = windows[i].id;
179 record[SG_WINDOW_X] = (uint32_t)rect.left; record[SG_WINDOW_Y] = (uint32_t)rect.top;235 record[SG_WINDOW_X] = (uint32_t)rect.left; record[SG_WINDOW_Y] = (uint32_t)rect.top;
180 record[SG_WINDOW_WIDTH] = (uint32_t)(rect.right - rect.left); record[SG_WINDOW_HEIGHT] = (uint32_t)(rect.bottom - rect.top);236 record[SG_WINDOW_WIDTH] = windows[i].before && !windows[i].shell ? windows[i].width : (uint32_t)(rect.right - rect.left);
237 record[SG_WINDOW_HEIGHT] = windows[i].before && !windows[i].shell ? windows[i].height : (uint32_t)(rect.bottom - rect.top);
181 for (i = SG_WINDOW_CAPTION_LEFT; i <= SG_WINDOW_CAPTION_BOTTOM; ++i) record[i] = 0;238 for (i = SG_WINDOW_CAPTION_LEFT; i <= SG_WINDOW_CAPTION_BOTTOM; ++i) record[i] = 0;
182 if ((GetWindowLongW(hwnd, GWL_STYLE) & WS_CAPTION) == WS_CAPTION) {239 if ((GetWindowLongW(hwnd, GWL_STYLE) & WS_CAPTION) == WS_CAPTION) {
183 TITLEBARINFO title = {0};240 TITLEBARINFO title = {0};
...@@ -187,6 +244,16 @@ static BOOL CALLBACK enumerate(HWND hwnd, LPARAM unused) {...@@ -187,6 +244,16 @@ static BOOL CALLBACK enumerate(HWND hwnd, LPARAM unused) {
187 record[SG_WINDOW_CAPTION_TOP] = (uint32_t)title.rcTitleBar.top;244 record[SG_WINDOW_CAPTION_TOP] = (uint32_t)title.rcTitleBar.top;
188 record[SG_WINDOW_CAPTION_RIGHT] = (uint32_t)(title.rcTitleBar.right - GetSystemMetrics(SM_CXSIZE) * 3);245 record[SG_WINDOW_CAPTION_RIGHT] = (uint32_t)(title.rcTitleBar.right - GetSystemMetrics(SM_CXSIZE) * 3);
189 record[SG_WINDOW_CAPTION_BOTTOM] = (uint32_t)title.rcTitleBar.bottom;246 record[SG_WINDOW_CAPTION_BOTTOM] = (uint32_t)title.rcTitleBar.bottom;
247 LONG left = (LONG)record[SG_WINDOW_CAPTION_LEFT], right = (LONG)record[SG_WINDOW_CAPTION_RIGHT];
248 LONG y = (title.rcTitleBar.top + title.rcTitleBar.bottom) / 2;
249 for (int sample = 0; sample < 3; ++sample) {
250 LONG x = left + 1 + sample * (right - left - 2) / 2;
251 DWORD_PTR hit;
252 if (right <= left || !SendMessageTimeoutW(hwnd, WM_NCHITTEST, 0, MAKELPARAM(x, y), SMTO_ABORTIFHUNG | SMTO_BLOCK, 1, &hit) || hit != HTCAPTION) {
253 for (i = SG_WINDOW_CAPTION_LEFT; i <= SG_WINDOW_CAPTION_BOTTOM; ++i) record[i] = 0;
254 break;
255 }
256 }
190 }257 }
191 }258 }
192 return TRUE;259 return TRUE;
...@@ -196,46 +263,147 @@ static int start_window(struct window *window) {...@@ -196,46 +263,147 @@ static int start_window(struct window *window) {
196 void *item = NULL, *cursor = NULL;263 void *item = NULL, *cursor = NULL;
197 struct size size;264 struct size size;
198 int success = 0;265 int success = 0;
199 if (FAILED(METHOD(interop, 3, window_fn)(interop, window->hwnd, &item_iid, &item))) goto done;266 HWND target = window->hwnd ? window->hwnd : (HWND)MonitorFromPoint((POINT){0, 0}, MONITOR_DEFAULTTOPRIMARY);
267 if (FAILED(METHOD(interop, window->hwnd ? 3 : 4, window_fn)(interop, target, &item_iid, &item))) goto done;
200 if (FAILED(METHOD(item, 7, size_fn)(item, &size)) || size.width <= 0 || size.height <= 0 || size.width > (INT32)SG_MAX_DIMENSION || size.height > (INT32)SG_MAX_DIMENSION) goto done;268 if (FAILED(METHOD(item, 7, size_fn)(item, &size)) || size.width <= 0 || size.height <= 0 || size.width > (INT32)SG_MAX_DIMENSION || size.height > (INT32)SG_MAX_DIMENSION) goto done;
201 if (FAILED(METHOD(pool_factory, 6, pool_fn)(pool_factory, capture_device, DXGI_FORMAT_B8G8R8A8_UNORM, 2, size, &window->pool))) goto done;269 if (FAILED(METHOD(pool_factory, 6, pool_fn)(pool_factory, capture_device, DXGI_FORMAT_B8G8R8A8_UNORM, 2, size, &window->pool))) goto done;
202 if (FAILED(METHOD(window->pool, 10, session_fn)(window->pool, item, &window->session))) goto done;270 if (FAILED(METHOD(window->pool, 10, session_fn)(window->pool, item, &window->session))) goto done;
203 if (FAILED(METHOD(window->session, 0, query_fn)(window->session, &cursor_iid, &cursor))) goto done;271 if (FAILED(METHOD(window->session, 0, query_fn)(window->session, &cursor_iid, &cursor))) goto done;
204 if (FAILED(METHOD(cursor, 7, bool_fn)(cursor, FALSE)) || FAILED(METHOD(window->session, 6, close_fn)(window->session))) goto done;272 if (FAILED(METHOD(cursor, 7, bool_fn)(cursor, FALSE)) || FAILED(METHOD(window->session, 6, close_fn)(window->session))) goto done;
205 window->pool_size = size;273 window->pool_size = size;
274 window->waiting = GetTickCount();
206 success = 1;275 success = 1;
207done:276done:
208 release(cursor); release(item);277 release(cursor); release(item);
209 return success;278 return success;
210}279}
211280
212int sg_windows_poll(sg_emit emit, int refresh) {281static int latest_frame(struct window *window, void **frame) {
282 void *next = NULL;
283 for (int i = 0; i < 2; ++i) {
284 if (FAILED(METHOD(window->pool, 7, object_fn)(window->pool, &next))) return 0;
285 if (!next) return 1;
286 close_object(*frame);
287 *frame = next;
288 }
289 return 1;
290}
291
292int sg_desktop_capture(uint8_t *pixels, uint32_t width, uint32_t height, int refresh) {
293 reap_paints();
294 void *frame = NULL, *surface = NULL, *access = NULL;
295 ID3D11Texture2D *texture = NULL;
296 D3D11_TEXTURE2D_DESC desc;
297 D3D11_MAPPED_SUBRESOURCE mapped;
298 struct size size;
299 int result = -1;
300 if (desktop_retry && (DWORD)(GetTickCount() - desktop_retry) < 1000) return -1;
301 if (!device && !sg_capture_start()) { desktop_retry = GetTickCount(); return -1; }
302 if (!desktop.pool && !start_window(&desktop)) goto done;
303 if (!latest_frame(&desktop, &frame)) goto done;
304 result = 0;
305 if (!frame) {
306 if (refresh && desktop.staging && desktop.width == width && desktop.height == height) goto readback;
307 if ((DWORD)(GetTickCount() - desktop.waiting) > 500 && (!desktop.staging || desktop.width != width || desktop.height != height)) result = -1;
308 goto done;
309 }
310 if (FAILED(METHOD(frame, 8, size_fn)(frame, &size))) { result = -1; goto done; }
311 if (size.width != (INT32)width || size.height != (INT32)height) { result = -1; goto done; }
312 if (desktop.pool_size.width != size.width || desktop.pool_size.height != size.height) {
313 if (FAILED(METHOD(desktop.pool, 6, recreate_fn)(desktop.pool, capture_device, DXGI_FORMAT_B8G8R8A8_UNORM, 2, size))) { result = -1; goto done; }
314 desktop.pool_size = size;
315 goto done;
316 }
317 result = -1;
318 if (FAILED(METHOD(frame, 6, object_fn)(frame, &surface)) ||
319 FAILED(METHOD(surface, 0, query_fn)(surface, &access_iid, &access)) ||
320 FAILED(METHOD(access, 3, query_fn)(access, &texture_iid, (void **)&texture))) goto done;
321 ID3D11Texture2D_GetDesc(texture, &desc);
322 if (desc.Width < width || desc.Height < height) goto done;
323 if (desktop.width != width || desktop.height != height) {
324 if (desktop.staging) ID3D11Texture2D_Release(desktop.staging);
325 desktop.staging = NULL;
326 desc.Width = width; desc.Height = height;
327 desc.BindFlags = 0; desc.MiscFlags = 0; desc.Usage = D3D11_USAGE_STAGING; desc.CPUAccessFlags = D3D11_CPU_ACCESS_READ;
328 if (FAILED(ID3D11Device_CreateTexture2D(device, &desc, NULL, &desktop.staging))) goto done;
329 desktop.width = width; desktop.height = height;
330 }
331 D3D11_BOX box = {0, 0, 0, width, height, 1};
332 ID3D11DeviceContext_CopySubresourceRegion(context, (ID3D11Resource *)desktop.staging, 0, 0, 0, 0, (ID3D11Resource *)texture, 0, &box);
333readback:
334 result = -1;
335 if (FAILED(ID3D11DeviceContext_Map(context, (ID3D11Resource *)desktop.staging, 0, D3D11_MAP_READ, 0, &mapped))) goto done;
336 for (uint32_t y = 0; y < height; ++y) {
337 uint8_t *src = (uint8_t *)mapped.pData + (size_t)y * mapped.RowPitch;
338 uint8_t *dst = pixels + (size_t)y * width * 4;
339 for (uint32_t x = 0; x < width * 4; ++x) dst[x] = src[x];
340 }
341 ID3D11DeviceContext_Unmap(context, (ID3D11Resource *)desktop.staging, 0);
342 result = 1;
343done:
344 if (texture) ID3D11Texture2D_Release(texture);
345 release(access); release(surface); close_object(frame);
346 if (result < 0) { drop(&desktop); desktop_retry = GetTickCount(); }
347 else desktop_retry = 0;
348 return result;
349}
350
351static int atlas_fits(void) {
352 uint32_t width = layout[SG_WINDOWS_WIDTH], height = 0, x = 0, row_height = 0, last_id = 0;
353 for (uint32_t i = 0; i < count; ++i) {
354 uint32_t w = layout[SG_WINDOWS_WORDS + i * SG_WINDOW_WORDS + SG_WINDOW_WIDTH];
355 if (w > width) width = w;
356 }
357 for (uint32_t i = 0; i < count; ++i) {
358 uint32_t *next = NULL;
359 for (uint32_t n = 0; n < count; ++n) {
360 uint32_t *record = layout + SG_WINDOWS_WORDS + n * SG_WINDOW_WORDS;
361 if (record[SG_WINDOW_ID] > last_id && (!next || record[SG_WINDOW_ID] < next[SG_WINDOW_ID])) next = record;
362 }
363 uint32_t w = next[SG_WINDOW_WIDTH], h = next[SG_WINDOW_HEIGHT];
364 if (!w || !h || w > SG_MAX_DIMENSION || h > SG_MAX_DIMENSION) return 0;
365 if (x + w > width) { x = 0; height += row_height; row_height = 0; }
366 x += w;
367 if (h > row_height) row_height = h;
368 last_id = next[SG_WINDOW_ID];
369 }
370 height += row_height;
371 return width <= SG_MAX_DIMENSION && height <= SG_MAX_DIMENSION && (size_t)width * height <= SG_MAX_PIXELS;
372}
373
374int sg_windows_poll(sg_emit emit, int refresh, const uint8_t *pixels, uint32_t width, uint32_t height) {
213 uint32_t i, n, words[SG_SURFACE_WORDS] = {0};375 uint32_t i, n, words[SG_SURFACE_WORDS] = {0};
214 if (retry && (DWORD)(GetTickCount() - retry) < 1000 && !refresh) return 1;376 if (retry && (DWORD)(GetTickCount() - retry) < 1000 && !refresh) return 1;
215 retry = 0;377 retry = 0;
216 count = 0; invalid = 0;378 count = 0; invalid = 0;
217 layout[SG_WINDOWS_WIDTH] = (uint32_t)GetSystemMetrics(SM_CXSCREEN);379 layout[SG_WINDOWS_WIDTH] = width;
218 layout[SG_WINDOWS_HEIGHT] = (uint32_t)GetSystemMetrics(SM_CYSCREEN);380 layout[SG_WINDOWS_HEIGHT] = height;
219 for (i = 0; i < SG_MAX_WINDOWS; ++i) windows[i].seen = 0;381 for (i = 0; i < SG_MAX_WINDOWS; ++i) windows[i].seen = 0;
220 EnumWindows(enumerate, 0);382 HWND foreground = GetForegroundWindow();
383 if (foreground && foreground != GetShellWindow() && foreground != FindWindowW(L"Shell_TrayWnd", NULL) &&
384 (GetWindowLongW(foreground, GWL_STYLE) & WS_CAPTION) != WS_CAPTION) invalid = 1;
385 if (!invalid) EnumWindows(enumerate, 0);
221 for (i = 0; i < SG_MAX_WINDOWS; ++i) if (windows[i].hwnd && !windows[i].seen) drop(windows + i);386 for (i = 0; i < SG_MAX_WINDOWS; ++i) if (windows[i].hwnd && !windows[i].seen) drop(windows + i);
387 if (!invalid && !atlas_fits()) invalid = 1;
222 layout[SG_WINDOWS_COUNT] = invalid ? 0 : count;388 layout[SG_WINDOWS_COUNT] = invalid ? 0 : count;
223 n = SG_WINDOWS_WORDS + layout[SG_WINDOWS_COUNT] * SG_WINDOW_WORDS;389 n = SG_WINDOWS_WORDS + layout[SG_WINDOWS_COUNT] * SG_WINDOW_WORDS;
224 int changed = refresh;390 int changed = refresh;
225 for (i = 0; i < n; ++i) if (layout[i] != prior[i]) changed = 1;391 for (i = 0; i < n; ++i) if (layout[i] != prior[i]) changed = 1;
226 if (changed && !emit(SG_WINDOWS, layout, n, NULL, 0)) return 0;392 if (changed && !emit(SG_WINDOWS, layout, n, NULL, 0)) return 0;
227 for (i = 0; i < n; ++i) prior[i] = layout[i];393 for (i = 0; i < n; ++i) prior[i] = layout[i];
228 if (invalid) return 1;394 if (invalid) { sg_windows_stop(); retry = GetTickCount(); return 1; }
229 for (i = 0; i < SG_MAX_WINDOWS; ++i) {395 for (i = 0; i < SG_MAX_WINDOWS * 2; ++i) {
230 struct window *window = windows + i;396 struct window *window = windows + i % SG_MAX_WINDOWS;
397 if (window->shell != (i < SG_MAX_WINDOWS)) continue;
231 void *frame = NULL, *surface = NULL, *access = NULL;398 void *frame = NULL, *surface = NULL, *access = NULL;
232 ID3D11Texture2D *texture = NULL;399 ID3D11Texture2D *texture = NULL;
233 D3D11_TEXTURE2D_DESC desc;400 D3D11_TEXTURE2D_DESC desc;
401 D3D11_BOX box;
234 D3D11_MAPPED_SUBRESOURCE mapped;402 D3D11_MAPPED_SUBRESOURCE mapped;
235 struct size size;403 struct size size;
236 struct sg_rect rect;404 struct sg_rect rect;
237 uint32_t y;405 uint32_t y;
238 int success = 1;406 int success = 1, mapped_texture = 0;
239 if (!window->hwnd) continue;407 if (!window->hwnd) continue;
240 uint32_t *record = NULL;408 uint32_t *record = NULL;
241 for (n = 0; n < count; ++n) {409 for (n = 0; n < count; ++n) {
...@@ -244,22 +412,24 @@ int sg_windows_poll(sg_emit emit, int refresh) {...@@ -244,22 +412,24 @@ int sg_windows_poll(sg_emit emit, int refresh) {
244 }412 }
245 if (!record) continue;413 if (!record) continue;
246 if (window->shell) {414 if (window->shell) {
247 RECT bounds;415 size.width = capture_bounds[n].right - capture_bounds[n].left;
248 if (!GetWindowRect(window->hwnd, &bounds)) goto frame_failed;416 size.height = capture_bounds[n].bottom - capture_bounds[n].top;
249 size.width = bounds.right - bounds.left; size.height = bounds.bottom - bounds.top;
250 mapped.pData = NULL;417 mapped.pData = NULL;
251 } else {418 } else {
252 if (!window->pool && !start_window(window)) goto frame_failed;419 if (!window->pool && !start_window(window)) goto frame_failed;
253 struct size target = {(INT32)record[SG_WINDOW_WIDTH], (INT32)record[SG_WINDOW_HEIGHT]};420 if (!latest_frame(window, &frame)) goto frame_failed;
254 if (window->pool_size.width != target.width || window->pool_size.height != target.height) {421 if (!frame) {
255 if (FAILED(METHOD(window->pool, 6, recreate_fn)(window->pool, capture_device, DXGI_FORMAT_B8G8R8A8_UNORM, 2, target))) goto frame_failed;422 if (window->initial && (DWORD)(GetTickCount() - window->waiting) > 500) goto frame_failed;
256 window->pool_size = target;423 continue;
257 window->initial = 1;
258 }424 }
259 if (FAILED(METHOD(window->pool, 7, object_fn)(window->pool, &frame))) goto frame_failed;
260 if (!frame) continue;
261 if (FAILED(METHOD(frame, 8, size_fn)(frame, &size))) goto frame_failed;425 if (FAILED(METHOD(frame, 8, size_fn)(frame, &size))) goto frame_failed;
262 if (size.width != target.width || size.height != target.height) goto frame_done;426 if (window->pool_size.width != size.width || window->pool_size.height != size.height) {
427 if (size.width <= 0 || size.height <= 0 || size.width > (INT32)SG_MAX_DIMENSION || size.height > (INT32)SG_MAX_DIMENSION) goto frame_failed;
428 if (FAILED(METHOD(window->pool, 6, recreate_fn)(window->pool, capture_device, DXGI_FORMAT_B8G8R8A8_UNORM, 2, size))) goto frame_failed;
429 window->pool_size = size;
430 window->initial = 1; window->waiting = GetTickCount();
431 goto frame_done;
432 }
263 if (FAILED(METHOD(frame, 6, object_fn)(frame, &surface)) ||433 if (FAILED(METHOD(frame, 6, object_fn)(frame, &surface)) ||
264 FAILED(METHOD(surface, 0, query_fn)(surface, &access_iid, &access)) ||434 FAILED(METHOD(surface, 0, query_fn)(surface, &access_iid, &access)) ||
265 FAILED(METHOD(access, 3, query_fn)(access, &texture_iid, (void **)&texture))) goto frame_failed;435 FAILED(METHOD(access, 3, query_fn)(access, &texture_iid, (void **)&texture))) goto frame_failed;
...@@ -268,13 +438,26 @@ int sg_windows_poll(sg_emit emit, int refresh) {...@@ -268,13 +438,26 @@ int sg_windows_poll(sg_emit emit, int refresh) {
268 if (texture) {438 if (texture) {
269 ID3D11Texture2D_GetDesc(texture, &desc);439 ID3D11Texture2D_GetDesc(texture, &desc);
270 if (desc.Width < (UINT)size.width || desc.Height < (UINT)size.height) goto frame_failed;440 if (desc.Width < (UINT)size.width || desc.Height < (UINT)size.height) goto frame_failed;
441 LONG left = 0, top = 0;
442 LONG right = bounds[n].right - bounds[n].left, bottom = bounds[n].bottom - bounds[n].top;
443 if (size.width == capture_bounds[n].right - capture_bounds[n].left && size.height == capture_bounds[n].bottom - capture_bounds[n].top) {
444 left = bounds[n].left - capture_bounds[n].left; top = bounds[n].top - capture_bounds[n].top;
445 right += left; bottom += top;
446 } else if (size.width != right || size.height != bottom) goto frame_failed;
447 if (left < 0 || top < 0 || right > size.width || bottom > size.height || right <= left || bottom <= top) goto frame_failed;
448 box = (D3D11_BOX){(UINT)left, (UINT)top, 0, (UINT)right, (UINT)bottom, 1};
449 size.width = right - left; size.height = bottom - top;
450 }
451 if (record[SG_WINDOW_WIDTH] != (uint32_t)size.width || record[SG_WINDOW_HEIGHT] != (uint32_t)size.height) {
452 record[SG_WINDOW_WIDTH] = (uint32_t)size.width; record[SG_WINDOW_HEIGHT] = (uint32_t)size.height;
453 if (!atlas_fits()) goto frame_failed;
454 if (!emit(SG_WINDOWS, layout, SG_WINDOWS_WORDS + count * SG_WINDOW_WORDS, NULL, 0)) { success = 0; goto frame_done; }
455 for (n = 0; n < SG_WINDOWS_WORDS + count * SG_WINDOW_WORDS; ++n) prior[n] = layout[n];
271 }456 }
272 if (window->width != (uint32_t)size.width || window->height != (uint32_t)size.height) {457 if (window->width != (uint32_t)size.width || window->height != (uint32_t)size.height) {
273 if (window->staging) ID3D11Texture2D_Release(window->staging);458 if (window->staging) ID3D11Texture2D_Release(window->staging);
274 window->staging = NULL;459 window->staging = NULL;
275 if (window->selected) SelectObject(window->dc, window->selected);460 detach_paint(window);
276 if (window->dib) DeleteObject(window->dib);
277 window->dib = window->selected = NULL; window->bits = NULL;
278 if (window->before) { allocated -= (size_t)window->width * window->height; HeapFree(heap, 0, window->before); window->before = NULL; }461 if (window->before) { allocated -= (size_t)window->width * window->height; HeapFree(heap, 0, window->before); window->before = NULL; }
279 if (allocated + (size_t)size.width * size.height > SG_MAX_PIXELS) goto frame_failed;462 if (allocated + (size_t)size.width * size.height > SG_MAX_PIXELS) goto frame_failed;
280 window->width = (uint32_t)size.width; window->height = (uint32_t)size.height;463 window->width = (uint32_t)size.width; window->height = (uint32_t)size.height;
...@@ -283,12 +466,21 @@ int sg_windows_poll(sg_emit emit, int refresh) {...@@ -283,12 +466,21 @@ int sg_windows_poll(sg_emit emit, int refresh) {
283 allocated += (size_t)window->width * window->height;466 allocated += (size_t)window->width * window->height;
284 if (window->shell) {467 if (window->shell) {
285 BITMAPINFO info = {0};468 BITMAPINFO info = {0};
286 if (!window->dc) window->dc = CreateCompatibleDC(NULL);469 size_t pixels = (size_t)window->width * window->height;
470 if (painted + pixels > SG_MAX_PIXELS) goto frame_failed;
471 window->paint = HeapAlloc(heap, HEAP_ZERO_MEMORY, sizeof(struct shell_paint));
472 if (!window->paint) goto frame_failed;
473 struct shell_paint *paint = window->paint;
474 paint->pixels = pixels; painted += pixels; paint->hwnd = window->hwnd;
475 paint->dc = CreateCompatibleDC(NULL);
287 info.bmiHeader.biSize = sizeof(BITMAPINFOHEADER); info.bmiHeader.biWidth = size.width;476 info.bmiHeader.biSize = sizeof(BITMAPINFOHEADER); info.bmiHeader.biWidth = size.width;
288 info.bmiHeader.biHeight = -size.height; info.bmiHeader.biPlanes = 1; info.bmiHeader.biBitCount = 32;477 info.bmiHeader.biHeight = -size.height; info.bmiHeader.biPlanes = 1; info.bmiHeader.biBitCount = 32;
289 window->dib = CreateDIBSection(window->dc, &info, DIB_RGB_COLORS, (void **)&window->bits, NULL, 0);478 paint->dib = CreateDIBSection(paint->dc, &info, DIB_RGB_COLORS, (void **)&paint->bits, NULL, 0);
290 if (!window->dc || !window->dib) goto frame_failed;479 if (!paint->dc || !paint->dib) goto frame_failed;
291 window->selected = SelectObject(window->dc, window->dib);480 paint->selected = SelectObject(paint->dc, paint->dib);
481 paint->started = GetTickCount();
482 paint->thread = CreateThread(NULL, 0, paint_shell, paint, 0, NULL);
483 if (!paint->thread) goto frame_failed;
292 } else {484 } else {
293 desc.Width = window->width; desc.Height = window->height;485 desc.Width = window->width; desc.Height = window->height;
294 desc.BindFlags = 0; desc.MiscFlags = 0; desc.Usage = D3D11_USAGE_STAGING; desc.CPUAccessFlags = D3D11_CPU_ACCESS_READ;486 desc.BindFlags = 0; desc.MiscFlags = 0; desc.Usage = D3D11_USAGE_STAGING; desc.CPUAccessFlags = D3D11_CPU_ACCESS_READ;
...@@ -297,14 +489,45 @@ int sg_windows_poll(sg_emit emit, int refresh) {...@@ -297,14 +489,45 @@ int sg_windows_poll(sg_emit emit, int refresh) {
297 window->initial = 1;489 window->initial = 1;
298 }490 }
299 if (window->shell) {491 if (window->shell) {
300 DWORD_PTR result;492 if (WaitForSingleObject(window->paint->thread, 0) != WAIT_OBJECT_0) {
301 if (!SendMessageTimeoutW(window->hwnd, WM_NULL, 0, 0, SMTO_ABORTIFHUNG | SMTO_BLOCK, 20, &result) || !PrintWindow(window->hwnd, window->dc, 2)) goto frame_failed;493 if ((DWORD)(GetTickCount() - window->paint->started) >= 500) goto frame_failed;
302 GdiFlush();494 return 1;
303 mapped.pData = window->bits; mapped.RowPitch = window->width * 4;495 }
496 DWORD painted_ok;
497 if (!GetExitCodeThread(window->paint->thread, &painted_ok) || !painted_ok) goto frame_failed;
498 for (y = 0; y < window->height; ++y) {
499 LONG screen_y = (LONG)record[SG_WINDOW_Y] + (LONG)y;
500 if (screen_y < 0 || screen_y >= (LONG)layout[SG_WINDOWS_HEIGHT]) continue;
501 LONG x = (LONG)record[SG_WINDOW_X], right = x + (LONG)window->width;
502 if (x < 0) x = 0;
503 if (right > (LONG)layout[SG_WINDOWS_WIDTH]) right = (LONG)layout[SG_WINDOWS_WIDTH];
504 while (x < right) {
505 LONG stop = right;
506 for (n = 0; n < count; ++n) {
507 uint32_t *other = layout + SG_WINDOWS_WORDS + n * SG_WINDOW_WORDS;
508 if (other[SG_WINDOW_ID] == window->id) break;
509 LONG left = capture_bounds[n].left - 2, top = capture_bounds[n].top - 2;
510 LONG edge = capture_bounds[n].right + 2;
511 if (screen_y < top || screen_y >= capture_bounds[n].bottom + 2) continue;
512 if (left <= x && edge > x) { x = edge; stop = x; break; }
513 if (left > x && left < stop) stop = left;
514 }
515 if (x >= stop) continue;
516 uint8_t *dst = window->paint->bits + ((size_t)y * window->width + (x - (LONG)record[SG_WINDOW_X])) * 4;
517 const uint8_t *src = pixels + ((size_t)screen_y * layout[SG_WINDOWS_WIDTH] + x) * 4;
518 for (LONG byte = 0; byte < (stop - x) * 4; ++byte) dst[byte] = src[byte];
519 x = stop;
520 }
521 }
522 mapped.pData = window->paint->bits; mapped.RowPitch = window->width * 4;
304 } else {523 } else {
305 D3D11_BOX box = {0, 0, 0, window->width, window->height, 1};
306 ID3D11DeviceContext_CopySubresourceRegion(context, (ID3D11Resource *)window->staging, 0, 0, 0, 0, (ID3D11Resource *)texture, 0, &box);524 ID3D11DeviceContext_CopySubresourceRegion(context, (ID3D11Resource *)window->staging, 0, 0, 0, 0, (ID3D11Resource *)texture, 0, &box);
307 if (FAILED(ID3D11DeviceContext_Map(context, (ID3D11Resource *)window->staging, 0, D3D11_MAP_READ, 0, &mapped))) goto frame_failed;525 if (FAILED(ID3D11DeviceContext_Map(context, (ID3D11Resource *)window->staging, 0, D3D11_MAP_READ, 0, &mapped))) goto frame_failed;
526 mapped_texture = 1;
527 for (y = 8; y + 8 < window->height; ++y) {
528 const uint8_t *row = (const uint8_t *)mapped.pData + (size_t)y * mapped.RowPitch;
529 for (uint32_t x = 8; x + 8 < window->width; ++x) if (row[x * 4 + 3] != 255) goto frame_failed;
530 }
308 }531 }
309 rect.x = window->width; rect.y = window->height; rect.width = rect.height = 0;532 rect.x = window->width; rect.y = window->height; rect.width = rect.height = 0;
310 if (refresh || window->initial) { rect.x = rect.y = 0; rect.width = window->width; rect.height = window->height; }533 if (refresh || window->initial) { rect.x = rect.y = 0; rect.width = window->width; rect.height = window->height; }
...@@ -330,17 +553,18 @@ int sg_windows_poll(sg_emit emit, int refresh) {...@@ -330,17 +553,18 @@ int sg_windows_poll(sg_emit emit, int refresh) {
330 }553 }
331 window->initial = 0;554 window->initial = 0;
332 }555 }
333 if (texture) ID3D11DeviceContext_Unmap(context, (ID3D11Resource *)window->staging, 0);
334 goto frame_done;556 goto frame_done;
335frame_failed:557frame_failed:
336 invalid = 1;558 invalid = 1;
337frame_done:559frame_done:
560 if (mapped_texture) ID3D11DeviceContext_Unmap(context, (ID3D11Resource *)window->staging, 0);
338 if (texture) ID3D11Texture2D_Release(texture);561 if (texture) ID3D11Texture2D_Release(texture);
339 release(access); release(surface); close_object(frame);562 release(access); release(surface); close_object(frame);
340 if (!success) return 0;563 if (!success) return 0;
341 if (invalid) { drop(window); break; }564 if (invalid) { drop(window); break; }
342 }565 }
343 if (invalid) {566 if (invalid) {
567 sg_windows_stop();
344 retry = GetTickCount();568 retry = GetTickCount();
345 uint32_t disabled[SG_WINDOWS_WORDS] = {layout[SG_WINDOWS_WIDTH], layout[SG_WINDOWS_HEIGHT], 0};569 uint32_t disabled[SG_WINDOWS_WORDS] = {layout[SG_WINDOWS_WIDTH], layout[SG_WINDOWS_HEIGHT], 0};
346 if (!emit(SG_WINDOWS, disabled, SG_WINDOWS_WORDS, NULL, 0)) return 0;570 if (!emit(SG_WINDOWS, disabled, SG_WINDOWS_WORDS, NULL, 0)) return 0;
guest/capture.h+4-2
...@@ -3,7 +3,9 @@...@@ -3,7 +3,9 @@
3#include "cursor.h"3#include "cursor.h"
44
5typedef int (*sg_emit)(uint32_t type, const uint32_t *words, uint32_t count, const uint8_t *pixels, uint32_t stride);5typedef int (*sg_emit)(uint32_t type, const uint32_t *words, uint32_t count, const uint8_t *pixels, uint32_t stride);
6int sg_windows_start(void);6int sg_capture_start(void);
7int sg_windows_poll(sg_emit emit, int refresh);7int sg_windows_poll(sg_emit emit, int refresh, const uint8_t *pixels, uint32_t width, uint32_t height);
8int sg_desktop_capture(uint8_t *pixels, uint32_t width, uint32_t height, int refresh);
9void sg_capture_stop(void);
8void sg_windows_stop(void);10void sg_windows_stop(void);
9#endif11#endif
guest/windows.c+18-10
...@@ -194,7 +194,7 @@ static void stream(DWORD session) {...@@ -194,7 +194,7 @@ static void stream(DWORD session) {
194 if (capturing != request[15]) {194 if (capturing != request[15]) {
195 capturing = request[15];195 capturing = request[15];
196 initial = 1; last_cursor = NULL; last_flags = ~0u;196 initial = 1; last_cursor = NULL; last_flags = ~0u;
197 if (!capturing) { sg_windows_stop(); windows_active = 0; }197 if (!capturing) { sg_capture_stop(); windows_active = 0; }
198 }198 }
199 } else {199 } else {
200 int enabled = request[15] && modern_windows;200 int enabled = request[15] && modern_windows;
...@@ -222,7 +222,7 @@ static void stream(DWORD session) {...@@ -222,7 +222,7 @@ static void stream(DWORD session) {
222 if (!active || !GetCursorInfo(&cursor)) {222 if (!active || !GetCursorInfo(&cursor)) {
223 if (!suspended && !packet(SG_SUSPEND, 0, NULL, 0)) goto done;223 if (!suspended && !packet(SG_SUSPEND, 0, NULL, 0)) goto done;
224 suspended = 1; initial = 1; last_cursor = NULL; last_flags = ~0u;224 suspended = 1; initial = 1; last_cursor = NULL; last_flags = ~0u;
225 if (windows_active) sg_windows_stop();225 sg_capture_stop();
226 windows_active = 0;226 windows_active = 0;
227 Sleep(100); continue;227 Sleep(100); continue;
228 }228 }
...@@ -270,14 +270,20 @@ static void stream(DWORD session) {...@@ -270,14 +270,20 @@ static void stream(DWORD session) {
270 if (!before || !dib) goto done;270 if (!before || !dib) goto done;
271 previous = SelectObject(dc, dib); width = w; height = h; initial = 1;271 previous = SelectObject(dc, dib); width = w; height = h; initial = 1;
272 }272 }
273 if (!BitBlt(dc, 0, 0, (int)width, (int)height, screen, 0, 0, SRCCOPY | CAPTUREBLT)) {273 int captured = modern_windows ? sg_desktop_capture(pixels, width, height, initial) : -1;
274 if (!captured) {
275 if (initial) continue;
276 goto window_capture;
277 }
278 if (captured < 0 && !BitBlt(dc, 0, 0, (int)width, (int)height, screen, 0, 0, SRCCOPY | CAPTUREBLT)) {
274 if (!suspended && !packet(SG_SUSPEND, 0, NULL, 0)) goto done;279 if (!suspended && !packet(SG_SUSPEND, 0, NULL, 0)) goto done;
275 suspended = 1; initial = 1; Sleep(100); continue;280 suspended = 1; initial = 1; Sleep(100); continue;
276 }281 }
277 GdiFlush();282 if (captured < 0) GdiFlush();
278 QueryPerformanceCounter(&scanned);283 QueryPerformanceCounter(&now);
279 ++stats[SG_CAPTURE_STATS_FRAMES];284 ++stats[SG_CAPTURE_STATS_FRAMES];
280 stats[SG_CAPTURE_STATS_CAPTURE_US] += microseconds(scanned.QuadPart - started.QuadPart, frequency.QuadPart);285 stats[SG_CAPTURE_STATS_CAPTURE_US] += microseconds(now.QuadPart - started.QuadPart, frequency.QuadPart);
286 scanned = now;
281 if (initial) { rect.x = 0; rect.y = 0; rect.width = width; rect.height = height; }287 if (initial) { rect.x = 0; rect.y = 0; rect.width = width; rect.height = height; }
282 else if (!sg_difference(before, pixels, width, height, &rect)) rect.width = rect.height = 0;288 else if (!sg_difference(before, pixels, width, height, &rect)) rect.width = rect.height = 0;
283 QueryPerformanceCounter(&written);289 QueryPerformanceCounter(&written);
...@@ -290,25 +296,26 @@ static void stream(DWORD session) {...@@ -290,25 +296,26 @@ static void stream(DWORD session) {
290 size_t offset = ((size_t)y * width + rect.x) * 4, i;296 size_t offset = ((size_t)y * width + rect.x) * 4, i;
291 for (i = 0; i < rect.width * 4; ++i) before[offset + i] = pixels[offset + i];297 for (i = 0; i < rect.width * 4; ++i) before[offset + i] = pixels[offset + i];
292 }298 }
299window_capture:
293 if (windows_enabled) {300 if (windows_enabled) {
294 if (!windows_active) {301 if (!windows_active) {
295 windows_active = sg_windows_start();302 windows_active = sg_capture_start();
296 if (!windows_active) {303 if (!windows_active) {
297 uint32_t disabled[SG_WINDOWS_WORDS] = {0};304 uint32_t disabled[SG_WINDOWS_WORDS] = {0};
298 if (!emit(SG_WINDOWS, disabled, SG_WINDOWS_WORDS, NULL, 0)) goto done;305 if (!emit(SG_WINDOWS, disabled, SG_WINDOWS_WORDS, NULL, 0)) goto done;
299 windows_enabled = 0;306 windows_enabled = 0;
300 }307 }
301 }308 }
302 if (windows_active && !sg_windows_poll(emit, initial)) goto done;309 if (windows_active && !sg_windows_poll(emit, initial, pixels, width, height)) goto done;
303 }310 }
304 QueryPerformanceCounter(&now);311 QueryPerformanceCounter(&now);
305 stats[SG_CAPTURE_STATS_WRITE_US] += microseconds(now.QuadPart - written.QuadPart, frequency.QuadPart);312 if (captured) stats[SG_CAPTURE_STATS_WRITE_US] += microseconds(now.QuadPart - written.QuadPart, frequency.QuadPart);
306 initial = 0; suspended = 0;313 initial = 0; suspended = 0;
307 }314 }
308 Sleep(1);315 Sleep(1);
309 }316 }
310done:317done:
311 sg_windows_stop();318 sg_capture_stop();
312 if (begin_period && end_period) end_period(1);319 if (begin_period && end_period) end_period(1);
313 if (winmm) FreeLibrary(winmm);320 if (winmm) FreeLibrary(winmm);
314 CancelIo(output);321 CancelIo(output);
...@@ -337,6 +344,7 @@ void mainCRTStartup(void) {...@@ -337,6 +344,7 @@ void mainCRTStartup(void) {
337 /* This optional user32 export supplies ANI step timing; static capture uses only documented APIs. */344 /* This optional user32 export supplies ANI step timing; static capture uses only documented APIs. */
338 cursor_frame = (cursor_frame_fn)symbol;345 cursor_frame = (cursor_frame_fn)symbol;
339 for (;;) {346 for (;;) {
347 sg_capture_stop();
340 if (session != WTSGetActiveConsoleSessionId()) { Sleep(1000); continue; }348 if (session != WTSGetActiveConsoleSessionId()) { Sleep(1000); continue; }
341 output = CreateFileW(L"\\\\.\\Global\\" SG_CHANNEL, GENERIC_READ | GENERIC_WRITE, 0, NULL, OPEN_EXISTING, FILE_FLAG_OVERLAPPED, NULL);349 output = CreateFileW(L"\\\\.\\Global\\" SG_CHANNEL, GENERIC_READ | GENERIC_WRITE, 0, NULL, OPEN_EXISTING, FILE_FLAG_OVERLAPPED, NULL);
342 if (output != INVALID_HANDLE_VALUE) { stream(session); CloseHandle(output); }350 if (output != INVALID_HANDLE_VALUE) { stream(session); CloseHandle(output); }