1# Guest display stream
2
3The Windows executable is freestanding C99. `cursor.c` contains pixel conversion and dirty-rectangle detection; `windows.c` owns desktop capture and transport; `capture.c` implements modern desktop capture and optional window surfaces. Zig supplies the cross compiler and Windows headers, with no Zig or C runtime in the executable. x86 and x64 target Windows 7; ARM64 targets Windows 10. PE subsystem versions and imported DLLs are checked by `python3 guest/test.py`. Runtime validation uses Windows 10 and 11; Windows 7 runtime behavior is unverified.
4
5Build with `python3 guest/build.py --arch x86_64` (also `x86` and `aarch64`). In an elevated PowerShell session inside the logged-in administrator's desktop, run `./install.ps1 -Binary ./build/snowglobe-guest-x86_64.exe`. The installer uses Task Scheduler COM APIs available to PowerShell 2, installs into protected Program Files, and starts an elevated task at that user's logon. `-Remove` removes that task and executable. Elevation is required by the VirtIO serial driver's device ACL; capture must run in the interactive session, not session 0. The guest needs a compatible VirtIO serial driver and the channel named in `protocol.json`. A standard-user launcher requires a privileged handle broker; it is not part of this first backend.
6
7Windows 10 build 19041 and later use Windows Graphics Capture for the primary monitor with cursor capture disabled. Older or unsupported systems use `BitBlt` into a CPU DIB without drawing the cursor. Only the latest available capture frames are read back; an unchanged desktop needs no readback. RGB comparisons ignore the unused fourth byte. Unchanged pixels produce no screen packets; changed pixels produce one bounding rectangle. Cursor polling runs every millisecond; desktop capture targets 60 Hz. The host's existing encoder supplies WebRTC video. Secure desktops, RDP sessions that differ from the physical console, and capture failures return the viewer to QEMU capture.
8
9`GetCursorInfo` supplies the shape, visibility, and observed position. `GetIconInfo` and black/white `DrawIconEx` renders recover RGBA and binary XOR planes, including colored XOR. Nonbinary color XOR and unsupported cursor sizes explicitly fall back to the browser's default cursor, preserving video. Animation timing uses the optional `GetCursorFrameInfo` user32 export, resolved by name. Missing or unusable animation metadata falls back to a static frame. Animation starts a browser-local clock; Windows does not expose the current animation phase through this interface. Ordinary shapes use the browser cursor; XOR shapes use local layers with difference blending. Neither follows streamed positions.
10
11## Linux cursor backend
12
13`linux.c` uses the same protocol and browser cursor renderer. Build natively with `python3 guest/build.py --platform linux`; install with `sudo sh guest/install.sh guest/build/snowglobe-guest`. Build dependencies are a C compiler, pkg-config, GLib/GIO, PipeWire/SPA, Xlib, XFixes, and Xext development packages. The executable links the distribution's libraries; build it on the oldest distribution that it must support. Linux builds use libc, while Windows retains its freestanding import boundary.
14
15The installer grants the active local user access to only the display VirtIO port through udev/logind and starts the agent in the graphical session through XDG autostart. Sign out and back in after installation. The VM needs the channel from `protocol.json`; there is no guest network listener. `--remove` removes the installation for subsequent logins.
16
17X11 uses XFixes cursor notifications and premultiplied ARGB images, with shared-memory root capture and an XGetImage fallback. Root images exclude the pointer. Wayland uses the ScreenCast portal's metadata cursor mode and CPU-mapped PipeWire BGRx frames. The first session requires selecting a monitor and approving sharing; a returned restore token is saved privately for later logins. Compositors without cursor metadata support keep QEMU capture. Cursor animation is forwarded as observed shape updates; these APIs do not supply a complete animation timeline.
18
19Desktop locking and inactive logind sessions suspend guest capture and release the port. Closing viewers pauses capture; reconnects resend the cached cursor and a complete desktop frame. Output is nonblocking and bounded, and a stalled transport falls back after three seconds. Linux advertises no separate-window capability and never sends window surfaces. Run `python3 guest/linux-test.py` on Linux for metadata bounds, alpha conversion, dirty frames, fragmented requests, cursor-free X11 capture, shape changes, and reconnect validation.
20
21## Wire format
22
23`protocol.json` is the only home for packet IDs, field order, channel name, flags, and bounds. The build generates its private C header; the host reads that same schema. Each packet begins with the four-byte magic, a big-endian uint32 type, and a big-endian uint32 payload length. Fields listed in the schema are big-endian uint32 values.
24
25- Screen: tightly packed top-down BGRx rectangle rows following the listed fields. A connection, refresh request, resize, or resumed capture starts with a full frame.
26- Cursor: each frame is a uint32 duration in milliseconds, width × height RGBA pixels, then one XOR bit mask byte per pixel. The schema assigns the channel bits; each selected channel inverts the underlying browser pixel. Zero dimensions and zero frames mean unsupported shape/default cursor.
27- Pointer: coordinates use signed two's-complement values in the uint32 fields. These are observations, not a cursor hit map.
28- Suspend and heartbeat have empty payloads. A heartbeat arrives at least once per second; four seconds without a packet falls back to QEMU.
29- Refresh has an empty payload. The host requests it on every connection and resends cached cursor state when WebRTC connects.
30- Stream pauses or resumes capture; heartbeats continue while paused.
31- Stop has an empty payload. The last viewer closing or service shutdown drains guest writes until the guest acknowledges stop, then closes the channel. The guest sends no further packets after its acknowledgement and closes its handle.
32- Capabilities advertises window capture and stop-handshake support. Older agents without stop support retain the original transport-close behavior; update the agent for safe channel retirement. Experiment requests enable or disable window capture.
33- Windows lists full surface dimensions, desktop position, stacking order, and native caption bounds. Surface contains tightly packed BGRx dirty rectangles; the first update is complete.
34- Capture statistics reports observed frame counts and time spent capturing, comparing, and writing.
35
36Transport writes are limited to 64 KiB so each request fits the VirtIO serial DMA descriptor budget.
37
38The host broker validates a running VM's libvirt-owned channel path. One guest reader fans out to that VM's viewers. Guest bytes are untrusted and bounded before decoding; viewer buffers are bounded independently. No guest network listener or host GPU capture process is required.
39
40## Separate windows experiment
41
42Windows 10 build 19041 and later can enable separate windows from the viewer’s View menu. Windows Graphics Capture supplies full unoccluded surfaces with cursor capture disabled. Its dependencies are loaded dynamically, preserving the Windows 7 import boundary. Desktop and taskbar surfaces initialize with a `PrintWindow` worker because Windows Graphics Capture rejects those shell HWNDs. Subsequent updates copy visible desktop spans, retaining covered pixels for local window dragging. Shell surfaces initialize before window capture sessions start, keeping capture indicators out of their initial pixels. Changed rectangles update a single host atlas and one existing video encoder. Metadata describes the atlas and stacking; the browser composes it and moves cached surfaces immediately during native caption drags. Atlas geometry is matched to the decoded RTP frame.
43
44The experiment is off by default. Unsupported, layered, translucent, shell-popup, or oversized window sets close all window sessions and fall back to desktop video. Frame-pool resizing follows captured content dimensions; a window whose first or resized surface never arrives also falls back. Secure desktops and session changes suspend guest capture. Shell painting runs off the capture thread and falls back after 500 ms; its bounded drawing resources remain owned until the worker exits. Windows capture indicators remain subject to the OS; fallback disposes the sessions instead of leaving indicators attached to inactive windows. Protected content and custom caption behavior remain subject to the application’s capture support.
45
46## Later backends
47
48Keep the core independent of an OS and add real platform implementations when their capture and cursor APIs can be tested. macOS capture permissions need a separate backend decision. A window rectangle does not imply a uniform cursor: application hit testing can change the shape anywhere inside it. Region prediction should be based on validated application information or observations with invalidation, not fabricated bounding boxes.