ContributingDraft

Glass plan (28 Sep)

1. What's landed -- none of it#

git cherry origin/main HEAD marks all 21 commits on work/glass-live as + (not on main), confirmed by content, not just SHA: origin/main's core/computers/driver/policy.js and docker.js have no COMPUTERD_TOKEN_FILE, no .boot seeding, no URLBlocklist / DownloadDirectory / http(s)-only fencing. Both HIGH fixes from the last e2e review (4b88f1d9) are absent from 0.1.0-rc.1 -- secrets still sit in the container's Env, and an agent's CDP session can still dump cookies, navigate file:///chrome://, and download unbounded. Reported to the lead already; this is an RC blocker independent of anything below.

Also not on main: the vault fill contract (computers.fill.begin/end, shielding during a fill), idle hand-back (computers.handback.*, the 10 s warning), computers.restart/limits, the tailnet egress gate hardening, and the native-Capsule-era copy/event fixes (a8b9f0e, f2b8a1b, ab983cb5). core/glass and deck/glass DO exist independently on main (a smaller, older line of the same module -- watch.js 393 lines there vs 407 here) -- main was never empty, but it's stale relative to this branch by about three weeks of work, and this branch is stale relative to main's newer neighbors (sight, Design A, the settings hub).

2. What's outdated on this branch#

• sight (cohesion, landed on main): sight.frame gives a still JPEG of an agent's screen for the phone, one screen service shared by both Glass's live view and sight's stills. This branch still treats Glass's watch/relay path as the only screen source; it should sit behind the same service sight now defines so a still and a live stream are two views of one thing, not two pipelines.
• Design A (picked 27 Sep, docs/design/one-app/DIRECTION.md): home is "Needs you", Glass is a place you go to, violet is the sole attention colour, one tree/three shapes. This branch's copy and layout predate that call -- GlassWatch.dc.html/GlassTakeover.dc.html boards need a pass against Design A's placement rules (top bar, card chrome, badge colour) and the Chat-place shell.
• the settings hub (refactor(settings) on main): modules now declare their own settings, the kernel checks and stores them. computers.handback.* in this branch predates that; it should become a glass/computers-declared setting, not its own ad hoc tool pair, to land inside the hub instead of beside it.
• per-session socket / registry: main's module loader (core/modules/index.js) is unchanged in shape, but PERSON_ONLY presence enforcement (computers.takeover, glass.take, glass.release) landed differently upstream than this branch assumed -- reconcile against current core/presence/index.js before merging, not after.

3. "True module"#

Glass is already close: core/glass talks to computers only through its providers (providers/computer.js, providers/box.js), declares tools/events/views in module.json, and deck/glass is a self-contained view tree with vendored noVNC. The frozen boundary edges that still leak:

• computerd /fs routes are computers-image surface, not glass's own -- contributed to the computers image rather than owned by glass. Fine as a documented contract, but it means glass can't ship or version independently of the computers image. Freeze the route shapes in team/archive/work-journals/computers.md explicitly as glass's contract, the way vault's fill contract is now written down.
• computers.shield/computers.helper are computers tools glass calls; the vault fill contract adds a third caller (vault) to the same shield. That's the registry working as intended -- but glass's guard.js (denied-path list) needs to stay equal to link's copy per team/archive/work-journals/glass.md's "Needs from others", which is a manual sync today, not enforced by a test. Add a same-file/checksum test so drift fails CI instead of getting found in review.
• presence (PERSON_ONLY) is enforced in the module loader, not in glass itself -- correct per the registry model, but computers.takeover/glass.take declaring their own presence at all (noted as removed in this branch's "Changed contracts") suggests glass used to duplicate a check the registry now owns. Confirm nothing in deck/glass still gates on a local presence check instead of trusting the tool call to fail closed.
• event naming: glass emits both computer.* and glass.* for the same take-over/hand-back (see watch.js's agentOf/switch handling both). That's a real seam, not tech debt -- computers owns the resource, glass owns the UI meaning -- but two event families for one user action is the kind of thing that should be one internal event translated at the module boundary, not duplicated at every listener.

4. "Feels like my own screen" -- concrete plan#

Already built, verify and extend rather than build from scratch:

• Fullscreen (watch.js:89, requestFullscreen/webkitRequestFullscreen on the stage element) and pinch zoom (pinchZoom(host, zoomView, ...), Fit reset button) exist today on Deck. Not yet: a native Mac window through the Capsule (Capsule is native-Swift now, no window chrome for Glass yet -- this is real, new work), and fullscreen has not been verified on the phone (deck/glass/phone.js) or measured for correctness on resize.
• Quality/compression already adapts by device and link (watch.js:34, laptop 6/2, phone 5/4, slow link 2/6) -- a latency-aware starting point exists; it is not closed-loop (doesn't react to measured fps/latency mid-session, only to link type at open).
• scaleViewport: true, resizeSession: false (watch.js:259-260) -- fit-to-window is the current default; 1:1 pixel and retina scale are not implemented as an explicit toggle.

New work, in priority order (reset 28 Sep per cohesion's binding interaction pass, docs/design/interaction.md 5debc1bc -- items 1 and 3 below are now top-2, not mid-list):

1. Close the latency badge's own loop (S→M). Shipped 47d90b0c as an open-time snapshot (link.latencyMs from glass.open, read once at connect); every other "what's happening now" surface in Vyre updates live, so this is the one that goes stale mid-session. Needs continuous sampling and one event, not a bigger UI change -- the render side is already there.
2. Native full screen (M): CORRECTED 28 Sep by the new browser harness (section 8) -- the Fullscreen button (fullBtn in watch.js) is built but only ever placed in the phone layout's markup; on Deck it exists as a JS object with nowhere to render, so Deck has no fullscreen control at all today. The earlier draft of this plan said it was "wired on Deck" -- that was wrong, caught by deck/test/glass-browser.js actually loading the desktop route and finding no such element. Work: add the same control to the desktop layout (gl-stagewrap header), verify iOS Safari's fullscreen restrictions on the phone one that already exists (iOS Safari has no real element fullscreen in some contexts -- may need a CSS "cover the viewport" fallback instead of the API). A Mac native window through the Capsule is L: needs a real window (not a panel/extension) hosting the same RFB canvas, which is new Capsule-native surface, not a Glass-side change alone -- coordinate with capsule-pro. This is the user's literal ask ("feels like my own screen"); reprioritized to the top per cohesion/the lead.
3. sight.frame for the reconnect still and the resting-tile preview (S): already agreed with cohesion (2026-09-28) -- call sight.frame right when the socket drops, show that as the frozen frame, swap to the live stream once sight.watch's ticket reconnects; same call, small maxWidth, for the resting tile, refetched on the next sight.stepped rather than a timer. Replaces the "reconnect without a black flash" item below 1:1 -- this is now that item's implementation, not a separate one.
4. Fit and zoom (S, DONE): a Deck 1:1 toggle shipped (f5fe3f1f), verified by the harness. DPR sizing is NOT the S fix this plan assumed -- corrected 28 Sep after actually reading vendor/novnc/core/display.js. noVNC's canvas backing store (canvas.width/height) already equals the remote framebuffer's real pixel size; scaleViewport only sets the canvas's CSS style.width/height to fit the container, via _rescale. So the canvas is not "DPR-unaware" in a way a client-side multiply fixes: on a 2x Retina screen, a 1024x768 remote desktop shown at ~1024 CSS px wide really is only 1024 physical pixels' worth of data, upscaled by the browser regardless of any canvas-sizing change here. The only way to a sharper Retina picture is more source pixels -- the container's Xvnc running at a higher resolution than today's config default -- which is a real tradeoff (more bytes over the relay for every viewer, not just Retina ones) and a computers/pool decision, not a glass client patch. Downgraded from S to a design question; not doing a cosmetic canvas-size change that would not actually help.
5. Latency/fps targets + measurement (S to define, M to instrument): propose p95 input-to- paint under 150ms on Tailscale-direct / 300ms over relay, sustained 24fps minimum during active use, under 5% dropped frames over a 10s window -- matching the smoothness bar Design A already set for the rest of the app (one-app/DIRECTION.md's 60fps/1% table is the sibling spec for page UI; Glass needs its own row in that same table, not a separate standard). Measure via RFB's existing frame timestamps plus a round-trip ping tool -- the same sampling loop item 1 needs, shared rather than built twice.
6. Input fidelity (M): keyboard shortcuts through noVNC's domkeytable/keysymdef are vendored and presumably complete; clipboard is called out in team/archive/work-journals/glass.md's Next as "clipboard to the holder only while shielded" -- still open. Two-way clipboard, IME support (noVNC's input layer is keysym-based, which is lossy for IME composition -- needs explicit testing with CJK input), and scroll fidelity (trackpad vs wheel) are all unverified.
7. Cursor (S): confirm noVNC's cursor decoder is on (local cursor rendering vs remote-only) -- cheap to check, meaningfully changes perceived latency since a local cursor never waits on the round trip.
8. Audio (not applicable today -- Chrome-in-a-container has no audio pipeline in this design; skip unless a future computer type needs it).
9. Instant take-over handoff (S, mostly done): shield/unshield and holder events already exist; the "instant" feel is mostly about not re-negotiating the RFB session on take-over, which the code already avoids (take-over is a permission change, not a reconnect). Verify no visible stall today; if there is one, it's likely the passkey/Touch ID round trip, not video.

5. Cheap opportunities (S/M) -- status 28 Sep#

rc.2 was narrowed to 14f1824c only (the HIGH-fix merge); everything below is 0.1.1, built on work/glass-live behind it.

• Live latency badge -- done (47d90b0c): latencyLabel(link) in deck/glass/watch.js, a pure function with a unit test (watch.test.js), rendered next to the connection badge. It is an open-time snapshot today, not the closed loop cohesion asked for (item 1 above) -- that part is still open.
• 1:1 zoom toggle -- done (f5fe3f1f): a 1:1/Fit button on Deck's panel, rfb.scaleViewport/ clipViewport flipped, .gl-1to1 in glass.css (a fixed-height scrollable box instead of the aspect-ratio one). Verified: harness click-and-check plus a real screenshot. Was blocked on deck/glass having no test harness for mountScreen's RFB/DOM wiring; built one first (section 8, deck/test/glass-browser.js, 9/9 on testbox including this toggle).
• DPR-aware canvas sizing -- downgraded from an S fix to a design question (section 4, item 4): not a client-side patch, needs a decision on Xvnc's container resolution.
• sight.frame reconnect still -- next.

6. Tests that improve UX (measurable)#

Built, in deck/test/glass-browser.js (section 8): mount/connect, fit ratio, take-over class, reconnect-keeps-last-frame, phone fullscreen wiring, and resize/layout at Deck (1440) and phone (390) widths, plus a screenshot of each state. Still open:

• Latency budget test: scripted RFB connect + input round-trip, assert p95 under the target in section 4.3, over both Tailscale-direct and relay paths (two runs, two budgets). Needs the closed-loop sampling from item 1 to have a number to assert on.
• Frame-drop budget test: capture N seconds of frame timestamps during a scripted mouse-drag, assert dropped-frame percentage under budget.
• Deck fullscreen toggle test: once item 2 ships a Deck control, add the same stub-and-click check glass-browser.js already does for the phone one.
• DPR/1:1 correctness: once those land, assert canvas backing-store size against devicePixelRatio and the toggle's state.

7. Coordinate with cohesion -- resolved 28 Sep#

• sight.frame reuse: agreed. Glass calls sight.frame {target: "agent:<name>", maxWidth} directly for both the reconnect still (call it the moment the socket drops, show the still, swap to the live stream once sight.watch's ticket reconnects -- no separate JPEG path or polling loop of Glass's own) and the resting-tile preview (small maxWidth, refetched on the next sight.stepped for that target, not a timer). No changes needed on cohesion's side. Note the target format is agent:<name> (confirmed via core/sight/sight.test.js), not glass's own computer:<name> -- the conversion happens at the call site.
• Inline chat pictures: item 18 in docs/design/cohesion.md (cohesion, sha baf5ec7c), now bound by the lead's interaction pass. Split as proposed: chat owns rendering (built once in chat-core), sessions passes image blocks through from the Agent SDK's own shape, sight owns the one capture path (sight.frame/sight.stepped) -- glass is a second caller of that same path (the reconnect tile, and now chat's per-step thumbnail), not a second capture contract. The other inline-image source (files the agent made -- Canva renders, saved screenshots, not a live screen) is explicitly unowned, open for 0.1.1, the lead's call.

8. Browser harness -- built 28 Sep#

deck/glass had no test that mounts mountScreen -- only pure helpers extracted from watch.js/input.js were unit-tested. Built one, real Chrome on testbox, real vyred, real Glass WS relay, no Docker:

• test/fixtures/fake-xvnc.js -- a minimal RFB server (extracted from core/computers/glass.test.js, which now imports it too rather than keeping its own copy), enough handshake for a real client to reach connected, plus sendFrame() (one raw-encoded rect, so a real canvas paints something) and crash() (kills the TCP connection, standing in for a dead container).
• deck/test/glass-world.js -- a throwaway VYRE_HOME with role: "box" (computers/glass are box-role modules; the default on a Mac is local, which silently runs neither -- first thing the harness caught) and computers: { driver: "fake", local: {...} } pointed at the fake Xvnc, an agent with a real computers.checkout, and the same HTTP+WS proxy deck/test/world.js uses so the real Deck page and the real Glass relay are exercised, not a mock. Prints {ready, url, agent} on stdout, same contract as deck/test/native-bar/world.js.
• deck/test/glass-browser.js -- drives it with deck/test/cdp.js (no Playwright dependency, matching the rest of the repo's browser checks), one JSON line per check plus a screenshot. Run: node deck/test/glass-browser.js --port 4757 (testbox only, one Chrome at a time).

Run 28 Sep on testbox: 8/8 checks pass -- mount/connect, fit ratio, take-over (gl-held, no passkey prompt, matching the PERSON_ONLY change), reconnect (gl-snap stays visible after the backend crashes, no black canvas), phone fullscreen (real click, stubbed requestFullscreen, called once on the stage element), and both resize/layout checks (phone lays out as gl-phone, Deck as gl-stagewrap). Screenshots confirmed real UI (take-over state, the "kit is paused while you drive" copy), not a blank page.

What the harness caught immediately: the fullscreen button (fullBtn in watch.js) is built but only placed in the phone layout's markup -- Deck has no fullscreen control at all today. An earlier draft of this plan (section 4, item 2) said it was "wired on Deck"; that was wrong, and the harness is what caught it by actually loading the desktop route and finding no such element. Corrected above. This is the harness earning its cost on its first run.

Two things it does not cover yet, needed before 1:1/DPR land: canvas pixel-content assertions (today's checks are structural -- classes, sizes, element presence -- not "does this pixel match"); and a way to fetch devicePixelRatio-scaled screenshots (CDP's Page.captureScreenshot needs an explicit Emulation.setDeviceMetricsOverride deviceScaleFactor, which cdp.js's openTab already takes as scale, so this is wiring, not new capability).

0.1.1 build list#