Document Sidecar

ContextCopy Anchor Link

Today an Align room shows only chat + a "Coverage" rail of section titles/statuses. Participants review a PDF or Outlook email thread but never see the document itself, and the moderator's per-topic understanding is only visible post-session (the annotated review record).

This adds a document sidecar: a slide-in card (open on click) that renders the room's uploaded PDF (self-hosted PDF.js) or email thread, with live moderator-generated topic summaries as bubbles anchored to the relevant section/email. The moderator gains a second concurrent lane (lane B) that summarizes the topic currently being discussed and pushes it to the sidecar, independent of the main-chat moderation (lane A).

Outcome: reviewers see the source beside the conversation, and the moderator's understanding of each topic surfaces as it forms, grounded in the document — not just at the end.

Research outcome — viewer selectionCopy Anchor Link

• PDF → self-hosted PDF.js (Mozilla). Adobe PDF Embed API rejected: no offline/self-hosted mode, requires internet + a domain-bound client ID — incompatible with Align's local-first, no-CDN design. PDF.js is fully offline once vendored (like GSAP/Three), and exposes a positioned text layer used for client-side section anchoring.
• Email → render from the existing server parse, not a new lib. No embeddable Outlook web renderer exists. server/execution/parsers/email.js already yields {messages:[{from,date,subject,body}]}, so the sidecar renders our own escaped/styled thread. (Reference libs if ever needed: postal-mime / eml-parser / msgreader + DOMPurify — not adopted v1.)

Locked decisions (confirmed with user)Copy Anchor Link

1. Section-anchored bubbles (v1). Clicking a topic scrolls the doc to that section and pins the bubble beside the heading, located client-side via PDF.js text-layer search for the section title. Email = message-anchored (scroll to the message referencing the topic). No pixel-precise server coordinate feed in v1.
2. Enterprise = direct-to-Resident. In TIER=enterprise the browser fetches document bytes AND summary text straight from the Resident (:3200); the coordinator stays document-blind and never handles bytes. In TIER=team everything runs in the one process.

Architecture — two lanes, plane-aware "side wall"Copy Anchor Link

• Lane A (existing, unchanged): main-chat moderation on the 1s room tick (rooms.js startChat). Guards: room.modBusy (moderator-context.js:104), room.nluBusy.
• Lane B (new): live topic summarization. Trigger = room.goal.lastDiscussedId change + section maturing (messageCount>=2, set in ontology/goals/docreview.js:89). Debounced, own single-flight guard, lighter model — can never starve lane A. Reuses the summary shape from knowledge.reviewSummaries(), made incremental.
  • Topic-scoped context (critical — multiple topics interleave): lane B must summarize a topic from only the messages attributed to that sectionId, never the raw last-N global chat. Attribution is the same semantic/matcher.matchEntities the goal engine already uses in docreview.reduce. Implementation: maintain a per-section rolling message buffer room.goal.sections[id].msgs (append the attributed message inside reduce where lastDiscussedId/messageCount are already updated), and feed that buffer + the section's own grounded excerpt from search(sectionTitle) into summarizeTopic. This keeps each bubble about its own topic even when discussion ping-pongs between sections.
• Plane split (the side wall): team → lane B + byte serving in the monolith, summary rides the socket inline. enterprise → lane B runs inside the Resident (it holds the document; the moderator is document-blind); the client pulls bytes + summary text directly from the Resident; the coordinator emits only a contentless topic:summary signal. Under TRUST_EPHEMERAL, summaries are sealed at rest with K_room (ephemeral-key.js:52-78) and crypto-shredded on dialog-end.

One event contract, discriminated by source, shared by both tiers:

topic:summary { sectionId, seq, source:'monolith'|'resident',
                ref:{kind:'pdf',sectionTitle} | {kind:'email',messageIndex},
                text? /* present ONLY when source==='monolith' */ }

Modularity & layered hierarchy (must hold)Copy Anchor Link

Follow the existing four-layer architecture — no cross-layer shortcuts, LLM only via the injected port ({chat,extractJSON,embed}), enforced by test/layers-test.js:

• semantic (semantic/matcher.js): message→section attribution — reused, not reimplemented.
• ontology (ontology/goals/docreview.js): topic lifecycle + the per-section message buffer (sec.msgs) that scopes lane-B context. No I/O, no LLM here — pure goal state.
• context (context/knowledge.js): summarizeTopic — retrieval + LLM summarization behind the port. Knows nothing about sockets, HTTP, or the sidecar.
• execution (execution/presenter.js, http-api.js, resident/index.js, kxp-client.js, rooms.js, index.js): wiring only — the tick hook, event emit, routes, plane branch, Resident.
• client (public/v4/, lib/align/client.ts): rendering only, over the SDK/protocol.

New client-side viewer logic is its own module (public/v4/sidecar.js, lazy-imported like scene4.js) so v4.js stays a thin wiring layer and a PDF.js load failure is isolated.

Deliverable locationCopy Anchor Link

This plan lives in docs/ alongside TRUST.md, DEPLOYMENT.md, and EPHEMERAL-ARCHITECTURE.md as the module's design document.

Phased implementationCopy Anchor Link

Phase 1 — Vendor PDF.js + raw-document route + viewer shell (team path)Copy Anchor Link

• server/index.js (after :75): app.use('/vendor/pdfjs', express.static('node_modules/pdfjs-dist/build')); add pdfjs-dist to package.json (pin a version; worker served same-origin, never CDN).
• server/execution/http-api.js (by :335): GET /api/rooms/:code/document/raw → application/pdf from room.doc.pdfBuffer, or {kind:'email',title,messages} from room.doc.emailMessages; 409 {error:'bytes-off-plane',plane:'resident'} when room.compliant (the enterprise discriminator); 409 {error:'not-retained'} for seeded/replayed rooms. Reuses retention at :156 / null-at :63.
• public/v4/v4.js: lazy loadPdfjs() mirroring the scene4.js lazy-import (:67) with GlobalWorkerOptions.workerSrc='/vendor/pdfjs/pdf.worker.min.mjs'; renderDoc(container) branches on content-type (PDF → page canvases + text layer; JSON → email list).

Phase 2 — Sidecar UI: slide-in, open-on-click, render, section scrollCopy Anchor Link

• public/v4/index.html (sibling of #takeaway, ~:507): #sidecar (.sc-head/#sc-title/#sc-close, #sc-doc, #sc-bubbles) + #sc-open tab. Show #sc-open only when goalType==='docreview' and doc.title.
• public/v4/v4.css: .sidecar fixed right, translateX(100%)→.open{translateX(0)} with transition: transform .28s (plain CSS, no GSAP, matching #takeaway); mobile full-width overlay under the existing 1-col breakpoint (never a 3rd .room-body column).
• public/v4/sidecar.js (new module, lazy-imported from v4.js): openSidecar()/close, renderDoc, scrollToSection(title) (walks each page text-layer for the best title match → scrollIntoView + return heading rect), scrollToMessage(idx) via data-msg-idx, renderBubble. v4.js only wires the #sc-open click + the topicSummary subscription to this module — keeping v4.js thin.

Phase 3 — Lane B live summarization (team/monolith)Copy Anchor Link

• server/ontology/goals/docreview.js (ontology layer): in reduce() where lastDiscussedId/messageCount are set (:89), also append the attributed message text to a bounded per-section buffer sec.msgs (cap ~12, drop oldest). This is the topic-scoped context source — attribution already happens here, so no second attribution pass.
• server/context/knowledge.js (context layer): add summarizeTopic(room, sectionId) → Promise<{sectionId,title,ref,text,seq}> (export at :617, prompt in PROMPTS :633). Context = room.goal.sections[sectionId].msgs (topic-scoped, from ontology) + the section's grounded excerpt via search(sectionTitle) — never the global transcript. One llm.chat({model:SUMMARY_MODEL,maxTokens:90,temperature:0.3,timeoutMs:15000}) behind the injected LLM port; deterministic fail-soft one-liner (mirror reviewSummaries :589). SUMMARY_MODEL=process.env.SUMMARY_MODEL||ollama.MODEL. Email ref.messageIndex by matching title→emailMessages; PDF ref={kind:'pdf',sectionTitle}.
• server/execution/presenter.js: add maybeSummarizeTopic(room) — debounce on room.goal.lastDiscussedId vs room.summaryLastId; require messageCount>=2; single-flight on new room.summaryBusy; ≥8s per-topic cooldown (room.summaryAt); bump room.summarySeq.
• server/index.js + server/execution/rooms.js: new manager handler onTopicTick(room)→presenter.maybeSummarizeTopic(room) called from startChat (:163) right after onCheckNudges — reuse the existing 1s tick, no new interval. Lane A's modBusy/nudge path untouched.

Phase 4 — topic:summary event + anchored bubblesCopy Anchor Link

• server/execution/presenter.js: emit io.to(channel(room)).emit('topic:summary', payload); persist room.topicSummaries[sectionId]=payload. Do NOT touch publicRoomState (:34, frozen).
• server/execution/http-api.js: GET /api/rooms/:code/topic-summaries?since=seq catch-up (409→resident under enterprise).
• lib/align/client.ts: one line in the wire→SDK event map: 'topic:summary':'topicSummary'.
• public/v4/v4.js: client.on('topicSummary', …) → renderBubble (reuse mdToHtml :592) into #sc-bubbles, then scrollToSection/scrollToMessage and pin beside the heading/message rect; de-dupe/replace by sectionId keeping latest seq (mirror the [data-msg-id] mutate pattern :464-511). If no text-layer match (scanned PDF) → pin to page top, flag "approx." — never drop the summary.

Phase 5 — Enterprise plane split (Resident routes, direct-to-Resident, sealing)Copy Anchor Link

• server/resident/index.js: bind configurable RESIDENT_BIND||'127.0.0.1' (:243); CORS (RESIDENT_CORS_ORIGIN + OPTIONS 204) on the new GET read routes only (keep KXP write routes HMAC/loopback). Add GET /kxp/document/raw?room= (bytes/email from d.pdfBuffer/d.emailMessages), GET /kxp/summaries?room=&since= (keyring.open decrypt-on-read; [] after shred), POST /kxp/summarize (HMAC-verified like /kxp/ask :177). Lane B here uses d.summaryBusy separate from d.busy (:57) so live summaries and @h asks never serialize against each other; reuse summarizeTopic against d.pseudoRoom; model RESIDENT_SUMMARY_MODEL||RESIDENT_MODEL. Seal on completion (keyring.seal(room, …)); drop d.summaries on dialog-end (:201) = crypto-shred.
• server/execution/presenter.maybeSummarizeTopic: branch on room.compliant — team = in-process + emit with text; enterprise = kxp.summarizeSigned(...) (add to kxp-client.js mirroring askSigned :62) then emit contentless {sectionId,seq,source:'resident',ref}.
• server/index.js: add residentPublicUrl: process.env.RESIDENT_PUBLIC_URL to /api/trust (:93) — the browser-reachable Resident origin (loopback residentUrl isn't a browser origin).
• public/v4/v4.js: store S.residentBase from /api/trust; docSource/renderDoc fall back to ${S.residentBase}/kxp/document/raw on 409 bytes-off-plane; topicSummary with source:'resident' → fetch ${S.residentBase}/kxp/summaries?since= then renderBubble; one catch-up pull on sidecar open.

Phase 6 — TestsCopy Anchor Link

• test/harness/sidecar-team-test.js (new): spawnServer({TIER:'team'}), upload gsPdf, drive chat that flips lastDiscussedId; assert document/raw is application/pdf, a topic:summary with text+ref arrives, and a moderator nudge + a summary both fire in one window (lane B doesn't block lane A).
• test/harness/sidecar-enterprise-test.js (new): moderator TIER=enterprise + spawned Resident (RESIDENT_BIND/RESIDENT_CORS_ORIGIN); assert monolith document/raw→409 bytes-off-plane, Resident /kxp/document/raw returns bytes, topic:summary arrives without text (source:'resident'), /kxp/summaries returns decrypted text, and [] after dialog-end (shred). Add both to test/run-all.js.
• test/v4-test.js (headless Chrome): click #sc-open → #sidecar.open, #sc-doc has PDF.js canvas/text-layer (proves offline /vendor/pdfjs worker), a real chat topic-switch yields an anchored #sc-bubbles node; scanned-PDF "approx." fallback; narrow-viewport full-width overlay with .room-body intact.

Critical filesCopy Anchor Link

• server/execution/http-api.js — raw-document + catch-up routes, 409 discriminators
• server/execution/presenter.js — maybeSummarizeTopic lane-B driver, topic:summary emit, team/enterprise branch
• server/context/knowledge.js — new summarizeTopic (reuse matcher/search, light model)
• server/resident/index.js — client-reachable bind+CORS, /kxp/document/raw, /kxp/summaries, /kxp/summarize, d.summaryBusy, K_room sealing
• public/v4/v4.js — PDF.js lazy loader, sidecar open/render, section/message anchoring, topicSummary handler, direct-to-Resident fetch
• Supporting: server/index.js (/vendor/pdfjs, residentPublicUrl), server/execution/kxp-client.js (summarize/summarizeSigned), lib/align/client.ts (one map line), public/v4/index.html + v4.css (sidecar + CSS slide), package.json (pdfjs-dist).

Key risks & mitigationsCopy Anchor Link

• PDF.js worker offline: same-origin worker only; lazy-import so a vendor 404 degrades to a "preview unavailable" panel, never crashes the module graph. Pin pdfjs-dist, verify worker filename.
• Large PDFs (≤15MB cap): render pages lazily on scroll (IntersectionObserver), cap concurrent renders.
• Heading not in text layer (scanned PDF): section search fails → page-top "approx." pin, never drop the summary.
• Resident not browser-reachable / CORS: default loopback+no-CORS today; Phase 5 makes bind+CORS explicit config and ships RESIDENT_PUBLIC_URL; sidecar stays chat-only if Resident unreachable.
• Lane B backlog: dedicated busy guard (team room.summaryBusy / Resident d.summaryBusy≠d.busy), debounce, ≥8s cooldown, lighter model, short maxTokens.
• Frozen wire shapes: never extend publicRoomState/docreview.snapshot; summaries travel on the new event + catch-up route only.

VerificationCopy Anchor Link

• Team: npm run team; upload a PDF and an .eml; open sidecar; confirm live bubbles anchor on topic switches.
• Enterprise: npm run resident (RESIDENT_BIND/RESIDENT_PUBLIC_URL/RESIDENT_CORS_ORIGIN) + npm run enterprise; confirm moderator holds no bytes (document/raw→409), the browser pulls bytes+summaries from :3200 directly, and summaries vanish on room end.
• Automated: the two new harness suites + test/v4-test.js additions above; npm test / npm run qa.