From 52bb56501e1c51108b3caa55e5eec50db0a1c487 Mon Sep 17 00:00:00 2001 From: david_bai Date: Thu, 18 Dec 2025 12:23:08 +0800 Subject: [PATCH] chore(doc):Add an English version of documents such as AGENTS.md and ai-playbook --- AGENTS.md | 29 +-- AGENTS.zh-CN.md | 18 ++ docs/ai-playbook/code-map.md | 170 +++++++++++++ docs/ai-playbook/collab-rules.md | 150 +++++++++++ docs/ai-playbook/flows.md | 233 ++++++++++++++++++ .../flows/backpressure-chunking.md | 99 ++++++++ docs/ai-playbook/flows/frontend.md | 178 +++++++++++++ .../flows/reconnect-consistency.md | 183 ++++++++++++++ docs/ai-playbook/flows/resume.md | 109 ++++++++ docs/ai-playbook/index.md | 52 ++++ 10 files changed, 1207 insertions(+), 14 deletions(-) create mode 100644 AGENTS.zh-CN.md create mode 100644 docs/ai-playbook/code-map.md create mode 100644 docs/ai-playbook/collab-rules.md create mode 100644 docs/ai-playbook/flows.md create mode 100644 docs/ai-playbook/flows/backpressure-chunking.md create mode 100644 docs/ai-playbook/flows/frontend.md create mode 100644 docs/ai-playbook/flows/reconnect-consistency.md create mode 100644 docs/ai-playbook/flows/resume.md create mode 100644 docs/ai-playbook/index.md diff --git a/AGENTS.md b/AGENTS.md index e20227b..da02705 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,18 +1,19 @@ -# AGENTS — PrivyDrop Repository Rules(简版) +# AGENTS — PrivyDrop Repository Rules (Short) -最重要的原则 +## First Principles -- 用中文沟通:与项目负责人沟通一律使用中文(简体)。代码注释、命名、提交信息、PR 标题与描述统一使用英文。 -- 方案与代码遵循最佳实践:优先选择与现有技术栈一致、被验证过的实现;小步迭代、易回滚。 -- 计划先行:任何实现前须提交并获批变更计划(目标、范围/文件、方案、风险、验收、回滚、需更新文档、验证方式)。模板见 docs/ai-playbook/collab-rules.zh-CN.md。 -- 单一主题:每次改动只解决一个明确目标,避免“顺手修复”无关问题,保持最小可回滚。 -- 隐私与架构红线:后端仅做信令与房间管理;严禁任何形式的文件数据中转、存储或上报至第三方。 -- 传输护栏:保持既定分片/背压/重试等关键参数与机制;任何破坏性或参数层变更需先获批。 -- 依赖与基建:未经批准不得新增依赖/组件库/基础设施或进行大规模重构。 -- 文档同步:涉及流程、接口或入口文件路径的改动,必须在同一 PR 内同步更新 docs/ai-playbook/flows.zh-CN.md 与 docs/ai-playbook/code-map.zh-CN.md。 -- 验证要求:前端需构建通过(next build);列出关键手测用例与回归点。 +- Communicate in Chinese: Always use Simplified Chinese when communicating with the project owner/maintainers. Use English for code comments, naming, commit messages, and PR titles/descriptions. +- Best practices, aligned with the existing stack: Prefer proven approaches consistent with what the repo already uses; iterate in small steps and keep changes easy to roll back. +- Plan first: Before implementing anything, propose a change plan and get approval (goals, scope/files, approach, risks, acceptance, rollback, docs updates, validation). Template: `docs/ai-playbook/collab-rules.md` (or `docs/ai-playbook/collab-rules.zh-CN.md`). +- One change, one purpose: Each change should solve one clear goal; avoid “while I’m here” fixes; keep it minimal and reversible. +- Privacy & architecture red line: The backend is for signaling and room coordination only. Do not relay, store, or upload any user file data to the server or third parties in any form. +- Transport guardrails: Keep established chunking/backpressure/retry parameters and mechanisms; any breaking change or parameter-level change must be approved first. +- Dependencies & infrastructure: Do not add new dependencies/component libraries/infrastructure or do large refactors without approval. +- Docs must stay in sync: If a change affects flows, interfaces, or entry file paths, update `docs/ai-playbook/flows.zh-CN.md` and `docs/ai-playbook/code-map.zh-CN.md` in the same PR. +- Verification required: Frontend must build (`next build`); list key manual test cases and regression points. -优先级与冲突 +## Priority & Conflicts + +- Explicit user instructions override this file; if there’s a conflict, call it out in the plan and get approval. +- For detailed rules, examples, and checklists, follow `docs/ai-playbook/collab-rules.md` (this file only keeps the highest-level principles). -- 显式用户指令优先于本文件;如有冲突需在计划中说明并征得同意。 -- 更多细则、示例与校验清单以 docs/ai-playbook/collab-rules.zh-CN.md 为准(本文件仅保留最原则条款)。 diff --git a/AGENTS.zh-CN.md b/AGENTS.zh-CN.md new file mode 100644 index 0000000..e20227b --- /dev/null +++ b/AGENTS.zh-CN.md @@ -0,0 +1,18 @@ +# AGENTS — PrivyDrop Repository Rules(简版) + +最重要的原则 + +- 用中文沟通:与项目负责人沟通一律使用中文(简体)。代码注释、命名、提交信息、PR 标题与描述统一使用英文。 +- 方案与代码遵循最佳实践:优先选择与现有技术栈一致、被验证过的实现;小步迭代、易回滚。 +- 计划先行:任何实现前须提交并获批变更计划(目标、范围/文件、方案、风险、验收、回滚、需更新文档、验证方式)。模板见 docs/ai-playbook/collab-rules.zh-CN.md。 +- 单一主题:每次改动只解决一个明确目标,避免“顺手修复”无关问题,保持最小可回滚。 +- 隐私与架构红线:后端仅做信令与房间管理;严禁任何形式的文件数据中转、存储或上报至第三方。 +- 传输护栏:保持既定分片/背压/重试等关键参数与机制;任何破坏性或参数层变更需先获批。 +- 依赖与基建:未经批准不得新增依赖/组件库/基础设施或进行大规模重构。 +- 文档同步:涉及流程、接口或入口文件路径的改动,必须在同一 PR 内同步更新 docs/ai-playbook/flows.zh-CN.md 与 docs/ai-playbook/code-map.zh-CN.md。 +- 验证要求:前端需构建通过(next build);列出关键手测用例与回归点。 + +优先级与冲突 + +- 显式用户指令优先于本文件;如有冲突需在计划中说明并征得同意。 +- 更多细则、示例与校验清单以 docs/ai-playbook/collab-rules.zh-CN.md 为准(本文件仅保留最原则条款)。 diff --git a/docs/ai-playbook/code-map.md b/docs/ai-playbook/code-map.md new file mode 100644 index 0000000..af40c85 --- /dev/null +++ b/docs/ai-playbook/code-map.md @@ -0,0 +1,170 @@ +# PrivyDrop AI Playbook — Code Map + +This map is designed for quick orientation. It lists directories and key entry files with short notes, and intentionally does not cover “frequently changed spots” or detailed impact analysis. + +## Frontend (Next.js, TypeScript) + +- `frontend/app/` — App Router routes and pages. + + - `frontend/app/[lang]/page.tsx` — Home page entry; generates metadata and SEO structured data (JsonLd), with multilingual canonical links. + - `frontend/app/[lang]/*/page.tsx` — Static pages: features, about, faq, help, terms, privacy; each generates multilingual SEO metadata. + - `frontend/app/[lang]/blog/page.tsx` — Blog list page; renders multilingual post lists. + - `frontend/app/[lang]/blog/[slug]/page.tsx` — Blog post page; MDX rendering, TOC, breadcrumbs, and JSON-LD structured data. + - `frontend/app/[lang]/blog/tag/[tag]/page.tsx` — Blog tag page; lists posts by tag. + - `frontend/app/[lang]/layout.tsx` — Global layout & providers (ThemeProvider, Header/Footer); generates organization/site structured data. + - `frontend/app/[lang]/HomeClient.tsx` — Main client component composing the home layout (Hero, ClipboardApp, HowItWorks, video demo, system diagram, features, FAQ), with multi-platform video links (YouTube/Bilibili). + - `frontend/app/api/health/route.ts` — Basic health API. + - `frontend/app/api/health/detailed/route.ts` — Detailed health API. + - `frontend/app/sitemap.ts` — Sitemap generator; multilingual URLs and dynamic blog entries. + - `frontend/middleware.ts` — i18n and routing middleware. + - `frontend/app/config/environment.ts` — Runtime/env config (ICE, endpoints, etc.). + - `frontend/app/config/api.ts` — Backend API client wrapper. + +- `frontend/components/` — UI layer, including the orchestrator and child components. + + - `frontend/components/ClipboardApp.tsx` — Top-level UI orchestrator. Integrates 5 business hooks (useWebRTCConnection/useFileTransferHandler/useRoomManager/usePageSetup/useClipboardAppMessages), and handles global drag events plus the Send/Retrieve tabs. + - UX: when switching to Retrieve and all of the following hold—“not in a room, no roomId in URL, empty input, cached ID exists”—it auto-fills and joins (reads `frontend/lib/roomIdCache.ts`). + - Connection feedback: integrates `useConnectionFeedback` (`frontend/hooks/useConnectionFeedback.ts`) to map WebRTC states to UI messages (negotiating, 8s slow hint, disconnect/reconnect/restored hints when visible). Slow hints reuse `frontend/utils/useOneShotSlowHint.ts`. + +- `frontend/hooks/` — Business logic hub (React Hooks). + - `useRoomManager.ts` + - Join flow: `join_inProgress` (immediate), `join_slow` (3s, reuses `useOneShotSlowHint`), `join_timeout` (15s); timers are cleared on both success and failure. + - Equivalent success signals: before `joinResponse`, receiving `ready/recipient-ready/offer` is treated as “joined”, and clears the 3s/15s timers. + - Others: room status copy, share-link generation, leave room, input validation (750ms debounce). + - `useConnectionFeedback.ts` + - State normalization: `new/connecting` → `negotiating`; `failed/closed` → `disconnected` (reuses `utils/rtcPhase.ts`). + - Negotiation slow hint: an 8s timer (`rtc_slow`), shown at most once per negotiation attempt; if it fires in background, it’s deferred and emitted once on foreground if still negotiating (reuses `useOneShotSlowHint`). + - One-shot hints: first `connected` (`rtc_connected`) is shown once; foreground reconnecting (`rtc_reconnecting`) and restored (`rtc_restored`) hints. + +- i18n copy & types + - Copy: `frontend/constants/messages/*.{ts}` (zh/en/ja/es/de/fr/ko filled). + - Types: `frontend/types/messages.ts` (ClipboardApp includes `join_*` and `rtc_*` message keys). + - `frontend/components/ClipboardApp/SendTabPanel.tsx` — Send panel: rich-text editor, file upload, room ID generation (4-digit vs UUID), share-link generation. + - UX: clicking “Use cached ID” triggers join immediately on the sender side, saving one manual click. + - `frontend/components/ClipboardApp/RetrieveTabPanel.tsx` — Retrieve panel: room join, file receiving, directory selection (File System Access API), rich-text display. + - `frontend/components/ClipboardApp/FileListDisplay.tsx` — File list: file/folder grouping, progress tracking, browser-specific download strategies (Chrome auto download; others prompt manual save), download count stats. + - `frontend/components/ClipboardApp/FullScreenDropZone.tsx` — Full-screen drag overlay/feedback. + - `frontend/components/ClipboardApp/*` — Other subcomponents: FileUploadHandler, ShareCard (QR code sharing), TransferProgress, CachedIdActionButton, FileTransferButton. + - `frontend/components/Editor/` — Rich-text editor module: RichTextEditor, toolbar components (BasicFormatTools/FontTools/AlignmentTools/InsertTools), SelectMenu, types, and editor hooks. + - `frontend/components/blog/` — Blog components: TableOfContents (Chinese heading ID generation + scroll tracking), Mermaid rendering, MDXComponents, ArticleListItem, etc. + - `frontend/components/common/` — Shared components: clipboard_btn (clipboard read/write buttons), AutoPopupDialog, LazyLoadWrapper, YouTubePlayer. + - `frontend/components/web/` — Site components: Header (responsive nav + language), Footer (copyright + language links), FAQSection, HowItWorks, SystemDiagram, KeyFeatures, theme-provider. + - `frontend/components/web/ThemeToggle.tsx` — Theme toggle (single Light/Dark button), used in Header (desktop & mobile). + - `frontend/components/seo/JsonLd.tsx` — SEO structured data component for multiple JSON-LD types. + - `frontend/components/LanguageSwitcher.tsx` — Language switcher. + - `frontend/components/ui/*` — Base UI atoms (Radix UI + shadcn/ui): Button (variants), Accordion, Dialog, Card, Tooltip, Select, Input, Textarea, Checkbox, DropdownMenu, Toast system, AnimatedButton. + +- `frontend/hooks/` — Business logic hub (React Hooks). + + - `frontend/hooks/useWebRTCConnection.ts` — WebRTC lifecycle and orchestration APIs. + - `frontend/hooks/useRoomManager.ts` — Room create/join/validate and UI state; supports cached-ID reconnect (≥8 chars auto-sends initiator-online). + - `frontend/hooks/useFileTransferHandler.ts` — File/text payload orchestration and callbacks; uses getState() to avoid stale closures; supports JSZip folder downloads. + - `frontend/hooks/useClipboardActions.ts` — Clipboard actions/state; supports modern APIs and document.execCommand fallback; handles HTML/rich-text paste. + - `frontend/hooks/useClipboardAppMessages.ts` — App messaging (shareMessage/retrieveMessage) with a 4-second auto-dismiss mechanism. + - `frontend/hooks/useLocale.ts` — Language selection by parsing pathname. + - `frontend/hooks/usePageSetup.ts` — Page setup & SEO; auto-join from URL roomId; referrer tracking. + - `frontend/hooks/useRichTextToPlainText.ts` — Rich-text → plain-text helper; block-level line breaks and text-node wrapping. + +- `frontend/lib/` — Core libraries and utilities. + + - WebRTC base & roles + - `frontend/lib/webrtc_base.ts` — WebRTC base class: Socket.IO signaling, RTCPeerConnection management, ICE candidate queues, dual-disconnect reconnection, wake lock management, DataChannel send retries (5 attempts with increasing delays), graceful disconnect tracking (`gracefullyDisconnectedPeers` Set), and multi-format payload compatibility (ArrayBuffer/Blob/Uint8Array/TypedArray). joinRoom uses a 15s timeout and an “equivalent success signal” fallback: Initiator treats `ready/recipient-ready` as joined; Recipient treats `offer` as joined; once triggered it sets inRoom and clears listeners/timers to reduce false timeouts on weak networks. + - `frontend/lib/webrtc_Initiator.ts` — Initiator role: handles `ready`/`recipient-ready`, creates RTCPeerConnection and a proactive DataChannel, sends offers, handles answers, supports a 256KB buffer threshold. + - `frontend/lib/webrtc_Recipient.ts` — Recipient role: handles `offer`, creates RTCPeerConnection and a reactive DataChannel (`ondatachannel`), generates and sends answers, handles `initiator-online` reconnect signals and connection cleanup. + - `frontend/lib/webrtcService.ts` — WebRTC service singleton (persists across routes): manages sender/receiver instances, exposes a unified business API, handles connection-state changes, broadcasting, file requests, and disconnect cleanup. + - Sending (sender) + - `frontend/lib/fileSender.ts` — Backward-compatible sender wrapper; internally uses FileTransferOrchestrator. + - `frontend/lib/transfer/FileTransferOrchestrator.ts` — Sender main orchestrator; manages the file transfer lifecycle. + - `frontend/lib/transfer/StreamingFileReader.ts` — High-performance streaming reader using the 32MB batch + 64KB network chunk dual-layer buffer design. + - `frontend/lib/transfer/NetworkTransmitter.ts` — Network transmitter; uses native WebRTC backpressure control and supports embedded-metadata chunk packets. + - `frontend/lib/transfer/StateManager.ts` — State hub; tracks peer state, pending files, folder metadata. + - `frontend/lib/transfer/ProgressTracker.ts` — Progress tracker; computes file/folder progress and speed stats; triggers callbacks. + - `frontend/lib/transfer/MessageHandler.ts` — Message routing (fileRequest/fileReceiveComplete/folderReceiveComplete). + - `frontend/lib/transfer/TransferConfig.ts` — Transfer config: 4MB file read chunks, 32MB batches, 64KB network chunks. + - Receiving (receiver) + - `frontend/lib/fileReceiver.ts` — Backward-compatible receiver wrapper; internally uses FileReceiveOrchestrator. + - `frontend/lib/receive/FileReceiveOrchestrator.ts` — Receiver main orchestrator; manages reception lifecycle with resume support and streaming disk writes. + - `frontend/lib/receive/ReceptionStateManager.ts` — State hub; manages file metadata, active reception state, folder progress, and save-mode config. + - `frontend/lib/receive/ChunkProcessor.ts` — Chunk processor: payload conversion, embedded-metadata parsing, validation, and index mapping. + - `frontend/lib/receive/StreamingFileWriter.ts` — Streaming writer with SequencedDiskWriter for strict in-order disk writes; supports large streaming files. + - `frontend/lib/receive/FileAssembler.ts` — In-memory assembler for small files; reassembles, checks integrity, and creates a File object. + - `frontend/lib/receive/MessageProcessor.ts` — Message routing (fileMeta/stringMetadata/fileRequest/folderReceiveComplete). + - `frontend/lib/receive/ProgressReporter.ts` — Progress reporter: progress/speed stats and throttled callbacks. + - `frontend/lib/receive/ReceptionConfig.ts` — Reception config: 1GB “large file” threshold, 64KB chunks, buffer sizes, debug toggles. + - Tools & helpers + - `frontend/lib/fileReceiver.ts`, `frontend/lib/fileUtils.ts`, `frontend/lib/speedCalculator.ts`, `frontend/lib/utils.ts` — general utilities. + - `frontend/lib/roomIdCache.ts` — room ID cache management. + - `frontend/lib/wakeLockManager.tsx` — wake lock manager (mobile optimization). + - `frontend/lib/utils/ChunkRangeCalculator.ts` — chunk-range calculations. + - `frontend/lib/browserUtils.ts` — browser compatibility helpers. + - `frontend/lib/tracking.ts` — user behavior tracking. + - `frontend/lib/dictionary.ts`, `frontend/lib/mdx-config.ts`, `frontend/lib/blog.ts` — i18n/content/SEO helpers. + +- `frontend/stores/` — Shared app state (Zustand). + + - `frontend/stores/fileTransferStore.ts` — Single source of truth for transfer progress/state (Zustand singleton, persists across routes). + +- `frontend/types/`, `frontend/constants/` — Types and constants. + + - `frontend/types/global.d.ts` — Global types (lodash module, FileSystemDirectoryHandle). + - `frontend/types/messages.ts` — i18n message and UI content types (Meta, Text, Messages, etc.). + - `frontend/types/webrtc.ts` — WebRTC transfer protocol types (metadata, chunk shape, state machine interfaces). + - `frontend/constants/messages/` — i18n message files (7 languages: en, zh, de, es, fr, ja, ko). + - `frontend/constants/i18n-config.ts` — i18n config (default language, supported languages, display-name mapping). + +- `frontend/content/` — Content. + + - `frontend/content/blog/` — Blog posts (MDX, multilingual), including OSS release, WebRTC file transfer, resume, etc. + - `frontend/lib/blog.ts` — Blog utilities: multilingual post loading, frontmatter parsing, tag extraction, content validation. + +- **Config & build** + - `frontend/package.json`, `frontend/tsconfig.json`, `frontend/tailwind.config.ts` — project configuration. + - `frontend/next.config.mjs`, `frontend/postcss.config.mjs`, `frontend/components.json` — Next.js and component config. + - `frontend/.eslintrc.json` — lint configuration. + - `frontend/Dockerfile`, `frontend/health-check.js` — Docker deploy and health checks. + +## Backend (Express, Socket.IO, Redis) + +- `backend/src/server.ts` — Server entry: Express + Socket.IO init and listen. +- `backend/src/config/env.ts`, `backend/src/config/server.ts` — Environment and server config. + - `backend/src/config/env.ts` — Env var validation (port, CORS, Redis), supports per-env `.env` loading. + - `backend/src/config/server.ts` — CORS config for dev/prod, supports multi-origin and LAN regex matching. +- `backend/src/routes/api.ts` — REST: room create/validate, tracking, debug logs. +- `backend/src/routes/health.ts` — Health checks. +- `backend/src/socket/handlers.ts` — Signaling events: `join`, `initiator-online`, `recipient-ready`, `offer`, `answer`, `ice-candidate`. +- `backend/src/services/redis.ts` — Redis client. +- `backend/src/services/room.ts` — Room/member storage and helpers. +- `backend/src/services/rateLimit.ts` — Redis Sorted Set IP rate limiter. +- `backend/src/types/room.ts`, `backend/src/types/socket.ts` — Types and interfaces. + + - `backend/src/types/socket.ts` — Socket.IO types: JoinData, SignalingData (offer/answer/candidate), InitiatorData, RecipientData. + - `backend/src/types/room.ts` — Room types: RoomInfo (createdAt), ReferrerTrack, LogMessage. + +- **Backend config & scripts** + - `backend/package.json`, `backend/tsconfig.json` — project configuration. + - `backend/Dockerfile`, `backend/.dockerignore` — Docker configuration. + - `backend/health-check.js` — health-check script. + - `backend/scripts/export-tracking-data.js` — data export script. + - `backend/docker/` — Docker-related configs/scripts (Nginx, TURN server config). + +## Deployment & Ops + +- **Root-level config** + + - `docker-compose.yml`, `ecosystem.config.js` — Docker Compose and PM2 config. + - `build-and-deploy.sh`, `deploy.sh` — build and deploy scripts. + - `deploy.config_prod`, `deploy.config_test` — prod/test deployment config. + +- **Docker infrastructure** + + - `docker/nginx/` — Nginx reverse proxy config. + - `docker/scripts/` — deployment scripts (env checks, config generation, deployment tests). + - `docker/ssl/` — SSL certificate directory. + - `docker/coturn/` — TURN server config. + - `docker/letsencrypt-www/` — Let’s Encrypt config. + +- **Build & docs** + - `build/` — ignore this temporary directory. + - `test-health-apis.sh` — health API test script. + - `README.md`, `README.zh-CN.md`, `ROADMAP.md`, `ROADMAP.zh-CN.md` — project docs. + diff --git a/docs/ai-playbook/collab-rules.md b/docs/ai-playbook/collab-rules.md new file mode 100644 index 0000000..a0740a2 --- /dev/null +++ b/docs/ai-playbook/collab-rules.md @@ -0,0 +1,150 @@ +# PrivyDrop AI Playbook — Collaboration Rules + +These rules govern how “human developers + AI assistants” collaborate to evolve the codebase efficiently and safely, without breaking the privacy stance or technical baseline. This document complements the index (`index.md`) and the flow map (`flows.md`): it constrains “how we work” without re-stating “what the system does”. + +- Scope: the entire repository (code + docs) +- Audience: human developers, AI assistants, reviewers +- Change principles: best practices first, one goal per change, reversible, verifiable + +## 1. Collaboration Principles + +- Best practices first: prefer proven approaches consistent with the existing stack; avoid reinventing the wheel. +- One change, one purpose: each change should focus on one goal; avoid “while I’m here” fixes. +- Privacy stance: never introduce (or suggest) server-relayed file transfers; the backend is for signaling and room coordination only. +- Small steps: keep PRs small and easy to roll back; prefer the minimum viable change. +- Traceability: commit messages, PR descriptions, and code comments should be clear and reproducible. + +## 2. Plan First (Hard Requirement) + +Before implementing anything, you must propose a “change plan” and get approval. The plan must include: goals, scope + file list, approach, risks + mitigations, acceptance criteria, rollback, docs to update, and validation. + +Recommended reading before implementation (and reference in the plan): + +- `docs/ai-playbook/index.md` +- `docs/ai-playbook/code-map.md` +- `docs/ai-playbook/flows.md` + +## 3. Language & Comments + +- Communication: always use Simplified Chinese when communicating with the project owner/maintainers. +- Code comments, exported symbol naming, commit messages, and PR titles/descriptions must be in English. +- User/marketing docs may be bilingual; this collaboration guide is maintained in both languages. +- Use TSDoc/JSDoc (English) for exported functions, complex flows, and shared types to keep APIs readable. + +## 4. Next.js (Frontend) Conventions + +- App Router defaults to Server Components; use `"use client"` only when interaction truly requires it. +- Reuse existing UI (Tailwind + shadcn/ui + Radix); do not introduce new UI libraries without approval. +- i18n: all visible copy must go through dictionaries and the `frontend/app/[lang]` routes; do not hardcode strings in components. +- Naming & files: + - Components: PascalCase file names and exports (ExampleCard.tsx) + - Hooks: camelCase file names; exports start with use* (useSomething.ts) + - Centralize types/constants; avoid circular deps +- SEO: use Next Metadata and `frontend/components/seo/JsonLd.tsx`; pages must include canonical and multilingual links. +- Performance & a11y: dynamically import heavy components when needed; ensure basic aria/focus behavior. + +## 5. TypeScript & Style + +- Keep types strict; avoid any. Use unknown when needed and narrow explicitly; exported functions should have explicit return types. +- Follow existing ESLint/Prettier and path aliases (`@/...`); do not introduce new formatters. +- Keep functions small and clear; move complex logic into service/util layers. Components should consume state, not mutate global state. +- Avoid 1-letter variable names; avoid magic numbers—centralize them as constants. + +## 6. WebRTC / Transfer Guardrails (Do Not Break) + +- Keep established strategy: 32MB batches + 64KB network chunks; do not casually change DataChannel bufferedAmountLowThreshold or maxBuffer strategy. +- Resume, strict sequential disk writes, and multi-format compatibility are baseline capabilities—do not downgrade/remove them. +- Signaling and message names (offer/answer/ice-candidate, etc.) must stay compatible; any breaking change must follow the “must ask” process. +- Reconnect and queue handling (ICE candidate caching, backpressure, send retries) must remain consistent; changes require risk assessment and thorough validation. +- Never send file contents (in any form) to the backend or third-party services. + +## 7. Backend Constraints (Signaling Service) + +- Signaling + room management only. Never persist user file data; logs must not include sensitive content or raw payloads. +- Keep rate limiting and abuse protection; if extending APIs, ensure backward compatibility or provide a migration path. + +## 8. Dependencies & Security + +- New dependencies require justification in the plan: size (ESM/SSR compatibility), maintenance health, license, alternatives, security impact. +- Do not add telemetry/tracking; do not log sensitive data; follow least privilege. +- Inject config via env vars; never hardcode secrets or service endpoints in the repo. + +## 9. Documentation Sync + +- If code changes affect flows, interfaces, or key entry points, update in the same PR: + - `docs/ai-playbook/flows.zh-CN.md` + - `docs/ai-playbook/code-map.zh-CN.md` +- PRs must list “docs impacted” to avoid the playbook going stale; keep the index page lean and only update it when adding new links. + +## 10. Validation & Regression + +- Frontend: must build (`next build`); include manual verification for key paths (at least: create/join room, single/multi file, folder, large files, resume, cross-browser transfers, i18n routes + SEO metadata). +- Backend: Socket.IO core flow works. +- Regression checklist: reconnect flow, download counts + state cleanup, store single-source-of-truth constraint, browser compatibility (Chromium/Firefox). + +## 11. Must Ask First (Approval Required) + +- Breaking changes to protocols/message names/public APIs/storage formats. +- Architecture changes impacting privacy stance or crossing boundaries (any relay or persistence). +- New dependencies, new infrastructure, or large refactors. +- Changes to transfer guardrail parameters (chunking, backpressure, retries, etc.). + +## 12. Common Pitfalls + +- Mutating global state from inside components (breaks one-way dataflow). +- Changing code without updating docs, leaving the playbook stale. +- Using any to bypass type and boundary checks. +- Hardcoding UI copy in components instead of using dictionaries/i18n. +- Tweaking critical WebRTC parameters without validation, causing silent regressions. + +## 13. Templates + +Change Plan Template + +``` +Title: + +Goals +- + +Scope / Files +- + +Approach +- + +Risks & Mitigations +- -> + +Acceptance Criteria +- + +Rollback +- + +Docs to Update +- code-map.zh-CN.md / flows.zh-CN.md / README(.zh-CN).md / others? + +Validation +- Build: next build / backend health +- Manual: +``` + +PR Checklist + +``` +- [ ] Single-topic change only +- [ ] Code comments and commit messages are in English +- [ ] No unapproved dependencies/UI libraries added +- [ ] i18n and SEO follow conventions (if applicable) +- [ ] Transfer guardrails are unchanged (or approved + validated) +- [ ] flows / code-map docs are updated in sync +- [ ] Validation notes and regression checklist included +``` + +## 14. References & Quick Entry Points + +- Index & context: `docs/ai-playbook/index.md` +- Code map: `docs/ai-playbook/code-map.md` +- Key flows: `docs/ai-playbook/flows.md` + diff --git a/docs/ai-playbook/flows.md b/docs/ai-playbook/flows.md new file mode 100644 index 0000000..162ac85 --- /dev/null +++ b/docs/ai-playbook/flows.md @@ -0,0 +1,233 @@ +# PrivyDrop AI Playbook — Flows (with Micro-Plan Template) + +This page summarizes the core P2P transfer flows and signaling/reconnection sequences, plus practical debugging notes and a compact “micro-plan template”. Use it to align on phases, events, and entry points before making changes. + +## Quick Navigation + +- Fast path: this page contains 1–5 (key flows/messages/debug notes) and 11 (micro-plan template). +- Deep dives (split out from this page): + - Frontend component collaboration (was Section 6): [`docs/ai-playbook/flows/frontend.md`](./flows/frontend.md) + - Backpressure & chunking (was Section 7): [`docs/ai-playbook/flows/backpressure-chunking.md`](./flows/backpressure-chunking.md) + - Resume / partial transfer (was Section 9): [`docs/ai-playbook/flows/resume.md`](./flows/resume.md) + - Reconnect consistency (was Section 10): [`docs/ai-playbook/flows/reconnect-consistency.md`](./flows/reconnect-consistency.md) + +## 1) File Transfer (Single File) + +Sequence (via DataChannel, sender ↔ receiver): + +1. Sender → `fileMetadata` (id, name, size, type, fullName, folderName). +2. Receiver → `fileRequest` (ack metadata; supports offset-based resume). +3. Sender → chunk stream (high-performance, dual-layer buffering): + - StreamingFileReader reads in 32MB batches and sends 64KB network chunks + - NetworkTransmitter uses native WebRTC backpressure (bufferedAmountLowThreshold) + - Each send embeds metadata (chunkIndex, totalChunks, fileOffset, fileId) +4. Receiver → integrity checks & assembly (strict sequential disk writes or in-memory assembly; supports resume). +5. Receiver → `fileReceiveComplete` (success receipt, includes receivedSize). +6. Sender → MessageHandler fires 100% progress callback and clears sending state. + +Sender-side detailed flow: + +1. FileTransferOrchestrator.sendFileMeta() → StateManager records folder/file sizes +2. Receive `fileRequest` → FileTransferOrchestrator.handleFileRequest() +3. Initialize StreamingFileReader (supports startOffset resume) +4. processSendQueue() loop: + - getNextNetworkChunk() returns the next 64KB chunk (efficient slicing within a batch) + - NetworkTransmitter.sendEmbeddedChunk() sends with backpressure control + - ProgressTracker.updateFileProgress() updates progress and speed +5. Wait for `fileReceiveComplete`, then clear isSending state + +Entry points: + +- Sender: `frontend/lib/fileSender.ts` (compat wrapper) → `frontend/lib/transfer/FileTransferOrchestrator.ts` (main orchestrator) +- Key components: StreamingFileReader (fast reads), NetworkTransmitter (backpressure sending), StateManager (state), ProgressTracker (progress) + +Receiver-side detailed flow: + +1. MessageProcessor.handleFileMetadata() → ReceptionStateManager stores file metadata +2. FileReceiveOrchestrator.requestFile() → check resume (getPartialFileSize) +3. Initialize reception: compute expected chunk count; choose storage mode (memory vs disk) based on size +4. Send `fileRequest` (with offset if needed) → wait for sender to start +5. handleBinaryChunkData() loop: + - ChunkProcessor.convertToArrayBuffer() handles multiple payload types (Blob/Uint8Array/ArrayBuffer) + - ChunkProcessor.parseEmbeddedChunkPacket() parses the “embedded metadata” packet format + - ChunkProcessor.validateChunk() validates fileId, chunkIndex, chunkSize + - Store chunks in an array (or write sequentially via SequencedDiskWriter) + - ProgressReporter.updateFileProgress() throttles progress updates (100ms) +6. Auto completion detection: checkAndAutoFinalize() validates completeness +7. Finalize based on storage mode: + - Large/disk: StreamingFileWriter.finalizeWrite() + - Small/memory: FileAssembler.assembleFileFromChunks() +8. Send `fileReceiveComplete` with receivedSize and receivedChunks + +Entry points: + +- Sender: `frontend/lib/fileSender.ts` (compat wrapper) → `frontend/lib/transfer/FileTransferOrchestrator.ts` (main orchestrator) +- Receiver: `frontend/lib/fileReceiver.ts` (compat wrapper) → `frontend/lib/receive/FileReceiveOrchestrator.ts` (main orchestrator) +- Key components: StreamingFileReader (fast reads), NetworkTransmitter (backpressure), ChunkProcessor (format handling), StreamingFileWriter (disk writes), FileAssembler (memory assembly) + +Notes: + +- **Sender**: dual-layer buffering (32MB batches + 64KB network chunks), native WebRTC backpressure, resume support +- **Receiver**: strict sequential disk writer (SequencedDiskWriter), multi-format conversion, smart storage selection (≥1GB auto disk mode) +- **Compatibility**: ChunkProcessor supports Blob/Uint8Array/ArrayBuffer to address Firefox quirks +- **Progress throttling**: ProgressReporter updates at different rates (file 100ms, folder 200ms) to avoid UI overload +- **Resume**: getPartialFileSize() checks local partial files; fileRequest.offset drives resume +- **Debug support**: ReceptionConfig provides verbose chunk/progress logs for investigation + +## 2) File Transfer (Folder) + +Sequence (run the single-file flow for each file): + +1. Sender → send `fileMetadata` for all files in the folder. +2. Receiver → `folderRequest` (confirm starting the batch transfer). +3. For each file: run the single-file flow, but do not mark global 100% on individual file completion. +4. Receiver → after all files finish, send `folderReceiveComplete`. +5. Sender → mark folder-level progress as 100% (fire final callback). + +## 3) Signaling & Reconnect (Socket.IO) + +High-level sequence: + +1. Client → REST: create or fetch a `roomId` (`backend/src/routes/api.ts`). +2. Client → Socket.IO: `join` the room (backend validates and binds socket ↔ room). +3. Online state & reconnection coordination within the room: + - Initiator → `initiator-online` (online/ready; tells the peer it can rebuild the connection). + - Recipient → `recipient-ready` (ready; initiator may start offer). +4. WebRTC negotiation relay: + - `offer` → backend → peer. + - `answer` → backend → peer. + - `ice-candidate` → backend → peer. + +### Join Success Conditions & Timeout Strategy (Frontend Fault-Tolerance) + +- Primary success condition: receive `joinResponse(success=true)` (backend completed validation and socket↔room binding). +- Equivalent “success signals” (fault-tolerance; any one means “we’re effectively in” and should clear listeners/timers): + - Initiator: receives `ready` or `recipient-ready` + - Recipient: receives `offer` +- Timeout: 15 seconds. This covers weak networks, mobile, and Socket.IO polling fallback where joinResponse can arrive late. +- Why it’s safe: `ready/recipient-ready` are room broadcast events; `offer` is the P2P handshake starting point. If you can receive these, you’re in the room and negotiation has begun—treat it as success to avoid false “Join room timeout” errors. + +Reconnect mechanics (mobile network switching support): + +- **Dual disconnect detection**: Socket.IO `disconnect` → set `isSocketDisconnected = true`; P2P disconnect → set `isPeerDisconnected = true` and call `cleanupExistingConnection()` +- **Reconnect trigger**: only when both socket and P2P are disconnected → `attemptReconnection()`, guarded by `reconnectionInProgress` to avoid concurrent reconnects +- **State restoration**: reconnect calls `joinRoom(roomId, isInitiator, sendInitiatorOnline)`; the initiator auto-sends `initiator-online` and the recipient replies `recipient-ready` +- **ICE candidate queue**: cache candidates in `iceCandidatesQueue` until ready, then flush; support re-queuing invalid candidates with connection-state validation +- **Wake lock**: request via WakeLockManager when connected; release on disconnect to stabilize mobile transfers +- **Graceful disconnect tracking**: `gracefullyDisconnectedPeers` tracks intentionally closed peers; send retries skip them +- **DataChannel send retries**: up to 5 attempts with backoff from 100ms to 1000ms; skip peers marked as gracefully disconnected + +**Backend signaling & room management**: + +Socket.IO event handling flow: + +1. **join**: IP rate limit → validate room existence → bind socket-room → success response → broadcast `ready` +2. **Reconnect state sync**: initiator reconnect sends `initiator-online`; recipient replies `recipient-ready` +3. **Relay signaling**: offer/answer/ice-candidate are forwarded with `socket.to(peerId).emit()`, including a `from` field +4. **Disconnect cleanup**: broadcast `peer-disconnected` → unbind socket-room → delete empty rooms after 15 minutes + +**Room management**: + +- Redis structures: + - `room:` (Hash): room creation time + - `room::sockets` (Set): sockets in the room + - `socket:` (String): the roomId for a socket +- ID generation: prefer 4-digit numeric IDs; fall back to 4-character alphanumeric on collision +- Idempotency: long IDs (≥8 chars) can be reused across reconnects +- TTL: 24 hours, refreshed on activity + +**Rate limiting**: + +- Redis Sorted Set based IP rate limit +- Up to 2 requests per 5-second window +- Uses pipeline to keep operations atomic + +Entry points: + +- Frontend: `frontend/hooks/useWebRTCConnection.ts`, `frontend/lib/webrtc_base.ts`, `frontend/lib/webrtc_Initiator.ts`, `frontend/lib/webrtc_Recipient.ts` +- Backend: `backend/src/socket/handlers.ts` (all signaling events), `backend/src/services/room.ts`, `backend/src/routes/api.ts` + +## 4) DataChannel Messages & Constraints (Overview) + +- Messages (example names): `fileMetadata`, `fileRequest`, `chunk`, `fileReceiveComplete`, `folderRequest`, `folderReceiveComplete`, plus potential flow-control/keepalive messages. +- Core fields: file/folder id, indices/ranges, sizes, names, optional checksums. +- Key constraints: + - Chunk size: choose a safe range per browser/network; mind channel buffer thresholds. + - Backpressure: check `RTCDataChannel.bufferedAmount` and throttle as needed. + - Completion: mark 100% only after `fileReceiveComplete` / `folderReceiveComplete`. + - Resume: `fileRequest` can carry offset/range to support partial transfers. + +## 5) Debugging Notes (Distilled from Experience) + +- Download races / double counting: treat `frontend/stores/fileTransferStore.ts` as the single source of truth; provide cleanup APIs at the store level (e.g. `clearSendProgress`, `clearReceiveProgress`) rather than deleting objects locally in components. +- Recipient reconnect & room state: reset state correctly; UI must strictly derive from the store; leaving/rejoining should clear related state; follow `initiator-online`/`recipient-ready` ordering before starting an offer; verify room membership after reconnect. +- Reconnect with cached roomId: if `roomId` is cached, ensure renegotiation is triggered via online state sync (`initiator-online`/`recipient-ready`); backend must correctly clean and restore socket↔room mappings on disconnect/reconnect. +- Multiple transfers: don’t over-dedupe and mask real “second downloads”; rely on correct state cleanup. +- Dataflow principle: one-way dataflow (Store → Hooks → Components); hooks adapt, components consume but do not mutate shared state. +- **Practical debugging tactics**: + - Add structured logs for connection state changes and store updates; for timing/race issues, `setTimeout(..., 0)` can help reorder updates + - DataChannel send retries: `sendToPeer()` supports 5 retries with backoff from 100ms→1000ms; skip gracefully disconnected peers + - WebRTC payload compatibility: handle `ArrayBuffer`/`Blob`/`Uint8Array`/`TypedArray` to cover Firefox quirks + - Connection state monitoring: react to `connectionState` changes (connected/disconnected/failed/closed) + - Backpressure control: DataChannel uses `bufferedAmountLowThreshold = 256KB`; check `bufferedAmount` before sending + - Join false positives: if “Join room timeout” appears but you immediately see `offer/answer/connected` logs, it’s often a late joinResponse rather than a real failure; the 15s window plus “equivalent success signals” corrects it automatically + +## 6) Frontend Component System & Core-Orchestrator Collaboration + +This section is split into: [`docs/ai-playbook/flows/frontend.md`](./flows/frontend.md) + +- Use it for: understanding boundaries and collaboration among UI components, hooks, and the store +- Includes: ClipboardApp orchestrator, hook layering, connection feedback state machine, dataflow patterns, etc. + +## 7) Backpressure & Chunking Strategy (Deep Dive) + +This section is split into: [`docs/ai-playbook/flows/backpressure-chunking.md`](./flows/backpressure-chunking.md) + +- Use it for: verifying thresholds, chunk/batch strategy, embedded packet format, and performance tuning points +- Includes: sender dual-buffering, receiver storage strategy, debugging/monitoring tips, etc. + +## 9) Resume / Partial Transfer (Deep Dive) + +This section is split into: [`docs/ai-playbook/flows/resume.md`](./flows/resume.md) + +- Use it for: verifying resume detection, offset negotiation, and chunk-range calculations +- Includes: ChunkRangeCalculator, sender/receiver resume flows, limitations, and debugging notes + +## 10) Reconnect & State Consistency (Deep Dive) + +This section is split into: [`docs/ai-playbook/flows/reconnect-consistency.md`](./flows/reconnect-consistency.md) + +- Use it for: verifying dual disconnect rules, ICE candidate queues, send retries, and consistency safeguards +- Includes: reconnect triggers, retry strategy, mobile-specific additions, debugging notes + +## 11) Micro-Plan Template (for Aligning on Small Changes) + +Title: + +Background / Problem + +- What user scenario or defect are we fixing? + +Goals & Non-Goals + +- What’s in scope, and what’s explicitly out of scope? + +Impacted Files & Messages + +- Code: list key files (e.g. `frontend/lib/webrtc_base.ts`, `backend/src/socket/handlers.ts`). +- Protocol: list DataChannel messages/fields to be changed. + +State Machine / Flow Changes + +- Add/remove/modify phases; include a short sequence diagram or steps. + +Tests & Regression Checklist + +- Unit/integration (if applicable), manual scenarios, performance/boundaries, reconnect cases. + +Docs to Update + +- `code-map.md` (if new entry points appear) +- `flows.md` (if flows/messages/constraints change) +- Other architecture or deployment docs (if involved) + diff --git a/docs/ai-playbook/flows/backpressure-chunking.md b/docs/ai-playbook/flows/backpressure-chunking.md new file mode 100644 index 0000000..cb234e4 --- /dev/null +++ b/docs/ai-playbook/flows/backpressure-chunking.md @@ -0,0 +1,99 @@ +# PrivyDrop AI Playbook — Backpressure & Chunking Strategy (Deep Dive) + +← Back to flow index: [`docs/ai-playbook/flows.md`](../flows.md) + +(This page is the English edition of content split out from `docs/ai-playbook/flows.zh-CN.md`, preserving the original section numbering and structure.) + +## 7) Backpressure & Chunking Strategy (Deep Dive) + +### Sender Dual-Layer Buffering Architecture + +**Design rationale**: + +- **File read layer**: 4MB chunks reduce FileReader calls; 8 chunks form a 32MB batch +- **Network layer**: 64KB pieces fit WebRTC DataChannel limits and avoid sendData failed errors +- **Performance**: efficient slicing inside each batch; a single FileReader.read() yields 512 network chunks + +**Key parameters**: + +```typescript +TransferConfig.FILE_CONFIG = { + CHUNK_SIZE: 4194304, // 4MB - file read chunks + BATCH_SIZE: 8, // 8 chunks = 32MB batch + NETWORK_CHUNK_SIZE: 65536, // 64KB - safe WebRTC send size +}; +``` + +**Backpressure control**: + +- **DataChannel thresholds**: `bufferedAmountLowThreshold = 256KB` (Initiator) and `512KB` (NetworkTransmitter) +- **Max buffer**: `maxBuffer = 1MB`; wait until pressure releases when exceeded +- **Async waiting**: listens to `bufferedamountlow`, with a timeout safeguard (10 seconds) + +**Embedded metadata packet format**: + +``` +[4-byte length][JSON metadata][payload bytes] +``` + +- Each network chunk includes: chunkIndex, totalChunks, fileOffset, fileId, isLastChunk +- Receiver can parse each packet independently without relying on extra shared state + +### Receiver Smart Storage Strategy + +**Storage decision logic**: + +```typescript +ReceptionConfig.shouldSaveToDisk(fileSize, hasSaveDirectory); +``` + +- **In-memory**: file < 1GB and no save directory chosen +- **Disk**: file ≥ 1GB or the user selected a save directory +- **Buffer cap**: up to 100 chunks buffered (≈ 6.4MB) + +**Chunk validation**: + +- **Format compatibility**: ArrayBuffer/Blob/Uint8Array/TypedArray supported +- **Integrity checks**: validate fileId, chunkIndex, chunkSize consistency +- **Firefox quirks**: Blob size checks and conversion error handling + +**Strict sequential disk writes**: + +- **SequencedDiskWriter**: guarantees in-order writes; enables streaming for large files +- **Resume**: `getPartialFileSize()` checks existing partial files +- **Auto completion**: `checkAndAutoFinalize()` verifies completeness + +### Performance Tuning Details + +**Sender-side optimizations**: + +- **Batch reads**: 32MB batches reduce I/O operations and improve large file read throughput +- **Network fit**: 64KB balances transfer efficiency with cross-browser compatibility +- **Backpressure response**: leverages native WebRTC backpressure to prevent drops + +**Receiver-side optimizations**: + +- **Unified format conversion**: ChunkProcessor handles multiple payload formats in one place +- **Progress throttling**: 100ms for files, 200ms for folders to avoid UI overload +- **Memory management**: small files assemble in memory; large files stream to disk + +**Error handling**: + +- **Send retries**: NetworkTransmitter returns boolean for upper-layer retry logic +- **Conversion tolerance**: when Blob conversion fails, return null instead of aborting the transfer +- **Timeout safeguards**: 30s completion timeout; 5s graceful close timeout + +### Debugging & Monitoring + +**Dev logs**: + +- **Chunk tracking**: log details every 100 chunks and for the last chunk +- **Backpressure monitoring**: buffer size changes and wait-time stats +- **Performance metrics**: transfer speed, batch processing time, conversion cost + +**Production optimizations**: + +- **Conditional logging**: `ENABLE_CHUNK_LOGGING` and `ENABLE_PROGRESS_LOGGING` +- **Error reporting**: critical errors are sent to the backend via `postLogToBackend` +- **Performance sampling**: use `performance.now()` for precise timings + diff --git a/docs/ai-playbook/flows/frontend.md b/docs/ai-playbook/flows/frontend.md new file mode 100644 index 0000000..53ae6e8 --- /dev/null +++ b/docs/ai-playbook/flows/frontend.md @@ -0,0 +1,178 @@ +# PrivyDrop AI Playbook — Frontend Component System & Core-Orchestrator Collaboration + +← Back to flow index: [`docs/ai-playbook/flows.md`](../flows.md) + +(This page is the English edition of content split out from `docs/ai-playbook/flows.zh-CN.md`, preserving the original section numbering and structure.) + +## 6) Frontend Component System & Core-Orchestrator Collaboration + +### Component Architecture Layers + +``` +App Router (page.tsx/layout.tsx) + ↓ +HomeClient (layout & SEO) + ↓ +ClipboardApp (top-level UI orchestrator) + ↓ +SendTabPanel/RetrieveTabPanel (feature panels) + ↓ +Business Hooks (state + orchestration) + ↓ +Core Services (webrtcService) + Store (fileTransferStore) +``` + +### ClipboardApp Orchestrator Pattern + +**Core responsibilities**: + +- Integrates 5 key business hooks: useWebRTCConnection, useFileTransferHandler, useRoomManager, usePageSetup, useClipboardAppMessages +- Handles global drag events: dragenter/dragleave/dragover/drop, supports multi-file and folder-tree traversal +- Manages the Send/Retrieve tabs via activeTab +- Unified messaging: shareMessage/retrieveMessage auto-dismiss after 4 seconds + +### Hook Layering & Separation of Concerns + +**useWebRTCConnection** (state bridge): + +- Computes global transfer state (isAnyFileTransferring) +- Exposes webrtcService methods (broadcastDataToAllPeers, requestFile, requestFolder) +- Provides reset methods (resetSenderConnection, resetReceiverConnection) + +**useFileTransferHandler** (files and content): + +- File ops: addFilesToSend (dedupe), removeFileToSend +- Downloads: handleDownloadFile (supports folder ZIP downloads) +- Key fix: uses `useFileTransferStore.getState()` to read the latest state and avoid stale closures +- Retry: up to 3 retries with 50ms interval and detailed error logs + +**useRoomManager** (room lifecycle): + +- Room ops: joinRoom (supports cached-ID reconnect), processRoomIdInput (750ms debounce) +- Leave protection: confirmation prompt while transferring (checks isAnyFileTransferring) +- Status text: dynamic room status copy +- Link generation: auto-generates share links + +**usePageSetup** (page initialization): + +- i18n dictionary loading and error handling +- URL param handling: extracts roomId and auto-joins (200ms delay to ensure DOM readiness) +- Referrer tracking (trackReferrer) + +**useClipboardAppMessages** (messages): + +- Split message states: shareMessage (send side) and retrieveMessage (receive side) +- Unified API: putMessageInMs(message, isShareEnd, displayTimeMs) +- Auto cleanup: clears message state after 4 seconds + +### Panel-Specific Design + +**SendTabPanel**: + +- Dual-mode room ID generation: 4-digit numbers (via backend API) and UUID (via Web Crypto) +- Rich-text editor integration (dynamic import, SSR disabled) +- File upload handling + file list management +- Share link + QR code + +**RetrieveTabPanel**: + +- File System Access API integration: directory selection and direct disk saves +- Rich-text rendering (dangerouslySetInnerHTML) +- File requests + download state management +- Save-location selection and large file/folder hints + +**FileListDisplay**: + +- Smart grouping and stats for files/folders +- Cross-browser download strategy: Chrome auto-download; other browsers show manual save guidance +- Download count stats + transfer progress tracking +- Resume state and storage mode display (memory/disk) + +### Key UX Improvements + +1. **Stale-closure fix for download state**: `useFileTransferHandler.ts:110` uses `useFileTransferStore.getState()` +2. **Debounced roomId validation**: `useRoomManager.ts:247` uses lodash debounce (750ms) +3. **Leaving while transferring**: `useRoomManager.ts:164,218` checks `isAnyFileTransferring` and shows a confirmation dialog +4. **Cached-ID reconnect**: `useRoomManager.ts:91` detects long IDs (≥8 chars) and auto-sends `initiator-online` +5. **Folder ZIP downloads**: `useFileTransferHandler.ts:89` builds ZIPs on the fly with JSZip +6. **Global drag-and-drop robustness**: ClipboardApp uses dragCounter to avoid mis-detecting drag state; supports webkitGetAsEntry folder traversal +7. **Clipboard compatibility**: useClipboardActions supports modern navigator.clipboard APIs with document.execCommand fallback +8. **Rich-text safety**: useRichTextToPlainText is safe on server render; client-side DOM conversion handles block elements +9. **In-app navigation without breaking transfers (same tab)**: relies on `frontend/stores/fileTransferStore.ts` (Zustand singleton) and `frontend/lib/webrtcService.ts` (service singleton). App Router navigation keeps transfers and selected/received content intact. Avoid calling `webrtcService.leaveRoom()` or resetting the store in route-change side effects. Refresh/new tab is not covered. + +### UI Connection Feedback State Machine (Weak Network / VPN Hints) + +- Join phase + - Immediate: `join_inProgress` (“Joining the room…”). + - Not finished after 3s: `join_slow` (“Connection seems slow—check your network/VPN…”). + - Timeout after 15s: `join_timeout` (“Join timed out…”). + - Equivalent success signal: while waiting for `joinResponse`, if `ready/recipient-ready/offer` arrives, treat it as “joined” and immediately clear the 3s/15s timers and hints to avoid “slow/timeout hints after success”. +- Negotiation phase (WebRTC) + - Enter `new/connecting`: normalize to “negotiating” → `rtc_negotiating`. + - Not connected after 8s: `rtc_slow` (“Your network may be restricted—try turning off VPN or try again later”). Only fires when the page is visible. Only once per negotiation attempt (timer starts when either side enters negotiating; ownership goes to the side that entered negotiating first). +- Connection & reconnection + - First `connected`: `rtc_connected` (one-time). + - Foreground disconnect: `rtc_reconnecting` → upon recovery `rtc_restored`. + - Background disconnect does not notify; when returning to foreground, if still disconnected, notify `rtc_reconnecting` immediately. + - If the page is backgrounded during a disconnect, when returning to foreground, if still negotiating and the slow timer had fired, emit `rtc_slow` once and mark it as already shown to avoid repeats. + +Implementation locations: + +- `frontend/hooks/useRoomManager.ts`: join-phase hints and timers (3s slow, 15s timeout), cleared on join success/failure; supports early “equivalent success signals” (`ready/recipient-ready/offer`). +- `frontend/hooks/useConnectionFeedback.ts`: maps WebRTC connection states to UI hints. + - Phase normalization (mapPhase): `new/connecting` → `negotiating`; `failed/closed` → `disconnected`. + - Negotiation slow hint: 8s timer, foreground/background throttling, only once per attempt (including deferred background → emitted on foreground). + - One-shot hints: first `connected` only once; disconnected → restored shows `rtc_restored`; `rtc_reconnecting` only in foreground. + - Shared helpers: timer + visibility control via `frontend/utils/useOneShotSlowHint.ts`; phase normalization via `frontend/utils/rtcPhase.ts`. + +Copy & i18n: + +- Message keys live in `frontend/constants/messages/*.{ts}`; types in `frontend/types/messages.ts`. +- Key messages: `join_inProgress`, `join_slow`, `join_timeout`, `rtc_negotiating`, `rtc_slow`, `rtc_connected`, `rtc_reconnecting`, `rtc_restored` (filled across en/ja/es/de/fr/ko). + +Throttling & display: + +- All hints auto-dismiss after ~4–6 seconds; use `useClipboardAppMessages.putMessageInMs(message, isShareEnd, ms)` as the unified display channel. +- Connection feedback fires under three constraints: state transition + ever/wasDisc markers + visibility checks, preventing “hint storms”. + +10. **Auto-join on switching to Retrieve (cached ID)**: when switching to Retrieve, not in a room, no `roomId` in URL, empty input, and a cached ID exists locally, auto-fill and call joinRoom. Entry: `frontend/components/ClipboardApp.tsx` (watches activeTab, reads `frontend/lib/roomIdCache.ts`). +11. **Sender “Use cached ID” joins immediately**: clicking “Use cached ID” in SendTabPanel triggers joining right away (not just filling the input). Entry: `frontend/components/ClipboardApp/CachedIdActionButton.tsx` (`onUseCached`) + `frontend/components/ClipboardApp/SendTabPanel.tsx`. +12. **Dark theme toggle**: single-button Light/Dark toggle in `frontend/components/web/ThemeToggle.tsx`, integrated into `frontend/components/web/Header.tsx` (desktop & mobile). Some local styles are migrated from hardcoded colors to tokens (e.g. retrieve panel uses `bg-card text-card-foreground`). + +### Frontend Architecture Specializations + +**Rich-text editor module**: + +- **RichTextEditor**: main editor component; contentEditable, image paste, formatting tools; SSR disabled +- **Toolbar separation**: BasicFormatTools (bold/italic/underline), FontTools (font/size/color), AlignmentTools, InsertTools (link/image/code block) +- **Type-safe design**: complete TypeScript types (FormatType, AlignmentType, FontStyleType, CustomClipboardEvent) +- **Editor hooks**: useEditorCommands (commands), useSelection (selection), useStyleManagement (style) + +**Website page components**: + +- **Header responsive nav**: desktop horizontal nav + mobile hamburger; integrates GitHub link and language switcher +- **Footer i18n**: dynamic copyright year and multilingual support links via languageDisplayNames +- **FAQSection**: configurable “tool page vs standalone page”, heading level control, and automatic FAQ array generation +- **Content components**: HowItWorks (animated steps + video), SystemDiagram, KeyFeatures + +**UI component library architecture**: + +- **Built on Radix UI**: Button (CVA variants), Accordion, Dialog, Select, DropdownMenu +- **Design-system consistency**: shared cn utility, theme token system, animation transitions +- **Composable patterns**: DialogHeader/DialogFooter/DialogTitle/DialogDescription +- **Lazy-load optimizations**: LazyLoadWrapper uses react-intersection-observer with rootMargin tuning to reduce layout shifts + +**Shared components as utilities**: + +- **clipboard_btn**: WriteClipboardButton/ReadClipboardButton split; integrates useClipboardActions; supports i18n messages +- **TableOfContents**: Chinese heading ID generation, scroll tracking, indentation, IntersectionObserver +- **JsonLd SEO**: multi-type support, suppressHydrationWarning, array vs single object handling +- **AutoPopupDialog/YouTubePlayer**: scenario-driven wrappers designed for reuse + +### Dataflow Pattern + +- **One-way dataflow**: Store → Hooks → Components +- **Centralized state**: all state is owned by `useFileTransferStore` +- **Standardized error handling**: unified message channel (putMessageInMs) +- **i18n integration**: useLocale + getDictionary provide multilingual content + diff --git a/docs/ai-playbook/flows/reconnect-consistency.md b/docs/ai-playbook/flows/reconnect-consistency.md new file mode 100644 index 0000000..dd8ba6a --- /dev/null +++ b/docs/ai-playbook/flows/reconnect-consistency.md @@ -0,0 +1,183 @@ +# PrivyDrop AI Playbook — Reconnect & State Consistency (Deep Dive) + +← Back to flow index: [`docs/ai-playbook/flows.md`](../flows.md) + +(This page is the English edition of content split out from `docs/ai-playbook/flows.zh-CN.md`, preserving the original section numbering and structure.) + +## 10) Reconnect & State Consistency (Deep Dive) + +### WebRTC Base-Layer Reconnect Mechanics + +**Dual disconnect detection**: + +```typescript +// webrtc_base.ts +private isSocketDisconnected = false; // Socket.IO connection state +private isPeerDisconnected = false; // P2P connection state +private gracefullyDisconnectedPeers = new Set(); // peers closed gracefully +``` + +**Reconnect trigger**: only start reconnection when both Socket.IO and P2P are disconnected: + +```typescript +// Avoid duplicate reconnects: socket disconnect != P2P disconnect +if ( + this.isSocketDisconnected && + this.isPeerDisconnected && + !this.reconnectionInProgress +) { + this.attemptReconnection(); +} +``` + +### ICE Candidate Queue Management + +**Candidate caching strategy**: + +- **Before ready**: cache candidates in the `iceCandidatesQueue` Map, grouped by peerId +- **After ready**: flush cached candidates and add them to RTCPeerConnection in order +- **Invalid handling**: re-queue invalid candidates and retry after validating connection state + +**Implementation detail**: + +```typescript +private iceCandidatesQueue = new Map(); +// Cache candidates until the connection is ready +if (dataChannel?.readyState !== 'open') { + this.queueIceCandidate(candidate, peerId); +} else { + this.addIceCandidate(candidate, peerId); +} +``` + +### DataChannel Send-Retry Mechanism + +**5-attempt retry policy**: + +```typescript +async sendToPeer(data: string | ArrayBuffer, peerId: string): Promise { + for (let attempt = 1; attempt <= 5; attempt++) { + try { + dataChannel.send(data); + return true; + } catch (error) { + if (this.gracefullyDisconnectedPeers.has(peerId)) { + return false; // skip peers that were closed gracefully + } + if (attempt === 5) throw error; + await new Promise(resolve => setTimeout(resolve, attempt * 100)); // 100ms→1000ms + } + } +} +``` + +**Backoff**: 100ms → 200ms → 300ms → 400ms → 500ms, up to 5 attempts + +### Room-Layer Reconnect Support + +**Idempotency**: + +- **Long IDs**: roomId length ≥ 8 supports room reuse across reconnects +- **Short IDs**: 4-digit numeric IDs must be re-generated after disconnect to avoid collisions + +**Cached-ID reconnect optimization**: + +```typescript +// useRoomManager.ts +if (roomId.length >= 8) { + // long IDs auto-send initiator-online + this.sendInitiatorOnline(); +} +``` + +**State sync sequence**: + +1. **Initiator reconnects**: sends `initiator-online` to signal readiness +2. **Recipient replies**: `recipient-ready` confirms readiness +3. **WebRTC negotiation**: re-run offer/answer/ICE exchange +4. **Transfer continues**: resume file transfer on the new DataChannel + +### State Consistency Safeguards + +**Store as the single source of truth**: + +```typescript +// fileTransferStore.ts +export const useFileTransferStore = create((set, get) => ({ + sendProgress: new Map(), + receiveProgress: new Map(), + // cleanup APIs to avoid double counting + clearSendProgress: (fileId: string) => + set((state) => { + const newProgress = new Map(state.sendProgress); + newProgress.delete(fileId); + return { sendProgress: newProgress }; + }), +})); +``` + +**Connection state machine**: + +```typescript +type ConnectionStatus = 'connecting' | 'connected' | 'disconnected' | 'failed' | 'closed'; + +// react to state transitions +connectionStateChangeHandler(status: ConnectionStatus) { + switch (status) { + case 'connected': + this.gracefullyDisconnectedPeers.clear(peerId); + this.resetReconnectionState(); + break; + case 'disconnected': + case 'failed': + this.cleanupExistingConnection(peerId); + break; + } +} +``` + +### Mobile Optimizations + +**Wake lock management**: + +```typescript +// WakeLockManager +async requestWakeLock(): Promise { + try { + this.wakeLock = await navigator.wakeLock.request('screen'); + this.wakeLock.addEventListener('release', () => { + this.wakeLock = null; + }); + } catch (error) { + console.warn('Wake lock request failed:', error); + } +} +``` + +**Adapting to network changes**: + +- **Detection**: listen to `connectionstatechange` to infer network quality changes +- **Auto-reconnect**: `connectionState: 'disconnected' | 'failed' | 'closed'` all route into the same reconnect path (attemptReconnection) +- **Restore state**: after reconnect, restore room status and transfer progress + +**Mobile background/foreground addendum**: + +- **Auto re-join on socket reconnect**: on `socket.on('connect')`, if a `roomId` exists and (`lastJoinedSocketId !== socket.id` or `!isInRoom`), force `joinRoom(roomId, isInitiator, isInitiator)`. The initiator auto-broadcasts `initiator-online`; the recipient replies `recipient-ready`. +- **Identity tracking**: after a successful `joinRoom`, record `lastJoinedSocketId = socket.id` to detect “socketId changed after background resume”. +- **Lowered threshold**: `attemptReconnection` can start as long as `roomId` exists and any of the following hold: P2P disconnected / socket disconnected / socketId changed. It no longer requires “socket and P2P disconnected at the same time”. + +### Reconnect Debugging Notes + +**Key log points**: + +- **Dual disconnect detection**: record timestamps for Socket.IO vs P2P disconnects +- **Candidate queue**: count cached ICE candidates and flush durations +- **Send retries**: record retry attempts, delays, and the final result +- **State restoration**: trace `initiator-online` → `recipient-ready` ordering + +**Common diagnostics**: + +- **Duplicate reconnects**: check `reconnectionInProgress` and the `gracefullyDisconnectedPeers` set +- **Invalid candidates**: validate `iceConnectionState` and `iceGatheringState` +- **State divergence**: confirm store progress cleanup and connection-state synchronization + diff --git a/docs/ai-playbook/flows/resume.md b/docs/ai-playbook/flows/resume.md new file mode 100644 index 0000000..79f6dad --- /dev/null +++ b/docs/ai-playbook/flows/resume.md @@ -0,0 +1,109 @@ +# PrivyDrop AI Playbook — Resume / Partial Transfer (Deep Dive) + +← Back to flow index: [`docs/ai-playbook/flows.md`](../flows.md) + +(This page is the English edition of content split out from `docs/ai-playbook/flows.zh-CN.md`, preserving the original section numbering and structure.) + +## 9) Resume / Partial Transfer (Deep Dive) + +### Core Resume Mechanism + +**Resume detection & state restoration**: + +- **Sender init**: `StreamingFileReader constructor(file, startOffset)` supports starting from any offset +- **Receiver detection**: `StreamingFileWriter.getPartialFileSize()` checks partial files via the File System Access API +- **State sync**: the fileRequest message includes an offset parameter to tell the sender where to continue + +**Chunk index calculation**: + +```typescript +// unified chunk calculation logic +const startChunk = Math.floor(startOffset / chunkSize); +const expectedChunks = Math.ceil((fileSize - startOffset) / chunkSize); +``` + +### ChunkRangeCalculator (Single Source of Truth) + +**Purpose**: ensure sender and receiver use the exact same chunk-range math + +```typescript +getChunkRange(fileSize, startOffset, chunkSize) { + const startChunk = Math.floor(startOffset / chunkSize); + const endChunk = Math.floor((fileSize - 1) / chunkSize); + return { startChunk, endChunk, totalChunks: endChunk - startChunk + 1 }; +} +``` + +**Key methods**: + +- `getRelativeChunkIndex()`: convert absolute index to relative index for receiver-side array mapping +- `isChunkIndexValid()`: validate that a chunk index is within the expected range +- `calculateExpectedChunks()`: compute expected chunk count, aligned with ReceptionConfig + +### Receiver-Side Resume Flow + +**Partial-file detection**: + +1. **Prepare directories**: `createFolderStructure()` ensures the target directory exists +2. **Lookup file**: `getFileHandle(fileName, {create: false})` checks if a file already exists +3. **Get size**: `file.getFile()` returns the current size as the resume starting point + +**Resume decision logic**: + +```typescript +// FileReceiveOrchestrator.ts +const offset = await this.streamingFileWriter.getPartialFileSize( + fileInfo.name, + fileInfo.fullName +); +if (offset === fileInfo.size) { + // file is already complete; skip transfer + return; +} +if (offset > 0) { + // partial file found; resume + // send fileRequest with offset +} +``` + +### Sender-Side Resume Response + +**Preparation**: + +- **Reset reader**: `StreamingFileReader.reset(startOffset)` starts reading from the new offset +- **Batch alignment**: `currentBatchStartOffset` and `totalFileOffset` are updated in sync +- **Chunk indices**: `startChunkIndex` records the transfer start point for boundary checks + +**Resume log**: + +```typescript +const chunkRange = ChunkRangeCalculator.getChunkRange( + fileSize, + startOffset, + chunkSize +); +postLogToBackend( + `[SEND-SUMMARY] File: ${file.name}, offset: ${startOffset}, startChunk: ${chunkRange.startChunk}, endChunk: ${chunkRange.endChunk}` +); +``` + +### Benefits & Limitations + +**Benefits**: + +- **Saves bandwidth**: avoids re-sending already received bytes +- **Faster recovery**: large transfers can resume quickly after interruption +- **Better UX**: transient network issues don’t reset progress to zero + +**Limitations / caveats**: + +- **File consistency**: assumes file content hasn’t changed; consider validating size/mtime before resuming +- **Save location requirement**: supported when the user chose a save directory via the File System Access API +- **Browser support**: File System Access API is mainly Chrome/Edge; other browsers fall back to in-memory storage + +**Debug support**: + +- **Verbose logs**: record resume offset, chunk range, and expected transfer volume in dev +- **Error handling**: if file access fails, fall back to a full transfer from the start +- **State tracking**: the store records resume state and actual received size + diff --git a/docs/ai-playbook/index.md b/docs/ai-playbook/index.md new file mode 100644 index 0000000..3680183 --- /dev/null +++ b/docs/ai-playbook/index.md @@ -0,0 +1,52 @@ +# PrivyDrop AI Playbook — Context & Index + +This playbook is a high signal-to-noise entry point for AI and developers, helping you jump to the right place in the codebase fast. It contains project context and an index of links, not step-by-step task guides. + +## Project Snapshot + +- Product: WebRTC-based P2P file/text sharing. Data transfers directly between browsers via RTCDataChannel with end-to-end encryption. +- Frontend: Next.js 14 (App Router), React 18, TypeScript, Tailwind, shadcn/ui. +- Backend: Node.js, Express, Socket.IO, Redis; optional STUN/TURN for NAT traversal. +- Privacy stance: The server must never relay file data; the backend is for signaling and room coordination only. + +## Document Index + +- README + + - `README.md` + +- AI Playbook + + - Code map: `docs/ai-playbook/code-map.md` + - Flows (includes micro-plan template): `docs/ai-playbook/flows.md` + - Flows (deep dives, split out): `docs/ai-playbook/flows/frontend.md`, `docs/ai-playbook/flows/backpressure-chunking.md`, `docs/ai-playbook/flows/resume.md`, `docs/ai-playbook/flows/reconnect-consistency.md` + - Collaboration rules: `docs/ai-playbook/collab-rules.md` + +- System & Architecture + + - System architecture: `docs/ARCHITECTURE.md` / `docs/ARCHITECTURE.zh-CN.md` + - Frontend architecture: `docs/FRONTEND_ARCHITECTURE.md` / `docs/FRONTEND_ARCHITECTURE.zh-CN.md` + - Backend architecture: `docs/BACKEND_ARCHITECTURE.md` / `docs/BACKEND_ARCHITECTURE.zh-CN.md` + +- Deployment + - Deployment guide: `docs/DEPLOYMENT.md` / `docs/DEPLOYMENT.zh-CN.md` + - Docker deployment: `docs/DEPLOYMENT_docker.md` / `docs/DEPLOYMENT_docker.zh-CN.md` + +## Key Modules at a Glance + +- Frontend core + - Hooks: `frontend/hooks/useWebRTCConnection.ts` (connection orchestration), `useRoomManager.ts` (room lifecycle), `useFileTransferHandler.ts` (payload orchestration). + - WebRTC base: `frontend/lib/webrtc_base.ts` (Socket.IO signaling, RTCPeerConnection, data channel). + - Roles: `frontend/lib/webrtc_Initiator.ts`, `frontend/lib/webrtc_Recipient.ts` (initiator/recipient behavior). + - Sending: `frontend/lib/transfer/*`, `frontend/lib/fileSender.ts` (metadata, chunking, progress). + - Receiving: `frontend/lib/receive/*`, `frontend/lib/fileReceiver.ts` (assembly, validation, persistence). + - Store: `frontend/stores/fileTransferStore.ts` (single source of truth for progress/state). +- Backend core + - Socket.IO: `backend/src/socket/handlers.ts` (join, initiator-online, recipient-ready, offer/answer/ice-candidate). + - Services: `backend/src/services/{room,redis,rateLimit}.ts`. + - REST: `backend/src/routes/api.ts` (rooms, tracking, debug logs). + +## Maintenance + +- Keep it lean and factual; avoid duplicating system-level docs. +- This playbook exists to support collaboration and quick orientation.