Adds a microphone button beside the composer. Click to record, click to stop; the audio is resampled to 16 kHz mono PCM16 WAV, posted to the local server and transcribed by a SenseVoice worker process, and the text lands in the draft without being sent. If the draft changed or an IME is composing, the result is held behind an insert button instead of overwriting the user's text.
Server: a small provider registry behind /api/voice/* (catalog, preferences, prepare/cancel/status/remove, transcribe). The engine and model are downloaded at runtime to <config>/cc-haha/voice with pinned sha256/sha512, HuggingFace plus hf-mirror and npm plus npmmirror, HTTP Range resume, automatic retry after interruptions, and a partial file kept across cancels. Recognition runs in a separate worker process (sidecar --voice-worker), started on demand and reclaimed when idle.
Desktop: an independent Voice input settings tab with enable switch, model download progress and resume, language, microphone selection and a transcription test with a live waveform. Uses the shared Dropdown/Card/Button components; adds a danger-ghost Button variant. Preferences live in desktop-ui.json (schemaVersion 6); the microphone device id stays in localStorage.
Electron: main-window media permission handler limited to app pages, main frame and audio only, plus the audio-input entitlement and NSMicrophoneUsageDescription.
Scope: Electron desktop only. Not verified on Windows or Linux, with a real microphone, or in a signed and notarized package.
* feat(desktop): show per-format file icons on chat file cards
Generated-file cards, the turn change card and message attachments used
one grey Material glyph for every file. They now render a folded-corner
document icon with a colored body and the extension label (PDF red,
Word blue, Markdown blue, Excel green, PowerPoint orange, archives
amber, code purple, ...), built on react-file-icon instead of
hand-drawn art. Extensions the library does not know are mapped to a
close sibling or to a category color, and jsonl now counts as code.
Brand colors live in lib/fileTypePalette.ts because react-file-icon
writes them into SVG attributes, where CSS variables are not reliable.
* fix(desktop): open workspace documents written through a symlinked path
Clicking an output card for /tmp/app/report.pdf opened the system app
instead of the workspace preview when the session's canonical workdir is
/private/tmp/app. The gate compared path strings, so a symlinked form,
a workdir that had not loaded yet, or a registered access root all
looked like "outside the workspace".
The string check stays as the fast path. When it says "outside", ask the
server through getWorkspaceFile, which resolves real paths: an accepted
document opens in the workspace, a 403 still goes to the system app.
* fix(desktop): load local images in the web UI and H5
A bare <img src> cannot send Authorization, and the server refuses a
credential-less cross-site subresource load, so the browser blocked the
response (net::ERR_BLOCKED_BY_ORB) and chat showed "unable to load
image". Only the Electron shell worked, because its main process injects
the credential for an allowlist of media routes.
When an <img> fails, retry once through the credentialed client and show
the result as a blob: URL; the failure notice appears only if that also
fails. Covers the inline image gallery, the lightbox, image generation
slots and Markdown images. The credential is only ever sent to the local
server's own origin.
* fix(desktop): keep spreadsheet numbers whole in the workspace preview
Columns without a stored width fell back to a fixed 72px, and stored
widths were chosen for Excel's font, which is narrower than the
preview's. Values such as 12,000.00 were clipped to "12,000...".
A column the file gives no width now fits its widest cell, and a number
column never ends up narrower than its numbers. Text in a column whose
width the file sets is still clipped, as Excel clips it, and merged
headings that spill over their span do not widen a column.
Documents the agent writes open in the workspace panel instead of another
application, and local images the agent mentions show up in the conversation.
Workspace preview
- PDF (pdf.js with its own layout and text layer), Word (docx-preview inside a
scripts-disabled sandboxed iframe) and Excel (SheetJS; .xlsx, .xlsm, .xls) open
in the side panel with zoom and fit, per-file scroll/zoom/sheet memory, and a
refresh when the agent rewrites the file. The engines load lazily.
- Bytes come from a new GET /api/sessions/:id/workspace/raw route, with an
extension allowlist, size caps, the workspace boundary and canonical-path
checks. The file endpoint returns metadata and a version for documents. The
client fetches with the bearer credential, so it works in Electron, LAN H5 and
remote access alike.
- Chat links, output cards and the change card open pdf/docx/xlsx in the
workspace; documents outside the workdir still go to the system application.
- Image viewer with fit, zoom and pan, and "open in system app".
Chat images
- Markdown images outside the workdir, at ~/, C:\ and file:// paths render, open
in the viewer, and offer "open original" (pictures only).
- Images returned by tools such as Read appear as thumbnails under the call.
Hardening found in review
- previewFsUrl escapes each path segment; a double-escaped %2e%2e used to leave
/preview-fs/<session>/.
- The CORS, API timing and remote-access header decorators set headers in place.
Rebuilding the response buffered whole files in memory and dropped
Content-Length.
- The engine owns the pdf.js worker, so closing one document no longer fails the
next open.
- Office archives are inflated in steps to check their real sizes, not the sizes
they declare.
- A viewer that fails to load stays in its panel instead of taking the window down.
Adds pdfjs-dist, docx-preview, xlsx (SheetJS 0.20.3 tarball) and fflate as
renderer dev dependencies; Vite bundles them.
Refs #1397
main is 87 commits ahead and carries a large amount of fixed behaviour this
branch should not be re-deciding. The rule applied throughout: this worktree
owns Computer Use, main owns everything else.
Only 12 files were touched on both sides, and Git merged all of them without
reporting a conflict — but two of those silent merges were wrong, and neither
was visible until the checks ran.
`desktop/src/api/client.ts` ended up with two `apiGetBlob` implementations.
Both sides had independently hit the same problem (an `<img src>` pointed at an
API endpoint is a cross-origin subresource, so it carries no Authorization
header and the server's fetch-metadata policy refuses it) and both had written
the same fix. Git saw two additions in different places and kept both, which
does not even compile. main's version survives: it builds its headers through
the shared `buildHeaders()` rather than assembling them inline, so it inherits
whatever main adds there later.
`src/server/api/computer-use.ts` still imported `runtime/mac_helper.py` and
`runtime/requirements.txt` as compile-time text, both deleted on this branch.
Nothing at runtime referenced them, which is why the deletion looked clean; the
bundler resolves those imports when the server module is loaded, so the failure
surfaced only when the tests actually imported it. That path is now Windows-only
in the same sense the rest of the Python bridge is, and it also ships
`win_cursor_badge.py`, which the badge needs because it runs as its own process.
`computer-use-requirements.test.ts` drops its darwin half for the same reason —
the pins it guards still matter, but only one requirements file is left.
Verified: server 3869 tests / 331 files, desktop 4612 tests / 319 files
(lint + tsc + build), Swift 272 XCTest + 14 Swift Testing, Python 25.
Claude-Session: https://claude.ai/code/session_015j1yxxaoonyAS2iZ7qGnTS
Two things, in one commit because both touch `src/server/api/computer-use.ts`
and the halves cannot be split without rewriting the file twice.
## The regression
`desktop/package.json` had lost `"claude-sidecar-[^/]+$"` from `mac.signIgnore`.
That entry is not a hardening nicety — it is load-bearing for attestation:
1. `build-sidecars.ts` signs the sidecar with an explicit
`--identifier com.claude-code-haha.desktop.sidecar`
2. `ClientAttestation.swift` compares that identifier EXACTLY in
`validDesktopChain`
3. without the exclusion, electron-builder re-signs the sidecar and drops the
flag, so codesign derives the identifier from the file name
(`claude-sidecar-aarch64-apple-darwin`), which never matches
Measured on the shipped 0.5.3 build: host and helper identifiers were correct,
the sidecar's was not. `validDesktopChain` backs BOTH `authorizeOneShot` and
`authorizeDaemon`, so this was not merely a broken permission probe — every
Computer Use call in that build failed closed with `unauthorized_client`.
The existing guard was a literal `toEqual` on the whole signIgnore array, which
does not survive an edit that changes the array and the expectation together —
exactly how the entry was lost. The replacement asserts the behaviour instead:
real sidecar file names must match some exclusion pattern, and the identifier
constants in `sign-identity.ts` and `ClientAttestation.swift` must agree.
`checkCuHelperPermissions` swallowed the failure into nulls, which the settings
page renders as a permanent "checking…" — indistinguishable from a probe still
in flight, and the only symptom this bug ever produced. It now records an
error-level diagnostic: the helper binary is present, so the user did nothing
wrong and the check still did not complete.
## App icons
Rows in the picker and the authorized list showed a letter tile. They now show
the application's own icon, resolved the way Finder does it: `CFBundleIconFile`
from Info.plist, the `.icns` under `Contents/Resources`, rasterised with `sips`.
`openTargetService` already does this, but its resolver is keyed on a
`TargetDefinition` and cannot answer for an arbitrary installed app, so the
path-only half lives in `macAppIcon.ts` and that service is left alone.
The endpoint takes a bundle id and resolves the path itself. There is
deliberately no parameter that names a file — it rasterises and returns bytes,
so its input surface is a security property, and a test drives paths at it.
Enumeration is shared across concurrent lookups: opening the picker fires one
icon request per visible row while the cache is still cold, and a plain
check-then-fill cache would walk every application root once per row.
macOS only, matching where this engine exists. The Windows list renders no icon
slot at all, and Linux is not a supported Computer Use platform.
## Verification
- 208 installed applications on the dev machine: 204 icons resolved; the 4
misses are background bundles (Adobe sync extension, a URL handler, an
updater, a token host) that ship no icon and correctly fall back
- check:server 3345 pass, desktop 4101 pass, check:policy 243 pass, lint clean
(the 2 failures in each suite are `*.golden.test.ts`, pre-existing on main)
- mutation-checked that the new guards actually fail: removing the signIgnore
entry, dropping the icon `onError` fallback, and breaking the in-flight share
each turn a test red
Rebuilt against main so the branch carries the Computer Use work and no other
divergence. Three unrelated efforts had been sitting uncommitted in this
worktree and were swept into an earlier commit; they are preserved on
cu-worktree-full-backup and belong on their own branches — adapter control
credentials, Electron asar sealing, and the sidecar code-loading audit. Every
file outside Computer Use now matches main exactly.
The engine
A Swift helper drives apps through the accessibility tree, with coordinate
actuation for the Chromium and Electron apps whose tree is a bare window
frame. Ten primitives matching the shape Codex uses, so an app's guidance and
the model's habits transfer.
Coordinate actions resolve their target window once and refuse when none can
be named. The unbound event they used to fall back to is discarded by custom
renderers, so a minimized target produced a whole session of "Action
completed" with nothing behind it.
Input acceptance is established for typing and key presses as well as clicks:
each MCP call is seconds apart, so the keyboard cannot inherit the focus a
click established. The synthetic focus notification is gated on the target
not already being active — sent unconditionally it names window 0 at an app
that already owns a key window, and nine window-bound clicks were discarded
with the traffic lights fully lit.
State the model can trust
An off-screen target says so, and says which tools still reach it: element
actions need no on-screen geometry, so an app with a real tree can still be
driven from the Dock. A fully covered window is recovered once, then left
alone — burying it again is the user wanting their screen back. A repeated
capture is reported with the cause that actually applies rather than both,
because coverage is something we compute.
Signing
The helper is signed under a stable identity before electron-builder sees it,
and excluded from re-signing: macOS ties Accessibility and Screen Recording
grants to the signing identity, so rotating it drops both on every update.
Discoverability
The desktop slash menu falls back to a directory scan while a session's CLI
has not started, which is when the menu is first opened. Built-ins and
bundled skills live in the binary, so /computer-use was absent until after
the first message.
Two real conflicts, one of them structural.
desktop/src/pages/Settings.tsx — git offered the whole 4128-line pre-split file
as "theirs" against the 183-line shell, which is not a merge anyone can review.
Resolved by keeping the split and porting main's nine hunks to where that code
now lives: the rail width, its comment and TabButton's padding stay in
Settings.tsx; the ModelIdCombobox import and the five ProviderFormModal changes
(the canFetchModels split into hasModelsBaseUrl/hasModelsApiKey, modelPickerItems
becoming modelPickerGroups, the two new hint branches, and the Input+Dropdown pair
collapsing into ModelIdCombobox) go to settings/ProviderSettings.tsx.
Verified rather than assumed: every line main added is present somewhere in the
split, every construct it removed is gone (modelPickerItems, canFetchModels =
Boolean(...), the supplementary Dropdown), and the import specifier gained the
level the new directory needs.
desktop/package.json — taking main's version wholesale dropped the eslint setup
from f36c9cb49. Reconstructed with both sides: main's six prosemirror packages and
the three eslint devDependencies, with lint back to eslint + tsc.
Everything else merged clean, including the files both sides touched:
src/server/ws/handler.ts (main's four title-generation changes all sit in code the
three splits left behind), desktop/src/stores/chatStore.ts (main's mention/
repository-launch state alongside the turn-scoped replay guard), and the five
locales — 2526 - 9 removed + 5 added = 2522, still aligned across all languages.
Checks: desktop lint + 4049 tests + build, check:electron, check:policy all green.
The one server failure, workspace-service.test.ts "rejects a file outside the
workdir", fails identically on main — confirmed against a temporary worktree at
main, which fails it plus three more. It passes 3/3 in isolation; the registry it
asserts on is a module-level Set shared across test files.
`bun run lint` was `tsc --noEmit` alone. tsc types the code but knows nothing
about React's rules, which is how a conditional useMemo sat in
AskUserQuestion.tsx long enough to be a latent crash — the fix in cf5200922.
One rule, react-hooks/rules-of-hooks, as an error. It is the one React rule with
no judgement in it: a conditional hook is always a bug, never a preference, so
it needs no suppressions and the tree is already at zero.
exhaustive-deps stays off. It has 27 hits here, most deliberate, and a
permanently-yellow lint is one nobody reads. The 13 existing eslint-disable
comments name rules this config does not enable, so unused-directive reporting
is off too — they are dormant records of intent, not mistakes.
The composer was a plain textarea, so @-selected files and folders could
not render inline: they became attachment chips above the input and were
sent as a detached @"path" prefix. The composer now runs on ProseMirror
(ChatGPT desktop's architecture): @ selections insert an inline mention
pill (an atom node carrying label/path/isDirectory), and the pill is
serialized back to the @"absolute path" text the CLI already parses at
its exact position in the prompt.
- MentionComposer + composerEditor: schema, doc<->projection mapping,
controlled bridge (doc changes project to text+mentions; external
writes rebuild the editor state, resetting undo history), single-
backspace whole-pill deletion, placeholder/disabled/aria plumbing
- composerMentions: tokenOrdinal keeps pills distinct from literal text
that happens to spell the same token, so serialization and rebuilds
never rewrite the wrong occurrence
- ChatInput and EmptySession composers both migrated; drafts persist
mentions; drag-and-drop attachment chips unchanged
The shadcn migration (ad597ffe) and its follow-up (5b4e224f) were
mega-commits: only ~16% of the insertions were UI work. Rolling either
one back wholesale also deletes ~3400 lines of unrelated server,
Electron, and store hardening -- and because each of those changes
shipped with its own tests, the suite stays green while the hardening
silently disappears.
This rolls back the UI layer only.
Reverted (back to c2774fc1):
- desktop/src/components, pages, features, theme, i18n
- components/ui/**, components.json, src/lib/utils.ts (cn helper)
- radix-ui, class-variance-authority, tailwind-merge deps
- src-tauri/src/lib.rs hardening: dead code, nothing compiles it
(no @tauri-apps dep, no cargo invocation anywhere, and
scripts/pr/release-workflow.test.ts asserts Tauri's absence)
Kept (identical to main):
- src/** -- WhatsApp authDir escape allowing arbitrary recursive
directory removal, computer-use tier bypass granting every
pre-authorized app full tier, plugin uninstall keepData inversion
that deleted data when asked to keep it, plugin project-scope
writing to the sidecar cwd, diagnostics share leaking provider
config, memory API TOCTOU plus atomic writes
- desktop/electron/** -- PTY leak on renderer destruction, IPC
boundary validation rejecting NaN and unbounded input
- desktop/src/stores/** -- request-id race guards
- desktop/src/api, lib/desktopHost -- kept in step with the stores
Follows main's b64069a3 in dropping the installed-skills overview.
Updated two memorySettings assertions to expect the optimistic-lock
fields the retained memory store sends.
check:desktop pass (lint clean, 225 files / 2519 passed / 1 skipped,
build ok). check:server pass (224 files / 2375 passed / 0 failed).
Avoid Bun filter-mode repository scans that exhaust macOS file descriptors and corrupt subprocess test evidence. Apply rooted filters across server, contract, coverage, persistence, policy, desktop native, and adapter test entrypoints.
Confidence: high
Scope-risk: narrow
Tested: bun run check:policy; bun run check:server; bun run check:chat-contract
Prepare the v0.4.6 desktop release note, bump the desktop package version, and refresh README/docs guidance for signed releases and updater validation.
Tested: bun run scripts/release.ts 0.4.6 --dry
Tested: bun test scripts/pr/release-workflow.test.ts scripts/release-update-metadata.test.ts scripts/quality-gate/package-smoke/index.test.ts
Tested: bun run check:policy
Tested: bun run check:docs
Not-tested: bun run verify; release prep was validated with docs and release-focused gates only.
Confidence: high
Scope-risk: narrow
Tested: bun run scripts/release.ts 0.4.5 --dry
Tested: bun run verify (quality report artifacts/quality-runs/2026-07-02T18-46-23-146Z/report.md, passed=9 failed=0 skipped=1)
Tested: bun run quality:gate --mode release --allow-live --provider-model codingplan:main:codingplan-main (report artifacts/quality-runs/2026-07-02T18-54-51-367Z/report.md, passed=18 failed=1 skipped=0; live provider lanes passed)
Not-tested: local macOS Gatekeeper package-smoke did not pass because the local canonical macOS artifact directory contained stale v0.4.4 artifacts and the current shell has no Apple notarization credentials to produce a stapled local build; the tag-triggered GitHub release workflow remains the source of truth for signed/notarized assets.
Confidence: medium
Scope-risk: moderate
Tested: bun run scripts/release.ts 0.4.4 --dry
Tested: bun run verify (artifacts/quality-runs/2026-07-01T12-34-24-275Z/report.md)
Confidence: high
Scope-risk: broad
Tested: bun test scripts/pr/release-workflow.test.ts
Tested: git diff --check
Tested: bun run scripts/release.ts 0.4.3 --dry
Tested: env DEBUG=electron-builder,electron-osx-sign node ./node_modules/electron-builder/out/cli/cli.js --mac zip --arm64 --publish never -c.mac.notarize=false
Tested: bun run test:package-smoke --platform macos --package-kind release --artifacts-dir desktop/build-artifacts/electron
Confidence: high
Scope-risk: narrow
Tested: bun run scripts/release.ts 0.4.3 --dry
Tested: git diff --check
Tested: bun run verify (artifacts/quality-runs/2026-06-16T16-48-12-824Z/report.md; passed=8 failed=0 skipped=2)
Confidence: high
Scope-risk: narrow
程序员阿江(Relakkes)