dsh-web-speech-input
leozou320-ai
Voice-to-text for the DeepSeek Harness Web UI — live, editable, never auto-sends. | DeepSeek Harness 网页语音输入
PROJECT TOPICS
PROJECT README
English | 中文
An optional desktop companion for deepseek-harness. It shows a small animated pet as an ambient status indicator: the pet relaxes when the harness is idle, "thinks" while the model reasons, "works" while tools run, waits when the harness needs input, and celebrates (or frowns) when a turn finishes.
It is not a second chat UI, a task manager, or a full desktop app. It is a status indicator.
assets/pets/ are discovered at startup, so adding a pet is dropping a folder in — no rebuild.The plugin is a Cordis bundle that ships both a host half (the pet window)
and a client half (the settings card). dsh plugin add installs it and, because
the manifest declares dsh.bundle, adds it to the profile's bundle list
automatically.
dsh plugin --profile <name> add dsh-desktop-pet
Install the plugin directly from its source directory (the profile keeps it as
a link: dependency):
dsh plugin --profile <name> add /path/to/dsh-desktop-pet
On Windows, for example:
dsh plugin --profile web add D:/deepseek-pet
You can also run dsh plugin --profile <name> add . from inside the plugin
directory.
Pack the plugin, then install the tarball:
npm pack
dsh plugin --profile <name> add /path/to/dsh-desktop-pet-0.1.0.tgz
dsh plugin --profile <name> add github:sereinmono/dsh-desktop-pet
If the repository does not commit build artifacts, configure a prepare
script so dsh plugin add builds the plugin during install.
dsh --profile <name>
If you are running Harness from a source checkout, prefix the commands above with
pnpm— i.e. runpnpm dsh plugin ...andpnpm dsh ...from the Harness repository.
The settings card requires Harness to expose the
desktop-petsettings namespace (via itsWEB_SETTINGS_NAMESPACESallowlist). The pet window itself does not depend on that allowlist.
Set enabled: false in the plugin's config, or remove the bundle from the
profile. The plugin then loads but shows nothing; removing it entirely leaves
Harness fully functional.
Pets are plain folders following a fixed sprite-sheet format. Three common ways to obtain one are covered in the dedicated guide, each with a copy-paste prompt and manual steps:
assets/pets/.pet.json + spritesheet.webp into assets/pets/.See Adding a Pet for the full walkthrough, and the
asset format reference for the
exact pet.json and sprite-sheet layout.
| Platform | Status |
|---|---|
| Windows 11 | ✅ primary target (Win32 layered window via koffi) |
| Linux (X11 / XWayland) | ✅ (XCB ARGB overlay; requires a compositor) |
| macOS | ❌ not implemented (backend interface is reserved) |
Linux per-pixel transparency needs a running compositor (GNOME/KDE ship one by default; lightweight WMs need picom or similar). On Wayland the overlay runs through XWayland.
All fields are optional and validated with a Schemastery schema (invalid values fail loudly at load). The user-editable fields (enabled, petScale, petId, hideWhenIdle) are exposed on the Web settings card.
| Field | Default | Description |
|---|---|---|
enabled |
true |
Master switch. |
alwaysOnTop |
true |
Keep the pet above other windows. |
petScale |
1 |
Pet size, 0.5–4× in 0.25 steps. |
petId |
text |
Which pet to display (a directory name under assets/pets/). |
hideWhenIdle |
false |
Automatically hide the pet when it sleeps (no task), and show it again on activity. |
animationEnabled |
true |
Run the frame animation (static frame when false). |
idleFrequencySec |
20 |
Seconds (≥8) between randomized idle variations. |
clickThrough |
false |
Pass pointer events through (Windows only). |
startSleeping |
false |
Start in the sleeping state. |
animationSpeed |
1 |
Global speed multiplier (0.25–4). |
Example:
- insert:
- id: desktop-pet
name: dsh-desktop-pet
config:
petScale: 1
petId: text
idleFrequencySec: 30
Window position is persisted privately under ~/.dsh/desktop-pet/position.json (best-effort; failures are ignored). It does not depend on any Harness storage service.
When the core ctx.commands service is present, the plugin registers a /pet <state> command to simulate states without any LLM:
/pet thinking
/pet working
/pet waiting_for_user
/pet success
/pet error
/pet reset
Valid states: STARTING IDLE THINKING WORKING CODING RUNNING_COMMAND WAITING_FOR_USER SUCCESS ERROR SLEEPING.
harness events / lifecycle
↓ (the only harness-specific layer)
integration/ HarnessBridge · capability-detection · event-mapping
↓ NormalizedEvent
core/ PetStateResolver · PetStateMachine · TaskStateRegistry
↓ SemanticState
renderer/ AnimationController · PetWindow
↓ finished RGBA frames
renderer/backend/ Win32Backend · X11Backend (native overlays via koffi)
↑
renderer/codex-pet/ PetContract · PetLoader (pet.json + sprite sheet)
HarnessBridge is the only module that knows raw harness event names. Everything above it is harness-independent.core/) is a standalone library: testable with no harness, no window, no network.WindowBackend; the renderer never sees Win32 or X11 details.src/client/) is a separate browser bundle registered through the harness module loader; the host and client halves communicate through the settings namespace.Only the Cordis plugin lifecycle and these core services/events are used:
apply(ctx, config) + name / inject / Config.ctx.effect(), ctx.on(), ctx.logger(name).session/event, agent/status.desktop-pet settings namespace (host-side), bound by the client card.ctx.agents, ctx.sessions, ctx.approval, ctx.commands.No non-core plugin is required. If an optional service is absent, the pet degrades gracefully (coarser states, no /pet command).
| Package | Purpose | Runtime |
|---|---|---|
koffi |
Win32 + X11 FFI for the overlay window | Node ≥22 |
sharp |
Decode WebP/PNG sprite sheets to RGBA | Node ≥22 |
@deepseek-ai/schemastery |
Config schema validation | Node ≥22 |
clsx |
Class-name helper for the client card (inlined into the browser bundle) | build |
Peer (type-only, not bundled): @deepseek-ai/cordis.
The client bundle's react and @deepseek-ai/dsh-client-* imports are externalized:
they are provided at runtime by the harness module loader, so the plugin does not
ship them as runtime dependencies (they appear only as dev dependencies for type
checking and bundling).
Explicitly avoided: Electron, Tauri, WebView2/webview, GLFW/SDL/raylib, game engines, GPU/OpenGL, Docker, databases, Redis, any external server, browser automation.
| Normalized event (from harness) | Pet state (semantic → animation) |
|---|---|
| startup | STARTING → waving |
idle (agent/status: idle) |
IDLE → idle |
assistant/chunk (text/reasoning/tool-call delta) |
THINKING → running |
tool/call (editing tools) |
CODING → running |
tool/call (shell/command tools) |
RUNNING_COMMAND → running |
tool/call (other) |
WORKING → running |
approval/asked / waiting |
WAITING_FOR_USER → waiting |
turn/end reason completed |
SUCCESS → review |
turn/end reason error/aborted |
ERROR → failed |
| long quiet period | SLEEPING → idle |
SUCCESS / ERROR / STARTING are transient (default 2s) then return to IDLE. Concurrent agents are tracked per session/task and folded by priority WAITING_FOR_USER > ERROR > WORKING > THINKING > SUCCESS > IDLE.
SemanticState in src/core/types.ts, its resolver mapping in src/core/PetStateResolver.ts, and its renderer pose in SEMANTIC_TO_CODEX.WindowBackend (src/renderer/backend/WindowBackend.ts) and register it in src/renderer/backend/selectBackend.ts.npm test # vitest unit tests (core + loader + integration)
npm run typecheck # tsc --noEmit (host half)
npm run typecheck:client # tsc -p tsconfig.client.json --noEmit (client half)
npm run build # tsdown bundle (host + client)
npm run gen:assets # regenerate the bundled text pet
The pet core is tested without a harness or a display. The native overlay backends require a real desktop session and are not exercised by the headless test suite — they need manual verification on Windows/Linux.
Publishing to npm is automated with GitHub Actions via npm Trusted
Publishing (OIDC): GitHub mints an id token and npm publishes on its behalf,
so the workflow needs no NODE_AUTH_TOKEN secret. A v* tag pushed to
master triggers npm ci → typecheck → test → build → npm publish --provenance.
One-time setup on npm (do this before the first tag push):
0.1.0 once with a manual
token (npm publish --access public) — Trusted Publishing is configured per
package, so an npmjs package page must exist first.sereinmono, repository dsh-desktop-pet. Pin the
workflow publish.yml and branch master if the form offers it.Release locally — bump the version, tag, and push the tag:
npm version patch # or minor / major; creates vX.Y.Z tag
git push origin master --tags
The workflow runs only when the tag's commit is on master; tags on other
branches are skipped. A manual backup entry is available at
Actions → Publish → Run workflow (the tag must still be on master).
text test pet only — original SVG-drawn text, with no OpenAI/Codex/DeepSeek character artwork or trademarks.MIT.
CLASSIFICATION EVIDENCE
系统优先读取 GitHub Topics,再与站内分类词典和词根规则比对。