deepseek-eyes

Give a text-only model in DeepSeek Harness sight — then just paste the image.

Pasted images are normally refused the moment the selected model is text-only, before any tool or plugin can react. deepseek-eyes adds a DeepSeek（视觉桥） entry to the model picker: select it, keep your usual text model underneath, and paste or drag images exactly like a native vision model. Set it up once in the Settings UI — no YAML, no new provider, no extra model download — and it just works from then on.

Community plugin — not affiliated with DeepSeek. This is a community-maintained, third-party plugin. It is not developed, sponsored, or endorsed by DeepSeek (深度求索), and it is not part of the official @deepseek-ai npm scope.

中文说明见 README.zh.md。

Quick start

Three steps, mostly point-and-click:

1. Install

git clone https://github.com/<you>/deepseek-eyes.git
dsh plugin --profile web add "file:$PWD/deepseek-eyes"

Restart the running Web profile and refresh the page. On Windows PowerShell, pass the absolute checkout path directly to dsh plugin — keep the file: prefix, it matters (see Install).

2. Configure once, in the UI. Open Settings → DeepSeek Eyes and fill in just two things:

• Vision endpoint — base URL + model id of any OpenAI-compatible vision API (OpenRouter, DashScope, a self-hosted VL model, …).
• API key — paste it into the API key field and press Save key. It is stored through the DSH credentials seam and never sent back to the browser.

Press Test connection to verify — settings save and apply live, no restart needed.

3. Select the bridge and paste. In the model picker choose DeepSeek（视觉桥） with your usual text model id (e.g. deepseek-v4-pro), paste or drag an image into the composer, ask a question in the same message, and send.

That's it. Text-only requests pass through untouched; only messages containing images are rewritten. Stuck? See Troubleshooting.

The only two things you bring: a vision endpoint and its key. Everything else — provider wiring, the Settings page, credential storage, image admission — is taken care of by the plugin.

See it work

Configure the vision endpoint once on the Settings page — fill it in, test the connection, save the key:

Then pick DeepSeek（视觉桥） in the model selector and paste an image as usual:

And here is the bridge in action: paste an image straight into the chat — DeepSeek-eyes describes it and hands the description to the text-only model, which then reads and analyzes it:

How it works

flowchart LR
    A[Paste / drag an image<br/>bridge provider selected] --> B[DSH prompt admission<br/>inputModalities: text + image]
    B --> C[Bridge adapter stream]
    C -->|has images| D[Focus hint from user text]
    D --> E[Vision API<br/>OpenAI-compatible]
    E --> F[Image block → text description]
    F --> G[Forward text-only request]
    C -->|no images| G
    G --> H[Target provider<br/>e.g. deepseek-official]

• The bridge provider declares inputModalities: ['text', 'image'], so the host admits image content instead of answering Model "..." does not support image input. — the rejection shown in the Web UI as an attachment-error.
• In stream(), the adapter walks the request messages (including tool-result content), extracts a focus hint from the message's own text or the nearest preceding user text, and asks the vision API: "the agent is looking at this image because: ". The returned description replaces the image block, framed as evidence (text inside images is data, never instructions).
• The rewritten request is delegated to ctx.llm.stream under the target provider route, so the real adapter's serialization, streaming, retries, and telemetry stay intact. Text-only requests forward with untouched messages.
• The image description is cached in memory, keyed by attachment + vision model + language + hint.

Install

git clone https://github.com/<you>/deepseek-eyes.git
dsh plugin --profile web add "file:$PWD/deepseek-eyes"
# headless profiles too, if you run them:
dsh plugin --profile headless add "file:$PWD/deepseek-eyes"

# store the vision API key through the credentials seam:
dsh credentials set VISION_API_KEY

Restart the running Web profile, refresh the page, and configure the vision endpoint (see above). On Windows PowerShell, dsh plugin accepts the absolute checkout path directly.

Use the file: prefix. A bare directory path makes pnpm install a link: dependency — a symlink from the profile to your checkout. Released DSH runtimes load plugins with plain Node ESM, which resolves that symlink to its real location outside the profile tree, so the plugin's peer imports (@deepseek-ai/dsh-settings, @deepseek-ai/dsh-llm, …) can no longer find the profile's module fallback, and the harness fails to start with Cannot find package '@deepseek-ai/dsh-settings'. The file: prefix makes pnpm copy the checkout into the profile (a real directory — resolution stays inside the profile), which works on every runtime. The bare-path form only appears to work under the source-checkout dev harness (tsx). Tarball installs are equivalent: pnpm pack, then dsh plugin add ./deepseek-eyes-0.1.0.tgz.

Because file: installs a copy, rebuild (pnpm run build) and re-run dsh plugin add "file:…" after changing src/.

Configure

Preferred: configure in the Web UI. After restarting, open Settings → DeepSeek Eyes: a full form (vision endpoint, model, credential reference with its configured state, output language, timeout and image limits, description cache) that saves and applies live, plus a Test connection button (GET /models against the configured endpoint — no image, no completion). The same page has an API key field: paste the vision key there and press Save key — it stores through the DSH credentials seam under the configured reference and is never sent back to the browser (a Clear stored key button removes it). The Models settings page also gains a DeepSeek Eyes row showing the bridge route's model list and credential state.

The UI edits the same deepseek-eyes settings section as hand-written YAML — override in your profile patch row with the same id (e.g. %DSH_HOME%\profiles\web\cordis.patch.yml):

- id: deepseek-eyes
  config:
    provider: deepseek-vision          # bridge route shown in the model picker
    displayName: 'DeepSeek（视觉桥）'
    targetProvider: deepseek-official  # the real text-only provider route
    apiKeyEnv: VISION_API_KEY          # credential reference, not the key value
    vision:
      baseUrl: https://your-vision-provider.example.com/v1   # any OpenAI-compatible endpoint
      model: your-vision-model
      language: zh                      # zh | en
      timeoutMs: 60000
      maxTokens: 2048
      maxImageBytes: 10485760
      maxImagePixels: 40000000
      cacheSize: 16

The API key never lives in the patch: paste it into the API key field on the DeepSeek Eyes page, or store it with dsh credentials set VISION_API_KEY, or export it in the launching environment. Settings-page saves apply live; a changed provider re-registers the route in place.

Use

1. In the model picker, select DeepSeek（视觉桥） (or your displayName) with your usual text model id (e.g. deepseek-v4-pro).
2. Paste or drag an image into the composer, optionally with a question ("这个按钮为什么是灰的？").
3. Send. The text model receives the vision description framed as evidence and answers as if it had seen the image.

Tip: put the question in the same message as the image — it becomes the focus hint and materially improves the description.

Requirements & dependencies

deepseek-eyes is a thin layer: it ships no model and no provider of its own. Everything it needs falls into four groups.

1. Provided by the consuming DSH profile (peer dependencies — already installed in any standard profile):

2. Local environment:

3. External services you must bring:

4. Development only (not needed at runtime): TypeScript, Vitest, @types/node, React types — plus the deepseek-harness checkout as a sibling (its built lib/ type declarations pin the typecheck to the exact harness API line; see Development).

Comparison with dsh-vision-toolkit

deepseek-eyes and dsh-vision-toolkit solve different halves of the problem and can run side by side:

The bridge gives the model a description; the toolkit gives the agent pixel-precise tools. For engineering-grade vision work, install both and use the toolkit's tools for measurements.

Error codes

Failures surface as terminal LLM errors with stable codes:

Troubleshooting

Limitations

• The text model receives a description, not pixels: fine-grained geometry, exact colors, and pixel-level layout are out of scope (use dsh-vision-toolkit alongside).
• The delegated call is intentionally unmarked, so sessions on the bridge route lose adapter replay state for cached-response replay (correctness is unaffected; replayed responses fall back to a fresh provider call).
• Only PNG / JPEG / WebP / GIF image parts admitted by the DSH attachment path are handled; the vision endpoint must accept the chosen formats.
• Descriptions are generated per image block, sequentially, inside the request path: budget vision.timeoutMs accordingly.
• The in-memory cache is process-local and clears on restart.

Security

• Image content is treated as untrusted data: the vision prompt is instructed — and the injected description is framed — so that text inside images is never executed as instructions.
• Keys are resolved through the DSH credentials seam per call; they never appear in configuration, logs, or error messages. The Settings page's API key field writes one way into the credentials seam (like the first-party Models page); stored values are never sent back to the browser.
• Upstream error bodies are bounded to 300 characters before they reach a message.
• Sizes are fenced (maxImageBytes, maxImagePixels) before any upload.

Development

pnpm install      # devDependencies only; harness peers resolve from the consuming profile
pnpm typecheck    # tsc over src + tests (host side)
pnpm build        # emits lib/ (host) and lib/client.js (browser Settings card)
pnpm test         # vitest unit tests (rewrite, vision client, config)

• lib/ is committed on purpose: dsh plugin add "file:<path>" installs the checkout and loads main: lib/index.js; the Web frontend discovers the browser bundle through the dsh.client declaration and exports["./client"].
• Typechecking pins to the harness API line: the tsconfig files map @deepseek-ai/dsh-* type imports to a sibling deepseek-harness checkout's built lib/ declarations (registry-published versions predate parts of that API). Clone github.com/deepseek-ai/deepseek-harness next to this repo and build it once; the same layout is what CI assumes.
• Run pnpm run build after any src/ change before installing into a profile; file: installs copy the checkout, so re-run dsh plugin add "file:…" to pick the rebuilt lib/ up.
• CI runs install and tests; typecheck/build additionally run where the harness sibling is present — see .github/workflows/ci.yml.

License

MIT — see LICENSE.

Acknowledgements

The focus-hint idea (send why the model is looking at the image to the vision model instead of asking for a generic description) comes from Anionex/agent-vision-toolkit. This plugin implements that idea natively inside the DeepSeek Harness LLM adapter layer.