dsh-minecraft-agent

A DeepSeek Harness plugin system that lets AI agents live and act autonomously in Minecraft — not just one bot, but a whole world of "transmigrators" governed by a goddess, with magic that human players can chant too.

The key differentiators

• Zero API cost — agents are driven by a local LLM (e.g. Qwen via llama.cpp), not a cloud API.
• Text as the only interface — an AI interacts with the world exactly like a human player: by talking in chat. No server privileges, no command access.
• World of AIs — a two-process architecture separates the world (server side, one privileged process) from transmigrators (client side, one unprivileged process per AI player). Anyone can point their own agent at the server and join.

让 AI 在《我的世界》里像穿越者一样自主生活：本地模型驱动、零 API 成本；AI 与真人平权——想施法就得在聊天框里念咒，想祷告就得低语。任何人都可以让自己的 Agent 接入同一个世界一起玩。

Why this over Mindcraft?

Mindcraft is the established "AI plays Minecraft" project, but it is a monolithic runtime. This project re-implements the same capability the DeepSeek Harness way — as composable plugins — and goes further: multi-agent worlds, a chat-driven magic system, and an observation deck for spectators.

Architecture

Two kinds of processes, connected only by Minecraft chat:

                    ┌─────────────────────────────────────────────┐
                    │  Minecraft Server (vanilla, RCON enabled)   │
                    └───────▲─────────────────────▲───────────────┘
                            │ RCON (sole holder)  │  chat / whispers
        ┌───────────────────┴──────────┐   ┌──────┴──────────────────────┐
        │  WORLD PROCESS               │   │  TRANSMIGRATOR PROCESS ×N   │
        │  bootstrap-world.mts         │   │  bootstrap-mc.mts           │
        │                              │   │                ┌────────────┼─────────┐
        │  mc-rcon   shared RCON svc   │◄──┼── chat only ───┤ mc-bot     │ mineflayer client
        │  mc-magic  fast-path spells  │   │                 │ mc-tools   │ agent tool layer
        │  mc-god    slow-path oracle  │   │                 │ mc-memory  │ durable memory
        │  mc-ritual birthright rites  │   │                 │ mc-transmigrator (persona)
        │  goddess avatar (spectator)  │   │                 │ mc-mystic  │ chant/pray/choose
        └──────────────────────────────┘   │                 │ mc-loop    │ autonomous loop
                                           │                 │ mc-vision / mc-camera (eyes)│
                                           │                 └────────────┴─────────┘
                                           └──────────────────────────────┘

World process (bootstrap-world.mts) — the server side

Runs once per world, on the machine hosting the MC server. It is the only holder of the RCON password, and serves AI players and human players through the same channel: public chat.

The spell catalog lives in data/magic-atoms.json — data-driven, no code changes needed to add spells, tweak costs, or rename incantations. Costs are split per-resource into base + per-parameter unit_cost; mana regenerates lazily (state file + last_update timestamp); food/hp are read live from MC entities. All visual effects are pure vanilla (particles, sounds, titles) — no mods, no client changes, zero barrier for human players.

Transmigrator process (bootstrap-mc.mts) — the client side

Runs one per AI player (start-bot.bat <username> <viewerPort>). Holds zero server privileges: no RCON, no server commands, no magic-ID tables. Every world interaction is literally speaking — public chat to chant, /msg to pray.

The invariant

The world process is the single privilege holder. Transmigrators interact with the world exactly like human players — by talking. Anyone chanting the same words gets the same magic, AI or human. The server never needs to know who (or what) is logged in.

This is what makes "bring your own agent" possible: the server-side contract is just chat.

Observation panel (web-panel.mjs)

A zero-dependency web deck on :9090 for human spectators (the audience in this world of AIs):

• Per-bot tabs — click a transmigrator to view their first-person stream (iframe into their viewer port).
• Interactive top-down minimap — canvas map from bot memory: current bot (green + heading), other transmigrators (purple), bases (gold), public chests (blue), resource points by type. Click to switch view, scroll to zoom, drag to pan, double-click to re-follow.
• Live mind stream — current goal, thinking, recent chat, and per-step mc_see screenshots from data/status-<username>.json.

Requirements

• DeepSeek Harness (developer preview)
• Node.js 22.19+ / 24+
• A Minecraft server (Java edition, tested on 1.21.11; RCON enabled). Offline mode is fine for bots.
• An LLM endpoint. Any OpenAI-compatible URL works:
  • Local (recommended, free): llama.cpp / Ollama, e.g. http://localhost:8890/v1
  • Cloud: the public DeepSeek API

Quick start

1. Clone this repo next to your DeepSeek Harness checkout (..\node_modules is expected — see start-*.bat).
2. Start the world process on the MC server machine (edit MC_HOST/RCON settings first):

start-world.bat

3. Start one transmigrator per AI player:

start-bot.bat Kirito 3001
start-bot.bat Naruto 3002

4. Open the observation deck at http://localhost:9090.

Environment knobs: MC_HOST, MC_PORT, MC_USERNAME, MC_VIEWER_PORT, MC_GOD_NAME. Runtime state (memory, mana, status snapshots, logs) lives under data/ and is git-ignored.

Local model setup (free)

The default assumption is a local llama.cpp server exposing an OpenAI-compatible API:

llama-server -m Qwen3.8-27B.gguf --host 0.0.0.0 --port 8890 -c 524288

Then point DEEPSEEK_BASE_URL=http://localhost:8890/v1 and use any placeholder DEEPSEEK_API_KEY. No cloud key, no per-token billing.

Supporting a newer MC version in the web viewer

The bundled prismarine-viewer browser assets only know versions up to 1.21.4 out of the box. Two tools in tools/ bring 1.21.11 (or any newer version) to life:

• gen_viewer_assets.py — bakes blocksStates/<v>.json + textures/<v>.png from PrismarineJS/minecraft-assets, faithfully replicating prismarine-viewer's own model/atlas builder.
• patch_viewer_bundle.cjs — injects the new version into the browser bundle's version tables (PC versions list + lazy data table, aliased to the closest known version's data modules).

python tools/gen_viewer_assets.py 1.21.11
node tools/patch_viewer_bundle.cjs

Examples

examples/ contains standalone scripts (no Harness required) to sanity-check your Minecraft server + bot setup. Run them with npx tsx examples/test-mineflayer.mts after setting MC_HOST / MC_PORT / MC_USERNAME.

Roadmap

• [x] Tool hardening: every tool body runs behind try-catch + bot-alive guard
• [x] Core tools: moving, gathering, building, fighting, looting, crafting, equipping, chest storage, trading
• [x] mc-loop: continuous autonomous loop, multimodal decisions with embedded screenshots
• [x] Durable per-bot memory
• [x] Multi-transmigrator infrastructure: per-bot persona registry, status, viewer ports, start scripts
• [x] World/transmigrator process split — world side holds the only RCON; transmigrators are chat-only
• [x] Magic system: 29-atom data-driven spell catalog, three-resource costs, vanilla-only effects
• [x] Goddess slow path: whisper → LLM oracle → structured verdict → RCON miracle
• [x] Birthright ritual for new transmigrators
• [x] Observation deck: multi-bot tabs, interactive minimap, live mind stream
• [x] First-person vision (mc_see) via offscreen WebGL camera
• [ ] Vector-based chant matching (bge-m3) to tolerate typos
• [ ] More tools: useToolOn (block interaction)
• [ ] A demo video

License

MIT — see LICENSE. Example persona files (data/transmigrators/) reference third-party fictional characters for demonstration purposes only.

A project by jcs130. Built on DeepSeek Harness and the mineflayer ecosystem.