Add mcp-loader extension: generic MCP server registration via settings.json

Reads an `mcp` block from ~/.pi/agent/settings.json (same shape as
opencode and Claude Desktop) and connects to each declared MCP server,
exposing all of their tools to pi as native tools namespaced as
<server-name>_<tool-name>.

Why: pi has no built-in MCP loader. Adding each new MCP server as a
hand-rolled extension (the way mempalace.ts does it) doesn't scale.
This is the config-driven generalization — one extension, any number
of servers, no per-server boilerplate.

Settings.json schema matches opencode and Claude Desktop verbatim:

  {
    "mcp": {
      "searxng": {
        "type": "local",
        "command": ["uvx", "mcp-searxng"],
        "env": { "SEARXNG_URL": "https://searxng.your-host.lan" }
      },
      "context7": {
        "type": "remote",
        "url": "https://mcp.context7.com/mcp"
      }
    }
  }

Per-server keys: type (local/remote), command, url, enabled, env.

Implementation:
  • StdioMcpClient class spawns subprocess, performs MCP initialize
    handshake (protocol 2024-11-05), lists tools, exposes a callTool()
    method. Newline-delimited JSON-RPC over stdio.
  • Each MCP tool registered via pi.registerTool with the server-
    namespaced name, the upstream MCP inputSchema passed through
    via Type.Unsafe (TypeBox is JSON-Schema-compatible at runtime).
  • Per-server fail-soft: a server that won't start logs one stderr
    line and is skipped; others continue.
  • SIGTERM all subprocesses on session_shutdown so /reload doesn't
    leak processes.

Tool naming: prefix with <serverName>_ except when the upstream tool
name already starts with that prefix (mempalace's tools are already
mempalace_search, mempalace_kg_query, etc — avoids double-prefixing).

Coexists with mempalace.ts but does not replace it. The mempalace
bridge has bespoke agent-identity injection that's worth preserving.

v1 limitations:
  • Stdio transport only. Remote (streamable-HTTP) servers are
    detected and skipped with a warning. v2 will add streamable-HTTP.
  • No reconnect on subprocess death — same limitation as mempalace.ts.

Verification:
  • node --check syntax clean
  • Standalone smoke test against `uvx mcp-server-time`: handshake +
    tools/list (2 tools) + tools/call (get_current_time) all green
    on the same JSON-RPC code that lives inside the loader.

Debug: set PI_MCP_LOADER_DEBUG=1 to surface per-server stderr.
This commit is contained in:
2026-05-08 20:02:21 +02:00
parent ba994014a7
commit 141bf64d81
3 changed files with 470 additions and 0 deletions
+61
View File
@@ -23,6 +23,7 @@ extensions/
notify.ts # Native terminal notification when agent finishes
ext-toggle.ts # /ext slash command — list & toggle extensions at runtime
todo.ts # `todo` tool for the agent + /todos for the user (copy of upstream example)
mcp-loader.ts # Generic MCP server loader — reads `mcp` block from settings.json
install.sh # Idempotent installer — symlinks extensions/ into ~/.pi/agent/extensions/
package.json # pi package manifest — enables `pi install /path` as an alternative
README.md # User-facing docs.
@@ -246,6 +247,66 @@ cp /opt/homebrew/Cellar/pi-coding-agent/*/libexec/lib/node_modules/@mariozechner
# review diff, then commit
```
### `mcp-loader.ts`
Generic MCP stdio client — reads an `mcp` block from `~/.pi/agent/settings.json`
and connects to each declared server, exposing the tools as namespaced pi tools.
**Why this exists:** pi has no built-in MCP loader (unlike opencode and Claude
Desktop). Adding each new MCP server as a hand-rolled extension (the way
`mempalace.ts` does it) doesn't scale past 2-3 servers. This loader is the
config-driven generalization — one extension, any number of servers.
**Key design decisions:**
- **Settings.json shape matches opencode / Claude Desktop verbatim.** A user
can copy-paste their `mcp` block from one harness's config to the other.
- **Tool names are prefixed with the server name** to avoid collisions when
multiple servers expose tools with the same short name. Tools that already
start with `<servername>_` (the convention some servers like mempalace
follow internally) skip the prefix to avoid double-prefixing.
- **Fail-soft per server.** A server that won't start (binary missing, init
handshake fails) logs one stderr line and is skipped. Other servers
continue. Pi keeps working.
- **Stdio transport only in v1.** Remote/streamable-HTTP servers are
detected and skipped with a warning. Adding remote support is the
obvious v2 (context7 is the prime motivator). Avoided in v1 because
streamable-HTTP needs SSE parsing, session management, and reconnect
logic that triples the implementation size.
- **Does not replace `mempalace.ts`.** The mempalace bridge has bespoke
agent-identity injection from `$MEMPALACE_AGENT_NAME` that's worth
preserving. Loader and bridge coexist. Listing `mempalace` in the `mcp`
block would produce duplicate tool registrations — don't.
- **No reconnect on subprocess death.** If a server's subprocess crashes
mid-session, its tools become permanently unavailable until pi `/reload`s.
Same limitation as the mempalace bridge today; not worth complicating v1.
**API used:**
- `pi.registerTool({ name, label, description, parameters, execute })` for
each MCP tool exposed by each connected server.
- `pi.on("session_shutdown", ...)` to SIGTERM all subprocesses cleanly.
- `Type.Unsafe<...>(inputSchema)` from typebox to pass the MCP server's
JSON schema through to pi without conversion (TypeBox schemas are
plain JSON Schema at runtime).
**Internal MCP client:**
- `class StdioMcpClient` — spawn subprocess, write newline-delimited
JSON-RPC, parse newline-delimited responses, match by request `id`.
- Sends `initialize` handshake with `protocolVersion: 2024-11-05`, then
`notifications/initialized`, then `tools/list` to discover tools.
- Stderr drained silently unless `PI_MCP_LOADER_DEBUG=1`.
**Future v2 extensions:**
- Streamable-HTTP transport for remote servers (context7).
- Per-tool enable/disable in settings.json (e.g.
`"mcp.searxng.tools": ["web_search"]` to expose only a subset).
- Reconnect-on-crash with exponential backoff.
- Schema sanitization for MCP servers that emit malformed `inputSchema`
(some return `{type: "object", properties: {...}}` without `required`).
No framework. Manual:
```bash
+47
View File
@@ -203,6 +203,53 @@ State lives in the session's tool result `details`, not an external file. So:
This is a verbatim copy of the upstream `examples/extensions/todo.ts` shipped with `pi-coding-agent`. Refresh from upstream when desired (see `AGENTS.md`).
### `mcp-loader.ts`
Generic MCP server loader. Reads an `mcp` block from `~/.pi/agent/settings.json` (same shape as opencode and Claude Desktop) and connects to each declared server, exposing all of their tools to pi as native tools — namespaced as `<server-name>_<tool-name>` to avoid collisions.
**Settings.json shape:**
```jsonc
{
// … existing pi settings …
"mcp": {
"searxng": {
"type": "local",
"command": ["uvx", "mcp-searxng"],
"env": { "SEARXNG_URL": "https://searxng.your-host.lan" }
},
"gitea": {
"type": "local",
"command": ["gitea-mcp", "-t", "stdio"],
"enabled": false,
"env": { "GITEA_ACCESS_TOKEN": "...", "GITEA_HOST": "https://gitea.example.com" }
},
"context7": {
"type": "remote",
"url": "https://mcp.context7.com/mcp"
}
}
}
```
**Per-server keys:**
| Key | Description |
|---|---|
| `type` | `"local"` (stdio subprocess) or `"remote"` (streamable-http). Default `"local"`. |
| `command` | Argv array. First element is the executable, rest are args. Local servers only. |
| `url` | Remote MCP endpoint URL. Remote servers only. |
| `enabled` | Default `true`. Set `false` to disable a server without removing the entry. |
| `env` | Optional object of env vars injected into the subprocess. Inherits parent env first, then overlays these keys. Local servers only. |
**Limitations (v1):**
- **Stdio only.** Remote/streamable-HTTP transport is detected and skipped with a warning. Server like `context7` configured as `"type": "remote"` will not load until v2.
- **No reconnect** if a subprocess dies mid-session — those tools become unavailable until `/reload` (same as `mempalace.ts`).
- **Coexists with `mempalace.ts`** but does not replace it. The mempalace bridge has bespoke handling (agent identity injection) that's worth keeping. Don't list `mempalace` in the `mcp` block too — you'd get duplicate tool registrations.
**Debug:** set `PI_MCP_LOADER_DEBUG=1` in the environment to surface per-server stderr and connection logs.
Each extension is a TypeScript module loaded by [jiti](https://github.com/unjs/jiti) — no compilation step. See the [pi extensions docs](https://github.com/mariozechner/pi-coding-agent/blob/main/docs/extensions.md) and the [built-in examples](https://github.com/mariozechner/pi-coding-agent/tree/main/examples/extensions) for the API surface.
---
+362
View File
@@ -0,0 +1,362 @@
/**
* MCP Loader Extension
*
* Reads an `mcp` block from pi's settings.json (same shape as opencode and
* Claude Desktop) and connects to each declared server, exposing all of
* their tools to pi as native tools.
*
* Settings.json shape:
*
* {
* "mcp": {
* "searxng": {
* "type": "local",
* "command": ["uvx", "mcp-searxng"],
* "env": { "SEARXNG_URL": "https://searxng.your-host.lan" }
* },
* "gitea": {
* "type": "local",
* "command": ["gitea-mcp", "-t", "stdio"],
* "enabled": false,
* "env": { "GITEA_ACCESS_TOKEN": "...", "GITEA_HOST": "..." }
* },
* "context7": {
* "type": "remote",
* "url": "https://mcp.context7.com/mcp"
* }
* }
* }
*
* Per-server keys:
* type "local" (stdio subprocess) | "remote" (streamable-http)
* command argv array — first element is the executable, rest are args.
* For local servers only.
* url remote MCP endpoint URL. For remote servers only.
* enabled default true. Set false to disable a server without
* removing it from settings.json.
* env optional object of environment variables to inject into
* the spawned subprocess. Inherits parent env first, then
* overlays these keys.
*
* Tool naming
*
* Each MCP tool is registered with pi as `<server-name>_<tool-name>` to
* avoid collisions across servers. If the tool name already begins with
* `<server-name>_` (e.g. mempalace's tools are `mempalace_search` etc.)
* the prefix is not duplicated.
*
* Lifecycle
*
* • Servers are spawned at extension load time (pi startup or /reload).
* • Each server's `tools/list` is awaited before pi finishes registering.
* • Subprocess stderr is silenced unless PI_MCP_LOADER_DEBUG=1.
* • Subprocesses receive SIGTERM at session_shutdown.
* • A server that fails to start (binary missing, init handshake error)
* logs a single line to stderr and is skipped. Other servers continue.
* Pi keeps working without that server's tools.
*
* Limitations (v1)
*
* • Only local/stdio transport is implemented. Remote (streamable-http,
* SSE) servers are detected and skipped with a warning. v2 will add
* remote support — context7 is the prime motivator.
* • No reconnect: if a subprocess dies mid-session, its tools become
* unavailable until pi is reloaded. The mempalace.ts extension has
* the same limitation today.
*
* Coexistence with mempalace.ts
*
* The mempalace.ts extension already spawns mempalace-mcp directly with
* bespoke handling (agent identity injection from $MEMPALACE_AGENT_NAME,
* wake-up context). This loader does NOT touch mempalace — leave the
* bespoke extension in place. If you also list mempalace under the `mcp`
* block in settings.json the loader will register a parallel set of
* tools, which is harmless but redundant. Don't.
*/
import { type ChildProcessWithoutNullStreams, spawn } from "node:child_process";
import * as fs from "node:fs";
import * as os from "node:os";
import * as path from "node:path";
import type { ExtensionAPI } from "@mariozechner/pi-coding-agent";
import { Type } from "typebox";
// ── MCP types ────────────────────────────────────────────────────────────────
type McpTool = {
name: string;
description?: string;
inputSchema?: Record<string, unknown>;
};
type LocalServerConfig = {
type?: "local";
command: string[];
env?: Record<string, string>;
enabled?: boolean;
};
type RemoteServerConfig = {
type: "remote";
url: string;
enabled?: boolean;
};
type ServerConfig = LocalServerConfig | RemoteServerConfig;
const SETTINGS_PATH = path.join(os.homedir(), ".pi", "agent", "settings.json");
const DEBUG = process.env.PI_MCP_LOADER_DEBUG === "1";
function dlog(msg: string) {
if (DEBUG) process.stderr.write(`[mcp-loader] ${msg}\n`);
}
function warn(msg: string) {
process.stderr.write(`[mcp-loader] ${msg}\n`);
}
// ── Settings.json reader ─────────────────────────────────────────────────────
function readMcpServers(): Record<string, ServerConfig> {
if (!fs.existsSync(SETTINGS_PATH)) {
dlog(`no settings.json at ${SETTINGS_PATH}`);
return {};
}
let raw: string;
try {
raw = fs.readFileSync(SETTINGS_PATH, "utf8");
} catch (err) {
warn(`could not read ${SETTINGS_PATH}: ${(err as Error).message}`);
return {};
}
let parsed: any;
try {
parsed = JSON.parse(raw);
} catch (err) {
warn(`settings.json parse error: ${(err as Error).message}`);
return {};
}
const block = parsed?.mcp;
if (!block || typeof block !== "object") {
dlog("no `mcp` block in settings.json");
return {};
}
return block as Record<string, ServerConfig>;
}
// ── Stdio MCP client ─────────────────────────────────────────────────────────
class StdioMcpClient {
private proc: ChildProcessWithoutNullStreams | null = null;
private nextId = 1;
private pending = new Map<number, { resolve: (v: any) => void; reject: (e: Error) => void }>();
private stdoutBuf = "";
private serverName: string;
public tools: McpTool[] = [];
constructor(serverName: string) {
this.serverName = serverName;
}
async start(command: string, args: string[], env?: Record<string, string>): Promise<void> {
const childEnv = { ...process.env, ...(env ?? {}) };
this.proc = spawn(command, args, { stdio: ["pipe", "pipe", "pipe"], env: childEnv });
this.proc.on("error", (err) => this.failAll(err));
this.proc.on("exit", (code, signal) => {
this.failAll(new Error(`${this.serverName}: subprocess exited (code=${code}, signal=${signal})`));
});
this.proc.stdout.setEncoding("utf8");
this.proc.stdout.on("data", (chunk: string) => this.onStdout(chunk));
this.proc.stderr.setEncoding("utf8");
if (DEBUG) {
this.proc.stderr.on("data", (chunk: string) => {
process.stderr.write(`[${this.serverName} stderr] ${chunk}`);
});
} else {
this.proc.stderr.resume(); // drain without logging
}
// MCP initialize handshake
await this.request("initialize", {
protocolVersion: "2024-11-05",
capabilities: {},
clientInfo: { name: "pi-mcp-loader", version: "0.1.0" },
});
this.notify("notifications/initialized", {});
const listed = await this.request("tools/list", {});
this.tools = (listed?.tools as McpTool[]) ?? [];
}
private onStdout(chunk: string) {
this.stdoutBuf += chunk;
let nl: number;
while ((nl = this.stdoutBuf.indexOf("\n")) !== -1) {
const line = this.stdoutBuf.slice(0, nl).trim();
this.stdoutBuf = this.stdoutBuf.slice(nl + 1);
if (!line) continue;
let msg: any;
try {
msg = JSON.parse(line);
} catch {
continue;
}
if (typeof msg.id === "number" && this.pending.has(msg.id)) {
const { resolve, reject } = this.pending.get(msg.id)!;
this.pending.delete(msg.id);
if (msg.error) reject(new Error(msg.error.message ?? "MCP error"));
else resolve(msg.result);
}
// notifications (no id) ignored.
}
}
private failAll(err: Error) {
for (const { reject } of this.pending.values()) reject(err);
this.pending.clear();
}
private write(obj: unknown) {
if (!this.proc) throw new Error(`${this.serverName}: not started`);
this.proc.stdin.write(`${JSON.stringify(obj)}\n`);
}
private notify(method: string, params: unknown) {
this.write({ jsonrpc: "2.0", method, params });
}
request(method: string, params: unknown): Promise<any> {
const id = this.nextId++;
return new Promise((resolve, reject) => {
this.pending.set(id, { resolve, reject });
this.write({ jsonrpc: "2.0", id, method, params });
});
}
async callTool(name: string, args: Record<string, unknown>): Promise<any> {
return this.request("tools/call", { name, arguments: args });
}
stop() {
if (this.proc) {
try {
this.proc.kill("SIGTERM");
} catch {}
this.proc = null;
}
}
}
// ── MCP result → pi tool result helpers ──────────────────────────────────────
function extractText(mcpResult: any): string {
if (!mcpResult) return "";
const content = mcpResult.content;
if (!Array.isArray(content)) return JSON.stringify(mcpResult);
return content
.map((c: any) => {
if (c?.type === "text" && typeof c.text === "string") return c.text;
return JSON.stringify(c);
})
.join("\n");
}
function namespacedToolName(serverName: string, toolName: string): string {
// Avoid double-prefix: if the MCP server already names its tools
// `<serverName>_<...>`, leave them alone.
const prefix = `${serverName}_`;
if (toolName.startsWith(prefix)) return toolName;
return `${prefix}${toolName}`;
}
// ── Extension entry ──────────────────────────────────────────────────────────
export default async function mcpLoaderExtension(pi: ExtensionAPI) {
const servers = readMcpServers();
const serverNames = Object.keys(servers);
if (serverNames.length === 0) {
dlog("no MCP servers configured — nothing to do");
return;
}
const clients: StdioMcpClient[] = [];
for (const [name, cfg] of Object.entries(servers)) {
if (cfg.enabled === false) {
dlog(`${name}: disabled in settings.json, skipping`);
continue;
}
if (cfg.type === "remote") {
warn(
`${name}: remote MCP transport not yet supported by mcp-loader (v1 is stdio-only); skipping`,
);
continue;
}
const local = cfg as LocalServerConfig;
if (!Array.isArray(local.command) || local.command.length === 0) {
warn(`${name}: invalid \`command\` (must be non-empty array), skipping`);
continue;
}
const [bin, ...args] = local.command;
const client = new StdioMcpClient(name);
try {
await client.start(bin, args, local.env);
} catch (err) {
warn(`${name}: failed to start — ${(err as Error).message}`);
client.stop();
continue;
}
dlog(`${name}: connected, ${client.tools.length} tools`);
clients.push(client);
// Register each MCP tool as a pi tool.
for (const tool of client.tools) {
const piName = namespacedToolName(name, tool.name);
const schema =
tool.inputSchema && typeof tool.inputSchema === "object"
? (Type.Unsafe<Record<string, unknown>>(
tool.inputSchema as object,
) as unknown as ReturnType<typeof Type.Object>)
: Type.Object({}, { additionalProperties: true });
pi.registerTool({
name: piName,
label: piName,
description: tool.description ?? `MCP tool from ${name}`,
parameters: schema,
async execute(_toolCallId, params) {
try {
const result = await client.callTool(tool.name, params as Record<string, unknown>);
const text = extractText(result);
return {
content: [{ type: "text", text }],
details: { server: name, tool: tool.name, raw: result },
};
} catch (err) {
return {
content: [
{
type: "text",
text: `MCP call failed (${name}/${tool.name}): ${(err as Error).message}`,
},
],
details: { server: name, tool: tool.name, error: (err as Error).message },
isError: true,
};
}
},
});
}
}
// Tear down subprocesses on session shutdown so reloads don't leak procs.
pi.on("session_shutdown", async () => {
for (const c of clients) c.stop();
});
}