mcp-loader v2: streamable-HTTP transport for remote MCP servers (context7)
- New RemoteMcpClient implementing MCP streamable-HTTP per spec 2025-03-26:
POST JSON-RPC, parse application/json or text/event-stream responses,
round-trip optional Mcp-Session-Id, optional auth via 'headers' config.
- Refactor StdioMcpClient to share an IMcpClient interface with the remote
client; extension entry dispatches on cfg.type. Drops the v1 'remote
skipped with warning' code path.
- Bump MCP_PROTOCOL_VERSION to 2025-11-25 (single constant, both clients).
- 404 self-heal: when a remote returns 404 to a request carrying our
Mcp-Session-Id, drop the id, re-initialize, retry the request once
(per spec 2025-11-25 \u00a72.2). allowReinitOn404=false on the retry path
prevents recursion. Verified via mock-server smoke test.
- Sanitize pi-facing tool names to ^[A-Za-z][A-Za-z0-9_]{0,63}$. Anthropic
allows hyphens but Bedrock's Anthropic shim rejects them, causing entire
turns to 4xx silently when context7's hyphenated tools (resolve-library-id,
query-docs) were registered. Original MCP-side names are preserved in the
tool-execute closure, so sanitization is purely pi-facing.
- /mcp slash command: drop 'remote (skipped)' status label.
- Docs: README and AGENTS updated for transports, headers config, 404
self-heal, tool-name sanitization rationale, OAuth limitation.
End-to-end verified: context7 connects through pi, returns useful docs
(Bun streaming/SSE example fetched successfully).
This commit is contained in:
@@ -249,13 +249,15 @@ cp /opt/homebrew/Cellar/pi-coding-agent/*/libexec/lib/node_modules/@mariozechner
|
|||||||
|
|
||||||
### `mcp-loader.ts`
|
### `mcp-loader.ts`
|
||||||
|
|
||||||
Generic MCP stdio client — reads an `mcp` block from `~/.pi/agent/settings.json`
|
Generic MCP client — reads an `mcp` block from `~/.pi/agent/settings.json`
|
||||||
and connects to each declared server, exposing the tools as namespaced pi tools.
|
and connects to each declared server (stdio subprocess or streamable-HTTP
|
||||||
|
endpoint), exposing the tools as namespaced pi tools.
|
||||||
|
|
||||||
**Why this exists:** pi has no built-in MCP loader (unlike opencode and Claude
|
**Why this exists:** pi has no built-in MCP loader (unlike opencode and Claude
|
||||||
Desktop). Adding each new MCP server as a hand-rolled extension (the way
|
Desktop). Adding each new MCP server as a hand-rolled extension (the way
|
||||||
`mempalace.ts` does it) doesn't scale past 2-3 servers. This loader is the
|
`mempalace.ts` does it) doesn't scale past 2-3 servers. This loader is the
|
||||||
config-driven generalization — one extension, any number of servers.
|
config-driven generalization — one extension, any number of servers, two
|
||||||
|
transports.
|
||||||
|
|
||||||
**Key design decisions:**
|
**Key design decisions:**
|
||||||
|
|
||||||
@@ -265,42 +267,72 @@ config-driven generalization — one extension, any number of servers.
|
|||||||
multiple servers expose tools with the same short name. Tools that already
|
multiple servers expose tools with the same short name. Tools that already
|
||||||
start with `<servername>_` (the convention some servers like mempalace
|
start with `<servername>_` (the convention some servers like mempalace
|
||||||
follow internally) skip the prefix to avoid double-prefixing.
|
follow internally) skip the prefix to avoid double-prefixing.
|
||||||
|
- **Tool names are sanitized to `^[A-Za-z][A-Za-z0-9_]{0,63}$`** before
|
||||||
|
registering with pi. This is the strictest regex we have to satisfy:
|
||||||
|
Anthropic's Messages API allows hyphens, but AWS Bedrock's Anthropic shim
|
||||||
|
rejects them outright — a single hyphenated tool name causes the whole
|
||||||
|
turn to 4xx silently, manifesting as "no output at all" once the offending
|
||||||
|
server is enabled. context7 surfaces this because its tools are named
|
||||||
|
`resolve-library-id` and `query-docs`. We replace any non-`[A-Za-z0-9_]`
|
||||||
|
char with `_`, prepend `t_` if the result doesn't start with a letter, and
|
||||||
|
truncate to 64 chars. The original MCP-side name is kept in the closure
|
||||||
|
used by `client.callTool`, so the sanitization is purely pi-facing.
|
||||||
- **Fail-soft per server.** A server that won't start (binary missing, init
|
- **Fail-soft per server.** A server that won't start (binary missing, init
|
||||||
handshake fails) logs one stderr line and is skipped. Other servers
|
handshake fails, remote 4xx/5xx) logs one stderr line and is skipped.
|
||||||
continue. Pi keeps working.
|
Other servers continue. Pi keeps working.
|
||||||
- **Stdio transport only in v1.** Remote/streamable-HTTP servers are
|
- **Two transports behind one interface.** `IMcpClient` (`start`, `tools`,
|
||||||
detected and skipped with a warning. Adding remote support is the
|
`callTool`, `stop`) is implemented by `StdioMcpClient` (subprocess +
|
||||||
obvious v2 (context7 is the prime motivator). Avoided in v1 because
|
newline-delimited JSON-RPC) and `RemoteMcpClient` (HTTP POST + JSON or
|
||||||
streamable-HTTP needs SSE parsing, session management, and reconnect
|
SSE response). The extension entry dispatches on `cfg.type` and the
|
||||||
logic that triples the implementation size.
|
rest of the code is transport-agnostic.
|
||||||
- **Does not replace `mempalace.ts`.** The mempalace bridge has bespoke
|
- **Does not replace `mempalace.ts`.** The mempalace bridge has bespoke
|
||||||
agent-identity injection from `$MEMPALACE_AGENT_NAME` that's worth
|
agent-identity injection from `$MEMPALACE_AGENT_NAME` that's worth
|
||||||
preserving. Loader and bridge coexist. Listing `mempalace` in the `mcp`
|
preserving. Loader and bridge coexist. Listing `mempalace` in the `mcp`
|
||||||
block would produce duplicate tool registrations — don't.
|
block would produce duplicate tool registrations — don't.
|
||||||
- **No reconnect on subprocess death.** If a server's subprocess crashes
|
- **No stdio reconnect on death.** If a stdio subprocess crashes
|
||||||
mid-session, its tools become permanently unavailable until pi `/reload`s.
|
mid-session, its tools become permanently unavailable until pi `/reload`s.
|
||||||
Same limitation as the mempalace bridge today; not worth complicating v1.
|
Same limitation as the mempalace bridge today.
|
||||||
|
- **Remote sessions self-heal on 404.** Per spec 2025-11-25 §2.2, a server
|
||||||
|
that has terminated a session MUST respond with HTTP 404 to requests
|
||||||
|
carrying the stale `Mcp-Session-Id`. `RemoteMcpClient` catches this
|
||||||
|
condition (404 + sessionAtStart was non-null), drops the id, runs a fresh
|
||||||
|
`initialize` + `notifications/initialized`, and retries the original
|
||||||
|
request once. Persistent 404s after refresh surface as errors.
|
||||||
|
|
||||||
**API used:**
|
**API used:**
|
||||||
|
|
||||||
- `pi.registerTool({ name, label, description, parameters, execute })` for
|
- `pi.registerTool({ name, label, description, parameters, execute })` for
|
||||||
each MCP tool exposed by each connected server.
|
each MCP tool exposed by each connected server.
|
||||||
- `pi.on("session_shutdown", ...)` to SIGTERM all subprocesses cleanly.
|
- `pi.on("session_shutdown", ...)` to SIGTERM stdio subprocesses and DELETE
|
||||||
|
remote sessions cleanly.
|
||||||
- `Type.Unsafe<...>(inputSchema)` from typebox to pass the MCP server's
|
- `Type.Unsafe<...>(inputSchema)` from typebox to pass the MCP server's
|
||||||
JSON schema through to pi without conversion (TypeBox schemas are
|
JSON schema through to pi without conversion (TypeBox schemas are
|
||||||
plain JSON Schema at runtime).
|
plain JSON Schema at runtime).
|
||||||
|
|
||||||
**Internal MCP client:**
|
**Internal MCP clients:**
|
||||||
|
|
||||||
- `class StdioMcpClient` — spawn subprocess, write newline-delimited
|
- `class StdioMcpClient` — spawn subprocess, write newline-delimited
|
||||||
JSON-RPC, parse newline-delimited responses, match by request `id`.
|
JSON-RPC, parse newline-delimited responses, match by request `id`.
|
||||||
- Sends `initialize` handshake with `protocolVersion: 2024-11-05`, then
|
Stderr drained silently unless `PI_MCP_LOADER_DEBUG=1`.
|
||||||
`notifications/initialized`, then `tools/list` to discover tools.
|
- `class RemoteMcpClient` — streamable-HTTP per MCP spec 2025-03-26.
|
||||||
- Stderr drained silently unless `PI_MCP_LOADER_DEBUG=1`.
|
POST JSON-RPC body to a single URL with `Accept: application/json,
|
||||||
|
text/event-stream`. Server replies either with one JSON response or an
|
||||||
|
SSE stream containing `event: message` / `data: <json>` blocks; we
|
||||||
|
consume the stream until the response with our `id` arrives, then
|
||||||
|
cancel the reader. Optional `Mcp-Session-Id` header is captured on
|
||||||
|
initialize and round-tripped on every subsequent request, then DELETEd
|
||||||
|
on stop. Per-server `headers` config (e.g. `Authorization`) is merged
|
||||||
|
into every request. No GET-stream subscription — server-initiated
|
||||||
|
notifications are not consumed.
|
||||||
|
- Both share `interface IMcpClient { serverName, tools, start, callTool, stop }`.
|
||||||
|
Both send `initialize` handshake with `protocolVersion: MCP_PROTOCOL_VERSION`
|
||||||
|
(currently `2025-11-25`, per the constant at the top of the file),
|
||||||
|
then `notifications/initialized`, then `tools/list` to discover tools.
|
||||||
|
|
||||||
**Future v2 extensions:**
|
**Future extensions:**
|
||||||
|
|
||||||
- Streamable-HTTP transport for remote servers (context7).
|
- OAuth flow for remote servers that require it (today: pre-issued bearer
|
||||||
|
tokens via `headers` only).
|
||||||
- Per-tool enable/disable in settings.json (e.g.
|
- Per-tool enable/disable in settings.json (e.g.
|
||||||
`"mcp.searxng.tools": ["web_search"]` to expose only a subset).
|
`"mcp.searxng.tools": ["web_search"]` to expose only a subset).
|
||||||
- Reconnect-on-crash with exponential backoff.
|
- Reconnect-on-crash with exponential backoff.
|
||||||
|
|||||||
@@ -205,7 +205,7 @@ This is a verbatim copy of the upstream `examples/extensions/todo.ts` shipped wi
|
|||||||
|
|
||||||
### `mcp-loader.ts`
|
### `mcp-loader.ts`
|
||||||
|
|
||||||
Generic MCP server loader. Reads an `mcp` block from `~/.pi/agent/settings.json` (same shape as opencode and Claude Desktop) and connects to each declared server, exposing all of their tools to pi as native tools — namespaced as `<server-name>_<tool-name>` to avoid collisions.
|
Generic MCP server loader. Reads an `mcp` block from `~/.pi/agent/settings.json` (same shape as opencode and Claude Desktop) and connects to each declared server, exposing all of their tools to pi as native tools — namespaced as `<server-name>_<tool-name>` to avoid collisions, with non-`[A-Za-z0-9_]` characters replaced by `_` so the names pass the strictest provider tool-name regex (e.g. AWS Bedrock).
|
||||||
|
|
||||||
**Settings.json shape:**
|
**Settings.json shape:**
|
||||||
|
|
||||||
@@ -239,13 +239,20 @@ Generic MCP server loader. Reads an `mcp` block from `~/.pi/agent/settings.json`
|
|||||||
| `type` | `"local"` (stdio subprocess) or `"remote"` (streamable-http). Default `"local"`. |
|
| `type` | `"local"` (stdio subprocess) or `"remote"` (streamable-http). Default `"local"`. |
|
||||||
| `command` | Argv array. First element is the executable, rest are args. Local servers only. |
|
| `command` | Argv array. First element is the executable, rest are args. Local servers only. |
|
||||||
| `url` | Remote MCP endpoint URL. Remote servers only. |
|
| `url` | Remote MCP endpoint URL. Remote servers only. |
|
||||||
|
| `headers` | Optional object of HTTP headers (e.g. `Authorization`, `X-API-Key`) sent with every request. Remote servers only. |
|
||||||
| `enabled` | Default `true`. Set `false` to disable a server without removing the entry. |
|
| `enabled` | Default `true`. Set `false` to disable a server without removing the entry. |
|
||||||
| `env` | Optional object of env vars injected into the subprocess. Inherits parent env first, then overlays these keys. Local servers only. |
|
| `env` | Optional object of env vars injected into the subprocess. Inherits parent env first, then overlays these keys. Local servers only. |
|
||||||
|
|
||||||
**Limitations (v1):**
|
**Transports:**
|
||||||
|
|
||||||
- **Stdio only.** Remote/streamable-HTTP transport is detected and skipped with a warning. Server like `context7` configured as `"type": "remote"` will not load until v2.
|
- `local` — stdio JSON-RPC subprocess (mcp-searxng, gitea-mcp, mcp-server-time…).
|
||||||
- **No reconnect** if a subprocess dies mid-session — those tools become unavailable until `/reload` (same as `mempalace.ts`).
|
- `remote` — streamable-HTTP per MCP spec 2025-03-26: POST JSON-RPC, server replies either `application/json` or `text/event-stream`. Optional `Mcp-Session-Id` round-trip if the server issues one. No GET subscription stream (server-initiated notifications are not consumed).
|
||||||
|
|
||||||
|
**Limitations:**
|
||||||
|
|
||||||
|
- **No stdio reconnect** if a subprocess dies mid-session — those tools become unavailable until `/reload` (same as `mempalace.ts`).
|
||||||
|
- **Remote sessions self-heal on 404.** If a streamable-HTTP server forgets our session id (e.g. server restart), the client transparently re-initializes and retries the request once.
|
||||||
|
- **No OAuth flow.** Remote servers requiring OAuth must be accessed with a pre-issued bearer token via `headers`.
|
||||||
- **Coexists with `mempalace.ts`** but does not replace it. The mempalace bridge has bespoke handling (agent identity injection) that's worth keeping. Don't list `mempalace` in the `mcp` block too — you'd get duplicate tool registrations.
|
- **Coexists with `mempalace.ts`** but does not replace it. The mempalace bridge has bespoke handling (agent identity injection) that's worth keeping. Don't list `mempalace` in the `mcp` block too — you'd get duplicate tool registrations.
|
||||||
|
|
||||||
**Debug:** set `PI_MCP_LOADER_DEBUG=1` in the environment to surface per-server stderr and connection logs.
|
**Debug:** set `PI_MCP_LOADER_DEBUG=1` in the environment to surface per-server stderr and connection logs.
|
||||||
@@ -255,7 +262,6 @@ Generic MCP server loader. Reads an `mcp` block from `~/.pi/agent/settings.json`
|
|||||||
- `running · N tools` — connected, tools registered
|
- `running · N tools` — connected, tools registered
|
||||||
- `failed: <message>` — start handshake threw
|
- `failed: <message>` — start handshake threw
|
||||||
- `disabled in settings` — `enabled: false`
|
- `disabled in settings` — `enabled: false`
|
||||||
- `remote (skipped — v1 stdio only)` — type `remote`, awaiting v2
|
|
||||||
- `invalid: <message>` — malformed config (read-only row)
|
- `invalid: <message>` — malformed config (read-only row)
|
||||||
|
|
||||||
UX matches `/ext`: **space** stages a toggle, **enter** writes back to `settings.json` and reloads pi, **esc** cancels. Toggling re-enables a previously-disabled server by removing the explicit `enabled` key (the default is `true`).
|
UX matches `/ext`: **space** stages a toggle, **enter** writes back to `settings.json` and reloads pi, **esc** cancels. Toggling re-enables a previously-disabled server by removing the explicit `enabled` key (the default is `true`).
|
||||||
|
|||||||
+287
-48
@@ -22,7 +22,8 @@
|
|||||||
* },
|
* },
|
||||||
* "context7": {
|
* "context7": {
|
||||||
* "type": "remote",
|
* "type": "remote",
|
||||||
* "url": "https://mcp.context7.com/mcp"
|
* "url": "https://mcp.context7.com/mcp",
|
||||||
|
* "headers": { "Authorization": "Bearer …" }
|
||||||
* }
|
* }
|
||||||
* }
|
* }
|
||||||
* }
|
* }
|
||||||
@@ -32,11 +33,13 @@
|
|||||||
* command argv array — first element is the executable, rest are args.
|
* command argv array — first element is the executable, rest are args.
|
||||||
* For local servers only.
|
* For local servers only.
|
||||||
* url remote MCP endpoint URL. For remote servers only.
|
* url remote MCP endpoint URL. For remote servers only.
|
||||||
|
* headers optional object of HTTP headers to send with every request to
|
||||||
|
* a remote server (e.g. Authorization, X-API-Key). Remote only.
|
||||||
* enabled default true. Set false to disable a server without
|
* enabled default true. Set false to disable a server without
|
||||||
* removing it from settings.json.
|
* removing it from settings.json.
|
||||||
* env optional object of environment variables to inject into
|
* env optional object of environment variables to inject into
|
||||||
* the spawned subprocess. Inherits parent env first, then
|
* the spawned subprocess. Inherits parent env first, then
|
||||||
* overlays these keys.
|
* overlays these keys. Local only.
|
||||||
*
|
*
|
||||||
* Tool naming
|
* Tool naming
|
||||||
*
|
*
|
||||||
@@ -47,22 +50,35 @@
|
|||||||
*
|
*
|
||||||
* Lifecycle
|
* Lifecycle
|
||||||
*
|
*
|
||||||
* • Servers are spawned at extension load time (pi startup or /reload).
|
* • Servers are connected at extension load time (pi startup or /reload).
|
||||||
* • Each server's `tools/list` is awaited before pi finishes registering.
|
* • Each server's `tools/list` is awaited before pi finishes registering.
|
||||||
* • Subprocess stderr is silenced unless PI_MCP_LOADER_DEBUG=1.
|
* • Local subprocess stderr is silenced unless PI_MCP_LOADER_DEBUG=1.
|
||||||
* • Subprocesses receive SIGTERM at session_shutdown.
|
* • Local subprocesses receive SIGTERM at session_shutdown; remote
|
||||||
* • A server that fails to start (binary missing, init handshake error)
|
* servers receive an MCP DELETE if a session id was issued.
|
||||||
* logs a single line to stderr and is skipped. Other servers continue.
|
* • A server that fails to start (binary missing, init handshake error,
|
||||||
* Pi keeps working without that server's tools.
|
* remote 4xx/5xx) logs a single line to stderr and is skipped. Other
|
||||||
|
* servers continue. Pi keeps working without that server's tools.
|
||||||
*
|
*
|
||||||
* Limitations (v1)
|
* Transports
|
||||||
*
|
*
|
||||||
* • Only local/stdio transport is implemented. Remote (streamable-http,
|
* local — stdio JSON-RPC subprocess (mcp-searxng, gitea-mcp, …).
|
||||||
* SSE) servers are detected and skipped with a warning. v2 will add
|
* remote — streamable-HTTP per MCP spec 2025-03-26: POST JSON-RPC, server
|
||||||
* remote support — context7 is the prime motivator.
|
* replies either application/json or text/event-stream. Optional
|
||||||
* • No reconnect: if a subprocess dies mid-session, its tools become
|
* Mcp-Session-Id header round-tripped if the server issues one.
|
||||||
* unavailable until pi is reloaded. The mempalace.ts extension has
|
* No GET-stream subscription (we don't consume server-initiated
|
||||||
* the same limitation today.
|
* notifications).
|
||||||
|
*
|
||||||
|
* Limitations
|
||||||
|
*
|
||||||
|
* • No stdio reconnect: if a stdio subprocess dies mid-session, its tools
|
||||||
|
* become unavailable until pi is reloaded. The mempalace.ts extension
|
||||||
|
* has the same limitation today.
|
||||||
|
* • Remote sessions self-heal on 404: if a streamable-HTTP server
|
||||||
|
* forgets our session id (e.g. server restart), the next request gets
|
||||||
|
* HTTP 404; the client transparently re-initializes and retries the
|
||||||
|
* request once. Persistent 404s after refresh surface as errors.
|
||||||
|
* • No OAuth flow: remote servers requiring OAuth must be accessed with
|
||||||
|
* a pre-issued bearer token via `headers`.
|
||||||
*
|
*
|
||||||
* Coexistence with mempalace.ts
|
* Coexistence with mempalace.ts
|
||||||
*
|
*
|
||||||
@@ -76,7 +92,7 @@
|
|||||||
* Slash command
|
* Slash command
|
||||||
*
|
*
|
||||||
* `/mcp` lists configured MCP servers with their runtime status
|
* `/mcp` lists configured MCP servers with their runtime status
|
||||||
* (running / failed / disabled / remote-skipped / invalid) and lets
|
* (running / failed / disabled / invalid) and lets
|
||||||
* you toggle the `enabled` flag in settings.json. Same UX as `/ext`:
|
* you toggle the `enabled` flag in settings.json. Same UX as `/ext`:
|
||||||
* space stages, enter applies + reloads, esc cancels.
|
* space stages, enter applies + reloads, esc cancels.
|
||||||
*/
|
*/
|
||||||
@@ -116,6 +132,7 @@ type LocalServerConfig = {
|
|||||||
type RemoteServerConfig = {
|
type RemoteServerConfig = {
|
||||||
type: "remote";
|
type: "remote";
|
||||||
url: string;
|
url: string;
|
||||||
|
headers?: Record<string, string>;
|
||||||
enabled?: boolean;
|
enabled?: boolean;
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -124,6 +141,13 @@ type ServerConfig = LocalServerConfig | RemoteServerConfig;
|
|||||||
const SETTINGS_PATH = path.join(os.homedir(), ".pi", "agent", "settings.json");
|
const SETTINGS_PATH = path.join(os.homedir(), ".pi", "agent", "settings.json");
|
||||||
const DEBUG = process.env.PI_MCP_LOADER_DEBUG === "1";
|
const DEBUG = process.env.PI_MCP_LOADER_DEBUG === "1";
|
||||||
|
|
||||||
|
// MCP protocol version we advertise. Servers we've tested:
|
||||||
|
// - searxng (mcp-searxng) accepts both 2024-11-05 and 2025-11-25
|
||||||
|
// - context7 accepts both
|
||||||
|
// - mempalace bridge: not touched by this loader
|
||||||
|
// Bump as the spec evolves; both stdio and remote clients use this.
|
||||||
|
const MCP_PROTOCOL_VERSION = "2025-11-25";
|
||||||
|
|
||||||
function dlog(msg: string) {
|
function dlog(msg: string) {
|
||||||
if (DEBUG) process.stderr.write(`[mcp-loader] ${msg}\n`);
|
if (DEBUG) process.stderr.write(`[mcp-loader] ${msg}\n`);
|
||||||
}
|
}
|
||||||
@@ -162,21 +186,35 @@ function readMcpServers(): Record<string, ServerConfig> {
|
|||||||
|
|
||||||
// ── Stdio MCP client ─────────────────────────────────────────────────────────
|
// ── Stdio MCP client ─────────────────────────────────────────────────────────
|
||||||
|
|
||||||
class StdioMcpClient {
|
interface IMcpClient {
|
||||||
|
readonly serverName: string;
|
||||||
|
tools: McpTool[];
|
||||||
|
start(): Promise<void>;
|
||||||
|
callTool(name: string, args: Record<string, unknown>): Promise<any>;
|
||||||
|
stop(): void | Promise<void>;
|
||||||
|
}
|
||||||
|
|
||||||
|
class StdioMcpClient implements IMcpClient {
|
||||||
private proc: ChildProcessWithoutNullStreams | null = null;
|
private proc: ChildProcessWithoutNullStreams | null = null;
|
||||||
private nextId = 1;
|
private nextId = 1;
|
||||||
private pending = new Map<number, { resolve: (v: any) => void; reject: (e: Error) => void }>();
|
private pending = new Map<number, { resolve: (v: any) => void; reject: (e: Error) => void }>();
|
||||||
private stdoutBuf = "";
|
private stdoutBuf = "";
|
||||||
private serverName: string;
|
public readonly serverName: string;
|
||||||
|
private command: string;
|
||||||
|
private args: string[];
|
||||||
|
private env: Record<string, string> | undefined;
|
||||||
public tools: McpTool[] = [];
|
public tools: McpTool[] = [];
|
||||||
|
|
||||||
constructor(serverName: string) {
|
constructor(serverName: string, command: string, args: string[], env?: Record<string, string>) {
|
||||||
this.serverName = serverName;
|
this.serverName = serverName;
|
||||||
|
this.command = command;
|
||||||
|
this.args = args;
|
||||||
|
this.env = env;
|
||||||
}
|
}
|
||||||
|
|
||||||
async start(command: string, args: string[], env?: Record<string, string>): Promise<void> {
|
async start(): Promise<void> {
|
||||||
const childEnv = { ...process.env, ...(env ?? {}) };
|
const childEnv = { ...process.env, ...(this.env ?? {}) };
|
||||||
this.proc = spawn(command, args, { stdio: ["pipe", "pipe", "pipe"], env: childEnv });
|
this.proc = spawn(this.command, this.args, { stdio: ["pipe", "pipe", "pipe"], env: childEnv });
|
||||||
|
|
||||||
this.proc.on("error", (err) => this.failAll(err));
|
this.proc.on("error", (err) => this.failAll(err));
|
||||||
this.proc.on("exit", (code, signal) => {
|
this.proc.on("exit", (code, signal) => {
|
||||||
@@ -197,9 +235,9 @@ class StdioMcpClient {
|
|||||||
|
|
||||||
// MCP initialize handshake
|
// MCP initialize handshake
|
||||||
await this.request("initialize", {
|
await this.request("initialize", {
|
||||||
protocolVersion: "2024-11-05",
|
protocolVersion: MCP_PROTOCOL_VERSION,
|
||||||
capabilities: {},
|
capabilities: {},
|
||||||
clientInfo: { name: "pi-mcp-loader", version: "0.1.0" },
|
clientInfo: { name: "pi-mcp-loader", version: "0.2.0" },
|
||||||
});
|
});
|
||||||
this.notify("notifications/initialized", {});
|
this.notify("notifications/initialized", {});
|
||||||
|
|
||||||
@@ -266,6 +304,195 @@ class StdioMcpClient {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ── Streamable-HTTP MCP client (remote) ──────────────────────────────────────
|
||||||
|
//
|
||||||
|
// Per MCP spec 2025-03-26: a single endpoint URL accepts POST with a
|
||||||
|
// JSON-RPC body. The server responds either with application/json (a
|
||||||
|
// single response) or text/event-stream (an SSE stream containing the
|
||||||
|
// response, possibly preceded by notifications, terminated by stream
|
||||||
|
// close). We do NOT open a separate GET subscription stream — we don't
|
||||||
|
// consume server-initiated notifications today.
|
||||||
|
//
|
||||||
|
// Session id: if the server returns an Mcp-Session-Id header on
|
||||||
|
// initialize, we round-trip it on every subsequent request and DELETE
|
||||||
|
// it on stop().
|
||||||
|
|
||||||
|
class RemoteMcpClient implements IMcpClient {
|
||||||
|
public readonly serverName: string;
|
||||||
|
private url: string;
|
||||||
|
private extraHeaders: Record<string, string>;
|
||||||
|
private sessionId: string | null = null;
|
||||||
|
private nextId = 1;
|
||||||
|
public tools: McpTool[] = [];
|
||||||
|
|
||||||
|
constructor(serverName: string, url: string, headers?: Record<string, string>) {
|
||||||
|
this.serverName = serverName;
|
||||||
|
this.url = url;
|
||||||
|
this.extraHeaders = headers ?? {};
|
||||||
|
}
|
||||||
|
|
||||||
|
async start(): Promise<void> {
|
||||||
|
await this.request("initialize", {
|
||||||
|
protocolVersion: MCP_PROTOCOL_VERSION,
|
||||||
|
capabilities: {},
|
||||||
|
clientInfo: { name: "pi-mcp-loader", version: "0.2.0" },
|
||||||
|
});
|
||||||
|
await this.notify("notifications/initialized", {});
|
||||||
|
const listed = await this.request("tools/list", {});
|
||||||
|
this.tools = (listed?.tools as McpTool[]) ?? [];
|
||||||
|
}
|
||||||
|
|
||||||
|
async callTool(name: string, args: Record<string, unknown>): Promise<any> {
|
||||||
|
return this.request("tools/call", { name, arguments: args });
|
||||||
|
}
|
||||||
|
|
||||||
|
async stop(): Promise<void> {
|
||||||
|
if (!this.sessionId) return;
|
||||||
|
try {
|
||||||
|
await fetch(this.url, { method: "DELETE", headers: this.buildHeaders() });
|
||||||
|
} catch {}
|
||||||
|
}
|
||||||
|
|
||||||
|
private buildHeaders(): Record<string, string> {
|
||||||
|
const h: Record<string, string> = {
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
Accept: "application/json, text/event-stream",
|
||||||
|
"MCP-Protocol-Version": MCP_PROTOCOL_VERSION,
|
||||||
|
...this.extraHeaders,
|
||||||
|
};
|
||||||
|
if (this.sessionId) h["Mcp-Session-Id"] = this.sessionId;
|
||||||
|
return h;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Re-establish a session after the server returned 404 to a request that
|
||||||
|
* carried our `Mcp-Session-Id`. Per spec 2025-11-25, the client MUST start
|
||||||
|
* a new session by sending a fresh InitializeRequest without a session id.
|
||||||
|
*
|
||||||
|
* Caller must clear `this.sessionId` BEFORE invoking this so the new
|
||||||
|
* initialize POST goes out without the stale id.
|
||||||
|
*/
|
||||||
|
private async reinitialize(): Promise<void> {
|
||||||
|
dlog(`${this.serverName}: session lost (404) — reinitializing`);
|
||||||
|
await this.request("initialize", {
|
||||||
|
protocolVersion: MCP_PROTOCOL_VERSION,
|
||||||
|
capabilities: {},
|
||||||
|
clientInfo: { name: "pi-mcp-loader", version: "0.2.0" },
|
||||||
|
}, /*allowReinitOn404=*/ false);
|
||||||
|
await this.notify("notifications/initialized", {});
|
||||||
|
}
|
||||||
|
|
||||||
|
private async request(method: string, params: unknown, allowReinitOn404 = true): Promise<any> {
|
||||||
|
const id = this.nextId++;
|
||||||
|
const body = JSON.stringify({ jsonrpc: "2.0", id, method, params });
|
||||||
|
const sessionAtStart = this.sessionId;
|
||||||
|
let res: Response;
|
||||||
|
try {
|
||||||
|
res = await fetch(this.url, { method: "POST", headers: this.buildHeaders(), body });
|
||||||
|
} catch (err) {
|
||||||
|
throw new Error(`${this.serverName}: fetch failed on ${method}: ${(err as Error).message}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
const sid = res.headers.get("Mcp-Session-Id") ?? res.headers.get("mcp-session-id");
|
||||||
|
if (sid) this.sessionId = sid;
|
||||||
|
|
||||||
|
// 404 + we sent a session id → server forgot us. Drop our id, re-init,
|
||||||
|
// retry once. `allowReinitOn404=false` is passed by the recovery path to
|
||||||
|
// prevent infinite recursion if the server is permanently broken.
|
||||||
|
if (res.status === 404 && sessionAtStart && allowReinitOn404) {
|
||||||
|
try { await res.arrayBuffer(); } catch {}
|
||||||
|
this.sessionId = null;
|
||||||
|
await this.reinitialize();
|
||||||
|
return this.request(method, params, /*allowReinitOn404=*/ false);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!res.ok) {
|
||||||
|
let detail = "";
|
||||||
|
try { detail = (await res.text()).slice(0, 200); } catch {}
|
||||||
|
throw new Error(`${this.serverName}: HTTP ${res.status} on ${method}${detail ? `: ${detail}` : ""}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
const ct = (res.headers.get("content-type") ?? "").toLowerCase();
|
||||||
|
if (ct.includes("text/event-stream")) {
|
||||||
|
return await this.readSseForResponse(res, id);
|
||||||
|
}
|
||||||
|
if (ct.includes("application/json")) {
|
||||||
|
const json: any = await res.json();
|
||||||
|
if (json.error) throw new Error(json.error.message ?? "MCP error");
|
||||||
|
return json.result;
|
||||||
|
}
|
||||||
|
if (res.status === 202) return undefined;
|
||||||
|
throw new Error(`${this.serverName}: unexpected content-type "${ct}" on ${method}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
private async notify(method: string, params: unknown): Promise<void> {
|
||||||
|
const body = JSON.stringify({ jsonrpc: "2.0", method, params });
|
||||||
|
let res: Response;
|
||||||
|
try {
|
||||||
|
res = await fetch(this.url, { method: "POST", headers: this.buildHeaders(), body });
|
||||||
|
} catch (err) {
|
||||||
|
throw new Error(`${this.serverName}: fetch failed on notify ${method}: ${(err as Error).message}`);
|
||||||
|
}
|
||||||
|
try { await res.arrayBuffer(); } catch {}
|
||||||
|
if (!res.ok && res.status !== 202) {
|
||||||
|
throw new Error(`${this.serverName}: HTTP ${res.status} on notify ${method}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private async readSseForResponse(res: Response, expectedId: number): Promise<any> {
|
||||||
|
if (!res.body) throw new Error(`${this.serverName}: SSE response had no body`);
|
||||||
|
const reader = res.body.getReader();
|
||||||
|
const decoder = new TextDecoder();
|
||||||
|
let buf = "";
|
||||||
|
let dataLines: string[] = [];
|
||||||
|
|
||||||
|
const tryDispatch = (): { matched: boolean; result?: any } => {
|
||||||
|
if (dataLines.length === 0) return { matched: false };
|
||||||
|
const data = dataLines.join("\n");
|
||||||
|
dataLines = [];
|
||||||
|
let msg: any;
|
||||||
|
try {
|
||||||
|
msg = JSON.parse(data);
|
||||||
|
} catch {
|
||||||
|
return { matched: false };
|
||||||
|
}
|
||||||
|
if (typeof msg.id === "number" && msg.id === expectedId) {
|
||||||
|
if (msg.error) throw new Error(msg.error.message ?? "MCP error");
|
||||||
|
return { matched: true, result: msg.result };
|
||||||
|
}
|
||||||
|
return { matched: false };
|
||||||
|
};
|
||||||
|
|
||||||
|
try {
|
||||||
|
while (true) {
|
||||||
|
const { done, value } = await reader.read();
|
||||||
|
if (done) break;
|
||||||
|
buf += decoder.decode(value, { stream: true });
|
||||||
|
let nl: number;
|
||||||
|
while ((nl = buf.indexOf("\n")) !== -1) {
|
||||||
|
const rawLine = buf.slice(0, nl);
|
||||||
|
buf = buf.slice(nl + 1);
|
||||||
|
const line = rawLine.endsWith("\r") ? rawLine.slice(0, -1) : rawLine;
|
||||||
|
if (line === "") {
|
||||||
|
const out = tryDispatch();
|
||||||
|
if (out.matched) return out.result;
|
||||||
|
} else if (line.startsWith(":")) {
|
||||||
|
// SSE comment — ignore (often a keepalive)
|
||||||
|
} else if (line.startsWith("data:")) {
|
||||||
|
dataLines.push(line.slice(5).replace(/^ /, ""));
|
||||||
|
}
|
||||||
|
// event:, id:, retry: — ignored; only `data` matters for our use
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const out = tryDispatch();
|
||||||
|
if (out.matched) return out.result;
|
||||||
|
} finally {
|
||||||
|
try { reader.cancel(); } catch {}
|
||||||
|
}
|
||||||
|
throw new Error(`${this.serverName}: SSE stream closed without response for id=${expectedId}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// ── MCP result → pi tool result helpers ──────────────────────────────────────
|
// ── MCP result → pi tool result helpers ──────────────────────────────────────
|
||||||
|
|
||||||
function extractText(mcpResult: any): string {
|
function extractText(mcpResult: any): string {
|
||||||
@@ -284,8 +511,20 @@ function namespacedToolName(serverName: string, toolName: string): string {
|
|||||||
// Avoid double-prefix: if the MCP server already names its tools
|
// Avoid double-prefix: if the MCP server already names its tools
|
||||||
// `<serverName>_<...>`, leave them alone.
|
// `<serverName>_<...>`, leave them alone.
|
||||||
const prefix = `${serverName}_`;
|
const prefix = `${serverName}_`;
|
||||||
if (toolName.startsWith(prefix)) return toolName;
|
const raw = toolName.startsWith(prefix) ? toolName : `${prefix}${toolName}`;
|
||||||
return `${prefix}${toolName}`;
|
// Sanitize for the strictest tool-name regex we have to satisfy.
|
||||||
|
// Anthropic's Messages API allows `^[a-zA-Z0-9_-]{1,64}$` (hyphens OK),
|
||||||
|
// but AWS Bedrock's Anthropic shim rejects names containing `-` outright
|
||||||
|
// — it accepts `^[a-zA-Z][a-zA-Z0-9_]{0,63}$`. Hyphenated MCP tool names
|
||||||
|
// (context7's `resolve-library-id`, `query-docs`) caused entire turns to
|
||||||
|
// 4xx silently on Bedrock, manifesting as no output at all once the
|
||||||
|
// server was enabled. Replace any non-[A-Za-z0-9_] with `_`, then prefix
|
||||||
|
// a leading underscore with `t` if the result starts with a non-letter.
|
||||||
|
// Truncate to 64 to also satisfy Anthropic's length cap.
|
||||||
|
let sanitized = raw.replace(/[^A-Za-z0-9_]/g, "_");
|
||||||
|
if (!/^[A-Za-z]/.test(sanitized)) sanitized = `t_${sanitized}`;
|
||||||
|
if (sanitized.length > 64) sanitized = sanitized.slice(0, 64);
|
||||||
|
return sanitized;
|
||||||
}
|
}
|
||||||
|
|
||||||
// ── Settings.json writer ─────────────────────────────────────────────────────
|
// ── Settings.json writer ─────────────────────────────────────────────────────
|
||||||
@@ -316,7 +555,6 @@ type ServerRuntimeState =
|
|||||||
| { kind: "running"; toolCount: number }
|
| { kind: "running"; toolCount: number }
|
||||||
| { kind: "failed"; message: string }
|
| { kind: "failed"; message: string }
|
||||||
| { kind: "disabled" }
|
| { kind: "disabled" }
|
||||||
| { kind: "remote-skipped" }
|
|
||||||
| { kind: "invalid"; message: string };
|
| { kind: "invalid"; message: string };
|
||||||
|
|
||||||
// ── Extension entry ──────────────────────────────────────────────────────────
|
// ── Extension entry ──────────────────────────────────────────────────────────
|
||||||
@@ -324,7 +562,7 @@ type ServerRuntimeState =
|
|||||||
export default async function mcpLoaderExtension(pi: ExtensionAPI) {
|
export default async function mcpLoaderExtension(pi: ExtensionAPI) {
|
||||||
const servers = readMcpServers();
|
const servers = readMcpServers();
|
||||||
const runtime = new Map<string, ServerRuntimeState>();
|
const runtime = new Map<string, ServerRuntimeState>();
|
||||||
const clients: StdioMcpClient[] = [];
|
const clients: IMcpClient[] = [];
|
||||||
|
|
||||||
// Always register /mcp — even when no servers are configured, so users
|
// Always register /mcp — even when no servers are configured, so users
|
||||||
// discover the surface. Handler reads settings.json fresh on each invoke.
|
// discover the surface. Handler reads settings.json fresh on each invoke.
|
||||||
@@ -342,30 +580,33 @@ export default async function mcpLoaderExtension(pi: ExtensionAPI) {
|
|||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
let client: IMcpClient;
|
||||||
if (cfg.type === "remote") {
|
if (cfg.type === "remote") {
|
||||||
warn(
|
const remote = cfg as RemoteServerConfig;
|
||||||
`${name}: remote MCP transport not yet supported by mcp-loader (v1 is stdio-only); skipping`,
|
if (typeof remote.url !== "string" || remote.url.length === 0) {
|
||||||
);
|
warn(`${name}: remote server missing \`url\`, skipping`);
|
||||||
runtime.set(name, { kind: "remote-skipped" });
|
runtime.set(name, { kind: "invalid", message: "remote server missing url" });
|
||||||
continue;
|
continue;
|
||||||
|
}
|
||||||
|
client = new RemoteMcpClient(name, remote.url, remote.headers);
|
||||||
|
} else {
|
||||||
|
const local = cfg as LocalServerConfig;
|
||||||
|
if (!Array.isArray(local.command) || local.command.length === 0) {
|
||||||
|
warn(`${name}: invalid \`command\` (must be non-empty array), skipping`);
|
||||||
|
runtime.set(name, { kind: "invalid", message: "command must be non-empty array" });
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
const [bin, ...args] = local.command;
|
||||||
|
client = new StdioMcpClient(name, bin, args, local.env);
|
||||||
}
|
}
|
||||||
|
|
||||||
const local = cfg as LocalServerConfig;
|
|
||||||
if (!Array.isArray(local.command) || local.command.length === 0) {
|
|
||||||
warn(`${name}: invalid \`command\` (must be non-empty array), skipping`);
|
|
||||||
runtime.set(name, { kind: "invalid", message: "command must be non-empty array" });
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
const [bin, ...args] = local.command;
|
|
||||||
|
|
||||||
const client = new StdioMcpClient(name);
|
|
||||||
try {
|
try {
|
||||||
await client.start(bin, args, local.env);
|
await client.start();
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
const message = (err as Error).message;
|
const message = (err as Error).message;
|
||||||
warn(`${name}: failed to start — ${message}`);
|
warn(`${name}: failed to start — ${message}`);
|
||||||
runtime.set(name, { kind: "failed", message });
|
runtime.set(name, { kind: "failed", message });
|
||||||
client.stop();
|
try { await client.stop(); } catch {}
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -413,9 +654,9 @@ export default async function mcpLoaderExtension(pi: ExtensionAPI) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Tear down subprocesses on session shutdown so reloads don't leak procs.
|
// Tear down clients (stdio subprocesses, remote sessions) on session shutdown.
|
||||||
pi.on("session_shutdown", async () => {
|
pi.on("session_shutdown", async () => {
|
||||||
for (const c of clients) c.stop();
|
await Promise.allSettled(clients.map((c) => Promise.resolve(c.stop())));
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -434,8 +675,6 @@ function statusLabel(state: ServerRuntimeState | undefined, configEnabled: boole
|
|||||||
return `failed: ${state.message}`;
|
return `failed: ${state.message}`;
|
||||||
case "disabled":
|
case "disabled":
|
||||||
return "disabled in settings";
|
return "disabled in settings";
|
||||||
case "remote-skipped":
|
|
||||||
return "remote (skipped — v1 stdio only)";
|
|
||||||
case "invalid":
|
case "invalid":
|
||||||
return `invalid: ${state.message}`;
|
return `invalid: ${state.message}`;
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user