feat: host-agnostic LAN access (base) + fork/recall in pi variants
Validate / base-change-warning (push) Successful in 22s
Validate / docs-check (push) Successful in 44s
Validate / validate-base (push) Successful in 3m27s
Validate / validate-omos (push) Successful in 7m3s
Validate / validate-with-pi (push) Failing after 4m33s
Validate / validate-omos-with-pi (push) Failing after 8m29s
Validate / base-change-warning (push) Successful in 22s
Validate / docs-check (push) Successful in 44s
Validate / validate-base (push) Successful in 3m27s
Validate / validate-omos (push) Successful in 7m3s
Validate / validate-with-pi (push) Failing after 4m33s
Validate / validate-omos-with-pi (push) Failing after 8m29s
Item A — LAN access (base image): - New rootfs/usr/local/lib/opencode-devbox/setup-lan-access.sh, invoked non-fatally from entrypoint-user.sh. On VM-backed hosts (macOS OrbStack / Docker Desktop, detected via host.docker.internal) it generates a writable ~/.ssh-local/config that uses the host as an SSH jump to reach LAN peers; no-op on native Linux. Ships the mechanism (generic 'host' jump alias), not policy (targets stay in the user's bind-mounted ~/.ssh/config). - New env knobs: DEVBOX_LAN_ACCESS (auto|jump|off), HOST_SSH_USER, DEVBOX_HOST_ALIAS. dssh/dscp aliases in .bash_aliases (guarded). Item B — pi-fork (fork) + pi-observational-memory (recall) in pi variants: - Dockerfile.variant clones both elpapi42 repos to /opt and runs npm install there at build time (local-path 'pi install' does not npm-install, so deps must be present to load). New args PI_FORK_REPO/REF, PI_OBSMEM_REPO/REF. - entrypoint-user.sh registers them at runtime via 'pi install /opt/<pkg>' (instant, in-place, idempotent; tools bind on next pi start). - CI resolve-versions resolves each repo's master HEAD to a commit SHA and passes PI_FORK_REF/PI_OBSMEM_REF — same cache-hit guard as PI_VERSION. - smoke-test asserts /opt clones + node_modules + settings.json registration; size thresholds bumped (with-pi 2700->2900, omos-with-pi 3700->3900). Versions unchanged (opencode 1.15.13, pi 0.78.0 — both still latest). Docs: README LAN section + env table, .env.example, AGENTS.md, CHANGELOG. Plan recorded in docs/plan-lan-access-and-pi-extensions.md.
This commit is contained in:
@@ -0,0 +1,218 @@
|
||||
# Plan: LAN-access mechanism + pi-fork/pi-observational-memory in the builds
|
||||
|
||||
Status: PROPOSED (2026-06-03, decisions folded in). Author: pi (devbox session).
|
||||
Scope: opencode-devbox base + variant, pi-devbox. Two independent work items.
|
||||
|
||||
---
|
||||
|
||||
## Layering decision
|
||||
|
||||
| Capability | Lives in | Why |
|
||||
|---|---|---|
|
||||
| **LAN-access (smart-detect host-jump)** | opencode-devbox **base** | Both opencode-devbox and pi-devbox inherit it; not pi-specific. |
|
||||
| **pi-fork + pi-observational-memory** | **pi layer** (variant `with-pi`/`omos-with-pi` + pi-devbox/Dockerfile) | Only meaningful when `pi` is present. Runtime deploy via the shared base `entrypoint-user.sh`, guarded by `command -v pi`. |
|
||||
|
||||
Guiding principle for LAN access: **ship the mechanism, not the policy.**
|
||||
The image provides a generic `host` jump alias + writable SSH config + detection.
|
||||
A user's *specific* targets (e.g. pve/pve-2) come from their bind-mounted
|
||||
`~/.ssh/config` (`ProxyJump host`) or an env list — never hardcoded in the image.
|
||||
|
||||
---
|
||||
|
||||
## ITEM A — LAN access (opencode-devbox base)
|
||||
|
||||
### Why it can't "just work" unattended
|
||||
- macOS (OrbStack / Docker Desktop): container is in a Linux VM behind the host's
|
||||
stack. Directly-attached LAN peers are not bridged by default; only the host +
|
||||
routed subnets are reachable.
|
||||
- Linux Docker: default bridge already NATs container egress onto the host's LAN,
|
||||
so LAN peers are usually directly reachable. The jump is unnecessary.
|
||||
- The jump path needs the host running sshd + the container's pubkey authorized.
|
||||
The average DockerHub t"kick the tires" user has neither → setup must be
|
||||
**opt-in / non-fatal**, never block startup.
|
||||
|
||||
### New file: `rootfs/usr/local/lib/opencode-devbox/setup-lan-access.sh`
|
||||
COPY'd automatically (base already does `COPY rootfs/usr/local/lib/opencode-devbox/`).
|
||||
|
||||
Behavior, driven by `DEVBOX_LAN_ACCESS=auto|jump|off` (default `auto`):
|
||||
|
||||
1. `off` → return immediately.
|
||||
2. Detect environment:
|
||||
- VM-backed Docker (OrbStack / Docker Desktop) iff `getent hosts host.docker.internal`
|
||||
resolves (OrbStack also exposes `host.orb.internal`). Native Linux → no resolution
|
||||
(unless the user added `extra_hosts: host.docker.internal:host-gateway`).
|
||||
3. `auto` + native Linux → do nothing (direct LAN works); print one info line.
|
||||
4. `auto` + VM-backed, or `jump` forced →
|
||||
- Create writable `~/.ssh-local/{,cm/}`, `chmod 700`.
|
||||
- Generate `~/.ssh-local/devbox_jump_ed25519` if absent (preserve across restarts).
|
||||
- Render `~/.ssh-local/config`:
|
||||
```
|
||||
Host *
|
||||
UserKnownHostsFile ~/.ssh-local/known_hosts
|
||||
StrictHostKeyChecking accept-new
|
||||
Host host mac # 'mac' kept as friendly alias
|
||||
HostName host.docker.internal
|
||||
User ${HOST_SSH_USER} # REQUIRED for auth; see below
|
||||
IdentityFile ~/.ssh-local/devbox_jump_ed25519
|
||||
IdentitiesOnly yes
|
||||
ControlMaster auto
|
||||
ControlPath ~/.ssh-local/cm/%r@%h:%p
|
||||
ControlPersist 4h
|
||||
# Optional per-target blocks generated from DEVBOX_LAN_HOSTS (see below)
|
||||
Include ~/.ssh/config # user's bind-mounted targets still resolve
|
||||
```
|
||||
- If `HOST_SSH_USER` unset → still render config but print a clear hint block:
|
||||
the generated **public key** + the one-liner to authorize it on the host
|
||||
(`echo '<pubkey>' >> ~/.ssh/authorized_keys`) + "enable Remote Login".
|
||||
- Idempotent: re-render config each start (cheap); never regenerate the key.
|
||||
- DECISION #5: NO `DEVBOX_LAN_HOSTS` env. Keep the image policy-free. Users add
|
||||
`ProxyJump host` to their own target entries in the bind-mounted `~/.ssh/config`
|
||||
(pulled in by the `Include ~/.ssh/config` line).
|
||||
|
||||
### `entrypoint-user.sh`
|
||||
Call `setup-lan-access.sh` right after the existing `/tmp/sshcm` block
|
||||
(non-fatal: `… || true`). It's environment-gated so it self-skips on Linux.
|
||||
|
||||
### `rootfs/home/developer/.bash_aliases` (per your note — alias goes HERE)
|
||||
Append, guarded:
|
||||
```bash
|
||||
# dssh — ssh using the container's writable LAN-access config (host-jump).
|
||||
# Only useful when setup-lan-access.sh generated ~/.ssh-local/config.
|
||||
if [ -r "$HOME/.ssh-local/config" ]; then
|
||||
alias dssh='ssh -F "$HOME/.ssh-local/config"'
|
||||
alias dscp='scp -F "$HOME/.ssh-local/config"'
|
||||
fi
|
||||
```
|
||||
Migration caveat: skel `.bash_aliases` is only copied when absent, so existing
|
||||
volumes/containers won't get `dssh` until they `rm ~/.bash_aliases` and recreate,
|
||||
OR drop the alias into the host-shared `~/.config/devbox-shell/bash_aliases`
|
||||
(already sourced at the top of the skel file).
|
||||
|
||||
### Dockerfile.base
|
||||
No structural change required (script ships via existing rootfs COPY). Optionally
|
||||
document `DEVBOX_LAN_ACCESS` / `HOST_SSH_USER` / `DEVBOX_LAN_HOSTS` in `.env.example`
|
||||
and README.
|
||||
|
||||
---
|
||||
|
||||
## ITEM B — pi-fork + pi-observational-memory (pi layer)
|
||||
|
||||
Sources (pinned this week):
|
||||
- `github.com/elpapi42/pi-fork` (registers `fork`; ~v0.1.0)
|
||||
- `github.com/elpapi42/pi-observational-memory` (registers `recall`; default branch **master**, v3.0.2)
|
||||
|
||||
### B1 RESOLVED (verified live 2026-06-03 in this container)
|
||||
- `pi install <local-path>` is INSTANT (~0.5s): NO copy, NO npm install. pi registers
|
||||
the path and loads the extension IN PLACE from that dir.
|
||||
- settings.json stores a RELATIVE path (e.g. `../../../opt/pi-fork` from ~/.pi/agent).
|
||||
Points into the image-layer `/opt` → stable across volume recreate. Good.
|
||||
- Idempotent: a second `pi install <same path>` does NOT duplicate the entry.
|
||||
- CONSEQUENCE: because pi does NOT npm-install a local path, deps must already exist
|
||||
at `/opt/<pkg>/node_modules`. pi-fork imports `@sinclair/typebox` + `@earendil-works/*`
|
||||
peers; git-install produced a 148 MB node_modules. So we MUST `npm install` inside
|
||||
each `/opt/<pkg>` AT BUILD TIME.
|
||||
- BAKE RECIPE: clone to /opt -> `npm install` there (build) -> `pi install /opt/<pkg>`
|
||||
at runtime (instant, idempotent).
|
||||
- (Optional size win, verify-first: prune to external-only deps if pi provides the
|
||||
`@earendil-works/*` peers from its own runtime resolution. ~148M is mostly those.)
|
||||
|
||||
### DECISION #3: refactor to remove duplication
|
||||
`pi-devbox/Dockerfile` currently duplicates the pi-install + /opt-clone logic from
|
||||
`Dockerfile.variant`. Refactor `pi-devbox/Dockerfile` to `FROM` the `with-pi` variant
|
||||
image so pi-install logic (incl. the new fork/obsmem clones) lives in ONE place.
|
||||
|
||||
### Build time — clone to /opt + npm install (mirror pi-toolkit/extensions pattern)
|
||||
Add to the single `INSTALL_PI=true` block in `opencode-devbox/Dockerfile.variant`
|
||||
(after refactor, pi-devbox inherits it):
|
||||
```dockerfile
|
||||
ARG PI_FORK_REPO=https://github.com/elpapi42/pi-fork.git
|
||||
ARG PI_FORK_REF=<pin: tag or commit SHA>
|
||||
ARG PI_OBSMEM_REPO=https://github.com/elpapi42/pi-observational-memory.git
|
||||
ARG PI_OBSMEM_REF=master # pin to SHA in CI to dodge cache-hit footgun
|
||||
# ... inside the INSTALL_PI / pi-install RUN, after the pi-toolkit/extensions clones:
|
||||
git_clone_retry "$PI_FORK_REPO" "$PI_FORK_REF" /opt/pi-fork && \
|
||||
git_clone_retry "$PI_OBSMEM_REPO" "$PI_OBSMEM_REF" /opt/pi-observational-memory && \
|
||||
(cd /opt/pi-fork && npm install --no-audit --no-fund) && \
|
||||
(cd /opt/pi-observational-memory && npm install --no-audit --no-fund) && \
|
||||
echo "pi-fork at $(cd /opt/pi-fork && git rev-parse --short HEAD)" && \
|
||||
echo "pi-obsmem at $(cd /opt/pi-observational-memory && git rev-parse --short HEAD)"
|
||||
```
|
||||
NOTE: `git_clone_retry` uses `--branch "$ref"`, which accepts tags & branches but
|
||||
NOT arbitrary commit SHAs. For SHA pinning use `git clone <url> <dest> && git -C
|
||||
<dest> checkout <sha>` for these two repos.
|
||||
|
||||
### Why not bake the install result
|
||||
`~/.pi` is a named volume mounted at runtime — anything `pi install`'d into
|
||||
`~/.pi/agent/...` at BUILD time is hidden by the volume. Same reason
|
||||
pi-toolkit/extensions deploy at runtime via `entrypoint-user.sh`. So:
|
||||
|
||||
### Runtime deploy — `entrypoint-user.sh` (shared base, in the `command -v pi` block)
|
||||
After the pi-extensions `install.sh` call, add an idempotent install of each /opt pkg:
|
||||
```bash
|
||||
for pkg in /opt/pi-fork /opt/pi-observational-memory; do
|
||||
[ -d "$pkg" ] || continue
|
||||
name=$(basename "$pkg")
|
||||
# skip if already registered in settings.json packages
|
||||
if ! grep -q "$name" "$HOME/.pi/agent/settings.json" 2>/dev/null; then
|
||||
(cd "$HOME" && pi install "$pkg") || echo "WARN: pi install $name failed (continuing)"
|
||||
fi
|
||||
done
|
||||
```
|
||||
`fork` + `recall` tools register on the NEXT pi start after deploy (exts bind at
|
||||
startup). First deploy after a volume recreate pays an `npm install` cost
|
||||
(pi-fork pulls ~133 deps) — acceptable, one-time per volume lifetime.
|
||||
|
||||
OPEN ITEM B1 (verify before finalizing): exact `pi install <local-path>` semantics
|
||||
— does it copy/symlink, and does it npm-install at run each time? If it re-resolves
|
||||
deps every start, pre-populate `/opt/<pkg>/node_modules` at build (`npm install
|
||||
--omit=dev`) and confirm the runtime install reuses it. Quick test in this container:
|
||||
`pi install /opt/pi-fork` twice, observe settings.json + timing + tool registration.
|
||||
|
||||
### CI — `.gitea/workflows/docker-publish-split.yml` (DECISION #2: latest-but-pinned)
|
||||
- USE LATEST CONTENT, BUT RESOLVE TO A SHA IN CI (same pattern as PI_VERSION/OMOS).
|
||||
The existing `resolve-versions` job curls npm `latest` for pi/omos to defeat the
|
||||
build-arg cache-hit footgun. Add an analogous resolve for the two git repos:
|
||||
query the GitHub API for the HEAD commit SHA of the tracked branch (master) and
|
||||
pass it as `PI_FORK_REF` / `PI_OBSMEM_REF` build-args, so the layer hash changes
|
||||
when upstream moves AND we still get newest-at-build-time.
|
||||
- Passing a bare branch name would be byte-identical across builds -> stale cached
|
||||
layer (the documented footgun). SHA resolution fixes both.
|
||||
- Pass the new build-args in the `with-pi` and `omos-with-pi` build steps.
|
||||
- The resolved SHAs print in build logs (and ideally as image labels) so a bad
|
||||
upstream is diagnosable and we can pin back to a known-good SHA.
|
||||
|
||||
### Version coupling risk (carry-over from prior session)
|
||||
pi-fork/obsmem extensions are coupled to the host pi version (AGENTS.md warns).
|
||||
pi-fork had a `fix/effort-string-enum-schema` branch from recent API churn. So:
|
||||
- Pin against the SAME `PI_VERSION` the image ships.
|
||||
- smoke-test must assert the tools actually register (below), not just that files exist.
|
||||
|
||||
### Smoke test — `scripts/smoke-test.sh`
|
||||
Add (for `with-pi`/`omos-with-pi`/pi-devbox):
|
||||
1. `/opt/pi-fork/package.json` and `/opt/pi-observational-memory/package.json` exist.
|
||||
2. Run a container, then assert `~/.pi/agent/settings.json` "packages" includes both.
|
||||
3. Best-effort: headless `pi` tool-list contains `fork` and `recall` (if pi exposes a
|
||||
non-interactive list; otherwise step 2 is the gate).
|
||||
|
||||
---
|
||||
|
||||
## Decisions — RESOLVED 2026-06-03
|
||||
1. **B1**: VERIFIED. Local-path install is instant/in-place; bake `npm install` into
|
||||
`/opt/<pkg>` at build; runtime `pi install /opt/<pkg>` is instant + idempotent. ✓
|
||||
2. **Latest-but-pinned**: track latest (master HEAD), resolve to SHA in CI build-arg. ✓
|
||||
3. **Refactor**: pi-devbox/Dockerfile -> `FROM` the with-pi variant; pi-install in ONE place. ✓
|
||||
4. **LAN default** `DEVBOX_LAN_ACCESS=auto`: generate config + print authorize hint when
|
||||
`HOST_SSH_USER` unset; silent no-op on native Linux. ✓
|
||||
5. **No `DEVBOX_LAN_HOSTS`**: rely on user's bind-mounted `~/.ssh/config` (`ProxyJump host`). ✓
|
||||
|
||||
## Remaining verify-before-merge items
|
||||
- Confirm the fork/recall extensions LOAD at runtime from `/opt/<pkg>` WITH the baked
|
||||
node_modules (smoke test asserts tool registration, not just files).
|
||||
- Optional: confirm whether pi supplies `@earendil-works/*` peers at runtime so /opt
|
||||
node_modules can be pruned to external-only deps (size optimization, ~148M -> small).
|
||||
|
||||
## Rollout order
|
||||
1. Verify B1 in this live container (cheap, no build).
|
||||
2. Land ITEM A in base (rootfs script + entrypoint call + alias) → rebuild base → smoke.
|
||||
3. Land ITEM B in variant + pi-devbox + CI resolve + smoke assertions.
|
||||
4. CHANGELOG + tag both repos; CI rebuild; verify fork+recall+dssh survive a volume recreate.
|
||||
Reference in New Issue
Block a user