feed: chase the symlink, or fail-closed takes the whole fleet's feed down
The image installs /usr/local/bin/mempalace-pi-session as a symlink into
/opt/mempalace-toolkit/bin, and ${BASH_SOURCE[0]} reports the path the script was
INVOKED as, not the resolved target. So the sibling-module lookup pointed at
/usr/local/bin, the redactor was not there, and the fail-closed import did exactly
what it was told: refused to stage.
MEASURED, not theorised: /usr/local/bin/mempalace-pi-session --dry-run exited
"[FATAL] secret scrubber unavailable ... refusing to stage" while
/opt/mempalace-toolkit/bin/mempalace-pi-session --dry-run scrubbed 40 findings on
the same input. The symlink is how every device invokes it, so at the next image
bake every feeder tick on every machine would have stopped staging — a silent,
fleet-wide memory outage, which is a worse outcome than the leak the scrubber
exists to prevent. My own tests missed it by calling bin/... directly from the
checkout, i.e. the one invocation path the fleet never uses.
FIX: chase the symlink chain in portable shell and offer fallbacks instead of
betting on a single answer. MEMPALACE_REDACT_DIR is now colon-separated —
resolved dir, invoked dir, then the image's known install path /opt/... — and the
Python side inserts the first existing candidate. readlink -f is deliberately NOT
used: it is GNU/newer-BSD only and this script also runs directly on macOS hosts,
so the chase is a plain while [ -L ] loop handling relative link targets.
Verified on all three invocation shapes: via the /usr/local/bin symlink, via the
direct /opt path, and via a second-hop symlink in an unrelated directory. All
three now report the same 40 redactions.
LESSON worth keeping: fail-closed is correct for a secret scrubber, but it
converts "module not found" into an outage, so the module lookup becomes
load-bearing infrastructure and must be tested through the real invocation path,
not the convenient one.
This commit is contained in:
@@ -536,7 +536,31 @@ fi
|
|||||||
# `mempalace mine --mode convos` is still the authoritative dedup.
|
# `mempalace mine --mode convos` is still the authoritative dedup.
|
||||||
# The redactor is a sibling module, imported by the heredoc below. Exported
|
# The redactor is a sibling module, imported by the heredoc below. Exported
|
||||||
# rather than passed as argv so the argv unpack stays stable.
|
# rather than passed as argv so the argv unpack stays stable.
|
||||||
MEMPALACE_REDACT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
#
|
||||||
|
# ${BASH_SOURCE[0]} reports the path the script was INVOKED as, and the image
|
||||||
|
# installs /usr/local/bin/mempalace-pi-session as a symlink into
|
||||||
|
# /opt/mempalace-toolkit/bin. A naive dirname therefore yields /usr/local/bin,
|
||||||
|
# where the redactor does not exist — and because the import is fail-closed, that
|
||||||
|
# turns every feeder tick on every device into "refusing to stage". Measured: the
|
||||||
|
# symlinked invocation exited FATAL while the direct one worked, i.e. it would
|
||||||
|
# have stopped the whole fleet's memory feed at the next image bake. So chase the
|
||||||
|
# symlink chain, and offer fallbacks rather than betting on one answer.
|
||||||
|
#
|
||||||
|
# readlink -f is avoided deliberately: it is GNU/newer-BSD only, and this script
|
||||||
|
# also runs directly on macOS hosts.
|
||||||
|
_mp_resolve() {
|
||||||
|
local p="$1" target
|
||||||
|
while [ -L "$p" ]; do
|
||||||
|
target="$(readlink "$p")" || break
|
||||||
|
case "$target" in
|
||||||
|
/*) p="$target" ;;
|
||||||
|
*) p="$(dirname "$p")/$target" ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
printf '%s' "$p"
|
||||||
|
}
|
||||||
|
_mp_self="$(_mp_resolve "${BASH_SOURCE[0]}")"
|
||||||
|
MEMPALACE_REDACT_DIR="$(cd "$(dirname "$_mp_self")" && pwd):$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd):/opt/mempalace-toolkit/bin"
|
||||||
export MEMPALACE_REDACT_DIR
|
export MEMPALACE_REDACT_DIR
|
||||||
export_count=$(python3 - "$PI_SESSIONS_DIR" "$STAGE" "$SESSION_ID" "$SINCE" "$MIN_MESSAGES" "$MIN_ASSISTANT_CHARS" "$MODE" <<'PY'
|
export_count=$(python3 - "$PI_SESSIONS_DIR" "$STAGE" "$SESSION_ID" "$SINCE" "$MIN_MESSAGES" "$MIN_ASSISTANT_CHARS" "$MODE" <<'PY'
|
||||||
import json, os, sqlite3, sys
|
import json, os, sqlite3, sys
|
||||||
@@ -553,7 +577,11 @@ from pathlib import Path
|
|||||||
# point of this step is that a secret must not reach a shared palace. Override
|
# point of this step is that a secret must not reach a shared palace. Override
|
||||||
# deliberately with MEMPALACE_FEED_ALLOW_UNSCRUBBED=1 if you ever need to feed a
|
# deliberately with MEMPALACE_FEED_ALLOW_UNSCRUBBED=1 if you ever need to feed a
|
||||||
# machine whose toolkit is older than this file.
|
# machine whose toolkit is older than this file.
|
||||||
sys.path.insert(0, os.environ.get("MEMPALACE_REDACT_DIR", ""))
|
# Colon-separated candidates: symlink-resolved dir, invoked dir, then the image's
|
||||||
|
# known install path. First one that has the module wins.
|
||||||
|
for _cand in os.environ.get("MEMPALACE_REDACT_DIR", "").split(":"):
|
||||||
|
if _cand and os.path.isdir(_cand):
|
||||||
|
sys.path.insert(0, _cand)
|
||||||
try:
|
try:
|
||||||
import mempalace_redact as _redact
|
import mempalace_redact as _redact
|
||||||
except Exception as _e: # noqa: BLE001 - any import failure is fatal by design
|
except Exception as _e: # noqa: BLE001 - any import failure is fatal by design
|
||||||
|
|||||||
Reference in New Issue
Block a user