ship: don't trust the mtime the exporter deliberately backdates
rsync --update skips a file whose mtime is not strictly newer than the receiver's. The stage file's mtime IS the source transcript's mtime (os.utime() at :903, "preserve session mtime for dedup stability"), so re-exporting a session that has not been appended to since its last ship produces a mtime that is not newer than what's already at the receiver — exactly the case a redactor upgrade needs to ship, because content differs while mtime does not. --update reports success and sends nothing. Reported and patched by pi@mbp-m1-2020 (evt_20260827T211925_9674a31da0b4, artifact art_20260827T211839_fa52af563105, sha256 f217e47e…), measured live: a scrubbed re-export of a dormant session (pi_01a03022-…f542) sat unshipped in the palace host's inbox while every local signal reported a clean stage, saved only because a host-side sweep happened to rewrite the remote copy independently that same day. --checksum compares content and ignores size/mtime entirely. Dropping --update outright was considered and rejected: rsync's default quick check already transfers on a SIZE difference alone, which is why the observed case (33-byte placeholder vs a 43-byte token) would have been masked as "fixed" by a change that only works until a redaction whose placeholder happens to match the secret's length. os.utime() at :903 is untouched — its backdating is a separate, load-bearing design call for dedup stability, out of scope for this fix. Added scripts/test-rsync-ship-idempotency.sh: ships a file, rewrites its content to an EQUAL-LENGTH string while restoring the original mtime (what os.utime() does), ships again, asserts the receiver's sha256 changed. Equal length is deliberate, not cosmetic — mismatched lengths would pass via the quick check alone and prove nothing about --checksum specifically; this is the same reasoning that ruled out dropping --update. Verified the test discriminates: passes against today's --checksum, fails against --update (checked by temporarily substituting the flag in a copy, not committed). Runs offline — a local rsync destination path exercises the same size/mtime/checksum comparison as the ssh transfer, no palace or network needed.
This commit is contained in:
@@ -974,15 +974,26 @@ fi
|
||||
|
||||
# ── Ship to the palace host (remote mode only) ───────────────────────
|
||||
# mempalace_mine expands its source path in the SERVER process, so in remote
|
||||
# mode the exports have to physically exist over there. rsync --update is the
|
||||
# mode the exports have to physically exist over there. --checksum is the
|
||||
# idempotent half; the mine is the other half.
|
||||
#
|
||||
# NOT --update: the stage file's mtime is deliberately the SOURCE transcript's
|
||||
# mtime (see the os.utime() in the exporter, "preserve session mtime for dedup
|
||||
# stability"), so a re-export of a session that has not been appended to since
|
||||
# the last ship carries an mtime that is NOT newer than the receiver copy. With
|
||||
# --update rsync then SKIPS it silently -- which is exactly the case that must
|
||||
# ship after a redactor change, because the content differs while the mtime does
|
||||
# not. Measured on mbp-m1-2020 2026-08-27: a scrubbed re-export of a dormant
|
||||
# session was skipped and unscrubbed bytes stayed in the palace host's inbox,
|
||||
# while every local signal reported a clean stage. --checksum compares content
|
||||
# and keeps the ship idempotent without trusting timestamps.
|
||||
MINE_SOURCE="$STAGE"
|
||||
if [[ "$MODE" == "remote" ]]; then
|
||||
ssh_cmd="ssh"
|
||||
[[ -n "$SSH_CONFIG" ]] && ssh_cmd="ssh -F $SSH_CONFIG"
|
||||
echo ""
|
||||
echo "Shipping stage to ${SSH_TARGET%/}/$DEVICE/ ..."
|
||||
if ! rsync -a --update --no-owner --no-group \
|
||||
if ! rsync -a --checksum --no-owner --no-group \
|
||||
-e "$ssh_cmd" \
|
||||
--include='*.jsonl' --exclude='*' \
|
||||
"$STAGE/" "${SSH_TARGET%/}/$DEVICE/"; then
|
||||
|
||||
Reference in New Issue
Block a user