Backups: stop storing credentials, and fix a dedup chain that never matched

The NTLARS/DNC registry carries an FTP password in cleartext and it was stored
verbatim. That put it in the backuprevisions table, on the revision panel which
renders values, and in the download route which hands back a ready-to-import
.reg - so anyone holding backups.view could read it off a web page.

Redaction happens at PARSE, so the secret never reaches the database at all.
Redacting on display would have left it at rest and one query away.

What is deliberately NOT done is dropping the value silently. This codec already
refuses to discard an unparseable value, precisely because a backup that looks
complete and restores a machine with a setting missing is the worst outcome. A
redacted value keeps its name, is listed in the projection, and renders as a
COMMENT naming what must be set by hand. Emitting a placeholder as the password
would produce a machine that looks restored and cannot connect, and the
technician would go debugging the network.

scripts/redact_backup_secrets.py scrubs revisions recorded before this, dry run
by default. It leaves contenthash alone: the hash identifies the configuration a
bay reported, so rewriting it would make every bay's next post look like a
change and insert a duplicate revision across the fleet.

Separately, sourcehostname is now normalized at ingest. samesource() maps '' to
IS NULL for the query while the insert stored the raw '', so rows written by a
client posting an empty hostname could never match their own next post: every
post inserted, the table grew without bound, and the history filled with
identical entries nobody could read. Whitespace did the same thing more quietly.
This commit is contained in:
cproudlock
2026-08-26 15:06:04 -04:00
parent e60d6b74d8
commit dffa2082be
5 changed files with 325 additions and 6 deletions

View File

@@ -410,7 +410,13 @@ class BackupsPlugin(BasePlugin):
# with_for_update serializes concurrent posts for the same chain: the
# fleet collects on one GE-Enforce cycle, so a retry could otherwise
# pass the hash check twice and insert the same revision.
sourcehostname = payload.get('sourcehostname')
# Normalize ONCE, and use the same value to query and to store.
# samesource() maps '' to IS NULL, but the insert below used to store
# the raw '' - so a client posting an empty hostname wrote rows that its
# own next post could never match. Every post then deduped against
# nothing, inserting forever: unbounded growth, and a history no
# technician can read. Whitespace does the same thing more quietly.
sourcehostname = (payload.get('sourcehostname') or '').strip() or None
latest = (db.session.query(BackupRevision)
.filter(BackupRevision.assetid == assetid,
BackupRevision.backupkind == kindkey,
@@ -476,7 +482,7 @@ class BackupsPlugin(BasePlugin):
sharepath=sharepath,
sourcefilename=sourcefilename,
bytesize=bytesize,
sourcehostname=payload.get('sourcehostname'),
sourcehostname=sourcehostname,
collectedat=collectedat or datetime.utcnow(),
lastseenat=datetime.utcnow(),
)