ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Backups: stop storing credentials, and fix a dedup chain that never matched

Browse Source 
The NTLARS/DNC registry carries an FTP password in cleartext and it was stored
verbatim. That put it in the backuprevisions table, on the revision panel which
renders values, and in the download route which hands back a ready-to-import
.reg - so anyone holding backups.view could read it off a web page.

Redaction happens at PARSE, so the secret never reaches the database at all.
Redacting on display would have left it at rest and one query away.

What is deliberately NOT done is dropping the value silently. This codec already
refuses to discard an unparseable value, precisely because a backup that looks
complete and restores a machine with a setting missing is the worst outcome. A
redacted value keeps its name, is listed in the projection, and renders as a
COMMENT naming what must be set by hand. Emitting a placeholder as the password
would produce a machine that looks restored and cannot connect, and the
technician would go debugging the network.

scripts/redact_backup_secrets.py scrubs revisions recorded before this, dry run
by default. It leaves contenthash alone: the hash identifies the configuration a
bay reported, so rewriting it would make every bay's next post look like a
change and insert a duplicate revision across the fleet.

Separately, sourcehostname is now normalized at ingest. samesource() maps '' to
IS NULL for the query while the insert stored the raw '', so rows written by a
client posting an empty hostname could never match their own next post: every
post inserted, the table grew without bound, and the history filled with
identical entries nobody could read. Whitespace did the same thing more quietly.
This commit is contained in:
cproudlock
2026-08-26 15:06:04 -04:00
parent e60d6b74d8
commit dffa2082be
5 changed files with 325 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
plugins/backups/migrations/versions/0002_backups_lastseenat.py
View File

@@ -27,8 +27,10 @@ depends_on = None
def upgrade():
columns = {c['name'] for c in
sa.inspect(op.get_bind()).get_columns('backuprevisions')}
inspector = sa.inspect(op.get_bind())
if 'backuprevisions' not in inspector.get_table_names():
return
columns = {c['name'] for c in inspector.get_columns('backuprevisions')}
if 'lastseenat' not in columns:
op.add_column('backuprevisions',
sa.Column('lastseenat', sa.DateTime(), nullable=True))
@@ -37,7 +39,9 @@ def upgrade():
def downgrade():
columns = {c['name'] for c in
sa.inspect(op.get_bind()).get_columns('backuprevisions')}
inspector = sa.inspect(op.get_bind())
if 'backuprevisions' not in inspector.get_table_names():
return
columns = {c['name'] for c in inspector.get_columns('backuprevisions')}
if 'lastseenat' in columns:
op.drop_column('backuprevisions', 'lastseenat')

10
plugins/backups/plugin.py
View File

@@ -410,7 +410,13 @@ class BackupsPlugin(BasePlugin):
# with_for_update serializes concurrent posts for the same chain: the
# fleet collects on one GE-Enforce cycle, so a retry could otherwise
# pass the hash check twice and insert the same revision.
sourcehostname = payload.get('sourcehostname')
# Normalize ONCE, and use the same value to query and to store.
# samesource() maps '' to IS NULL, but the insert below used to store
# the raw '' - so a client posting an empty hostname wrote rows that its
# own next post could never match. Every post then deduped against
# nothing, inserting forever: unbounded growth, and a history no
# technician can read. Whitespace does the same thing more quietly.
sourcehostname = (payload.get('sourcehostname') or '').strip() or None
latest = (db.session.query(BackupRevision)
.filter(BackupRevision.assetid == assetid,
BackupRevision.backupkind == kindkey,
@@ -476,7 +482,7 @@ class BackupsPlugin(BasePlugin):
sharepath=sharepath,
sourcefilename=sourcefilename,
bytesize=bytesize,
sourcehostname=payload.get('sourcehostname'),
sourcehostname=sourcehostname,
collectedat=collectedat or datetime.utcnow(),
lastseenat=datetime.utcnow(),
)

64
plugins/backups/services/ntlars.py
View File

@@ -32,6 +32,31 @@ import re
SCHEMA = 'ntlars/1'
# SECRETS ARE NOT BACKED UP.
#
# The DNC registry carries an FTP password in cleartext. Stored verbatim it
# reached three places it must not: the backuprevisions table, the revision
# panel (which renders values), and the download route (which hands back a
# ready-to-import .reg). Anyone with backups.view could read the fleet's FTP
# password off a web page.
#
# Redaction happens at PARSE, so the secret never enters the database at all.
# Redacting only on display would leave it at rest and one query away.
#
# What is deliberately NOT done: dropping the value silently. This module's
# parser refuses to discard an unparseable value precisely because a backup that
# looks complete and restores a machine with a setting missing is the worst
# outcome. A redacted value keeps its NAME and records itself in
# `redacted`, so both the panel and a restore can say plainly that this one
# field must be set by hand.
REDACTEDMARKER = '***REDACTED***'
SECRETNAMEPATTERN = re.compile(r'passw|pwd|secret|apikey|token', re.IGNORECASE)
def issecretname(name):
"""Whether a registry value name holds a credential."""
return bool(SECRETNAMEPATTERN.search(name or ''))
REGROOT = 'HKEY_LOCAL_MACHINE'
DNCPATH = r'SOFTWARE\GE Aircraft Engines\DNC'
DNCPATHWOW = r'SOFTWARE\WOW6432Node\GE Aircraft Engines\DNC'
@@ -245,6 +270,20 @@ def parse(raw):
dialect = 'wow6432node' if sawwow else ('ntlars' if sawplain else 'unknown')
# Strip credentials before the projection is returned - i.e. before anything
# persists it. Recorded by path\name so the omission is visible rather than
# discovered during a restore.
redacted = []
for path in keys:
for name, value in keys[path].items():
if not issecretname(name):
continue
if value.get('data') in ('', None):
continue # nothing to hide
keys[path][name] = {'type': value.get('type', 'REG_SZ'),
'data': REDACTEDMARKER}
redacted.append('{}\\{}'.format(path, name) if path else name)
return {
'schema': SCHEMA,
'sourcedialect': dialect,
@@ -252,6 +291,7 @@ def parse(raw):
{'path': path, 'values': dict(sorted(keys[path].items()))}
for path in sorted(keys)
],
'redacted': sorted(redacted),
}
@@ -291,6 +331,12 @@ def render(projection, dialect='ntlars', comments=None):
dialect='ntlars' omits WOW6432Node - use with the NTLARS Load... button
dialect='wow6432node' includes it - use with `reg import` on 64-bit
Redacted credentials are rendered as a comment naming the value, NOT as the
marker string. Importing '***REDACTED***' as the FTP password would leave a
machine that looks restored and cannot connect, and the technician would be
debugging the network. A commented-out value fails loudly at the one moment
somebody is looking at the file.
"""
if dialect not in ('ntlars', 'wow6432node'):
raise ValueError('unknown dialect: {!r}'.format(dialect))
@@ -302,10 +348,28 @@ def render(projection, dialect='ntlars', comments=None):
if comments:
out.append('')
redacted = projection.get('redacted') or []
if redacted:
out.append('; ---------------------------------------------------------------')
out.append('; {} credential value(s) were NOT backed up and are NOT in this'
.format(len(redacted)))
out.append('; file. Set them by hand after importing:')
for name in redacted:
out.append('; {}'.format(name))
out.append('; ---------------------------------------------------------------')
out.append('')
for key in projection.get('keys', []):
path = key.get('path', '')
out.append('[{}]'.format(root + ('\\' + path if path else '')))
for name, entry in (key.get('values') or {}).items():
# A redacted credential is commented out, never emitted as the
# marker. Importing the marker as a password produces a machine that
# looks restored and cannot log in.
if (entry or {}).get('data') == REDACTEDMARKER:
out.append('; "{}" was not backed up - set it by hand'.format(
_escape(name)))
continue
out.append(_rendervalue(name, entry))
out.append('')