backups plugin: per-asset config backups with revision history
Adds a kind-pluggable backups plugin. Configuration captured from a PC is filed against the MACHINE it controls, with a revision history and download back to the native format. NTLARS/DNC is the first kind. Settings live in the controlling PC's registry but describe the machine, so revisions attach to the machine's asset and carry no foreign key to the PC: history survives a PC being replaced or deleted, and sourcehostname records the handover. Storage splits by kind. Parseable kinds store a dialect-neutral JSON projection in ShopDB and re-render on download; opaque vendor formats (part marker and similar) keep their bytes on the SFLD share with ShopDB holding metadata and the UNC pointer. Two .reg dialects exist in the wild: NTLARS's own Save... export omits the WOW6432Node path segment, scripted exports include it. Parsing strips whichever root matched, so a stored revision commits to neither and download offers both (NTLARS Load... by default, WOW6432Node for direct reg import). Getting this backwards is silent, so the dedup hash deliberately excludes sourcedialect and both dialects of one config dedup to a single revision. Dedup is load-bearing: the collector runs every GE-Enforce cycle across the fleet, so a revision is inserted only when the content hash differs from that asset's latest for that kind. A freshly imaged PC opens NTLARS with a blank General tab. Recording that would make an empty config the newest revision exactly when someone needs the last good one, so a blank MachineNo is rejected rather than accepted as a change. Two of the 320 known-good backups on the share already have that shape. DNC Info card summarises the latest revision on the machine page: General (Cnc, NcIF, HostType), eFocas, Serial, NTSHR when populated (only 18 of 147 machines), and MARK when the machine is a marker. MARK is gated on Cnc=MARKER or the ShopDB machine type, not on the MARK key having content: MARK carries serial defaults on 145 of 147 machines and DncPatterns reads YES on 103 including ordinary lathes, so neither identifies a marker. The info card is owned by the kind (BackupKind.infopanel/buildinfo) and served by a generic endpoint, so the expected successor to DNC ships its own card by adding a class rather than changing the plugin or the panel wiring. Also: schedule and retention settings with a prune that never drops the newest or the oldest revision, and scripts/import_ntlars_backups.py to seed history from the existing per-machine .reg files (144 of 147 resolve to assets). Codec verified against all 320 real backups: round-trips clean through both dialects. Bay-side generation verified on Windows against reg.exe export.
This commit is contained in:
137
scripts/import_ntlars_backups.py
Normal file
137
scripts/import_ntlars_backups.py
Normal file
@@ -0,0 +1,137 @@
|
||||
#!/usr/bin/env python
|
||||
"""Seed backup history from a directory of existing NTLARS .reg files.
|
||||
|
||||
The share already holds ~150 per-machine backups collected by hand over years
|
||||
(S:\\DT\\RegFiles\\Dnc\\Backup Copies\\<machinenumber>.reg and the pxe-images
|
||||
mirrors). Importing them gives every machine a baseline revision on day one,
|
||||
so the feature is useful before the fleet collector has run even once.
|
||||
|
||||
Files are keyed by machine number in the FILENAME, which is how the share names
|
||||
them. The MachineNo embedded in the file is compared against it and any
|
||||
disagreement is reported: in the curated set the two always agree, so a
|
||||
mismatch means the file is misfiled and should not be trusted as that machine's
|
||||
baseline.
|
||||
|
||||
Dry run by default. Nothing is written without --commit.
|
||||
|
||||
venv/bin/python scripts/import_ntlars_backups.py /home/camp/pxe-images/ntlars-deploy
|
||||
venv/bin/python scripts/import_ntlars_backups.py /path/to/regs --commit
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import sys
|
||||
from datetime import datetime
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from shopdb import create_app # noqa: E402
|
||||
from shopdb.api import db # noqa: E402
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('directory', help='directory holding <machinenumber>.reg files')
|
||||
parser.add_argument('--commit', action='store_true',
|
||||
help='actually write (default is a dry run)')
|
||||
parser.add_argument('--hostname', default='imported',
|
||||
help="sourcehostname to record (default 'imported')")
|
||||
args = parser.parse_args()
|
||||
|
||||
if not os.path.isdir(args.directory):
|
||||
parser.error('not a directory: {}'.format(args.directory))
|
||||
|
||||
regs = sorted(f for f in os.listdir(args.directory) if f.lower().endswith('.reg'))
|
||||
if not regs:
|
||||
parser.error('no .reg files in {}'.format(args.directory))
|
||||
|
||||
app = create_app()
|
||||
with app.app_context():
|
||||
from shopdb.core.models import Asset
|
||||
from plugins.backups.models import BackupRevision
|
||||
from plugins.backups.services.registry import getkind, canonicalhash
|
||||
|
||||
kind = getkind('ntlars')
|
||||
|
||||
created = skipped = unresolved = rejected = duplicate = 0
|
||||
mismatches = []
|
||||
|
||||
for filename in regs:
|
||||
path = os.path.join(args.directory, filename)
|
||||
machinenumber = os.path.splitext(filename)[0]
|
||||
|
||||
with open(path, 'rb') as handle:
|
||||
raw = handle.read()
|
||||
|
||||
try:
|
||||
projection = kind.parse(raw)
|
||||
except ValueError as exc:
|
||||
# Unconfigured NTLARS installs are in the corpus; they are not
|
||||
# a usable baseline for anyone.
|
||||
print(' REJECT {:<12} {}'.format(machinenumber, exc))
|
||||
rejected += 1
|
||||
continue
|
||||
|
||||
embedded = kind.embeddedmachineno(projection)
|
||||
if embedded and embedded != machinenumber:
|
||||
mismatches.append((machinenumber, embedded))
|
||||
|
||||
asset = db.session.query(Asset).filter(
|
||||
Asset.assetnumber == machinenumber).first()
|
||||
if asset is None:
|
||||
print(' NOASSET {:<12} no asset with that assetnumber'.format(
|
||||
machinenumber))
|
||||
unresolved += 1
|
||||
continue
|
||||
|
||||
contenthash = canonicalhash(projection)
|
||||
latest = (db.session.query(BackupRevision)
|
||||
.filter(BackupRevision.assetid == asset.assetid,
|
||||
BackupRevision.backupkind == 'ntlars')
|
||||
.order_by(BackupRevision.backuprevisionid.desc())
|
||||
.first())
|
||||
if latest is not None and latest.contenthash == contenthash:
|
||||
duplicate += 1
|
||||
continue
|
||||
|
||||
if args.commit:
|
||||
revision = BackupRevision(
|
||||
assetid=asset.assetid,
|
||||
backupkind='ntlars',
|
||||
storagebackend='shopdb',
|
||||
contenthash=contenthash,
|
||||
sourcefilename=filename,
|
||||
bytesize=len(raw),
|
||||
sourcehostname=args.hostname,
|
||||
# The file's mtime is the closest thing to when the config
|
||||
# was actually captured; createdat records the import.
|
||||
collectedat=datetime.utcfromtimestamp(os.path.getmtime(path)),
|
||||
)
|
||||
revision.payload = projection
|
||||
db.session.add(revision)
|
||||
created += 1
|
||||
|
||||
if args.commit:
|
||||
db.session.commit()
|
||||
|
||||
print()
|
||||
print('files : {}'.format(len(regs)))
|
||||
print('would create : {}'.format(created) if not args.commit
|
||||
else 'created : {}'.format(created))
|
||||
print('already current : {}'.format(duplicate))
|
||||
print('no matching asset: {}'.format(unresolved))
|
||||
print('rejected (blank) : {}'.format(rejected))
|
||||
if mismatches:
|
||||
print()
|
||||
print('filename/embedded MachineNo disagreements ({}):'.format(
|
||||
len(mismatches)))
|
||||
for filename_no, embedded_no in mismatches[:20]:
|
||||
print(' file={:<10} embedded={}'.format(filename_no, embedded_no))
|
||||
print(' These are likely misfiled; review before trusting them.')
|
||||
if not args.commit:
|
||||
print()
|
||||
print('DRY RUN - nothing written. Re-run with --commit to apply.')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
Reference in New Issue
Block a user