Files
shopdb-flask/plugins/backups/api/routes.py
cproudlock fca775c737
Some checks failed
CI / backend (push) Failing after 8s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 7s
backups plugin: per-asset config backups with revision history
Adds a kind-pluggable backups plugin. Configuration captured from a PC is
filed against the MACHINE it controls, with a revision history and download
back to the native format.

NTLARS/DNC is the first kind. Settings live in the controlling PC's registry
but describe the machine, so revisions attach to the machine's asset and carry
no foreign key to the PC: history survives a PC being replaced or deleted, and
sourcehostname records the handover.

Storage splits by kind. Parseable kinds store a dialect-neutral JSON
projection in ShopDB and re-render on download; opaque vendor formats (part
marker and similar) keep their bytes on the SFLD share with ShopDB holding
metadata and the UNC pointer.

Two .reg dialects exist in the wild: NTLARS's own Save... export omits the
WOW6432Node path segment, scripted exports include it. Parsing strips whichever
root matched, so a stored revision commits to neither and download offers both
(NTLARS Load... by default, WOW6432Node for direct reg import). Getting this
backwards is silent, so the dedup hash deliberately excludes sourcedialect and
both dialects of one config dedup to a single revision.

Dedup is load-bearing: the collector runs every GE-Enforce cycle across the
fleet, so a revision is inserted only when the content hash differs from that
asset's latest for that kind.

A freshly imaged PC opens NTLARS with a blank General tab. Recording that would
make an empty config the newest revision exactly when someone needs the last
good one, so a blank MachineNo is rejected rather than accepted as a change.
Two of the 320 known-good backups on the share already have that shape.

DNC Info card summarises the latest revision on the machine page: General
(Cnc, NcIF, HostType), eFocas, Serial, NTSHR when populated (only 18 of 147
machines), and MARK when the machine is a marker. MARK is gated on Cnc=MARKER
or the ShopDB machine type, not on the MARK key having content: MARK carries
serial defaults on 145 of 147 machines and DncPatterns reads YES on 103
including ordinary lathes, so neither identifies a marker.

The info card is owned by the kind (BackupKind.infopanel/buildinfo) and served
by a generic endpoint, so the expected successor to DNC ships its own card by
adding a class rather than changing the plugin or the panel wiring.

Also: schedule and retention settings with a prune that never drops the newest
or the oldest revision, and scripts/import_ntlars_backups.py to seed history
from the existing per-machine .reg files (144 of 147 resolve to assets).

Codec verified against all 320 real backups: round-trips clean through both
dialects. Bay-side generation verified on Windows against reg.exe export.
2026-08-07 14:24:42 -04:00

307 lines
11 KiB
Python

"""Backups API.
Read + download only. Revisions are created exclusively through the ADR-006
collector endpoint (POST /api/collector/backups), so there is deliberately no
create route here: a hand-posted "backup" that never came off a machine would
poison the history this feature exists to provide.
Downloads differ by storage backend. 'shopdb' kinds re-render from the stored
projection, which is what lets one revision produce either .reg dialect. 'share'
kinds are not streamed - ShopDB returns the UNC path for the tech to open,
because requiring the app server to mount the SFLD share would turn a
permissions slip into an unexplained empty download.
"""
from flask import Blueprint, current_app, request, Response
from flask_jwt_extended import jwt_required
from shopdb.api import (
db, Asset,
success_response, error_response, ErrorCodes,
require_permission,
)
from ..models import BackupRevision
from ..services.registry import REGISTRY, getkind
backups_bp = Blueprint('backups', __name__)
def _label(revision):
"""Panel list title: kind-agnostic, readable at a glance."""
when = revision.collectedat or revision.createdat
stamp = when.strftime('%Y-%m-%d %H:%M') if when else 'unknown time'
if revision.sourcefilename:
return '{} - {}'.format(stamp, revision.sourcefilename)
return stamp
def _summary(revision, islatest=False):
data = revision.to_dict()
data['label'] = _label(revision)
data['islatest'] = islatest
kind = getkind(revision.backupkind)
data['formats'] = kind.formats() if kind else []
return data
@backups_bp.route('/kinds', methods=['GET'])
@jwt_required()
@require_permission('backups.view')
def list_kinds():
"""Registered backup kinds, for UI that needs to enumerate them."""
return success_response([
{
'key': kind.key,
'displayname': kind.displayname,
'storagebackend': kind.storagebackend,
'assettypes': list(kind.assettypes),
'formats': kind.formats(),
}
for kind in REGISTRY.values()
])
@backups_bp.route('/asset/<int:assetid>', methods=['GET'])
@jwt_required()
@require_permission('backups.view')
def asset_revisions(assetid):
"""Revision history for one asset, newest first.
This is the asset-panel endpoint. Optional ?kind= narrows to a single kind,
which is how each per-kind panel scopes itself.
"""
asset = db.session.get(Asset, assetid)
if asset is None:
return error_response(ErrorCodes.NOT_FOUND, 'Asset not found', http_code=404)
query = db.session.query(BackupRevision).filter(
BackupRevision.assetid == assetid)
kindkey = (request.args.get('kind') or '').strip().lower()
if kindkey:
if getkind(kindkey) is None:
return error_response(ErrorCodes.VALIDATION_ERROR,
'Unknown kind: {}'.format(kindkey))
query = query.filter(BackupRevision.backupkind == kindkey)
revisions = query.order_by(BackupRevision.backuprevisionid.desc()).all()
# "Latest" is per kind, not per asset, so an unfiltered listing still marks
# the current revision of each kind correctly.
seen = set()
out = []
for revision in revisions:
islatest = revision.backupkind not in seen
seen.add(revision.backupkind)
out.append(_summary(revision, islatest=islatest))
return success_response(out)
@backups_bp.route('/asset/<int:assetid>/info', methods=['GET'])
@jwt_required()
@require_permission('backups.view')
def asset_info(assetid):
"""A kind's 'at a glance' card, built from its LATEST revision.
Generic across kinds: the requested kind owns both the panel declaration
and the payload (see BackupKind.infopanel / buildinfo), so a successor to
NTLARS/DNC gets its own card without a new endpoint.
Empty when the machine has no revision of that kind yet, which the generic
renderer turns into the panel's empty text.
"""
kindkey = (request.args.get('kind') or 'ntlars').strip().lower()
kind = getkind(kindkey)
if kind is None:
return error_response(ErrorCodes.VALIDATION_ERROR,
'Unknown kind: {}'.format(kindkey))
revision = (db.session.query(BackupRevision)
.filter(BackupRevision.assetid == assetid,
BackupRevision.backupkind == kindkey)
.order_by(BackupRevision.backuprevisionid.desc())
.first())
if revision is None:
return success_response({'fields': [], 'sectioncount': 0})
data = kind.buildinfo(
revision.payload, assetid,
partmarkertypes=current_app.config.get('BACKUPS_PARTMARKER_TYPES'))
data['backuprevisionid'] = revision.backuprevisionid
data['collectedat'] = (revision.collectedat.isoformat()
if revision.collectedat else None)
return success_response(data)
@backups_bp.route('/revisions/<int:backuprevisionid>', methods=['GET'])
@jwt_required()
@require_permission('backups.view')
def get_revision(backuprevisionid):
"""One revision, including its parsed payload where there is one."""
revision = db.session.get(BackupRevision, backuprevisionid)
if revision is None:
return error_response(ErrorCodes.NOT_FOUND, 'Revision not found',
http_code=404)
data = revision.to_dict(includepayload=True)
data['label'] = _label(revision)
kind = getkind(revision.backupkind)
data['formats'] = kind.formats() if kind else []
data['displayname'] = kind.displayname if kind else revision.backupkind
return success_response(data)
@backups_bp.route('/revisions/<int:backuprevisionid>/download', methods=['GET'])
@jwt_required()
@require_permission('backups.download')
def download_revision(backuprevisionid):
"""Render a revision back to its native file format.
?format=ntlars (default for NTLARS) omits WOW6432Node - the form the
NTLARS Load... button expects
?format=wow6432node includes WOW6432Node - for `reg import` on 64-bit
"""
revision = db.session.get(BackupRevision, backuprevisionid)
if revision is None:
return error_response(ErrorCodes.NOT_FOUND, 'Revision not found',
http_code=404)
kind = getkind(revision.backupkind)
if kind is None:
return error_response(ErrorCodes.VALIDATION_ERROR,
'Unknown kind: {}'.format(revision.backupkind))
if revision.storagebackend == 'share':
# Not an error: the file exists, ShopDB just is not the one serving it.
return success_response({
'storagebackend': 'share',
'sharepath': revision.sharepath,
'sourcefilename': revision.sourcefilename,
'message': 'This backup lives on the SFLD share. Open the path directly.',
})
formats = kind.formats()
formatid = (request.args.get('format') or '').strip().lower()
if not formatid:
formatid = formats[0]['id'] if formats else ''
if not any(f['id'] == formatid for f in formats):
return error_response(
ErrorCodes.VALIDATION_ERROR,
'Unknown format {!r} for kind {}'.format(formatid, kind.key))
asset = db.session.get(Asset, revision.assetid)
assetnumber = asset.assetnumber if asset else str(revision.assetid)
when = revision.collectedat or revision.createdat
comments = [
'{} backup from ShopDB'.format(kind.displayname),
'Asset: {}'.format(assetnumber),
'Captured: {}'.format(when.strftime('%Y-%m-%d %H:%M:%S') if when else 'unknown'),
'Source PC: {}'.format(revision.sourcehostname or 'unknown'),
'Revision: {} ({})'.format(revision.backuprevisionid,
(revision.contenthash or '')[:12]),
]
try:
raw, ext, mimetype = kind.render(revision.payload, formatid,
comments=comments)
except ValueError as exc:
return error_response(ErrorCodes.VALIDATION_ERROR, str(exc))
filename = revision.sourcefilename or '{}-{}{}'.format(
assetnumber, kind.key, ext)
if formatid == 'wow6432node':
filename = '{}-{}-wow6432node{}'.format(assetnumber, kind.key, ext)
return Response(
raw,
mimetype=mimetype,
headers={'Content-Disposition': 'attachment; filename="{}"'.format(filename)},
)
@backups_bp.route('/revisions/<int:backuprevisionid>/diff', methods=['GET'])
@jwt_required()
@require_permission('backups.view')
def diff_revision(backuprevisionid):
"""Diff two revisions of the same asset+kind.
?against=<id> picks the comparison revision; default is the immediately
preceding one, which answers "what changed" without the user choosing.
"""
revision = db.session.get(BackupRevision, backuprevisionid)
if revision is None:
return error_response(ErrorCodes.NOT_FOUND, 'Revision not found',
http_code=404)
if revision.storagebackend != 'shopdb':
return error_response(
ErrorCodes.VALIDATION_ERROR,
'Kind {} stores opaque files and cannot be diffed'.format(
revision.backupkind))
againstid = request.args.get('against', type=int)
if againstid:
other = db.session.get(BackupRevision, againstid)
else:
other = (db.session.query(BackupRevision)
.filter(BackupRevision.assetid == revision.assetid,
BackupRevision.backupkind == revision.backupkind,
BackupRevision.backuprevisionid < revision.backuprevisionid)
.order_by(BackupRevision.backuprevisionid.desc())
.first())
if other is None:
return success_response({
'backuprevisionid': revision.backuprevisionid,
'againstid': None,
'changes': [],
'message': 'No earlier revision to compare against.',
})
changes = _diffprojections(other.payload, revision.payload)
return success_response({
'backuprevisionid': revision.backuprevisionid,
'againstid': other.backuprevisionid,
'changes': changes,
'changecount': len(changes),
})
def _flatten(projection):
"""{(subkey, valuename): (type, data)} from a stored projection."""
flat = {}
for key in (projection or {}).get('keys', []):
path = key.get('path', '')
for name, entry in (key.get('values') or {}).items():
flat[(path, name)] = (entry.get('type'), entry.get('data'))
return flat
def _diffprojections(old, new):
oldflat = _flatten(old)
newflat = _flatten(new)
changes = []
for ref in sorted(set(oldflat) | set(newflat)):
path, name = ref
before = oldflat.get(ref)
after = newflat.get(ref)
if before == after:
continue
if before is None:
change = 'added'
elif after is None:
change = 'removed'
else:
change = 'changed'
changes.append({
'keypath': path,
'valuename': name,
'change': change,
'before': None if before is None else before[1],
'after': None if after is None else after[1],
'beforetype': None if before is None else before[0],
'aftertype': None if after is None else after[0],
})
return changes