Adds a kind-pluggable backups plugin. Configuration captured from a PC is filed against the MACHINE it controls, with a revision history and download back to the native format. NTLARS/DNC is the first kind. Settings live in the controlling PC's registry but describe the machine, so revisions attach to the machine's asset and carry no foreign key to the PC: history survives a PC being replaced or deleted, and sourcehostname records the handover. Storage splits by kind. Parseable kinds store a dialect-neutral JSON projection in ShopDB and re-render on download; opaque vendor formats (part marker and similar) keep their bytes on the SFLD share with ShopDB holding metadata and the UNC pointer. Two .reg dialects exist in the wild: NTLARS's own Save... export omits the WOW6432Node path segment, scripted exports include it. Parsing strips whichever root matched, so a stored revision commits to neither and download offers both (NTLARS Load... by default, WOW6432Node for direct reg import). Getting this backwards is silent, so the dedup hash deliberately excludes sourcedialect and both dialects of one config dedup to a single revision. Dedup is load-bearing: the collector runs every GE-Enforce cycle across the fleet, so a revision is inserted only when the content hash differs from that asset's latest for that kind. A freshly imaged PC opens NTLARS with a blank General tab. Recording that would make an empty config the newest revision exactly when someone needs the last good one, so a blank MachineNo is rejected rather than accepted as a change. Two of the 320 known-good backups on the share already have that shape. DNC Info card summarises the latest revision on the machine page: General (Cnc, NcIF, HostType), eFocas, Serial, NTSHR when populated (only 18 of 147 machines), and MARK when the machine is a marker. MARK is gated on Cnc=MARKER or the ShopDB machine type, not on the MARK key having content: MARK carries serial defaults on 145 of 147 machines and DncPatterns reads YES on 103 including ordinary lathes, so neither identifies a marker. The info card is owned by the kind (BackupKind.infopanel/buildinfo) and served by a generic endpoint, so the expected successor to DNC ships its own card by adding a class rather than changing the plugin or the panel wiring. Also: schedule and retention settings with a prune that never drops the newest or the oldest revision, and scripts/import_ntlars_backups.py to seed history from the existing per-machine .reg files (144 of 147 resolve to assets). Codec verified against all 320 real backups: round-trips clean through both dialects. Bay-side generation verified on Windows against reg.exe export.
307 lines
11 KiB
Python
307 lines
11 KiB
Python
"""Backups API.
|
|
|
|
Read + download only. Revisions are created exclusively through the ADR-006
|
|
collector endpoint (POST /api/collector/backups), so there is deliberately no
|
|
create route here: a hand-posted "backup" that never came off a machine would
|
|
poison the history this feature exists to provide.
|
|
|
|
Downloads differ by storage backend. 'shopdb' kinds re-render from the stored
|
|
projection, which is what lets one revision produce either .reg dialect. 'share'
|
|
kinds are not streamed - ShopDB returns the UNC path for the tech to open,
|
|
because requiring the app server to mount the SFLD share would turn a
|
|
permissions slip into an unexplained empty download.
|
|
"""
|
|
|
|
from flask import Blueprint, current_app, request, Response
|
|
from flask_jwt_extended import jwt_required
|
|
|
|
from shopdb.api import (
|
|
db, Asset,
|
|
success_response, error_response, ErrorCodes,
|
|
require_permission,
|
|
)
|
|
|
|
from ..models import BackupRevision
|
|
from ..services.registry import REGISTRY, getkind
|
|
|
|
backups_bp = Blueprint('backups', __name__)
|
|
|
|
|
|
def _label(revision):
|
|
"""Panel list title: kind-agnostic, readable at a glance."""
|
|
when = revision.collectedat or revision.createdat
|
|
stamp = when.strftime('%Y-%m-%d %H:%M') if when else 'unknown time'
|
|
if revision.sourcefilename:
|
|
return '{} - {}'.format(stamp, revision.sourcefilename)
|
|
return stamp
|
|
|
|
|
|
def _summary(revision, islatest=False):
|
|
data = revision.to_dict()
|
|
data['label'] = _label(revision)
|
|
data['islatest'] = islatest
|
|
kind = getkind(revision.backupkind)
|
|
data['formats'] = kind.formats() if kind else []
|
|
return data
|
|
|
|
|
|
@backups_bp.route('/kinds', methods=['GET'])
|
|
@jwt_required()
|
|
@require_permission('backups.view')
|
|
def list_kinds():
|
|
"""Registered backup kinds, for UI that needs to enumerate them."""
|
|
return success_response([
|
|
{
|
|
'key': kind.key,
|
|
'displayname': kind.displayname,
|
|
'storagebackend': kind.storagebackend,
|
|
'assettypes': list(kind.assettypes),
|
|
'formats': kind.formats(),
|
|
}
|
|
for kind in REGISTRY.values()
|
|
])
|
|
|
|
|
|
@backups_bp.route('/asset/<int:assetid>', methods=['GET'])
|
|
@jwt_required()
|
|
@require_permission('backups.view')
|
|
def asset_revisions(assetid):
|
|
"""Revision history for one asset, newest first.
|
|
|
|
This is the asset-panel endpoint. Optional ?kind= narrows to a single kind,
|
|
which is how each per-kind panel scopes itself.
|
|
"""
|
|
asset = db.session.get(Asset, assetid)
|
|
if asset is None:
|
|
return error_response(ErrorCodes.NOT_FOUND, 'Asset not found', http_code=404)
|
|
|
|
query = db.session.query(BackupRevision).filter(
|
|
BackupRevision.assetid == assetid)
|
|
|
|
kindkey = (request.args.get('kind') or '').strip().lower()
|
|
if kindkey:
|
|
if getkind(kindkey) is None:
|
|
return error_response(ErrorCodes.VALIDATION_ERROR,
|
|
'Unknown kind: {}'.format(kindkey))
|
|
query = query.filter(BackupRevision.backupkind == kindkey)
|
|
|
|
revisions = query.order_by(BackupRevision.backuprevisionid.desc()).all()
|
|
|
|
# "Latest" is per kind, not per asset, so an unfiltered listing still marks
|
|
# the current revision of each kind correctly.
|
|
seen = set()
|
|
out = []
|
|
for revision in revisions:
|
|
islatest = revision.backupkind not in seen
|
|
seen.add(revision.backupkind)
|
|
out.append(_summary(revision, islatest=islatest))
|
|
return success_response(out)
|
|
|
|
|
|
@backups_bp.route('/asset/<int:assetid>/info', methods=['GET'])
|
|
@jwt_required()
|
|
@require_permission('backups.view')
|
|
def asset_info(assetid):
|
|
"""A kind's 'at a glance' card, built from its LATEST revision.
|
|
|
|
Generic across kinds: the requested kind owns both the panel declaration
|
|
and the payload (see BackupKind.infopanel / buildinfo), so a successor to
|
|
NTLARS/DNC gets its own card without a new endpoint.
|
|
|
|
Empty when the machine has no revision of that kind yet, which the generic
|
|
renderer turns into the panel's empty text.
|
|
"""
|
|
kindkey = (request.args.get('kind') or 'ntlars').strip().lower()
|
|
kind = getkind(kindkey)
|
|
if kind is None:
|
|
return error_response(ErrorCodes.VALIDATION_ERROR,
|
|
'Unknown kind: {}'.format(kindkey))
|
|
|
|
revision = (db.session.query(BackupRevision)
|
|
.filter(BackupRevision.assetid == assetid,
|
|
BackupRevision.backupkind == kindkey)
|
|
.order_by(BackupRevision.backuprevisionid.desc())
|
|
.first())
|
|
if revision is None:
|
|
return success_response({'fields': [], 'sectioncount': 0})
|
|
|
|
data = kind.buildinfo(
|
|
revision.payload, assetid,
|
|
partmarkertypes=current_app.config.get('BACKUPS_PARTMARKER_TYPES'))
|
|
data['backuprevisionid'] = revision.backuprevisionid
|
|
data['collectedat'] = (revision.collectedat.isoformat()
|
|
if revision.collectedat else None)
|
|
return success_response(data)
|
|
|
|
|
|
@backups_bp.route('/revisions/<int:backuprevisionid>', methods=['GET'])
|
|
@jwt_required()
|
|
@require_permission('backups.view')
|
|
def get_revision(backuprevisionid):
|
|
"""One revision, including its parsed payload where there is one."""
|
|
revision = db.session.get(BackupRevision, backuprevisionid)
|
|
if revision is None:
|
|
return error_response(ErrorCodes.NOT_FOUND, 'Revision not found',
|
|
http_code=404)
|
|
data = revision.to_dict(includepayload=True)
|
|
data['label'] = _label(revision)
|
|
kind = getkind(revision.backupkind)
|
|
data['formats'] = kind.formats() if kind else []
|
|
data['displayname'] = kind.displayname if kind else revision.backupkind
|
|
return success_response(data)
|
|
|
|
|
|
@backups_bp.route('/revisions/<int:backuprevisionid>/download', methods=['GET'])
|
|
@jwt_required()
|
|
@require_permission('backups.download')
|
|
def download_revision(backuprevisionid):
|
|
"""Render a revision back to its native file format.
|
|
|
|
?format=ntlars (default for NTLARS) omits WOW6432Node - the form the
|
|
NTLARS Load... button expects
|
|
?format=wow6432node includes WOW6432Node - for `reg import` on 64-bit
|
|
"""
|
|
revision = db.session.get(BackupRevision, backuprevisionid)
|
|
if revision is None:
|
|
return error_response(ErrorCodes.NOT_FOUND, 'Revision not found',
|
|
http_code=404)
|
|
|
|
kind = getkind(revision.backupkind)
|
|
if kind is None:
|
|
return error_response(ErrorCodes.VALIDATION_ERROR,
|
|
'Unknown kind: {}'.format(revision.backupkind))
|
|
|
|
if revision.storagebackend == 'share':
|
|
# Not an error: the file exists, ShopDB just is not the one serving it.
|
|
return success_response({
|
|
'storagebackend': 'share',
|
|
'sharepath': revision.sharepath,
|
|
'sourcefilename': revision.sourcefilename,
|
|
'message': 'This backup lives on the SFLD share. Open the path directly.',
|
|
})
|
|
|
|
formats = kind.formats()
|
|
formatid = (request.args.get('format') or '').strip().lower()
|
|
if not formatid:
|
|
formatid = formats[0]['id'] if formats else ''
|
|
if not any(f['id'] == formatid for f in formats):
|
|
return error_response(
|
|
ErrorCodes.VALIDATION_ERROR,
|
|
'Unknown format {!r} for kind {}'.format(formatid, kind.key))
|
|
|
|
asset = db.session.get(Asset, revision.assetid)
|
|
assetnumber = asset.assetnumber if asset else str(revision.assetid)
|
|
when = revision.collectedat or revision.createdat
|
|
|
|
comments = [
|
|
'{} backup from ShopDB'.format(kind.displayname),
|
|
'Asset: {}'.format(assetnumber),
|
|
'Captured: {}'.format(when.strftime('%Y-%m-%d %H:%M:%S') if when else 'unknown'),
|
|
'Source PC: {}'.format(revision.sourcehostname or 'unknown'),
|
|
'Revision: {} ({})'.format(revision.backuprevisionid,
|
|
(revision.contenthash or '')[:12]),
|
|
]
|
|
|
|
try:
|
|
raw, ext, mimetype = kind.render(revision.payload, formatid,
|
|
comments=comments)
|
|
except ValueError as exc:
|
|
return error_response(ErrorCodes.VALIDATION_ERROR, str(exc))
|
|
|
|
filename = revision.sourcefilename or '{}-{}{}'.format(
|
|
assetnumber, kind.key, ext)
|
|
if formatid == 'wow6432node':
|
|
filename = '{}-{}-wow6432node{}'.format(assetnumber, kind.key, ext)
|
|
|
|
return Response(
|
|
raw,
|
|
mimetype=mimetype,
|
|
headers={'Content-Disposition': 'attachment; filename="{}"'.format(filename)},
|
|
)
|
|
|
|
|
|
@backups_bp.route('/revisions/<int:backuprevisionid>/diff', methods=['GET'])
|
|
@jwt_required()
|
|
@require_permission('backups.view')
|
|
def diff_revision(backuprevisionid):
|
|
"""Diff two revisions of the same asset+kind.
|
|
|
|
?against=<id> picks the comparison revision; default is the immediately
|
|
preceding one, which answers "what changed" without the user choosing.
|
|
"""
|
|
revision = db.session.get(BackupRevision, backuprevisionid)
|
|
if revision is None:
|
|
return error_response(ErrorCodes.NOT_FOUND, 'Revision not found',
|
|
http_code=404)
|
|
if revision.storagebackend != 'shopdb':
|
|
return error_response(
|
|
ErrorCodes.VALIDATION_ERROR,
|
|
'Kind {} stores opaque files and cannot be diffed'.format(
|
|
revision.backupkind))
|
|
|
|
againstid = request.args.get('against', type=int)
|
|
if againstid:
|
|
other = db.session.get(BackupRevision, againstid)
|
|
else:
|
|
other = (db.session.query(BackupRevision)
|
|
.filter(BackupRevision.assetid == revision.assetid,
|
|
BackupRevision.backupkind == revision.backupkind,
|
|
BackupRevision.backuprevisionid < revision.backuprevisionid)
|
|
.order_by(BackupRevision.backuprevisionid.desc())
|
|
.first())
|
|
|
|
if other is None:
|
|
return success_response({
|
|
'backuprevisionid': revision.backuprevisionid,
|
|
'againstid': None,
|
|
'changes': [],
|
|
'message': 'No earlier revision to compare against.',
|
|
})
|
|
|
|
changes = _diffprojections(other.payload, revision.payload)
|
|
return success_response({
|
|
'backuprevisionid': revision.backuprevisionid,
|
|
'againstid': other.backuprevisionid,
|
|
'changes': changes,
|
|
'changecount': len(changes),
|
|
})
|
|
|
|
|
|
def _flatten(projection):
|
|
"""{(subkey, valuename): (type, data)} from a stored projection."""
|
|
flat = {}
|
|
for key in (projection or {}).get('keys', []):
|
|
path = key.get('path', '')
|
|
for name, entry in (key.get('values') or {}).items():
|
|
flat[(path, name)] = (entry.get('type'), entry.get('data'))
|
|
return flat
|
|
|
|
|
|
def _diffprojections(old, new):
|
|
oldflat = _flatten(old)
|
|
newflat = _flatten(new)
|
|
|
|
changes = []
|
|
for ref in sorted(set(oldflat) | set(newflat)):
|
|
path, name = ref
|
|
before = oldflat.get(ref)
|
|
after = newflat.get(ref)
|
|
if before == after:
|
|
continue
|
|
if before is None:
|
|
change = 'added'
|
|
elif after is None:
|
|
change = 'removed'
|
|
else:
|
|
change = 'changed'
|
|
changes.append({
|
|
'keypath': path,
|
|
'valuename': name,
|
|
'change': change,
|
|
'before': None if before is None else before[1],
|
|
'after': None if after is None else after[1],
|
|
'beforetype': None if before is None else before[0],
|
|
'aftertype': None if after is None else after[0],
|
|
})
|
|
return changes
|