backups plugin: per-asset config backups with revision history

Adds a kind-pluggable backups plugin. Configuration captured from a PC is
filed against the MACHINE it controls, with a revision history and download
back to the native format.

NTLARS/DNC is the first kind. Settings live in the controlling PC's registry
but describe the machine, so revisions attach to the machine's asset and carry
no foreign key to the PC: history survives a PC being replaced or deleted, and
sourcehostname records the handover.

Storage splits by kind. Parseable kinds store a dialect-neutral JSON
projection in ShopDB and re-render on download; opaque vendor formats (part
marker and similar) keep their bytes on the SFLD share with ShopDB holding
metadata and the UNC pointer.

Two .reg dialects exist in the wild: NTLARS's own Save... export omits the
WOW6432Node path segment, scripted exports include it. Parsing strips whichever
root matched, so a stored revision commits to neither and download offers both
(NTLARS Load... by default, WOW6432Node for direct reg import). Getting this
backwards is silent, so the dedup hash deliberately excludes sourcedialect and
both dialects of one config dedup to a single revision.

Dedup is load-bearing: the collector runs every GE-Enforce cycle across the
fleet, so a revision is inserted only when the content hash differs from that
asset's latest for that kind.

A freshly imaged PC opens NTLARS with a blank General tab. Recording that would
make an empty config the newest revision exactly when someone needs the last
good one, so a blank MachineNo is rejected rather than accepted as a change.
Two of the 320 known-good backups on the share already have that shape.

DNC Info card summarises the latest revision on the machine page: General
(Cnc, NcIF, HostType), eFocas, Serial, NTSHR when populated (only 18 of 147
machines), and MARK when the machine is a marker. MARK is gated on Cnc=MARKER
or the ShopDB machine type, not on the MARK key having content: MARK carries
serial defaults on 145 of 147 machines and DncPatterns reads YES on 103
including ordinary lathes, so neither identifies a marker.

The info card is owned by the kind (BackupKind.infopanel/buildinfo) and served
by a generic endpoint, so the expected successor to DNC ships its own card by
adding a class rather than changing the plugin or the panel wiring.

Also: schedule and retention settings with a prune that never drops the newest
or the oldest revision, and scripts/import_ntlars_backups.py to seed history
from the existing per-machine .reg files (144 of 147 resolve to assets).

Codec verified against all 320 real backups: round-trips clean through both
dialects. Bay-side generation verified on Windows against reg.exe export.

plugins/backups/services/__init__.py

@@ -0,0 +1,6 @@
+"""Backups plugin services: kind registry and per-kind codecs."""
+
+from .registry import REGISTRY, BackupKind, getkind, canonicalhash, byteshash, DEFAULTSHAREROOT
+
+__all__ = ['REGISTRY', 'BackupKind', 'getkind', 'canonicalhash', 'byteshash',
+           'DEFAULTSHAREROOT']

plugins/backups/services/dncinfo.py

@@ -0,0 +1,161 @@
+"""DNC Info card.
+
+Surfaces the handful of NTLARS settings a tech actually asks about on the
+machine's page, so the common question ("what is this machine's controller
+talking to?") is answered without downloading and reading a .reg.
+
+Sections, drawn from the machine's LATEST ntlars revision:
+
+  eFocas   Fanuc ethernet link - IP, socket, dual-path. Present on 143 of the
+           147 known-good backups, so it is shown whenever it has content.
+  Serial   RS-232 link parameters. Always populated (Baud, Data Bits and
+           friends carry defaults even where the link is unused), so it is
+           always shown.
+  NTSHR    Network share the controller pulls programs from. Populated on only
+           18 of 147, hence the has-content gate: showing an empty NTSHR block
+           on 129 machines would be noise.
+  MARK     Part-marker settings. Gated on the ASSET being a Part Marker in
+           ShopDB, NOT on the key having content.
+
+WHY MARK IS GATED ON THE ASSET, NOT THE KEY:
+    The obvious rule - show MARK when it has content - does not work. MARK is
+    populated on 145 of 147 machines because Baud/Data Bits carry serial
+    defaults everywhere, and the fields that would identify a marker
+    (CageCode, DataHost, DataPath, MarkMasterPath) are empty across the entire
+    corpus. The one field that is set, DncPatterns, reads YES on 103 of 147
+    including ordinary lathes, so it is a DNC pattern-matching option and not a
+    marker flag. No value in the DNC tree distinguishes a part marker, so the
+    machine's type in ShopDB is the only reliable signal.
+"""
+
+# Values equal to these (case-insensitively) count as "no content".
+EMPTYISH = ('', '0', 'no', 'none')
+
+DEFAULTPARTMARKERTYPES = ('Part Marker',)
+
+
+def _keyvalues(projection, path):
+    """Values under one subkey of the stored projection, or {}."""
+    for key in (projection or {}).get('keys', []):
+        if (key.get('path') or '').lower() == path.lower():
+            return key.get('values') or {}
+    return {}
+
+
+def _hascontent(values):
+    """True when at least one value carries something meaningful."""
+    for entry in values.values():
+        data = entry.get('data')
+        if isinstance(data, (list, tuple)):
+            if any(str(x).strip() for x in data):
+                return True
+            continue
+        if str(data).strip().lower() not in EMPTYISH:
+            return True
+    return False
+
+
+def _fields(values, mono=()):
+    """Render a subkey's values as keyvalue panel fields, empties dropped."""
+    out = []
+    for name in sorted(values):
+        data = values[name].get('data')
+        if isinstance(data, (list, tuple)):
+            data = ', '.join(str(x) for x in data)
+        text = '' if data is None else str(data)
+        if not text.strip():
+            continue
+        out.append({
+            'label': name,
+            'value': text,
+            'mono': name in mono,
+        })
+    return out
+
+
+def ispartmarker(assetid, typenames=None):
+    """True when this asset is a Part Marker according to ShopDB.
+
+    Reads the machines plugin defensively: a lean per-site build (ADR-014) may
+    not install it, and the DNC Info card must degrade to "no MARK section"
+    rather than erroring the whole panel.
+    """
+    typenames = tuple(t.lower() for t in (typenames or DEFAULTPARTMARKERTYPES))
+    try:
+        from shopdb.api import db
+        from plugins.machines.models import Machine, MachineType
+    except ImportError:
+        return False
+    try:
+        row = (db.session.query(MachineType.machinetype)
+               .join(Machine, Machine.machinetypeid == MachineType.machinetypeid)
+               .filter(Machine.assetid == assetid)
+               .first())
+    except Exception:
+        return False
+    return bool(row) and (row[0] or '').strip().lower() in typenames
+
+
+def _cncismarker(projection):
+    """True when NTLARS itself says the controller is a marker.
+
+    General\\Cnc reads 'MARKER' on the part markers (0600 and 0614 in the
+    known-good corpus) and a controller family - Fanuc 30, Fanuc 16, OKUMA,
+    Fidia - everywhere else. This is the one place the DNC tree does
+    distinguish a marker; nothing inside the MARK key does, since MARK carries
+    serial defaults on nearly every machine.
+    """
+    general = _keyvalues(projection, 'General')
+    cnc = str((general.get('Cnc') or {}).get('data') or '').strip().upper()
+    return cnc == 'MARKER'
+
+
+def build(projection, assetid, partmarkertypes=None):
+    """Build the DNC Info card payload from a stored ntlars projection.
+
+    Returns {'fields': [...]} in the shape the generic keyvalue renderer wants,
+    with section headings inlined as labelless separators.
+    """
+    sections = []
+
+    # General first: what the controller IS, before what it talks to. Cnc gives
+    # the controller family, NcIF the interface actually in use (EFOCAS on 127
+    # of 147, NTSHR on 16, SERIAL on 3, HSSB on 1), HostType the DNC host.
+    general = _keyvalues(projection, 'General')
+    wanted = ('Cnc', 'NcIF', 'HostType')
+    generalfields = _fields(
+        {n: v for n, v in general.items() if n in wanted})
+    if generalfields:
+        sections.append(('General', generalfields))
+
+    efocas = _keyvalues(projection, 'eFocas')
+    if _hascontent(efocas):
+        sections.append(('eFocas (ethernet link)', _fields(efocas, mono=('IpAddr',))))
+
+    serial = _keyvalues(projection, 'Serial')
+    if serial:
+        sections.append(('Serial (RS-232)', _fields(serial)))
+
+    ntshr = _keyvalues(projection, 'NTSHR')
+    if _hascontent(ntshr):
+        sections.append(('NTSHR (program share)',
+                         _fields(ntshr, mono=('ShrFolder', 'ShrFolder2',
+                                              'ShrFolder3', 'ShrHost'))))
+
+    # Either signal is enough: NTLARS's own Cnc=MARKER works before anyone has
+    # set the machine's type in ShopDB and on lean builds with no machines
+    # plugin, while the ShopDB type still covers a marker whose Cnc says
+    # something else.
+    if _cncismarker(projection) or ispartmarker(assetid, partmarkertypes):
+        mark = _keyvalues(projection, 'MARK')
+        if mark:
+            sections.append(('MARK (part marker)',
+                             _fields(mark, mono=('DataPath', 'MarkMasterPath'))))
+
+    fields = []
+    for title, entries in sections:
+        if not entries:
+            continue
+        fields.append({'label': title, 'value': '', 'heading': True})
+        fields.extend(entries)
+    return {'fields': fields, 'sectioncount': len(sections)}

plugins/backups/services/ntlars.py

@@ -0,0 +1,308 @@
+"""NTLARS / DNC registry backup codec.
+
+Converts between Windows .reg files and a dialect-neutral JSON projection.
+
+WHY DIALECT-NEUTRAL: NTLARS is a 32-bit app, so its settings physically live
+under HKLM\\SOFTWARE\\WOW6432Node\\GE Aircraft Engines\\DNC. But NTLARS's own
+Save... button exports them WITHOUT the WOW6432Node segment (it writes the path
+it asks for, before the WOW64 redirector rewrites it). Both dialects therefore
+exist in the wild:
+
+  NTLARS Save... output      HKLM\\SOFTWARE\\GE Aircraft Engines\\DNC
+  scripted / reg export      HKLM\\SOFTWARE\\WOW6432Node\\GE Aircraft Engines\\DNC
+
+Parsing strips whichever root matched and stores subkeys RELATIVE to it, so the
+stored revision commits to neither. render() then re-attaches whichever root the
+consumer needs:
+
+  dialect='ntlars'      no WOW6432Node - what the NTLARS Load... button expects
+  dialect='wow6432node' explicit - what `reg import` needs on a 64-bit box
+
+Getting this backwards is silent: a reg import of the NTLARS dialect on 64-bit
+writes to the 64-bit hive, where NTLARS will never look, and reports success.
+
+CANONICAL ORDERING: keys and value names are sorted on parse. MySQL's JSON type
+normalizes object key order anyway, so preserving source order is not possible
+end-to-end; sorting makes it deterministic instead, which is what makes diffs
+between revisions stable. Re-rendered files are semantically identical to their
+source, not byte-identical.
+"""
+
+import re
+
+SCHEMA = 'ntlars/1'
+
+REGROOT = 'HKEY_LOCAL_MACHINE'
+DNCPATH = r'SOFTWARE\GE Aircraft Engines\DNC'
+DNCPATHWOW = r'SOFTWARE\WOW6432Node\GE Aircraft Engines\DNC'
+
+ROOTNTLARS = '{}\\{}'.format(REGROOT, DNCPATH)
+ROOTWOW = '{}\\{}'.format(REGROOT, DNCPATHWOW)
+
+# Order is not significant: the two roots diverge immediately after
+# 'SOFTWARE\\' (GE vs WOW), so neither is a string prefix of the other and
+# _striproot's startswith test cannot match the wrong one. Listed longest-first
+# only for readability.
+KNOWNROOTS = (ROOTWOW, ROOTNTLARS)
+
+HEADER = 'Windows Registry Editor Version 5.00'
+
+# hex(N): type codes that appear in .reg files, mapped to registry type names.
+HEXTYPES = {
+    0: 'REG_NONE',
+    1: 'REG_SZ',
+    2: 'REG_EXPAND_SZ',
+    3: 'REG_BINARY',
+    4: 'REG_DWORD',
+    7: 'REG_MULTI_SZ',
+    11: 'REG_QWORD',
+}
+HEXTYPECODES = {v: k for k, v in HEXTYPES.items()}
+
+
+class NtlarsParseError(ValueError):
+    """Raised when input is not a .reg file we can make sense of."""
+
+
+def decodereg(raw):
+    """Decode .reg bytes to text.
+
+    .reg files are conventionally UTF-16LE with a BOM (that is what both
+    regedit and NTLARS emit), but hand-edited ones show up as UTF-8. Sniff the
+    BOM rather than trusting the extension.
+    """
+    if isinstance(raw, str):
+        return raw
+    if raw.startswith(b'\xff\xfe'):
+        return raw.decode('utf-16-le')[1:]
+    if raw.startswith(b'\xfe\xff'):
+        return raw.decode('utf-16-be')[1:]
+    if raw.startswith(b'\xef\xbb\xbf'):
+        return raw.decode('utf-8-sig')
+    # No BOM. UTF-16LE ASCII text has a NUL in every other byte.
+    if b'\x00' in raw[:64]:
+        return raw.decode('utf-16-le', errors='replace')
+    return raw.decode('utf-8', errors='replace')
+
+
+def _unescape(s):
+    return s.replace('\\\\', '\x00').replace('\\"', '"').replace('\x00', '\\')
+
+
+def _escape(s):
+    return s.replace('\\', '\\\\').replace('"', '\\"')
+
+
+def _joincontinuations(text):
+    """Fold .reg line continuations (trailing backslash) into single lines."""
+    out = []
+    for line in text.replace('\r\n', '\n').replace('\r', '\n').split('\n'):
+        if out and out[-1].endswith('\\'):
+            out[-1] = out[-1][:-1] + line.strip()
+        else:
+            out.append(line)
+    return out
+
+
+def _parsehexvalue(body):
+    """Parse the body of a hex:/hex(N): value into (typename, data)."""
+    m = re.match(r'^hex(?:\((?P<code>[0-9a-fA-F]+)\))?:(?P<bytes>.*)$', body, re.S)
+    if not m:
+        raise NtlarsParseError('unparseable hex value: {!r}'.format(body))
+    code = int(m.group('code'), 16) if m.group('code') else 3
+    tokens = [t.strip() for t in m.group('bytes').split(',') if t.strip()]
+    try:
+        data = bytes(int(t, 16) for t in tokens)
+    except ValueError as exc:
+        raise NtlarsParseError('bad hex byte in value: {}'.format(exc))
+
+    typename = HEXTYPES.get(code, 'REG_BINARY')
+
+    # Wide-string hex types decode back to text so diffs stay readable.
+    if typename in ('REG_SZ', 'REG_EXPAND_SZ'):
+        return typename, data.decode('utf-16-le', errors='replace').rstrip('\x00')
+    if typename == 'REG_MULTI_SZ':
+        text = data.decode('utf-16-le', errors='replace')
+        return typename, [p for p in text.split('\x00') if p]
+    # REG_QWORD must come back as an int: _rendervalue turns it back into
+    # little-endian bytes via int(), so storing the "aa,bb" byte form here
+    # would raise at download time - i.e. precisely when someone is trying to
+    # restore a machine.
+    if typename == 'REG_QWORD':
+        return typename, int.from_bytes(data, 'little')
+    return typename, ','.join('{:02x}'.format(b) for b in data)
+
+
+def _parsevalue(body):
+    """Parse the right-hand side of a .reg value assignment."""
+    body = body.strip()
+    if body.startswith('"'):
+        if not body.endswith('"') or len(body) < 2:
+            raise NtlarsParseError('unterminated string value: {!r}'.format(body))
+        return 'REG_SZ', _unescape(body[1:-1])
+    if body.lower().startswith('dword:'):
+        try:
+            return 'REG_DWORD', int(body.split(':', 1)[1].strip(), 16)
+        except ValueError:
+            raise NtlarsParseError('bad dword value: {!r}'.format(body))
+    if body.lower().startswith('hex'):
+        return _parsehexvalue(body)
+    if body == '-':
+        return 'DELETE', None
+    raise NtlarsParseError('unrecognised value form: {!r}'.format(body))
+
+
+def _striproot(keypath):
+    """Strip a known DNC root, returning the relative subkey path.
+
+    Returns None for keys outside the DNC tree so callers can ignore them
+    rather than silently folding unrelated hives into the backup.
+    """
+    for root in KNOWNROOTS:
+        if keypath.upper() == root.upper():
+            return ''
+        prefix = root.upper() + '\\'
+        if keypath.upper().startswith(prefix):
+            return keypath[len(prefix):]
+    return None
+
+
+def parse(raw):
+    """Parse .reg bytes/text into the dialect-neutral JSON projection.
+
+    Returns {'schema', 'sourcedialect', 'keys': [{'path', 'values': {...}}]}
+    with keys and value names sorted for deterministic diffing.
+    """
+    text = decodereg(raw)
+    lines = _joincontinuations(text)
+
+    if not any(line.strip().lower().startswith('windows registry editor')
+               or line.strip().lower().startswith('regedit4')
+               for line in lines[:5]):
+        raise NtlarsParseError('missing "Windows Registry Editor" header')
+
+    keys = {}
+    current = None
+    sawwow = False
+    sawplain = False
+
+    for line in lines:
+        stripped = line.strip()
+        if not stripped or stripped.startswith(';'):
+            continue
+
+        if stripped.startswith('[') and stripped.endswith(']'):
+            keypath = stripped[1:-1].strip()
+            if keypath.startswith('-'):
+                current = None          # key deletion, not a backup concern
+                continue
+            if keypath.upper().startswith(ROOTWOW.upper()):
+                sawwow = True
+            elif keypath.upper().startswith(ROOTNTLARS.upper()):
+                sawplain = True
+            rel = _striproot(keypath)
+            if rel is None:
+                current = None          # outside the DNC tree - ignore
+                continue
+            current = rel
+            keys.setdefault(current, {})
+            continue
+
+        if current is None or '=' not in stripped:
+            continue
+
+        # Match the QUOTED name and split at the '=' that follows its closing
+        # quote. A plain split('=', 1) breaks on any value name containing '='
+        # or an escaped quote - both legal in the registry - and silently drops
+        # the value.
+        match = re.match(r'^(?:@|"((?:[^"\\]|\\.)*)")\s*=\s*(.*)$', stripped, re.S)
+        if not match:
+            continue
+        name = '' if match.group(1) is None else _unescape(match.group(1))
+        body = match.group(2)
+
+        # Deliberately NOT caught. A backup that silently dropped an
+        # unparseable value would present as complete and restore a machine
+        # with a setting missing - the exact silent-failure class this project
+        # has repeatedly been bitten by. Fail the whole parse instead; the
+        # collector reports it and the previous good revision stays newest.
+        typename, data = _parsevalue(body)
+        if typename == 'DELETE':
+            continue
+        keys[current][name] = {'type': typename, 'data': data}
+
+    if not keys:
+        raise NtlarsParseError(
+            'no keys under {} or {} - not an NTLARS DNC backup'.format(
+                ROOTNTLARS, ROOTWOW))
+
+    dialect = 'wow6432node' if sawwow else ('ntlars' if sawplain else 'unknown')
+
+    return {
+        'schema': SCHEMA,
+        'sourcedialect': dialect,
+        'keys': [
+            {'path': path, 'values': dict(sorted(keys[path].items()))}
+            for path in sorted(keys)
+        ],
+    }
+
+
+def _rendervalue(name, entry):
+    typename = entry.get('type', 'REG_SZ')
+    data = entry.get('data')
+    lhs = '@' if name == '' else '"{}"'.format(_escape(name))
+
+    if typename == 'REG_SZ':
+        return '{}="{}"'.format(lhs, _escape('' if data is None else str(data)))
+    if typename == 'REG_DWORD':
+        return '{}=dword:{:08x}'.format(lhs, int(data) & 0xFFFFFFFF)
+    if typename == 'REG_QWORD':
+        return '{}=hex(b):{}'.format(lhs, _hexbytes(
+            int(data).to_bytes(8, 'little')))
+    if typename == 'REG_EXPAND_SZ':
+        payload = ('' if data is None else str(data)).encode('utf-16-le') + b'\x00\x00'
+        return '{}=hex(2):{}'.format(lhs, _hexbytes(payload))
+    if typename == 'REG_MULTI_SZ':
+        parts = data if isinstance(data, list) else [str(data)]
+        payload = ''.join(p + '\x00' for p in parts).encode('utf-16-le') + b'\x00\x00'
+        return '{}=hex(7):{}'.format(lhs, _hexbytes(payload))
+
+    # REG_BINARY / REG_NONE: data is the "aa,bb,cc" form parse() produced.
+    raw = bytes(int(t, 16) for t in str(data).split(',') if t.strip()) if data else b''
+    code = HEXTYPECODES.get(typename, 3)
+    prefix = 'hex:' if code == 3 else 'hex({:x}):'.format(code)
+    return '{}={}{}'.format(lhs, prefix, _hexbytes(raw))
+
+
+def _hexbytes(raw):
+    return ','.join('{:02x}'.format(b) for b in raw)
+
+
+def render(projection, dialect='ntlars', comments=None):
+    """Render the JSON projection back to .reg bytes (UTF-16LE, BOM, CRLF).
+
+    dialect='ntlars'      omits WOW6432Node - use with the NTLARS Load... button
+    dialect='wow6432node' includes it - use with `reg import` on 64-bit
+    """
+    if dialect not in ('ntlars', 'wow6432node'):
+        raise ValueError('unknown dialect: {!r}'.format(dialect))
+    root = ROOTWOW if dialect == 'wow6432node' else ROOTNTLARS
+
+    out = [HEADER, '']
+    for line in (comments or []):
+        out.append('; {}'.format(line))
+    if comments:
+        out.append('')
+
+    for key in projection.get('keys', []):
+        path = key.get('path', '')
+        out.append('[{}]'.format(root + ('\\' + path if path else '')))
+        for name, entry in (key.get('values') or {}).items():
+            out.append(_rendervalue(name, entry))
+        out.append('')
+
+    text = '\r\n'.join(out)
+    if not text.endswith('\r\n'):
+        text += '\r\n'
+    return b'\xff\xfe' + text.encode('utf-16-le')

plugins/backups/services/registry.py

@@ -0,0 +1,237 @@
+"""Backup kind registry.
+
+A "kind" is one category of configuration backup (NTLARS/DNC registry, part
+marker, UDC, CMM ...). Each kind declares three things the rest of the plugin
+needs and nothing else:
+
+  where its bytes live      'shopdb' (parsed into payloadjson) or 'share'
+                            (opaque vendor file left on the SFLD share)
+  how to resolve an asset   the collector reports an identifier; the kind turns
+                            it into the assetid the revision belongs to
+  how to render downloads   one or more output formats
+
+Adding a kind is a class plus one REGISTRY entry. Nothing else in the plugin
+knows kind names.
+"""
+
+import hashlib
+import json
+
+from . import ntlars as ntlarscodec
+
+# Default root of the opaque-backup tree on the SFLD share. Connected PCs write
+# here directly (they need SFLD creds - a SYSTEM process hitting a UNC path
+# without them gets an access-denied that Test-Path reports as "not found").
+#
+# This is the WEST JEFFERSON path and is only a DEFAULT: the live value is the
+# backups_shareroot setting, because a bundled plugin in a multi-site product
+# must not hardcode one site's file server.
+DEFAULTSHAREROOT = r'\\tsgwp00525.wjs.geaerospace.net\shared\dt\shopfloor\backups'
+
+
+def canonicalhash(projection):
+    """sha256 over the SEMANTIC content only. Dedup key for 'shopdb' kinds.
+
+    Deliberately hashes just schema + keys, excluding sourcedialect. The same
+    settings exported through NTLARS's Save... button and through a scripted
+    reg export parse to identical keys but differing sourcedialect; hashing the
+    whole projection would record that as a change and produce a spurious
+    revision every time the collection route changed. Storage is
+    dialect-neutral, so the dedup key has to be too.
+    """
+    subset = {
+        'schema': projection.get('schema'),
+        'keys': projection.get('keys'),
+    }
+    blob = json.dumps(subset, sort_keys=True, separators=(',', ':'))
+    return hashlib.sha256(blob.encode('utf-8')).hexdigest()
+
+
+def byteshash(raw):
+    """sha256 over raw bytes. Dedup key for 'share' kinds."""
+    return hashlib.sha256(raw).hexdigest()
+
+
+class BackupKind:
+    """Base class. Subclasses override what applies to them."""
+
+    key = None
+    displayname = None
+    storagebackend = 'shopdb'
+    assettypes = ['*']
+    # Human note shown on the panel when there is nothing yet.
+    emptytext = 'No backups on record.'
+
+    def parse(self, raw):
+        """Opaque kinds return None; parseable kinds return the projection."""
+        return None
+
+    def formats(self):
+        """Downloadable formats: [{'id','label','ext','mimetype'}]."""
+        return []
+
+    def render(self, projection, formatid, comments=None):
+        """Return (bytes, extension, mimetype) for a 'shopdb' kind."""
+        raise NotImplementedError
+
+    def resolveassetid(self, payload):
+        """Map a collector payload to the assetid the backup belongs to."""
+        raise NotImplementedError
+
+    def infopanel(self):
+        """Optional 'at a glance' card built from this kind's LATEST revision.
+
+        Returns an ADR-010 panel dict, or None when the kind has no summary
+        worth surfacing. Declared by the kind rather than hardcoded in the
+        plugin so a successor technology (NTLARS/DNC is expected to give way to
+        Shopfloor Connect) ships its own card by adding a class, without
+        touching the plugin or the panel wiring.
+        """
+        return None
+
+    def buildinfo(self, projection, assetid, **options):
+        """Build this kind's info-card payload. Only called when infopanel()."""
+        return {'fields': [], 'sectioncount': 0}
+
+    def sharedir(self, machinetype, identifier, shareroot=None):
+        """UNC directory a 'share' kind's files are expected under.
+
+        Advisory only - the authoritative path is the sharepath the collector
+        reports, since the PC is what actually wrote the file. This builds the
+        conventional location for display and for validating a reported path.
+        """
+        return '{}\\{}\\{}\\{}'.format(
+            shareroot or DEFAULTSHAREROOT,
+            machinetype or 'unknown', identifier or 'unknown', self.key)
+
+
+class NtlarsKind(BackupKind):
+    """NTLARS / DNC registry settings.
+
+    Collected from the CONTROLLING PC's registry but belongs to the MACHINE:
+    the settings describe how to talk to that machine's controller, so they
+    follow the machine when a PC is swapped. resolveassetid therefore keys on
+    the reported machine number, not the hostname.
+    """
+
+    key = 'ntlars'
+    displayname = 'NTLARS / DNC Settings'
+    storagebackend = 'shopdb'
+    assettypes = ['machine']
+    emptytext = 'No NTLARS settings captured yet.'
+
+    @staticmethod
+    def embeddedmachineno(projection):
+        """The MachineNo NTLARS itself is configured with (General tab)."""
+        for key in (projection or {}).get('keys', []):
+            if (key.get('path') or '').lower() == 'general':
+                entry = (key.get('values') or {}).get('MachineNo') or {}
+                return str(entry.get('data') or '').strip()
+        return ''
+
+    def parse(self, raw):
+        """Parse, then refuse to record an unconfigured NTLARS install.
+
+        A freshly imaged PC opens NTLARS with a blank General tab (no MachineNo,
+        CNC, Host or Interface type) until a tech restores the config. Storing
+        that would make an empty config the newest revision at exactly the
+        moment someone needs the last good one - so a blank MachineNo is
+        rejected as "nothing worth backing up" rather than accepted as a change.
+        Two of the 320 known-good backups on the share already have this shape.
+        """
+        projection = ntlarscodec.parse(raw)
+        if not self.embeddedmachineno(projection):
+            raise ValueError(
+                'NTLARS General\\MachineNo is empty - unconfigured install, '
+                'refusing to record it as a revision')
+        return projection
+
+    def formats(self):
+        return [
+            {
+                'id': 'ntlars',
+                'label': 'NTLARS Load... (.reg)',
+                'ext': '.reg',
+                'mimetype': 'application/octet-stream',
+                'hint': 'Restore with the Load... button in the NTLARS settings dialog.',
+            },
+            {
+                'id': 'wow6432node',
+                'label': 'Direct reg import (.reg, WOW6432Node)',
+                'ext': '.reg',
+                'mimetype': 'application/octet-stream',
+                'hint': 'Use when importing outside NTLARS on a 64-bit machine.',
+            },
+        ]
+
+    def render(self, projection, formatid, comments=None):
+        if formatid not in ('ntlars', 'wow6432node'):
+            raise ValueError('unknown format for ntlars kind: {}'.format(formatid))
+        raw = ntlarscodec.render(projection, dialect=formatid, comments=comments)
+        return raw, '.reg', 'application/octet-stream'
+
+    def infopanel(self):
+        return {
+            'id': 'backups-dncinfo',
+            'title': 'DNC Info',
+            'assettypes': ['machine'],
+            'endpoint': '/api/backups/asset/{assetid}/info?kind=ntlars',
+            'render': 'keyvalue',
+            'empty': 'No NTLARS settings captured for this machine yet.',
+            # Above the history panels: this answers the question a tech
+            # arrives with, while history is for the rarer restore case.
+            'position': 38,
+        }
+
+    def buildinfo(self, projection, assetid, **options):
+        from . import dncinfo
+        return dncinfo.build(projection, assetid,
+                             partmarkertypes=options.get('partmarkertypes'))
+
+    def resolveassetid(self, payload):
+        from shopdb.api import db, Asset
+
+        machinenumber = (payload.get('machinenumber') or '').strip()
+        if not machinenumber:
+            return None, 'no machinenumber in payload'
+        asset = db.session.query(Asset).filter(
+            Asset.assetnumber == machinenumber).first()
+        if asset is None:
+            return None, 'no asset with assetnumber {!r}'.format(machinenumber)
+        return asset.assetid, None
+
+
+class PartMarkerKind(BackupKind):
+    """Telesis part marker configuration.
+
+    Opaque vendor filetype with no useful JSON representation, so ShopDB stores
+    metadata and a UNC pointer while the file itself stays on the share. The
+    original filename and extension are preserved because the vendor tool
+    rejects a renamed file.
+    """
+
+    key = 'partmarker'
+    displayname = 'Part Marker Configuration'
+    storagebackend = 'share'
+    assettypes = ['machine', 'measuring_tool']
+    emptytext = 'No part marker backups on record.'
+
+    def resolveassetid(self, payload):
+        from shopdb.api import db, Asset
+
+        identifier = (payload.get('machinenumber')
+                      or payload.get('assetnumber') or '').strip()
+        if not identifier:
+            return None, 'no machinenumber/assetnumber in payload'
+        asset = db.session.query(Asset).filter(
+            Asset.assetnumber == identifier).first()
+        if asset is None:
+            return None, 'no asset with assetnumber {!r}'.format(identifier)
+        return asset.assetid, None
+
+
+REGISTRY = {k.key: k for k in (NtlarsKind(), PartMarkerKind())}
+
+
+def getkind(key):
+    return REGISTRY.get((key or '').strip().lower())

plugins/backups/services/retention.py

@@ -0,0 +1,68 @@
+"""Revision retention.
+
+Dedup already keeps growth low - a revision appears only when a setting really
+changed - so retention exists for the pathological case, not the normal one: a
+value that flaps (or two PCs alternately claiming one machine number) would
+otherwise append a revision every collection cycle forever.
+
+Two independent limits, both off by default at 0:
+
+  retentioncount  keep at most N revisions per asset per kind
+  retentiondays   additionally drop anything older than N days
+
+The NEWEST and the OLDEST revision are never pruned. The newest is the one a
+tech restores from; the oldest is the earliest known-good baseline, which is
+usually the most valuable row in the table and the one a naive "keep last N"
+would delete first.
+"""
+
+from datetime import datetime, timedelta
+
+from shopdb.api import db
+
+from ..models import BackupRevision
+
+
+def prune(assetid, backupkind, retentioncount=0, retentiondays=0):
+    """Delete surplus revisions for one asset+kind. Returns the number removed.
+
+    Caller commits. Returns 0 when both limits are disabled.
+    """
+    retentioncount = int(retentioncount or 0)
+    retentiondays = int(retentiondays or 0)
+    if retentioncount <= 0 and retentiondays <= 0:
+        return 0
+
+    revisions = (db.session.query(BackupRevision)
+                 .filter(BackupRevision.assetid == assetid,
+                         BackupRevision.backupkind == backupkind)
+                 .order_by(BackupRevision.backuprevisionid.desc())
+                 .all())
+    if len(revisions) <= 2:
+        return 0                      # newest + oldest are both protected
+
+    newest = revisions[0]
+    oldest = revisions[-1]
+    protected = {newest.backuprevisionid, oldest.backuprevisionid}
+
+    doomed = []
+
+    if retentioncount > 0 and len(revisions) > retentioncount:
+        # Walk from the oldest END of the middle, so the surplus dropped is
+        # always the least recent, never the newest.
+        for revision in revisions[retentioncount:]:
+            if revision.backuprevisionid not in protected:
+                doomed.append(revision)
+
+    if retentiondays > 0:
+        cutoff = datetime.utcnow() - timedelta(days=retentiondays)
+        for revision in revisions:
+            if revision.backuprevisionid in protected:
+                continue
+            stamp = revision.collectedat or revision.createdat
+            if stamp and stamp < cutoff and revision not in doomed:
+                doomed.append(revision)
+
+    for revision in doomed:
+        db.session.delete(revision)
+    return len(doomed)