Files
shopdb-flask/plugins/backups/plugin.py
cproudlock 13c46af5ce Rebuild the dashboard around what needs a person
The board was one card per source in a fixed grid, and both halves of that hurt.

Every source got the same small box whatever it had to say. Warranties showed 5
of 50 behind "and 45 more"; the crash card truncated at 4; a one-row card sat
beside a four-row card and left a hole. Nothing had room, which is the complaint
that started this: not enough space to relay the data.

And a box per source flattened priority. An enforcement failure and a warranty
notice looked equally urgent because each owned a card, while the largest type
on the page belonged to counts that are true every day and need nobody.

WHAT THE PAGE IS NOW

A severity rollup, one worst-first queue of everything that needs a person, and
a row of context beneath it.

The queue merges every critical and warning row and sorts them, so reading order
is work order. It is full width, which is where the room came from. Rows are
capped per source, because fifty expiring warranties must not bury two crashed
PCs, and what did not fit is named per source with a link to it - a bare "and 45
more" that leads nowhere is worse than saying nothing.

The rollup counts ROWS, not cards, and filters the board in place rather than
navigating away, so the counts and the rows cannot disagree.

Severity, position and permission were already declared per card and already
sorted on. The old layout threw that away; this only stops it.

WHAT WENT, AND WHY IT SHOULD NOT COME BACK

Four asset tiles plus an "All assets" tile: machines + PCs + printers + network
IS all assets, so the page showed a total beside its own addends at the same
weight. It is now a line under a rule, where a total belongs.

The printer strip. It carried the printer count a third time - the tile row and
"Total Printers" were already the same number - and "Online" equalled total, so
three tiles held one fact. A genuinely low cartridge already reaches the queue
through the printers-supplies card, which is the moment anyone wants it.

Three cards whose whole job was to display 0.

Recent Devices: the largest block on the page, listing five machines that were
all "In Use" with business unit "TBD". Recency is not a reason to look at
something.

CONTRACT 0.26.0: 'shortname'

Cards may declare a short label for their rows in the merged queue. Their
'title' is written as a heading - "Endpoints with crashes or shutdowns" - and as
a row badge it ran wider than the hostname it qualified, putting the badge
rather than the asset at the start of every line. CSS-truncating the title was
tried first and was worse: four consecutive rows reading "ENDPOINTS WITH CR..."
say nothing at all. Optional and defaults to the title, so a plugin that does
not declare one is unaffected. Additive field, minor bump.

ALSO: "silent 1 days"

A meta suffix is written for the plural case and the card cannot know the value.
One row saying it was a shrug; the merged queue puts every quiet endpoint on one
screen, so it said it five times in a column and read as broken. Now singularised
at exactly 1, on the first word, keeping what follows: "1 event (48h)", "silent 1
day". The test asserting ' ms' survives is what caught the first version turning
it into ' m'.

DashboardRow.vue is extracted because the queue and the context panels render
the same row, and the map tooltip, link fallbacks and chips are too easy to get
subtly different in a second copy.

Verified on screen against live data. Full suite 2178 passed, frontend 301.
2026-08-29 21:05:48 -04:00

522 lines
24 KiB
Python

"""Backups plugin main class.
Per-asset configuration backups with revision history, pluggable by kind
(services/registry.py). Accepts collector input over HTTPS per ADR-006 and
renders one asset panel per kind via the ADR-010 hook.
Dedup is the load-bearing behaviour: GE-Enforce runs the collector every cycle
across the fleet, so apply_collector_payload inserts a revision only when the
content hash differs from that asset's latest for that kind. Without it a
readable history would be buried under millions of identical rows.
"""
import base64
import json
import logging
from datetime import datetime, timezone
from pathlib import Path
from typing import Dict, List, Optional, Type
from flask import Flask, Blueprint
from shopdb.plugins.base import BasePlugin, PluginMeta
from shopdb.api import Setting
from .api import backups_bp
from .models import BackupRevision
from .services.registry import (REGISTRY, getkind, canonicalhash,
DEFAULTSHAREROOT)
from .services.retention import prune, samesource as _samesource
logger = logging.getLogger(__name__)
class BackupsPlugin(BasePlugin):
"""Configuration backup + revision history plugin."""
def __init__(self):
self._manifest = self._load_manifest()
def _load_manifest(self) -> Dict:
manifest_path = Path(__file__).parent / 'manifest.json'
if manifest_path.exists():
with open(manifest_path, 'r') as f:
return json.load(f)
return {}
@property
def meta(self) -> PluginMeta:
return PluginMeta(
name=self._manifest.get('name', 'backups'),
version=self._manifest.get('version', '1.0.0'),
description=self._manifest.get('description', 'Configuration backups'),
author=self._manifest.get('author', 'ShopDB Team'),
dependencies=self._manifest.get('dependencies', []),
core_version=self._manifest.get('core_version', '>=0.2.0,<1.0.0'),
api_prefix=self._manifest.get('api_prefix', '/api/backups'),
)
def get_blueprint(self) -> Optional[Blueprint]:
return backups_bp
def get_models(self) -> List[Type]:
return [BackupRevision]
def get_permissions(self) -> List:
"""RBAC for the human-facing routes.
Deliberately split: download is separate from view because a .reg is a
restorable artifact, so a read-only auditor should be able to see that a
machine's config changed without being able to pull a file that
reconfigures it.
No delete permission. Revisions are removed only by the retention
policy; a hand-delete route would let someone quietly drop the evidence
of a config change, and there is no operational need for it. Collector
INGEST is not covered here at all - that path authorizes on a
collector.ingest-scoped service token, not a user JWT.
"""
return [
('backups.view', 'View configuration backups', 'backups'),
('backups.download', 'Download configuration backups', 'backups'),
]
def get_asset_panels(self) -> List[Dict]:
"""One panel per kind, plus the DNC Info card.
DNC Info sits ABOVE the history panels (lower position) because it
answers the question a tech actually arrives with - what is this
machine's controller talking to - while the history panels are for the
rarer restore case.
"""
panels = [kind.infopanel() for kind in REGISTRY.values()
if kind.infopanel()]
for position, kind in enumerate(REGISTRY.values()):
panel = {
'id': 'backups-{}'.format(kind.key),
'title': kind.displayname,
'assettypes': list(kind.assettypes),
'endpoint': '/api/backups/asset/{assetid}?kind=' + kind.key,
'render': 'list',
'map': {
'title': 'label',
'meta': [
{'key': 'collectedat', 'label': 'Captured', 'format': 'date'},
# Captured is when this config was WRITTEN; verified is
# when a PC last posted it and the hash still matched.
# An unchanged config writes no revision, so without
# this a machine checked daily for a year shows one
# entry from last spring and reads as abandoned.
{'key': 'lastseenat', 'label': 'Verified', 'format': 'date'},
{'key': 'sourcehostname', 'label': 'From'},
{'key': 'shorthash', 'label': 'Hash', 'mono': True},
],
},
# The generic renderer has no per-item actions, and downloading
# a .reg needs a per-revision control plus a dialect choice, so
# the panel links to the plugin's own history view instead.
'manage': {
'to': '/backups/asset/{assetid}',
'label': 'History / download',
'emptylabel': 'View history',
},
'position': 40 + position,
}
# OMIT 'empty' entirely unless the kind sets it. The list renderer
# shows a panel when it has rows OR declares empty text, so a
# present-but-null 'empty' would still render the panel on every
# asset of the type. Most machines are not part markers and never
# will be, so that panel must disappear rather than announce it has
# nothing.
if kind.emptytext:
panel['empty'] = kind.emptytext
panels.append(panel)
return panels
def get_settings_cards(self) -> List[Dict]:
"""ADR-010: contribute the Backups settings card.
The core settings catalog (settingsNav.js) is a hardcoded list a plugin
cannot extend, so this plugin shipped five settings and no page for any
of them - backups_shareroot and the collection interval were reachable
only through the API or a shell on the server, which is not a
configurable setting in any useful sense.
Filed under Machines because that is where the assets these back up
live, and it puts the card next to Machine Types rather than in a
platform group nobody visits.
"""
return [
{
'group': 'Machines',
'to': '/settings/backups',
'icon': 'package',
'title': 'Backups',
'description': 'Collection interval, share root for part-marker '
'snapshots, staleness threshold and revision retention',
'position': 21,
},
]
def get_settings_defaults(self) -> List[Dict]:
"""Schedule + retention, read by the collector and the prune job.
The interval lives here rather than in the manifest entry because
GE-Enforce fires every cycle (several times a day on a shift change).
The collector reads backups_intervalhours and no-ops when its marker
file is younger than that, so the cadence is changed centrally in
ShopDB instead of by editing a script on the share.
"""
return [
{
'key': 'backups_intervalhours',
'value': '24',
'valuetype': 'integer',
'category': 'backups',
'description': 'Minimum hours between collection attempts on a '
'PC. GE-Enforce runs every cycle; the collector '
'skips until this much time has passed.',
# MUST be public. The collecting script reads this from
# /api/settings/public BEFORE it has any credential, and its
# Get-IntervalHours falls back to 24 on any failure - silently.
# Left off the allowlist, the setting looked configurable and
# was not: every PC used 24 no matter what the UI said. A
# collection cadence is not a secret; the share root beneath it
# is internal topology and stays private.
'public': True,
},
{
'key': 'backups_shareroot',
'value': DEFAULTSHAREROOT,
'valuetype': 'string',
'category': 'backups',
'description': 'UNC root that opaque (non-JSON) backups are '
'written under by the collecting PC, e.g. '
'\\\\fileserver\\shopfloor\\backups. Blank until '
'the site sets it: no default can be right at '
'more than one site (ADR-015).',
},
{
'key': 'backups_staledays',
'value': '3',
'valuetype': 'integer',
'category': 'backups',
'description': 'Days without a CONFIRMED check before a backup '
'is listed as stopped on the dashboard. Measured '
'from the last check, not the last change - an '
'unchanged config writes no revision. 0 disables '
'the card.',
},
{
'key': 'backups_retentioncount',
'value': '50',
'valuetype': 'integer',
'category': 'backups',
'description': 'Distinct revisions kept per asset per kind. '
'0 keeps everything. The newest and the oldest '
'are never pruned.',
},
{
'key': 'backups_retentiondays',
'value': '0',
'valuetype': 'integer',
'category': 'backups',
'description': 'Also prune revisions older than this many days. '
'0 disables age-based pruning.',
},
]
def get_dashboard_widgets(self) -> List[Dict]:
"""Dashboard card: backups that have stopped running.
Keyed on the last CONFIRMED check, never on the last change. Dedup means
an unchanged config writes no revision, so a card keyed on revision age
would flag most of a healthy fleet.
"""
return [
{
'id': 'backups-stale',
'title': 'Backups stopped',
'shortname': 'Backups',
'endpoint': '/api/backups/dashboard/stale',
'render': 'exceptions',
'severity': 'warning',
'permission': 'backups.view',
'empty': 'hide',
'position': 30,
'map': {
'title': 'assetnumber',
'detail': 'backupkind',
'meta': [
{'key': 'sourcehostname', 'label': 'from'},
{'key': 'quietdays', 'label': 'last checked',
'suffix': ' days ago'},
],
'link': '/backups/asset/{assetid}',
},
},
]
# ---- ADR-006 collector contract -------------------------------------
def get_collector_schema(self) -> Optional[dict]:
return {
'identityfield': 'machinenumber',
'fields': {
'type': 'object',
'required': ['machinenumber', 'backupkind'],
'properties': {
'machinenumber': {
'type': 'string',
'description': 'Machine number the config belongs to. '
'Resolved against assets.assetnumber.',
},
'backupkind': {
'type': 'string',
'enum': sorted(REGISTRY.keys()),
},
'contentbase64': {
'type': 'string',
'description': "Raw backup file, base64. Required for "
"'shopdb' kinds (parsed server-side).",
},
'contenthash': {
'type': 'string',
'description': "sha256 of the file. Required for 'share' "
"kinds, whose bytes stay on the SFLD share.",
},
'sharepath': {
'type': 'string',
'description': "Full UNC path. Required for 'share' kinds.",
},
'manifest': {
'type': 'object',
'description': "Optional description of a 'share' kind's "
"contents - what is in the snapshot, not "
"the bytes themselves. Lets an asset panel "
"list a snapshot and diff it against the "
"previous one without the app server "
"mounting the share.",
},
'sourcefilename': {'type': 'string'},
'sourcehostname': {
'type': 'string',
'description': 'PC the backup was read from. For NTLARS '
'this is the controlling PC, not the machine.',
},
'collectedat': {'type': 'string', 'format': 'date-time'},
'bytesize': {'type': 'integer'},
},
},
}
def apply_collector_payload(self, payload: dict) -> dict:
from shopdb.api import db
warnings = []
kindkey = (payload.get('backupkind') or '').strip().lower()
kind = getkind(kindkey)
if kind is None:
raise ValueError('unknown backupkind: {!r}'.format(kindkey))
assetid, err = kind.resolveassetid(payload)
if assetid is None:
# Unresolvable is reported, never silently dropped - an unmatched
# machine number means a real gap between the fleet and ShopDB.
raise ValueError('could not resolve asset: {}'.format(err))
projection = None
sharepath = None
sourcefilename = payload.get('sourcefilename')
bytesize = payload.get('bytesize')
if kind.storagebackend == 'shopdb':
encoded = payload.get('contentbase64')
if not encoded:
raise ValueError(
"contentbase64 is required for kind '{}'".format(kindkey))
raw = base64.b64decode(encoded)
bytesize = bytesize or len(raw)
projection = kind.parse(raw)
contenthash = canonicalhash(projection)
# The fleet's machine number (pc-config.txt) is authoritative for
# WHICH asset this belongs to, but NTLARS carries its own MachineNo.
# A disagreement means the PC is running another machine's config -
# worth surfacing rather than quietly filing under the reported one.
embedded = getattr(kind, 'embeddedmachineno', None)
if embedded is not None:
embeddedno = embedded(projection)
reported = (payload.get('machinenumber') or '').strip()
if embeddedno and reported and embeddedno != reported:
warnings.append(
'reported machine number {!r} but NTLARS is configured '
'for {!r}'.format(reported, embeddedno))
else:
contenthash = (payload.get('contenthash') or '').strip().lower()
sharepath = (payload.get('sharepath') or '').strip()
if not contenthash or not sharepath:
raise ValueError(
"contenthash and sharepath are required for kind '{}'".format(
kindkey))
# A share kind may also describe itself. The bytes stay on the
# share - that part is unchanged - but a manifest of what the
# snapshot CONTAINS is small, structured, and the only way an asset
# panel can list a part marker's patterns or say which one
# disappeared last month. The alternative is the app server
# mounting SFLD, which the read routes deliberately refuse to do.
#
# It is advisory: dedup still keys on contenthash, so a client that
# sends no manifest behaves exactly as before.
manifest = payload.get('manifest')
if manifest is not None:
# Validated at the door, not at render time. A shallow check
# here would let a malformed manifest become the LATEST
# revision, and every view of the asset panel would then fail
# until the content next changed - a client bug turning into a
# broken page that outlives it.
if not isinstance(manifest, dict):
raise ValueError('manifest must be an object')
files = manifest.get('files')
if files is not None:
if not isinstance(files, list):
raise ValueError('manifest files must be a list')
for entry in files:
if not isinstance(entry, dict):
raise ValueError(
'each manifest file must be an object')
if not isinstance(entry.get('name'), str) or not entry['name']:
raise ValueError(
'each manifest file needs a name')
size = entry.get('bytes')
if size is not None and not isinstance(size, int):
raise ValueError(
'manifest file {!r} has a non-integer size'.format(
entry['name']))
projection = manifest
# Dedup is against the LATEST row of THIS SOURCE's chain, not the
# asset's. Several PCs legitimately share one machine number here - the
# part markers on 0613, 0615 and WJPRT do, and their configs genuinely
# differ, typically by COM port. Keyed on
# asset alone, each marker's post differed from whichever marker posted
# last, so nothing ever deduped and the chain grew by one revision per
# PC per collection cycle. Keyed on the source, an unchanged config is a
# no-op again and each marker keeps its own history against the machine.
#
# Dedup is against the latest row only, not any row, because a config
# reverting to an earlier state is a legitimate new revision.
#
# with_for_update serializes concurrent posts for the same chain: the
# fleet collects on one GE-Enforce cycle, so a retry could otherwise
# pass the hash check twice and insert the same revision.
# Normalize ONCE, and use the same value to query and to store.
# samesource() maps '' to IS NULL, but the insert below used to store
# the raw '' - so a client posting an empty hostname wrote rows that its
# own next post could never match. Every post then deduped against
# nothing, inserting forever: unbounded growth, and a history no
# technician can read. Whitespace does the same thing more quietly.
sourcehostname = (payload.get('sourcehostname') or '').strip() or None
latest = (db.session.query(BackupRevision)
.filter(BackupRevision.assetid == assetid,
BackupRevision.backupkind == kindkey,
_samesource(sourcehostname))
.order_by(BackupRevision.backuprevisionid.desc())
.with_for_update()
.first())
if latest is not None and latest.contenthash == contenthash:
# Unchanged, so no revision - but RECORD THE CHECK. Without this the
# only evidence a backup still runs is a line in a log on the PC,
# and a config stable for six months is indistinguishable from a
# backup that died six months ago. Cheap: one column on a row that
# already exists, no new history.
latest.lastseenat = datetime.utcnow()
# BACKFILL, on a no-op. The contenthash attests the BYTES, not the
# manifest, so a client that starts describing itself posts the same
# hash as before and would otherwise no-op forever - leaving the
# index unstored on exactly the markers that are healthy and
# unchanging, which is most of them. A revision that already has a
# manifest is left alone; this only fills a hole.
#
# Honest by construction: the hash proves the content is identical,
# so indexing it now describes the same snapshot. lastseenat
# semantics are untouched.
if projection is not None and latest.payload is None:
latest.payload = projection
# Same reasoning for the pointer: a snapshot that moved on the share
# (a changed layout, a renamed asset directory) reports a new
# sharepath with unchanged bytes, and the stored path would
# otherwise go on pointing at a directory nobody writes to.
if sharepath and latest.sharepath != sharepath:
latest.sharepath = sharepath
db.session.flush()
return {
'action': 'noop',
'assetid': assetid,
'backuprevisionid': latest.backuprevisionid,
'warnings': warnings,
}
collectedat = None
rawcollected = payload.get('collectedat')
if rawcollected:
try:
collectedat = datetime.fromisoformat(rawcollected.replace('Z', '+00:00'))
if collectedat.tzinfo is not None:
# Convert, don't just drop the offset: createdat is utcnow(),
# so keeping local wall time would skew a -04:00 bay by 4h
# against its own history.
collectedat = collectedat.astimezone(timezone.utc).replace(
tzinfo=None)
except ValueError:
warnings.append('unparseable collectedat: {!r}'.format(rawcollected))
revision = BackupRevision(
assetid=assetid,
backupkind=kindkey,
storagebackend=kind.storagebackend,
contenthash=contenthash,
sharepath=sharepath,
sourcefilename=sourcefilename,
bytesize=bytesize,
sourcehostname=sourcehostname,
collectedat=collectedat or datetime.utcnow(),
lastseenat=datetime.utcnow(),
)
revision.payload = projection
db.session.add(revision)
# flush, not commit: the collector dispatcher owns the transaction and
# commits after writing its AuditLog row. Committing here would leave an
# unaudited revision behind if that write then failed.
db.session.flush()
pruned = prune(
assetid, kindkey,
# Setting.get, NOT self.get_setting. The BasePlugin helper
# namespaces what it reads to `plugin.backups.<key>`, while
# get_settings_defaults declares - and the settings page writes -
# the bare key. So this read never found the value an operator had
# set, always fell back to 0, and 0 means "keep everything": the
# retention policy was configurable in the UI and did nothing.
retentioncount=Setting.get('backups_retentioncount', 0),
retentiondays=Setting.get('backups_retentiondays', 0),
)
if pruned:
warnings.append('pruned {} old revision(s) per retention policy'
.format(pruned))
return {
'action': 'created',
'assetid': assetid,
'backuprevisionid': revision.backuprevisionid,
'warnings': warnings,
}
def init_app(self, app: Flask, db_instance) -> None:
logger.info('Backups plugin initialized (v%s, kinds: %s)',
self.meta.version, ', '.join(sorted(REGISTRY)))