1bf4efa9e1a2d04a59ba5c4ff32dc09bd36a78b0
11 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
cproudlock
|
13c46af5ce |
Rebuild the dashboard around what needs a person
The board was one card per source in a fixed grid, and both halves of that hurt. Every source got the same small box whatever it had to say. Warranties showed 5 of 50 behind "and 45 more"; the crash card truncated at 4; a one-row card sat beside a four-row card and left a hole. Nothing had room, which is the complaint that started this: not enough space to relay the data. And a box per source flattened priority. An enforcement failure and a warranty notice looked equally urgent because each owned a card, while the largest type on the page belonged to counts that are true every day and need nobody. WHAT THE PAGE IS NOW A severity rollup, one worst-first queue of everything that needs a person, and a row of context beneath it. The queue merges every critical and warning row and sorts them, so reading order is work order. It is full width, which is where the room came from. Rows are capped per source, because fifty expiring warranties must not bury two crashed PCs, and what did not fit is named per source with a link to it - a bare "and 45 more" that leads nowhere is worse than saying nothing. The rollup counts ROWS, not cards, and filters the board in place rather than navigating away, so the counts and the rows cannot disagree. Severity, position and permission were already declared per card and already sorted on. The old layout threw that away; this only stops it. WHAT WENT, AND WHY IT SHOULD NOT COME BACK Four asset tiles plus an "All assets" tile: machines + PCs + printers + network IS all assets, so the page showed a total beside its own addends at the same weight. It is now a line under a rule, where a total belongs. The printer strip. It carried the printer count a third time - the tile row and "Total Printers" were already the same number - and "Online" equalled total, so three tiles held one fact. A genuinely low cartridge already reaches the queue through the printers-supplies card, which is the moment anyone wants it. Three cards whose whole job was to display 0. Recent Devices: the largest block on the page, listing five machines that were all "In Use" with business unit "TBD". Recency is not a reason to look at something. CONTRACT 0.26.0: 'shortname' Cards may declare a short label for their rows in the merged queue. Their 'title' is written as a heading - "Endpoints with crashes or shutdowns" - and as a row badge it ran wider than the hostname it qualified, putting the badge rather than the asset at the start of every line. CSS-truncating the title was tried first and was worse: four consecutive rows reading "ENDPOINTS WITH CR..." say nothing at all. Optional and defaults to the title, so a plugin that does not declare one is unaffected. Additive field, minor bump. ALSO: "silent 1 days" A meta suffix is written for the plural case and the card cannot know the value. One row saying it was a shrug; the merged queue puts every quiet endpoint on one screen, so it said it five times in a column and read as broken. Now singularised at exactly 1, on the first word, keeping what follows: "1 event (48h)", "silent 1 day". The test asserting ' ms' survives is what caught the first version turning it into ' m'. DashboardRow.vue is extracted because the queue and the context panels render the same row, and the map tooltip, link fallbacks and chips are too easy to get subtly different in a second copy. Verified on screen against live data. Full suite 2178 passed, frontend 301. |
||
|
cproudlock
|
dffa2082be |
Backups: stop storing credentials, and fix a dedup chain that never matched
The NTLARS/DNC registry carries an FTP password in cleartext and it was stored verbatim. That put it in the backuprevisions table, on the revision panel which renders values, and in the download route which hands back a ready-to-import .reg - so anyone holding backups.view could read it off a web page. Redaction happens at PARSE, so the secret never reaches the database at all. Redacting on display would have left it at rest and one query away. What is deliberately NOT done is dropping the value silently. This codec already refuses to discard an unparseable value, precisely because a backup that looks complete and restores a machine with a setting missing is the worst outcome. A redacted value keeps its name, is listed in the projection, and renders as a COMMENT naming what must be set by hand. Emitting a placeholder as the password would produce a machine that looks restored and cannot connect, and the technician would go debugging the network. scripts/redact_backup_secrets.py scrubs revisions recorded before this, dry run by default. It leaves contenthash alone: the hash identifies the configuration a bay reported, so rewriting it would make every bay's next post look like a change and insert a duplicate revision across the fleet. Separately, sourcehostname is now normalized at ingest. samesource() maps '' to IS NULL for the query while the insert stored the raw '', so rows written by a client posting an empty hostname could never match their own next post: every post inserted, the table grew without bound, and the history filled with identical entries nobody could read. Whitespace did the same thing more quietly. |
||
|
cproudlock
|
500f7d6ad6 |
Give the backups plugin a settings page, and show where a snapshot is
The plugin shipped five settings and no page for any of them. The core settings catalog is a hardcoded list a plugin cannot extend, so backups_shareroot and the collection interval were reachable only through the API or a flask shell on the server - which is not a configurable setting in any useful sense, and it is how today went: the setting simply was not there to set. The page is contributed the ADR-010 way rather than by editing core - a card from get_settings_cards plus a route in the plugin's own routes.js, both of which disappear when the plugin is disabled. Filed under Machines, next to Machine Types, because that is where the assets these back up live. It also states the two things the settings do not say for themselves: that health is judged on when a backup was last CONFIRMED rather than when it last changed, and that retention prunes database revisions only - files on the share stay, so enabling it on a share-backed kind leaves orphaned snapshot directories. Snapshot paths are now shown as full UNC text in both places: a Snapshot row on the Marker Contents card, and the existing path on the history page, which gains a copy button. Deliberately NOT a link - browsers refuse to navigate from an https page to file://, so a clickable path looks actionable and silently does nothing, which is worse than no link. Copy, then paste into Explorer. The path comes off the revision, never composed from the configured share root: a snapshot written before the layout changed still lives where it was put. |
||
|
cproudlock
|
512cc9be76 |
Let a share-backed backup say what is in it
A 'share' kind keeps its bytes on the file share and gives ShopDB only a hash and a UNC path. That is right for an opaque vendor file, but it means the app cannot answer the question a tech actually arrives with - what does this part marker hold, and which pattern disappeared last month. Reading the share from the app server is not an option: the download routes refuse to mount it on purpose, because a permissions slip there becomes an unexplained empty panel. So the collector now accepts an optional manifest alongside the hash: the index, not the content. Names, sizes, per-file hashes, and the marking system each pattern was authored for. Advisory throughout - a client that sends none behaves exactly as before, and dedup still keys on contenthash. The marker asset gains a Marker Contents card built from it: counts by type, the controller family, and a diff against the previous snapshot in the same source chain. Removed files are named rather than counted, because "1 removed" only tells someone to go and look. Files edited in place are named too - the file set can be identical while the content hash moved, and reporting no change there would contradict the reason the revision exists. The manifest is deliberately NOT covered by contenthash, which attests the bytes on the share. A client that starts describing itself therefore posts an unchanged hash, so a no-op backfills a manifest the row does not yet have - otherwise the card would appear only on markers that had recently changed, and stay hidden on every healthy one. The same no-op refreshes sharepath, so a snapshot that moved on the share stops being pointed at where it used to be. Manifests are validated at ingest rather than at render: a malformed one would otherwise become the latest revision and fail every view of the asset page until the content next changed, outliving the client bug that caused it. sharedir() for the marker kind is overridden to the layout the site actually uses, so what ShopDB displays and validates matches what the collector writes. |
||
|
cproudlock
|
4d6ab741cc |
backups: retention reads the key the settings page actually writes
get_setting on BasePlugin namespaces what it reads to plugin.backups.<key>, while get_settings_defaults declares - and the settings page writes - the bare key. So the retention read never found the operator's value and always fell back to 0, and 0 means keep everything. Retention was configurable in the UI and did nothing. It is the only place in the codebase using the namespaced helper. The share root also stops shipping one site's file server as its default. That put a site's internal topology in a bundled plugin and in the public mirror, and pointed a second site at a server it cannot reach. Blank now, per ADR-015, and a share kind with no configured root returns nothing rather than composing a path from somebody else's hostname - a path built on an empty root is not a lesser answer, it is a wrong one. |
||
|
cproudlock
|
294ddbb38e |
backups: show that a check happened, not just that a change did
lastseenat already recorded it and the API already returned it; nothing displayed it, so from the UI a healthy machine still looked abandoned - one revision from last spring and no sign anything had looked at it since. The history page gains a Last verified column beside Captured, and the asset panel a Verified field. Only the CURRENT revision carries one: an older revision was superseded, so saying it was verified today would be false - what was verified is the configuration the PC holds now. A current revision with no check yet says "not yet checked" rather than showing a blank or borrowing the captured date. That state is real and temporary: the column is new, so every chain reports it until its PC next posts. |
||
|
cproudlock
|
6c975a107c |
backups: record that a config was checked, not only that it changed
The stale-backup card could not be built as designed, and the reason is more important than the card. Dedup means an unchanged configuration writes no revision, so collectedat moves only on a CHANGE. A machine stable for six months has a six-month-old newest revision and is perfectly healthy. Keying a staleness card on revision age would have flagged most of the fleet - exactly the noise that makes a board worth ignoring. Underneath that: ShopDB could not distinguish those cases at all. On a no-op the server returned "unchanged" and wrote nothing, so "we checked yesterday and it matched" was discarded. That fact is the one thing a backup system must be able to prove, and the only record of it was a line in a log file on the PC. lastseenat records the check rather than the change. Touched on every matching post including the no-op; set on creation, since a new revision has by definition just been seen; backfilled from collectedat or createdat so existing rows start from the last moment the config can be PROVEN current, rather than from now - claiming a check that never happened would be worse than silence. The card keys on it, one row per CHAIN rather than per asset: a machine with two part markers can have one still reporting while the other stopped, and a per-asset view would report the machine as fine. It stays deliberately silent about assets never backed up, because whether one SHOULD be is a question only the manifest can answer, and guessing would list a hundred healthy machines. The rule lives in services/staleness.py rather than the route, so it is testable without an auth layer in the way - the same split retention.py uses. Threshold is backups_staledays, default 3, and 0 disables the card. |
||
|
cproudlock
|
6516e76bf4 |
backups: the collection interval was never actually readable
Backup-NtlarsSettings reads backups_intervalhours from /api/settings/public, because it runs before it holds any credential, and its Get-IntervalHours falls back to 24 on any failure. The plugin never declared the key public, so the endpoint did not return it, the fallback fired on every PC, and the setting looked configurable in the UI while changing nothing. The fleet log shows the symptom plainly: "Throttled: last attempt under 24h ago", every cycle, regardless of what the setting said. Declared public. A collection cadence is not a secret. backups_shareroot stays private - it is internal topology - and the test asserts both directions so a later edit cannot quietly widen it. Same defect as the 3D parts kiosk label prefix already in this changelog: a logged-out reader against an allowlist its key was not on. Worth noticing that the pattern has now bitten twice. |
||
|
cproudlock
|
d429c882b4 |
backups: a revision chain belongs to a PC, not just a machine
Dedup compared a posted config against the latest revision for the ASSET, which is only correct when a machine number means one PC. Several PCs share one here: the part markers on 0613, 0615 and WJPRT are separate devices, differing by COM port, filed under one machine number. Each marker's post therefore differed from whichever marker had posted last, nothing ever deduped, and the table grew by one row per PC per collection cycle. A chain is now (asset, kind, source hostname). An unchanged config is a no-op again, and each PC keeps its own history against the machine. NULL sources - rows written before the column was populated, and hand-loaded ones - form their own chain via IS NULL; `column == None` never matches in SQL, so without that those rows would have re-posted forever. Two consumers assumed the old key and are fixed with it. Retention pruned per asset, so a busy marker's revisions could evict a quiet marker's only backup; it now prunes each chain separately, protecting the newest and oldest of each. The revision diff compared against the previous revision on the machine, which across two markers reported one device's COM port as a change on the other; it now compares within the source's own chain. scripts/collapse_duplicate_backup_revisions.py cleans up what the old rule wrote. It removes only a revision whose hash repeats the one before it in the same chain - rows the fixed code would never have written - and keeps every genuine change, every chain's newest and oldest, and every source. Dry run by default. Its --report mode explains what grew each chain, which separates a legitimately shared machine number from two PCs wrongly carrying the same one, and from a value inside the config that changes on its own. |
||
|
cproudlock
|
efe2f7e6ca |
backups: show a kind's panels only when that asset has data
Every machine was getting a Part Marker Configuration panel, and only two of the 147 known machines are part markers. A kind applies to an asset TYPE, but whether a given asset ever carries that kind of backup is a property of the individual machine, so type alone cannot decide what to show. The generic renderer already handled this: a list panel is visible when it has rows OR declares empty text. Declaring emptytext on both kinds defeated it and forced them to render everywhere. emptytext now defaults to None on the base class, neither bundled kind sets one, and the panel builder OMITS the key rather than emitting null - a present-but-null 'empty' would still have kept the panel on screen. The DNC Info card carried empty text that could never be displayed, since keyvalue visibility is decided purely on field count. Removed rather than left to mislead. A lathe now shows DNC Info and NTLARS history; 0600 and 0614 additionally show Part Marker once something collects for them; a machine with no NTLARS data shows no backup panels at all. The emptytext mechanism stays available for a kind that genuinely wants to say "expected here, nothing yet". |
||
|
cproudlock
|
fca775c737 |
backups plugin: per-asset config backups with revision history
Adds a kind-pluggable backups plugin. Configuration captured from a PC is filed against the MACHINE it controls, with a revision history and download back to the native format. NTLARS/DNC is the first kind. Settings live in the controlling PC's registry but describe the machine, so revisions attach to the machine's asset and carry no foreign key to the PC: history survives a PC being replaced or deleted, and sourcehostname records the handover. Storage splits by kind. Parseable kinds store a dialect-neutral JSON projection in ShopDB and re-render on download; opaque vendor formats (part marker and similar) keep their bytes on the SFLD share with ShopDB holding metadata and the UNC pointer. Two .reg dialects exist in the wild: NTLARS's own Save... export omits the WOW6432Node path segment, scripted exports include it. Parsing strips whichever root matched, so a stored revision commits to neither and download offers both (NTLARS Load... by default, WOW6432Node for direct reg import). Getting this backwards is silent, so the dedup hash deliberately excludes sourcedialect and both dialects of one config dedup to a single revision. Dedup is load-bearing: the collector runs every GE-Enforce cycle across the fleet, so a revision is inserted only when the content hash differs from that asset's latest for that kind. A freshly imaged PC opens NTLARS with a blank General tab. Recording that would make an empty config the newest revision exactly when someone needs the last good one, so a blank MachineNo is rejected rather than accepted as a change. Two of the 320 known-good backups on the share already have that shape. DNC Info card summarises the latest revision on the machine page: General (Cnc, NcIF, HostType), eFocas, Serial, NTSHR when populated (only 18 of 147 machines), and MARK when the machine is a marker. MARK is gated on Cnc=MARKER or the ShopDB machine type, not on the MARK key having content: MARK carries serial defaults on 145 of 147 machines and DncPatterns reads YES on 103 including ordinary lathes, so neither identifies a marker. The info card is owned by the kind (BackupKind.infopanel/buildinfo) and served by a generic endpoint, so the expected successor to DNC ships its own card by adding a class rather than changing the plugin or the panel wiring. Also: schedule and retention settings with a prune that never drops the newest or the oldest revision, and scripts/import_ntlars_backups.py to seed history from the existing per-machine .reg files (144 of 147 resolve to assets). Codec verified against all 320 real backups: round-trips clean through both dialects. Bay-side generation verified on Windows against reg.exe export. |