Commit Graph

397 Commits

Author SHA1 Message Date
benjamin.baker
7f84e8ca1e Admit serviceprocess server-side and give Operator Reports its own tab
asset_panels() rewrites any group missing from ASSET_PANEL_GROUPS to 'details' before the client sees it, and Phase 3 never added 'serviceprocess' there. Both panels were relabelled on the wire, so the tab could not appear no matter how consistent the frontend was. That is why the tab was still missing after the schema fix - a second, unrelated cause wearing the same symptom.

Operator Reports moves out of System Monitors into a tab of its own: that tab is machine-collected health, this is a person reporting something, and the thumbnail grid dwarfed the fact cards it sat between. Full width, since it is alone in the tab.

panelGroups.spec.js claimed to cover four files and checked two; it now reads the server tuple as well. The plugin test's hand-copied set was a fifth copy and now derives from the server tuple instead.
2026-09-05 07:37:01 -04:00
benjamin.baker
f77a288bdc Show empty-state text on empty table panels
A table panel with no rows rendered its column headers over an empty body. On a live PC Details page that made Network Interfaces and Listening Ports look like cards that had failed to load, when the real answer was that the agent had not reported any yet. List panels already honoured panel.empty; table panels did not.

Also records the simulation-suite requirement uncovered when rolling containers: the pytest suite builds its schema from the models, so an unapplied migration is invisible to it while every query against the affected table 500s in the running image.
2026-09-05 07:22:58 -04:00
benjamin.baker
0c9bd1ed32 add mapped drives card
Mapped drives existed only as a comma-joined line in the user context card, which is fine for three drives and unreadable for a dozen. They now get their own list card in Monitors: one row per letter, remote path as meta, health as a badge. Reuses the statecolor/levels vocabulary the tracked-services cards already declare rather than new CSS. Only unhealthy drives are coloured. Green on every working drive makes the broken one just another badge in a row of badges, and the card exists to make the exception findable at a glance. Unhealthy rows sort first for the same reason - a user with a dozen mappings should not hunt for the broken one - and the sort is on health alone, so drive-letter order survives within each group and the list does not reshuffle between refreshes. The merge logic is untouched. _mapped_drive_rows now returns structured rows and the user-context summary line is built from them, so the card and the line cannot drift apart about a drive's status. The two silent-failure paths are mutation-tested: forcing every badge green failed 7 tests, and the first sort test passed a mutation that added a secondary key because every broken drive in it said the same word - it now uses statuses that would reorder under a text sort. Also corrects a stale warning claiming the merge is untested; it has been covered since Phase 1.
2026-09-05 07:12:21 -04:00
benjamin.baker
e00bb44d6b merge storage and disk health into one card with two sections
Volume capacity and physical drive health were two adjacent cards, and the operator reading a full disk is usually the operator reading a dying disk in the same minute. They are now one Storage card with two sections. Visual grouping only - the two tables share no key, storage being per volume and health per physical device, and one drive can back several volumes or none the operator sees. Nothing is reconciled across the sections; this is not the mapped-drive merge where two sources described the same drive and the worse status won. The tabs render type was the only sections-in-one-card mechanism and emitted label/value fields with no table path, so it now takes table sections too. That is the renderer every plugin uses, so both behaviour changes in it are mutation-tested. Two things only surfaced on contact: the empty-section filter had to grow a table branch, because a table section declares columns up front and dropping it when it has no rows takes its tab with it and reads as no disks rather than nothing reported; and full-width span was keyed off the declared render type, but a tabs panel holding a six-column table needs the same span and that is only knowable from fetched data. The diskhealth route stays. It no longer backs a panel, but it is a stable read for callers outside the renderer and removing it for a UI-only change would break them.
2026-09-05 07:05:30 -04:00
benjamin.baker
af42bf5abd split monitors tab and surface boot type and nic link state
Tracked Services and Tracked Processes were sharing a tab with system health cards, so what is running on the box and how the box itself is doing read as one list. They now sit in a Service/Process Monitors tab and the old tab is System Monitors. The group vocabulary is duplicated across the renderer, the tab map and the panel declarations with nothing checking they agree, and a group named in one but not the others is dropped in silence - that has eaten cards twice. Tests now pin the vocabulary on both sides. Boot type and NIC link state were collected in Phase 2 but shown nowhere. Boot type only renders when fast startup applies, because uptime measured from a resume understates how long the kernel has been up and that is the case worth naming. Link state leaves a disconnected NIC unknown rather than half duplex: live hardware returns empty speed and duplex when unplugged, and half duplex is the cable fault signature this column exists to catch.
2026-09-05 06:58:43 -04:00
benjamin.baker
9ab1b4f702 Document Phase 2 system telemetry
Records why rebootreasons is not redundant with pendingreboot, why bootinfo carries two facts that must not be collapsed, and why defender is stored unsurfaced.

Also corrects the notes: pending file-rename was already being collected and OR-ed into pendingreboot, so the gap was that the stronger signal was flattened into the weaker one rather than missing. Records that the columns landed as 0015 because 0014 was taken by disk health earlier in this phase.
2026-09-05 06:41:28 -04:00
benjamin.baker
2c4d8ab4cb Persist Phase 2 system telemetry
Adds rebootreasonsjson, bootinfojson and defenderjson with their migration in the same change, so the model never references a column the schema has not applied.

rebootreasons is not redundant with the existing pendingreboot column. That column is a single bool OR-ed from three registry signals and cannot say which fired, and the difference matters: a staged file rename means the box is mid-servicing, while a lone servicing key can sit set for weeks on a machine nobody rebooted.

Null means never reported and is deliberately distinct from an empty list, which means checked and nothing pending. Backfilling [] would assert every pre-existing endpoint was confirmed clean.
2026-09-05 06:41:20 -04:00
benjamin.baker
f1e8fe6b4a Collect NIC link state, boot type and Defender health
Link speed and duplex are paired onto the existing interface entries by MAC from MSFT_NetAdapter, which is a second query because the IPEnabled config class carries no link state. Both fields stay null when unreadable: a disconnected adapter returns empty values on the live box, so defaulting would report every unplugged NIC as half-duplex.

Boot info keeps two facts separate rather than collapsing them. The registry says whether fast startup is configured; Kernel-Boot event 27 says what the last boot actually did. The registry cannot answer the second and the event can age out of the log, so neither substitutes for the other. This exists because uptime lies under fast startup.

Defender is collected and persisted but not surfaced, as specified: known false positives here would train operators to ignore the card. Ages are in days and a real zero means updated today, so the collector sends null when Defender did not report.
2026-09-05 06:41:13 -04:00
benjamin.baker
1c836e1c1b Scope the Storage and Disk health grouping
Records the agreed plan to combine the two cards into one with two collapsible sections, as VISUAL GROUPING ONLY. The two tables cannot share rows: Storage is per volume, Disk health is per physical device, and one drive can back several volumes or none the operator sees. Stating that explicitly so this is never mistaken for the Phase 1 mapped-drive merge, where two sources described the same drive and the worse status won.

Deferred to its own phase because the tabs render type emits label/value fields only and has no table path, so this means extending the renderer every plugin uses. Bundling that with the Phase 2 telemetry work would turn a UI regression and a telemetry regression into one commit to bisect.

Also scopes the not nested cards density rule to the header block it was written for, having checked it does not block this.
2026-09-05 06:32:11 -04:00
benjamin.baker
71847d940a Document the disk health field
Records why there are two sources and why they are not reconciled, so the next person does not simplify the ATA SMART path back in and silently blind the fleet on NVMe hardware.

Also notes that diskhealth is per physical device and is not the same list as disks, and that a null predictfailure is unknown rather than healthy.
2026-09-05 06:26:06 -04:00
benjamin.baker
c2ec29e236 Test disk health rows
Every test pins a way the card could start showing a failing or unreadable drive as fine, which is the only failure mode that matters here: nobody investigates a disk that looks healthy.

Covers failure-first ordering, null predictfailure reading as unknown rather than OK, unknown outranking OK, both sources surviving for one drive, order independence, and malformed payloads. The ranking was mutation-checked: swapping the unknown and OK ranks fails the suite.

Also updates the panel count assertion, which correctly caught the new card.
2026-09-05 06:26:00 -04:00
benjamin.baker
8edc6956da Add the disk health panel
Its own card rather than folded into Storage: Storage is per volume and this is per physical device, and one drive can back several volumes or none the operator sees, so the two do not share a key.

Rows sort failure-first, then unknown, then OK. A failing drive must not sit below the fold on a multi-disk box, and a drive that stopped answering is a weaker claim of health than one that said OK. An unrecognised verdict sorts as unknown rather than last, so a future verdict string cannot bury itself under the healthy drives.

Both agent sources are shown rather than merged, matching the mapped-drive card: when two sources disagree about one drive, that disagreement is the thing worth looking at.
2026-09-05 06:25:55 -04:00
benjamin.baker
024910d4f2 Persist endpoint disk health
Adds diskhealthjson with its migration in the same change, so the model never references a column the schema has not applied.

Kept separate from disksjson rather than merged into it. That column is per volume and answers how full a drive is; this one is per physical device and answers whether the hardware is dying. They come from different WMI sources with different failure modes, so merging them would make a capacity collection failure indistinguishable from a health collection failure.

Nullable with no backfill: a drive whose health has never been reported is not a healthy drive, and defaulting to an empty list would read as no disks have problems on every endpoint predating the column.
2026-09-05 06:25:49 -04:00
benjamin.baker
b297dea822 Collect physical disk health from two sources
ATA SMART alone is not enough. MSStorageDriver_FailurePredictStatus returns zero instances on an NVMe machine, verified on a two-NVMe box where both drives were present and healthy, so an agent collecting only that class would report an empty disk health list on every modern endpoint and read as no problems.

MSFT_PhysicalDisk is the only NVMe-aware source; Win32_DiskDrive.Status covers ATA/SATA and hosts where the Storage namespace is unavailable. Both are collected and neither is reconciled away, so a disagreement between them stays visible.

MSFT_PhysicalDisk returns raw enum integers rather than the friendly strings Get-PhysicalDisk prints, so the mapping is done here to keep the wire self-describing. predictfailure is precomputed so an alert is a boolean test instead of a string match against a Windows enum; null means the drive did not answer, which is unknown rather than healthy.
2026-09-05 06:25:42 -04:00
benjamin.baker
5d7a426bb2 Cut time sync collection, pin Windows Update history
w32time is Stopped on provisioned endpoints, confirmed on more than one box, so w32tm returns 0x80070426 fleet-wide and there is no offset to read. It appears to be stopped by policy after provisioning. Collecting it would return null everywhere and give us an alerting signal that can never fire, which is worse than not having one: a permanently-null health check reads as no problem found.

Implemented first, then removed - collector, both parsers, the process helper, and the contract field. Kept one note in the notes file: parse the offset by label, not by first value ending in s, because w32tm prints Root Delay and Root Dispersion in the same format beforehand and a first-match parser reports network latency as clock skew. That bug was written and caught here.

Windows Update history is pinned rather than cut. The COM update searcher is slow, but the blocking concern is that some endpoints host COM server applications where driving it risks interfering with collections or the hosted application. Needs its own investigation. The Phase 4 card that depended on it is marked blocked so the two do not drift apart.
2026-09-05 06:00:40 -04:00
benjamin.baker
4a8dd3f934 Warn that mapped-drive ranks must stay distinct
Two statuses sharing a rank makes min() break the tie by argument order, so the same drive reads healthy or broken depending on which source enumerated first. Deliberate synonyms are safe because either answer is the same answer; a new status landing on an existing rank is not, and the rank table is where someone adding one will be looking.

Names the test that catches it, since the single-direction tests stay green through this and would otherwise suggest the change was safe.
2026-09-05 05:53:09 -04:00
benjamin.baker
06395dc2b6 Pin the mapped-drive merge against silently rendering a broken drive as OK
_mapped_drives decides whether an operator is shown a fault, and had no tests.
Its only failure mode that matters is rendering a broken drive as working -
nobody reports a drive that looks fine, so that failure is silent by nature.

Covers the disagreement from the real capture (Get-SmbMapping OK vs net use
Disconnected), single-source drives, a missing smbmapping key entirely (the
live shape until the agent republish lands), absent status becoming Unknown
rather than blank, and malformed payload rows.

Each test was checked by mutation rather than trusted for being green. That
caught a bad one: the unrecognised-status test paired the unknown against
Unavailable, which wins on severity however unknowns are ranked, so the test
proved nothing. Pairing it against OK exposed something worse - with unknowns
ranked equal to OK the two TIE, and min() then resolves by argument order, so
the verdict depended on which source happened to report which status rather
than on severity. A drive would read healthy or broken based on enumeration
order.

Hence test_which_source_reported_a_status_does_not_change_the_verdict, which
asserts both directions of each pair. A single-direction assertion passes on
luck; asserting both is what makes an accidental tie visible. It is the test
that fails when the ranking is broken - the others stay green.
2026-09-05 05:51:25 -04:00
benjamin.baker
aa901c4111 Report mapped drives from both Windows sources, worse status wins
Windows exposes mapped drives twice and the two views disagree. Captured on one
box at one moment, same two mappings: Get-SmbMapping reported OK for both while
net use reported Disconnected and Reconnecting. Collecting only Get-SmbMapping -
the nicer, more structured one - would have reported a healthy Q: on a machine
whose Q: was down, which is the exact failure this telemetry exists to catch.

So collect both and keep them apart. The probe emits netuse and smbmapping as
separate keys and does not reconcile them; the disagreement is the signal and
normalising it in the agent would discard it before anyone could see it.

The server merges by drive letter and shows the WORSE of the two. A drive that
either source calls broken is broken for the operator, and a drive neither
source gave a status for renders as Unknown rather than blank - an unannotated
row reads as healthy, so a failed check would present as a working drive.
Drives seen by only one source are kept, not dropped: Get-SmbMapping is absent
on some builds and session types, and requiring agreement would silently hide
every mapped drive on those machines.

No active reachability testing. Windows already knows the status, so touching
each share would be wasted work and would reintroduce the hang risk the bounded
probe exists to avoid.

The probe stays unelevated in the operator's session. UAC token separation
gives an elevated context a different mapped-drive table, often empty, so
elevating this later would quietly empty the card.

Adds a commented-out remap_drive seam in the command catalog. Not implemented,
but placed so the next person finds the user-session and elevation constraints
before writing it rather than after.

The merge logic has no test coverage yet; the docstring names the three
behaviours that must not be broken.
2026-09-05 05:47:31 -04:00
benjamin.baker
511ded1ba9 Collect user context from the tray instead of descending from the service
The service ran as SYSTEM and tried to launch the probe into the interactive
user's session. That path is fragile under CyberArk and produced no spool file
at all on a live endpoint: session-scoped data has to be gathered by something
already in the session, not reached down into from outside it.

Invert it. The tray already runs unelevated as the operator, so it invokes
shopdbagent.exe with the user-context verb on a timer and the service only
reads the resulting spool file. Nothing descends between security contexts.

That makes the tray the writer and the service an independent reader, so the
spool can no longer be consumed on read - deleting it would race the collector
and drop data the tray had just gathered. It is now read non-destructively and
aged out by its own collectedat value instead, which also means freshness is
decided by when the data was collected rather than by when it happened to be
read. The 60-minute read throttle goes with it; it delayed data that was
already sitting on disk.

Agent 1.8.0, tray 1.1.0. Both ship together - the tray invokes a verb this
agent build is the first to understand.
2026-09-05 05:46:58 -04:00
benjamin.baker
539af3073f Read listAll results as rows, not as a response
listAll goes through fetchAllPages, which pages past the server's 100-row cap
and returns the rows themselves. Three asset forms read .data.data off that
array. An array has no .data, so the second access threw and the surrounding
Promise.all rejected, taking the whole load into the catch.

On the PC edit page the throw landed before the record fetch, so no request was
made for the computer at all. The form rendered with its reference data and no
record, and the banner said only "Failed to load data" - which reads like a
backend or schema fault and was twice diagnosed as one. The server logs settled
it: every reference endpoint answered 200 and the record was never requested.

MachineForm already carried a comment describing this exact trap, added when
pcsRes hit it. Two adjacent lines had the same defect and were missed, so the
comment now sits above all of them.

Report the failure honestly while here. apiError only reads err.response, so a
client-side TypeError produced nothing but the fallback string no matter how
many times the page was reloaded. Fall back to err.message when there is no
response, which is what identified this.
2026-09-05 04:39:11 -04:00
benjamin.baker
2a8dcc6db9 Load site settings once and recover from a failed fetch
The cache held only the resolved settings object, so every caller arriving
before the first response fired its own request: nothing populated the cache
until that response landed. applyBranding awaits five settings in sequence at
bootstrap, getServicenowUrls four and getEntraDeviceUrls two, all of which
became separate GETs on a cold load.

That race also hid the Intune and Entra buttons on PC Details. The page loads
the URL templates without awaiting them so a settings failure cannot block
rendering, and on a cold cache the response could arrive after the record had
rendered, leaving the templates empty and the links hidden until a refresh
warmed the cache.

Cache the in-flight promise so concurrent callers share one request, and hold a
failed load for 30s instead of the page session. A transient blip used to blank
every settings-driven link until the operator reloaded, which they have no
reason to know would help; the window still bounds a hard-down backend to two
requests a minute per client.
2026-09-05 04:38:48 -04:00
benjamin.baker
3d5c180e08 Restore the indentation on the gunicorn worker block
The threaded-worker arguments were pasted at column 0 instead of under the api service's command list, so the file stopped parsing as YAML: compose failed with 'did not find expected key' at the services mapping and no build or up could run at all.

Indentation only - the arguments themselves are unchanged, and the stack still runs gthread with 4 workers and 8 threads on a 30 second timeout.
2026-09-04 19:14:23 -04:00
benjamin.baker
9ac711ce2d Surface the endpoint inventory, and let an event show its message
The hardware, OS and storage inventory had columns and data but nowhere to
be read: the only per-PC card was the status keyvalue, so a machine's
processor, firmware, disks and listening ports were reachable only by
querying the database.

Adds four Monitors cards - Hardware, Operating system, Storage, Listening
ports - split by the question each answers rather than by which JSON blob
they came from. OS and BIOS share one card deliberately; apart they are
three short rows each. A card yields no fields when its columns are null,
so an endpoint on an older agent hides them rather than showing a grid of
blanks.

User context is declared last on the page. The probe feeding it is not yet
producing output, so it reads empty for now, and being last means an empty
card sits below the populated ones instead of pushing them down.

The Activity feed showed a title and an event id, but the title is the
catalog name for that id - the same text on every occurrence - so two
different faults read identically. Rows now carry the provider name, and
the stored message body expands in place when the severity badge is
clicked. Commands expand to their result message, which is the reason a
failed one failed.

The badge doubles as the expander rather than gaining a control beside it,
and it keeps its severity label when open: swapping that for "Hide" would
cost the at-a-glance severity scan down the feed. Being a button rather
than a span, it is also now keyboard reachable.

map.body is additive in the shared renderer - panels that do not declare
it render exactly as before.
2026-09-04 19:14:16 -04:00
benjamin.baker
862b290545 Report each Windows event once, and persist the inventory already sent
Curated events were re-read from the newest 25 entries per channel on every
collect, and the server appended whatever arrived. One real event was
therefore stored again on every cadence tick for as long as it stayed in
that window, so event counts measured how long a box had been up rather
than what had happened to it.

The agent now keeps a per-channel watermark of the last Windows event
record id it reported, persisted under ProgramData so a service restart
does not replay history, and sends eventrecordid with each event. The
server dedupes on (assetid, eventchannel, eventrecordid) behind a unique
index, so a replay from an older agent cannot double-store either.

Six inventory keys the agent had been sending all along - bios, disks,
domain, memory, osinfo, processor - were parsed, logged as unpersisted,
and dropped. They now have columns and are written on check-in, along
with user context, pending reboot, and the cpupercent/memorypercent/
idleseconds fields that were declared in the contract but always null.

Session-scoped data cannot be read from the agent's LocalSystem session 0
at all: mapped drives and user-owned listeners do not exist there. A probe
runs in the interactive user's session and hands results back through a
spool file. It is not yet producing output on a live endpoint; the column
and panel are in place and read empty until that is fixed.

Also bounds the user-context script wait. It was WaitForSingleObject with
INFINITE, so a hung script pinned the command worker permanently.
2026-09-04 19:09:12 -04:00
benjamin.baker
3714b4bc99 Merge origin/wj into wilm
wilm was one commit behind wj. The only conflict was in the gunicorn command,
where both branches had already made the identical change (gthread, 4 workers,
8 threads, timeout 30) and only the explanatory comments differed. Kept the wj
wording, which records the production diagnosis behind the change.
2026-09-04 16:52:21 -04:00
benjamin.baker
d3f69bbb71 Trust the binary PE version over the typed agent version
Uploading an agent build whose product version carried semver build metadata
(1.5.0+abcdef) was rejected, because the typed version string was validated as
a dotted-numeric value before the file was ever read. That string is only a
hint from the operator, and the build metadata it carries is not a legal
directory name for the version store.

The authority is the dotted-numeric FILE version stamped in the binary PE
resource, so validation now happens after the upload is parsed: when the typed
value is missing or not dotted-numeric, fall back to the detected version, and
reject only when neither yields a usable one.
2026-09-04 16:19:28 -04:00
benjamin.baker
b973561cb7 Gate restore on the dump's data revision, not the code head
A restore could load a database dump whose schema was older than the running
code. The manifest stamped schemarev from the code image on disk, so the
compatibility check compared the code against itself and always passed. A set
captured from a database at 7d37_import_batches_and_journal restored under
code at 7d41_drop_guest_tools_secrets, leaving the data behind the code: every
query on a table the newer migrations touched failed with 1054 Unknown column,
which surfaced as empty configuration-policy dropdowns and a failure to load
data on edit PC.

The authoritative signal is the alembic_version of the database that was
dumped, so manifests now carry datarev alongside schemarev, backup.sh reads it
from the database and warns when it has drifted from the image, and the gate
asks whether the running code knows that revision. Data behind the code is
allowed because it can be rolled forward; data ahead of the code is refused
because it cannot.

Rolling it forward has to happen somewhere with an app context. The backup
sidecar is mysql plus restic plus alembic only and has no Flask, so restore.sh
loads the dump and parks the run in an upgrading phase behind an
.upgrade-requested sentinel rather than declaring a success it cannot verify.
The api container owns the Flask CLI and already runs these exact idempotent
commands on every deploy, so a finalize endpoint there runs db upgrade plus
plugin upgrade-all and stamps the terminal phase. The UI drives it on seeing
the upgrading phase, so recovery needs no operator intervention.

The live simulation runs one codebase against itself, so its dump is always at
the same revision as the code restoring it and it can never reproduce this
drift. The gate therefore gets a standalone tester that builds throwaway
migration chains and dumps offline, invoked as a simulation preflight. It
caught a real bug in the legacy dump-parse fallback, where the revision was
matched loosely enough to return the alembic_version table name instead of the
value it holds.
2026-09-04 15:53:58 -04:00
benjamin.baker
3c9321f16c Add developer-only restore simulation harness
Some checks failed
CI / backend (push) Has been cancelled
CI / naming (push) Has been cancelled
CI / gates (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / lean-build (push) Has been cancelled
FOR DEVELOPER USE ONLY. Adds a live restore round-trip diagnostic that fingerprints every base table via CHECKSUM TABLE, backs up and restores through the real sentinel path, and proves all 110 tables return to backup-time state.

Documents expected drift so future runs are not misread: apitokens last-used churn, external endpoint-agent writes (computers/endpointstates/events/applications) that cannot be quiesced, and that live asset-row deletion is not permitted under RESTRICT/CMMC. Restore remains a full drop-and-recreate replace, so a missing or corrupted asset is still fully restored - verified, no restore.sh change needed.

Writes simulate-restore-results.txt to the script dir and ships a lightweight README with the testing cadence and parameters.
2026-09-04 05:01:03 -04:00
benjamin.baker
debe1fce69 it is risen - add restore progress and backup/restore history
The watcher scripts now write live status and append-only run history onto the shared volume; the api exposes them via /shopdbbackups/status and /history. The restore modal shows an indeterminate spinner with phase labels, and two cards list backup and restore run records. Timestamps render in UTC to match the Zulu-stamped set ids, the history directory is excluded from the set list, and a stale-status guard stops a previous run's done phase from collapsing the spinner on the next restore.
2026-09-03 20:43:42 -04:00
benjamin.baker
7e4469c44e Wire restore into the backup watcher and fix backup timestamps
The api queued .restore-requested but nothing consumed it, so restores silently did nothing. The watcher now services the restore sentinel by running restore.sh, and its instance_data mount is read-write so restore can lay files back down (the manual one-shot stays read-only). Backup stamps use a colon-free UTC form that Date could not parse, so the UI showed raw strings; normalize them before formatting.
2026-09-03 20:26:01 -04:00
benjamin.baker
231fd27f68 Fix restore compat check reporting empty schema heads
alembic.ini is a Flask-Migrate config with no script_location, so ScriptDirectory.from_config raised and _current_schema_revisions swallowed it and returned no heads. Every set then failed the subset check and was wrongly blocked as taken on a newer schema than the running code. Set script_location to the migrations directory - the same versions tree the manifest scans - so live heads resolve and compatible sets restore.
2026-09-03 20:11:52 -04:00
benjamin.baker
19071ec6c6 Add opt-in backup overlay with third-container restic store
Backups are per-site and not part of the base stack, so they ship as a separate compose overlay with an isolated restic REST server the app can only reach through the backup utility (no shell or db creds). The sidecar image trusts the corporate proxy root CA at build time so its Oracle yum, github, and PyPI fetches succeed behind TLS interception; shell scripts are pinned to LF via .gitattributes so the container entrypoints do not die on a CRLF shebang.
2026-09-03 20:06:45 -04:00
37ba30ee63 Threaded gunicorn workers so a retry storm cannot starve the UI 2026-09-03 17:38:20 -04:00
benjamin.baker
d5d823fd8b Merge remote-tracking branch 'origin/wj' into wilm 2026-09-03 17:35:48 -04:00
ff9ae693a6 Remediate the 2026-09-03 feature-test findings 2026-09-03 17:14:53 -04:00
benjamin.baker
ab949f8bec Merge remote-tracking branch 'origin/wj' into wilm
Some checks failed
CI / backend (push) Has been cancelled
CI / naming (push) Has been cancelled
CI / gates (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / lean-build (push) Has been cancelled
2026-09-03 13:46:17 -04:00
benjamin.baker
db75ca79d1 [ALERT] Serve gunicorn with threaded workers so bad clients cannot starve the UI
Untrusting or half-deployed agents that fail the TLS handshake retry in a loop. With 4 blocking sync workers a few of them pinned every worker, and real UI requests queued for 20+ seconds while CPU sat idle. Switch to gthread (4 workers x 8 threads) and drop the timeout to 30s so a stuck connection frees its thread quickly and one misbehaving client cannot exhaust request capacity.
2026-09-03 12:36:28 -04:00
ddf41c3e65 Make a partly-undone import retryable, and refuse a plan it cannot deliver 2026-09-03 09:16:01 -04:00
b0bd73f62d Import the wilm agent store, WINC adoption work and batches CLI, with four fixes 2026-09-03 08:16:38 -04:00
benjamin.baker
17825ec8f1 Add WINC batches CLI and a CLI how-to README
The web import panel can list applied/undone batches, but the CLI had no equivalent, so an operator working from the shell had no way to find a batchid to feed into the undo command. Add a read-only batches command that reuses the same importbatches query, and a short README documenting the four WINC commands and their switches.
2026-09-03 08:07:01 -04:00
benjamin.baker
2998e35ddf Add reversible Maximo/WINC machine-adoption import
Machine data arrives as a Maximo asset export (validated against the WINC
site) that must update existing machines in place, not create canonical
per-table rows, so it does not fit the ShopDB-native/Azure/Thor CSV adapter
path. Bulk edits to live machines also need an escape hatch, so the whole
commit is journaled and can be undone as one batch.

Add a dedicated dry-run analyzer, approval packet, resolver/suggest helpers,
and a reversible writer/undo backed by a new importbatches/importjournalentries
schema (with migration). Expose preview/commit/batches/undo settings routes and
a matching frontend source card and panel, and surface the batch/journal
contract through shopdb.api for plugins.
2026-09-03 05:04:22 -04:00
benjamin.baker
6f9d1de6da Stamp agent version into download name and collapse binary management UI
Append the staged agent version to the site-agent ZIP download name so DT operators see the build at a glance; the exe name inside the ZIP is unchanged so install.ps1 is untouched. Read the versioned name from Content-Disposition on the client. Group agent/tray binary management under a collapsed 'Upload New/Legacy Agents' section and default the key explainers to collapsed.
2026-09-02 20:41:22 -04:00
benjamin.baker
729e488903 Rework Build Site Agent binary UI with version auto-detect and rollback
Restructure the agent and tray binary blocks around an auto-detected current-release upload plus a separate legacy/rollback upload that stages a version without seizing current. Order staged versions by version number for the active-version dropdown, add a PE version detect route, and drop stale dev-status tags and manual-staging copy.
2026-09-02 20:27:55 -04:00
benjamin.baker
7a11851fbe Document migration-on-deploy contract for plugin schema changes
A plugin model change that references a column its migration has not applied yet 500s every query on that table. Make the deploy order explicit: run flask db upgrade then flask plugin upgrade-all before serving traffic, and require model+migration to ship together.
2026-09-02 20:27:47 -04:00
benjamin.baker
f4cf3d0b6a add agent binary upload and previous-versions UI
Surface versioned agent/tray uploads, current-version display, and an expandable retained-version list with rollback in the Build Site Agent card, consuming the new siteagent status fields.
2026-09-02 19:41:39 -04:00
benjamin.baker
04d5bdd212 stage and retain versioned agent binaries
Add per-version staging, retention (keep 3), and upload/rollback routes for shopdbagent.exe and the tray exe. Keep upload under endpoints.agentpackage and declare a distinct endpoints.agentupdate permission for the future live-push pipe so app-deploy rights never imply SYSTEM agent replacement.
2026-09-02 19:35:14 -04:00
benjamin.baker
51ad18ca81 report and cache the endpoint tray version
The agent already reported agentversion but the interactive tray app, which ships and updates independently, had no reported version. The agent now reads the tray exe file version and reports it as trayversion; EndpointState caches it (migration 0011) and the Fleet card surfaces it beside Agent. The tray csproj gains an explicit Version so the file version is stamped. Groundwork for per-PC version surfacing and push-update targeting.
2026-09-02 19:04:02 -04:00
benjamin.baker
27ccf8962e exclude gitignored devnotes from the naming/style gate
The gate walks the working tree with grep, which does not honor .gitignore, so it scanned devnotes/ (local scratch notes, already gitignored and never committed) and failed on pasted logs full of em-dashes and arrows. Exclude devnotes/ the same way venv, node_modules, and dist are already excluded.
2026-09-02 18:39:23 -04:00
benjamin.baker
86a47b6bc9 return to Fleet after editing a PC opened from Fleet
Editing or retiring a PC launched from the Fleet view returned focus to the PCs list instead of Fleet. Fleet now tags its PC navigation with from=fleet, and PC Details and the edit form honor that origin for Back, Edit, Cancel, Save, and Retire. Absent the marker, every path still falls back to the PCs list.
2026-09-02 18:37:59 -04:00
benjamin.baker
364c62d09c show the DEV badge on source-build dev stacks
The base compose file is the wilm/dev-be source build; stable and edge ship through the pull/airgap files. A raw source build is never a release, so default SHOPDB_ENV_LABEL to DEV here while leaving it overridable from .env.
2026-09-02 18:25:50 -04:00