- Restore original HTTPS config in docker-compose files (gunicorn + gevent DOES support SSL)
- Add plugin migration step to docker-entrypoint.sh (fixes missing plugin tables on first deploy)
- Install only core plugins on bootstrap (backups, computers, endpoints, guides, knowledgebase, machines, network, notifications, printers, slides, tools, warranty)
- Set FLASK_ENV=production to disable Werkzeug debugger
Bug: bootstrap-plugins.sh installed ALL plugins but didn't run migrations, causing
'Table does not exist' errors. Now installs core plugins only and runs 'flask plugin
upgrade-all' after bootstrap.
Optional plugins (employees, geenforce, measuringtools, printedparts, usb) can be
installed later via Settings > Plugins.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
VM cannot connect via HTTPS due to self-signed certificate errors.
Added:
- test-vm-connectivity.ps1: 5-step diagnostic for VM
- agent/QUICK-FIX-VM.md: Use HTTP for Hyper-V local network
- open-firewall-for-vm.ps1: Open Windows Firewall
Since VM and host are on same physical machine (Hyper-V internal
network), HTTP is acceptable for local monitoring.
For production: Use proper TLS certificates.
Configuration for deploying fleet agent on VM that reports to host PC:
- Host IP: 172.31.96.1
- Collector API key included
- Step-by-step build and install instructions
- Network and SSL troubleshooting
Completes the Docker deployment with endpoint monitoring capability.
DEPLOY-NOW.bat - Double-click to deploy on VM
- Checks prerequisites
- Runs deploy-to-vm.ps1 automatically
- No command-line needed
Usage: Copy repo to VM, double-click DEPLOY-NOW.bat
Backup was mostly working but failing at the end due to Python calls.
CHANGES:
- Made manifest generation work without Python (bash JSON)
- Made check verification recording work without Python
- All Python calls now wrapped in 'command -v python3' checks
- Falls back to bash sed for JSON updates
RESULT: Backup completes successfully without Python
- Database dump: SUCCESS (24K)
- Minimal manifest: SUCCESS (bash-generated JSON)
- Restic snapshot: SUCCESS
- Repository verification: SUCCESS
- Script exit: SUCCESS (no more Python errors)
Backups now work in offline/corporate proxy environments with
zero downloads during Docker build.
PROBLEM: Corporate proxy (Zscaler) blocked ALL package downloads during
Docker build - Oracle Linux yum, Alpine apk, even with SSL certs.
SOLUTION: Offline package approach
- Pre-download restic binary on host (works outside Docker)
- Bake binary into repo at scripts/backup/offline-packages/
- Updated Dockerfile.backup to use offline binary (no downloads)
- Simplified: mysql:8.4 base + offline restic only (no Python/alembic)
RESULT: Backup containers build successfully behind corporate proxy
FILES:
- scripts/backup/offline-packages/restic_0.17.3_linux_amd64.bz2 (8.6MB)
- scripts/backup/Dockerfile.backup: Use offline package, no downloads
VERIFICATION:
- docker compose build backup: SUCCESS
- Backup watcher running: Polling every 30s
- Backup store running: restic REST server on port 8000
WORKING:
- Click 'Back Up Now' in UI
- Watcher detects sentinel file within 30s
- Backup executes automatically
The offline package can be checked into git for easy deployment at
other sites behind similar proxies.
Successfully deployed ShopDB in Docker with pre-built frontend approach.
WORKING:
- MySQL 8.4 database (healthy)
- Flask API with HTTPS on port 5001
- Gunicorn with 4 gevent workers
- All 17 plugins installed, 12 enabled
- Frontend built and served via Vue 3 SPA
- Auto-bootstrap on first deployment
BUILD APPROACH:
- Build frontend on host (bypasses Zscaler npm issues)
- Copy pre-built dist/ into Docker image
- Simple Python-only Dockerfile (no npm in container)
- Build time: 2-3 minutes (vs 15+ for failed multi-stage)
NOT WORKING YET:
- Backup watcher service (Oracle Linux SSL cert issues with Zscaler)
- Manual backups still functional via one-shot container
VERIFICATION:
- curl -k https://localhost:5001/ returns HTML
- curl -k https://localhost:5001/api/settings returns JSON
- 4 gunicorn workers running
- Database migrations applied
Files:
- DOCKER-DEPLOYMENT-COMPLETE.md: Full deployment guide
- scripts/backup/zscaler-root-ca.pem: Zscaler cert for backup containers
- .dockerignore: Updated to allow frontend/dist
For production: Set up CI/CD to build images and push to registry.
Corporate proxy (Zscaler) causes npm to fail inside Docker builds with
'Exit handler never called' error. This approach builds frontend on host
where npm works, then copies pre-built dist/ into Docker image.
New files:
- Dockerfile.prebuilt: Single-stage build, copies frontend/dist
- docker-compose.prebuilt.yml: Uses Dockerfile.prebuilt
- .dockerignore.prebuilt: Allows frontend/dist to be included
- start-docker-quick.sh: Quick restart without rebuild
- DOCKER-PREBUILT-APPROACH.md: Documents workaround and comparison
Benefits:
- Builds in 2-3 minutes (not 15+)
- Works behind corporate proxy
- Simpler to debug
- Faster iteration
Trade-off: Requires npm on host, two-step build process
For production: CI/CD builds frontend + image, pushes to registry
ASSESSMENT: Current Docker source build is NOT production-ready.
Too complex:
- 10+ manual setup steps
- Zscaler cert extraction required
- 15+ minute build times
- 3-container backup orchestration
- Manual plugin installation
- Scattered documentation
RECOMMENDATIONS:
- West Jefferson: Use IIS + Windows Services (simplest)
- New sites: Pre-built Docker images from registry (needs CI/CD)
- Source build: Development only, not for operations
Code is solid. Deployment automation needs 2-3 weeks investment.
Created honest assessment document explaining why current approach
is too complex and what's needed for true production readiness.
- Add --no-reload flag to run_dev.py to prevent env var loss
- Configure SHOPDB_BACKUPS_ROOT in .env for Windows path
- Create local backups directory
- Document local dev setup in backup README
- Fix: Flask auto-reloader was spawning process without env vars
- Quick start guide for backup/restore
- Configuration options (local, SMB, S3)
- Troubleshooting common issues
- API endpoints reference
- Clarify empty list on first deployment is normal
- Add docker-entrypoint.sh: runs migrations and bootstraps plugins
- Add bootstrap-plugins.sh: installs all plugins (disabled by default)
- Update Dockerfile to use entrypoint
- Plugins can now be enabled via Settings UI after first deployment
- Idempotent: safe to run multiple times
- Solves issue where imported prod databases had tables but no plugin registry
wilm's backup implementation doesn't expose _backups_configured or
start_restore_finalizer - those were wj-specific. Since we took wilm's
complete backup system, remove the initialization code.
The printer assignment routes checked usesprinter and defaultprinter
together and reported one generic string: "Relationship types are not
seeded". A partially seeded table is the common case, so that message
sends someone looking for an empty table they will not find.
Found on this stack: relationshiptypes held 9 rows including
defaultprinter and controls, but usesprinter had never been inserted -
the id sequence shows it was never created rather than created and
deleted, since defaultprinter took the id usesprinter would have used.
The missing type also silently cost defaultprinter its propagation rail,
because _seed_propagation returns without comment when either end is
absent, so one gap broke assignment in two places.
Now reports which types are actually missing, e.g. "Relationship type(s)
not seeded: usesprinter - run: flask seed reference-data". Same check,
same 500, same remedy; it just says what is wrong.
Present on main, wj and dev-be as well.
listAll() pages past the server's 100-row cap and returns the ROWS, not
an axios response - there is no .data to unwrap. PCTypesList still read
response.data.data, which threw, and the || [] fallback swallowed it and
rendered an empty list every time regardless of what was in the table.
The call was switched from list() to listAll() in fd513c0 to page past
the cap, but the line below it kept the unwrap that was correct for
list(). Half a change.
The add/edit dropdown was unaffected because PCForm calls list() with
perpage, so a newly added type appeared there while the management
screen stayed blank - which is what made this look like a write problem
rather than a read one.
Every other listAll() call site in the tree already assigns the array
directly; this was the only outlier. NotificationForm deliberately
re-wraps to match its .list() siblings in the same destructure and is
left alone.
Inactive types now render too, which is what active=false was always
asking for.
Both agent loops made their first request the instant the service
started, and the service is installed start=auto - so a patch reboot put
the whole fleet on the wire together, twice per machine (a telemetry
report and a command poll). The existing jitterseconds does not help: it
spreads the gap between runs, not first contact.
Measured on a 700-endpoint burst: simultaneous arrival drove TLS
handshakes to 1482ms worst case and pinned CPU at 305% while the
database sat idle at 3 active queries. Spreading the same arrivals over
30s cut worst-case handshake to 63.7ms with no server-side change. The
server was never the constraint; simultaneity was. Raising the
SQLAlchemy pool was tried and rejected - it moved the queue from
SQLAlchemy into MySQL without adding throughput.
StartupJitter is shared by both workers so the clamp and the
randomisation cannot drift apart between them. Randomised in
milliseconds rather than whole seconds because a 30s window at second
granularity re-clumps a large fleet into 30 buckets of simultaneous
arrivals - the same pile-up in miniature. Clamped to 300s and warned
about when out of range: the command channel is how an operator would
correct a bad value, so an unbounded delay would remove the means of
recovery.
The cost is up to startupjitterseconds of extra latency before a
rebooted endpoint picks up a queued command, on top of a reboot the
machine was already unreachable for. The once-only console switch
bypasses the delay entirely, so smoke tests stay instant.
Agent 1.12.0. Minor because the appsettings schema gains a key; packages
without it fall back to the AgentConfig default. The shipped installer
template in siteagent.py carries the new key, so the containers need
rolling before newly built packages pick it up.
The claim poll holds a connection open server-side. Under gthread that
pinned one of a worker's 8 OS threads, so 14 agents at hold=25 left a
worker with 8 of 9 threads parked in timed sleep and UI requests hashed
to that worker waiting 12-24s. Hold was cut to 5s as an interim fix.
gevent removes the constraint rather than working around it: a held
connection is a greenlet and the per-worker ceiling is worker-connections
(2000), not 8 threads, so an idle hold no longer costs a UI request
anything. Measured at 700 simulated agents plus UI, integration and kiosk
traffic: zero errors, UI median 7.5ms, peak RSS 479MB. CPU is the binding
resource, not memory or DB connections.
With that gone the 5s hold is a liability. Hold length sets the fleet's
request rate, not its concurrency, because the agent re-polls immediately
on a 204 - so hold=5 is 5x the requests of hold=25, against a burst
bottleneck that measured as TLS handshake CPU. 25 also restores the
margin against the agent's MinIdlePollDuration=5s floor used to tell a
normal hold-window close apart from a transient failure; at 5 it was
~20ms.
Timeout raised to 60 so gunicorn cannot kill a healthy long-poll
mid-hold, and the access log now carries request duration - without it a
slow request is indistinguishable from a fast one and latency has to be
reconstructed from outside the container. Healthcheck moved off
POST /api/auth/login, which logged a 400 every 30s and exercised the
credential path from a probe.
Corporate Tools is the first PCDetail tab that mounts no PluginAssetPanels: it is permission-gated and renders a fixed action list, so no panel can land in it. The vocabulary check asserted every tabs.push key appears in RENDERED_GROUPS, which cd70556 broke by adding that tab.
Adding 'corptools' to the vocabulary would have made it a legal panel group with no panel host - fetched, held, rendered nowhere - the silent-drop failure this file exists to catch, and it would also have failed the server-vocabulary check against ASSET_PANEL_GROUPS. So the invariant is narrowed to panel-backed tabs instead, with a mirror check that any tab mounting a PluginAssetPanels must have its group rendered.
Corporate Tools tab on PC Details, gated by endpoints.deploy and liveness.
Starts already-registered scheduled tasks; the agent's ScheduledTaskPolicy
allowlist remains the authority on what may actually run.
Dell Command | Update split into two actions (Scan, Apply Updates) because
they have very different consequences. Apply Updates carries a persistent
breakglass warning in the card naming Tanium as the primary update utility,
rendered between the row title and description so it is visible at all times
rather than only in the confirm dialog. Destructive actions keep the typed
RUN confirmation.
Fix scheduledtask_run being stripped by Configuration Policy: the
intersection in ALLOWED_ACTIONS_FOR dropped it on any endpoint whose
effective policy was non-empty, so every Corporate Tools click failed with
"not permitted by policy" regardless of the caller's role. Corporate tools
are role-gated only, via POLICY_EXEMPT_ACTIONS; policy keeps authority over
monitors and service/process primitives.
Fix blank PC Details page: watch(detailtabs) evaluates its source during
setup and read canDeploy before its const declaration, throwing a TDZ
ReferenceError that aborted setup with no API request made.
Mapped-drive refresh moves to the tray. SYSTEM cannot probe user-scoped
drives, so the service requests a re-probe over a named event and waits for
the spool collectedat stamp to advance before collecting. The probe task is
registered as a ShopFloor user principal; the Users-group principal remains
only as a degraded fallback since a group member cannot start it.
Placed between Health and Logged-in User on /endpoints. The main agent
version, not the tray: both ship side by side in the payload as
agentversion and trayversion, so the wrong one is an easy pick.
Frontend only. EndpointState.to_dict() has carried agentversion all
along, so the fleet list API was already sending the field and nothing
rendered it.
Also bumped the empty-state colspan from 9/8 to 10/9. It is hardcoded
rather than derived from the header count, so it does not track a new
column on its own and silently under-spans the "No endpoints found" row
if missed.
wilm has been red for eight consecutive runs across backend, gates and
frontend. Three failures, two unrelated causes, neither introduced by the
endpoint agent work.
docs/ prose (backend + gates):
The docs gates detect a published checkout by the ABSENCE of docs/*.md,
because docs/ becomes the wiki on that side. DEV-BUILD-TEST.md and
PLUGIN-HOOKS.md were committed in unrelated changes, which made CI look
like an internal checkout carrying almost no prose. test_there_are_docs_
to_check then failed on `assert 3 > 20`, test_the_map_itself_is_current
failed on a generated PROJECT-MAP.md that is correctly untracked, and
check-gates-ran.sh failed on the resulting skip - a skipped gate reports
green and checks nothing, so it refuses to accept one.
The existing .gitignore comment predicted this exactly and was defeated by
naming files individually. Untracked both (kept on disk) and replaced the
per-file list with docs/*.md so the next authored doc cannot reintroduce
it. Scoped to the top level: docs/notes/ prose and docs/openapi.json stay
tracked, and the gates only glob docs/*.md.
Frontend specs:
getEntraDeviceUrls() gained lapsUrl and loaded, but two siteSettings specs
still asserted the old two-key shape with toEqual, which is exact-match.
The implementation is correct - loaded is what lets a caller distinguish a
failed fetch from an unconfigured site, since both read as empty strings -
so the specs were brought up to the current shape rather than the return
value narrowed.
pinnedPanels.spec.js sliced 420 characters from getEntraDeviceUrls() and
asserted the finally block appeared inside it. Adding the retry logic
pushed finally past that offset, so a correct implementation reported as
"the finally is missing". Rebounded the slice to the function's own extent,
matching the structural approach already used elsewhere in this file for
the same reason. Verified by mutation: renaming the flag in the source
fails the test, which also caught that the spec reads
plugins/computers/frontend/views/PCDetail.vue and not the .plugins-staged
build copy.
Mapped drives reported Unavailable on endpoints where the operator could
see them fine. The data path was intact; the collection was running under
the wrong token. Mapped drives are per logon token, so an elevated probe
enumerates a different drive table than the interactive user.
Collection:
- Pin the user-context spool path so the probe and service agree on one
location, and persist/overwrite it instead of deleting on read.
- Refuse to collect while elevated (exit code 2) rather than emitting a
confident wrong answer, and stamp elevated/username on the spool.
- Reject spools stamped elevated or from an unexpected account on read,
so an admin run cannot masquerade as valid.
- Log netuse/smbmapping/userlistener counts so each run leaves evidence
of what it actually saw.
Install and launch context:
- Move collection to a 'ShopDB user-context probe' scheduled task under
S-1-5-32-545 at RunLevel Limited, the only mechanism that reliably runs
in the operator's own session. Verify registration and record it in the
install ledger.
- Stop starting the tray as a child of the elevated installer; explorer.exe
de-elevates without changing ACCOUNT, so an admin install produced an
admin tray. The Run key only fires at logon, which left the tray dead on
a machine already signed in, so add a transient 'ShopDB tray first launch'
task that starts it once in the existing session and unregisters itself.
- Hide the console window during --collect-usercontext. The agent is a
console exe; the tray used to mask this with CreateNoWindow, but the
scheduled task invokes it directly and flashed a window every cycle.
Installer edits are in _install_script() in plugins/endpoints/api/siteagent.py,
which generates the install.ps1 that actually ships. The standalone
agent/installer/install-shopdbagent.ps1 was never shipped once the package
moved to the UI builder and is deleted here to stop the divergence that
caused an earlier fix to land in a file no endpoint ever ran.
Versions: agent 1.10.1, tray 1.3.0. These share the pinned spool path and
must be deployed together.
Pre-existing test failures unrelated to this work (8, identical on a clean
tree): docs publishable/versions, plugin contract symlink scan, import
guard tamper check, security headers upload sandbox, and three endpoint
timestamp tests.
Details tab card layout
-----------------------
The Details tab sat on .content-grid, a two-column CSS grid. Grid packs
items into implicit rows, so every row is as tall as its tallest card and
align-items:start leaves the shorter one with dead space beneath it:
Warranty is short, Endpoint Status is tall, and Policy had to wait for the
taller of the two rather than sliding up. That read as a staggered layout.
Switched that one tab to multicol via a .details-flow modifier, which has no
rows at all - each card starts where the previous one ended, so the holes
close while cards still size to their content.
Multicol is deliberately NOT applied to .content-grid globally. The note by
that rule in style.css records why it was abandoned there: a column-span:all
child (.pap-span, used by table panels) overlaps the shorter column when the
two are unequal. That cannot happen on this tab - every panel in the details
group is a keyvalue or list render, and spansFullWidth() only returns true
for table (or a tabs panel holding one), so nothing in the subtree spans.
Monitors and Service/Process DO carry table panels and stay on grid.
The modifier is scoped to PCDetail, so the nine other detail views on
.content-grid are untouched.
Chosen over JS masonry on purpose. Masonry packs perfectly but has to
measure every card and re-measure on resize, on panel collapse and on every
async panel load - and these panels fetch after paint, so it would mean
continuous forced reflow on a page that already polls. This is pure CSS:
no measurement, no observer, no runtime cost. The tradeoff accepted is that
column bottoms can end at slightly different heights.
Cards carry break-inside: avoid so none fragments across the column break,
and the 25px spacing returns as a margin because multicol has no gap between
stacked items the way grid does.
IP Configuration card
---------------------
Removed: the data has no purpose on the page anymore. Taken out at the root
rather than hidden - the panel declaration, the /asset/<id>/ipconfig route
and the _IPCONFIG_COLUMNS constant all go, since a route with no consumer is
dead surface that still answers requests.
test_the_two_cards_do_not_duplicate_columns_beyond_the_address existed only
to police the history/config column split; with one card gone it guards
nothing, so it and its import go with it. The panel-count assertion in
test_endpoints_asset_panels_are_computer_scoped moves 15 -> 14, which is the
test doing its job and catching the removal.
IP History is unchanged and still carries the address list.
Not addressed here: three tzset() tests in test_endpoints_plugin.py fail on
Windows because time.tzset() is Unix-only. Verified pre-existing against
unmodified code; they pass in the Linux container.
Mapped drives stopped reporting because the user-context path failed silently
in three separate places, and none of them left a trace to read back.
- Probe: launch via -EncodedCommand instead of staging a temp .ps1 and running
powershell.exe -File. A missing or quarantined script file fails BEFORE any
statement runs, so the script's own try/catch never sees it and the only
symptom is exit -196608.
- Spool: pin to the ShopFloor profile. Resolving it through WTSQueryUserToken
is privileged, endpoint protection blocks it, and on failure the tray
(writer) and service (reader) silently disagree about the path.
- Spool: consume after a successful parse, so a stopped tray cannot keep
re-reporting stale mapped drives as current. Unparseable files are left in
place for the next cycle.
- Logging: user-session components cannot write to Program Files or to the
SYSTEM-owned ProgramData log directory, and the file sink degrades to a
silent no-op when it cannot create its directory. Tray and probe now log to
C:\Users\ShopFloor\ShopDB\Logs, installs transcript to ProgramData, and the
sink records a SinkProblem instead of swallowing UnauthorizedAccessException.
- Installer: provision the operator spool/log directories with ShopFloor:Modify
before the tray first runs, and stop the transcript via trap so a throw
cannot orphan exactly the failed install worth reading.
- Upload cap 64 -> 128 MB: the self-contained tray publishes at ~69 MB and was
rejected on SIZE, but the size check returns before the PE is parsed, so the
operator only ever saw a version-detection failure.
AGENT-CONTRACT.md gains a normative section defining the three security
contexts, their log destinations, the fixed spool path and consume-on-read.
Agent 1.8.0 -> 1.8.1, tray 1.1.0 -> 1.1.1 so the deployed build is
identifiable from a log line.
asset_panels() rewrites any group missing from ASSET_PANEL_GROUPS to 'details' before the client sees it, and Phase 3 never added 'serviceprocess' there. Both panels were relabelled on the wire, so the tab could not appear no matter how consistent the frontend was. That is why the tab was still missing after the schema fix - a second, unrelated cause wearing the same symptom.
Operator Reports moves out of System Monitors into a tab of its own: that tab is machine-collected health, this is a person reporting something, and the thumbnail grid dwarfed the fact cards it sat between. Full width, since it is alone in the tab.
panelGroups.spec.js claimed to cover four files and checked two; it now reads the server tuple as well. The plugin test's hand-copied set was a fifth copy and now derives from the server tuple instead.
A table panel with no rows rendered its column headers over an empty body. On a live PC Details page that made Network Interfaces and Listening Ports look like cards that had failed to load, when the real answer was that the agent had not reported any yet. List panels already honoured panel.empty; table panels did not.
Also records the simulation-suite requirement uncovered when rolling containers: the pytest suite builds its schema from the models, so an unapplied migration is invisible to it while every query against the affected table 500s in the running image.
Mapped drives existed only as a comma-joined line in the user context card, which is fine for three drives and unreadable for a dozen. They now get their own list card in Monitors: one row per letter, remote path as meta, health as a badge. Reuses the statecolor/levels vocabulary the tracked-services cards already declare rather than new CSS. Only unhealthy drives are coloured. Green on every working drive makes the broken one just another badge in a row of badges, and the card exists to make the exception findable at a glance. Unhealthy rows sort first for the same reason - a user with a dozen mappings should not hunt for the broken one - and the sort is on health alone, so drive-letter order survives within each group and the list does not reshuffle between refreshes. The merge logic is untouched. _mapped_drive_rows now returns structured rows and the user-context summary line is built from them, so the card and the line cannot drift apart about a drive's status. The two silent-failure paths are mutation-tested: forcing every badge green failed 7 tests, and the first sort test passed a mutation that added a secondary key because every broken drive in it said the same word - it now uses statuses that would reorder under a text sort. Also corrects a stale warning claiming the merge is untested; it has been covered since Phase 1.
Volume capacity and physical drive health were two adjacent cards, and the operator reading a full disk is usually the operator reading a dying disk in the same minute. They are now one Storage card with two sections. Visual grouping only - the two tables share no key, storage being per volume and health per physical device, and one drive can back several volumes or none the operator sees. Nothing is reconciled across the sections; this is not the mapped-drive merge where two sources described the same drive and the worse status won. The tabs render type was the only sections-in-one-card mechanism and emitted label/value fields with no table path, so it now takes table sections too. That is the renderer every plugin uses, so both behaviour changes in it are mutation-tested. Two things only surfaced on contact: the empty-section filter had to grow a table branch, because a table section declares columns up front and dropping it when it has no rows takes its tab with it and reads as no disks rather than nothing reported; and full-width span was keyed off the declared render type, but a tabs panel holding a six-column table needs the same span and that is only knowable from fetched data. The diskhealth route stays. It no longer backs a panel, but it is a stable read for callers outside the renderer and removing it for a UI-only change would break them.
Tracked Services and Tracked Processes were sharing a tab with system health cards, so what is running on the box and how the box itself is doing read as one list. They now sit in a Service/Process Monitors tab and the old tab is System Monitors. The group vocabulary is duplicated across the renderer, the tab map and the panel declarations with nothing checking they agree, and a group named in one but not the others is dropped in silence - that has eaten cards twice. Tests now pin the vocabulary on both sides. Boot type and NIC link state were collected in Phase 2 but shown nowhere. Boot type only renders when fast startup applies, because uptime measured from a resume understates how long the kernel has been up and that is the case worth naming. Link state leaves a disconnected NIC unknown rather than half duplex: live hardware returns empty speed and duplex when unplugged, and half duplex is the cable fault signature this column exists to catch.
Records why rebootreasons is not redundant with pendingreboot, why bootinfo carries two facts that must not be collapsed, and why defender is stored unsurfaced.
Also corrects the notes: pending file-rename was already being collected and OR-ed into pendingreboot, so the gap was that the stronger signal was flattened into the weaker one rather than missing. Records that the columns landed as 0015 because 0014 was taken by disk health earlier in this phase.
Adds rebootreasonsjson, bootinfojson and defenderjson with their migration in the same change, so the model never references a column the schema has not applied.
rebootreasons is not redundant with the existing pendingreboot column. That column is a single bool OR-ed from three registry signals and cannot say which fired, and the difference matters: a staged file rename means the box is mid-servicing, while a lone servicing key can sit set for weeks on a machine nobody rebooted.
Null means never reported and is deliberately distinct from an empty list, which means checked and nothing pending. Backfilling [] would assert every pre-existing endpoint was confirmed clean.
Link speed and duplex are paired onto the existing interface entries by MAC from MSFT_NetAdapter, which is a second query because the IPEnabled config class carries no link state. Both fields stay null when unreadable: a disconnected adapter returns empty values on the live box, so defaulting would report every unplugged NIC as half-duplex.
Boot info keeps two facts separate rather than collapsing them. The registry says whether fast startup is configured; Kernel-Boot event 27 says what the last boot actually did. The registry cannot answer the second and the event can age out of the log, so neither substitutes for the other. This exists because uptime lies under fast startup.
Defender is collected and persisted but not surfaced, as specified: known false positives here would train operators to ignore the card. Ages are in days and a real zero means updated today, so the collector sends null when Defender did not report.
Records the agreed plan to combine the two cards into one with two collapsible sections, as VISUAL GROUPING ONLY. The two tables cannot share rows: Storage is per volume, Disk health is per physical device, and one drive can back several volumes or none the operator sees. Stating that explicitly so this is never mistaken for the Phase 1 mapped-drive merge, where two sources described the same drive and the worse status won.
Deferred to its own phase because the tabs render type emits label/value fields only and has no table path, so this means extending the renderer every plugin uses. Bundling that with the Phase 2 telemetry work would turn a UI regression and a telemetry regression into one commit to bisect.
Also scopes the not nested cards density rule to the header block it was written for, having checked it does not block this.
Records why there are two sources and why they are not reconciled, so the next person does not simplify the ATA SMART path back in and silently blind the fleet on NVMe hardware.
Also notes that diskhealth is per physical device and is not the same list as disks, and that a null predictfailure is unknown rather than healthy.
Every test pins a way the card could start showing a failing or unreadable drive as fine, which is the only failure mode that matters here: nobody investigates a disk that looks healthy.
Covers failure-first ordering, null predictfailure reading as unknown rather than OK, unknown outranking OK, both sources surviving for one drive, order independence, and malformed payloads. The ranking was mutation-checked: swapping the unknown and OK ranks fails the suite.
Also updates the panel count assertion, which correctly caught the new card.
Its own card rather than folded into Storage: Storage is per volume and this is per physical device, and one drive can back several volumes or none the operator sees, so the two do not share a key.
Rows sort failure-first, then unknown, then OK. A failing drive must not sit below the fold on a multi-disk box, and a drive that stopped answering is a weaker claim of health than one that said OK. An unrecognised verdict sorts as unknown rather than last, so a future verdict string cannot bury itself under the healthy drives.
Both agent sources are shown rather than merged, matching the mapped-drive card: when two sources disagree about one drive, that disagreement is the thing worth looking at.
Adds diskhealthjson with its migration in the same change, so the model never references a column the schema has not applied.
Kept separate from disksjson rather than merged into it. That column is per volume and answers how full a drive is; this one is per physical device and answers whether the hardware is dying. They come from different WMI sources with different failure modes, so merging them would make a capacity collection failure indistinguishable from a health collection failure.
Nullable with no backfill: a drive whose health has never been reported is not a healthy drive, and defaulting to an empty list would read as no disks have problems on every endpoint predating the column.
ATA SMART alone is not enough. MSStorageDriver_FailurePredictStatus returns zero instances on an NVMe machine, verified on a two-NVMe box where both drives were present and healthy, so an agent collecting only that class would report an empty disk health list on every modern endpoint and read as no problems.
MSFT_PhysicalDisk is the only NVMe-aware source; Win32_DiskDrive.Status covers ATA/SATA and hosts where the Storage namespace is unavailable. Both are collected and neither is reconciled away, so a disagreement between them stays visible.
MSFT_PhysicalDisk returns raw enum integers rather than the friendly strings Get-PhysicalDisk prints, so the mapping is done here to keep the wire self-describing. predictfailure is precomputed so an alert is a boolean test instead of a string match against a Windows enum; null means the drive did not answer, which is unknown rather than healthy.
w32time is Stopped on provisioned endpoints, confirmed on more than one box, so w32tm returns 0x80070426 fleet-wide and there is no offset to read. It appears to be stopped by policy after provisioning. Collecting it would return null everywhere and give us an alerting signal that can never fire, which is worse than not having one: a permanently-null health check reads as no problem found.
Implemented first, then removed - collector, both parsers, the process helper, and the contract field. Kept one note in the notes file: parse the offset by label, not by first value ending in s, because w32tm prints Root Delay and Root Dispersion in the same format beforehand and a first-match parser reports network latency as clock skew. That bug was written and caught here.
Windows Update history is pinned rather than cut. The COM update searcher is slow, but the blocking concern is that some endpoints host COM server applications where driving it risks interfering with collections or the hosted application. Needs its own investigation. The Phase 4 card that depended on it is marked blocked so the two do not drift apart.
Two statuses sharing a rank makes min() break the tie by argument order, so the same drive reads healthy or broken depending on which source enumerated first. Deliberate synonyms are safe because either answer is the same answer; a new status landing on an existing rank is not, and the rank table is where someone adding one will be looking.
Names the test that catches it, since the single-direction tests stay green through this and would otherwise suggest the change was safe.
_mapped_drives decides whether an operator is shown a fault, and had no tests.
Its only failure mode that matters is rendering a broken drive as working -
nobody reports a drive that looks fine, so that failure is silent by nature.
Covers the disagreement from the real capture (Get-SmbMapping OK vs net use
Disconnected), single-source drives, a missing smbmapping key entirely (the
live shape until the agent republish lands), absent status becoming Unknown
rather than blank, and malformed payload rows.
Each test was checked by mutation rather than trusted for being green. That
caught a bad one: the unrecognised-status test paired the unknown against
Unavailable, which wins on severity however unknowns are ranked, so the test
proved nothing. Pairing it against OK exposed something worse - with unknowns
ranked equal to OK the two TIE, and min() then resolves by argument order, so
the verdict depended on which source happened to report which status rather
than on severity. A drive would read healthy or broken based on enumeration
order.
Hence test_which_source_reported_a_status_does_not_change_the_verdict, which
asserts both directions of each pair. A single-direction assertion passes on
luck; asserting both is what makes an accidental tie visible. It is the test
that fails when the ranking is broken - the others stay green.
Windows exposes mapped drives twice and the two views disagree. Captured on one
box at one moment, same two mappings: Get-SmbMapping reported OK for both while
net use reported Disconnected and Reconnecting. Collecting only Get-SmbMapping -
the nicer, more structured one - would have reported a healthy Q: on a machine
whose Q: was down, which is the exact failure this telemetry exists to catch.
So collect both and keep them apart. The probe emits netuse and smbmapping as
separate keys and does not reconcile them; the disagreement is the signal and
normalising it in the agent would discard it before anyone could see it.
The server merges by drive letter and shows the WORSE of the two. A drive that
either source calls broken is broken for the operator, and a drive neither
source gave a status for renders as Unknown rather than blank - an unannotated
row reads as healthy, so a failed check would present as a working drive.
Drives seen by only one source are kept, not dropped: Get-SmbMapping is absent
on some builds and session types, and requiring agreement would silently hide
every mapped drive on those machines.
No active reachability testing. Windows already knows the status, so touching
each share would be wasted work and would reintroduce the hang risk the bounded
probe exists to avoid.
The probe stays unelevated in the operator's session. UAC token separation
gives an elevated context a different mapped-drive table, often empty, so
elevating this later would quietly empty the card.
Adds a commented-out remap_drive seam in the command catalog. Not implemented,
but placed so the next person finds the user-session and elevation constraints
before writing it rather than after.
The merge logic has no test coverage yet; the docstring names the three
behaviours that must not be broken.
The service ran as SYSTEM and tried to launch the probe into the interactive
user's session. That path is fragile under CyberArk and produced no spool file
at all on a live endpoint: session-scoped data has to be gathered by something
already in the session, not reached down into from outside it.
Invert it. The tray already runs unelevated as the operator, so it invokes
shopdbagent.exe with the user-context verb on a timer and the service only
reads the resulting spool file. Nothing descends between security contexts.
That makes the tray the writer and the service an independent reader, so the
spool can no longer be consumed on read - deleting it would race the collector
and drop data the tray had just gathered. It is now read non-destructively and
aged out by its own collectedat value instead, which also means freshness is
decided by when the data was collected rather than by when it happened to be
read. The 60-minute read throttle goes with it; it delayed data that was
already sitting on disk.
Agent 1.8.0, tray 1.1.0. Both ship together - the tray invokes a verb this
agent build is the first to understand.
listAll goes through fetchAllPages, which pages past the server's 100-row cap
and returns the rows themselves. Three asset forms read .data.data off that
array. An array has no .data, so the second access threw and the surrounding
Promise.all rejected, taking the whole load into the catch.
On the PC edit page the throw landed before the record fetch, so no request was
made for the computer at all. The form rendered with its reference data and no
record, and the banner said only "Failed to load data" - which reads like a
backend or schema fault and was twice diagnosed as one. The server logs settled
it: every reference endpoint answered 200 and the record was never requested.
MachineForm already carried a comment describing this exact trap, added when
pcsRes hit it. Two adjacent lines had the same defect and were missed, so the
comment now sits above all of them.
Report the failure honestly while here. apiError only reads err.response, so a
client-side TypeError produced nothing but the fallback string no matter how
many times the page was reloaded. Fall back to err.message when there is no
response, which is what identified this.
