listAll goes through fetchAllPages, which pages past the server's 100-row cap
and returns the rows themselves. Three asset forms read .data.data off that
array. An array has no .data, so the second access threw and the surrounding
Promise.all rejected, taking the whole load into the catch.
On the PC edit page the throw landed before the record fetch, so no request was
made for the computer at all. The form rendered with its reference data and no
record, and the banner said only "Failed to load data" - which reads like a
backend or schema fault and was twice diagnosed as one. The server logs settled
it: every reference endpoint answered 200 and the record was never requested.
MachineForm already carried a comment describing this exact trap, added when
pcsRes hit it. Two adjacent lines had the same defect and were missed, so the
comment now sits above all of them.
Report the failure honestly while here. apiError only reads err.response, so a
client-side TypeError produced nothing but the fallback string no matter how
many times the page was reloaded. Fall back to err.message when there is no
response, which is what identified this.
The cache held only the resolved settings object, so every caller arriving
before the first response fired its own request: nothing populated the cache
until that response landed. applyBranding awaits five settings in sequence at
bootstrap, getServicenowUrls four and getEntraDeviceUrls two, all of which
became separate GETs on a cold load.
That race also hid the Intune and Entra buttons on PC Details. The page loads
the URL templates without awaiting them so a settings failure cannot block
rendering, and on a cold cache the response could arrive after the record had
rendered, leaving the templates empty and the links hidden until a refresh
warmed the cache.
Cache the in-flight promise so concurrent callers share one request, and hold a
failed load for 30s instead of the page session. A transient blip used to blank
every settings-driven link until the operator reloaded, which they have no
reason to know would help; the window still bounds a hard-down backend to two
requests a minute per client.
The threaded-worker arguments were pasted at column 0 instead of under the api service's command list, so the file stopped parsing as YAML: compose failed with 'did not find expected key' at the services mapping and no build or up could run at all.
Indentation only - the arguments themselves are unchanged, and the stack still runs gthread with 4 workers and 8 threads on a 30 second timeout.
The hardware, OS and storage inventory had columns and data but nowhere to
be read: the only per-PC card was the status keyvalue, so a machine's
processor, firmware, disks and listening ports were reachable only by
querying the database.
Adds four Monitors cards - Hardware, Operating system, Storage, Listening
ports - split by the question each answers rather than by which JSON blob
they came from. OS and BIOS share one card deliberately; apart they are
three short rows each. A card yields no fields when its columns are null,
so an endpoint on an older agent hides them rather than showing a grid of
blanks.
User context is declared last on the page. The probe feeding it is not yet
producing output, so it reads empty for now, and being last means an empty
card sits below the populated ones instead of pushing them down.
The Activity feed showed a title and an event id, but the title is the
catalog name for that id - the same text on every occurrence - so two
different faults read identically. Rows now carry the provider name, and
the stored message body expands in place when the severity badge is
clicked. Commands expand to their result message, which is the reason a
failed one failed.
The badge doubles as the expander rather than gaining a control beside it,
and it keeps its severity label when open: swapping that for "Hide" would
cost the at-a-glance severity scan down the feed. Being a button rather
than a span, it is also now keyboard reachable.
map.body is additive in the shared renderer - panels that do not declare
it render exactly as before.
Curated events were re-read from the newest 25 entries per channel on every
collect, and the server appended whatever arrived. One real event was
therefore stored again on every cadence tick for as long as it stayed in
that window, so event counts measured how long a box had been up rather
than what had happened to it.
The agent now keeps a per-channel watermark of the last Windows event
record id it reported, persisted under ProgramData so a service restart
does not replay history, and sends eventrecordid with each event. The
server dedupes on (assetid, eventchannel, eventrecordid) behind a unique
index, so a replay from an older agent cannot double-store either.
Six inventory keys the agent had been sending all along - bios, disks,
domain, memory, osinfo, processor - were parsed, logged as unpersisted,
and dropped. They now have columns and are written on check-in, along
with user context, pending reboot, and the cpupercent/memorypercent/
idleseconds fields that were declared in the contract but always null.
Session-scoped data cannot be read from the agent's LocalSystem session 0
at all: mapped drives and user-owned listeners do not exist there. A probe
runs in the interactive user's session and hands results back through a
spool file. It is not yet producing output on a live endpoint; the column
and panel are in place and read empty until that is fixed.
Also bounds the user-context script wait. It was WaitForSingleObject with
INFINITE, so a hung script pinned the command worker permanently.
wilm was one commit behind wj. The only conflict was in the gunicorn command,
where both branches had already made the identical change (gthread, 4 workers,
8 threads, timeout 30) and only the explanatory comments differed. Kept the wj
wording, which records the production diagnosis behind the change.
Uploading an agent build whose product version carried semver build metadata
(1.5.0+abcdef) was rejected, because the typed version string was validated as
a dotted-numeric value before the file was ever read. That string is only a
hint from the operator, and the build metadata it carries is not a legal
directory name for the version store.
The authority is the dotted-numeric FILE version stamped in the binary PE
resource, so validation now happens after the upload is parsed: when the typed
value is missing or not dotted-numeric, fall back to the detected version, and
reject only when neither yields a usable one.
A restore could load a database dump whose schema was older than the running
code. The manifest stamped schemarev from the code image on disk, so the
compatibility check compared the code against itself and always passed. A set
captured from a database at 7d37_import_batches_and_journal restored under
code at 7d41_drop_guest_tools_secrets, leaving the data behind the code: every
query on a table the newer migrations touched failed with 1054 Unknown column,
which surfaced as empty configuration-policy dropdowns and a failure to load
data on edit PC.
The authoritative signal is the alembic_version of the database that was
dumped, so manifests now carry datarev alongside schemarev, backup.sh reads it
from the database and warns when it has drifted from the image, and the gate
asks whether the running code knows that revision. Data behind the code is
allowed because it can be rolled forward; data ahead of the code is refused
because it cannot.
Rolling it forward has to happen somewhere with an app context. The backup
sidecar is mysql plus restic plus alembic only and has no Flask, so restore.sh
loads the dump and parks the run in an upgrading phase behind an
.upgrade-requested sentinel rather than declaring a success it cannot verify.
The api container owns the Flask CLI and already runs these exact idempotent
commands on every deploy, so a finalize endpoint there runs db upgrade plus
plugin upgrade-all and stamps the terminal phase. The UI drives it on seeing
the upgrading phase, so recovery needs no operator intervention.
The live simulation runs one codebase against itself, so its dump is always at
the same revision as the code restoring it and it can never reproduce this
drift. The gate therefore gets a standalone tester that builds throwaway
migration chains and dumps offline, invoked as a simulation preflight. It
caught a real bug in the legacy dump-parse fallback, where the revision was
matched loosely enough to return the alembic_version table name instead of the
value it holds.
FOR DEVELOPER USE ONLY. Adds a live restore round-trip diagnostic that fingerprints every base table via CHECKSUM TABLE, backs up and restores through the real sentinel path, and proves all 110 tables return to backup-time state.
Documents expected drift so future runs are not misread: apitokens last-used churn, external endpoint-agent writes (computers/endpointstates/events/applications) that cannot be quiesced, and that live asset-row deletion is not permitted under RESTRICT/CMMC. Restore remains a full drop-and-recreate replace, so a missing or corrupted asset is still fully restored - verified, no restore.sh change needed.
Writes simulate-restore-results.txt to the script dir and ships a lightweight README with the testing cadence and parameters.
The watcher scripts now write live status and append-only run history onto the shared volume; the api exposes them via /shopdbbackups/status and /history. The restore modal shows an indeterminate spinner with phase labels, and two cards list backup and restore run records. Timestamps render in UTC to match the Zulu-stamped set ids, the history directory is excluded from the set list, and a stale-status guard stops a previous run's done phase from collapsing the spinner on the next restore.
The api queued .restore-requested but nothing consumed it, so restores silently did nothing. The watcher now services the restore sentinel by running restore.sh, and its instance_data mount is read-write so restore can lay files back down (the manual one-shot stays read-only). Backup stamps use a colon-free UTC form that Date could not parse, so the UI showed raw strings; normalize them before formatting.
alembic.ini is a Flask-Migrate config with no script_location, so ScriptDirectory.from_config raised and _current_schema_revisions swallowed it and returned no heads. Every set then failed the subset check and was wrongly blocked as taken on a newer schema than the running code. Set script_location to the migrations directory - the same versions tree the manifest scans - so live heads resolve and compatible sets restore.
Backups are per-site and not part of the base stack, so they ship as a separate compose overlay with an isolated restic REST server the app can only reach through the backup utility (no shell or db creds). The sidecar image trusts the corporate proxy root CA at build time so its Oracle yum, github, and PyPI fetches succeed behind TLS interception; shell scripts are pinned to LF via .gitattributes so the container entrypoints do not die on a CRLF shebang.
Untrusting or half-deployed agents that fail the TLS handshake retry in a loop. With 4 blocking sync workers a few of them pinned every worker, and real UI requests queued for 20+ seconds while CPU sat idle. Switch to gthread (4 workers x 8 threads) and drop the timeout to 30s so a stuck connection frees its thread quickly and one misbehaving client cannot exhaust request capacity.
The web import panel can list applied/undone batches, but the CLI had no equivalent, so an operator working from the shell had no way to find a batchid to feed into the undo command. Add a read-only batches command that reuses the same importbatches query, and a short README documenting the four WINC commands and their switches.
Machine data arrives as a Maximo asset export (validated against the WINC
site) that must update existing machines in place, not create canonical
per-table rows, so it does not fit the ShopDB-native/Azure/Thor CSV adapter
path. Bulk edits to live machines also need an escape hatch, so the whole
commit is journaled and can be undone as one batch.
Add a dedicated dry-run analyzer, approval packet, resolver/suggest helpers,
and a reversible writer/undo backed by a new importbatches/importjournalentries
schema (with migration). Expose preview/commit/batches/undo settings routes and
a matching frontend source card and panel, and surface the batch/journal
contract through shopdb.api for plugins.
Append the staged agent version to the site-agent ZIP download name so DT operators see the build at a glance; the exe name inside the ZIP is unchanged so install.ps1 is untouched. Read the versioned name from Content-Disposition on the client. Group agent/tray binary management under a collapsed 'Upload New/Legacy Agents' section and default the key explainers to collapsed.
Restructure the agent and tray binary blocks around an auto-detected current-release upload plus a separate legacy/rollback upload that stages a version without seizing current. Order staged versions by version number for the active-version dropdown, add a PE version detect route, and drop stale dev-status tags and manual-staging copy.
A plugin model change that references a column its migration has not applied yet 500s every query on that table. Make the deploy order explicit: run flask db upgrade then flask plugin upgrade-all before serving traffic, and require model+migration to ship together.
Surface versioned agent/tray uploads, current-version display, and an expandable retained-version list with rollback in the Build Site Agent card, consuming the new siteagent status fields.
Add per-version staging, retention (keep 3), and upload/rollback routes for shopdbagent.exe and the tray exe. Keep upload under endpoints.agentpackage and declare a distinct endpoints.agentupdate permission for the future live-push pipe so app-deploy rights never imply SYSTEM agent replacement.
The agent already reported agentversion but the interactive tray app, which ships and updates independently, had no reported version. The agent now reads the tray exe file version and reports it as trayversion; EndpointState caches it (migration 0011) and the Fleet card surfaces it beside Agent. The tray csproj gains an explicit Version so the file version is stamped. Groundwork for per-PC version surfacing and push-update targeting.
The gate walks the working tree with grep, which does not honor .gitignore, so it scanned devnotes/ (local scratch notes, already gitignored and never committed) and failed on pasted logs full of em-dashes and arrows. Exclude devnotes/ the same way venv, node_modules, and dist are already excluded.
Editing or retiring a PC launched from the Fleet view returned focus to the PCs list instead of Fleet. Fleet now tags its PC navigation with from=fleet, and PC Details and the edit form honor that origin for Back, Edit, Cancel, Save, and Retire. Absent the marker, every path still falls back to the PCs list.
The base compose file is the wilm/dev-be source build; stable and edge ship through the pull/airgap files. A raw source build is never a release, so default SHOPDB_ENV_LABEL to DEV here while leaving it overridable from .env.
Bring main's endpoint hardening, operator-report validation, launch-path
policy, and theme-variable CSS into wilm. Channel model reconciled to this
repo's shipping (stable=release tag, edge=main trunk) while keeping wilm's
remote-BuildKit publish step and GHCR pull-based launcher.
arc-runner-set pods have no docker daemon and cannot run a privileged DinD sidecar under cluster policy, so Set up Buildx failed to connect to /var/run/docker.sock. Point Buildx at a persistent rootless buildkitd Service (driver: remote, endpoint from the BUILDKIT_ENDPOINT secret) and add its non-privileged Kubernetes manifest. Infra only; no application code changes.
Pilots behind the inspecting proxy cannot reliably build the frontend from source; npm's many small fetches get reset mid-install. Build the image once in CI on the clean runners and have pilots pull the finished channel image. One resumable image pull survives the proxy where hundreds of npm requests do not, and a pilot now needs only Docker.
behind Zscaler each package fetch takes 40-50s; npm's default parallel
sockets pile up and the proxy resets them mid-stream, aborting npm with
'Exit handler never called' and a truncated node_modules. maxsockets=1
serializes fetches so each completes; a whole-command retry loop rides out
any residual transient reset. verified with a --no-cache build: npm ci
completes in ~55s and vite build succeeds.
a clone first created from an elevated prompt is owned by Administrators; a
later normal-user run then fails every git op with 'dubious ownership'
(exit 128), which the launcher misreported as an auth error. detect that
case and remove the clone for a clean re-clone.
behind an inspecting proxy npm's parallel fetches intermittently drop and
npm can abort with 'Exit handler never called', leaving an incomplete
node_modules with no vite - which then surfaces as a confusing
'vite: not found' at build time. raise fetch retries/timeouts to ride out
flaky connections and hard-verify node_modules/.bin/vite so a truncated
install fails loudly at the install step.
docker/compose emit warnings on stderr (e.g. nothing to remove), which
tripped ErrorActionPreference=Stop and killed the run before the clone.
soften the preference around the best-effort cleanup steps.
proxy timeouts can kill a build mid-flight, leaving orphaned containers on
the channel ports, dangling build cache, and half-written clones. clean
those on every start, scoped to the channel and keeping the DB volume, so a
re-run starts from a clean slate.
vite and the build toolchain live under devDependencies; proxied/base-image
npm defaults to omit=dev, leaving 'vite: not found' at build time. Force
--include=dev so the build stage always has its toolchain.
GE/Zscaler proxy inspection breaks npm ci silently, leaving node_modules
incomplete and vite missing at build time. Detect the root CA from the
host's chain to the npm registry and stage it as corp-root-ca.pem so the
Dockerfile trusts it; harmless on clean networks.
The frontend stage set NODE_EXTRA_CA_CERTS unconditionally, but the cert file it points at is only created when corp-root-ca.pem is supplied (gitignored, absent from a clean checkout). On a clean build - every pilot's - Node found the env var pointing at a missing file and printed a 'load failed: No such file or directory' warning on each npm step. The var is now exported inline in npm ci / npm run build only when the cert actually exists, so a clean build is quiet and a proxy-inspected build still trusts the corp CA.
The iex (irm ...) one-liner could never work against a private repo: raw.githubusercontent.com returns 404 without a token, and a git operation's Credential Manager auth does not apply to a plain irm. The prior token-passing workaround (get.ps1, 717387f) asked pilots to paste a PAT, which they will not do and may not have.
Start-ShopDBDev.ps1 is a downloaded FILE, not a web pipe, so only its git clone/fetch need auth - and those go through Git Credential Manager using the login pilots already have from cloning this repo or signing in via Visual Studio / VS Code. No token, no paste. It takes -Channel stable|edge and hands off to dev-bootstrap.ps1. DEV-BUILD-TEST.md is rewritten for the save-and-run flow with a GCM-seeding fallback.
raw.githubusercontent.com returns 404 for a private repo without a token, so the anonymous iex one-liner could never fetch get.ps1, and even once fetched its git clone of the same private repo would fail. get.ps1 now reads a GitHub PAT from SHOPDB_TOKEN (or GITHUB_TOKEN) and uses it for the clone/fetch; the raw fetch is authenticated by irm -Headers in the one-liner. The token is kept off disk: origin is reset to the clean URL after clone, and the update path fetches by explicit URL rather than a tokenized origin.
Follows the config-driven-badge decision through to packaging: a single channel-parameterized builder emits two distinct installers (stable from main, edge from dev-be) rather than two drifting scripts. Each package ships the image bundle, the airgap+channel compose templates, a pre-filled .env, and a one-command site installer.
Secrets ship as CHANGE_ME_* sentinels and Install-ShopDB.ps1 refuses to start until every one is rotated, so a distributable can never come up on shipped credentials. docker-compose.airgap.yml now threads SHOPDB_ENV_LABEL through the shared app-env anchor, otherwise a distributed edge stack would run the badge-capable image but never light the badge.
get.ps1 is the web launcher that makes the iex one-liners work: dev-bootstrap.ps1 operates on a local clone, so get.ps1 clones/updates the channel branch and hands off. docs/DEV-BUILD-TEST.md is the short reviewer handout with the two one-liners for Cameron to try.
