ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: ge-aerospace:lab-stage-02
Branches Tags
ge-aerospace:main
ge-aerospace:v0.11.2 ge-aerospace:v0.11.1 ge-aerospace:v0.11.0 ge-aerospace:v0.10.0 ge-aerospace:v0.9.0 ge-aerospace:v0.8.1 ge-aerospace:v0.8.0 ge-aerospace:lab-stage-17 ge-aerospace:lab-stage-16 ge-aerospace:lab-stage-15 ge-aerospace:lab-stage-14 ge-aerospace:lab-stage-13 ge-aerospace:lab-stage-12 ge-aerospace:lab-stage-11 ge-aerospace:lab-stage-10 ge-aerospace:lab-stage-09 ge-aerospace:lab-stage-08 ge-aerospace:lab-stage-07 ge-aerospace:lab-stage-06 ge-aerospace:lab-stage-05 ge-aerospace:lab-stage-04 ge-aerospace:lab-stage-03 ge-aerospace:lab-stage-02 ge-aerospace:lab-stage-01 ge-aerospace:v0.7.0 ge-aerospace:v0.6.0 ge-aerospace:v0.5.0
...
compare: ge-aerospace:v0.10.0
Branches Tags
ge-aerospace:main
ge-aerospace:v0.11.2 ge-aerospace:v0.11.1 ge-aerospace:v0.11.0 ge-aerospace:v0.10.0 ge-aerospace:v0.9.0 ge-aerospace:v0.8.1 ge-aerospace:v0.8.0 ge-aerospace:lab-stage-17 ge-aerospace:lab-stage-16 ge-aerospace:lab-stage-15 ge-aerospace:lab-stage-14 ge-aerospace:lab-stage-13 ge-aerospace:lab-stage-12 ge-aerospace:lab-stage-11 ge-aerospace:lab-stage-10 ge-aerospace:lab-stage-09 ge-aerospace:lab-stage-08 ge-aerospace:lab-stage-07 ge-aerospace:lab-stage-06 ge-aerospace:lab-stage-05 ge-aerospace:lab-stage-04 ge-aerospace:lab-stage-03 ge-aerospace:lab-stage-02 ge-aerospace:lab-stage-01 ge-aerospace:v0.7.0 ge-aerospace:v0.6.0 ge-aerospace:v0.5.0

344 Commits
lab-stage- ... v0.10.0

Author SHA1 Message Date
cproudlock
7d9a54ca0f Release 0.10.0
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
A privilege-escalation fix in the Windows installer, the multi-site portability
work, and a documentation and API surface that a second site can rely on.

The installer created C:\shopdb-flask, C:\Python314 and C:\MySQL84 without
breaking inheritance from the drive root, whose default DACL grants
Authenticated Users an inherit-only Modify. Two files inside those roots are
executed by someone else - shopdb-admin.ps1 self-elevates, and instance
config.py is loaded unconditionally by the app - so any logged-on user could
replace either and wait. Verified fixed as SYSTEM in the Windows 11 VM against
the shipped functions.

Contract 0.19.0 is breaking: get_dashboard_widgets returns data and shape, not a
component name. The change itself shipped earlier without a bump, with the
contract docs still teaching the dead shape; both are corrected.

The version and the changelog are the release; the detail is in the entry.
 2026-08-17 08:01:52 -04:00
cproudlock
a895bde6fe README: count the plugins the tree has, and point somewhere useful 
It claimed thirteen bundled plugins and listed thirteen; there are fifteen, and
the two missing ones - backups and tools - were undiscoverable from the first
page anyone reads. The count is gone rather than corrected, because a number
typed here goes stale exactly the way it just did; the generated map carries it.

The version and count guard now covers README.md and CONTRIBUTING.md, not just
docs/. A rule that leaves the most-read file in the repository outside it is
most of a rule.

Also names the reference site as the reference site rather than by plant, adds a
pointer to START-HERE, and corrects the distribution paragraph: it told a reader
of the public mirror that the code is only available on an internal server,
which is both wrong and unhelpful now that the installer ships as a release
asset.
 2026-08-14 16:34:46 -04:00
cproudlock
e7b8933588 Stop the publication scrub failing on the rules that enforce it
Some checks failed
CI / backend (push) Has been cancelled
Details
CI / naming (push) Has been cancelled
Details
CI / frontend (push) Has been cancelled
Details
CI / migrations-mysql (push) Has been cancelled
Details 
The export gate now carries the site patterns, and three kinds of file tripped
it - two legitimately, one by construction.

Two test files held real internal subnets as fixtures. They are documentation
ranges now, which test the same logic and disclose nothing.

The project-map generator and a changelog entry named a file that is excluded
from publication, so a public reader was pointed at something they cannot see.
Both now describe what happened without naming it.

And the naming script has to CONTAIN the site patterns in order to grep for
them, so written literally the rule's own definition fails the gate that
enforces it. The patterns are assembled from fragments, the same trick the docs
publishability test already uses for the same reason. Verified the hard way: a
planted literal is still caught, so the fragmentation did not quietly turn the
rule into one that matches nothing - which is the obvious way for this fix to
have gone wrong.
 2026-08-14 16:27:58 -04:00
cproudlock
1d8c983d04 Record the documentation and API-surface sweep in the changelog 2026-08-14 16:21:16 -04:00
cproudlock
96df19702e docs: one manual runbook, and ADR statuses that mean something 
DEPLOY-WINDOWS-IIS was a second copy of the manual IIS procedure that had
diverged from the first: a different MySQL version (8.0, which reached end of
life in April), a different port, a different plugin list, and a profile file
that does not exist. Two runbooks for one procedure means a reader follows
whichever they found, and one of them was wrong. INSTALL-WINDOWS-IIS covers
everything it did plus a preflight step and the subpath method, so the one
section it uniquely had - redeploying a hand-built server - is folded in there,
with the plugin-chain step it was missing and a note to back up first, and the
duplicate is gone. Everything that pointed at it now points at the survivor.

Three ADR statuses said something untrue.

ADR-013 said PROPOSED while half of it had shipped and ADR-014 had been accepted
on top of it. A decision that has been implemented and depended upon is not
proposed, and leaving one that way devalues every other status in the index. The
catalog half is still unbuilt, which is the ordinary state of an accepted
decision: accepted means settled, not delivered.

ADR-016 said ACCEPTED for a design where nothing is built - the endpoint and
permissions it describes do not exist, so a reader goes looking for them. The
status stands, because the decision does; the header now says so plainly and
points at where today's credentials actually live.

ADR-003 and ADR-004 were ACCEPTED with their own Decision lines still opening
"**PROPOSED:**", which reads as though the decision was never taken.

And the dashboard proposal carried Status: ACCEPTED, which belongs to a decision
record. A proposal is a proposal; the contract it produced is the ADR.
 2026-08-14 16:20:59 -04:00
cproudlock
928a50c16e docs: what a site needs that no page answered 
Four gaps a second site hits and cannot resolve by reading.

**Restoring on Windows** was one sentence - "the standard mysql < dump.sql" -
with no ordering. Restoring a database under running code that expects a
different schema turns a restore into a second incident, so the steps are now
ordered and each says why. It also says what `.env` costs if it is lost, which
is the part nobody discovers until they are already rebuilding: the dump does
not contain it, and without the JWT secrets every issued token dies, so every
collector and every GE-Enforce client on the fleet needs a new key.

**Rolling back** had a paragraph saying downgrades are refused and a backup is
the way back, but not the procedure. Rollback is restoring a matched pair, code
and the schema it expects, in that order - and the doc now separates it from the
case it gets confused with: a migration that failed mid-update has already been
rolled back by the installer, and fixing forward is the only move.

**Sizing, acquisition and support** were absent from the install guide entirely.
A reader could not learn how big a server to ask for, where the .exe comes from,
or where to raise a problem. The sizing is small and the reasons are stated, so
a site does not over-provision a VM for a load that is a few dozen users.

**Credentials** were described in three documents from three ends, so three
answers existed for where a key lives. One table, both ends - server and PC -
plus the two rules behind it: what a shop-floor PC holds is scoped to exactly
what it does, and a credential is delivered rather than typed, because a value
entered per machine is a value that is wrong on some machine.
 2026-08-14 16:16:01 -04:00
cproudlock
62c5454f00 docs: a front door, and one page on how the pieces relate 
The wiki had no entry point worth the name. Its Home page was hardcoded inside
the generator, had drifted into recommending INSTALL-WINDOWS-IIS - the manual
procedure these same docs tell you not to follow for a new site, because it
produces a server the installer then refuses to upgrade - and the page carrying
that warning, llms.txt, was never published at all, because the generator copied
only .md files. The adoption guide had zero inbound links.

START-HERE routes by what a reader came to do: stand up a site, deploy the
shop-floor tools, write a plugin, integrate with the API, understand a decision,
or fix something. It is the shortest correct path per role, not an index - the
sidebar is already the index.

FLEET-ARCHITECTURE is the page nothing else could assume. The server, GE-Enforce,
the asset reporter, the backup collectors and EventSaver were each documented
alone, each assuming a reader who already knew the other four. It draws the
shape, states the rule that explains most of the behaviour - nothing on the
server reaches out to a PC, the fleet asks - and ends with the table that says
which of the five programs to open for a given symptom, since knowing that is
most of the diagnosis.

wikigen renders START-HERE as the landing page rather than a list maintained in
a second place, and publishes llms.txt as LLM-GUIDE.
 2026-08-14 16:12:04 -04:00
cproudlock
ad335cfc9e api: document the twenty-two routes that were serving traffic in silence 
The inventory is hand-maintained, and should stay that way - its value is the
prose, and what an endpoint is FOR cannot be derived from the code. An audit of
all 372 documented operations found zero phantom routes and zero wrong parameter
names, so the maintenance is in good order.

What hand-maintenance cannot do is notice a route somebody added. Twenty-two
were undocumented: the entire backups plugin surface, every one of the dashboard
card endpoints added with contract 0.19.0, the GE-Enforce publish preflight, the
employee SSO resolver, the protocol update verbs, and the four /api/docs routes -
so the spec did not describe how to fetch the spec.

Coverage is now a test. It walks the live url_map and fails when a served route
has no entry, which means adding an endpoint includes describing it, in the same
commit, while the author still knows what it is for. The reverse direction is
checked too: a documented route that no longer exists sends a reader to a 404.

Writing that test found one more thing. The inventory writes multi-verb routes
as "PUT|PATCH", and neither the parity check nor the generator split on the pipe
- so those operations were absent from the published spec entirely, with nothing
reporting it. The spec now carries all 394 operations the code serves, which is
the first time the two numbers have matched.

The generator's own docstring claimed the inventory could be regenerated. It
cannot; nothing generates it. That sentence is why nobody noticed it was falling
behind.
 2026-08-14 16:09:47 -04:00
cproudlock
36b0265668 openapi: emit a spec a machine can actually use 
The generated spec carried no `parameters` and no `requestBody` on any of its
372 operations. That is invalid OpenAPI 3.1, and the practical cost was worse
than the formal one: the MCP server builds its tools from this file, so every
tool had an empty input schema and silently dropped whatever the caller passed.
A request for one asset returned the list, and nothing anywhere reported an
error. All 118 templated paths now declare their path parameters, typed from the
Flask converter that named them, and write verbs declare a JSON body.

The body is an open object carrying the prose description rather than an
invented schema. The inventory describes bodies in sentences, and a field list
this generator guessed at would be worse than none - but "an object, described
here" is the difference between a client that can send a body and one that
cannot send anything.

Security was wrong on 123 operations. `jwt-optional` means "works logged out,
returns more logged in", which OpenAPI expresses as the empty requirement
alongside the scheme; publishing them as bearer-required told every reader that
a public endpoint needs a token.

Responses were one hardcoded 200, so a generated client had no idea a call could
fail. Every operation now documents the error envelope - and the envelope itself
is a defined schema, because its error nests under `data.error` rather than at
the top level, which is the single thing people get wrong when writing against
this API.

95 summaries were cut at 120 characters mid-word, which is what a tool picker
shows a user as the whole description of a call. They now end on a word.

Tests pin the shape rather than the prose. One of them contradicted an older
test that REQUIRED the contract version as a literal in PLUGIN-HOOKS.md - the
same copying that left nine documents stale - so that test now asserts the doc
points at the generated map instead.
 2026-08-14 15:56:27 -04:00
cproudlock
05be4c4489 docs: fix the examples that fail if you paste them 
Six procedures that could not be followed as written.

Eighty-nine curl examples single-quoted `Authorization: Bearer $TOKEN`, so the
shell never expanded it and the server answered 422 "Not enough segments". Nine
more did the same with X-API-Key. The other 129 examples in the same file
already used double quotes, so this was drift rather than a convention, and the
spec regenerated from it carried the fault onward.

The GE-Enforce report example put a `//` comment inside a JSON body. The server
parses with silent=True, so it saw `{}` and answered "hostname is required"
about a body that plainly has one - the worst kind of error message, one that
sends the reader to the wrong field entirely.

The IIS install ran `flask db upgrade` and a per-plugin install loop but never
`flask plugin upgrade-all`, leaving every plugin's own chain unapplied. That is
precisely the 1054 "Unknown column" a deploy then hits somewhere else, days
later, on the page that uses the new column.

The pilot runbook looped `flask plugin enable` over plugins that were not yet
installed; enable refuses those, so on a fresh database it exited 1 on every
iteration and enabled nothing. ADR-013 had already recorded that defect.
`apply-profile` installs and enables in dependency order, which is what the step
was reaching for.

DEPLOY-WINDOWS-IIS named a profile file that does not exist; the shipped ones do.

And PLUGIN-EXTERNAL-REPO never mentioned PLUGIN_TABLE_OWNERS, while the
migration engine raises for any plugin missing from it - so the guide's own
step 5 fails for any external plugin that owns a table. That the registry lives
in the framework repo is deliberate, so the guide now says so, and says what it
costs: a table-owning external plugin is a two-repository change, and a plugin
that owns no tables avoids it entirely.
 2026-08-14 15:43:27 -04:00
cproudlock
d1ba3a1a02 docs: stop typing versions the code already knows 
Nine documents carried a hand-typed contract version and every one was stale.
One was load-bearing: PLUGIN-EXTERNAL-REPO.md told an external author to pin
">=0.13.0,<0.14.0" while the contract is at 0.19.0, so a plugin built by
following that guide is refused by the loader at startup. The plugin count was
wrong in six more.

They now point at docs/PROJECT-MAP.md, which is generated. A test enforces it:
no document may declare a version literal, a stated current version must match
the code, and a stated plugin count must match the tree. ADRs are exempt from
the current-version rule, because an ADR states the version a decision was taken
AT - that is a record of the past, and rewriting it would falsify the record
ADRs exist to keep.

CONTRACT-STABILITY.md was missing 0.17.0, 0.18.0 and 0.19.0 - including the only
BREAKING change in the series - in the one document a site reads to choose its
pin. All three are recorded, with 0.19.0 called out: it took something away, and
it shipped before it was written down, which is the argument for pinning tight
rather than trusting that a minor bump is safe.
 2026-08-14 15:41:27 -04:00
cproudlock
4995456136 docs: take one site's name, hosts and paths off the public wiki 
The publishability gate caught internal tooling names and developer paths but
nothing site-specific, so roughly sixty leaks reached the wiki: the site name in
ten documents, real fleet hostnames in the collector and GE-Enforce examples, an
internal database name through the whole import guide, imaging-share paths, and
a maintainer's username as the Deciders line of every ADR and inside a generated
curl example.

None of it is a security matter on an air-gapped fleet. It matters because these
pages are read by engineers at other plants, and a document that names one site
throughout reads as that site's notes rather than a product's documentation -
which is exactly what it then gets treated as.

Examples now use neutral hostnames, the site is "the reference site" where the
distinction carries meaning, and ADRs are decided by "ShopDB maintainers". The
gate carries all of these patterns, so the next one fails a build.

Two documents leave docs/ because they were never written for an outside reader.
PROJECT-REVIEW.md is an internal health memo pinned to a commit from July, whose
headline finding (an untracked playbook) has since been fixed - it is history,
and git holds it. PILOT-DEPLOY.md is one site's own cutover runbook, complete
with a "re-measure before publishing" placeholder; it moves next to the loader
it belongs to, in scripts/site_imports/wjf/.

ADR-015 is AMENDED rather than rewritten. Its enforcement section still said
report-only and its backlog still listed hardcodes that are now cleared, which
left the record contradicting itself. The amendment says what changed and why
the report-only period ended; the original text stays, because what the decision
looked like when it was taken is the part worth keeping.

Also corrects llms.txt's response envelope, which had errors at the top level
and pagination at meta.total. Both are nested one deeper, so anything written
against that description read undefined on every error it tried to handle.
 2026-08-14 15:38:27 -04:00
cproudlock
c829d96128 docs: take two leaks off the public wiki, and say where the files come from 
An ACCEPTED ADR ended with a stray tool-call fragment - it said nothing to a
reader and everything about how the document was produced, which is not what a
decision record is for. And the cutover playbook carried a real internal IP in a
file otherwise correctly scrubbed to documentation ranges. Both were published.

The publishability gate now catches all three shapes, so the next one fails a
build instead of reaching the wiki.

The adoption guide also told a site to copy two files that ship from nowhere:
the asset reporter and EventSaver are both on the reference site's imaging
share, not in this repository, and the guide never said so - it could not be
followed as written. It now states, per artifact, where each comes from, which
of them are pending a move into this repository, and that EventSaver can be
rebuilt from its one source file rather than taken on trust as a binary.
 2026-08-14 15:23:04 -04:00
cproudlock
ce6d44e69f docs: EventSaver, and how GE-Enforce reaches a bay in the first place 
EventSaver was undocumented anywhere a reader could find it. What it is, the two
source modes and why HTTP is the one to use, how order.txt and the ShopDB feed
each decide the running order, the three command-line switches Windows calls it
with, where it must NOT run (a screensaver over a live dashboard or a metrology
run is a defect), how to build it with the in-box compiler, and a table of
symptoms with the first thing to check.

The adoption guide also stopped short of the question it invites: the fleet
tools assume GE-Enforce is already on the PC, so how does GE-Enforce get there.
Both real paths are now written down - imaging time, which is how the shop-floor
pipeline does it, and Intune, which is how the display cohort is deployed today
because those PCs are Entra-joined with no share. Machine Configuration is
possible and unshipped, with the caveat that matters: DSC and GE-Enforce are
both convergence loops, so give each a layer to own rather than pointing them at
the same configuration.
 2026-08-14 14:19:16 -04:00
cproudlock
c648bdf560 labels: one module knows how to draw a code, seven views stop guessing 
Three core pages and four plugin pages each imported qrcode and jsbarcode
directly, and each carried its own answer to the same questions: what margin,
what width, which error correction, how big a module must be before a scanner
can read it. The answers had already drifted - margin 0 in one place and 2 in
another, width 150 against 160 - and on a label that is the difference between
a sticker that scans and one that does not.

frontend/src/utils/codes.js owns it now: the label-stock presets, the quiet-zone
and margin defaults, CODE128 with no printed value, and the printer-resolution
arithmetic that only the Tech Tools generator had. A view passes what is
specific to its own label and nothing else - MachineBadge still asks for CODE39,
because the badge readers predate the shop-floor scanners and decode nothing
else, and that is exactly the kind of thing a call site should say out loud.

views/print/qrLogo.js is folded in rather than left as a second half-shared
helper that only some of the pages reached into.

The check script now fails a build that imports either library outside that
module. Without it this re-forks within a month: the next label page starts by
copying the nearest existing one, which is how it happened the first time.

Tests cover the part no amount of looking at a screen verifies - a QR that
looks fine at 96 dpi on a monitor can be unreadable at 203 dpi on half-inch
stock.
 2026-08-14 14:03:49 -04:00
cproudlock
b37c08eb5b docs: how another site points these tools at its own ShopDB 
The asset reporter and EventSaver are both already built to be repointed - the
server URL, the API key and the targeting are parameters, an ini file and
manifest targeting, not code. Nothing said so, so the question "can another shop
use this" had no answer that did not involve reading PowerShell.

Worked examples for all three deployment paths, because sites have different
management planes and the choice is not ours to make: Intune (a remediation for
the reporter, a Win32 app for the screensaver, plus a Machine Configuration/DSC
form for estates already governed that way), a GE-Enforce manifest entry, and
manual installation for a pilot or a single bay.

The two traps are written down rather than left to be discovered. EventSaver
falls back to a path compiled into the binary when its ini is missing, and that
path belongs to the reference site - a missing ini is not a neutral default. And
a config enforced by hash reverts a hand edit on the next cycle, which is the
feature working correctly and reads exactly like a bug.

Also notes the reporter's -ApiUrl default still points at the reference site, so
every example passes it explicitly until that is fixed.
 2026-08-14 13:53:17 -04:00
cproudlock
8f9805d964 Generate the facts that go stale, and record the fortnight in the changelog 
CLAUDE.md is read at the start of every session and claimed contract 0.16.0 and
1159 tests while the code was at 0.18.0 and 1567, along with a plugin count and
a chain head that were both wrong. Hand-written facts about a moving codebase
are wrong within a fortnight, and a reader who cannot tell which lines are stale
has to re-derive all of them.

So they are derived. scripts/gen_project_map.py reads the versions, the plugin
inventory, every Alembic chain head, the ADR index with status and the size of
the codebase out of the code, and --check fails a build where the committed map
no longer matches. CLAUDE.md now points at it, leads with the multi-site rule,
and lists the gates to run instead of restating the conventions they enforce.

The changelog's Unreleased section covered 8 of the 46 commits since 0.9.0 and
had no Changed, Fixed or Security sections at all. It now carries the whole
fortnight, including both contract bumps - which had never been recorded even
though ADR-002 makes contract versions their own series.
 2026-08-14 13:47:48 -04:00
cproudlock
035419fa51 ADR-015: stop shipping one site's values, and make the rule a gate 
The scanner has been reporting the same count for weeks, which is what a rule
that only prints becomes. It now FAILS the build, and it looks where the leaks
actually were: PowerShell, the installer, the seeds, generated JSON, the
frontend - case-insensitively, across plugins, shopdb, scripts, deploy, tools.
A line that is deliberate declares itself with an ADR-015-OK marker and a
reason, so the claim is visible in review instead of tolerated in silence.

What it found, fixed here:

- The shadow client wrote one site's ShopDB URL into HKLM whenever the registry
  disagreed. At the site it was written for that reads as healing drift;
  anywhere else it overwrites the site's own address on every enforce cycle,
  and the site cannot win because the cycle repeats. The bay's value now wins,
  an explicit -BaseUrl seeds it, and with neither there is nothing honest to
  write, so it says so and skips.
- The kiosk dispatcher fell back to one plant's host when HKLM was unset, so a
  kiosk elsewhere quietly opened a server it has no business reaching. The
  fallback is now this site's site_base_url, baked in at seed time, and the
  dispatcher refuses rather than guessing when neither is set. Its legacy
  shortcut matcher derives the host from that URL instead of naming one.
- The OpenAPI generator hardcoded a production hostname into every spec it
  generated, which then published to a public wiki. The relative mount is the
  only server it can honestly name; a site passes its own by environment.
- Placeholders and examples in the UI and the client help offered real internal
  subnets and a real production URL. They now use documentation ranges.

Both publication gates - the export scrub and the docs publishability test -
carry the site patterns, which neither did. One plant's hostname, FQDN and
internal networks are out of the documentation and the generated specs.

Comments naming the reference site are reworded rather than deleted: the
reasoning is worth keeping, the plant name is not what makes it true.
 2026-08-14 13:47:39 -04:00
cproudlock
4d6ab741cc backups: retention reads the key the settings page actually writes 
get_setting on BasePlugin namespaces what it reads to plugin.backups.<key>,
while get_settings_defaults declares - and the settings page writes - the bare
key. So the retention read never found the operator's value and always fell back
to 0, and 0 means keep everything. Retention was configurable in the UI and did
nothing. It is the only place in the codebase using the namespaced helper.

The share root also stops shipping one site's file server as its default. That
put a site's internal topology in a bundled plugin and in the public mirror, and
pointed a second site at a server it cannot reach. Blank now, per ADR-015, and a
share kind with no configured root returns nothing rather than composing a path
from somebody else's hostname - a path built on an empty root is not a lesser
answer, it is a wrong one.
 2026-08-14 13:47:19 -04:00
cproudlock
838932a72d geenforce: gate publishing on the library version, not on the manifest's own 
The publish gate exists because a minor version bump that NARROWS behaviour is
not backward compatible: _CmmVersion arrived in lib 2.6, and an older lib does
not know the field, so every gated entry looks unfiltered and it installs every
PC-DMIS version it cannot detect, on every CMM, in one cycle.

It was comparing the fleet's reported library versions against manifestversion.
That is the manifest's own 'Version' field. For a share-imported manifest the
two numbering schemes happen to coincide; for a scope authored in code they do
not, and seed_display_scope writes '2.0' - which every kiosk exceeds. So the
gate passed on the scope that most needed it.

A scope now declares minlibversion. Unset, the requirement is DERIVED from what
the manifest actually uses, so a scope written before this column existed is
still judged on its contents rather than on a number that says nothing about the
library. Only features that narrow behaviour belong in that table; an addition
an old lib ignores harmlessly needs no floor. manifestversion remains the last
fallback, which preserves what share-imported manifests already relied on.
 2026-08-14 13:47:11 -04:00
cproudlock
2df5028883 relationships: the cleanup tools stop acting on links that were deleted 
Deleting a relationship is soft, so the row survives with isactive False, and
three things read them without knowing that.

Re-adding a deleted link answered 409 "this relationship already exists" about a
link the page no longer shows, and there was no way forward from the UI at all -
the row cannot simply be inserted again, since the triple is unique.
Reactivating IS the create for an inactive row.

The inverse guard blocked on a deleted inverse, which made "remove the existing
one first" - the instruction in its own message - fail to unblock anything.

fix-controls-direction retired the reversed row whenever a correctly-directed
one existed, without checking whether that one was itself deleted. So it removed
the only live link and reported a successful clean-up. It now reactivates the
row pointing the right way before retiring the one pointing the wrong way.

These are the commands the docs tell an operator to run against production.
 2026-08-14 13:47:01 -04:00
cproudlock
c7dffce81e Serve an uploaded file as data, not as a document that can run 
An SVG is an XML document that may carry a script, and it is an accepted image
type because floor-plan maps and branding genuinely want vector. Loaded through
an img tag that script never runs, so the tiles and maps were never the risk.
Opening the file's own URL is - and the application image route is public, so
that URL needs no session.

Every route that serves an upload now goes through one helper that sends
Content-Security-Policy: default-src 'none'; sandbox, and nosniff. Seven routes
across core and five plugins, so a new one added later starts from the same
place rather than repeating the reasoning. Banning the format instead would
have cost the maps their only sensible one.

The app also sent no security headers at all. It now sets nosniff,
frame-ancestors self (as X-Frame-Options too, for the display bays' browsers)
and a referrer policy. Deliberately NOT a page-wide CSP: this serves an SPA with
inline styles, so a real script-src policy is a change worth making with the
frontend in front of you, and a permissive header claiming one would be worse
than having none.

Contract 0.19.0. send_upload is on the shopdb.api surface, because a plugin
serving user-supplied bytes should not have to remember these headers. The same
bump records that get_dashboard_widgets has taken data and shape rather than a
component name since the dashboard was rebuilt - that shipped without a bump,
while BasePlugin and PLUGIN-HOOKS.md both still documented the shape nothing
renders, which is how five plugins came to declare widgets pointing at
components nobody had written.
 2026-08-14 13:46:53 -04:00
cproudlock
d830dd49a9 collector: adopt an instrument before minting one, and stop the 500 loops 
Minting derived a measuring tool's asset number from the HOSTNAME, so a
permanent instrument inherited the identity of whichever PC drove it that week:
replace the PC and either the number lies or a second tool appears for the same
physical unit. And because idempotency was keyed on the collector's own label,
it could not see a tool it had not created - on prod that left 43 legacy MT-####
tools shadowed by minted <HOST>-CMM twins, three records deep in places.

Resolution order is now most-stable-identity-first: the instrument named by
measuringtool-id.txt, then a prior collector link, then a tool this PC already
controls that somebody else created, then the reported machine number, and only
then mint. What minting produces should be read as a placeholder until a real
identifier is recorded.

Three separate 500 loops came out of the same mistake, looking a relationship up
by LABEL when assetrelationships is unique on (source, target, type):

- On a CMM the instrument IS the reported bay, so the machine sync has already
  made a row for that exact triple - and it finds its own rows by that label.
  Relabelling hid the link, so the next cycle built a second row for the same
  triple and MySQL rejected it: 200 once, then 500 forever. The machine link is
  now recognised and left alone; adoption only needs the identity.
- A part-marker PC hit it twice over, once on its partof row and once because
  the marker's asset number is derived from the PC and could already be taken.
  Both are get-or-create on the triple now, and an existing asset of that number
  is adopted rather than duplicated.

A named instrument also supersedes a minted twin properly: the stale link is
archived by TARGET, not by object identity, which is what left a PC reading as
the controller of two instruments.

Reported identifiers are matched exactly rather than with ilike. They arrive
from a text file on a shopfloor PC, and ilike reads _ and % as wildcards, so
MT-600_ adopted MT-6001 and a bare % adopted whatever active asset came first.
A named id that is not a measuring tool is refused with a warning rather than
linked as one.
 2026-08-14 13:46:40 -04:00
cproudlock
38deefe619 migrations: commit the plugin chains too, and bound what re-ran 
Core's Alembic env got connection.commit() when the stamp bug was found; the
per-plugin template did not. MySQL commits DDL implicitly, which flushes
everything queued before it including the previous migration's version stamp,
and the LAST migration of a run has no DDL after it - so its stamp rolled back
at close while its schema change survived. flask plugin upgrade-all then exited
0 having silently re-run that migration, and re-ran it again on every deploy
after. Invisible for exactly as long as every plugin head happened to be
idempotent.

Two were not.

backups 0003 cleared lastseenat for EVERY row, which is correct once and
destroys evidence on each repeat. It is now scoped to the backfill's actual
signature, COALESCE(collectedat, createdat) - the expression 0002 wrote - plus a
date bound. Both conditions are needed. Matching on collectedat alone misses
every row whose collectedat is NULL, so precisely the rows carrying the most
invented value would have kept it forever; and value equality is not a signature
on MySQL, where db.DateTime is second-precision and the collector writes both
stamps in one statement, so a genuinely fresh revision would read as a backfill
and be wiped. SQLite keeps microseconds, which is why no test could show it.

geenforce 0003 added a column unconditionally, so it failed on a fresh database
built from the models and on any re-run. Guarded like network0003prefix.
 2026-08-14 13:46:23 -04:00
cproudlock
0c574e0f49 installer: break ACL inheritance on the roots it creates 
A directory created directly under C:\ inherits the drive root's DACL, and the
default carries an inherit-only Modify grant for Authenticated Users. So
C:\shopdb-flask, C:\Python314 and C:\MySQL84 were writable by every logged-on
user on the server.

That is a local privilege escalation here specifically, because two things this
installer puts inside those roots are executed by someone else: shopdb-admin.ps1
self-elevates with -Verb RunAs, and instance\config.py is loaded unconditionally
by the app through from_pyfile. Replace either, wait for an administrator or the
app pool to run it, and the code runs as them.

Hardening is applied at creation and RE-APPLIED on every run, so an upgrade over
an installation that predates this repairs the ACL rather than leaving the hole
in place. The MySQL data directory is locked down too: it holds the users table
and its password hashes, and it inherits ProgramData's read-for-Users.

The app pool is granted RX on the Python root alongside the app root. A venv's
python.exe is a copy but the DLL and the standard library are still read from
the base install, so without that grant the site 500s on every request once the
roots stop inheriting.

A Python installation this installer did not create is reported, not seized:
something else may depend on the ACL it has.

Verified as SYSTEM in the Windows 11 VM against the functions as shipped
(extracted by AST, not retyped): the inherited ACL really does grant non-admins
write; after hardening only Administrators and SYSTEM remain; a service identity
granted afterwards keeps access and later-created files inherit it; re-running
is a no-op; a missing path is silently skipped. 14 checks, all passing.
 2026-08-14 13:46:03 -04:00
cproudlock
ab301df9ac network: generate a device's asset number instead of asking twice 
Every network device on this fleet already follows one convention, applied by
hand: AP-<name>, SW-<name>, SVR-<name>, IDF-<name>. 45 records, no exceptions.
The create form demanded the asset number anyway, so the same value was typed
twice and the convention held only as long as everyone remembered it.

The prefix now lives on the device type, and a blank asset number is generated
as <PREFIX>-<name>. Left explicit, an asset number always wins: a device
carrying a real identifier of its own - a vendor tag, a controller name, a
serial - keeps it. That is the platform rule, adopt where an identifier exists
and derive only where none does.

The prefix is NOT derived from the type name. "Access Point" and "Access Panel"
both initialise to AP, and assetnumber is unique, so the second type would
collide with the first on every device it created. It is nullable, so a type
that wants no prefix generates the bare name rather than needing one invented.

Names are sanitised before they reach a business key - the existing data
already shows why, with IDF-Telco-Demarc-#1 carrying a '#' into an identifier.
An existing prefix is never stacked: IDF-03 under type IDF stays IDF-03.
 2026-08-14 13:45:51 -04:00
cproudlock
1078ac03df geenforce: shadow runs inside the enforce cycle, not on its own clock 
Test bays reported 0 installed / 0 skipped / 0 failed and nothing was being
processed. The share is a TRANSIENT mount: GE-Enforce.ps1 maps it for the length
of its cycle and unmounts at the end - mounted 12:10:02, unmounted 12:10:27 on
FB9TP7V3. The shadow task ran on its own 15-minute schedule, so it woke with the
drive gone, handed the engine a path that no longer resolved, and the summary
zero-filled. A silent nothing, indistinguishable from a healthy no-op, which is
why it read as "not processing" rather than as a failure.

Register-ShopdbShadow.ps1 is replaced by Invoke-ShopdbShadow.ps1, which runs AS
a manifest entry and invokes the runner directly. The share is mounted because
the enforce cycle is what called us; there is one cadence instead of two; and
there is no task to register, drift, or heal. It also unregisters the superseded
task, so bays that already carry it clean themselves up rather than keeping a
run that cannot see the share and overwrites the real result with 0/0/0.

The runner gains the Test-Path guard that should have been there: an unreachable
-ShareManifestPath now logs, writes an event, and reports a real failure naming
the reason, instead of running the engine against nothing. That silent zero is
what made this expensive to find.

Verified on the win11 VM with a substituted drive removed between runs - the
same call gives real counts while mapped and the guard when not. The earlier
task self-heal is moot now; it was a fix at the wrong layer, and testing against
a local path is what hid a mount I had already noted was dynamic.
 2026-08-13 14:52:56 -04:00
cproudlock
dfbc12a172 printers: part numbers for the HP 3201 / 3301 series 
A 3201dw is monitored and alerts correctly - Zabbix reports its cartridge names
and levels straight off the printer - but the Toner Report had no reorder chip
for it, because part numbers come only from modelsupplies and no family matched.
lookupsupplies returns [] for a model with no rows, so the printer showed levels
with nothing to order.

Adds the 218 family: 218A standard and 218X high yield in all four colours,
matched on 3201 and 3301 so the MFP siblings are covered by the same entry.
Part numbers and yields cross-checked against three independent sources rather
than inferred from the series - a wrong toner number means someone orders a
cartridge that does not fit, which is worse than the blank chip it replaces.

No drum or waste unit: none is listed as a separately orderable part for this
series, so the family is toners only.
 2026-08-13 13:45:45 -04:00
cproudlock
4bded210e4 geenforce: the backup hover names both dates, because they mean different things 
"Checked 13 Aug 1:20 PM. Verified the backup taken 12 Aug 5:20 PM is still
current." Two facts, and one date could not carry both.

lastseenat moves on every successful confirmation and proves the check is still
running. collectedat moves only when the configuration actually CHANGED and says
what is stored. Showing lastseenat alone hid what is in the backup; showing
collectedat alone read as "last backup was a month ago" on a machine that is
perfectly healthy, because a stable config writes no new revision. The hover now
states the check and the capture separately and says outright that the second
being older is the normal case.

The stale wording gets the same treatment: it names the check that stopped AND
the newest copy that exists, which is the thing someone recovering from it
actually needs.

collectedat is exposed as backupcollectedat. Both stay tooltip-only - the badge
is still just the kind and a colour, so nothing here changes what a new backup
kind has to do to inherit it.
 2026-08-13 13:35:05 -04:00
cproudlock
1d7191c2d3 geenforce: the fleet table links where it says, and judges backups instead of dating them 
Two fixes to the same table, in the same regions of the same files.

ASSET LINK POINTED AT THE WRONG RECORD. The Asset chip linked
/machines/<assetid>, but /machines/:id keys on machineid - the plugin extension
id - as MachineDetail itself does everywhere. So the link landed on whichever
machine happened to carry that number: a wrong page that looks right, which is
worse than a 404. Same for /measuringtools/. The API now returns
machinepluginid / toolpluginid beside the asset ids and the view links on those.
Both lookups are import-guarded, and with no plugin id the number renders as
plain text rather than a link that misleads. AssetRelationships already resolved
this correctly; this brings the reports table in line.

BACKUP COLUMN READ AS NEGLECT. It showed a raw date, and a revision is only
written when the config CHANGES - dedup means a machine stable for months has a
months-old newest revision and is perfectly healthy. The column already used
lastseenat, the last time the collector CONFIRMED the config, but a bare
timestamp says "nothing has happened since", which at the default 24h collection
interval IS the healthy steady state. It made a working system look stalled and
made the reader do arithmetic against a setting they would have to go and find.

It now returns backupok and shows a badge naming the kind, green when confirmed
recently, red when not, with the date and an explanation in the hover. backupok
is tri-state on purpose: null means no revision at all, and renders as NO badge
rather than a green one, because "never seen" must not read as healthy. The
threshold is the backups plugin's own backups_staledays, read through its
service so there is one definition of stale rather than a second drifting here.

Nothing in the badge is kind-specific, so a backup kind added later inherits it
by existing. docs/BACKUP-KINDS.md records that, the BackupKind contract, and why
the rule is time-based rather than per-kind.
 2026-08-13 13:20:15 -04:00
cproudlock
6dc363411d geenforce: the shadow task actually runs, and says so on disk 
Registered fine and never fired once. Three faults, all found on the win11 VM.

A `-Once -At (Get-Date)` trigger does NOT run immediately: its first run is the
start boundary PLUS the repetition interval, 15 minutes out. This ships as a
DetectionMethod=Always entry, so it ran every enforce cycle, 5 minutes apart,
and each Register-ScheduledTask -Force reset the start boundary to "now" -
pushing the first run back to +15 before the previous +15 could elapse. 5 < 15,
so the task sat Ready at LastTaskResult 267011 (SCHED_S_TASK_HAS_NOT_RUN)
forever. It now registers only when the task is missing or its arguments
changed, and starts it once on first registration rather than waiting out the
first interval.

A bay provisioned by the broken version carries a task with correct arguments
that has never run, so "leave it alone if it matches" would have stranded
exactly the machines that hit the bug. If the task has never run it is kicked
once; after that LastRunTime is set and the check is a no-op.

None of this was visible. The engine records only "ps1: <path>" and an exit
code for a PS1 entry, so Write-Host reached nothing, and with the fail-safe
`exit 0` on every path a silent early-out was indistinguishable from success.
It now also writes C:\Logs\Shopfloor\shadow-setup-<date>.log.

Scope is no longer hardcoded either: this script is shipped by more than one
scope now, and a wrong value would shadow the wrong manifest silently. It is
derived from the script's own directory, the same way the share manifest path
already was, so the two cannot disagree.
 2026-08-13 13:19:59 -04:00
cproudlock
3d83806135 Make the toner forecast an order, not a table 
The report answers a purchasing question, and it was answering it in seven
columns, two tables and a rowspan. What someone actually needs from it is a
short list of what to buy.

So it opens with that list, grouped by part number with a quantity. Two
cartridges of the same part in different printers is a quantity of two, which
is the number an order needs and the one a per-printer table made the reader
count by hand. It covers what is empty plus what goes within a fortnight -
ordering only what is already empty means running empty. There is a copy
button, because it ends up pasted into a mail.

Below it the cartridges sit in urgency bands rather than in one long list
sorted by a number. The question is which pile a thing is in, and a pile that
is empty is worth seeing as empty. Everything past "empty" starts collapsed;
the order list above already covers the same ground in a tenth of the height.

The row is a cartridge now, not a printer, so it can carry its own part number,
its own level bar and its own countdown. Nesting supplies under a printer meant
opening a printer to find out whether anything on it needed doing.

Cartridges with no part mapped are counted on a single line rather than given
one each. They cannot be dropped, since that would quietly shorten the order,
and they cannot be ordered from here either - the job they represent is
mapping them, which is one job however many there are.

Bands and the order horizon are decided server-side, next to the arithmetic
that produces them, so a heading cannot disagree with what got added to the
list.

Checked against a fleet of 43 dev printers with real part mappings, driven by
a stub Zabbix - live Zabbix is not reachable from the dev box.
 2026-08-13 13:08:39 -04:00
cproudlock
e67fe47fe2 relationships: refuse links that cannot both be true, and report the ones already stored
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Prod grew rows saying both "PC controls 2005" and "2005 controls PC", and a CMM
PC showing "<- controls from CMM4" beside its own outgoing link. Only one
direction can be true: a PC drives a machine, never the reverse.

Nothing stopped it. The duplicate check was keyed on (source, target, type), so
the inverse inserted cleanly, and the Add Relationship dialog offers an incoming
direction that writes exactly that. The legacy import stores controls the wrong
way round as well. Directional creates now refuse the reverse with a 409 naming
the row that already holds it, and refuse self-links, which render as a
duplicate on the asset's own page and mean nothing. Symmetric types are exempt:
Dualpath stores both directions on purpose and the card collapses them. The
propagation fan-out got the same guard so a rail meant to spread one direction
across sibling bays cannot manufacture a pair.

fix-controls-direction only matched source assettype 'machine', so every
measuring_tool, printer and network_device row it was written to clean survived
it - which is why running it would never have fixed the CMM. It now matches any
non-computer controlled BY a computer.

New `flask relationships audit` reports what is already stored: reciprocal
pairs, self-links, and PCs controlling several assets of one type. Read-only,
and it prints each row's label because that usually names the writer outright -
collector:* means this code made it, anything else means a person or the import
did. That distinction decides the fix for duplicate device assets, which is not
in this commit: the collector keys idempotency on its own label, so a device
somebody created by hand is invisible to it and it mints another, and the
adoption rule needs the audit run against prod before it can be written.

Two false positives were found writing it, against the dev database, and both
would have made the report useless. A self-link is its own inverse, so it was
counted as a reciprocal pair AND printed twice. And Dualpath siblings looked
like duplicate devices - a dual-bay machine is one physical machine with one
controller and controls is propagated to both bays deliberately. That was 30 of
32 findings, consecutive bay numbers pair by pair.
 2026-08-13 12:25:24 -04:00
cproudlock
6cdbea449a geenforce: ship shadow mode through the share, and correct the cutover doc
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
The cheap cutover the doc described - manifest over API, payloads still on the
share - does not work. Measured on the win11 VM against lib 2.6 with the same
manifest and engine, varying only InstallerRoot: the payload cache gives
0 installed / 2 failed ("CMD/BAT not found" under the cache), the scope
directory gives 2 installed / 0 failed.

Two causes. Invoke-ShopdbEnforce.ps1 takes no -InstallerRoot and hardcodes the
payload cache outside shadow mode, which is right for displays where every
entry is inline and wrong for every smb entry. And -IncludeCommon cannot work
for smb payloads at all: they live under their own scope directory, so a merged
manifest needs two roots where the engine takes one, and Join-InstallerPath
rejects rooted values by design so absolute paths are not a way out. The real
dispatcher never merges - it calls the engine once per scope with that scope's
root - and a fix should follow that shape.

Both claims are corrected in the doc rather than deleted, because reading it
was what made this look like a twenty-minute job; it would have failed 29 of 31
entries on the pilot bay.

Shadow mode is unaffected - it points InstallerRoot at the share scope dir,
which is the working path - so the pilot proceeds. Register-ShopdbShadow.ps1
is the reference copy of the script that puts a share-attached bay into shadow:
it writes BaseUrl, then registers the task as SYSTEM. It derives the share
manifest from $PSScriptRoot rather than hardcoding a drive, because GE-Enforce
mounts the share to a letter it picks at run time and SYSTEM has no standing
UNC access. Delivered as a manifest entry gated to one hostname, so the pilot
needs no hands on the bay and heals if it drifts.
 2026-08-13 11:21:28 -04:00
cproudlock
7d66551622 Forecast from the right end of the window, and from the level shown
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 6s
Details 
Four defects stacked into one nonsense report: cartridges at 20% claiming four
days, cartridges at 1% claiming weeks.

The root cause is a Zabbix API detail. `limit` caps the whole result set rather
than each item, and the query sorted ascending, so the cap kept the OLDEST rows
in the window. A four-cartridge printer polled every five minutes writes over
100k readings in 90 days; the forecast was fitted to the first few days of that
and nothing since. Every rate was real and every rate described a cartridge
thrown away three months ago. Nothing in the output looks wrong, which is why
it needed pinning in a test rather than a comment.

A 90-day burn rate does not need every individual poll, so a long window now
reads hourly trends - the table meant for this, a tenth of the rows, and kept
longer. Raw history serves short windows and any item a site keeps no trends
for. Both are fetched newest-first with the budget scaled per item.

Second, the countdown was computed from the last stored reading while the level
displayed was the live one, so the two could disagree by a whole cartridge. The
live level is now what the countdown divides. A live level far above the stored
run means it was swapped since the last reading, and that is reported as a
replacement rather than as a collapse in the burn rate.

Third, at or below 5% a cartridge reads as empty rather than as a slow drain.
At 1% losing a tenth of a point a day the arithmetic says ten days. The printer
is out of toner, and it is the first thing to order.

Fourth, the days-left column spanned the printer's rows, so the printer's
soonest figure was printed beside every supply it had. That alone accounts for
the shape of both complaints: a healthy cartridge wearing its neighbour's
deadline, and an empty one wearing a number that belonged to nothing on its row.

Also fixes float-typed supplies vanishing from any printer that also had an
integer-typed one - they live in different history tables and the fetch stopped
at whichever answered first.

Not verified against live data: Zabbix is not reachable from the dev box.
 2026-08-13 10:54:23 -04:00
cproudlock
1e884dc02a geenforce: judge silence on both clocks, not just the server's
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 10s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
The fleet table has two time columns and staleness only looked at one. A PC can
go quiet in either direction: silent to the server, or still posting while its
own cycle has stopped advancing. Only the first was caught, so a machine whose
Last check-in had not moved since the previous morning still showed 'ok'.

The two are tested independently rather than by taking the newer of them.
receivedat is the server's own record and cannot be argued with; lastcheckin is
asserted by the client, so a wrong clock there must not be able to vouch for a
PC the server has not actually heard from. Checking them separately means a
future-dated lastcheckin cannot mask real silence.

The tooltip now names both times, since which one stopped says what kind of
failure it is.
 2026-08-13 09:53:56 -04:00
cproudlock
962979d483 geenforce: a PC that has gone quiet stops reading as healthy 
A report records how ONE cycle went. Nothing ages it, so a PC that stops
reporting keeps the status of its last good cycle: switch a machine off after a
clean run and it shows 'ok' indefinitely. One had been offline more than a day
and still read 'ok'.

Silence is a different fact from the last cycle's outcome, so it is computed
separately rather than by rewriting the stored status. receivedat - the server's
own clock, not anything a client asserts - is compared against
geenforce_reportstaleminutes, default 30, which is roughly two missed cycles at
the usual cadence. Set it to 0 to turn the check off.

In the table 'stale' takes the badge, because a status from a machine that has
not spoken since is not evidence of anything. What it last reported stays in the
tooltip with the time it was heard. The stored status is untouched: it is still
a true record of that cycle, just not proof the PC is alive.

A site whose scope enforces less often than the threshold will read stale while
healthy, which is what the setting is for.
 2026-08-13 09:28:37 -04:00
cproudlock
20a95013ad contract 0.18.0: one name per display role, the kiosk's own 
Core called the roles dashboard / lobby / partskiosk. The kiosks call them
Dashboard / Lobby / 3DPrintRoom, which are the literal contents of
C:\Enrollment\display-type.txt, read by the GE-Enforce dispatcher to pick a
target. Two vocabularies for three kiosks, each with its own copy of the same
route map.

That is not cosmetic. A display reporting its own type sends what its file
says, so it could report a role core would not accept, and core could store
'partskiosk', a value no dispatcher would ever match. The enforcement report
column would have shown one vocabulary from the device and the other from the
DashboardDefault fallback, in the same column.

The machine's file wins, because that is what a person edits. DISPLAY_ROLE_PATHS
takes the kiosk spelling and the display scope now uses that dict rather than
holding a second one, so the two cannot drift again. normalize_display_role
resolves any casing and the retired 'partskiosk' forward; the dispatcher already
matched its map case-insensitively and the server now agrees with it.

Nothing is turned away over a capital: the API accepts any spelling and stores
the canonical one, displaypath resolves through the normalizer so rows written
before this keep working, and the settings dropdown canonicalises on open so an
old value does not render as a blank select.

A reported subtype is normalised on the way in, but an UNRECOGNISED one is kept
verbatim. That is a kiosk with a typo in its file or a role nobody declared, and
both are worth seeing in the fleet table rather than blanked or guessed at.

Contract bumped for the added names. DashboardDefault is finally listed in
__all__ too - 0.17.0 put it on the surface and never exported it.
 2026-08-13 09:28:26 -04:00
cproudlock
9336577abe Commit the migration run instead of trusting MySQL to 
SQLAlchemy 2.0 connections do not autocommit, and on MySQL alembic reports
"non-transactional DDL", so context.begin_transaction() is a no-op. Nothing in
a migration run committed.

It looked like it worked because MySQL implicitly commits on DDL: every
ALTER/CREATE flushed whatever was queued before it, including the PREVIOUS
migration's version stamp. The LAST migration of a run has no DDL after it, so
its stamp was rolled back when the connection closed. `flask db upgrade` then
exited 0 with the schema change really applied and alembic_version one revision
behind, and re-ran that same migration on the next deploy. A migration that is
not idempotent applies twice.

Found while adding a core migration: the column default really changed and
alembic_version still named its parent, with UPDATE alembic_version followed
immediately by ROLLBACK in the log.

Existing databases upgraded before this are one revision behind their real
schema. Compare `flask db current` against `flask db heads`; re-running the
upgrade re-applies the final migration once and lands the stamp.
 2026-08-13 09:28:11 -04:00
cproudlock
5de3594425 displays: the client module updates itself 
Install-ShopdbKiosk.ps1 lays the enforce client down once at bootstrap and
never refreshes it. So a client change rode the code deploy to the server
and then sat one directory away from where kiosks actually fetch, waiting
for someone to re-stage the installer bundle by hand - which is how the new
display-type reporting reached prod and changed nothing on any kiosk.

The module now ships as a manifest entry like everything else in this
scope: inline over HTTPS, Hash detection against the exact bytes shipped,
written to the same path the installer uses so bootstrap and self-update
cannot disagree. Ordered first, so a stale client refreshes before anything
leans on it. The installer keeps its real job - a fresh kiosk still needs
something that can talk to shopdb - it just stops being the update path.

Self-modifying by design: this module is what stages payloads, but
PowerShell loads it into memory at start, so rewriting the file mid-run is
harmless and lands on the next cycle. Pilot a client change on ONE kiosk
before the fleet: a broken module cannot fetch its own replacement, and on
a share-less display that means a site visit.
 2026-08-12 17:21:29 -04:00
cproudlock
84bf5d04ed geenforce: let a kiosk say what it is instead of guessing 
A display knows whether it is a Dashboard, a Lobby screen or the 3D print
room - the dispatcher reads C:\Enrollment\display-type.txt to choose which
page to open. It never told shopdb, so the fleet table inferred it from the
DashboardDefault fqdn mapping, which is empty unless somebody added a row
per kiosk. The column was blank for every display.

The client now reads that file and reports it, the report stores it, and
the API prefers the reported value with the old mapping left as a fallback
for hosts still on an older client. Reported by the device beats inferred
from a lookup table, the same way enforcerversion already works. A PC with
no display-type.txt reports nothing rather than something invented, and an
empty string lands as NULL.

Two guards had to learn about it. The DDL parity check read only the 0001
baseline, so a column added by a later revision looked like drift even
though its migration existed; it now runs the whole chain, which is what
'do the models match what the migrations build' means. 0002 added a whole
table rather than a column, which is why this is the first time it bit.
 2026-08-12 17:07:21 -04:00
cproudlock
d572c913e5 geenforce: let the wide pages use the page 
Two caps, one inside the other: the section shell at 1400px and the reports
view at 1100px. Fine for nine columns; the reports table now carries twelve,
including the asset a PC drives, its location and its backup state, so it
was being squeezed while empty page sat to the right. Both are gone, and
the manifests editor gets the width too.

The Asset column no longer falls back to the PC's own asset number. The
collector stores a PC's hostname AS its assetnumber, so for any PC that
drives nothing that column just repeated the Host column next to it. It now
shows a dash, and carries content only when it says something Host does
not: the machine a bay PC controls, the tool a measuring PC controls, or
the role of a display.
 2026-08-12 16:56:45 -04:00
cproudlock
f1f573862d geenforce: order backup revisions in Python, not in MySQL 
The reports table 500'd on every load: the backup lookup ordered with
ORDER BY lastseenat DESC NULLS LAST, which SQLite accepts and MySQL
rejects outright. Every test passed and the real database refused the
query - the tests run on SQLite, so the dialect difference was invisible.

Sorting in Python removes the dependency for nothing: the rows are one per
host per kind. The regression test pins which revision wins, including
that one never confirmed does not, and says why the sort lives here so it
does not get helpfully moved back into SQL.
 2026-08-12 16:49:58 -04:00
cproudlock
52eb10f5ca contract 0.17.0: expose DashboardDefault to plugins 
The enforcement reports needed to name what a display IS, and reached
straight into shopdb.core.models.dashboarddefault to do it. Plugins may
only touch core through shopdb.api, and the contract test said so.

The role belongs on the surface rather than behind it: it lives in core,
no plugin owns it, and a plugin reporting on displays has no other way to
resolve it. Added there and the version bumped, which the docs test pins.
 2026-08-12 16:41:09 -04:00
cproudlock
598c2c98bc geenforce: reports say what the PC is, where, and whether it is backed up 
A report row carried a hostname and some counts. Everything an operator
wants next hangs off the asset behind that host, and none of it was there.

Host now links to its PC page. Beside it, what the PC IS or DRIVES: the
machine number for a bay or part-marker PC, the measuring tool for a tool
PC, the role for a display - each linking to that asset's own page. A map
pin appears only when the asset has coordinates and hovers to the same
floor-plan preview the asset pages use; an icon that opened an empty map
would be worse than no icon.

The backup column is LAST CONFIRMED, not last changed. Dedup means an
unchanged config writes no revision, so a machine stable for six months has
a six-month-old newest revision and is perfectly healthy - the one to worry
about is the machine whose backup stopped running. It reads lastseenat and
names the kind rather than assuming ntlars, since udc/file kinds on the
share are coming.

Resolution is bulk, never per row: this table shows the whole fleet, so a
lookup inside the loop would be one query per PC. It reads the collector's
existing 'controls' relationship rather than re-deriving which machine a PC
drives - that same resolution living in two places is what put a wrong
subtype filter on the map. Every plugin it touches is optional, so each
lookup is ImportError-guarded and a lean build renders the table without
those columns. A host ShopDB has no asset for still shows: the enforcement
result is real even when the inventory is behind.
 2026-08-12 16:31:24 -04:00
cproudlock
523e3e4ecc geenforce: give the manifest entry rows room to breathe 
Every column in the entries table was sized to its button text with nothing
spare, so with flex-wrap on the action cell the Edit and Delete buttons
wrapped out of their 128px column and sat on top of the entry description.
The order column was tighter still, and its Up/Down buttons carried almost
no padding, which made them hard to hit as well as hard to read.

Widened the order, type and action columns to fit their contents on one
line, stopped the action cell wrapping, gave the buttons real padding, and
added vertical cell padding with top alignment so a two-line entry no
longer runs into the row beneath it.
 2026-08-12 15:47:36 -04:00
cproudlock
9e34fafce5 geenforce: refuse to publish a manifest the fleet's lib cannot read 
The engine treats a minor-newer manifest as backward compatible and carries
on. That holds for additions which WIDEN behaviour - an old lib skips a Type
it does not know - and inverts for one that NARROWS it. _CmmVersion arrived
in lib 2.6 as a minor bump, so a PC on 2.5 does not recognise the field,
reads every gated entry as unfiltered, and installs every PC-DMIS version it
cannot detect, on every CMM, within one cycle.

The share runbook already says push the lib first. A runbook is not a
control, and the failure is silent, fleet-wide and about five minutes fast.

ShopDB already had the evidence and was not using it: every enforcement
report carries the enforcer version, and publish_scope had no gate at all.
It now compares the scope's manifest version against the versions PCs
actually report for that scope and refuses when any is behind, naming the
hosts. force=True for someone who knows why. A report with no or an
unreadable version counts as behind - that field arrived with the
summary-emitting engine, so its absence IS an old lib, and treating unknown
as safe is precisely how this fails open.

A scope nobody has reported for still publishes, or a fresh site could
never publish anything. Versions compare numerically, since as text '2.10'
sorts below '2.9'.

Also exposed as a preflight endpoint so the UI can warn before someone
clicks publish, and as a 409 with the offending hosts rather than a 500.
 2026-08-12 15:23:57 -04:00
cproudlock
787f475208 displays: clear a pending Edge update without waiting for 02:00 
Every kiosk was sitting on Edge's restart-to-update prompt. The scope
already sets RelaunchNotification=2 so Edge restarts unattended, but
RelaunchWindow defers that restart to 02:00-04:00, so during the day the
update waits and there is nobody on site to dismiss the prompt.

A one-shot entry runs the Edge updater and stops the browser. It does not
relaunch it - the enforce task is SYSTEM in session 0, where a launched
browser is invisible - so it leans on the watchdog that already relaunches
the kiosk from the Startup shortcut. That relaunch is what applies the
staged update.

One-shot is DetectionMethod=MarkerFile. The engine writes the marker only
after a 0 exit, so a failed run retries next cycle instead of being
recorded as done, and the script exits 0 when no Edge was running - that
is a success, and failing it would withhold the marker and re-kill Edge on
every cycle from then on. The marker path carries a date, which is the
re-arm mechanism for a future update.

Ordered after the watchdog entry: a display seeing both for the first time
must have its relauncher registered before anything stops the browser.
 2026-08-12 12:26:29 -04:00
cproudlock
457349d258 export: stop an unanchored exclude from eating a plugin 
The publication rsync excluded 'tools' to keep the repo-root tools/ dir
off GitHub. rsync patterns without a leading slash match at any depth, so
naming a plugin 'tools' meant plugins/tools/ was dropped too. It was
committed, exported, bundled and deployed, and the first sign of trouble
was `plugin install tools` on the server failing on a manifest.json that
had never been shipped.

The root-intended excludes are anchored, and the sync now verifies that
every plugins/*/manifest.json in the working tree came out the other side.
A silent omission of a whole plugin should not be something a deploy
discovers for us.
 2026-08-12 12:13:23 -04:00
cproudlock
a64796f060 Document the upload, proof and forecast endpoints
Some checks failed
CI / backend (push) Failing after 9s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 10s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Ten routes shipped over the last few commits without reaching the inventory,
which is the file everything else is generated from - openapi.json, the /api/docs
browser, and the per-site build. An endpoint absent from it is invisible to
anyone integrating against the API even though it answers requests.

The auth notes matter more than the paths. API-REFERENCE tells a deployment
reviewer that essentially every GET is optional-auth; two of these are not, and
an exception buried in a table of 372 operations is one nobody reads. The
installer download and the warranty proof are called out with why: one is
licensed vendor software, the other carries pricing and a service tag. The
application image is listed with the fully public endpoints, since tiles render
before login.
 2026-08-12 12:06:25 -04:00
cproudlock
ea35a134fe Show the fiscal week beside the site name 
The classic ASP site prints a week number under the site title, and people
quote it in conversation and on paperwork. Anyone with both sites open needs
the two to agree, so this is a port of the old arithmetic rather than a fresh
interpretation of what a fiscal week is: ISO 8601, week 1 contains 4 January,
and the week's Thursday decides which year it belongs to. That rule is what
makes late December and early January land in the right year, which is exactly
where a naive day-of-year count goes wrong, so it is what the tests cover.

Worth recording: a true GE fiscal calendar need not follow ISO weeks. Nobody
has asked for a different rule, and inventing one here would silently disagree
with the site people compare against.

Computed in local time on purpose. The number people quote is the one on the
wall where they stand, and a UTC week rolls over hours early in the evening at
a US site. The sidebar re-checks every half hour; the shop-floor board picks it
up with the clock it already ticks.
 2026-08-12 12:06:25 -04:00
cproudlock
d109314123 Forecast when a printer runs out, and count what it has been through 
The toner report says what is empty now. It could not say what to order, and
nothing recorded how fast anything drains - every level read was cached for
five minutes and then discarded.

Zabbix has been keeping the history all along; we simply never asked. One
history.get gives both answers, because a cartridge only goes DOWN while it is
in use: a rise is a replacement. Count the rises and you have how many
cartridges a printer has been through; fit a slope to the readings SINCE the
last rise and you have days-to-empty. Fitting across a replacement averages a
spent cartridge with a fresh one and describes neither.

Sorted by days left, which is the point. A cartridge at 60% dropping 5% a day
needs ordering before one sitting at 8% that has not moved in months, and a
level-sorted list ranks those backwards.

It refuses to guess. Too few readings, a level that has not moved enough - many
printers report in 10% steps and sit on a plateau for a fortnight - or a recent
replacement each produce no estimate and say which. Those printers are listed
separately rather than sorted in as 0 or as 999, since a printer without an
estimate is neither urgent nor safe. Estimates show what they rest on, because
"9 days from 21 days of readings" and "9 days from 2 readings" are not the same
claim.

A separate report card, not an extension of the toner report: that one is an
exceptions list a tech acts on today, this is an ordering view read monthly,
and the history query is heavier than the live read it would have slowed down.

The analysis is pure arithmetic over a list of readings, so the 14 tests cover
the noise wobble, the plateau, the swap, junk rows and division by zero without
needing Zabbix. Zabbix being unreachable is reported as such rather than
rendering an empty table that reads as "nothing is due".
 2026-08-12 11:45:40 -04:00
cproudlock
2fce81f33f Attach proof of cover to a warranty 
A provider lookup answers whether a unit is covered. It does not produce the
invoice or the extended-warranty certificate, and a manually entered warranty
had nowhere to keep one - so the proof stayed in somebody's mailbox until they
left.

Two columns rather than one: the served URL of the stored document, and the
name the vendor sent it under, because "Dell invoice 4471.pdf" is what a person
recognises a year later and "warranty-12.pdf" is not. The download route sends
the original name back.

Authenticated in both directions, unlike an asset photo: an invoice carries
pricing and a service tag. One document per warranty, replacing any prior
extension so a re-upload as .pdf does not leave the old .png behind claiming to
be current. Capped at 25MB - a certificate is a document, not a disk image.

Office formats are allowed because purchase records genuinely arrive as .msg
and .xlsx, not only as PDFs.
 2026-08-12 11:45:40 -04:00
cproudlock
c28b02e45b Upload an application's image and installer instead of typing paths 
Adding an application meant typing an image FILENAME and trusting someone had
dropped the file into the frontend's own directory by hand, and typing an
install path from memory. Both are uploads now, following the model-image trio
that models and part photos already use.

The two differ deliberately. The image is public, because application tiles
render before anything is authenticated. The installer is not: it is licensed
vendor software, an open URL would publish it to anything that can reach the
site, and it is always sent as an attachment rather than rendered.

Installers are capped at 500MB and the size is measured by seeking the stream
rather than trusting Content-Length, which a chunked upload does not send and a
client can understate. Anything larger belongs on the share, and the error says
so rather than just refusing.

Files are chosen before a new application exists, so they are held and uploaded
once there is an id to attach them to. A failed upload leaves the saved record
alone and reports, rather than losing what saved fine.

Removing an installer only clears installpath when it pointed at the upload - a
share path was typed by a person and is not ours to wipe. The detail page reads
both shapes, since entries from the classic site hold a bare filename that is
still served from /images/applications/.
 2026-08-12 11:45:17 -04:00
cproudlock
2693eb28d6 Cap the dashboard at four cards, and shrink the hover map 
The card track was 28rem, which fits four across a full-width page but only
three once the sidebar takes its 250px - and 1920 with the sidebar is the
common case here, so the board showed three. The track is 22rem now, with an
explicit four-column cap above 96rem: left to auto-fit alone a wide screen
reaches five, and a fifth column only makes the cards narrower until the rows
they hold start truncating again.

Measured at 1366, 1600, 1920 and 2560: three, four, four, four.

The floor-plan preview drops from 500x385 to 390x300. At the old size it
covered the row it was launched from, which is the row you are trying to read.
 2026-08-12 11:45:17 -04:00
cproudlock
27f76ee964 frontend: declare the empty toplevel export the codegen reads 
routes.gen.js spreads `.toplevel` for every plugin uniformly, but only a
few own full-screen routes, so Rollup warned on eleven of them every
build. The `|| []` guard was always doing its job - the warning was noise,
and noise in a build log is where a real warning goes to hide.
 2026-08-12 11:43:57 -04:00
cproudlock
f66763e6df tools: a Tech Tools section, starting with codes on label stock 
A place for the small utilities a technician reaches for at a bench. The
plugin owns no API and no tables: every tool runs entirely in the browser,
so an air-gapped site gets them for free and a bad network cannot break
them. Adding the next tool is a view, a route, and one entry in tools.js.

First tool is a barcode/QR generator. Content is typed text, a URL, or a
CSV (content,label,copies - quoted fields and an optional header both
handled), so a batch of a few hundred is one paste. Label stock is
adjustable in inches with five presets, and the code renders to an SVG
data URI rather than a PNG: a bitmap gets downscaled to label size and
smears the module edges a scanner reads, where SVG rasterizes at the
printer's resolution with hard edges.

It also carries the dot-grid rule that is easy to get wrong by eye. A
thermal head cannot render a fraction of a dot, so a code sized off the
grid gets uneven modules; pick a DPI and the page says what the current
size lands on and what to use instead. The quiet zone is blank label
rather than white baked into the code, so it can be tuned - and it applies
to CODE128 too, which needs clear space at each end and was letting bars
run into the caption.

Tech Tools is the first bundled plugin that owns no schema, which two
guards did not model: it belongs in the universal installer profile, and
upgrade-all reports it 'no-migrations' where every plugin was assumed to
report 'ok'. The migration test now asserts that status explicitly for
schema-less plugins, so a table-owning plugin whose chain went missing
still fails.
 2026-08-12 11:43:48 -04:00
cproudlock
4d807ccb4b toner report: the part to order, and where the printer is 
The report exists to answer "what needs replacing and where do I get it",
and it was answering neither. The part numbers were already in the
lowsupplies payload and simply never rendered; a chip per part now shows
them, with capacity tier and page yield on hover, since a model can list
several tiers for one colour.

Asset # and Location columns are gone. Location is replaced by the
floor-plan preview the asset pages already use, hung off the printer name
via its mapx/mapy.

The IP is now the site's FQDN (printer_hostname_template, built from the
IP exactly as PrinterForm does) and links to the printer's own web page in
a new tab - the report is a worklist, and losing your place in it to visit
one printer means finding your row again. The raw IP stays on hover.

Cartridge names were ellipsised inside a fixed 120px column, hiding the
one thing being reordered. The supplies cell is a grid with a max-content
name column, so names show in full and still line up across a printer's
rows.

CSV and emailed exports follow the screen, with one row per part number so
the result is a copy-pasteable order list.
 2026-08-12 11:43:25 -04:00
cproudlock
94d8d6c9b6 dashboard: numbers that agree, a map on hover, wider cards
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
"All assets 704" sat beside "all assets in use 737", and both were correct
about different populations. The totals summed five specific asset types and
subtracted dual-bay secondaries; the status counts took every asset row of any
type with no collapse, so USB devices and hidden secondary bays inflated one
side of a comparison the layout invites. Status is now counted over exactly the
same assets the totals describe.

Warranty rows fell back to asset.name when the covered asset had no hostname,
and an asset's name is usually the MACHINE's descriptive name - which is how a
column meant to identify a PC ended up showing a machine. Hostname, else the
asset number, never the name. The machine number loses its label too: the row
is hostname, machine, state, and "machine 3015" spends a word on what position
already conveys.

Printer names now carry the floor-plan preview on hover, the same
LocationMapTooltip the printer's own page uses - a location name tells you the
room, the map tells you where to walk. Declared as map.maphover on the card, so
any card with coordinates gets it; a row without them shows a plain link rather
than being dropped.

Cards are four across rather than five. At five columns a row holding a
hostname, a machine number and a state truncates on exactly the rows that
matter. auto-fit, so two cards fill the width instead of leaving empty tracks.

Not covered by a test: the count fix. I started one and it was interrupted, and
I have not gone back for it - the assertion worth having is that in-use can
never exceed the total.
 2026-08-11 16:27:14 -04:00
cproudlock
221bbb226e employees: an endpoint that says WHY a name did not resolve
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
The shopfloor board lost every photo and started showing lowercase SSOs where
names belong, and nothing in the system could say why. Both resolvers returned
None on any problem and three separate bare excepts threw the cause away, so an
unreachable HR host, a rotated credential, a renamed column and a genuinely
unknown SSO all produced identical output and no log line. That is not a bug in
the resolution so much as a hole where the diagnosis should be.

GET /api/employees/resolve/<sso> returns the same answer the board gets, plus
which source produced it (directory, hrdirectory, useaccount), which mode the
directory is in, and the exception text when a source failed. It is the
difference between "the board is broken" and "the HR host refused the
connection".

The two bare excepts in the shared resolvers now log rather than pass. The
external-directory branch is the one that fails on a live site; it was the one
saying nothing.

No behaviour change to the board itself - it still falls back exactly as
before. What changed is that the fallback is now visible.
 2026-08-11 16:10:20 -04:00
cproudlock
42c050a9f4 dashboard: tighten the printer, warranty and notification cards
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
PRINTER CARD is now supplies at or below 5 percent, a new
printers_dashboardpercent setting. The report and the card want different
scopes: the report lists anything the thresholds call low, which is right for
planning an order, while the dashboard is asking what to walk out and change
today - and a cartridge at 18 percent is not that. Lowest first.

PRINTER LOCATION WAS ALWAYS EMPTY. The lookup went through db.session.get on
locationid and produced nothing even where a location is set; the printers list
has always read it through the asset relationship, so the card does too now.

WARRANTY ROWS are identified the way the floor identifies them: the PC's
hostname and the MACHINE it drives, reusing the same lookup behind the warranty
page's machine column so the board and the report cannot disagree about which
bay a PC belongs to. No dates - expired or expiring is the whole decision when
scanning a board, and the exact day belongs on the report you order from.

NOTIFICATIONS are stacked: the type in full on one line, the message beneath,
trimmed to 100 characters with the rest on hover. Inline, the type was
truncated to make room for prose that was then truncated anyway, and neither
read. A tooltip is omitted when the text was not trimmed, because one repeating
what is already on screen is noise.

Two general additions: layout: 'stacked' on a card, and map.detailtooltip.
 2026-08-11 15:57:14 -04:00
cproudlock
8623db3ee2 dashboard: printer rows are a name and its cartridges, with the answer on hover
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
The printer card was a name followed by a comma-joined string of cartridges and
a location - the widest row on the board, and the one running past the card
edge.

Now: the printer name links to its page and reveals its location on hover, and
each depleted cartridge is its own chip showing "Black 4%", revealing the part
number to order on hover. The percentage says something is wrong; the part
number says what to do about it, which today means opening the printer's page
to find out. Every capacity tier is listed, as the report has always done.

Two additions to the card contract, both general: 'chips' maps a row key to a
list of {text, title, level}, and 'titletooltip' puts context on the row title.
Nothing load-bearing goes in a tooltip - hover is not discoverable and does not
exist on touch - so a chip always states the fact and only explains it on hover.

Chips are bordered rather than filled: a row of solid red pills reads as an
emergency even when a cartridge is merely low.

Also repaired a self-inflicted mess. A string-slice edit used a marker that
appears EARLIER in the file, so the slice was empty and two helpers were
injected at line 1, above the module docstring. Removed; the file parses and
the helpers live beside the route they serve.
 2026-08-11 15:46:21 -04:00
cproudlock
c34815b87e dashboard: overflow links somewhere, tiles say what they count, rows stay inside
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Four fixes, all from looking at the real board.

"and N more" now links to a page showing them all. Telling someone 35 more PCs
are silent and leaving them to find the list is worse than not saying it. Each
card names its own destination and a test checks it against the routes that
actually exist - a viewall pointing at a route nobody wrote is the same rot the
endpoint check already guards, just failing in the browser instead of the API.

PRINTER ROWS ESCAPED THE CARD. A flex child will not shrink below its content
width unless told to, so text-overflow never engaged and a row carrying three
cartridge readings plus a location simply ran past the border. min-width:0 on
the row parts is what enables the ellipsis; meta shrinks first because it
matters least, and the card clips as a backstop.

THE STAT TILES WERE INCOHERENT. Two counted asset TYPES, two counted asset
STATUSES, and nothing said which - with the status one labelled "Active", which
reads as "not deleted" but meant status = In Use across every type. Each tile
now counts one thing and its label says so.

PCs GONE SILENT IS NARROWER, and better for it. A PC that never reported at all
is usually a hand-made or imported record rather than a bay that broke, and a
PC that is not In Use is silent ON PURPOSE - that is the status doing its job.
Both were burying the real signal: a machine that was working, is not now, and
nobody has marked as anything else.
 2026-08-11 15:28:29 -04:00
cproudlock
294ddbb38e backups: show that a check happened, not just that a change did
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
lastseenat already recorded it and the API already returned it; nothing
displayed it, so from the UI a healthy machine still looked abandoned - one
revision from last spring and no sign anything had looked at it since.

The history page gains a Last verified column beside Captured, and the asset
panel a Verified field. Only the CURRENT revision carries one: an older
revision was superseded, so saying it was verified today would be false - what
was verified is the configuration the PC holds now.

A current revision with no check yet says "not yet checked" rather than showing
a blank or borrowing the captured date. That state is real and temporary: the
column is new, so every chain reports it until its PC next posts.
 2026-08-11 15:00:25 -04:00
cproudlock
e0e4cce8bd dashboard: fix what a real fleet showed, which tests could not
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 10s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Three faults, visible only once the board ran against production data.

BACKUPS SAID THE WHOLE FLEET HAD STOPPED. The lastseenat backfill was wrong. It
seeded from collectedat, reasoning that the last change was the last provable
moment - but an unchanged config writes no revision, so a machine whose settings
last changed nine months ago got a nine-month-old lastseenat and was instantly
reported as a dead backup. Every chain lit up at once, which is worse than no
card: it says the site is broken when it is fine.

The honest value is NULL. Before the column existed nothing recorded when a
config was last confirmed, and inventing a date does not change that. Migration
0003 clears the backfill, and staleness now IGNORES a NULL chain rather than
substituting timestamps that mean something else. A chain becomes measurable the
first time its PC posts, which for NTLARS is within a day.

TONER READ "None%". The supply dict has no 'percent' key - it is 'remaining'.
Supply names are also shortened, because "Black Toner Level 4%" spends three
words saying what the card already says.

THE CARDS READ AS WALLS OF TEXT. Rows wrapped into paragraphs and a card with
forty PCs pushed everything below it off the screen. Now: at most five rows with
"and N more", one line per row that truncates rather than wraps, meta pushed
right and dropped first since it matters least, and severity reduced to a small
dot beside an uppercase label instead of a coloured card - six severity-painted
cards read as a crisis, which is how a board stops being read.

Worth recording that none of this could fail in a test. Every one needed real
data on a real fleet.
 2026-08-11 14:45:28 -04:00
cproudlock
5eb84873e8 dashboard: convert the last dead widgets, and delete the one that had nothing
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Three plugins still declared widgets naming Vue components nobody wrote.
Converting them honestly meant three different answers, not one.

notifications gets a real card: the active notifications themselves, not a
count. "4 active" tells an admin nothing; knowing WHICH message the shop is
looking at is the point, and it is how a stale one gets noticed and taken down.

machines gets machines out of service - anything not In Use, excluding
Inventory, because a spare on a shelf is stock rather than a problem. Someone is
supposed to be chasing each of those and today they are visible only to whoever
thinks to filter the list by status.

network gets NOTHING, and its declaration is deleted rather than converted.
Network devices carry no live status - no polling, no reachability check,
nothing that can be wrong - so the only possible card is a count of how many
exist, which is precisely the always-true number this dashboard exists to get
away from. A comment records that, so the next person does not re-add it. If
reachability is ever collected, that is the card.

Also adds a contract test over every declared card: no component names, a valid
renderer and severity, and - the one that matters - the endpoint must be a REAL
route. A declaration pointing at a route nobody wrote is exactly how the old
widgets rotted unnoticed for months, and now it fails the build instead.
 2026-08-11 14:27:29 -04:00
cproudlock
7151b68bdd dashboard: printer supplies, expiring warranties, mis-numbered bays
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Wave one complete. Three cards, no new data and no migrations.

Printer supplies reuses the existing low-supplies query and its five-minute
cache; a Zabbix round-trip per printer on every dashboard load would make this
the slowest page in the app. One row per printer listing every depleted
cartridge, criticals first - a row per cartridge would report one printer three
times and read as three problems, and showing only the worst class would hide a
low cartridge behind a critical one on the same machine when whoever walks out
there wants to carry both.

While there: the low-supplies REPORT itself was including healthy cartridges. A
printer with one empty black and three full colour ones listed all four, so the
reader had to find the problem inside the row. It now lists only what needs
replacing, and the test that asserted the old behaviour now asserts the new.

Expiring warranties keeps already-expired entries on the list rather than
dropping them the day they lapse, which is how they get missed. Horizon is
warranty_expiringdays, default 90, because that suits a site budgeting
quarterly and nobody else.

Mis-numbered bays promotes check-shared-machines out of a CLI command nobody
will remember to run - it found seven bays that had been wrong for weeks. It
reports only numbers with NO child assets, so part markers legitimately sharing
an operation stay silent: that distinction is the whole card, and without it it
would list correct data beside faults and be ignored.

Printers also loses its dead component-named widget; notifications, network and
machines still have theirs.
 2026-08-11 14:13:26 -04:00
cproudlock
6c975a107c backups: record that a config was checked, not only that it changed
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 10s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
The stale-backup card could not be built as designed, and the reason is more
important than the card. Dedup means an unchanged configuration writes no
revision, so collectedat moves only on a CHANGE. A machine stable for six
months has a six-month-old newest revision and is perfectly healthy. Keying a
staleness card on revision age would have flagged most of the fleet - exactly
the noise that makes a board worth ignoring.

Underneath that: ShopDB could not distinguish those cases at all. On a no-op
the server returned "unchanged" and wrote nothing, so "we checked yesterday and
it matched" was discarded. That fact is the one thing a backup system must be
able to prove, and the only record of it was a line in a log file on the PC.

lastseenat records the check rather than the change. Touched on every matching
post including the no-op; set on creation, since a new revision has by
definition just been seen; backfilled from collectedat or createdat so existing
rows start from the last moment the config can be PROVEN current, rather than
from now - claiming a check that never happened would be worse than silence.

The card keys on it, one row per CHAIN rather than per asset: a machine with
two part markers can have one still reporting while the other stopped, and a
per-asset view would report the machine as fine. It stays deliberately silent
about assets never backed up, because whether one SHOULD be is a question only
the manifest can answer, and guessing would list a hundred healthy machines.

The rule lives in services/staleness.py rather than the route, so it is
testable without an auth layer in the way - the same split retention.py uses.

Threshold is backups_staledays, default 3, and 0 disables the card.
 2026-08-11 13:52:07 -04:00
cproudlock
1ca8a9b8e8 dashboard: PCs not reporting, and the card styling standard it broke
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Second wave-one card. GET /api/computers/dashboard/quiet lists two populations
and deliberately does not merge them into one count. A PC that reported and
went quiet is probably off, moved or broken. A PC that has NEVER reported is
worse: not enrolled, or enrolled against the wrong pc-type, so nothing enforces
anything on it and no backup of it exists. That one hides indefinitely because
nothing about it fails loudly - the same shape as the bay that carried a wrong
machine number for weeks.

Never-reported sorts above the merely quiet, then longest silence first: the
order someone should work down the list, not the order rows left the table. A
soft-deleted PC is excluded - a decommissioned machine is silent on purpose,
and listing it would train people to ignore the card, which is the failure this
whole board exists to avoid.

The window is computers_quietreporthours, default 24, because every site will
disagree with any number picked here (ADR-015). A malformed value falls back
rather than failing the card.

This also replaces the computers plugin's old widget declaration, which named a
component nobody ever wrote. Four such declarations remain and will convert as
their cards arrive.

Two fixes to the renderer found while wiring this up. Meta specs now support a
trailing unit, so a row reads 'quiet for 3 days' rather than 'quiet for 3'. And
the card styles hardcoded hex colours against the frontend standard, including
a var(--card-bg) that DOES NOT EXIST - the variable is --bg-card - so the
fallback would have painted every card white and broken dark mode entirely.
Now --bg-card, --border, --danger, --warning, --primary and --link throughout.
 2026-08-11 13:40:41 -04:00
cproudlock
05c150c663 dashboard: render plugin-declared cards, starting with enforcement failures
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
The frontend now calls /api/dashboard/widgets. It never had, which is why five
plugins have been declaring widgets into a void for months, pointing at
components nobody ever wrote.

Core owns three generic renderers - exceptions, metric, list - and a plugin
declares data, a shape and a link template. The mapping logic lives in a plain
module beside the component, the same split as pluginAssetPanels.js, so it is
unit tested without mounting anything: 16 tests covering row mapping, empty
handling, ordering and gating.

The behaviours worth naming, because each is a decision rather than an
implementation detail:

Cards fetch INDEPENDENTLY and a failure becomes null. One hung endpoint - a
Zabbix call, a plugin mid-upgrade - cannot blank the board. A card whose fetch
failed HIDES rather than drawing empty, because "nothing wrong" and "I could
not tell" must not look the same.

Empty cards disappear by default. A card reporting nothing every day teaches
people to stop reading the page, which is precisely how a fleet log reached
3,234 lines with 17 that mattered. A card opts into a one-line presence only
when its absence is itself news.

Severity outranks position, so an info card can never sit above a failure.

Permission filtering happens BEFORE fetching: no point firing a request that
would only 403, and the dashboard must not become a way around RBAC.

An unknown render mode is skipped, so a plugin built against a newer core
degrades instead of leaving a hole.

A row whose link substitution is missing keeps the row and drops the link -
a PC shopdb does not know still reports its failure, and that is the bay most
likely to be misconfigured.

Cards sit ABOVE the totals: what needs a person first, context second. The
existing stat cards are untouched for now.
 2026-08-11 13:09:46 -04:00
cproudlock
8b50e6fe2a geenforce: first dashboard card, and the widget contract it proves
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Wave one of the dashboard proposal, built as a vertical slice so the contract
is proven by something real before the other five cards follow.

GET /api/geenforce/dashboard/failures lists entries that FAILED on their PC's
most recent enforcement cycle. Per ENTRY, not per report: "three PCs failed" is
a number, while "Install OpenText failed with exit 1603 on WJSF1234" is
something a person can act on. Only current reports count, so a failure that
has since been fixed clears itself instead of needing dismissing. Hostnames
resolve to computerids in one query so each row links to the PC, and a PC
shopdb does not know still appears - the failure is real even when the
inventory is behind, and that is the bay most likely to be misconfigured.

The data has been there all along. The only way to see any of it was to open
one PC's report modal, one PC at a time.

The widget declaration is the contract change. The old shape named a Vue
component per widget, which cannot survive a lean build where a plugin's
component may never be staged into the bundle - which is exactly why five
plugins declare widgets pointing at components nobody ever wrote. This declares
data, a generic renderer, a permission and a link template, the way ADR-010
already does for asset panels. A test asserts no 'component' key, so the old
shape cannot creep back.

empty: hide is part of the contract, not decoration. A card reporting "nothing
wrong" daily teaches people to stop reading the page, which is how a fleet log
reached 3,234 lines with 17 that mattered.

Frontend rendering comes next; the endpoint and declaration stand alone and
change nothing that exists.
 2026-08-11 13:01:04 -04:00
cproudlock
02d7619b1a Propose a dashboard that shows the fleet, not the row count
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
The dashboard reports totals that are true every day and actionable on none of
them. Meanwhile a bay returned 500 to every collector report for a day and a
half, eleven markers' backups overwrote each other for weeks, and GE-Enforce
records failures, stale manifests and repeated self-heals that surface nowhere.
Every one of those was found by someone going to look.

Two things found while writing this shaped it. The plugin mechanism already
exists - get_dashboard_widgets, an endpoint that aggregates across enabled
plugins and isolates failures, and five plugins declaring widgets. And nothing
consumes it: the frontend never calls the endpoint and not one of the declared
components was ever written. So this finishes a half-built feature rather than
starting one.

It also proposes fixing the contract while it is still unused. Naming a
component per widget does not survive a lean build, where a plugin's component
may not be staged into the bundle at all. ADR-010 already solved this for asset
panels - declare data and a render mode, let a generic core component draw it -
and the same three renderers cover every card listed.

The rest is principles that came out of this week: empty cards shrink, because a
card that says "nothing wrong" daily trains people to stop reading, which is how
a log reached 3,234 lines with 17 that mattered; every row links to the thing;
cards declare a permission so the dashboard cannot become a way around RBAC;
thresholds are settings with neutral defaults.

Wave one is six cards whose data is already reliable. Wave two is the valuable
one: diffing what a manifest says a PC should have against what it reports
having, which nothing answers today and which is what GE-Enforce exists to
guarantee.
 2026-08-11 12:50:16 -04:00
cproudlock
105345fb3d Release 0.9.0
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Product version to 0.9.0, frontend in lock-step, Unreleased notes moved into a
dated section per ADR-007. The plugin contract stays at 0.16.0: it moved after
v0.8.1 and is already recorded in this release's notes, and nothing since
touched the contract surface.

A minor rather than a patch: collector behaviour changed in ways an integrator
must know about. A reported machine number no longer becomes the PC's asset
number, it builds a controls link instead; a second PC claiming a machine is
treated as a claim rather than a handover; and a backup revision chain is now
per source PC rather than per asset.
 2026-08-11 12:41:35 -04:00
cproudlock
bf75cdc43b docs: keep real fleet hostnames out of a page that publishes to the wiki
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
The check-shared-machines example carried two actual prod hostnames. docs/ is
excluded from the code bundle and goes to the GitHub wiki, so an example is the
wrong place for live fleet identifiers. Replaced with placeholders.
 2026-08-11 12:10:04 -04:00
cproudlock
db2b9280e7 docs: write down the composition pattern, not just the one case
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
The MECHANISM was already a documented platform contract - ADR-001 defines
partof as composition, makes controls propagate through it, and walks it first
for map-position inheritance. The part-marker work used that rail rather than
inventing one.

What was undocumented is the PATTERN built on it: several devices answering to
one identifier, each becoming its own asset filed under a parent. It existed
only as a collector behaviour for part markers plus a docstring in the device
map, so nothing told anyone how to apply it to another device type, or when not
to.

ASSET-COMPOSITION.md covers when to reach for it and when the shared identifier
is a numbering fault instead, what propagation buys, how to declare a device
type through the map or a per-site setting, what a backup kind must do to
follow the device rather than the parent, how to find the next case with
check-shared-machines, and why the parent is not disposable once devices hang
off it - deactivating it breaks filing, and a hard delete cascades through
backuprevisions.
 2026-08-11 12:04:35 -04:00
cproudlock
c90ebcbc7c computers: declare subordinate devices instead of coding each one
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 6s
Details 
A PC that drives a device which is its own asset had been implemented twice.
METROLOGY_TOOL_MAP covered CMM, Keyence, Genspect and wax-trace, minting a
measuring_tool. A separate path keyed on one hardcoded pc-type minted a Part
Marker machine and filed it under its operation. Both create a device, link the
PC with controls, and archive that link when the PC is re-imaged: one mechanism
with different nouns, written out twice because the second case arrived later.

That is the same trap as the site literals in ADR-015 - a pattern implemented
per instance rather than declared - and it has a known next occurrence. Part
markers already share operation numbers, and any site with two marking lasers
or two wax-trace units on one number needs identical treatment.

One SUBORDINATE_DEVICE_MAP now declares asset type, type name, naming suffix,
whether the device files partof the operation, and the relationship label. The
labels are unchanged per case on purpose: those values are in the production
database and only rows carrying them are archived by a collector push. A site
overrides or adds an entry through subordinatedevice_<pctype> settings, per
ADR-015, so the next case needs no code. A malformed override falls back to the
default rather than failing the push, because a bad setting must not stop a bay
reporting its inventory.

metrology_tool_for stays as a shim over the same map: filters.py and the older
tests read it, and unifying must not change what it returns. A test pins that.

Also adds flask relationships check-shared-machines, which finds the next 0615
rather than waiting for someone to notice duplicate backups. Several devices
legitimately sharing a number and two PCs mis-numbered at imaging look the same
from outside; the difference is whether child assets exist, so that is what it
reports. Read-only.
 2026-08-11 11:13:12 -04:00
cproudlock
91143d94fb ADR-016: credential delivery to the fleet
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Controller and share credentials are currently edited as cleartext into a
script on the SFLD share, one value for the whole fleet, re-applied by
GE-Enforce every cycle. Rotating means editing that file and hoping - nothing
reports which bays picked the new value up - and sites need per-bay variation,
which a manifest scope cannot express because scopes are per PC TYPE. A scope
per bay is a spreadsheet, not a model.

ShopDB therefore owns the definitions and the targeting, and a PC receives a
decided answer rather than rules to evaluate. Targeting uses the axes
GE-Enforce already filters on plus those the asset model knows: hostname
(including -like globs), machine, model, controller OS, PC type with alias
expansion, GE-Enforce profile, and a selector over the stored DNC projection -
"every machine whose DataHost is that share" is answerable from data already
held, and it also answers which machines need a credential at all.

Overlapping matches are guaranteed rather than exceptional, so precedence is
explicit: an integer priority, then scope specificity, then rule id for
stability. The resolver must be able to explain which rule won and why, or
nobody will trust it. It reuses plugins/geenforce/filters.py; there are already
two implementations of this matching logic kept honest by a parity harness, and
a third would drift silently.

Targeting stays separate from detection, mirroring PCTypes versus
DetectionMethod: who gets a credential is a server question, whether it is
already applied is a local one, and conflating them rewrites the secret every
cycle instead of healing drift.

Stored Fernet-encrypted with the key OUTSIDE the database, write-only through
the API and masked on read, fetched with a dedicated credentials.fetch scope so
a leaked collector key does not yield controller passwords, and every fetch
audited.

PROPOSED, not accepted: it makes ShopDB a credential store, and the key becomes
a single point of loss - restore the database without it and every credential
is unrecoverable, which docs/BACKUP-RESTORE.md must state in the same change
that implements this. Registry-only provisioning is recommended as a first step
regardless, since it removes the cleartext from the share immediately and the
client helper is identical either way.
 2026-08-11 10:54:42 -04:00
cproudlock
6516e76bf4 backups: the collection interval was never actually readable
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Backup-NtlarsSettings reads backups_intervalhours from /api/settings/public,
because it runs before it holds any credential, and its Get-IntervalHours falls
back to 24 on any failure. The plugin never declared the key public, so the
endpoint did not return it, the fallback fired on every PC, and the setting
looked configurable in the UI while changing nothing. The fleet log shows the
symptom plainly: "Throttled: last attempt under 24h ago", every cycle,
regardless of what the setting said.

Declared public. A collection cadence is not a secret. backups_shareroot stays
private - it is internal topology - and the test asserts both directions so a
later edit cannot quietly widen it.

Same defect as the 3D parts kiosk label prefix already in this changelog: a
logged-out reader against an allowlist its key was not on. Worth noticing that
the pattern has now bitten twice.
 2026-08-11 09:53:43 -04:00
cproudlock
13e6e039fe ADR-015: where a site's own data is allowed to live
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
ShopDB is a product several sites adopt, developed at one site against that
site's live fleet, so every feature arrives carrying West Jefferson's
hostnames, networks and share paths. A scan of plugins/ and shopdb/ returns 19
site literals, of which about 8 are a shipped default or a hardcode rather than
a comment: a kiosk URL baked into a PowerShell payload a sister site cannot
override, a WJ UNC path as the backup share default, and device.geaerospace.net
shipped as a SETTING DEFAULT - inherited silently by anyone who adopts.

The mechanisms already existed; the rule did not, so each value landed wherever
was convenient. The rule: a setting with a NEUTRAL default first, a
site-namespaced directory second, seed data third. Blank must be a working
state. Fleet-wide vocabulary such as the gea-shopfloor-* imaging types is not
site data and is out of scope - it already has a per-site override.

Enforced by an addition to check-naming-and-style.sh, which CI already runs, in
REPORT-ONLY mode. Making a 19-item backlog a hard gate the day it lands blocks
every unrelated commit until someone clears it; SITE_LITERALS_ENFORCE=1 turns
it into a gate once the listed hardcodes are done. The ADR carries that backlog,
including that the display FQDN domain is defined three times in three files.
 2026-08-11 09:21:23 -04:00
cproudlock
78c1c4709f docs: the collector contract described the machine-number defect as the rule
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
COLLECTOR-INTEGRATION.md told integrators that machinenumber maps to
Asset.assetnumber and that a real value updates it on an existing PC. That is
precisely the behaviour that returned 500 to every bay reporting a machine
ShopDB already knew, and it is the document another site reads to build its
reporter.

Rewritten to say what the field is for: it identifies the machine, the PC keeps
its hostname, and the number builds a controls link. Adds a section covering
the claim-versus-handover rule and its quiet window, the alert gate defaulting
off, and how a part-marker PC gets its own marker asset under the operation.
 2026-08-11 09:13:00 -04:00
cproudlock
300f9123d2 backups: re-file historical revisions onto the part markers they came from
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 11s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Backups collected before markers became assets were filed against the operation
number the PC reported. The collector now files a marker PC's backup against
its marker, but only for backups collected since; everything already stored
stayed where it was, so the new marker assets read as empty while the operation
holds a mixed history.

The attribution is exact rather than a guess. A revision records the PC it was
read from, and the collector records which marker each PC drives, so the move
is that mapping applied. Revisions with no source hostname stay put, since
nothing says which marker they came from.

Moving a revision can duplicate one the collector has already written against
the marker, so the script collapses those as part of the move, keeping the
earliest so the date a config was first seen survives. Dry run by default.
 2026-08-11 08:13:21 -04:00
cproudlock
738f30dca3 computers: never file a part marker under its own PC
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
A PC first seen before the machine-number fix was created with the machine
number as its OWN asset number, and the fix deliberately does not overwrite an
existing PC's asset number. So resolving the reported number can return the
reporting PC itself, and the marker was then filed partof its own PC - which
reads, on the machine page, as the PC being the operation.

The machine-link path already guarded this case; the marker path did not. It
now refuses and says why, naming the repair: rename the PC asset to its
hostname, or create the operation asset.
 2026-08-10 17:01:23 -04:00
cproudlock
a61739d1ab computers: a part marker is its own asset, under the operation it serves
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Several Telesis markers serve one operation number - 0613, 0615 and WJPRT each
have more than one - so treating the operation as the marker collapsed separate
devices into a single record. Their configurations differ by COM port, so in
the backup history they overwrote each other, and no question about an
individual marker could be asked at all: how many there are, which port one is
on, which one failed.

There is one marker per PC, which makes the PC the marker's identity, so the
collector can mint the marker the same way it already mints a CMM or a Keyence
unit for a metrology PC. A marker PC now gets a Part Marker machine asset, the
PC controls it, and the marker is partof the operation whose number the PC
reports. An operation holds any number of markers.

A marker PC therefore does not claim the operation directly. controls
propagates through partof, which reference-data already seeds, so control of
the operation still follows from controlling its marker - without two markers
contesting a link only one of them can hold.

Backups from a marker PC resolve to the marker rather than the operation, and
fall back to the machine number whenever the marker cannot be resolved: no
hostname on the payload, a lean build without the computers or machines plugin,
or a marker PC that has not reported to the computers collector yet. Filing
under the operation is the old behaviour and beats rejecting a backup.

Moving a marker to another operation archives the old membership rather than
deleting it, so where a marker used to live stays answerable.
 2026-08-10 15:54:53 -04:00
cproudlock
d429c882b4 backups: a revision chain belongs to a PC, not just a machine 
Dedup compared a posted config against the latest revision for the ASSET,
which is only correct when a machine number means one PC. Several PCs share one
here: the part markers on 0613, 0615 and WJPRT are separate devices, differing
by COM port, filed under one machine number. Each marker's post therefore
differed from whichever marker had posted last, nothing ever deduped, and the
table grew by one row per PC per collection cycle.

A chain is now (asset, kind, source hostname). An unchanged config is a no-op
again, and each PC keeps its own history against the machine. NULL sources -
rows written before the column was populated, and hand-loaded ones - form their
own chain via IS NULL; `column == None` never matches in SQL, so without that
those rows would have re-posted forever.

Two consumers assumed the old key and are fixed with it. Retention pruned per
asset, so a busy marker's revisions could evict a quiet marker's only backup;
it now prunes each chain separately, protecting the newest and oldest of each.
The revision diff compared against the previous revision on the machine, which
across two markers reported one device's COM port as a change on the other; it
now compares within the source's own chain.

scripts/collapse_duplicate_backup_revisions.py cleans up what the old rule
wrote. It removes only a revision whose hash repeats the one before it in the
same chain - rows the fixed code would never have written - and keeps every
genuine change, every chain's newest and oldest, and every source. Dry run by
default. Its --report mode explains what grew each chain, which separates a
legitimately shared machine number from two PCs wrongly carrying the same one,
and from a value inside the config that changes on its own.
 2026-08-10 15:02:38 -04:00
cproudlock
5108ba8aaa computers: a second PC claiming a machine is a claim, not a handover 
Treating "another PC is linked to this machine" as proof of replacement was
wrong. A PC imaged for machine 3010 carries that number from the bench, before
it has replaced anything, and several PCs sharing one machine number is a
normal state at this site: the part markers do it. Both PCs then reported on
their own schedules, each report moved the link and raised an alert, and the
pair traded the machine back and forth for as long as both were alive.

The PC holding a machine now keeps it while it is still alive. Alive means it
has reported within MACHINE_CLAIM_QUIET_HOURS and its asset is still In Use. A
challenger is recorded as a dormant link instead, which doubles as the marker
saying the claim has already been announced, so a PC sitting on a bench does
not alert on every collector cycle.

The handover still happens on its own once the old PC has been quiet for a day,
which is what a PC pulled off a machine does. Moving the old PC off In Use -
Retired, Inventory, In Repair - hands the machine over on the next report,
which gives IT a one-step way to force a swap the moment it happens rather than
waiting out the window. A day is long enough that a PC switched off overnight,
or one behind a network outage, never loses its bay to a spare.

Alerts for both cases are gated on a new computers_machinelink_alerts setting
and ship OFF. Several part markers legitimately share a machine number here, so
the alerts would fire on correct data. Links, warnings in the collector
response, and archived history are unaffected; only the sending is gated.

Also: the alert goes through send_alert rather than resolving recipients by
hand, which had missed the SMTP_ALERT_RECIPIENTS environment fallback, so a
site configuring SMTP by environment would have got the webhook and no email.
 2026-08-10 14:48:51 -04:00
cproudlock
9512b0bdb3 computers: the machine number identifies the machine, not the PC 
A bay reporting machinenumber 3015 got a 500 from the collector every five
minutes since it was imaged, and would have forever: the reported number was
written to the PC's own assets.assetnumber, which is uniquely indexed and
already held by machine 3015, so the insert failed with "Duplicate entry '3015'
for key 'ix_assets_assetnumber'" and the entire report was discarded. Operating
system, boot time, applications, printers and access protocols never landed.
Every retry did the same thing, so there was no path out of it.

A new PC now takes its hostname as its asset number, which is what the data
already shows: of 289 computers none has a numeric asset number and 214 use
their hostname. An existing PC's asset number is left alone; overwriting it
renamed the PC onto the machine's identifier, changing how that PC is
identified everywhere else.

The machine number instead does what it was collected for. It resolves the
machine and links the PC to it with a 'controls' relationship carrying a
collector:machine origin label, the same discipline the printer and
measuring-tool links use, so a link made by hand is never archived by a
collector push. Reporting a different machine archives this PC's previous link;
a machine ShopDB does not know is reported as a warning rather than invented.

When another PC was already linked to that machine it has been replaced. The
old link is archived rather than deleted, so which PC ran a machine in a given
month remains answerable, and an alert goes out by email and webhook. The
retired PC's status is deliberately not changed: the collector cannot tell
whether it was shelved, sent for repair or re-imaged for another bay, and
guessing would overwrite what a person set.
 2026-08-10 14:24:55 -04:00
cproudlock
b8398a36eb backups: an empty value in a .reg export must not fail the file 
A line of the form "Name=" with nothing after the sign is not strictly legal,
but it occurs in real exports - the part marker's WJPRT.reg has KRelay1 like
this. The parser raised on it, which failed the whole file, which meant that
machine could never be backed up at all. Read it as an empty string so the
value name is still preserved.
 2026-08-10 14:24:41 -04:00
cproudlock
2785c0463e warranty: use the shared LocationMapTooltip for the machine preview
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
The machine map preview was a bespoke popover. LocationMapTooltip already
existed and is what the machine detail page uses, so the warranty tables now
show the same thing.

That is what was asked for - the preview zooms, as it does on the machine page -
and it comes with behaviour the bespoke one did not have: scroll-wheel zoom, and
staying open while the pointer is on the tooltip itself so it can actually be
read and panned.

It also solves the clipping properly. The previous commit reached for
position: fixed with a hand-rolled flip because .table-container's overflow-x
clipped an absolute child; LocationMapTooltip teleports to body, which avoids
the clipping context altogether rather than escaping it. All of that
positioning code, and the blueprint rendering, is deleted - 64 lines from 140.

An unplaced machine keeps its chip and now says why in the native tooltip
rather than showing an empty panel: 17 of the 142 linked machines have no map
position, and those rows still need their number.
 2026-08-10 10:05:15 -04:00
cproudlock
4b1a64d298 warranty: stop the machine map preview being clipped on the last row
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
The bottom row's preview was cut off behind the pagination controls.

.table-container sets overflow-x: auto, and ANY non-visible overflow makes an
element a clipping context, so the absolutely-positioned preview was cropped at
the container's edge. The last row had nowhere to open into.

The preview is now positioned fixed from the chip's bounding rect, which
escapes the clip entirely, and flips ABOVE the chip when there is not room
below. It also clamps horizontally, since the column sits well to the right on
a wide table and the panel is nearly 300px.

Height is measured from the rendered element rather than assumed: it depends on
the blueprint's aspect ratio, which differs per site. That means one tick where
the element exists but is unplaced, so it starts hidden and is revealed once
positioned - otherwise it flashed in the corner of the screen.

Also sets white-space: normal. The table sets nowrap for its cells, which the
preview inherited and which ran the location line off the panel.
 2026-08-10 09:59:30 -04:00
cproudlock
2bfb5e7e69 settings: make the user and audit-log dialogs opaque
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 6s
Details 
The Add User dialog was see-through over its own overlay in dark mode. Both it
and the audit-log detail dialog set background: var(--bg-card), which is
translucent in dark mode - frontend/CLAUDE.md already names this trap and says
a modal panel must use var(--bg-card-solid). The shared Modal.vue and the
.modal rule in style.css were already correct; these two hand-rolled panels
were not.

Audited every modal-ish panel in the app while here. The remaining
var(--bg-card) uses are correct and deliberate: .relationships-section and
.permission-category are cards, and the audit-log .filters rules are form
controls - translucency is the intended look on all of them.

One more was worth changing: the warranty asset-search dropdown floats above
the form INSIDE a modal, so a translucent panel let the fields show through the
suggestions. That one is now solid too.
 2026-08-10 09:41:59 -04:00
cproudlock
9e271b4c03 warranty: show the machine a covered PC drives, with a map on hover 
A shopfloor PC is bought, warranted and replaced as a PC, but it is FOUND by
the machine it drives - nobody walks the floor looking for an asset number. The
warranty tables listed the covered asset and left the reader to work out where
that is.

Both tables gain a Machine # column. The payload resolves it by walking the
asset relationship graph in BOTH directions: the canonical edge is
PC --controls--> machine, but a dual-bay pair carries controls on both bays and
hand-made links are not reliably oriented.

Hovering the chip shows the floor map with the machine marked, so the row
answers "where do I go" without opening anything. The blueprint follows the
viewer's theme and the marker is placed from mapx/mapy as a percentage of the
configured map dimensions, since the preview is a few hundred pixels wide
rather than the full plan. It renders only while hovered, so a long table does
not build a blueprint per row.

A machine with no map position still gets its chip and says so, rather than
being dropped: against real data 142 PCs resolve to a machine and 125 of those
are placed, so 17 rows would otherwise have silently lost their number.

The chip is deliberately not a link. /machines/:id is keyed by machineid, not
assetid, and resolving one to the other here would make the warranty plugin
import the machines plugin (ADR-014). Worth noting separately: the existing
assetLink() in these tables already sends machine-type assets to
/machines/<assetid>, which is that same mismatch and predates this change.
 2026-08-10 09:41:48 -04:00
cproudlock
c0a7655aab geenforce: kiosk watchdog must see a WINDOW, not just a process
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Reported from a real display: Edge closed on the desktop but still listed in
Task Manager, and the watchdog never relaunched it.

That is this watchdog's own bug. It asked only whether a --kiosk process
existed. After an Edge update the window can be gone while the process lingers,
so the check said "kiosk is up" and returned - every cycle, forever. Matching on
the command line was chosen to stop a stray renderer masking a dead kiosk; it
does not help when the orphan is the parent.

A kiosk now counts as up only when a --kiosk process still owns a visible
window (MainWindowHandle). Windowless ones are killed BEFORE relaunching:
leaving them would satisfy the next cycle's check again, and a second browser
would fight the first for the display.

Also refuses to run as SYSTEM. MainWindowHandle is session-scoped, so a SYSTEM
caller reads 0 for a perfectly healthy kiosk and would kill and relaunch it on
every cycle. The task uses an interactive Users principal so this cannot
normally happen; the guard makes a mis-registered task fail loudly instead of
thrashing a display in a hallway.

Verified on Windows this time by letting the SCHEDULED TASK do the work rather
than invoking the script by hand - which is what the first version was missing:
  - task fired unattended (rc=0) and launched the kiosk into session 1
  - the next cycle saw the healthy kiosk and did nothing, no relaunch loop
  - a windowless --kiosk process was killed and replaced
  - run as SYSTEM, it refused and the healthy kiosk survived
 2026-08-10 09:03:10 -04:00
cproudlock
4d3985ce22 shopfloor dashboard: show site time, and align the state badge
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Every time on the board was wrong, from two faults stacked.

The shopfloor feed serialised starttime/endtime with a bare isoformat(). Those
columns are stored NAIVE but hold UTC, so an untagged string is read by the
browser as LOCAL and every card shifted by the tz offset. The model's to_dict
already learned this - its _utc_iso helper documents the exact symptom, a 14:34
notification showing 18:34 - but the feed had not, so the feed now uses it too.

The dashboard then formatted with toLocaleString, i.e. the VIEWER's zone. A
board hangs on a wall in the plant: it has to read plant time whatever the
machine driving it is set to, and a kiosk with a wrong system timezone would
otherwise show wrong times to the floor with nothing to reveal it. It now
loads site_timezone and formats through formatInZone, the wall clock included -
a header disagreeing with the cards beneath it is worse than either being
wrong alone.

startsWhen was worse still: it decided TODAY/TOMORROW from browser-local
calendar days, so the wording itself could differ between the board and a
remote admin looking at the same card. That arithmetic now runs on the site's
calendar day.

Separately, the type chip carried a margin-bottom while the state chip beside
it did not. .chip-row centres each item's MARGIN box, so that margin lifted the
type chip about 4px and left "Starts Thu, Aug 13 8:00 PM" looking low. The row
already provides the spacing, so the chip's own margin is gone.
 2026-08-10 08:34:31 -04:00
cproudlock
efe34034e3 computers: collect remote-access protocols (RealVNC and friends) 
The accessprotocols / computeraccess tables replaced the old isvnc/iswinrm
booleans and the PC page already badges what a machine exposes, but nothing
kept them current: the 574 rows in place all came from the legacy migration and
have not moved since. The collector schema had no field for them.

Adds 'accessprotocols', a list of catalog names, synced with the same
discipline as the printer links. A reported protocol is activated; a
catalogued one the PC did NOT report is deactivated rather than deleted, so a
manual portoverride survives a service being briefly down. An unknown name
warns and is skipped: the catalog is admin-managed, and a typo on one bay must
not invent a protocol for the whole site.

Presence of the key is what drives the sync. A payload without it leaves every
existing row untouched, which is what protects the migrated rows from a
collector that does not report protocols yet.

Six tests cover recording, case-insensitive matching, deactivation on removal,
the omitted-key no-op, an explicit empty list meaning "exposes nothing", and
that an unknown name never creates a protocol.
 2026-08-10 08:34:19 -04:00
cproudlock
939cdd0882 geenforce: relaunch the display kiosk when Edge goes away
Some checks failed
CI / backend (push) Failing after 9s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 6s
Details 
Kiosks were updating, closing Edge, and never coming back - the display stayed
dead until the next logon or reboot.

The kiosk is launched by an all-users Startup shortcut, which runs ONCE at
logon, and nothing supervised the browser afterwards. RelaunchNotification=2
was meant to cover the update case and does not: that policy drives Edge's own
update-restart, which depends on session restore to return to where it was.
Kiosk mode restores no session and has no UI to show the notification in, so
Edge honours the close and never the relaunch. The same gap swallowed crashes
and anyone closing the window.

Adds a scope entry that registers a scheduled task in the INTERACTIVE session -
SYSTEM cannot launch a visible browser, which is why the dispatcher writes a
shortcut rather than calling Start-Process. The task relaunches from that same
shortcut, so the target URL keeps one source of truth: retarget a subtype in
DISPLAY_TYPE_TARGETS and the watchdog follows unchanged.

Two details that matter. It matches on the COMMAND LINE, not the image name:
Edge runs a crowd of msedge.exe children and only the parent carries --kiosk,
so testing "is msedge running" would let a stray renderer mask a dead kiosk
forever - verified against a real kiosk PC showing 7 processes and 1 match. And
it avoids -RepetitionDuration [TimeSpan]::MaxValue, which serialises out of
range and is rejected, exactly as the kiosk installer documents.

A launch debounce stops a display that fails to start from spawning a browser
every cycle, the log is size-bounded because this runs forever on a PC nobody
watches, and it does nothing at all when no kiosk shortcut is present so it
cannot put Edge on a PC that never asked for one.

Verified on Windows: registers with the right principal and triggers, relaunches
when the kiosk is gone, debounces an immediate re-run, and is idempotent across
cycles (the staged script compare is trimmed - Set-Content adds a trailing
newline the here-string lacks, so an untrimmed compare rewrote it every cycle).
 2026-08-10 07:49:43 -04:00
cproudlock
178dacd55e Make '+N more' actually show the events behind it
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
The link had a custom hover tooltip that took the hidden events with
dayEvents.slice(3) - correct while dayMaxEvents was a fixed 3, wrong the moment
that cap started varying with the row height. On a short row showing one chip,
'+4 more' sliced from index 3 and listed the wrong events; on a day with three
events it sliced to nothing, hit the empty-list guard and rendered no tooltip at
all. A link with nothing behind it.

FullCalendar's own popover replaces it rather than the arithmetic being fixed:
no index to drift out of step with the cap, a header and a close button, every
event for that day listed, and it works on a touch screen - which a hover
tooltip never did on a kiosk. Clicking an event in the popover still opens the
detail modal.

That takes the hover handlers, the container mouseenter/mouseleave delegation,
the tooltip markup and styles, and the by-date index that existed only to feed
them: 108 lines out, 25 in. The popover is themed through the CSS variables,
since FullCalendar ships it light.
 2026-08-08 15:36:01 -04:00
cproudlock
a4a862b42a Take the last scrollbar off the calendar
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
An audit of every scrollable element on the page, rather than the document
alone, found what the previous two attempts kept missing: FullCalendar's
day-grid scroller sits 16px short of its own content at every window size.

It is phantom. The table inside measures exactly the scroller's height, the
last week sits flush with the bottom, and the horizontal axis is clean - there
is nothing to scroll to, but the browser paints a bar for the gap. Shrinking
the calendar to close it does not work either: the gap stays 16px however small
the grid gets, so the previous pass had started shrinking the month for
nothing. That correction is reverted to page overflow only.

The grid is sized to fit its six rows, so the scroller has nothing to reveal;
hiding it is safe and is what the earlier arithmetic was failing to buy.

Audited with 60 events across 12 days at 1920x1080, 1600x900, 1366x768,
1280x720 and 1920x1200: no scrollable element left in the calendar, six weeks
visible, last week fully in view, badges shown with a '+N more' where a short
row cannot hold them all.

The nav sidebar still scrolls on a short window - 100vh of viewport against
about 1384px of menu - but that is every page, not this one.
 2026-08-08 15:26:11 -04:00
cproudlock
686966b3b1 Fit the calendar to the space it actually has
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
The previous attempt at this was verified against an empty month, which is why
it measured clean and still scrolled: put events in the month and the fixed
height clipped the grid, leaving the last week behind FullCalendar's own
scroller. The media-query floors it added are gone.

Two things were guessed and are now measured. The height came from
calc(100vh - 230px), a stand-in for chrome that is not 230px tall; it is taken
from the container's real top at mount and on resize, then corrected against
whatever the page still overflows by. And dayMaxEvents was a fixed 3, which
does not fit an 80px row - hence the overflow. It is `true` now, so
FullCalendar shows as many chips as the row genuinely holds and rolls the rest
into a '+N more'; that is the only setting that cannot outgrow the box. The
chips and the day-number strip are tighter, so more fit before that happens.

With 44 events across 11 days, at 1920x1080, 1600x900, 1366x768 and 1280x720:
all six weeks visible, no page scrollbar anywhere. A full-height display shows
every chip; a laptop shows one and a link.
 2026-08-08 15:12:01 -04:00
cproudlock
f67d975058 Stop the calendar growing a scrollbar on a short window
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
The floors that keep the month grid from squashing - 620px on the calendar and
110px per day cell - add up to more than calc(100vh - 230px) yields on a 768 or
720 tall window. The floors won, the page overflowed, and a month that used to
fit needed scrolling to see.

They relax below 900px tall: 78px cells, which still show the day number and
three events, so the grid stays readable rather than collapsing to the strips
the floors were guarding against. Taller windows keep the roomy cells.

Measured at 1920x1080, 1600x900, 1366x768, 1280x720 and 1080x1920 portrait:
six rows and no page scrollbar in all five.
 2026-08-08 14:52:19 -04:00
cproudlock
65f53e91cd Colour every board row from its own types, and say what has not started yet
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 6s
Details 
Two of the board's colours were still hardcoded: carousel headings were gold
and grid headings blue, left over from when carousel meant Recognition and grid
meant Recertification. A type wearing another type's colour is worse than no
colour at all, and it was the whole of the "why did my category turn blue"
puzzle - it had turned from one style's hardcoded colour to another's.

Headings now take a colour from the types beneath them. Where a row holds
several types, the one with the most cards wins, ties going to whichever sorts
first on the board. Live cards decide it; the upcoming set only gets a say when
nothing on the row has started yet, so a heading always describes what is
happening now. That covers the standard rows too - the old getSectionClass
painted the list red off a legacy 'danger' keyword.

Upcoming cards say so plainly: STARTS TOMORROW 2:06 PM, or the weekday and date
beyond that, on an amber chip, with a dashed border and a hollow carousel pip.
Shape as well as colour, so it reads from across the floor and does not depend
on telling amber from white.

Cards and tiles also carry the type's colour as a bar down the left edge, the
way the legacy board marked them; a 2px border reads as chrome at distance.
 2026-08-08 14:20:36 -04:00
cproudlock
442a631557 Let a site choose the order its board rows run in 
Rows ran in display-style order and then alphabetically, so what led the screen
was an accident of styling and the alphabet - a new type called Awareness
landed above Recertification for no better reason than the letter A.

Each type now carries a board position, lowest first, set on the Notification
Types page. The migration seeds Recognition at 10 and Recertification at 20 and
leaves everything else at 100, so an existing board keeps the order sites
already expect. Steps of ten leave room to slot a row in without renumbering
the rest.

A row shared by several types sits wherever its earliest-ordered type puts it,
so a category moves as a unit.
 2026-08-08 14:20:35 -04:00
cproudlock
adb8542018 backups: show timestamps in the site timezone
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Backup timestamps read wrong because of two faults stacked, which is why it
looked like a single offset.

The API serialised naive ISO ("2026-08-07T12:00:00"), with nothing saying the
value was UTC. JavaScript's new Date() parses that as BROWSER-LOCAL, so every
timestamp shifted by the viewer's offset before any timezone formatting ran.
Every datetime this plugin stores is naive UTC, so the wire format now carries
a trailing Z.

The history view then formatted with toLocaleString(), i.e. the viewer's zone,
ignoring the site_timezone setting entirely. It now loads that setting and
formats through the shared formatInZone helper, matching NotificationsList.

The panel list label is built server-side with strftime, so a client cannot
correct it afterwards. It now converts to the site zone using the same Setting
lookup the notifications plugin uses - without that it showed UTC, four hours
out at West Jefferson.

Tests cover the wire format and that 16:30Z renders as 12:30 in
America/New_York.
 2026-08-08 14:08:33 -04:00
cproudlock
c93ec7a949 backups: one tabbed DNC card, machine-numbered downloads, themed history
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
DNC Info becomes a single tabbed card - General, eFocas, Serial, NTSHR and
MARK - instead of a flat wall of every value. On 3204 that is 11 rows visible
rather than 22, and on 0600 eleven rather than 27, which also stops the card
unbalancing the detail page's two-column layout.

Everything DNC now lives on that one card, so the Part Marker panel is gone:
its settings are the MARK tab. The partmarker KIND is untouched and still
stores, dedupes and serves revisions - they are listed on the backup history
page - it simply contributes no card of its own, which on 145 of 147 machines
would have been an empty box.

Downloads are named for the machine: 3204.reg, and 3204-wow6432node.reg for
the dialect that imports outside NTLARS. The view had been rebuilding the name
from sourcefilename and producing 3204.reg-wow6432node.reg, so the revision now
carries assetnumber and both ends agree. That needed a viewonly relationship to
Asset - no backref, so the core asset side gains no dependency on this plugin.

The history page was hardcoded to light colours (#e0e0e0, #f4f9ff, #666) and
rendered as a white table on a dark page. It now uses the palette variables
throughout, per frontend/CLAUDE.md. The current-revision tint is a color-mix
against --primary so it reads in both themes rather than a baked light blue
that disappears on dark, and the diff columns are headed as well as red/green,
since colour alone does not survive a colourblind reader.
 2026-08-07 15:22:45 -04:00
cproudlock
a33470a34b asset panels: add a tabs render mode, align the keyvalue grid 
A panel with many fields was unusable. The detail page is a two-column
multicol (.content-column is display:contents, so the left/right wrappers are
dissolved and cards flow into one balanced flow), and a card cannot be split
across a column break. One 30-row panel therefore became a single unbreakable
block that dragged its column far past the other.

The tabs mode renders sections one at a time, so such a panel stays the height
of its largest section. Payload is {sections: [{label, fields}]}; sections that
end up with no fields are dropped rather than left as empty tabs. The active
tab is marked by an underline AND weight, not colour alone.

keyvalue rows now sit in two aligned columns instead of space-between: a ragged
right edge is hard to scan, and one long value (a UNC path) otherwise pushed
its label away from every other row.

Four specs cover the new mode, including that it drives panel visibility.
 2026-08-07 15:22:31 -04:00
cproudlock
efe2f7e6ca backups: show a kind's panels only when that asset has data
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Every machine was getting a Part Marker Configuration panel, and only two of
the 147 known machines are part markers. A kind applies to an asset TYPE, but
whether a given asset ever carries that kind of backup is a property of the
individual machine, so type alone cannot decide what to show.

The generic renderer already handled this: a list panel is visible when it has
rows OR declares empty text. Declaring emptytext on both kinds defeated it and
forced them to render everywhere. emptytext now defaults to None on the base
class, neither bundled kind sets one, and the panel builder OMITS the key
rather than emitting null - a present-but-null 'empty' would still have kept
the panel on screen.

The DNC Info card carried empty text that could never be displayed, since
keyvalue visibility is decided purely on field count. Removed rather than left
to mislead.

A lathe now shows DNC Info and NTLARS history; 0600 and 0614 additionally show
Part Marker once something collects for them; a machine with no NTLARS data
shows no backup panels at all. The emptytext mechanism stays available for a
kind that genuinely wants to say "expected here, nothing yet".
 2026-08-07 14:53:37 -04:00
cproudlock
565611e3d0 add backups to the universal site profile, and guard the list
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
site-profile-universal.json is what the released Windows installer is built
from, so a bundled plugin missing from it is invisible to the install wizard.
Worse, `flask plugin prune-schema` drops the tables owned by plugins the site
did not install (ADR-014), so backuprevisions would have been dropped at
provisioning on every new site - a table that shipped in the build, removed
because the profile never named it.

The same omission explains why `flask plugin upgrade-all` skipped backups:
upgrade_all_plugins iterates the REGISTRY, not the plugins directory, on
purpose - a plugin folder merely sitting on disk unadopted must not have its
DDL run as a side effect of a deploy. instance/ is gitignored, so any machine
that never ran `flask plugin install backups` has it on disk but unadopted.

Nothing in the suite caught the stale profile, so this adds two guards: every
bundled plugin carrying a manifest must appear in the universal profile, and
the profile must not name a plugin that does not exist. Verified the first one
fails with the profile as it was.
 2026-08-07 14:44:33 -04:00
cproudlock
34f08d5306 backups: fix the history view's back link
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
The view linked to /assets/<assetid>, which is not a route. Each asset type
has its own detail route keyed by its own id (/machines/<machineid>), and
there is no generic asset detail page to return to, so the link 404'd.

Uses router.back() instead: this page is only ever reached from an asset
panel, so history is the correct destination regardless of the asset type,
and it needs no per-type route table. Falls back to the machines list when
opened directly from a pasted URL.

The /api/assets/<id> call that fetches the heading name is unaffected - that
endpoint does exist; it was only the frontend route that did not.
 2026-08-07 14:40:27 -04:00
cproudlock
bb4169cd8c backups: keep the seed script's docstring site-neutral 
The import script is bundled in a multi-site product, so its usage examples
should not name one site's share layout or a dev box path.
 2026-08-07 14:30:14 -04:00
cproudlock
a10e695d7c shopfloor dashboard: show upcoming cards, state chips, mixed-row headings
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 6s
Details 
Scheduled notifications with a special display style rendered nowhere at all.
styledGroups only read notifications.current, so anything upcoming fell through
to the standard Upcoming list, which drops banner/carousel/grid styles. A
scheduled banner was therefore invisible until the moment it went live. Upcoming
items now join the styled groups, tagged so they can be told apart, and each row
sorts live cards ahead of not-yet-started ones.

State chips name what a card is doing: UPCOMING with its start time, or
RESOLVED for the tail a type's grace window buys it. Pending cards also dim to
0.72 so a glance never mistakes one for live. The dim rule is scoped to
.recognition-card.active.pending on purpose: inactive carousel cards sit at
opacity 0, and an unscoped .pending outranks that, which stacks the whole
carousel back into view at once.

Rows keyed by board category can hold more than one type, and the heading then
speaks for none of them. Such rows keep a neutral heading instead of wearing
the first type's colour, and each card carries its own type chip. The chip is
named rather than only coloured, since colour alone does not survive a
colourblind reader at board distance.

Carousel rows gain one pip per card, so a reader can see how many are in the
rotation and how long until theirs returns.
 2026-08-07 14:28:01 -04:00
cproudlock
fca775c737 backups plugin: per-asset config backups with revision history
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Adds a kind-pluggable backups plugin. Configuration captured from a PC is
filed against the MACHINE it controls, with a revision history and download
back to the native format.

NTLARS/DNC is the first kind. Settings live in the controlling PC's registry
but describe the machine, so revisions attach to the machine's asset and carry
no foreign key to the PC: history survives a PC being replaced or deleted, and
sourcehostname records the handover.

Storage splits by kind. Parseable kinds store a dialect-neutral JSON
projection in ShopDB and re-render on download; opaque vendor formats (part
marker and similar) keep their bytes on the SFLD share with ShopDB holding
metadata and the UNC pointer.

Two .reg dialects exist in the wild: NTLARS's own Save... export omits the
WOW6432Node path segment, scripted exports include it. Parsing strips whichever
root matched, so a stored revision commits to neither and download offers both
(NTLARS Load... by default, WOW6432Node for direct reg import). Getting this
backwards is silent, so the dedup hash deliberately excludes sourcedialect and
both dialects of one config dedup to a single revision.

Dedup is load-bearing: the collector runs every GE-Enforce cycle across the
fleet, so a revision is inserted only when the content hash differs from that
asset's latest for that kind.

A freshly imaged PC opens NTLARS with a blank General tab. Recording that would
make an empty config the newest revision exactly when someone needs the last
good one, so a blank MachineNo is rejected rather than accepted as a change.
Two of the 320 known-good backups on the share already have that shape.

DNC Info card summarises the latest revision on the machine page: General
(Cnc, NcIF, HostType), eFocas, Serial, NTSHR when populated (only 18 of 147
machines), and MARK when the machine is a marker. MARK is gated on Cnc=MARKER
or the ShopDB machine type, not on the MARK key having content: MARK carries
serial defaults on 145 of 147 machines and DncPatterns reads YES on 103
including ordinary lathes, so neither identifies a marker.

The info card is owned by the kind (BackupKind.infopanel/buildinfo) and served
by a generic endpoint, so the expected successor to DNC ships its own card by
adding a class rather than changing the plugin or the panel wiring.

Also: schedule and retention settings with a prune that never drops the newest
or the oldest revision, and scripts/import_ntlars_backups.py to seed history
from the existing per-machine .reg files (144 of 147 resolve to assets).

Codec verified against all 320 real backups: round-trips clean through both
dialects. Bay-side generation verified on Windows against reg.exe export.
 2026-08-07 14:24:42 -04:00
cproudlock
d180a693ee Paint each board row in its own notification type's colour
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 6s
Details 
The carousel gold and the grid blue were hardcoded in the dashboard's CSS,
which was fine while carousel meant Recognition and grid meant Recertification.
Now that any type can take either style, a type arrived wearing somebody else's
colour: a green Kudos row came out gold, and every grid row came out
Recertification blue regardless of what was picked in the type settings.

Row headings now take the type's colour, and cards and tiles take theirs
through a --accent property, so a shared category row shows a red incident tile
beside a purple change tile. Heading text colour is computed from the
background's luma rather than listed per colour, since a site picks its own
hexes - gold needs dark text, navy needs white.

The card backgrounds were gold- and blue-tinted gradients; they are neutral
dark now so the accent is what carries the colour. The old values remain as CSS
fallbacks for a card that somehow arrives with no type.
 2026-08-07 11:05:47 -04:00
cproudlock
245f94d344 Stop two dialogs going see-through in dark mode
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
--bg-card is deliberately translucent in dark mode (rgba(0,0,61,0.4)) so cards
glass over the page; --bg-card-solid exists for the things that must not. Two
hand-rolled modal panels used the former, leaving the notification-type and
access-protocol editors transparent over the overlay with the table legible
through them. The shared .modal in style.css already got this right.

Also writes down the page-vs-modal rule the codebase already follows, since
nothing stated it: a record with a detail page gets a routed form page, a lookup
row that only exists inside its list gets a modal over that list. Plus the modal
rules from the overlay-close fix - data entry never closes on a stray click,
confirmations may, and panels are painted solid.
 2026-08-07 10:30:45 -04:00
cproudlock
ed9c91c47d Give a network device somewhere to put its IP 
The network device form had no IP field, so the one thing people look up a
switch for could not be entered. The API had accepted an ipaddress since the
legacy import work - create, update and every read already carried it - but
nothing in the UI ever sent one, which also left the device off the "Devices on
this network" list, since that matches on the IP a device does not have.

The field sits next to Hostname and round-trips through the Communication row
the platform keeps IPs in, the same way the PC and printer forms do.

Clearing it now clears the stored address. The upsert helper returned early on a
blank value despite a docstring promising it cleared - harmless while no form
could submit one, wrong the moment this field existed.
 2026-08-07 10:30:38 -04:00
cproudlock
76c184fe91 Give every notification type its own row on the board 
The shopfloor board grouped cards by display style alone, so every type set to
grid landed inside the Recertification row and every carousel type inside
Recognition's - under a heading naming somebody else's type. Setting Awareness
to grid put awareness messages under "Recertification Required".

Each type now gets a row of its own, titled by its own name, and rotation state
is per row: two carousel rows advance on their own indexes instead of sharing
one counter, and two grid rows page independently.

For the other direction there is notificationtypes.boardcategory. Types sharing
a category share one row under the category name, so Change, Awareness and
Incident can sit together while Recognition and Recertification keep their own.
Blank - the default - means a row of its own. The category is part of the
grouping key along with the display style, since a category cannot merge a
banner with a row of tiles.

A card that names no employee now renders its message as the tile or card,
rather than a placeholder face above a blank name, which is what a grid type
like Awareness looked like before.

The layout fingerprint that makes open kiosks reload now covers the category
and the grace window, so a re-grouped board reaches screens that are already up.
 2026-08-07 10:30:28 -04:00
cproudlock
a52e192501 Revert "Show recertification four at a time, two by two" 
This reverts commit af60a77399.
 2026-08-07 09:44:26 -04:00
cproudlock
af60a77399 Show recertification four at a time, two by two 
Eight tiles across left each one too narrow to read at floor distance. The row
now shows four per page in a 2x2 block, cycling on the same seven-second timer,
so a name is legible from where people actually stand.

The tiles lay the photo beside the name rather than above it. Two rows of
stacked tiles would roughly double the section's height, and the board is
scaled to fit one screen, so height spent here shrinks everything else. The
photo grows to 120px and the name to 26px in the space that buys.
 2026-08-07 09:22:41 -04:00
cproudlock
fef5e28952 Let an end time mean the card leaves the board 
The shopfloor feed kept every ended notification up for a hardcoded 30 minutes,
flagged resolved. A card with an 8:00 end time was still on the board at 8:29,
which reads as an expiry that did not work - and in the carousel, grid and
banner sections it read that way with no visual sign at all, since only the
standard cards render the resolved state.

The tail is now notificationtypes.gracewindowminutes, set per type on the
Notification Types page and defaulting to 0, so an end time means what it says.
A type whose cards are worth acknowledging after they clear - an incident, say
- opts into a tail, and only that type's cards get one.

The feed widens its query to the largest configured tail and then holds each
row to its own type's window. That keeps one portable query rather than a
per-type interval expression in SQL, and with every type at 0 it collapses to
"still showing".

Also fixes resolved serializing as null rather than false for a card with no
end time, which the and-chain produced.
 2026-08-07 09:22:34 -04:00
cproudlock
536a8f0825 Centre the parts kiosk on the screen instead of hugging the top 
The kiosk runs full-screen on a wall-mounted display, where the interface sat
against the top edge with the rest of the screen empty below it. It now
centres as one block, vertically and horizontally.

The title was the other half of it: the header was space-between, which reads
as centred only on the steps that show the Start over button. On the first
step, with no button to balance it, the title sat alone at the left edge of
the column. The title is centred and Start over is taken out of the flow so it
keeps its corner without shifting the title on the steps that have it.

Centring uses `safe center`, with plain `center` as the fallback line. On a
screen too short for the content, plain `center` overflows in both directions
and the header ends up above the scroll origin, unreachable. `safe` falls back
to top-aligned there and the page scrolls normally.
 2026-08-07 08:38:55 -04:00
cproudlock
593dd46525 Show the kiosk label prefix, and let a plugin declare the settings it owns 
Three defects, all found on printedparts_label_prefix, all one root cause:
nothing in the framework knew that setting existed.

The parts kiosk runs logged out. An unauthenticated read of a setting is
limited to an allowlist, the key was not on it, so the kiosk got a 404 and
fell back to no prefix. An admin previewing the same page while logged in saw
the prefix, which is why it looked like it worked.

The same setting also looked like it would not save. The row did not exist on
a site that installed the plugin before the setting was added, so the first
save created it - under the placeholder category the settings API uses for
keys it does not recognise, where the plugin's settings page, which lists by
category, could no longer see it. The value was in the database the whole
time.

And the row was missing in the first place because seeding ran from
on_install / on_enable, which fire only on a state transition. Neither runs
again on an upgrade, so a setting added in a later plugin version never
reached a site that installed an earlier one. The comment claiming enable ran
every upgrade cycle was simply wrong.

A plugin now declares the settings it owns in get_settings_defaults(): key,
default, type, category, description, and whether a logged-out page may read
it. The framework seeds declared keys at install, at enable, and on every
flask plugin upgrade-all; files a first-time write under the declared
category; re-homes any row left in the placeholder category, value untouched;
and answers an anonymous read for keys marked public. Core carries no list of
any plugin's keys.

Contract 0.16.0 (additive optional hook). printedparts and printers move to
the hook and floor their core_version at 0.16.0. The dev database had two rows
in the misfiled state (printedparts_alert_email, employee_db_host); the first
repairs itself on the next upgrade pass.
 2026-08-06 18:17:49 -04:00
cproudlock
1aeb3bd1d4 Reorder slides by dragging 
Reordering meant clicking the up arrow repeatedly - moving a slide from the
bottom of a long playlist to the top was a dozen clicks and a page of
re-rendering. Rows are now draggable, with a grip so it looks it.

Applies to BOTH surfaces: the manager already switches between Lobby Display and
Shopfloor Screensaver, so one change covers the lobby TV and the EventSaver
playlist.

The drop target is shown as a line on the row being dropped against rather than
by shuffling rows under the cursor, which reads as the list fighting the drag.
The hover preview is dismissed when a drag starts, or it would sit over the list
for the whole gesture.

Drag and the up/down buttons now share reorderTo(), so both persist through the
same call and both recover the same way: a failed save reloads from the server
rather than leaving an order on screen that looks saved and is not.

dataTransfer.setData is set because Firefox starts no drag at all without it.
 2026-08-06 10:11:09 -04:00
cproudlock
8fd2b3bcd2 Show a full-size slide on hover in the slide manager 
The thumbnails are 120x68 and cropped with object-fit: cover, so any text on a
slide is unreadable and the edges are cut off. Picking the right slide to
reorder or delete meant opening images by hand to tell them apart.

Hovering a thumbnail now shows the whole slide, bounded by the viewport rather
than the image so a 3300x2550 upload does not fill the screen, with the filename
underneath.

Fixed position rather than inside the row: the list scrolls and a
relatively-positioned parent would clip it. pointer-events: none so the preview
can never sit between the cursor and the move or delete buttons.
 2026-08-06 10:04:21 -04:00
cproudlock
a18335b61c Step through slides with the arrow keys 
Somebody standing at a display who wants to hold a slide, or go back to one that
has just passed, had to wait for the whole rotation to come round again.

Left and right arrows step back and forward; PageUp and PageDown do the same, so
a presenter remote works without being told about it. The listener is on window
rather than an element because a display has nothing focused.

Stepping restarts the rotation timer instead of leaving it running. Advancing by
hand and then having it move again a second later, because the existing timer
was nearly up, reads as the display ignoring the keypress.

Ignored entirely when there is less than one slide to move to.
 2026-08-06 07:27:59 -04:00
cproudlock
8e6f6ad58b Floor the calendar height so a short window cannot squash it 
The viewport-relative height can resolve smaller than the grid needs on a short
window - a laptop with a docked browser, or a display in portrait - which would
have reintroduced the squashing it was meant to fix. Three floors now: the
container, the calendar root, and each day cell.

The day cell is the one that matters: six week rows cannot render shorter than
about 660px however short the window, so the grid stays readable rather than
collapsing back to strips.
 2026-08-05 15:38:38 -04:00
cproudlock
658e5c2224 Give the calendar room, and put new notifications on the shopfloor board 
The calendar was set to height 'auto', which sizes each week row to its own
content, so a month of mostly empty days collapsed into thin strips. It now
takes a viewport-relative height and expandRows shares that evenly across the
weeks, with a floor under each day cell so a short window squeezes the grid back
down rather than the rows vanishing.

New notifications now have "Show on Shopfloor Dashboard" ticked. The board is
where these are meant to be read, and starting unticked meant most were written
and then never appeared on it.

Only the default for a NEW notification. Editing an existing one still loads its
stored value, so nothing that was deliberately turned off gets flipped back on,
and the column default is left alone so an API or import caller that omits the
field keeps the behaviour it has today.
 2026-08-05 15:37:46 -04:00
cproudlock
959db2922b Release 0.8.1
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
The installer on the share was stamped 0.8.0 and contained sixteen commits that
were not in the v0.8.0 tag. A build that misreports its own version is exactly
what the version check in docs/RELEASING-WINDOWS.md exists to prevent, and it
would have left two sites unable to say what they were running.

Everything in 0.8.1 landed after v0.8.0 was tagged this morning, driven by two
sites entering real data for the first time: the blank-code 500, the two-slide
display that never rotated, modals discarding a part-filled form, filters
returning an empty page, model photos that could not be saved, and the shared
equipment catalog that lets a new site start with vendors, models and printer
supply part numbers already present.

CHANGELOG gains a 0.8.1 section, and the OpenAPI document follows __version__
rather than being restated.
 2026-08-05 14:50:06 -04:00
cproudlock
512d5fafac Stop an empty box appearing after Notes on the network device form
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 6s
Details 
CustomFieldsInputs renders nothing when a site has defined no custom fields,
which is the normal state. The network form was the only one wrapping it in a
fieldset, and a fieldset draws its border whether or not anything is inside, so
an empty bordered box sat below Notes on every new network device.

Machines, PCs, printers and measuring tools all place the component bare. This
now matches them.
 2026-08-05 14:25:10 -04:00
cproudlock
d8fe0a48b2 Stop a stray click outside a modal discarding what was typed
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Operators reported losing a part-filled form by clicking slightly outside it.
Every data-entry modal closed on a backdrop click with no warning and no way
back - the worst possible response to a misplaced click, and it happens most to
someone adding their first records at a new site.

Close-on-overlay is removed from 35 modals across 30 files: anything containing
an input, textarea, select or v-model. They still close by Cancel or the X.

Confirmation dialogs keep it, because a delete prompt holds nothing to lose and
dismissing one by clicking away is the behaviour people expect. VendorsList
shows the distinction - its edit form no longer closes that way, its delete
confirmation still does.

The shared Modal component now defaults closeOnOverlay to FALSE. Every current
caller holds a form, a checkout, a stock adjustment or a map position being
picked, and not one passed the prop, so all of them had the same fault. A modal
that genuinely wants dismissing that way opts in explicitly.

Also regroups the operator console menu, which had grown to numbers 1-9 plus
three letters bolted on with no order to them. Actions are now grouped by what
they touch, keyed by their first letter, and the old numbers still work so
nobody who has used it for months is stopped by a rearrangement.

The menu also warns when the server is not fully provisioned and names the key
that fixes it, instead of reporting it as ordinary status lines that read as
normal unless you already knew what to look for. That check is cached for the
session because it shells out to flask twice and the answer does not change
while somebody reads the screen.
 2026-08-05 13:42:20 -04:00
cproudlock
23b1dfff41 Load the equipment catalog from the console
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
The installer offers the catalog as a tick-box, which covers new sites and
nobody else. A site that installed before it existed, or unticked the box and
later changed its mind, had to be talked through an RDP session and a flask
command - which is exactly the sort of thing the console exists to avoid.

`shopdb-admin.ps1 catalog`, and C on the menu. It runs the dry run FIRST and
prints what would be added, then asks before writing: somebody running this on a
site that has been live for years deserves to see what it would touch before it
touches anything. Answering anything but yes leaves it alone.

An older build without the seed-catalog command is reported as such and told to
update, rather than the failure being read as an empty catalog.

Repair is now on the menu too. It was reachable only by typing the verb, which
is little use to the operator most likely to need it.
 2026-08-05 13:30:33 -04:00
cproudlock
367bc56a6d Ship the equipment catalog so a new site does not start empty
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
`flask seed reference-data` wrote a dozen generic model types and no vendors or
models at all, so adopting this platform began by retyping a catalog another
site had already spent a year building. That is the largest single obstacle to
standing a new facility up.

scripts/export_catalog.py dumps the catalog from a live instance to
shopdb/data/catalog.json, and `flask seed catalog` loads it. What travels:

  vendors 53, models 128, modelsupplies 146, modeltypes 35, machinetypes 21,
  computertypes 10, printertypes 9, networkdevicetypes 5, locationtypes 11,
  operatingsystems 14, measuringtooltypes 8, notificationtypes 3,
  accessprotocols 3

The 146 printer supplies are the most useful part after the models themselves:
every toner, drum and maintenance kit with its part number, colour, capacity
tier and page yield, already matched to the right model, instead of somebody
reading them off spent cartridges.

IDEMPOTENT and ADDITIVE. Records match on a natural key - a vendor by name, a
model by vendor plus model number, a supply by model plus part number - so a
second run adds nothing, and it never updates or deletes: a site that corrected
a description or pointed a model at its own photo keeps its version.

Catalog only. No assets, locations, employees, business units or anything with a
serial number: nobody wants one plant's machines appearing at another. Vendor
contact details are excluded too, since a rep's name and number belong to
whoever holds that relationship. supportteams, printerdrivers and customfields
are site-specific and deliberately absent.

Models and supplies reference their vendor by NAME rather than id, because ids
differ between databases and an id-keyed catalog would silently attach part
numbers to the wrong printer.

The installer offers it as a tick-box on a new "Starter data" page, defaulting
to on, passing -SeedCatalog to stage 3. Offered rather than assumed: a site that
machines nothing does not want 21 machine types cluttering its dropdowns.

Verified by loading into an empty database and running twice: every group
populated on the first pass, "Catalog already present, nothing to add" on the
second.
 2026-08-05 13:16:57 -04:00
cproudlock
53c1f6476c Pick a network device's map position, and stop hardcoding one site's label prefix 
Two things a second site ran into.

The network device form asked for the map position as two raw numbers, so
placing a device meant reading coordinates off another screen and typing them
in. Machines, PCs and printers have had a "Set Location on Map" picker all
along, and the network API already accepted mapx and mapy - only the form was
missing. Same picker, same modal.

The 3D parts kiosk hardcoded 'WJ' as the prefix shown before the number box,
with a comment inviting whoever needed something else to edit the source. That
is West Jefferson's gage-lab tag format and nobody else's, so another site's
operators were told to expect letters that are not on their labels.

It is now printedparts_label_prefix, set in Settings, defaulting to EMPTY - a
site that has not set one sees no prefix rather than inheriting another site's
convention. West Jefferson sets it to WJ once. The kiosk hides the prefix
entirely when unset and falls back to no prefix if the setting cannot be read,
because a cosmetic hint must never stop a kiosk working.

Not to be confused with printedparts_code_prefix, which mints item codes like
3DP0042 and was already configurable. That is the code we generate; this is the
tag already printed on the label.
 2026-08-05 13:16:41 -04:00
cproudlock
b44108f70c Run one slideshow timer, not two, and honour each slide's own duration 
A site added two slides to the lobby display and it never changed between them.

onMounted awaited fetchSlides, which starts the slideshow itself once it has
more than one slide, and then started it AGAIN unconditionally. Two timer chains
ran, and the second assignment to slideTimer lost the handle to the first, so
nothing could ever cancel it. Both fired about ten seconds later, milliseconds
apart, and each advanced one slide.

With exactly two slides that is 0 -> 1 -> 0 every cycle: the display looked
frozen. With three or more it advanced by two and merely skipped one, which is
why this survived so long - and why adding a third slide would have appeared to
"fix" it.

onMounted no longer starts it; fetchSlides owns that. scheduleNextSlide also
cancels any pending timer before setting a new one, so a future double-call
replaces the chain rather than leaking an untracked one.

While here: the feed has always sent a per-slide duration and the display
ignored it, hardcoding ten seconds, so a slide set to hold for a minute changed
after ten. It now uses the slide's own value, and the progress bar animates over
that same duration instead of finishing early and sitting full.
 2026-08-05 13:16:41 -04:00
cproudlock
ead5bd8f58 Give the console a repair verb, and something real to check 
A server whose migrations or seeds never finished does not fail politely. Most
pages answer 500 and settings endpoints answer 404 for keys that were never
created, which reads as a broken application rather than an unfinished install.
One site spent a morning being debugged that way.

`shopdb-admin.ps1 repair` runs what stage 3 of the installer runs: db upgrade,
plugin upgrade-all, and the three seeds. Every step is idempotent, so running it
on a healthy server changes nothing, and each step runs independently so one
failure does not silently skip the rest.

`check` now says so before anyone has to infer it:

    THIS SERVER IS NOT FULLY PROVISIONED
      - seed data is missing (permissions, settings or reference data)
    Most pages will answer 500 until this is fixed. Run:
      shopdb-admin.ps1 repair

That needs a real test to sit on, so `flask db-utils seed-state` reports each
seed group and exits non-zero when any is missing. Verified by emptying the
settings table inside a transaction: MISSING, exit 1, rollback clean. Without it
the console check would have looked reassuring while testing nothing - an older
build with no such command reports UNKNOWN rather than healthy, for the same
reason.
 2026-08-05 13:16:24 -04:00
cproudlock
705dd771bd Store a blank optional unique field as NULL, and answer a duplicate with 409 
A site reported "internal server error" adding a second business unit. It was
reproducible: create one with a blank code, create another with a blank code,
500.

A column that is unique and nullable accepts any number of NULLs - that is what
makes "optional but unique" work - and exactly ONE empty string. The form sent
'', so the first blank code saved and every one after it collided with it. The
field showed no asterisk because it genuinely is optional; the database just
behaved as though it were not.

This is not specific to business units. A dozen columns across core and the
plugins are unique and nullable - asset numbers, hostnames, item codes, subnet
names, gage-lab tags - and each was one blank form away from the same 500.
Fixing them an endpoint at a time would have left the next to be found by a
user, so a before_flush listener normalises blank to NULL on any unique nullable
text column. Listening on Session rather than on individual mappers covers
plugin models imported later, and avoids mapper-event semantics that differ
between SQLAlchemy versions.

A genuine duplicate is now a 409 with a readable message rather than a bare 500
with a traceback in the log: reusing a code that is taken is the caller's
mistake, not a server fault.

Verified against the development database: three business units with blank codes
all save, the blank stores as NULL, and a real duplicate code returns 409.
 2026-08-05 13:16:23 -04:00
cproudlock
85ff25462e Reset to page one when a filter changes, and let the catalog carry a real type
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Two unrelated things found while looking at blank printer types.

Selecting a filter while past page one returned an empty list. The filter asked
the server for page 5 of a result set that now had one page, and the screen said
nothing matched. useListQuery already resets the page - setSearch and setExtra
both do - but the filter dropdowns bypassed it and called the loader directly.
Nine list pages now route through applyFilter, which calls setPage(1) when it
needs to and loads directly when already on page one, so the composable's URL
watcher does not also fire and fetch twice.

scripts/retype_models.py addresses why printer types cannot be derived. The
catalog types every printer model "Printer": true, and useless, since it does not
say whether the product is a laser, a plotter or a label printer. That answer is
a property of the model - every VersaLink C405 is a laser MFP - but nothing
recorded it, so nothing could derive it. Recording it on the MODEL means the
existing backfill fills every printer by exact name match, and a printer added
later inherits the right type the moment its model is chosen.

It exports the models needing a decision to CSV with a type suggested from the
model number, a person corrects the column, and applying it is a dry run unless
given --commit. A suggested type is refused unless it already exists in that
asset class's own vocabulary, which is what keeps the later name match working.

The suggestion order matters and got this wrong first time: a generic plotter
pattern matched "Zebra ZT411" and filed a label printer as a plotter. Brands now
come before generic patterns, and the review step exists precisely because a
confident wrong guess would type every asset using that model.

Verified on the development database: 24 printer models need a decision, 22 got
a sensible suggestion, applying them let all 42 printers match a printertype by
name, and the transaction rolled back cleanly.
 2026-08-05 11:26:23 -04:00
cproudlock
e22322dcc9 Show model type in the machines list
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
The machine's own type is blank for the 134 machines that came from the classic
ASP database on machinetypeid=1, a LocationOnly placeholder the import refuses
to carry across as a real subtype. The catalog model knows what those machines
are, and its type is populated, so the column reads modeltypename under a
heading that says so.

Where both values exist they are identical - all 262 machines in the development
database match exactly - so nothing is lost by showing the one that is reliably
filled in.

This does not fix the underlying gap. A null machinetypeid also excludes a
machine from the map's subtype filter and drops its marker to the default
colour, and no column heading affects that. Only populating machinetypeid does,
which is what the backfill script is for.
 2026-08-05 10:45:35 -04:00
cproudlock
cb18d170cf Say whose type it is
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Two fields on the same page were both labelled "Type": the asset's own, and the
catalog model's. Only one of them was vague. "Model type" already says exactly
what it is; the bare "Type" did not say whose.

So the unqualified one is the one that changes. No new vocabulary, and "Model
type" reads correctly against it:

  Type  ->  Machine Type      (machines)
  Type  ->  PC Type           (computers)
  Type  ->  Printer Type      (printers)
  Type  ->  Device Type       (network devices)

Left alone everywhere the word is not ambiguous - measuring tools, subnets,
VLANs, notifications, supply types and the manifest editor have no model type on
screen to be confused with.

This is a labelling change only. It does not address the blank type column on
machines imported from the classic ASP database, which is a data gap the
backfill script fills; renaming a column heading was never going to put values
in it.
 2026-08-05 10:33:47 -04:00
cproudlock
24266146d8 Show the model's type only when it differs from the asset's own
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Adding a "Model type" row next to "Type" put two rows in the Hardware section
that read identically. They come from different tables - modeltypes is the
catalog-wide list spanning every kind of asset, machinetypes is machine-only -
but the names line up in practice: all 262 machines in the development database
match exactly, which is the same fact that makes the type backfill safe.

So the row now appears only when the two disagree, which is the case worth
seeing: a model catalogued as one thing fitted to an asset recorded as another.
When they agree it says nothing and is hidden. Applied to machines, PCs,
printers and network devices, each compared against its own type table.
 2026-08-05 10:16:21 -04:00
cproudlock
58b460fe3d Backfill an asset's type from its model, by exact name only 
Correcting an earlier judgement. I said the model's type could not be used to
fill an asset's type, because modeltypes is the catalog-wide list covering every
kind of asset - it holds "Access Point", "Camera" and "Desktop PC" - and only
about two thirds of its names exist as machine types.

That is true across the whole catalog and misleading in practice. Restricted to
the models an asset class actually uses, the picture is different: all 262
machines in the development database map exactly, because the non-machine
entries are never used by machines. The blanks on the machines list are rows
whose type the database could already have supplied.

So the backfill now fills the type as well, under a rule that cannot mistype
anything: exact name match or nothing. A model type with no identically named
entry in the asset's own type table is reported with a count and left untouched,
so somebody can decide rather than have a guess written into their data. The
same shape covers computers, printers and network devices, each against its own
type table.

Verified against the development database by nulling one machine's type inside a
transaction: it was detected as fillable, the proposal read "LocationOnly" ->
"LocationOnly", the update restored exactly the original id, and the rollback
left the row unchanged.

Still a dry run unless given --commit, and a table missing the model column is
skipped, so it runs against a server whose network migration is not yet applied.
 2026-08-05 10:11:50 -04:00
cproudlock
3f320fcc8b Derive an asset's vendor from its catalog model, and show the model's own type
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
An asset that carries a model but no vendor was showing a blank the database
could already answer: the model records its vendor, and both sides reference the
same vendors table. Machines, PCs, printers and network devices now fall back to
it.

The fallback is FLAGGED, not merged silently. to_dict sets vendorfrommodel and
the detail pages render "(from model)" beside the value, because the record
itself is still empty: the edit form shows an empty vendor box, and a page
implying the vendor is stored would be lying about where it came from.

The model's type is exposed under its own name, modeltypename, and shown as a
separate "Model type" row. It is deliberately NOT used to fill in the asset's
own type. modeltypes is the catalog-wide list covering every kind of asset - it
holds "Access Point", "Camera" and "Desktop PC" alongside the machine entries -
so it is a different taxonomy from machinetypes. Only about two thirds of the
names overlap, and mapping one onto the other would mistype the remainder, with
the failure mode being a machine labelled "Desktop PC".

scripts/backfill_vendor_from_model.py writes the derived vendor down for real,
since the display fallback leaves reports that read vendorid still seeing
nothing. It is a dry run unless given --commit, fills only rows where the
asset's vendor is NULL and the model names one, and never overwrites a vendor
somebody chose. It skips a table lacking either column, so it runs against a
server whose network migration has not been applied yet.

Verified against the development database by nulling one machine's vendor inside
a transaction: it was detected as fillable, restored to exactly its original
value, and the rollback left the row untouched.

FLASK_ENV is not forced by the script. The app already reads it from .env, and
overriding it demanded a SECRET_KEY the environment had no reason to supply.
 2026-08-05 09:59:39 -04:00
cproudlock
f8c4246483 Fix model photo upload, and give network devices the model link the page assumed
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 6s
Details 
Three faults around vendor-model photos, found while looking at why an uploaded
image did not appear.

Saving a model was blocked after uploading a photo. The Image URL field was
type="url", and an upload sets it to an application path such as
/api/models/image/model-120.png. Native url validation demands an absolute URL
with a scheme, so the browser refused to submit the form with "Please enter a
URL" for a value the page had just written itself. The field is now type="text",
which is what it always needed to be: it holds either a full web address or a
path on this server. documentationurl stays type="url".

The upload button did not appear when adding a model, only when editing one.
That was deliberate - the photo is stored as model-<id>.<ext>, so it cannot be
sent before the record has an id - but it reads as a missing feature, and the
hint explaining it was easy to miss. A photo chosen while creating is now held
and uploaded as soon as the model is saved, and it is dropped if the dialog is
cancelled, so it cannot land on the next model created in the same session.

Network devices could never show a photo. NetworkDeviceDetail.vue binds its hero
image to networkdevice.imageurl, but networkdevices carried only vendorid, with
no link to a catalog model, so nothing could populate it - a feature that looked
present and could not work. Machines, PCs and printers have carried
modelnumberid since July. This adds the same column and relationship, the
to_dict branch that exposes modelname and imageurl, the field on the API, and a
Model selector on the form so the link can actually be set.

The migration is guarded the same way employees0002photo is: on a fresh database
the tables come from the SQLAlchemy models, which already declare the column, so
an unconditional add fails with "duplicate column name". The foreign key is
created only on databases that can add one by ALTER; routing it through
batch_alter_table made Alembic's column sort raise "Circular dependency
detected" on the fresh-database test.

Deploying this needs `flask db upgrade` and `flask plugin upgrade-all` on the
server, not just a file copy.
 2026-08-05 09:08:40 -04:00
cproudlock
92a90fcec6 Document publishing the installer as a release asset
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 6s
Details 
The exe is attached to a release, never committed: most forges reject files over
100 MB inside a repository while allowing release assets far larger, and a
committed binary would sit in every future clone forever.

Also records that tags must be pushed explicitly. A plain push of the branch
does not carry them, so a release had nothing to hang off.
 2026-08-05 07:44:57 -04:00
cproudlock
89e880afc3 Release 0.8.0
Some checks failed
CI / backend (push) Failing after 6s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 6s
Details 
The Windows installer has never shipped under a version: v0.7.0 was tagged
before any of it existed, so every build handed out so far stamped a server with
0.7.0. Two servers running different builds were indistinguishable, and the
installer logged each upgrade as "same version already installed" rather than
recording what changed. This cuts the release that fixes that.

0.8.0 rather than a patch: the air-gapped installer is a new capability, and
pre-1.0 semantic versioning puts that in the minor slot (ADR-007).

CHANGELOG gains a 0.8.0 section covering the twelve defects a real Windows
Server 2019 install surfaced, the move from inferring "is this a re-run of my
install?" to recording it, and the operator documentation.

deploy/site-profile-universal.json is now in the repository. Released builds
were being produced from a profile in a temporary directory, so the next release
could not have been reproduced once that file was cleaned up.
docs/RELEASING-WINDOWS.md points at the committed profile and says why.

scripts/gen_openapi.py reads __version__ out of shopdb/__init__.py instead of
restating it. Its hardcoded copy had already drifted a release behind, which is
the same mistake that once shipped an installer stamped with the wrong version.
 2026-08-05 07:34:05 -04:00
cproudlock
fb53161578 Answer "is this a re-run of my install?" from a record, not from the machine
Some checks failed
CI / backend (push) Failing after 9s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 10s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
The installer inferred that question from whatever the server happened to look
like: a MySQL service exists, the database has tables, the site exists, the venv
exists. None of those record who created them. A retry after a failed first
install was therefore taken for an upgrade of somebody else's working system,
which produced two dead ends on exactly the retry the wizard invites: stage 3
demanded a mandatory backup of a database its own failed attempt had written,
and then refused to prune tables it had created minutes earlier, because core
migration 7d05 seeds access protocols owned by the computers plugin and any
profile without that plugin hit the refusal every single time.

An install record at ProgramData\ShopDB-Flask\install-state.json answers it
instead. It is written when provisioning STARTS rather than when it finishes,
because the run that dies halfway is precisely the run whose retry needs it, and
it records what this installer created as it goes, so a crashed run no longer
leaves the next one guessing from the machine.

During unfinished first provisioning the pre-migration backup becomes advisory
and prune may force, since every row present was written by an earlier attempt
of the same install. On an established install both stay exactly as they were.
The classification is deliberately asymmetric: an install predating this record
carries a version stamp and probably real data, so it is treated as established
and keeps the mandatory backup. Guessing "first run" there would arm
prune --force against live tables.

Get-CreatedItems comma-protects its return. A zero-length array returned from a
PowerShell function unrolls to $null, and $null.Count is fatal under StrictMode
2.0 - the same fault that made bundle verification fail on every install
earlier. The harness caught it before it shipped.

Tests: deploy/windows/installer/tests/test-install-state.ps1 exercises new
servers, retries, completed installs, unrecorded-but-stamped installs, records
naming another directory, corrupt records, and persistence across a crash.
tests/test_installer_state.py runs it wherever pwsh exists and asserts the
invariants as text everywhere else. Both were confirmed to fail when the prune
gate or the comma protection is removed.

pytest.ini stops collection walking into deploy/windows/installer/bundle, which
is build output holding a complete second copy of the application. Importing
every plugin twice made SQLAlchemy refuse a redefined table and the whole suite
fail to collect, on a tree with nothing wrong in it, purely because an installer
had been built first. It surfaced only when the bundle grew from four plugins to
thirteen.
 2026-08-04 21:42:49 -04:00
cproudlock
412c2dc877 Record the code-signing decision
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 6s
Details 
Waiting for a certificate from the organisation's own certificate authority
rather than buying one from a public CA. Every server this installer runs on is
centrally managed and already trusts that root, so an internally issued
Authenticode certificate removes the unknown-publisher warning exactly where it
matters; a public certificate would buy trust on machines this software never
reaches.

Notes the interim measure that costs nothing: publish the SHA-256 through a
channel separate from the installer, since a hash beside the file is only as
trustworthy as write access to that location.

Wording avoids naming internal infrastructure, since docs/ is published.
 2026-08-04 21:25:14 -04:00
cproudlock
1c04ff28b9 Close the remaining installer review findings
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Eleven findings, grouped by the root cause each belongs to.

Wizard input reaching a command line unchecked (ShopDBFlask.iss). Port fields
were spliced in bare and arrive as [int] parameters, so a blank or mistyped
port shifted every argument after it; both port fields are now validated as
1-65535 digits. A path ending in a backslash, which is what a drive root looks
like, ended its argument with \" and CommandLineToArgvW read that as an escaped
quote, so paths are now quoted through a helper that doubles the trailing
backslash. A drive root is refused outright as well: uninstall deletes the
application directory recursively, so installing to D:\ would have wiped the
drive on removal. The password handoff was written with SaveStringToFile, which
writes an AnsiString, and read back as UTF-8, so a correct non-ASCII password
was reported as wrong; it now goes out as UTF-8 without a BOM.

Launching without checking the result. Plugin deregistration invoked "flask
plugin uninstall" without --yes, and the command carries a click
confirmation_option that aborts with exit 1 when nothing can answer the prompt,
so it could never once have succeeded; the bare 2>&1 under EAP Stop then turned
that into a terminating error which the catch downgraded to a warning while the
plugin directory was deleted regardless. It now passes --yes, brackets the
error preference, restores the location in a finally, and keeps the code on
disk unless deregistration actually succeeded. MarkShortcutRunAs had four
quotes where it needed three, which kept the whole command inside one Pascal
literal so LnkPath was never interpolated and no shortcut ever got the
elevation flag; its exit code is now logged too.

Comparing IIS physical paths as raw strings. IIS stores the path as typed, so
it may carry environment variables or a trailing backslash. A Test-SamePath
helper now normalises both sides. That closes a real hazard in uninstall, which
matched applications on alias alone and would remove an unrelated application
of the same name under another site, unattended, since -OnFailure never
suppresses the confirmation.

Accepting existing IIS state without reconciling it. "Site already exists" took
the site however it was, so re-running with a different port left the old
binding while CORS_ORIGINS, the firewall rule and the smoke test all used the
new one, failing a working server. It now refuses with both ports named rather
than silently re-binding, and refuses a site of that name serving a different
directory.

Preflight rows drawn past the panel. The failures loop had no cap at all and
the warnings loop capped at 6, a number unrelated to the panel, which holds
about three rows. The cap is now measured from the panel height, applies to
both loops, and the footer counts what was actually left out instead of
inferring it.

Also: a failed upgrade now says the application pool is still stopped and how
to start it, rather than only "part-configured", since stage 2 stops a pool
that was serving. It is deliberately not restarted automatically, because after
a stage 3 failure the deployed code and the schema may disagree. shopdb-admin
Restart-App starts a stopped pool or site instead of recycling, which is a
no-op on a stopped pool and then reported the application as unresponsive. A
dead Write-Log line that parsed as three arguments is gone, and a preflight
warning no longer tells the operator to add a directory to a compiled exe.
 2026-08-04 21:16:46 -04:00
cproudlock
ce521e84a5 Lock down backup directory ACLs, and let the uninstaller reach IIS
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 6s
Details 
Two findings from the installer review, both of which fail silently.

Database dumps were readable by every authenticated user. A directory created
under ProgramData inherits BUILTIN\Users:(I)(OI)(CI)(RX), and a dump contains
every row including the users table and its password hashes. The installer
applied an owner-only ACL, but only in the branch that CREATED the directory,
so a directory created first by the console (shopdb-admin.ps1 backup) kept the
inherited permissions and the installer could never repair it. The ACL is now
re-applied on every run rather than only on creation, and the grants are made
inheritable with (OI)(CI) so dumps written into the directory later are covered
too. shopdb-admin.ps1 applies the same hardening for the default location, and
for an operator-named path says the dump holds password hashes rather than
silently rewriting the ACL of a directory that is theirs.

Verified on Windows: before, the directory carried BUILTIN\Users:(I)(OI)(CI)
(RX); after, only SYSTEM and Administrators, and a file created inside inherits
exactly those two. Without (OI)(CI) that file would not have been covered.

The uninstaller could not remove anything in IIS. [UninstallRun] launched a
bare "powershell.exe", and the Inno uninstaller is a 32-bit process, so WOW64
resolved it to the 32-bit PowerShell, which cannot see the IIS provider. The
site, application pool and application survived, pointing at a directory that
HAD been deleted, while Windows reported a clean uninstall. It now uses the
same Sysnative path as the [Run] entry, which was the last unshielded launch
site in the file.
 2026-08-04 21:04:54 -04:00
cproudlock
1d73bd477e Document what future Windows releases look like, for operators and for builders
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Two audiences, two documents. Both were only in people's heads.

UPDATES-WINDOWS.md is for whoever runs a server: updates arrive as one
self-contained exe, an update takes two to four minutes, the site is down for
that time, .env and data and any hand-edited web.config are kept, unticking a
feature never removes it, the database is backed up and verified first, and a
downgrade is refused because migrations only go forwards. It covers both kinds
of security release, application and third-party, and explains that the
CycloneDX inventory staged on every server is what answers a published
vulnerability question. It also says plainly that the exe is not signed and the
checksum is the integrity check to rely on today.

It answers one question the existing docs did not address at all: the effect on
other sites sharing the same IIS server. The application pool is isolated and
the configuration is scoped to its own path, so other sites keep their own
handlers. What IS shared gets named rather than glossed: installing the IIS
modules and writing server-level configuration recycles application pools
across the server, which can drop requests in flight and clears in-memory
session state, though IIS is never stopped and no iisreset is issued. The two
IIS modules and the single permitted rewrite server variable are machine-wide
and stay behind on uninstall, deliberately, since another site may have come to
depend on them. The bundled database option collides on port 3306 with an
existing MySQL.

RELEASING-WINDOWS.md is for whoever builds releases: the three kinds of change
and the commands for each, why bundle-lock.json must be committed, the two
dependency traps that have each already cost a release, which generated files
must never be hand-edited, and the pre-release checks. It records the two known
gaps honestly - no code signing, and compiling still requires Windows and a
person.

UPGRADE.md and OPERATE-WINDOWS.md link to the operator document.
 2026-08-04 20:54:05 -04:00
cproudlock
aeee210cf6 Repair a web.config that an earlier build made unusable
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Stage 4 deliberately leaves an existing web.config alone, because operators put
real changes in it: extra MIME maps, a /installers location, bindings, a
proxy-specific rule. Overwriting reverts those silently.

That rule had no exception, and earlier builds of this installer wrote an
<allowedServerVariables> block which is fatal on its own: the section is Deny
by default, so IIS rejects the entire file with 500.52 before
httpPlatformHandler runs. Any server already installed would therefore keep the
broken file forever, with re-running the fixed installer powerless to help,
since the first thing stage 4 does is decline to touch it.

Strip just that element, keeping every other edit, and only when it contains
nothing besides the variable this installer adds. A block holding anything else
is somebody's deliberate change and is left alone with a warning. The previous
file is copied to web.config.before-xff-fix first.

Exercised against four inputs: the file earlier builds wrote, which is repaired
and still parses as XML with the rewrite rule intact; a block with an
operator-added variable, which is left unchanged; an empty block, which is the
$null.Count trap under Set-StrictMode 2.0 and is why the filter is wrapped in
@(); and an already-correct file, which is a no-op.
 2026-08-04 20:06:47 -04:00
cproudlock
95b0b77c13 Allow HTTP_X_FORWARDED_FOR at server level instead of declaring it per-application 
The stage 5 smoke test failure was a locked config section, but not one of the
two the installer unlocks. A diagnostic collected from the server returned:

  HTTP 500.52 - URL Rewrite Module Error
  Module RewriteModule, Handler httpplatformhandler
  Error Code 0x80070021
  Config Error: This configuration section cannot be used at this path.
  Config File: \\?\C:\shopdb-flask\web.config

handlers and httpPlatform were both overrideMode Allow and locked false, so
the unlock had worked. The section at fault was a third one,
system.webServer/rewrite/allowedServerVariables, which ships
overrideModeDefault="Deny". web.config declared <allowedServerVariables>
locally for the X-Forwarded-For rule, and IIS rejects that declaration
outright, failing the entire configuration before httpPlatformHandler ran.
python was therefore never launched and C:\shopdb-flask\logs stayed empty,
which reads as a dead application or a permissions fault and is neither.

Unlocking the section would let every site on the machine declare arbitrary
server variables. The installer now adds the single variable to the
server-level allow list, checking first because a duplicate add is an error,
and web.config no longer declares it. The rewrite rule is unchanged.

Verified by applying the installer's own uncommenting to the template and
parsing the result: one rewrite element, no allowedServerVariables, the rule
still setting HTTP_X_FORWARDED_FOR from REMOTE_ADDR.

shopdb-diagnose.py checked only the two sections the installer unlocks, so it
could not have named this one; the IIS error page did. It now reports the
lock state of the rewrite sections as well.
 2026-08-04 20:04:19 -04:00
cproudlock
10ee3a3c58 Add a stage 5 diagnostic collector
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
The stage 5 smoke test failing tells us only that IIS did not return 200. The
cause is in one of four places, and finding out which has taken a round trip
per guess. This gathers all four in one pass and writes a single report.

It records what IIS actually answers on localhost, 127.0.0.1, ::1 and the
machine name, including the status code and the parsed text of the IIS error
page; the site, application, pool and module state from appcmd, plus the
override state of the two config sections httpPlatformHandler needs; the
contents of web.config and the resolved httpPlatform processPath; whether the
venv can import shopdb and call create_app; the application logs, separating a
missing log from an empty one; the ACLs the pool identity depends on; and
recent HttpPlatform, WAS and W3SVC event log entries.

Secrets never reach the report. Values are read from .env first, then scrubbed
from every section before the file is written, which covers command output and
tracebacks that might quote them. A password embedded in any connection URL is
also masked whether or not it came from .env.

Standard library only, so it runs on the bundled runtime or any system Python.
Verified end to end on a Windows VM: it correctly reported a 404 with the IIS
error code for an absent application, and that localhost resolves to ::1 first.
 2026-08-04 19:56:13 -04:00
cproudlock
97391cdee4 Unlock IIS config after the application exists, and report why the smoke test failed
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Two defects found in the stage 4 and stage 5 logs from a Windows Server 2019
install.

The scoped config unlock ran before the thing it unlocks existed. appcmd
resolves its location argument against applicationHost.config, but the unlock
was issued from the ACL block, ahead of New-WebApplication. On a first install
"Default Web Site/shopdb" is not there yet, so appcmd returned 80070003, "the
system cannot find the path specified", and the code fell through to unlocking
the section for the entire machine. That fallback exists for servers which
refuse the scoped form; it was instead the only path a first install could
take, so every install silently granted handler delegation server-wide. Moving
the block below site and application creation lets the scoped unlock work.

The smoke test discarded the diagnosis. Invoke-WebRequest raises on any
non-2xx, and the catch block kept nothing from the exception, so a fault IIS
had already identified by status code was reported as "site did not return
200 ... check the logs". It now records the status code and the text of the
IIS error page, and prints the tail of the HttpPlatform stdout log, which is
where a Python traceback lands. It also distinguishes a missing log from an
empty one: the first means the pool never launched python, the second that
python started and wrote nothing.

A non-200 that did not raise, such as a redirect, skipped the retry delay, so
the loop could spend all twelve attempts at once and report a timeout without
having waited.
 2026-08-04 19:43:30 -04:00
cproudlock
a352a21a10 Declare packaging as a runtime dependency
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
shopdb/plugins/loader.py imports packaging.specifiers and packaging.version
at module scope, but packaging was never listed in requirements.in. It was
present in every development and CI environment as a transitive dependency of
pytest, so the full suite passed while a venv built from requirements.txt
alone could not import shopdb at all.

The Windows installer builds exactly such a venv, so stage 3 failed on a
customer server with ModuleNotFoundError: No module named 'packaging', after
the runtime and all wheels had installed successfully.

Add packaging to requirements.in, recompile the hashed lockfile, and add the
wheel to the offline wheelhouse with the matching bundle-lock entry. The
recompile also picked up newer uv formatting: inline environment markers on
cffi and greenlet and shorter "via" comments. The pinned distribution set and
every existing hash are unchanged.

tests/test_runtime_dependencies.py guards the general case by scanning
shopdb/, plugins/ and scripts/ for unconditional third-party imports and
asserting each maps to a distribution pinned in requirements.txt. Test
dependencies are the blind spot for this class of failure, since they are
present wherever the suite runs and absent wherever it does not.
 2026-08-04 19:12:00 -04:00
cproudlock
f6b621d126 fix(installer): keep -Wait, and use the stage-0 handoff even when .env exists
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 6s
Details 
Two defects from a Server 2019 run that got further than any before it - the
payload verified against the lock on a real server for the first time.

EMPTY EXIT CODE. "Python install failed (exit )" on an install that had actually
worked. Dropping -Wait to make -TimeoutSec enforceable left $p.ExitCode
unreadable: PowerShell only reliably populates it on a waited process. -Wait is
restored and the trade is now explicit - exit codes are load-bearing here, 1639
vs 1603 vs 3010 is the entire diagnosis, and a bounded wait is not worth losing
them for. -TimeoutSec is advisory: logged as an expected duration so a hang is
identifiable, not enforced. The code is also read defensively now, and an
unreadable one fails loudly rather than being taken for success.

That timeout has never worked - -Wait made the block dead code from the start -
so nothing is lost that was ever there. Trying to fix it broke something that
was working, which was the wrong trade to make silently.

STALE .env PREFERRED OVER A GOOD HANDOFF. The .dbpass fallback sat in the else
of "if .env exists", so it was consulted only when .env was absent. A
part-finished install HAS an .env, holding whatever password stage 2 last wrote;
if stage 0 has since regenerated the credential, .env is stale and .dbpass is
correct - and the installer preferred the stale one, giving "Access denied" with
the right password sitting unread on disk. The handoff is now applied before the
branch, so it covers both, and only when .env points at the local server so it
can never redirect a site whose database lives elsewhere.
 2026-08-04 13:57:13 -04:00
cproudlock
21110b86eb fix(installer): clear the retry path, which is the path everyone is actually on
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 6s
Details 
Five defects from the Windows-defect review, each confirmed against the code
before changing it. Four of the five only fire on a RE-RUN - and after eight
attempts a re-run is the normal case, not an edge case, which is exactly why
they survived.

Invoke-Native, three defects in one function:
  - Any non-zero exit was failure. 3010 and 1641 mean "done, reboot required",
    and the VC++ redistributable returns 3010 on a server with a pending file
    rename - an ordinary state on a freshly patched box. It is now an accepted
    outcome for the installers that can report it, logged as a warning so the
    operator knows a reboot is owed.
  - -Wait blocks inside Start-Process until the child exits, so the -TimeoutSec
    block below it could never run. Every timeout on every MSI was decorative.
    The wait is now bounded here, followed by a parameterless WaitForExit so the
    redirected output is flushed before it is read.
  - The Python bootstrapper ran /quiet with no /norestart, free to reboot the
    server mid-install.

Stage 0 refused to run when a MySQL service existed - including the MySQL84 it
had registered itself. Every bundled-database retry dead-ended while the wizard
promised that re-running was safe. A foreign MySQL still blocks; ours is started
if stopped, and the create-the-server block is skipped. It also no longer tries
to bootstrap through a root account whose password it set on the previous run:
with the handoff present there is nothing to do, and without it there is no safe
automatic recovery, so it says what to do instead of guessing.

Stage 4's appcmd unlock used '2>&1' under $ErrorActionPreference = 'Stop', which
turns any appcmd stderr into a terminating error - so the exit-code test and the
server-wide fallback, the whole reason the block exists, were unreachable, and
the stage aborted after Python, the venv, the schema and the ACLs had been
changed.

Stage 3 ran prune-schema and treated its refusal as a failure. Refusing is the
designed outcome when a table holds rows, signalled with SystemExit(1), so
Invoke-Native killed the stage and the reporting written to explain the refusal
was unreachable. Core migration 7d05 seeds access protocols owned by the
computers plugin, so any profile omitting computers hit this on every retry.

The preflight's MySQL 5.6 index-flag check is a warning, not a blocker. It
inspects the LOCAL MySQL, which may not be the database being installed against;
stage 3 checks the one actually chosen. Same class as the HttpPlatformHandler
blocker fixed earlier.
 2026-08-04 13:45:37 -04:00
cproudlock
5f350179b1 fix(installer): undo a fix applied twice, and load the checker at script scope 
A review of the installer for Windows-only defect classes found seven live
issues. These two would have stopped the next attempt on any server.

DOUBLE-APPLIED GUARD. Yesterday's $null.Count fix was applied at BOTH ends:
Test-BundleLock returns ,$problems, and the call site also wrapped it in @().
The comma already hands the array back intact, so the extra @() nests it and
.Count becomes 1 regardless of how many problems there are. Every install would
have failed with "the bundle does not match bundle-lock.json (1 problem(s))" on
a byte-perfect payload. Applying the same guard at both ends was worse than
applying it at neither. Verified in a Windows VM against a real bundle: clean 0,
tampered 1, restored 0.

DOT-SOURCE SCOPE. bundle-lock.ps1 was dot-sourced INSIDE
Assert-BundleIntegrity, which loads it into that function's scope - every helper
it defines disappears when the function returns. Assert-BundleIntegrity itself
worked; the next caller, Get-WheelhousePythonTag, died with "The term
'Get-JsonProperty' is not recognized". It only fires where a venv already
exists, so greenfield was fine and every retry after a part-completed install
was not. Now loaded once at script scope, guarded so the stages that run without
a bundle still work.

Both were confirmed by running them rather than by reading: the nesting with a
three-case pwsh test, the scoping with a minimal repro.
 2026-08-04 13:35:30 -04:00
cproudlock
14fedcee4c fix(installer): a clean payload crashed stage 2 
Reported from Server 2019: "The property 'Count' cannot be found on this object"
immediately into stage 2.

Test-BundleLock returns an array of problems, and an EMPTY array means the
payload is exactly right. PowerShell unrolls a zero-element return into $null,
and under Set-StrictMode 2.0 $null.Count throws - so the branch that runs when
everything is correct was the one that could not run. Every failing bundle got
past it fine, which is why nothing caught it until the 8.3 path fix made
verification succeed for the first time on a real server.

Fixed at both ends: the call site wraps in @(), and Test-BundleLock returns
,$problems so no caller can be handed $null or a bare string depending on how
many problems there happen to be.

The other .Count uses in this file were already @()-wrapped and are unaffected.
 2026-08-04 13:04:05 -04:00
cproudlock
fe091e751a fix(installer): the bundled database rejected connections it should have accepted 
Two defects in the stage 0 bootstrap, both surfacing as "Access denied" on a
server where the operator was holding the correct password.

CREATE USER IF NOT EXISTS is a no-op on an existing user - it does NOT change
the password. Stage 0 generates a fresh password every run and overwrites
.dbpass with it unconditionally, so any path that re-runs the bootstrap over an
existing account left the handoff holding a password the server had never been
told. ALTER USER now follows each CREATE, so the stored password and the handoff
always agree.

The user was also only created for 'localhost' and '127.0.0.1'. On current
Windows, 'localhost' resolves to the IPv6 loopback FIRST, so an operator who
types localhost rather than 127.0.0.1 arrives as '<user>'@'::1' - an account
that did not exist - and MySQL answers "Access denied" naming a host they never
typed. The ::1 account is now created and granted alongside the other two.

Note the datadir guard means the first defect could not fire on a straightforward
re-run - stage 0 refuses a non-empty data directory before reaching the
bootstrap. It was still wrong, and reachable once the directory has been cleared
by hand, which is what the failure message tells operators to do.
 2026-08-04 12:59:45 -04:00
cproudlock
189a474082 fix(installer): do not demand a password the installer already holds 
Asked why the password box does not pre-fill from .dbpass. It should not - but
it should not have been demanding a password either.

.dbpass is the ACL'd handoff stage 0 writes when it creates the database itself,
and stage 2 already reads it automatically when no password is supplied. The
wizard, though, required a password whenever .env was absent, without checking
for the handoff. On a server where stage 0 had completed but stage 2 had not -
which is exactly what a partly-failed install leaves - the operator was blocked
on a secret the installer already had, and sent hunting for a generated password
they were never meant to handle.

Blank is now accepted when either .env or .dbpass is present, and the sign-in
page says so when it sees a handoff.

Deliberately NOT pre-filled into the password box, for two reasons. It is the
only copy of a generated password, so round-tripping it through a UI control and
back out through a temporary password file adds exposure for no benefit - stage 2
reads the file directly. And .dbpass belongs to the BUNDLED database; on the
existing-database page the operator is pointing at someone else's server, where
a locally generated password is simply the wrong answer.
 2026-08-04 12:57:00 -04:00
cproudlock
5f18ca27a1 fix(installer): split the database page so every field is reachable
Some checks failed
CI / backend (push) Failing after 6s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Trimming the description brought the Username box back and left Password off the
bottom. CreateInputQueryPage stacks its fields below the description and neither
scrolls nor shrinks, so a field that does not fit is drawn past the surface and
simply never appears - no error, no scrollbar. Sizing the description against a
pixel budget that varies with DPI and font scaling is guesswork, and it had now
failed twice.

Connection details (host, port, database) and sign-in (username, password) are
now two pages of three and two fields. Both fit under any reasonable
description, at any scaling, without anyone having to estimate.

The upgrade hint about leaving the password blank moves to the sign-in page,
where the password field actually is. ShouldSkipPage hides both pages for the
bundled-database option, and the stage arguments read the values from their new
homes.
 2026-08-04 12:51:39 -04:00
cproudlock
e650eb0220 fix(installer): database page lost its Username and Password boxes
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Reported from the Server 2019 test: the Existing database page rendered a
truncated "Username:" label and no input boxes at all below it, so there was no
way to enter connection details.

CreateInputQueryPage lays its fields out BELOW the description text. The
description had grown to include a five-line CREATE DATABASE / CREATE USER /
GRANT block, added so a DBA could be handed the exact SQL. With five fields
underneath, the last two fell past the bottom of the page surface, where they
are simply not drawn - no error, no scrollbar, just missing controls.

The description is back to three lines. The SQL moves to
docs/INSTALL-WINDOWS.md, which is where someone would look for it anyway and
where it can be copied without being retyped from a wizard page.

Wizard page descriptions are a fixed budget: anything long enough to be worth
reading twice belongs in the guide, not on the page.
 2026-08-04 12:44:37 -04:00
cproudlock
c5797bb339 fix(installer): payload verification broke on 8.3 short paths
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 6s
Details 
A Server 2019 install reported all 96 payload files as simultaneously missing
and unexpected, with mangled names - wheels/heels/flask.whl,
python/ython/python-3.14.6-amd64.exe, mysqlclient/lient/mysql.exe. Exactly five
characters of each directory name survived, which is the difference between
ADMINI~1 and Administrator.

Inno extracts the bundle under C:\Users\ADMINI~1\AppData\Local\Temp\..., an 8.3
SHORT path. Resolve-Path kept that short form while Get-ChildItem returned the
long one, so the root was five characters shorter than the prefix being sliced
off every FullName, and every relative key came out wrong. The payload was
correct; the comparison was not - the verifier refused a perfectly good bundle.

The root now comes from Get-Item, which goes through the same provider as
Get-ChildItem so their path forms agree, and the prefix is checked with
StartsWith before being trimmed. If the two ever disagree again this throws
instead of inventing paths.

Verified against the real failure mode rather than assumed: running the check
through C:\SHOPDB~3\bundle in a Windows VM now passes.

Nothing on Linux or in a normally-pathed Windows directory could have caught
this - the short name only appears under a profile directory long enough to need
one, which is where Setup extracts.
 2026-08-04 12:26:53 -04:00
cproudlock
263ae8e3b4 fix(installer): install the Visual C++ runtime before MySQL, and log the MSI
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 6s
Details 
Second failure from the Server 2019 test. The previous fix worked - msiexec went
from exit 1639 (ERROR_INVALID_COMMAND_LINE, which is why it printed its usage
dialog) to exit 1603 (ERROR_INSTALL_FAILURE), so the command line parses now and
the MSI itself is failing.

It failed in 1.1 seconds. An MSI that dies that fast has not begun installing;
it has failed a launch condition. MySQL 8.4 requires the Visual C++
redistributable and a bare Windows Server does not ship it - the same runtime
mysql.exe and mysqldump.exe import, which was visible when their DLL
dependencies were trimmed and went unnoticed.

Stage 0 now installs VC_redist.x64.exe from the bundle before touching MySQL,
skipping it when vcruntime140.dll is already present, and fails with a sentence
naming the requirement if the redistributable is absent from the bundle
altogether. vcredist\ is an optional locked payload.

msiexec also gets /l*v now. A bare 1603 names neither the failing action nor the
reason, and it is the most common MySQL install failure - diagnosing this one
took a launch-condition inference rather than a log. The MSI log lands beside
the installer's own in ProgramData, so the next failure is readable instead of
guessed at.
 2026-08-04 12:17:36 -04:00
cproudlock
8d0afc40d3 fix(installer): bundled MySQL install failed on a malformed msiexec command line 
Reported from a Windows Server 2019 test: a "Windows Installer" dialog listing
every msiexec /Option appeared, then the wizard reported that the bundled MySQL
database could not be installed. That dialog is msiexec's usage help - it prints
it when the command line does not parse - so the install never started.

Cause: $MysqlRoot defaulted to 'C:\Program Files\MySQL\MySQL Server 8.4', which
contains spaces. Invoke-Native wraps any argument containing whitespace in
quotes, producing "INSTALLDIR=C:\Program Files\...". msiexec takes public
properties as PROPERTY=value and expects the VALUE quoted -
INSTALLDIR="C:\Program Files\..." - so it rejected the line, printed usage, and
exited non-zero.

This file already carried the rule, next to the Python target: "Never put a
space in a path this installer controls." I broke it setting the 8.4 path.

Two fixes, because one of them alone leaves the trap in place:

- $MysqlRoot is now C:\MySQL84, space-free like C:\Python314. The MySQL client
  search paths in the installer, the preflight and the operator console all look
  there first, keeping backups working against the bundled server.
- Invoke-Native now quotes PROPERTY=value correctly, so passing a spaced path
  explicitly no longer produces an unparseable command line.

tests/test_installer_defaults.py fails if an installer-controlled path default
ever contains a space again.
 2026-08-04 12:06:36 -04:00
cproudlock
5321649e02 fix(installer): stop blocking the wizard on things the installer itself installs 
The preflight page began refusing to continue while any check was failing, which
is right for something the operator must go and fix. HttpPlatformHandler was
marked FAIL when absent - so on a server without it the wizard stopped dead,
telling the operator the server was not ready, over a module the bundle carries
and stage 4 installs a few pages later. The only way forward was to go and
install by hand the exact thing the installer was about to install.

It is now INFO: reported, not blocking, matching how URL Rewrite is already
handled. Nothing the installer SUPPLIES may block the wizard, and
tests/test_installer_defaults.py now fails if that rule is broken again.

The site-port conflict check is downgraded from FAIL to WARN for the same class
of reason: it runs before the operator reaches the Address page, so it tests the
DEFAULT port rather than the one they intend to use, and blocking refuses an
install over a conflict the very next page lets them resolve.

Genuine blockers are unchanged - no IIS, no WebAdministration, wrong Windows
edition or architecture, no disk, and the MySQL 5.6 index flags. Those the
operator really does have to fix first.
 2026-08-04 10:00:44 -04:00
cproudlock
4a8bd138a9 feat(import): load a site's data from spreadsheets
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 6s
Details 
Adopting a site means getting its asset register in. The HTTP import API suits a
site with a source system and someone to script against it; a sister site with a
spreadsheet and no developer needs something else, and that is the common case.

FOREIGN KEYS TAKE NAMES. This is the whole design. A CSV row has to say where an
asset is, and the database stores locationid, an integer. Requiring the number
means importing locations, reading back the generated ids and pasting them into
the asset sheet - a workflow nobody finishes. Every foreign key here accepts
either a numeric id or the referenced row's name:

    assetnumber,assettypeid,statusid,locationid
    CMM-01,Measuring Tool,Active,Gage Lab

The column keeps its database name, per CONTRIBUTING.md; the value is whatever
the operator actually knows. Names resolve across files in one run, so
assets.csv can reference a location that only exists because locations.csv was
read moments earlier. A name that does not resolve is reported with its line,
column and value, not as a foreign key violation from three layers down.

Dry run is the default, and writes go into the transaction either way - the
rollback is what makes it a dry run. Skipping the writes instead made every
cross-file reference fail, which is the one thing a folder-wide check exists to
verify. Validation covers every row before anything is written, so a typo on
line 400 cannot leave 399 rows imported. Files are matched on a natural key, so
correcting a spreadsheet and re-running updates rather than duplicates.

TEMPLATES ARE GENERATED, NOT MAINTAINED. "flask csv templates" builds them from
the live schema, annotated with required/optional and which file each foreign
key refers to. The prompt for this was a hand-written template set that had
invented columns on seven of eleven tables and named a table that does not
exist, while looking entirely plausible - and described an import mechanism
(a Data Import page, a flask import-csv command) that had never existed. A test
fails the build if a generated template ever offers a column the schema lacks.

User accounts are deliberately not importable: passwords do not belong in a
spreadsheet in either direction.

Verified end to end against MySQL 5.6 - a folder dry run catching one bad
reference, the fix, the commit, and a re-run reporting updates rather than
inserts. 16 tests.
 2026-08-04 09:13:03 -04:00
cproudlock
f72813ed9c feat(installer): bundle the database - MySQL 8.4 LTS, not 8.0 
The bundled-database option could not actually be built. Stage 0 looks for
mysql\mysql-8.0.x-winx64.msi, and Oracle no longer publishes a standalone server
MSI for 8.0 - every 8.0.x returns 404. What remains for 8.0 is the MySQL
Installer bundle, which is an installer-manager: 'msiexec /i INSTALLDIR=' would
install THAT rather than a database, and stage 0 would then fail on a missing
mysqld.exe.

MySQL 8.0 also reached end of life in April 2026, so bundling it would have put
an unsupported database on every new site.

8.4 LTS still ships the standalone MSI (129MB, which is what the '125MB' note in
stage 0 was written against) and is supported into 2032. Defaults follow it:
install root MySQL Server 8.4, service MySQL84. The operator console still looks
for an 8.0 install path as a fallback, for sites already running one.

Also bundles mysqlclient\ - mysql.exe and mysqldump.exe with the two OpenSSL
DLLs they actually import, 20MB rather than the 51MB of debug and auth-plugin
libraries the archive ships. Stage 2 stages it onto the server, so a site whose
database is on ANOTHER host can still take the pre-upgrade backup that every
upgrade depends on. That was the gap the preflight had started warning about.

Bundle is now 221MB.
 2026-08-04 07:56:39 -04:00
cproudlock
8d9d1d3439 test(docs): skip the publishability gate where there is no docs/ to check
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 6s
Details 
The GitHub backend job failed on test_there_are_docs_to_check, correctly. docs/
is stripped from the published repository - it lives in the wiki on that side -
so on the mirror the glob matched nothing and the guard fired exactly as
designed.

An absent docs/ and a glob that silently matches nothing in a tree that HAS docs
are different conditions, and the test conflated them. The module now skips when
the directory is not there at all, and the guard still fails when it is there and
empty. Verified all three ways: 7 pass here, 7 skip in a docs-less checkout, and
the guard still fails against a docs/ containing no markdown.

The gate has to ship rather than be excluded from publication, because the
published tree is where the GitHub CI that would catch a regression runs.
 2026-08-04 07:28:47 -04:00
cproudlock
2073d0dbe8 build(export): purge stale generated paths from the publication tree
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
rsync --exclude also PROTECTS a path from --delete, so anything that reached the
publication tree before its exclude existed stayed there permanently - invisible
to the sync and surfacing only as a scrub-gate failure. That cost two rounds of
'add the exclude, still fails' on the installer bundle and again on
.pytest_cache. The generated paths are now purged before the sync, so adding an
exclude is sufficient on its own.

tests/test_docs_publishable.py assembles its search terms from fragments: a file
containing the literal strings the scrub greps for tripped that scrub on itself.
Excluding the file from publication would have removed the check from the
repository it protects.
 2026-08-03 15:17:09 -04:00
cproudlock
ee083ea80e docs: stop publishing internal references to a public wiki 
docs/ is excluded from the code bundle and its scrub gate, because it goes to
the GitHub wiki instead - via a generator that has no gate at all. So the one
part of the repository written in prose, by people, about internal
infrastructure, was the one part nothing checked.

What was reaching a public wiki: the internal git server's URL and hostname,
.gitea workflow paths, developer home directories in the GE-Enforce cutover
reference, and a dev database root password inside a copy-pasteable command in
the import guide.

All replaced with neutral equivalents. tests/test_docs_publishable.py is now the
gate, at the source, in CI - a wiki page cannot be un-published, so catching this
after the fact is not good enough.

PROJECT-REVIEW.md also referred to internal tooling by name throughout; those
references are generalised. It remains an internal candid assessment of this
project that is nonetheless published, which is worth a separate decision.
 2026-08-03 15:12:25 -04:00
cproudlock
bec138f5ac build(export): keep the installer's staged bundle out of publication 
The sync walks the working tree rather than git, so deploy/windows/installer/
bundle came through despite being gitignored - about 100MB of build output
containing a copy of the whole application tree, the wheels and the vendor
installers. Its copies of config.py and requirements.txt then tripped the scrub
gate, which is the only reason it was noticed.

Excluded along with the installer's other generated files. Note that rsync
--exclude also protects a path from --delete, so a copy already in the
publication tree has to be removed by hand once.
 2026-08-03 15:09:13 -04:00
cproudlock
e158cb21f9 deps: keep build-machine paths out of the lockfiles
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
The '# via' annotations recorded the absolute path of the temporary file the
lockfile was compiled from, which is meaningless to anyone else and does not
belong in a published artifact. They now read 'requirements.in', which is where
these requirements actually come from.

Pins and hashes are unchanged - verified by a hash-checked dry-run install.
 2026-08-03 15:06:02 -04:00
cproudlock
2c415a1712 fix(installer): correct a false security claim, and clear the should-fix list
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
CLIENT IP / SPOOFABILITY. docs/geenforce-api-cutover.md claimed that removing the
IIS rewrite rule made the allowlist fail closed and that it does NOT become
spoofable. The opposite is true. IIS never sets X-Forwarded-For on its own; the
rule is the only thing that does. Remove it and IIS still forwards whatever
X-Forwarded-For the CALLER sent, waitress trusts it because it arrives from
127.0.0.1, and remote_addr becomes attacker-controlled - so a token-less caller
can fetch manifests from anywhere on the network. The document and the
_trusted_client_ip docstring now say so, waitress runs with
--trusted-proxy-count=1, and stage 5 checks the rule is actually live rather than
assuming it. The wizard question is rephrased to something an operator can verify
with their network team instead of guessing at.

NON-ASCII. The style gate only ever checked .py/.vue/.js/.ts, so documentation
accumulated em-dashes, arrows and box-drawing characters against this repo's own
convention - including in files added this week. Cleaned, and the gate now uses
INCLUDES_ALL so Markdown, JSON and YAML are covered.

PLUGIN DEFAULTS. The wizard pre-ticked measuringtools and printedparts, both of
which ship default_enabled=false, so every site taking the defaults installed and
enabled them against their manifests. Inno has no JSON parser so the list must be
hardcoded, but tests/test_installer_defaults.py now fails when it drifts.

UPGRADES. The payload copy merges, so a plugin dropped from a site's profile kept
its code forever - which defeats a lean build and leaves core's optional-import
guards succeeding for a plugin the site no longer has. Stale plugin directories
are now deregistered and removed before the copy.

add-plugin used 'plugin install', which for the five default_enabled=false
plugins left them installed but DISABLED - and printed a green success line
anyway. It now goes through apply-profile, and the success line is gated on the
exit code. Invoke-Flask records its own exit status, because $LASTEXITCODE keeps
a stale value when flask.exe is missing and no native command runs.

CHARSET. The utf8mb4 compiler hook lived inline in migrations/env.py, so it
covered the CORE chain only: plugin baselines inherited the server default, which
on a latin1 server means two charsets in one database. It is now
shopdb/utils/mysql_charset.py, imported by both, and preflight reports the
database's default charset.

BACKUP HONESTY. The dump was described as 'all of your asset data'. Uploaded
branding and floor-map images live in instance\ on disk, not in the database, so
a restore from the .sql alone comes back with no map. backup now archives
instance\ alongside it and says both are needed.

VERSIONING. AppVersion was hardcoded at 0.9.0 while the product, the frontend and
the newest tag said 0.7.0 - and 0.9.0 collides with a retired contract version.
Both builders now generate version.iss from shopdb/__init__.py.

Smaller: rollback overwrites .env before deleting it, as uninstall already did;
appcmd unlocks are scoped to this site's location rather than server-wide, with
the wide unlock as a fallback; DEVELOPMENT-SETUP says Python 3.14; the README
plugin list gains printedparts; prune-schema --force is documented as
first-provisioning-only; HTTPS is documented as not-the-default with the steps to
add it; the DBA SQL is on the wizard's database page; the features page says
unticking does not remove an installed feature; and the installer README states
that bundle-lock cannot vouch for the exe itself - that needs signing or an
out-of-band hash, neither of which is wired up.
 2026-08-03 14:57:38 -04:00
cproudlock
aea2905de0 fix(installer): stop it lying, stop it leaking, and make it findable
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Nine fixes from a review of the installer against its actual audience: DT leads
at sister sites who are not Windows, IIS or Python specialists and who will lean
on an AI assistant to get through it.

TRUTHFULNESS. The preflight was advisory - an operator read 'IIS is not
installed', pressed Next, answered five more pages and the install died partway
through with Python already on the box. The results page now blocks while
anything is failing, repaints on every run instead of latching after the first,
and offers 'Check again' so a fixed problem does not mean starting over. On
failure the wizard said 'Nothing was left running', which is false in every path
because the stages run with -OnFailure never: it now says the server is
part-configured, that re-running is safe, and how to remove it. The final page no
longer reads 'ShopDB-Flask is ready' after a failed install.

SECRETS. The generated MySQL root password went to Write-Host in a process the
wizard runs hidden - so nobody saw it - and stdout is forwarded into the setup
log operators are told to send to support, so it was permanently recorded for
everyone who did not need it. It now goes to an ACL'd file. Database dumps, which
contain every user password hash, landed in a ProgramData directory readable by
every user on the box; the directory is now locked at creation.

UPGRADES ON REMOTE-DATABASE SITES. mysqldump was looked for only under local
MySQL install paths, so a site whose database is on another host silently skipped
every pre-upgrade backup - after stage 2 had already stopped the pool and
replaced the tree. Find-MysqlTool now prefers a client shipped in the bundle,
stage 2 stages it onto the server, preflight reports when it is missing, and
mysqlclient\ is an optional locked payload.

UNINSTALL. A subpath install is an IIS Application, not a site; removing only the
site left the application pointing at a deleted directory, so the parent site -
at West Jefferson, the live classic ASP - served 503 on that path forever while
Add/Remove Programs reported success. Uninstall now reads MOUNT_PATH and removes
the application. The firewall rule was created as "$SiteName $SitePort" and
removed as the literal 'ShopDB-Flask 8090', which matches nothing.

DAY-2 TOOLING. Every shortcut now passes -AppRoot and -SitePort, and the console
forwards them through its own elevation and 32-bit relaunches instead of
discarding them - a non-default directory or port made it report a healthy site
as broken, from a shortcut the installer wrote. 'Open ShopDB-Flask' resolved to a
hardcoded localhost:8090 that was wrong for every subpath install; it now asks
the console, which reads the address the installer recorded, and no longer
demands administrator to open a browser.

SMOKE TEST. The parent-site port lookup filtered for an http binding and
defaulted to 80, so an https-only parent site failed a working install with a red
dialog.

DOCS AND /api/docs. The installer was invisible: nothing in docs/, README.md or
CLAUDE.md mentioned it, so a DT lead or their assistant landed on the manual IIS
runbook and hand-built the very server the installer then refuses to upgrade.
docs/INSTALL-WINDOWS.md and docs/OPERATE-WINDOWS.md are now the canonical route,
the two manual runbooks are bannered as reference-only, README and CLAUDE.md
route by target, and llms.txt tells an assistant which document to follow and to
ask for 'check -Json' before diagnosing. Both ship on the server, along with
openapi.json and llms.txt - without those the self-hosted /api/docs was broken on
every installed box, which matters most to the sites least able to debug it.
Stage 5 now checks it actually serves.

shopdb-admin.ps1 gains 'check -Json': one structured, secret-free block covering
version, publishing method, IIS state, HTTP reachability, database, Python
version, plugins and errors. That is the cheapest useful answer to 'the operator
will ask an LLM' - it works with no infrastructure, which a install-time MCP
server could not.
 2026-08-03 14:39:38 -04:00
cproudlock
e58f376643 fix(installer): the adopt-existing guard never fired 
Stage 4 decided whether the IIS objects it was about to reconcile were its own by
testing for .installed-version. Stage 2 writes that file, and stage 2 always runs
first in a '-Stage all' install - so by the time the guard looked, the stamp it
had just written made every server look like one this installer built, including
the hand-built ones the guard exists to protect.

Stage 2 now records whether a stamp was present BEFORE it writes its own, and
stage 4 reads that observation. Running stage 4 alone still tests the file, which
is correct there: no stage 2 has run to disturb it.

Found by review, not by test - the guard has no coverage, because exercising it
needs a live IIS.
 2026-08-03 13:47:07 -04:00
cproudlock
3606d8d696 feat(sbom): ship a CycloneDX bill of materials with every build
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
An air-gapped site cannot be scanned from anywhere else, so when a CVE lands the
only way to answer 'is that component here, and at what version' was to RDP in
and go looking. The frontend was the real blind spot: nothing recorded which
version of leaflet, dompurify, jspdf or html2canvas ends up inside the compiled
SPA.

scripts/generate_sbom.py emits CycloneDX 1.6 covering both ecosystems - every pin
in requirements.txt with the sha256 the installer enforces, and every package in
package-lock.json. Build-only npm packages are marked scope 'excluded' rather
than dropped, so 'not here' stays distinguishable from 'not looked for'.
Dependency edges are real: uv's '# via' comments give the Python graph and
package-lock gives the npm one.

Hand-rolled rather than cyclonedx-py plus cyclonedx-npm because both inputs are
already pinned and committed - this is a format translation, not a scan - and
because the build box may be a work PC with nothing but Python and Node. It is
deterministic by construction: same inputs, byte-identical output, so
regenerating does not churn.

Staged into the application tree by both builders, so it installs onto the
server with the app. shopdb-admin.ps1 verify reports it and searches it by
component name, which is the question actually being asked.

Packages appearing at several depths in package-lock (node_modules/vite and
node_modules/vitest/node_modules/vite) are merged, and a copy reachable outside
the dev tree makes the component count as shipped. Emitting both produced
duplicate bom-refs, which CycloneDX forbids and scanners reject; getting the dev
merge backwards would have hidden a shipped package from a CVE search.

Not covered by bundle-lock.json on purpose: its provenance is git, not the
third-party payload.
 2026-08-03 13:15:27 -04:00
cproudlock
1bf3cb2e1c feat(installer): refuse to damage an installation it did not create
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Two guards for a server deployed by hand, which the West Jefferson production
box is.

An existing venv is reused, which is right for a repair or an upgrade of an
install this made, and wrong when the venv belongs to a different Python. The
wheelhouse is tagged for one minor version, so pip finds no candidate for the
compiled packages and dies partway through - after Python has been installed and
the application tree replaced. The two versions are now compared up front and
the run stops with both numbers and what to do about it.

Switching deployment method removes the other method's IIS artifact. That is
correct when this installer owns both and dangerous when it does not: a wrong
-MountAlias would call Remove-WebApplication on a live mount with no prompt and
no error, and the first sign would be the site returning 404. It now refuses
unless a version stamp shows this installer made the install, or -AdoptExisting
is passed, and the refusal lists exactly what it would have removed.
 2026-08-03 11:46:48 -04:00
cproudlock
13d831eb90 feat(installer): require the wheelhouse to satisfy requirements.txt, and lock the real payload 
The lock records what IS in the wheelhouse, not what the application NEEDS, so
an incomplete wheelhouse was locked, blessed and shipped - and only failed on an
air-gapped server.

That is not hypothetical. Assembling the wheelhouse anywhere other than Windows
silently omits colorama, a win32-only dependency of click, because pip evaluates
environment markers against the machine doing the downloading rather than the
machine being targeted. The bundle built here was short exactly that one wheel.

Both verifiers now cross-check wheels/ against the staged requirements.txt,
ignoring markers, since a requirement guarded by sys_platform == 'win32' is
precisely the one that must be present. Names are normalised to PEP 427 wheel
form, so mysql-connector-python matches mysql_connector_python.

bundle-lock.json is the first real lock: 42 files, cp314/win_amd64 - 39 wheels,
Python 3.14.6, HttpPlatformHandler 1.2 and URL Rewrite. MySQL is absent and
optional; a site choosing the bundled-database option adds it and re-locks.

The naming gate now skips the installer's build output. It contains a staged
copy of the application plus a second SPA build under dist-subpath, which
--exclude-dir=dist does not match, so a staged bundle failed the gate on
vendored minified JS nobody in this repository wrote.
 2026-08-03 11:46:39 -04:00
cproudlock
44237b5cbd feat(installer): bundle URL Rewrite, ask where client IPs come from, verify installs 
IIS does not set X-Forwarded-For on its own and HttpPlatformHandler connects
from loopback, so without a rewrite rule every client reads as 127.0.0.1. The
GE-Enforce IP allowlist, the dashboard visitor-location lookup and per-host
login rate limiting all stop working, silently. The rule needed URL Rewrite,
which the installer told operators to download - from an air-gapped server.

URL Rewrite now ships in the bundle, and the wizard asks which case applies,
because the two answers are mutually exclusive. Directly exposed: install it and
set X-Forwarded-For from REMOTE_ADDR, which is what stops a client spoofing its
own. Behind a proxy: leave the rule off, since REMOTE_ADDR is the proxy and
applying it would discard the real client IP.

The rule is enabled by deleting two explicit marker lines rather than by a regex
over the surrounding comment, so editing that prose cannot silently disable it.

An existing web.config is no longer overwritten. It is the one file on a server
that legitimately carries hand-edits, and replacing it reverted them without a
word - on a server where the X-Forwarded-For rule had been enabled by hand, that
alone would have turned the GE-Enforce IP allowlist off. The installer reports
what it found instead.

pip now runs with --require-hashes and --only-binary=:all:. Hash-checking is
requested explicitly rather than inferred from the lockfile, so shipping an
unhashed requirements.txt fails loudly instead of quietly dropping the check.

shopdb-admin.ps1 gains a verify command: which bundle this server was installed
from, and whether the installed packages still match what shipped.

The .iss states its compiler floor. WizardStyle uses the built-in windows11
custom style, which needs Inno Setup 6.6.0; older compilers now fail with that
sentence rather than 'WizardStyle is invalid'.
 2026-08-03 11:17:58 -04:00
cproudlock
88af7fd9ce feat(installer): lock the third-party payload, and build on Windows without Bash 
The bundle carries ~40 wheels, a Python installer and two MSIs. All of them run
as SYSTEM on the target server, and nothing verified any of them. A missing
wheelhouse printed MISSING and the script still exited 0, so an empty bundle
compiled into a shippable installer and the failure surfaced on an air-gapped
server with no way to fix it.

bundle-lock.json now records that payload exactly - sha256 and byte size per
file - and verification is set equality: a missing file, an unexpected extra
file, or changed content all fail. Both builders check it and refuse to produce
an unverified bundle; the lock ships inside the bundle and shopdb-install.ps1
re-checks it on the server before running any of it.

This is deliberately a layer above requirements.txt hashes. pip lists every
artifact of a pinned version (cffi 2.1.0 alone has 100 hashes), so it proves a
wheel is genuine, not that it is the wheel this bundle was built and tested
with; it ignores extra files in the wheelhouse; and it covers none of the
executables.

refresh-bundle-lock.ps1 regenerates the lock but refuses to overwrite one until
the operator has seen the diff, because the commit is the review - it is the
only place a change to what runs as SYSTEM becomes visible to a human.

build-installer.ps1 is the whole build natively on Windows, so a work PC needs
no Bash. It shares the plugin closure resolver with build-site.sh.

Both builders now copy the installer scripts from the repository. They were
copied from a downloads folder, so the logic that shipped was not the logic that
was committed and the build worked on exactly one machine.

Two verifiers exist because PowerShell is the only thing guaranteed present on
the target server, while the Linux builder should not need pwsh.
tests/test_bundle_lock.py runs both against the same fixtures and fails if they
disagree.
 2026-08-03 11:17:45 -04:00
cproudlock
75f0a57821 build(site): share the plugin closure resolver, stage only web.config 
Two fixes to the lean-site build.

The closure resolution moves out of an inline heredoc into
scripts/resolve_plugin_closure.py. The Windows builder needs the same answer,
and a PowerShell reimplementation would have been a second copy of the rules,
free to drift and produce a bundle whose plugin set did not match its profile.

The backend staging step copied all of deploy/ into the output tree. The Windows
installer stages its bundle at deploy/windows/installer/bundle, so that copy
recursed into its own destination and cp aborted with 'cannot copy a directory
into itself' - the documented build could not complete. Only
deploy/windows/web.config is read at install time, so only that is staged; the
rest of deploy/ is installer source and does not belong on an application
server.
 2026-08-03 11:17:28 -04:00
cproudlock
9c2c21c2cc fix(employees): resolve User through the contract surface 
test_plugins_only_import_contract_surface has been failing on main since
9a2d0cc: the employee name resolver imported shopdb.core.models directly.
shopdb.api already exports User (contract 0.13.0), so this is the same object
reached the way ADR-001 requires.
 2026-08-03 11:17:28 -04:00
cproudlock
6ebc79a2de deps: hash-pin the lockfiles and stop dev and prod drifting apart 
Both files are recompiled with --universal --generate-hashes, preserving every
pinned version. Three things change.

Hashes put pip into hash-checking mode, so a wheel whose sha256 is not listed is
refused rather than installed. The offline Windows install previously took
whatever file in the wheelhouse satisfied the version pin.

--universal means one lockfile serves Linux (dev, Docker, CI) and the Windows
wheelhouse. The Linux-only resolve had silently omitted colorama, a win32-only
dependency of click; in hash-checking mode a missing entry is a hard error, so
that omission would have broken every Windows install.

requirements-dev.txt is now compiled with -c requirements.txt, pinning shared
dependencies to the versions production runs. The two had been compiled at
different times and drifted: CI tested against alembic 1.18.5 while sites
installed 1.18.4.

Hashes pin the version and prove the artifact is one upstream published. They do
not pin WHICH artifact of that version is used, and they say nothing about extra
files in the wheelhouse - bundle-lock.json covers both.
 2026-08-03 11:17:18 -04:00
cproudlock
5c4fdcb15e Merge branch 'installer-prereqs' into feat/installer-bundle-lock 2026-08-03 10:06:56 -04:00
cproudlock
4d5b6c3c55 build(site): also stage a subpath frontend build
Some checks failed
CI / backend (push) Failing after 2m1s
Details
CI / naming (push) Failing after 1s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
Vite compiles the mount path into the bundle, so it cannot be chosen at install
time from a single build - a page served under /shopdb would load and then
request its assets from /assets/, and render nothing.

build-site.sh now produces both:
  frontend-dist           base /        - the app on its own IIS site
  frontend-dist-subpath   base /<alias> - an IIS Application under an existing
                          site, e.g. http://<server-fqdn>/shopdb/

SUBPATH_ALIAS (default 'shopdb') is fixed per bundle and written into the staged
build as .alias, so the three places that must agree - the IIS application alias,
MOUNT_PATH in .env, and this compiled base - cannot drift apart. The installer
checks that marker and refuses rather than serving a page that cannot load.

The subpath build runs FIRST and is held in a temp dir: the root build has to be
last so frontend/dist is left in the state a developer expects, and the copy into
$OUT has to happen after the staging step that does rm -rf "$OUT".
 2026-08-03 01:55:21 -04:00
cproudlock
0fa5f1e910 feat(deploy): add the air-gapped Windows installer 
Roughly 2500 lines of tested installer had been living in ~/Downloads and an
untracked folder - nothing was under version control.

It goes here rather than in a repo of its own because it depends on application
internals: the `flask plugin` verbs, site-profile.json, MOUNT_PATH, and the
plugin registry. Versioned separately it would drift out of step with the thing
it installs.

Contents: the read-only preflight, the staged installer (bundled MySQL, runtime,
schema, IIS, verify, uninstall), the operator console, the Inno Setup wizard, the
bundle builder and the artwork generator.

bundle/ and Output/ are ignored - regenerable, and ~220MB. plugins.iss is ignored
because build-installer.sh generates it from the staged payload. The artwork IS
committed so a Windows build box does not need Python and cairosvg.

Verified end to end on Windows Server 2025 against a bundled MySQL 8.0 and an
existing MySQL 5.6: fresh install, upgrade with backup and rollback, re-run
idempotency, uninstall, and both deployment methods including switching between
them. Not yet verified: a hypervisor-level air-gapped run, and any load from a
real browser (every HTTP check so far used curl, which sends no Origin header).
 2026-08-03 01:47:34 -04:00
cproudlock
0f766cf977 fix(setup): stop the wizard step promising data it does not create 
The step was called "Starter Data" and offered to "seed the data a new site
needs", but seed_starter inserts eight vendor rows and nothing else: no assets,
locations, departments or statuses. An operator ran it, saw every dashboard
count stay at zero, and reasonably concluded the seed was broken.

Rename the step to Reference Data, describe what is actually seeded, and say
outright that no assets are created and that an empty dashboard is expected
here. Assets arrive later via the import API.
 2026-08-02 18:56:38 -04:00
cproudlock
888b15a7dd build(site): stage a deployable tree and the profile in build-site.sh 
build-site.sh staged only shopdb/, the chosen plugins/ and frontend-dist, so the
output could be imported but not run or migrated. The Windows installer had to
assemble wsgi.py, requirements.txt, migrations/ and deploy/ separately, which
meant it could assemble a payload whose plugin set did not match the profile the
tree was staged from.

Stage those runtime files, and copy the profile in as site-profile.json so the
set is self-describing: `flask plugin apply-profile` at provisioning reads the
same profile the tree was built from, so installed plugins and shipped plugin
code cannot drift.

frontend-dist keeps its name; CI reads that path (ci.yml:79).

The closing hint now spells out `prune-schema --yes --force`. ADR-014's prose
says lean provisioning "uses --force", but --force alone only permits dropping
non-empty tables; without --yes the command is a dry run that prints a preview
and exits, so following the ADR literally silently skips the prune.
 2026-08-02 16:08:42 -04:00
cproudlock
94d6878a03 feat(frontend): gate first run on needs-admin so a fresh instance shows the wizard 
A fresh install landed on the anonymous dashboard instead of prompting to create
the first admin, so an operator had no way to discover /setup.

The router now asks /api/setup/needs-admin before rendering any unauthenticated
route and redirects to /login?firstrun=1 while no user exists. The result is
cached in a composable so it costs one request per session, and the lookup fails
open (a backend that cannot answer must not lock the login screen). Racing it
against a 4s timeout keeps a slow or hung backend from blocking the first paint.

Login.vue clears the flag after creating the admin so the gate stops firing
without a reload.
 2026-08-02 16:08:42 -04:00
cproudlock
11f3d00a04 Installer prerequisites: REQ-D through REQ-G
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
REQ-D: restore waitress and tzdata to requirements.in. They existed ONLY in the
generated requirements.txt (hand-added in bf9e60e), so the next
`uv pip compile` would have silently removed the WSGI server and the IANA
timezone database from every Windows install.

REQ-E: split production and development requirements. requirements.txt was
installing pytest, pytest-cov, pytest-flask, coverage, iniconfig and pluggy onto
production servers. Verified on a real Windows Server box before this change.
CI, scripts/test-external-plugin.sh and the dev docs now use requirements-dev.txt.

REQ-F: standardise on Python 3.14. The repo declared four different versions
(Dockerfile 3.12, DEPLOY-WINDOWS-IIS 3.12, INSTALL-WINDOWS-IIS 3.13, CI 3.13,
plus README, web.config and PLUGIN-EXTERNAL-REPO). 3.14 is in active bugfix
support until ~Apr 2027 and supported to Oct 2030; 3.13 entered security-only in
Apr 2026. All four compiled dependencies publish win_amd64 wheels for 3.14
(cryptography via an abi3 wheel), verified by building an offline wheelhouse and
installing it on Windows Server 2025.

REQ-G: state MySQL 8.0 as the standard for new installs; 5.7+/5.6 remain
supported on an existing server.

Lockfiles regenerated with uv pip compile. Production deps 44 -> 38.
 2026-08-02 14:15:18 -04:00
cproudlock
6639afd1f0 map: drop the marker popup that was never meant to be reached
Some checks failed
CI / backend (push) Failing after 1m55s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Each marker bound both a hover tooltip and a click popup. On the map page the
click handler routes to the detail page, so the popup opened and the
navigation discarded it in the same tick - it was never visible. Where it did
render (the map editor and the picker forms) its 'View Details' link only
served to pull the user off an unsaved form.

Keep hover as a glance and leave the click to the consumer. Removes the popup
markup, its styles, and the two now-unused detail-route helpers.
 2026-07-31 10:17:17 -04:00
cproudlock
2528b7e556 map: thin the marker and legend rings 
The ring only has to separate the mark from the surface; at 2px it read as
part of the mark. Markers go to 1.25, legend dots to 1.5px (they are larger,
so the same visual weight needs slightly more), and the PDF marker and swatch
strokes drop to match.
 2026-07-31 10:12:30 -04:00
cproudlock
a7fe2c8353 fix: navigation dying after an app-pool restart 
Two independent ways a restart leaves the SPA unable to navigate, both of
which look identical to a user - a click that does nothing.

1. The router awaits loadEnabledPlugins() to gate plugin routes. An app-pool
   restart leaves that request hanging (IIS queues it while the worker starts)
   and axios sets no timeout, so the navigation never resolves. Worse, the
   promise is cached, so every later navigation awaited the same dead request
   and stayed frozen long after the backend recovered. Bound the wait and fail
   open on expiry, and drop the cached promise when an attempt times out or
   fails so the next navigation retries. The setup-state probe in the guard
   gets the same bound (it already fails open, defaulting to "complete").

2. A deploy replaces the content-hashed chunk files, so a tab open across it
   asks for chunks that no longer exist and the dynamic import rejects with
   nothing handling it. Reload once on a chunk-load error, via router.onError
   and Vite's preloadError, guarded by a sessionStorage flag against a reload
   loop and cleared on the next successful navigation.

Also stop index.html being cached: it names the hashed chunks, so a stale copy
points at files the deploy already deleted. It now revalidates while the
hashed assets under assets/ cache for a year.
 2026-07-31 10:10:51 -04:00
cproudlock
cbf90be7ec map: ring markers by surface so a theme rings every marker alike
Some checks failed
CI / backend (push) Has been cancelled
Details
CI / naming (push) Has been cancelled
Details
CI / frontend (push) Has been cancelled
Details
CI / migrations-mysql (push) Has been cancelled
Details 
Keying the ring off the fill singled out the light colors: on the map the
orange network-device marker took a black ring while its neighbours kept white
ones, and the legend dot for the same type wore a surface-colored border, so
the key did not match the markers.

Ring by the SURFACE instead - dark on the white blueprint, light on the dark
one - which is uniform within a theme and still works for every fill, since a
fill that resembles the ring is by definition far from the background. Legend
swatches take the same ring, the theme watcher redraws the markers (their ring
now depends on it), and the PDF uses the light-surface ring throughout because
it prints on white.
 2026-07-31 09:28:38 -04:00
cproudlock
c80c612922 map: markers legible on both the white and the dark blueprint
Some checks failed
CI / backend (push) Has been cancelled
Details
CI / naming (push) Has been cancelled
Details
CI / frontend (push) Has been cancelled
Details
CI / migrations-mysql (push) Has been cancelled
Details 
Markers sit on two very different surfaces - the light blueprint on white and
the dark blueprint over the near-black navy card - but the palette only ever
suited one. The grey "no subtype" default sat at 1.88:1 against white and the
orange asset-type step at 2.16:1, so both effectively disappeared on the light
blueprint. The fixed white ring made it worse: on white it added nothing.

Re-step the asset-type colors to versions of the SAME hues that clear 3:1
against both surfaces, replace the grey default with a neutral that clears
5.4:1 / 3.7:1, and derive the ring from the fill's luminance (light fill ->
dark ring, dark fill -> light ring) so every marker keeps a hard edge on
either background. The ring also rescues a washed-out color a user picks by
hand for a subtype, which no palette change can reach. The PDF export applies
the same rule to its markers and legend swatches, keeping print in parity.

Colors were chosen against a contrast/CVD validator rather than by eye. Note
that five simultaneous hues cannot all stay distinguishable under color-blind
simulation - past roughly five subtypes on screen, the legend and the hover
tooltip carry identity.

Adds computed contrast assertions so a future palette edit cannot
reintroduce a washed-out step.
 2026-07-31 09:03:24 -04:00
cproudlock
d5635a4306 map: fix PDF export 404 on the blueprint under a subpath mount 
exportPdf passed the raw map_blueprint_light setting value, which is a
root-relative /api path. Under /ops or /shopdb that resolves to the server
root and 404s, so the export died with "Failed to load blueprint image".
The on-screen map was unaffected because it goes through blueprintUrlFor,
which applies withBase - use that here too.

Same file, so this also carries the subtype auto-palette replacement that
goes with the marker-legibility change in the next commit.
 2026-07-31 09:03:14 -04:00
cproudlock
b16f143467 search: asset lists search the type column they display 
Every asset list shows a Type column (and printers a Model, machines and
network a Vendor), but the search filters only looked at the asset number,
name, serial and hostname. Searching a type returned zero rows: 'Part Washer'
on machines, 'Standard' on PCs, 'Thermal' on printers.

Extend the search on machines, computers, printers, network devices,
measuring tools and the unified asset list to cover the type name plus the
vendor/model where the list shows them. Joins are outer joins so an asset
missing a type or vendor still matches on its own fields; the core list uses
a correlated EXISTS instead, since its type-name filter already joins
AssetType.
 2026-07-31 09:02:51 -04:00
cproudlock
71982fc0f1 map: fix subtype filter dropping every measuring tool
Some checks failed
CI / backend (push) Has been cancelled
Details
CI / naming (push) Has been cancelled
Details
CI / frontend (push) Has been cancelled
Details
CI / migrations-mysql (push) Has been cancelled
Details 
MapView carried its own copy of the per-type subtype-id lookup and it
never gained a Measuring Tool branch, so selecting any measuring-tool
subtype filtered out all assets. Marker coloring and the PDF export were
unaffected because both already used the shared getSubtypeId helper.

Point the filter at that shared helper and delete the duplicate copy in
ShopFloorMap too, so one definition serves filter, coloring and export.
Adds a table-driven spec covering every subtype-carrying asset type.
 2026-07-31 07:56:09 -04:00
cproudlock
86697a4e7b docs: remove WIKI-UPDATE-PLAN.md (executed) 2026-07-30 16:07:37 -04:00
cproudlock
802256f929 docs: wiki update for API docs, printer installer, geenforce cutover, timezone 
Execute WIKI-UPDATE-PLAN.md (14 items):
- NEW docs/PRINTER-INSTALLER.md: install-list / pc-default / install-batch
  contract + public installer map page.
- NEW-shape docs/API-REFERENCE.md: index + pointer to the live generated docs
  (/api/docs Redoc, openapi.json, llms.txt, MCP), replacing a stale full dump.
- geenforce cutover + GE-ENFORCE-DISPLAY/CLIENT/DEPLOY: server-first display
  dispatcher (display-role by FQDN, display-type.txt fallback), dashboarddefaults
  FQDN keying, legacy kiosk autostart self-heal (Wow6432Node), per-PC-type
  cutover status.
- PLUGINS: printers/slides rows + plugin-permissions note (slides.manage).
- IMPORT-API: dashboarddefaults FQDN-first keying.
- CONFIG: word-wise search, site_timezone setting.
- PILOT-DEPLOY: servers-to-network reclassify step. IMPORT-ADOPTION: fixup note.
- CLAUDE.md: test count 1077->1159, HTTPS-cutover state. CHANGELOG: timezone +
  kiosk-autostart fixes, site_timezone setting.
 2026-07-30 16:05:21 -04:00
cproudlock
af6bcd4726 geenforce display dispatcher: purge legacy autostart in Wow6432Node + all hives 
The old kiosk kept relaunching the dead URL from an HKLM Run value the 32-bit
Inno installer wrote - WOW64-redirected into SOFTWARE\Wow6432Node, which 64-bit
tooling (and the earlier purge) never saw. Broaden the sweep to both registry
views, every loaded user hive, Run/RunOnce/Policies-Explorer-Run, matching by
legacy name AND by any value pointing at the old URLs, plus every per-user and
common Startup folder.
 2026-07-30 15:54:18 -04:00
cproudlock
01a545f507 geenforce display dispatcher: revert to direct Edge shortcut, drop VBS launcher 
The white-on-login was the old Dashboard/Lobby installer's leftover autostart
relaunching the dead old URL (404 -> white), not a network race - so the
wait-for-URL launcher solved the wrong problem. Go back to the plain direct
Edge kiosk shortcut and clean up any stale launcher file. The real fix (the
legacy HKLM Run-key + old .lnk purge) stays; it just has to be published.
 2026-07-30 15:37:26 -04:00
cproudlock
95df51fddf geenforce display dispatcher: wait for kiosk URL before launching Edge 
At auto-login the Startup shortcut fired before the network was up, so Edge
--kiosk navigated to nothing and sat on a blank white page with no retry.
Point the shortcut at a hidden VBS launcher (wscript, no console flash) that
polls the kiosk URL until it responds (up to ~3 min) and only then launches
Edge fullscreen, so the first paint is the real page. Falls through to launch
anyway after the timeout so a display is never left dark.
 2026-07-30 15:33:49 -04:00
cproudlock
ea6fae91c3 notifications: correct timezone handling + configurable site timezone 
Notification start/end times displayed and stored wrong by the tz offset
(a 2:34 PM entry showed 6:34 PM). Two stacked bugs: to_dict emitted stored
UTC as naive ISO (no offset) so the browser read it as local, and the form
filled the datetime-local input from toISOString() (UTC).

Fix and generalize to a configurable site timezone (multi-site):
- New setting site_timezone (default America/New_York), public, editable in
  Settings > Site > Localization (common-zone dropdown).
- Backend tags datetimes UTC (_utc_iso); parse normalizes to naive UTC
  (_parse_utc); daily-reset expiry uses the site zone (_next_site_time);
  calendar allDay events key off the site-local day (_site_date).
- Shared frontend util datetime.js (Intl-based, DST-safe) converts between a
  UTC instant and a site-zone wall clock. Notification form, list, and
  calendar all render/enter in the site zone.
 2026-07-30 15:08:59 -04:00
cproudlock
3ad26ba010 export-github: exclude mcp/ from public publication 
The MCP server README names Claude Desktop / Claude Code (it is an MCP
server for those clients), which trips the scrub gate. It is a standalone
local tool distributed via pxe-images/mcp/ + setup-mcp.cmd, not part of the
shipped product, so exclude it from the public repo like docs/ and tools/.
 2026-07-30 14:30:12 -04:00
cproudlock
e6533d5205 geenforce display dispatcher: purge legacy HKLM Run autostart 
The old LobbyDisplay/Dashboard Inno installers planted an HKLM
...\CurrentVersion\Run value (plus a Startup .lnk). The dispatcher already
swept the stale .lnk/.url launchers but never the Run value, so a display
with our new ShopDB Kiosk.lnk still relaunched the old kiosk URL at logon
(the Run key beats the Startup shortcut). Remove the two legacy Run values
and kill any running old-URL Edge so the display self-heals to the resolved
target on the next enforce cycle.
 2026-07-30 14:28:44 -04:00
cproudlock
516e33a6c3 geenforce display dispatcher: kiosk .lnk uses only --kiosk + --edge-kiosk-type=fullscreen (drop the extra Edge flags) 2026-07-30 13:53:04 -04:00
cproudlock
5f02e488eb mcp: resolve openapi.json from env/in-repo/same-dir so a standalone copy works 2026-07-30 09:30:39 -04:00
cproudlock
f0b5465917 mcp: read-only ShopDB MCP server generated from the OpenAPI spec 
A separate tool (not shipped in the app) that exposes a curated set of read
endpoints as MCP tools, so an LLM client can query the asset DB directly. Built
with FastMCP.from_openapi over docs/openapi.json; auth via a scoped PAT
(SHOPDB_TOKEN) or managed X-API-Key. Read-only: only GETs on the curated
allowlist become tools, all writes excluded. Runs anywhere that can reach the
API - never on the air-gapped box. Needs `pip install fastmcp` + testing in that
env (not installed in this repo's venv).
 2026-07-30 07:52:53 -04:00
cproudlock
b507884ad6 api: serve interactive OpenAPI docs at /api/docs (offline) + llms.txt 
Generate docs/openapi.json (3.1, 362 operations) from the API inventory via
scripts/gen_openapi.py, and serve it with a self-hosted Redoc bundle at
/api/docs - no CDN, works on the air-gapped box. Also serve docs/llms.txt (a
concise LLM entrypoint) at /api/docs/llms.txt. New core 'docs' blueprint;
staticdocs/ excluded from the naming check (vendored minified JS).
 2026-07-30 07:51:12 -04:00
cproudlock
8575837d8e docs: add project health review, wiki update plan, API reference (Fable review) 2026-07-30 07:51:12 -04:00
cproudlock
ecf4ef6edd scripts: match servers by name prefix (SVR-) as well as computer type 2026-07-30 07:12:51 -04:00
cproudlock
346c428409 scripts: reclassify server 'computer' assets to network_device 
Servers were imported as computers (a PC type) so they show under PCs, not
Network. This one-shot re-points each server's asset in place - assetid is
unchanged, so comms/relationships/map/name/location/audit all carry over; only
the extension row is swapped (computers -> networkdevices), the asset type is
flipped, and the device gets the 'Server' networkdevicetype (created if absent).

Identify servers by their computer type name (--type, default 'Server'). Dry-run
by default; --commit applies. Run on the target instance.
 2026-07-30 07:10:16 -04:00
cproudlock
c075658ca6 printers: drop the trailing pause in the install .bat so it self-closes 2026-07-29 15:34:07 -04:00
cproudlock
cd79e610e9 printers: make the installer map public (no login) 
The /printer-installer map only reads the public install-list and downloads
the install .bat - both jwt-optional endpoints - so requiring auth was an
unnecessary gate. Drop requiresAuth; it now matches the other display/kiosk
tools (public).
 2026-07-29 15:16:16 -04:00
cproudlock
ad84c9060a printers: add format=text to install-list + pc-default; vendor via model 
The Inno printer installers hand-parsed JSON in Pascal (brittle brace-counting).
Add ?format=text to install-list (one printer per line, pipe-delimited:
printerid|windowsname|vendorname|modelnumber|hostname|ipaddress|mapx|mapy) and
to pc-default (printerid|windowsname), so the installer side is a split() with
no JSON parser. The web map keeps the default JSON.

Also resolve install-list's vendorname via the model (as the batch already does),
since the import sets the model, not the printer's direct vendorid - otherwise
the installers' HP/Xerox/Brother filter drops every prod printer.
 2026-07-29 13:51:01 -04:00
cproudlock
cb075a278f reports: pc-relationships matches PC<->machine links in either direction 
Prod had 331 relationships, 268 computers, 204 machines, but the report came
back empty. The query only matched computer(source) -> machine(target), while
the import stores the general machinerelationships as machine(source) ->
PC(target) (only the synthetic measuring-tool links are PC -> tool). So the real
shop-floor edges never matched.

Make the query direction-agnostic (UNION of both orientations); a PC-runs-machine
report is conceptually undirected. Also drop the comtypeid=1 filter so the IP is
taken from the primary communication regardless of its type.

Test: a machine(source) -> PC(target) edge now appears in the report.
 2026-07-29 13:16:15 -04:00
cproudlock
3eaaee0e50 printers: resolve installer vendor via the model + fix batch download base URL 
Two fixes for the printer install-batch on prod data:

1. Vendor was read only from the printer's direct vendorid, which the legacy
   import never sets (it sets the model; legacy resolved vendor through the
   model). Every prod printer came back vendor "unknown", so all fell into the
   manual group and the universal PrinterInstaller.exe block never emitted. Now
   resolve vendor via the model's vendor when the printer has no direct one, as
   the classic installprinter.asp did.

2. Harden the download base URL. Behind IIS the app sees http on a loopback
   port and url_root drops the /shopdb mount, giving a broken download URL when
   site_base_url is unset. Fall back to https + the forwarded Host + script_root.

Test: a printer with no vendorid but an HP/Xerox model now groups universal.
 2026-07-29 12:56:31 -04:00
cproudlock
0d40780f53 printers: printer installer map + install-batch endpoint 
Rebuilds the classic printer-installer feature: pick printers on the shopfloor
map, download a .bat that installs them.

Backend (asset_routes.py): GET /api/printers/install-batch?printerids=1,2,3
returns a .bat attachment. Groups printers the way the classic installprinter.asp
did - HP/Xerox via the universal PrinterInstaller.exe /PRINTER="a,b,c", printers
with a .exe installpath via that installer /SILENT, and anything else (no
installpath, or a .zip) listed for manual install instead of being run blindly.
Download URLs derive from the site_base_url setting + the IIS-served /installers
folder (no hardcoded host). Reuses the existing install-list query shape.

Frontend: PrinterInstallerMap.vue - full-screen Leaflet shopfloor map (reuses
mapConfig), a marker per network printer at its mapx/mapy, click to toggle-select,
sidebar with the selection + an Install button that downloads the batch. Toplevel
route /printer-installer, printersApi.installList(), and an Installer Map button
on the printers list.

Tests: install-batch grouping (universal/specific/manual) + requires-ids.
 2026-07-29 12:38:51 -04:00
cproudlock
1ee9328bf9 applications: fix relative installer/link hrefs + serve /installers via IIS
Some checks failed
CI / backend (push) Failing after 1m53s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 9s
Details 
Two problems with application download/launch/doc links:

1. Stored paths like 'installers/Foo.exe' are relative, so an <a href> on
   /shopdb/applications/6 resolved to /shopdb/applications/installers/Foo.exe.
   New basePath.fileHref() mounts a relative path under the app base
   (-> /shopdb/installers/Foo.exe) while leaving full URLs and UNC/file paths
   untouched. Applied to installpath, applicationlink, and documentationpath in
   the list and detail views.

2. Even the correct /shopdb/installers/Foo.exe 404s: httpPlatformHandler is
   path="*", so IIS forwards it to Flask, which has no such route. Add a
   web.config <location path="installers"> that clears the handler and serves
   that subpath as IIS static (with .exe/.msi MIME), from a physical
   APP_ROOT\installers folder.
 2026-07-29 10:25:46 -04:00
cproudlock
071d40488b chore: sync package-lock for dompurify direct dependency
Some checks failed
CI / backend (push) Failing after 1m53s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details
2026-07-29 10:10:35 -04:00
cproudlock
bf8842e1d7 applications: render Application Notes as sanitized HTML
Some checks failed
CI / backend (push) Failing after 1m55s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
The notes field is authored as HTML (the form says "HTML supported") but the
detail page interpolated it with {{ }}, so tags like <BR> showed as literal
text. Render via v-html through a DOMPurify sanitizer (utils/sanitizeHtml):
allow-list of formatting tags + links only, forces target=_blank
rel=noopener on links, strips scripts/handlers. Promote dompurify to a direct
dependency (was transitive via jspdf).
 2026-07-29 10:06:18 -04:00
cproudlock
ced356882c net: strip the ephemeral source port from the forwarded client IP 
IIS ARR sets X-Forwarded-For to clientip:port, and the port changes every
connection. Left in, the audit log showed IP:PORT, the dashboard IP fallback
never matched a stored (portless) DashboardDefault.ipaddress, and login rate
limiting keyed per-connection instead of per-host. Add an IPv6-safe
clientip.client_ip / strip_port helper and use it in the audit log, the
dashboard resolver, and the login rate-limit key.
 2026-07-29 10:06:18 -04:00
cproudlock
8dce622392 knowledgebase: include the topic (application name) in list search 
The KB list search matched only shortdescription + keywords, so searching a
topic (e.g. "Spotfire", the Application name) surfaced just the one article
whose title/keywords contained the word, not the others tied to it by topic.
Match the topic too via an appid IN (apps named like the term) subquery - used
instead of a join so it does not collide with the sort=topic join, and articles
with no app still match on title/keywords.
 2026-07-29 10:06:18 -04:00
cproudlock
c5ee164565 ui: format application knowledge-base card (was a wall of text)
Some checks failed
CI / backend (push) Failing after 1m53s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
The .kb-* classes had no styles, so the KB entries rendered as bare inline
spans - shortdescription and keywords (both up to 500 chars) wrapped and mashed
into one block. Style each entry as a bordered clickable card: description as
the link title clamped to 2 lines, keywords split on whitespace into small
muted chips below.
 2026-07-29 09:26:58 -04:00
cproudlock
b63690996a fix: location dropdowns rendered blank (wrong field) + require printer model
Some checks failed
CI / backend (push) Failing after 1m52s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
The location option label read l.location, but the Location.to_dict() field is
locationname, so every option rendered blank - the dropdown looked empty and
"massive" (a long list of blank rows). Fixed across all five affected forms:
printers, computers, network devices, network device form, and the subnets
location filter. Other .location uses (printer-driver URL, search-result label,
report bylocation key) are legitimately different fields, left alone.

Also require a model on the printer form: asterisk + required attr, plus a JS
guard in savePrinter (the native required is skipped while the select is
disabled with no vendor picked) that points the user at the vendor first.
 2026-07-29 09:07:43 -04:00
cproudlock
b9d0cfac6a ui: stop audit-log columns clipping + widen ge-enforce report modal
Some checks failed
CI / backend (push) Failing after 1m53s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
AuditLogs: the scoped table-layout:fixed + width:100% forced the table to fit
the settings pane, so cells ellipsis-clipped (Timestamp/User/IP fell off) rather
than scrolling. Drop it so columns size to content and the container scrolls
horizontally (global .table-container is overflow-x:auto). Only the free-form
Name/ID cell stays bounded (320px + title tooltip) so one long value cannot blow
the table width out.

EnforcementReports: the per-entry detail modal capped at 640px, too narrow for
the 5-column table. Widen to min(1000px, 92vw) and let the Message column wrap
instead of forcing horizontal scroll inside the modal.
 2026-07-29 08:55:47 -04:00
cproudlock
174c6c0b9a slides: gate management on slides.manage permission (grantable to non-admin curator)
Some checks failed
CI / backend (push) Failing after 1m54s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 10s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
The lobby-display and screensaver slide manager was admin-only. Add a shared
slides.manage permission so a curator can manage both surfaces without full
admin. Admins keep access via the require_permission admin bypass.

Backend:
- plugins/slides/api/routes.py: all 5 management routes require slides.manage
- plugins/slides/plugin.py: declare it via get_permissions(); nav item carries
  the permission so the frontend can gate visibility
- shopdb/core/api/auth.py: login response now returns the user's permissions
  (matches /me) so the frontend authStore has them on fresh login

Frontend:
- stores/auth.js: hasPermission(name) getter (admin true, else granted list)
- router/index.js: guard supports requiresPermission
- views/AppLayout.vue: hide nav items whose permission the user lacks
- plugins/slides/frontend/routes.js: slide manager gated requiresPermission

Tests: no-perm user 403, curator role with the perm 200 (+ login advertises
it), admin 200 via bypass.

Deploy: run `flask seed permissions` to create the row, then grant it to a
role in Settings > Users & Roles.
 2026-07-29 08:41:11 -04:00
cproudlock
7a7c7f37d5 search: multi-word queries match by word, not the exact phrase 
Global search did a single ilike('%CSF Roles%'), so any query with more than one
word required the exact contiguous phrase and usually returned nothing. Add
_word_match: split the query into words and AND them (OR across the searched
columns per word), so 'CSF Roles' matches a record with both words in any field,
any order. Applied across every domain (assets, applications, KB, employees
[selfhosted + external HR], notifications, hostnames, IP, custom fields,
vendor/model/type). External HR path uses a parameterized per-word LIKE.
 2026-07-29 07:43:31 -04:00
cproudlock
b22701a444 geenforce display dispatcher: resolve role from the server by FQDN, fall back to display-type.txt 
The dispatcher now derives its FQDN (F<BIOS serial>.<domain>) and asks
/api/dashboarddefaults/display-role for its role/path, so changing a display's
type/location in Settings > Dashboard Defaults takes effect with no reimage. If
there is no serial, no server mapping, or the lookup fails, it falls back to the
local display-type.txt map (offline-safe). VM-verified both paths.
 2026-07-29 07:34:21 -04:00
cproudlock
e237cc2c05 dashboarddefaults: pick a kiosk from a dropdown (auto-fill FQDN) + Location wording 
Adds GET /api/computers/display-kiosks - the displays that reported in (Kiosk
type), each with its derived FQDN (F<serial>.<domain>). The Dashboard Defaults
form gets a kiosk dropdown that fills the FQDN so admins pick a display instead
of typing an IP; IP stays an optional manual field. Table shows FQDN or IP.
'Business Unit' label -> 'Location' on this page + the settings nav.
 2026-07-29 07:29:34 -04:00
cproudlock
3ba808028c dashboarddefaults: key display mappings by stable FQDN (from BIOS serial), IP fallback 
A display's DHCP IP can change; its FQDN (F<serial>.<domain>, domain from the
display_fqdn_domain setting) is stable and the collector already reports the
serial. Add a nullable unique fqdn column (varchar191 so the index fits utf8mb4
without innodb_large_prefix), make ipaddress nullable, and require fqdn OR ip.
visitor-location + display-role resolve by FQDN first, then IP; create/update
accept fqdn. Core migration 7d31, verified up/down/idempotent on MySQL 5.6.
'Business unit' wording -> 'location' in the validation messages.
 2026-07-29 07:20:52 -04:00
cproudlock
b539e36096 test: assert display seed keeps exactly two inline payloads across rebuilds 2026-07-28 18:56:29 -04:00
cproudlock
89a1617103 geenforce: fix re-publish FK crash on MySQL (stale entries in draft rebuild) 
replace_scope_draft deleted old draft entries with per-object db.session.delete
but left the deleted objects in scope.entries. On a re-publish a caller
(seed_display_scope) then matched a stale deleted entry via next() and
store_inline_payload attached a payload to its dead entryid, failing the
manifestpayloads->manifestentries FK on MySQL (1452); SQLite does not enforce
it so the idempotency test passed. Clear the collection via the delete-orphan
cascade instead, and flush pending inserts before the bulk payload delete so its
autoflush cannot interleave a half-built insert. Verified publish + re-publish
x3 on MySQL 5.6.
 2026-07-28 18:55:17 -04:00
cproudlock
9a2d0ccebb dashboard: resolve employee names from directory/user, GE monogram photo fallback, kiosk sweep + label 
- notifications shopfloor feed: resolve the employee name live when the stored
  value is a bare SSO (WJ notifications imported as SSOs, never converted), for
  both single and split-per-employee cards
- employee name resolver: after a directory miss, fall back to the shopdb User
  account (firstname/lastname, keyed by SSO username) so users from other
  locations still show a name
- shopfloor dashboard: employee photo falls back to the GE monogram (own asset,
  independent of the site_logo setting) with a loop-guarded onerror; recognition
  + recert tiles both covered
- shopfloor dashboard: 'All Business Units' filter label -> 'All Locations'
- geenforce display dispatcher: startup sweep also matches the imaging
  installers' 'GE Aerospace Dashboard/Lobby' shortcuts by name
 2026-07-28 18:21:47 -04:00
cproudlock
3a8df166cf geenforce: broaden kiosk startup sweep to match single-dash -kiosk and shopdb-URL launchers 
The prior sweep only matched '--kiosk'; the imaging installers (Inno
GEAerospaceDashboardSetup / lobby) create Startup shortcuts with single-dash
'-kiosk' pointing at /shopdb/shopfloor-dashboard, so they survived. Match any
msedge/chrome Startup .lnk whose args contain -kiosk (one or two dashes) OR a
shopdb kiosk URL (tsgwp00525 / /shopdb/ / shopfloor-dashboard). Unrelated
Startup items are left untouched (VM-verified).
 2026-07-28 18:09:12 -04:00
cproudlock
5712f72ccf geenforce: fix http-payload path doubling + sweep stale kiosk startup shortcuts 
- Resolve-ShopdbPayloads wrote an absolute local path into the entry, and the
  engine resolves it as Join-Path InstallerRoot <field>, doubling it
  (C:\...\payloads\C:\...\payloads\<sha>.ps1 -> PS1 not found). Write the leaf
  filename instead; the runner already sets InstallerRoot to that payloads dir.
- display dispatcher now removes leftover kiosk launchers from prior installs
  (any Startup .lnk that runs Edge --kiosk, plus .url to a shopdb kiosk page),
  not just its own, so two kiosks do not fight.
 2026-07-28 17:49:47 -04:00
cproudlock
4c0cc672a2 geenforce: harden allowlist + fix share-less kiosk client and display scope 
- allowlist auth uses remote_addr, not the spoofable first X-Forwarded-For hop
  (adds _trusted_client_ip + a regression test); rate-limit path unchanged
- client psm1: fix Set-StrictMode crashes reading absent keys in Get-ShopdbConfig
  (token-less mode) and Resolve-ShopdbPayloads (no-payload entries); validate
  the manifest response is JSON before overwriting the last-known-good cache
- runner: pass the engine its required -InstallerRoot/-LogFile; create the log
  directory so enforce logging is not silently lost on a fresh kiosk
- display scope: dispatcher writes an all-users Startup shortcut instead of
  Start-Process (SYSTEM cannot show a window in session 0), resolves the base
  URL from HKLM, and adds an always-on power/no-lock entry; tests updated for
  the 6-entry scope
 2026-07-28 17:09:21 -04:00
cproudlock
f533af82cd export: --dist builds both /ops and /shopdb frontend bases 
Two instances run on the box (dev /ops + prod /shopdb); each needs its own
base-path build. Build both on every --dist so prod never ships a stale
frontend. tools/ is excluded from publication, so this is dev-tooling only.
 2026-07-28 08:28:42 -04:00
cproudlock
67de46dfb9 geenforce: import-share recognizes the display scope folder 
discover_share only matched 'common' and 'gea-shopfloor-*', so a display/
manifest.json on the share was silently skipped and 'flask geenforce publish
display' failed with 'No scope display/runtime'. The display scope is a
first-class HTTPS-pull target (kiosks fetch pctype=display), so accept it.
 2026-07-27 14:54:13 -04:00
cproudlock
d9080a59ca geenforce: move settings into the Settings rail 
The client IP allowlist config was a tab inside the GE-Enforce section; move
it to the Settings rail via get_settings_cards (matches printedparts / zabbix /
dell). Route relocated from /geenforce/settings to /settings/geenforce; the
in-section Settings tab is removed. Card: Settings > GE-Enforce.
 2026-07-27 14:44:26 -04:00
cproudlock
2d675720b7 geenforce client: make ApiToken optional for IP-allowlisted kiosks 
Get-ShopdbConfig required both BaseUrl AND ApiToken, so a token-less kiosk
(authorized by the server's IP allowlist) got a null config and never ran.
Now BaseUrl alone is a valid config; X-API-Key is sent only when a token is
present (New-ShopdbAuthHeaders), so token-authorized sites are unchanged and
vaulted-network sites need no per-PC token.
 2026-07-27 14:17:51 -04:00
cproudlock
0860aa85c5 geenforce: IP allowlist for client endpoints + admin Settings tab 
Fleet PCs on a trusted (vaulted) network can now reach the GE-Enforce client
endpoints (manifest, payload, report) without a per-PC token: the auth path
accepts a valid geenforce.fetch/report token OR a source IP in the configured
allowlist (setting geenforce_allowed_cidrs). Fail-closed; an empty allowlist
means the token stays the only path, so existing deployments are unchanged.

Rationale: the client token lives in HKLM on every kiosk, so it does not
defend against a compromised kiosk anyway - network-perimeter trust is the
same practical strength with far less provisioning + no token-rotation churn
on a DB wipe. Documented in-UI that this is perimeter trust, not per-device
identity.

- _ip_allowlisted() (ipaddress, X-Forwarded-For-aware via _client_ip)
- /geenforce/config GET/PUT extended with allowedcidrs, server-validated +
  normalized (bad CIDR -> 400)
- new GE-Enforce > Settings tab (GeEnforceSettings.vue) to edit the allowlist
  in admin, no SQL
- 3 regression tests (allow by IP, reject outside list, empty = token required)
 2026-07-27 14:06:40 -04:00
cproudlock
19876a5640 warranty: fix bulk Dell re-check duplicating non-dell warranties 
The bulk /sync/dell reuse check only matched an existing warranty when its
provider was exactly 'dell'. Warranties added by hand or via import default to
provider 'manual', so re-check-all did not recognize them and created a brand
new Dell warranty for every asset - duplicating the whole set.

Broaden the reuse match to treat a warranty as Dell by any signal (provider,
matching service tag, or a 'Dell' vendor), and canonicalize the reused row to
provider 'dell' so later re-checks match by provider and never duplicate.
 2026-07-24 13:24:36 -04:00
cproudlock
6534590fca docker: air-gapped deploy kit (image bundle + offline compose + runbook)
Some checks failed
CI / backend (push) Successful in 1m51s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 9s
Details 
Air-gapped sites cannot pip install / npm ci / docker pull, so a build-at-site
compose (build: .) fails and reports 'service api is not running'. Add a
build-once-ship-image path:

- scripts/build-offline-bundle.ps1: on a connected box, build shopdb-flask +
  pull mysql:8.0, docker save both into one gzipped tarball with a sha256.
- docker-compose.airgap.yml: runs pre-loaded images (image:, never build:),
  drops the ./plugins bind mount (which would mask the image's baked-in plugins
  with an empty host dir and load zero plugins at an image-only site), and adds
  a one-shot migrate service (db upgrade + plugin upgrade-all + seed) that api
  waits on via service_completed_successfully, so 'up -d' brings a working site.
- docs/DEPLOY-AIRGAP.md: full runbook (build, transfer+verify, load+run, admin,
  verify, upgrade, troubleshooting), incl the Zscaler in-build cert caveat.
- .env.example: IMAGE_TAG for the air-gap compose to pin the loaded image tag.
 2026-07-23 14:15:34 -04:00
cproudlock
75386d2f51 geenforce: resource-scope binding for fetch tokens (0.15.0) 
A geenforce.fetch token can now be pinned to specific manifest scopes so a
fleet-wide key (a display's, delivered by DSC or baked into the image) is not a
skeleton key for the whole content store. NULL binding = unrestricted, so every
existing service token keeps working.

Core:
- ApiToken.resourcescopes column + resourcescopelist property (migration
  7d30_apitoken_resourcescopes; NULL = unrestricted).
- apitokens API create/update accept + persist an optional resourcescopes list
  (a resource-name allowlist; not permission-catalog names).
- New contract helper authorized_service_token(scope): same check as
  service_token_authorized but returns the ApiToken so a plugin can read its
  binding. Contract 0.14.0 -> 0.15.0; also export SupportTeam.

GE-Enforce enforcement:
- get_manifest: a bound token requesting a scope outside its allowlist -> 403.
- get_payload: a bound token may only pull a blob its own scope(s) reference
  (service.blob_referenced_by_scopes); anything else -> 404 (no hash probing).
- Decorator stashes the authorized token on g for the route to read.

Also fixes a pre-existing contract-surface violation: the printers/printedparts
alert helpers imported shopdb.core.models / shopdb.extensions directly; now
via shopdb.api (SupportTeam newly exported). Docs: GE-ENFORCE-DISPLAY.md
provisioning note, PLUGIN-HOOKS.md, CLAUDE.md.

9 new resource-binding tests; full suite 1131 passing.
 2026-07-23 09:02:42 -04:00
cproudlock
d0bf37ced7 geenforce: display scope is self-sufficient, no common inheritance 
Per decision: displays need none of the fleet-wide common scope's software, so
the gea-shopfloor-display scope carries everything it enforces and does not
inherit common. This avoids repackaging common's SMB-backed payloads for a
share-less display.

- Invert the client common-merge switch: -NoCommon (default-on) becomes
  -IncludeCommon (default OFF). A scope now enforces alone unless opted in.
  The capability stays for a future share-less non-display PC; displays omit it.
- Drop the common SMB-payload audit + inheritance sections from the display
  seed comments and docs (GE-ENFORCE-DISPLAY.md); document self-sufficiency.
- GE-ENFORCE-CLIENT.md: common-scope inheritance is now opt-in.
 2026-07-23 08:22:23 -04:00
cproudlock
9d65ef103d geenforce: display-readiness batch (server hardening, PS client wiring, display scope) 
Get GE-Enforce closer to running on credential-less Intune/Entra display PCs
that pull manifest + payloads over HTTPS instead of SMB.

Server (plugins/geenforce/api/routes.py):
- Rate-limit + 512MB served-size ceiling on GET /payload/<sha256> (reuses the
  login limiter's cache pattern, config-overridable via GEENFORCE_PAYLOAD_*).
- New tests: payload hardening, manifestblobs model-vs-migration parity, and a
  report-contract test locking the lowercase per-entry report keys.

PS client (plugins/geenforce/client/):
- Fix New-ShopdbReport per-entry key casing to lowercase (name/action/selfhealed/
  exitcode/message) to match what the server reads; the engine emits PascalCase.
- Enforce TLS 1.2 in the network functions.
- Fetch + merge the fleet-wide common scope alongside the pctype scope
  (pctype wins on conflict; -NoCommon opt-out).
- Normalize whatever the engine returns into a well-formed summary.
- Make the empty-cache fail-safe observable: event-log entry + report ping
  instead of a silent exit 0.

Manifest (plugins/geenforce/seed_display_scope.py + docs/GE-ENFORCE-DISPLAY.md):
- Seed a gea-shopfloor-display scope: 4 Edge kiosk drift-heal registry entries
  + 1 data-driven dispatcher (Dashboard/Lobby/3DPrintRoom via display-type.txt).
  Kiosk EXEs stay image-baked; the manifest heals policy/config drift only.
- Documents the common SMB-payload audit (entries needing http/inline before a
  share-less display can inherit common).

Migration registry (shopdb/plugins/alembic_template.py + test):
- Register the pre-existing manifestblobs and the new printersupplyalerts tables
  in PLUGIN_TABLE_OWNERS; update EXPECTED_HEAD_REVISION for geenforce (0002blobs),
  printers (0002supplyalerts), and printedparts (0004txnrev) which had drifted.
 2026-07-23 08:16:38 -04:00
cproudlock
b211e817d5 printers: low-toner alerts with configurable thresholds + support-team routing 
Poll Zabbix for toner levels on a schedule and email/webhook on a downward
crossing. Warning fires at or below the warning threshold (default 5%),
critical at the critical threshold (default 0%); both thresholds are settings.
State lives in printersupplyalerts so an alert fires once per crossing and
re-arms after a refill.

Recipients mirror the printedparts pattern: plugin-scoped shopdb users +
roles + free-text emails (falling back to the site alert_recipients), and a
chosen support team's webhook (falling back to the site alert_webhook_url).

- PrinterSupplyAlert model + migration printers0002supplyalerts
- alerttier(remaining, warning, critical) + check_supplies poller
- flask printers check-toner-alerts CLI (run via scheduled task/cron)
- printers alert settings + Low-Toner Alerts settings page
- 7 tests: tier boundaries, once-per-crossing + re-arm, toner-only scope,
  custom thresholds, support-team webhook routing
 2026-07-22 14:47:59 -04:00
cproudlock
fb188fd302 alerts: per-support-team webhook; printedparts routes low-stock to a chosen team 
Support teams gain a webhookurl (migration 7d29 + API + settings-page field), so
a team is a notification target. send_webhook(url=) lets a caller override the
site default with a team's webhook. Printedparts gains a 'alert support team'
setting (printedparts_alert_supportteamid) + selector on its settings page;
low-stock alerts post to that team's webhook, falling back to the site
alert_webhook_url. Email leg unchanged. Same pattern extends to other alerting
plugins (printers low-toner next).
 2026-07-22 13:32:00 -04:00
cproudlock
824863a95a settings UI: add Site Base URL + Alert Webhook URL/format fields 
The webhook + site_base_url settings existed in the backend but the Email
Settings page had no inputs (hardcoded fields), and the settings composable's
reactive map didn't include the keys so they never loaded. Add the three inputs
(site base URL, webhook URL, webhook format select) and register the keys.
 2026-07-22 12:55:50 -04:00
cproudlock
38c7ec347b alerts: Teams webhook fan-out (contract 0.14.0) + printedparts detail revision column 
send_webhook(title,text) posts alerts to an optional webhook (Teams Incoming
Webhook / Workflow, or generic JSON) via alert_webhook_url + alert_webhook_format
settings; send_alert fans out to it alongside email; exposed on shopdb.api
(0.13.0->0.14.0, PLUGIN-HOOKS synced); low-stock posts on its custom-recipient
path too. Also: recent-transactions table shows the consumed print-file revision.
 2026-07-22 11:01:53 -04:00
cproudlock
d141fef203 printedparts: show revision at kiosk; low-stock email uses gage tag + item link 
Kiosk now displays the scanned revision (badge, quantity, and done screens) so
the operator sees which revision they checked out - it was already recorded on
the take, just not shown. Low-stock email now uses the gage lab tag (was the
internal item code) and links to the item page when site_base_url is set (new
core setting, category email; emails have no request context to derive the URL).
 2026-07-22 10:12:51 -04:00
cproudlock
baf6862151 printedparts: QR label with gage tag + revision; record taken rev 
Label switches from CODE128 to a QR encoding 'TAG|rev' (gage lab tag + latest
print-file revision), so a physical part carries which revision it was printed
from - short payload stays low-version + reliable at 0.5in (margin quiet zone,
EC M, no logo). Item exposes latestrevision; kiosk strips the |rev to resolve
and records the scanned revision on the take (migration 0004 adds
printeditemtransactions.revision) for traceability of which rev was consumed.
Manual entry records a null revision.
 2026-07-22 07:52:17 -04:00
cproudlock
e85553e5e7 notifications: show start/end date fields for ALL types 
The time fields were hidden for the employee types (recognition, recert); show
them for every type. Backend already honors start/end on create and update and
only auto-fills the end when left blank (recognition = next 8 AM, recert = two
weeks), so nothing server-side changes.
 2026-07-21 11:09:19 -04:00
cproudlock
0bb906a37c notifications: let Recognition set start/end dates; geenforce B2 client payload fetch 
Recognition edit hid the time fields (grouped with Recertification), so start/end
could not be adjusted even though the backend honors them. Show the time fields
for every type except Recertification (due-date driven); Recognition end still
auto-fills to the next 8 AM reset when blank.

Also GE-Enforce B2 client (HTTPS payload consume): ShopdbEnforceClient.psm1 gains
Get-ShopdbPayload (fetch by sha256, verify, cache) + Resolve-ShopdbPayloads
(rewrite http/inline entries to local staged files so the engine installs from
local, no SMB); Invoke-ShopdbEnforce resolves payloads before running the engine;
importer parses PayloadSource/PayloadSha256/PayloadRef. VM-verified: a SYSTEM
Windows client fetched a payload over HTTP by hash, hash matched.
 2026-07-21 10:56:00 -04:00
cproudlock
b00ef72581 geenforce: HTTPS payload delivery (content-addressed blob store + endpoint) 
Lets share-less (Intune/local-account) PCs pull installers the manifest
references over HTTPS instead of SMB - the general capability the whole fleet
migrates toward. New ManifestBlob registry (migration 0002) with bytes on disk
at instance/geenforce/payloads/<sha256> (deduped by content); service.store_blob
+ blob_path; client-facing GET /api/geenforce/payload/<sha256> (geenforce.fetch
token, ETag=hash, serves the blob store or an inline DB payload by hash). The
serializer now emits PayloadSource/PayloadSha256/PayloadRef for http/inline
entries only (smb entries round-trip unchanged - parity green). CLI
'flask geenforce add-payload <file>' registers a blob and prints its sha256.
This is the shopdb half (B1); the PS client/engine fetch is B2.
 2026-07-21 10:10:59 -04:00
cproudlock
60e2947fc7 displays: single display type with IP-driven role (dashboard/lobby/kiosk) 
One 'display' image resolves what it shows from its own IP, like the existing
visitor-location BU mapping. Extend DashboardDefault with displayrole
(dashboard|lobby|partskiosk; migration 7d28, businessunitid now nullable since
only the dashboard role needs one) + a role->path map. New unauthenticated
GET /api/dashboarddefaults/display-role returns {role, path, businessunitid}
for the caller IP. Settings UI gains a Display selector, showing the business
unit only for the dashboard role.
 2026-07-21 09:49:14 -04:00
cproudlock
b05fa33278 shopfloor dashboard: Verdana font + full left-edge type-color bar 
Switch the dashboard to Verdana (system font on the Windows kiosks - built for
on-screen distance reading, zero bundle) and drop the Archivo package. Restyle
the event card's type-color indicator to span the whole left edge of the card
(old-site style) instead of a small pill.
 2026-07-20 16:01:58 -04:00
cproudlock
eb1c46c053 shopfloor dashboard: fix recert-name height so long names don't shift the grid 
A wrapped (2-line) name made its tile taller and shifted the whole recert grid.
Reserve a fixed two-line height on .recert-name (line-clamp 2 + ellipsis) so
every tile is the same height regardless of name length.
 2026-07-20 15:40:28 -04:00
cproudlock
d39d34b55f shopfloor dashboard: use Archivo display font for TV legibility 
Bundle @fontsource-variable/archivo (air-gap safe) and apply it to the shopfloor
dashboard only - a sturdy grotesque built for signage/displays, more legible
from across the shop than Inter. Rest of the app stays on Inter.
 2026-07-20 15:19:28 -04:00
cproudlock
0dc4dba265 shopfloor dashboard: bigger text for distance + footer at bottom 
Shrink the fit surface to 16:9 1600x900 so the fit-scaler upscales the whole
board ~1.2x (more readable from across the shop) and it fills a 1080p TV exactly.
Footer was position:fixed inside a transformed ancestor so it floated at the
content bottom; make it in-flow with margin-top:auto (flex column) so it pins to
the bottom of the surface = the screen bottom.
 2026-07-20 15:16:26 -04:00
cproudlock
482d6d4bfe shopfloor dashboard: fit-to-viewport scaling for TV kiosks 
The board was a fixed-pixel layout with an internal overflow-y scroll, so on a
TV (no scrolling) content past the fold was unreachable. Wrap it in a fixed
1920-wide surface and scale it to fill the viewport (ResizeObserver + resize),
so the whole board is visible edge-to-edge at any resolution. Drop the
.dashboard-content max-height/overflow scroll.
 2026-07-20 14:49:33 -04:00
cproudlock
a50e5b0ec1 slides: fix manager thumbnail 404 under /ops + single-column order list 
SlideManager rendered <img :src=slide.url> raw, so the root-relative
/api/slides/img/... path 404'd under the /ops subpath mount (the /tv display
already wrapped withBase; the manager did not). Wrap the thumbnail in withBase.
Also switch the multi-column grid to a single-column list with order numbers so
the top-to-bottom play order is clear to arrange.
 2026-07-20 14:16:12 -04:00
cproudlock
8496441ddc slides: add /screensaver route for the shopfloor slide surface 
TVDashboard hardcoded surface=lobby, so only the lobby display was reachable.
Read the surface from route meta (/screensaver -> shopfloor) or a ?surface=
query, defaulting to lobby. Adds a /screensaver toplevel route so the shopfloor
screensaver surface can be displayed on a kiosk.
 2026-07-20 13:16:31 -04:00
cproudlock
18f027c951 printedparts kiosk: show gage lab tag (not item code) on the SSO prompt 
The badge step card still showed the internal item code; show the gage lab tag
(fallback to item code) to match the label and list. The take still posts the
item code as the stable identifier.
 2026-07-20 11:34:12 -04:00
cproudlock
108d4d396c printedparts: kiosk tag prefix + labels use the gage lab tag 
Kiosk item lookup shows a fixed "WJ" prefix addon so operators type only the
number off the label. The label page - and the detail "Part Label" button,
renamed from "Bin Label" - now barcodes and prints the gage lab tag, falling
back to the internal item code when a part has no tag assigned.
 2026-07-20 11:18:16 -04:00
cproudlock
31139267d1 Fix model images 404 under a subpath mount (withBase) 
Model image URLs are root-relative (/api/models/image/...), so on an /ops
subpath deploy the raw <img src> resolved to the server root and 404'd. Wrap
every model-image src in withBase(): machine/printer/PC/network detail heroes,
the models settings preview, and the machine-badge / asset-label print pages.
withBase leaves external http(s)/data URLs untouched.
 2026-07-20 11:14:00 -04:00
cproudlock
91fa3b9115 printedparts kiosk: one visible entry input for scanner and keyboard 
The bin step hid entry behind a "Type the number" link, and a plugged-in
keyboard could not drive the visible fields. Replace the hidden wedge input
with one visible, always-focused input per step that a wedge scanner, a
physical keyboard/numpad, and the on-screen keypad all feed; Enter submits.
inputmode="none" keeps the OS soft keyboard from popping on a touchscreen.
 2026-07-20 11:14:00 -04:00
cproudlock
89b3156ea1 printedparts: show gage lab tag in the catalog list 
The list showed the internal auto-minted itemcode; the gage lab works from the
WJRP gage lab tag. Show gagelabtag as the primary identifier, falling back to
itemcode when a row has no tag assigned.
 2026-07-20 11:03:32 -04:00
cproudlock
a7ff882e21 Add per-role badge colors 
Role badges rendered gray for everything except admin, with no way to tell
roles apart. Add an optional color per role, matching how statuses and types
carry one: new roles.color column (migration 7d27_roles_color), color threaded
through the role API and the user serializer, and a ColorSwatchPicker in the
role editor. Badges use the role's color with contrast-aware text and fall
back to the old admin/gray classes when unset.
 2026-07-20 10:05:58 -04:00
cproudlock
0b247ed96f CI: run GitHub Actions on self-hosted arc-runner-set 
The org IP allow list blocks GitHub-hosted runner IPs (checkout 403), so
point all jobs at the self-hosted arc-runner-set. Drop the rsync dependency
in build-site.sh (cp + bytecode prune; the ARC runner image has no rsync)
and remove the migrations-mysql job - ARC/Kubernetes has no service
containers, so that MySQL 8 coverage stays on the internal CI.
 2026-07-20 10:05:58 -04:00
cproudlock
3c830244f8 CLAUDE.md: refresh state header (1077 tests, 13 plugins, 16 WJF stages, lean-build/ADR-014) 2026-07-19 13:18:52 -04:00
cproudlock
e005d1846a docs: wiki staleness sweep (Fable-orchestrated Opus audit) 
Audited all 40 docs/ against the live codebase; fixed factual staleness in 23,
14 were clean. Highlights (all verified against code):
- equipment -> machines (ADR-011 rename) in INSTALL/DEPLOY-WINDOWS-IIS,
  PLUGIN-GUIDE, GE-ENFORCE, ROADMAP.
- Versions refreshed: contract 0.10.0 -> 0.13.0, product 0.5.0 -> 0.7.0, plus
  plugin example core_version pins.
- Bundled set corrected to the current 13 (PLUGINS.md 7 -> 13 rows; DEPLOY
  eleven -> thirteen).
- Per-plugin Alembic chain workflow (ADR-008) replacing stale core-chain steps
  in PLUGIN-QUICKSTART / BACKUP-RESTORE; deploy adds plugin upgrade-all.
- Frontend plugin staging (ADR-010) replacing 'no frontend plugin system yet'
  in PLUGIN-GUIDE; view/route paths repointed to plugins/<name>/frontend/.
- Corrected file paths (MapView.vue, manifest_schema.json), CLI (shelf-list),
  API gating (GET /api/plugins is optional-jwt), WJF 15 -> 16 stages, and
  retired Collector/PC-Types settings pages (ADR-012).
- ge-enforce proposal marked ACCEPTED/built.
 2026-07-19 12:54:53 -04:00
cproudlock
49a0206b9f docs: lean-build behavior + nav placement + fix stale prod plugin list 
- PLUGINS.md: new 'Lean per-site builds' section (backend/frontend/DB layers,
  manifest-less core frontends always ship, menus gated to staged routes).
- PLUGIN-HOOKS.md: get_navigation_items sidebar placement (position ranges ->
  Assets/Information sections, section override, icon key).
- DEPLOY-WINDOWS-IIS.md: fix stale plugin list (equipment -> machines, complete
  the bundled set), add apply-profile + prune-schema flow.
 2026-07-19 12:35:43 -04:00
cproudlock
212165befd Lean build: gate Shopfloor Dashboard on the notifications plugin 
Shopfloor Dashboard is a core view but its content is entirely
notificationsApi.getShopfloor() + the calendar (both owned by the notifications
plugin). Without notifications the display is empty, so gate the Displays link
on the notifications/calendar route being staged. On a site without it the link
- and the Displays header when nothing else is present - drops.
 2026-07-19 12:25:10 -04:00
cproudlock
a5ba973974 Lean build: gate Displays links by staged route, not plugin-enabled state 
The Displays section is hardcoded in AppLayout (not plugin nav). TV Slideshow
(/tv, slides) had no gate at all and Parts Kiosk (/parts-kiosk, printedparts)
was gated on isPluginEnabled - which a registry copied from a full site reports
true even when the plugin was never staged, so both showed on a lean site and
dead-ended blank. Now each Displays link (Shopfloor, TV Slideshow, Parts Kiosk)
is gated by whether its route is registered in this build (router.getRoutes),
and the Displays header hides when none are present. Route existence is the true
'is it in this build' test.
 2026-07-19 12:22:15 -04:00
cproudlock
5d86bc86b3 Lean build: hide settings cards whose route was not staged 
settingsNav.js hardcodes plugin settings (PC Access Protocols, Machine Types,
VLANs, Employee Directory, ...). A lean per-site build only stages the chosen
plugins' settings routes, so the settings rail showed cards for absent plugins
that dead-ended on a blank page. useSettingsCatalog now filters the catalog to
cards whose target route is registered in this build's router (router.getRoutes),
dropping now-empty groups. Generic - gates every settings card by staged routes
with no per-plugin logic; full builds keep every card. Found testing a live
machines+printers lean site.
 2026-07-19 12:17:51 -04:00
cproudlock
d009ac94fb Lean build: always ship core (manifest-less) frontends 
A frontend dir under plugins/ with no manifest.json is a CORE feature, not a
per-site plugin - applications is one (backend is shopdb/core/api/applications.py,
nav is advertised as core in dashboard.py). stage-frontend.mjs treated it like
a plugin and dropped it under SITE_PLUGINS, so a lean site showed the core
Applications nav item but had no route for it -> blank page. Now manifest-less
frontends always stage regardless of SITE_PLUGINS; SITE_PLUGINS selection applies
only to real plugins. CI lean-build job asserts ApplicationsList ships in a lean
bundle. Found while testing a live machines+printers lean site.
 2026-07-19 12:10:40 -04:00
cproudlock
c386e211df ADR-014 Phase 2: flask plugin prune-schema for lean per-site DBs 
A lean site still gets every plugin's tables from the shared core Alembic
baseline. prune-schema drops the tables of plugins not installed on this
site, leaving core + chosen-plugin tables, with no edit to any released
migration (the relocate-into-plugin-baselines alternative would mean
rewriting ~15 released core migrations for a cosmetic gain - see ADR-014).

- shopdb/plugins/cli.py: prune-schema command. Dry-run by default; --yes to
  execute; refuses non-empty tables without --force. Drops by table name (no
  plugin import) so it works on a lean image. MySQL: private AUTOCOMMIT engine
  (db.engine's pooled connections sit idle-in-transaction in a CLI context and
  would deadlock the DROP on a metadata lock). SQLite: db.engine, restoring the
  prior foreign_keys pragma so the StaticPool connection is not left changed.
- tests/test_plugin_prune_schema.py: drop-only-not-installed, full no-op,
  refuse-non-empty, force-drops-non-empty.
- docs/DEPLOY.md: lean provisioning step after upgrade-all.
- ADR-014 ACCEPTED; index updated.

Verified on MySQL: full install then prune = no-op (86 tables); lean install
(machines+printers) then prune drops the other 19 plugin tables; second run
no-op. Full suite 1077 passed.
 2026-07-19 11:39:46 -04:00
cproudlock
42ca8d75c3 ADR-014: schema-lean per-site (investigation + idempotent create_plugin_tables) 
Cross-plugin FK blocker ADR-013 cited is already resolved: the FKs into
machines were held only by dead legacy tables (machinerelationships,
printerdata, installedapps, communications.machineid) that existing
migrations 7a01/7c01 already drop. No live plugin table hard-FKs another
plugin. Schema-lean is unblocked.

Enabling change: create_plugin_tables now skips already-existing tables
(idempotent) so a plugin anchor can create its tables on a fresh lean
install and no-op on a database that has them from the pre-cutover
baseline. The load-bearing baseline lift is staged as ADR-014 Phase 2.
 2026-07-19 00:36:27 -04:00
cproudlock
5861caf78f ADR-013 Phase 5: CI lean-build job (delete-a-plugin guarantee) 
New CI job builds a lean site (machines + printers) via build-site.sh and
asserts omitted-plugin code (PartsKiosk, ManifestEditor, USBLabelBatch,
KnowledgeBaseDetail) is absent from the bundle while chosen-plugin code is
present, and that only chosen plugin dirs stage into the backend. Locks the
lean-build guarantee so a future change cannot silently pull an unchosen plugin
into a per-site build.
 2026-07-19 00:09:23 -04:00
cproudlock
da3cb37be8 ADR-013 Phase 5: lean per-site builds - build-site.sh + import-guard audit 
The lean-build endgame: a site ships carrying only the plugins it chose.

- scripts/build-site.sh: reads a site profile, resolves the hard-dependency
  closure from manifests, builds the frontend with SITE_PLUGINS (stage-frontend
  carries only those plugins), and stages a backend tree of core + only the
  chosen plugin dirs. An unchosen plugin is in neither the bundle nor the tree.
- Core lazy-import guard: `flask seed demo` hard-imported the 5 asset subtype
  models, which would crash a lean build missing any of those plugins. Now
  guarded (a missing model skips its demo section).
- test_lean_build_guards.py: statically asserts NO core (shopdb/core, shopdb/cli)
  import of a plugin is unguarded - a lean build omitting that plugin would
  otherwise crash. 0 unguarded today.

Pilot verified: a lean build (machines + printers) carries only machines +
printers code - PartsKiosk / ManifestEditor / USBLabelBatch / KnowledgeBaseDetail
/ EmployeeDirectory are absent from the bundle, and only machines/printers plugin
dirs stage into the backend. (Sidebar labels for absent plugins remain - the
accepted small plugin-aware core remainder.) Guard test + naming green.
 2026-07-19 00:08:35 -04:00
cproudlock
c6a1e07a6c ADR-013 Phase 4: lint plugin-frontend imports (self-contained rule) 
The naming/style check now fails a plugin frontend (plugins/<name>/frontend/)
that imports with an escaping ../../ or another plugin's path. Plugin frontends
must reach core only through the @/ alias and otherwise import only their own
tree, so a per-site build can drop a plugin cleanly. All 14 plugin frontends
pass.
 2026-07-19 00:03:33 -04:00
cproudlock
592ff49abe ADR-013 Phase 4: extract the 11 plugin routes embedded in core.js 
core.js still routed plugin-owned pages directly. Extracted all 11 into the
owning plugin's route file + moved their views into plugins/<name>/frontend/:
- computers: reports/pc-relationships, settings/pctypemapping
- printers: reports/toner, settings/printertypes, settings/zabbix (toner/supply
  monitoring)
- machines: settings/machinetypes
- network: settings/networktypes
- warranty: settings/dellwarranty
- slides: settings/slides (its route file gains a default export; it was
  toplevel-only)
- employees: NEW plugin frontend (employees/:sso + settings/employeedirectory) -
  employees had no route file before; its pages lived only in core.js.

core.js now holds only core routes; all 14 bundled plugins are self-contained
under plugins/<name>/frontend/. Verified live: the extracted Machine Types
settings page renders in the settings rail from the machines plugin frontend.
Build + 58 vitest + naming green.
 2026-07-19 00:02:32 -04:00
cproudlock
ebca0b00b0 ADR-013 Phase 4: relocate the remaining 9 plugin frontends (all 13 done) 
Relocate warranty, measuringtools, network, printers, usb, notifications,
computers, and slides into plugins/<name>/frontend/. Each plugin's views are
pulled from wherever they lived (own dir, plus the shared views/settings/,
views/reports/, views/print/ dirs, and top-level views) into the plugin's
frontend/views/, and its route file becomes the self-contained routes.js.

Handled the messy cases:
- computers: name mismatch (its views live in views/pcs/) - moved by following
  the route file's own imports, so the dir name did not matter. Its OS/access-
  protocol/PC-type settings views move with it (only computers.js routed them).
- network: NetworkHub's sibling sub-views (NetworkDevicesList, SubnetsBrowse,
  not directly routed) moved too so its `./` imports resolve.
- printers: the qrLogo helper is SHARED with core AssetLabel, so it stays in
  views/print/ and PrinterQR imports it via @/views/print/qrLogo.
- slides: route file is toplevel-only (TVDashboard); SlideManager stays core
  (core.js routes /settings/slides).

frontend/src/views/ now holds only core views; frontend/src/router/routes/ holds
only core.js. All 13 plugins are self-contained under plugins/<name>/frontend/.
Verified live: Network (hub + moved sub-views), Computers (name mismatch),
GE-Enforce (helper), printedparts all render from their staged frontends. Build +
58 vitest + naming green.
 2026-07-18 23:56:07 -04:00
cproudlock
23dc9fa379 ADR-013 Phase 4: relocate 4 self-contained plugin frontends 
Relocate applications, geenforce, knowledgebase, and machines - each owns only
its own views dir, so a clean move to plugins/<name>/frontend/ (views/ +
routes.js, core imports rewritten to @/). geenforce's entryForm.js helper + its
vitest spec move with it (ManifestEditor imports it as a sibling).

Machinery fixes this batch surfaced:
- routes.gen.js codegen uses namespace imports (import * as p_x). A route file
  without a `toplevel` export is undefined on the namespace instead of a strict-
  ESM missing-binding build error.
- vitest gains a `pretest` stage so plugin-frontend specs (now under
  plugins/<name>/frontend/) run from their staged copy in src/.plugins-staged/.

Verified live: GE-Enforce (the most complex, uses the entryForm sibling helper)
renders fully from its staged frontend. Build + 58 vitest + naming green.
 2026-07-18 23:51:23 -04:00
cproudlock
af9a3b190b ADR-013 Phase 4: frontend staging machinery + relocate printedparts; fix router crash 
The staging step that makes lean per-site frontend builds possible, plus the
first plugin relocated as the pilot.

- scripts/stage-frontend.mjs: copies each chosen plugin's plugins/<name>/frontend/
  into frontend/src/.plugins-staged/<name>/ and codegens routes.gen.js. Plugin
  selection via SITE_PLUGINS (comma-separated); empty = all plugins that have a
  frontend/ (the full build). Wired as npm predev/prebuild; outputs gitignored.
- Router imports routes.gen.js and merges staged routes with the in-tree
  ./routes/*.js glob - dual-location during the transition.
- printedparts relocated: its 6 views (list/detail/form/kiosk + the settings and
  labels views from the shared dirs) moved into plugins/printedparts/frontend/
  views/, core imports rewritten to the @/ alias; routes.js is the self-contained
  route module. Its old in-tree route file is removed.

Also fixes a crash the previous commit (37c764b) shipped: slides.js exports only
`toplevel` (its child routes live in core.js), so the router's
flatMap(m => m.default) produced an undefined child and threw
"Cannot read properties of undefined (reading 'path')" at load - the whole SPA
went blank. Guarded with `m.default || []`. (The earlier "print pages are blank"
reading was this crash, not page nature.)

Verified live: /machines renders again; the relocated /printedparts list renders
identically from the staged plugin frontend; SITE_PLUGINS=machines excludes
printedparts from routes.gen. Build (via npm, runs stage) + vitest + naming green.
 2026-07-18 23:42:43 -04:00
cproudlock
37c764ba8d ADR-013 Phase 4: move hardcoded plugin top-level routes into plugin route files 
Core-router surgery (the Phase 4 prerequisite for lean builds): index.js
hardcoded six plugin-owned full-screen routes (parts-kiosk, TV, printer-qr x2,
usb-labels, printedparts-labels), so pruning any of those plugins broke the SPA
build on an unresolvable import. The router now also collects a `toplevel`
named export from each plugin route file (alongside the existing default =
AppLayout children) and spreads it into the top-level routes. Each of the six
routes moved into its owning plugin's route file (printedparts, printers, usb,
slides); index.js keeps only the core print pages that span asset types
(machine-badge, asset-label, asset-label-batch).

index.js now references zero plugin view components. Verified: all six route
paths are present in the built bundle and the moved routes resolve exactly like
the unchanged core print routes. Build + vitest + naming green.
 2026-07-18 23:26:17 -04:00
cproudlock
296b6e024b ADR-013 Phase 3: generic map-overlays renderer (Path A) 
Wires the ADR-010 get_map_overlays hook into the floor map so a plugin decorates
markers as JSON, no map code. ShopFloorMap fetches /api/pluginui/map-overlays,
then each overlay's endpoint (per-asset [{assetid, color, label}]), joins by
assetid, and draws a ring or badge circleMarker on matching markers plus a
legend entry - all as extra Leaflet layers cleared and redrawn with the markers.

Aligned the measuringtools calibration overlay endpoint to the documented
contract: it now returns {assetid, color, label} (was {calibrationstatus,
statuscolor}) and only decorates due/overdue tools.

Additive + guarded (assetid null check, per-endpoint try/catch, cleanup on
re-render), so the map degrades to no decorations on any failure. Verified: the
overlay endpoint serves the contract shape, the map renders without error, and
the frontend builds. A populated badge needs a site that actually places
measuring tools on its map (this dataset places none). 38 measuringtools/pluginui
tests, 58 vitest, build + naming green.
 2026-07-18 23:10:09 -04:00
cproudlock
8a2f984393 ADR-013 Phase 3: search routes via get_asset_presentation, not a hardcoded map 
Global-search rows built the plugin detail URL from a hardcoded url_map of
plugin routes in core. Now core prefers a plugin's declared
get_asset_presentation route (ADR-010), substituting the core assetid via the
plugin's by-asset resolver; types that have not declared fall back to the legacy
id-keyed map, so nothing breaks. Measuring tools (which declare the route) link
through it now; machines/PCs/printers/network migrate off the hardcode as they
add a by-asset route + declaration. Presentation map is collected once per
search (cached on flask.g). 2 consumer tests; 26 search tests green.
 2026-07-18 22:56:05 -04:00
cproudlock
b669561421 ADR-013 Phase 3: migrate all detail pages to PluginAssetPanels; drop WarrantyPanel 
Rolls the generic renderer into the remaining four detail pages (PCDetail,
PrinterDetail, NetworkDeviceDetail, MeasuringToolDetail), replacing the
hand-composed <WarrantyPanel> with <PluginAssetPanels>. The warranty hero badge
(useWarrantyBadge) stays on the pages that show it; MeasuringToolDetail dropped
its now-unused warranty composable usage.

WarrantyPanel.vue is deleted - warranty now renders entirely from its
get_asset_panels JSON declaration through the generic renderer. Verified live on
a PC with a warranty: the card is identical to the old bespoke panel (vendor
title, Expiring Soon status badge with color, servicelevel/ends/tag meta, manage
link) with no warranty-specific frontend code. Build clean, 58 vitest, naming green.
 2026-07-18 22:50:27 -04:00
cproudlock
e3c4b90afe ADR-013 Phase 3: generic asset-panels renderer (Path A) 
Wires the ADR-010 get_asset_panels hook to a generic frontend renderer so a
plugin adds detail-page UI as JSON, no Vue. This is the Path A foundation that
lets simple plugins ship UI without a frontend build.

- components/PluginAssetPanels.vue + pluginAssetPanels.js: fetches
  /api/pluginui/asset-panels for an asset, then each panel's data endpoint, and
  renders by mode: list (title + status badge + meta lines via a field map),
  keyvalue, table (declared or inferred columns), badge. Pure mapping logic is
  in the .js module and unit tested (9 specs), same pattern as entryForm.js.
- New 'list' render mode with a declarative field map (title/badge/meta),
  documented on the hook in base.py.
- Warranty migrated to it: get_asset_panels now declares a 'list' panel + map
  that reproduces WarrantyPanel's output (vendor title, status badge with color
  + label map, servicelevel/ends/tag meta, manage link) with zero
  warranty-specific frontend code.
- MachineDetail swapped from <WarrantyPanel> to <PluginAssetPanels> (pilot); the
  hero warranty badge is unchanged. Verified end to end: the API serves the list
  panel + map and the warranty rows; the page renders without error.

Rollout of the other 4 detail pages (PCDetail, PrinterDetail, NetworkDeviceDetail,
MeasuringToolDetail) and the map-overlays / asset-presentation renderers are
follow-up Phase 3 commits. 58 vitest, build clean, 1067 backend pass, naming green.
 2026-07-18 22:43:54 -04:00
cproudlock
beea6c0c9f ADR-013 Phase 2: guard hash-gates a .py sibling of an init-less dir 
Fourth review found the last import-path bypass: the is_dir() branch returned
None for a name whose dir has no __init__.py, without checking a same-name
sibling file. FileFinder loads a file over an init-less namespace dir, so an
attacker could overwrite a signed foo.py with malicious bytes, mkdir an empty
foo/ next to it (PROVENANCE untouched, still verifies), and any import of that
name ran the unverified foo.py - RCE with only plugins/ write access.

Fix: the dir-with-no-__init__.py branch no longer returns early; it falls
through to the leaf .py hash gate and the non-source refuse check. Invariant:
find_spec returns None for a plugins.* name ONLY where FileFinder would also
find nothing on the same __path__.

Everything else was confirmed sound this round: the owned plugins root, exec of
exact verified bytes (never .pyc/.so), the extension/bytecode refusal, plugin.py
read-once, the provenance signature gate, dev-exemption scoping, and #3/#4.
Symlink, suffix-ordering, cache-lifecycle, and loader-internal angles cleared.
2 regression tests (tampered .py + sibling dir; unsigned .py + sibling dir). All
13 bundled plugins still load under enforcement; 1067 pass, naming green.
 2026-07-18 22:28:33 -04:00
cproudlock
c59d2dab56 ADR-013 Phase 2: import guard fails closed on non-.py + owns package root 
Third review found the meta_path guard leaked exactly where it delegated to the
stdlib import system:

1. Non-.py submodules (CRITICAL). When a name had no dir and no .py, find_spec
   returned None and the stdlib loaded a planted .so (ExtensionFileLoader) or a
   sourceless .pyc unverified - an attacker deletes a signed .py and drops a
   same-named .so with arbitrary init code, run on a normal request via core's
   `from plugins.<name>.models import ...`. The guard now refuses any name for
   which a non-source importable candidate (EXTENSION_SUFFIXES + BYTECODE_
   SUFFIXES) exists on disk; None is reserved for genuinely-absent modules.

2. Top-level plugins/__init__.py (CRITICAL). It is in no plugin's provenance,
   is attacker-writable, and Python runs it before any guarded submodule. The
   guard now owns `plugins`: it execs an EMPTY package body (search points at
   the plugins dir), so an overwritten plugins/__init__.py never runs.

Also: specs are built with spec_from_file_location so loaded modules get
__file__/__path__ (Flask blueprint root paths need it) while the loader still
execs the verified in-memory bytes - never re-reading the file.

Verified end to end: under PLUGIN_REQUIRE_SIGNED with all 13 bundled plugins
stamped, the app boots and loads every plugin through the guard; a tampered
plugin file is refused at load. 4 new guard tests (planted .so, sourceless
.pyc, absent-module defer, neutralized package root). Prior fixes #3/#4
confirmed still sound by the review. 1065 pass, naming green.
 2026-07-18 22:12:20 -04:00
cproudlock
dd30ca0c3f ADR-013 Phase 2: verify every plugins.* import via a meta_path guard 
A re-review showed the previous "single import choke point" claim was wrong:
`plugins` is a normal importable package, so core request handlers that do
`from plugins.<name>.models import ...` never passed through the loader and ran
unverified - an attacker who dropped a file into plugins/<name>/ got arbitrary
in-process code execution on an ordinary HTTP request (and a planted .pyc ran
from cache). Gating load_plugin_class covered only plugin.py, one path of many.

Fix: importguard.py installs a sys.meta_path finder (under enforcement) that
intercepts EVERY plugins.<name>.* import, verifies the plugin's signed
provenance once, then verifies each module file against it and execs the exact
bytes it hashed - read once, compiled, exec'd, never a .pyc, never a re-opened
file. This closes the submodule bypass and the planted-bytecode read, and the
read-once exec closes the verify-vs-exec TOCTOU on the import path. The import
system, not one method, is the real choke point.

- init_app installs the guard when PLUGIN_REQUIRE_SIGNED, clears it otherwise.
- load_plugin_class now verifies plugin.py from a single read and execs that
  buffer (finding #3 on that file); its submodule imports flow through the guard.
- docs: stamp-bundled must cover every plugin dir present (a disabled plugin's
  module can be imported by core); recommend a read-only plugins/ owned by the
  deploy user as defense in depth (closes the residual migrate-time race an
  attacker with concurrent write could otherwise attempt).

Earlier review's fixes #3 (migrate code paths) and #4 (shelf content binding)
were confirmed sound and are unchanged. 7 import-guard tests (submodule verify,
tamper, unsigned refused, planted .pyc ignored, real import through the guard,
install/uninstall). 1061 pass, naming green.
 2026-07-18 21:47:32 -04:00
cproudlock
55a6f1b8d3 ADR-013 Phase 2: fix four bypasses found by adversarial review 
An adversarial security review of the Phase 2 trust model found four real
bypasses (two remote-triggerable to in-process code execution). Root cause for
three: the set of bytes verification covered was smaller than the set that
determined execution. Fixes:

1. Bytecode-cache blind spot (CRITICAL). verify_dir excluded __pycache__/.pyc,
   so a planted cache ran while escaping the hash map. verify_dir now flags any
   bytecode as an unexpected file; the loader strips bytecode before verify and
   imports under sys.dont_write_bytecode, so only verified source executes.

2. Unauthenticated verify-at-load bypass (CRITICAL). load_plugin_class imported
   plugin.py with no gate, reachable via discover_available / an anonymous GET
   /api/plugins. The verify+strip gate moved INTO load_plugin_class - the single
   import choke point every path flows through - so an unsigned/tampered plugin
   is never imported. discover_available skips a refused plugin instead of 500.

3. Ungated migration entrypoints (HIGH). downgrade_plugin and get_current_head
   (ScriptDirectory imports version modules) ran plugin code with no check. All
   alembic-invoking methods now pass through _verify_ok (strip + verify) first
   and run under no-bytecode.

4. Revocation/content bypass (HIGH). The signed index bound a filename, not
   content; adopt did not bind the delivered bytes to the resolved version, so
   revoked bytes could be served under a live filename. The index now records a
   per-artifact SHA-256; adopt verifies the on-disk digest and requires the
   artifact's own signed manifest version to equal the resolved version.

Enforcement stays default-off; strip/no-bytecode run only under enforcement, so
the unsigned path is unchanged. 6 regression tests (planted bytecode, the
discover import path, downgrade gate, version-swap). 1054 pass, naming green.
 2026-07-18 21:06:27 -04:00
cproudlock
5b19f3b554 ADR-013 Phase 2: enforcement + signed shelf + adopt 
Completes the marketplace security model. Verification stops being advisory:
a plugin only loads or migrates when its tree matches a trusted signature, and
plugins are pulled from a signed shelf with anti-rollback and revocation.

Enforcement (default OFF - existing deploys unchanged):
- verification.py PluginVerifier, shared by the loader (verify-at-load, before
  plugin.py is imported) and the migration manager (verify-at-migrate, before
  any DDL). Fail-closed: an unsigned/tampered/wrong-key plugin does not run.
- Gated by PLUGIN_REQUIRE_SIGNED. PLUGIN_DEV_TRUST_DIRS exempts named dirs but
  only under DEBUG/TESTING; production ignores it.
- flask plugin stamp-bundled writes provenance into in-tree plugins so
  verify-at-load applies to bundled plugins too (image build step).
- tier:core manifest guard: uninstall/disable refuse a core-tier plugin.

Shelf (shelf.py):
- Signed shelf-index.json (+ .sig): monotonic serial (a site refuses an older
  index - anti-rollback), revoked list carried across builds, per-entry
  version/tier/core_version for browse. Index is a browse layer only; adopt
  reads security-bearing fields from the verified artifact.
- flask plugin shelf-build / shelf-list / adopt / audit. adopt verifies index +
  artifact (signature + every file hash), unpacks to staging, re-verifies, then
  atomically moves into place and installs+enables the closure. Refuses a
  downgrade without --force-downgrade. Anti-rollback serial stored in
  instance/shelf-state.json.
- config PLUGIN_SHELF_DIR; the app only reads the folder, never speaks a
  network. .env.example + docs/PLUGIN-SIGNING.md document the flow.

22 tests: verifier policy (off / no-keys / signed / tampered / wrong-key /
dev-exempt), verify-at-load + verify-at-migrate integration, tier guard, index
sign/verify + tamper/wrong-key, serial state, revocation, version resolution,
verified atomic unpack + tamper refusal. Live-smoked keygen->pack->shelf-build
->list->adopt->audit + serial guard. 1050 pass, naming green.
 2026-07-18 20:44:54 -04:00
cproudlock
86f5f1be68 ADR-013 Phase 1: signed plugin artifacts (pack/validate/keygen) 
Packaging + provenance for the plugin marketplace. No runtime behavior change
yet - verification is available on demand; enforcing it at plugin load/migrate
and pulling from a shelf are Phase 2.

- signing.py: ed25519 key pairs + provenance. Provenance is a sorted per-file
  SHA-256 map plus metadata; the detached signature covers the exact
  serialized provenance bytes, so verifying is re-hash files, re-serialize,
  check signature. verify() accepts any of several trusted keys (rotation).
  Uses cryptography (already a dependency).
- packaging.py: pack() builds a signed <name>-<version>.shopdbplugin (zip +
  PROVENANCE.json + PROVENANCE.sig). verify_artifact()/verify_dir() re-hash
  and check the signature, and flag a tampered file, an unexpected file, a
  wrong/absent key - all fail closed.
- CLI: `flask plugin keygen` (publisher key pair), `flask plugin pack <name>
  --key` (validates then signs), and `flask plugin validate` extended to a
  signed artifact by path (--pubkey, else PLUGIN_TRUSTED_KEYS).
- config PLUGIN_TRUSTED_KEYS: os.pathsep-separated public-key PEM paths,
  delivered with the site config, never read from the shelf. .env.example
  documents it.
- docs/PLUGIN-SIGNING.md: curator flow (keygen offline, review, pack, publish,
  pin keys, rotate).

The signature proves an artifact is exactly what a curator signed, not that the
code is safe - human review before signing is the control. 11 tests: sign/verify
round trip, wrong key, provenance excludes noise, serialize determinism, pack +
verify, tamper -> hash mismatch, extra file, no-key fail-closed, verify_dir.
1028 pass, naming green.
 2026-07-18 20:21:57 -04:00
cproudlock
d178726687 ADR-013 Phase 0: plugin lifecycle groundwork 
Additive, zero-risk-to-running-sites prep for the plugin catalog. No
distribution or lean-build behavior yet; fixes latent bugs and adds the
declarative + validate tooling later phases build on.

Fixes:
- upgrade_all_plugins iterates registry.get_all(); only adopted plugins are
  migrated. Removes the phantom hasattr(registry, 'list_installed') probe
  that always fell through to migrating every folder on disk (unadopted DDL
  ran with full DB rights on every deploy).
- Reverse-dependency checks on uninstall/disable read dependencies from the
  manifest on disk via _installed_dependents, so an installed-but-unloaded or
  disabled dependent is counted. Uninstall blocks on any installed dependent;
  disable blocks on an enabled dependent.
- _sort_by_dependencies detects a dependency cycle (back edge in the DFS) and
  raises PluginDependencyError instead of looping or dropping a plugin.

New:
- flask plugin validate <name>: manifest loads + name match, manifest-schema
  check, core_version admits the framework contract, declared dependencies
  exist on disk. No new dependency (lightweight checker); schema ships in the
  package at shopdb/plugins/manifest_schema.json (docs/ is stripped on
  publish). The check caught that provides is an object, not an array.
- flask plugin apply-profile <file>: declarative install AND enable of a
  chosen plugin set plus its hard-dependency closure, in dependency order,
  idempotent. Replaces the hand-ordered runbook sequences that could enable a
  plugin that was never installed. deploy/site-profile.example.json template.
- Dockerfile header corrected (all 13 catalog plugins, not "eleven core").

10 new lifecycle tests (reverse-deps from disk, cycle detection, upgrade-all
scope, profile closure, schema, all 13 manifests match schema). 1018 pass,
naming green.
 2026-07-18 18:08:34 -04:00
cproudlock
3ac5ed2580 Add ADR-013: plugin catalog, curated shelf, lean per-site builds 
Design record for distributing optional plugins across GE sites: a small
mandatory core plus a catalog of optional plugins, packaged as signed
versioned artifacts, served from a transport-agnostic read-only shelf (a
SharePoint-synced or sneakernet folder - untrusted either way because every
decision-bearing byte is signed), verified at adopt AND at every load and
migrate. Lean per-site builds stage only chosen plugins into the backend
image and SPA bundle.

Status PROPOSED. Grounds the design in the real loader/contract/migration/
frontend code and records defects to fix along the way (upgrade-all
migrating unadopted folders, enable-without-install, reverse-dep checks
blind to unloaded plugins, missing cycle detection and dependency closure,
hardcoded plugin imports in the SPA router). Honest on scope: the frontend
re-org is the long pole (one core-router change plus per-plugin relocation),
not a mechanical move. Phased 0-5 with schema-lean and runtime-JS delivery
explicitly deferred.
 2026-07-18 17:40:29 -04:00
cproudlock
603872ff76 Add Copilot custom instructions 
Repo-level instructions so GitHub Copilot follows the LOCKED naming rules
(lowercase concatenated DB columns, allowed-acronym list, banned shorthand),
the ASCII-only style policy, and the plugin/migration/contract architecture.
Without this Copilot suggests snake_case columns, em-dashes, and
db.create_all(), which the naming hook and CI then reject. Distilled from
CONTRIBUTING.md; that file stays the authority.
 2026-07-17 20:56:41 -04:00
cproudlock
6ab1046ef4 Add flask seed demo sample-data command 
New dev/eval seeder populates a small, broad dataset so a fresh site has
something on every screen: ~25 assets across machines, computers,
printers, network devices, and measuring tools, plus supporting
vendors/business-units/locations, six 3D-printed parts (two below their
low-stock threshold to exercise the alert), and a few relationships for
the map and relationship cards. Idempotent, keyed on a DEMO- assetnumber
prefix; skips the plugin sections that are not installed.

`flask seed demo-clear` removes exactly what it created: bulk-deletes the
DEMO- assets so the DB-level ON DELETE CASCADE drops each plugin subtype
row (per-object ORM delete would try to NULL the NOT NULL child assetid),
after clearing the demo relationships first. Leaves reference data,
settings, users, and any imported rows untouched.

Documented as an optional step in the dev setup guide.
 2026-07-17 20:32:01 -04:00
cproudlock
83141bacb7 Widen settings.description to TEXT; run seeders in CI 
The dualpath_single_machine setting description is 257 chars but
settings.description was varchar(255). On strict MySQL 8 an over-length
insert is a hard error 1406 (Data too long), so `flask seed settings`
failed on a fresh install; older/relaxed MySQL truncated silently and
hid it. Widen the column to TEXT (matches value, already TEXT) via core
migration 7d26.

CI only ran `flask db upgrade` + plugin install, never the seeders, so it
missed this. Add a seed step to the migrations-mysql job so a seeded row
that violates a column constraint fails CI on strict MySQL 8 instead of
shipping.
 2026-07-17 20:31:48 -04:00
cproudlock
804c066de4 Add cryptography dependency for MySQL 8 auth
Some checks failed
CI / backend (push) Successful in 1m44s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
pymysql needs the cryptography package to speak MySQL 8's default
caching_sha2_password, so 'flask db upgrade' against a stock MySQL 8
failed with 'cryptography package is required'. Make it a real
dependency (dev, prod, CI all connect cleanly) and drop the CI
native-auth workaround that stood in for it.
 2026-07-17 19:41:42 -04:00
cproudlock
9deb194580 Standardize on Python 3.13 (matches prod 3.13.7)
Some checks failed
CI / backend (push) Successful in 1m45s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
Prod runs Python 3.13.7, not the originally planned 3.12. Align the
stack: CI both jobs 3.12->3.13, the IIS install runbook and the dev
setup guide to 3.13 (winget Python.Python.3.13). NOTE for whoever
maintains the offline kit: its wheels are still cp312 and must be
regenerated as cp313 before the next air-gapped deploy.
 2026-07-17 19:30:19 -04:00
cproudlock
314f339ba9 Dev setup: winget install commands for the Windows toolchain
Some checks failed
CI / backend (push) Successful in 1m45s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
Add a winget block to the prerequisites (Git/Python/Node/VS Code/MySQL
or Docker) so a Windows dev provisions the whole toolchain from one
terminal, with a note that the LTS Node may be newer than CI's 20 and
it does not matter for this SPA (nvm-windows to pin if wanted).
 2026-07-17 18:58:06 -04:00
cproudlock
1361dc6004 Lab intro: tag range through lab-stage-17 (last audit finding)
Some checks failed
CI / backend (push) Successful in 1m45s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details
2026-07-17 18:15:59 -04:00
cproudlock
efb879d44a Docs audit fixes: kiosk code drift, PowerShell chains, broken links, leaks
Some checks failed
CI / backend (push) Successful in 1m45s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
From the Fable/Opus documentation audit (8 confirmed + verified
lab-drift the run's session limit had cut short):
- HIGH: the lab's kiosk _kiosk_find_item block showed the pre-stage-17
  row-id resolver as current; replace with the shipped gagelabtag /
  numeric-tail resolver, fix the stale 'resolved by row id' prose and
  the 'stage-7 code is corrected' note.
- MED: the badge _external_lookup block used dict-only row access that
  breaks on a tuple cursor; use the tuple-or-dict form shipped. Split
  '&&' command chains (fail in PowerShell 5.1) in the lab.
- LOW/link: the Windows note's [DEVELOPMENT-SETUP] link dropped the .md
  and 404'd in four docs; fix. Correct the stage-6a->16a comment and
  the lab-stage tag range (..16 -> ..17).
- Leaks: drop /home/camp path from ADR-006, the internal gitea host
  from PLUGINS.md.
- Windows: add an mklink junction note for the external-plugin symlink
  dev loop.
- CI: prime root to mysql_native_password so pymysql connects to the
  MySQL 8 service without the cryptography package (and its kit wheel).
 2026-07-17 18:05:47 -04:00
cproudlock
aba588cc07 Neutralize internal-host references for publication
Some checks failed
CI / backend (push) Successful in 1m45s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
The CI workflow comment named the internal server, and
PLUGIN-EXTERNAL-REPO carried internal gitea clone URLs (it becomes a
public wiki page). Point both at the GitHub home / a generic CI
mention so the publication scrub gate passes and the wiki does not
expose internal infrastructure.
 2026-07-17 15:13:34 -04:00
cproudlock
f77f0a8d90 Add GitHub Actions CI + Windows notes on the developer docs
Some checks failed
CI / naming (push) Has been cancelled
Details
CI / frontend (push) Has been cancelled
Details
CI / migrations-mysql (push) Has been cancelled
Details
CI / backend (push) Has been cancelled
Details 
GitHub had no CI, so naming/tests/build were unenforced on the public
mirror. Add .github/workflows/ci.yml mirroring the internal pipeline:
backend pytest, the naming gate, frontend vitest+build, and the
migrations-mysql job that proves a fresh flask db upgrade + every
plugin chain on utf8mb4 MySQL 8 is idempotent. Flip the dev-setup CI
note to reflect it. Add an identical Windows/VS Code convention note to
the four developer docs (venv\Scripts vs venv/bin, $env: vs export,
pointer to DEVELOPMENT-SETUP).
 2026-07-17 15:12:36 -04:00
cproudlock
0e194c3237 Dev setup: note GitHub has no CI yet, so local checks are the gate there
Some checks failed
CI / backend (push) Successful in 1m46s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details
2026-07-17 15:09:16 -04:00
cproudlock
9a60100cb2 Dev setup: correct the hook claim, ship an opt-in committed hook
Some checks failed
CI / naming (push) Has been cancelled
Details
CI / frontend (push) Has been cancelled
Details
CI / migrations-mysql (push) Has been cancelled
Details
CI / backend (push) Has been cancelled
Details 
The naming check was documented as an auto-running pre-commit hook,
but .git/hooks is never cloned and no installer existed - a fresh
clone had nothing, and the real enforcement is CI. Say that plainly.
Ship .githooks/pre-commit (LF-pinned) so a dev who wants the local
check can opt in with 'git config core.hooksPath .githooks'; CI stays
the backstop that fails the build on a bad name.
 2026-07-17 15:08:50 -04:00
cproudlock
0c37ef9057 Dev setup: Windows-first (most devs are on VS Code / Windows)
Some checks failed
CI / backend (push) Successful in 1m45s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
PowerShell commands lead, bash equivalents in comments: venv
Activate.ps1 + execution-policy note, copy/$env:, a PowerShell
plugin-enable loop, and how the bash naming hook runs under Git Bash
(plus the pre-commit hook catching it automatically). The VS Code
Check task gets a Windows variant (venv\Scripts, bash for the .sh).
Pin shell scripts to LF in .gitattributes so a Windows checkout does
not CRLF-corrupt them into 'bad interpreter' failures.
 2026-07-17 15:01:49 -04:00
cproudlock
f764c5d3e9 Add development setup guide + shared VS Code config
Some checks failed
CI / backend (push) Successful in 1m45s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
New docs/DEVELOPMENT-SETUP.md: clone-to-first-change onboarding
(Docker fast path, manual venv+Node daily driver, VS Code, the dev
loop, first-change pointer at the plugin lab, troubleshooting). Ship
.vscode/ launch/tasks/extensions so F5 debugs the backend on 5001 and
a task runs both servers; personal settings.json stays ignored. Fix
the README manual path - it ran the backend on the default 5000, but
the frontend dev server proxies to 5001, so nothing loaded; also add
the plugin upgrade-all step and a VS Code pointer.
 2026-07-17 14:40:55 -04:00
cproudlock
3a3dff285e printedparts stage 17: gage-lab asset tag + print-files redesign
Some checks failed
CI / backend (push) Successful in 1m45s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
The gage lab assigns real WJRP asset numbers, so identity splits: the
internal itemcode stays auto-minted and a new optional unique
gagelabtag (migration 0003) carries the lab's number - settable on
create/edit, searchable, and resolved by the kiosk for scans and bare
keypad digits against the numeric tail of either identifier
(unique-match only). The print-files table becomes stacked revision
cards - filename with rev/current badges, one meta line, delete pinned
right - ending the horizontal scroll in that column.
 2026-07-17 14:01:09 -04:00
cproudlock
6160a5142a Dark mode: dropdown arrow no longer tiles across selects
Some checks failed
CI / backend (push) Successful in 1m45s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
The dark .form-control override used the background shorthand, which
resets a select's background-repeat and position; the dark select rule
then re-added the arrow image without them, tiling it from the top
left. Use background-color in the overrides and restate
no-repeat/position on the select rule.
 2026-07-17 13:37:24 -04:00
cproudlock
d75e80ce79 Plugin lab rewritten as the literal type-along walkthrough
Some checks failed
CI / backend (push) Successful in 1m45s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
The milestone workbook becomes a from-scratch guide with the actual
code inline for every core stage: models, the real migration baseline,
read routes and the list page, mutations and minting, the badge
resolver (final mode-aware form), the single-commit ledger invariant,
RBAC gating, both kiosk endpoints with the wedge-input and focus-guard
mechanics, the 1x0.5in label CSS, and the reconcile query. Field
extensions stay summarized against their tags. New section: how to
contribute a plugin through GitHub (branch, stage commits, the three
CI gates, PR expectations, review checklist, and how publication
folds PRs into release commits).
 2026-07-17 13:35:02 -04:00
cproudlock
ee80d684d4 Shopfloor feed resolves employee names live when none is stored
Some checks failed
CI / backend (push) Successful in 1m45s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
Photos already resolved through the directory at read time, but names
only came from the stored employeename column - empty after a
shopdb-only import, so recertification/recognition cards showed bare
SSOs. New resolve_employee_display_name in the employees plugin
(mode-aware: self-hosted table or external HR) backs a fallback in
both the single-card and split-per-employee paths; stored names still
win when present.
 2026-07-17 13:29:34 -04:00
cproudlock
bc9159742c printedparts lab: post-stage polish addendum and closing lesson
Some checks failed
CI / backend (push) Successful in 1m45s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details
2026-07-17 13:24:11 -04:00
cproudlock
e9235de8ec Floor-map previews honor the mount path
Some checks failed
CI / backend (push) Successful in 1m44s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
The uploaded-blueprint thumbnails on the map settings page and the
setup wizard used the raw setting value (/api/settings/map-blueprint/
...), which resolves at the server root and 404s under a subpath
mount - while the map itself resolves through blueprintUrlFor and
worked. Wrap the previews in withBase.
 2026-07-17 11:43:06 -04:00
cproudlock
4f3ea2848a GE monogram avatar fallback + per-page document titles
Some checks failed
CI / backend (push) Successful in 1m44s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 7s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Users without a profile photo (and broken photo URLs) show the GE
monogram instead of nothing/initials - sidebar identity, employee
detail hero, and the directory list thumbs; the shopfloor cards
already did this. Document titles become
'<Facility> ShopDB - <Page>' via a router afterEach (facility from
public settings, page label from meta.title or a prettified route
name with spellings for PCs/USB/GE-Enforce/3D Printed Parts/...), so
copied links and browser tabs identify the page.
 2026-07-17 11:31:38 -04:00
cproudlock
5625608bd0 Employees: external photo base URL is a setting
Some checks failed
CI / backend (push) Successful in 1m45s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
External HR Picture values are relative paths; the resolver hardcoded
/static/employees/ (which the SPA then mounts under the subpath, e.g.
/ops/static/...), but sites like WJ serve those photos from the
classic EmployeeDBAPP on another URL entirely. New setting
employee_photo_base_url (blank keeps the old behavior; a full URL like
https://host/EmployeeDBAPP/images/ passes through withBase untouched),
declared in the plugin config schema.
 2026-07-17 11:16:04 -04:00
cproudlock
0cc205d25e Users: deleting a user clears their API tokens and detaches audit rows
Some checks failed
CI / backend (push) Successful in 1m45s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
Deleting any user who owned an API token or appeared in the audit log
hit the users FK and 500ed - the import's 'importer' account being the
guaranteed case (its PAT plus every audit row the import wrote).
Tokens are revoked outright; audit history is kept but detached
(userid NULL), so the trail survives the account.
 2026-07-17 10:54:47 -04:00
cproudlock
bb5308bae0 printedparts: badge resolution honors the employee directory mode
Some checks failed
CI / backend (push) Successful in 1m44s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
The resolver only read the self-hosted directory table, which is empty
at sites running the external HR directory - every kiosk badge fell to
the deny policy. It now branches on employee_directory_mode like the
usb plugin: selfhosted looks up DirectoryEmployee by SSO; external
queries the HR directory via employee_connection, resolving PayNo
badges by their real PayNo column and recovering the employee's SSO.
 2026-07-17 10:39:11 -04:00
cproudlock
d297c5b75d IIS runbook: app pool needs Modify on instance/
Some checks failed
CI / backend (push) Successful in 1m44s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
The plugin registry (instance/plugins.json), uploaded logos, floor
plans, item photos, and print files all write under instance/; with
the app pool at read-only, toggling a plugin in Settings surfaces as
an internal error and every upload fails. Grant Modify in step 7.3
and add the troubleshooting row.
 2026-07-17 10:34:52 -04:00
cproudlock
deb6dd2162 Ignore the publication clone's _transfer bundle folder
Some checks failed
CI / backend (push) Successful in 1m43s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 7s
Details
CI / migrations-mysql (push) Failing after 7s
Details
2026-07-17 10:01:34 -04:00
cproudlock
02ed88c7c5 Merge printedparts: 3D-printed parts storefront, kiosk, labels, alerts
Some checks failed
CI / backend (push) Successful in 1m44s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
The plugin-lab exemplar built end to end: catalog with photos and
print-file revisions, badge-attributed stock ledger, touch kiosk with
an open decrement-only take endpoint (decision record), 1x0.5in bin
labels, low-stock alerts to users/roles/emails, reports with a
reconcile check, per-plugin migrations 0001+0002, contract 0.13.0
(mailer + User/Role on the plugin surface).
 2026-07-17 09:20:43 -04:00
cproudlock
d1357defc4 printedparts: catalog access is printedparts.view-gated
Some checks failed
CI / backend (push) Successful in 1m43s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
Browsing the catalog (item list, detail, file listings) now requires
authentication plus the view permission, and the /printedparts pages
and the label print page require login. Still deliberately open: the
kiosk endpoints per the decision record, the image serve and file
download (img tags and anchor downloads cannot carry a JWT), and the
reports (product-wide jwt-optional convention). Grant
printedparts.view to the roles that should see the catalog.
 2026-07-17 09:19:30 -04:00
cproudlock
96e48e0f50 printedparts kiosk: keypad and entry-panel visual polish
Some checks failed
CI / backend (push) Successful in 1m44s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
The keypad becomes a proper terminal pad: fixed 3-column grid of
rounded square buttons with tabular numerals, press feedback, and
muted Clear/backspace actions. Each manual step (item number, SSO,
quantity) shares one card panel - boxed entry display with placeholder
styling, keypad, and a full-width action button.
 2026-07-17 09:13:27 -04:00
cproudlock
4dfdb167d5 printedparts stage 16: kiosk touch fixes from first hands-on use
Some checks failed
CI / backend (push) Successful in 1m44s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
The tap-anywhere wedge refocus stole focus from the manual-entry field
the moment it was tapped - the handler now only reclaims focus from
dead space, never from a real control. Manual entry works without a
physical keyboard: badge entry uses the TouchKeypad (an SSO is
digits), and item lookup accepts bare digits resolved by row id - the
digits in a minted code are the id, which also keeps labels printed
under an older prefix scannable after the prefix changes.
 2026-07-17 09:04:52 -04:00
cproudlock
aa4bfcd41c printedparts stage 15: print-file revision history + role-based alerts
Some checks failed
CI / backend (push) Successful in 1m44s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
printeditemfiles lands as the plugin's first incremental migration
(0002 on the plugin chain - the ADR-008 payoff). Revisions are
append-only per item: upload assigns the next number, records the
uploader from the JWT, enforces an extension allowlist and a 100 MB
cap; download serves the original filename; a permission-gated delete
covers wrong-file mistakes. The detail page gains the revision table
with a current badge. Unique storedfilename is sized 191 so the index
fits MySQL's 767-byte prefix - the per-plugin chain does not apply the
core env's ROW_FORMAT hook.

Alert recipients gain roles: Role joins the 0.13.0 surface, a role
picker on the settings page, and every active member of the selected
roles is folded into the deduped recipient list.
 2026-07-17 08:47:41 -04:00
cproudlock
26b6b6b32f printedparts: Parts Kiosk link in the sidebar Displays section 
Beside Shopfloor Dashboard and TV Slideshow, opening in a new tab and
shown only while the plugin is enabled - kiosk-style pages get
launched from the Displays group, not the Information nav.
 2026-07-17 08:38:57 -04:00
cproudlock
eab225e1e6 printedparts stage 14: retire/restore in the UI, dashless item codes
Some checks failed
CI / backend (push) Successful in 1m42s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 7s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
Retire button with confirmation on the detail page (item leaves the
storefront and the kiosk rejects its code; ledger history and label
survive), Restore on retired items, and an Include-retired list toggle
with a badge. Restore is its own permission-gated POST - the generic
update still cannot flip isactive. New codes mint as WJRP0042 style
without the dash; existing codes are immutable bin labels and keep
their form.
 2026-07-17 08:35:31 -04:00
cproudlock
a8a6baf979 printedparts stage 13: pick alert recipients from shopdb users
Some checks failed
CI / backend (push) Successful in 1m42s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
Contract 0.13.0 puts the User model on the plugin surface. The
settings page gains a checkbox picker over the user list; selected
users receive low-stock alerts at their account email, merged and
deduped with the free-text address list, inactive accounts skipped,
site alert_recipients still the fallback when both are empty.
 2026-07-17 08:30:04 -04:00
cproudlock
427eb0de8c printedparts stage 12: admin settings page + settings-rail card
Some checks failed
CI / backend (push) Successful in 1m42s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 7s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
PrintedPartsSettings edits the four plugin settings (code prefix,
default threshold, kiosk badge policy, alert recipients) through the
core settings API; the route rides the plugin's router file and the
settings shell nests it into the rail; get_settings_cards contributes
the catalog card while the plugin is enabled.
 2026-07-17 08:25:33 -04:00
cproudlock
df918ed38f printedparts stage 11: low-stock email alerts on threshold crossing
Some checks failed
CI / backend (push) Successful in 1m43s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
Contract 0.12.0: send_email/send_alert join the plugin surface (the
mailer was core-only), PLUGIN-HOOKS and status docs updated, manifest
pins the new floor. The alert fires inside _ledger_write only when a
decrement CROSSES the item's threshold - one alert per depletion,
rearmed by restocking above - and is best-effort after the commit so
mail trouble can never fail a take. Recipients come from
printedparts_alert_email, falling back to the site alert_recipients.
on_enable re-seeds settings idempotently so existing installs pick up
new keys. Crossing/rearm semantics proven by test.
 2026-07-17 08:15:50 -04:00
cproudlock
fc0d48a6a7 printedparts stage 10: closeout - lab guide rewritten from the real build
Some checks failed
CI / backend (push) Successful in 1m42s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
The lab is now a build-along mirroring what actually happened: ten
stages, each with the goal, the divergences, a see-it-work check, and
the errors genuinely hit while building (empty Migration error from a
broken model import, the migration-guard KeyError, the missing Lucide
icon, nested-app-context test writes, Decimal sums, and the authz
sweep catching the deliberately open kiosk take). That last one gets
its explicit EXEMPT_ENDPOINTS entry with a pointer to the decision
record - the net stays, the exception is reviewable. Full suite: 993
backend tests, 49 vitest, frontend build, naming hook, all green.
 2026-07-17 08:11:36 -04:00
cproudlock
b68e927ef6 printedparts stage 9: reports - stock w/ reconcile, consumption, by-person
Some checks failed
CI / backend (push) Failing after 1m42s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
Three jwt-optional endpoints with ?format=csv, merged into the reports
hub via get_reports while the plugin is enabled. The stock report's
ledgerdelta column is the reconcile check: 0 for every item whose
stock moved through the ledger, nonzero for anything that bypassed it
(the hand-seeded dev rows demonstrate the catch). MySQL SUM returns
Decimal - cast to int or the delta serializes as a string.
 2026-07-17 08:04:09 -04:00
cproudlock
6439d1ccd9 printedparts stage 8: 1x0.5in bin labels
Some checks failed
CI / backend (push) Failing after 1m42s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 7s
Details
CI / migrations-mysql (push) Failing after 7s
Details 
New public print view at /print/printedparts-labels following the
plugin-owned USB label precedent: multi-select with per-item copies,
CODE128 of the item code via JsBarcode (a QR at this size is at the
edge of scanner tolerance), one label per page on 1in x 0.5in roll
stock via a new @page size. The Detail page's Bin Label button
preselects its item through ?item=<id>; the list header gains a batch
Print Labels button.
 2026-07-17 08:00:35 -04:00
cproudlock
6ed3da1b64 printedparts stage 7: the kiosk - scan bin, scan badge, keypad, take
Some checks failed
CI / backend (push) Failing after 1m43s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
Two open endpoints: an item lookup by scanned code and the take POST -
the product's first unauthenticated write, held to the decision
record's bar (decrement-only, badge-attributed server-side, bounded,
physically rate-limited; justification in the plugin README). The
/parts-kiosk route is a full-screen no-auth view beside /shopfloor: a
hidden always-focused input consumes keyboard-wedge scans for
whichever step is active, TouchKeypad (net-new 3x4 grid) takes the
quantity, and a success screen resets after a few seconds. Manual
type-in fallbacks cover damaged labels. Kiosk test proves open access,
the over-take guard, the badge policy, and cache==ledger afterward.
 2026-07-17 07:49:13 -04:00
cproudlock
d6a78a72ff printedparts stage 6: RBAC - declared permissions gate every mutation
Some checks failed
CI / backend (push) Successful in 1m42s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
get_permissions declares view/create/edit/delete/restock (seeded on
install/enable and by flask seed permissions); every write route adds
require_permission on top of jwt_required. New test proves
authentication alone is not authorization: a role-less member gets
403 where an admin succeeds.
 2026-07-17 07:42:07 -04:00
cproudlock
6dfc8906c4 printedparts stage 5: the ledger - restock/adjust with badge attribution
Some checks failed
CI / backend (push) Failing after 1m42s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
Badge resolver copied from the USB contract (SSO digits, 0<digits>BZ
PayNo wrap) with names from the employees directory and the
unknown-badge policy setting; deliberately copied rather than
cross-imported so the contract test stays green. Restock and adjust
write the ledger row and move the cached quantity in one commit -
the single-commit invariant every write path must use. Adjust
requires a reason and refuses to drive stock below zero. Detail page
gains Restock/Adjust modals. Seven tests cover minting, the
cache==ledger invariant, badge shapes, policy toggle, and auth.
 2026-07-17 07:41:18 -04:00
cproudlock
cb367a38f9 printedparts stage 4: catalog mutations, item photos, detail + form
Some checks failed
CI / backend (push) Failing after 1m40s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
POST/PUT/DELETE for items: create mints the itemcode from the
configured prefix plus the flushed row id, update refuses
quantityonhand (ledger-managed - restock/adjust arrive next stage),
delete soft-retires. The image upload/serve/delete trio replicates the
models.py pattern into instance/printedpartsimages/ with a public GET.
PrintedItemDetail follows the unified detail skeleton (hero photo,
info list, transaction history table); PrintedItemForm covers
create/edit plus photo management on edit.
 2026-07-17 07:36:54 -04:00
cproudlock
d1c844d533 printedparts stage 3: read API + list page (first visible win)
Some checks failed
CI / backend (push) Successful in 1m39s
Details
CI / naming (push) Successful in 1s
Details
CI / frontend (push) Successful in 8s
Details
CI / migrations-mysql (push) Failing after 8s
Details 
GET /items (paginated, search across code/name/description/bin,
lowstock filter) and GET /items/<id> with recent transactions, both
open reads. printedpartsApi client, router file repointed at the
renamed views, PrintedItemsList with image thumbs and a red/green
quantity badge against the per-item threshold. Nav entry '3D Parts'
with a new 'box' Lucide icon mapping (the sidebar renders nothing for
unknown icon names - lab gotcha).
 2026-07-16 17:10:42 -04:00
520 changed files with 85697 additions and 6810 deletions
Expand all files Collapse all files

23
.env.example
View File

@@ -49,6 +49,10 @@ MYSQL_ROOT_PASSWORD=CHANGE_ME_ROOT_PASSWORD
MYSQL_PASSWORD=CHANGE_ME_APP_PASSWORD
MYSQL_PORT=3306
API_PORT=5001
# Air-gapped deploy only (docker-compose.airgap.yml): the loaded image tag,
# which MUST match what build-offline-bundle.ps1 -Version produced. Ignored by
# the connected build template (docker-compose.yml builds from source).
IMAGE_TAG=0.7.0
# ---- Zabbix integration (optional, for printer supply monitoring) ----
ZABBIX_URL=
@@ -61,6 +65,25 @@ ZABBIX_TOKEN=
# COLLECTOR_API_KEY=
# COLLECTOR_API_KEY_COMPUTERS=
# ---- Trusted plugin publisher keys (ADR-013, optional) ----
# Public-key PEM paths (OS path separator: ':' on Linux, ';' on Windows) used
# to verify signed plugin artifacts. Delivered with this config, NEVER from the
# plugin shelf. Empty on a site that does not adopt marketplace plugins.
# PLUGIN_TRUSTED_KEYS=/etc/shopdb/keys/curator.pub
#
# Enforce signatures: a plugin only loads/migrates if its tree matches a
# trusted signature. Default off. Turn on only after stamping plugins
# (flask plugin stamp-bundled) and pinning keys above.
# PLUGIN_REQUIRE_SIGNED=false
#
# Dev-only: directories whose UNSIGNED plugins are trusted, honored ONLY under
# DEBUG/TESTING (external-repo/symlink dev). Production ignores this.
# PLUGIN_DEV_TRUST_DIRS=/home/dev/my-plugin-repo
#
# Read-only folder the app pulls plugin artifacts from (a SharePoint-synced or
# copied shelf). The app reads this folder; it never speaks any network.
# PLUGIN_SHELF_DIR=/srv/shopdb/plugin-shelf
# ---- Employee directory database (optional, read-only) ----
# Separate HR/employee lookup DB consumed by the notifications plugin and the
# public shopfloor kiosks. Leave unset if the feature is not used; there is no

5
.gitattributes vendored
View File

@@ -1 +1,6 @@
.env filter=git-crypt diff=git-crypt
# Shell scripts must stay LF so Git Bash on Windows can run them
*.sh text eol=lf
scripts/check-naming-and-style.sh text eol=lf
.githooks/pre-commit text eol=lf

5
.githooks/pre-commit Executable file
View File

@@ -0,0 +1,5 @@
#!/usr/bin/env bash
# Committed pre-commit hook. Activate per clone with:
# git config core.hooksPath .githooks
# Runs the naming/style gate; blocks the commit on failure.
exec bash "$(git rev-parse --show-toplevel)/scripts/check-naming-and-style.sh"

87
.github/copilot-instructions.md vendored Normal file
View File

@@ -0,0 +1,87 @@
# Copilot instructions for shopdb-flask
Follow these when suggesting code. They are enforced by a naming/style hook and
by CI (`.github/workflows/ci.yml`) - suggestions that break them fail the build.
`CONTRIBUTING.md` is the full authority; this is the short version.
## Naming (LOCKED - the hook rejects violations)
- **DB tables**: lowercase, concatenated, plural. No underscores, no dashes.
`machines`, `networkdevices`, `businessunits` - NOT `machine_types`, `BusinessUnits`.
- **DB columns**: lowercase, concatenated, singular. No underscores.
`machineid`, `lastzabbixsync`, `isactive` - NOT `machine_id`, `last_zabbix_sync`.
- **Foreign keys**: referenced table (singular) + `id`: `locationid`, `vendorid`.
- **Booleans**: `is`/`has` prefix: `isactive`, `isshopfloor`.
- **Index names**: `idx_<table>_<column>` (underscores allowed here only).
### Python
- A variable, attribute, function, or dict key that holds a DB value MUST match
the column name exactly - do NOT convert to snake_case.
Column `machineid` -> `Machine.machineid`, `{"machineid": 1}`, local `machineid`.
- Pure code that does NOT mirror a DB field uses normal snake_case (PEP 8):
`loop_count`, `current_user`, `validate_input()`.
- Classes: PascalCase, spelled out (`NetworkDevice`, `AssetType`).
### JavaScript / Vue
- A JS variable holding an API field value matches the API key exactly - do NOT
camelCase it. API `{"machineid": 1}` -> `response.machineid`, never `machineId`.
- Components: PascalCase (`AssetDetail.vue`). CSS classes: lowercase-with-dashes.
### API
- Endpoints: lowercase plural nouns, no underscores/dashes: `/api/networkdevices`.
- Query params + response keys match column names: `?locationid=5`,
`{"machineid": 1, "lastzabbixsync": "..."}`.
### Allowed acronyms only
Universal: id url api http https json jwt sql os ip dns csv pdf cors ttl uuid
html css orm. Domain: cmm cnc pc usb vnc winrm ssh ssl tls tcp udp smtp ldap
vlan sso dnc focas clm mtconnect. Anything else: spell it out.
### Banned shorthand
Never use `cfg ctx mgr req res env util helper` or `db` as a standalone variable
name. Spell out: `config context manager request response environment utilities`.
`_bp` is fine only as a suffix with a meaningful prefix (`printers_bp`).
## Style (ASCII only)
- NO emojis anywhere - code, comments, strings, UI.
- NO em-dashes, en-dashes, Unicode arrows, or smart quotes. Plain ASCII only.
- Comments default to NONE. Add one only when the WHY is non-obvious; keep inline
`#`/`//` comments terse. Docstrings stay normal English.
- Dark theme is the default; keep UI functional and professional.
## Architecture (do not violate)
- **Plugins are the product.** Plugin code lives in `plugins/<name>/{models,api,services,schemas}/`
with a `manifest.json` (single source of truth: name, version, dependencies,
api_prefix) and a `BasePlugin` subclass in `plugin.py`.
- **Plugins never import core internals.** Use the contract surface `shopdb.api`
(e.g. `from shopdb.api import db, Asset, success_response`). Adding to that
surface is a contract-version bump + a `docs/PLUGIN-HOOKS` update in the same PR.
- **Migrations, never `db.create_all()`.** The core Alembic chain is in
`migrations/versions/`; each plugin owns its own chain under
`plugins/<name>/migrations/`. New schema = a new migration with an idempotent
guard and a real downgrade. Migrations must run clean on strict MySQL 8.
- **Asset model is the platform contract.** Physical things are an `Asset` plus a
plugin subtype row linked by `assetid` (FK, `ON DELETE CASCADE`). Consumables
with quantities are standalone tables, not assets.
- **Ledger pattern**: a cached `quantityonhand` moves in the SAME commit as the
signed transaction row it reflects.
## Before you finish a change
Run the three gates (CI runs the same):
```
python -m pytest tests/ -q
cd frontend && npx vitest run && npm run build && cd ..
bash scripts/check-naming-and-style.sh
```
Commits: short present-tense subject, body says WHY. No AI/tool attribution in
commit messages, code comments, or docs.

106
.github/workflows/ci.yml vendored Normal file
View File

@@ -0,0 +1,106 @@
# CI for shopdb-flask on GitHub Actions.
#
# Mirrors the internal CI pipeline. Four jobs on push + pull_request:
# backend - pytest (SQLite via TestingConfig, no DB service needed)
# naming - the CONTRIBUTING.md naming/style gate
# frontend - vitest + Vue build
# migrations-mysql - the REAL multi-site deploy path: fresh flask db upgrade
# + every plugin's chain on utf8mb4 MySQL 8, idempotent on
# a second run. The pytest suite only exercises SQLite
# create_all(), so this is what catches an Alembic
# regression on MySQL before it ships.
name: CI
# Jobs run on the org's self-hosted "arc-runner-set" (enterprise
# ge-aerospace-runner-group, Linux). GitHub-hosted runners are blocked by the
# org IP allow list (hosted Azure runner IPs are not allow-listed -> checkout
# 403), so ubuntu-latest cannot be used here. arc-runner-set checks out from an
# internal allow-listed IP and, being Linux, still supports service containers.
on:
push:
pull_request:
jobs:
backend:
runs-on: arc-runner-set
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.14'
cache: pip
- run: pip install -r requirements-dev.txt
- run: python -m pytest -q
naming:
runs-on: arc-runner-set
steps:
- uses: actions/checkout@v4
- run: bash scripts/check-naming-and-style.sh
frontend:
runs-on: arc-runner-set
defaults:
run:
working-directory: frontend
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'
cache: npm
cache-dependency-path: frontend/package-lock.json
- run: npm ci
- run: npx vitest run
- run: npm run build
lean-build:
# ADR-013 Phase 5: prove a per-site build carries only its chosen plugins.
# Builds a lean site (machines + printers) and asserts an omitted plugin's
# code is absent from the bundle - the delete-a-plugin guarantee in CI.
runs-on: arc-runner-set
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'
cache: npm
cache-dependency-path: frontend/package-lock.json
- run: cd frontend && npm ci
- name: Build lean site (machines + printers)
run: |
printf '{ "site": "ci-lean", "plugins": ["machines", "printers"] }' \
> /tmp/lean-profile.json
bash scripts/build-site.sh /tmp/lean-profile.json /tmp/leansite
- name: Assert omitted plugin code is absent, chosen present
run: |
assets=/tmp/leansite/frontend-dist/assets
for code in PartsKiosk ManifestEditor USBLabelBatch KnowledgeBaseDetail; do
if grep -rqoh "$code" "$assets"/*.js; then
echo "FAIL: omitted-plugin code '$code' leaked into the lean bundle"
exit 1
fi
done
for code in MachineDetail PrinterDetail; do
grep -rqoh "$code" "$assets"/*.js || {
echo "FAIL: chosen-plugin code '$code' missing from the lean bundle"
exit 1; }
done
# Core frontends (no manifest, e.g. applications) must ship in EVERY
# build regardless of SITE_PLUGINS, or a lean site loses a core page.
grep -rqoh "ApplicationsList" "$assets"/*.js || {
echo "FAIL: core page 'ApplicationsList' missing from the lean bundle"
exit 1; }
test -d /tmp/leansite/plugins/machines
test ! -d /tmp/leansite/plugins/printedparts
echo "lean build verified: only chosen plugins present"
# NOTE: the MySQL-8 migration/seed job (fresh `flask db upgrade` + every
# plugin chain + strict-mode seeders on a real MySQL 8) runs on the internal
# CI server, which supports service containers. The org's arc-runner-set is
# Kubernetes/ARC without docker-in-docker, so GitHub Actions service
# containers ("services: mysql") are unavailable here ("Job Container is
# required"). That coverage stays on the internal CI rather than being
# duplicated on GitHub.

9
.gitignore vendored
View File

@@ -28,7 +28,11 @@ env/
# IDE
.idea/
.vscode/
.vscode/*
# Share the team's launch/tasks/extensions; keep personal settings out.
!.vscode/launch.json
!.vscode/tasks.json
!.vscode/extensions.json
*.swp
*.swo
*~
@@ -76,3 +80,6 @@ secrets.yml
*_secrets
credentials.json
scripts/site_imports/wjf/idmap.json
# work-PC publication clone: bundle drop folder for the transfer pipeline
_transfer/

9
.vscode/extensions.json vendored Normal file
View File

@@ -0,0 +1,9 @@
{
"recommendations": [
"ms-python.python",
"ms-python.vscode-pylance",
"Vue.volar",
"dbaeumer.vscode-eslint",
"ms-azuretools.vscode-docker"
]
}

30
.vscode/launch.json vendored Normal file
View File

@@ -0,0 +1,30 @@
{
"version": "0.2.0",
"configurations": [
{
"name": "Flask API (:5001)",
"type": "debugpy",
"request": "launch",
"module": "flask",
"cwd": "${workspaceFolder}",
"env": {
"FLASK_APP": "shopdb",
"FLASK_ENV": "development",
"FLASK_DEBUG": "1"
},
"args": ["run", "--port", "5001", "--no-reload"],
"jinja": true,
"justMyCode": false,
"console": "integratedTerminal"
},
{
"name": "Pytest (current file)",
"type": "debugpy",
"request": "launch",
"module": "pytest",
"cwd": "${workspaceFolder}",
"args": ["${file}", "-v"],
"console": "integratedTerminal"
}
]
}

46
.vscode/tasks.json vendored Normal file
View File

@@ -0,0 +1,46 @@
{
"version": "2.0.0",
"tasks": [
{
"label": "Backend: flask run (:5001)",
"type": "shell",
"command": "${workspaceFolder}/venv/bin/flask run --port 5001",
"options": {
"cwd": "${workspaceFolder}",
"env": { "FLASK_APP": "shopdb" }
},
"windows": {
"command": "${workspaceFolder}\\venv\\Scripts\\flask.exe run --port 5001"
},
"isBackground": true,
"problemMatcher": []
},
{
"label": "Frontend: npm run dev (:5173)",
"type": "shell",
"command": "npm run dev",
"options": { "cwd": "${workspaceFolder}/frontend" },
"isBackground": true,
"problemMatcher": []
},
{
"label": "Dev site (backend + frontend)",
"dependsOn": [
"Backend: flask run (:5001)",
"Frontend: npm run dev (:5173)"
],
"dependsOrder": "parallel",
"problemMatcher": []
},
{
"label": "Check: naming + tests + build",
"type": "shell",
"command": "bash scripts/check-naming-and-style.sh && venv/bin/python -m pytest tests/ -q && cd frontend && npx vitest run && npm run build",
"windows": {
"command": "bash scripts/check-naming-and-style.sh && venv\\Scripts\\python -m pytest tests/ -q && cd frontend && npx vitest run && npm run build"
},
"options": { "cwd": "${workspaceFolder}" },
"problemMatcher": []
}
]
}

549
CHANGELOG.md
View File

@@ -10,6 +10,538 @@ ADR-007 and ADR-002.
## [Unreleased]
## [0.10.0] - 2026-08-17
A security release. The Windows installer left the directories it creates
writable by every logged-on user, which made two of the files it puts there a
local privilege escalation; that is fixed and verified on Windows. Alongside it,
the work that makes this product genuinely adoptable by a second site: one
site's hostnames, share paths and networks are out of the product code and out
of the published documentation, and the rule that keeps them out now fails the
build rather than printing a warning nobody read.
The plugin contract moved to 0.19.0, which is a BREAKING change - see the
Changed section and CONTRACT-STABILITY.md before upgrading a plugin you wrote.
### Added
- Applications accept an uploaded image and an uploaded installer, rather than
only a path typed into a text box. The path field kept working only while
someone maintained a share by hand, and a link to a file nobody could still
find was indistinguishable from a link to one that worked. The installer
download is gated on `applications.view` because it is licensed vendor
software; the image is public, since tiles render before login.
- Warranty records accept an uploaded proof document - the invoice or
certificate that actually settles a claim. The documentation path stays as it
was: it points at a vendor portal or a share, which an upload cannot replace.
Downloads require `warranty.view`, since a proof carries pricing and a service
tag, and it comes back under the vendor's own filename so it is recognisable
in a support ticket.
- Toner forecast report: days until each cartridge runs out, soonest first,
from Zabbix level history. Ordering by days left rather than by level is the
point - a cartridge at 60% draining fast needs ordering before one sitting at
8% that has not moved in a month. It also counts how many cartridges each
printer has been through in the window, which is the number a purchasing
conversation asks for. Printers with no honest estimate are listed
separately with the reason (too little history, gauge has not moved, replaced
recently) instead of being sorted in as zero or as safe, and an unreachable
Zabbix says so rather than rendering an empty table that reads as "nothing is
due".
- Fiscal week under the site name in the sidebar and on the shop-floor board,
as the classic ASP site showed it. People quote a week number across both
sites, so the arithmetic is a port of the old one (ISO 8601: week 1 contains
4 January, the week's Thursday decides the year) rather than a fresh
interpretation.
- Backup history distinguishes CAPTURED from LAST VERIFIED. Captured is when a
configuration was written; verified is when a PC last posted it and the hash
still matched. They differ by design, because an unchanged config writes no
new revision - so a machine checked daily for a year shows one revision from
last spring, and previously read as abandoned. Shown on the history page and
on the asset panel; only the current revision carries it, since an older one
is not what the PC holds now.
- A backup now records that it was CHECKED, not only that it changed. Dedup
means an unchanged configuration writes no revision, so the stored timestamps
moved only on a change: a machine whose settings had been stable for six
months was indistinguishable from a machine whose backup died six months ago.
The only evidence a backup still ran was a line in a log file on the PC.
`lastseenat` is touched on every matching post, including the no-op, and is
backfilled from the timestamps that already exist.
- The low-supplies report lists only the cartridges that need replacing. A
printer with one empty black and three full colour cartridges was listing all
four, so the reader had to hunt for the problem inside the row. The report
exists to answer "what needs replacing".
- Dashboard cards for enforcement failures, machine numbers claimed by two PCs,
PCs not reporting, backups that have stopped, printer supplies, expiring
warranties, machines out of service, and the notifications currently showing
on the floor. The dashboard now renders cards plugins declare, which it never
did before - five plugins had been declaring widgets into a void, pointing at
components nobody wrote. Cards are ordered by severity, hide themselves when
there is nothing to report, gate on a permission, and each fetches
independently so one broken endpoint cannot blank the board.
- Tech Tools: a section for the small conversions and code generation that
otherwise happen in a browser tab on somebody's phone. Starts with barcode and
QR codes laid out for the label stock actually in the printers.
- Kiosks report what they are. The dispatcher already read
`C:\Enrollment\display-type.txt` to pick its target; now it tells ShopDB, so
the Asset column on an enforcement report shows a reported value instead of
one inferred from a mapping table nobody filled in.
- Enforcement reports carry what the PC is, where it is, and whether it is
backed up: the host links to its own page, the Asset column names the machine
or instrument the PC controls, a map pin shows the location on hover, and the
backup column judges the chain rather than printing its last date.
- `flask relationships audit`, read-only, listing links that cannot both be
true, plus an inverse guard that refuses to create one.
- An employees endpoint that says WHY a name did not resolve, rather than
returning nothing and leaving the caller to guess.
- The display client module updates itself through the manifest, so a client
change no longer means hands on every kiosk.
- `docs/START-HERE.md` routes a reader by what they came to do, and is now the
wiki's landing page - which previously recommended the one install document
the docs themselves forbid for a new site. `docs/FLEET-ARCHITECTURE.md` is the
page nothing else could assume: how the server, GE-Enforce, the asset
reporter, the backup collectors and EventSaver relate, where credentials live
at both ends, and which of the five to open for a given symptom.
- `docs/ADOPTING-AT-ANOTHER-SITE.md`: how another site repoints the asset
reporter and the EventSaver screensaver at its own ShopDB, with worked
examples for Intune (including Machine Configuration/DSC), a GE-Enforce
manifest entry, and manual installation. Both tools were already built to be
repointed - the URL, the key and the targeting are inputs - but nothing said
so, and the two things that catch people out are written down: EventSaver's
compiled-in fallback path is not neutral, and a config enforced by hash
reverts a hand edit by design.
- `docs/PROJECT-MAP.md`, generated by `scripts/gen_project_map.py`: versions,
the plugin inventory, every Alembic chain head, the ADR index and the size of
the codebase, derived from the code. The hand-written equivalents had drifted
by two contract releases and 400 tests.
### Changed
- Plugin contract **0.19.0**: `get_dashboard_widgets` returns data and shape,
not a component name. The old shape named a Vue component per widget, which
cannot survive a lean build (ADR-013) and had five plugins pointing at
components nobody wrote. The change itself shipped earlier without a bump and
with `BasePlugin` and PLUGIN-HOOKS.md still teaching the dead shape; this
records it and corrects both.
- Plugin contract **0.17.0**: `DashboardDefault` is part of the `shopdb.api`
surface, so a plugin can resolve a display without reaching into core.
- Plugin contract **0.18.0**: one name per display role, and it is the kiosk's
own (`Dashboard`, `Lobby`, `3DPrintRoom`). Server and dispatcher had three
vocabularies for the same three kiosks, and a display could report a role core
could not store. `normalize_display_role` resolves case and the retired
spelling; an unknown value is kept verbatim so a typo stays visible.
Contract versions are their own series (ADR-002) and belong in this file;
0.17.0 and 0.18.0 shipped without an entry, which is what this note corrects.
- Publishing a GE-Enforce manifest is refused when PCs reporting for that scope
run an older client library than the manifest needs, naming the hosts. A minor
version bump that NARROWS behaviour is not backward compatible: an old library
reads a gated entry as ungated and installs every version it cannot detect.
- Shadow mode runs inside the enforce cycle instead of on its own schedule. The
share is mounted only for the length of a cycle, so a task on its own clock
woke to a missing drive and reported 0 installed / 0 skipped / 0 failed, which
is indistinguishable from a healthy no-op.
- A GE-Enforce report is judged stale on both clocks, the server's and the
client's asserted check-in, so a PC cannot mask real silence with a future
timestamp. A PC that has gone quiet stops reading as healthy.
### Fixed
- Documentation a reader could not follow. 98 curl examples single-quoted
`$TOKEN`, so the shell never expanded it and the server answered 422; a JSON
body carried a `//` comment, which made the server report a missing hostname
about a body that has one; the IIS install omitted `flask plugin upgrade-all`,
which is the 1054 error a deploy hits days later; a pilot loop exited 1 on
every iteration; and the external-plugin guide pinned a contract version six
releases stale, so a plugin built by following it is refused at startup. Nine
documents now point at the generated project map rather than typing a version,
and a test enforces that. The API inventory gained the 22 routes that were
serving traffic undocumented, with a parity test against the live URL map so
the next one cannot slip in silently.
- **Every plugin migration chain re-ran its head on each MySQL deploy.** Core's
Alembic env got a `connection.commit()`; the per-plugin template did not, so
the last migration of every run lost its version stamp while its DDL survived.
Invisible for as long as every head happened to be idempotent - and two were
not. The backups head cleared a column for every row, and the geenforce head
added a column unconditionally and failed on the second attempt. Both are now
bounded and guarded, and the template commits.
- A part-marker PC reported 200 on the cycle that created its marker and 500 on
every cycle after. Both of the links it makes were looked up by label, so a row
made by hand or by the legacy import was invisible and the insert violated the
uniqueness of (source, target, type); and the marker's asset number is derived
from the PC, so it could already be taken.
- A CMM reported the same way: the instrument IS the reported bay, so adoption
was relabelling the row the machine sync owns and finding it by that label.
- A reported identifier is matched exactly rather than with `ilike`, so `_` and
`%` in a file on a shopfloor PC are no longer wildcards. `MT-600_` had been
adopting MT-6001, and a bare `%` adopted whatever active asset came first.
- Measuring tools are adopted before they are minted. Minting derived the asset
number from the HOSTNAME, so a permanent instrument took the identity of
whichever PC drove it that week, and the collector could not see a tool it had
not created - which left legacy instruments shadowed by minted twins.
- An unanchored rsync exclude in the export was matching at any depth, so a
whole plugin was silently dropped from the published payload. Root excludes
are anchored and the export now aborts if a plugin manifest fails to survive.
- Backup revisions are ordered in Python, not with `ORDER BY ... NULLS LAST`,
which SQLite accepts and MySQL rejects.
### Security
- The Windows installer breaks ACL inheritance on the roots it creates. A
directory made directly under `C:\` inherits an inherit-only Modify grant for
Authenticated Users, so `C:\shopdb-flask`, `C:\Python314` and `C:\MySQL84`
were writable by any logged-on user. Two files inside them make that a local
privilege escalation: `shopdb-admin.ps1`, which self-elevates, and
`instance/config.py`, which the app executes unconditionally. The MySQL data
directory is locked down as well, since it holds the password hashes. A Python
installation this installer did not create is reported rather than seized.
- Site-specific values no longer ship in product code (ADR-015). The backups
share root defaults to blank instead of one site's file server, the shadow
client no longer overwrites a site's own ShopDB URL in the registry on every
cycle, the kiosk dispatcher takes its URL from the `site_base_url` setting and
refuses rather than guessing, and one site's hostname, FQDN and internal
subnets are out of the documentation, the generated API specs and the UI
placeholders. The ADR-015 scanner now covers PowerShell, the installer, the
seeds and generated JSON, case-insensitively, and FAILS the build; deliberate
organisation-wide defaults are declared with an `ADR-015-OK` marker. Both
publication gates carry the site patterns.
## [0.9.0] - 2026-08-11
Driven by a fleet that had been failing quietly. A bay had been returning 500
to every collector report since it was imaged, backups were accumulating
duplicate revisions on any machine number more than one PC reported against,
and a setting that looked configurable had never been readable by the script
that consumed it. The theme of the release is making those failures visible:
each fix ships with the check, report or document that would have surfaced it.
### Changed
- The toner forecast is read as an order, not as a table. It opens with what to
buy - grouped by part number, with a quantity - because two cartridges of the
same part in different printers is a quantity of two, and that was a number
the reader had to work out by hand. Below it, cartridges sit in urgency bands
(empty / two weeks / thirty days / later) rather than in one long sortable
list, since the question being asked is which pile a thing is in. A row is
now a cartridge rather than a printer, carries its own part number and a
level bar, and everything past "empty" starts collapsed. Cartridges with no
part mapped are counted on one line instead of a line each: they still have
to be ordered, but not from this page - the job is to map them.
### Fixed
- The toner forecast was reading the wrong end of the window. Zabbix applies a
query's `limit` to the whole answer rather than to each item, and combined
with an ascending sort that kept the OLDEST rows: a four-cartridge printer
polled every five minutes writes over 100k readings in 90 days, so a "90 day"
forecast was fitted to the first few days of the window and nothing since.
That is where "4 days left" beside a cartridge at 20% came from - the rate
was real, it just described a cartridge thrown away in May. Long windows now
read hourly trends, which is the table meant for this and is a tenth of the
rows; short ones read raw history, newest first, with a budget scaled per
item.
- The forecast now counts down from the level it displays. It showed the live
reading but computed days-left from the last stored one, so the two could
disagree by a whole cartridge. A live level far above the stored run is
treated as a swap that happened since the last reading rather than as a
collapse in the burn rate.
- A cartridge at or below 5% reads as empty rather than as a slow drain. At 1%
losing a tenth of a point a day the arithmetic said ten days; the printer is
out of toner, and it is the first thing that should be ordered.
- Days-left in the forecast is per cartridge again. It was rendered in a cell
spanning the printer's rows, which put the printer's soonest figure beside
every supply it had - a cartridge at 20% displaying "4 days" that belonged to
the black beside it, and a cartridge at 1% displaying weeks that belonged to
nothing on that row at all.
- Supplies typed as floats were dropped from the forecast on any printer that
also had an integer-typed one. The two live in different Zabbix history
tables and the fetch stopped at whichever answered first, so half a printer's
cartridges silently had no history at all.
- A shop-floor PC that reported the machine number of a machine ShopDB already
knew got a 500 from the collector, on every report, forever. The reported
machine number was being written to the PC's own `assets.assetnumber`, which
is uniquely indexed and already held by the machine, so the insert failed with
a duplicate-key error and the whole report was lost: operating system, boot
time, applications, printers and access protocols never landed. The machine
number now identifies the machine only. A new PC takes its hostname as its
asset number, which is what the existing data already does, and an existing
PC's asset number is never overwritten.
- Configuration backups piled up duplicate revisions on any machine number
reported by more than one PC. A posted config was compared against the latest
revision for the machine, so where several PCs share a number - the part
markers do, and their configs differ by COM port - each PC's post differed
from whichever PC posted last, nothing deduped, and the table grew by a row
per PC per collection cycle. A backup chain is now per source PC, so an
unchanged config is a no-op again and each PC keeps its own history against
the machine it belongs to. Retention prunes each chain separately, so a busy
PC's revisions can no longer push out a quiet PC's only backup, and the
revision diff compares against the same PC's previous revision rather than
another PC's.
- The backup collection interval could not actually be changed. The collecting
script reads `backups_intervalhours` from the public settings endpoint before
it holds any credential, but the plugin did not declare the key as public, so
the endpoint never returned it and the script fell back to its built-in 24
hours - silently. The setting looked configurable in the UI and was not.
- A backup export containing a value with an empty right-hand side (`Name=`)
failed to parse, and with it the whole file, so that machine could never be
backed up. The form is not strictly legal but occurs in real exports. It is
now read as an empty string, preserving the value name.
- The 3D parts kiosk label prefix never appeared on the kiosk. The kiosk runs
logged out, and an unauthenticated read of a setting is limited to an
allowlist the key was not on, so the kiosk got a 404 and fell back to no
prefix. An admin previewing the same page while logged in saw it, which is
why it looked like it worked. A plugin now declares which of its settings a
logged-out page may read.
- The same setting also looked like it would not save. The row did not exist on
a site that installed the plugin before the setting was added, so the first
save created it - under a placeholder category, where the settings page, which
lists by category, could no longer see it. The value was in the database the
whole time. A first-time save now files the key under the category its owner
declares. Any row already misfiled is repaired in place on upgrade, value
untouched. The 3D parts alert email and the employee directory host were in
that state.
- Settings added in a later plugin version never reached a site that installed
an earlier one. The seeding ran from `on_install` / `on_enable`, which fire
only on a state transition, so neither ran again on an upgrade - and the
comment claiming enable ran every upgrade cycle was wrong.
`flask plugin upgrade-all` now seeds missing declared settings, without
touching values a site has configured.
### Added
- A PC reporting a machine number is now linked to that machine. Until now the
number was collected and then discarded, so a replaced PC never took over its
bay: the retired PC kept the link and everything that walks PC to machine, the
warranty machine column and the DNC info card among them, pointed at hardware
that had been pulled out. The link is a `controls` relationship carrying a
`collector:machine` origin label, so a link made by hand is never touched.
Superseding a PC archives the old link rather than deleting it, keeping the
history of which PC ran a machine and when. The retired PC's status is
deliberately left alone: the collector cannot tell whether it was shelved,
sent for repair or re-imaged for another bay, so a person decides.
A second PC reporting a machine another PC already holds is treated as a
claim, not a handover. A PC imaged for a machine carries that number before
it ever reaches the floor, so the PC actually running the machine keeps the
link while it is still reporting and still In Use, and the challenger is
recorded as a dormant link. The handover completes on its own once the old PC
has been quiet for a day, or immediately when someone moves it off In Use.
Without this the two PCs traded the link back and forth on every collector
cycle.
Alerts for both cases go out by email and webhook, gated on a new
`computers_machinelink_alerts` setting that ships OFF: a site may legitimately
run several PCs on one machine number, and there the alerts fire on correct
data. The collector response warns either way.
- A PC that drives a subordinate device is now declared, not coded. Two cases
had arrived separately and been written twice: a metrology PC drives a CMM or
Keyence unit that becomes a measuring-tool asset, and a part-marker PC drives
a marker that becomes a machine asset filed under its operation. Both mint a
device, link the PC with `controls` and archive that link on a re-image - one
mechanism with different nouns. They are now one map, and a site can add or
retarget an entry through a `subordinatedevice_<pctype>` setting, so the next
case - two marking lasers or two wax-trace units on one machine number - is
configuration rather than a code change. A malformed override falls back to
the default instead of failing the collector push.
- `flask relationships check-shared-machines` reports machine numbers claimed
by more than one PC, and says which are modelled and which are faults. Several
devices genuinely sharing a number and two PCs mis-numbered at imaging look
identical from the outside; the difference is whether anything is filed under
the operation, which is what the check reports. Read-only.
- A part marker is its own asset. Several Telesis markers can serve one
operation number, so treating the operation as the marker collapsed separate
devices into one record: their configurations, which differ by COM port,
overwrote each other in the backup history, and no question about an
individual marker - how many there are, which port one is on, which one
failed - could be asked at all. There is one marker per PC, so the collector
now mints a Part Marker machine asset for a marker PC the same way it already
mints a CMM or a Keyence unit for a metrology PC: the PC `controls` the
marker, and the marker is `partof` the operation it serves. Because `controls`
propagates through `partof`, control of the operation still follows from
controlling its marker, and two markers on one operation no longer contest a
link only one of them can hold. Backups from a marker PC file against the
marker. Moving a marker to another operation archives the old membership
rather than deleting it.
- Plugin contract 0.16.0: `get_settings_defaults()` lets a plugin declare the
settings it owns (key, default, type, category, description, and whether an
unauthenticated caller may read it). See `docs/PLUGIN-HOOKS.md`.
## [0.8.1] - 2026-08-05
Everything here shipped after v0.8.0 was tagged the same morning, driven by two
sites putting real data in for the first time. Most of it is defects that only
appear when somebody who did not build the software tries to use it.
### Fixed
- A blank optional code could be saved once and never again. A unique nullable
column accepts any number of NULLs but exactly one empty string, so the second
business unit with no code collided with the first and returned 500 from a
field the form correctly showed as optional. Blank now stores as NULL, which
covers every unique nullable column - asset numbers, hostnames, item codes,
subnet names, gage-lab tags - not just the one that was reported. A genuine
duplicate answers 409 with a readable message instead of a bare 500.
- A lobby display with exactly two slides never changed between them. The
slideshow was started twice, and two timer chains advancing one slide each
landed back on the first every cycle. With three or more slides it advanced by
two and merely skipped one, which is why it went unnoticed. The per-slide
duration the feed has always sent was also ignored in favour of a hardcoded
ten seconds.
- Clicking slightly outside a modal discarded a part-filled form, in 35 modals
across 30 files. Confirmation dialogs still dismiss that way, since they hold
nothing to lose.
- Selecting a filter while past page one returned an empty list, because the
filter asked for page five of a result set that now had one page.
- Model photos could not be saved: an uploaded photo sets the field to an
application path, and the input was type="url", which demands an absolute
address. The upload button was also hidden when creating a model.
- An empty bordered box appeared after Notes on the network device form.
### Added
- The equipment catalog travels. `flask seed catalog` loads 53 vendors, 128
models with photos, 146 printer supply part numbers and every type vocabulary,
so a new site can start adding printers immediately instead of retyping a
catalog another site spent a year building. Idempotent and additive; catalog
only, with nothing site-specific. Offered by the installer as a tick-box and
by the operator console.
- `shopdb-admin.ps1 repair` completes an interrupted provisioning, and `check`
now says when a server is not fully provisioned instead of leaving it to be
inferred from 500s on unrelated pages.
- Network devices can be linked to a catalog model, and their map position is
picked on the floor plan rather than typed as coordinates.
- An asset's vendor and type can be derived from its catalog model, by exact
name match only, with a reviewable backfill script.
### Changed
- Labels say whose type they mean: Machine Type, PC Type, Printer Type, Device
Type, next to the catalog's Model type.
- The 3D parts kiosk label prefix is a setting rather than one site's initials
hardcoded in the source.
- The operator console menu is grouped by what each action touches.
## [0.8.0] - 2026-08-05
First release to carry the Windows installer. Everything below shipped after
v0.7.0 was tagged, and a complete install was exercised end to end on Windows
Server 2019 before this release was cut.
### Fixed - installer, from a real Server 2019 install
- `packaging` was imported by the plugin loader but declared nowhere. It reached
development and CI only as a dependency of pytest, so the whole suite passed
while a virtual environment built from `requirements.txt` alone - which is
exactly what the installer builds - could not import the application at all.
`tests/test_runtime_dependencies.py` now fails on any runtime import that is
not a declared dependency.
- IIS returned 500.52 before the application was ever launched. `web.config`
declared `<allowedServerVariables>` for the X-Forwarded-For rule, and that
section ships `overrideModeDefault="Deny"`, so the whole file was rejected.
The installer now permits the single variable at server level instead of
unlocking the section for every site on the machine, and repairs a
`web.config` an earlier build had made unusable.
- The config unlock ran before the application it unlocks existed, so the scoped
form could never succeed on a first install and every install silently fell
back to granting handler delegation server-wide.
- The stage 5 smoke test discarded the status code and error page it had already
received, reporting "site did not return 200" for a fault IIS had named. It
now records both, plus the tail of the application log.
- Database dumps were readable by every authenticated user: a directory created
under ProgramData inherits `Users:RX`, and the owner-only ACL was applied only
when the installer itself created it.
- The uninstaller ran the 32-bit PowerShell, which cannot see IIS, so the site,
pool and application survived a "successful" uninstall pointing at a deleted
directory.
- Uninstall matched applications by alias alone and would remove an unrelated
application of the same name under another site.
- Wizard input reached a command line unchecked: unvalidated ports, a drive-root
path that escaped its own quote, and a password written as ANSI but read back
as UTF-8, which reported a correct non-ASCII password as wrong.
- Plugin deregistration could never succeed - it omitted `--yes` against a
command that prompts - while the plugin directory was deleted regardless.
- Preflight rows were drawn past the bottom of the panel and silently vanished;
the failures loop had no cap at all.
### Changed - installer behaviour
- "Is this a re-run of my install?" is answered from a durable install record
rather than inferred from the state of the machine. Nothing on a server says
who created its database tables, so a retry after a failed first install was
taken for an upgrade of a working system: it demanded a mandatory backup of a
database its own failed attempt had written, then refused to prune tables it
had created minutes earlier. During unfinished first provisioning the backup
is advisory and prune may force; on an established install both are unchanged.
- The installer offers every bundled plugin, so one build serves any site
instead of one build per plugin profile.
### Added - operator documentation and diagnostics
- `docs/UPDATES-WINDOWS.md` - what operators should expect from future updates,
bug fixes and security releases, including downtime, what is preserved, and
the effect on other sites sharing the same IIS server.
- `docs/RELEASING-WINDOWS.md` - how to build and release, and the two known gaps.
- `deploy/windows/shopdb-diagnose.py` - collects what IIS answers, the config
lock state, the application logs and the ACLs in one pass, scrubbing secrets
before writing anything.
### Added - the installer itself
- Air-gapped Windows installer (`deploy/windows/installer/`). One `.exe` per
site, built from that site's plugin profile, containing Python, the wheels,
the SPA, the IIS modules and optionally MySQL. Operator docs:
`docs/INSTALL-WINDOWS.md` and `docs/OPERATE-WINDOWS.md`, both shipped onto the
server. `docs/INSTALL-WINDOWS-IIS.md` and `docs/DEPLOY-WINDOWS-IIS.md` are now
reference-only, for hand-built servers.
- `bundle-lock.json`: an exact sha256 + size record of the installer's
third-party payload (wheels, Python installer, IIS MSIs). Verified as set
equality by both builders and again on the server before anything runs; there
is no install-time override. Regenerate with `refresh-bundle-lock.ps1`.
- CycloneDX SBOM (`sbom.cdx.json`) generated on every build from
`requirements.txt` and `package-lock.json`, covering both ecosystems, staged
into the application tree so an air-gapped server can answer "do we carry this
component" locally: `shopdb-admin.ps1 verify -Path <name>`.
- `shopdb-admin.ps1`, the operator console: status, start/stop/restart, logs,
health check, backup, plugins, verify. `check -Json` emits secret-free
structured state for pasting into a support ticket or an AI assistant.
- `build-installer.ps1`, the whole build natively on Windows, so a work PC needs
no Bash. Shares `scripts/resolve_plugin_closure.py` with `build-site.sh`.
- URL Rewrite is bundled and the wizard asks where client IPs come from
(`-ClientIpSource direct|proxy`). Without the rule IIS sends no
`X-Forwarded-For` at all and every client reads as 127.0.0.1.
### Changed
- `requirements.txt` and `requirements-dev.txt` are compiled `--universal
--generate-hashes`. Installs run under `pip --require-hashes`, so a wheel whose
sha256 is not listed is refused. The dev lockfile is constrained to the
production pins; the two had drifted.
- Python 3.14 across the Dockerfile, CI, `web.config` and the docs, which
previously declared four different versions.
- The naming/style gate covers Markdown, JSON and YAML, not just code.
### Fixed
- `plugins/employees` imported `shopdb.core.models` directly, failing the
contract-surface test on `main` since the dashboard employee-name resolver
landed.
- `scripts/build-site.sh` copied all of `deploy/` into its own output directory,
which recursed when the output was staged inside it - the documented Windows
build could not complete.
- Plugin baseline migrations inherited the MySQL server's default charset: the
utf8mb4 compiler hook lived in `migrations/env.py` and so covered the core
chain only. It is now `shopdb/utils/mysql_charset.py`, imported by both.
`flask db-utils preflight` reports the database's default charset.
### Added
- Configurable site timezone: a `site_timezone` site setting (default
`America/New_York`, public-readable), editable in Settings > Site >
Localization. Notification start/end times are entered and displayed in this
zone, and daily-reset notification expiry is computed in it. A shared
`frontend/src/utils/datetime.js` (Intl-based, DST-safe) does the conversion.
### Changed
- Asset detail pages (machines, PCs, printers, network devices, measuring
@@ -24,6 +556,15 @@ ADR-007 and ADR-002.
### Fixed
- Notification start/end times were off by the timezone offset (a 2:34 PM entry
displayed as 6:34 PM). Times are now stored UTC and shown/entered in the site
timezone; the calendar keys all-day events off the site-local day.
- Kiosk displays showed a white screen on login: a legacy 32-bit kiosk
installer's autostart kept relaunching Edge at a now-dead URL. The install's
HKLM Run value was WOW64-redirected into `SOFTWARE\Wow6432Node\...\Run` and
survived earlier cleanup. The `gea-shopfloor-display` dispatcher now purges
the legacy autostarts every enforce cycle across both registry views, all
user hives, Run/RunOnce/policy-Run, and every Startup folder.
- List pages keep the current page (and search term) in the URL query, so
paging to page 9, opening an item, and hitting browser Back returns to page 9
instead of resetting to page 1. Applies to all 18 list views via a shared
@@ -36,7 +577,7 @@ ADR-007 and ADR-002.
- Shopfloor kiosk header text is readable (light on the dark navy header).
### Added
### Added (continued)
- Collector-driven PC -> printer relationships. The computers collector schema
gained optional `defaultprinter` (string) and `printers` (array of strings)
@@ -474,7 +1015,11 @@ letting other GE Aerospace sites stand up their own self-hosted instance
integration that passed the key as a query parameter. See
`docs/COLLECTOR-INTEGRATION.md`.
[Unreleased]: https://gitea.proudtech.net/ge-aerospace/shopdb-flask/compare/v0.7.0...HEAD
[Unreleased]: https://gitea.proudtech.net/ge-aerospace/shopdb-flask/compare/v0.10.0...HEAD
[0.10.0]: https://gitea.proudtech.net/ge-aerospace/shopdb-flask/compare/v0.9.0...v0.10.0
[0.9.0]: https://gitea.proudtech.net/ge-aerospace/shopdb-flask/compare/v0.8.1...v0.9.0
[0.8.1]: https://gitea.proudtech.net/ge-aerospace/shopdb-flask/compare/v0.8.0...v0.8.1
[0.8.0]: https://gitea.proudtech.net/ge-aerospace/shopdb-flask/compare/v0.7.0...v0.8.0
[0.7.0]: https://gitea.proudtech.net/ge-aerospace/shopdb-flask/compare/v0.6.0...v0.7.0
[0.6.0]: https://gitea.proudtech.net/ge-aerospace/shopdb-flask/compare/v0.5.0...v0.6.0
[0.5.0]: https://gitea.proudtech.net/ge-aerospace/shopdb-flask/releases/tag/v0.5.0

227
CLAUDE.md
View File

@@ -1,138 +1,131 @@
# ShopDB Flask Project
Modern rewrite of the classic-ASP shopdb. Built as a framework so sister GE Aerospace sites can adopt it. Plugin system is the product.
Modern rewrite of the classic-ASP shopdb. Built as a framework so sister GE
Aerospace sites can adopt it. The plugin system is the product.
**This site is not the only site.** Anything that names West Jefferson - a
hostname, a share path, a subnet, a machine-number format, a label prefix - is a
defect unless it is a setting with a neutral default, a site-namespaced
directory, or seed data. That is ADR-015, and it is the rule most often broken
in the places tooling cannot see: PowerShell, the installer, JSON, the imaging
share.
## Facts that move live in the map
`docs/PROJECT-MAP.md` is GENERATED. It carries the product and contract
versions, the plugin inventory, every Alembic chain head, the ADR index with
status, and the size of the codebase. Read it instead of asking; regenerate it
rather than correcting it by hand:
```bash
venv/bin/python scripts/gen_project_map.py # write
venv/bin/python scripts/gen_project_map.py --check # CI form
```
Hand-written version numbers in this file were wrong by two contract releases
and 400 tests before the map existed. Nothing that the generator can derive
should be typed here again.
## Where to look
| question | read |
|---|---|
| what a name may be called | `CONTRIBUTING.md` |
| why the architecture is like that | `docs/adr/` (index in the map) |
| what a plugin may touch | `docs/PLUGIN-HOOKS.md` |
| what the collector sends | `docs/COLLECTOR-INTEGRATION.md`, ADR-006 |
| how a site is deployed | `docs/INSTALL-WINDOWS.md`, `docs/OPERATE-WINDOWS.md` |
| how code reaches GitHub | `tools/export-github.sh` |
## Standards are gates, not prose
Compliance is checked, not remembered. Run these rather than re-reading the
rules:
```bash
bash scripts/check-naming-and-style.sh # naming, style, site literals (ADR-015)
venv/bin/python -m pytest -q # includes the docs publishability gate
venv/bin/python -m pytest tests/test_plugin_migrations.py # chain heads registry
venv/bin/python scripts/gen_project_map.py --check
```
A new plugin table means updating `PLUGIN_TABLE_OWNERS` and
`EXPECTED_HEAD_REVISION` in `tests/test_plugin_migrations.py`.
## Database
- **Active database:** `shopdb_flask` (MySQL, asset-based schema)
- **Legacy database:** `shopdb` (Classic ASP schema, used only for one-time data import via `scripts/import_from_mysql.py`)
- **Connection:** `.env` file. See `.env.example`.
- **Active:** `shopdb_flask` (MySQL, asset-based schema)
- **Legacy:** `shopdb` (classic ASP schema; one-time import only)
- **Connection:** `.env`, see `.env.example`
Architecture decisions live in `docs/adr/`. Read those before making schema or contract changes.
Core migrations run with `flask db upgrade`; every plugin owns its own chain
(ADR-008) and runs with `flask plugin upgrade-all`. **A deploy needs both** -
skipping the second is how a new column reaches production as a 1054 error.
- ADR-001: Asset model is the platform contract (Machine retires) - ACCEPTED
- ADR-002: Plugin contract versioning (semver) - ACCEPTED
- ADR-003: Plugin distribution model (in-tree bundled + filesystem-based external) - ACCEPTED
- ADR-004: Deployment topology (per-site instances, not multi-tenant) - ACCEPTED
- ADR-005: Equipment vs measuringtools plugin scope - ACCEPTED
- ADR-006: Plugin collector contract pattern - ACCEPTED
- ADR-007: Product versioning and releases - ACCEPTED
- ADR-008: Plugin migration ownership (per-plugin chains) - ACCEPTED
- ADR-009: Frontend plugin route gating - ACCEPTED
- ADR-010: Frontend plugin hook contract - ACCEPTED
- ADR-011: Machines rename + modeltypes retyping - ACCEPTED
- ADR-012: GE-Enforce manifest ownership in shopdb - ACCEPTED
Tests run on SQLite, production is MySQL. Dialect drift is not caught by the
suite: `ORDER BY ... NULLS LAST` parses on SQLite and is rejected by MySQL. Sort
in Python when in doubt.
## Coding convention
`CONTRIBUTING.md` defines naming rules (DB tables, columns, Python, JS, Vue, API). Pre-commit hook at `scripts/check-naming-and-style.sh` enforces them. Read `CONTRIBUTING.md` before naming any new identifier.
## Current state (as of 2026-07-13)
Refactor phases 0-5 landed; phase 6 (multi-site distribution readiness) largely complete; the last big milestone is the legacy-data import + a production pilot.
### Phases done
- **Phase 0**: 6 ADRs accepted, naming convention v1, pre-commit style hook
- **Phase 1**: 8 smoke tests + 7 production-config tests, Flask-SQLAlchemy 3 fixtures, uv lockfile, hardened ProductionConfig
- **Phase 2**: contract surface defined (`__contract_version__`, `BasePlugin` hooks, `docs/PLUGIN-HOOKS.md`), 51 contract tests
- **Phase 3**: manifest-first loader, fail-loud/isolate policy, contract-version range checking, auto-register core blueprints, `shopdb.api` namespace, `BasePlugin.get_setting/set_setting` helpers
- **Phase 4**: `flask plugin new <name>` CLI, scaffold templates, 14 canary tests, `docs/PLUGIN-QUICKSTART.md`
- **Phase 5**: ADRs moved to `docs/adr/`, Alembic baseline migration, per-site deploy artifacts (`Dockerfile`, `docker-compose.yml`, `docs/DEPLOY.md`)
### Active state
- 966 tests, naming/style check green, Gitea Actions CI (backend + naming + frontend build + a migrations-mysql job that runs the real fresh upgrade on utf8mb4 MySQL 8)
- `__contract_version__` at 0.11.0 (product `__version__` 0.7.0, tags v0.5.0/v0.6.0/v0.7.0 - distinct series, ADR-007)
- 12 bundled plugins all satisfy contract: computers, employees, geenforce, knowledgebase, machines, measuringtools, network, notifications, printers, slides, usb, warranty
- Core Alembic chain: baseline `68b3947ae14f` -> head `7d25_drop_redundant_indexes` (32 core migrations). Each plugin owns its own chain (ADR-008); deploy runs `flask db upgrade` then `flask plugin upgrade-all`. Reproducible + idempotent from empty (env.py relaxes session sql_mode so the chain runs on strict MySQL 8).
- Legacy import: `docs/IMPORT-API.md` is the schema-agnostic import contract; `docs/IMPORT-ADOPTION.md` + `docs/PILOT-DEPLOY.md` cover adopting a site; `scripts/site_imports/wjf/` is the West Jefferson reference loader (all 15 stages, validated end-to-end including on a Windows + MySQL 8 VM).
- API is migration-complete: an admin PAT + docs/IMPORT-API.md let a script import the whole legacy DB (X-Import-Mode preserves timestamps).
- Pre-1.0 framework; sister sites should pin tight `core_version` ranges until contract reaches 1.0
### Deferred
- Equipment data migration (one-shot script for legacy ASP shopdb -> assets). Per ADR-001, only `category='Equipment' AND machinenumber IS NOT NULL` migrates. Skill `migrating-asset-schema` documents the pattern; the actual one-shot script lives in `scripts/migration/` when run.
- Printers retirement: legacy `PrinterData` model + frontend changes. Coordinated with the equipment data migration.
- (DONE 2026-07-11) `measuringtools` plugin (ADR-005) is built and bundled; docs/PLUGIN-GUIDE.md narrates its construction as the plugin tutorial.
- (DONE) Frontend plugin hook contract (ADR-010): get_settings_cards / get_asset_panels / get_map_overlays / get_asset_presentation shipped; generic renderers for panels/overlays land incrementally.
- (DONE) Per-plugin Alembic chains (ADR-008): every bundled plugin carries its own chain; no plugin uses db.create_all().
- Legacy ASP data import against the renamed schema (unblocked; run via docs/IMPORT-API.md) + a production pilot deployment.
## Quick start
```bash
# Start dev environment
~/start-dev-env.sh
# Activate venv and install deps
cd /home/camp/projects/shopdb-flask
source venv/bin/activate
pip install -r requirements.txt
# Configure environment
cp .env.example .env
# Edit .env with DB credentials, JWT secrets
# Create / update database tables via the Alembic chain
flask db upgrade
# Seed RBAC, default settings, and reference data (all idempotent)
flask seed permissions
flask seed settings
flask seed reference-data
# Restart services
pm2 restart shopdb-flask-api shopdb-flask-ui
```
## Service URLs
- Flask API: http://localhost:5001
- Flask UI: http://localhost:5173
- Legacy ASP (data source for one-time import): http://192.168.122.151:8080
`CONTRIBUTING.md` is authoritative: lowercase concatenated DB columns, snake_case
pure-code identifiers, a closed acronym list, banned shorthand. The pre-commit
hook `scripts/check-naming-and-style.sh` enforces what it can.
## Plugin structure
```
plugins/
<plugin_name>/
__init__.py
plugin.py # BasePlugin implementation
manifest.json # Plugin metadata (name, version, dependencies, api_prefix)
models/
__init__.py # Export all models
<model>.py # SQLAlchemy models
api/
__init__.py
routes.py # Flask Blueprint
services/ # Optional, business logic
schemas/ # Optional, marshmallow schemas
migrations/ # Optional, plugin-specific Alembic migrations
plugins/<name>/
manifest.json # metadata, single source of truth (ADR-002)
plugin.py # BasePlugin implementation
models/ # __init__.py exports every model
api/routes.py # Flask blueprint
migrations/ # this plugin's own Alembic chain
services/ schemas/ frontend/ client/ # as needed
```
Each plugin must have:
- `models/__init__.py` exports all models
- `plugin.py` extends `BasePlugin`
- `manifest.json` with metadata (single source of truth per ADR-002)
- No direct imports from core code (use the contract surface defined in ADR-001)
No plugin imports core directly - everything goes through the `shopdb.api`
contract surface. A manifest-less directory under `plugins/` is core frontend
surface and always ships.
## Key files
- `shopdb/__init__.py` - app factory, blueprint registration
- `shopdb/plugins/base.py` - BasePlugin ABC, PluginMeta dataclass
- `shopdb/plugins/loader.py` - filesystem discovery, dependency-aware loading
- `shopdb/core/api/assets.py` - example of optional plugin imports
- `frontend/src/router/index.js` - frontend routing
- `frontend/src/components/AppSidebar.vue` - navigation menu
- `docs/adr/` - architecture decision records
- `shopdb/__init__.py` - app factory, version constants
- `shopdb/plugins/base.py` - BasePlugin, PluginMeta
- `shopdb/plugins/loader.py` - discovery and dependency-aware loading
- `shopdb/plugins/alembic_template.py` - the env every plugin chain runs through
- `frontend/src/router/index.js`, `frontend/src/components/AppSidebar.vue`
- `tools/export-github.sh` - scrub and publication gate
## Migration notes
## State
- `migrations/DATA_MIGRATION_GUIDE.md` - one-time import from legacy ASP shopdb
- `migrations/MIGRATE_USB_DEVICES_FROM_EQUIPMENT.md` - USB device migration from equipment table
- `migrations/FIX_LOCATIONONLY_EQUIPMENT_TYPES.md` - LocationOnly equipment type fix
- `migrations/PRODUCTION_MIGRATION_GUIDE.md` - production import methods
- `migrations/rename_underscore_columns.sql` - one-time rename of snake_case columns to lowercase concatenated (per CONTRIBUTING.md)
- `migrations/versions/` - the core Alembic chain (baseline `68b3947ae14f` -> head `7d25_drop_redundant_indexes`). Run `flask db upgrade` to apply.
Refactor phases 0-5 landed; phase 6 (multi-site distribution readiness) largely
complete. The last milestone before 1.0 is unchanged and still open: the
legacy-data import plus a production pilot (`scripts/site_imports/wjf/` is the
reference loader, validated end to end).
Pre-1.0 framework. Sister sites pin tight `core_version` ranges until the
contract reaches 1.0.
### Deferred
- Equipment data migration (legacy ASP -> assets). Per ADR-001 only
`category='Equipment' AND machinenumber IS NOT NULL` migrates.
- Printers retirement: legacy `PrinterData` model, coordinated with the above.
## Quick start
```bash
source venv/bin/activate
pip install -r requirements.txt
cp .env.example .env # DB credentials, JWT secrets
flask db upgrade # core chain
flask plugin upgrade-all # every plugin chain
flask seed permissions && flask seed settings && flask seed reference-data
pm2 restart shopdb-flask-api shopdb-flask-ui
```
Dev API on :5001, dev UI on :5173. `venv/bin/python tools/shot.py /route` logs
in and screenshots a page.

12
Dockerfile
View File

@@ -2,10 +2,12 @@
#
# One image, one site. Per ADR-004, each adopting facility runs its own
# stack with its own DB, secrets, and enabled-plugin list. This image
# bundles all eleven core plugins (computers, employees, knowledgebase,
# machines, measuringtools, network, notifications, printers, slides, usb,
# warranty);
# install them at runtime with `flask plugin install <name>`.
# bundles all 13 catalog plugins (computers, employees, geenforce,
# knowledgebase, machines, measuringtools, network, notifications,
# printedparts, printers, slides, usb, warranty); a site installs + enables
# the ones it wants with `flask plugin install <name>` (or, declaratively,
# `flask plugin apply-profile <profile.json>`). Per ADR-013 a future lean
# build stages only the chosen plugin directories into this image.
#
# The frontend is built in a first stage and its dist output is copied into
# the final image so Flask can serve the SPA (register_frontend_routes in
@@ -30,7 +32,7 @@ RUN npm run build
# Output lands in /build/dist (Vite default), copied into the final stage below.
# ---- Stage 2: Python application image ----
FROM python:3.12-slim AS base
FROM python:3.14-slim AS base
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \

87
README.md
View File

@@ -20,7 +20,7 @@ ShopDB tracks and manages:
## Tech Stack
**Backend:**
- Python 3.12 with Flask
- Python 3.14 with Flask
- SQLAlchemy ORM
- MySQL 5.7+ database (5.6 works with extra utf8mb4 config; see docs/DEPLOY.md)
- JWT authentication
@@ -36,26 +36,26 @@ ShopDB tracks and manages:
```
shopdb-flask/
├── shopdb/ # Flask application
│ ├── core/
│ │ ├── api/ # REST API endpoints
│ │ ├── models/ # SQLAlchemy models
│ │ ├── schemas/ # Validation schemas
│ │ └── services/ # Business logic
│ ├── plugins/ # Plugin system
│ └── utils/ # Shared utilities
├── frontend/ # Vue 3 application
│ ├── src/
│ │ ├── api/ # API client
│ │ ├── components/ # Reusable components
│ │ ├── views/ # Page components
│ │ ├── router/ # Route definitions
│ │ └── stores/ # Pinia stores
│ └── public/ # Static assets
├── plugins/ # Bundled and external plugins
├── migrations/ # Alembic migration chain (flask db upgrade)
├── scripts/ # Import and utility scripts
└── tests/ # Test suite
+-- shopdb/ # Flask application
| +-- core/
| | +-- api/ # REST API endpoints
| | +-- models/ # SQLAlchemy models
| | +-- schemas/ # Validation schemas
| | `-- services/ # Business logic
| +-- plugins/ # Plugin system
| `-- utils/ # Shared utilities
+-- frontend/ # Vue 3 application
| +-- src/
| | +-- api/ # API client
| | +-- components/ # Reusable components
| | +-- views/ # Page components
| | +-- router/ # Route definitions
| | `-- stores/ # Pinia stores
| `-- public/ # Static assets
+-- plugins/ # Bundled and external plugins
+-- migrations/ # Alembic migration chain (flask db upgrade)
+-- scripts/ # Import and utility scripts
`-- tests/ # Test suite
```
## Naming Conventions
@@ -98,7 +98,7 @@ To maintain consistency with the legacy ShopDB database and codebase, the follow
### Prerequisites
- Python 3.12
- Python 3.14
- Node.js 18+
- MySQL 5.7+ (5.6 works with extra utf8mb4 config; see docs/DEPLOY.md)
@@ -108,8 +108,10 @@ SQLite). Do not run dev or production against SQLite.
### Distribution
The application is distributed through the internal GE Aerospace git server. Clone
it from there; there is no public package or image registry.
Source lives here. The Windows installer ships as a release asset rather than a
file in the repository - it is around 240 MB, well past what a repository takes
and well inside what a release asset does. There is no package or image
registry; a site installs from that `.exe` or builds from source.
### Fast path (Docker)
@@ -148,11 +150,13 @@ pip install -r requirements.txt
cp .env.example .env
# Edit .env with your database credentials and secrets.
export FLASK_APP=shopdb
flask db upgrade
flask plugin upgrade-all # per-plugin schema (ADR-008)
flask seed permissions
flask seed settings
flask seed reference-data
flask run
flask run --port 5001 # MUST be 5001 - the frontend dev server proxies here
# Frontend (separate terminal)
cd frontend
@@ -162,15 +166,26 @@ npm run build # production build into frontend/dist (served by Flask)
```
Complete first-run setup at `/setup`, or run `flask seed admin` for a headless
admin account.
admin account. The repo ships VS Code config in `.vscode/` (F5 debugs the
backend; a "Dev site" task runs both servers). A fuller day-one walkthrough,
including VS Code and troubleshooting, is the DEVELOPMENT-SETUP page in the
project wiki.
To import a site's legacy data, use the HTTP import surface: an admin API
token plus [docs/IMPORT-API.md](docs/IMPORT-API.md) drive the whole migration
through documented endpoints (`X-Import-Mode` preserves original timestamps).
`scripts/site_imports/wjf/` is the West Jefferson reference loader.
`scripts/site_imports/wjf/` is the reference site's loader, kept as a worked
example of the whole sequence.
For the full per-site deployment runbook see [docs/DEPLOY.md](docs/DEPLOY.md);
for every environment variable and Setting key see [docs/CONFIG.md](docs/CONFIG.md).
### Which deployment route
| Target | Use |
|---|---|
| **Windows Server + IIS** (how sister sites run) | **[docs/INSTALL-WINDOWS.md](docs/INSTALL-WINDOWS.md)** - one installer `.exe`, offline, no manual IIS work. Day 2: [docs/OPERATE-WINDOWS.md](docs/OPERATE-WINDOWS.md) |
| Linux / Docker, air-gapped | [docs/DEPLOY-AIRGAP.md](docs/DEPLOY-AIRGAP.md) |
| Linux / Docker, connected | [docs/DEPLOY.md](docs/DEPLOY.md) |
For every environment variable and Setting key see [docs/CONFIG.md](docs/CONFIG.md).
## Configuration
@@ -208,12 +223,22 @@ Query parameters for list endpoints:
- `search` - Search term
- `assettype` - Filter by asset type (computer, printer, machine, networkdevice, measuringtool)
## Where to start
[docs/START-HERE.md](docs/START-HERE.md) routes by what you are here to do -
standing up a site, deploying the shop-floor tools, writing a plugin,
integrating with the API, or diagnosing something. On the published mirror the
same page is the wiki's front door.
## Plugin System
ShopDB supports plugins for extending functionality. See `CONTRIBUTING.md` for plugin development guidelines.
The image bundles twelve plugins; only the ones a site installs are loaded:
The image bundles these plugins; only the ones a site installs are loaded. The
current count and each plugin's version and migration head are in
[docs/PROJECT-MAP.md](docs/PROJECT-MAP.md), which is generated:
- **backups** - Machine configuration backups (NTLARS/DNC, CMM, part marker, UDC) with revision history and diffs
- **computers** - Shopfloor PCs and workstations, collector fleet ingest
- **employees** - Employee directory
- **geenforce** - GE-Enforce imaging/software manifests and fleet compliance
@@ -222,8 +247,10 @@ The image bundles twelve plugins; only the ones a site installs are loaded:
- **knowledgebase** - Documentation and troubleshooting guides
- **network** - Network devices and subnets
- **notifications** - Shopfloor notifications and recognition feed
- **printedparts** - 3D-printed part catalogue, kiosk issue tracking and stock alerts
- **printers** - Extended printer management with Zabbix integration
- **slides** - TV/kiosk slideshows
- **tools** - Tech Tools: barcode and QR generation laid out for real label stock
- **usb** - CMMC USB check-in/out tracking
- **warranty** - Warranty records with Dell lookups

22
deploy/site-profile-universal.json Normal file
View File

@@ -0,0 +1,22 @@
{
"site": "universal",
"plugins": [
"backups",
"computers",
"employees",
"geenforce",
"knowledgebase",
"machines",
"measuringtools",
"network",
"notifications",
"printedparts",
"printers",
"slides",
"tools",
"usb",
"warranty"
],
"locked": [],
"_comment": "The profile the released Windows installer is built from. Every bundled plugin that carries a manifest, so ONE exe serves any site: the wizard offers all of them and the operator ticks what that site uses. Plugins left unticked are never installed, and 'flask plugin prune-schema' drops their tables at provisioning (ADR-014). 'applications' is deliberately absent - it is manifest-less core and always ships. Build with: deploy/windows/installer/build-installer.sh deploy/site-profile-universal.json <repo>. Use site-profile.example.json instead only when a site genuinely needs a lean build; see ADR-013."
}

11
deploy/site-profile.example.json Normal file
View File

@@ -0,0 +1,11 @@
{
"site": "example-site",
"plugins": [
"machines",
"computers",
"printers",
"network"
],
"locked": [],
"_comment": "Declarative plugin selection for a site (ADR-013). Apply with: flask plugin apply-profile deploy/site-profile.example.json. 'plugins' is the set this site wants; their hard dependencies are pulled in automatically and everything installs + enables in dependency order (idempotent). Naming a plugin not on disk fails loudly. 'locked' is reserved for a future guard against removing a site-mandated plugin. apply-profile never removes plugins absent from the list - removal stays an explicit flask plugin uninstall. Run flask plugin upgrade-all after applying, then restart."
}

6
deploy/windows/installer/.gitignore vendored Normal file
View File

@@ -0,0 +1,6 @@
# Build output and staged payload - regenerable, and ~220MB.
bundle/
Output/
# Generated by build-installer.sh from the staged bundle.
plugins.iss
version.iss

247
deploy/windows/installer/README.md Normal file
View File

@@ -0,0 +1,247 @@
# Windows installer
Builds a single self-contained `.exe` that installs ShopDB-Flask on an
**air-gapped** Windows Server. Nothing here ever touches the network at install
time: Python, the wheels, the SPA and (optionally) MySQL all ship inside it.
Lives with the application on purpose. The installer depends on app internals -
`flask plugin` verbs, `site-profile.json`, `MOUNT_PATH`, the plugin registry -
so a separate repo would drift out of step with the thing it installs.
## Files
| File | What it is |
|---|---|
| `shopdb-preflight.ps1` | Stage 1. Read-only. Changes nothing, reports what this server is missing. |
| `shopdb-install.ps1` | Stages 0 and 2-5 plus `uninstall`. All the actual work. |
| `shopdb-admin.ps1` | Operator console installed alongside the app: status, restart, logs, backup, plugins. |
| `ShopDBFlask.iss` | Inno Setup wizard. A thin wrapper - it collects input and runs the stages. |
| `build-installer.sh` | Stages the bundle from a site profile. |
| `make-branding.py` | Generates wizard artwork and the icon from `frontend/public/*.svg`. |
| `*.bmp`, `shopdb.ico` | Generated artwork, committed so a Windows build box needs no Python. |
| `build-installer.ps1` | The same build, natively on Windows. No Bash needed. |
| `bundle-lock.json` | The exact third-party payload this installer ships. Reviewed by commit. |
| `bundle-lock.ps1` | Creates and checks that lock. Also runs on the target server. |
| `refresh-bundle-lock.ps1` | Regenerates the lock, after showing what changed. |
| `verify_bundle_lock.py` | The same check for the Bash builder, so Linux needs no pwsh. |
## Building
Two builders, same result. Use whichever machine you are on - `build-installer.ps1`
does the whole job natively so a Windows work PC needs no Bash.
```bash
# Linux / WSL
./build-installer.sh ../../site-profile.example.json
```
```powershell
# Windows
.\build-installer.ps1 -Profile ..\..\site-profile.example.json
```
Both stage the app tree, build the SPA twice, write `plugins.iss`, copy the
installer scripts **from this directory**, and then verify the third-party
payload against `bundle-lock.json`. They exit non-zero if it does not match.
The payload itself is added by hand, and no longer needs Windows to produce:
```
bundle/wheels/ pip download -r requirements.txt --only-binary=:all: \
--platform win_amd64 --python-version 314 \
--implementation cp --abi cp314 -d wheels
bundle/python/ python-3.14.x-amd64.exe
bundle/httpplatformhandler/ httpPlatformHandler_amd64.msi
bundle/urlrewrite/ rewrite_amd64.msi (client-IP rule; see below)
bundle/vcredist/ VC_redist.x64.exe (MySQL will not install without it)
bundle/mysql/ mysql-8.4.x-winx64.msi (bundled-database option only)
bundle/mysqlclient/ mysql.exe, mysqldump.exe + libcrypto/libssl (backups)
```
Then compile on Windows:
```
iscc ShopDBFlask.iss
```
**Inno Setup 6.6.0 or newer.** The wizard uses the built-in `windows11` custom
style, which earlier versions reject. The script fails at compile time with that
sentence rather than with a bare "WizardStyle is invalid".
The wheelhouse is **cp314-locked**. A different Python minor version means a
different wheelhouse; the installer will not use a Python it did not install.
## What is pinned, and where
Three layers, because no one of them covers the whole problem.
| Layer | Covers | Enforced |
|---|---|---|
| `requirements.txt` sha256 per package | every wheel is genuinely what upstream published | `pip --require-hashes` at install; aborts on mismatch |
| `bundle-lock.json` | the EXACT payload: wheels, Python installer, MSIs | both builders, and again on the server before anything runs |
| git | the application tree | code review |
`--require-hashes` alone is not enough. pip lists every artifact of a pinned
version - `cffi 2.1.0` has 100 hashes - so it proves the wheel is genuine, not
that it is the wheel this bundle was built and tested with. It also ignores
extra files in the wheelhouse, and says nothing about the Python installer or
the MSIs, all of which run as SYSTEM on the target server.
So `bundle-lock.json` records an exact file set with a sha256 and a byte size
each, and verification is **set equality**: a missing file, an unexpected extra
file, or changed content all fail. There is no install-time override.
The lockfile is `--universal`, so one file serves Linux (dev, Docker, CI) and the
Windows wheelhouse. A Linux-only resolve had silently omitted `colorama`, a
win32-only dependency of `click` - which in hash-checking mode is a hard error
rather than a quiet omission.
### Verifying the installer itself
`bundle-lock.json` is **inside** the thing it describes, so it proves the payload
was not altered between build and install - not that the `.exe` you received is
the one that was built. That needs something out of band. Two options, in order
of preference:
1. **Authenticode-sign the `.exe`** with a GE code-signing certificate. Windows
then shows a real publisher instead of "Unknown", which is also what stops an
operator learning to click through the SmartScreen warning.
2. **Publish a sha256 per release** through a different channel than the file
itself, and have the receiving site check it:
`Get-FileHash ShopDBFlask_Installer_*.exe -Algorithm SHA256`
Neither is wired up yet. Until one is, an installer is only as trustworthy as
the share it arrived on.
### Changing what ships
```powershell
.\refresh-bundle-lock.ps1 # show what changed, write nothing
.\refresh-bundle-lock.ps1 -Yes # write it
```
Then **commit `bundle-lock.json`**. That commit is the review - it is the only
place a change to what runs as SYSTEM on a customer's server becomes visible to
a human. `refresh-bundle-lock.ps1` refuses to overwrite an existing lock until
you have seen the diff, for that reason.
To stage a bundle before its lock exists: `ALLOW_UNLOCKED=1` (Bash) or
`-AllowUnlocked` (PowerShell). Bundles built that way must not be shipped.
Verifying a live server, months later and offline:
```powershell
shopdb-admin.ps1 verify
```
## Servers this installer did not build
It is built for greenfield: its own Python, its own venv, its own IIS objects.
Its upgrade path assumes the thing being upgraded came out of a previous run.
Two guards keep it from damaging a server that was deployed by hand.
**Python minor version.** An existing venv is reused, which is right for a repair
or an upgrade. It is wrong when the venv belongs to a different Python - the
wheelhouse is tagged for one minor version, so pip would die at the first
compiled package, *after* Python was installed and the app tree replaced. The
installer compares the two up front and stops with both version numbers.
**IIS objects.** Switching deployment method removes the other method's artifact,
which is correct when the installer owns both and dangerous when it does not: a
wrong `-MountAlias` would delete a live mount with no prompt. It now refuses
unless there is a version stamp proving it made the install, or you pass
`-AdoptExisting`. The refusal lists exactly what it would have removed.
An existing `web.config` is never overwritten in either case.
For the West Jefferson production server specifically, this is a **migration, not
an upgrade** - prod runs Python 3.13 against a hand-built deployment, so it needs
a deliberate window, a database backup, and web.config reconciled by hand.
## Bill of materials
Every build stages a CycloneDX 1.6 SBOM at `sbom.cdx.json`, inside the
application tree, so it installs onto the server with the app. Both ecosystems,
in one document:
- **Python** - every pin in `requirements.txt`, with the sha256 the installer
enforces. Environment markers are ignored: a `sys_platform == 'win32'`
dependency still installs on the target.
- **npm** - every package in `frontend/package-lock.json`. Build-only packages
are marked `scope: excluded` rather than dropped, so "not here" is
distinguishable from "not looked for".
It ships to the server because an air-gapped site cannot be scanned from
anywhere else. When a CVE lands, the answer is already on the box:
```powershell
shopdb-admin.ps1 verify # counts, and which bundle this is
shopdb-admin.ps1 verify -Path leaflet # is that component here, at what version
```
Generated by `scripts/generate_sbom.py` from files that are already pinned and
committed, so it is a translation rather than a scan - no network, no extra
toolchain on the build box, and byte-identical output for the same inputs. It is
deliberately not in `bundle-lock.json`: its provenance is git, not the payload.
## Client IP addresses
IIS does not set `X-Forwarded-For` on its own, and HttpPlatformHandler connects
from loopback. Without a rule, **every client reads as 127.0.0.1** - so the
GE-Enforce IP allowlist, the dashboard's visitor-location lookup and per-host
login rate limiting all stop working, silently.
The wizard asks, because the two answers are mutually exclusive:
- **Clients connect directly** (`-ClientIpSource direct`, the default) - installs
URL Rewrite from the bundle and sets `X-Forwarded-For` from `REMOTE_ADDR`.
Overwriting the header is what stops a client spoofing its own.
- **A proxy sits in front** (`-ClientIpSource proxy`) - leaves the rule off.
Behind ARR or a load balancer `REMOTE_ADDR` is the *proxy*, so applying the
rule would discard the real client IP.
An existing `web.config` is never overwritten - it is the one file on a server
that legitimately carries hand-edits. The installer reports what it found
instead.
## Deployment methods
Chosen in the wizard, and the bundle carries a SPA build for each because Vite
compiles the base path in - it cannot be switched at install time.
- **Its own site** on a port (default 8090).
- **Subpath** under an existing site, e.g. `http://<server-fqdn>/shopdb/`. Needs
no new DNS record. Three things must agree - the IIS application alias,
`MOUNT_PATH` in `.env`, and the SPA's build-time base - so the alias is fixed
per bundle (`SUBPATH_ALIAS`, default `shopdb`) and the installer refuses if the
bundle's build does not match what was asked for.
Switching between methods removes the other one's IIS artifact and reconciles
`MOUNT_PATH` and `CORS_ORIGINS`, so a server never ends up with both.
## Upgrades
Run a newer installer over an existing install. It:
- backs the database up first, **verifies** the dump is complete, and refuses to
migrate if it cannot;
- restores from that backup if migrations fail, and reports honestly that DDL
the failed migration committed cannot be undone;
- refuses to run a bundle older than what is installed;
- keeps `.env` unless new credentials are supplied, and copies it aside first;
- stops the app pool before replacing files, then starts it again.
Whether a run is an upgrade is decided by probing the **target database**, not by
whether the app directory exists - a rebuilt server pointed at an existing
database is an upgrade, and treating it as fresh would drop tables.
## Testing notes
Verified end to end on Windows Server 2025 against both a bundled MySQL 8.4 LTS and
an existing MySQL 5.6: fresh install, upgrade, re-run idempotency, failure and
rollback, uninstall, and both deployment methods including switching between
them.
**Not yet verified:** a fully air-gapped run with the network disabled at the
hypervisor, and any load in a real browser (all HTTP checks so far used curl,
which sends no `Origin` header - so `CORS_ORIGINS` is untested in anger).

1340
deploy/windows/installer/ShopDBFlask.iss Normal file
View File

File diff suppressed because it is too large Load Diff

301
deploy/windows/installer/build-installer.ps1 Normal file
View File

@@ -0,0 +1,301 @@
<#
.SYNOPSIS
Stage a lean per-site installer bundle on Windows, and verify its third-party
payload against bundle-lock.json.
.DESCRIPTION
The Windows equivalent of build-installer.sh, for a work PC with no Bash. It
does the whole job natively - it does NOT shell out to build-site.sh - so the
only tools needed are Python, Node and (to compile) Inno Setup.
What it does, in order:
1. Resolves the site's plugin closure from its profile.
2. Builds the SPA twice, because Vite compiles the base path in and it
therefore cannot be chosen at install time: once for /<alias> (subpath
deployment) and once for / (its own site).
3. Stages the backend tree: core, the chosen plugins only, and the runtime
files a deployable tree needs.
4. Writes plugins.iss so the wizard's plugin page matches the payload.
5. Copies the installer scripts from THIS directory.
6. Verifies wheels\, python\ and the MSIs against bundle-lock.json, and
FAILS if the bundle is not exactly what the lock describes.
Step 6 is the point of the script. A bundle that does not match its lock is
not shipped, and "the wheelhouse was missing" is caught here rather than by
an operator halfway through installing on a server with no network.
.PARAMETER Profile
Path to the site profile (see deploy\site-profile.example.json).
.PARAMETER RepoRoot
The repository. Defaults to four levels up from this script, which is correct
for a normal checkout.
.PARAMETER SkipFrontend
Reuse the SPA builds already staged in the bundle. For iterating on the
installer itself, where two Vite builds per run is most of the wall clock.
Never use it for a bundle you intend to ship.
.PARAMETER AllowUnlocked
Stage the bundle and report payload problems WITHOUT failing. For assembling
a bundle before its lock exists. A bundle built this way must not be shipped;
run refresh-bundle-lock.ps1, commit the lock, then build again without this.
.EXAMPLE
.\build-installer.ps1 -Profile ..\..\site-profile.example.json
.\build-installer.ps1 -Profile C:\sites\wjf.json -SkipFrontend
#>
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)] [string] $Profile,
[string] $RepoRoot = (Resolve-Path (Join-Path $PSScriptRoot '..\..\..')).Path,
[string] $BundleRoot = (Join-Path $PSScriptRoot 'bundle'),
[string] $SubpathAlias = 'shopdb',
[switch] $SkipFrontend,
[switch] $AllowUnlocked
)
$ErrorActionPreference = 'Stop'
. (Join-Path $PSScriptRoot 'bundle-lock.ps1')
function Say { param($m, $c = 'Gray') Write-Host $m -ForegroundColor $c }
function Step { param($m) Write-Host ''; Write-Host "==> $m" -ForegroundColor Cyan }
function Die { param($m, $fix = '') Write-Host ''; Write-Host " $m" -ForegroundColor Red
if ($fix) { Write-Host " $fix" -ForegroundColor Yellow }; exit 1 }
function Invoke-Tool {
# Runs a build tool and stops on a non-zero exit. npm writes progress to
# stderr on SUCCESS, so stderr alone must never be treated as failure.
param([string] $Exe, [string[]] $Arguments, [string] $WorkDir, [string] $What)
Push-Location $WorkDir
try {
$prev = $ErrorActionPreference
$ErrorActionPreference = 'Continue'
& $Exe @Arguments 2>&1 | ForEach-Object { Say " $_" 'DarkGray' }
$code = $LASTEXITCODE
$ErrorActionPreference = $prev
if ($code -ne 0) { Die "$What failed (exit $code)" }
} finally { Pop-Location }
}
if (-not (Test-Path $Profile)) { Die "profile not found: $Profile" }
if (-not (Test-Path $RepoRoot)) { Die "repo not found: $RepoRoot" }
$Profile = (Resolve-Path $Profile).Path
$RepoRoot = (Resolve-Path $RepoRoot).Path
$AppOut = Join-Path $BundleRoot 'app'
$python = (Get-Command python -ErrorAction SilentlyContinue)
if (-not $python) { $python = Get-Command py -ErrorAction SilentlyContinue }
if (-not $python) { Die 'Python not found on PATH' 'Install Python 3.14 and re-open the shell.' }
$npm = Get-Command npm.cmd -ErrorAction SilentlyContinue
if (-not $npm -and -not $SkipFrontend) { Die 'npm not found on PATH' 'Install Node, or pass -SkipFrontend to reuse the staged SPA.' }
Say ''
Say " repo : $RepoRoot"
Say " profile : $Profile"
Say " bundle : $BundleRoot"
# --- 1. plugin closure ------------------------------------------------------
# Same resolver the Linux builder uses, so both produce the same set from one
# profile instead of two implementations of the closure rules.
Step 'Resolving plugin closure'
$closure = (& $python.Source (Join-Path $RepoRoot 'scripts\resolve_plugin_closure.py') $Profile $RepoRoot)
if ($LASTEXITCODE -ne 0 -or -not $closure) { Die 'could not resolve the plugin closure from the profile' }
$closure = $closure.Trim()
Say " $closure" 'White'
# --- 2. frontend ------------------------------------------------------------
$frontend = Join-Path $RepoRoot 'frontend'
$subStaged = Join-Path $BundleRoot 'spa-subpath'
$rootStaged= Join-Path $BundleRoot 'spa-root'
if ($SkipFrontend) {
if (-not (Test-Path $subStaged) -or -not (Test-Path $rootStaged)) {
Die '-SkipFrontend was passed but no SPA build is staged' 'Run once without it.'
}
Say ''
Say ' Reusing the staged SPA builds (-SkipFrontend). NOT shippable if the frontend changed.' 'Yellow'
} else {
Step "Building the SPA for /$SubpathAlias/"
$env:SITE_PLUGINS = $closure
$env:VITE_BASE_PATH = "/$SubpathAlias/"
try { Invoke-Tool $npm.Source @('run','build','--silent') $frontend 'subpath frontend build' }
finally { Remove-Item Env:\VITE_BASE_PATH -ErrorAction SilentlyContinue }
Remove-Item $subStaged -Recurse -Force -ErrorAction SilentlyContinue
Copy-Item (Join-Path $frontend 'dist') $subStaged -Recurse -Force
# The alias is fixed at BUILD time; the installer reads this and refuses to
# publish under a different one rather than serving a page that cannot load
# its own assets.
Set-Content -Path (Join-Path $subStaged '.alias') -Value $SubpathAlias -Encoding ASCII
# Root build LAST, so frontend\dist is left in the state a developer expects.
Step 'Building the SPA for /'
try { Invoke-Tool $npm.Source @('run','build','--silent') $frontend 'root frontend build' }
finally { Remove-Item Env:\SITE_PLUGINS -ErrorAction SilentlyContinue }
Remove-Item $rootStaged -Recurse -Force -ErrorAction SilentlyContinue
Copy-Item (Join-Path $frontend 'dist') $rootStaged -Recurse -Force
}
# --- 3. backend tree --------------------------------------------------------
Step "Staging the application tree"
Remove-Item $AppOut -Recurse -Force -ErrorAction SilentlyContinue
New-Item -ItemType Directory -Path (Join-Path $AppOut 'plugins') -Force | Out-Null
Copy-Item (Join-Path $RepoRoot 'shopdb') $AppOut -Recurse -Force
foreach ($name in $closure.Split(',')) {
$src = Join-Path $RepoRoot ('plugins\' + $name.Trim())
if (-not (Test-Path $src)) { Die "plugin in the closure is not on disk: $name" }
Copy-Item $src (Join-Path $AppOut 'plugins') -Recurse -Force
}
# Runtime files a deployable tree needs beyond the Python packages. Without
# these the tree imports but cannot be run or migrated.
foreach ($f in @('wsgi.py', 'requirements.txt')) {
Copy-Item (Join-Path $RepoRoot $f) $AppOut -Force
}
Copy-Item (Join-Path $RepoRoot 'migrations') $AppOut -Recurse -Force
# ONLY web.config, not all of deploy\. The bundle is staged INSIDE deploy\, so
# copying the whole tree would recurse into its own output, and the rest of
# deploy\ is installer source that has no business on an application server.
# shopdb-install.ps1 reads it from exactly this path.
$cfgSrc = Join-Path $RepoRoot 'deploy\windows\web.config'
if (Test-Path $cfgSrc) {
New-Item -ItemType Directory -Path (Join-Path $AppOut 'deploy\windows') -Force | Out-Null
Copy-Item $cfgSrc (Join-Path $AppOut 'deploy\windows') -Force
}
# A CycloneDX SBOM of everything this tree depends on, Python and npm together.
# Staged INTO the tree so it installs onto the server with the application: an
# air-gapped site cannot be scanned remotely, so the only way to answer "are we
# exposed to this CVE, and where" is for the answer to be sitting on the box.
Step 'Generating SBOM'
& $python.Source (Join-Path $RepoRoot 'scripts\generate_sbom.py') $RepoRoot `
-o (Join-Path $AppOut 'sbom.cdx.json') | ForEach-Object { Say " $_" 'White' }
if ($LASTEXITCODE -ne 0) { Die 'SBOM generation failed' }
# Docs the running site serves, plus the runbooks an air-gapped server has no
# other way to reach. Without openapi.json and llms.txt the self-hosted /api/docs
# page is broken on every installed server.
Step 'Staging docs'
$docsOut = Join-Path $AppOut 'docs'
New-Item -ItemType Directory -Path $docsOut -Force | Out-Null
foreach ($doc in @('openapi.json', 'llms.txt', 'api-inventory.json',
'INSTALL-WINDOWS.md', 'OPERATE-WINDOWS.md',
'BACKUP-RESTORE.md', 'UPGRADE.md')) {
$src = Join-Path $RepoRoot ('docs\' + $doc)
if (Test-Path $src) { Copy-Item $src $docsOut -Force; Say " $doc" }
}
# Stage the profile INTO the tree: `flask plugin apply-profile` at provisioning
# reads the same profile the tree was staged from, so the installed plugin set
# and the shipped plugin code cannot drift.
Copy-Item $Profile (Join-Path $AppOut 'site-profile.json') -Force
# The installer expects frontend\dist and frontend\dist-subpath.
$feOut = Join-Path $AppOut 'frontend'
New-Item -ItemType Directory -Path $feOut -Force | Out-Null
Copy-Item $rootStaged (Join-Path $feOut 'dist') -Recurse -Force
Copy-Item $subStaged (Join-Path $feOut 'dist-subpath') -Recurse -Force
Get-ChildItem $AppOut -Recurse -Directory -Filter '__pycache__' -ErrorAction SilentlyContinue |
Remove-Item -Recurse -Force -ErrorAction SilentlyContinue
Get-ChildItem $AppOut -Recurse -File -Filter '*.pyc' -ErrorAction SilentlyContinue |
Remove-Item -Force -ErrorAction SilentlyContinue
# --- 4. plugins.iss ---------------------------------------------------------
Step 'Writing plugins.iss'
$shipped = (Get-ChildItem (Join-Path $AppOut 'plugins') -Directory | Select-Object -ExpandProperty Name) -join ','
$aliasBuilt = ''
$aliasFile = Join-Path $feOut 'dist-subpath\.alias'
if (Test-Path $aliasFile) { $aliasBuilt = (Get-Content $aliasFile -TotalCount 1).Trim() }
@"
; GENERATED by build-installer.ps1 - do not edit.
; The plugins present in bundle\app\plugins at build time.
#define AvailablePlugins "$shipped"
; The alias the subpath SPA was built for, or empty if this bundle has no
; subpath build - in which case the wizard must not offer that option.
#define SubpathAlias "$aliasBuilt"
"@ | Set-Content -Path (Join-Path $PSScriptRoot 'plugins.iss') -Encoding ASCII
Say " $shipped" 'White'
# The product version, read from the code rather than restated in the .iss.
Step 'Writing version.iss'
$initText = Get-Content (Join-Path $RepoRoot 'shopdb\__init__.py') -Raw
if ($initText -notmatch "(?m)^__version__\s*=\s*'([^']+)'") { Die 'could not read __version__ from shopdb\__init__.py' }
$appVersion = $Matches[1]
@"
; GENERATED by build-installer.ps1 from shopdb/__init__.py - do not edit.
#define AppVersion "$appVersion"
"@ | Set-Content -Path (Join-Path $PSScriptRoot 'version.iss') -Encoding ASCII
Say " $appVersion" 'White'
# --- 5. installer scripts ---------------------------------------------------
# From THIS directory, which is the reviewed copy under version control. They
# used to be copied from a downloads folder, so the logic that shipped was not
# the logic that was committed and the build only worked on one machine.
Step 'Copying installer scripts'
foreach ($f in @('shopdb-install.ps1', 'shopdb-preflight.ps1', 'bundle-lock.ps1')) {
$src = Join-Path $PSScriptRoot $f
if (-not (Test-Path $src)) { Die "installer script missing from the repo: $f" }
Copy-Item $src $BundleRoot -Force
Say " $f"
}
# --- 6. payload verification ------------------------------------------------
Step 'Verifying the third-party payload against bundle-lock.json'
$lockPath = Join-Path $PSScriptRoot 'bundle-lock.json'
$lock = Read-BundleLock $lockPath
if (-not $lock) {
$msg = "no bundle-lock.json at $lockPath"
if ($AllowUnlocked) { Say " $msg - continuing because -AllowUnlocked was passed" 'Yellow' }
else {
Die $msg @'
Add the wheels and installers to the bundle, then:
.\refresh-bundle-lock.ps1 (review the list)
.\refresh-bundle-lock.ps1 -Yes (write it)
and commit bundle-lock.json. Pass -AllowUnlocked to stage a bundle without one -
it must not be shipped.
'@
}
} else {
$problems = Test-BundleLock -BundleRoot $BundleRoot -Lock $lock
if ($problems.Count -eq 0) {
Say (" payload matches the lock ({0}, {1})" -f `
(Get-JsonProperty $lock 'pythontag' 'unknown'), (Get-JsonProperty $lock 'platform' 'unknown')) 'Green'
# Ships WITH the bundle: the installer re-checks the payload on the
# target server before running any of it, so tampering between build and
# install is caught too.
Copy-Item $lockPath $BundleRoot -Force
} else {
Say ''
foreach ($p in $problems) { Say " $p" 'Red' }
Say ''
if ($AllowUnlocked) {
Say ' -AllowUnlocked: continuing anyway. DO NOT SHIP this bundle.' 'Yellow'
} else {
Die ("{0} payload problem(s) - the bundle is not what the lock describes" -f $problems.Count) @'
Either the payload is wrong (fix the bundle) or it changed on purpose (run
refresh-bundle-lock.ps1, read the diff, and commit the new lock).
'@
}
}
}
# --- summary ----------------------------------------------------------------
Write-Host ''
Say " Bundle staged at: $BundleRoot" 'White'
foreach ($d in @('app', 'wheels', 'python', 'httpplatformhandler', 'urlrewrite', 'vcredist', 'mysqlclient', 'mysql')) {
$p = Join-Path $BundleRoot $d
if (Test-Path $p) {
$mb = ((Get-ChildItem $p -Recurse -File | Measure-Object Length -Sum).Sum / 1MB)
Say (" {0,-22} {1,8:N1} MB" -f $d, $mb)
} else {
Say (" {0,-22} {1}" -f $d, 'absent') 'DarkGray'
}
}
Write-Host ''
Say ' Compile with: iscc ShopDBFlask.iss' 'White'
Say ' (Inno Setup 6.6.0 or newer - the wizard uses the windows11 custom style.)' 'DarkGray'
Write-Host ''

141
deploy/windows/installer/build-installer.sh Executable file
View File

@@ -0,0 +1,141 @@
#!/bin/bash
# Stage a lean per-site bundle next to ShopDBFlask.iss, ready for Inno Setup.
#
# The bundle is built FOR ONE SITE from its plugin profile (ADR-013): plugins the
# site did not choose are absent from the payload entirely. Build one installer
# per site, not one universal installer.
#
# Usage: build-installer.sh <site-profile.json> [repo-path]
#
# The wheelhouse cannot be built here. Wheels are cp314 win_amd64 and must be
# produced ON Windows with the matching Python:
# pip download -r requirements.txt -d wheels --only-binary=:all:
# Copy that wheels\ directory in before compiling.
set -euo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROFILE="${1:?usage: build-installer.sh <site-profile.json> [repo-path]}"
REPO="${2:-$HOME/projects/shopdb-flask}"
BUNDLE="$HERE/bundle"
[ -f "$PROFILE" ] || { echo "profile not found: $PROFILE"; exit 1; }
[ -d "$REPO" ] || { echo "repo not found: $REPO"; exit 1; }
echo "==> Staging lean app tree from $PROFILE"
rm -rf "$BUNDLE/app"
bash "$REPO/scripts/build-site.sh" "$PROFILE" "$BUNDLE/app"
# build-site.sh emits the SPA as frontend-dist; the installer's web.config and
# static route expect frontend\dist.
if [ -d "$BUNDLE/app/frontend-dist" ]; then
mkdir -p "$BUNDLE/app/frontend"
rm -rf "$BUNDLE/app/frontend/dist"
mv "$BUNDLE/app/frontend-dist" "$BUNDLE/app/frontend/dist"
fi
# The /shopdb-based build, used when the operator picks the subpath deployment.
if [ -d "$BUNDLE/app/frontend-dist-subpath" ]; then
rm -rf "$BUNDLE/app/frontend/dist-subpath"
mv "$BUNDLE/app/frontend-dist-subpath" "$BUNDLE/app/frontend/dist-subpath"
fi
# Tell the .iss which plugins this bundle actually carries, so the wizard's
# plugin page always matches the payload instead of a hand-maintained list.
echo "==> Writing plugins.iss"
PLUGINS=$(ls "$BUNDLE/app/plugins" 2>/dev/null | tr '\n' ',' | sed 's/,$//')
# The subpath SPA is built with its base path compiled in, so whether the wizard
# can OFFER a subpath install is a property of the bundle, not a runtime choice.
SUBPATH_ALIAS_BUILT=""
if [ -f "$BUNDLE/app/frontend/dist-subpath/.alias" ]; then
SUBPATH_ALIAS_BUILT="$(cat "$BUNDLE/app/frontend/dist-subpath/.alias")"
fi
cat > "$HERE/plugins.iss" <<EOF
; GENERATED by build-installer.sh - do not edit.
; The plugins present in bundle\\app\\plugins at build time.
#define AvailablePlugins "$PLUGINS"
; The alias the subpath SPA was built for, or empty if this bundle has no
; subpath build - in which case the wizard must not offer that option.
#define SubpathAlias "$SUBPATH_ALIAS_BUILT"
EOF
echo " $PLUGINS"
# The product version, read from the code rather than restated here. A hardcoded
# AppVersion in the .iss had drifted two minor versions from shopdb/__init__.py.
echo "==> Writing version.iss"
APPVERSION=$(sed -n "s/^__version__ = '\\(.*\\)'/\\1/p" "$REPO/shopdb/__init__.py" | head -1)
[ -n "$APPVERSION" ] || { echo "could not read __version__ from shopdb/__init__.py"; exit 1; }
cat > "$HERE/version.iss" <<EOF
; GENERATED by build-installer.sh from shopdb/__init__.py - do not edit.
#define AppVersion "$APPVERSION"
EOF
echo " $APPVERSION"
# From THIS directory, which is the reviewed copy under version control. These
# used to be copied from $HOME/Downloads, so the installer logic that shipped was
# not the logic that was committed, and the build only worked on one machine.
echo "==> Copying installer scripts"
mkdir -p "$BUNDLE"
for f in shopdb-install.ps1 shopdb-preflight.ps1 bundle-lock.ps1; do
[ -f "$HERE/$f" ] || { echo "installer script missing from the repo: $f"; exit 1; }
cp "$HERE/$f" "$BUNDLE/"
done
echo ""
echo "Bundle staged at: $BUNDLE"
for d in app wheels python httpplatformhandler urlrewrite vcredist mysqlclient mysql; do
if [ -d "$BUNDLE/$d" ]; then
printf ' %-20s %s\n' "$d" "$(du -sh "$BUNDLE/$d" | cut -f1)"
else
printf ' %-20s absent\n' "$d"
fi
done
echo ""
echo " plugins shipped: $(ls "$BUNDLE/app/plugins" 2>/dev/null | tr '\n' ' ')"
# --- payload verification ---------------------------------------------------
# The third-party payload is the part git does not record: the wheels, the Python
# installer and the MSIs that run as SYSTEM on the target server. It must be
# EXACTLY what bundle-lock.json describes - no missing file, no stale extra wheel
# left over from a previous build, no changed content - or this is not a bundle
# anyone reviewed. Previously a missing wheelhouse printed MISSING and the script
# still exited 0, so an empty bundle compiled into a shippable installer and the
# failure surfaced on an air-gapped server with no way to fix it.
#
# ALLOW_UNLOCKED=1 downgrades this to a warning, for assembling a bundle before
# its lock exists. A bundle built that way must not be shipped.
echo ""
echo "==> Verifying the third-party payload against bundle-lock.json"
if python3 "$HERE/verify_bundle_lock.py" "$BUNDLE" "$HERE/bundle-lock.json"; then
echo " payload matches the lock"
# Ships WITH the bundle: the installer re-checks the payload on the target
# server before running any of it, so tampering between build and install is
# caught too.
cp "$HERE/bundle-lock.json" "$BUNDLE/"
elif [ "${ALLOW_UNLOCKED:-0}" = "1" ]; then
echo ""
echo " ALLOW_UNLOCKED=1: continuing anyway. DO NOT SHIP this bundle."
else
echo ""
echo " The bundle is not what the lock describes."
echo ""
echo " Add the missing pieces by hand:"
echo " wheels/ pip download -r requirements.txt --only-binary=:all: \\"
echo " --platform win_amd64 --python-version 314 \\"
echo " --implementation cp --abi cp314 -d wheels"
echo " python/ python-3.14.x-amd64.exe"
echo " httpplatformhandler/ httpPlatformHandler_amd64.msi"
echo " urlrewrite/ rewrite_amd64.msi (client-IP rule; see README)"
echo " vcredist/ VC_redist.x64.exe (MySQL requires it)"
echo " mysql/ mysql-8.4.x-winx64.msi (bundled-database option only)"
echo ""
echo " If the payload changed ON PURPOSE, regenerate and COMMIT the lock:"
echo " pwsh ./refresh-bundle-lock.ps1 # review the diff"
echo " pwsh ./refresh-bundle-lock.ps1 -Yes # write it"
echo ""
echo " To stage a bundle before its lock exists: ALLOW_UNLOCKED=1 $0 ..."
exit 1
fi
echo ""
echo "Then compile on Windows: iscc ShopDBFlask.iss"
echo "(Inno Setup 6.6.0 or newer - the wizard uses the windows11 custom style.)"

240
deploy/windows/installer/bundle-lock.json Normal file
View File

@@ -0,0 +1,240 @@
{
"schema": 1,
"generated": "2026-08-04T23:11:22Z",
"pythontag": "cp314",
"platform": "win_amd64",
"note": "Exact third-party payload of the installer bundle. Regenerate with refresh-bundle-lock.ps1 and COMMIT the change as a reviewed dependency bump.",
"payloads": {
"httpplatformhandler": {
"files": {
"httpPlatformHandler_amd64.msi": {
"sha256": "90f8d4905a0ab4f2c95223b3c79e2807a0b74507747d240e43c4302e8db4b5ef",
"size": 557056
}
},
"required": true
},
"python": {
"files": {
"python-3.14.6-amd64.exe": {
"sha256": "14b3e9a710a3fcf0bd9b55ab6b60412bd91227563f813fc49040cabc0209e0bd",
"size": 30774112
}
},
"required": true
},
"vcredist": {
"files": {
"VC_redist.x64.exe": {
"sha256": "cc0ff0eb1dc3f5188ae6300faef32bf5beeba4bdd6e8e445a9184072096b713b",
"size": 25635768
}
},
"required": false
},
"mysql": {
"files": {
"mysql-8.4.6-winx64.msi": {
"sha256": "868885b2e221409f0170729bb30b2b2ce83440a1a4c735d19b3c225b51045762",
"size": 135081984
}
},
"required": false
},
"wheels": {
"files": {
"email_validator-2.3.0-py3-none-any.whl": {
"sha256": "80f13f623413e6b197ae73bb10bf4eb0908faf509ad8362c5edeb0be7fd450b4",
"size": 35604
},
"cachelib-0.13.0-py3-none-any.whl": {
"sha256": "8c8019e53b6302967d4e8329a504acf75e7bc46130291d30188a6e4e58162516",
"size": 20914
},
"flask_caching-2.4.0-py3-none-any.whl": {
"sha256": "d15b8135f055c4f28f6f7dbcf8d36a3de4af1224def975ae6e0b43cbfa684486",
"size": 28727
},
"typing_extensions-4.15.0-py3-none-any.whl": {
"sha256": "f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548",
"size": 44614
},
"greenlet-3.5.0-cp314-cp314-win_amd64.whl": {
"sha256": "3bc59be3945ae9750b9e7d45067d01ae3fe90ea5f9ade99239dabdd6e28a5033",
"size": 239835
},
"charset_normalizer-3.4.7-cp314-cp314-win_amd64.whl": {
"sha256": "92a0a01ead5e668468e952e4238cccd7c537364eb7d851ab144ab6627dbbe12f",
"size": 159634
},
"colorama-0.4.6-py2.py3-none-any.whl": {
"sha256": "4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6",
"size": 25335
},
"Flask_Migrate-4.1.0-py3-none-any.whl": {
"sha256": "24d8051af161782e0743af1b04a152d007bad9772b2bca67b7ec1e8ceeb3910d",
"size": 21237
},
"markupsafe-3.0.3-cp314-cp314-win_amd64.whl": {
"sha256": "bdc919ead48f234740ad807933cdf545180bfbe9342c2bb451556db2ed958581",
"size": 15341
},
"flask_marshmallow-1.5.0-py3-none-any.whl": {
"sha256": "99951c77e5654111ed733811c6dc9310bfb4c3688c78a9e76f80b5ae0b2279a6",
"size": 12161
},
"tzdata-2026.3-py2.py3-none-any.whl": {
"sha256": "dc096730c87af6cab1b171c9d532be840741ff5d459015e7f6947bd7d7e54931",
"size": 348168
},
"python_dotenv-1.2.2-py3-none-any.whl": {
"sha256": "1d8214789a24de455a8b8bd8ae6fe3c6b69a5e3d64aa8a8e5d68e694bbcb285a",
"size": 22101
},
"dnspython-2.8.0-py3-none-any.whl": {
"sha256": "01d9bbc4a2d76bf0db7c1f729812ded6d912bd318d3b1cf81d30c0f845dbf3af",
"size": 331094
},
"pymysql-1.1.3-py3-none-any.whl": {
"sha256": "8164ba62c552f6105f3b11753352d0f16b90d1703ba67d81923d5a8a5d1c5289",
"size": 45356
},
"packaging-26.3-py3-none-any.whl": {
"sha256": "d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c",
"size": 129956
},
"flask_jwt_extended-4.7.3-py2.py3-none-any.whl": {
"sha256": "905ac807b52b5409bc9244dbcca434968c13ca6f9d91bffe7d4cb71e0e6231cb",
"size": 22698
},
"urllib3-2.7.0-py3-none-any.whl": {
"sha256": "9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897",
"size": 131087
},
"marshmallow-4.3.0-py3-none-any.whl": {
"sha256": "46c4fe6984707e3cbd485dfebbf0a59874f58d695aad05c1668d15e8c6e13b46",
"size": 49148
},
"waitress-3.0.2-py3-none-any.whl": {
"sha256": "c56d67fd6e87c2ee598b76abdd4e96cfad1f24cacdea5078d382b1f9d7b5ed2e",
"size": 56232
},
"idna-3.13-py3-none-any.whl": {
"sha256": "892ea0cde124a99ce773decba204c5552b69c3c67ffd5f232eb7696135bc8bb3",
"size": 68629
},
"flask-3.1.3-py3-none-any.whl": {
"sha256": "f4bcbefc124291925f1a26446da31a5178f9483862233b23c0c96a20701f670c",
"size": 103424
},
"werkzeug-3.1.8-py3-none-any.whl": {
"sha256": "63a77fb8892bf28ebc3178683445222aa500e48ebad5ec77b0ad80f8726b1f50",
"size": 226459
},
"flask_sqlalchemy-3.1.1-py3-none-any.whl": {
"sha256": "4ba4be7f419dc72f4efd8802d69974803c37259dd42f3913b0dcf75c9447e0a0",
"size": 25125
},
"marshmallow_sqlalchemy-1.5.0-py3-none-any.whl": {
"sha256": "3865232672f3dd38c4d5e4e85fdedce76904200742c3594948a2d11d0af93258",
"size": 16582
},
"alembic-1.18.4-py3-none-any.whl": {
"sha256": "a5ed4adcf6d8a4cb575f3d759f071b03cd6e5c7618eb796cb52497be25bfe19a",
"size": 263893
},
"tabulate-0.10.0-py3-none-any.whl": {
"sha256": "f0b0622e567335c8fabaaa659f1b33bcb6ddfe2e496071b743aa113f8774f2d3",
"size": 39814
},
"cffi-2.1.0-cp314-cp314-win_amd64.whl": {
"sha256": "1b96bfe2c4bd825681b7d311ad6d9b7280a091f43e8f63da5729638083cd3bfb",
"size": 187937
},
"jinja2-3.1.6-py3-none-any.whl": {
"sha256": "85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67",
"size": 134899
},
"cryptography-50.0.0-cp311-abi3-win_amd64.whl": {
"sha256": "bd1c592e4d5974f0d08d4888e432157adba757c66da0246918e43677fafa2d30",
"size": 3840395
},
"certifi-2026.4.22-py3-none-any.whl": {
"sha256": "3cb2210c8f88ba2318d29b0388d1023c8492ff72ecdde4ebdaddbb13a31b1c4a",
"size": 135707
},
"sqlalchemy-2.0.49-cp314-cp314-win_amd64.whl": {
"sha256": "77641d299179c37b89cf2343ca9972c88bb6eef0d5fc504a2f86afd15cd5adf5",
"size": 2144204
},
"blinker-1.9.0-py3-none-any.whl": {
"sha256": "ba0efaa9080b619ff2f3459d1d500c57bddea4a6b424b60a91141db6fd2f08bc",
"size": 8458
},
"pycparser-3.0-py3-none-any.whl": {
"sha256": "b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992",
"size": 48172
},
"flask_cors-6.0.2-py3-none-any.whl": {
"sha256": "e57544d415dfd7da89a9564e1e3a9e515042df76e12130641ca6f3f2f03b699a",
"size": 13257
},
"requests-2.33.1-py3-none-any.whl": {
"sha256": "4e6d1ef462f3626a1f0a0a9c42dd93c63bad33f9f1c1937509b8c5c8718ab56a",
"size": 64947
},
"pyjwt-2.12.1-py3-none-any.whl": {
"sha256": "28ca37c070cad8ba8cd9790cd940535d40274d22f80ab87f3ac6a713e6e8454c",
"size": 29726
},
"click-8.3.3-py3-none-any.whl": {
"sha256": "a2bf429bb3033c89fa4936ffb35d5cb471e3719e1f3c8a7c3fff0b8314305613",
"size": 110502
},
"mysql_connector_python-9.7.0-cp314-cp314-win_amd64.whl": {
"sha256": "5a5abbc152bc28cb2e64a04605ecd9941eff6b0dc5f9528cb84adb873e9a1e49",
"size": 18197576
},
"mako-1.3.12-py3-none-any.whl": {
"sha256": "8f61569480282dbf557145ce441e4ba888be453c30989f879f0d652e39f53ea9",
"size": 78521
},
"itsdangerous-2.2.0-py3-none-any.whl": {
"sha256": "c6242fc49e35958c8b15141343aa660db5fc54d4f13a1db01a3f5891b98700ef",
"size": 16234
}
},
"required": true
},
"urlrewrite": {
"files": {
"rewrite_amd64_en-US.msi": {
"sha256": "37342ff2f585f263f34f48e9de59eb1051d61015a8e967dbde4075716230a32a",
"size": 6078464
}
},
"required": false
},
"mysqlclient": {
"files": {
"libcrypto-3-x64.dll": {
"sha256": "e84ec61fc1b07a3c899ae39c1dc3f38591cdd310c043a388e7ab40a6828aa297",
"size": 5187216
},
"mysqldump.exe": {
"sha256": "329410b7a8ae3d68d38e6f5b781610770854d3f11b975060e231dae62c20dc46",
"size": 7142528
},
"mysql.exe": {
"sha256": "bd6880253c1853fc17b1e19129d6e5d9f9c86a2dccf2c7c9e6b878a10baa9770",
"size": 7177336
},
"libssl-3-x64.dll": {
"sha256": "e24fcd6c8ec31a14c97f520b5b960ed360c55ddf0935040eff297e511697633e",
"size": 788616
}
},
"required": false
}
}
}

261
deploy/windows/installer/bundle-lock.ps1 Normal file
View File

@@ -0,0 +1,261 @@
<#
.SYNOPSIS
Hash manifest for the installer's third-party payload: create it, and check a
bundle against it.
.DESCRIPTION
Dot-source this. It defines two functions and runs nothing on its own:
New-BundleLock hash a staged bundle and return the lock object
Test-BundleLock compare a staged bundle against a lock, return problems
WHAT IT COVERS, and why pip's own hash checking is not enough.
requirements.txt carries a sha256 for every wheel, so pip refuses an artifact
upstream did not publish. Three gaps remain, and all three are what actually
goes wrong with a hand-assembled offline bundle:
1. pip lists EVERY artifact of a pinned version - cffi 2.1.0 alone has 100
hashes. It proves the wheel is genuine, not that it is the wheel this
bundle was built and tested with.
2. pip ignores extra files in the wheelhouse. A stale wheel left behind by
a previous build sits there unnoticed until a resolve picks it up.
3. pip says nothing about the rest of the payload - the Python installer,
the HttpPlatformHandler MSI, URL Rewrite, MySQL. Those are executables
that run as SYSTEM on the target server and were, until this file, the
only unverified thing the installer would run.
So the lock records an exact file set with a sha256 and a byte size each, and
verification is SET EQUALITY: a missing file, an unexpected extra file, or a
changed file all fail. Nothing is skipped and nothing is "close enough".
The app tree is deliberately NOT covered. It is built from the repository on
every run and changes with every commit; hashing it would make the lock churn
constantly and train everyone to regenerate it without reading it. Git is the
record for the app tree. This file is the record for everything that comes
from outside the repository.
.NOTES
Stock Windows PowerShell 5.1, and also runs under pwsh on Linux so the Bash
builder can call the same checker.
#>
# Payload directories under the bundle root. 'required' means the installer
# cannot work without it; the optional ones are per-deployment choices, and an
# absent optional directory is fine. A PRESENT directory is always checked in
# full, optional or not.
$script:BundlePayloads = @(
@{ Name = 'wheels'; Required = $true; What = 'Python wheels for the offline install' }
@{ Name = 'python'; Required = $true; What = 'the Python installer' }
@{ Name = 'httpplatformhandler'; Required = $true; What = 'the IIS module that launches waitress' }
@{ Name = 'urlrewrite'; Required = $false; What = 'IIS URL Rewrite, for the client-IP rule' }
@{ Name = 'mysqlclient'; Required = $false; What = 'mysql/mysqldump, for backups against a remote database' }
@{ Name = 'vcredist'; Required = $false; What = 'the Visual C++ runtime MySQL requires' }
@{ Name = 'mysql'; Required = $false; What = 'MySQL, for the bundled-database option' }
)
function Get-JsonProperty {
# shopdb-install.ps1 runs under Set-StrictMode 2.0, where reading a property
# that does not exist on a PSCustomObject THROWS instead of returning $null.
# A truncated or hand-edited bundle-lock.json would therefore blow up with
# "Property 'payloads' cannot be found" rather than saying what is wrong with
# the lock. Every read of parsed JSON goes through here.
param($Object, [string] $Name, $Default = $null)
if ($null -eq $Object) { return $Default }
$prop = $Object.PSObject.Properties[$Name]
if ($null -eq $prop) { return $Default }
return $prop.Value
}
function Get-FileDigest {
param([string] $Path)
$sha = [System.Security.Cryptography.SHA256]::Create()
$stream = [System.IO.File]::OpenRead($Path)
try { return (-join ($sha.ComputeHash($stream) | ForEach-Object { $_.ToString('x2') })) }
finally { $stream.Dispose(); $sha.Dispose() }
}
function Get-PayloadFiles {
<#
Every file in the directory, keyed by its path RELATIVE to that directory
with forward slashes, so a lock generated on Windows reads the same from
the Bash builder.
The root comes from Get-Item, NOT Resolve-Path, and the prefix is checked
before it is trimmed. Both matter, and a real install proved it:
Inno extracts the bundle under C:\Users\ADMINI~1\AppData\Local\Temp\... -
an 8.3 SHORT path. Resolve-Path kept that short form while Get-ChildItem
returned the long one (Administrator), so the root was five characters
shorter than the prefix it was slicing off. Every relative path came out
mangled - 'wheels/heels/flask.whl' - and the verifier reported all 96 files
as simultaneously missing and unexpected. The payload was fine; the
comparison was not.
Get-Item and Get-ChildItem go through the same provider, so their path
forms agree. The StartsWith guard means that if they ever disagree again
this fails loudly instead of inventing paths.
#>
param([string] $Dir)
$out = @{}
if (-not (Test-Path $Dir)) { return $out }
$rootItem = Get-Item -LiteralPath $Dir
$root = $rootItem.FullName.TrimEnd('\', '/')
foreach ($f in (Get-ChildItem -LiteralPath $rootItem.FullName -Recurse -File)) {
if (-not $f.FullName.StartsWith($root, [System.StringComparison]::OrdinalIgnoreCase)) {
throw ("cannot place '{0}' beneath '{1}' - path forms disagree (8.3 short name?)" -f $f.FullName, $root)
}
$rel = $f.FullName.Substring($root.Length).TrimStart('\', '/').Replace('\', '/')
$out[$rel] = @{ sha256 = (Get-FileDigest $f.FullName); size = $f.Length }
}
return $out
}
function New-BundleLock {
<#
Hash a staged bundle. The caller writes the result to bundle-lock.json;
this returns the object so a caller can diff it against the committed lock
before overwriting anything.
#>
param(
[Parameter(Mandatory = $true)] [string] $BundleRoot,
[string] $PythonTag = 'cp314',
[string] $Platform = 'win_amd64'
)
$payloads = @{}
foreach ($p in $script:BundlePayloads) {
$dir = Join-Path $BundleRoot $p.Name
if (-not (Test-Path $dir)) {
if ($p.Required) { throw ("payload directory is missing: {0} ({1})" -f $p.Name, $p.What) }
continue
}
$files = Get-PayloadFiles $dir
if ($files.Count -eq 0 -and $p.Required) {
throw ("payload directory is empty: {0} ({1})" -f $p.Name, $p.What)
}
$payloads[$p.Name] = @{ required = $p.Required; files = $files }
}
return [ordered]@{
schema = 1
generated = (Get-Date).ToUniversalTime().ToString('yyyy-MM-ddTHH:mm:ssZ')
pythontag = $PythonTag
platform = $Platform
note = 'Exact third-party payload of the installer bundle. Regenerate with refresh-bundle-lock.ps1 and COMMIT the change as a reviewed dependency bump.'
payloads = $payloads
}
}
function Test-BundleLock {
<#
Compare a staged bundle against a lock. Returns an array of problem
strings - EMPTY means the bundle is exactly what the lock describes.
Returning problems rather than throwing is deliberate: an operator fixing a
wheelhouse wants the whole list at once, not one failure per rebuild.
#>
param(
[Parameter(Mandatory = $true)] [string] $BundleRoot,
[Parameter(Mandatory = $true)] $Lock
)
$problems = @()
$payloads = Get-JsonProperty $Lock 'payloads'
# ,@(...) for the same reason as the final return: a bare one-element array
# unrolls to a string, and the caller's .Count then measures the wrong thing.
if ($null -eq $payloads) { return ,@('bundle-lock.json has no "payloads" section') }
foreach ($p in $script:BundlePayloads) {
$name = $p.Name
$dir = Join-Path $BundleRoot $name
$present = Test-Path $dir
$entry = Get-JsonProperty $payloads $name
$locked = ($null -ne $entry)
if (-not $locked) {
# Not in the lock at all. An unlocked directory that exists is a
# payload nobody reviewed, which is exactly what this is here to stop.
if ($present) { $problems += "$name/ is present but is not in bundle-lock.json - regenerate the lock" }
elseif ($p.Required) { $problems += "$name/ is required but is in neither the bundle nor the lock" }
continue
}
if (-not $present) {
if ($p.Required -or (Get-JsonProperty $entry 'required' $false)) {
$problems += "$name/ is in the lock but missing from the bundle ($($p.What))"
}
continue
}
$expected = @{}
$lockedFiles = Get-JsonProperty $entry 'files'
if ($null -ne $lockedFiles) {
foreach ($prop in $lockedFiles.PSObject.Properties) { $expected[$prop.Name] = $prop.Value }
}
$actual = Get-PayloadFiles $dir
# Sorted so the report reads the same way twice, and so it matches the
# order verify_bundle_lock.py produces.
foreach ($rel in ($expected.Keys | Sort-Object)) {
if (-not $actual.ContainsKey($rel)) { $problems += "$name/$rel is in the lock but missing from the bundle"; continue }
if ($actual[$rel].sha256 -ne $expected[$rel].sha256) {
$problems += "$name/$rel does NOT match the lock (expected sha256 $($expected[$rel].sha256.Substring(0,12))..., got $($actual[$rel].sha256.Substring(0,12))...)"
} elseif ([int64] $actual[$rel].size -ne [int64] $expected[$rel].size) {
# Cannot happen for a matching sha256, so it means the lock itself
# was hand-edited. Say so rather than passing it.
$problems += "$name/$rel size disagrees with the lock - the lock has been edited by hand"
}
}
foreach ($rel in ($actual.Keys | Sort-Object)) {
if (-not $expected.ContainsKey($rel)) { $problems += "$name/$rel is in the bundle but NOT in the lock (unexpected extra file)" }
}
}
$problems += Test-WheelhouseCoversRequirements -BundleRoot $BundleRoot
# The comma keeps this an ARRAY through the return. Without it PowerShell
# unrolls an empty result to $null and a single result to a bare string, and
# every caller that asks for .Count then behaves differently depending on how
# many problems there happen to be.
return ,$problems
}
function Test-WheelhouseCoversRequirements {
<#
The lock records what IS in the wheelhouse, not what the application NEEDS.
Without this an incomplete wheelhouse gets locked, blessed, and shipped,
and the install fails on an air-gapped server.
Not hypothetical: assembling the wheelhouse anywhere other than Windows
silently omits colorama, a win32-only dependency of click, because pip
evaluates environment markers against the machine doing the downloading
rather than the machine being targeted. Markers are therefore IGNORED here
- a requirement guarded by sys_platform == 'win32' is precisely the one
that has to be present.
#>
param([Parameter(Mandatory = $true)] [string] $BundleRoot)
$wheels = Join-Path $BundleRoot 'wheels'
$reqs = Join-Path $BundleRoot 'app\requirements.txt'
if (-not (Test-Path $wheels) -or -not (Test-Path $reqs)) { return @() }
$have = @(Get-ChildItem $wheels -File -ErrorAction SilentlyContinue | ForEach-Object { $_.Name.ToLower() })
$problems = @()
$pins = @{}
foreach ($line in (Get-Content $reqs)) {
$trimmed = $line.Trim()
if (-not $trimmed -or $trimmed.StartsWith('#')) { continue }
if ($trimmed -match '^([A-Za-z0-9._-]+)==([^\s;\\]+)') {
# PEP 427 wheel filename form: runs of non-alphanumerics become one _.
$pins[([regex]::Replace($Matches[1], '[^A-Za-z0-9.]+', '_')).ToLower()] = $Matches[2]
}
}
foreach ($name in ($pins.Keys | Sort-Object)) {
$prefix = "$name-$($pins[$name])-"
if (-not ($have | Where-Object { $_.StartsWith($prefix) })) {
$problems += ("wheels/ has no wheel for {0}=={1}, which requirements.txt pins " +
"(a marked-out dependency still installs on Windows)") -f $name, $pins[$name]
}
}
return $problems
}
function Read-BundleLock {
param([Parameter(Mandatory = $true)] [string] $Path)
if (-not (Test-Path $Path)) { return $null }
return (Get-Content $Path -Raw | ConvertFrom-Json)
}

159
deploy/windows/installer/make-branding.py Normal file
View File

@@ -0,0 +1,159 @@
#!/usr/bin/env python3
"""Generate Inno Setup wizard artwork from the app's own brand assets.
Everything here is derived from frontend/public/*.svg so the installer and the
running application are visibly the same product. Nothing is redrawn by hand.
Inno stretches artwork to fit and does not resample well, so render at the exact
sizes it asks for and supply the 125%/250% variants for high-DPI displays.
WizardImageFile 164x314, 192x386, 384x772
WizardSmallImageFile 55x55, 64x64, 138x138
SetupIconFile .ico with 16/24/32/48/64/128/256
Usage: python3 make-branding.py [output-dir]
"""
import io
import sys
from pathlib import Path
import cairosvg
from PIL import Image, ImageDraw, ImageFont
ASSETS = Path.home() / "projects/shopdb-flask/frontend/public"
OUT = Path(sys.argv[1] if len(sys.argv) > 1 else Path(__file__).parent)
# Sampled from the application's own palette so the installer does not look like
# a different product wearing the same badge.
NAVY = (10, 34, 74) # deep base
BLUE = (16, 74, 150) # GE blue
CYAN = (0, 158, 224) # accent
WHITE = (255, 255, 255)
MUTED = (176, 197, 226)
def render_svg(name, width=None, height=None):
png = cairosvg.svg2png(url=str(ASSETS / name), output_width=width, output_height=height)
return Image.open(io.BytesIO(png)).convert("RGBA")
def recolour(img, colour):
"""Replace RGB while keeping the alpha mask. Source marks are dark-on-light;
on a dark panel they must be inverted or they disappear."""
solid = Image.new("RGBA", img.size, colour + (255,))
solid.putalpha(img.getchannel("A"))
return solid
def font(size, bold=False):
for path in (
f"/usr/share/fonts/truetype/dejavu/DejaVuSans{'-Bold' if bold else ''}.ttf",
f"/usr/share/fonts/truetype/liberation/LiberationSans{'-Bold' if bold else '-Regular'}.ttf",
):
if Path(path).exists():
return ImageFont.truetype(path, size)
return ImageFont.load_default()
def vertical_gradient(size, top, bottom):
w, h = size
img = Image.new("RGB", size)
draw = ImageDraw.Draw(img)
for y in range(h):
t = y / max(1, h - 1)
# Ease the ramp so the middle does not look flat.
t = t * t * (3 - 2 * t)
draw.line(
[(0, y), (w, y)],
fill=tuple(int(top[i] + (bottom[i] - top[i]) * t) for i in range(3)),
)
return img
def banner(w, h):
img = vertical_gradient((w, h), BLUE, NAVY)
draw = ImageDraw.Draw(img)
k = w / 164.0 # scale factor from the 100% design
# Faint diagonal wash: stops the flat area under the text reading as empty.
glow = Image.new("RGBA", (w, h), (0, 0, 0, 0))
gd = ImageDraw.Draw(glow)
gd.polygon([(0, int(h * 0.52)), (w, int(h * 0.30)), (w, h), (0, h)],
fill=(255, 255, 255, 10))
img = Image.alpha_composite(img.convert("RGBA"), glow).convert("RGB")
draw = ImageDraw.Draw(img)
margin = int(22 * k)
# GE Aerospace wordmark at the top, above the product name - the corporate
# mark leads, the product sits under it. (Previously the bare monogram was
# here and the wordmark was stranded at the bottom.)
mark_w = w - (margin * 2)
mark = render_svg("ge-aerospace-logo.svg", mark_w, int(mark_w * 32 / 138))
mark = recolour(mark, WHITE)
img.paste(mark, (margin, int(34 * k)), mark)
# Product name, directly beneath it.
y = int(34 * k) + mark.height + int(30 * k)
draw.text((margin, y), "ShopDB", font=font(int(26 * k), bold=True), fill=WHITE)
y += int(31 * k)
# Hairline rule, then the descriptor. Cheap way to look considered.
draw.rectangle([margin, y, margin + int(30 * k), y + max(1, int(2 * k))], fill=CYAN)
y += int(14 * k)
for line in ("Asset management", "for the shop floor"):
draw.text((margin, y), line, font=font(int(10.5 * k)), fill=MUTED)
y += int(15 * k)
# Accent bar flush to the bottom edge.
bar = max(2, int(4 * k))
draw.rectangle([0, h - bar, w, h], fill=CYAN)
return img
def small(size):
"""Header mark on every page after the welcome page. White plate so it sits
correctly on the wizard's own header, in light or dark mode."""
img = Image.new("RGB", (size, size), WHITE)
m = int(size * 0.80)
mono = recolour(render_svg("ge-monogram.svg", m, m), BLUE)
off = (size - m) // 2
img.paste(mono, (off, off), mono)
return img
def icon(path):
"""Installer icon. Rounded navy tile with the monogram, so it reads at 16px
instead of turning into mush."""
base = 256
img = Image.new("RGBA", (base, base), (0, 0, 0, 0))
d = ImageDraw.Draw(img)
d.rounded_rectangle([0, 0, base - 1, base - 1], radius=int(base * 0.22), fill=BLUE + (255,))
d.rounded_rectangle([0, 0, base - 1, int(base * 0.5)], radius=int(base * 0.22),
fill=(30, 96, 175, 255))
d.rounded_rectangle([0, int(base * 0.3), base - 1, base - 1], radius=int(base * 0.22),
fill=BLUE + (255,))
m = int(base * 0.62)
mono = recolour(render_svg("ge-monogram.svg", m, m), WHITE)
img.paste(mono, ((base - m) // 2, (base - m) // 2), mono)
img.save(path, sizes=[(16, 16), (24, 24), (32, 32), (48, 48), (64, 64),
(128, 128), (256, 256)])
def main():
OUT.mkdir(parents=True, exist_ok=True)
made = []
for w, h, name in [(164, 314, "wizard-image.bmp"),
(192, 386, "wizard-image@125.bmp"),
(384, 772, "wizard-image@250.bmp")]:
banner(w, h).save(OUT / name, "BMP"); made.append((name, f"{w}x{h}"))
for s, name in [(55, "wizard-small.bmp"), (64, "wizard-small@125.bmp"),
(138, "wizard-small@250.bmp")]:
small(s).save(OUT / name, "BMP"); made.append((name, f"{s}x{s}"))
icon(OUT / "shopdb.ico"); made.append(("shopdb.ico", "multi-size"))
for name, dims in made:
print(f" {name:<26} {dims:<10} {(OUT / name).stat().st_size // 1024} KB")
if __name__ == "__main__":
main()

110
deploy/windows/installer/refresh-bundle-lock.ps1 Normal file
View File

@@ -0,0 +1,110 @@
<#
.SYNOPSIS
Regenerate bundle-lock.json from the staged bundle, after showing what would
change.
.DESCRIPTION
Run this on the machine that assembled the wheelhouse, once the bundle holds
the payload you intend to ship:
bundle\wheels\ the wheels, built with the matching Python
bundle\python\ the Python installer
bundle\httpplatformhandler\ the IIS module MSI
bundle\urlrewrite\ URL Rewrite MSI (optional)
bundle\mysql\ MySQL MSI (optional)
Then COMMIT the resulting bundle-lock.json. That commit is the review: it is
the only place a change to what runs as SYSTEM on a customer's server becomes
visible to a human. A lock regenerated and committed without reading the diff
provides nothing, so this refuses to overwrite an existing lock until you
have seen the change and passed -Yes.
.EXAMPLE
.\refresh-bundle-lock.ps1 # show the diff, write nothing
.\refresh-bundle-lock.ps1 -Yes # write it
.NOTES
Building the wheelhouse itself no longer requires Windows. From any machine:
pip download -r requirements.txt -d wheels --only-binary=:all: `
--platform win_amd64 --python-version 314 --implementation cp --abi cp314
Do it wherever you like; this script records what came out.
#>
[CmdletBinding()]
param(
[string] $BundleRoot = (Join-Path $PSScriptRoot 'bundle'),
[string] $LockPath = (Join-Path $PSScriptRoot 'bundle-lock.json'),
[string] $PythonTag = 'cp314',
[string] $Platform = 'win_amd64',
[switch] $Yes
)
$ErrorActionPreference = 'Stop'
. (Join-Path $PSScriptRoot 'bundle-lock.ps1')
function Say { param($m, $c = 'Gray') Write-Host $m -ForegroundColor $c }
if (-not (Test-Path $BundleRoot)) {
Say "bundle not found: $BundleRoot" 'Red'
Say 'Stage it first with build-installer.ps1 (or build-installer.sh), then add' 'Yellow'
Say 'the wheels and installers by hand.' 'Yellow'
exit 1
}
Say ''
Say " Hashing $BundleRoot" 'Cyan'
$new = New-BundleLock -BundleRoot $BundleRoot -PythonTag $PythonTag -Platform $Platform
foreach ($name in $new.payloads.Keys) {
Say (" {0,-22} {1,4} files" -f $name, $new.payloads[$name].files.Count)
}
$old = Read-BundleLock $LockPath
if (-not $old) {
Say ''
Say ' No existing lock - this will be the first one.' 'Yellow'
} else {
# Diff by file, per payload, so the operator sees exactly which artifacts
# changed rather than "the lock is different".
Say ''
Say ' Changes against the committed lock:' 'Cyan'
$changes = 0
foreach ($name in $new.payloads.Keys) {
$oldFiles = @{}
if ($old.payloads.PSObject.Properties.Name -contains $name) {
foreach ($p in $old.payloads.$name.files.PSObject.Properties) { $oldFiles[$p.Name] = $p.Value.sha256 }
}
$newFiles = $new.payloads[$name].files
foreach ($rel in ($newFiles.Keys | Sort-Object)) {
if (-not $oldFiles.ContainsKey($rel)) { Say " + $name/$rel" 'Green'; $changes++ }
elseif ($oldFiles[$rel] -ne $newFiles[$rel].sha256) { Say " ~ $name/$rel (content changed)" 'Yellow'; $changes++ }
}
foreach ($rel in ($oldFiles.Keys | Sort-Object)) {
if (-not $newFiles.ContainsKey($rel)) { Say " - $name/$rel" 'Red'; $changes++ }
}
}
foreach ($p in $old.payloads.PSObject.Properties.Name) {
if (-not $new.payloads.Contains($p)) { Say " - $p/ (whole payload gone)" 'Red'; $changes++ }
}
if ($changes -eq 0) {
Say ' none - the bundle already matches the lock' 'Green'
exit 0
}
Say ''
Say (" {0} change(s)." -f $changes) 'White'
}
if (-not $Yes) {
Say ''
Say ' Nothing written. Read the list above, then re-run with -Yes.' 'Yellow'
Say ' Commit the resulting bundle-lock.json - that commit IS the review.' 'Yellow'
exit 2
}
# ConvertTo-Json defaults to a depth of 2, which silently flattens the per-file
# entries into "System.Collections.Hashtable" strings and produces a lock that
# verifies against nothing.
$new | ConvertTo-Json -Depth 8 | Set-Content -Path $LockPath -Encoding UTF8
Say ''
Say " Written: $LockPath" 'Green'
Say ' Commit it.' 'Green'

1045
deploy/windows/installer/shopdb-admin.ps1 Normal file
View File

File diff suppressed because it is too large Load Diff

2805
deploy/windows/installer/shopdb-install.ps1 Normal file
View File

File diff suppressed because it is too large Load Diff

562
deploy/windows/installer/shopdb-preflight.ps1 Normal file
View File

@@ -0,0 +1,562 @@
<#
.SYNOPSIS
ShopDB-Flask installer - Stage 1: read-only preflight.
.DESCRIPTION
Discovers everything the installer needs to know about this box and reports
it. Makes NO changes: no installs, no config edits, no service restarts.
Safe to run on a production server.
Written for stock Windows PowerShell 5.1 (Windows Server ships it). No
pwsh-only syntax, no external modules, no network access.
.PARAMETER SitePort
The port the ShopDB site will listen on. Default 8090 (the runbook's example;
the classic ASP site keeps 8080).
.PARAMETER AppRoot
Intended install directory. Default C:\shopdb-flask.
.PARAMETER Json
Emit machine-readable JSON instead of the human report. Later installer
stages consume this.
.EXAMPLE
powershell -ExecutionPolicy Bypass -File .\shopdb-preflight.ps1
powershell -ExecutionPolicy Bypass -File .\shopdb-preflight.ps1 -Json > preflight.json
.NOTES
Exit 0 = no blocking problems. Exit 1 = at least one FAIL.
#>
[CmdletBinding()]
param(
[int] $SitePort = 8090,
[string] $AppRoot = 'C:\shopdb-flask',
# Needed so the port check can tell OUR site apart from a stranger's.
[string] $SiteName = 'shopdb-flask',
[switch] $Json,
# Machine-readable output for a GUI caller: one record per line,
# STATUS|AREA|CHECK|DETAIL|FIX
# The console rendering below aligns columns with padding spaces, which only
# works in a fixed-width font at console width. A GUI must do its own layout,
# so give it DATA and let it decide - do not make it parse a formatted table.
# (-Json exists too, but Inno's Pascal Script has no JSON parser.)
[switch] $Delimited
)
Set-StrictMode -Version 2.0
$ErrorActionPreference = 'Stop'
# --- result collection -------------------------------------------------------
# Every check appends one record. Status is PASS / WARN / FAIL / INFO.
$script:Results = New-Object System.Collections.ArrayList
$script:IisPresent = $false
function Add-Result {
param(
[string] $Area,
[string] $Check,
[ValidateSet('PASS','WARN','FAIL','INFO','SKIP')] [string] $Status,
[string] $Detail,
[string] $Fix = ''
)
$null = $script:Results.Add([PSCustomObject]@{
Area = $Area
Check = $Check
Status = $Status
Detail = $Detail
Fix = $Fix
})
}
# Wrap a check so one failure cannot abort the whole run. On an unfamiliar box
# an unexpected exception is itself a finding, not a crash.
function Invoke-Check {
param([string] $Area, [string] $Check, [scriptblock] $Body)
try { & $Body }
catch {
Add-Result $Area $Check 'WARN' "check could not run: $($_.Exception.Message)" `
'Report this output; the installer needs to handle this box shape.'
}
}
# =============================================================================
# 1. Operator context
# =============================================================================
Invoke-Check 'System' 'Elevation' {
$id = [Security.Principal.WindowsIdentity]::GetCurrent()
$adm = (New-Object Security.Principal.WindowsPrincipal($id)).IsInRole(
[Security.Principal.WindowsBuiltInRole]::Administrator)
if ($adm) { Add-Result 'System' 'Elevation' 'PASS' 'running as Administrator' }
else {
Add-Result 'System' 'Elevation' 'FAIL' 'not elevated' `
'Re-run PowerShell as Administrator. IIS and service changes require it.'
}
}
Invoke-Check 'System' 'Windows version' {
$os = Get-CimInstance Win32_OperatingSystem
$name = $os.Caption
$ver = $os.Version
# ProductType: 1 = workstation, 2 = domain controller, 3 = server
$isServer = ($os.ProductType -ne 1)
$detail = "$name (build $ver), $(if ($isServer) {'Server'} else {'Client'})"
$supported = $false
if ($isServer -and [version]$ver -ge [version]'10.0.17763') { $supported = $true } # 2019+
if (-not $isServer -and [version]$ver -ge [version]'10.0.19045') { $supported = $true } # Win10 22H2+
if ($supported) { Add-Result 'System' 'Windows version' 'PASS' $detail }
else {
Add-Result 'System' 'Windows version' 'FAIL' $detail `
'Supported: Windows Server 2019/2022+, or Windows 10 22H2 / 11 Pro+.'
}
# Client SKUs must be Pro/Enterprise/Education for IIS.
if (-not $isServer -and $name -match 'Home') {
Add-Result 'System' 'Windows edition' 'FAIL' 'Windows Home edition' `
'IIS is not available on Home editions. Pro or higher is required.'
}
}
Invoke-Check 'System' 'Architecture' {
if ([Environment]::Is64BitOperatingSystem) {
Add-Result 'System' 'Architecture' 'PASS' '64-bit'
} else {
Add-Result 'System' 'Architecture' 'FAIL' '32-bit' `
'The bundled Python and wheels are 64-bit (win_amd64) only.'
}
}
Invoke-Check 'System' 'PowerShell version' {
$v = $PSVersionTable.PSVersion
Add-Result 'System' 'PowerShell version' 'INFO' "$v"
if ($v.Major -lt 5) {
Add-Result 'System' 'PowerShell version' 'FAIL' "$v" `
'PowerShell 5.1 or later is required.'
}
}
# =============================================================================
# 2. Disk and ports
# =============================================================================
Invoke-Check 'Disk' 'Free space' {
$drive = (Split-Path -Qualifier $AppRoot)
$d = Get-CimInstance Win32_LogicalDisk -Filter "DeviceID='$drive'"
if ($null -eq $d) {
Add-Result 'Disk' 'Free space' 'FAIL' "drive $drive not found" `
"Choose an -AppRoot on an existing volume."
return
}
$freeGB = [math]::Round($d.FreeSpace / 1GB, 1)
$detail = "$freeGB GB free on $drive"
if ($freeGB -ge 5) { Add-Result 'Disk' 'Free space' 'PASS' $detail }
else { Add-Result 'Disk' 'Free space' 'FAIL' $detail 'At least 5 GB is required.' }
}
Invoke-Check 'Disk' 'AppRoot' {
if (Test-Path $AppRoot) {
$existing = @(Get-ChildItem $AppRoot -Force -ErrorAction SilentlyContinue)
if ($existing.Count -gt 0) {
$hasEnv = Test-Path (Join-Path $AppRoot '.env')
if ($hasEnv) {
# An existing install is the NORMAL state for an upgrade. Reporting
# it as a warning makes a routine update look like a problem.
$ver = ''
$vf = Join-Path $AppRoot '.installed-version'
if (Test-Path $vf) { $ver = ' version ' + (Get-Content $vf -TotalCount 1).Trim() }
Add-Result 'Disk' 'AppRoot' 'INFO' `
"existing ShopDB-Flask install found$ver - it will be upgraded in place, and your settings and database are kept"
} else {
Add-Result 'Disk' 'AppRoot' 'WARN' "$AppRoot exists and is not empty" `
'Confirm this directory is safe to install into.'
}
} else { Add-Result 'Disk' 'AppRoot' 'PASS' "$AppRoot exists and is empty" }
} else { Add-Result 'Disk' 'AppRoot' 'PASS' "$AppRoot does not exist yet" }
}
function Test-PortFree {
param([int] $Port)
# Get-NetTCPConnection is the reliable listener check on Server 2012R2+.
try {
$listening = @(Get-NetTCPConnection -State Listen -LocalPort $Port -ErrorAction SilentlyContinue)
return ($listening.Count -eq 0)
} catch {
# Fall back to a bind attempt if the cmdlet is unavailable.
try {
$l = New-Object System.Net.Sockets.TcpListener([System.Net.IPAddress]::Any, $Port)
$l.Start(); $l.Stop(); return $true
} catch { return $false }
}
}
Invoke-Check 'Network' 'Site port' {
if (Test-PortFree $SitePort) {
Add-Result 'Network' 'Site port' 'PASS' "TCP $SitePort is free"
} else {
$owner = ''
try {
$c = Get-NetTCPConnection -State Listen -LocalPort $SitePort -ErrorAction SilentlyContinue | Select-Object -First 1
if ($c) { $owner = " (pid $($c.OwningProcess): $((Get-Process -Id $c.OwningProcess -ErrorAction SilentlyContinue).ProcessName))" }
} catch { }
# Is the listener OUR OWN site? On a reinstall or upgrade the port is held
# by the very application being upgraded, and blocking on that makes the
# installer refuse to update anything it previously installed.
$ours = $false
try {
Import-Module WebAdministration -ErrorAction SilentlyContinue
$site = Get-Website -Name $SiteName -ErrorAction SilentlyContinue
if ($site) {
foreach ($b in $site.bindings.Collection) {
# ${} is required: "$SitePort:" parses as a DRIVE-qualified variable.
if ($b.bindingInformation -match ":${SitePort}:") { $ours = $true }
}
}
} catch { }
if ($ours) {
Add-Result 'Network' 'Site port' 'INFO' `
"TCP $SitePort is used by the existing $SiteName site - this will be upgraded in place"
} else {
# WARN, not FAIL. This check runs before the operator has reached the
# Address page, so it is testing the DEFAULT port, not necessarily
# the one they intend to use. Blocking here would refuse an install
# over a conflict the very next page lets them resolve.
Add-Result 'Network' 'Site port' 'WARN' "TCP $SitePort is in use$owner" `
"Pick a different port on the Address page later in this wizard, or stop whatever is holding it."
}
}
}
# =============================================================================
# 3. IIS
# =============================================================================
Invoke-Check 'IIS' 'Installed' {
$svc = Get-Service -Name W3SVC -ErrorAction SilentlyContinue
$script:IisPresent = ($null -ne $svc)
if ($null -eq $svc) {
Add-Result 'IIS' 'Installed' 'FAIL' 'W3SVC service not found' `
'Install IIS. Server: Install-WindowsFeature Web-Server -IncludeManagementTools. Client: enable Internet Information Services in Windows Features.'
return
}
Add-Result 'IIS' 'Installed' 'PASS' "W3SVC present, status $($svc.Status)"
if ($svc.Status -ne 'Running') {
Add-Result 'IIS' 'Running' 'WARN' "W3SVC is $($svc.Status)" 'Start-Service W3SVC'
}
}
Invoke-Check 'IIS' 'WebAdministration module' {
$m = Get-Module -ListAvailable -Name WebAdministration
if ($m) { Add-Result 'IIS' 'WebAdministration module' 'PASS' 'available' }
else {
Add-Result 'IIS' 'WebAdministration module' 'FAIL' 'not available' `
'Install the IIS management tools (Web-Mgmt-Console / IIS Management Scripts and Tools).'
}
}
Invoke-Check 'IIS' 'HttpPlatformHandler' {
# The handler registers itself as a global module. Check the module list.
$appcmd = Join-Path $env:windir 'system32\inetsrv\appcmd.exe'
if (-not (Test-Path $appcmd)) {
Add-Result 'IIS' 'HttpPlatformHandler' 'SKIP' 'appcmd.exe not present (IIS not installed)' `
'Re-run this preflight after installing IIS.'
return
}
$modules = & $appcmd list module 2>$null
if ($modules -match 'httpPlatformHandler') {
Add-Result 'IIS' 'HttpPlatformHandler' 'PASS' 'installed'
} else {
# NOT a blocker: the MSI is in the bundle and stage 4 installs it. This
# was a FAIL, which - once the preflight page started blocking on any
# failure - stopped the wizard dead over something the installer was
# about to do by itself, with no way forward but to go and install it by
# hand. Nothing the installer SUPPLIES may be a blocker.
Add-Result 'IIS' 'HttpPlatformHandler' 'INFO' 'not installed yet' `
'The installer installs it from the bundle. No action needed.'
}
}
Invoke-Check 'IIS' 'Locked config sections' {
# The authoritative source is applicationHost.config. `appcmd list config
# /section:X` prints the section CONTENTS, not its lock state, so grepping
# that output silently reports every section as unlocked.
$cfg = Join-Path $env:windir 'system32\inetsrv\config\applicationHost.config'
if (-not (Test-Path $cfg)) {
Add-Result 'IIS' 'Locked config sections' 'SKIP' 'applicationHost.config not found'
return
}
foreach ($name in @('handlers','httpPlatform')) {
$line = Select-String -Path $cfg -Pattern ('<section name="' + $name + '"') |
Select-Object -First 1
if ($null -eq $line) {
# httpPlatform is registered by the HttpPlatformHandler MSI. Before
# that, `appcmd unlock config /section:system.webServer/httpPlatform`
# fails with "Unknown config section".
Add-Result 'IIS' "Section $name" 'SKIP' 'section not registered yet' `
'Install HttpPlatformHandler FIRST; only then can this section be unlocked.'
} elseif ($line.Line -match 'overrideModeDefault="Deny"') {
Add-Result 'IIS' "Section $name" 'WARN' 'locked (overrideModeDefault="Deny")' `
"Installer must run: appcmd unlock config /section:system.webServer/$name (else IIS 500.19)"
} else {
Add-Result 'IIS' "Section $name" 'PASS' 'not locked'
}
}
}
Invoke-Check 'IIS' 'URL Rewrite module' {
$appcmd = Join-Path $env:windir 'system32\inetsrv\appcmd.exe'
if (-not (Test-Path $appcmd)) {
Add-Result 'IIS' 'URL Rewrite module' 'SKIP' 'IIS not installed; cannot check'
return
}
$modules = & $appcmd list module 2>$null
if ($modules -match 'RewriteModule') {
Add-Result 'IIS' 'URL Rewrite module' 'PASS' 'installed (the X-Forwarded-For rule can be enabled)'
} else {
# Not a FAIL: it is only needed for -ClientIpSource direct, and the
# installer carries the MSI and installs it itself. Worth reporting
# because without the rule IIS sends no X-Forwarded-For at all, so every
# client reads as 127.0.0.1 and the GE-Enforce IP allowlist, the
# visitor-location lookup and per-host login rate limiting go quiet.
Add-Result 'IIS' 'URL Rewrite module' 'INFO' 'not installed' `
'The installer installs it from the bundle when -ClientIpSource is direct. Behind a reverse proxy that already sets X-Forwarded-For, use -ClientIpSource proxy and leave it out.'
}
}
Invoke-Check 'IIS' 'Existing sites' {
if (-not $script:IisPresent) {
Add-Result 'IIS' 'Existing sites' 'SKIP' 'IIS not installed'
return
}
try {
Import-Module WebAdministration -ErrorAction Stop
$sites = @(Get-Website)
if ($sites.Count -eq 0) { Add-Result 'IIS' 'Existing sites' 'INFO' 'none' ; return }
$desc = ($sites | ForEach-Object {
$b = ($_.bindings.Collection | ForEach-Object { $_.bindingInformation }) -join ','
"$($_.Name) [$($_.State)] $b"
}) -join '; '
Add-Result 'IIS' 'Existing sites' 'INFO' $desc
# Adoption sites typically run the classic ASP shopdb here already.
if ($desc -match '8080') {
Add-Result 'IIS' 'Classic ASP site' 'INFO' 'a site is bound on 8080 (likely the classic ASP shopdb)' `
'Install ShopDB as a separate site on its own port; do not disturb this one.'
}
} catch {
Add-Result 'IIS' 'Existing sites' 'WARN' "could not enumerate: $($_.Exception.Message)" ''
}
}
# =============================================================================
# 4. MySQL (detect BEFORE offering bundled vs existing)
# =============================================================================
Invoke-Check 'MySQL' 'Service' {
$svcs = @(Get-Service -ErrorAction SilentlyContinue | Where-Object { $_.Name -match '^MySQL' -or $_.DisplayName -match 'MySQL' })
if ($svcs.Count -eq 0) {
Add-Result 'MySQL' 'Service' 'INFO' 'no MySQL service found' `
'Bundled MySQL 8.4 LTS is the appropriate choice on this box.'
return
}
foreach ($s in $svcs) {
Add-Result 'MySQL' 'Service' 'WARN' "$($s.Name) ($($s.DisplayName)) is $($s.Status)" `
'MySQL already present. Default to the EXISTING-server option; installing bundled MySQL will collide on port 3306.'
}
}
Invoke-Check 'MySQL' 'Port 3306' {
if (Test-PortFree 3306) {
Add-Result 'MySQL' 'Port 3306' 'INFO' 'nothing listening on 3306'
} else {
Add-Result 'MySQL' 'Port 3306' 'WARN' 'something is listening on 3306' `
'Bundled MySQL cannot use the default port. Use the existing server, or pick another port.'
}
}
Invoke-Check 'MySQL' 'Backup client' {
# mysqldump is what takes the mandatory pre-upgrade backup. Without it every
# upgrade skips the backup - and skips it AFTER the application pool has been
# stopped and the tree replaced, so the site is down and there is nothing to
# restore from. A site whose database is on another server typically has no
# MySQL client installed here at all, which is exactly the case that needs it.
$names = @('mysqldump.exe')
$found = ''
foreach ($root in @((Join-Path $PSScriptRoot 'mysqlclient'),
(Join-Path $AppRoot 'mysqlclient'),
'C:\MySQL84', 'C:\Program Files\MySQL', 'C:\mysql56\bin',
'C:\Program Files (x86)\MySQL')) {
if (-not (Test-Path $root)) { continue }
$hit = Get-ChildItem $root -Filter $names[0] -Recurse -ErrorAction SilentlyContinue |
Select-Object -First 1
if ($hit) { $found = $hit.FullName; break }
}
if ($found) {
Add-Result 'MySQL' 'Backup client' 'PASS' "mysqldump found ($found)"
} else {
Add-Result 'MySQL' 'Backup client' 'WARN' 'mysqldump not found on this server' `
'Needed for the automatic pre-upgrade backup and for "shopdb-admin.ps1 backup". A first install works without it; upgrades will not be protected. It is not on this server yet - the installer stages its own copy, so this normally resolves itself during installation.'
}
}
Invoke-Check 'MySQL' 'Version and config' {
# Find mysqld.exe via the service binary path; read the version and locate my.ini.
$svc = Get-CimInstance Win32_Service -ErrorAction SilentlyContinue |
Where-Object { $_.PathName -match 'mysqld' } | Select-Object -First 1
if ($null -eq $svc) { return }
$path = $svc.PathName
$exe = ''
if ($path -match '"([^"]+mysqld[^"]*)"') { $exe = $matches[1] }
elseif ($path -match '(\S+mysqld\S*)') { $exe = $matches[1] }
$ver = ''
if ($exe -and (Test-Path $exe)) {
try { $ver = (& $exe --version 2>$null | Out-String).Trim() } catch { }
}
if ($ver) { Add-Result 'MySQL' 'Version' 'INFO' $ver }
# my.ini path is passed as --defaults-file in the service command line.
$ini = ''
if ($path -match '--defaults-file="?([^"]+\.ini)"?') { $ini = $matches[1] }
if ($ini -and (Test-Path $ini)) {
Add-Result 'MySQL' 'Config file' 'INFO' $ini
# MySQL 5.6 needs three flags or `flask db upgrade` dies with error 1071.
$is56 = ($ver -match '\b5\.6\.')
if ($is56) {
$content = Get-Content $ini -Raw
$need = @('innodb_file_per_table','innodb_file_format','innodb_large_prefix')
$missing = @()
foreach ($k in $need) { if ($content -notmatch $k) { $missing += $k } }
if ($missing.Count -eq 0) {
# Present in the FILE is not the same as ACTIVE. MySQL must be
# restarted for them to take effect, and the app's own
# `flask db-utils preflight` queries the live server - trust that.
Add-Result 'MySQL' '5.6 index flags' 'WARN' 'all three present in my.ini' `
'Present in the file only. They take effect after a MySQL RESTART, which interrupts the classic ASP app. Confirm with SHOW VARIABLES or flask db-utils preflight.'
} else {
# WARN, not FAIL. This inspects the LOCAL MySQL, which may not be the
# database the operator is about to install against - a bundled 8.4,
# or a remote server. Blocking the wizard here refused an install
# over a server that had nothing to do with it. Stage 3 runs
# 'flask db-utils preflight' against the database actually chosen,
# which is the check that can genuinely block.
Add-Result 'MySQL' '5.6 index flags' 'WARN' ("missing: " + ($missing -join ', ')) `
"Add to [mysqld] in $ini and restart MySQL, or 'flask db upgrade' fails with error 1071. NOTE: restarting interrupts the classic ASP app."
}
}
}
}
# =============================================================================
# 5. Python (detect, but the installer uses its OWN bundled interpreter)
# =============================================================================
Invoke-Check 'Python' 'On PATH' {
$cmd = Get-Command python -ErrorAction SilentlyContinue
if ($null -eq $cmd) {
Add-Result 'Python' 'On PATH' 'INFO' 'no python on PATH' `
'Expected. The installer supplies its own interpreter.'
return
}
$v = ''
try { $v = (& $cmd.Source --version 2>&1 | Out-String).Trim() } catch { }
$detail = "$v at $($cmd.Source)"
# A per-user install under %LOCALAPPDATA% is unreadable by the IIS app-pool
# identity. That produces a 500 with an empty HttpPlatform log.
if ($cmd.Source -like "$env:LOCALAPPDATA*") {
Add-Result 'Python' 'On PATH' 'WARN' "$detail (PER-USER install)" `
'The IIS app-pool identity cannot read %LOCALAPPDATA%. The installer must install Python for ALL USERS and use absolute paths.'
} elseif ($cmd.Source -like '*WindowsApps*') {
Add-Result 'Python' 'On PATH' 'WARN' "$detail (Microsoft Store)" `
'Store Python misbehaves under service identities. The installer will not use it.'
} else {
Add-Result 'Python' 'On PATH' 'INFO' $detail `
'Not used by the installer, but a manual `flask` command later would resolve to this interpreter.'
}
}
Invoke-Check 'Python' 'Registered installs' {
$found = @()
foreach ($hive in @('HKLM:\SOFTWARE\Python\PythonCore','HKCU:\SOFTWARE\Python\PythonCore')) {
if (Test-Path $hive) {
foreach ($k in Get-ChildItem $hive -ErrorAction SilentlyContinue) {
$ip = Join-Path $k.PSPath 'InstallPath'
if (Test-Path $ip) {
$loc = (Get-ItemProperty $ip -ErrorAction SilentlyContinue).'(default)'
$scope = if ($hive -like 'HKLM*') { 'all-users' } else { 'per-user' }
$found += "$($k.PSChildName) ($scope) $loc"
}
}
}
}
if ($found.Count -eq 0) { Add-Result 'Python' 'Registered installs' 'INFO' 'none' }
else { Add-Result 'Python' 'Registered installs' 'INFO' ($found -join '; ') }
}
# =============================================================================
# Report
# =============================================================================
$fails = @($script:Results | Where-Object { $_.Status -eq 'FAIL' })
$warns = @($script:Results | Where-Object { $_.Status -eq 'WARN' })
$skips = @($script:Results | Where-Object { $_.Status -eq 'SKIP' })
if ($Delimited) {
# Data only. No padding, no colour, no alignment - the caller lays it out.
# Pipes are stripped from field values so the record can be split naively.
foreach ($r in $script:Results) {
$fix = ''
if ($r.Fix) { $fix = $r.Fix }
$fields = @($r.Status, $r.Area, $r.Check, $r.Detail, $fix) | ForEach-Object {
([string]$_) -replace '\|', '/' -replace '\s*\r?\n\s*', ' '
}
Write-Output ($fields -join '|')
}
} elseif ($Json) {
[PSCustomObject]@{
Timestamp = (Get-Date).ToString('s')
Computer = $env:COMPUTERNAME
SitePort = $SitePort
AppRoot = $AppRoot
Failures = $fails.Count
Warnings = $warns.Count
Skipped = $skips.Count
Results = $script:Results
} | ConvertTo-Json -Depth 5
} else {
Write-Host ''
Write-Host 'ShopDB-Flask preflight' -ForegroundColor Cyan
Write-Host (" host {0} site port {1} approot {2}" -f $env:COMPUTERNAME, $SitePort, $AppRoot)
Write-Host ''
$area = ''
foreach ($r in $script:Results) {
if ($r.Area -ne $area) { $area = $r.Area; Write-Host "[$area]" -ForegroundColor White }
$colour = 'Gray'
if ($r.Status -eq 'PASS') { $colour = 'Green' }
if ($r.Status -eq 'WARN') { $colour = 'Yellow' }
if ($r.Status -eq 'FAIL') { $colour = 'Red' }
if ($r.Status -eq 'SKIP') { $colour = 'DarkGray' }
Write-Host (" {0,-5} {1,-28} {2}" -f $r.Status, $r.Check, $r.Detail) -ForegroundColor $colour
if ($r.Fix -and $r.Status -ne 'PASS' -and $r.Status -ne 'INFO' -and $r.Status -ne 'SKIP') {
Write-Host (" -> {0}" -f $r.Fix) -ForegroundColor DarkGray
}
}
Write-Host ''
if ($fails.Count -eq 0) {
Write-Host "No blocking problems. $($warns.Count) warning(s), $($skips.Count) skipped." -ForegroundColor Green
} else {
Write-Host "$($fails.Count) blocking problem(s), $($warns.Count) warning(s), $($skips.Count) skipped." -ForegroundColor Red
}
if ($skips.Count -gt 0) {
Write-Host " Skipped checks were NOT verified. Re-run once their prerequisite is installed." -ForegroundColor DarkGray
}
Write-Host ''
}
if ($fails.Count -gt 0) { exit 1 } else { exit 0 }

BIN
deploy/windows/installer/shopdb.ico Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 52 KiB

BIN
deploy/windows/installer/swatch-bad.bmp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 822 B

BIN
deploy/windows/installer/swatch-ok.bmp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 822 B

BIN
deploy/windows/installer/swatch-warn.bmp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 822 B

130
deploy/windows/installer/tests/test-install-state.ps1 Normal file
View File

@@ -0,0 +1,130 @@
# Tests the installer's durable install record.
#
# The record answers one question: "is this a re-run of MY install?". Getting it
# wrong is destructive in one direction - answering "first provisioning" for a
# server holding real data lets prune-schema --force loose on it - and a dead
# end in the other, which is what it was built to fix.
#
# Run directly: pwsh -File test-install-state.ps1
# pytest runs it through tests/test_installer_state.py wherever pwsh exists.
Set-StrictMode -Version 2.0
$ErrorActionPreference = 'Stop'
$Here = Split-Path -Parent $MyInvocation.MyCommand.Path
$Installer = Join-Path (Split-Path -Parent $Here) 'shopdb-install.ps1'
if (-not (Test-Path $Installer)) { throw "installer not found at $Installer" }
$Sandbox = Join-Path ([System.IO.Path]::GetTempPath()) ('shopdb-state-' + [System.Guid]::NewGuid().ToString('N'))
# Load ONLY the state block, so the rest of the installer does not run. Bounded
# by the same markers the installer uses, and it fails loudly if either moves
# rather than silently testing nothing.
$lines = Get-Content $Installer
$start = ($lines | Select-String -Pattern '^\$script:StateFile ' | Select-Object -First 1)
$end = ($lines | Select-String -Pattern '^function Invoke-Native' | Select-Object -First 1)
if (-not $start -or -not $end) { throw 'could not locate the install-record block in shopdb-install.ps1' }
$block = $lines[($start.LineNumber - 1)..($end.LineNumber - 2)] -join "`n"
if ($block -notmatch 'function Test-FirstProvisioning') { throw 'extracted block does not contain the state functions' }
# Stand-ins for the surrounding script.
$script:Created = New-Object System.Collections.ArrayList
function Write-Log { param($m, $l = 'INFO') }
function Protect-File { param([string] $Path, [switch] $Directory) }
function Test-SamePath {
param([string] $A, [string] $B)
if (-not $A) { return -not $B }
return ($A.TrimEnd('\', '/').ToLowerInvariant() -eq $B.TrimEnd('\', '/').ToLowerInvariant())
}
if (-not $env:ProgramData) { $env:ProgramData = [System.IO.Path]::GetTempPath() }
Invoke-Expression $block
$script:Failures = 0
function Check {
param([string] $Name, $Expected, $Actual)
if ($Expected -eq $Actual) {
Write-Host (" PASS " + $Name)
} else {
Write-Host (" FAIL {0}: expected [{1}], got [{2}]" -f $Name, $Expected, $Actual)
$script:Failures++
}
}
function Reset-Case {
param([switch] $Stamped)
if (Test-Path $Sandbox) { Remove-Item $Sandbox -Recurse -Force }
New-Item -ItemType Directory -Path $Sandbox -Force | Out-Null
$script:AppRoot = $Sandbox
$script:StateFile = Join-Path $Sandbox 'install-state.json'
$script:InstallState = $null
if ($Stamped) { Set-Content (Join-Path $Sandbox '.installed-version') '0.7.0' }
}
Write-Host 'brand new server'
Reset-Case
Initialize-InstallState
Check 'first provisioning' $true (Test-FirstProvisioning)
Check 'record written' $true (Test-Path $script:StateFile)
Write-Host 'retry of a failed first install'
$script:InstallState = $null
Initialize-InstallState
Check 'still first provisioning' $true (Test-FirstProvisioning)
Write-Host 'genuine upgrade after a completed install'
Complete-InstallState
$script:InstallState = $null
Initialize-InstallState
Check 'no longer first provisioning' $false (Test-FirstProvisioning)
Write-Host 'install predating the record - must take the safe side'
Reset-Case -Stamped
Initialize-InstallState
Check 'treated as established' $false (Test-FirstProvisioning)
Write-Host 'record naming a different directory'
Reset-Case
Initialize-InstallState
$other = Get-Content $script:StateFile -Raw | ConvertFrom-Json
$other.approot = 'C:\somewhere-else'
($other | ConvertTo-Json -Depth 6) | Set-Content $script:StateFile
$script:InstallState = $null
Check 'foreign record rejected' $null (Get-InstallState)
Write-Host 'corrupt record'
Reset-Case
Set-Content $script:StateFile '{ this is not json'
$script:InstallState = $null
Check 'corrupt record rejected' $null (Get-InstallState)
Reset-Case
Set-Content $script:StateFile '{ this is not json'
Initialize-InstallState
Check 'recovers and rewrites' $true ($null -ne $script:InstallState)
Write-Host 'created items survive a crash'
Reset-Case
Initialize-InstallState
Track 'site' 'shopdb-flask'
Track 'apppool' 'shopdbflask'
Track 'site' 'shopdb-flask'
$script:InstallState = $null
Initialize-InstallState
Check 'site remembered' 'shopdb-flask' ((Get-CreatedItems 'site') -join ',')
Check 'apppool remembered' 'shopdbflask' ((Get-CreatedItems 'apppool') -join ',')
# A zero-length array returned from a function unrolls to $null, and $null.Count
# is fatal under StrictMode. This is the check that catches losing the comma.
Check 'unknown kind is an empty array' 0 ((Get-CreatedItems 'firewall')).Count
Write-Host 'no record at all'
$script:InstallState = $null
Check 'empty list, not null' 0 ((Get-CreatedItems 'site')).Count
Check 'not first provisioning' $false (Test-FirstProvisioning)
if (Test-Path $Sandbox) { Remove-Item $Sandbox -Recurse -Force }
if ($script:Failures -gt 0) {
Write-Host ("{0} check(s) failed" -f $script:Failures)
exit 1
}
Write-Host 'all checks passed'
exit 0

171
deploy/windows/installer/verify_bundle_lock.py Normal file
View File

@@ -0,0 +1,171 @@
#!/usr/bin/env python3
"""Check a staged installer bundle against bundle-lock.json.
Prints one line per problem and exits non-zero if there are any. Exits 0 only
when the bundle's third-party payload is EXACTLY what the lock describes: no
missing file, no unexpected extra file, no changed content.
Why this exists alongside bundle-lock.ps1, which does the same job:
- bundle-lock.ps1 is canonical. It runs at INSTALL time on the target server,
where PowerShell is the only thing guaranteed to be present - Python is not
installed until stage 2, and verifying the payload after running part of it
would defeat the purpose.
- This file lets the Linux builder (build-installer.sh) do the same check
without adding pwsh as a build dependency.
The two are kept honest by tests/test_bundle_lock.py, which runs BOTH against
the same fixtures and fails if they disagree.
Usage: verify_bundle_lock.py <bundle-root> <bundle-lock.json>
"""
import hashlib
import json
import os
import re
import sys
# Must match $script:BundlePayloads in bundle-lock.ps1.
PAYLOADS = [
('wheels', True, 'Python wheels for the offline install'),
('python', True, 'the Python installer'),
('httpplatformhandler', True, 'the IIS module that launches waitress'),
('urlrewrite', False, 'IIS URL Rewrite, for the client-IP rule'),
('mysqlclient', False, 'mysql/mysqldump, for backups against a remote database'),
('vcredist', False, 'the Visual C++ runtime MySQL requires'),
('mysql', False, 'MySQL, for the bundled-database option'),
]
def digest(path):
sha = hashlib.sha256()
with open(path, 'rb') as fh:
for chunk in iter(lambda: fh.read(1024 * 1024), b''):
sha.update(chunk)
return sha.hexdigest()
def payload_files(directory):
"""Every file under the directory, keyed by forward-slashed relative path."""
found = {}
if not os.path.isdir(directory):
return found
for root, _dirs, files in os.walk(directory):
for name in files:
full = os.path.join(root, name)
rel = os.path.relpath(full, directory).replace(os.sep, '/')
found[rel] = {'sha256': digest(full), 'size': os.path.getsize(full)}
return found
def normalize(name):
"""PEP 427 wheel filename form: runs of non-alphanumerics become one _."""
return re.sub(r'[^A-Za-z0-9.]+', '_', name).lower()
def requirement_pins(requirements_path):
"""Every 'name==version' pinned in a lockfile, including marked-out ones.
Markers are deliberately IGNORED. A requirement guarded by
sys_platform == 'win32' is exactly the case that must be present, because the
target is Windows and the wheelhouse is usually assembled somewhere else.
"""
pins = {}
with open(requirements_path) as fh:
for line in fh:
line = line.strip()
if not line or line.startswith('#'):
continue
match = re.match(r'^([A-Za-z0-9._-]+)==([^\s;\\]+)', line)
if match:
pins[normalize(match.group(1))] = match.group(2)
return pins
def check_wheelhouse_covers_requirements(bundle_root):
"""The lock records what IS in the wheelhouse, not what the app NEEDS.
Without this, an incomplete wheelhouse gets locked and blessed, and the
install fails on an air-gapped server. That is not hypothetical: assembling
the wheelhouse on Linux silently omits colorama, a win32-only dependency of
click, because pip evaluates environment markers against the machine doing
the downloading rather than the machine being targeted.
"""
wheels = os.path.join(bundle_root, 'wheels')
requirements = os.path.join(bundle_root, 'app', 'requirements.txt')
if not os.path.isdir(wheels) or not os.path.exists(requirements):
return []
have = os.listdir(wheels)
problems = []
for name, version in sorted(requirement_pins(requirements).items()):
prefix = '%s-%s-' % (name, version)
if not any(f.lower().startswith(prefix) for f in have):
problems.append(
'wheels/ has no wheel for %s==%s, which requirements.txt pins '
'(a marked-out dependency still installs on Windows)' % (name, version))
return problems
def verify(bundle_root, lock):
problems = []
locked = lock.get('payloads')
if not locked:
return ['bundle-lock.json has no "payloads" section']
for name, required, what in PAYLOADS:
directory = os.path.join(bundle_root, name)
present = os.path.isdir(directory)
if name not in locked:
if present:
problems.append(
'%s/ is present but is not in bundle-lock.json - regenerate the lock' % name)
elif required:
problems.append(
'%s/ is required but is in neither the bundle nor the lock' % name)
continue
if not present:
if required or locked[name].get('required'):
problems.append('%s/ is in the lock but missing from the bundle (%s)' % (name, what))
continue
expected = locked[name].get('files', {})
actual = payload_files(directory)
for rel, want in sorted(expected.items()):
got = actual.get(rel)
if got is None:
problems.append('%s/%s is in the lock but missing from the bundle' % (name, rel))
elif got['sha256'] != want['sha256']:
problems.append(
'%s/%s does NOT match the lock (expected sha256 %s..., got %s...)'
% (name, rel, want['sha256'][:12], got['sha256'][:12]))
elif int(got['size']) != int(want['size']):
# Impossible for a matching sha256, so the lock was hand-edited.
problems.append(
'%s/%s size disagrees with the lock - the lock has been edited by hand'
% (name, rel))
for rel in sorted(actual):
if rel not in expected:
problems.append(
'%s/%s is in the bundle but NOT in the lock (unexpected extra file)'
% (name, rel))
problems.extend(check_wheelhouse_covers_requirements(bundle_root))
return problems
def main():
if len(sys.argv) != 3:
sys.exit('usage: verify_bundle_lock.py <bundle-root> <bundle-lock.json>')
bundle_root, lock_path = sys.argv[1], sys.argv[2]
if not os.path.exists(lock_path):
print('no bundle-lock.json at %s' % lock_path)
return 1
with open(lock_path) as fh:
lock = json.load(fh)
problems = verify(bundle_root, lock)
for problem in problems:
print(problem)
return 1 if problems else 0
if __name__ == '__main__':
sys.exit(main())

BIN
deploy/windows/installer/wizard-image.bmp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 151 KiB

BIN
deploy/windows/installer/wizard-image@125.bmp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 217 KiB

BIN
deploy/windows/installer/wizard-image@250.bmp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 869 KiB

BIN
deploy/windows/installer/wizard-small.bmp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 9.1 KiB

BIN
deploy/windows/installer/wizard-small@125.bmp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 12 KiB

BIN
deploy/windows/installer/wizard-small@250.bmp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 56 KiB

422
deploy/windows/shopdb-diagnose.py Normal file
View File

@@ -0,0 +1,422 @@
"""Collect everything needed to diagnose a ShopDB-Flask stage 5 failure.
Stage 5 is the smoke test: the installer asks IIS for the site and expects 200.
When it does not get one, the cause is always in one of four places, and this
script reads all four in one pass so the answer arrives in a single round trip:
1. What IIS actually answers, and with which status code and error page.
2. Whether the config sections httpPlatformHandler needs are unlocked.
3. Whether the app-pool identity can read the app and run its venv.
4. Whether the app itself imports and starts.
Run it ON THE SERVER, as Administrator:
C:\\Python314\\python.exe shopdb-diagnose.py
It writes shopdb-diagnose-<timestamp>.txt next to itself and prints the path.
Send that file back.
SECRETS: the report never contains them. Values from .env (database password,
SECRET_KEY, JWT_SECRET_KEY) are read first, then scrubbed out of every section
of the report before it is written, including command output and tracebacks
that might quote them.
Standard library only, so it runs on the bundled runtime or any system Python.
"""
import os
import re
import socket
import subprocess
import sys
import time
from datetime import datetime
APP_ROOT = os.environ.get('SHOPDB_APPROOT', r'C:\shopdb-flask')
ALIAS = 'shopdb'
TIMEOUT = 25
# Filled from .env, then scrubbed from the whole report.
SECRETS = []
WINDIR = os.environ.get('WINDIR', r'C:\Windows')
# Sysnative gives a 32-bit process the real 64-bit System32. Harmless on 64-bit.
APPCMD_CANDIDATES = [
os.path.join(WINDIR, 'Sysnative', 'inetsrv', 'appcmd.exe'),
os.path.join(WINDIR, 'System32', 'inetsrv', 'appcmd.exe'),
]
def find_appcmd():
for path in APPCMD_CANDIDATES:
if os.path.isfile(path):
return path
return None
class Report(object):
def __init__(self):
self.chunks = []
def head(self, title):
self.chunks.append('\n' + '=' * 72 + '\n' + title + '\n' + '=' * 72)
def line(self, text=''):
self.chunks.append(str(text))
def block(self, title, body):
self.chunks.append('\n--- %s ---' % title)
if body is None or str(body).strip() == '':
self.chunks.append('(no output)')
else:
self.chunks.append(str(body).rstrip())
def text(self):
return '\n'.join(self.chunks) + '\n'
def run(cmd, timeout=60):
"""Run a command, return combined output. Never raises."""
try:
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, shell=False)
out, _ = proc.communicate(timeout=timeout)
text = out.decode('utf-8', 'replace') if out else ''
return '[exit %s]\n%s' % (proc.returncode, text)
except subprocess.TimeoutExpired:
try:
proc.kill()
except Exception:
pass
return '[TIMED OUT after %ss]' % timeout
except Exception as exc:
return '[could not run: %s]' % exc
def powershell(script, timeout=90):
exe = os.path.join(WINDIR, 'Sysnative', 'WindowsPowerShell', 'v1.0', 'powershell.exe')
if not os.path.isfile(exe):
exe = 'powershell.exe'
return run([exe, '-NoProfile', '-ExecutionPolicy', 'Bypass', '-Command', script],
timeout=timeout)
def load_secrets():
"""Read .env so its secret VALUES can be scrubbed from the report."""
env_path = os.path.join(APP_ROOT, '.env')
found = {}
if not os.path.isfile(env_path):
return found, None
try:
with open(env_path, 'r', encoding='utf-8', errors='replace') as handle:
raw = handle.read()
except Exception as exc:
return found, '[could not read .env: %s]' % exc
for line in raw.splitlines():
line = line.strip()
if not line or line.startswith('#') or '=' not in line:
continue
key, value = line.split('=', 1)
key, value = key.strip(), value.strip().strip('"').strip("'")
found[key] = value
if not value:
continue
upper = key.upper()
if 'SECRET' in upper or 'PASSWORD' in upper or 'TOKEN' in upper or 'KEY' in upper:
SECRETS.append(value)
if upper == 'DATABASE_URL':
# mysql+pymysql://user:PASSWORD@host/db -- the password only.
match = re.match(r'^[^:]+://([^:@/]+):([^@]+)@', value)
if match:
SECRETS.append(match.group(2))
return found, raw
def scrub(text):
"""Remove every known secret value from the report."""
for secret in SECRETS:
if secret and len(secret) >= 4:
text = text.replace(secret, '<REDACTED>')
# Catch a password inside any URL that did not come from .env.
text = re.sub(r'(://[^:@/\s]+:)[^@\s]+(@)', r'\1<REDACTED>\2', text)
return text
def http_probe(url):
"""Fetch a URL, returning status, headers and body even for an error page."""
import urllib.error
import urllib.request
started = time.time()
try:
request = urllib.request.Request(url, headers={'User-Agent': 'shopdb-diagnose'})
with urllib.request.urlopen(request, timeout=TIMEOUT) as response:
body = response.read(4000).decode('utf-8', 'replace')
return {'status': response.status, 'reason': response.reason,
'headers': dict(response.headers), 'body': body,
'seconds': time.time() - started}
except urllib.error.HTTPError as exc:
body = ''
try:
body = exc.read(4000).decode('utf-8', 'replace')
except Exception:
pass
return {'status': exc.code, 'reason': exc.reason,
'headers': dict(exc.headers or {}), 'body': body,
'seconds': time.time() - started}
except Exception as exc:
return {'status': None, 'reason': '%s: %s' % (type(exc).__name__, exc),
'headers': {}, 'body': '', 'seconds': time.time() - started}
def summarise_iis_error(body):
"""Pull the meaningful bits out of an IIS error page."""
if not body:
return None
import html
# style and script blocks first: their contents survive plain tag stripping
# and drag CSS into the summary.
flat = re.sub(r'(?is)<(script|style)[^>]*>.*?</\1>', ' ', body)
flat = re.sub(r'(?s)<!--.*?-->', ' ', flat)
flat = re.sub(r'(?s)<[^>]*>', ' ', flat)
# Any unterminated tag left by the 4000-byte body truncation.
flat = re.sub(r'(?s)<[^>]*$', ' ', flat)
flat = html.unescape(flat)
flat = re.sub(r'\s+', ' ', flat).strip()
hints = []
for pattern in (r'\b\d{3}\.\d+\b', r'0x[0-9a-fA-F]{8}',
r'Error Code[^.]{0,80}', r'Config (?:Error|File)[^.]{0,120}',
r'Requested URL[^.]{0,120}', r'Physical Path[^.]{0,120}'):
for match in re.findall(pattern, flat):
cleaned = re.sub(r'\s+', ' ', match).strip(' :-')
if cleaned and cleaned not in hints:
hints.append(cleaned)
return {'flat': flat[:1200], 'hints': hints}
def main():
report = Report()
stamp = datetime.now().strftime('%Y%m%d-%H%M%S')
env_values, env_raw = load_secrets()
report.line('ShopDB-Flask stage 5 diagnostic')
report.line('generated %s' % datetime.now().strftime('%Y-%m-%d %H:%M:%S'))
report.line('host %s' % socket.gethostname())
report.line('app root %s' % APP_ROOT)
report.line('python %s' % sys.version.replace('\n', ' '))
report.line('process is %d-bit' % (64 if sys.maxsize > 2 ** 32 else 32))
# ---------------------------------------------------------------- 1. HTTP
report.head('1. WHAT IIS ANSWERS')
report.line('This is the single most important section. The status code names')
report.line('the fault: 500.19 = config locked, 502.3/503 = the app did not')
report.line('start, 404 = application or handler mapping missing.')
hostname = socket.gethostname()
targets = [
'http://localhost/%s/' % ALIAS,
'http://127.0.0.1/%s/' % ALIAS,
'http://[::1]/%s/' % ALIAS,
'http://%s/%s/' % (hostname, ALIAS),
'http://localhost/',
]
for url in targets:
result = http_probe(url)
report.line('\n%s' % url)
report.line(' status : %s %s' % (result['status'], result['reason']))
report.line(' time : %.1fs' % result['seconds'])
server = result['headers'].get('Server')
if server:
report.line(' server : %s' % server)
summary = summarise_iis_error(result['body'])
if summary and summary['hints']:
report.line(' hints : %s' % ' | '.join(summary['hints'][:8]))
if summary and summary['flat']:
report.line(' body : %s' % summary['flat'][:600])
# localhost resolving to ::1 first has bitten this install before.
report.block('name resolution for localhost', run(
['nslookup', 'localhost'], timeout=20))
# ------------------------------------------------------------- 2. IIS state
report.head('2. IIS STATE')
appcmd = find_appcmd()
if not appcmd:
report.line('appcmd.exe NOT FOUND - is the IIS role installed?')
else:
report.line('appcmd: %s' % appcmd)
report.block('sites', run([appcmd, 'list', 'sites']))
report.block('applications', run([appcmd, 'list', 'apps']))
report.block('app pools', run([appcmd, 'list', 'apppools']))
report.block('worker processes (empty means nothing is running)',
run([appcmd, 'list', 'wp']))
report.block('modules: httpPlatformHandler present?',
run([appcmd, 'list', 'modules']))
# overrideMode tells us whether the unlock actually took effect.
#
# allowedServerVariables is in this list because it caused a 500.52 that
# the first two sections could not explain: it is Deny by default, so an
# <allowedServerVariables> block in the app's web.config is rejected
# before httpPlatformHandler runs. Checking only the sections we unlock
# would have missed the one we do not.
for section in ('system.webServer/handlers',
'system.webServer/httpPlatform',
'system.webServer/rewrite/allowedServerVariables',
'system.webServer/rewrite/rules'):
report.block('lock state of %s' % section,
run([appcmd, 'list', 'config', '/section:%s' % section,
'/text:*']))
report.block('W3SVC / WAS services', powershell(
"Get-Service W3SVC,WAS | Format-Table Name,Status,StartType -AutoSize | Out-String"))
report.block('listeners on port 80', powershell(
"Get-NetTCPConnection -LocalPort 80 -State Listen -EA SilentlyContinue | "
"Format-Table LocalAddress,LocalPort,OwningProcess -AutoSize | Out-String"))
report.block('app pool detail', powershell(
"Import-Module WebAdministration -EA SilentlyContinue; "
"Get-Item IIS:\\AppPools\\shopdbflask -EA SilentlyContinue | "
"Select-Object name,state,managedRuntimeVersion,enable32BitAppOnWin64,"
"@{n='identity';e={$_.processModel.identityType}} | Format-List | Out-String"))
# --------------------------------------------------------- 3. app + config
report.head('3. APPLICATION AND CONFIG')
web_config = os.path.join(APP_ROOT, 'web.config')
if os.path.isfile(web_config):
try:
with open(web_config, 'r', encoding='utf-8', errors='replace') as handle:
report.block('web.config', handle.read())
except Exception as exc:
report.block('web.config', '[could not read: %s]' % exc)
else:
report.block('web.config', 'MISSING at %s' % web_config)
if env_raw is None:
report.block('.env', 'MISSING at %s' % os.path.join(APP_ROOT, '.env'))
else:
# Keys and non-secret values only. Secret values are scrubbed anyway.
lines = []
for key in sorted(env_values):
upper = key.upper()
secretish = ('SECRET' in upper or 'PASSWORD' in upper
or 'TOKEN' in upper or 'KEY' in upper
or upper == 'DATABASE_URL')
if secretish:
lines.append('%s = <set, %d chars>' % (key, len(env_values[key])))
else:
lines.append('%s = %s' % (key, env_values[key]))
report.block('.env (secret values withheld)', '\n'.join(lines))
venv_python = os.path.join(APP_ROOT, 'venv', 'Scripts', 'python.exe')
report.line('\nvenv python exists: %s' % os.path.isfile(venv_python))
if os.path.isfile(venv_python):
# The exact failure stage 3 used to hit. Proves the app imports.
report.block('venv: import shopdb', run(
[venv_python, '-c',
'import shopdb; print("import OK"); '
'app = shopdb.create_app(); print("create_app OK")'], timeout=120))
report.block('venv: waitress present', run(
[venv_python, '-c', 'import waitress; print(waitress.__version__)'],
timeout=60))
# What httpPlatformHandler is told to launch, and whether it exists.
if os.path.isfile(web_config):
try:
with open(web_config, 'r', encoding='utf-8', errors='replace') as handle:
raw = handle.read()
match = re.search(r'processPath\s*=\s*"([^"]+)"', raw)
args = re.search(r'arguments\s*=\s*"([^"]*)"', raw)
if match:
path = os.path.expandvars(match.group(1))
report.line('\nhttpPlatform processPath : %s' % match.group(1))
report.line(' resolved : %s' % path)
report.line(' exists : %s' % os.path.isfile(path))
if args:
report.line('httpPlatform arguments : %s' % args.group(1))
except Exception as exc:
report.line('[could not parse web.config: %s]' % exc)
# ------------------------------------------------------------- 4. app logs
report.head('4. APPLICATION LOGS')
log_dir = os.path.join(APP_ROOT, 'logs')
if not os.path.isdir(log_dir):
report.line('MISSING: %s' % log_dir)
else:
entries = []
for name in sorted(os.listdir(log_dir)):
full = os.path.join(log_dir, name)
try:
entries.append((os.path.getmtime(full), full, name,
os.path.getsize(full)))
except OSError:
pass
if not entries:
report.line('%s is EMPTY.' % log_dir)
report.line('No stdout log at all means httpPlatformHandler never')
report.line('launched python - look at the pool identity and ACLs.')
for _, full, name, size in sorted(entries, reverse=True)[:5]:
if size == 0:
report.block('%s (0 bytes)' % name,
'EMPTY - python was launched but wrote nothing.')
continue
try:
with open(full, 'r', encoding='utf-8', errors='replace') as handle:
tail = handle.readlines()[-60:]
report.block('%s (%d bytes, last 60 lines)' % (name, size),
''.join(tail))
except Exception as exc:
report.block(name, '[could not read: %s]' % exc)
# ---------------------------------------------------------------- 5. ACLs
report.head('5. PERMISSIONS')
report.line('The pool runs as "IIS AppPool\\shopdbflask". It needs RX on the')
report.line('tree, Modify on logs and instance, and Read on .env.')
for target in (APP_ROOT, log_dir, os.path.join(APP_ROOT, '.env'),
os.path.join(APP_ROOT, 'venv', 'Scripts')):
if os.path.exists(target):
report.block('icacls %s' % target, run(['icacls', target], timeout=40))
# ------------------------------------------------------------ 6. event log
report.head('6. EVENT LOG')
report.block('recent application errors', powershell(
"Get-WinEvent -FilterHashtable @{LogName='Application';"
"StartTime=(Get-Date).AddHours(-6)} -EA SilentlyContinue | "
"Where-Object { $_.ProviderName -match 'HttpPlatform|IIS|W3SVC|WAS|\\.NET' "
"-or $_.LevelDisplayName -eq 'Error' } | Select-Object -First 25 "
"TimeCreated,ProviderName,LevelDisplayName,Message | Format-List | Out-String",
timeout=180))
report.block('system log: WAS / W3SVC', powershell(
"Get-WinEvent -FilterHashtable @{LogName='System';"
"StartTime=(Get-Date).AddHours(-6)} -EA SilentlyContinue | "
"Where-Object { $_.ProviderName -match 'WAS|W3SVC|HTTP' } | "
"Select-Object -First 20 TimeCreated,ProviderName,LevelDisplayName,Message | "
"Format-List | Out-String", timeout=180))
# ------------------------------------------------------------------ write
body = scrub(report.text())
out_path = os.path.join(os.path.dirname(os.path.abspath(__file__)),
'shopdb-diagnose-%s.txt' % stamp)
try:
with open(out_path, 'w', encoding='utf-8') as handle:
handle.write(body)
except Exception:
out_path = os.path.join(os.environ.get('TEMP', r'C:\Windows\Temp'),
'shopdb-diagnose-%s.txt' % stamp)
with open(out_path, 'w', encoding='utf-8') as handle:
handle.write(body)
print('')
print('Report written to:')
print(' %s' % out_path)
print('')
print('%d secret value(s) were scrubbed from it.' % len(SECRETS))
print('Send that file back.')
return 0
if __name__ == '__main__':
sys.exit(main())

89
deploy/windows/web.config
View File

@@ -11,7 +11,7 @@
- HttpPlatformHandler IIS module installed
(https://www.iis.net/downloads/microsoft/httpplatformhandler)
- URL Rewrite module installed (only for the optional X-Forwarded-For rule)
- Python 3.12 + a venv at APP_ROOT\venv with requirements.txt + waitress
- Python 3.14 + a venv at APP_ROOT\venv with requirements.txt + waitress
- Secrets live in APP_ROOT\.env (wsgi.py load_dotenv() reads it). Keep them
OUT of this file. Lock .env ACLs to the app pool identity + admins.
@@ -28,7 +28,7 @@
<httpPlatform
processPath="C:\shopdb-flask\venv\Scripts\waitress-serve.exe"
arguments="--port=%HTTP_PLATFORM_PORT% --host=127.0.0.1 --threads=8 --trusted-proxy=127.0.0.1 --trusted-proxy-headers=x-forwarded-for wsgi:app"
arguments="--port=%HTTP_PLATFORM_PORT% --host=127.0.0.1 --threads=8 --trusted-proxy=127.0.0.1 --trusted-proxy-headers=x-forwarded-for --trusted-proxy-count=1 wsgi:app"
stdoutLogEnabled="true"
stdoutLogFile="C:\shopdb-flask\logs\httpplatform"
startupTimeLimit="120"
@@ -49,20 +49,46 @@
</httpPlatform>
<!--
OPTIONAL: forward the real client IP so audit logs and the kiosk
visitor-location feature (IP -> business unit) see the caller, not the
loopback that HttpPlatformHandler connects from.
Forward the real client IP, so the audit log, the kiosk visitor-location
feature (IP -> business unit), the GE-Enforce IP allowlist and per-host
login rate limiting all see the caller rather than the loopback address
HttpPlatformHandler connects from.
This block is COMMENTED OUT by default because it needs the URL Rewrite
module; with it uncommented but URL Rewrite not installed, IIS returns
HTTP 500.19 ("configuration section not well-formed / cannot be read").
Install URL Rewrite (https://www.iis.net/downloads/microsoft/url-rewrite)
and then uncomment the <rewrite> block below to enable it.
IIS does not set X-Forwarded-For on its own. Without the rule below there
is no such header at all, and every client looks like 127.0.0.1 - so the
allowlist and the visitor-location lookup silently stop working.
ONLY CORRECT WHEN IIS IS DIRECTLY EXPOSED. It overwrites the header with
REMOTE_ADDR, which is what stops a client spoofing its own X-Forwarded-For.
Behind a reverse proxy (ARR, a load balancer) REMOTE_ADDR is the PROXY, so
this rule would destroy the real client IP - there, leave it disabled and
let the proxy set the header.
It ships DISABLED because it needs the URL Rewrite module; enabled without
it, IIS returns HTTP 500.19 ("configuration section not well-formed").
There is deliberately NO <allowedServerVariables> block below. Setting a
server variable requires that variable to be allowed, but the section
system.webServer/rewrite/allowedServerVariables ships with
overrideModeDefault="Deny", so declaring it in an application's own
web.config is refused outright: IIS answered 500.52 with error 0x80070021,
"this configuration section cannot be used at this path", BEFORE it ever
reached httpPlatformHandler - so python was never launched and the stdout
log stayed empty, which looks like an application fault and is not one.
The installer instead allows the single variable at server level, which
grants exactly HTTP_X_FORWARDED_FOR rather than unlocking the section and
letting every site on the machine declare arbitrary server variables.
The installer handles both: -ClientIpSource direct installs URL Rewrite
from the bundle, allows the variable, and enables this; -ClientIpSource
proxy leaves it alone. By hand: install URL Rewrite, run
appcmd set config /section:system.webServer/rewrite/allowedServerVariables ^
/+"[name='HTTP_X_FORWARDED_FOR']" /commit:apphost
then delete the two marker lines below.
-->
<!-- SHOPDB-CLIENTIP-BEGIN
<rewrite>
<allowedServerVariables>
<add name="HTTP_X_FORWARDED_FOR" />
</allowedServerVariables>
<rules>
<rule name="Set X-Forwarded-For" stopProcessing="false">
<match url=".*" />
@@ -73,7 +99,42 @@
</rule>
</rules>
</rewrite>
-->
SHOPDB-CLIENTIP-END -->
</system.webServer>
<!--
Installer downloads: serve /installers/* as IIS static files instead of
forwarding them to Flask. The handler above is path="*", so without this a
request for /installers/Foo.exe goes to waitress, which has no such route
(SPA fallback), and large binaries would stream through a Python thread.
This <location> clears the httpPlatformHandler for that one subpath and puts
the static file handler back, so IIS serves the bytes directly (kernel-mode,
range/resume, no Python thread held).
Requires a physical folder at APP_ROOT\installers (the site's physical path
is APP_ROOT). Drop the installer binaries there, e.g. robocopy them from the
classic wwwroot\installers. The stored installpath 'installers/Foo.exe' then
resolves to <mount>/installers/Foo.exe (e.g. /shopdb/installers/Foo.exe).
.exe/.msi are given an explicit MIME map; if the parent site has a Request
Filtering rule that denies executable extensions, also allow them there.
-->
<location path="installers">
<system.webServer>
<handlers>
<clear />
<add name="StaticFile" path="*" verb="*"
modules="StaticFileModule" resourceType="File"
requireAccess="Read" />
</handlers>
<staticContent>
<remove fileExtension=".exe" />
<mimeMap fileExtension=".exe" mimeType="application/octet-stream" />
<remove fileExtension=".msi" />
<mimeMap fileExtension=".msi" mimeType="application/octet-stream" />
</staticContent>
</system.webServer>
</location>
</configuration>

90
docker-compose.airgap.yml Normal file
View File

@@ -0,0 +1,90 @@
# shopdb-flask AIR-GAPPED single-site stack.
#
# For a site with NO internet. Nothing is built or pulled here: the images are
# built on a connected box (scripts/build-offline-bundle.ps1), shipped as a
# tarball, and `docker load`ed at the site. This file only RUNS pre-loaded
# images. See docs/DEPLOY-AIRGAP.md for the full runbook.
#
# Differences from docker-compose.yml (the connected/build template):
# - api uses `image:` (a loaded image), never `build: .` (build needs the net).
# - NO ./plugins bind mount. The image already carries every plugin baked in;
# binding a host ./plugins (which does not exist at an image-only site) would
# mask the baked plugins with an empty dir and load ZERO plugins.
# - a one-shot `migrate` service runs db upgrade + plugin upgrade-all + seed
# BEFORE api starts, so `up -d` alone brings up a working site (no manual
# `docker compose exec ... flask db upgrade` to forget).
#
# Usage at the site:
# docker load < shopdb-stack-<version>.tar.gz
# cp .env.example .env # then edit: secrets, CORS_ORIGINS, IMAGE_TAG
# docker compose -f docker-compose.airgap.yml up -d
# docker compose -f docker-compose.airgap.yml exec api flask seed admin <user> <email> <password>
# Shared application environment, reused by the migrate one-shot and the api
# service so the two never drift. A YAML anchor, not a container.
x-app-env: &app-env
FLASK_APP: wsgi.py
FLASK_ENV: production
DATABASE_URL: mysql+pymysql://shopdb:${MYSQL_PASSWORD}@db:3306/shopdb_flask?charset=utf8mb4
SECRET_KEY: ${SECRET_KEY:?SECRET_KEY must be set}
JWT_SECRET_KEY: ${JWT_SECRET_KEY:?JWT_SECRET_KEY must be set}
CORS_ORIGINS: ${CORS_ORIGINS:?CORS_ORIGINS must be set}
LOG_LEVEL: ${LOG_LEVEL:-INFO}
ZABBIX_URL: ${ZABBIX_URL:-}
ZABBIX_TOKEN: ${ZABBIX_TOKEN:-}
services:
db:
image: mysql:8.0
command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
restart: unless-stopped
environment:
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:?MYSQL_ROOT_PASSWORD must be set}
MYSQL_DATABASE: shopdb_flask
MYSQL_USER: shopdb
MYSQL_PASSWORD: ${MYSQL_PASSWORD:?MYSQL_PASSWORD must be set}
volumes:
- db_data:/var/lib/mysql
ports:
- "127.0.0.1:${MYSQL_PORT:-3306}:3306"
healthcheck:
test: ["CMD", "mysqladmin", "ping", "-h", "localhost", "-u", "root", "-p${MYSQL_ROOT_PASSWORD}"]
interval: 10s
timeout: 5s
retries: 5
# One-shot schema + seed. Runs to completion and exits; api waits for it.
# Every step is idempotent, so it is safe to run on every `up`.
migrate:
image: shopdb-flask:${IMAGE_TAG:-0.7.0}
restart: "no"
depends_on:
db:
condition: service_healthy
environment:
<<: *app-env
command:
- sh
- -c
- >
flask db upgrade &&
flask plugin upgrade-all &&
flask seed permissions &&
flask seed settings &&
flask seed reference-data
api:
image: shopdb-flask:${IMAGE_TAG:-0.7.0}
restart: unless-stopped
depends_on:
db:
condition: service_healthy
migrate:
condition: service_completed_successfully
environment:
<<: *app-env
ports:
- "${API_PORT:-5001}:5001"
volumes:
db_data:

445
docs/ADOPTING-AT-ANOTHER-SITE.md Normal file
View File

@@ -0,0 +1,445 @@
# Adopting the fleet tools at another site
Two tools run on shop-floor PCs and talk to a ShopDB instance: the **asset
reporter**, which posts what a PC is to the collector API, and **EventSaver**,
a screensaver that shows a slideshow served by ShopDB. Both are designed to be
repointed at another site's server. This page is how.
Neither tool is site-specific by design. What is site-specific is the URL, the
API key, and which PCs it runs on - and all three are inputs, not code.
> **Prerequisite.** The reporter's `-ApiUrl` parameter still carries a default
> pointing at the reference site's server. Adopt it with an explicit `-ApiUrl`
> (every example below does) until that default is removed. See
> [ADR-015](adr/ADR-015-site-specific-configuration.md).
## Where these files come from
The commands below copy files. This is where each one is obtained, because they
do not all ship from the same place and two of them are not in this repository
at all yet.
| Artifact | Where it is today |
|---|---|
| GE-Enforce client (`Install-GEEnforce.ps1`, `Invoke-ShopdbEnforce.ps1`, `ShopdbEnforceClient.psm1`) | This repository, `plugins/geenforce/client/`. Present on any installed server under the install directory. |
| `Report-AssetToShopDB.ps1` | **Not in this repository.** It lives on the reference site's imaging share and is provided on request. It is planned to move to `plugins/computers/client/` so it versions with the collector contract it implements. |
| EventSaver (`EventSaver.scr`, `EventSaver.ini`, `EventSaver.cs`) | **Not in this repository.** Provided on request; the source is a single C# file that builds with the in-box .NET Framework compiler, so a site can rebuild it rather than trust a binary. See [EVENTSAVER.md](EVENTSAVER.md). |
Ask the maintainers for the two that are not here. A site that would rather not
run a binary it cannot rebuild should take EventSaver's source and compile it
locally - the build needs no SDK and is one command.
---
## Part 1: the asset reporter
### What it does
Runs as SYSTEM, collects hostname, BIOS serial, PC type, logged-in user, IP
addresses and the machine number where one is configured, and POSTs them to:
```
POST https://<your-shopdb>/api/collector/computers
X-API-Key: <collector token>
Content-Type: application/json
```
The payload contract is [COLLECTOR-INTEGRATION.md](COLLECTOR-INTEGRATION.md) and
[ADR-006](adr/ADR-006-collector-contract.md). A field your site does not collect
is simply absent; the server upserts on `hostname` and leaves the rest alone.
### What you must provide
| Input | Where it comes from |
|---|---|
| Server URL | `-ApiUrl https://<your-shopdb>/api/collector/computers` |
| API key | `-ApiKey`, or `HKLM:\SOFTWARE\GE\ShopDB` value `CollectorKey` |
### Minting the key on your ShopDB
Create a **managed personal access token scoped to `collector.ingest` and
nothing else**, and use it as the `X-API-Key` value. That scope authorises the
collector ingest API and no normal route, so a token recovered off a shop-floor
PC cannot read your asset register. A shared `COLLECTOR_API_KEY` environment
variable also works and is simpler for a pilot, but it cannot be rotated per
fleet or revoked individually.
### Deploying it
The three paths below are alternatives - pick one. All of them end with the same
two facts on the PC: the script exists somewhere it can run from, and the key is
in the registry.
#### Option A: Microsoft Intune
**A1. The key, as a platform script.** Devices > Scripts and remediations >
Platform scripts > Add > Windows 10 and later. Run as SYSTEM, do not run in the
64-bit context only.
```powershell
# Set-ShopdbCollectorKey.ps1 (Intune platform script, runs once per device)
$RegPath = 'HKLM:\SOFTWARE\GE\ShopDB'
$Key = 'shopdb_pat_REPLACE_WITH_YOUR_TOKEN'
if (-not (Test-Path $RegPath)) { New-Item -Path $RegPath -Force | Out-Null }
$current = (Get-ItemProperty -Path $RegPath -Name CollectorKey -ErrorAction SilentlyContinue).CollectorKey
if ($current -ne $Key) { Set-ItemProperty -Path $RegPath -Name CollectorKey -Value $Key }
exit 0
```
Putting a token in an Intune script body means anyone who can read the Intune
configuration can read the token. That is an argument for the `collector.ingest`
scope, not against Intune: the blast radius of that token is one API.
**A2. The reporter, as a remediation on a schedule.** Devices > Scripts and
remediations > Remediations. A remediation is the natural fit because it re-runs
on a schedule, which is exactly what reporting is.
*Detection script* - forces the remediation to run every cycle:
```powershell
# Always "not compliant": reporting is not a state to converge on, it is an
# event that should happen on every schedule tick.
exit 1
```
*Remediation script* - the reporter itself, with your URL:
```powershell
& "$PSScriptRoot\Report-AssetToShopDB.ps1" `
-ApiUrl 'https://shopdb.example.net/api/collector/computers'
exit 0
```
Assign it with a daily schedule. Run as SYSTEM, 64-bit.
**A3. Intune with DSC (Machine Configuration).** If your estate is already
governed by Azure Machine Configuration, express the *key* as configuration and
leave the *reporting* to a scheduled task, because a report is an event and DSC
converges state:
```powershell
Configuration ShopdbCollector
{
Import-DscResource -ModuleName PSDesiredStateConfiguration
Node localhost
{
Registry CollectorKey
{
Key = 'HKEY_LOCAL_MACHINE\SOFTWARE\GE\ShopDB'
ValueName = 'CollectorKey'
ValueData = 'shopdb_pat_REPLACE_WITH_YOUR_TOKEN'
ValueType = 'String'
Ensure = 'Present'
}
Registry CollectorUrl
{
Key = 'HKEY_LOCAL_MACHINE\SOFTWARE\GE\ShopDB'
ValueName = 'BaseUrl'
ValueData = 'https://shopdb.example.net'
ValueType = 'String'
Ensure = 'Present'
}
Script ReportingTask
{
GetScript = { @{ Result = (Get-ScheduledTask -TaskName 'ShopDB asset report' -ErrorAction SilentlyContinue) } }
TestScript = { [bool](Get-ScheduledTask -TaskName 'ShopDB asset report' -ErrorAction SilentlyContinue) }
SetScript = {
$action = New-ScheduledTaskAction -Execute 'powershell.exe' `
-Argument '-NoProfile -ExecutionPolicy Bypass -File "C:\ProgramData\ShopDB\Report-AssetToShopDB.ps1" -ApiUrl "https://shopdb.example.net/api/collector/computers"'
$trigger = New-ScheduledTaskTrigger -Daily -At 7am
Register-ScheduledTask -TaskName 'ShopDB asset report' -Action $action `
-Trigger $trigger -User 'SYSTEM' -RunLevel Highest -Force
}
}
}
}
```
#### Option B: GE-Enforce
If you already run the GE-Enforce client, this is one manifest entry, and it is
how the reference site does it. `Type: PS1` with `DetectionMethod: Always` runs
the script from the share every cycle with no local copy and no hash to bump
when the script changes.
```json
{
"_comment": "Reports host, BIOS serial, pc-type, logged-in user, IPs and machine number to ShopDB every cycle, as SYSTEM, straight off the share.",
"Name": "Report asset to ShopDB",
"Type": "PS1",
"Script": "apps/Report-AssetToShopDB.ps1",
"Args": "-ApiUrl https://shopdb.example.net/api/collector/computers",
"DetectionMethod": "Always"
}
```
Provision the key with a second entry, ordered **before** it, so the key is
present the same cycle the first report fires:
```json
{
"_comment": "Writes the collector key from an ACL'd share file. The token is NOT in this manifest - manifests sync broadly. Rotate by replacing configs/collector-key.txt on the share; every PC picks it up next cycle.",
"Name": "Provision the ShopDB collector key",
"Type": "PS1",
"Script": "scripts/Set-CollectorKey.ps1",
"DetectionMethod": "Always"
}
```
Keep the token in `configs/collector-key.txt` on your share, locked to SYSTEM
and administrators - not in the manifest JSON.
#### Option C: manual
For a pilot, a single bay, or an estate with no management plane:
```powershell
# As administrator, once per PC.
New-Item -Path 'C:\ProgramData\ShopDB' -ItemType Directory -Force | Out-Null
Copy-Item .\Report-AssetToShopDB.ps1 'C:\ProgramData\ShopDB\'
New-Item -Path 'HKLM:\SOFTWARE\GE\ShopDB' -Force | Out-Null
Set-ItemProperty -Path 'HKLM:\SOFTWARE\GE\ShopDB' -Name CollectorKey `
-Value 'shopdb_pat_REPLACE_WITH_YOUR_TOKEN'
$action = New-ScheduledTaskAction -Execute 'powershell.exe' -Argument @'
-NoProfile -ExecutionPolicy Bypass -File "C:\ProgramData\ShopDB\Report-AssetToShopDB.ps1" -ApiUrl "https://shopdb.example.net/api/collector/computers"
'@
$trigger = New-ScheduledTaskTrigger -Daily -At 7am
Register-ScheduledTask -TaskName 'ShopDB asset report' -Action $action -Trigger $trigger `
-User 'SYSTEM' -RunLevel Highest -Force
Start-ScheduledTask -TaskName 'ShopDB asset report'
```
Lock down `C:\ProgramData\ShopDB` if you put it there: a script a user can edit,
run by SYSTEM on a schedule, is a local privilege escalation. Break inheritance
and grant Administrators and SYSTEM only.
### Verifying
```powershell
Invoke-RestMethod -Uri 'https://shopdb.example.net/api/computers/by-hostname/<HOSTNAME>'
```
The PC should exist with its serial, type and last-reported timestamp. The
script also logs to `C:\Logs\Shopfloor`.
---
## Part 2: EventSaver
### What it does
A Windows screensaver (`EventSaver.scr`) that shows a slideshow. It reads
`EventSaver.ini` from the directory it lives in, **on every launch**, so
retargeting it needs no recompile:
```ini
# HTTP mode (recommended): pull slides from ShopDB, cache locally. No share.
url=https://shopdb.example.net/api/slides/feed?surface=shopfloor
# Folder mode (fallback): used only when url is blank. SMB or local path.
# folder=\\fileserver\shopfloor\tv
interval=10 # seconds per image, when a slide carries no time of its own
shuffle=0 # 1 = random, 0 = ordered
fadems=600 # crossfade length, reserved
```
HTTP mode is the one to adopt: slides are managed in ShopDB (Slides), and the
screensaver caches them locally, so a PC that cannot reach the server keeps
showing the last set instead of going black.
### The two things that catch people out
**The compiled-in fallback is not yours.** If `EventSaver.ini` is missing or
both `url` and `folder` are blank, the binary falls back to a folder path
compiled into `EventSaver.cs`, which is the reference site's file server. Ship
the ini. A missing ini is not a neutral default.
**If you enforce the ini by hash, your own edits revert.** The reference
manifest pins `EventSaver.ini` with `DetectionMethod: Hash`, which is what makes
a retune propagate to the fleet automatically - and equally means a hand edit on
one PC is undone next cycle. That is the feature working. Edit the copy on
**your** share and update `DetectionValue` to its new SHA-256:
```powershell
(Get-FileHash .\configs\EventSaver.ini -Algorithm SHA256).Hash
```
### Deploying it
#### Option A: Intune
Package the three files as a Win32 app (`EventSaver.scr`, `EventSaver.ini`, and
the enable script) with this install command:
```powershell
# install.ps1 for the Win32 app
Copy-Item .\EventSaver.scr 'C:\Windows\System32\EventSaver.scr' -Force
Copy-Item .\EventSaver.ini 'C:\Windows\System32\EventSaver.ini' -Force
exit 0
```
Detection rule: file `C:\Windows\System32\EventSaver.scr` exists **and** matches
the expected SHA-256, so a changed binary reinstalls.
The screensaver itself is per-user, so set it with a configuration profile
(Settings catalog > Screen saver) or a user-context script:
```powershell
Set-ItemProperty 'HKCU:\Control Panel\Desktop' -Name SCRNSAVE.EXE -Value 'C:\Windows\System32\EventSaver.scr'
Set-ItemProperty 'HKCU:\Control Panel\Desktop' -Name ScreenSaveActive -Value '1'
Set-ItemProperty 'HKCU:\Control Panel\Desktop' -Name ScreenSaveTimeOut -Value '480'
```
With Machine Configuration/DSC, express the two files with `File` resources and
the screensaver with a `Registry` resource in the user hive - noting that DSC
runs as SYSTEM, so a per-user setting needs the logged-on user's hive or a
default-user template, which is usually more trouble than a login script.
#### Option B: GE-Enforce
Three entries, which is exactly how the reference fleet does it. Note the
per-PC-type targeting: a kiosk showing a live dashboard must **not** get a
screensaver over the top of it.
```json
{
"Name": "EventSaver screensaver (binary)",
"Type": "File",
"Source": "apps/EventSaver.scr",
"Destination": "C:\\Windows\\System32\\EventSaver.scr",
"DetectionMethod": "Hash",
"DetectionPath": "C:\\Windows\\System32\\EventSaver.scr",
"DetectionValue": "<sha256 of your staged .scr>",
"PCTypes": ["gea-shopfloor-common"]
},
{
"Name": "EventSaver screensaver (config)",
"Type": "File",
"Source": "configs/EventSaver.ini",
"Destination": "C:\\Windows\\System32\\EventSaver.ini",
"DetectionMethod": "Hash",
"DetectionPath": "C:\\Windows\\System32\\EventSaver.ini",
"DetectionValue": "<sha256 of YOUR .ini, pointing at YOUR shopdb>",
"PCTypes": ["gea-shopfloor-common"]
},
{
"Name": "EventSaver enable (per-user screensaver)",
"Type": "PS1",
"Script": "scripts/Set-EventSaverScreensaver.ps1",
"Args": "-TimeoutSeconds 480",
"DetectionMethod": "Always",
"PCTypes": ["gea-shopfloor-common"]
}
```
#### Option C: manual
```powershell
Copy-Item .\EventSaver.scr 'C:\Windows\System32\' -Force
Copy-Item .\EventSaver.ini 'C:\Windows\System32\' -Force # with YOUR url=
# Per user, in the user's own session:
Set-ItemProperty 'HKCU:\Control Panel\Desktop' -Name SCRNSAVE.EXE -Value 'C:\Windows\System32\EventSaver.scr'
Set-ItemProperty 'HKCU:\Control Panel\Desktop' -Name ScreenSaveActive -Value '1'
Set-ItemProperty 'HKCU:\Control Panel\Desktop' -Name ScreenSaveTimeOut -Value '480'
```
Run `EventSaver.scr /c` to see which source it resolved and where it is caching.
### Verifying
Open the feed in a browser from the PC itself:
```
https://shopdb.example.net/api/slides/feed?surface=shopfloor
```
It is a public endpoint - the screensaver has no credentials - and returns the
playlist with per-slide durations. An empty `slides` array means slides have not
been uploaded for that surface, not that the PC is misconfigured.
---
---
## Part 3: bootstrapping GE-Enforce itself
The two parts above assume the tools are already on the PC. GE-Enforce is what
puts them there and keeps them there - so the question is how GE-Enforce gets
onto a bay in the first place. Two paths are supported, and they are the two
that actually occur: imaging time, and a management plane such as Intune.
Whichever you use, the end state is the same three things:
1. The client files in `C:\Program Files\GE\Shopfloor`.
2. `HKLM:\SOFTWARE\GE\ShopDB` carrying `BaseUrl`, and `ApiToken` where the scope
is served over HTTPS.
3. A scheduled task running the enforce cycle as SYSTEM.
### At imaging time (the usual path)
The shop-floor imaging pipeline registers the enforce task as its last step,
once the PC type is known - the type is what decides which manifest scope the
bay enforces. Self-contained types are skipped deliberately: a display kiosk
gets its configuration at imaging time and no share, so registering an enforce
task on one would give it a cycle with nothing to do.
If you are building your own imaging pipeline, the equivalent step is: copy the
engine into a runtime directory, write the PC type where the client can read it,
and register the task. That is what `Install-GEEnforce.ps1` does in one call.
### Through Intune
`Install-GEEnforce.ps1` is parameterised for exactly this, and this is how the
display cohort is deployed today - those PCs are Entra-joined, have no file
share, and fetch their manifest over HTTPS:
```powershell
.\Install-GEEnforce.ps1 `
-PCType 'gea-shopfloor-common' `
-ShopdbUrl 'https://shopdb.example.net' `
-ShopdbToken 'shopdb_pat_REPLACE_WITH_YOUR_TOKEN'
```
Package it as a Win32 app with the client files. Detection rule: the scheduled
task exists **and** `HKLM:\SOFTWARE\GE\ShopDB\BaseUrl` matches your server, so a
PC that was imaged for another site is repaired rather than skipped.
The token is a managed service token scoped to the GE-Enforce fetch scope, not
an admin credential. Scope it that way and a token read off a bay buys the
reader a manifest they could have read anyway.
### With DSC (Machine Configuration)
There is no shipped DSC configuration, but nothing about the end state resists
one - it is two registry values, a set of files and a scheduled task, which is
ordinary DSC territory. Express it the same way as the reporter example above:
`Registry` resources for `BaseUrl` and `ApiToken`, `File` resources for the
client, and a `Script` resource that registers the task.
One caution. DSC converges state on a schedule and GE-Enforce is itself a
convergence loop, so running both against the same PC means two things fighting
to own the same configuration. Use DSC to install and configure the client, and
let GE-Enforce own everything downstream of that. Deciding which tool owns what
is the whole job; splitting it by layer is what keeps it answerable.
## Which path to choose
| You have | Use |
|---|---|
| Intune, no GE-Enforce | Option A. A remediation for the reporter, a Win32 app for EventSaver. |
| GE-Enforce | Option B. One manifest entry each, and the fleet converges on its own cycle. |
| Neither, or a pilot of a few bays | Option C, then move to A or B once it earns its place. |
GE-Enforce and Intune are not exclusive. The reference site uses Intune for
enrolment and imaging and GE-Enforce for shop-floor configuration state, because
Intune is not on the shop floor network path in the same way. If you have both,
the deciding question is which one you would look at first to answer "why is
this bay wrong".

147
docs/API-REFERENCE.md Normal file
View File

@@ -0,0 +1,147 @@
# API reference (index)
This page is an index and a pointer, not a full specification. It answers three
questions: what endpoints exist, who calls them, and what auth they require. The
detailed request and response shapes live in the live generated docs and in the
per-surface contract docs linked from each table below.
## Live and generated docs
The repo ships hosted, generated API docs. Start there:
- **Interactive spec:** `GET /api/docs` - a self-hosted Redoc page over the
generated OpenAPI spec. The Redoc bundle is vendored under
`shopdb/core/api/staticdocs/`, so it renders fully offline on the air-gapped
prod box (no CDN).
- **Raw spec:** `GET /api/docs/openapi.json` - OpenAPI 3.1, roughly 238 paths and
362 operations. Generated by `scripts/gen_openapi.py` from
`docs/api-inventory.json`; regenerate after any API change.
- **LLM / agent entry point:** `GET /api/docs/llms.txt` - a concise API guide
following the llms.txt convention, plus a read-only MCP server
(`mcp/shopdb_mcp.py`, built with `FastMCP.from_openapi` over the same spec).
The MCP server exposes a curated set of GET endpoints as tools for an agent to
query the asset database over HTTPS with a scoped read token; it never runs on
the prod box. Set it up on a work PC with
`<imaging-share>/github/setup-mcp.cmd`.
The docs blueprint is `shopdb/core/api/docs.py` (a core blueprint, always
mounted regardless of which plugins are staged into a site build).
Contract docs (linked per table below) hold the deep semantics: field mappings,
idempotency rules, rotation, error envelopes, staged rollout. This page only
routes you to the right one.
---
## 1. Fleet and client contracts (unauthenticated or token)
These are the endpoints the shopfloor PC fleet, kiosks, displays, and printer
installers call. They are consumed by machines, not by the interactive UI, and
they authenticate with a scoped service token or nothing at all.
| Endpoint | Auth | Purpose | Contract doc |
|---|---|---|---|
| `GET /api/geenforce/manifest?pctype=<scope>` | `geenforce.fetch` service token (`X-API-Key` or Bearer PAT) | Serve the current published manifest for a PC-type scope. ETag / 304 supported. | GE-ENFORCE-CLIENT.md, geenforce-api-cutover.md |
| `GET /api/geenforce/payload/<sha256>` | `geenforce.fetch` service token | Serve a payload blob (installer) by content hash so share-less PCs pull over HTTPS instead of SMB. Rate limited and size capped; the sha256 is the integrity guarantee. | GE-ENFORCE-CLIENT.md, geenforce-api-cutover.md |
| `POST /api/geenforce/report` | `geenforce.report` service token | Record one PC's enforcement cycle: applied manifest version plus per-entry self-heal outcomes. | GE-ENFORCE-CLIENT.md, geenforce-api-cutover.md |
| `POST /api/collector/<plugin>` | `X-API-Key` env key or a `collector.ingest` managed token | Generic idempotent inventory upsert; the PC fleet targets `/api/collector/computers`. | COLLECTOR-INTEGRATION.md |
| `POST /api/collector/pc`, `/apps`, `/heartbeat`, `/bulk` | `X-API-Key` or `collector.ingest` token | Legacy computers-only collector paths (predate ADR-006); deprecated in favor of `/api/collector/computers`. | COLLECTOR-INTEGRATION.md |
| `GET /api/collector/status` | `X-API-Key` or `collector.ingest` token | Collector liveness and endpoint list. | COLLECTOR-INTEGRATION.md |
| `GET /api/printers/install-list` | optional JWT (anonymous fleet or logged-in browser) | Flat list of network printers with floor-map positions for the signed installer. `?format=text` returns a pipe-delimited variant. | PRINTER-INSTALLER.md |
| `GET /api/printers/pc-default?machine=NNNN` | optional JWT | The PC's default printer by machine (asset) number, via the `defaultprinter` relationship. `?format=text` supported. | PRINTER-INSTALLER.md |
| `GET /api/printers/install-batch?printerids=1,2,3` | optional JWT | Generate a self-deleting Windows `.bat` that installs the selected printers. | PRINTER-INSTALLER.md |
| `GET /api/dashboarddefaults/display-role?fqdn=<fqdn>` | public (none) | Resolve what a display PC should show (role `dashboard`/`lobby`/`partskiosk`, frontend path, business unit). FQDN-first, IP fallback. | GE-ENFORCE-DISPLAY.md |
| `GET /api/dashboarddefaults/visitor-location?fqdn=<fqdn>` | public (none) | Resolve the business unit for a lobby display by FQDN (IP fallback). | GE-ENFORCE-DISPLAY.md |
The `geenforce.fetch` and `geenforce.report` scopes accept both `X-API-Key` and
`Authorization: Bearer` transports, the same managed-token pattern the collector
uses (see COLLECTOR-INTEGRATION.md for how to mint, deploy, and rotate a scoped
token). A fetch token may be further resource-bound to specific scopes; a bound
token is denied (403 on manifest, 404 on payload) anything outside its scopes.
---
## 2. Import API
The import surface (an admin PAT plus `X-Import-Mode` to preserve legacy
timestamps) lets a script load an entire legacy database through the same
endpoints the UI uses. It is documented in full, per resource, in **IMPORT-API.md**
and is not duplicated here. The dashboarddefaults import fields (FQDN-preferred
keying) are covered there as well.
---
## 3. Core UI API
Everything else is the core UI API: the endpoints the Vue frontend calls. As a
rule these are JWT-authenticated (a login token or a managed Personal Access
Token) and versioned by the plugin contract (`__contract_version__`; the
current value is in [PROJECT-MAP.md](PROJECT-MAP.md), which is generated). Behavior and stability guarantees are in **CONTRACT-STABILITY.md**;
sister sites should pin tight `core_version` ranges until the contract reaches
1.0.
Two auth patterns dominate the reads:
- **Public (no token ever).** The endpoints below are reachable with no
credential at all. This is the surface a firewall or deployment reviewer asks
about, so it is enumerated in full.
- **Optional JWT (`jwt_required(optional=True)`).** Nearly every core and plugin
GET (list / detail / report / dashboard-summary) is optional-auth: it serves
reads anonymously and only requires a JWT for writes. There are well over a
hundred of these; rather than reprint them, enumerate them from the live spec
at `/api/docs` (filter to the `GET` operations). All product reports
(`/api/reports/*` and every plugin `.../report*`) are optional-auth by the same
convention.
Every mutating endpoint (POST / PUT / PATCH / DELETE) requires a JWT and is
gated by `require_role` or `require_permission`; none are public.
Two GETs deliberately break the optional-auth convention and require a
permission, because what they return is not asset metadata but a file that
carries value or liability of its own:
| Endpoint | Requires | Why |
|---|---|---|
| `GET /api/applications/package/<filename>` | `applications.view` | Licensed vendor installers. An anonymous URL is a redistribution channel. |
| `GET /api/warranty/proof/<filename>` | `warranty.view` | Invoices and certificates carry pricing and a service tag. |
Their sibling upload/delete routes are `applications.edit` / `warranty.edit`
like any other mutation.
### Fully public endpoints (auth = none)
| Endpoint | Purpose |
|---|---|
| `POST /api/auth/login` | Obtain a JWT. |
| `GET /api/setup/needs-admin` | First-run check: does the instance have zero users. |
| `POST /api/setup/create-admin` | First-run only; creates the first admin, then 403s forever. |
| `GET /api/settings/map-blueprint/<filename>` | Serve the floor-map blueprint image. |
| `GET /api/settings/branding/<filename>` | Serve site branding assets (logo, etc.). |
| `GET /api/settings` and `GET /api/settings/<key>` | Read-only, and only the public allowlist: the `branding` and `map` categories, a few named site keys, plus any key a plugin declares `public` in `get_settings_defaults` (e.g. `printedparts_label_prefix`, which the logged-out parts kiosk renders). Every other key answers 404 to an anonymous caller. |
| `GET /api/models/image/<filename>` | Serve a model image. |
| `GET /api/applications/image/<filename>` | Serve an application image (tiles render before login). |
| `GET /api/dashboard/navigation` | Public navigation tree. |
| `GET /api/dashboard/health` | Liveness / health probe. |
| `GET /api/plugins/enabled` | List enabled plugins (no claims used). |
| `GET /api/dashboarddefaults/display-role` | Display role resolution (see section 1). |
| `GET /api/dashboarddefaults/visitor-location` | Lobby business-unit resolution (see section 1). |
| `GET /api/employees/search`, `/lookup/<sso>`, `/lookup` | Employee directory lookups (kiosk / sign-in flows). |
| `GET /api/employees/photo/<filename>` | Serve an employee photo. |
| `GET /api/notifications` (and `/types`, `/<id>`, `/active`, `/calendar`, `/dashboard/summary`, `/employee/<sso>`, `/shopfloor`) | Read shop-floor notifications for the public display and kiosks. |
| `GET /api/slides/feed` | Slide feed for the lobby display and screensaver. |
| `GET /api/slides/img/<surface>/<filename>` | Serve a slide image. |
| `GET /api/printedparts/image/<filename>` | Serve a printed-part image. |
| `GET /api/printedparts/kiosk/item/<itemcode>` | Kiosk part lookup (deliberately open; a kiosk carries no JWT). |
| `POST /api/printedparts/kiosk/take` | Kiosk part checkout (deliberately open, per decision record). |
---
## See also
- `docs/adr/README.md` - architecture decision records index.
- `docs/DEPLOY.md` - deployment; the public-endpoint inventory in section 3 above
is the site-exposure surface a deploy reviewer needs.
- `docs/PLUGINS.md` - the plugin catalog.
- Contract docs: GE-ENFORCE-CLIENT.md, geenforce-api-cutover.md,
COLLECTOR-INTEGRATION.md, PRINTER-INSTALLER.md, GE-ENFORCE-DISPLAY.md,
IMPORT-API.md, CONTRACT-STABILITY.md.

154
docs/ASSET-COMPOSITION.md Normal file
View File

@@ -0,0 +1,154 @@
# Composition: several devices under one parent
When more than one physical device answers to a single identifier, model each
device as its own asset and file it under a parent. This page is the pattern:
when to reach for it, what you get for free, and how to make the collector do
it without writing code.
The mechanism is not new - `partof` and its propagation rails are the ADR-001
platform contract. What is new is the recipe.
## The problem it solves
A machine number is supposed to identify one thing. Sometimes it does not.
At the reference site, several Telesis part markers serve one operation number:
0613, 0615 and PRTMK01 each have more than one. Their configurations differ, most
often by COM port. Treating the operation as the device collapsed them into a
single record, and the damage was quiet:
- Their backups overwrote each other, so the history read as one device
flip-flopping between configurations that were really two devices.
- No question about an individual device could be asked at all - how many there
are, which port one is on, which one failed.
- Every PC driving one contested the single `controls` link to the operation,
so whichever reported last appeared to own it.
None of that announced itself. It surfaced as duplicate backup rows, weeks
later.
## When to use it
Reach for composition when ALL of these hold:
- Several physical devices share one identifier, and that is CORRECT - not a
numbering mistake. Two PCs carrying the same machine number by accident is a
fault to fix on the PC, not a model to build. `flask relationships
check-shared-machines` tells the two apart.
- The devices are individually interesting: they fail, get replaced, carry
their own configuration or calibration.
- Something already identifies each device. One device per PC is the easy case,
because the PC names it.
Do NOT reach for it when the parent is simply a location. A room holding six
printers wants a Location, not a parent asset.
## What you get
Filing a device `partof` a parent buys behaviour already built:
- **Control propagates.** `controls` propagates through `partof` (ADR-001, and
seeded by `flask seed reference-data`). A PC controlling a device therefore
controls its parent by inheritance, so the PC does NOT need - and must not
have - a direct link to the parent. That is what stops several devices
contesting a link only one can hold.
- **Position inherits.** Map-position resolution walks `partof` first, so a
device with no coordinates of its own shows at its parent's position.
- **History separates.** Anything keyed on the asset - backup revisions,
relationships, notes, audit - is per device instead of merged.
## Making the collector do it
A PC that drives a subordinate device declares it in
`plugins/computers/pctypemap.py`:
```python
SUBORDINATE_DEVICE_MAP = {
'gea-shopfloor-partmarker': {
'assettype': 'machine', # core AssetType for the device
'typename': 'Part Marker', # type within that vocabulary
'description': 'Telesis part marker',
'suffix': 'PARTMARKER', # device asset number = <PC>-<suffix>
'partof': True, # file under the reported machine number
'label': 'collector:partmarker',
},
}
```
`partof: True` is the switch that matters. It files the device under the
operation the PC reports AND stops that PC claiming the operation directly.
Leave it False for a device that does not share an identifier - a CMM is a
subordinate device too, but no two CMMs answer to one number, so it links to
its PC and nothing more.
`label` is the relationship origin marker. Only rows carrying it are archived
by a collector push, so links made by hand are never touched. **Do not change
an existing label**: those exact strings are in production databases.
A site adds or retargets an entry without a code change, per ADR-015:
```
Setting: subordinatedevice_<pctype> category: pctypemapping
Value: {"assettype": "machine", "typename": "Marking Laser",
"suffix": "LASER", "partof": true, "label": "collector:partmarker"}
```
A malformed override falls back to the built-in default rather than failing the
collector push - a bad setting must not stop a bay reporting its inventory.
## Making a backup kind follow the device
A backup posted by a PC that drives a device belongs to the DEVICE, not the
parent. `plugins/backups/services/registry.py` resolves this through
`markerforsource`: it finds the PC by the reported `sourcehostname`, follows the
PC's active device link, and files against what it finds. It falls back to the
machine number whenever the device cannot be resolved - no hostname on the
payload, a lean build without the computers plugin, or a PC that has not
reported yet - because filing under the parent beats rejecting a backup.
Two things that matter if you add a kind:
- **`sourcehostname` is load-bearing**, not informational. Without it a backup
files against the parent and merges two devices' histories.
- **A revision chain is (asset, kind, source hostname)**, so two devices on one
parent keep separate chains even before they become separate assets.
## Finding the next one
```
flask relationships check-shared-machines
```
Read-only. Lists every machine number claimed by more than one PC and separates
the two cases by whether child assets exist:
```
0615 11 PCs, 11 child asset(s) - modelled
2026 2 PCs, NO child assets - PCONE, PCTWO
```
The first is composition working. The second is two PCs carrying one number -
fix that on the PC.
## The parent is not disposable
Once devices are filed under it, the parent is doing a real job even though it
may look empty: it is the thing that says those devices belong together.
Deactivating it (`isactive = 0`, which is what "delete" does in the UI) breaks
filing, because the resolver requires an ACTIVE parent - every subsequent
report warns and files nothing.
A hard SQL `DELETE` is worse: `backuprevisions.assetid` is `ON DELETE CASCADE`,
so it destroys history still attached to the parent.
If the parent looks wrong in a list - an operation number appearing among
machines - that is a CLASSIFICATION question, not a deletion one.
## See also
- `docs/adr/ADR-001-asset-as-platform-contract.md` - relationship types,
propagation rails, position inheritance
- `docs/adr/ADR-015-site-specific-configuration.md` - why the device map is
settings-overridable
- `docs/COLLECTOR-INTEGRATION.md` - the collector contract and the part-marker
case end to end

125
docs/BACKUP-KINDS.md Normal file
View File

@@ -0,0 +1,125 @@
# Fleet config backups: adding a kind, and how health is judged
This is about the `backups` plugin - the per-PC CONFIG backups the collector
takes off shopfloor machines (NTLARS settings, part-marker configs). It is not
about backing up the ShopDB database itself; that is `docs/BACKUP-RESTORE.md`.
Two things this covers:
1. [Adding a new backup kind](#adding-a-new-backup-kind)
2. [How the Backup badge decides good vs stale](#how-the-backup-badge-decides-good-vs-stale)
---
## Adding a new backup kind
A kind is one class in `plugins/backups/services/registry.py`, added to
`REGISTRY` at the bottom of that file:
```python
REGISTRY = {k.key: k for k in (NtlarsKind(), PartMarkerKind())}
```
Subclass `BackupKind` and override what applies. `NtlarsKind` is the fullest
example (parseable, renderable, has an info panel); `PartMarkerKind` is the
lean one.
| Member | What it is |
|--------|------------|
| `key` | Wire value. This is what lands in `backuprevisions.backupkind` and what the fleet table shows on the badge. Keep it short and lowercase. |
| `displayname` | Human label for the UI. |
| `storagebackend` | `'shopdb'` (bytes live in the DB, deduped on a semantic hash) or `'share'` (file lives on the SMB share, deduped on a raw byte hash). |
| `assettypes` | Which core asset types this kind can attach to; `['*']` for any. |
| `emptytext` | Text when an asset has no revisions of this kind. `None` HIDES the panel, which is the right default - a kind applies to an asset TYPE, but whether a given machine ever has that backup is per-machine. A part-marker panel on all 144 machines is noise. |
| `parse(raw)` | Opaque kinds return `None`; parseable kinds return a projection dict. |
| `formats()` / `render(...)` | Download formats, for `shopdb` kinds. |
| `resolveassetid(payload)` | Map a collector payload to the asset the backup belongs to. |
| `infopanel()` / `buildinfo(...)` | Optional at-a-glance card (ADR-010). Declared by the KIND, not hardcoded in the plugin, so a successor technology ships its own card by adding a class. |
| `sharedir(...)` | Conventional UNC directory for a `share` kind. Advisory - the authoritative path is whatever the collector reported, because the PC is what actually wrote the file. |
### What you do NOT have to touch
The GE-Enforce **Enforcement Reports** Backup column needs no change. It reads
`backuprevisions` generically: newest revision per host, whatever the kind, and
shows `backupkind` on the badge. A new kind inherits the badge, the colour and
the tooltip with no work.
Same for the staleness rule below - it is time-based and kind-agnostic.
---
## How the Backup badge decides good vs stale
### The trap this is built around
**A revision is only written when the config CHANGES.** Dedup compares against
the latest revision for the chain `(asset, kind, sourcehostname)`, so a machine
whose config has been stable for six months has a six-month-old newest revision
and is perfectly healthy. The question worth answering is not "when was the last
backup taken" - it is "is this still being checked".
So the timestamp the fleet table uses is `backuprevisions.lastseenat`: the last
time the collector CONFIRMED this config, whether or not anything changed. It
moves on every successful collection; the revision does not.
That was shown as a raw date at first, and it read as neglect. At the default
`backups_intervalhours` of **24**, the collector only attempts once a day, so a
day-old confirmation IS the healthy steady state. The date made a working system
look like a stalled one, and made the reader do arithmetic against a setting
they would have to go and look up.
### What it does now
`GET /api/geenforce/reports` returns, per host:
| Field | Meaning |
|-------|---------|
| `backupkind` | Which kind was most recently confirmed. `null` = no backup at all. |
| `backuplastseen` | When it was last CONFIRMED still current (ISO). Tooltip only. |
| `backupcollectedat` | When the config was last CAPTURED (ISO). Only moves on a real change. Tooltip only. |
| `backupok` | `true` good, `false` stale, `null` nothing to judge. |
| `backupstaleafterdays` | The threshold in force, so the UI can explain itself. |
Both dates are reported because they answer different questions and one cannot
stand for both. The hover reads *"ntlars: checked 13 Aug 1:20 PM. Verified the
backup taken 12 Aug 5:20 PM is still current."* - the first date proves the
check is running, the second says what is actually stored. Collapsing them was
what made a healthy machine look neglected.
`backupok` is deliberately **tri-state**. `null` means there is no revision for
that host, or the check is disabled - and it renders as no badge, never green.
"Never seen" must not read as healthy.
The threshold is the backups plugin's own `backups_staledays` setting (default
**3**), read through `plugins.backups.services.staleness.staledays()` rather
than re-derived, so there is ONE definition of stale. `0` disables the check.
The import is guarded, so a lean site build without the backups plugin returns
`null` instead of failing.
### Why time-based rather than per-kind
Every kind answers the same question the same way: something confirmed this
recently, or it did not. Making the rule per-kind would mean each new kind has
to define health before it can show a badge, for no gain. If a kind ever needs
its own window - a weekly backup that should not be judged on a 3-day rule -
add a threshold override on `BackupKind` and have `_backup_stale_cutoff` prefer
it; the tri-state contract stays as it is.
### Where the code lives
- `plugins/geenforce/api/routes.py` - `_attach_backup_state` (newest revision
per host), `_backup_stale_cutoff` (threshold), `_backup_ok` (verdict)
- `plugins/geenforce/frontend/views/EnforcementReports.vue` - `backupClass`
(green / red / none), `backupTitle` (the hover text)
- `plugins/backups/services/staleness.py` - the shared threshold, also behind
the dashboard's stale-backups card
Tests: `tests/test_plugins/test_geenforce_reporting.py`, the backup-verdict
block - recent-is-good, older-than-threshold-is-stale, and none-is-not-green.
---
## See also
- `docs/BACKUP-RESTORE.md` - backing up the ShopDB database itself
- `docs/geenforce-api-cutover.md` - the fleet reporting path these fields ride on

105
docs/BACKUP-RESTORE.md
View File

@@ -96,18 +96,42 @@ CREATE DATABASE shopdb_flask CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
tar xzf instance-2026-07-10.tar.gz # restores ./instance/
```
The docker-compose api container reads `instance/` from the repo working
directory; make sure it is present before starting `api`.
The default `docker-compose.yml` does NOT bind-mount `instance/` into the api
container (its only volume is `- ./plugins:/app/plugins:ro`, and the image never
copies `instance/`), so the container's Flask instance path is an empty
`/app/instance` and a restored host `./instance` is invisible to it. To make the
restored `instance/` visible, add a bind mount to the api service before starting
it:
```yaml
api:
volumes:
- ./plugins:/app/plugins:ro
- ./instance:/app/instance
```
Make sure `./instance` is present on the host before starting `api`.
### Step 4: Bring up the API and reconcile migrations
```bash
docker compose up -d api
docker compose exec api flask db upgrade
docker compose exec api flask plugin upgrade-all
```
For a non-docker deploy:
```bash
flask db upgrade
flask plugin upgrade-all
```
`flask db upgrade` is a safety net: if the dump predates the current code, this
applies any newer migrations. If the dump is at the same version it is a no-op.
applies only the core Alembic chain. `flask plugin upgrade-all` then applies any
newer per-plugin migrations (each bundled plugin owns its own chain, ADR-008);
without it, plugin-owned tables stay un-migrated. If the dump is at the same
version both are no-ops.
### Step 5: Verify
@@ -118,8 +142,83 @@ applies any newer migrations. If the dump is at the same version it is a no-op.
- `curl -s -X POST -H "Content-Type: application/json" -d '{}' http://localhost:5001/api/auth/login | jq .`
should return a `VALIDATION_ERROR`, not a 500.
## Windows sites (installer-built)
On a server installed from the Windows installer, everything above is wrapped by
the operator console. Do not run mysqldump by hand:
```powershell
cd C:\shopdb-flask
.\shopdb-admin.ps1 backup # C:\ProgramData\ShopDB-Flask\backups
.\shopdb-admin.ps1 backup D:\backups
```
The dump is verified complete before it is reported as good; a truncated one is
deleted rather than left to be discovered when it is needed. An upgrade takes its
own backup automatically before touching the schema, and restores from it if a
migration fails.
Two Windows-specific notes:
- The backup directory is locked to Administrators and SYSTEM, because a dump
contains every row including user password hashes. Keep it that way.
- `mysqldump` must be present. It ships with the bundled-database option; a site
using a remote MySQL needs `mysqlclient\` in its installer bundle, or the
pre-upgrade backup is skipped. `shopdb-admin.ps1 check` reports this.
### Restoring on Windows, in order
Order matters. Restoring the database under running code that expects a
different schema is how a restore turns into a second incident.
```powershell
cd C:\shopdb-flask
# 1. Stop serving. The pool, not the whole site: other applications on this
# IIS server are unaffected.
.\shopdb-admin.ps1 stop
# 2. Restore the database. Use the dump taken closest BEFORE the problem,
# not the newest one - the newest may already contain it.
mysql -u root -p shopdb_flask < C:\ProgramData\ShopDB-Flask\backups\shopdb_flask-pre-upgrade-20260814-0730.sql
# 3. Restore instance\ if you are rebuilding a server rather than just
# reverting data. It holds uploaded branding, map blueprints, application
# images and warranty proofs - none of which are in the database.
robocopy D:\backups\instance C:\shopdb-flask\instance /MIR
# 4. Start, then prove it.
.\shopdb-admin.ps1 start
.\shopdb-admin.ps1 check -Json
```
`check -Json` reports version, publishing method, IIS and pool state, HTTP
reachability, database reachability, Python version and installed plugins. If it
passes, the restore worked; if the version it reports is not the version you
expect, see "Rolling back a release" in [UPDATES-WINDOWS.md](UPDATES-WINDOWS.md),
because a restored database and newer code is the one combination the installer
cannot fix for you.
### What is lost if `.env` is lost
`.env` is not in the database dump, and rebuilding it is not simply retyping it:
| Value | If it is lost |
|---|---|
| `SECRET_KEY`, `JWT_SECRET_KEY` | Every issued token and session becomes invalid. Users log in again; managed API tokens must be reissued, which means every collector and GE-Enforce client needs its key replaced. Recoverable, but it is a fleet-wide job. |
| `DATABASE_URL` password | Recoverable: reset the MySQL user's password and write the new one in. |
| `MYSQL_ROOT_PASSWORD` | Recoverable through MySQL's own reset procedure, which requires stopping the server. |
| `ZABBIX_TOKEN` and similar integration tokens | Reissue at the far end. Nothing else breaks. |
So back it up with the database, to somewhere as protected as the dump - it is
ACL'd to Administrators and SYSTEM on the server for the same reason. A dump
without its `.env` restores the data and locks everyone out of it.
See [OPERATE-WINDOWS.md](OPERATE-WINDOWS.md).
## See also
- [DEPLOY.md](DEPLOY.md) - first-time deploy
- [INSTALL-WINDOWS.md](INSTALL-WINDOWS.md) - Windows Server install
- [UPGRADE.md](UPGRADE.md) - upgrade procedure (back up first)
- [CONFIG.md](CONFIG.md) - environment variables and Setting keys

76
docs/COLLECTOR-INTEGRATION.md
View File

@@ -78,9 +78,10 @@ suspended for it, so the token is contained to the collector API even though its
owner is an admin - it cannot act with admin authority anywhere.
1. Settings > API Tokens > New Token.
2. Click the **Collector service token** preset (pre-selects only
`collector.ingest`), name it (e.g. `wj-fleet-collector`), optionally set an
expiry, Create.
2. Check **Restrict permissions**, then in the permissions grid tick only
**Submit collector payloads (fleet reporting)** (the `collector.ingest`
permission under the Collector category). Name it (e.g. `fleet-collector`),
optionally set an expiry, Create.
3. Copy the `shopdb_pat_...` secret (shown once) and deploy it to the fleet the
same way as the env key: the `collectorApiKey` field in per-site
`site-config.json` (see "Delivering the API key to clients" below). The
@@ -137,7 +138,7 @@ Response body (HTTP 200), wrapped in the standard envelope
"status": "ok",
"action": "created",
"assetid": 12345,
"identityvalue": "WJRP2335",
"identityvalue": "SHOPPC2335",
"warnings": ["unknown operating system: Microsoft Windows 11 Enterprise 23H2 (build 22631)"]
},
"message": "computers collector created",
@@ -197,7 +198,7 @@ a column.
| Payload field | Type | Server behaviour (`apply_collector_payload`) |
|---|---|---|
| `hostname` (required) | string | Identity. Matches `Computer.hostname` (case-insensitive), then falls back to `Asset.assetnumber`. New asset created if no match. |
| `machinenumber` | string | Business tag -> `Asset.assetnumber`. The placeholder `9999` and empty string are skipped; when skipped a new PC falls back to `assetnumber = hostname`. On an existing PC a real value updates `assetnumber`. |
| `machinenumber` | string | Identifies the MACHINE this PC drives, never the PC. A new PC always takes `assetnumber = hostname`, and an existing PC's `assetnumber` is left alone. The number resolves a machine asset and builds a PC -> machine `controls` relationship; an unknown number warns rather than creating a machine. The placeholder `9999` and empty string link nothing. See "Machine links" below. |
| `pctype` | string | `gea-shopfloor-*` imaging type -> `Computer.computertypeid` via the configurable `pctypemap` settings. Unmapped value -> warning, not error. |
| `pcsubtype` | string | Accepted but not stored yet -> warning. |
| `serialnumber` | string | `Asset.serialnumber`. |
@@ -215,6 +216,51 @@ a column.
Schema source of truth: `get_collector_schema` in `plugins/computers/plugin.py`.
If you change the payload, change it there and re-check this table.
### PC -> machine links, and part markers
The reported `machinenumber` says which machine the PC drives. It is not the
PC's own identifier, and writing it as one is a defect this contract used to
describe: `assets.assetnumber` is uniquely indexed and the machine already holds
that value, so the insert failed and the bay got a 500 on every report, forever.
A PC takes its hostname as its asset number and keeps it.
- Link: PC -> machine as `controls`, tagged
`assetrelationships.label = 'collector:machine'`. Only tagged rows are ever
archived by a push, so a link made by hand is never touched.
- Unknown number: warns and links nothing. A machine is never invented, because
a mistyped number would create equipment nobody can account for.
- A second PC reporting a machine another PC holds is a CLAIM, not a handover.
A PC imaged for a machine carries that number before it reaches the floor, so
the PC currently running the machine keeps the link while it is still alive -
reporting within 24 hours (`MACHINE_CLAIM_QUIET_HOURS`) and still `In Use`.
The challenger is recorded as a dormant link, which also marks the claim as
already announced so it does not alert every cycle.
- Handover completes on its own once the incumbent has been quiet past that
window, or immediately when someone moves it off `In Use` - which is the
one-step way to force a swap at the moment it happens. The old link is
archived, never deleted, so "which PC ran this machine in June" stays
answerable. The retired PC's STATUS is deliberately not changed: the collector
cannot tell shelved from broken from re-imaged.
- Alerts for handover and contested claims are gated on the
`computers_machinelink_alerts` setting, which ships OFF. A site may run
several PCs on one machine number legitimately, and there the alerts fire on
correct data. Warnings ride in the collector response either way.
The part-marker case below is one instance of a general pattern - several
devices under one parent. `docs/ASSET-COMPOSITION.md` covers when to use it,
what propagation buys you, and how to declare a new device type without a code
change.
A PC reporting `pctype = gea-shopfloor-partmarker` is handled differently,
because several markers can serve one machine number and an operation holds any
number of them. Such a PC gets its own Part Marker machine asset
(`<HOSTNAME>-PARTMARKER`), `controls` it, and the marker is filed `partof` the
operation the PC reported. Both links are tagged `collector:partmarker`. The PC
does NOT claim the operation directly: `controls` propagates through `partof`,
so control still follows from controlling the marker, without two markers
contesting a link only one can hold. Backups from a marker PC file against the
marker rather than the operation.
### PC -> printer relationship sync
When a payload carries `defaultprinter` and/or `printers`, the collector syncs
@@ -244,10 +290,12 @@ the PCs that use it (both render in the shared Relationships card).
### pc-type mapping (configurable per site)
`pctype` (e.g. `gea-shopfloor-cmm`) maps to a shopdb Computer Type through
`pctypemap_<pxetype>` settings (Settings > Collector PC Types).
Defaults live in `plugins/computers/pctypemap.py` and are seeded on plugin
install; edit per site in the UI. Unmapped pc-types produce a warning, not a
failure.
`pctypemap_<pxetype>` settings. The "Collector PC Types" settings page is
retired (ADR-012): pc-type-to-Computer-Type handling now lives in GE-Enforce
(each imaging PC type is a manifest scope with its own `computertypeid`). The
built-in defaults in `plugins/computers/pctypemap.py` are still seeded on plugin
install and the collector still reads them, so existing enrollment keeps
working. Unmapped pc-types produce a warning, not a failure.
### Classic api.asp field mapping (for porting the PowerShell reporter)
@@ -289,7 +337,7 @@ Before (no longer works - the key is ignored and the request is rejected 401):
POST /api/collector/computers?api_key=SECRET
Content-Type: application/json
{ "hostname": "WJRP2335", "machinenumber": "2335" }
{ "hostname": "SHOPPC2335", "machinenumber": "2335" }
```
After (correct):
@@ -299,7 +347,7 @@ POST /api/collector/computers
X-API-Key: SECRET
Content-Type: application/json
{ "hostname": "WJRP2335", "machinenumber": "2335" }
{ "hostname": "SHOPPC2335", "machinenumber": "2335" }
```
PowerShell before/after:
@@ -422,8 +470,8 @@ function Get-ShopdbCorpIPv4 {
# Pick the corp/AESFMA NIC IP. Same allowed-range gate as
# Report-AssetToShopDB.ps1 - update the ranges if the site re-VLANs.
$allowedRanges = @(
@{ Network = '10.134.48.0'; PrefixLen = 23 },
@{ Network = '10.48.249.0'; PrefixLen = 26 }
@{ Network = '192.0.2.0'; PrefixLen = 24 },
@{ Network = '198.51.100.0'; PrefixLen = 26 }
)
function ConvertTo-Uint32([string]$ip) {
$bytes = ([System.Net.IPAddress]::Parse($ip)).GetAddressBytes()
@@ -447,7 +495,7 @@ function Get-ShopdbCorpIPv4 {
function Send-ShopdbCollectorReport {
param(
[string]$SiteHost = 'tsgwp00525.wjs.geaerospace.net',
[string]$SiteHost = 'shopdb.example.net',
[string]$ApiKey = (Get-ShopdbCollectorApiKey),
[int]$TimeoutSec = 30,
[string]$LogFile = ('C:\Logs\Shopfloor\collector-{0}.log' -f (Get-Date -Format 'yyyyMMdd'))

13
docs/CONFIG.md
View File

@@ -145,6 +145,14 @@ read back through the API.
| `printer_hostname_template` | `Printer-{ip}.printer.geaerospace.net` | Printer hostname template. `{ip}` is the dash-separated IP address. |
| `contact_email_domain` | `geaerospace.com` | Email domain appended to a support contact's SSO to build email (`sso@domain`) and Teams-chat links. Blank hides the contact action buttons. |
| `dualpath_single_machine` | `true` | Treat a Dualpath pair (a dual-bay machine with one controller) as a single machine in the machines list, dashboard/report counts, and the floor map (the secondary bay is hidden). The data model always keeps both bay records; detail pages stay per-bay with a sibling banner. `false` lists and counts both bays separately. |
| `site_timezone` | `America/New_York` | IANA timezone for the site. Notification start/end times are entered and displayed in this zone (not the viewer's browser zone), and daily-reset notification expiry (`expirymode=dailytime`) is computed here. Editable in Settings > Site > Localization. Public-readable so kiosks/clients can resolve it. |
Notification times are stored and served in UTC; the frontend converts to
`site_timezone` via `frontend/src/utils/datetime.js` (Intl-based, DST-safe).
Change note: notification times are now timezone-correct (stored UTC, shown in
`site_timezone`); this fixes the prior offset bug where a 2:34 PM entry displayed
as 6:34 PM.
### branding
@@ -319,6 +327,11 @@ One boolean key per search domain, keyed `search_<type>_enabled` (default
`true`). Toggles whether a domain appears in global search results. The set is
generated from `SEARCH_DOMAINS` in `shopdb/core/api/settings.py`.
Search terms are matched word-wise: a multi-word query returns rows containing
EVERY word, each word anywhere in the searched fields, in any order ("CSF Roles"
matches a row with "CSF" and "Roles" in different columns). Quoting does not
force a contiguous phrase.
## Custom fields
Site-defined extra attributes per asset type (Settings > Custom Fields, table

30
docs/CONTRACT-STABILITY.md
View File

@@ -8,12 +8,14 @@ the live code, not aspiration. The authoritative hook reference is
## Current version
The plugin contract is at **0.10.0**, declared in `shopdb/__init__.py` as
`__contract_version__`. It is pre-1.0, which under semver means any 0.x minor
bump is allowed to break the contract, and this project has used that latitude.
The plugin contract version is declared in `shopdb/__init__.py` as
`__contract_version__`, and the current value is in
[PROJECT-MAP.md](PROJECT-MAP.md), which is generated. It is pre-1.0, which under
semver means any 0.x minor bump is allowed to break the contract, and this
project has used that latitude - see 0.19.0 below.
The product release version (`__version__`, currently 0.5.0) is a separate
series with its own bump rules; see [ADR-007](adr/ADR-007-product-versioning-and-releases.md).
The product release version (`__version__`) is a separate series with its own
bump rules; see [ADR-007](adr/ADR-007-product-versioning-and-releases.md).
Do not pin against it for compatibility - pin against `__contract_version__`.
### 0.x history
@@ -28,8 +30,24 @@ Recorded in the comment block in `shopdb/__init__.py`:
| 0.7.0 | Added the four ADR-010 frontend-contribution hooks (`get_settings_cards`, `get_asset_panels`, `get_map_overlays`, `get_asset_presentation`), consumed by the `GET /api/pluginui/*` endpoints | additive optional hooks (minor) |
| 0.9.0 | Exposed the dualpath pair-resolution helpers on `shopdb.api` for the machines plugin | additive surface (minor) |
| 0.10.0 | Added the `get_permissions` hook so plugins declare their own RBAC permissions; the catalog is resolved dynamically from core + enabled plugins | additive optional hook (minor) |
| 0.11.0 | Added `service_token_authorized(scope)` to `shopdb.api` so a plugin's unattended endpoints (e.g. the GE-Enforce fetch API) can authorize a scoped managed service token without importing core token internals | additive surface (minor) |
| 0.12.0 | Added the mailer helpers (`send_email`, `send_alert`) to `shopdb.api` | additive surface (minor) |
| 0.13.0 | Added the `User` model to the `shopdb.api` surface | additive surface (minor) |
| 0.14.0 | Added `send_webhook` to the `shopdb.api` surface | additive surface (minor) |
| 0.15.0 | Added `authorized_service_token` / the `SupportTeam` model to the `shopdb.api` surface | additive surface (minor) |
| 0.16.0 | Added the `get_settings_defaults` hook so a plugin declares the Setting rows it owns; the framework seeds them at install, at enable, and on `flask plugin upgrade-all`, files a first-time write under the declared category, and honours `public: True` for pages that render before login | additive optional hook (minor) |
| 0.17.0 | Added `DashboardDefault` to the `shopdb.api` surface, so a plugin can resolve a display without reaching into core | additive surface (minor) |
| 0.18.0 | Added `DISPLAY_ROLES`, `DISPLAY_ROLE_PATHS` and `normalize_display_role`, and finally exported `DashboardDefault`, which 0.17.0 imported but left out of `__all__`. The role vocabulary became the kiosk's own - `Dashboard`, `Lobby`, `3DPrintRoom` - so a plugin holding its own copy of that map reads core's instead of drifting from it | additive surface (minor) |
| 0.19.0 | **BREAKING.** `get_dashboard_widgets` returns DATA AND SHAPE, not a component name. The old shape (`name` + `component` + `size`) named a Vue component per widget, which cannot survive a lean build - a plugin's component may never be staged into the frontend bundle (ADR-013) - and five plugins were declaring widgets that pointed at components nobody had written. A card now declares `id` / `title` / `endpoint` / `render` / `severity` / `permission` / `empty` / `position`; see PLUGIN-HOOKS.md. Also added `send_upload` so a plugin serving user-supplied bytes gets the headers that keep an SVG from running as script | **contract change (minor, pre-1.0)** |
The source comment block documents 0.3.0, 0.4.0, 0.6.0, 0.7.0, 0.9.0, and 0.10.0. Earlier points
The source comment block documents 0.3.0, 0.4.0, 0.6.0, 0.7.0, 0.9.0, 0.10.0,
0.11.0, 0.16.0, 0.18.0 and 0.19.0; 0.12.0 through 0.15.0 and 0.17.0 are recorded
in this table only.
**0.19.0 is the one to read before pinning.** It is the only entry in this
series that took something away, and it shipped before it was recorded here -
which is precisely the reason to pin tight rather than to trust that a minor
bump is safe. Earlier points
(0.1.x / 0.2.x) predate that recorded rationale; `PluginMeta`'s fallback
`core_version` default of `>=0.2.0,<1.0.0` is the only remaining trace of the
0.2 baseline.

142
docs/CSV-IMPORT.md Normal file
View File

@@ -0,0 +1,142 @@
# Loading a site's data from spreadsheets
For getting a new site's starting data in when you have a spreadsheet rather
than a source database to script against. No developer needed.
If the site *does* have a source system worth reading, the HTTP import API is
the better tool - see [IMPORT-ADOPTION.md](IMPORT-ADOPTION.md).
---
## The short version
```bash
cd C:\shopdb-flask # or your install directory
venv\Scripts\flask csv templates --out csv-templates
```
Fill in the templates. Then:
```bash
venv\Scripts\flask csv import --dir csv-templates
```
That **checks only** and changes nothing. It tells you what it would create and
what is wrong. When you are happy:
```bash
venv\Scripts\flask csv import --dir csv-templates --commit
```
---
## Write names, not numbers
This is the part that makes the difference. Every column that points at another
table accepts the **name** of the thing:
```
assetnumber,name,assettypeid,statusid,locationid
CMM-01,Zeiss Contura,Measuring Tool,Active,Gage Lab
MILL-07,Haas VF-2,Machine,Active,Bay 3
```
`assettypeid` gets `Measuring Tool`. `locationid` gets `Gage Lab`. You never
have to import a file, read back the numbers it generated, and paste them into
the next one.
The column keeps its database name so it matches the rest of the system, but
the value is whatever you actually know. Numeric ids still work if you have
them - useful when re-importing something this system exported.
Names resolve **across files in the same run**, so `assets.csv` can reference a
location that only exists because `locations.csv` was loaded moments earlier.
## Nothing is half-imported
Every row is checked before anything is written. If one row is wrong, nothing is
written at all - you fix the file and run it again. A mistake on line 400 never
leaves 399 rows loaded.
## Running it twice is safe
Each file is matched on a natural key - `assetnumber` for assets, `locationname`
for locations, and so on. Re-importing an edited file **updates** those rows
rather than creating second copies. Correcting a spreadsheet and re-running is
the expected workflow, not a mistake.
## What the errors look like
```
assets: 0 new, 0 updated, 1 problem(s)
line 4, column 'locationid': nothing in locations is named 'Bay 9'
- add it to locations.csv, or import that file first
```
Line, column, value, and what to do. Not a foreign key constraint violation.
---
## What you can import
Fourteen tables, in the order the importer handles them. You only need the ones
you have; skip any file you do not care about.
| Order | File | Matched on |
|---|---|---|
| 1 | `assetstatuses.csv` | `status` |
| 2 | `assettypes.csv` | `assettype` |
| 3 | `locationtypes.csv` | `locationtype` |
| 4 | `modeltypes.csv` | `modeltype` |
| 5 | `computertypes.csv` | `computertype` |
| 6 | `machinetypes.csv` | `machinetype` |
| 7 | `businessunits.csv` | `businessunit` |
| 8 | `locations.csv` | `locationname` |
| 9 | `vendors.csv` | `vendor` |
| 10 | `models.csv` | `modelnumber` |
| 11 | `operatingsystems.csv` | `osname` |
| 12 | `assets.csv` | `assetnumber` |
| 13 | `computers.csv` | `assetid` |
| 14 | `machines.csv` | `assetid` |
`--dir` handles the order for you. Use `--file` with `--table` for one file.
**User accounts are deliberately not importable.** Passwords do not belong in a
spreadsheet, in either direction. Create the first administrator through the
first-run page and the rest in the application.
## The templates are generated, not maintained
`flask csv templates` builds them from the live database schema each time. Every
column offered exists; every required one is marked; every foreign key says
which file it refers to.
This matters because the alternative does not work. A hand-written template set
was tried, and it had invented columns on seven of eleven tables and named a
table that does not exist - while looking entirely plausible. Templates that are
generated cannot drift from the schema, and a test fails the build if they ever
do.
## Editing the files
- **UTF-8**, no BOM. Excel: "CSV UTF-8 (Comma delimited)".
- Lines starting with `#` are ignored, so the notes and the example row in each
template can stay where they are.
- Booleans are `1` or `0`.
- Dates are `YYYY-MM-DD` (`2026-08-04`). `YYYY-MM-DD HH:MM:SS` also works, as do
`DD/MM/YYYY` and `MM/DD/YYYY`.
- Leave a cell **empty** for "no value". Not `NULL`, not `N/A`.
- Quote anything containing a comma: `"Bay 3, North"`.
## If it will not run
**"the 'assets' table does not exist in this database"** - the schema has not
been created. Run `flask db upgrade` first, and check `DATABASE_URL` points at
the site you meant.
**"unknown column(s): ..."** - a column that does not exist, usually from an
older template. Regenerate with `flask csv templates`; the message lists what
the table does accept.
**"required column(s) missing: ..."** - a column that must be present has been
deleted from the header. Regenerate and copy your data across.

149
docs/DEPLOY-AIRGAP.md Normal file
View File

@@ -0,0 +1,149 @@
# Air-gapped Docker deploy
For a site with **no internet**. The site cannot `pip install`, `npm ci`, or
`docker pull`, so nothing is built or pulled there. You build a fully
self-contained image on a **connected** box, ship it as one tarball, and
`docker load` + run it at the site.
This is the counterpart to `docker-compose.yml` (the connected/build template).
The air-gapped stack uses `docker-compose.airgap.yml`, which differs in three
ways that matter:
- **`image:` not `build: .`** - the site runs a loaded image; it never builds.
- **No `./plugins` bind mount** - the image already carries every plugin baked
in. Binding a host `./plugins` (which does not exist at an image-only site)
would mask the baked plugins with an empty dir and load **zero** plugins.
- **A one-shot `migrate` service** runs `db upgrade` + `plugin upgrade-all` +
seed before `api` starts, so `up -d` alone brings up a working site.
---
## 1. Build the bundle (connected box)
On a box with clean access to Docker Hub + PyPI + npm, from the repo root:
```powershell
pwsh scripts/build-offline-bundle.ps1 -Version 0.7.0
```
This builds `shopdb-flask:0.7.0` (frontend + all Python deps baked in), pulls
`mysql:8.0`, and writes:
- `shopdb-stack-0.7.0.tar.gz` - both images in one archive
- `shopdb-stack-0.7.0.tar.gz.sha256` - checksum to verify after transfer
The image is self-contained: **no pip/npm/registry access is needed at the
site.**
### Building behind Zscaler
If the build box is itself behind GE Zscaler, the in-build `pip`/`npm` may fail
with `CERTIFICATE_VERIFY_FAILED` - the build container has its own cert store
and does not trust GE's re-signing root (host `PIP_CERT` / `NODE_EXTRA_CA_CERTS`
do NOT carry into a `docker build`). Easiest: build from a box with clean
internet (home, cloud, or CI). Otherwise the corp root CA must be trusted
**inside** the build (a Dockerfile change to accept the CA - not wired today;
see docs/DEVELOPMENT-SETUP.md section 0b for the all-roots PEM bundle).
---
## 2. Transfer + verify
Carry `shopdb-stack-0.7.0.tar.gz` (+ the `.sha256`), plus
`docker-compose.airgap.yml` and `.env.example`, to the site on approved media.
Verify the archive survived the trip:
```powershell
# PowerShell
(Get-FileHash shopdb-stack-0.7.0.tar.gz -Algorithm SHA256).Hash.ToLower()
# compare against the .sha256 file
```
```bash
# Linux site
sha256sum -c shopdb-stack-0.7.0.tar.gz.sha256
```
---
## 3. Load + run (air-gapped site)
```bash
# 1) Load both images into the local Docker.
docker load -i shopdb-stack-0.7.0.tar.gz
docker image ls | grep -E 'shopdb-flask|mysql' # confirm both present
# 2) Configure the site.
cp .env.example .env
# Edit .env - REQUIRED:
# IMAGE_TAG=0.7.0 # MUST match the loaded image tag
# MYSQL_ROOT_PASSWORD=... # strong, unique
# MYSQL_PASSWORD=... # strong, unique (the app's db user)
# SECRET_KEY=... # 32+ random bytes
# JWT_SECRET_KEY=... # 32+ random bytes, different from SECRET_KEY
# CORS_ORIGINS=https://shopdb.site.example # the site's browser origin(s)
# Optional: API_PORT (default 5001), LOG_LEVEL, ZABBIX_URL/ZABBIX_TOKEN.
# 3) Bring it up. The migrate one-shot runs the schema + seed, then api starts.
docker compose -f docker-compose.airgap.yml up -d
# 4) Watch the one-shot finish (it exits 0 when the schema + seed are done).
docker compose -f docker-compose.airgap.yml logs -f migrate
```
`IMAGE_TAG` in `.env` must equal the loaded tag (`0.7.0` here); otherwise compose
looks for an image that was never loaded and `api`/`migrate` will not start.
---
## 4. Create the first admin
Seeding creates permissions, settings, and reference data, but not a login. Make
one admin (choose the password; it is not automatable):
```bash
docker compose -f docker-compose.airgap.yml exec api \
flask seed admin --username <username> --email <email> --password <password>
# Omit --password to have a strong one generated and printed once.
```
---
## 5. Verify
```bash
docker compose -f docker-compose.airgap.yml ps # db + api "running", migrate "exited (0)"
docker compose -f docker-compose.airgap.yml logs api # gunicorn started, no tracebacks
curl -sf http://localhost:${API_PORT:-5001}/api/dashboard/health # or browse the site origin
```
Then log in at the site origin with the admin created in step 4.
---
## Upgrading to a new version
Build a new bundle on the connected box (`-Version 0.8.0`), transfer, then at the
site:
```bash
docker load -i shopdb-stack-0.8.0.tar.gz
# set IMAGE_TAG=0.8.0 in .env
docker compose -f docker-compose.airgap.yml up -d
```
The `migrate` one-shot re-runs `db upgrade` + `plugin upgrade-all` + seed (all
idempotent) against the existing data before the new `api` starts. Back up first
(see docs/BACKUP-RESTORE.md); the `db_data` volume persists across upgrades.
---
## Troubleshooting
| Symptom | Cause / fix |
| --- | --- |
| `service api is not running` / `manifest ... not found` | The image was not loaded, or `IMAGE_TAG` in `.env` does not match a loaded image. `docker image ls`, fix `IMAGE_TAG`. Also: never use `docker-compose.yml` here - its `build: .` needs internet. |
| Site loads but **no plugins / empty nav** | You used the wrong compose file. `docker-compose.yml` bind-mounts `./plugins` (absent here) over the baked plugins. Use `docker-compose.airgap.yml`. |
| `api` never starts, `migrate` shows an error | Read `logs migrate`. A DB-connection error means `db` is not healthy yet (`logs db`) or `MYSQL_PASSWORD` in `.env` differs from what the `db` volume was first initialised with. A fresh site with a stale `db_data` volume needs the volume removed (`docker compose ... down -v` - DESTROYS data). |
| `SECRET_KEY must be set` (and similar) on `up` | A required `.env` var is empty. Fill every REQUIRED key in step 3. |
| Build fails on the connected box at `pip`/`npm` | Zscaler cert - see "Building behind Zscaler" above. |

196
docs/DEPLOY-WINDOWS-IIS.md
View File

@@ -1,196 +0,0 @@
# Deploy shopdb-flask to Windows IIS (MySQL 5.6)
Runbook for standing up a single-site instance on the production Windows Server
that already runs the classic ASP shopdb, using IIS + HttpPlatformHandler +
waitress, against the existing MySQL 5.6. This is the test-instance path; keep
developing on the Linux dev box and redeploy as needed.
The Docker path in `DEPLOY.md` does NOT apply on Windows (gunicorn is Linux
only, and there is no MySQL container here). This file replaces it for IIS.
Notation: `APP_ROOT` = the deploy folder, e.g. `C:\shopdb-flask`. The IIS site
physical path must be `APP_ROOT` (where `wsgi.py` lives).
## 0. Prerequisites on the box
- Python 3.12 (same minor as dev). `py -3.12 --version` to confirm.
- IIS with the **HttpPlatformHandler** module:
https://www.iis.net/downloads/microsoft/httpplatformhandler
- **URL Rewrite** module (only for the optional real-client-IP rule).
- Network access to the MySQL 5.6 server.
- If the box is air-gapped, you cannot `pip install` live. On the dev box run
`pip download -r requirements.txt waitress -d wheels\` (on a matching
Windows/Python target, or use `--platform` wheels), copy `wheels\` over, and
install with `pip install --no-index --find-links wheels\ ...`.
## 1. Copy the code
Copy the repo to `APP_ROOT`, INCLUDING `frontend/dist` (the built SPA the API
serves). Build it on dev first if stale:
```bash
# on the dev box
cd frontend && npm run build # produces frontend/dist
```
Ship `frontend/dist` with the code (Node is not needed on the prod box).
## 2. Python venv + dependencies
```powershell
cd C:\shopdb-flask
py -3.12 -m venv venv
venv\Scripts\python -m pip install --upgrade pip
venv\Scripts\pip install -r requirements.txt
venv\Scripts\pip install waitress
```
The DB driver is `pymysql` (pure Python) so no C compiler / MySQL client libs
are needed. `waitress` is the WSGI server (installed separately, same as the
Docker image installs gunicorn separately).
## 3. Prepare MySQL 5.6 (the utf8mb4 gotcha)
MySQL 5.6 defaults cannot index utf8mb4 VARCHAR(255) columns (767-byte prefix
limit) and often defaults the server charset to latin1. The schema is utf8mb4,
so the server needs Barracuda + large-prefix, made durable in `my.ini` under
`[mysqld]`, then restart the MySQL service:
```ini
[mysqld]
innodb_file_per_table = 1
innodb_file_format = Barracuda
innodb_large_prefix = 1
```
Then create the database as utf8mb4 and a least-privilege app user:
```sql
CREATE DATABASE shopdb_flask CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'shopdb'@'%' IDENTIFIED BY 'CHANGE_ME';
GRANT ALL PRIVILEGES ON shopdb_flask.* TO 'shopdb'@'%';
FLUSH PRIVILEGES;
```
Without the `[mysqld]` flags, `flask db upgrade` fails with error 1071
("Specified key was too long"). The migration chain emits `ROW_FORMAT=DYNAMIC`
per table (see `migrations/env.py`), which fits the 3072-byte prefix those
flags unlock.
## 4. Configure secrets and connection (.env)
Create `APP_ROOT\.env` (loaded by `wsgi.py` via `load_dotenv()`). Keep secrets
here, not in `web.config`. Lock the file's ACLs to the IIS app-pool identity +
administrators.
```
FLASK_ENV=production
SECRET_KEY=<64+ random chars>
JWT_SECRET_KEY=<another 64+ random chars>
DATABASE_URL=mysql+pymysql://shopdb:CHANGE_ME@<mysql-host>:3306/shopdb_flask?charset=utf8mb4
CORS_ORIGINS=https://<the site's own hostname>
```
`ProductionConfig.validate()` refuses to boot if any of `SECRET_KEY`,
`JWT_SECRET_KEY`, `DATABASE_URL`, `CORS_ORIGINS` is missing or left at a dev
default. `CORS_ORIGINS` is the browser origin users hit (the IIS binding).
Generate a key: `venv\Scripts\python -c "import secrets;print(secrets.token_urlsafe(64))"`.
## 5. Initialize schema, data, plugins, admin
Run from `APP_ROOT` with the venv active and `.env` present:
```powershell
$env:FLASK_APP="shopdb"
venv\Scripts\flask db upgrade
venv\Scripts\flask seed reference-data
# Enable the plugins this site tracks (registry lives in the gitignored
# instance/plugins.json, so a fresh box starts with none enabled):
venv\Scripts\flask plugin list
venv\Scripts\flask plugin install computers
venv\Scripts\flask plugin install equipment
venv\Scripts\flask plugin install network
venv\Scripts\flask plugin install notifications
venv\Scripts\flask plugin install printers
venv\Scripts\flask plugin install usb
venv\Scripts\flask plugin install knowledgebase
venv\Scripts\flask plugin install slides
venv\Scripts\flask plugin install employees
# First admin (password is generated and printed once):
venv\Scripts\flask seed admin --username admin --email admin@yourfacility.example.com
```
(Alternatively copy the dev box's `instance/plugins.json` to `APP_ROOT\instance\`
to reproduce the exact enabled set, then just run `flask plugin upgrade-all`.)
## 6. Create the IIS site + web.config
This describes the own-site method (the app gets its own IIS site + port). To
mount the app at a subpath under an existing site instead (e.g.
`https://<host>/ops/` sharing the classic site's binding and cert), see
**docs/INSTALL-WINDOWS-IIS.md section 7b**: same web.config, but the site is a
`New-WebApplication` under the parent, `MOUNT_PATH=/ops` is set (web.config or
`.env`), and the frontend is built with `VITE_BASE_PATH=/ops/`.
1. In IIS Manager, add a new **Site** (separate from the classic ASP site):
- Physical path: `APP_ROOT`
- Binding: a free port or a dedicated hostname (e.g. `https` 443 with the
facility cert, or `http` on a test port like 8081 to start).
- App pool: No Managed Code, and an identity that can read `APP_ROOT`.
2. Copy `deploy\windows\web.config` to `APP_ROOT\web.config` and edit the paths
(`C:\shopdb-flask` -> your `APP_ROOT`). It launches
`waitress-serve --port=%HTTP_PLATFORM_PORT% wsgi:app` and sets
`FLASK_ENV=production` + `PYTHONPATH`.
3. Create `APP_ROOT\logs` for the HttpPlatform stdout log.
4. **Unlock the handler sections** (locked server-wide by default; without this
IIS returns **HTTP 500.19** "section cannot be used at this path"):
```powershell
%windir%\system32\inetsrv\appcmd unlock config /section:system.webServer/handlers
%windir%\system32\inetsrv\appcmd unlock config /section:system.webServer/httpPlatform
```
5. Grant the app-pool identity read/execute on `APP_ROOT` and modify on
`APP_ROOT\logs` (e.g. `icacls APP_ROOT /grant "IIS AppPool\<pool>:(OI)(CI)RX" /T`).
6. Recycle the app pool / restart the site.
TLS terminates at the IIS binding. The `X-Forwarded-For` URL Rewrite rule in the
web.config (real client IP for audit logs / kiosk visitor-location) is
**commented out by default** because it needs the URL Rewrite module - with it
active but URL Rewrite absent, IIS returns HTTP 500.19. Install URL Rewrite and
uncomment the `<rewrite>` block to enable it.
## 7. Smoke test
```powershell
# SPA loads:
curl.exe -k https://<host>/ # returns index.html
# API rejects an empty login with a validation error (health signal):
curl.exe -k -X POST https://<host>/api/auth/login -H "Content-Type: application/json" -d "{}"
# expect JSON containing VALIDATION_ERROR
```
Then log in through the browser as the admin from step 5 and confirm the
dashboard renders.
## 8. Redeploying as dev advances
Because this is a test instance you keep iterating on:
1. Pull/copy new code to `APP_ROOT` (rebuild `frontend/dist` on dev if the UI
changed).
2. `venv\Scripts\pip install -r requirements.txt` (if deps changed).
3. `venv\Scripts\flask db upgrade` (if new migrations).
4. Recycle the app pool.
## Troubleshooting
| Symptom | Cause / fix |
|---|---|
| Site 502 / process won't start | Check `APP_ROOT\logs\httpplatform*`. Usually a bad `processPath`, missing waitress, or `wsgi:app` not importable (set `PYTHONPATH`). |
| Boots but SQL echoes / debug on | `FLASK_ENV` not `production` (web.config env var or `.env`). |
| `flask db upgrade` error 1071 | MySQL 5.6 `[mysqld]` flags in step 3 not applied / server not restarted. |
| ConfigError on boot | A required var (SECRET_KEY / JWT_SECRET_KEY / DATABASE_URL / CORS_ORIGINS) missing or left at a dev default in `.env`. |
| Login works, CORS errors in browser | `CORS_ORIGINS` does not match the exact origin (scheme + host + port) the browser used. |
| Audit logs show 127.0.0.1 | Expected without the URL Rewrite X-Forwarded-For rule (step 6). |

32
docs/DEPLOY.md
View File

@@ -7,12 +7,12 @@ shopdb-flask is single-tenant per ADR-004. Each adopting facility runs its own s
- Docker 24+ and Docker Compose v2 (or equivalent container runtime)
- A reverse proxy with TLS termination (nginx, traefik, Caddy, GE corporate LB) -- the framework does not terminate TLS itself
- A MySQL backup destination (offsite recommended)
- Access to the GE Aerospace Gitea or a clone of the repo
- Access to the internal GE Aerospace git server, or a clone of the repo
## Step 1: Clone and configure
```bash
git clone https://gitea.proudtech.net/ge-aerospace/shopdb-flask.git
git clone <internal-git-server>/ge-aerospace/shopdb-flask.git
cd shopdb-flask
cp .env.example .env
```
@@ -73,6 +73,24 @@ any plugin-specific migrations added after the ownership cutover. Both commands
are idempotent, so re-running them is safe. See ADR-008 for why plugin schema
splits into per-plugin chains from the cutover forward.
**Lean sites (ADR-014):** the core chain creates every bundled plugin's tables,
so a site that ships only some plugins still has the others' (empty) tables. To
carry only core + chosen-plugin tables, prune the rest once, at initial
provisioning, after the two commands above:
```bash
docker compose exec api flask plugin prune-schema # dry-run, review
docker compose exec api flask plugin prune-schema --yes --force
```
It drops the tables of every plugin not installed on this site. `--force` is
needed because the core chain seeds a few plugin reference tables (default
access protocols, etc.); at first provisioning those hold only seeded defaults,
before any site data. It refuses to drop a table that holds rows without
`--force`, so it is safe to leave out of routine upgrades - run it only when
provisioning a lean site or after deliberately removing a plugin. Installing a
pruned plugin later recreates its tables automatically.
**Charset:** the schema is utf8mb4 (`utf8mb4_unicode_ci`). The docker-compose `db` service sets `--character-set-server=utf8mb4`, so the auto-created `shopdb_flask` database is utf8mb4. If you point at an external MySQL instead of the bundled container, create the database as utf8mb4 first, or it inherits the server default (often latin1) and the schema silently drifts:
```sql
@@ -98,13 +116,15 @@ docker compose exec api flask seed reference-data
- `seed settings` - writes the default Setting rows (branding, ServiceNow
integration, floor-map placeholders, search toggles, site identity). A site
overrides these later in Settings or the setup wizard.
- `seed reference-data` - creates default `Vendor`, `Location`, `BusinessUnit`,
`OperatingSystem`, `AssetStatus`, `RelationshipType` rows seeded with the
platform contract values (`partof`, `controls`, `connectedto`).
- `seed reference-data` - creates default `ModelType`, `AssetStatus`,
`LocationType`, `CommunicationType`, `OperatingSystem`, `RelationshipType` rows
seeded with the platform contract values (`partof`, `controls`, `connectedto`).
(`Vendor`, `Location`, and `BusinessUnit` are not seeded here; they come from
`seed demo`.)
## Step 5: Pick plugins to enable
The image bundles eleven plugins (computers, employees, knowledgebase, machines, measuringtools, network, notifications, printers, slides, usb, warranty). Only enabled plugins are loaded.
The image bundles thirteen plugins (computers, employees, geenforce, knowledgebase, machines, measuringtools, network, notifications, printedparts, printers, slides, usb, warranty). Only enabled plugins are loaded.
```bash
docker compose exec api flask plugin list

301
docs/DEVELOPMENT-SETUP.md Normal file
View File

@@ -0,0 +1,301 @@
# Development setup: from clone to first change
The goal of this page: a new developer clones the repo and has a working dev
site plus a change they can see in the browser, in one sitting. Reference
material lives elsewhere - naming rules in `CONTRIBUTING.md`, every config
variable in the CONFIG guide, plugin authoring in the PLUGIN docs - this is
just the on-ramp.
**Most developers here are on Windows in VS Code** - commands below are
PowerShell first, with the bash equivalent in a comment where they differ.
Install **Git for Windows** (it ships Git Bash, which VS Code and the git
hooks use to run the shell-based naming check) and **VS Code** with the
extensions this repo recommends (you'll be prompted - section 2c).
Two ways to run it. **Docker** (Docker Desktop on Windows) is the fastest to a
working site. **Manual (venv + Node)** is the daily driver - frontend
hot-reloads, backend restarts on save. Do Docker once to confirm the box is
sane, then use manual for day-to-day work.
---
## 0. Prerequisites
| Need | Version | Check |
| --- | --- | --- |
| Python | 3.14 (64-bit) - matches CI, the container image and the Windows installer wheelhouse | `python --version` |
| Node.js | 18+ | `node --version` |
| MySQL | 8.0 (or Docker, below) | `mysql --version` |
| Git | any recent | `git --version` |
On Windows, install all of them with winget (accept each license, then
reopen the terminal so PATH updates):
```powershell
winget install Git.Git # includes Git Bash (naming hook needs it)
winget install Python.Python.3.14
winget install OpenJS.NodeJS.LTS # LTS; may install v24, fine for this SPA
winget install Microsoft.VisualStudioCode
winget install Oracle.MySQL # or Docker.DockerDesktop for the DB
```
CI runs Node 20; the LTS package may be newer. This Vite/Vue frontend builds
identically across 20-24, so it does not matter. To pin exactly:
`winget install CoreyButler.NVMforWindows` then `nvm install 20; nvm use 20`.
---
## 0b. Corp network (SSL cert) - if you are behind a GE/Zscaler proxy
A proxy that inspects HTTPS (Zscaler on GE PCs) re-signs every connection
with a corporate root CA. `git`, `npm`, `pip`, and Node each keep their own
trust store and do not trust that CA by default, so downloads fail:
| Tool | Symptom |
| --- | --- |
| npm | `UNABLE_TO_GET_ISSUER_CERT_LOCALLY` |
| git | `SSL certificate problem: unable to get local issuer certificate` |
| pip | `SSLError` / `CERTIFICATE_VERIFY_FAILED` |
Fix once - export the corp root CA, point every tool at it. PowerShell
mangles multi-line pastes, so each step below is **one physical line**: paste
it, press Enter, then the next. Do not paste both at once.
```powershell
# 1) Bundle EVERY trusted root into one PEM (one line). Guessing which single cert is the proxy's is fragile; bundling all always includes it.
$sb = New-Object System.Text.StringBuilder; Get-ChildItem Cert:\LocalMachine\Root | ForEach-Object { [void]$sb.AppendLine("-----BEGIN CERTIFICATE-----"); [void]$sb.AppendLine([Convert]::ToBase64String($_.RawData,'InsertLineBreaks')); [void]$sb.AppendLine("-----END CERTIFICATE-----") }; [IO.File]::WriteAllText("$HOME\corp-root-ca.pem", $sb.ToString())
```
Confirm it has many certs (dozens, not 1):
`(Select-String "BEGIN CERTIFICATE" $HOME\corp-root-ca.pem).Count`
```powershell
# 2) Point every tool at it (one line, persistent). NODE_EXTRA_CA_CERTS also fixes Vite / npm run dev.
git config --global http.sslCAInfo "$HOME\corp-root-ca.pem"; npm config set cafile "$HOME\corp-root-ca.pem"; setx NODE_EXTRA_CA_CERTS "$HOME\corp-root-ca.pem"; setx PIP_CERT "$HOME\corp-root-ca.pem"
```
Reopen the terminal so `setx` takes effect. Quick unblock if you cannot
export right now (skips verification - use briefly, then set back):
`npm config set strict-ssl false`, `git config --global http.sslVerify false`.
---
## 1. Get the code
```powershell
git clone https://github.com/ge-aero/shopdb-flask.git
cd shopdb-flask
```
Never work on `main`. Branch for your change:
```powershell
git checkout -b feat/<short-description>
```
---
## 2a. Fast path - Docker (a working site in one command)
```powershell
copy .env.example .env
# Edit .env: set SECRET_KEY, JWT_SECRET_KEY, and the MYSQL_* passwords.
# Generate a secret: python -c "import secrets;print(secrets.token_urlsafe(64))"
docker compose up -d --build # MySQL + the app (frontend built in-image)
# Schema + platform data (idempotent, safe to re-run):
docker compose exec api flask db upgrade
docker compose exec api flask plugin upgrade-all
docker compose exec api flask seed permissions
docker compose exec api flask seed settings
docker compose exec api flask seed reference-data
docker compose exec api flask seed admin --username admin --email you@example.com
docker compose exec api flask seed demo # OPTIONAL: sample data (undo: flask seed demo-clear)
```
The app is on the port the compose file maps (see `docker-compose.yml`). Good
for a smoke test; for active development use the manual path so the frontend
hot-reloads.
---
## 2b. Manual path - venv + Node (the daily driver)
### Database
Either point at an existing MySQL 8, or bring one up with just the db service
from compose:
```powershell
docker compose up -d db # MySQL on 127.0.0.1:3306
```
Create the database + app user (skip if compose already did via env):
```sql
CREATE DATABASE shopdb_flask CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'shopdb'@'%' IDENTIFIED BY 'devpassword';
GRANT ALL PRIVILEGES ON shopdb_flask.* TO 'shopdb'@'%';
FLUSH PRIVILEGES;
```
### Backend
```powershell
python -m venv venv
venv\Scripts\Activate.ps1 # bash/mac: source venv/bin/activate
# If PowerShell blocks the activate script (execution policy), run once:
# Set-ExecutionPolicy -Scope CurrentUser RemoteSigned
pip install -r requirements-dev.txt
copy .env.example .env # bash/mac: cp .env.example .env
# Edit .env - set SECRET_KEY, JWT_SECRET_KEY, and
# DATABASE_URL=mysql+pymysql://shopdb:devpassword@127.0.0.1:3306/shopdb_flask?charset=utf8mb4
# CORS_ORIGINS=http://localhost:5173
$env:FLASK_APP = "shopdb" # bash/mac: export FLASK_APP=shopdb
flask db upgrade # core schema
flask plugin upgrade-all # per-plugin schema (ADR-008)
flask seed permissions
flask seed settings
flask seed reference-data
flask seed admin --username admin --email you@example.com # password printed once
flask seed demo # OPTIONAL: ~25 sample assets across plugins + printed parts (undo: flask seed demo-clear)
```
Enable the plugins you want visible (they install on a fresh box; some ship
disabled). To turn on everything for development:
PowerShell:
```powershell
foreach ($p in "computers","employees","machines","measuringtools","network",
"notifications","printers","slides","usb","warranty",
"knowledgebase","geenforce","printedparts") {
flask plugin install $p; flask plugin enable $p
}
```
(bash/mac: a `for p in ...; do flask plugin install "$p"; ...; done` loop.)
Run the backend ON PORT 5001 - the frontend dev server proxies `/api` and
`/static` there (a bare `flask run` uses 5000 and nothing will load):
```powershell
flask run --port 5001
```
**Leave this running.** `flask run` does not return to a prompt - that is
correct, not a hang. The server holds this terminal until you stop it. Do
NOT press Ctrl+C to move on; that kills the backend. Open the frontend in a
separate terminal (next section) and leave this one alone. Ctrl+C only when
you are done for the day.
### Frontend (a second terminal - leave the backend running)
```powershell
cd frontend
npm install
npm run dev # http://localhost:5173
```
Open http://localhost:5173, log in as `admin` with the printed password.
> Convenience: instead of two terminals you can run both under a process
> manager (pm2, honcho, foreman). Keep the backend on 5001.
---
## 2c. VS Code (turnkey)
The repo ships shared VS Code config in `.vscode/` (personal `settings.json`
stays git-ignored):
- **Recommended extensions** - on first open VS Code offers to install them
(Python + Pylance, Vue Volar, ESLint, Docker). Accept.
- **Run the dev site** - Command Palette > "Tasks: Run Task" >
**Dev site (backend + frontend)** starts both servers in parallel (backend
on 5001, frontend on 5173). Individual tasks exist too.
- **Debug the backend** - the Run panel's **Flask API (:5001)** config runs
the app under the debugger (breakpoints in routes/services, full
stepping); **Pytest (current file)** debugs the open test file.
- **The CI gate** - task **Check: naming + tests + build** runs the same
three checks CI runs, before you commit.
Prerequisite: the venv and `npm install` from 2b must be done first (the
tasks call `venv/` and `frontend/node_modules`).
---
## 3. The development loop
1. Make a change. Backend: `flask run` auto-reloads. Frontend: Vite hot-reloads.
2. Before committing, run the three gates. Easiest: in VS Code, Command
Palette > "Tasks: Run Task" > **Check: naming + tests + build**. By hand
in PowerShell:
```powershell
venv\Scripts\python -m pytest tests/ -q # backend
cd frontend; npx vitest run; npm run build; cd ..
bash scripts/check-naming-and-style.sh # naming - runs via Git Bash
```
There is NO auto-installed git hook - you run these yourself (or the
VS Code task). CI runs all three on every push and pull request
(`.github/workflows/ci.yml` on GitHub Actions; the same gate runs on the
internal server) and fails the build on a bad name, so nothing bad
reaches `main` - running them locally just saves the round trip. The
naming check is a shell script, so that one line needs Git Bash
(installed with Git for Windows).
Want it automatic? The repo ships a hook; enable it once per clone:
```powershell
git config core.hooksPath .githooks
```
Now every `git commit` runs the naming check first (Git for Windows
executes the hook under its bundled bash) and blocks the commit if a name
is wrong. Purely local convenience; CI is the real backstop.
3. Commit in small, working steps. Subject: short, present tense, plain
English; body says WHY. Read `CONTRIBUTING.md` before naming anything - the
naming hook will reject snake_case DB columns, banned shorthand, and
non-ASCII.
Seeing a change in the real app (not just tests) is the bar for "done" -
drive the actual flow in the browser.
---
## 4. Your first change (suggested)
Add a field to an existing list page, or better, build a plugin end to end:
`docs/PLUGIN-LAB-PRINTEDPARTS.md` is a literal type-along that constructs the
3D-printed-parts plugin from scratch, with the finished code on branch
`feat/printedparts-plugin` (tags `lab-stage-01`..`lab-stage-17`) as the
answer key. It touches every hook the framework has.
---
## 5. Contributing back
```powershell
git push -u origin feat/<short-description>
```
Open a Pull Request against `main` on GitHub. Describe what changed, any
plugin hooks implemented, and any contract additions (those need a version
bump + `docs/PLUGIN-HOOKS.md` update in the same PR). See the contributor
section of the plugin lab for the full review checklist.
---
## Common setup problems
| Symptom | Cause / fix |
| --- | --- |
| Frontend loads but every API call fails / CORS error | backend not on 5001 (`flask run --port 5001`), or `CORS_ORIGINS` missing `http://localhost:5173`. |
| App refuses to boot in production config | a required `.env` var (`SECRET_KEY`, `JWT_SECRET_KEY`, `DATABASE_URL`, `CORS_ORIGINS`) missing or a dev default. |
| `flask db upgrade` error 1071 (key too long) | MySQL 5.6 without the `innodb_large_prefix`/Barracuda flags; use MySQL 8 for dev. |
| Nav missing Machines/PCs/... | plugins not installed/enabled (step 2b), or the backend not restarted after enabling. |
| "No time zone found with key America/New_York" | `tzdata` not installed - `pip install -r requirements.txt` includes it. |
| npm/git/pip SSL error (`UNABLE_TO_GET_ISSUER_CERT_LOCALLY`, `unable to get local issuer certificate`) | corp proxy (Zscaler) intercepts HTTPS - point each tool at the corp root CA. See section 0b. |
| Naming hook rejects a commit | you used snake_case on a DB-mirrored field or a banned acronym - see `CONTRIBUTING.md`. |
| Plugin toggle throws an internal error | app cannot write `instance/` (the plugin registry lives there) - fix directory permissions. |

132
docs/EVENTSAVER.md Normal file
View File

@@ -0,0 +1,132 @@
# EventSaver
A rotating-image screensaver for shop-floor PCs. Idle bays show the slides
managed in ShopDB, so a safety notice or a shift message reaches the floor
without anyone walking to each machine.
It is a small C# WinForms screensaver (`.scr`), not part of the ShopDB server.
It is documented here because ShopDB serves its content and because a site
adopting the fleet tools needs to know how it is configured.
For how to deploy it at a new site, see
[ADOPTING-AT-ANOTHER-SITE.md](ADOPTING-AT-ANOTHER-SITE.md). This page is what it
is, how it decides what to show, and how to diagnose it.
## Two sources, one of them preferred
EventSaver reads `EventSaver.ini` from the directory it lives in, **on every
launch**. Retargeting it never needs a recompile.
```ini
# HTTP mode: pull the playlist from ShopDB and cache it locally.
url=https://shopdb.example.net/api/slides/feed?surface=shopfloor
# Folder mode: used only when url is blank. SMB or local path.
# folder=\\fileserver\shopfloor\tv
interval=10 # seconds per image, unless a slide carries its own time
shuffle=0 # 1 = random order, 0 = ordered
fadems=600 # crossfade length in ms, reserved (v1 hard-cuts)
```
**HTTP mode is the one to use.** Slides are managed in ShopDB (Slides), the
feed is public so the screensaver needs no credentials, and each PC caches what
it fetched under its own `LocalApplicationData`. A bay that cannot reach the
server keeps showing the last set it saw instead of going black - which matters,
because the failure would otherwise be visible to the whole floor.
**Folder mode** predates the server and stays as a fallback for a site with no
ShopDB instance yet, or for content nobody wants in the database.
> If `EventSaver.ini` is missing, or both `url` and `folder` are blank, the
> binary falls back to a path compiled into `EventSaver.cs` - and that path
> belongs to the site it was first built for. Ship the ini. A missing ini is not
> a neutral default.
## What decides the running order
| Situation | Behaviour |
|---|---|
| No `order.txt` | Every image shows, sorted by filename. Number them `001_`, `002_` to sequence. |
| `shuffle=1` | Random order, ignored when a playlist is present. |
| `order.txt` present | ONLY the listed files show, in the listed order, looped. |
`order.txt` is the strict-sequence option and is the whole playlist - a file not
listed does not appear. Per-slide duration is appended with a pipe:
```
# Lines starting with # or ; are ignored.
001_welcome.jpg
002_safety_week.jpg|15
003_quality_board.jpg
```
In HTTP mode you do not write `order.txt` by hand: ShopDB's feed carries the
order and each slide's own seconds, and EventSaver writes the equivalent
`order.txt` into its cache so the same playlist logic runs either way.
A running screensaver rescans about every 30 seconds, so a slide change reaches
the floor without touching any PC.
## Command-line switches
Windows calls a screensaver with these, and EventSaver answers all three:
| Switch | Meaning |
|---|---|
| `/s` | Show fullscreen. What Windows uses on idle. |
| `/c` | Configuration. Reports the resolved source and cache directory, and points at the ini. |
| `/p <hwnd>` | Preview pane. Deliberately a no-op - it keeps the Settings dialog happy without drawing a thumbnail nobody looks at. |
`EventSaver.scr /c` is the first diagnostic: it says which source it resolved
and where it is caching, which answers most "why is it showing the wrong thing"
questions immediately.
## Where it should and should not run
A screensaver over a live display is a defect, not a feature. The reference
fleet excludes:
- **Kiosks and lobby displays**, which show a live dashboard.
- **Metrology bays** - CMM, Genspect, Keyence, wax-trace - where an operator
watches a measurement run.
- **Bays that sleep**, where the screensaver never wins anyway.
Targeting is per PC type in the GE-Enforce manifest, or by group assignment in
Intune. Decide it deliberately: the cost of getting it wrong is a screensaver
covering something someone needed to see.
## Building it
No SDK required - it compiles with the in-box .NET Framework compiler on any
Windows 10 or 11 machine:
```
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\csc.exe ^
/target:winexe /out:EventSaver.scr ^
/reference:System.dll,System.Drawing.dll,System.Windows.Forms.dll ^
EventSaver.cs
```
Ship `EventSaver.scr` and `EventSaver.ini` together. A `.scr` is an executable
with a different extension, so it goes to `C:\Windows\System32\`.
## Diagnosing it
| Symptom | Cause to check first |
|---|---|
| Blank screen, no images | Source unreachable and cache empty. Run `/c` and open the `url` in a browser on that PC. |
| Old slides only | Feed reachable at first run, not since. The cache is doing its job; fix the network path. |
| Wrong content | Pointing at another surface, or another site's server. `/c` reports which. |
| Local ini edits revert | Correct behaviour if the config is enforced by hash. Edit the copy on the share and update its `DetectionValue`. |
| Screensaver never starts | Per-user setting missing. Check `HKCU:\Control Panel\Desktop` for `SCRNSAVE.EXE`, `ScreenSaveActive` and `ScreenSaveTimeOut`. |
The feed is a plain public endpoint, so it can always be checked from the PC
itself:
```
https://shopdb.example.net/api/slides/feed?surface=shopfloor
```
An empty `slides` array means no slides have been uploaded for that surface -
the PC is configured correctly and there is nothing to show.

118
docs/FLEET-ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,118 @@
# How the pieces fit together
ShopDB is a server. Around it are four things that run on shop-floor PCs, each
documented in its own page, and each of those pages assumes you already know how
it relates to the others. This page is that relationship.
## The shape
```
+-------------------------------+
| ShopDB (one per site) |
| Flask API + Vue UI + MySQL |
+-------------------------------+
^ ^ ^ |
reports | | | | serves
(X-API-Key) | | | v
+----------------+ | | | +------------------+
| asset reporter |---+ | | | kiosks, displays |
| on every PC | | | | (browser, HTTPS) |
+----------------+ | | +------------------+
| |
+----------------+ | | +------------------+
| backup |----------+ +----| EventSaver |
| collectors | config revisions | screensaver |
+----------------+ +------------------+
pulls the slide feed
+---------------------------+
| GE-Enforce client | pulls its manifest, applies it,
| scheduled task, SYSTEM | reports the cycle back
+---------------------------+
^ |
| v
manifest the PC's
(share or actual state
HTTPS)
```
Every arrow is one direction on purpose. **Nothing on the server reaches out to
a PC.** The server holds state and answers questions; the fleet asks. That is
what lets a bay behind a firewall, asleep, or on a different network segment be
merely out of date rather than broken - and it is why every one of these tools
degrades to "stale" instead of "failed".
## What each piece is for
| Piece | Direction | What it does | Page |
|---|---|---|---|
| **Asset reporter** | PC to server | Says what this PC is: hostname, serial, type, user, IPs, machine number. Creates the PC record the rest of the system hangs off. | [ADOPTING-AT-ANOTHER-SITE](ADOPTING-AT-ANOTHER-SITE.md) |
| **GE-Enforce client** | both | Pulls the manifest for this PC's type, makes the PC match it, reports what it installed, skipped or failed. | [GE-ENFORCE](GE-ENFORCE.md) |
| **Backup collectors** | PC to server | Post machine configuration (NTLARS/DNC, CMM, part marker, UDC) as revisions, so a controller can be restored and a change can be seen. | [BACKUP-RESTORE](BACKUP-RESTORE.md) |
| **EventSaver** | PC to server | Pulls the slide playlist and shows it when a bay is idle. | [EVENTSAVER](EVENTSAVER.md) |
| **Kiosks and displays** | PC to server | A browser in kiosk mode on a ShopDB route. No client software beyond the browser. | [GE-ENFORCE-DISPLAY](GE-ENFORCE-DISPLAY.md) |
## The three ways a PC talks to the server
**The collector API**, `POST /api/collector/*`, with `X-API-Key`. Used by the
asset reporter and the backup collectors. The key is a managed token scoped to
`collector.ingest` and nothing else, so one recovered off a bay cannot read the
asset register. Idempotent by design: the same report twice changes nothing.
See [COLLECTOR-INTEGRATION](COLLECTOR-INTEGRATION.md) and ADR-006.
**The GE-Enforce API**, `/api/geenforce/*`, with a token scoped to the fetch
scope. Manifests out, cycle reports back. Most PC types still take their
manifest from the SMB share and use this only to report; the display cohort
takes everything over HTTPS, because those PCs have no share.
**Plain public reads.** The slide feed and the kiosk dashboard routes need no
credentials at all, because a screensaver and a wall display have nowhere to
keep one. They are read-only and carry nothing a floor visitor should not see.
## Where the credentials live
Three documents each named a different place, which is what happens when a
credential is described from whichever end the author was working on. Both ends
are here.
| Credential | On the server | On the PC | Rotate by |
|---|---|---|---|
| Collector key | A managed token scoped to `collector.ingest` only. `COLLECTOR_API_KEY` in `.env` also works and is simpler for a pilot, but cannot be revoked per fleet. | `HKLM:\SOFTWARE\GE\ShopDB` value `CollectorKey` | Replacing the token file the fleet reads from, so every PC picks it up on its next cycle. Never put it in the manifest - manifests sync broadly. |
| GE-Enforce fetch token | A managed service token scoped to the GE-Enforce fetch scope | `HKLM:\SOFTWARE\GE\ShopDB` value `ApiToken` | Same path as the collector key |
| Server URL | n/a | `HKLM:\SOFTWARE\GE\ShopDB` value `BaseUrl` | Setting it once at install; nothing should overwrite a bay's own value |
| Application secrets | `.env`: `SECRET_KEY`, `JWT_SECRET_KEY`, `DATABASE_URL`, integration tokens | n/a | See [BACKUP-RESTORE](BACKUP-RESTORE.md) - losing these invalidates every issued token |
Two rules behind that table. A shop-floor PC is not a trusted place, so what it
holds is scoped to exactly what it does: a collector key that leaks buys the
reader the ability to file a PC report, not to read the asset register. And a
credential is delivered, never typed - a value that has to be entered per machine
is a value that is wrong on some machine.
## What owns what
The question that decides most arguments: **who is allowed to change this?**
- **ShopDB owns the record.** What a PC is, what it drives, what it should have.
- **The manifest owns the PC's configuration.** If GE-Enforce enforces a value,
editing it on the PC is temporary - the next cycle puts it back. That is the
feature, and it is the single most common surprise.
- **The PC owns its own identity.** Serial, hostname, installed software: the
server records what it is told rather than deciding.
Where two systems both converge state - GE-Enforce and Intune DSC, say - decide
which owns which layer before deploying both, or they will fight in a way that
is very hard to see. [ADOPTING-AT-ANOTHER-SITE](ADOPTING-AT-ANOTHER-SITE.md) has
the recommended split.
## When something looks wrong
| Symptom | The piece to look at first |
|---|---|
| A PC is missing from ShopDB entirely | Asset reporter: never ran, or no collector key |
| A PC is there but its details are old | Asset reporter: stopped running. The dashboard's "PCs not reporting" card is this |
| A PC has the wrong software | GE-Enforce: check its last cycle report, then its manifest scope |
| A bay shows the wrong thing on screen | EventSaver config, or the kiosk dispatcher - both are per PC type |
| A machine has no recent config backup | Backup collector on the controlling PC. "Backups that have stopped" on the dashboard |
| Two PCs claim one machine number | Neither: it is a claim in progress, or a typo. The dashboard card names both |
Each row is a different program on a different schedule. Knowing which one to
open is most of the diagnosis.

61
docs/GE-ENFORCE-CLIENT.md
View File

@@ -63,14 +63,17 @@ being edited in shopdb, so a half-finished edit can never reach a PC.
Each enforcement cycle, POST the result (best-effort; a failed report never
fails the cycle):
`appliedversion` is the published version the client actually ran, not the
latest one available.
```
POST /api/geenforce/report
X-API-Key: <token>
Content-Type: application/json
{
"hostname": "WJCMM01",
"hostname": "CMMPC01",
"scopename": "gea-shopfloor-cmm",
"appliedversion": 3, // the published version you actually ran
"appliedversion": 3,
"enforcerversion": "2.6",
"counts": { "installed": 1, "skipped": 3, "failed": 0, "filtered": 2 },
"results": [
@@ -88,13 +91,63 @@ POST /api/geenforce/report
present), `skipped` (detected present), `failed`, `filtered`. `selfhealed`
marks a drift correction.
- shopdb keeps the latest report per (hostname, scope, phase) plus history, and
surfaces it under Settings > Enforcement Reports.
surfaces it under GE-Enforce > Enforcement Reports.
The engine already computes these counts (`installed/skipped/failed/pcFiltered`
at the end of its main loop) and knows each entry's action; shape them into the
`results` list at the call site (`New-ShopdbReport` in the kit takes a summary
with `Installed/Skipped/Failed/Filtered` + a `Results` list).
The engine emits per-entry outcomes in PascalCase (`Name/Action/SelfHealed/
ExitCode/Message`); `New-ShopdbReport` maps every per-entry key down to the
lowercase names above (`name/action/selfhealed/exitcode/message`) before POST,
so the entire wire contract shopdb reads is lowercase. `ConvertTo-ShopdbSummary`
first normalizes whatever the engine returns (a well-formed summary, a bare
return code, `$null`, or several emitted objects) into the count/results shape
`New-ShopdbReport` expects, so a not-yet-compliant engine still produces a valid
report.
## Common-scope inheritance (opt-in, OFF by default)
By default a PC enforces its `-Scope` ALONE. Pass `-IncludeCommon` to also fetch
the fleet-wide `common` scope and merge it on top, mirroring the real
GE-Enforce.ps1 (which applies `common\manifest.json` first, then the pctype's).
When enabled, `Invoke-ShopdbEnforce.ps1` fetches `common` in addition to
`-Scope` and merges it via `Merge-ShopdbManifests`:
- entries are keyed by `Name` (case-insensitive);
- common's unique entries come first, then all pctype entries (common enforces
ahead of the pctype, as on the share);
- on a `Name` conflict the pctype entry wins (its override replaces common's).
Common is fetched over the same fail-safe path (ETag + last-known-good cache).
`-CommonScope <name>` inherits a different fleet scope; a run whose `-Scope`
already is the common scope does not merge itself.
Displays do NOT use this: the `gea-shopfloor-display` scope is self-sufficient,
so the display scheduled task omits `-IncludeCommon`. Common-merge exists for a
future share-less non-display PC that genuinely needs the fleet-wide entries
(which would first require repackaging common's SMB payloads as http/inline).
The three display subtypes (Dashboard, Lobby, 3D Print Room), selected by
`C:\Enrollment\display-type.txt`, carry their shared policy inside the display
scope itself, not via common.
## Fail-safe is observable, not silent
Any error still exits 0 - a bad web app never blocks or breaks a PC. But a fresh
display with an EMPTY cache (first boot, shopdb unreachable or the token
rejected with 401 / a TLS-trust failure) would otherwise enforce nothing
*silently*. When no manifest and no cache are available, the kit:
- writes a Windows Application event-log entry (source `ShopdbEnforce`, event id
1001, type Error) naming the scope and the reason (HTTP status or transport
error), and
- fires a best-effort report ping (counts `failed: 1`, a single
`(manifest-fetch)` result carrying the reason) so the miss surfaces under
GE-Enforce > Enforcement Reports.
The cycle still exits 0; the signal just makes the no-enforcement state visible.
## Cutover (safe, staged)
1. **Configure** the registry values on a canary PC; mint the token.
@@ -105,7 +158,7 @@ with `Installed/Skipped/Failed/Filtered` + a `Results` list).
3. **Read cutover**: drop `-ShadowMode`. The engine now runs against the
shopdb-sourced manifest; payloads still come from the share. Rollback is a
one-line revert to the share-sourced call. Keep exporting manifests from
shopdb to the share (Settings > Imaging PC Types > Export to Share) so the
shopdb to the share (GE-Enforce > Manifests > Export to Share) so the
share stays a break-glass copy.
4. **Payload migration** (optional, later): move small scripts/configs to
`http`/`inline` payloads, verified by `payloadsha256`. Big MSIs stay on SMB.

8
docs/GE-ENFORCE-DEPLOY.md
View File

@@ -69,7 +69,7 @@ selected:
```
powershell -ExecutionPolicy Bypass -File Install-GEEnforce.ps1 `
-PCType gea-shopfloor-cmm -MachineNumber 0615 -CmmVersion 2019 `
-ShareRoot \\server\share\dt\shopfloor -Site "West Jefferson" `
-ShareRoot \\server\share\dt\shopfloor -Site "the reference site" `
-ShopdbUrl https://shopdb.site.geaerospace.net -ShopdbToken shopdb_pat_xxx `
-EngineSource \\server\share\dt\shopfloor\common
```
@@ -106,6 +106,12 @@ retrofit it. `-NoTask` provisions identity + kit without registering the task.
Re-running it updates identity/config and re-registers the task in place.
For PC types that have cut over to HTTPS manifest delivery (currently
displays/kiosks), the full server-side setup, auth model (IP allowlist vs
ApiToken), and per-PC-type cutover playbook live in `geenforce-api-cutover.md`.
This doc covers what gets laid on the PC; that doc covers where the manifest
comes from.
---
## 5. The engine boundary

133
docs/GE-ENFORCE-DISPLAY.md Normal file
View File

@@ -0,0 +1,133 @@
# GE-Enforce: the gea-shopfloor-display scope
Displays are the share-less corner of the fleet. They are Entra-joined,
credential-less kiosk PCs that pull their manifest over HTTPS on port 443 and
authenticate with a read-only service PAT scoped `geenforce.fetch`, sent as
`X-API-Key`. They have no SMB share mount. The kiosk engine and the kiosk
browser are baked into the display image, not shipped over HTTPS, so the display
manifest heals POLICY / CONFIG drift only, never EXEs. It is self-sufficient and
does not inherit the fleet-wide `common` scope (see below).
## The display fetch token MUST be resource-bound
The same read-only key ships to every display (delivered by DSC, or baked into
the image), so it must not be a skeleton key for the whole content store. Mint
the display token bound to just this scope, so a leak cannot pull any other
scope's manifest or any blob by hash:
```
POST /api/apitokens
{ "name": "display fetch", "scopes": ["geenforce.fetch"],
"resourcescopes": ["gea-shopfloor-display"] }
```
With `resourcescopes` set, `GET /manifest?pctype=<other>` returns 403 and
`GET /payload/<sha>` returns 404 for any blob the display scope does not
reference. `resourcescopes` NULL (unset) = unrestricted, for back-compat with
existing service tokens. Rotate by minting a new bound token and revoking the
old one (deactivate it server-side); DSC re-delivers, or re-image.
There are three display subtypes, selected by `C:\Enrollment\display-type.txt`:
`Dashboard`, `Lobby`, and `3DPrintRoom`.
## Authoring the scope
The scope is authored programmatically by
`plugins/geenforce/seed_display_scope.py`, which builds a manifest dict and
hands it to `service.replace_scope_draft` (the same call the `import-share` CLI
uses), then attaches the inline dispatcher payload. From a Flask app context:
```python
from plugins.geenforce.seed_display_scope import seed_display_scope
seed_display_scope(publish=True) # publish=False leaves it as a draft
```
`replace_scope_draft` is an idempotent draft rebuild. `publish=True` additionally
freezes an immutable published snapshot (that step is not idempotent: it always
creates a new version).
### What the scope contains
1. Four `Registry` drift-heal entries that re-assert the Microsoft Edge kiosk
relaunch policies set at imaging by `09-Setup-Display.ps1`. Each writes the
value and detects drift with `DetectionMethod = ValueMatches` against the
same path/name, so a display that loses a policy self-heals on the next
enforce cycle with no keyboard or mouse on site:
- `RelaunchNotification = 2` (DWord, Required auto-restart)
- `RelaunchNotificationPeriod = 3600000` (DWord, 1 hour)
- `RelaunchHeadsUpPeriod = 60000` (DWord, 1 minute)
- `RelaunchWindow` (String, JSON, 02:00 start, 120 minute duration)
2. One `PS1` dispatcher, delivered inline over HTTPS. It reads
`C:\Enrollment\display-type.txt` and launches the kiosk target for the
subtype. The subtype -> route map is a data-driven table
(`DISPLAY_TYPE_TARGETS`) at the top of both the seed module and the generated
script, so targets are easy to edit. `DetectionMethod = Always` so it
re-asserts each cycle, but the script is idempotent (it skips relaunch if a
kiosk process is already serving the target URL).
### Role resolution: server first, display-type.txt fallback
The dispatcher first asks the server: `GET
/api/dashboarddefaults/display-role?fqdn=<fqdn>` (public, unauthenticated). A row
in `dashboarddefaults` keyed by the display's FQDN (IP fallback) wins and returns
the role and frontend path directly. Only when the server is unreachable or has
no mapping does the dispatcher fall back to the local `display-type.txt` map
below. To repurpose a display, edit its `dashboarddefaults` row; the change takes
effect on the next enforce cycle.
Fallback map (local file):
| display-type.txt | kiosk route | notes |
| --- | --- | --- |
| `Dashboard` | `/shopfloor` | core ShopfloorDashboard, standalone full-screen |
| `Lobby` | `/tv` | slides plugin TV dashboard (surface `lobby`) |
| `3DPrintRoom` | `/parts-kiosk` | **PLACEHOLDER, TODO-confirm** printedparts parts kiosk route; confirm the real 3D-print-room target with the floor team before publishing to production displays |
### Dashboard-defaults FQDN keying
`dashboarddefaults` rows were historically keyed by IP. Migration
`7d31_dashboarddefault_fqdn` added an `fqdn` column; resolution is now FQDN-first
with IP as fallback (`_resolve_default` in
`shopdb/core/api/dashboarddefaults.py`). FQDNs are stored lowercase. This
survives DHCP churn on kiosk subnets. `POST /api/dashboarddefaults` accepts
`fqdn`, `ipaddress`, `displayrole` (`dashboard`|`lobby`|`partskiosk`),
`businessunitid`, and `description`; `displaypath` is not stored but derived from
the role (`DISPLAY_ROLE_PATHS`). Two public read endpoints consume it:
`/api/dashboarddefaults/display-role` (dispatcher) and
`/api/dashboarddefaults/visitor-location` (lobby business-unit lookup). The
server derives a display's FQDN from its reported BIOS serial as
`F<serial>.<domain>` (`derive_display_fqdn`, domain from the `display_fqdn_domain`
setting); the dispatcher in `plugins/geenforce/seed_display_scope.py` builds the
same FQDN client-side for its lookup.
### Legacy autostart self-heal
The dispatcher also cleans up after the old GE Aerospace Dashboard / Lobby
Display Inno installers, which planted autostarts (a Public-Desktop `.lnk`, an
all-users Startup `.lnk`, and an `HKLM ...\CurrentVersion\Run` value) that
relaunch Edge at now-dead URLs (`/shopfloor-dashboard/`, `/tv-dashboard/`) and
white-screen. The 32-bit installer's Run value was WOW64-redirected into
`Wow6432Node`, which is why it survived earlier cleanup. Every enforce cycle the
dispatcher sweeps both registry views, all loaded user hives, Run/RunOnce/policy
Run keys, and every per-user and common Startup folder, matching by legacy name
and by the old URLs, then kills any old-URL Edge. The kiosk shortcut it writes is
a direct Edge shortcut (no launcher or VBS). The fix ships by re-publishing this
code-authored scope (`seed_display_scope(publish=True)`), not an import-share.
`<imaging-share>/github/find-legacy-kiosk-autostart.ps1` is a read-only locator for
stragglers.
## Self-sufficient: displays do NOT inherit common
The `gea-shopfloor-display` scope carries everything a display enforces. It does
NOT inherit the fleet-wide `common` scope. Displays run the enforcer with
common-merge off (the client default; common-merge is opt-in via
`Invoke-ShopdbEnforce.ps1 -IncludeCommon`), so `common`'s SMB-backed fleet
entries (Adobe, Oracle, OpenText, Defect Tracker, EventSaver, printer map,
self-update, asset-reporting, ...) never reach a share-less display.
This was a deliberate decision: a display needs none of common's software, and
inheriting common would have forced repackaging every SMB `common` payload as
`http`/`inline` for a share-less box. Keeping the display scope self-sufficient
avoids all of that. If a future non-display share-less PC genuinely needs the
fleet-wide entries, that is what `-IncludeCommon` plus a per-entry SMB->http
payload conversion would be for -- but displays do not use it.

17
docs/GE-ENFORCE.md
View File

@@ -231,9 +231,9 @@ because it is a full management surface.
### 4.1 Manifests - authoring (GE-Enforce > Manifests)
- **PC Types (scopes):** each imaging PC type is a row; add/edit/delete. (The
scope carries an optional `computertypeid` reference field, but the collector's
imaging-pc-type -> ComputerType mapping is configured separately at
Settings > Collector PC Types.)
scope's `computertypeid` field is now the imaging-pc-type -> ComputerType
mapping mechanism, per ADR-012. The old Settings > Collector PC Types page is
retired; there is nothing to configure there anymore.)
- **Entries:** an ordered list (Up/Down = the execution-order contract). Add/Edit
opens a typed form: the payload fields switch on `Type` (MSI shows Installer +
InstallArgs, PS1 shows Script + Args, File shows Source + Destination, Registry
@@ -288,8 +288,15 @@ The engine sources the manifest and reports results using the reference kit in
(BaseUrl + a `geenforce.fetch`/`geenforce.report` service token). See
`docs/GE-ENFORCE-CLIENT.md` for the fetch/report contract, the last-known-good
cache, shadow mode, and the staged cutover from share-sourced to shopdb-sourced
manifests. Until that cutover, the client only REPORTS; the manifest still comes
from the share via Export to Share (4.2).
manifests.
The cutover from share-sourced to shopdb-sourced manifests is per PC type. The
**displays/kiosks cohort has cut over**: share-less display PCs fetch their
manifest and payloads entirely over HTTPS (see `docs/geenforce-api-cutover.md`
and `docs/GE-ENFORCE-DISPLAY.md`). All other fleet PC types (cmm, collections,
keyence, genspect, heattreat, partmarker, nocollections, common) still enforce
from the SFLD SMB share via Export to Share (4.2) and only REPORT to shopdb. The
playbook for moving the next PC type is `geenforce-api-cutover.md` section 11.
---

34
docs/IMPORT-ADOPTION.md
View File

@@ -1,5 +1,30 @@
# Importing a site's legacy data
## Two routes in, and which one you want
**If the site has a spreadsheet and no developer**, use the CSV import. It is
the common case, and it needs nothing beyond the templates:
```bash
flask csv templates --out csv-templates # generated from the live schema
# fill them in
flask csv import --dir csv-templates # checks only, changes nothing
flask csv import --dir csv-templates --commit
```
Foreign keys take a NAME, not an id - write `Bay 3`, not `locationid=7`. The
importer resolves them, including across files in the same run, and a name it
cannot find is reported with the line, the column and the value. Nothing is
written unless every row passes, and re-running an edited file updates rows
rather than duplicating them. See [CSV-IMPORT.md](CSV-IMPORT.md).
**If the site has a source database to read from**, and someone able to script
against it, the HTTP import API below is the better tool: it carries the whole
history, preserves original timestamps, and handles relationships the CSV set
does not model.
---
Every adopting site has its own source database - it will not match another
site's schema. So the import is split in two layers:
@@ -11,7 +36,7 @@ site's schema. So the import is split in two layers:
2. **A per-site loader is thin glue.** It reads *your* source database and POSTs
to those endpoints. Nobody runs another site's loader - you copy the pattern.
The West Jefferson loader in `scripts/site_imports/wjf/` is reference
The the reference site loader in `scripts/site_imports/wjf/` is reference
implementation #1. Read it alongside this guide.
## The shape of a loader
@@ -25,6 +50,11 @@ implementation #1. Read it alongside this guide.
- `run.py` - ordered `stage_*` functions. Each reads a slice of the source,
POSTs it, and records the crosswalk later stages resolve foreign keys against.
Post-import fixups that re-point existing assets (example:
`scripts/reclassify_servers_to_network.py`, servers imported as PCs moved to
network devices in place) belong in the site loader's verify stage, not in the
stable API layer.
### Stage order matters
Reference/lookup tables first (so foreign keys resolve), then the entity hub,
@@ -59,7 +89,7 @@ onboarding path.
UI spot-check (log in, eyeball the lists / map / a detail page).
5. Only then point a real instance at the imported database.
## What the WJ loader demonstrates
## What the reference loader demonstrates
- Fanning one legacy "machine" table out to the flask asset types
(computer/machine/network/measuring-tool) by a routing rule, with the

13
docs/IMPORT-API.md
View File

@@ -1,7 +1,7 @@
# Import API: migrating the classic ASP shopdb through HTTP alone
This is the operator manual for importing the legacy Classic-ASP shopdb database
(`prodscratch` on the dev MySQL container) into shopdb-flask using ONLY the HTTP
(`legacyshopdb` on the dev MySQL container) into shopdb-flask using ONLY the HTTP
API. No direct writes to the `shopdb_flask` database are needed or wanted: every
row is created through a documented endpoint so authorization, validation,
auditing, and plugin hooks all run exactly as they do for a human operator.
@@ -234,10 +234,13 @@ upload them after import via `POST /api/models/<modelid>/image` (multipart
| `pctype` | `POST /api/computers/types` | `typename` -> `computertype`, `description` | `computertype` |
| `subnettypes` | (see subnets) | used as `subnettype` string on subnets | - |
| `subnets` | `POST /api/network/subnets` | `cidr`, `description` -> `name`/`description`, `vlan` -> create VLAN first (`POST /api/network/vlans`) then `vlanid`, `subnettypeid` -> `subnettype` name | `cidr` |
| `dashboarddefaults` | `POST /api/dashboarddefaults` | `ipaddress` -> `ipaddress`, `businessunitid` (remapped), `description` | `ipaddress` |
| `dashboarddefaults` | `POST /api/dashboarddefaults` | `fqdn` (preferred key, stored lowercase), `ipaddress` (fallback key), `displayrole` (`dashboard`/`lobby`/`partskiosk`), `businessunitid` (remapped; only the `dashboard` role uses it), `description` | `fqdn`, else `ipaddress` |
| `controllertypes` | remap into `vendors` + `models` | e.g. "Fanuc" -> a Vendor; the controller model -> a Model; then set `controllervendorid`/`controllermodelid` on the machine | - |
| `comstypes` | `communicationtypes` (seeded, no API) | ensure `flask seed reference-data` created IP/Serial/USB/... before importing comms | - |
Resolution at runtime is FQDN-first with IP fallback (migration
`7d31_dashboarddefault_fqdn`); import both when the legacy source has them.
Note on communication types: the classic `comstypes.typename` values
(IP, Serial, Network_Interface, USB, Parallel, VNC, FTP, DNC) correspond to the
seeded `communicationtypes.comtype`. They are created by the reference-data seed,
@@ -362,7 +365,7 @@ dispositions below are DECIDED, not open questions.
### DECIDED: skip (structure only or low value)
- **`compliance`, `compliancescans`** - 0 rows in `prodscratch`. No data to
- **`compliance`, `compliancescans`** - 0 rows in `legacyshopdb`. No data to
migrate; a future compliance plugin would own them. Skip.
- **`ednc_installations`, `ednc_logs`** - 0 rows, and they belong to the eDNC
tooling rather than the asset catalog. Skip.
@@ -479,7 +482,7 @@ if __name__ == "__main__":
args = parser.parse_args()
client = ImportClient(args.token, dryrun=args.dry_run)
# read legacy rows from prodscratch (read-only) and call the import_* fns
# read legacy rows from legacyshopdb (read-only) and call the import_* fns
# in the order of section 2, keeping a legacy-id -> new-id map as you go.
```
@@ -494,7 +497,7 @@ need it to remap foreign keys (a machine's `businessunitid`, a checkout's
After each phase, compare counts. Legacy side (read-only), for example:
```bash
docker exec dev-mysql mysql -uroot -prootpassword prodscratch \
docker exec dev-mysql mysql -uroot -p"$MYSQL_ROOT_PASSWORD" legacyshopdb \
-e "SELECT COUNT(*) FROM vendors;"
```

41
docs/INSTALL-WINDOWS-IIS.md
View File

@@ -1,5 +1,13 @@
# ShopDB - Windows + IIS install runbook
> **Not the route for a new site.** Sister sites install from the Windows
> installer - one `.exe`, no manual IIS work: **[INSTALL-WINDOWS.md](INSTALL-WINDOWS.md)**.
>
> This document is the **manual** procedure, kept for reference and for
> hand-built servers that predate the installer. Note that the installer will not
> adopt a server built this way without `-AdoptExisting`, on purpose.
A step-by-step, **tested** install for a new site on Windows Server / Windows 11
with IIS in front of the Flask app (HttpPlatformHandler -> waitress), backed by
MySQL. This runbook was validated end to end on a win11 + IIS + MySQL 5.6 box.
@@ -13,9 +21,9 @@ lives). Run PowerShell as Administrator.
| Need | Notes |
| --- | --- |
| **Python 3.12** (64-bit) | `python --version` |
| **Python 3.14** (64-bit) | `python --version` |
| **IIS** with **HttpPlatformHandler** | https://www.iis.net/downloads/microsoft/httpplatformhandler (direct MSI: `download.microsoft.com/download/8/1/3/813AC4E6-9203-4F7A-8DD5-F3D54D10C5CD/httpPlatformHandler_amd64.msi`) |
| **MySQL 5.7+/8.0** (or 5.6 with the flags in step 1) | reachable from the app host |
| **MySQL 8.4 LTS** (standard for new installs) | reachable from the app host. 8.0 reached end of life in April 2026 and no longer ships a standalone server MSI. 5.7+ still works on an existing server; 5.6 needs the flags in step 1. |
| URL Rewrite (optional) | only for the real-client-IP rule; skip it and the app still runs |
The app itself pulls in `waitress` and `tzdata` from `requirements.txt` (step 4).
@@ -113,10 +121,15 @@ venv\Scripts\flask seed settings
# enable the plugins this site tracks (registry is empty on a fresh box).
# usb + employees install DISABLED by default - enable them later in the wizard
# if the site wants those (they create extra tables).
foreach ($p in "computers","equipment","network","notifications","printers","knowledgebase","slides","warranty") {
foreach ($p in "computers","machines","network","notifications","printers","knowledgebase","slides","warranty") {
venv\Scripts\flask plugin install $p
}
# EVERY plugin owns its own Alembic chain (ADR-008), and `flask db upgrade`
# above ran only the core one. Skipping this is how a new column reaches
# production as a 1054 "Unknown column" error on the page that uses it.
venv\Scripts\flask plugin upgrade-all
# first admin (password generated + printed once - store it):
venv\Scripts\flask seed admin --username admin --email admin@yourfacility.example.com
```
@@ -151,6 +164,8 @@ Two supported deployment methods:
```powershell
icacls APP_ROOT /grant "IIS AppPool\shopdbflask:(OI)(CI)RX" /T
icacls APP_ROOT\logs /grant "IIS AppPool\shopdbflask:(OI)(CI)M" /T
mkdir APP_ROOT\instance 2>NUL
icacls APP_ROOT\instance /grant "IIS AppPool\shopdbflask:(OI)(CI)M" /T
```
4. **Unlock the handler sections** (locked server-wide by default; without this
IIS returns **HTTP 500.19**):
@@ -227,6 +242,23 @@ each gets its own site, app pool, port, and venv.
---
## Redeploying a hand-built server
A hand-built server has no installer to run, so an update is done by hand in the
same order the installer would:
1. Copy the new code to the application root, rebuilding `frontend/dist` first
if the UI changed.
2. `venv\Scripts\pip install -r requirements.txt`, if dependencies changed.
3. `venv\Scripts\flask db upgrade` for the core chain, then
`venv\Scripts\flask plugin upgrade-all` for the plugin chains. Both, every
time - the second is the one people skip, and it surfaces days later as a
1054 "Unknown column".
4. Recycle the application pool.
Take a database backup before step 3. The installer does this automatically and
restores from it when a migration fails; by hand, it is yours to remember.
## Troubleshooting
| Symptom | Cause / fix |
@@ -236,8 +268,9 @@ each gets its own site, app pool, port, and venv.
| IIS **500.52** after enabling the rewrite block | `allowedServerVariables` locked at server level - `appcmd unlock config -section:system.webServer/rewrite/allowedServerVariables`. |
| Audit log shows only **127.0.0.1** with the rewrite block active | waitress strips untrusted proxy headers - `--trusted-proxy=127.0.0.1 --trusted-proxy-headers=x-forwarded-for` missing from the waitress `arguments`. |
| **500** with an empty HttpPlatform log | app-pool identity can't read `APP_ROOT` / run the venv (step 7.3), or `.env` missing/invalid. |
| "internal error" toggling plugins, or uploads fail | app pool cannot WRITE `APP_ROOT\instance` (plugin registry, logos, photos, files live there) - step 7.3 grants it Modify. |
| "No time zone found with key America/New_York" | `tzdata` not installed (`pip install tzdata`). |
| Nav missing Equipment/PCs/... | plugins not installed (step 6 `flask plugin install`), or site not recycled. |
| Nav missing Machines/PCs/... | plugins not installed (step 6 `flask plugin install`), or site not recycled. |
| Method B: blank page / assets 404 under `/ops` | frontend `dist` built without `VITE_BASE_PATH=/ops/` (step 7b.1). |
| Method B: SPA loads but every API call 404s | `MOUNT_PATH` unset or not matching the Application alias (step 7b.3). |
| ConfigError on boot | a required `.env` var missing or left at a dev default. |

257
docs/INSTALL-WINDOWS.md Normal file
View File

@@ -0,0 +1,257 @@
# Install ShopDB-Flask on Windows Server
**This is the route for a new site.** You run one `.exe`, answer a few questions,
and get a working application. Nothing here needs an internet connection, and you
do not need to know IIS, Python or MySQL.
If you are looking after an existing hand-built server, see
[INSTALL-WINDOWS-IIS.md](INSTALL-WINDOWS-IIS.md) instead - that is the manual
procedure, and the installer will not adopt a server it did not build.
---
## Before you start
You need **four things**. The installer supplies everything else.
| | What | How to check |
|---|---|---|
| 1 | Windows Server 2019 or newer | `winver` |
| 2 | The **IIS Web Server role** installed | Server Manager -> Manage -> Add Roles -> Web Server (IIS). Or run the PowerShell below. |
| 3 | Administrator rights on the box | Right-click PowerShell -> "Run as administrator" works |
| 4 | A decision about the database - see [Which database?](#which-database) | - |
Installing IIS, if it is missing (this needs no internet):
```powershell
Install-WindowsFeature -Name Web-Server -IncludeManagementTools
```
### Sizing the server
Small, and the reasons are worth stating so a site does not over-provision.
ShopDB serves one plant's own staff, not the internet: a busy instance handles a
few dozen concurrent users and a fleet reporting a few hundred PCs a day, and
both are trivial loads. What actually consumes resources is MySQL and the
uploads.
| | Minimum | Comfortable | Why |
|---|---|---|---|
| Disk | 5 GB free (the installer refuses below this) | 40 GB | The application and Python are about 1 GB. The rest is MySQL, config-backup revisions, and uploads - branding, floor-plan blueprints, application installers and warranty proofs, which are the only ones that grow without bound. |
| RAM | 4 GB | 8 GB | MySQL wants most of it. The application itself is a Python worker per request. |
| CPU | 2 cores | 4 cores | Reports and the map are the heaviest reads and are cached. |
A VM is entirely normal. The one thing worth insisting on is that the database
directory and the backup directory are not on the same disk as anything you
would be sad to fill up.
### Getting the installer
The `.exe` is a release asset on the GitHub repository, not a file in it - it is
around 240 MB, which is well past what a repository accepts and well inside what
a release asset does. Take it from the release page for the version you want,
and keep it: you need the previous one to roll a release back, and you cannot
download it from your own server.
Every build is stamped with its version and ships a CycloneDX SBOM onto the
server, so `shopdb-admin.ps1 verify -Path <name>` answers "does this server
carry component X" without a network call - which is the question that arrives
on a Friday afternoon after a CVE announcement.
### If you get stuck
Raise an issue on the repository. Include the output of
`shopdb-admin.ps1 check -Json`, which reports version, IIS and pool state,
database reachability, Python version and installed plugins, and contains no
secrets - it is the single most useful thing you can attach, and it is designed
to be pasted.
---
The installer **checks all of this before it changes anything**, and it will not
let you continue until the check passes. You do not have to get it right first
time.
### Which database?
Two options. Pick before you start, because they ask different questions.
- **Use the bundled MySQL** - the installer puts MySQL 8.4 LTS on this server and
creates the database for you. Choose this when the server has no database
today. Simplest option, nothing to arrange in advance.
- **Use an existing MySQL** - the database already exists somewhere, and you have
a hostname, a database name, a username and a password for it. Choose this if
your site already runs MySQL, or a DBA looks after it.
If you are unsure: if nobody has given you database credentials, you want the
bundled option.
### SQL for your DBA (existing-MySQL option only)
The installer does not create the database or the user - it never needs
administrative rights on your database server. Ask your DBA to run:
```sql
CREATE DATABASE shopdb_flask CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'shopdb'@'%' IDENTIFIED BY '<a password you choose>';
GRANT ALL PRIVILEGES ON shopdb_flask.* TO 'shopdb'@'%';
FLUSH PRIVILEGES;
```
The `utf8mb4` charset matters - the default on older servers is `latin1`, and a
latin1 database mangles any non-ASCII text you store.
---
## Installing
1. Copy the installer `.exe` onto the server. It is one file and needs no
network.
2. **Right-click it -> Run as administrator.** Without this it cannot configure
IIS, and it will tell you so.
3. Work through the wizard. The pages are:
| Page | What it wants | If unsure |
|---|---|---|
| **Server check** | Nothing - it reports what it found | Fix anything red, then "Check again". You cannot continue while something is red, and nothing has been changed yet. |
| **Features** | Which parts of the product this site uses | The defaults are fine. You can add more later; removing needs a new installer. |
| **Database** | Bundled or existing - see above | Bundled |
| **Database details** | Host, port, name, user, password | Only asked for the existing-database option |
| **Address** | How people reach the site | See [Own address or subpath?](#own-address-or-subpath) |
| **Client addresses** | Whether a proxy sits in front | See [Client addresses](#client-addresses) |
| **Location** | Where to install | `C:\shopdb-flask` is fine |
4. The install takes a few minutes. Most of it is Python and the database schema.
5. At the end you get the address to open. **Write it down** - it is also on the
Start Menu as "Open ShopDB-Flask".
### Own address or subpath?
- **Its own address** - `http://yourserver:8090/`. Choose this on a server that
is not already running a website. Simplest.
- **Under this server's existing address** - `http://yourserver/shopdb/`. Choose
this when the server already serves something else and you do not want a second
port or a new DNS name. This is what the reference site uses.
You cannot change your mind later without re-running the installer, because the
web interface has the address compiled into it.
### Client addresses
The application records who connects, and some features decide what to show based
on it. The wizard asks one question:
- **Clients connect to this server directly** - the normal answer. Pick this
unless you know otherwise.
- **A proxy or load balancer sits in front** - pick this only if your network
team has told you traffic reaches this server through something else first.
Getting this wrong is not dangerous, but the site will record every visitor as
coming from the server itself, and features that depend on location will not
work. It can be changed later by re-running the installer.
---
## First login
Open the address the installer gave you. With no users in the database yet, the
page offers to **create the first administrator**, then runs a short setup wizard
for site details, features and the floor map.
That first account is a normal administrator account. Use a real password -
this is the account that creates everyone else.
---
## Did it work?
From the Start Menu, open **ShopDB-Flask Console** and pick option 1, or:
```powershell
cd C:\shopdb-flask
.\shopdb-admin.ps1 status
```
You want to see the site started, the pool started, and `responding : yes`.
Day-to-day tasks - restarting, backups, logs, upgrades - are in
[OPERATE-WINDOWS.md](OPERATE-WINDOWS.md).
---
## If the install fails
**The server is left part-configured.** Whatever had been done before the failure
is still there. That is deliberate: it means re-running is able to pick up where
it stopped.
1. Read the error. It names the cause and what to do about it.
2. Fix that, then **run the same installer again**. Re-running is safe - it skips
what is already done and does not touch your database or `.env`.
3. If you would rather start clean, remove it from **Settings -> Apps** first.
The full log is at:
```
C:\ProgramData\ShopDB-Flask\logs\shopdb-install-<date>.log
```
It records every step, including everything that was created. Send this if you
need help.
### Getting help from an AI assistant
These installs are often done with an assistant open in another window. Give it
real state rather than a description:
```powershell
.\shopdb-admin.ps1 check -Json
```
That prints one structured block covering the version, how the site is published,
IIS state, database reachability, Python version, installed features and any
errors. Paste it in. **It contains no passwords.** The install log is also safe
to share - the installer keeps secrets out of it deliberately.
Offline API reference for this server is served at `/api/docs` on the site
itself, and `docs\` in the install directory holds these runbooks.
---
## Upgrading
Run a newer installer over the top. It:
- backs the database up first, **verifies the dump is complete**, and refuses to
continue if it cannot;
- restores from that backup if the schema migration fails;
- refuses to install an **older** build over a newer one;
- keeps your `.env`, your data and your `web.config`.
Nothing else is required. See [UPGRADE.md](UPGRADE.md).
> **Before your first upgrade:** confirm `mysqldump` is available - the console's
> health check reports it. Without it the pre-upgrade backup is skipped, and that
> is the one you would want if a migration went wrong. It ships with the bundled
> database option; for an existing remote database, ask for `mysqlclient\` to be
> included in your installer bundle.
---
## Removing it
**Settings -> Apps -> ShopDB-Flask**, or Add/Remove Programs.
That removes the website, the application pool, the firewall rule and the
application directory. It deliberately **does not** drop the database and does
not uninstall MySQL, so your data survives.
Take a backup first: `.\shopdb-admin.ps1 backup`
---
## Notes for the person who builds the installer
Building a bundle for a site is a separate job, documented in
[../deploy/windows/installer/README.md](../deploy/windows/installer/README.md).
Sites receive a finished `.exe`; they do not build one.

241
docs/OPERATE-WINDOWS.md Normal file
View File

@@ -0,0 +1,241 @@
# Running ShopDB-Flask on Windows Server
Day-to-day operation of a site installed with the Windows installer. If you are
installing for the first time, start with [INSTALL-WINDOWS.md](INSTALL-WINDOWS.md).
Everything here goes through one tool, installed alongside the application:
```
C:\shopdb-flask\shopdb-admin.ps1
```
The Start Menu folder **ShopDB-Flask** has shortcuts for the common tasks. Run it
with no arguments for a menu, or pass a command directly. It needs
Administrator - it will ask, except for `open`.
---
## The commands
| Command | What it does | Safe at any time |
|---|---|---|
| `status` | Is it published, running, responding; database and table count | yes |
| `restart` | Recycles the application pool. **Use this after any config change.** | yes - drains requests rather than cutting them off |
| `stop` / `start` | Takes the site down / brings it back | yes, but `stop` makes it unavailable |
| `logs` | Last lines of the application and install logs | yes |
| `check` | Full health check | yes |
| `check -Json` | The same, machine-readable - see [Getting help](#getting-help) | yes |
| `verify` | Which build this is, and whether what is installed still matches it | yes |
| `sessions` | IIS worker processes and memory | yes |
| `plugins` | Which features are installed, which are available | yes |
| `add-plugin -Path <name>` | Turns on a feature this build ships | changes the site; restarts it |
| `backup [-Path <dir>]` | Writes a verified `.sql` dump | yes, but see below |
| `open` | Opens the site in a browser | yes |
Examples:
```powershell
.\shopdb-admin.ps1 status
.\shopdb-admin.ps1 restart
.\shopdb-admin.ps1 backup D:\backups
.\shopdb-admin.ps1 verify -Path leaflet
```
---
## Backups
```powershell
.\shopdb-admin.ps1 backup
```
Writes to `C:\ProgramData\ShopDB-Flask\backups` unless you pass a directory. The
dump is **verified complete** before it is reported as good - a truncated backup
is deleted rather than left to be discovered later.
Two things to know:
- **The dump contains everything, including user password hashes.** The directory
is locked to Administrators and SYSTEM. Keep it that way, and treat copies as
sensitive.
- **Store it off this server.** A backup on the server does not survive the
server.
An upgrade takes its own backup automatically, before it touches the schema.
Restore, and the Linux/Docker equivalents, are in
[BACKUP-RESTORE.md](BACKUP-RESTORE.md).
---
## Upgrading
Run a newer installer over the top. Nothing else. It backs up first, refuses to
go backwards, and restores if a migration fails. See
[UPGRADE.md](UPGRADE.md).
---
## Adding a feature
```powershell
.\shopdb-admin.ps1 plugins # what is here
.\shopdb-admin.ps1 add-plugin -Path warranty # turn one on
```
Only features **shipped in this build** can be added. Each site's installer is
built for that site's chosen feature set, so a feature nobody asked for is not on
the server at all - adding it means a new installer built from an updated
profile. `plugins` shows you which is which.
---
## When something is wrong
Work down this list.
**1. Is it actually down?**
```powershell
.\shopdb-admin.ps1 status
```
`responding : NO` with the pool `Started` usually means the application failed to
start, not that IIS is broken.
**2. What does it say?**
```powershell
.\shopdb-admin.ps1 logs
```
Application logs are in `C:\shopdb-flask\logs`, install logs in
`C:\ProgramData\ShopDB-Flask\logs`.
**3. Try a restart.** It fixes anything that is a stuck worker, and tells you
immediately if it is not:
```powershell
.\shopdb-admin.ps1 restart
```
**4. Check the database is reachable** - `status` reports the host and whether it
could count tables. A site that starts but shows no data is usually a database
problem, not an application one.
**5. Confirm nothing has drifted:**
```powershell
.\shopdb-admin.ps1 verify
```
This flags packages that no longer match what shipped - which usually means
somebody ran a `pip install` on the server by hand.
---
## Getting help
Give an assistant real state rather than describing the symptom:
```powershell
.\shopdb-admin.ps1 check -Json
```
One structured block: version, how the site is published, IIS and pool state,
whether it responds, database host and reachability, Python version, installed
features, and any errors. **It contains no passwords** and is safe to paste into
a chat window or a ticket.
The install log is also safe to share - secrets are deliberately kept out of it.
Offline reference on the server itself:
- `/api/docs` on the site - the full API reference, self-hosted, no internet.
- `C:\shopdb-flask\docs\` - these runbooks.
- `C:\shopdb-flask\sbom.cdx.json` - every component this build contains.
---
## Answering "are we affected by this vulnerability?"
The server carries its own bill of materials, so this does not need the build box
or an internet connection:
```powershell
.\shopdb-admin.ps1 verify -Path <component-name>
```
It reports whether the component is here, at what version, and whether it
actually **ships** or is only used to build the software. Example:
```
matches for 'leaflet':
leaflet 1.9.4 SHIPPED
```
Nothing found means this server does not carry it.
---
## Adding HTTPS
**The installer publishes over HTTP.** It has no certificate to use and no way to
get one on an air-gapped server, so it does not pretend otherwise. On an internal
network behind the site firewall that is often accepted; confirm it against your
own policy rather than assuming.
If you installed **under an existing site** (the subpath option) and that site
already has a certificate, you are already on HTTPS - nothing to do.
For a site of its own, once you have a certificate in the machine store:
```powershell
Import-Module WebAdministration
# 1. Add the binding. Get the thumbprint from the certificate you imported.
New-WebBinding -Name shopdb-flask -Protocol https -Port 443
$cert = Get-ChildItem Cert:\LocalMachine\My | Where-Object { $_.Subject -like '*yourserver*' }
Get-Item "IIS:\SslBindings\0.0.0.0!443" -EA SilentlyContinue | Remove-Item -EA SilentlyContinue
New-Item "IIS:\SslBindings\0.0.0.0!443" -Value $cert
# 2. Open the port.
New-NetFirewallRule -DisplayName "shopdb-flask 443" -Direction Inbound `
-Protocol TCP -LocalPort 443 -Action Allow
```
Then **update `CORS_ORIGINS` in `C:\shopdb-flask\.env`** to the `https://` address
and restart:
```powershell
.\shopdb-admin.ps1 restart
```
That last step is not optional. `CORS_ORIGINS` is an exact origin match, so a
site reached over `https://` while `.env` still says `http://` loads the page and
then fails every data request - which looks like the application is broken rather
than a configuration mismatch.
## Where things live
| | |
|---|---|
| Application | `C:\shopdb-flask` |
| Configuration and secrets | `C:\shopdb-flask\.env` (locked down - do not loosen) |
| Application logs | `C:\shopdb-flask\logs` |
| Install logs | `C:\ProgramData\ShopDB-Flask\logs` |
| Backups | `C:\ProgramData\ShopDB-Flask\backups` |
| Bill of materials | `C:\shopdb-flask\sbom.cdx.json` |
| Which build this is | `C:\shopdb-flask\.installed-version` |
If the bundled MySQL was installed, its generated root password was written once
to `C:\ProgramData\ShopDB-Flask\mysql-root-password.txt`. **Move it into your
password manager and delete that file.** It cannot be recovered.
## See also
- [UPDATES-WINDOWS.md](UPDATES-WINDOWS.md) - what future updates, bug fixes and
security releases will look like, including downtime and the effect on other
sites on the same IIS server
- [BACKUP-RESTORE.md](BACKUP-RESTORE.md) - what to back up and how to restore
- [INSTALL-WINDOWS.md](INSTALL-WINDOWS.md) - installing a new site

72
docs/PLUGIN-EXTERNAL-REPO.md
View File

@@ -7,6 +7,12 @@ the framework ships a bundled set, and you drop your own plugin into
`<framework>/plugins/<name>/` by clone, submodule, or symlink. No pip packaging
is required for v1 (pip distribution is deferred to v2 per ADR-003).
> **Windows / VS Code:** command examples use the Linux venv path
> `venv/bin/python`; on Windows use `venv\Scripts\python` and
> `$env:FLASK_APP="shopdb"` (not `export`). Full Windows onboarding:
> [DEVELOPMENT-SETUP](DEVELOPMENT-SETUP.md).
If you have not written a plugin before, start with
[PLUGIN-QUICKSTART.md](PLUGIN-QUICKSTART.md) and the hook reference in
[PLUGIN-HOOKS.md](PLUGIN-HOOKS.md). This document only covers the parts that are
@@ -63,18 +69,23 @@ live. The loader discovers a symlinked directory the same as a real one.
```bash
# 1. Clone the framework and your plugin repo side by side.
git clone https://gitea.proudtech.net/ge-aerospace/shopdb-flask.git
git clone https://gitea.proudtech.net/wjsf/wjsf-shipping.git
git clone https://github.com/ge-aero/shopdb-flask.git
git clone https://github.com/ge-aero/wjsf-shipping.git
# 2. Symlink your repo into the framework's plugins/ directory.
# The link name is the plugin name from your manifest.json.
cd shopdb-flask
ln -s ../../wjsf-shipping plugins/shipping
# (use an absolute path if you prefer: ln -s "$(pwd)/../wjsf-shipping" plugins/shipping)
#
# Windows: use a directory junction instead of ln -s. In an ADMIN prompt
# (or with Developer Mode on) from the shopdb-flask dir:
# mklink /D plugins\shipping ..\..\wjsf-shipping
# The plugin loader treats a junction the same as a real directory.
# 3. Set up the framework as usual.
python3 -m venv venv
venv/bin/pip install -r requirements.txt
venv/bin/pip install -r requirements-dev.txt
# 4. Install (enable) your plugin.
venv/bin/flask plugin install shipping
@@ -91,6 +102,35 @@ venv/bin/python -m pytest ../wjsf-shipping/tests
Edits in `../wjsf-shipping` are picked up on the next framework restart, because
the symlink points back at your working tree.
### If your plugin owns tables, step 5 needs a change to the framework
This is the one place an external plugin is not self-contained, and it is better
said plainly than discovered at the first migration.
`flask plugin upgrade-all` builds each plugin's metadata from
`PLUGIN_TABLE_OWNERS` in `shopdb/plugins/alembic_template.py`, and raises rather
than guessing when a plugin has no entry:
```
RuntimeError: PLUGIN_TABLE_OWNERS has no entry for plugin 'shipping'.
Update shopdb/plugins/alembic_template.py.
```
So a plugin that owns tables needs its table names registered in that dictionary
in the framework repository. It is deliberate - the registry is what stops one
plugin's migration touching another's tables, and `tests/test_plugin_migrations.py`
tests it - but it does mean a table-owning external plugin is a two-repository
change: yours, plus a one-line addition upstream.
Two ways to live with it:
- **Send the entry upstream.** One line in `PLUGIN_TABLE_OWNERS` plus one in
`EXPECTED_HEAD_REVISION`, and your plugin is a normal citizen from then on.
- **Own no tables.** A plugin that stores nothing of its own - a report, a
dashboard card, a settings page over existing models - has nothing to
register and stays entirely in your repository. More plugins fit this than
expect to.
## core_version pinning for sister sites
The framework is pre-1.0. Under semver, any 0.x minor bump is allowed to break
@@ -98,20 +138,22 @@ the contract, and this project uses that latitude (see the history in
[CONTRACT-STABILITY.md](CONTRACT-STABILITY.md)). So pin a TIGHT range that
admits only the contract minor you tested against, not the whole 0.x line.
The current contract version is declared in `shopdb/__init__.py`:
The current contract version is declared in `shopdb/__init__.py` as
`__contract_version__`, and is reported in
[PROJECT-MAP.md](PROJECT-MAP.md), which is generated from the code. Read it
there - a version typed into this page is wrong within a fortnight, and a
plugin pinned to a stale one is refused at startup.
```python
__contract_version__ = '0.6.0'
```
Recommended pin in your `manifest.json`, per ADR-002 (pip-style `>=,<` ranges):
Pin a tight range in your `manifest.json`, per ADR-002 (pip-style `>=,<`),
admitting only the contract minor you tested against. With the contract at
0.19.0 that would be:
```json
{
"name": "shipping",
"version": "1.0.0",
"description": "Tracks shipping-station scanners and label printers",
"core_version": ">=0.6.0,<0.7.0",
"core_version": ">=0.19.0,<0.20.0",
"dependencies": []
}
```
@@ -177,14 +219,14 @@ The full script:
# PLUGIN_DIR ($1) path to the plugin directory (holds manifest.json). Required.
# FRAMEWORK_REF ($2) git ref to test against in CI mode. Default: main.
# FRAMEWORK_URL framework git URL for CI mode.
# Default: https://gitea.proudtech.net/ge-aerospace/shopdb-flask.git
# Default: https://github.com/ge-aero/shopdb-flask.git
# LOCAL_FRAMEWORK path to an existing framework checkout. Set it to run offline.
set -eu
PLUGIN_DIR="${PLUGIN_DIR:-${1:-}}"
FRAMEWORK_REF="${FRAMEWORK_REF:-${2:-main}}"
FRAMEWORK_URL="${FRAMEWORK_URL:-https://gitea.proudtech.net/ge-aerospace/shopdb-flask.git}"
FRAMEWORK_URL="${FRAMEWORK_URL:-https://github.com/ge-aero/shopdb-flask.git}"
LOCAL_FRAMEWORK="${LOCAL_FRAMEWORK:-}"
if [ -z "$PLUGIN_DIR" ]; then
@@ -228,7 +270,7 @@ else
python3 -m venv "$WORKDIR/venv"
PYTHON="$WORKDIR/venv/bin/python"
"$PYTHON" -m pip install --upgrade pip >/dev/null
"$PYTHON" -m pip install -r "$FRAMEWORK/requirements.txt"
"$PYTHON" -m pip install -r "$FRAMEWORK/requirements-dev.txt"
fi
echo "==> Linking plugin '$PLUGIN_NAME' into framework plugins/"
@@ -312,14 +354,14 @@ jobs:
runs-on: ubuntu-latest
env:
FRAMEWORK_REF: v0.5.0
FRAMEWORK_URL: https://gitea.proudtech.net/ge-aerospace/shopdb-flask.git
FRAMEWORK_URL: https://github.com/ge-aero/shopdb-flask.git
steps:
- name: Check out the plugin
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
python-version: '3.14'
- name: Fetch the harness from the framework
run: |
git clone --depth 1 --branch "$FRAMEWORK_REF" "$FRAMEWORK_URL" /tmp/framework

44
docs/PLUGIN-GUIDE.md
View File

@@ -5,6 +5,10 @@ running feature with its own list, detail, form, settings page, and report. It i
the companion to [PLUGIN-QUICKSTART.md](PLUGIN-QUICKSTART.md): the quickstart gets
you moving with `flask plugin new`; this guide explains *why* each piece looks the
way it does by walking the shipped code of the exemplar plugin.
> **Windows / VS Code:** command examples use the Linux venv path
> `venv/bin/python`; on Windows use `venv\Scripts\python` and
> `$env:FLASK_APP="shopdb"` (not `export`). Full Windows onboarding:
> [DEVELOPMENT-SETUP](DEVELOPMENT-SETUP.md).
`measuringtools` was chosen as the exemplar on purpose. It is the first plugin
built after the framework matured (ADR-005 scoped it; ADR-008 changed how plugin
@@ -74,7 +78,8 @@ plugin's identity (ADR-002):
Two fields deserve attention.
`core_version` is a semver range against the framework's `__contract_version__`
(declared in `shopdb/__init__.py`, currently `0.6.0`). The loader refuses to load
(declared in `shopdb/__init__.py`; the current value is in
[PROJECT-MAP.md](PROJECT-MAP.md), which is generated). The loader refuses to load
a plugin whose range excludes the running framework. We pin `>=0.6.0` because this
plugin uses the `get_reports` hook, which was added to the contract in 0.6.0
(see [PLUGIN-HOOKS.md](PLUGIN-HOOKS.md), "get_reports"). We cap at `<1.0.0` because
@@ -268,7 +273,7 @@ assertion to a frozen `CUTOVER_PLUGINS` list rather than to all discovered plugi
and to expect `measuringtools`'s real baseline revision in the upgrade-all test:
```python
CUTOVER_PLUGINS = ('computers', 'employees', 'equipment', 'knowledgebase',
CUTOVER_PLUGINS = ('computers', 'employees', 'knowledgebase', 'machines',
'network', 'notifications', 'printers', 'slides', 'usb', 'warranty')
@pytest.mark.parametrize('plugin', CUTOVER_PLUGINS) # not all plugins
@@ -277,7 +282,10 @@ def test_anchor_migration_is_noop(plugin):
```
Freezing the list (rather than deriving it) is intentional: a newly discovered
plugin should not silently be treated as a cutover no-op.
plugin should not silently be treated as a cutover no-op. Note that `machines`
(renamed from `equipment`, ADR-011) keeps its original cutover anchor but carries
a `machines0002rename` revision on top of it, so its expected head is that rename
revision rather than the bare anchor (`tests/test_plugin_migrations.py`).
---
@@ -495,23 +503,25 @@ the plugin small.
## 9. Frontend integration
There is no frontend plugin system yet (see
[ADR-009](adr/ADR-009-frontend-plugin-gating.md), "Future direction"). A plugin's
Vue routes and views ship in the core bundle. The plugin's job is to add them
correctly and gate them.
A plugin ships its own Vue routes and views under
`plugins/<name>/frontend/` (ADR-010's frontend plugin hook contract). At build
time those files are staged into `frontend/src/.plugins-staged/<name>/` so the
core bundle picks them up; you author them in the plugin tree, not in core
`frontend/src/`. The plugin's job is to add them correctly and gate them.
**Route module with `meta.plugin` gating (ADR-009).** A new file
`frontend/src/router/routes/measuringtools.js` is auto-discovered by the router's
**Route module with `meta.plugin` gating (ADR-009).** The file
`plugins/measuringtools/frontend/routes.js` is staged into
`frontend/src/.plugins-staged/measuringtools/` and auto-discovered by the router's
`import.meta.glob('./routes/*.js')`. Every route carries `meta.plugin =
'measuringtools'`:
```js
export default [
{ path: 'measuringtools', name: 'measuringtools',
component: () => import('../../views/measuringtools/MeasuringToolsList.vue'),
component: () => import('./views/MeasuringToolsList.vue'),
meta: { plugin: 'measuringtools' } },
{ path: 'measuringtools/new', name: 'measuringtool-new',
component: () => import('../../views/measuringtools/MeasuringToolForm.vue'),
component: () => import('./views/MeasuringToolForm.vue'),
meta: { requiresAuth: true, plugin: 'measuringtools' } },
{ path: 'measuringtools/:id', ..., meta: { plugin: 'measuringtools' } },
{ path: 'measuringtools/:id/edit', ..., meta: { requiresAuth: true, plugin: 'measuringtools' } },
@@ -535,18 +545,18 @@ reorganize the file; just add the block, mirroring `machinesApi`.
**Views mirror the master templates.** The frontend has master templates
(`PrintersList.vue` for lists, `PrinterDetail.vue` for detail pages). `measuringtools` mirrors the equivalent equipment views:
- `views/measuringtools/MeasuringToolsList.vue` - table with search, a type filter,
- `plugins/measuringtools/frontend/views/MeasuringToolsList.vue` - table with search, a type filter,
and a calibration-status filter; the status badge uses `utils/colorStyle` with
the color the API derived.
- `views/measuringtools/MeasuringToolDetail.vue` - hero + Identity card +
- `plugins/measuringtools/frontend/views/MeasuringToolDetail.vue` - hero + Identity card +
Calibration card (with the derived badge) + Location card, plus the shared
`CustomFieldsSection` and `WarrantyPanel` (section 10).
- `views/measuringtools/MeasuringToolForm.vue` - asset core fields + type +
- `plugins/measuringtools/frontend/views/MeasuringToolForm.vue` - asset core fields + type +
location + the calibration fields, plus `CustomFieldsInputs`.
- `views/reports/CalibrationReport.vue` - the four buckets (overdue / due soon /
- `plugins/measuringtools/frontend/views/CalibrationReport.vue` - the four buckets (overdue / due soon /
current / unknown), mirroring `WarrantyReport.vue`.
**Settings subtype page.** `views/settings/MeasuringToolTypesList.vue` mirrors
**Settings subtype page.** `plugins/measuringtools/frontend/views/MeasuringToolTypesList.vue` mirrors
`PCTypesList.vue`: add / edit / delete with a `ColorSwatchPicker`. It is linked from
`settingsNav.js` with a "Measuring Tools" card group, so it appears in the settings
rail and landing overview.
@@ -574,7 +584,7 @@ detail page drops in `<CustomFieldsSection :assetid="tool.assetid" />` and the f
drops in `<CustomFieldsInputs :assettypeid="assettypeid" :assetid="currentAssetId" />`,
then calls `customFieldsRef.value.save(assetId)` after the tool saves. The one
subtlety: `CustomFieldsInputs` needs the asset-type id. Rather than hardcode it (the
equipment form hardcodes `EQUIPMENT_ASSETTYPEID = 1`), the measuringtools form
machines form hardcodes `MACHINE_ASSETTYPEID = 1`), the measuringtools form
resolves it dynamically from `GET /api/assets/types`, finding the row whose
`assettype === 'measuring_tool'`. Dynamic lookup is preferred because seeded ids are
not stable across sites.

129
docs/PLUGIN-HOOKS.md
View File

@@ -6,11 +6,11 @@ The contract is locked in [ADR-001](../docs/adr/ADR-001-asset-as-platform-contra
## Contract version
The framework declares its contract version in `shopdb/__init__.py`:
```python
__contract_version__ = '0.11.0'
```
The framework declares its contract version in `shopdb/__init__.py` as
`__contract_version__`. The current value is in
[PROJECT-MAP.md](PROJECT-MAP.md), which is generated from the code - this page
does not restate it, because a version copied into prose is stale within a
fortnight and a plugin pinned against a stale one is refused at startup.
Each plugin's `manifest.json` declares the range of contract versions it supports:
@@ -154,27 +154,48 @@ class PrintersPlugin(BasePlugin):
### `get_dashboard_widgets() -> List[Dict]`
Returns dashboard widget definitions for the home page.
Returns dashboard card definitions for the home page.
A card declares DATA AND SHAPE, never a component name. Core owns a small set of
generic renderers and draws the card; the plugin says what to show, where it
comes from, and how to link it.
**Changed in contract 0.19.0.** The previous shape named a Vue component per
widget (`'component': 'NotificationsWidget'`). That cannot survive a lean build,
because a plugin's component may never be staged into the frontend bundle
(ADR-013), and in practice five plugins declared widgets pointing at components
nobody had written - so they rendered as nothing. A card using the old shape is
ignored. This is the same correction ADR-010 already made for asset panels.
```python
class NotificationsPlugin(BasePlugin):
class GeEnforcePlugin(BasePlugin):
def get_dashboard_widgets(self):
return [{
'name': 'recent_notifications',
'component': 'NotificationsWidget',
'endpoint': '/api/notifications/recent',
'size': 'medium',
'position': 1,
'id': 'geenforce-failures', # stable, unique across plugins
'title': 'Enforcement failures',
'endpoint': '/api/geenforce/dashboard/failures',
'render': 'exceptions', # a core renderer, not a component
'severity': 'critical', # orders cards on the page
'permission': 'geenforce.manage', # hidden without it
'empty': 'hide', # say nothing when there is nothing
'position': 10,
'viewall': '/geenforce', # optional link behind the heading
'map': {'title': 'hostname', 'detail': 'entryname'},
}]
```
`empty: 'hide'` is not cosmetic. A card that reports "nothing wrong" every day
teaches people to stop reading the page.
Consumed by `GET /api/dashboard/widgets`, which merges widgets from all enabled
plugins sorted by `position` (disabled plugins are skipped; a broken plugin is
isolated in prod, re-raised in dev/test).
### `get_navigation_items() -> List[Dict]`
Returns navigation menu items.
Returns navigation menu items. A plugin owns its own sidebar entry here, so it
appears when the plugin is installed and disappears when it is not (including in
a lean per-site build that omits the plugin).
```python
class ComputersPlugin(BasePlugin):
@@ -187,6 +208,26 @@ class ComputersPlugin(BasePlugin):
}]
```
**Placement.** `position` (int) sets both the sort order and which section the
item lands in - the core sidebar (`AppLayout.vue:buildNavItems`) assigns section
headers by position range:
| `position` | section |
|-----------|---------|
| `< 10` | top, above any header (Dashboard is 0, Map is 4) |
| `10-29` | **Assets** |
| `30-49` | **Information** |
| `>= 50` | trailing, below Information |
Lower number sorts higher within a section. An explicit `'section':
'information'` forces the Information group regardless of position. Only these
two named sections exist; a new section needs a core edit to `buildNavItems`.
The **Displays** group (kiosk/TV links) is hardcoded in `AppLayout.vue`, not
plugin-driven.
`icon` is a string key mapped to a Lucide component core-side (same idea as
`get_settings_cards`); an unknown key renders with no icon.
> Removed in contract 0.4.0: `get_searchable_fields`. Global search
> (`/api/search`) is a core concern that queries the asset model directly and
> already covers every bundled asset type; no plugin ever implemented the hook.
@@ -393,6 +434,46 @@ class PrintersPlugin(BasePlugin):
]
```
### `get_settings_defaults() -> List[Dict]` (0.16.0)
Declares the `Setting` rows this plugin owns. Return `[]` (the default) if it
owns none. Each entry is a dict:
| Key | Meaning |
|-----|---------|
| `key` | the Setting key |
| `value` | default value in string form |
| `valuetype` | `'string'` / `'boolean'` / `'integer'` / `'json'` |
| `category` | grouping the plugin's settings page filters on |
| `description` | what the setting does |
| `public` | `True` if an unauthenticated caller may read it; default `False` |
```python
class PrintedpartsPlugin(BasePlugin):
def get_settings_defaults(self):
return [
{'key': 'printedparts_label_prefix', 'value': '', 'valuetype': 'string',
'category': 'printedparts', 'public': True,
'description': 'Leading text on the physical labels, shown at the kiosk'},
]
```
The framework seeds declared keys at install, at enable, and on every
`flask plugin upgrade-all`, so a key added in a later plugin version reaches a
site that installed an earlier one. Existing values are never overwritten.
Declaring a key is also what tells the settings API which category and type to
use when an admin's save creates the row for the first time. Do not seed
settings by hand in `on_install` / `on_enable`: those hooks fire only on a state
transition, so a hand-seeded key added later never reaches an existing site, and
the row the first save creates lands in the placeholder `plugin` category where
the plugin's own settings page (which filters by category) cannot see it.
`public: True` puts the key on the unauthenticated read allowlist of
`GET /api/settings/<key>` and `GET /api/settings`. Use it only for cosmetic
values that a page rendering before login needs (a kiosk, a print page). Never
mark a credential, a hostname, or an integration URL public.
### `get_collector_schema() -> Optional[Dict]`
Declares the JSON Schema for an external collector pushing to `/api/collector/<pluginname>`. See [ADR-006](../docs/adr/ADR-006-collector-contract.md) for the contract.
@@ -465,7 +546,8 @@ What `shopdb.api` exposes:
- Model bases: `BaseModel`, `AuditMixin`
- Core models: `Asset`, `AssetType`, `AssetStatus`, `Vendor`, `Model`,
`Communication`, `CommunicationType`, `Location`, `Setting`, `AuditLog`,
`Application`, `AppVersion`, `OperatingSystem`
`Application`, `AppVersion`, `OperatingSystem`, `AssetRelationship`,
`RelationshipType`
- Responses: `success_response`, `error_response`, `paginated_response`,
`ErrorCodes`
- Pagination: `get_pagination_params`, `paginate_query`
@@ -474,11 +556,30 @@ What `shopdb.api` exposes:
(`service_token_authorized(scope)` returns True when the request carries a
managed service token scoped for `scope` whose owner holds that permission -
for unattended plugin endpoints like the GE-Enforce fetch API)
- `authorized_service_token(scope)` (0.15.0) - same check as
`service_token_authorized` but returns the `ApiToken` itself (or None), so a
plugin can honor the token's optional resource binding
(`token.resourcescopelist`: an allowlist of resource names the token may
reach, NULL = unrestricted). GE-Enforce uses it to pin a display's fetch
token to its own manifest scope + that scope's blobs.
- Helpers: `audit_log`, `resolve_asset_position`, `resolve_dualpath_pairs`,
`dualpath_single_machine_enabled`
- Import mode: `apply_import_timestamps`, `import_mode_active`,
`parse_import_datetime`
- Legacy employee directory: `employee_connection`
- CMMC USB check-in/out DB (read-write, used by the usb plugin):
`cmmc_usb_connection`
- `User` / `Role` (0.13.0) - the account and role models, e.g. resolving
alert recipients' emails from selected user ids or role membership
- `SupportTeam` (0.15.0) - the support-team model (carries a `webhookurl`), so
an alerting plugin can route a notification to a chosen team's Teams webhook
- Mailer (0.12.0): `send_email(to, subject, html, text=None)` and
`send_alert(subject, html, text=None)` - settings-first, no-op safe when
email is unconfigured; send_alert targets the site's alert_recipients
- `send_webhook(title, text)` (0.14.0) - POST an alert to the configured
`alert_webhook_url` (Teams Incoming Webhook / Workflow, or generic JSON via
the `alert_webhook_format` setting); best-effort, no-op when unset.
`send_alert` fans out to this automatically alongside email.
```python
from shopdb.api import db, Asset, AssetType, success_response, paginate_query

1333
docs/PLUGIN-LAB-PRINTEDPARTS.md
View File

File diff suppressed because it is too large Load Diff

13
docs/PLUGIN-QUICKSTART.md
View File

@@ -3,6 +3,12 @@
Build a working shopdb-flask plugin in 30 minutes. This walks through generating, customizing, installing, and testing a plugin from scratch.
For the full hook reference, see [PLUGIN-HOOKS.md](PLUGIN-HOOKS.md).
> **Windows / VS Code:** command examples below use the Linux venv path
> `venv/bin/python`; on Windows use `venv\Scripts\python` and
> `$env:FLASK_APP="shopdb"` (not `export`). Full Windows onboarding:
> [DEVELOPMENT-SETUP](DEVELOPMENT-SETUP.md).
For the architectural decisions behind the contract, see [docs/adr/](../docs/adr/).
## Step 1: Generate the skeleton
@@ -85,13 +91,14 @@ audit_log(action='created', entitytype='Camera', entityid=asset.assetid, entityn
## Step 4: Install the plugin
First add `plugins/cameras/migrations/` with a per-plugin Alembic chain that creates the plugin's tables, and register those tables in `PLUGIN_TABLE_OWNERS` (per ADR-008; the plugin chain owns plugin schema, never the core chain). Then:
```bash
flask plugin install cameras
flask db migrate -m "Add cameras plugin tables"
flask db upgrade
flask plugin upgrade-all
```
`install` runs the plugin's `on_install` hook (which seeds the AssetType row), registers it in the plugin registry, and runs migrations.
`install` runs the plugin's `on_install` hook (which seeds the AssetType row), registers it in the plugin registry, and runs the plugin's own migration chain. `flask db migrate`/`flask db upgrade` is reserved for core tables and must not be used for plugin schema.
## Step 5: Verify it works

141
docs/PLUGIN-SIGNING.md Normal file
View File

@@ -0,0 +1,141 @@
# Plugin signing and packaging (curator guide)
ADR-013 Phase 1. How a plugin becomes a signed, verifiable artifact and how a
site trusts it. The signature proves an artifact is EXACTLY what a curator
reviewed and signed - it does not prove the code is safe. Human review before
signing is the actual safety control; the signature makes that review's verdict
tamper-evident all the way to the point of execution.
Requires the `cryptography` package (already a dependency).
## One-time: create the publisher key pair
```
flask plugin keygen --out ./keys --name curator
```
Writes `keys/curator.key` (PRIVATE) and `keys/curator.pub` (public).
- Keep the `.key` OFFLINE with the curator. It is the only thing that can sign a
trusted artifact. Never put it on the plugin shelf or in the repo.
- Distribute the `.pub` with each site's deployed config and pin it (below).
- Rotation: generate a new pair, pin BOTH public keys on sites for an overlap
window (`verify` accepts any trusted key), then retire the old one.
## Per plugin: review, then pack
1. Review the plugin's source. This is the security gate - read what it does.
2. Validate and package in one step:
```
flask plugin pack printers --key ./keys/curator.key --publisher west-jefferson
```
`pack` refuses to sign a directory that does not validate (manifest schema,
name/dir match, core_version, dependencies on disk). On success it writes
`printers-<version>.shopdbplugin` - a zip of the plugin plus:
- `PROVENANCE.json`: name, version, publisher, created, and a sorted
`{file: sha256}` map of every packaged file.
- `PROVENANCE.sig`: a detached ed25519 signature over the exact
`PROVENANCE.json` bytes.
3. Publish the artifact to the shelf (a SharePoint-synced or copied folder).
Transport is untrusted; the signature is what makes it safe.
## Verify an artifact
```
flask plugin validate dist/printers-1.0.0.shopdbplugin --pubkey ./keys/curator.pub
```
Checks, fail-closed: signature against the trusted key(s), every file's hash,
no unexpected files, manifest schema, and that the plugin's `core_version`
admits this framework's contract version. Any changed byte in any file fails
the hash check; a signature from an untrusted key fails the signature check.
## Pin trusted keys on a site
Set `PLUGIN_TRUSTED_KEYS` to one or more public-key PEM paths, separated by the
OS path separator (`:` on Linux, `;` on Windows), in the site's environment:
```
PLUGIN_TRUSTED_KEYS=/etc/shopdb/keys/curator.pub:/etc/shopdb/keys/curator-next.pub
```
Keys are read only from this deployed config, never from the shelf - a folder an
attacker could write must not also carry the keys that authenticate it. With no
keys set, `validate` on an artifact fails closed (unverifiable).
## Enforce signatures (Phase 2)
By default nothing is enforced - plugins load unsigned, as before. To require
signatures on a site:
1. Stamp the plugins the image ships with, so verify-at-load applies to them
too (run at image build with the site/build key):
```
flask plugin stamp-bundled --key ./keys/curator.key
```
This writes `PROVENANCE.json` + `PROVENANCE.sig` into each in-tree plugin.
2. Pin the public key(s) and turn enforcement on (site config):
```
PLUGIN_TRUSTED_KEYS=/etc/shopdb/keys/curator.pub
PLUGIN_REQUIRE_SIGNED=true
```
Now a plugin only loads or migrates when its tree matches a trusted signature.
Under enforcement a `sys.meta_path` guard verifies EVERY `plugins.<name>.*`
import (not just `plugin.py`) - including the `from plugins.<name>.models import
...` that core request handlers do - against the plugin's signed provenance, and
executes the exact bytes it hashed (never a `.pyc`). An unsigned, tampered, or
wrong-key plugin is refused, fail-closed. `PLUGIN_DEV_TRUST_DIRS` exempts named
directories, but ONLY under DEBUG/TESTING (the external-repo dev workflow);
production ignores it.
Because every `plugins.*` import is verified, `stamp-bundled` must cover EVERY
plugin directory present (its no-argument form does), not only the enabled ones
- core code can import a disabled plugin's module, and an unstamped one would be
refused.
Defense in depth - set filesystem permissions so the app's runtime user CANNOT
write the `plugins/` directory (owned by the deploy user). Import-time
verification closes the "attacker drops a file, a request imports it" path; a
strict read-only `plugins/` also closes the narrow verify-vs-migrate race where
an attacker with concurrent write to `plugins/` swaps a migration script between
the check and alembic re-reading it.
## The shelf and adopt (Phase 2)
A shelf is a read-only folder of artifacts plus a signed index. The app reads
`PLUGIN_SHELF_DIR`; it never talks to SharePoint - a sync (or robocopy/USB)
populates that folder, and the signature makes the transport untrusted and
interchangeable.
Publish (curator, after packing artifacts into the shelf folder):
```
flask plugin shelf-build --dir /srv/shelf --key ./keys/curator.key --serial 3
```
The index carries a monotonic `serial` (a site refuses an index older than the
last it saw) and a `revoked` list (carried forward across builds). Bump
`--serial` on every publish.
On a site:
```
flask plugin shelf-list # browse (verifies index + serial)
flask plugin adopt printers # or printers==1.2.0
flask plugin audit # warn if an installed version is revoked
```
`adopt` verifies the shelf index and the artifact (signature + every file
hash), unpacks into a staging area, re-verifies, then atomically moves it into
place and installs + enables it with its dependency closure. It refuses a
downgrade unless `--force-downgrade`. Run `flask plugin upgrade-all` and restart
afterward so migrations apply and routes register.

48
docs/PLUGINS.md
View File

@@ -10,11 +10,24 @@ These plugins are in `plugins/` in this repo. Enable per site with `flask plugin
|--------|--------|-------|
| `machines` | Manufacturing machinery: 5-axis mills, lathes, broachers, heat treatment ovens | Manually entered. See [ADR-005](adr/ADR-005-equipment-vs-measuringtools.md). Subtype tables for FOCAS / CLM / MTConnect controller protocols (planned). |
| `computers` | Shop-floor PCs and engineering workstations | Fed by the PXE pipeline collector per [ADR-006](adr/ADR-006-collector-contract.md). |
| `printers` | Network and shop-floor printers | Optional Zabbix integration for supply tracking. Legacy `PrinterData` retiring per ADR-001. |
| `printers` | Network and shop-floor printers | Public installer map page + fleet install contract (`/api/printers/install-list`, `/pc-default`, `/install-batch`; see [PRINTER-INSTALLER.md](PRINTER-INSTALLER.md)). Optional Zabbix integration for supply tracking. Legacy `PrinterData` retiring per ADR-001. |
| `network` | Switches, routers, access points, IDFs as locations | Asset-only; cleanest of the bundled set. |
| `usb` | USB devices issued to shop-floor users | Lightweight checkout / check-in. |
| `notifications` | Shop-floor notifications, recognitions, kiosk feed | Used by `ShopfloorDashboard.vue`. |
| `measuringtools` | Metrology and inspection instruments: calipers, micrometers, thread/bore/height gages, indicators | Per [ADR-005](adr/ADR-005-equipment-vs-measuringtools.md). Calibration lifecycle with derived status. First plugin built on the matured scaffold; its walkthrough is [PLUGIN-GUIDE.md](PLUGIN-GUIDE.md). Ships `default_enabled: false`. |
| `employees` | Read-only employee directory lookup | Backed by a separate HR database. Ships `default_enabled: false`. |
| `geenforce` | GE-Enforce manifest store: imaging PC-type scopes and their install manifests (apps, scripts, files, registry, version gates) | Per [ADR-012](adr/ADR-012-geenforce-manifest-ownership.md). Served to the GE-Enforce client as JSON. Requires GE-Enforce lib >= 2.6 on target PCs. Ships `default_enabled: false`. |
| `knowledgebase` | Knowledge Base articles linking to external resources | Lightweight article store. |
| `printedparts` | 3D-printed parts inventory | Kiosk checkout / check-in. Ships `default_enabled: false`. |
| `slides` | Slides for the lobby display and shop-floor screensaver | Upload / reorder / delete per surface. Management gated on the `slides.manage` permission, grantable to non-admin curators; display routes are public. |
| `warranty` | Asset warranty tracking | Manual entry now, Dell / Lenovo / HP provider lookups later. Derived coverage status with report buckets. |
## Plugin permissions
Plugins may register their own permissions (for example `slides.manage`). Admins
implicitly hold them; grant them to specific roles or users to delegate curation
without admin. Each plugin's registered permissions appear in its `plugin.py`
`get_permissions()`.
## Building your own
@@ -62,11 +75,42 @@ which revisions each plugin has applied in `migrations_applied`.
For sister-site plugins (per [ADR-003](adr/ADR-003-plugin-distribution.md)):
- Plugin lives in its own git repo: `gitea.proudtech.net/<your-site>/<pluginname>`
- Plugin lives in its own git repo: `<git-host>/<your-site>/<pluginname>`
- Adopting site clones or symlinks into their `<repo>/plugins/<name>/`
- Plugin manifest declares `core_version` range matching the framework version they target
- Plugin readme explains: what it tracks, who maintains it, where to file issues
## Lean per-site builds
A site ships only the plugins it chose; a site that never wants printedparts /
usb / network never carries that code (see
[ADR-013](adr/ADR-013-plugin-catalog-and-lean-builds.md) and
[ADR-014](adr/ADR-014-schema-lean-per-site.md)). Three layers make a build lean:
- **Backend code** - `scripts/build-site.sh <profile>` stages `shopdb/core` plus
only the chosen plugins' directories (and their hard-dependency closure). A
plugin a site did not choose is absent from the backend tree.
- **Frontend code** - `SITE_PLUGINS=machines,printers npm run build` (via
`scripts/stage-frontend.mjs`) stages only those plugins' `frontend/` dirs and
codegens the route table. **Exception:** a `plugins/<name>/frontend/` dir with
**no `manifest.json`** is a CORE feature (e.g. `applications`), not a per-site
plugin, and is ALWAYS staged regardless of `SITE_PLUGINS` - otherwise a lean
build would lose a core page.
- **Database** - the shared core Alembic baseline creates every plugin's tables,
so a lean site provisions them and then drops the ones it does not use with
`flask plugin prune-schema` (ADR-014). Run it once at provisioning, after
`flask db upgrade` and `flask plugin upgrade-all`; see
[DEPLOY.md](DEPLOY.md).
**Menus follow the build, not a plugin flag.** The sidebar nav, the settings
rail, and the Displays links all gate on whether the target route was actually
staged into this build (the router's own route table), not on a registry
"enabled" flag. So a lean site never shows a menu entry that dead-ends on a
blank page - an omitted plugin's nav item, settings cards, and kiosk links all
disappear together. Shopfloor Dashboard is a core view but is gated on the
notifications plugin (its only data source), so it drops when notifications is
not in the build.
## Naming policy
Plugin names follow the framework's naming convention (lowercase concatenated, no underscores or dashes; full words preferred over acronyms). See [CONTRIBUTING.md](../CONTRIBUTING.md). Plugin name collisions across sites are not enforced; the convention recommends prefixing site-specific plugins with the site code (e.g., `wjsf-shippingstation`) when there is risk of overlap.

104
docs/PRINTER-INSTALLER.md Normal file
View File

@@ -0,0 +1,104 @@
# Printer installer map and install endpoints
How the shop-floor fleet installs network printers from shopdb-flask, replacing
the classic ASP `apiprinters.asp` / `apipcdefaultprinter.asp` / `installprinter.asp`
contract. Shopfloor 2.0 PCs cannot run unsigned `.bat` maps, so a signed
installer EXE (and the public web map page) drives installs from three endpoints
in the printers plugin.
- Server code: `plugins/printers/api/asset_routes.py`
(`printer_install_list`, `pc_default_printer`, `printer_install_batch`)
- Consumed as a fleet manifest entry: the `common` scope's `printer map`
entry (see `GE-ENFORCE-DISPLAY.md`).
All three endpoints are `@jwt_required(optional=True)`: an anonymous fleet
client works, and a logged-in browser (the public map page) works too.
---
## 1. The public map page
`PrinterInstallerMap` is a public (no-login) frontend page: the floor map with
printer hotspots positioned at each printer's `mapx` / `mapy`. The user clicks
the printers they want, and the page requests an install batch. The PC's default
printer is preselected via `pc-default`.
---
## 2. `GET /api/printers/install-list`
Flat, unpaginated list of active NETWORK printers. A printer counts as network
only if it has a hostname or a non-USB IP; USB-only printers are excluded.
Fields per row:
| Field | Notes |
|---|---|
| `printerid` | Printer id (the token `install-batch` takes). |
| `name` | Asset name, else asset number. |
| `machinenumber` | The asset number. |
| `windowsname` | Standardized Windows printer name. |
| `sharename` | Share / CSF name. |
| `hostname` | Print-queue host. |
| `ipaddress` | Primary IP (falls back to any communication row). |
| `vendorname` | Direct vendor, else the model's vendor. |
| `modelnumber` | Model name. |
| `installpath` | Installer path for this printer (see install-batch). |
| `iscsf` | CSF flag. |
| `locationname` | Location name, if the asset has one. |
| `mapx` / `mapy` | Floor-map hotspot position. |
`?format=text` returns a pipe-delimited line per printer, one printer per line,
with a fixed field order so the Inno / Pascal installer does a `split()` instead
of parsing JSON:
```
printerid|windowsname|vendorname|modelnumber|hostname|ipaddress|mapx|mapy
```
Any pipe or newline inside a value is neutralized to a space so the field count
stays fixed. The web map uses the default JSON.
---
## 3. `GET /api/printers/pc-default?machine=NNNN`
The PC's default printer, by machine (asset) number persisted at PXE enrollment.
Parity with classic `apipcdefaultprinter.asp`: the installer preselects a PC's
default-printer hotspot on the site-map wizard. The link is resolved through the
`defaultprinter` asset relationship (PC asset -> printer asset), so it stays
inside the contract surface (no cross-plugin model import).
Returns `{printerid, windowsname}`, or `{}` when the machine is unknown or has no
active default printer set.
`?format=text` returns one pipe-delimited line (`printerid|windowsname`), or an
EMPTY body when there is no default (so the installer's split yields nothing).
---
## 4. `GET /api/printers/install-batch?printerids=1,2,3`
Returns a self-deleting Windows `.bat` that installs the selected printers,
grouped the same way classic `installprinter.asp` grouped them:
- HP / Xerox: one universal `PrinterInstaller.exe /PRINTER="a,b,c"` call.
- Printers with a `.exe` `installpath`: a PowerShell `Invoke-WebRequest` download
(using the caller's Windows credentials, against the site base URL + the
IIS-served `/installers` folder) followed by running it `/SILENT`.
- No `installpath`, or a non-`.exe` payload (e.g. a `.zip`): listed as a manual
install rather than run blindly.
`printerids` is required, comma-separated; non-numeric tokens are ignored. An
empty / missing list is a validation error.
The install name preference is `windowsname`, else `sharename`, else the asset
name / number.
---
## 5. Fleet wiring
The `common` scope's `printer map` manifest entry (see `GE-ENFORCE-DISPLAY.md`)
lays down the signed installer that consumes these endpoints. The web map page
covers the same install flow for a human at a browser.

89
docs/PROJECT-MAP.md Normal file
View File

@@ -0,0 +1,89 @@
<!-- GENERATED by scripts/gen_project_map.py - do not edit by hand.
Regenerate with: venv/bin/python scripts/gen_project_map.py -->
# Project map
Derived from the code on every regeneration. If something here is wrong, the
code changed and the map was not regenerated - fix it by running the generator,
never by editing this file.
## Versions
| series | value | governed by |
|---|---|---|
| product `__version__` | `0.10.0` | ADR-007 |
| plugin contract `__contract_version__` | `0.19.0` | ADR-002 |
They move independently. A contract bump is not a release.
## Alembic chains (ADR-008)
Core runs with `flask db upgrade`; every plugin chain runs
with `flask plugin upgrade-all`. Both are needed on a deploy.
| chain | head |
|---|---|
| core | `7d32_displayrole_kiosk_vocabulary` |
| backups | `backups0003clearlastseen` |
| computers | `computers0001anchor` |
| employees | `employees0002photo` |
| geenforce | `geenforce0004minlib` |
| knowledgebase | `knowledgebase0001anchor` |
| machines | `machines0002rename` |
| measuringtools | `measuringtools0001baseline` |
| network | `network0003prefix` |
| notifications | `notifications0005boardorder` |
| printedparts | `printedparts0004txnrev` |
| printers | `printers0002supplyalerts` |
| slides | `slides0001anchor` |
| usb | `usb0002dropmachineid` |
| warranty | `warranty0002proof` |
## Bundled plugins (15)
| plugin | version | core_version | owns migrations |
|---|---|---|---|
| backups | 1.0.0 | >=0.16.0,<1.0.0 | yes |
| computers | 1.0.0 | >=0.1.0,<1.0.0 | yes |
| employees | 1.0.0 | >=0.1.0,<1.0.0 | yes |
| geenforce | 0.1.0 | >=0.18.0,<1.0.0 | yes |
| knowledgebase | 1.0.0 | >=0.1.0,<1.0.0 | yes |
| machines | 1.0.0 | >=0.1.0,<1.0.0 | yes |
| measuringtools | 1.0.0 | >=0.6.0,<1.0.0 | yes |
| network | 1.0.0 | >=0.1.0,<1.0.0 | yes |
| notifications | 1.0.0 | >=0.1.0,<1.0.0 | yes |
| printedparts | 0.1.0 | >=0.16.0,<1.0.0 | yes |
| printers | 1.0.0 | >=0.16.0,<1.0.0 | yes |
| slides | 2.0.0 | >=0.2.0,<1.0.0 | yes |
| tools | 1.0.0 | >=0.16.0,<1.0.0 | no |
| usb | 1.0.0 | >=0.1.0,<1.0.0 | yes |
| warranty | 1.0.0 | >=0.2.0,<1.0.0 | yes |
Manifest-less directories under `plugins/` are core frontend surface and always ship: `applications`.
## Architecture decisions
| ADR | title | status |
|---|---|---|
| ADR-001-asset-as-platform-contract.md | ADR-001: Asset model is the platform contract | ACCEPTED |
| ADR-002-plugin-versioning.md | ADR-002: Plugin contract versioning | ACCEPTED |
| ADR-003-plugin-distribution.md | ADR-003: Plugin distribution model | ACCEPTED |
| ADR-004-deployment-topology.md | ADR-004: Deployment topology (per-site instances) | ACCEPTED |
| ADR-005-equipment-vs-measuringtools.md | ADR-005: Equipment plugin scope vs measuringtools plugin | ACCEPTED |
| ADR-006-collector-contract.md | ADR-006: Plugin collector contract pattern | ACCEPTED |
| ADR-007-product-versioning-and-releases.md | ADR-007: Product versioning and releases | ACCEPTED |
| ADR-008-plugin-migration-ownership.md | ADR-008: Plugin migration ownership (per-plugin chains from the cutover) | ACCEPTED |
| ADR-009-frontend-plugin-gating.md | ADR-009: Frontend plugin gating | ACCEPTED |
| ADR-010-frontend-plugin-hooks.md | ADR-010: Frontend plugin hook contract | ACCEPTED |
| ADR-011-machines-rename.md | ADR-011: Rename the equipment domain to machines; retype the models catalog with modeltypes | ACCEPTED |
| ADR-012-geenforce-manifest-ownership.md | ADR-012: GE-Enforce manifest ownership in shopdb | ACCEPTED |
| ADR-013-plugin-catalog-and-lean-builds.md | ADR-013: Plugin Catalog, Curated Shelf, and Lean Per-Site Builds | ACCEPTED |
| ADR-014-schema-lean-per-site.md | ADR-014: Schema-lean per-site builds (retire cross-plugin FKs, lift plugin tables) | ACCEPTED |
| ADR-015-site-specific-configuration.md | ADR-015: Where a site's own data is allowed to live | ACCEPTED |
| ADR-016-credential-delivery.md | ADR-016: Credential delivery to the fleet | ACCEPTED (decided; NOT yet implemented - |
## Size
- test functions defined: **1050** (parametrised cases collect higher)
- documented API paths: **265** (`docs/openapi.json`, regenerate with `scripts/gen_openapi.py`)

202
docs/RELEASING-WINDOWS.md Normal file
View File

@@ -0,0 +1,202 @@
# Building and releasing the Windows installer
For whoever builds releases. If you run a server rather than build releases, see
[UPDATES-WINDOWS.md](UPDATES-WINDOWS.md).
Every release is one self-contained `.exe`. It carries the Python runtime, a
hash-checked wheelhouse, MySQL, the IIS modules and the application, so nothing
is fetched from the network at install time. That is the point: sites are
air-gapped.
## What you need
- A checkout of this repository.
- Inno Setup 6.6.0 or newer, on Windows. Compiling needs Windows; staging the
bundle does not.
- PowerShell, for the bundle lock tools.
- `uv`, only when dependencies change.
## The three kinds of change
Almost every release is the first kind.
### 1. Application change: bug fix, feature, no new dependency
```bash
# bump __version__ in shopdb/__init__.py first
./deploy/windows/installer/build-installer.sh deploy/site-profile-universal.json <repo-path>
```
`deploy/site-profile-universal.json` is the profile released builds come from:
every bundled plugin, so one `.exe` serves any site and the operator ticks what
that site uses. Build from a narrower profile only when a site genuinely needs a
lean build (ADR-013). Do not build a release from a profile that is not in the
repository - the build stops being reproducible the moment that file is
somewhere else.
Then on Windows, in `deploy/windows/installer`:
```
iscc ShopDBFlask.iss
```
`build-installer.sh` restages the application, rebuilds the web interface for
the chosen feature set, regenerates `plugins.iss` and `version.iss`, and
re-verifies the third-party payload against `bundle-lock.json`. The lock is
untouched, because nothing third-party changed.
### 2. A dependency is added, removed or moved
The only case that touches the lock.
```bash
# edit requirements.in, then
uv pip compile requirements.in --universal --generate-hashes -o requirements.txt
# fetch the wheel for the target runtime, into the wheelhouse
pip download <name>==<version> -d deploy/windows/installer/bundle/wheels \
--only-binary=:all: --platform win_amd64 --python-version 314 --no-deps
```
Then regenerate and commit the lock:
```powershell
pwsh ./refresh-bundle-lock.ps1 # review the change
pwsh ./refresh-bundle-lock.ps1 -Yes # write it
```
**Commit `bundle-lock.json`. That commit is the review.** The lock records every
third-party file and its SHA-256; the installer refuses to run if the payload it
carries does not match, so an unreviewed substitution cannot reach a server.
Two traps, both of which have already cost a release:
- `--universal` is not optional. A resolve done only for the host platform drops
packages that exist only on Windows, and the install then fails hash checking
on a package with no entry.
- Declare **every** runtime import in `requirements.in`, including ones that
already work locally. `packaging` reached this project only as a test
dependency, so the full suite passed while a production virtual environment,
which has no test dependencies, could not import the application at all.
`tests/test_runtime_dependencies.py` is the gate for this.
### 3. A new plugin
Ordinary plugin work, plus three installer-specific steps:
1. **Stage it.** Add the plugin to the profile you build from. `plugins.iss` is
generated from what is actually in the bundle, so the wizard offers it with
no edit to the installer script.
2. **Decide whether it is ticked by default.** `PluginDefault()` in
`ShopDBFlask.iss` is an exclusion list: a new plugin defaults to **ticked**
unless you name it there. This is the one manual edit.
3. **Register its migrations.** Update `PLUGIN_TABLE_OWNERS` and
`EXPECTED_HEAD_REVISION`, and give the plugin its own migration chain
(ADR-008). Existing sites pick up its tables through `plugin upgrade-all`;
sites that do not tick it have them pruned (ADR-014).
## Version numbers
`build-installer.sh` reads `__version__` from `shopdb/__init__.py` and writes
`version.iss` from it. Do not edit `version.iss`: a hardcoded version in the
installer script had already drifted two minor versions from the code.
Bump the version for every release you hand out. The installer compares versions
and:
- refuses a build **older** than what is installed, because migrations only go
forwards;
- warns and continues on an **equal** version, treating it as a repair.
Equal-version rebuilds are useful while testing and are a poor idea in the
field, since the server cannot then tell you what it is running.
Follow ADR-007 for what each part means.
## Do not hand-edit generated files
Regenerate these; changes are overwritten without warning:
- `version.iss` - from `shopdb/__init__.py`
- `plugins.iss` - from the plugins actually present in the bundle
- `requirements.txt` - from `requirements.in` via `uv pip compile`
- `bundle-lock.json` - via `refresh-bundle-lock.ps1`
`waitress` and `tzdata` were once hand-added to `requirements.txt` and vanished
on the next compile, taking the Windows runtime with them.
## Before you hand a build out
```bash
python -m pytest -q # full suite
./scripts/check-naming-and-style.sh # naming and style
```
The suite includes gates worth knowing about:
- `tests/test_runtime_dependencies.py` - every runtime import is declared
- `tests/test_bundle_lock.py` - the lock covers the payload
- `tests/test_docs_publishable.py` - `docs/` carries no internal references,
because it is published
`build-installer.sh` refuses to finish if the payload does not match the lock.
That check is not advisory; do not work around it.
## Publishing
Alongside the `.exe`, publish:
- a `.sha256` file, so the operator can verify what they received;
- release notes covering what changed and anything needing attention;
- the version in the filename, so a server's build is identifiable on sight.
The installer is attached to a git-forge release as an ASSET, never committed.
Most forges reject files over 100 MB inside a repository while allowing release
assets far larger, and a committed binary would sit in every future clone
forever. Release creation is scripted next to the other deployment scripts and
verifies the SHA-256 before publishing, because a truncated upload is easiest to
catch before anyone can download it.
Tags have to reach the forge for a release to hang off one: a plain
`git push <remote> main` does not carry them, which is why the push step uses
`--follow-tags`.
The compiled installer stages a CycloneDX inventory (`sbom.cdx.json`) onto every
server, which is what answers a security question about a published
vulnerability without anyone guessing.
## Known gaps
**The installer is not code-signed.** Every install shows an unknown-publisher
warning, and the SHA-256 is the only integrity check. This is the significant
remaining gap before wider distribution: a checksum published next to the file
protects against corruption, not against someone who can write to that location.
The decision taken is to wait for a certificate from the organisation's own
certificate authority rather than buy one from a public CA. Every server this
installer runs on is centrally managed, and that CA's root is already trusted on
those machines, so an internally issued Authenticode certificate removes the
warning exactly where it matters. A public certificate would buy trust on
machines this software never reaches.
Until then, publish the SHA-256 through a channel SEPARATE from the installer
itself. A hash sitting beside the file is only as trustworthy as write access to
that location; a hash the operator gets another way means tampering has to
succeed twice.
Wiring it up afterwards is small: Inno has native SignTool support, so a
directive in the script and a signtool configuration on the build machine sign
the installer and its uninstaller. Include a timestamp server, or signatures
stop verifying when the certificate expires.
**Compiling requires Windows.** `build-installer.ps1` exists so the whole
process can run on a Windows workstation. Nothing about it runs in CI, so a
release is a deliberate act by a person.
## See also
- [UPDATES-WINDOWS.md](UPDATES-WINDOWS.md) - what operators should expect
- [INSTALL-WINDOWS.md](INSTALL-WINDOWS.md) - installing a new site
- [OPERATE-WINDOWS.md](OPERATE-WINDOWS.md) - running a site
- `docs/adr/` - ADR-007 versioning, ADR-008 plugin migrations, ADR-013 and
ADR-014 lean per-site builds

6
docs/ROADMAP.md
View File

@@ -1,6 +1,6 @@
# Roadmap
shopdb-flask is at `__contract_version__ = '0.11.0'` (pre-1.0; product `__version__ 0.7.0`, tags through v0.7.0). This document captures what stands between today and a stable `1.0.0` release. Maintained as scope evolves; supersedes nothing in the ADRs.
shopdb-flask is pre-1.0 on both series. Current contract and product versions are in [PROJECT-MAP.md](PROJECT-MAP.md), which is generated from the code rather than typed here. This document captures what stands between today and a stable `1.0.0` release. Maintained as scope evolves; supersedes nothing in the ADRs.
## Phase status
@@ -20,9 +20,9 @@ The last big milestone before 1.0 is the legacy-ASP data import plus a productio
### Must-have
- **Asset model fully wired**. `Asset.mapx, Asset.mapy` columns, `AssetRelationship.inheritsposition`, `AssetRelationship.propagatesthroughid` columns. Models match the locked ADR-001 surface that `resolve_asset_position` already targets.
- **Asset model fully wired**. `Asset.mapx, Asset.mapy` columns, `AssetRelationship.inheritsposition` column, and the `relationshiptypepropagations` M:N table (`RelationshipTypePropagation` model; propagation lives on `RelationshipType`, not `AssetRelationship`). Models match the locked ADR-001 surface that `resolve_asset_position` already targets.
- **Equipment data migration script** for facilities migrating from legacy ASP shopdb. One-shot script under `scripts/migration/`. Per ADR-001, only `category='Equipment' AND machinenumber IS NOT NULL` migrates.
- **Printers retirement**. Legacy `PrinterData` model, `printers_bp` legacy blueprint, and the frontend `PrinterForm.vue` references to `printer.printerdata.*` get removed in lockstep. Coordinated with the equipment migration.
- **Printers retirement**. The printers plugin already runs on the asset architecture (blueprint `printers_asset_bp`); any remaining legacy printer-table cleanup is coordinated with the equipment migration.
- **External plugin UI packaging**. The Vue-side hook contract ships (ADR-010: get_settings_cards / get_asset_panels / get_map_overlays / get_asset_presentation) and route gating is backend-driven (ADR-009), but plugin routes/views still live in core `frontend/src`. Let an external plugin ship its own Vue bundle so adopters can add UI without editing core.
### Nice-to-have

64
docs/START-HERE.md Normal file
View File

@@ -0,0 +1,64 @@
# Start here
ShopDB is a per-site asset platform: one instance per plant, plugins for what
each site tracks. These pages document the server, the plugin contract, and the
tools that run on shop-floor PCs.
Find yourself below. Each row is the shortest correct path, not everything that
exists.
## I am standing up ShopDB at a new site
1. [INSTALL-WINDOWS](INSTALL-WINDOWS.md) - one offline installer, start to finish.
2. [OPERATE-WINDOWS](OPERATE-WINDOWS.md) - restart, logs, backups, upgrades.
3. [CSV-IMPORT](CSV-IMPORT.md) if the site's data is in spreadsheets, or
[IMPORT-API](IMPORT-API.md) if there is a source database to script against.
**Do not** follow [INSTALL-WINDOWS-IIS](INSTALL-WINDOWS-IIS.md) for a new site.
That is the manual procedure, kept for hand-built servers that predate the
installer, and following it produces a server the installer then refuses to
upgrade.
## I am deploying the shop-floor tools
Read [FLEET-ARCHITECTURE](FLEET-ARCHITECTURE.md) first - it is one page and it
is what makes the rest make sense.
Then [ADOPTING-AT-ANOTHER-SITE](ADOPTING-AT-ANOTHER-SITE.md), which has worked
examples for Intune, GE-Enforce and manual installation, and says where each
artifact comes from.
## I am writing a plugin
1. [PLUGIN-QUICKSTART](PLUGIN-QUICKSTART.md) - scaffold and first run.
2. [PLUGIN-HOOKS](PLUGIN-HOOKS.md) - the contract. Authoritative.
3. [PLUGIN-LAB-PRINTEDPARTS](PLUGIN-LAB-PRINTEDPARTS.md) - a real plugin built
end to end, to type along with.
4. [PLUGIN-EXTERNAL-REPO](PLUGIN-EXTERNAL-REPO.md) if it lives in its own
repository, and [CONTRACT-STABILITY](CONTRACT-STABILITY.md) before you choose
a version pin.
## I am integrating with the API
1. [LLM-GUIDE](LLM-GUIDE.md) (`docs/llms.txt` in the repo, also served at
`/api/docs/llms.txt`) - auth, the response envelope, common recipes.
Short, and the envelope section is the part people get wrong.
2. `GET /api/docs` on any running instance - the full spec, browsable.
3. [API-REFERENCE](API-REFERENCE.md) for prose, [COLLECTOR-INTEGRATION](COLLECTOR-INTEGRATION.md)
if you are feeding data in from a PC.
## I am trying to understand why something is built this way
[The ADRs](ADR-001-asset-as-platform-contract.md). They are the decision record,
they say what was rejected and why, and they are the fastest way to avoid
relitigating a settled question. [PROJECT-MAP](PROJECT-MAP.md) lists them all
with their status, along with the current versions and every migration head -
it is generated, so it is never stale.
## Something is broken
- Server: [OPERATE-WINDOWS](OPERATE-WINDOWS.md), then `shopdb-admin.ps1 check -Json`.
- A PC or a bay: the symptom table at the end of
[FLEET-ARCHITECTURE](FLEET-ARCHITECTURE.md) says which piece to open first.
- A deploy that half-worked: [UPGRADE](UPGRADE.md) and
[BACKUP-RESTORE](BACKUP-RESTORE.md).

236
docs/UPDATES-WINDOWS.md Normal file
View File

@@ -0,0 +1,236 @@
# What to expect from updates (Windows sites)
For the people who run a ShopDB-Flask server. It covers how updates arrive, what
they do to your data and your server, how long they take, and what happens to
anything else running on the same machine.
If you are the person building releases, see
[RELEASING-WINDOWS.md](RELEASING-WINDOWS.md).
## Updates arrive as one file
Every release is a single `.exe`, the same kind of file you used to install.
There is no patch, no separate updater, and no download step during the install:
everything the server needs is inside that one file, including the Python
runtime, all library code and the application itself.
To update, run the newer `.exe` on the server as Administrator. That is the
whole procedure.
The installer works out for itself that this is an update rather than a first
install, and skips what is already correct. An update typically takes two to
four minutes, most of which is the database migration.
## What an update changes, and what it leaves alone
Changed:
- The application code and the web interface.
- The database schema, brought forward by migrations.
- The Python runtime and libraries, but only when that release moves them.
Left exactly as they are:
- `.env`, which holds your database connection and secret keys.
- Your data. Updates migrate the schema; they do not reset or reload content.
- `web.config`, if you have edited it. The installer only ever repairs a
specific fault in it that older builds created, and copies the file aside
first when it does.
- Which features are switched on. The feature list opens showing what this site
already has.
- Uploaded files and anything under `instance\`.
Unticking a feature during an update does **not** remove it. Adding is a tick;
removing is a deliberate, separate step. This is so an upgrade can never quietly
delete a feature and its data.
## Downtime
The site is down for the length of the update, so two to four minutes. The
installer stops the application pool before replacing files, because Windows
will not let it overwrite files a running process holds open, and starts it
again afterwards.
There is no reboot. If a release ever needs one, the installer says so rather
than restarting the machine itself.
## Your data is backed up first
Before applying migrations, the installer takes a database backup and checks the
dump is readable. If a migration fails, it restores from that backup and tells
you.
This depends on `mysqldump` being present. Confirm it once, before your first
update:
```powershell
.\shopdb-admin.ps1 check
```
Without it the update still runs, but the pre-update backup is skipped, and that
is precisely the backup you would want if a migration went wrong.
Afterwards:
```powershell
.\shopdb-admin.ps1 status
.\shopdb-admin.ps1 verify
```
## Going backwards is refused
Installing an older build over a newer one is blocked outright. Once migrations
have moved the schema forward, older code cannot read it, and the failures are
difficult to unpick.
Keep the previous `.exe` until you are satisfied with a release. You cannot
download it again from the server, and it is what a rollback needs.
### Rolling back a release
Rolling back is restoring a matched pair: the code AND the database it expects.
Doing one without the other is the state the installer refuses to create, and it
is worth understanding why - newer code against an older schema fails at the
first query that touches a new column, while older code against a newer schema
often appears to work until something silently misreads a column that changed
meaning.
The update takes its own backup before touching the schema, so the pair you need
already exists.
```powershell
cd C:\shopdb-flask
.\shopdb-admin.ps1 stop
# 1. Database first, from the pre-upgrade dump the update took.
mysql -u root -p shopdb_flask < C:\ProgramData\ShopDB-Flask\backups\shopdb_flask-pre-upgrade-<stamp>.sql
# 2. Then the code: run the PREVIOUS installer .exe. It is an install over a
# restored older schema, which is the ordinary case, not a downgrade.
.\ShopDBFlask_Installer_<previous-version>.exe
# 3. Prove the pair matches.
.\shopdb-admin.ps1 check -Json
```
If a migration failed mid-update, the installer has already restored its own
backup and there is nothing to roll back - read the log it names and fix
forward. Rollback is for a release that installed cleanly and then behaved
badly, which is a different situation and the only one this procedure is for.
**Uploaded files are not in the dump.** If the release you are leaving behind
accepted uploads, restore `instance\` from your own backup as well, or those
files stay while the rows referring to them do not. See
[BACKUP-RESTORE.md](BACKUP-RESTORE.md).
## Will an update affect other sites on the same IIS server?
Mostly no, and the exceptions are listed here rather than glossed over.
**Isolated from other sites:**
- The application runs in its own application pool under its own identity, so a
crash or a memory leak cannot reach another site's pool.
- Its configuration lives in its own folder and applies only to its own URL
path. A parent site's own pages, including classic ASP, keep their existing
handlers.
- File permissions are granted on the application folder only.
**Shared, and therefore worth knowing about:**
- **A brief application pool recycle across the server.** Installing the IIS
modules and writing server-level configuration causes IIS to reload its
configuration, which recycles application pools. Requests in flight at that
moment can be dropped, and any session state other sites hold in memory is
lost. It is a few seconds and there is no service outage: IIS itself is never
stopped and no `iisreset` is issued.
- **Two IIS modules are installed machine-wide** the first time:
HttpPlatformHandler and URL Rewrite. Both are standard Microsoft modules. They
do nothing to a site that does not reference them, and if a site already uses
URL Rewrite its rules are untouched.
- **One rewrite server variable is permitted machine-wide**,
`HTTP_X_FORWARDED_FOR`, so the application can see real client addresses
instead of the loopback address. This grants exactly that one variable rather
than opening the section up.
- **A Microsoft C++ runtime** may be installed, which is shared and backwards
compatible.
- **The bundled database option installs MySQL on port 3306.** If this server
already runs MySQL, choose the existing-database option instead. Two servers
will collide on that port. The wizard asks before doing anything.
Removing ShopDB-Flask takes away its own site, application, pool, folder and
firewall rule. It deliberately leaves the shared IIS modules in place, because
another site may have started depending on them.
If your server hosts something critical, schedule updates in a maintenance
window for the pool recycle, not for the application itself.
## Security updates
Two kinds reach you, both as an ordinary `.exe`.
**Application fixes** are built from the source and shipped like any other
release.
**Third-party fixes** cover the Python runtime, the libraries, MySQL and the IIS
modules. These are pinned to exact versions and checked by cryptographic hash at
install time, so a release contains precisely the versions it claims and nothing
substituted. When one of them publishes a fix that affects this application, it
is picked up and a new release is issued.
Each release ships a machine-readable inventory of every third-party component
and its version, installed on the server as `sbom.cdx.json`. If your security
team asks whether you are exposed to a published vulnerability, that file
answers it without anyone guessing.
An update that is only a dependency bump is still worth taking: the version
number moves and the application behaviour does not.
## Check the file before running it
Each release publishes a SHA-256 checksum beside the `.exe`. Verify it:
```powershell
certutil -hashfile ShopDBFlask_Installer_<version>.exe SHA256
```
Compare with the published `.sha256` file.
Windows will warn about an unknown publisher, because the installer is not yet
code-signed. The checksum is the integrity check to rely on today. Get the file
from the agreed location rather than from mail or a message.
## Version numbers
Three parts, for example `0.7.0`:
- The last part changes for bug fixes and security fixes. Nothing you use
behaves differently.
- The middle part changes for new features. Existing features keep working.
- The first part changes for something that needs you to read the notes first.
Before 1.0 the middle number can still bring changes that need attention. Read
the release notes for those.
## If an update fails
The installer stops at the first problem rather than continuing, and says what
failed and what to do. Nothing is left half-applied: either the change is
complete or it is rolled back, and the log records everything either way.
The log is at:
```
C:\ProgramData\ShopDB-Flask\logs\shopdb-install-<timestamp>.log
```
Re-running the same `.exe` is safe and picks up from where it stopped. If it
fails again, send that log with your report; it names the failing step, the
exit code and the relevant output.
## See also
- [OPERATE-WINDOWS.md](OPERATE-WINDOWS.md) - day-to-day running
- [UPGRADE.md](UPGRADE.md) - upgrade notes across all deployment types
- [BACKUP-RESTORE.md](BACKUP-RESTORE.md) - what to back up and how to restore
- [INSTALL-WINDOWS.md](INSTALL-WINDOWS.md) - first-time install

39
docs/UPGRADE.md
View File

@@ -24,7 +24,7 @@ cp -a instance/ instance-backup-$(date +%F)/
git pull origin main
```
The application is distributed through the internal GE Aerospace Gitea; pull
The application is distributed through the internal GE Aerospace git server; pull
from there. There is no external image registry.
## Step 2: Rebuild
@@ -89,9 +89,9 @@ Confirm the app is healthy (login page renders, `/api/auth/login` returns a
## Version-specific notes
### Upgrading to v0.5.0 or later: bundled West Jefferson floor plan removed
### Upgrading to v0.5.0 or later: bundled the reference site floor plan removed
Versions before 0.5 shipped the West Jefferson facility floor-plan PNGs as the
Versions before 0.5 shipped the the reference site facility floor-plan PNGs as the
map default (`/static/images/sitemap2025-light.png` and `-dark.png`). v0.5+
removes those bundled PNGs and ships a generic placeholder SVG instead.
@@ -105,12 +105,43 @@ default need to act.
To check what your instance points at:
```bash
docker compose exec api flask shell -c "from shopdb.core.models.setting import Setting; print(Setting.query.filter(Setting.key.like('map_blueprint%')).all())"
docker compose exec -T api flask shell <<'PY'
from shopdb.core.models.setting import Setting
print(Setting.query.filter(Setting.key.like('map_blueprint%')).all())
PY
```
## Windows sites (installer-built)
Run a newer installer `.exe` over the existing install. That is the whole
procedure - none of the manual steps above apply.
[UPDATES-WINDOWS.md](UPDATES-WINDOWS.md) is the operator-facing version of this:
downtime, what is and is not touched, security updates, and the effect on other
sites sharing the same IIS server.
It backs the database up first and verifies the dump, applies the core and plugin
migrations, restores from that backup if a migration fails, and refuses to
install an older build over a newer one. Your `.env`, your data and your
`web.config` are kept.
Before the first upgrade, confirm `mysqldump` is available
(`.\shopdb-admin.ps1 check`). Without it the pre-upgrade backup is skipped, which
is the one you would want if a migration went wrong.
Afterwards:
```powershell
.\shopdb-admin.ps1 status
.\shopdb-admin.ps1 verify
```
See [OPERATE-WINDOWS.md](OPERATE-WINDOWS.md).
## See also
- [BACKUP-RESTORE.md](BACKUP-RESTORE.md) - what to back up and how to restore
- [INSTALL-WINDOWS.md](INSTALL-WINDOWS.md) - Windows Server install
- [CONFIG.md](CONFIG.md) - environment variables and Setting keys
- [DEPLOY.md](DEPLOY.md) - first-time deploy runbook
- `CHANGELOG.md` - what changed in each release

4
docs/adr/ADR-001-asset-as-platform-contract.md
View File

@@ -2,7 +2,7 @@
- **Status:** ACCEPTED
- **Date:** 2026-05-08
- **Deciders:** cproudlock
- **Deciders:** ShopDB maintainers
- **Supersedes:** none
## Context
@@ -183,7 +183,7 @@ Skipped from migration:
## References
- `shopdb/core/models/asset.py`
- `shopdb/core/models/machine.py` (legacy, deprecated)
- (core `Machine` model retired per this decision; machine data now owned by the machines plugin at `plugins/machines/models/machine.py`)
- `shopdb/plugins/base.py`
- ADR-002 (versioning of the surface)
- ADR-003 (plugin distribution)

2
docs/adr/ADR-002-plugin-versioning.md
View File

@@ -2,7 +2,7 @@
- **Status:** ACCEPTED
- **Date:** 2026-05-08
- **Deciders:** cproudlock
- **Deciders:** ShopDB maintainers
- **Supersedes:** none
## Context

4
docs/adr/ADR-003-plugin-distribution.md
View File

@@ -2,7 +2,7 @@
- **Status:** ACCEPTED
- **Date:** 2026-05-08
- **Deciders:** cproudlock
- **Deciders:** ShopDB maintainers
## Context
@@ -25,7 +25,7 @@ Three viable distribution models:
## Decision
**PROPOSED:** Use a **hybrid model** with two clearly-labeled paths.
Use a **hybrid model** with two clearly-labeled paths.
1. **Bundled plugins**: a small set of plugins ships with the framework, in-tree at `plugins/`. These are the reference implementations and the default install (printers, computers, network, equipment, usb, notifications). A site that wants only what's bundled needs no extra work.
2. **External plugins**: sister sites or third parties build plugins in their own git repos. The site running the framework drops the plugin into `plugins/<name>/` (clone, submodule, or symlink) and runs `flask plugin install <name>`. No pip packaging required for v1.

27
docs/adr/ADR-004-deployment-topology.md
View File

@@ -2,7 +2,7 @@
- **Status:** ACCEPTED
- **Date:** 2026-05-08
- **Deciders:** cproudlock
- **Deciders:** ShopDB maintainers
## Context
@@ -17,7 +17,7 @@ The codebase today is single-tenant per deployment. There is no `siteid` column,
## Decision
**PROPOSED:** **Per-site instances.** Each adopting site runs its own dedicated stack. The framework does not support multi-tenancy.
**Per-site instances.** Each adopting site runs its own dedicated stack. The framework does not support multi-tenancy.
Each site:
@@ -63,16 +63,21 @@ The framework provides:
## Migration strategy (resolved)
Deploys run a single core Alembic chain: `flask db upgrade`. Bundled plugins do
NOT carry their own migration chains - their tables are folded into the core
chain (migration `7c04_fold_plugin_schema`). This was a deliberate resolution of
the Phase 7B footgun where bundled-plugin baselines and the core baseline both
created the same tables, so `flask plugin upgrade-all` would conflict. A fresh
`flask db upgrade` reproduces the live schema exactly (verified on a scratch DB).
Deploys run two commands: `flask db upgrade` then `flask plugin upgrade-all`
(lean/ADR-014 sites add an optional `flask plugin prune-schema` at initial
provisioning). The core Alembic chain applied by `flask db upgrade` creates the
full core AND bundled-plugin schema through the chain head (this includes
migration `7c04_fold_plugin_schema`). But every bundled plugin still carries its
own Alembic chain per ADR-008: `flask plugin upgrade-all` stamps each plugin's
own chain (the `alembic_version_<plugin>` tables) and applies any plugin-specific
migrations added after the ownership cutover. The earlier Phase 7B state that
folded everything into core with no per-plugin chains was superseded by ADR-008's
per-plugin ownership. A fresh `flask db upgrade` reproduces the live core schema
exactly (verified on a scratch DB).
External (out-of-tree) plugins per ADR-003 may still ship their own migrations;
the framework supports per-plugin chains for them. Only the in-tree bundled
plugins are consolidated into core.
External (out-of-tree) plugins per ADR-003 ship their own migrations too; the
framework runs the same per-plugin chain mechanism (ADR-008) for both bundled and
external plugins.
## Open questions

4
docs/adr/ADR-005-equipment-vs-measuringtools.md
View File

@@ -2,7 +2,7 @@
- **Status:** ACCEPTED
- **Date:** 2026-05-08
- **Deciders:** cproudlock
- **Deciders:** ShopDB maintainers
- **Supersedes:** none
## Context
@@ -145,5 +145,5 @@ Reclassification is one-shot, run once, archived. Like the original migration sc
- ADR-001 (Asset is platform contract)
- ADR-002 (versioning of the surface)
- `plugins/equipment/` (current placeholder)
- `plugins/machines/` (equipment plugin, renamed per ADR-011)
- `plugins/computers/` (existing example of plugin pattern)

4
docs/adr/ADR-006-collector-contract.md
View File

@@ -2,7 +2,7 @@
- **Status:** ACCEPTED
- **Date:** 2026-05-08
- **Deciders:** cproudlock
- **Deciders:** ShopDB maintainers
- **Supersedes:** none
## Context
@@ -145,4 +145,4 @@ Migration path:
- `shopdb/plugins/base.py` (`get_collector_schema` + `apply_collector_payload` hooks)
- ADR-001 (asset model the collectors target)
- ADR-002 (collector schema is part of plugin contract; changes to the hook signature are major bumps)
- The PXE project (`/home/camp/projects/pxe/`) which feeds the computers collector
- The PXE project (the PXE imaging project) which feeds the computers collector

8
docs/adr/ADR-007-product-versioning-and-releases.md
View File

@@ -2,7 +2,7 @@
- **Status:** ACCEPTED
- **Date:** 2026-07-10
- **Deciders:** cproudlock
- **Deciders:** ShopDB maintainers
- **Supersedes:** none
## Context
@@ -88,8 +88,8 @@ To cut release `X.Y.Z`:
### Neutral
- CI (`.gitea/workflows/ci.yml`) runs the backend tests, the naming/style
gate, and the frontend build on push and PR. It is best-effort: Gitea
- The internal CI workflow runs the backend tests, the naming/style
gate, and the frontend build on push and PR. It is best-effort: the internal CI
Actions availability on the host is unverified, so the workflow is
config-only until a runner is confirmed.
@@ -112,4 +112,4 @@ To cut release `X.Y.Z`:
- `shopdb/__init__.py` (`__version__`, `__contract_version__`)
- `CHANGELOG.md` (release record)
- `frontend/package.json` (frontend version, kept in lock-step)
- `.gitea/workflows/ci.yml` (CI gate)
- the internal CI workflow (CI gate)

2
docs/adr/ADR-008-plugin-migration-ownership.md
View File

@@ -2,7 +2,7 @@
- **Status:** ACCEPTED
- **Date:** 2026-07-10
- **Deciders:** cproudlock
- **Deciders:** ShopDB maintainers
- **Supersedes:** the "Migration strategy (resolved)" section of ADR-004
## Context

6
docs/adr/ADR-009-frontend-plugin-gating.md
View File

@@ -2,7 +2,7 @@
- **Status:** ACCEPTED
- **Date:** 2026-07-10
- **Deciders:** cproudlock
- **Deciders:** ShopDB maintainers
- **Supersedes:** none
## Context
@@ -41,7 +41,9 @@ list. This is the whole of what ships now.
anonymous callers is safe because `GET /api/dashboard/navigation`
already leaks the same enabled/disabled signal, and unauthenticated
kiosk routes (`/tv`) need the answer too. It carries no metadata, so it
reveals strictly less than the admin-gated `GET /api/plugins`.
reveals strictly less than the equally-anonymous but metadata-carrying
`GET /api/plugins` (also `jwt_required(optional=True)`; only the
`PUT /api/plugins/<name>` toggle is admin-gated).
2. **Route tagging.** Every plugin-owned route carries `meta.plugin =
'<pluginname>'`. This covers the per-plugin route modules

6
docs/adr/ADR-010-frontend-plugin-hooks.md
View File

@@ -3,7 +3,7 @@
- **Status:** ACCEPTED
- **Date:** 2026-07-11
- **Accepted:** 2026-07-11
- **Deciders:** cproudlock
- **Deciders:** ShopDB maintainers
- **Supersedes:** none
## Context
@@ -43,7 +43,7 @@ one would mean forking a core view:
printer detail page, a calibration-status card on a measuring tool. Today
`WarrantyPanel.vue` is composed in by hand-editing each detail view.
4. **Map marker / overlay contributions.** A calibration-due badge on the
shop-floor map (`frontend/src/views/map/MapView.vue`). The map draws type
shop-floor map (`frontend/src/views/MapView.vue`). The map draws type
colors but has no plugin decoration path.
5. **Search-result rendering / routing for plugin asset types.** Global search
returns assets, but core hardcodes how each type renders and where its detail
@@ -316,5 +316,3 @@ sites.
- docs/CONTRACT-STABILITY.md (the expected-churn line this ADR answers)
- shopdb/plugins/base.py (existing declarative hooks this pattern extends)
- shopdb/core/api/dashboard.py, shopdb/core/api/reports.py (consumer precedent)
</content>
</invoke>

2
docs/adr/ADR-011-machines-rename.md
View File

@@ -2,7 +2,7 @@
- **Status:** ACCEPTED
- **Date:** 2026-07-11
- **Deciders:** cproudlock
- **Deciders:** ShopDB maintainers
- **Relates to:** ADR-005 (equipment vs measuringtools scope), ADR-008 (plugin migration ownership)
## Context

4
docs/adr/ADR-012-geenforce-manifest-ownership.md
View File

@@ -2,7 +2,7 @@
- **Status:** ACCEPTED
- **Date:** 2026-07-13
- **Deciders:** cproudlock
- **Deciders:** ShopDB maintainers
- **Relates to:** ADR-002 (plugin contract versioning), ADR-004 (per-site
deployment), ADR-006 (collector contract), ADR-008 (per-plugin Alembic chains)
@@ -18,7 +18,7 @@ We want shopdb to own the manifests as data (author, version, publish, roll
back) and to observe fleet compliance, while NOT taking on the GE-Enforce engine
itself (which is the GE-Enforce framework's, maintained separately) and NOT
dictating any site's imaging path (per ADR-004, each site is single-tenant with
its own provisioning - PXE at West Jefferson, OOBE provisioning packages at
its own provisioning - PXE at the reference site, OOBE provisioning packages at
others).
The manifests are an enforcement PROGRAM, not an application inventory: entry

413
docs/adr/ADR-013-plugin-catalog-and-lean-builds.md Normal file
View File

@@ -0,0 +1,413 @@
# ADR-013: Plugin Catalog, Curated Shelf, and Lean Per-Site Builds
- Status: ACCEPTED
- Date: 2026-07-18
- Deciders: ShopDB maintainers
- Relates to: ADR-002 (contract versioning), ADR-003 (plugin distribution), ADR-004 (per-site instances), ADR-008 (per-plugin migrations), ADR-009 (frontend plugin gating), ADR-010 (frontend hook contract)
## Context
Every site today ships identical code. The backend image contains all 13 plugin
directories (Dockerfile COPY at line 55; the header comment listing "eleven core
plugins" is stale), and the SPA compiles every plugin's routes and views via a
static glob (frontend/src/router/index.js:11) plus hardcoded imports. A site's
"chosen set" exists only as runtime enable flags in instance/plugins.json.
Disabled is not absent: a site that never wants printedparts/usb/network still
ships, and can execute, that code.
Distribution per ADR-003 is "drop a directory into plugins/ by hand". There is
no artifact format, no signing, no catalog, no validate gate, and the loader
trusts whatever it finds on disk (loader.py:61-73 discovers any folder with a
plugin.py; loader.py:185-224 loads it; migrations.py:29-62 runs its DDL with
full DB privileges). Plugins run in-process with the full shopdb.api surface
including db, so the only tenable security model on air-gapped GE networks is
curation plus cryptographic provenance, enforced everywhere code can execute,
not sandboxing.
Known defects this ADR also resolves:
- upgrade_all_plugins checks hasattr(registry, 'list_installed') which never
exists (registry has only get_all/get_enabled_plugins), so it always falls
back to migrating every folder on disk, adopted or not (__init__.py:94).
- PILOT-DEPLOY.md enables plugins that were never installed; enable refuses.
There is no declarative "apply this chosen set" operation.
- Reverse-dependency checks on uninstall/disable read only LOADED plugin
instances, so an installed-but-unloaded dependent is invisible.
- Dependency install/enable is check-only; nothing computes a closure, and
install-a-dependent can fail at its own load step because the dependency was
installed disabled (default_enabled=false on employees).
- The dependency sort has no cycle detection.
- Soft couplings (geenforce -> computers, notifications -> employees) are
invisible to the manifest graph.
Frontend reality check (this drove the design below): plugin UI is NOT one
folder per plugin. Routes live in routes/<plugin>.js (slides has none), inside
the shared core.js (computers report, printers toner report, employees detail,
slides settings), and as six hardcoded top-level imports in index.js itself
(/parts-kiosk, /tv, /print/printer-qr x2, /print/usb-labels,
/print/printedparts-labels). View dirs mismatch plugin names (computers ->
views/pcs). Plugin settings cards sit in shared views/settings/
(DellWarrantySettings, ZabbixSettings, SlideManager, EmployeeDirectory,
MeasuringToolTypesList, PrintedPartsSettings), plugin print views in shared
views/print/, and some views span plugins (AssetLabel.vue serves five asset
types; PCDetail imports WarrantyPanel). Any lean-frontend design that only
moves routes/*.js and views/<plugin>/ fails the build the moment a plugin is
pruned. This ADR scopes that work honestly instead of calling it mechanical.
## Decision
### 1. Tiering: mandatory core is the core package; all plugins are catalog-optional
- The mandatory core is the non-plugin shopdb/core/ package (auth, users,
assets, locations, vendors, models, settings, audit, dashboard, search,
reports, plugin management). It already survives every plugin being absent
via hasattr/lazy-import guards. No plugin is promoted into it.
- New optional manifest field `tier: "core" | "optional"`, default "optional".
All 13 existing manifests are unchanged and unchanged in meaning. The
lifecycle gains a guard: uninstall_plugin and disable_plugin refuse a
tier:core plugin (alongside the reverse-dependency checks at
__init__.py:269-278 and :351-360). No plugin ships tier:core initially; the
field and guard exist so a future curation decision is a manifest edit, not a
framework change.
- Per-site mandates live in the site profile (section 5): a `locked` list the
profile applier refuses to remove. This preserves ADR-004 site autonomy: a
wing site can mandate usb without the framework mandating it fleet-wide.
- New manifest field `optional_dependencies: []` (names only, loader-ignored).
Declared for the verified soft couplings: geenforce lists computers
(service.py:32-43 loses app-detection gates without it), notifications lists
employees (routes.py:151-171 loses name/photo enrichment). Catalog listing
and adopt WARN on unmet optional deps; nothing blocks.
- Hard `dependencies` gains optional PEP440 ranges ("employees>=1.1").
validate/adopt honor ranges; the runtime loader keeps name-only semantics
(specifier stripped) so no loader behavior changes. The single existing hard
edge printedparts -> employees stays as-is; whether it can relax to optional
(badges.py has an external HR fallback) is a follow-up product question, not
blocked on this ADR.
- Dependency plumbing fixes: _sort_by_dependencies gains cycle detection
(raise PluginDependencyError on a back edge); reverse-dependency checks read
manifests of ALL installed plugins from disk, not loaded instances.
### 2. Packaging: signed, versioned artifacts
Artifact: `<name>-<version>.shopdbplugin` (a zip of the plugin directory:
manifest.json, plugin.py, api/, models/, migrations/, and frontend/ once
section 6 lands) plus two members generated at pack time:
- `PROVENANCE.json`: plugin name, version, publisher id, build timestamp, and
a sorted map of every packaged file path to its SHA-256. PROVENANCE.json is
not listed in its own map, so there is no circular-hash problem and no zip
canonicalization needed; determinism comes from sorted per-file hashes.
- `PROVENANCE.sig`: detached ed25519 signature over the exact PROVENANCE.json
bytes.
New CLI:
- `flask plugin pack <name> --key <path>` (producer side): runs validate on the
directory, then emits the artifact.
- `flask plugin validate <dir|artifact>` (the missing pre-publish gate),
fail-closed pipeline: signature (artifact mode) -> per-file hashes -> manifest
against a new docs/plugin-manifest.schema.json -> name == directory ->
core_version parses as a specifier and admits the target contract version ->
static import-surface scan reusing tests/test_plugin_contract.py logic ->
alembic versions parse. The schema types the known fields (name, version,
description, dependencies, optional_dependencies, tier, core_version,
api_prefix, display_name, default_enabled, provides, settings) and PERMITS
additional properties, so all 13 existing manifests pass unmodified.
The import-surface scan is documented as a lint, not a security control; it is
trivially bypassed by dynamic import. The security control is human review
before signing (section 4).
### 3. The shelf: a read-only folder, transport-agnostic by design
- One config knob: `PLUGIN_SHELF_DIR`. The app only ever reads this folder. It
never speaks SharePoint, OneDrive, or any network protocol.
- Transport is explicitly out of scope and explicitly untrusted. On networks
that can reach corporate M365, a SharePoint document library sync populates
the folder. On strictly air-gapped floors where no sync agent can run, the
folder is populated by robocopy/USB. Both are equally supported and equally
untrusted, because every decision-bearing byte is signed: swapping transport
changes nothing about the trust model.
- Layout: `<shelf>/<name>/<name>-<version>.shopdbplugin` plus
`shelf-index.json` and `shelf-index.sig`.
- The index is SIGNED with the same publisher key and carries a monotonically
increasing `serial` plus a `revoked` list of name-version pairs. Each site
records the last-seen serial in instance state and refuses an index with a
lower serial (anti-rollback of the catalog itself). The index also carries
per-entry version/tier/core_version so `flask plugin shelf-list` can display
compatibility without unpacking, but the index is a BROWSE layer only:
adopt reads dependencies, tier, and core_version from the signed manifest
inside the verified artifact, never from the index.
- Trusted keys: `PLUGIN_TRUSTED_KEYS` is a list of pinned public keys delivered
out-of-band in the site's deployed config/image. Keys are NEVER read from the
shelf; a folder that can be written by an attacker must not also carry the
keys that authenticate it. Multiple pinned keys allow overlap rotation.
Revocation of an artifact rides the signed index `revoked` list; a
`flask plugin audit` command warns when an installed version appears there.
- Partial-sync robustness: adopt copies the artifact to a temp location,
verifies signature and every file hash there, then unpacks to
plugins/.staging/<name> and renames into place atomically. OneDrive
placeholder stubs, zero-byte files, or an index referencing not-yet-synced
artifacts all fail closed with a clear "artifact not fully synced/verified"
error.
- `flask plugin adopt <name>[==version]`: resolve version from the shelf,
verify, compute the hard-dependency closure from signed manifests, then for
each closure member in topological order: unpack, INSTALL, and ENABLE (not
install-only; the load gate at loader.py:201-206 checks is_enabled, so an
install-only closure with default_enabled=false deps would fail its own
load). Migrations run via the unchanged per-plugin chain (ADR-008). Refuses
to adopt a version lower than the installed one unless
`--force-downgrade` is given interactively. Prints the restart notice.
- Adopt/install/uninstall remain CLI-only. The admin HTTP surface stays a
read-only catalog view plus the existing enable/disable toggle; because
Flask cannot register blueprints after the first request, any adopt or
enable takes full effect only on restart, and the UI says so. There is no
"install button" that pretends otherwise.
### 4. Trust model: verify at adopt AND at every load and migrate
Signing that gates only adoption is bypassable through every other write path
into plugins/ (git clone, symlink, USB drop) and defeated by post-adoption
tampering. Therefore verification is enforced where code executes:
- Adoption leaves PROVENANCE.json and PROVENANCE.sig inside plugins/<name>/
and records publisher + artifact hash in the registry entry.
- load_plugin verifies the signature against PLUGIN_TRUSTED_KEYS and re-hashes
the plugin tree against the provenance file map BEFORE importing plugin.py
(new step ahead of loader.py:185). Missing or invalid provenance is a
fail-closed refusal in production.
- run_plugin_migrations performs the same verification before executing any
revision, so a routine `flask plugin upgrade-all` can never run DDL from an
unverified folder.
- upgrade_all_plugins iterates registry.get_all() (fixing the phantom
list_installed fallback at __init__.py:94), so unadopted on-disk folders are
never migrated as a side effect of deploys.
- Development and the ADR-003 external-repo/symlink workflow (including
scripts/test-external-plugin.sh) are preserved via `PLUGIN_DEV_TRUST_DIRS`,
honored ONLY when DEBUG or TESTING is set. Production ignores it.
- Cost: hashing 13 small plugin trees at boot is milliseconds; accepted.
What signing does NOT claim: a valid signature proves the artifact is exactly
what a curator reviewed and signed, nothing more. Plugins remain in-process
Python with full DB access. The actual safety control is the human review
before signing; the signature makes that review's verdict tamper-evident all
the way to execution.
### 5. Declarative site profiles and lean backend builds
- `site-profile.json` per site (kept in the site's deploy config): site name,
list of chosen plugins, optional `locked` list. `flask plugin apply-profile
<file>` resolves the closure, installs AND enables in dependency order, runs
migrations, reports which changes need a restart. This replaces the
imperative CLI sequences in DEPLOY.md/PILOT-DEPLOY.md and fixes the
enable-without-install bug.
- Lean backend image: `scripts/build-site.sh` reads the profile and stages
only core + chosen plugin directories into the Docker build context
(correcting the Dockerfile COPY and its stale header comment). Discovery
needs no change; it already scans whatever exists.
- Prerequisite the naive version misses: core hardcodes plugin imports.
shopdb/core/api/search.py (~15 sites), reports.py, assets.py, collector.py,
applications.py, auditlogs.py, and shopdb/cli/__init__.py import
plugins.<name>.* lazily. Some already guard ImportError; ALL must, with
graceful degradation, before any site prunes a folder. This is audited and
enforced by a new CI job that deletes one plugin directory and runs the full
test suite (repeated per plugin). Longer term these aggregators should move
to registry-driven contract hooks (get_search_providers/get_report_sources)
so a new catalog plugin can join search/reports without core edits; that is
scoped as follow-up work, not a blocker for lean builds.
- Schema-lean is DEFERRED to its own ADR. The core baseline 68b3947ae14f
unconditionally creates the 10 pre-cutover plugins' tables, and lifting them
into plugin baselines collides with cross-plugin foreign keys (the
computers-owned installedapps table FKs machines.machineid while computers
declares no dependency on machines). Reversing the cutover would either
introduce undeclared hard deps or drop FKs; neither is decided here. A lean
site therefore carries a handful of empty pre-cutover tables. Accepted.
### 6. Frontend delivery: Path C for rich UIs, Path A for simple ones, Path B rejected
Path B (runtime-loaded JS / module federation) is REJECTED: it moves executable
UI delivery from a signed, statically auditable build artifact to runtime
fetching, which is exactly the wrong direction for an air-gapped,
review-then-sign posture, for zero benefit given restarts are already required.
Path A (declarative JSON UI over generic renderers) is COMMITTED and scheduled
EARLY: the three unwired ADR-010 endpoints (pluginui.py asset-panels:62,
map-overlays:88, asset-presentation:100) get generic core renderers, joining
the already-consumed settings-cards. After this, a simple plugin ships JSON-only
UI with zero frontend build involvement. Sequencing this before the relocation
gives every plugin an escape hatch during the migration instead of after it.
Path C (self-contained plugin frontend) is the primary mechanism, scoped
against the real code, not the idealized layout:
- Canonical home: plugins/<name>/frontend/ containing routes.js (the plugin's
complete route array, INCLUDING routes currently embedded in index.js and
core.js), views/, and settings views.
- A pre-Vite staging step (scripts/stage-frontend.mjs, run by build-site.sh
and the dev script) copies the CHOSEN plugins' frontend/ into
frontend/src/.plugins-staged/<name>/ (gitignored) and generates two files
inside the Vite root: routes.gen.js (aggregated plugin routes) and
meta.gen.js (plugin-supplied icon names, title spellings, settings-standalone
flags, replacing the hardcoded iconMap/TITLE_SPELLINGS/SETTINGS_STANDALONE in
AppLayout.vue, settingsCatalog.js, and index.js). This exists because
import.meta.glob requires a static literal inside the project root and
cannot select a per-site subset by itself.
- ONE-TIME core-router surgery, done first and called what it is: the six
hardcoded plugin-view imports in index.js (PartsKiosk, TVDashboard,
PrinterQRBatch/Single, USBLabelBatch, PrintedPartsLabels) and the plugin
routes embedded in core.js move into their owning plugins' routes.js. Without
this, pruning slides/printers/usb/printedparts fails the Vite build on
unresolvable imports; no amount of glob work fixes it.
- Per-plugin relocation PRs (13), each REAL WORK, not a file move: carve routes
out of shared files, move views (handling name mismatches like computers ->
views/pcs), move the plugin's settings views out of shared views/settings/,
and rewrite relative ../../ imports of core shared code to the @/ alias
(relative paths break at the staged depth). A lint rule enforces alias-only
core imports in plugin frontend code from then on.
- Shared plugin-aware code STAYS CORE and ships to every site: AssetLabel.vue
(spans five asset types), views/print helpers (assetLabel.js, qrLogo.js),
MachineBadge.vue, and cross-plugin panels like WarrantyPanel used by
PCDetail. These already null-guard or gate via isPluginEnabled and must keep
degrading when a peer plugin is absent; over time they migrate to ADR-010
asset-panels so the data becomes plugin-supplied. Lean v1 therefore prunes
plugin-EXCLUSIVE code; a small plugin-aware core remainder is accepted and
shrinks as Path A absorbs it.
- Dual-location transition: the staging step unions legacy locations
(routes/*.js glob, views/<plugin>/) with plugins/<name>/frontend/ until each
plugin has moved. The SPA builds green at every commit; each plugin's move is
independently revertable until the legacy glob is removed at the end.
- Nav and settings cards are already server-driven (dashboardApi.navigation,
settings-cards); the remaining hardcoded plugin entries in settingsNav.js
(/settings/zabbix, /settings/dellwarranty) move to those plugins'
get_settings_cards so pruning leaves no dead links.
### 7. Effect on the 13 existing plugins
- Backend: ZERO code changes required. tier/optional_dependencies/provenance
are additive; pack zips the directory as-is; all plugins keep passing
tests/test_plugin_contract.py. Bundled plugins in a site's image get
provenance stamped at build time by pack, so verify-at-load applies to them
identically.
- Frontend: one relocation PR each, of the honest scope above. Until a
plugin's PR lands it keeps working from its legacy location.
- Operationally nothing changes for a site that does nothing: default builds
remain all-plugins, apply-profile is opt-in, and enable/disable semantics
(including the restart requirement) are unchanged.
## Consequences
### Positive
- A real catalog: sites declare their set in site-profile.json and apply it in
one idempotent command; the chosen set drives backend image, SPA bundle, and
runtime state from one source of truth.
- Curated marketplace with end-to-end provenance: review -> sign -> any
transport -> verify at adopt, at load, and at migrate. Transport (SharePoint
sync or sneakernet) is untrusted and interchangeable, which is exactly right
for air-gapped sites.
- Lean per-site builds: unchosen plugins exist in neither the image nor the
bundle, shrinking attack surface and download size.
- Fixes shipped along the way: upgrade-all migrating unadopted folders,
PILOT-DEPLOY install/enable ordering, reverse-dep checks blind to unloaded
plugins, missing cycle detection, missing dependency closure, hardcoded
frontend plugin metadata.
- Path A completion makes simple plugins UI-capable with no build glue, which
is the cheapest possible marketplace onboarding.
### Negative
- Key management is a per-site operational burden: pinned keys delivered
out-of-band, rotation is a config change everywhere. Accepted as the price of
not trusting the distribution folder.
- The frontend re-org is the long pole: one core-router surgery plus 13
non-trivial PRs. It is sequenced to be always-green and per-plugin
revertable, but it is weeks of work, not a rename.
- Schema is not lean: pre-cutover plugin tables still appear at every site
until the deferred baseline re-org ADR.
- Restarts remain required after adopt/enable (Flask blueprint constraint);
the marketplace UX is honest about it rather than working around it.
- Boot adds a signature + tree-hash check per enabled plugin (milliseconds,
but nonzero).
### Risks
- Key compromise or curation failure: a signature proves provenance, not
safety; a compromised pinned key or a rubber-stamp review signs malware that
every gate will happily pass. Mitigations: multi-key pinning with overlap
rotation, signed revocation list with monotonic index serial, and keeping the
signing key offline with the curator. The static import scan is a lint and
must never be presented as a boundary.
- Rollback/downgrade: mitigated three ways: index serial monotonicity, adopt
refusing version downgrades without interactive --force-downgrade, and the
signed revoked list. Residual risk: a site that never syncs a newer index
cannot learn of revocations; `flask plugin audit` at deploy time narrows the
window.
- Version skew across ADR-004 sites: one shelf serves sites at different
contract versions. Adopt checks core_version from the signed manifest against
the site's own __contract_version__ (authoritative); shelf-list shows an
advisory compatibility column from the index. Incompatible artifacts are
listable but not adoptable.
- Partial/placeholder sync files: fail closed on hash verification; the error
message distinguishes "not fully synced" from "tampered" only by wording,
intentionally, since the app cannot tell.
- Dev-trust misuse: PLUGIN_DEV_TRUST_DIRS silently ignored outside
DEBUG/TESTING; a prod config carrying it gets a startup warning.
- Frontend closure drift: plugin views importing cross-plugin components is a
graph the manifest does not model. The lint rule (plugin frontend may import
core @/ paths and its own tree only, never another plugin's) prevents new
edges; existing shared plugin-aware code is explicitly core-owned.
## Implementation phases
- Phase 0, groundwork (small, days): upgrade_all_plugins uses
registry.get_all(); reverse-dep checks read installed manifests from disk;
cycle detection in _sort_by_dependencies; shopdb/plugins/manifest_schema.json +
`flask plugin validate` (directory mode); `flask plugin apply-profile` with
install+enable closure ordering; fix Dockerfile stale comment. All additive,
zero risk to running sites.
- Phase 1, packaging and signing (medium, about a week): PROVENANCE format,
`flask plugin pack`, validate artifact mode, PLUGIN_TRUSTED_KEYS config,
ed25519 signing tooling and curator docs. No runtime behavior change yet.
- Phase 2, shelf and enforcement (medium-large, one to two weeks):
PLUGIN_SHELF_DIR, signed shelf-index with serial + revoked list,
`flask plugin shelf-list` / `adopt` / `audit` with atomic verified unpack;
verify-at-load in load_plugin and verify-at-migrate in
run_plugin_migrations, fail-closed in prod; PLUGIN_DEV_TRUST_DIRS for
dev/test and the external-repo harness; tier:core lifecycle guard;
provenance stamping of bundled plugins at build. This phase completes the
security model; everything after it is delivery optimization.
- Phase 3, Path A completion (medium, one to two weeks): generic renderers for
asset-panels, map-overlays, asset-presentation; migrate settingsNav.js
hardcoded plugin cards to get_settings_cards. Done BEFORE relocation so
JSON-only UI is available during the migration.
- Phase 4, frontend re-org (large, the long pole, several weeks elapsed):
stage-frontend.mjs staging + routes.gen.js/meta.gen.js codegen; ONE core PR
moving the six index.js hardcoded plugin imports and the core.js-embedded
plugin routes into plugin route files; then 13 per-plugin relocation PRs
(views, settings views, name-mismatch dirs, @/ alias rewrite) under the
dual-location union; lint rule for plugin frontend imports. Always-green,
per-plugin revertable.
- Phase 5, lean builds end to end (medium, about a week after Phase 4):
build-site.sh staging backend dirs + frontend staging from site-profile.json;
core lazy-import guard audit finished, enforced by the delete-a-plugin CI
matrix; remove the legacy glob; pilot one real lean site (a location without
printedparts/usb/network) and diff its image and bundle against a full build.
Deferred, each to its own future decision: schema-lean core-baseline re-org
(blocked on the installedapps -> machines FK question), pip/entry-point
distribution (ADR-003 v2), hook-based search/report aggregation contract, and
any revisit of Path B.
## Amendment, 2026-08-14
Status corrected from PROPOSED to ACCEPTED. The lean-build half of this decision
shipped some time ago - `scripts/build-site.sh`, `SITE_PLUGINS` staging,
`flask plugin prune-schema` and `default_enabled: false` are all live, and
ADR-014 was accepted on top of it - while the record still said the decision was
under consideration. A decision that has been implemented and depended upon is
not proposed, whatever the header says, and leaving it that way makes every
other status in the index worth less.
The catalog and signed-artifact parts of the decision remain unbuilt. That is
the ordinary state of an accepted decision: accepted means settled, not
delivered.

142
docs/adr/ADR-014-schema-lean-per-site.md Normal file
View File

@@ -0,0 +1,142 @@
# ADR-014: Schema-lean per-site builds (retire cross-plugin FKs, lift plugin tables)
- Status: ACCEPTED
- Date: 2026-07-19
- Deciders: ShopDB maintainers
- Relates to: ADR-008 (per-plugin migration ownership), ADR-013 (plugin catalog + lean per-site builds), ADR-001 (asset model)
## Context
ADR-013 delivered lean per-site builds for plugin CODE (backend tree + frontend
bundle carry only chosen plugins). One residual was left, explicitly deferred:
the DATABASE. The core Alembic baseline (68b3947ae14f) creates EVERY table,
including ~30 plugin-owned tables (PLUGIN_TABLE_OWNERS). Each plugin's own
baseline is a stamp-only no-op (the core chain already made its tables, per
ADR-008). So a lean site that omits a plugin still creates that plugin's tables,
empty and unused.
The deferral cited a blocker: "the computers-owned installedapps table FKs
machines.machineid while computers declares no dependency on machines; reversing
the cutover would introduce undeclared hard deps or drop FKs; neither is
decided." Investigation refined this:
The cross-boundary foreign keys into the machines plugin table are ALL held by
DEAD legacy tables/columns that predate the asset model (ADR-001) and the
per-plugin cutover (ADR-008), and are queried nowhere in the codebase:
- `machinerelationships` (child/parentmachineid -> machines) - superseded by
`assetrelationships`. No model, no query.
- `printerdata` (machineid -> machines) - the pre-cutover printers table,
superseded by the printers plugin. No model, no query.
- `installedapps` (machineid -> machines) - a standalone machine-app link table;
the live relationship is `computerinstalledapps` (FK to computers only). The
standalone table has no model, no query.
- `communications.machineid` (-> machines) - a legacy column on the core
communications table (which is now assetid-based). Not read anywhere.
No LIVE plugin table hard-FKs another plugin's table. computerinstalledapps FKs
only computers.computerid (intra-plugin). So the blocker is dead cruft, not
live design.
## Decision
Two phases, both leaving existing databases correct.
### Phase 1: retire the dead cross-boundary cruft - ALREADY DONE
Investigation found this is already accomplished by existing migrations:
`7a01_adr001_position_contract` and `7c01_drop_legacy_machine` drop
`machinerelationships`, `printerdata`, `installedapps`, and
`communications.machineid` (with its FK). The current schema (verified on the
dev database) has none of them. So the cross-plugin FK blocker ADR-013 cited no
longer exists in the live schema - only in the baseline's transient
create-then-later-drop. No new migration is needed for Phase 1.
Precedent: ADR-001 dropped a cross-plugin FK the same way
(usbcheckouts.machineid -> machines became a soft sentinel).
### Enabling change (executed now): idempotent create_plugin_tables
`shopdb/plugins/alembic_template.py:create_plugin_tables` now skips any table
that already exists (inspects the bind first) instead of raising. This is the
mechanism Phase 2 needs: a plugin anchor can create its tables on a fresh lean
install AND be a safe no-op on an existing database that already has them from
the pre-cutover core baseline. Correct and inert regardless of Phase 2 (no
current caller creates against a populated schema). Verified against the
plugin-migration suite.
### Phase 2 (executed): prune not-installed plugin tables after upgrade
Two mechanisms were weighed to make a lean site's database carry only
core + chosen-plugin tables:
- **Relocate** (rejected): pull every plugin-table create/alter out of the core
chain into the plugin baselines, so the core chain never creates a
not-installed plugin's table. Measurement killed this: plugin tables are
created and altered across ~15 released core migrations (baseline plus 7c04,
7d05, 7d08, 7d13, 7d15, 7d16, 7d17, ...), not just the baseline. Because the
whole core chain runs before any plugin chain, removing a table's create from
core while a later core migration still alters it breaks FULL installs too, so
relocation means surgically rewriting ~15 released migrations - the highest
blast radius in the project - for a purely cosmetic gain (the omitted tables
are empty and the lean CODE build already never loads the plugin).
- **Prune-after-upgrade** (chosen): leave the entire core chain untouched. Add
`flask plugin prune-schema`, which drops the tables of every plugin in
PLUGIN_TABLE_OWNERS that is not installed on this site. Run once at deploy,
after `flask db upgrade` and `flask plugin upgrade-all`. Same end state
(core + chosen tables) with near-zero blast radius: no released migration is
edited, and an existing full site is unaffected because it never runs the
command.
`prune-schema` drops by table name (no plugin-code import), so it works on a
lean image where the omitted plugin's directory is absent. It is a dry-run by
default and refuses to drop a table that holds rows unless `--force`, so a
misfire on a populated site cannot silently delete data. Because the core chain
seeds a few plugin reference tables (e.g. 7d05 inserts default access
protocols), initial lean provisioning uses `--force` - at that point the tables
hold only migration-seeded defaults, before any site data exists.
The idempotent `create_plugin_tables` (enabling change above) is what lets a
lean site later ADD an omitted plugin: its anchor recreates the pruned tables.
Verified end to end on MySQL: fresh full install (86 tables) then prune is a
no-op; fresh lean install (machines + printers) then prune drops the other 19
plugin tables, leaving core + chosen; second prune is a no-op; the non-empty
guard refuses without `--force`. Four SQLite regression tests pin the behavior
(tests/test_plugin_prune_schema.py), running in the backend CI job via the real
CLI runner: drop-only-not-installed, full-site no-op, refuse-non-empty, and
force-drops-non-empty.
## Consequences
### Positive
- A lean site's database contains only core + chosen-plugin tables, with no edit
to any released migration (near-zero blast radius).
- The cross-plugin FK blocker ADR-013 cited is gone (dead cruft, dropped by
existing migrations), so plugin schemas are already FK-independent.
- Adding an omitted plugin to a lean site later just works: the idempotent
anchor recreates its tables.
### Negative / risk
- prune-schema is destructive by nature; the row-count guard + dry-run default +
required `--force` for non-empty tables contain that. It is a deploy-time
provisioning step, not something to run casually on a live populated site.
- A lean fresh install still transiently creates then drops the omitted plugins'
tables (the core chain builds them, prune removes them). Harmless and one-time
at provisioning; the trade for not touching the released baseline.
## Implementation
- Phase 1: nothing to do - the dead cross-boundary FK objects were already
dropped by existing migrations `7a01_adr001_position_contract` and
`7c01_drop_legacy_machine`; verified absent on a fresh full MySQL upgrade.
- Enabling change: `create_plugin_tables` made idempotent
(`shopdb/plugins/alembic_template.py`).
- Phase 2: `flask plugin prune-schema` (`shopdb/plugins/cli.py`), dry-run by
default, `--yes` to execute, `--force` for non-empty tables. Deploy order:
`flask db upgrade` -> `flask plugin upgrade-all` -> `flask plugin prune-schema
--yes --force`. Regression tests in `tests/test_plugin_prune_schema.py` (run in
the backend CI job).

148
docs/adr/ADR-015-site-specific-configuration.md Normal file
View File

@@ -0,0 +1,148 @@
# ADR-015: Where a site's own data is allowed to live
- Status: ACCEPTED
- Date: 2026-08-11
- Deciders: ShopDB maintainers
- Relates to: ADR-004 (per-site instances, not multi-tenant), ADR-013 (plugin catalog + lean per-site builds), ADR-014 (schema-lean per-site builds)
## Context
ShopDB is built as a product several GE Aerospace sites can adopt (ADR-004:
each site runs its own instance). It is also developed at one site, West
Jefferson, against that site's live fleet. Every feature therefore arrives
carrying that site's hostnames, networks, share paths and vocabulary, and the
fastest way to ship any of them is to write the value where the code needs it.
That is not a hypothetical drift. A scan of `plugins/` and `shopdb/` for a site
host, a site FQDN or a site network returns 19 hits, of which roughly 8 are a
shipped default or a hardcode rather than a comment:
- `plugins/geenforce/seed_display_scope.py` baked one site's own ShopDB URL and
`device.geaerospace.net` into an inline PowerShell payload. A sister site
could not override either without editing the source. FIXED 2026-08-14: the
kiosk base URL now comes from the `site_base_url` setting at seed time and
from HKLM at run time, and the dispatcher refuses rather than guessing when
neither is set.
- `plugins/backups/services/registry.py` defaults the backup share root to one site's own
UNC path.
- `shopdb/core/api/settings.py` ships `device.geaerospace.net` and
`Printer-{ip}.printer.geaerospace.net` as SETTING DEFAULTS, so another site
inherits that site's domain silently until someone notices.
- `shopdb/core/models/dashboarddefault.py` holds the same domain a second time
as a module constant, and `plugins/computers/api/routes.py` a third time as a
literal fallback.
The problem is not that site-specific values exist - they must. It is that the
repository already has three good mechanisms for them and no rule about which to
use, so each new value lands wherever was convenient. The result reads as drift
because it is unmanaged, not because any single choice was wrong.
Nothing here is a security matter: these are internal hostnames on an air-gapped
network, and `tests/test_docs_publishable.py` already gates what reaches the
public wiki. This is about whether a sister site can adopt the product without
reading our source.
## Decision
A site's own data lives in one of three places, in this order of preference.
**1. A setting, with a NEUTRAL default.** Anything an operator could plausibly
change: hostnames, domains, share paths, networks, base URLs, label prefixes.
The default that ships must be blank or generic, never one site's value. Blank
must be a working state - code reads the setting and degrades sensibly, the way
`display_fqdn_domain` already treats blank as "use the hostname as-is". A
site-specific default is the failure mode this ADR exists to stop: it is
invisible, it looks deliberate, and it is wrong everywhere except here.
**2. A site-namespaced directory.** Anything that is genuinely one site's and
will never be another's, such as the reference site's import loader at
`scripts/site_imports/wjf/`. Never reached by another site's build.
**3. Seed data, not code.** Vocabularies and catalogs, loaded by `flask seed`
and editable per site afterwards.
Fleet-wide convention is NOT site-specific. The `gea-shopfloor-*` imaging types
are a GE fleet taxonomy, not one site's data, and they already carry a per-site
override through the `pctypemap_*` settings. They are out of scope here.
Comments and documentation may name a site where it explains WHY - "part markers
share a machine number at the reference site" is a fact a maintainer needs. Prefer
phrasing the rule generally and the example specifically.
## Enforcement
`scripts/check-naming-and-style.sh`, which already runs in CI, greps `plugins/`
and `shopdb/` for a site host, a site FQDN or a site network, excluding tests
and `site_imports/`.
It shipped in REPORT-ONLY mode, because turning a 19-item backlog into a hard
gate on the day it lands blocks every unrelated commit until someone works
through it. See the amendment below: it now fails the build.
## Consequences
Positive:
- A sister site can adopt without reading our source for values it must change.
- The backlog is visible and counted rather than felt.
- New site-specifics are caught at commit time, in the same place the naming
rules are already enforced, so there is nothing new to remember.
Negative:
- Report-only means the rule is advisory until someone flips it, and an
advisory rule can be ignored.
- A neutral default is more code than a working one: every read needs a blank
case, which is a real cost paid at every call site.
- The pattern list is a denylist and will miss a site value it has not been
taught. It catches drift, not everything.
## Backlog
CLEARED - see the amendment below. Kept as the record of what the decision was
answering:
- `plugins/geenforce/seed_display_scope.py` - kiosk base URL, FQDN domain, and
the legacy-shortcut sweep pattern
- `plugins/backups/services/registry.py` - `DEFAULTSHAREROOT`
- `shopdb/core/api/settings.py` - `display_fqdn_domain` and the printer
hostname pattern defaults
- `shopdb/core/models/dashboarddefault.py` - `DEFAULT_DISPLAY_FQDN_DOMAIN`
- `plugins/computers/api/routes.py` - literal fallback duplicating that default
The domain appears three times in three files, which is its own defect: one
setting, one default, one read.
Cosmetic mentions in comments and UI placeholder text are acceptable under this
ADR and are not part of the backlog, with one exception worth fixing when
convenient: the GE-Enforce settings page shows one site's real CIDRs as its
placeholder, which every site sees.
## Amendment, 2026-08-14
The backlog above is cleared and the scanner now ENFORCES: it fails the build,
and `SITE_LITERALS_ENFORCE=0` is the escape hatch rather than the default. The
report-only period ended for the reason the Consequences section predicted - the
hit count did not move for weeks, because a rule that only prints is read as no
rule.
Three things changed beyond clearing the list:
- The scan covers PowerShell, the installer, the seeds, generated JSON and the
frontend, case-insensitively, across `plugins/`, `shopdb/`, `scripts/`,
`deploy/`, `frontend/src/` and `tools/`. Its original scope - three file types
under two directories - missed every literal that actually reached a second
site, because those were all in the client scripts and the installer.
- A line may declare itself deliberate with a trailing `ADR-015-OK: <reason>`
marker. That is for an organisation-wide default which is right at every site
and configurable anyway, such as the corporate device domain. It is not a
way to defer work: the marker puts the claim in the diff where a reviewer
sees it.
- Both publication gates - the export scrub and the docs publishability test -
carry the site patterns, which neither did. The ADR said this was not a
security matter because the values were internal to an air-gapped network;
that was true of the fleet and false of the public mirror, which had been
carrying a production hostname and a share path.
This amendment is appended rather than edited into the text above, because what
the decision looked like when it was taken is the part worth keeping.

182
docs/adr/ADR-016-credential-delivery.md Normal file
View File

@@ -0,0 +1,182 @@
# ADR-016: Credential delivery to the fleet
- Status: ACCEPTED (decided; NOT yet implemented - see Implementation status)
- Date: 2026-08-11
- Deciders: ShopDB maintainers
- Relates to: ADR-006 (collector contract), ADR-012 (GE-Enforce manifest ownership), ADR-015 (site-specific configuration)
## Context
Shopfloor bays need credentials they cannot prompt for. A collections bay loses
the Windows Credential Manager entry for its controller subnet periodically -
suspected Defender or Intune scrub - and without it, controller-side
connections need an interactive prompt, so an unattended bay simply stops
working. A machine whose NTLARS config points at a program share needs a
credential to reach that share for the same reason.
Today this is solved by `Set-ControllerCredential.ps1` on the SFLD share:
```powershell
$Target = '192.168.1.1'
$Username = 'CHANGEME'
$Password = 'CHANGEME'
```
A person edits the values into the file, GE-Enforce runs it every cycle, and
`cmdkey /add` re-applies it. It works, and it has three problems.
The password is **cleartext in a file on a share**, readable by anything that
can read the share - a wider exposure than the machine-local secret it becomes.
Rotating it means editing that file and hoping: there is no way to ask which
bays have picked up the new value. And it is **one value for everything**,
while the requirement is per-bay variation: sites will have machines needing
different credentials from their neighbours.
Per-bay variation is what rules out the obvious alternative. GE-Enforce scopes
are per PC TYPE, so expressing per-machine credentials as manifest scopes means
a scope per bay, which is not a model. The manifest can deliver a value
(`Registry` entries with inline payloads already do this for the display
scope), but it cannot decide WHICH value a given PC should get.
## Decision
ShopDB owns credential definitions and their targeting. A PC asks for its own
credentials and receives a decided answer; it never receives rules to evaluate.
### Targeting
A credential rule carries a targeting expression using the SAME axes
GE-Enforce already filters on, plus those the asset model knows:
| Axis | Source | Notes |
|---|---|---|
| hostname | reported | exact or `-like` glob (`WJRP*`) |
| machine | PC -> machine link | the machine the PC drives |
| model | asset model | e.g. every Okuma of one model |
| controller OS | asset / reported | version-valued, like the existing CmmVersion gate |
| PC type | ComputerType | with GE-Enforce ALIAS EXPANSION |
| GE-Enforce profile | manifest scope | the imaging profile |
| DNC/NTLARS setting | stored backup projection | e.g. any machine whose `DataHost` is a given UNC |
| site default | setting | the fallback |
The DNC axis is not a special case bolted on: the NTLARS projection ShopDB
already stores contains `DataHost`, `DataPath` and `MarkMasterPath`, so "every
machine that talks to \\ntshare" is a selector over data already held. It is
also the axis that answers "which machines even need a credential", which
nothing answers today.
### Precedence
Overlapping matches are guaranteed, not exceptional - a hostname glob and a
per-model rule will both hit the same bay. Precedence is therefore EXPLICIT,
never emergent:
1. Each rule carries an integer `priority`. Higher wins. This is the only
thing an operator needs to reason about.
2. Ties break on scope specificity, in the order of the table above (hostname
most specific, site default least).
3. Remaining ties break on the lowest rule id, so the result is stable rather
than dependent on row order.
A resolve endpoint must be able to EXPLAIN itself - "this PC got rule 12,
matched on model, beating rule 4 on priority" - or nobody will trust it. The
GE-Enforce simulator (`applicable_entry_names`) already set this precedent.
### Reuse the existing matcher
`plugins/geenforce/filters.py` already mirrors the engine's targeting: alias
groups, hostname globs, the rule that a machine-number filter with no machine
number EXCLUDES rather than defaults. The resolver reuses it. It must not
become a third implementation of the same matching logic - there are already
two (the PowerShell engine and its Python mirror), kept honest by a parity
harness, and a third would drift silently.
Alias expansion in particular is not optional: if a rule says `PCTypes:
Standard`, it must mean what it means in GE-Enforce, or the two systems will
disagree about who a rule covers.
### Targeting is not detection
Who gets a credential and whether it is already applied are separate
questions, exactly as `PCTypes` and `DetectionMethod` are separate in the
manifest. The client decides the second locally - is this `cmdkey` entry
already present and correct - and rewrites only on drift. Conflating them
rewrites the credential every cycle instead of healing it, and turns the audit
trail into noise.
### Storage
- Encrypted at rest with Fernet. `cryptography` is already a dependency.
- The key lives OUTSIDE the database - instance file or environment, like
`SECRET_KEY`. A key in the database it protects is not encryption.
- Write-only through the API. Masked on read, as `smtp_password` already is. A
fleet PC receives a value; a human never reads one back.
- A dedicated `credentials.manage` permission for authoring.
### Delivery
```
GET /api/collector/credentials X-API-Key: <credentials.fetch token>
```
- A DEDICATED `credentials.fetch` scope, not `collector.ingest`. A leaked
collector key must not yield controller passwords.
- The caller is resolved by hostname; it receives only what it is entitled to,
and never the rule set.
- Every fetch is audited: which PC, which credential, when. The audit log
exists; this belongs in it.
- The client half is a shared helper alongside `ShopdbBackupClient.psm1`:
fetch, compare, `cmdkey /add` on drift, log once per state.
## Consequences
Positive:
- The cleartext credential leaves the share.
- Rotation becomes a settings change plus a report of who has picked it up,
instead of a file edit and a hope.
- Per-bay, per-model and per-share credentials become expressible, which is the
actual requirement.
- "Which machines need a credential" becomes answerable from the DNC data
already stored.
Negative, and these are real:
- **ShopDB becomes a credential store.** That is a different security posture
from an asset database, and it earns the obligations above rather than
choosing them.
- **The key is a new single point of loss.** Restore the database without it
and every stored credential is unrecoverable. `docs/BACKUP-RESTORE.md` must
say so IN THE SAME CHANGE, not later.
- A PC that cannot reach ShopDB gets no credential. The client must keep the
last known good value rather than clearing a working entry on a failed fetch.
- Targeting complexity is real complexity. Eight axes with priorities will
produce a rule set someone has to debug, which is why the explain endpoint is
a requirement and not a nicety.
## Alternatives rejected
**Keep editing the script on the share.** Cannot express per-bay values, keeps
the cleartext, no rotation story.
**A manifest scope per bay.** Per-machine credentials as GE-Enforce scopes is a
spreadsheet, not a model, and the manifest cannot resolve WHICH value applies.
**Encrypted payload in the manifest.** DPAPI is per-machine, and one manifest
serves many machines, so the payload cannot be encrypted to its readers.
**Registry-only provisioning (no ShopDB).** A worthwhile FIRST STEP - it moves
the secret off the share immediately and needs no new endpoint - but it leaves
provisioning per-bay by hand and offers no rotation. Recommended as tier one
regardless, since the client helper is the same either way.
## Implementation status, 2026-08-14
Nothing in this ADR is built yet. The fetch endpoint and the `credentials.*`
permissions it describes do not exist in the code, and a reader searching for
them will not find them.
Recorded here rather than by changing the status, because the decision itself
stands: this is how credential delivery WILL work, and a plugin author designing
against it is designing correctly. What credentials the fleet uses today, and
where they live, is in [FLEET-ARCHITECTURE](../FLEET-ARCHITECTURE.md).

4
docs/adr/README.md
View File

@@ -25,6 +25,10 @@ Each ADR captures a single architectural decision: the context, the decision its
| [010](ADR-010-frontend-plugin-hooks.md) | Frontend plugin hook contract | ACCEPTED |
| [011](ADR-011-machines-rename.md) | Machines rename + modeltypes retyping | ACCEPTED |
| [012](ADR-012-geenforce-manifest-ownership.md) | GE-Enforce manifest ownership in shopdb | ACCEPTED |
| [013](ADR-013-plugin-catalog-and-lean-builds.md) | Plugin catalog, curated shelf, and lean per-site builds | PROPOSED |
| [014](ADR-014-schema-lean-per-site.md) | Schema-lean per-site builds (retire cross-plugin FKs, prune not-installed plugin tables) | ACCEPTED |
| [015](ADR-015-site-specific-configuration.md) | Where a site's own data is allowed to live (setting, site directory, or seed) | ACCEPTED |
| [016](ADR-016-credential-delivery.md) | Credential delivery to the fleet (ShopDB-resolved targeting, encrypted at rest) | ACCEPTED |
## Authoring

3236
docs/api-inventory.json Normal file
View File

File diff suppressed because it is too large Load Diff

506
docs/examples/toner-dashboard.html Normal file
View File

@@ -0,0 +1,506 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>ShopDB printer supplies API - reference and live dashboard example</title>
<style>
:root {
color-scheme: light dark;
--bg: #ffffff;
--card: #f6f7f9;
--text: #16181d;
--muted: #5b6472;
--border: #d8dce3;
--accent: #0d6efd;
--ok: #04b962;
--low: #ff8800;
--critical: #f5365c;
--code: #eef1f5;
}
@media (prefers-color-scheme: dark) {
:root {
--bg: #0d0d1f;
--card: #16162e;
--text: #e8eaf0;
--muted: #9aa3b5;
--border: #2a2a4a;
--code: #10102a;
}
}
* { box-sizing: border-box; }
body {
margin: 0;
padding: 2rem 1.25rem 4rem;
background: var(--bg);
color: var(--text);
font: 16px/1.6 system-ui, -apple-system, "Segoe UI", Roboto, sans-serif;
}
main { max-width: 62rem; margin: 0 auto; }
h1 { font-size: 1.9rem; margin: 0 0 .3rem; }
h2 { font-size: 1.35rem; margin: 2.5rem 0 .75rem; padding-top: .75rem; border-top: 1px solid var(--border); }
h3 { font-size: 1.05rem; margin: 1.5rem 0 .4rem; }
.lede { color: var(--muted); margin: 0 0 1.5rem; }
code, pre { font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; }
code { background: var(--code); padding: .12em .38em; border-radius: 4px; font-size: .9em; }
pre {
background: var(--code);
border: 1px solid var(--border);
border-radius: 8px;
padding: 1rem;
overflow-x: auto;
font-size: .86rem;
line-height: 1.5;
}
pre code { background: none; padding: 0; font-size: inherit; }
table { border-collapse: collapse; width: 100%; margin: .75rem 0; font-size: .93rem; }
th, td { text-align: left; padding: .5rem .6rem; border-bottom: 1px solid var(--border); vertical-align: top; }
th { color: var(--muted); font-weight: 600; }
td code { white-space: nowrap; }
.note {
background: var(--card);
border-left: 4px solid var(--accent);
border-radius: 0 6px 6px 0;
padding: .8rem 1rem;
margin: 1rem 0;
}
.note.warn { border-left-color: var(--low); }
.pill { display: inline-block; padding: .1rem .5rem; border-radius: 999px; font-size: .78rem; font-weight: 700; }
.pill.ok { background: var(--ok); color: #04220f; }
.pill.low { background: var(--low); color: #2b1600; }
.pill.critical { background: var(--critical); color: #fff; }
/* ---- live demo ---- */
.demo { background: var(--card); border: 1px solid var(--border); border-radius: 10px; padding: 1rem; }
.demo-controls { display: flex; gap: .5rem; flex-wrap: wrap; align-items: center; margin-bottom: 1rem; }
.demo input {
flex: 1 1 22rem; min-width: 14rem; padding: .45rem .6rem;
border: 1px solid var(--border); border-radius: 6px;
background: var(--bg); color: var(--text); font: inherit; font-size: .9rem;
}
.demo button {
padding: .45rem .9rem; border: 0; border-radius: 6px;
background: var(--accent); color: #fff; font: inherit; font-weight: 600; cursor: pointer;
}
.summary { display: flex; gap: 1.25rem; flex-wrap: wrap; margin-bottom: 1rem; color: var(--muted); font-size: .9rem; }
.summary b { color: var(--text); font-size: 1.3rem; display: block; }
.printer { border-top: 1px solid var(--border); padding: .75rem 0; }
.printer-name { font-weight: 600; }
.printer-meta { color: var(--muted); font-size: .85rem; }
.supply { display: grid; grid-template-columns: 11rem 1fr 4rem; gap: .6rem; align-items: center; margin: .35rem 0; font-size: .88rem; }
.bar { background: var(--border); border-radius: 999px; height: .55rem; overflow: hidden; }
.bar span { display: block; height: 100%; border-radius: 999px; }
.printer-rest { color: var(--muted); font-size: .8rem; margin-top: .35rem; }
.status-msg { color: var(--muted); font-style: italic; }
</style>
</head>
<body>
<main>
<h1>Printer supplies API</h1>
<p class="lede">
Reference for the toner and supply endpoints in ShopDB, and a working
single-page dashboard you can lift straight out of this file.
</p>
<div class="note">
<strong>Where the numbers come from.</strong> ShopDB does not poll printers itself.
Levels are read from Zabbix by IP address, so every endpoint below returns empty
results unless <code>zabbix_enabled</code>, <code>zabbix_url</code> and
<code>zabbix_token</code> are set under Settings &gt; Integrations. That failure is
deliberately soft: you get <code>200</code> with an empty list, never an error, so a
wall dashboard does not blow up when Zabbix is down.
</div>
<h2>Endpoints</h2>
<table>
<thead>
<tr><th>Method and path</th><th>Auth</th><th>What it returns</th></tr>
</thead>
<tbody>
<tr>
<td><code>GET /api/printers/lowsupplies</code></td>
<td>optional</td>
<td>Every printer with at least one supply below threshold, plus a summary. The toner report. Cached 5 minutes.</td>
</tr>
<tr>
<td><code>GET /api/printers/&lt;printerid&gt;/supplies</code></td>
<td>optional</td>
<td>Live levels for one printer, read from Zabbix on request. Not cached.</td>
</tr>
<tr>
<td><code>POST /api/printers/supplies/refresh</code></td>
<td><strong>required</strong> + <code>printers.create</code></td>
<td>Clears the cache so the next read is fresh. Backs the report's Refresh button.</td>
</tr>
<tr>
<td><code>GET /api/printers/supplies/meta</code></td>
<td>optional</td>
<td>Allowed supply types, colors and capacity tiers.</td>
</tr>
<tr>
<td><code>GET /api/printers/models/&lt;modelnumberid&gt;/supplies</code></td>
<td>optional</td>
<td>Cartridge part numbers for a model.</td>
</tr>
<tr>
<td><code>GET /api/printers/lookup?ip=</code> or <code>?fqdn=</code></td>
<td>optional</td>
<td>Resolve an IP or hostname to a printer id.</td>
</tr>
</tbody>
</table>
<p>
"optional" means the endpoint accepts a token but does not need one: a logged-out
page can read it. Only the refresh endpoint requires a token, and the account
behind it needs the <code>printers.create</code> permission.
</p>
<h2>Response shape</h2>
<p>Every ShopDB response is wrapped. Your data is under <code>data</code>:</p>
<pre><code>{
"status": "success",
"data": { ... },
"meta": { "requestid": "6d9d0bb6", "timestamp": "2026-08-11T13:47:41.250953Z" }
}</code></pre>
<h3>GET /api/printers/lowsupplies</h3>
<pre><code>{
"printers": [
{
"printerid": 42,
"printername": "Printer-10-129-22-15",
"assetnumber": "PRN-0042",
"ipaddress": "10.129.22.15",
"vendor": "Xerox",
"model": "AltaLink C8145",
"location": "Building 1 - Cell 4",
"supplies": [
{
"name": "Cyan Toner Cartridge",
"level": 8,
"remaining": 8.0,
"status": "low",
"color": "cyan",
"supplytype": "toner",
"iswaste": false,
"isdrum": false,
"partnumbers": ["006R01737"]
}
]
}
],
"summary": { "total_checked": 37, "low": 4, "critical": 1 }
}</code></pre>
<p>
Only printers with something below threshold appear in <code>printers</code>.
<code>total_checked</code> counts every printer Zabbix answered for, so
"37 checked, 5 listed" means 32 are healthy. A printer is counted once in the
summary, as <code>critical</code> if any of its supplies is critical, otherwise
as <code>low</code>.
</p>
<h3>GET /api/printers/&lt;printerid&gt;/supplies</h3>
<pre><code>{
"ipaddress": "10.129.22.15",
"pingstatus": "1",
"supplies": [ /* same supply objects as above */ ]
}</code></pre>
<p>
<code>pingstatus</code> is Zabbix's reachability value, <code>"-1"</code> when
Zabbix is unconfigured or unreachable. This endpoint hits Zabbix on every call,
so poll it for one printer on a detail view, not for forty on a dashboard - use
<code>lowsupplies</code> for that.
</p>
<h2>The supply object</h2>
<table>
<thead><tr><th>Field</th><th>Meaning</th></tr></thead>
<tbody>
<tr><td><code>name</code></td><td>Raw name from the printer, e.g. <code>"Black Toner Cartridge"</code>.</td></tr>
<tr><td><code>level</code></td><td>Raw value as reported. For most waste cartridges this is percent <em>full</em>.</td></tr>
<tr><td><code>remaining</code></td><td>Normalised percent remaining. <strong>Use this one</strong> for bars and text.</td></tr>
<tr><td><code>status</code></td><td><span class="pill ok">ok</span> <span class="pill low">low</span> <span class="pill critical">critical</span></td></tr>
<tr><td><code>color</code></td><td><code>black</code>, <code>cyan</code>, <code>magenta</code>, <code>yellow</code>, <code>none</code>.</td></tr>
<tr><td><code>supplytype</code></td><td><code>toner</code>, <code>drum</code>, <code>waste</code>, <code>maintenance</code>.</td></tr>
<tr><td><code>iswaste</code>, <code>isdrum</code></td><td>Convenience flags; a waste cartridge is usually worth showing differently.</td></tr>
<tr><td><code>partnumbers</code></td><td>Order codes from the <code>modelsupplies</code> table. Empty until someone fills them in for that model.</td></tr>
</tbody>
</table>
<div class="note warn">
<strong>Do not compute status from <code>level</code> yourself.</strong>
Thresholds are <code>remaining &lt;= 5</code> critical, <code>&lt;= 10</code> low.
The catch is direction: a full waste cartridge is bad, so for waste ShopDB
converts to <code>100 - level</code> - except on Xerox, which already reports
waste as capacity remaining. That vendor rule is why <code>remaining</code> and
<code>status</code> exist. Read them, do not re-derive them.
</div>
<h2>Before you build</h2>
<h3>Base path</h3>
<p>
Instances are served under a subpath, so the API is at
<code>https://shopdb.example.net/shopdb/api/...</code> - production on
this server - or <code>/ops/api/...</code> for the dev instance beside it, not at
the domain root. Make the base a variable; do not hardcode <code>/api</code>.
</p>
<h3>Same origin, or CORS</h3>
<p>
The simplest deployment is to drop your HTML file into the instance's web root so
it is served from the same origin - then <code>fetch</code> just works. A page
served from anywhere else is a cross-origin request, and the browser will block it
unless that origin is in the server's <code>CORS_ORIGINS</code> allowlist
(an env var; production refuses to start with a wildcard).
</p>
<h3>Caching and polling</h3>
<p>
<code>lowsupplies</code> is cached for 5 minutes server-side, so polling every
30 seconds gets you the same payload nine times out of ten and buys nothing.
Poll every 2 to 5 minutes. If you need a manual Refresh button, call the refresh
endpoint first - that one needs a token.
</p>
<h2>Minimal example</h2>
<pre><code>const BASE = 'https://shopdb.example.net/shopdb'; // no trailing /api
async function lowSupplies() {
const response = await fetch(`${BASE}/api/printers/lowsupplies`);
if (!response.ok) throw new Error(`HTTP ${response.status}`);
const body = await response.json();
return body.data; // { printers, summary }
}
lowSupplies().then(data =&gt; {
console.log(`${data.summary.critical} critical, ${data.summary.low} low`);
for (const printer of data.printers) {
for (const supply of printer.supplies) {
if (supply.status === 'ok') continue;
console.log(`${printer.printername}: ${supply.name} ${supply.remaining}%`);
}
}
});</code></pre>
<h2>Polling, with the failure cases handled</h2>
<pre><code>const BASE = 'https://shopdb.example.net/shopdb';
const POLL_MS = 3 * 60 * 1000; // server caches 5 min; faster buys nothing
async function tick() {
try {
const response = await fetch(`${BASE}/api/printers/lowsupplies`, {
headers: { 'Accept': 'application/json' }
});
if (!response.ok) throw new Error(`HTTP ${response.status}`);
const { data } = await response.json();
// Zabbix off or unreachable: 200 with nothing in it. Say so, rather than
// rendering "0 low" as if every printer were healthy.
if (data.summary.total_checked === 0) {
render.unavailable();
} else {
render.board(data);
}
} catch (err) {
// A wall display must survive a blip: keep the last good render.
console.error('supply poll failed', err);
render.stale(err);
} finally {
setTimeout(tick, POLL_MS); // chained, so a slow response cannot pile up
}
}
tick();</code></pre>
<h2>Refreshing on demand (needs a token)</h2>
<p>
Create a Personal Access Token under Settings &gt; API Tokens for an account with
<code>printers.create</code>. Treat it as a credential: a token embedded in a page
anyone can open is readable by anyone who opens it, so use this on an operator
page behind login, not on a lobby display.
</p>
<pre><code>async function refreshAndReload(token) {
await fetch(`${BASE}/api/printers/supplies/refresh`, {
method: 'POST',
headers: { 'Authorization': `Bearer ${token}` }
});
return lowSupplies(); // next read pulls fresh values from Zabbix
}</code></pre>
<h2>One printer, live</h2>
<pre><code>// Skips the cache and asks Zabbix now. Fine for a detail page, wrong for a loop.
async function printerSupplies(printerid) {
const response = await fetch(`${BASE}/api/printers/${printerid}/supplies`);
const { data } = await response.json();
if (data.pingstatus === '-1') return { offline: true, supplies: [] };
return data;
}
// Have an IP but not an id? Resolve it first.
async function printerIdByIp(ip) {
const response = await fetch(`${BASE}/api/printers/lookup?ip=${encodeURIComponent(ip)}`);
const { data } = await response.json();
return data?.printerid ?? null;
}</code></pre>
<h2>Live dashboard</h2>
<p>
The section below is the real thing, running in this page. Point it at an instance
and it will render. Served from a different origin than the API, it will fail on
CORS - that is the browser doing its job, and the fix is to host the file on the
instance itself.
</p>
<div class="demo">
<div class="demo-controls">
<input id="base" type="text" value="https://shopdb.example.net/shopdb" aria-label="ShopDB base URL"
placeholder="https://shopdb.example.net/shopdb" />
<button id="load">Load</button>
</div>
<div id="out"><p class="status-msg">Enter a base URL and press Load.</p></div>
</div>
<script>
// ---- the whole dashboard, in one place ----
const COLORS = { ok: '#04b962', low: '#ff8800', critical: '#f5365c' };
const el = id => document.getElementById(id);
const escapeHtml = text => String(text ?? '').replace(/[&<>"']/g,
ch => ({ '&': '&amp;', '<': '&lt;', '>': '&gt;', '"': '&quot;', "'": '&#39;' }[ch]));
async function fetchLowSupplies(base) {
const response = await fetch(`${base.replace(/\/$/, '')}/api/printers/lowsupplies`, {
headers: { 'Accept': 'application/json' }
});
if (!response.ok) throw new Error(`HTTP ${response.status} ${response.statusText}`);
const body = await response.json();
return body.data;
}
function supplyRow(supply) {
// remaining, not level: waste cartridges and Xerox invert differently and the
// server has already sorted that out.
const width = Math.max(0, Math.min(100, supply.remaining));
const parts = supply.partnumbers?.length
? ` &middot; ${escapeHtml(supply.partnumbers.join(', '))}` : '';
return `
<div class="supply">
<div>${escapeHtml(supply.name)}${parts}</div>
<div class="bar"><span style="width:${width}%;background:${COLORS[supply.status]}"></span></div>
<div><span class="pill ${supply.status}">${supply.remaining}%</span></div>
</div>`;
}
function printerBlock(printer) {
const meta = [printer.location, printer.model, printer.ipaddress]
.filter(Boolean).map(escapeHtml).join(' &middot; ');
// Only what needs action gets a row. A colour laser reports six supplies, so
// rendering all of them buries the empty cartridge among five healthy ones
// and costs the screen space of another printer.
const failing = printer.supplies.filter(s => s.status !== 'ok');
// The healthy ones are still worth a glance - magenta at 12% is not flagged
// today but is the same trip - so they collapse to one muted line rather
// than disappearing.
const healthy = printer.supplies.filter(s => s.status === 'ok');
const rest = healthy.length
? `<div class="printer-rest">also ok: ${healthy
.map(s => `${escapeHtml(shortName(s))} ${Math.round(s.remaining)}%`)
.join(' &middot; ')}</div>`
: '';
return `
<div class="printer">
<div class="printer-name">${escapeHtml(printer.printername || printer.assetnumber)}</div>
<div class="printer-meta">${meta}</div>
${failing.map(supplyRow).join('')}
${rest}
</div>`;
}
// 'Black Toner Cartridge' -> 'Black'. Enough to tell supplies apart on the
// one-line summary without repeating 'Toner Cartridge' four times.
function shortName(supply) {
if (supply.iswaste) return 'Waste';
if (supply.isdrum) return 'Drum';
if (supply.color && supply.color !== 'none') {
return supply.color.charAt(0).toUpperCase() + supply.color.slice(1);
}
return supply.name;
}
function render(data) {
if (data.summary.total_checked === 0) {
// Zabbix off or unreachable. "0 low" here would be a lie, not good news.
el('out').innerHTML =
'<p class="status-msg">No printers could be checked - Zabbix is not configured or not reachable.</p>';
return;
}
if (!data.printers.length) {
el('out').innerHTML =
`<p class="status-msg">All ${data.summary.total_checked} printers healthy.</p>`;
return;
}
el('out').innerHTML = `
<div class="summary">
<div><b>${data.summary.critical}</b> critical</div>
<div><b>${data.summary.low}</b> low</div>
<div><b>${data.summary.total_checked}</b> checked</div>
</div>
${data.printers.map(printerBlock).join('')}`;
}
async function load() {
el('out').innerHTML = '<p class="status-msg">Loading...</p>';
try {
render(await fetchLowSupplies(el('base').value));
} catch (err) {
el('out').innerHTML =
`<p class="status-msg">Could not read supplies: ${escapeHtml(err.message)}. ` +
'If this page is not served from the ShopDB instance, the browser blocked it on CORS.</p>';
}
}
el('load').addEventListener('click', load);
el('base').addEventListener('keydown', e => { if (e.key === 'Enter') load(); });
</script>
<h2>Notes for a wall display</h2>
<ul>
<li>Show the supplies that need action as rows, and collapse the healthy ones
into a single muted line. The endpoint returns every supply for a listed
printer, so this is the client's call: a colour laser reports six, and
rendering all six buries the empty one. Keep the healthy summary rather
than dropping it - a cartridge at 12% is not flagged today but is the same
walk.</li>
<li>Chain the poll with <code>setTimeout</code> in a <code>finally</code>, not
<code>setInterval</code>: a slow or hung request cannot then stack up behind itself.</li>
<li>Keep the last good render when a poll fails. A display that blanks on one
dropped request is worse than one showing five-minute-old numbers.</li>
<li>Distinguish "nothing is low" from "nothing was checked". They both look like
an empty list, and only one of them is good news.</li>
<li>The board reloads itself if you serve it as a ShopDB display; otherwise add
a daily <code>location.reload()</code> so a deployed fix reaches a screen
nobody touches.</li>
</ul>
</main>
</body>
</html>

785
docs/geenforce-api-cutover.md Normal file
View File

@@ -0,0 +1,785 @@
# GE-Enforce over HTTPS API: cutover reference
This is the operator/developer reference for moving a shopfloor PC type off the
SMB-share GE-Enforce delivery and onto the shopdb HTTPS API. The first cohort
cut over was the displays/kiosks (`gea-shopfloor-display`): share-less,
Intune/Entra-joined PCs with no SFLD share credentials. This doc captures
everything learned doing that, so extending the cutover to the other pc-types
(`gea-shopfloor-cmm`, `-collections`, `-keyence`, `-genspect`, `-heattreat`,
`-partmarker`, `-nocollections`, `common`) does not require re-learning it.
It pairs with the existing docs (which describe the pieces; this one describes
the CUTOVER):
- `docs/GE-ENFORCE.md` - concepts and the plugin
- `docs/GE-ENFORCE-CLIENT.md` - client fetch/report contract
- `docs/GE-ENFORCE-DEPLOY.md` - what must land on a PC
- `docs/GE-ENFORCE-DISPLAY.md` - the display scope specifics
- the PXE repo (`docs/ge-enforce-v2-architecture.md`) - the SMB world
being cut away from
Contents:
1. [Overview and why](#1-overview-and-why)
2. [Server architecture](#2-server-architecture)
3. [Delivery models: smb vs http/inline payloads](#3-delivery-models-smb-vs-httpinline-payloads)
4. [Authoring a scope](#4-authoring-a-scope)
5. [The on-PC client and engine](#5-the-on-pc-client-and-engine)
6. [Bootstrap for share-less PCs](#6-bootstrap-for-share-less-pcs)
7. [Asset reporting via the collector](#7-asset-reporting-via-the-collector)
8. [HARD-WON GOTCHAS](#8-hard-won-gotchas)
9. [How this was verified](#9-how-this-was-verified)
10. [Deploy](#10-deploy)
11. [PLAYBOOK: extending to a new pc-type](#11-playbook-extending-to-a-new-pc-type)
12. [Open items / TODO](#12-open-items--todo)
---
## 1. Overview and why
GE-Enforce v2 delivers desired-state manifests and installer payloads from the
SFLD SMB share (`\\shopdb.example.net\shared\dt\shopfloor\`).
Every PC mounts the share with Azure-DSC-provisioned SFLD credentials, reads
`<scope>\manifest.json`, and runs the engine
(`Install-FromManifest.ps1`). That works for the domain fleet but is a hard
dead end for share-less PCs.
The HTTPS API path replaces the share as the transport, keeping the engine and
its detection/self-heal behavior untouched:
| Concern | SMB share | HTTPS API |
|---------|-----------|-----------|
| Manifest source | `<scope>\manifest.json` on the share | `GET /api/geenforce/manifest?pctype=<scope>` (published snapshot) |
| Payload source | share paths (`apps\...`) | `GET /api/geenforce/payload/<sha256>` (content-addressed) |
| Auth | SFLD share credential (DSC) | `geenforce.fetch` service token OR source-IP allowlist |
| Result visibility | log files on the PC | `POST /api/geenforce/report` -> Enforcement Reports UI |
| Versioning | file overwrite + `_meta/history` | immutable published versions, rollback, ETag |
Which PCs MUST use the API: the share-less ones. Displays/kiosks are
Intune/Entra-joined with no SFLD credentials and no domain trust, so SMB is not
an option at all. The rest of the fleet CAN stay on the share (and currently
does); for them the API is an opt-in migration, not a forced one.
the reference site facts used throughout this doc:
| Fact | Value |
|------|-------|
| Prod host | `shopdb.example.net` |
| App mount | `/shopdb` (IIS, app dir `C:\inetpub\wwwroot\shopdb`, pool `shopdbflask-prod`) |
| BaseUrl clients use | `https://shopdb.example.net/shopdb` |
| Prod DB | `shopdb_flask` (MySQL) |
| Client allowlist CIDRs | `192.0.2.0/24,198.51.100.0/26` (this site s corp / shopfloor subnets - examples) |
| Dev/staging instance | `/ops` mount, DB `shopdb_flask_dev`, pool `shopdbflask` |
---
## 2. Server architecture
All server code is in `plugins/geenforce/` (routes: `plugins/geenforce/api/routes.py`).
### Client-facing endpoints (three)
| Endpoint | Method | Auth | What it does |
|----------|--------|------|--------------|
| `/api/geenforce/manifest?pctype=<scope>&phase=runtime` | GET | `geenforce.fetch` token OR IP allowlist | Serves the CURRENT PUBLISHED manifest snapshot for a scope (never the draft). `ETag: "<scopeid>-v<version>"`, `X-Manifest-Version` header, 304 on `If-None-Match`. |
| `/api/geenforce/payload/<sha256>` | GET | `geenforce.fetch` token OR IP allowlist | Streams a payload blob by content hash: blob store first (`service.blob_path`), then inline `ManifestPayload`. ETag = the hash. Per-IP rate limited (120/min default) and size-capped (512 MB default, 413 above). |
| `/api/geenforce/report` | POST | `geenforce.report` token OR IP allowlist | Records one enforcement cycle via `service.record_enforcement_report`: hostname, scopename, appliedversion, enforcerversion, counts, per-entry results. Upserts the current report per (hostname, scopename, phase); older reports kept as history. |
Plus the collector for asset reporting (section 7): `POST
/api/collector/computers` in `shopdb/core/api/collector.py` - a DIFFERENT auth
domain (`collector.ingest`), NOT covered by the geenforce allowlist.
### Auth model
`_require_service_token(scope)` in `routes.py` is the decorator factory. Two
paths, fail-closed (neither -> 401):
1. A managed service token with the scope (`geenforce.fetch` for
manifest/payload, `geenforce.report` for report), sent as `X-API-Key` or a
Bearer PAT (`authorized_service_token`). A token may carry
`resourcescopelist` bindings: a bound token can only fetch its own scope's
manifest (403 otherwise) and only blobs those scopes' published manifests
reference (`service.blob_referenced_by_scopes`, 404 so hashes cannot be
probed). Displays get a token bound to `gea-shopfloor-display` (see
GE-ENFORCE-DISPLAY.md).
2. The IP allowlist: `_ip_allowlisted()` checks the caller against the setting
`geenforce_allowed_cidrs` (comma-separated CIDRs/IPs, empty = disabled).
Network trust replaces the shared secret for a vaulted fleet. The
allowlisted path has no resource-scope binding (unrestricted).
The token paths use RBAC permissions the plugin registers in
`GeEnforcePlugin.get_permissions()`: `geenforce.manage` (edit),
`geenforce.publish` (ship), `geenforce.fetch` and `geenforce.report`
(client service tokens). Admin CRUD/publish/simulate/compliance routes are JWT
plus `geenforce.manage`/`geenforce.publish`.
### Why the allowlist uses remote_addr (the IIS XFF dependency)
`_trusted_client_ip()` returns `request.remote_addr`, NOT the raw
`X-Forwarded-For` header. Proxies APPEND to X-Forwarded-For, so its first hop
is attacker-controlled: parsing it (as `_client_ip()` does, acceptably, for
rate limiting only) would let any caller send `X-Forwarded-For:
<allowlisted-ip>` and bypass the token entirely.
`remote_addr` is trustworthy only because of a two-piece chain that MUST stay
in place:
1. The IIS URL-Rewrite rule in the `/shopdb` web.config OVERWRITES (not
appends) the inbound `X-Forwarded-For` with `REMOTE_ADDR`, the real TCP
peer.
2. waitress runs with `--trusted-proxy=127.0.0.1
--trusted-proxy-headers=x-forwarded-for`, so it derives `remote_addr` from
that overwritten header only when the request comes from IIS on localhost.
A client that somehow hits waitress directly is not a trusted proxy, so its
`remote_addr` is its own real peer address. Either way, real client IP.
**If the IIS rule is ever removed, the allowlist becomes SPOOFABLE.** This
document previously claimed the opposite; it was wrong, and the reasoning matters.
IIS does not set `X-Forwarded-For` on its own - the rewrite rule is the only
thing that does. Remove the rule and IIS still *forwards* whatever
`X-Forwarded-For` the caller sent. waitress trusts that header because it arrives
from `127.0.0.1`, which is IIS, and sets `remote_addr` from it. So a caller who
sends `X-Forwarded-For: 192.0.2.5` gets `remote_addr = 192.0.2.5`, matches
the allowlist and fetches manifests token-less from anywhere on the network.
The rule is not a nicety that improves logging. It is the control that makes
`remote_addr` trustworthy, and everything downstream - the allowlist, the
dashboard visitor-location lookup, per-host login rate limiting - depends on it.
Three consequences worth stating plainly:
- The Windows installer enables the rule when told IIS faces clients directly
(`-ClientIpSource direct`), installs URL Rewrite from the bundle to make that
possible offline, and its stage-5 check fails if the rule is not live.
- On a hand-built server, verify it: `deploy/windows/web.config` must have the
`<rewrite>` block ACTIVE, not inside the `SHOPDB-CLIENTIP` comment markers.
- Behind a real reverse proxy the rule is the wrong answer, because `REMOTE_ADDR`
is then the proxy. There, the proxy must set `X-Forwarded-For` itself and be
the only thing that can reach IIS. `-ClientIpSource proxy` covers that case.
---
## 3. Delivery models: smb vs http/inline payloads
Every `ManifestEntry` carries a `payloadsource` (`plugins/geenforce/serializer.py`
and `importer.py`):
| PayloadSource | Meaning | Manifest emission |
|---------------|---------|-------------------|
| `smb` (default) | Entry installs from the share exactly as v2 does; the entry's `Installer`/`Script`/`Source` is a share-relative path. | Nothing emitted - share manifests round-trip byte-identical, parity preserved. |
| `http` | Payload lives in the server's content-addressed blob store (`instance/geenforce/payloads/<sha256>`, registry row `ManifestBlob`). For big files (MSIs, EXEs). Upload via `flask geenforce add-payload <file>` or `service.store_blob`. | `PayloadSource`, `PayloadSha256`, `PayloadRef` keys on the entry. |
| `inline` | Payload bytes live IN the DB (`ManifestPayload`, <= 1 MB) - small scripts and configs. Attach via `service.store_inline_payload(entry, filename, contenttype, rawbytes)` or `POST /api/geenforce/entries/<id>/payload`. | Same three keys. |
Both `http` and `inline` are served from the same client URL:
`GET /api/geenforce/payload/<sha256>` (blob store checked first, then inline).
The sha256 IS the integrity contract - the client re-hashes after download.
### How the client stages payloads (Resolve-ShopdbPayloads)
`Resolve-ShopdbPayloads` in `plugins/geenforce/client/ShopdbEnforceClient.psm1`
is the bridge that lets the UNCHANGED engine install share-less:
1. For each entry with `PayloadSha256` and `PayloadSource` http/inline, call
`Get-ShopdbPayload`: download to
`C:\ProgramData\ShopDB\geenforce\payloads\<sha><ext>` (ext from
`PayloadRef`), verify the sha256, keep it as a content-addressed
last-known-good cache (a cache hit only counts if the bytes still hash
right).
2. Rewrite the entry's path field to the LEAF filename of the staged file
(`Split-Path -Leaf`) - NOT the absolute path. Field by Type:
`Installer` for MSI/EXE/CMD/BAT/INF, `Script` for PS1, `Source` for File.
3. Write a sibling `<scope>.resolved.json` manifest and return its path (or
the original path if nothing needed resolving). A payload that cannot be
fetched/verified THROWS - the runner's fail-safe catch decides what happens.
The runner (`Invoke-ShopdbEnforce.ps1`) then sets the engine's
`-InstallerRoot` to that same payloads directory, so the engine's
`Join-Path $InstallerRoot <leaf>` resolves to the staged file.
**`smb` entries in a mixed manifest DO NOT survive this.** An earlier version of
this doc claimed they were "left untouched and still resolve against the share";
that is false outside shadow mode and was measured wrong on the VM (see the
blockers in section 11 step 1). `Resolve-ShopdbPayloads` does leave them
untouched, but the runner has already pointed `-InstallerRoot` at the payload
cache, so a share-relative path resolves under the cache and is reported
`not found`. A manifest that mixes `smb` with http/inline cannot be served by a
single `InstallerRoot` at all.
---
## 4. Authoring a scope
Two authoring paths, both ending in `service.replace_scope_draft(scopename,
phase, manifest_dict)` (idempotent draft rebuild - published versions are never
touched by a re-import):
### A. import-share: adopt an existing SMB manifest
```
flask geenforce import-share --shareroot <path> [--scope <name>] [--preinstall <path>]
flask geenforce publish <scopename> [--phase runtime] [--notes "..."]
```
`importer.discover_share` walks the share root and ingests
`common/manifest.json`, `display/manifest.json`, and every
`gea-shopfloor-*/manifest.json` (skipping `.bak` variants). Entries come in as
`smb` payloads. Run `flask geenforce parity --shareroot <path>` first (Gate A):
proves import+re-export is behaviorally lossless before anything ships.
### B. authoring in code: seed_display_scope as the template
`plugins/geenforce/seed_display_scope.py` is the reference for a scope that
never existed on the share. Pattern:
- Build the manifest dict in Python (`build_display_manifest()`): four
`Type=Registry` drift-heal entries re-asserting the Edge kiosk relaunch
policies from imaging (`09-Setup-Display.ps1`), one inline PS1 dispatcher,
one inline PS1 always-on script. Registry heals use
`DetectionMethod=ValueMatches` against the same path/name they write, so
drift self-heals; the PS1s use `DetectionMethod=Always` and are idempotent.
- The dispatcher (`Invoke-DisplayKioskDispatch.ps1`, generated by
`build_dispatcher_script()`) reads `C:\Enrollment\display-type.txt`, maps
the subtype through the data-driven `DISPLAY_TYPE_TARGETS` table
(Dashboard -> `/shopfloor`, Lobby -> `/tv`, 3DPrintRoom -> `/parts-kiosk`),
and writes an all-users Startup shortcut (`ShopDB Kiosk.lnk`) launching Edge
`--kiosk` fullscreen at `{BaseUrl}{route}`. It does NOT Start-Process Edge
(see gotchas). Base URL comes from HKLM `BaseUrl`, falling back to the reference-site
host.
- `seed_display_scope(publish=False)`: `replace_scope_draft`, flush (entries
need entryids), then `service.store_inline_payload(...)` for each script
entry (sets `payloadsource='inline'`, `payloadsha256`, `payloadref`),
optionally `service.publish_scope(...)`, commit. Draft rebuild is
idempotent; publish always creates a NEW version.
Run it on the server:
```
cd C:\inetpub\wwwroot\shopdb
$env:FLASK_APP = 'shopdb'
'from plugins.geenforce.seed_display_scope import seed_display_scope; print(seed_display_scope(publish=True))' | venv\Scripts\python -m flask shell
```
Expected: `{scopeid, entrycount: 6, entrytypes: [Registry x4, PS1, PS1],
dispatchersha256, alwaysonsha256, publishedversion: N}`.
### Publishing
`service.publish_scope` freezes the draft (rendered by
`serializer.scope_to_json`) into an immutable `ManifestPublishedVersion` and
flips `iscurrent`. Clients only ever see published versions.
`rollback_scope` re-currents an older version. Also available over the API
(`POST /scopes/<id>/publish`, permission `geenforce.publish`) and the
GE-Enforce UI.
### Attaching payloads
- Inline (<= 1 MB): `service.store_inline_payload` in code, or
`POST /api/geenforce/entries/<entryid>/payload` (multipart file).
- Blob (`http`): `flask geenforce add-payload <filepath>` prints the sha256;
set `PayloadSource=http` + `PayloadSha256` (+ `PayloadRef` for the
filename/extension) on the entry.
Publish AFTER attaching - the published JSON is what carries the
`PayloadSha256` the client fetches, and blob access for resource-bound tokens
is checked against the CURRENT published manifest.
---
## 5. The on-PC client and engine
### Config: HKLM:\SOFTWARE\GE\ShopDB
| Value | Used by | Notes |
|-------|---------|-------|
| `BaseUrl` | enforce client + kiosk dispatcher | e.g. `https://shopdb.example.net/shopdb`. Required. |
| `ApiToken` | enforce client | `geenforce.fetch` (+ report) PAT. OPTIONAL - a token-less client relies on the IP allowlist (`Get-ShopdbConfig` treats BaseUrl-only as valid). |
| `CollectorKey` | `Report-AssetToShopDB.ps1` | `collector.ingest` PAT. REQUIRED for asset reporting (allowlist does not cover the collector). |
The key's ACL is restricted to SYSTEM + Administrators (the bootstrap does
this) so the kiosk auto-login user cannot read the PATs.
### The pieces on disk (kiosk layout, `C:\ProgramData\GE-Enforce`)
- `Invoke-ShopdbEnforce.ps1` - the runner
- `ShopdbEnforceClient.psm1` - the client module
- `lib\Install-FromManifest.ps1` - the engine (>= 2.6)
- `Report-AssetToShopDB.ps1` - the asset collector
- Cache: `C:\ProgramData\ShopDB\geenforce\` (`<scope>.json`, `.etag`,
`.version`, `payloads\`), logs `C:\Logs\Shopfloor\`
### Scheduled tasks (SYSTEM, RunLevel Highest)
| Task | Runs | Interval |
|------|------|----------|
| `ShopDB GE-Enforce` | `Invoke-ShopdbEnforce.ps1 -Scope <scope> -EnginePath <engine> -BaseUrl <url>` | AtStartup + every 15 min |
| `ShopDB Asset Report` | `Report-AssetToShopDB.ps1` | AtStartup + every 60 min |
Tokens are NOT in the task arguments (visible in task XML) - the scripts read
them from HKLM.
### The runner flow (Invoke-ShopdbEnforce.ps1)
1. `Get-ShopdbConfig` (params override registry). No BaseUrl -> exit 0, retry
next cycle.
2. `Sync-ShopdbManifest -Scope <scope>`: ETag-conditional GET; 200 validates
the JSON before overwriting the cache (a proxy error page served as 200
must not clobber last-known-good); 304 or any network failure -> cached
copy. Nothing at all -> Windows event log entry (source `ShopdbEnforce`,
id 1001) plus a best-effort failure report so it is visible server-side,
then exit 0.
3. `-ShadowMode` (with `-ShareManifestPath`): `Compare-ShopdbShadow` logs
name/order diffs, engine runs against the SHARE (zero behavior change).
This is the first step of every cutover.
4. Cutover mode: optional `-IncludeCommon` merges the fleet `common` scope
(`Merge-ShopdbManifests`: common's unique entries first, pctype wins on
Name conflict). OFF by default - a scope is enforced ALONE and displays are
self-sufficient. Then `Resolve-ShopdbPayloads` stages http/inline payloads
(section 3).
5. Engine call (the integration point):
`& $EnginePath -ManifestPath <resolved> -PCType $Scope -InstallerRoot <payloads dir> -LogFile <log>`.
6. `ConvertTo-ShopdbSummary` normalizes whatever came back (summary object,
array of emitted objects, bare int, $null) into
`@{Installed;Skipped;Failed;Filtered;Results;EnforcerVersion}`, then
`New-ShopdbReport` maps to the lowercase wire contract and
`Send-ShopdbReport` POSTs it. All best-effort; the whole script exits 0 no
matter what (fail-safe: a broken web app never breaks a PC).
### The engine contract (Install-FromManifest.ps1, lib 2.6)
Mandatory params: `-ManifestPath`, `-InstallerRoot`, `-LogFile`; optional
`-PCType`, `-PCSubType`. Entry Types: MSI, EXE, CMD/BAT, PS1, INF, File,
Registry. Detection: Registry, File, FileVersion, Hash, MarkerFile,
ValueMatches, pnputil, Always. Filters: PCTypes (with old/new-name alias
groups), TargetHostnames, TargetMachineNumbers, `_CmmVersion`. Exit 0/1/2
unchanged for the SMB path; NEW in the API cutover: the engine ends with
`Write-Output` of a summary pscustomobject
(`Installed/Skipped/Failed/Filtered/EnforcerVersion/Results`), which is the
only thing on the success stream (logs go via Write-Host), so
`& $EnginePath ...` captures it cleanly.
`SelfHealed` on a result means a REAL drift correction (a detected-missing
entry that got reinstalled). `Always`/no-detection entries install every cycle
by design and are not flagged, so the server-side status derivation
(`service.record_enforcement_report`: failed > selfhealed > ok) stays honest.
### The display dispatcher: server-resolved role, file fallback
For kiosks, per-subtype behavior does not fork the scope: ONE scope
(`gea-shopfloor-display`), one inline dispatcher entry (built by
`plugins/geenforce/seed_display_scope.py`) that resolves what the display should
show at enforce time, in two steps:
1. **Server (authoritative):** `GET
$KioskBaseUrl/api/dashboarddefaults/display-role?fqdn=<own-fqdn>`. This is a
PUBLIC endpoint (no token). The server matches the FQDN against the
`dashboarddefaults` table (IP fallback) and returns `{role, path,
businessunitid, businessunit}`. Roles: `dashboard`, `lobby`, `partskiosk`.
Changing a display's job is now a server-side edit; no touch on the PC.
2. **Fallback (offline, or unmapped):** the local
`C:\Enrollment\display-type.txt` value against the `DISPLAY_TYPE_TARGETS` map
baked into the script. If neither resolves, the dispatcher logs and
configures nothing.
The FQDN is built as `F<BIOS serial>.<domain>` (GE device naming); the domain
comes from HKLM `DisplayFqdnDomain` or the built-in default
(`device.geaerospace.net`). `DetectionMethod = Always`, but the script is
idempotent: it rewrites the all-users Startup shortcut (never Start-Process -
see gotchas) only when the resolved target changed.
### Legacy autostart self-heal
The old GE Aerospace Dashboard / Lobby Display Inno installers planted three
autostarts each: a Public-Desktop `.lnk`, an all-users Startup `.lnk`, and an
`HKLM ...\CurrentVersion\Run` value, all launching Edge at now-dead URLs
(`/shopfloor-dashboard/`, `/tv-dashboard/`) which 404 to a white screen. Because
those installers were 32-bit, the Run value was WOW64-redirected into
`HKLM\SOFTWARE\Wow6432Node\...\Run`, invisible to 64-bit tooling - the reason it
survived earlier cleanup. The dispatcher (`build_dispatcher_script` in
`seed_display_scope.py`) now sweeps, every enforce cycle: both the native and
Wow6432Node registry views, every loaded user hive (HKU), Run + RunOnce +
Policies\Explorer\Run, matching by the legacy value names AND by any value
pointing at the old URLs; plus every per-user and common Startup folder; then
kills any running old-URL Edge. A read-only locator,
`<imaging-share>/github/find-legacy-kiosk-autostart.ps1`, hunts all these locations
(and Edge startup-URL policy, scheduled tasks, Assigned Access) when a straggler
persists.
The kiosk shortcut is a direct Edge shortcut (no launcher/VBS); the fix ships by
re-publishing the code-authored `gea-shopfloor-display` scope
(`seed_display_scope(publish=True)`), not an import-share.
---
## 6. Bootstrap for share-less PCs
A share-less PC cannot pull its first files from the share, so the bootstrap
itself is downloadable from the web app.
`Install-ShopdbKiosk.ps1` (source of truth:
the imaging share (`shopdb-migration/kiosk-installer/`)) is hosted at
`C:\inetpub\wwwroot\shopdb\installers\kiosk\` and downloadable at
`{BaseUrl}/installers/kiosk/Install-ShopdbKiosk.ps1`. Run elevated on the PC:
```
Set-ExecutionPolicy Bypass -Scope Process -Force
$u = 'https://shopdb.example.net/shopdb/installers/kiosk/Install-ShopdbKiosk.ps1'
Invoke-RestMethod $u -OutFile "$env:TEMP\Install-ShopdbKiosk.ps1"
& "$env:TEMP\Install-ShopdbKiosk.ps1" -DisplayType Lobby -CollectorKey 'shopdb_pat_...'
# add -ShopdbToken 'shopdb_pat_...' only if the subnet is NOT allowlisted
```
What it does (idempotent; re-running is also the manual update path):
1. Writes `C:\Enrollment\display-type.txt` (the subtype) and
`C:\Enrollment\pc-type.txt` (the scope, default `gea-shopfloor-display`).
2. Writes HKLM `BaseUrl` [+ `ApiToken`] + `CollectorKey`, then locks the key
ACL to SYSTEM + Administrators.
3. Downloads runner + module + engine + collector from
`{BaseUrl}/installers/kiosk/` over HTTPS (TLS 1.2 forced).
4. Registers the two SYSTEM tasks (section 5).
5. Starts both once so the PC is live immediately.
IIS prerequisite: the `installers\kiosk` web.config MUST carry
`<staticContent>` MIME maps for `.ps1`/`.psm1` (`text/plain`) or IIS 404.3s
the downloads (see gotchas).
Delivery options for the bootstrap itself:
- Imaging-baked: the display image runs it (or lays down the same state) at
imaging time - see `project-display-self-contained`.
- Installer push: Intune/hand-run the one-liner above on an already-deployed
PC. This is how the pilot kiosks were done
(`shopdb-migration/run-on-kiosk-F.txt`).
---
## 7. Asset reporting via the collector
`Report-AssetToShopDB.ps1` (in the kiosk bundle; also deployed in the SMB
`common\` scope for the share fleet) POSTs the PC's identity to:
```
POST {BaseUrl}/api/collector/computers
X-API-Key: <collector.ingest PAT or COLLECTOR_API_KEY env key>
```
Auth (`shopdb/core/api/collector.py`, `_check_collector_auth`): a managed
token scoped `collector.ingest` (Bearer or X-API-Key) OR the
`COLLECTOR_API_KEY`/`COLLECTOR_API_KEY_COMPUTERS` env key. The GE-Enforce IP
allowlist does NOT apply here - the collector always needs a key, read from
HKLM `CollectorKey` (or the manifest entry's `Args -ApiKey`).
Schema (`plugins/computers/plugin.py`, `get_collector_schema`): identity field
`hostname` (required); optional `machinenumber`, `pctype`, `pcsubtype`,
`serialnumber`, `loggedinuser`, `lastboottime`, `lastcheckin`, `ipaddress`,
`vendorname`, `modelnumber`, `osname`, `installedsoftware`, `defaultprinter`,
`printers`. All lowercase concatenated (the project naming convention).
`apply_collector_payload` upserts idempotently by hostname (falls back to
`Asset.assetnumber`), creates the Asset+Computer when missing, maps
`machinenumber` -> `Asset.assetnumber` (imaging placeholder `9999` skipped both
client- and server-side), `pctype` -> ComputerType via the settings mapping,
and creates Vendor/Model/OS rows as needed. Fields not posted are not touched -
a partial read never blanks a good value, so the script only includes fields it
actually resolved.
Client details worth keeping: machine number resolution order is eDNC registry
`MachineNo` (WOW6432Node then native) -> `C:\Enrollment\cmm\cmmid.txt` ->
`C:\Enrollment\machine-number.txt`; the reported `ipaddress` is filtered to
the corp ranges (same two CIDRs as the allowlist) so a machine-LAN controller
NIC never lands in shopdb.
---
## 8. HARD-WON GOTCHAS
Read this section before touching ANY of the moving parts. Every bullet cost
real debugging time. Format: symptom -> cause -> fix.
- **PS crash "property 'X' cannot be found" under Set-StrictMode** ->
the module runs `Set-StrictMode -Version Latest`, and engine
results/summaries arrive as EITHER hashtables or PSCustomObjects with
varying key casing; direct `$obj.Key` access on an absent key throws ->
route every dynamic property read through `Get-ShopdbProperty` (handles
both shapes, case-insensitive, returns $null when absent). Never dot into
parsed JSON or engine output directly.
- **Register-ScheduledTask rejects the repeating trigger** -> passing
`-RepetitionDuration [TimeSpan]::MaxValue` serializes to an out-of-range
Duration the Task Scheduler XML schema rejects -> use
`-RepetitionInterval` ALONE; it defaults to indefinite repetition
(verified Win11 / PS 5.1). See `Register-SystemTask` in
`Install-ShopdbKiosk.ps1`.
- **Engine exits 2 / "InstallerRoot not found"** -> `-InstallerRoot` and
`-LogFile` are MANDATORY engine params and InstallerRoot must EXIST ->
the runner always passes both and pre-creates the payloads dir before the
engine call. Any new caller must do the same.
- **http payload "not found: C:\...\C:\..." (path doubling)** -> the engine
resolves entry paths as `Join-Path $InstallerRoot <field>`; writing the
staged payload's ABSOLUTE path into the entry made the engine double it ->
`Resolve-ShopdbPayloads` writes the LEAF filename only, and the runner sets
`-InstallerRoot` to the payloads cache dir. Keep those two in lockstep.
- **Manifest fetch "works" but parsing fails / cache garbage** -> if the
manifest is served with a non-JSON content type, PowerShell 5.1
`Invoke-WebRequest` `.Content` comes back as a `byte[]` instead of a string
-> the server route returns `mimetype='application/json'` (see
`get_manifest`); any mock server or proxy in the chain must do the same.
The client also validates JSON before overwriting last-known-good.
- **Bootstrap download 404 (HTTP 404.3)** -> IIS refuses to serve unknown
static extensions; `.ps1`/`.psm1` have no default MIME map -> add
`<staticContent><mimeMap fileExtension=".ps1" mimeType="text/plain" />`
(and `.psm1`) in the `installers\kiosk` web.config, with `<remove>` first
if inherited.
- **IP allowlist spoofable / mysteriously not matching** -> raw
`X-Forwarded-For` is attacker-controlled (proxies append; first hop is the
caller's to write) -> `_ip_allowlisted` uses `request.remote_addr` via
`_trusted_client_ip`, which is only correct because the IIS URL-Rewrite
rule OVERWRITES X-Forwarded-For with REMOTE_ADDR and waitress trusts only
127.0.0.1 as proxy. The rule is a hard dependency: never remove it, and
verify the spoof is closed after server changes
(`curl -H "X-Forwarded-For: 192.0.2.10"` from a non-allowlisted host
must get 401).
- **Kiosk browser never appears though the dispatcher "ran fine"** -> the
enforce task runs as SYSTEM in session 0, which has no interactive
desktop; `Start-Process msedge.exe` opens INVISIBLY there -> write an
all-users Startup shortcut
(`C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Startup\ShopDB Kiosk.lnk`)
and let the auto-login user launch it in a visible session. This is why
the dispatcher is shortcut-based.
- **TLS/transport errors on older images ("could not create SSL/TLS secure
channel")** -> Windows PowerShell 5.1 does not reliably negotiate TLS 1.2
by default -> every network helper calls `Set-ShopdbTls`
(`[Net.ServicePointManager]::SecurityProtocol = Tls12`) first; the
bootstrap and collector force it too. Any new script that touches the API
must do the same.
- **Secrets readable by the kiosk user / visible in Task Scheduler** ->
default HKLM\SOFTWARE ACL grants BUILTIN\Users read, and task arguments
are world-readable in the task XML -> restrict the
`HKLM:\SOFTWARE\GE\ShopDB` key ACL to SYSTEM + Administrators (the
bootstrap does), and NEVER put a token in a task's `-Argument` string -
scripts read `ApiToken`/`CollectorKey` from HKLM at run time.
- **Enforcement Reports show 0/0/0 with no per-entry detail** -> the engine
historically returned nothing on the success stream, so the runner had no
counts to report -> the 2.6 engine emits the summary object as its ONLY
`Write-Output` (all logging is Write-Host), and
`ConvertTo-ShopdbSummary` tolerates non-compliant engines by zero-filling.
If reports go 0/0/0 again, the engine on that PC is pre-summary - update
it.
- **Two kiosk browsers fighting / stale kiosk launch after retarget** -> old
installs left their own Startup launchers behind, in several flavors ->
the dispatcher's sweep must match ALL of: single- AND double-dash `-kiosk`
arguments (the regex `-kiosk` matches both), shortcuts whose args carry
shopdb URLs (`SHOPDBHOST`, `/shopdb/`, the dead `shopfloor-dashboard`
route), the imaging installers' `GE Aerospace Dashboard*` / `GE Aerospace
Lobby*` shortcut names, and `.url` files pointing at the kiosk routes.
Extend the sweep whenever a new launcher naming appears.
---
## 9. How this was verified
Two complementary verification passes; keep BOTH for future cohorts because
they catch different bug classes:
- **Code review** finds logic bugs: the XFF spoof hole, the StrictMode
absent-key crashes, the status-derivation trap (`installed>0` is not
self-heal), the report casing mismatch.
- **VM smoke test** finds integration/OS-version bugs: the
RepetitionDuration serialization rejection, the .ps1 MIME 404, session-0
invisibility, TLS negotiation, byte[] vs string response content - none of
which a read of the code surfaces.
The VM rig:
- The win11 virt-manager VM (see `project-win11-vm` /
`reference-vm-qga-as-system` memory), driven by
the imaging share (`ednc-bins/qga-run.py`) - qemu guest agent
`guest-exec`, which runs PowerShell AS SYSTEM. That matters: the scheduled
tasks run as SYSTEM, so testing as SYSTEM reproduced the session-0 and
profile-less behaviors an interactive test would have masked.
- A mock HTTP server on the host exposing `/api/geenforce/manifest` (served
`application/json`), `/api/geenforce/payload/<sha>`, and capturing the
`/api/geenforce/report` POST body. This exercised the full client loop -
ETag/304, cache fallback (kill the server mid-test), payload hash
verification, resolved-manifest rewrite, engine run, summary -> report
mapping - without touching prod.
- Full pass = bootstrap installer run end to end in the VM, then assert: both
tasks registered, HKLM written + ACLed, manifest cached, payloads staged by
sha, Startup shortcut written, report captured with real counts.
---
## 10. Deploy
TWO independent channels. Confusing them is the classic mistake: the client/
engine/collector/bootstrap are imaging-share artifacts, NOT deployed by the
git pipeline.
### Channel 1: backend + frontend (the git .cmd pipeline)
Prod is air-gapped from dev; code moves via a git bundle on the share
(`\\<imaging-share>\github\shopdb-flask-pub.bundle`) and the .cmd
scripts in the imaging share (`github/`), run on the work PC:
1. `pull-shopdb-bundle.cmd` - fetch the bundle into the local clone
(ff-only).
2. `update-shopdb-github.cmd` - push the clone to GitHub.
3. `update-dev-server.cmd` - robocopy the clone to `X:` (dev `/ops` tree) +
the `/ops`-base frontend dist. Validate on `/ops` FIRST.
4. `update-prod-server.cmd` - robocopy to `Y:` (`C:\inetpub\wwwroot\shopdb`)
+ the `/shopdb`-base dist (`frontend-dist-subpath-shopdb`). Both scripts
carry instance guards (web.config MOUNT_PATH check) - do not bypass them.
Then RDP: `Restart-WebAppPool shopdbflask-prod`, and if migrations/deps
changed: `flask db upgrade`, `flask plugin upgrade-all`, `flask seed
permissions`, `flask seed settings`.
(The fast path used during the pilot - robocopy just the changed plugin files
from `shopdb-migration\prod-patch-geenforce\` + pool restart, per
`deploy-server-patch.txt` - works, but the same commits must ALSO go through
the bundle pipeline or prod drifts from git.)
### Channel 2: client + engine + collector + bootstrap (the kiosk bundle)
These live at the imaging share (`shopdb-migration/kiosk-installer/`) and
deploy by robocopy from the work PC (Z: = share, Y: = prod app dir):
```
robocopy Z:\shopdb-migration\kiosk-installer Y:\installers\kiosk /E
```
That directory (bundle contents: `Install-ShopdbKiosk.ps1`,
`Invoke-ShopdbEnforce.ps1`, `ShopdbEnforceClient.psm1`,
`lib\Install-FromManifest.ps1`, `Report-AssetToShopDB.ps1`, `web.config` with
the MIME maps) IS the distribution point - PCs download from
`{BaseUrl}/installers/kiosk/`. Reference copies of the client kit also live in
the repo at `plugins/geenforce/client/` and the engine's source of truth is
`the imaging share, common/lib/Install-FromManifest.ps1`; when the engine
or client changes, update the kiosk bundle copy too (nothing syncs it
automatically). PCs pick up new bytes by re-running the bootstrap one-liner.
### Server prerequisites (once per site, all three or token-less clients 401)
1. Publish the scope(s) - `seed_display_scope(publish=True)` or
`flask geenforce publish <scope>`.
2. Seed `geenforce_allowed_cidrs` = `192.0.2.0/24,198.51.100.0/26`
(Settings rail > GE-Enforce Settings, or SQL upsert into `settings`).
3. Mint tokens (Settings > API Tokens, Restrict permissions ON):
`collector.ingest` (required, the kiosk `-CollectorKey`) and
`geenforce.fetch` (fallback for non-allowlisted subnets; resource-bind it
to the scope).
4. Keep the IIS URL-Rewrite XFF-overwrite rule enabled (section 2).
---
## 11. PLAYBOOK: extending to a new pc-type
Checklist for cutting any of the remaining scopes (`gea-shopfloor-cmm`,
`-collections`, `-keyence`, `-genspect`, `-heattreat`, `-partmarker`,
`-nocollections`, `common`) over to the API.
1. **Decide the delivery model.** Does this pc-type keep SMB access? If yes,
the intended cheap cutover is manifest-over-API + payloads-still-smb
(entries stay `smb`, nothing to upload). Only a genuinely share-less PC
needs http/inline payload conversion. Note the payload endpoint's 512 MB
default ceiling (`GEENFORCE_PAYLOAD_MAX_BYTES`) before promising huge
installers over HTTPS.
**THE CHEAP CUTOVER DOES NOT WORK WITH THE CURRENT RUNNER. Verified on the
win11 VM against lib 2.6, 2026-08-13.** Two blockers, both in delivery, not
in the manifests:
- `Invoke-ShopdbEnforce.ps1` has no `-InstallerRoot` parameter. Outside
shadow mode it hardcodes the payloads cache
(`Join-Path (Split-Path -Parent $manifestToRun) 'payloads'`). That is
right for displays, where every entry is inline, and wrong for every
`smb` entry: the engine resolves `Join-InstallerPath $InstallerRoot
<field>`, so `scripts/Install-AcroReader.cmd` is looked for under the
payload cache and logs `CMD/BAT not found`. Measured on the VM with the
same manifest and engine: InstallerRoot=payloads gave 0 installed /
2 failed, InstallerRoot=<scope dir> gave 2 installed / 0 failed.
- `-IncludeCommon` cannot work for `smb` payloads at all. Payloads live
under their own scope directory (`common/scripts/...` vs
`gea-shopfloor-nocollections/scripts/...`), so a merged manifest needs
two roots and the engine takes one. `Join-InstallerPath` also rejects
rooted values outright, so absolute paths are not a workaround - that
rejection is deliberate, to stop an entry escaping its share root.
This is why the real dispatcher does NOT merge: `common/GE-Enforce.ps1`
builds a target list and calls the engine ONCE PER SCOPE with
`-InstallerRoot $t.Root`. Any fix should follow that shape.
Until it is fixed, a share-attached cohort can still run SHADOW mode
(below), which is unaffected: shadow sets InstallerRoot to the share scope
dir, which is the working path.
2. **Get the scope into shopdb.** Existing share manifest:
`flask geenforce parity` then `flask geenforce import-share --scope
<name>`. New/reworked scope: author in code following
`seed_display_scope.py` (registry heals with ValueMatches detection,
idempotent Always PS1s, data-driven tables for anything per-subtype).
3. **Convert payloads (share-less only).** Small scripts/configs ->
`store_inline_payload` / the entry payload upload endpoint. Installers ->
`flask geenforce add-payload <file>`, set
`PayloadSource=http` + `PayloadSha256` + `PayloadRef` on the entry.
Remember: `PayloadRef`'s extension decides the staged filename's
extension.
4. **Publish.** New version every publish; clients converge within one
enforce cycle. Verify with
`curl "{BaseUrl}/api/geenforce/manifest?pctype=<scope>"` from an
allowlisted host.
5. **Auth for the PCs.** Subnet already inside
`192.0.2.0/24,198.51.100.0/26` -> token-less, nothing to do. New subnet
-> add its CIDR to `geenforce_allowed_cidrs` (Settings rail validates).
Not network-trustable -> mint a `geenforce.fetch` token resource-bound to
the scope and deliver it to HKLM `ApiToken`.
6. **Bootstrap the client.** Share-attached fleet: adapt the dispatcher /
`Install-GEEnforce.ps1` path (the pilot flow in
`shopdb-migration/kiosk-api-pilot.txt`: shadow first, then flip, then
DISABLE the old share enforce task so the two do not fight). Share-less:
the `Install-ShopdbKiosk.ps1` pattern - generalize `-Scope` and skip the
display-only pieces. Decide `-IncludeCommon`: displays run without it;
a non-display share-less PC that needs the fleet-wide common entries over
HTTPS turns it on AND requires common's entries to be payload-converted
first (an SMB-payload common entry will fail on a share-less PC).
7. **Run SHADOW mode first** on one pilot PC
(`-ShadowMode -ShareManifestPath <share manifest>`): fetch + compare +
report with zero behavior change. Watch the shadow diff log lines and the
Enforcement Reports row before flipping.
8. **Re-read section 8 (gotchas).** Especially: LEAF filenames, StrictMode
property access, SYSTEM/session-0, task trigger serialization, MIME maps
if you host new downloadables.
9. **Verify on the VM** (section 9) before the pilot PC: bootstrap +
enforce cycle against a mock or the dev `/ops` instance, as SYSTEM via
qga-run.py.
10. **Pilot one PC, then the cohort.** Keep the rollback in your pocket:
disable/remove the new task, re-enable the share task, remove HKLM
`BaseUrl` - the share path is untouched by all of this.
11. **pc-type mapping.** Make sure the collector's pctype mapping
(computers plugin settings, `pctypemap`) covers the scope name so asset
reports do not warn `no ComputerType mapping`.
---
## 12. Open items / TODO
- **Name resolution for reported users.** `loggedinuser` lands as a bare
username; resolving it to a display name depends on either the
`wjf_employees` `First_Name`/`Last_Name` data or shopdb User accounts
existing for shopfloor users. Not wired up; reports show raw usernames
until it is.
- **Old imaging-installer registry cleanup.** Displays imaged before the API
cutover carry leftover state from the old imaging-time kiosk installers
(superseded shortcuts are already swept by the dispatcher; stale registry
values are not yet cleaned). A cleanup entry in the display scope is the
natural vehicle.
- **The SMB fleet is still on the share.** Only displays/kiosks are on the
API. cmm/collections/keyence/genspect/heattreat/partmarker/nocollections/
common still enforce from SFLD; section 11 is the path. Shadow mode makes
each migration observable before it changes anything.
- **3DPrintRoom kiosk target is a placeholder.** `DISPLAY_TYPE_TARGETS`
points it at `/parts-kiosk`; confirm the real route with the floor team
before publishing to production 3D-print-room displays (flagged in
`seed_display_scope.py`).
- **Fast-path prod patches vs git.** The `prod-patch-geenforce` robocopy
path can leave prod ahead of the repo; reconcile by pushing the same
changes through the bundle pipeline (section 10, channel 1).

Some files were not shown because too many files have changed in this diff Show More