316 Commits

Author SHA1 Message Date
cproudlock
89e880afc3 Release 0.8.0
Some checks failed
CI / backend (push) Failing after 6s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 6s
The Windows installer has never shipped under a version: v0.7.0 was tagged
before any of it existed, so every build handed out so far stamped a server with
0.7.0. Two servers running different builds were indistinguishable, and the
installer logged each upgrade as "same version already installed" rather than
recording what changed. This cuts the release that fixes that.

0.8.0 rather than a patch: the air-gapped installer is a new capability, and
pre-1.0 semantic versioning puts that in the minor slot (ADR-007).

CHANGELOG gains a 0.8.0 section covering the twelve defects a real Windows
Server 2019 install surfaced, the move from inferring "is this a re-run of my
install?" to recording it, and the operator documentation.

deploy/site-profile-universal.json is now in the repository. Released builds
were being produced from a profile in a temporary directory, so the next release
could not have been reproduced once that file was cleaned up.
docs/RELEASING-WINDOWS.md points at the committed profile and says why.

scripts/gen_openapi.py reads __version__ out of shopdb/__init__.py instead of
restating it. Its hardcoded copy had already drifted a release behind, which is
the same mistake that once shipped an installer stamped with the wrong version.
2026-08-05 07:34:05 -04:00
cproudlock
fb53161578 Answer "is this a re-run of my install?" from a record, not from the machine
Some checks failed
CI / backend (push) Failing after 9s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 10s
CI / migrations-mysql (push) Failing after 7s
The installer inferred that question from whatever the server happened to look
like: a MySQL service exists, the database has tables, the site exists, the venv
exists. None of those record who created them. A retry after a failed first
install was therefore taken for an upgrade of somebody else's working system,
which produced two dead ends on exactly the retry the wizard invites: stage 3
demanded a mandatory backup of a database its own failed attempt had written,
and then refused to prune tables it had created minutes earlier, because core
migration 7d05 seeds access protocols owned by the computers plugin and any
profile without that plugin hit the refusal every single time.

An install record at ProgramData\ShopDB-Flask\install-state.json answers it
instead. It is written when provisioning STARTS rather than when it finishes,
because the run that dies halfway is precisely the run whose retry needs it, and
it records what this installer created as it goes, so a crashed run no longer
leaves the next one guessing from the machine.

During unfinished first provisioning the pre-migration backup becomes advisory
and prune may force, since every row present was written by an earlier attempt
of the same install. On an established install both stay exactly as they were.
The classification is deliberately asymmetric: an install predating this record
carries a version stamp and probably real data, so it is treated as established
and keeps the mandatory backup. Guessing "first run" there would arm
prune --force against live tables.

Get-CreatedItems comma-protects its return. A zero-length array returned from a
PowerShell function unrolls to $null, and $null.Count is fatal under StrictMode
2.0 - the same fault that made bundle verification fail on every install
earlier. The harness caught it before it shipped.

Tests: deploy/windows/installer/tests/test-install-state.ps1 exercises new
servers, retries, completed installs, unrecorded-but-stamped installs, records
naming another directory, corrupt records, and persistence across a crash.
tests/test_installer_state.py runs it wherever pwsh exists and asserts the
invariants as text everywhere else. Both were confirmed to fail when the prune
gate or the comma protection is removed.

pytest.ini stops collection walking into deploy/windows/installer/bundle, which
is build output holding a complete second copy of the application. Importing
every plugin twice made SQLAlchemy refuse a redefined table and the whole suite
fail to collect, on a tree with nothing wrong in it, purely because an installer
had been built first. It surfaced only when the bundle grew from four plugins to
thirteen.
2026-08-04 21:42:49 -04:00
cproudlock
412c2dc877 Record the code-signing decision
Some checks failed
CI / backend (push) Failing after 7s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 6s
Waiting for a certificate from the organisation's own certificate authority
rather than buying one from a public CA. Every server this installer runs on is
centrally managed and already trusts that root, so an internally issued
Authenticode certificate removes the unknown-publisher warning exactly where it
matters; a public certificate would buy trust on machines this software never
reaches.

Notes the interim measure that costs nothing: publish the SHA-256 through a
channel separate from the installer, since a hash beside the file is only as
trustworthy as write access to that location.

Wording avoids naming internal infrastructure, since docs/ is published.
2026-08-04 21:25:14 -04:00
cproudlock
1c04ff28b9 Close the remaining installer review findings
Some checks failed
CI / backend (push) Failing after 8s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 7s
Eleven findings, grouped by the root cause each belongs to.

Wizard input reaching a command line unchecked (ShopDBFlask.iss). Port fields
were spliced in bare and arrive as [int] parameters, so a blank or mistyped
port shifted every argument after it; both port fields are now validated as
1-65535 digits. A path ending in a backslash, which is what a drive root looks
like, ended its argument with \" and CommandLineToArgvW read that as an escaped
quote, so paths are now quoted through a helper that doubles the trailing
backslash. A drive root is refused outright as well: uninstall deletes the
application directory recursively, so installing to D:\ would have wiped the
drive on removal. The password handoff was written with SaveStringToFile, which
writes an AnsiString, and read back as UTF-8, so a correct non-ASCII password
was reported as wrong; it now goes out as UTF-8 without a BOM.

Launching without checking the result. Plugin deregistration invoked "flask
plugin uninstall" without --yes, and the command carries a click
confirmation_option that aborts with exit 1 when nothing can answer the prompt,
so it could never once have succeeded; the bare 2>&1 under EAP Stop then turned
that into a terminating error which the catch downgraded to a warning while the
plugin directory was deleted regardless. It now passes --yes, brackets the
error preference, restores the location in a finally, and keeps the code on
disk unless deregistration actually succeeded. MarkShortcutRunAs had four
quotes where it needed three, which kept the whole command inside one Pascal
literal so LnkPath was never interpolated and no shortcut ever got the
elevation flag; its exit code is now logged too.

Comparing IIS physical paths as raw strings. IIS stores the path as typed, so
it may carry environment variables or a trailing backslash. A Test-SamePath
helper now normalises both sides. That closes a real hazard in uninstall, which
matched applications on alias alone and would remove an unrelated application
of the same name under another site, unattended, since -OnFailure never
suppresses the confirmation.

Accepting existing IIS state without reconciling it. "Site already exists" took
the site however it was, so re-running with a different port left the old
binding while CORS_ORIGINS, the firewall rule and the smoke test all used the
new one, failing a working server. It now refuses with both ports named rather
than silently re-binding, and refuses a site of that name serving a different
directory.

Preflight rows drawn past the panel. The failures loop had no cap at all and
the warnings loop capped at 6, a number unrelated to the panel, which holds
about three rows. The cap is now measured from the panel height, applies to
both loops, and the footer counts what was actually left out instead of
inferring it.

Also: a failed upgrade now says the application pool is still stopped and how
to start it, rather than only "part-configured", since stage 2 stops a pool
that was serving. It is deliberately not restarted automatically, because after
a stage 3 failure the deployed code and the schema may disagree. shopdb-admin
Restart-App starts a stopped pool or site instead of recycling, which is a
no-op on a stopped pool and then reported the application as unresponsive. A
dead Write-Log line that parsed as three arguments is gone, and a preflight
warning no longer tells the operator to add a directory to a compiled exe.
2026-08-04 21:16:46 -04:00
cproudlock
ce521e84a5 Lock down backup directory ACLs, and let the uninstaller reach IIS
Some checks failed
CI / backend (push) Failing after 8s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 6s
Two findings from the installer review, both of which fail silently.

Database dumps were readable by every authenticated user. A directory created
under ProgramData inherits BUILTIN\Users:(I)(OI)(CI)(RX), and a dump contains
every row including the users table and its password hashes. The installer
applied an owner-only ACL, but only in the branch that CREATED the directory,
so a directory created first by the console (shopdb-admin.ps1 backup) kept the
inherited permissions and the installer could never repair it. The ACL is now
re-applied on every run rather than only on creation, and the grants are made
inheritable with (OI)(CI) so dumps written into the directory later are covered
too. shopdb-admin.ps1 applies the same hardening for the default location, and
for an operator-named path says the dump holds password hashes rather than
silently rewriting the ACL of a directory that is theirs.

Verified on Windows: before, the directory carried BUILTIN\Users:(I)(OI)(CI)
(RX); after, only SYSTEM and Administrators, and a file created inside inherits
exactly those two. Without (OI)(CI) that file would not have been covered.

The uninstaller could not remove anything in IIS. [UninstallRun] launched a
bare "powershell.exe", and the Inno uninstaller is a 32-bit process, so WOW64
resolved it to the 32-bit PowerShell, which cannot see the IIS provider. The
site, application pool and application survived, pointing at a directory that
HAD been deleted, while Windows reported a clean uninstall. It now uses the
same Sysnative path as the [Run] entry, which was the last unshielded launch
site in the file.
2026-08-04 21:04:54 -04:00
cproudlock
1d73bd477e Document what future Windows releases look like, for operators and for builders
Some checks failed
CI / backend (push) Failing after 8s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 7s
Two audiences, two documents. Both were only in people's heads.

UPDATES-WINDOWS.md is for whoever runs a server: updates arrive as one
self-contained exe, an update takes two to four minutes, the site is down for
that time, .env and data and any hand-edited web.config are kept, unticking a
feature never removes it, the database is backed up and verified first, and a
downgrade is refused because migrations only go forwards. It covers both kinds
of security release, application and third-party, and explains that the
CycloneDX inventory staged on every server is what answers a published
vulnerability question. It also says plainly that the exe is not signed and the
checksum is the integrity check to rely on today.

It answers one question the existing docs did not address at all: the effect on
other sites sharing the same IIS server. The application pool is isolated and
the configuration is scoped to its own path, so other sites keep their own
handlers. What IS shared gets named rather than glossed: installing the IIS
modules and writing server-level configuration recycles application pools
across the server, which can drop requests in flight and clears in-memory
session state, though IIS is never stopped and no iisreset is issued. The two
IIS modules and the single permitted rewrite server variable are machine-wide
and stay behind on uninstall, deliberately, since another site may have come to
depend on them. The bundled database option collides on port 3306 with an
existing MySQL.

RELEASING-WINDOWS.md is for whoever builds releases: the three kinds of change
and the commands for each, why bundle-lock.json must be committed, the two
dependency traps that have each already cost a release, which generated files
must never be hand-edited, and the pre-release checks. It records the two known
gaps honestly - no code signing, and compiling still requires Windows and a
person.

UPGRADE.md and OPERATE-WINDOWS.md link to the operator document.
2026-08-04 20:54:05 -04:00
cproudlock
aeee210cf6 Repair a web.config that an earlier build made unusable
Some checks failed
CI / backend (push) Failing after 8s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 7s
Stage 4 deliberately leaves an existing web.config alone, because operators put
real changes in it: extra MIME maps, a /installers location, bindings, a
proxy-specific rule. Overwriting reverts those silently.

That rule had no exception, and earlier builds of this installer wrote an
<allowedServerVariables> block which is fatal on its own: the section is Deny
by default, so IIS rejects the entire file with 500.52 before
httpPlatformHandler runs. Any server already installed would therefore keep the
broken file forever, with re-running the fixed installer powerless to help,
since the first thing stage 4 does is decline to touch it.

Strip just that element, keeping every other edit, and only when it contains
nothing besides the variable this installer adds. A block holding anything else
is somebody's deliberate change and is left alone with a warning. The previous
file is copied to web.config.before-xff-fix first.

Exercised against four inputs: the file earlier builds wrote, which is repaired
and still parses as XML with the rewrite rule intact; a block with an
operator-added variable, which is left unchanged; an empty block, which is the
$null.Count trap under Set-StrictMode 2.0 and is why the filter is wrapped in
@(); and an already-correct file, which is a no-op.
2026-08-04 20:06:47 -04:00
cproudlock
95b0b77c13 Allow HTTP_X_FORWARDED_FOR at server level instead of declaring it per-application
The stage 5 smoke test failure was a locked config section, but not one of the
two the installer unlocks. A diagnostic collected from the server returned:

  HTTP 500.52 - URL Rewrite Module Error
  Module RewriteModule, Handler httpplatformhandler
  Error Code 0x80070021
  Config Error: This configuration section cannot be used at this path.
  Config File: \\?\C:\shopdb-flask\web.config

handlers and httpPlatform were both overrideMode Allow and locked false, so
the unlock had worked. The section at fault was a third one,
system.webServer/rewrite/allowedServerVariables, which ships
overrideModeDefault="Deny". web.config declared <allowedServerVariables>
locally for the X-Forwarded-For rule, and IIS rejects that declaration
outright, failing the entire configuration before httpPlatformHandler ran.
python was therefore never launched and C:\shopdb-flask\logs stayed empty,
which reads as a dead application or a permissions fault and is neither.

Unlocking the section would let every site on the machine declare arbitrary
server variables. The installer now adds the single variable to the
server-level allow list, checking first because a duplicate add is an error,
and web.config no longer declares it. The rewrite rule is unchanged.

Verified by applying the installer's own uncommenting to the template and
parsing the result: one rewrite element, no allowedServerVariables, the rule
still setting HTTP_X_FORWARDED_FOR from REMOTE_ADDR.

shopdb-diagnose.py checked only the two sections the installer unlocks, so it
could not have named this one; the IIS error page did. It now reports the
lock state of the rewrite sections as well.
2026-08-04 20:04:19 -04:00
cproudlock
10ee3a3c58 Add a stage 5 diagnostic collector
Some checks failed
CI / backend (push) Failing after 7s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 7s
The stage 5 smoke test failing tells us only that IIS did not return 200. The
cause is in one of four places, and finding out which has taken a round trip
per guess. This gathers all four in one pass and writes a single report.

It records what IIS actually answers on localhost, 127.0.0.1, ::1 and the
machine name, including the status code and the parsed text of the IIS error
page; the site, application, pool and module state from appcmd, plus the
override state of the two config sections httpPlatformHandler needs; the
contents of web.config and the resolved httpPlatform processPath; whether the
venv can import shopdb and call create_app; the application logs, separating a
missing log from an empty one; the ACLs the pool identity depends on; and
recent HttpPlatform, WAS and W3SVC event log entries.

Secrets never reach the report. Values are read from .env first, then scrubbed
from every section before the file is written, which covers command output and
tracebacks that might quote them. A password embedded in any connection URL is
also masked whether or not it came from .env.

Standard library only, so it runs on the bundled runtime or any system Python.
Verified end to end on a Windows VM: it correctly reported a 404 with the IIS
error code for an absent application, and that localhost resolves to ::1 first.
2026-08-04 19:56:13 -04:00
cproudlock
97391cdee4 Unlock IIS config after the application exists, and report why the smoke test failed
Some checks failed
CI / backend (push) Failing after 8s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 7s
Two defects found in the stage 4 and stage 5 logs from a Windows Server 2019
install.

The scoped config unlock ran before the thing it unlocks existed. appcmd
resolves its location argument against applicationHost.config, but the unlock
was issued from the ACL block, ahead of New-WebApplication. On a first install
"Default Web Site/shopdb" is not there yet, so appcmd returned 80070003, "the
system cannot find the path specified", and the code fell through to unlocking
the section for the entire machine. That fallback exists for servers which
refuse the scoped form; it was instead the only path a first install could
take, so every install silently granted handler delegation server-wide. Moving
the block below site and application creation lets the scoped unlock work.

The smoke test discarded the diagnosis. Invoke-WebRequest raises on any
non-2xx, and the catch block kept nothing from the exception, so a fault IIS
had already identified by status code was reported as "site did not return
200 ... check the logs". It now records the status code and the text of the
IIS error page, and prints the tail of the HttpPlatform stdout log, which is
where a Python traceback lands. It also distinguishes a missing log from an
empty one: the first means the pool never launched python, the second that
python started and wrote nothing.

A non-200 that did not raise, such as a redirect, skipped the retry delay, so
the loop could spend all twelve attempts at once and report a timeout without
having waited.
2026-08-04 19:43:30 -04:00
cproudlock
a352a21a10 Declare packaging as a runtime dependency
Some checks failed
CI / backend (push) Failing after 8s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 7s
shopdb/plugins/loader.py imports packaging.specifiers and packaging.version
at module scope, but packaging was never listed in requirements.in. It was
present in every development and CI environment as a transitive dependency of
pytest, so the full suite passed while a venv built from requirements.txt
alone could not import shopdb at all.

The Windows installer builds exactly such a venv, so stage 3 failed on a
customer server with ModuleNotFoundError: No module named 'packaging', after
the runtime and all wheels had installed successfully.

Add packaging to requirements.in, recompile the hashed lockfile, and add the
wheel to the offline wheelhouse with the matching bundle-lock entry. The
recompile also picked up newer uv formatting: inline environment markers on
cffi and greenlet and shorter "via" comments. The pinned distribution set and
every existing hash are unchanged.

tests/test_runtime_dependencies.py guards the general case by scanning
shopdb/, plugins/ and scripts/ for unconditional third-party imports and
asserting each maps to a distribution pinned in requirements.txt. Test
dependencies are the blind spot for this class of failure, since they are
present wherever the suite runs and absent wherever it does not.
2026-08-04 19:12:00 -04:00
cproudlock
f6b621d126 fix(installer): keep -Wait, and use the stage-0 handoff even when .env exists
Some checks failed
CI / backend (push) Failing after 7s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 6s
Two defects from a Server 2019 run that got further than any before it - the
payload verified against the lock on a real server for the first time.

EMPTY EXIT CODE. "Python install failed (exit )" on an install that had actually
worked. Dropping -Wait to make -TimeoutSec enforceable left $p.ExitCode
unreadable: PowerShell only reliably populates it on a waited process. -Wait is
restored and the trade is now explicit - exit codes are load-bearing here, 1639
vs 1603 vs 3010 is the entire diagnosis, and a bounded wait is not worth losing
them for. -TimeoutSec is advisory: logged as an expected duration so a hang is
identifiable, not enforced. The code is also read defensively now, and an
unreadable one fails loudly rather than being taken for success.

That timeout has never worked - -Wait made the block dead code from the start -
so nothing is lost that was ever there. Trying to fix it broke something that
was working, which was the wrong trade to make silently.

STALE .env PREFERRED OVER A GOOD HANDOFF. The .dbpass fallback sat in the else
of "if .env exists", so it was consulted only when .env was absent. A
part-finished install HAS an .env, holding whatever password stage 2 last wrote;
if stage 0 has since regenerated the credential, .env is stale and .dbpass is
correct - and the installer preferred the stale one, giving "Access denied" with
the right password sitting unread on disk. The handoff is now applied before the
branch, so it covers both, and only when .env points at the local server so it
can never redirect a site whose database lives elsewhere.
2026-08-04 13:57:13 -04:00
cproudlock
21110b86eb fix(installer): clear the retry path, which is the path everyone is actually on
Some checks failed
CI / backend (push) Failing after 7s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 6s
Five defects from the Windows-defect review, each confirmed against the code
before changing it. Four of the five only fire on a RE-RUN - and after eight
attempts a re-run is the normal case, not an edge case, which is exactly why
they survived.

Invoke-Native, three defects in one function:
  - Any non-zero exit was failure. 3010 and 1641 mean "done, reboot required",
    and the VC++ redistributable returns 3010 on a server with a pending file
    rename - an ordinary state on a freshly patched box. It is now an accepted
    outcome for the installers that can report it, logged as a warning so the
    operator knows a reboot is owed.
  - -Wait blocks inside Start-Process until the child exits, so the -TimeoutSec
    block below it could never run. Every timeout on every MSI was decorative.
    The wait is now bounded here, followed by a parameterless WaitForExit so the
    redirected output is flushed before it is read.
  - The Python bootstrapper ran /quiet with no /norestart, free to reboot the
    server mid-install.

Stage 0 refused to run when a MySQL service existed - including the MySQL84 it
had registered itself. Every bundled-database retry dead-ended while the wizard
promised that re-running was safe. A foreign MySQL still blocks; ours is started
if stopped, and the create-the-server block is skipped. It also no longer tries
to bootstrap through a root account whose password it set on the previous run:
with the handoff present there is nothing to do, and without it there is no safe
automatic recovery, so it says what to do instead of guessing.

Stage 4's appcmd unlock used '2>&1' under $ErrorActionPreference = 'Stop', which
turns any appcmd stderr into a terminating error - so the exit-code test and the
server-wide fallback, the whole reason the block exists, were unreachable, and
the stage aborted after Python, the venv, the schema and the ACLs had been
changed.

Stage 3 ran prune-schema and treated its refusal as a failure. Refusing is the
designed outcome when a table holds rows, signalled with SystemExit(1), so
Invoke-Native killed the stage and the reporting written to explain the refusal
was unreachable. Core migration 7d05 seeds access protocols owned by the
computers plugin, so any profile omitting computers hit this on every retry.

The preflight's MySQL 5.6 index-flag check is a warning, not a blocker. It
inspects the LOCAL MySQL, which may not be the database being installed against;
stage 3 checks the one actually chosen. Same class as the HttpPlatformHandler
blocker fixed earlier.
2026-08-04 13:45:37 -04:00
cproudlock
5f350179b1 fix(installer): undo a fix applied twice, and load the checker at script scope
A review of the installer for Windows-only defect classes found seven live
issues. These two would have stopped the next attempt on any server.

DOUBLE-APPLIED GUARD. Yesterday's $null.Count fix was applied at BOTH ends:
Test-BundleLock returns ,$problems, and the call site also wrapped it in @().
The comma already hands the array back intact, so the extra @() nests it and
.Count becomes 1 regardless of how many problems there are. Every install would
have failed with "the bundle does not match bundle-lock.json (1 problem(s))" on
a byte-perfect payload. Applying the same guard at both ends was worse than
applying it at neither. Verified in a Windows VM against a real bundle: clean 0,
tampered 1, restored 0.

DOT-SOURCE SCOPE. bundle-lock.ps1 was dot-sourced INSIDE
Assert-BundleIntegrity, which loads it into that function's scope - every helper
it defines disappears when the function returns. Assert-BundleIntegrity itself
worked; the next caller, Get-WheelhousePythonTag, died with "The term
'Get-JsonProperty' is not recognized". It only fires where a venv already
exists, so greenfield was fine and every retry after a part-completed install
was not. Now loaded once at script scope, guarded so the stages that run without
a bundle still work.

Both were confirmed by running them rather than by reading: the nesting with a
three-case pwsh test, the scoping with a minimal repro.
2026-08-04 13:35:30 -04:00
cproudlock
14fedcee4c fix(installer): a clean payload crashed stage 2
Reported from Server 2019: "The property 'Count' cannot be found on this object"
immediately into stage 2.

Test-BundleLock returns an array of problems, and an EMPTY array means the
payload is exactly right. PowerShell unrolls a zero-element return into $null,
and under Set-StrictMode 2.0 $null.Count throws - so the branch that runs when
everything is correct was the one that could not run. Every failing bundle got
past it fine, which is why nothing caught it until the 8.3 path fix made
verification succeed for the first time on a real server.

Fixed at both ends: the call site wraps in @(), and Test-BundleLock returns
,$problems so no caller can be handed $null or a bare string depending on how
many problems there happen to be.

The other .Count uses in this file were already @()-wrapped and are unaffected.
2026-08-04 13:04:05 -04:00
cproudlock
fe091e751a fix(installer): the bundled database rejected connections it should have accepted
Two defects in the stage 0 bootstrap, both surfacing as "Access denied" on a
server where the operator was holding the correct password.

CREATE USER IF NOT EXISTS is a no-op on an existing user - it does NOT change
the password. Stage 0 generates a fresh password every run and overwrites
.dbpass with it unconditionally, so any path that re-runs the bootstrap over an
existing account left the handoff holding a password the server had never been
told. ALTER USER now follows each CREATE, so the stored password and the handoff
always agree.

The user was also only created for 'localhost' and '127.0.0.1'. On current
Windows, 'localhost' resolves to the IPv6 loopback FIRST, so an operator who
types localhost rather than 127.0.0.1 arrives as '<user>'@'::1' - an account
that did not exist - and MySQL answers "Access denied" naming a host they never
typed. The ::1 account is now created and granted alongside the other two.

Note the datadir guard means the first defect could not fire on a straightforward
re-run - stage 0 refuses a non-empty data directory before reaching the
bootstrap. It was still wrong, and reachable once the directory has been cleared
by hand, which is what the failure message tells operators to do.
2026-08-04 12:59:45 -04:00
cproudlock
189a474082 fix(installer): do not demand a password the installer already holds
Asked why the password box does not pre-fill from .dbpass. It should not - but
it should not have been demanding a password either.

.dbpass is the ACL'd handoff stage 0 writes when it creates the database itself,
and stage 2 already reads it automatically when no password is supplied. The
wizard, though, required a password whenever .env was absent, without checking
for the handoff. On a server where stage 0 had completed but stage 2 had not -
which is exactly what a partly-failed install leaves - the operator was blocked
on a secret the installer already had, and sent hunting for a generated password
they were never meant to handle.

Blank is now accepted when either .env or .dbpass is present, and the sign-in
page says so when it sees a handoff.

Deliberately NOT pre-filled into the password box, for two reasons. It is the
only copy of a generated password, so round-tripping it through a UI control and
back out through a temporary password file adds exposure for no benefit - stage 2
reads the file directly. And .dbpass belongs to the BUNDLED database; on the
existing-database page the operator is pointing at someone else's server, where
a locally generated password is simply the wrong answer.
2026-08-04 12:57:00 -04:00
cproudlock
5f18ca27a1 fix(installer): split the database page so every field is reachable
Some checks failed
CI / backend (push) Failing after 6s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 7s
Trimming the description brought the Username box back and left Password off the
bottom. CreateInputQueryPage stacks its fields below the description and neither
scrolls nor shrinks, so a field that does not fit is drawn past the surface and
simply never appears - no error, no scrollbar. Sizing the description against a
pixel budget that varies with DPI and font scaling is guesswork, and it had now
failed twice.

Connection details (host, port, database) and sign-in (username, password) are
now two pages of three and two fields. Both fit under any reasonable
description, at any scaling, without anyone having to estimate.

The upgrade hint about leaving the password blank moves to the sign-in page,
where the password field actually is. ShouldSkipPage hides both pages for the
bundled-database option, and the stage arguments read the values from their new
homes.
2026-08-04 12:51:39 -04:00
cproudlock
e650eb0220 fix(installer): database page lost its Username and Password boxes
Some checks failed
CI / backend (push) Failing after 8s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 7s
Reported from the Server 2019 test: the Existing database page rendered a
truncated "Username:" label and no input boxes at all below it, so there was no
way to enter connection details.

CreateInputQueryPage lays its fields out BELOW the description text. The
description had grown to include a five-line CREATE DATABASE / CREATE USER /
GRANT block, added so a DBA could be handed the exact SQL. With five fields
underneath, the last two fell past the bottom of the page surface, where they
are simply not drawn - no error, no scrollbar, just missing controls.

The description is back to three lines. The SQL moves to
docs/INSTALL-WINDOWS.md, which is where someone would look for it anyway and
where it can be copied without being retyped from a wizard page.

Wizard page descriptions are a fixed budget: anything long enough to be worth
reading twice belongs in the guide, not on the page.
2026-08-04 12:44:37 -04:00
cproudlock
c5797bb339 fix(installer): payload verification broke on 8.3 short paths
Some checks failed
CI / backend (push) Failing after 7s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 6s
A Server 2019 install reported all 96 payload files as simultaneously missing
and unexpected, with mangled names - wheels/heels/flask.whl,
python/ython/python-3.14.6-amd64.exe, mysqlclient/lient/mysql.exe. Exactly five
characters of each directory name survived, which is the difference between
ADMINI~1 and Administrator.

Inno extracts the bundle under C:\Users\ADMINI~1\AppData\Local\Temp\..., an 8.3
SHORT path. Resolve-Path kept that short form while Get-ChildItem returned the
long one, so the root was five characters shorter than the prefix being sliced
off every FullName, and every relative key came out wrong. The payload was
correct; the comparison was not - the verifier refused a perfectly good bundle.

The root now comes from Get-Item, which goes through the same provider as
Get-ChildItem so their path forms agree, and the prefix is checked with
StartsWith before being trimmed. If the two ever disagree again this throws
instead of inventing paths.

Verified against the real failure mode rather than assumed: running the check
through C:\SHOPDB~3\bundle in a Windows VM now passes.

Nothing on Linux or in a normally-pathed Windows directory could have caught
this - the short name only appears under a profile directory long enough to need
one, which is where Setup extracts.
2026-08-04 12:26:53 -04:00
cproudlock
263ae8e3b4 fix(installer): install the Visual C++ runtime before MySQL, and log the MSI
Some checks failed
CI / backend (push) Failing after 7s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 6s
Second failure from the Server 2019 test. The previous fix worked - msiexec went
from exit 1639 (ERROR_INVALID_COMMAND_LINE, which is why it printed its usage
dialog) to exit 1603 (ERROR_INSTALL_FAILURE), so the command line parses now and
the MSI itself is failing.

It failed in 1.1 seconds. An MSI that dies that fast has not begun installing;
it has failed a launch condition. MySQL 8.4 requires the Visual C++
redistributable and a bare Windows Server does not ship it - the same runtime
mysql.exe and mysqldump.exe import, which was visible when their DLL
dependencies were trimmed and went unnoticed.

Stage 0 now installs VC_redist.x64.exe from the bundle before touching MySQL,
skipping it when vcruntime140.dll is already present, and fails with a sentence
naming the requirement if the redistributable is absent from the bundle
altogether. vcredist\ is an optional locked payload.

msiexec also gets /l*v now. A bare 1603 names neither the failing action nor the
reason, and it is the most common MySQL install failure - diagnosing this one
took a launch-condition inference rather than a log. The MSI log lands beside
the installer's own in ProgramData, so the next failure is readable instead of
guessed at.
2026-08-04 12:17:36 -04:00
cproudlock
8d0afc40d3 fix(installer): bundled MySQL install failed on a malformed msiexec command line
Reported from a Windows Server 2019 test: a "Windows Installer" dialog listing
every msiexec /Option appeared, then the wizard reported that the bundled MySQL
database could not be installed. That dialog is msiexec's usage help - it prints
it when the command line does not parse - so the install never started.

Cause: $MysqlRoot defaulted to 'C:\Program Files\MySQL\MySQL Server 8.4', which
contains spaces. Invoke-Native wraps any argument containing whitespace in
quotes, producing "INSTALLDIR=C:\Program Files\...". msiexec takes public
properties as PROPERTY=value and expects the VALUE quoted -
INSTALLDIR="C:\Program Files\..." - so it rejected the line, printed usage, and
exited non-zero.

This file already carried the rule, next to the Python target: "Never put a
space in a path this installer controls." I broke it setting the 8.4 path.

Two fixes, because one of them alone leaves the trap in place:

- $MysqlRoot is now C:\MySQL84, space-free like C:\Python314. The MySQL client
  search paths in the installer, the preflight and the operator console all look
  there first, keeping backups working against the bundled server.
- Invoke-Native now quotes PROPERTY=value correctly, so passing a spaced path
  explicitly no longer produces an unparseable command line.

tests/test_installer_defaults.py fails if an installer-controlled path default
ever contains a space again.
2026-08-04 12:06:36 -04:00
cproudlock
5321649e02 fix(installer): stop blocking the wizard on things the installer itself installs
The preflight page began refusing to continue while any check was failing, which
is right for something the operator must go and fix. HttpPlatformHandler was
marked FAIL when absent - so on a server without it the wizard stopped dead,
telling the operator the server was not ready, over a module the bundle carries
and stage 4 installs a few pages later. The only way forward was to go and
install by hand the exact thing the installer was about to install.

It is now INFO: reported, not blocking, matching how URL Rewrite is already
handled. Nothing the installer SUPPLIES may block the wizard, and
tests/test_installer_defaults.py now fails if that rule is broken again.

The site-port conflict check is downgraded from FAIL to WARN for the same class
of reason: it runs before the operator reaches the Address page, so it tests the
DEFAULT port rather than the one they intend to use, and blocking refuses an
install over a conflict the very next page lets them resolve.

Genuine blockers are unchanged - no IIS, no WebAdministration, wrong Windows
edition or architecture, no disk, and the MySQL 5.6 index flags. Those the
operator really does have to fix first.
2026-08-04 10:00:44 -04:00
cproudlock
4a8bd138a9 feat(import): load a site's data from spreadsheets
Some checks failed
CI / backend (push) Failing after 7s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 6s
Adopting a site means getting its asset register in. The HTTP import API suits a
site with a source system and someone to script against it; a sister site with a
spreadsheet and no developer needs something else, and that is the common case.

FOREIGN KEYS TAKE NAMES. This is the whole design. A CSV row has to say where an
asset is, and the database stores locationid, an integer. Requiring the number
means importing locations, reading back the generated ids and pasting them into
the asset sheet - a workflow nobody finishes. Every foreign key here accepts
either a numeric id or the referenced row's name:

    assetnumber,assettypeid,statusid,locationid
    CMM-01,Measuring Tool,Active,Gage Lab

The column keeps its database name, per CONTRIBUTING.md; the value is whatever
the operator actually knows. Names resolve across files in one run, so
assets.csv can reference a location that only exists because locations.csv was
read moments earlier. A name that does not resolve is reported with its line,
column and value, not as a foreign key violation from three layers down.

Dry run is the default, and writes go into the transaction either way - the
rollback is what makes it a dry run. Skipping the writes instead made every
cross-file reference fail, which is the one thing a folder-wide check exists to
verify. Validation covers every row before anything is written, so a typo on
line 400 cannot leave 399 rows imported. Files are matched on a natural key, so
correcting a spreadsheet and re-running updates rather than duplicates.

TEMPLATES ARE GENERATED, NOT MAINTAINED. "flask csv templates" builds them from
the live schema, annotated with required/optional and which file each foreign
key refers to. The prompt for this was a hand-written template set that had
invented columns on seven of eleven tables and named a table that does not
exist, while looking entirely plausible - and described an import mechanism
(a Data Import page, a flask import-csv command) that had never existed. A test
fails the build if a generated template ever offers a column the schema lacks.

User accounts are deliberately not importable: passwords do not belong in a
spreadsheet in either direction.

Verified end to end against MySQL 5.6 - a folder dry run catching one bad
reference, the fix, the commit, and a re-run reporting updates rather than
inserts. 16 tests.
2026-08-04 09:13:03 -04:00
cproudlock
f72813ed9c feat(installer): bundle the database - MySQL 8.4 LTS, not 8.0
The bundled-database option could not actually be built. Stage 0 looks for
mysql\mysql-8.0.x-winx64.msi, and Oracle no longer publishes a standalone server
MSI for 8.0 - every 8.0.x returns 404. What remains for 8.0 is the MySQL
Installer bundle, which is an installer-manager: 'msiexec /i INSTALLDIR=' would
install THAT rather than a database, and stage 0 would then fail on a missing
mysqld.exe.

MySQL 8.0 also reached end of life in April 2026, so bundling it would have put
an unsupported database on every new site.

8.4 LTS still ships the standalone MSI (129MB, which is what the '125MB' note in
stage 0 was written against) and is supported into 2032. Defaults follow it:
install root MySQL Server 8.4, service MySQL84. The operator console still looks
for an 8.0 install path as a fallback, for sites already running one.

Also bundles mysqlclient\ - mysql.exe and mysqldump.exe with the two OpenSSL
DLLs they actually import, 20MB rather than the 51MB of debug and auth-plugin
libraries the archive ships. Stage 2 stages it onto the server, so a site whose
database is on ANOTHER host can still take the pre-upgrade backup that every
upgrade depends on. That was the gap the preflight had started warning about.

Bundle is now 221MB.
2026-08-04 07:56:39 -04:00
cproudlock
8d9d1d3439 test(docs): skip the publishability gate where there is no docs/ to check
Some checks failed
CI / backend (push) Failing after 8s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 6s
The GitHub backend job failed on test_there_are_docs_to_check, correctly. docs/
is stripped from the published repository - it lives in the wiki on that side -
so on the mirror the glob matched nothing and the guard fired exactly as
designed.

An absent docs/ and a glob that silently matches nothing in a tree that HAS docs
are different conditions, and the test conflated them. The module now skips when
the directory is not there at all, and the guard still fails when it is there and
empty. Verified all three ways: 7 pass here, 7 skip in a docs-less checkout, and
the guard still fails against a docs/ containing no markdown.

The gate has to ship rather than be excluded from publication, because the
published tree is where the GitHub CI that would catch a regression runs.
2026-08-04 07:28:47 -04:00
cproudlock
2073d0dbe8 build(export): purge stale generated paths from the publication tree
Some checks failed
CI / backend (push) Failing after 8s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 7s
rsync --exclude also PROTECTS a path from --delete, so anything that reached the
publication tree before its exclude existed stayed there permanently - invisible
to the sync and surfacing only as a scrub-gate failure. That cost two rounds of
'add the exclude, still fails' on the installer bundle and again on
.pytest_cache. The generated paths are now purged before the sync, so adding an
exclude is sufficient on its own.

tests/test_docs_publishable.py assembles its search terms from fragments: a file
containing the literal strings the scrub greps for tripped that scrub on itself.
Excluding the file from publication would have removed the check from the
repository it protects.
2026-08-03 15:17:09 -04:00
cproudlock
ee083ea80e docs: stop publishing internal references to a public wiki
docs/ is excluded from the code bundle and its scrub gate, because it goes to
the GitHub wiki instead - via a generator that has no gate at all. So the one
part of the repository written in prose, by people, about internal
infrastructure, was the one part nothing checked.

What was reaching a public wiki: the internal git server's URL and hostname,
.gitea workflow paths, developer home directories in the GE-Enforce cutover
reference, and a dev database root password inside a copy-pasteable command in
the import guide.

All replaced with neutral equivalents. tests/test_docs_publishable.py is now the
gate, at the source, in CI - a wiki page cannot be un-published, so catching this
after the fact is not good enough.

PROJECT-REVIEW.md also referred to internal tooling by name throughout; those
references are generalised. It remains an internal candid assessment of this
project that is nonetheless published, which is worth a separate decision.
2026-08-03 15:12:25 -04:00
cproudlock
bec138f5ac build(export): keep the installer's staged bundle out of publication
The sync walks the working tree rather than git, so deploy/windows/installer/
bundle came through despite being gitignored - about 100MB of build output
containing a copy of the whole application tree, the wheels and the vendor
installers. Its copies of config.py and requirements.txt then tripped the scrub
gate, which is the only reason it was noticed.

Excluded along with the installer's other generated files. Note that rsync
--exclude also protects a path from --delete, so a copy already in the
publication tree has to be removed by hand once.
2026-08-03 15:09:13 -04:00
cproudlock
e158cb21f9 deps: keep build-machine paths out of the lockfiles
Some checks failed
CI / backend (push) Failing after 7s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 7s
The '# via' annotations recorded the absolute path of the temporary file the
lockfile was compiled from, which is meaningless to anyone else and does not
belong in a published artifact. They now read 'requirements.in', which is where
these requirements actually come from.

Pins and hashes are unchanged - verified by a hash-checked dry-run install.
2026-08-03 15:06:02 -04:00
cproudlock
2c415a1712 fix(installer): correct a false security claim, and clear the should-fix list
Some checks failed
CI / backend (push) Failing after 8s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 7s
CLIENT IP / SPOOFABILITY. docs/geenforce-api-cutover.md claimed that removing the
IIS rewrite rule made the allowlist fail closed and that it does NOT become
spoofable. The opposite is true. IIS never sets X-Forwarded-For on its own; the
rule is the only thing that does. Remove it and IIS still forwards whatever
X-Forwarded-For the CALLER sent, waitress trusts it because it arrives from
127.0.0.1, and remote_addr becomes attacker-controlled - so a token-less caller
can fetch manifests from anywhere on the network. The document and the
_trusted_client_ip docstring now say so, waitress runs with
--trusted-proxy-count=1, and stage 5 checks the rule is actually live rather than
assuming it. The wizard question is rephrased to something an operator can verify
with their network team instead of guessing at.

NON-ASCII. The style gate only ever checked .py/.vue/.js/.ts, so documentation
accumulated em-dashes, arrows and box-drawing characters against this repo's own
convention - including in files added this week. Cleaned, and the gate now uses
INCLUDES_ALL so Markdown, JSON and YAML are covered.

PLUGIN DEFAULTS. The wizard pre-ticked measuringtools and printedparts, both of
which ship default_enabled=false, so every site taking the defaults installed and
enabled them against their manifests. Inno has no JSON parser so the list must be
hardcoded, but tests/test_installer_defaults.py now fails when it drifts.

UPGRADES. The payload copy merges, so a plugin dropped from a site's profile kept
its code forever - which defeats a lean build and leaves core's optional-import
guards succeeding for a plugin the site no longer has. Stale plugin directories
are now deregistered and removed before the copy.

add-plugin used 'plugin install', which for the five default_enabled=false
plugins left them installed but DISABLED - and printed a green success line
anyway. It now goes through apply-profile, and the success line is gated on the
exit code. Invoke-Flask records its own exit status, because $LASTEXITCODE keeps
a stale value when flask.exe is missing and no native command runs.

CHARSET. The utf8mb4 compiler hook lived inline in migrations/env.py, so it
covered the CORE chain only: plugin baselines inherited the server default, which
on a latin1 server means two charsets in one database. It is now
shopdb/utils/mysql_charset.py, imported by both, and preflight reports the
database's default charset.

BACKUP HONESTY. The dump was described as 'all of your asset data'. Uploaded
branding and floor-map images live in instance\ on disk, not in the database, so
a restore from the .sql alone comes back with no map. backup now archives
instance\ alongside it and says both are needed.

VERSIONING. AppVersion was hardcoded at 0.9.0 while the product, the frontend and
the newest tag said 0.7.0 - and 0.9.0 collides with a retired contract version.
Both builders now generate version.iss from shopdb/__init__.py.

Smaller: rollback overwrites .env before deleting it, as uninstall already did;
appcmd unlocks are scoped to this site's location rather than server-wide, with
the wide unlock as a fallback; DEVELOPMENT-SETUP says Python 3.14; the README
plugin list gains printedparts; prune-schema --force is documented as
first-provisioning-only; HTTPS is documented as not-the-default with the steps to
add it; the DBA SQL is on the wizard's database page; the features page says
unticking does not remove an installed feature; and the installer README states
that bundle-lock cannot vouch for the exe itself - that needs signing or an
out-of-band hash, neither of which is wired up.
2026-08-03 14:57:38 -04:00
cproudlock
aea2905de0 fix(installer): stop it lying, stop it leaking, and make it findable
Some checks failed
CI / backend (push) Failing after 7s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 7s
Nine fixes from a review of the installer against its actual audience: DT leads
at sister sites who are not Windows, IIS or Python specialists and who will lean
on an AI assistant to get through it.

TRUTHFULNESS. The preflight was advisory - an operator read 'IIS is not
installed', pressed Next, answered five more pages and the install died partway
through with Python already on the box. The results page now blocks while
anything is failing, repaints on every run instead of latching after the first,
and offers 'Check again' so a fixed problem does not mean starting over. On
failure the wizard said 'Nothing was left running', which is false in every path
because the stages run with -OnFailure never: it now says the server is
part-configured, that re-running is safe, and how to remove it. The final page no
longer reads 'ShopDB-Flask is ready' after a failed install.

SECRETS. The generated MySQL root password went to Write-Host in a process the
wizard runs hidden - so nobody saw it - and stdout is forwarded into the setup
log operators are told to send to support, so it was permanently recorded for
everyone who did not need it. It now goes to an ACL'd file. Database dumps, which
contain every user password hash, landed in a ProgramData directory readable by
every user on the box; the directory is now locked at creation.

UPGRADES ON REMOTE-DATABASE SITES. mysqldump was looked for only under local
MySQL install paths, so a site whose database is on another host silently skipped
every pre-upgrade backup - after stage 2 had already stopped the pool and
replaced the tree. Find-MysqlTool now prefers a client shipped in the bundle,
stage 2 stages it onto the server, preflight reports when it is missing, and
mysqlclient\ is an optional locked payload.

UNINSTALL. A subpath install is an IIS Application, not a site; removing only the
site left the application pointing at a deleted directory, so the parent site -
at West Jefferson, the live classic ASP - served 503 on that path forever while
Add/Remove Programs reported success. Uninstall now reads MOUNT_PATH and removes
the application. The firewall rule was created as "$SiteName $SitePort" and
removed as the literal 'ShopDB-Flask 8090', which matches nothing.

DAY-2 TOOLING. Every shortcut now passes -AppRoot and -SitePort, and the console
forwards them through its own elevation and 32-bit relaunches instead of
discarding them - a non-default directory or port made it report a healthy site
as broken, from a shortcut the installer wrote. 'Open ShopDB-Flask' resolved to a
hardcoded localhost:8090 that was wrong for every subpath install; it now asks
the console, which reads the address the installer recorded, and no longer
demands administrator to open a browser.

SMOKE TEST. The parent-site port lookup filtered for an http binding and
defaulted to 80, so an https-only parent site failed a working install with a red
dialog.

DOCS AND /api/docs. The installer was invisible: nothing in docs/, README.md or
CLAUDE.md mentioned it, so a DT lead or their assistant landed on the manual IIS
runbook and hand-built the very server the installer then refuses to upgrade.
docs/INSTALL-WINDOWS.md and docs/OPERATE-WINDOWS.md are now the canonical route,
the two manual runbooks are bannered as reference-only, README and CLAUDE.md
route by target, and llms.txt tells an assistant which document to follow and to
ask for 'check -Json' before diagnosing. Both ship on the server, along with
openapi.json and llms.txt - without those the self-hosted /api/docs was broken on
every installed box, which matters most to the sites least able to debug it.
Stage 5 now checks it actually serves.

shopdb-admin.ps1 gains 'check -Json': one structured, secret-free block covering
version, publishing method, IIS state, HTTP reachability, database, Python
version, plugins and errors. That is the cheapest useful answer to 'the operator
will ask an LLM' - it works with no infrastructure, which a install-time MCP
server could not.
2026-08-03 14:39:38 -04:00
cproudlock
e58f376643 fix(installer): the adopt-existing guard never fired
Stage 4 decided whether the IIS objects it was about to reconcile were its own by
testing for .installed-version. Stage 2 writes that file, and stage 2 always runs
first in a '-Stage all' install - so by the time the guard looked, the stamp it
had just written made every server look like one this installer built, including
the hand-built ones the guard exists to protect.

Stage 2 now records whether a stamp was present BEFORE it writes its own, and
stage 4 reads that observation. Running stage 4 alone still tests the file, which
is correct there: no stage 2 has run to disturb it.

Found by review, not by test - the guard has no coverage, because exercising it
needs a live IIS.
2026-08-03 13:47:07 -04:00
cproudlock
3606d8d696 feat(sbom): ship a CycloneDX bill of materials with every build
Some checks failed
CI / backend (push) Failing after 7s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 7s
An air-gapped site cannot be scanned from anywhere else, so when a CVE lands the
only way to answer 'is that component here, and at what version' was to RDP in
and go looking. The frontend was the real blind spot: nothing recorded which
version of leaflet, dompurify, jspdf or html2canvas ends up inside the compiled
SPA.

scripts/generate_sbom.py emits CycloneDX 1.6 covering both ecosystems - every pin
in requirements.txt with the sha256 the installer enforces, and every package in
package-lock.json. Build-only npm packages are marked scope 'excluded' rather
than dropped, so 'not here' stays distinguishable from 'not looked for'.
Dependency edges are real: uv's '# via' comments give the Python graph and
package-lock gives the npm one.

Hand-rolled rather than cyclonedx-py plus cyclonedx-npm because both inputs are
already pinned and committed - this is a format translation, not a scan - and
because the build box may be a work PC with nothing but Python and Node. It is
deterministic by construction: same inputs, byte-identical output, so
regenerating does not churn.

Staged into the application tree by both builders, so it installs onto the
server with the app. shopdb-admin.ps1 verify reports it and searches it by
component name, which is the question actually being asked.

Packages appearing at several depths in package-lock (node_modules/vite and
node_modules/vitest/node_modules/vite) are merged, and a copy reachable outside
the dev tree makes the component count as shipped. Emitting both produced
duplicate bom-refs, which CycloneDX forbids and scanners reject; getting the dev
merge backwards would have hidden a shipped package from a CVE search.

Not covered by bundle-lock.json on purpose: its provenance is git, not the
third-party payload.
2026-08-03 13:15:27 -04:00
cproudlock
1bf3cb2e1c feat(installer): refuse to damage an installation it did not create
Some checks failed
CI / backend (push) Failing after 8s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 7s
Two guards for a server deployed by hand, which the West Jefferson production
box is.

An existing venv is reused, which is right for a repair or an upgrade of an
install this made, and wrong when the venv belongs to a different Python. The
wheelhouse is tagged for one minor version, so pip finds no candidate for the
compiled packages and dies partway through - after Python has been installed and
the application tree replaced. The two versions are now compared up front and
the run stops with both numbers and what to do about it.

Switching deployment method removes the other method's IIS artifact. That is
correct when this installer owns both and dangerous when it does not: a wrong
-MountAlias would call Remove-WebApplication on a live mount with no prompt and
no error, and the first sign would be the site returning 404. It now refuses
unless a version stamp shows this installer made the install, or -AdoptExisting
is passed, and the refusal lists exactly what it would have removed.
2026-08-03 11:46:48 -04:00
cproudlock
13d831eb90 feat(installer): require the wheelhouse to satisfy requirements.txt, and lock the real payload
The lock records what IS in the wheelhouse, not what the application NEEDS, so
an incomplete wheelhouse was locked, blessed and shipped - and only failed on an
air-gapped server.

That is not hypothetical. Assembling the wheelhouse anywhere other than Windows
silently omits colorama, a win32-only dependency of click, because pip evaluates
environment markers against the machine doing the downloading rather than the
machine being targeted. The bundle built here was short exactly that one wheel.

Both verifiers now cross-check wheels/ against the staged requirements.txt,
ignoring markers, since a requirement guarded by sys_platform == 'win32' is
precisely the one that must be present. Names are normalised to PEP 427 wheel
form, so mysql-connector-python matches mysql_connector_python.

bundle-lock.json is the first real lock: 42 files, cp314/win_amd64 - 39 wheels,
Python 3.14.6, HttpPlatformHandler 1.2 and URL Rewrite. MySQL is absent and
optional; a site choosing the bundled-database option adds it and re-locks.

The naming gate now skips the installer's build output. It contains a staged
copy of the application plus a second SPA build under dist-subpath, which
--exclude-dir=dist does not match, so a staged bundle failed the gate on
vendored minified JS nobody in this repository wrote.
2026-08-03 11:46:39 -04:00
cproudlock
44237b5cbd feat(installer): bundle URL Rewrite, ask where client IPs come from, verify installs
IIS does not set X-Forwarded-For on its own and HttpPlatformHandler connects
from loopback, so without a rewrite rule every client reads as 127.0.0.1. The
GE-Enforce IP allowlist, the dashboard visitor-location lookup and per-host
login rate limiting all stop working, silently. The rule needed URL Rewrite,
which the installer told operators to download - from an air-gapped server.

URL Rewrite now ships in the bundle, and the wizard asks which case applies,
because the two answers are mutually exclusive. Directly exposed: install it and
set X-Forwarded-For from REMOTE_ADDR, which is what stops a client spoofing its
own. Behind a proxy: leave the rule off, since REMOTE_ADDR is the proxy and
applying it would discard the real client IP.

The rule is enabled by deleting two explicit marker lines rather than by a regex
over the surrounding comment, so editing that prose cannot silently disable it.

An existing web.config is no longer overwritten. It is the one file on a server
that legitimately carries hand-edits, and replacing it reverted them without a
word - on a server where the X-Forwarded-For rule had been enabled by hand, that
alone would have turned the GE-Enforce IP allowlist off. The installer reports
what it found instead.

pip now runs with --require-hashes and --only-binary=:all:. Hash-checking is
requested explicitly rather than inferred from the lockfile, so shipping an
unhashed requirements.txt fails loudly instead of quietly dropping the check.

shopdb-admin.ps1 gains a verify command: which bundle this server was installed
from, and whether the installed packages still match what shipped.

The .iss states its compiler floor. WizardStyle uses the built-in windows11
custom style, which needs Inno Setup 6.6.0; older compilers now fail with that
sentence rather than 'WizardStyle is invalid'.
2026-08-03 11:17:58 -04:00
cproudlock
88af7fd9ce feat(installer): lock the third-party payload, and build on Windows without Bash
The bundle carries ~40 wheels, a Python installer and two MSIs. All of them run
as SYSTEM on the target server, and nothing verified any of them. A missing
wheelhouse printed MISSING and the script still exited 0, so an empty bundle
compiled into a shippable installer and the failure surfaced on an air-gapped
server with no way to fix it.

bundle-lock.json now records that payload exactly - sha256 and byte size per
file - and verification is set equality: a missing file, an unexpected extra
file, or changed content all fail. Both builders check it and refuse to produce
an unverified bundle; the lock ships inside the bundle and shopdb-install.ps1
re-checks it on the server before running any of it.

This is deliberately a layer above requirements.txt hashes. pip lists every
artifact of a pinned version (cffi 2.1.0 alone has 100 hashes), so it proves a
wheel is genuine, not that it is the wheel this bundle was built and tested
with; it ignores extra files in the wheelhouse; and it covers none of the
executables.

refresh-bundle-lock.ps1 regenerates the lock but refuses to overwrite one until
the operator has seen the diff, because the commit is the review - it is the
only place a change to what runs as SYSTEM becomes visible to a human.

build-installer.ps1 is the whole build natively on Windows, so a work PC needs
no Bash. It shares the plugin closure resolver with build-site.sh.

Both builders now copy the installer scripts from the repository. They were
copied from a downloads folder, so the logic that shipped was not the logic that
was committed and the build worked on exactly one machine.

Two verifiers exist because PowerShell is the only thing guaranteed present on
the target server, while the Linux builder should not need pwsh.
tests/test_bundle_lock.py runs both against the same fixtures and fails if they
disagree.
2026-08-03 11:17:45 -04:00
cproudlock
75f0a57821 build(site): share the plugin closure resolver, stage only web.config
Two fixes to the lean-site build.

The closure resolution moves out of an inline heredoc into
scripts/resolve_plugin_closure.py. The Windows builder needs the same answer,
and a PowerShell reimplementation would have been a second copy of the rules,
free to drift and produce a bundle whose plugin set did not match its profile.

The backend staging step copied all of deploy/ into the output tree. The Windows
installer stages its bundle at deploy/windows/installer/bundle, so that copy
recursed into its own destination and cp aborted with 'cannot copy a directory
into itself' - the documented build could not complete. Only
deploy/windows/web.config is read at install time, so only that is staged; the
rest of deploy/ is installer source and does not belong on an application
server.
2026-08-03 11:17:28 -04:00
cproudlock
9c2c21c2cc fix(employees): resolve User through the contract surface
test_plugins_only_import_contract_surface has been failing on main since
9a2d0cc: the employee name resolver imported shopdb.core.models directly.
shopdb.api already exports User (contract 0.13.0), so this is the same object
reached the way ADR-001 requires.
2026-08-03 11:17:28 -04:00
cproudlock
6ebc79a2de deps: hash-pin the lockfiles and stop dev and prod drifting apart
Both files are recompiled with --universal --generate-hashes, preserving every
pinned version. Three things change.

Hashes put pip into hash-checking mode, so a wheel whose sha256 is not listed is
refused rather than installed. The offline Windows install previously took
whatever file in the wheelhouse satisfied the version pin.

--universal means one lockfile serves Linux (dev, Docker, CI) and the Windows
wheelhouse. The Linux-only resolve had silently omitted colorama, a win32-only
dependency of click; in hash-checking mode a missing entry is a hard error, so
that omission would have broken every Windows install.

requirements-dev.txt is now compiled with -c requirements.txt, pinning shared
dependencies to the versions production runs. The two had been compiled at
different times and drifted: CI tested against alembic 1.18.5 while sites
installed 1.18.4.

Hashes pin the version and prove the artifact is one upstream published. They do
not pin WHICH artifact of that version is used, and they say nothing about extra
files in the wheelhouse - bundle-lock.json covers both.
2026-08-03 11:17:18 -04:00
cproudlock
5c4fdcb15e Merge branch 'installer-prereqs' into feat/installer-bundle-lock 2026-08-03 10:06:56 -04:00
4d5b6c3c55 build(site): also stage a subpath frontend build
Some checks failed
CI / backend (push) Failing after 2m1s
CI / naming (push) Failing after 1s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 8s
Vite compiles the mount path into the bundle, so it cannot be chosen at install
time from a single build - a page served under /shopdb would load and then
request its assets from /assets/, and render nothing.

build-site.sh now produces both:
  frontend-dist           base /        - the app on its own IIS site
  frontend-dist-subpath   base /<alias> - an IIS Application under an existing
                          site, e.g. http://<server-fqdn>/shopdb/

SUBPATH_ALIAS (default 'shopdb') is fixed per bundle and written into the staged
build as .alias, so the three places that must agree - the IIS application alias,
MOUNT_PATH in .env, and this compiled base - cannot drift apart. The installer
checks that marker and refuses rather than serving a page that cannot load.

The subpath build runs FIRST and is held in a temp dir: the root build has to be
last so frontend/dist is left in the state a developer expects, and the copy into
$OUT has to happen after the staging step that does rm -rf "$OUT".
2026-08-03 01:55:21 -04:00
0fa5f1e910 feat(deploy): add the air-gapped Windows installer
Roughly 2500 lines of tested installer had been living in ~/Downloads and an
untracked folder - nothing was under version control.

It goes here rather than in a repo of its own because it depends on application
internals: the `flask plugin` verbs, site-profile.json, MOUNT_PATH, and the
plugin registry. Versioned separately it would drift out of step with the thing
it installs.

Contents: the read-only preflight, the staged installer (bundled MySQL, runtime,
schema, IIS, verify, uninstall), the operator console, the Inno Setup wizard, the
bundle builder and the artwork generator.

bundle/ and Output/ are ignored - regenerable, and ~220MB. plugins.iss is ignored
because build-installer.sh generates it from the staged payload. The artwork IS
committed so a Windows build box does not need Python and cairosvg.

Verified end to end on Windows Server 2025 against a bundled MySQL 8.0 and an
existing MySQL 5.6: fresh install, upgrade with backup and rollback, re-run
idempotency, uninstall, and both deployment methods including switching between
them. Not yet verified: a hypervisor-level air-gapped run, and any load from a
real browser (every HTTP check so far used curl, which sends no Origin header).
2026-08-03 01:47:34 -04:00
0f766cf977 fix(setup): stop the wizard step promising data it does not create
The step was called "Starter Data" and offered to "seed the data a new site
needs", but seed_starter inserts eight vendor rows and nothing else: no assets,
locations, departments or statuses. An operator ran it, saw every dashboard
count stay at zero, and reasonably concluded the seed was broken.

Rename the step to Reference Data, describe what is actually seeded, and say
outright that no assets are created and that an empty dashboard is expected
here. Assets arrive later via the import API.
2026-08-02 18:56:38 -04:00
888b15a7dd build(site): stage a deployable tree and the profile in build-site.sh
build-site.sh staged only shopdb/, the chosen plugins/ and frontend-dist, so the
output could be imported but not run or migrated. The Windows installer had to
assemble wsgi.py, requirements.txt, migrations/ and deploy/ separately, which
meant it could assemble a payload whose plugin set did not match the profile the
tree was staged from.

Stage those runtime files, and copy the profile in as site-profile.json so the
set is self-describing: `flask plugin apply-profile` at provisioning reads the
same profile the tree was built from, so installed plugins and shipped plugin
code cannot drift.

frontend-dist keeps its name; CI reads that path (ci.yml:79).

The closing hint now spells out `prune-schema --yes --force`. ADR-014's prose
says lean provisioning "uses --force", but --force alone only permits dropping
non-empty tables; without --yes the command is a dry run that prints a preview
and exits, so following the ADR literally silently skips the prune.
2026-08-02 16:08:42 -04:00
94d6878a03 feat(frontend): gate first run on needs-admin so a fresh instance shows the wizard
A fresh install landed on the anonymous dashboard instead of prompting to create
the first admin, so an operator had no way to discover /setup.

The router now asks /api/setup/needs-admin before rendering any unauthenticated
route and redirects to /login?firstrun=1 while no user exists. The result is
cached in a composable so it costs one request per session, and the lookup fails
open (a backend that cannot answer must not lock the login screen). Racing it
against a 4s timeout keeps a slow or hung backend from blocking the first paint.

Login.vue clears the flag after creating the admin so the gate stops firing
without a reload.
2026-08-02 16:08:42 -04:00
11f3d00a04 Installer prerequisites: REQ-D through REQ-G
Some checks failed
CI / backend (push) Failing after 7s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 7s
REQ-D: restore waitress and tzdata to requirements.in. They existed ONLY in the
generated requirements.txt (hand-added in bf9e60e), so the next
`uv pip compile` would have silently removed the WSGI server and the IANA
timezone database from every Windows install.

REQ-E: split production and development requirements. requirements.txt was
installing pytest, pytest-cov, pytest-flask, coverage, iniconfig and pluggy onto
production servers. Verified on a real Windows Server box before this change.
CI, scripts/test-external-plugin.sh and the dev docs now use requirements-dev.txt.

REQ-F: standardise on Python 3.14. The repo declared four different versions
(Dockerfile 3.12, DEPLOY-WINDOWS-IIS 3.12, INSTALL-WINDOWS-IIS 3.13, CI 3.13,
plus README, web.config and PLUGIN-EXTERNAL-REPO). 3.14 is in active bugfix
support until ~Apr 2027 and supported to Oct 2030; 3.13 entered security-only in
Apr 2026. All four compiled dependencies publish win_amd64 wheels for 3.14
(cryptography via an abi3 wheel), verified by building an offline wheelhouse and
installing it on Windows Server 2025.

REQ-G: state MySQL 8.0 as the standard for new installs; 5.7+/5.6 remain
supported on an existing server.

Lockfiles regenerated with uv pip compile. Production deps 44 -> 38.
2026-08-02 14:15:18 -04:00
cproudlock
6639afd1f0 map: drop the marker popup that was never meant to be reached
Some checks failed
CI / backend (push) Failing after 1m55s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 7s
Each marker bound both a hover tooltip and a click popup. On the map page the
click handler routes to the detail page, so the popup opened and the
navigation discarded it in the same tick - it was never visible. Where it did
render (the map editor and the picker forms) its 'View Details' link only
served to pull the user off an unsaved form.

Keep hover as a glance and leave the click to the consumer. Removes the popup
markup, its styles, and the two now-unused detail-route helpers.
2026-07-31 10:17:17 -04:00
cproudlock
2528b7e556 map: thin the marker and legend rings
The ring only has to separate the mark from the surface; at 2px it read as
part of the mark. Markers go to 1.25, legend dots to 1.5px (they are larger,
so the same visual weight needs slightly more), and the PDF marker and swatch
strokes drop to match.
2026-07-31 10:12:30 -04:00
cproudlock
a7fe2c8353 fix: navigation dying after an app-pool restart
Two independent ways a restart leaves the SPA unable to navigate, both of
which look identical to a user - a click that does nothing.

1. The router awaits loadEnabledPlugins() to gate plugin routes. An app-pool
   restart leaves that request hanging (IIS queues it while the worker starts)
   and axios sets no timeout, so the navigation never resolves. Worse, the
   promise is cached, so every later navigation awaited the same dead request
   and stayed frozen long after the backend recovered. Bound the wait and fail
   open on expiry, and drop the cached promise when an attempt times out or
   fails so the next navigation retries. The setup-state probe in the guard
   gets the same bound (it already fails open, defaulting to "complete").

2. A deploy replaces the content-hashed chunk files, so a tab open across it
   asks for chunks that no longer exist and the dynamic import rejects with
   nothing handling it. Reload once on a chunk-load error, via router.onError
   and Vite's preloadError, guarded by a sessionStorage flag against a reload
   loop and cleared on the next successful navigation.

Also stop index.html being cached: it names the hashed chunks, so a stale copy
points at files the deploy already deleted. It now revalidates while the
hashed assets under assets/ cache for a year.
2026-07-31 10:10:51 -04:00
cproudlock
cbf90be7ec map: ring markers by surface so a theme rings every marker alike
Some checks failed
CI / backend (push) Has been cancelled
CI / naming (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / migrations-mysql (push) Has been cancelled
Keying the ring off the fill singled out the light colors: on the map the
orange network-device marker took a black ring while its neighbours kept white
ones, and the legend dot for the same type wore a surface-colored border, so
the key did not match the markers.

Ring by the SURFACE instead - dark on the white blueprint, light on the dark
one - which is uniform within a theme and still works for every fill, since a
fill that resembles the ring is by definition far from the background. Legend
swatches take the same ring, the theme watcher redraws the markers (their ring
now depends on it), and the PDF uses the light-surface ring throughout because
it prints on white.
2026-07-31 09:28:38 -04:00
cproudlock
c80c612922 map: markers legible on both the white and the dark blueprint
Some checks failed
CI / backend (push) Has been cancelled
CI / naming (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / migrations-mysql (push) Has been cancelled
Markers sit on two very different surfaces - the light blueprint on white and
the dark blueprint over the near-black navy card - but the palette only ever
suited one. The grey "no subtype" default sat at 1.88:1 against white and the
orange asset-type step at 2.16:1, so both effectively disappeared on the light
blueprint. The fixed white ring made it worse: on white it added nothing.

Re-step the asset-type colors to versions of the SAME hues that clear 3:1
against both surfaces, replace the grey default with a neutral that clears
5.4:1 / 3.7:1, and derive the ring from the fill's luminance (light fill ->
dark ring, dark fill -> light ring) so every marker keeps a hard edge on
either background. The ring also rescues a washed-out color a user picks by
hand for a subtype, which no palette change can reach. The PDF export applies
the same rule to its markers and legend swatches, keeping print in parity.

Colors were chosen against a contrast/CVD validator rather than by eye. Note
that five simultaneous hues cannot all stay distinguishable under color-blind
simulation - past roughly five subtypes on screen, the legend and the hover
tooltip carry identity.

Adds computed contrast assertions so a future palette edit cannot
reintroduce a washed-out step.
2026-07-31 09:03:24 -04:00
cproudlock
d5635a4306 map: fix PDF export 404 on the blueprint under a subpath mount
exportPdf passed the raw map_blueprint_light setting value, which is a
root-relative /api path. Under /ops or /shopdb that resolves to the server
root and 404s, so the export died with "Failed to load blueprint image".
The on-screen map was unaffected because it goes through blueprintUrlFor,
which applies withBase - use that here too.

Same file, so this also carries the subtype auto-palette replacement that
goes with the marker-legibility change in the next commit.
2026-07-31 09:03:14 -04:00
cproudlock
b16f143467 search: asset lists search the type column they display
Every asset list shows a Type column (and printers a Model, machines and
network a Vendor), but the search filters only looked at the asset number,
name, serial and hostname. Searching a type returned zero rows: 'Part Washer'
on machines, 'Standard' on PCs, 'Thermal' on printers.

Extend the search on machines, computers, printers, network devices,
measuring tools and the unified asset list to cover the type name plus the
vendor/model where the list shows them. Joins are outer joins so an asset
missing a type or vendor still matches on its own fields; the core list uses
a correlated EXISTS instead, since its type-name filter already joins
AssetType.
2026-07-31 09:02:51 -04:00
cproudlock
71982fc0f1 map: fix subtype filter dropping every measuring tool
Some checks failed
CI / backend (push) Has been cancelled
CI / naming (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / migrations-mysql (push) Has been cancelled
MapView carried its own copy of the per-type subtype-id lookup and it
never gained a Measuring Tool branch, so selecting any measuring-tool
subtype filtered out all assets. Marker coloring and the PDF export were
unaffected because both already used the shared getSubtypeId helper.

Point the filter at that shared helper and delete the duplicate copy in
ShopFloorMap too, so one definition serves filter, coloring and export.
Adds a table-driven spec covering every subtype-carrying asset type.
2026-07-31 07:56:09 -04:00
cproudlock
86697a4e7b docs: remove WIKI-UPDATE-PLAN.md (executed) 2026-07-30 16:07:37 -04:00
cproudlock
802256f929 docs: wiki update for API docs, printer installer, geenforce cutover, timezone
Execute WIKI-UPDATE-PLAN.md (14 items):
- NEW docs/PRINTER-INSTALLER.md: install-list / pc-default / install-batch
  contract + public installer map page.
- NEW-shape docs/API-REFERENCE.md: index + pointer to the live generated docs
  (/api/docs Redoc, openapi.json, llms.txt, MCP), replacing a stale full dump.
- geenforce cutover + GE-ENFORCE-DISPLAY/CLIENT/DEPLOY: server-first display
  dispatcher (display-role by FQDN, display-type.txt fallback), dashboarddefaults
  FQDN keying, legacy kiosk autostart self-heal (Wow6432Node), per-PC-type
  cutover status.
- PLUGINS: printers/slides rows + plugin-permissions note (slides.manage).
- IMPORT-API: dashboarddefaults FQDN-first keying.
- CONFIG: word-wise search, site_timezone setting.
- PILOT-DEPLOY: servers-to-network reclassify step. IMPORT-ADOPTION: fixup note.
- CLAUDE.md: test count 1077->1159, HTTPS-cutover state. CHANGELOG: timezone +
  kiosk-autostart fixes, site_timezone setting.
2026-07-30 16:05:21 -04:00
cproudlock
af6bcd4726 geenforce display dispatcher: purge legacy autostart in Wow6432Node + all hives
The old kiosk kept relaunching the dead URL from an HKLM Run value the 32-bit
Inno installer wrote - WOW64-redirected into SOFTWARE\Wow6432Node, which 64-bit
tooling (and the earlier purge) never saw. Broaden the sweep to both registry
views, every loaded user hive, Run/RunOnce/Policies-Explorer-Run, matching by
legacy name AND by any value pointing at the old URLs, plus every per-user and
common Startup folder.
2026-07-30 15:54:18 -04:00
cproudlock
01a545f507 geenforce display dispatcher: revert to direct Edge shortcut, drop VBS launcher
The white-on-login was the old Dashboard/Lobby installer's leftover autostart
relaunching the dead old URL (404 -> white), not a network race - so the
wait-for-URL launcher solved the wrong problem. Go back to the plain direct
Edge kiosk shortcut and clean up any stale launcher file. The real fix (the
legacy HKLM Run-key + old .lnk purge) stays; it just has to be published.
2026-07-30 15:37:26 -04:00
cproudlock
95df51fddf geenforce display dispatcher: wait for kiosk URL before launching Edge
At auto-login the Startup shortcut fired before the network was up, so Edge
--kiosk navigated to nothing and sat on a blank white page with no retry.
Point the shortcut at a hidden VBS launcher (wscript, no console flash) that
polls the kiosk URL until it responds (up to ~3 min) and only then launches
Edge fullscreen, so the first paint is the real page. Falls through to launch
anyway after the timeout so a display is never left dark.
2026-07-30 15:33:49 -04:00
cproudlock
ea6fae91c3 notifications: correct timezone handling + configurable site timezone
Notification start/end times displayed and stored wrong by the tz offset
(a 2:34 PM entry showed 6:34 PM). Two stacked bugs: to_dict emitted stored
UTC as naive ISO (no offset) so the browser read it as local, and the form
filled the datetime-local input from toISOString() (UTC).

Fix and generalize to a configurable site timezone (multi-site):
- New setting site_timezone (default America/New_York), public, editable in
  Settings > Site > Localization (common-zone dropdown).
- Backend tags datetimes UTC (_utc_iso); parse normalizes to naive UTC
  (_parse_utc); daily-reset expiry uses the site zone (_next_site_time);
  calendar allDay events key off the site-local day (_site_date).
- Shared frontend util datetime.js (Intl-based, DST-safe) converts between a
  UTC instant and a site-zone wall clock. Notification form, list, and
  calendar all render/enter in the site zone.
2026-07-30 15:08:59 -04:00
cproudlock
3ad26ba010 export-github: exclude mcp/ from public publication
The MCP server README names Claude Desktop / Claude Code (it is an MCP
server for those clients), which trips the scrub gate. It is a standalone
local tool distributed via pxe-images/mcp/ + setup-mcp.cmd, not part of the
shipped product, so exclude it from the public repo like docs/ and tools/.
2026-07-30 14:30:12 -04:00
cproudlock
e6533d5205 geenforce display dispatcher: purge legacy HKLM Run autostart
The old LobbyDisplay/Dashboard Inno installers planted an HKLM
...\CurrentVersion\Run value (plus a Startup .lnk). The dispatcher already
swept the stale .lnk/.url launchers but never the Run value, so a display
with our new ShopDB Kiosk.lnk still relaunched the old kiosk URL at logon
(the Run key beats the Startup shortcut). Remove the two legacy Run values
and kill any running old-URL Edge so the display self-heals to the resolved
target on the next enforce cycle.
2026-07-30 14:28:44 -04:00
cproudlock
516e33a6c3 geenforce display dispatcher: kiosk .lnk uses only --kiosk + --edge-kiosk-type=fullscreen (drop the extra Edge flags) 2026-07-30 13:53:04 -04:00
cproudlock
5f02e488eb mcp: resolve openapi.json from env/in-repo/same-dir so a standalone copy works 2026-07-30 09:30:39 -04:00
cproudlock
f0b5465917 mcp: read-only ShopDB MCP server generated from the OpenAPI spec
A separate tool (not shipped in the app) that exposes a curated set of read
endpoints as MCP tools, so an LLM client can query the asset DB directly. Built
with FastMCP.from_openapi over docs/openapi.json; auth via a scoped PAT
(SHOPDB_TOKEN) or managed X-API-Key. Read-only: only GETs on the curated
allowlist become tools, all writes excluded. Runs anywhere that can reach the
API - never on the air-gapped box. Needs `pip install fastmcp` + testing in that
env (not installed in this repo's venv).
2026-07-30 07:52:53 -04:00
cproudlock
b507884ad6 api: serve interactive OpenAPI docs at /api/docs (offline) + llms.txt
Generate docs/openapi.json (3.1, 362 operations) from the API inventory via
scripts/gen_openapi.py, and serve it with a self-hosted Redoc bundle at
/api/docs - no CDN, works on the air-gapped box. Also serve docs/llms.txt (a
concise LLM entrypoint) at /api/docs/llms.txt. New core 'docs' blueprint;
staticdocs/ excluded from the naming check (vendored minified JS).
2026-07-30 07:51:12 -04:00
cproudlock
8575837d8e docs: add project health review, wiki update plan, API reference (Fable review) 2026-07-30 07:51:12 -04:00
cproudlock
ecf4ef6edd scripts: match servers by name prefix (SVR-) as well as computer type 2026-07-30 07:12:51 -04:00
cproudlock
346c428409 scripts: reclassify server 'computer' assets to network_device
Servers were imported as computers (a PC type) so they show under PCs, not
Network. This one-shot re-points each server's asset in place - assetid is
unchanged, so comms/relationships/map/name/location/audit all carry over; only
the extension row is swapped (computers -> networkdevices), the asset type is
flipped, and the device gets the 'Server' networkdevicetype (created if absent).

Identify servers by their computer type name (--type, default 'Server'). Dry-run
by default; --commit applies. Run on the target instance.
2026-07-30 07:10:16 -04:00
cproudlock
c075658ca6 printers: drop the trailing pause in the install .bat so it self-closes 2026-07-29 15:34:07 -04:00
cproudlock
cd79e610e9 printers: make the installer map public (no login)
The /printer-installer map only reads the public install-list and downloads
the install .bat - both jwt-optional endpoints - so requiring auth was an
unnecessary gate. Drop requiresAuth; it now matches the other display/kiosk
tools (public).
2026-07-29 15:16:16 -04:00
cproudlock
ad84c9060a printers: add format=text to install-list + pc-default; vendor via model
The Inno printer installers hand-parsed JSON in Pascal (brittle brace-counting).
Add ?format=text to install-list (one printer per line, pipe-delimited:
printerid|windowsname|vendorname|modelnumber|hostname|ipaddress|mapx|mapy) and
to pc-default (printerid|windowsname), so the installer side is a split() with
no JSON parser. The web map keeps the default JSON.

Also resolve install-list's vendorname via the model (as the batch already does),
since the import sets the model, not the printer's direct vendorid - otherwise
the installers' HP/Xerox/Brother filter drops every prod printer.
2026-07-29 13:51:01 -04:00
cproudlock
cb075a278f reports: pc-relationships matches PC<->machine links in either direction
Prod had 331 relationships, 268 computers, 204 machines, but the report came
back empty. The query only matched computer(source) -> machine(target), while
the import stores the general machinerelationships as machine(source) ->
PC(target) (only the synthetic measuring-tool links are PC -> tool). So the real
shop-floor edges never matched.

Make the query direction-agnostic (UNION of both orientations); a PC-runs-machine
report is conceptually undirected. Also drop the comtypeid=1 filter so the IP is
taken from the primary communication regardless of its type.

Test: a machine(source) -> PC(target) edge now appears in the report.
2026-07-29 13:16:15 -04:00
cproudlock
3eaaee0e50 printers: resolve installer vendor via the model + fix batch download base URL
Two fixes for the printer install-batch on prod data:

1. Vendor was read only from the printer's direct vendorid, which the legacy
   import never sets (it sets the model; legacy resolved vendor through the
   model). Every prod printer came back vendor "unknown", so all fell into the
   manual group and the universal PrinterInstaller.exe block never emitted. Now
   resolve vendor via the model's vendor when the printer has no direct one, as
   the classic installprinter.asp did.

2. Harden the download base URL. Behind IIS the app sees http on a loopback
   port and url_root drops the /shopdb mount, giving a broken download URL when
   site_base_url is unset. Fall back to https + the forwarded Host + script_root.

Test: a printer with no vendorid but an HP/Xerox model now groups universal.
2026-07-29 12:56:31 -04:00
cproudlock
0d40780f53 printers: printer installer map + install-batch endpoint
Rebuilds the classic printer-installer feature: pick printers on the shopfloor
map, download a .bat that installs them.

Backend (asset_routes.py): GET /api/printers/install-batch?printerids=1,2,3
returns a .bat attachment. Groups printers the way the classic installprinter.asp
did - HP/Xerox via the universal PrinterInstaller.exe /PRINTER="a,b,c", printers
with a .exe installpath via that installer /SILENT, and anything else (no
installpath, or a .zip) listed for manual install instead of being run blindly.
Download URLs derive from the site_base_url setting + the IIS-served /installers
folder (no hardcoded host). Reuses the existing install-list query shape.

Frontend: PrinterInstallerMap.vue - full-screen Leaflet shopfloor map (reuses
mapConfig), a marker per network printer at its mapx/mapy, click to toggle-select,
sidebar with the selection + an Install button that downloads the batch. Toplevel
route /printer-installer, printersApi.installList(), and an Installer Map button
on the printers list.

Tests: install-batch grouping (universal/specific/manual) + requires-ids.
2026-07-29 12:38:51 -04:00
cproudlock
1ee9328bf9 applications: fix relative installer/link hrefs + serve /installers via IIS
Some checks failed
CI / backend (push) Failing after 1m53s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 9s
Two problems with application download/launch/doc links:

1. Stored paths like 'installers/Foo.exe' are relative, so an <a href> on
   /shopdb/applications/6 resolved to /shopdb/applications/installers/Foo.exe.
   New basePath.fileHref() mounts a relative path under the app base
   (-> /shopdb/installers/Foo.exe) while leaving full URLs and UNC/file paths
   untouched. Applied to installpath, applicationlink, and documentationpath in
   the list and detail views.

2. Even the correct /shopdb/installers/Foo.exe 404s: httpPlatformHandler is
   path="*", so IIS forwards it to Flask, which has no such route. Add a
   web.config <location path="installers"> that clears the handler and serves
   that subpath as IIS static (with .exe/.msi MIME), from a physical
   APP_ROOT\installers folder.
2026-07-29 10:25:46 -04:00
cproudlock
071d40488b chore: sync package-lock for dompurify direct dependency
Some checks failed
CI / backend (push) Failing after 1m53s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
2026-07-29 10:10:35 -04:00
cproudlock
bf8842e1d7 applications: render Application Notes as sanitized HTML
Some checks failed
CI / backend (push) Failing after 1m55s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 8s
The notes field is authored as HTML (the form says "HTML supported") but the
detail page interpolated it with {{ }}, so tags like <BR> showed as literal
text. Render via v-html through a DOMPurify sanitizer (utils/sanitizeHtml):
allow-list of formatting tags + links only, forces target=_blank
rel=noopener on links, strips scripts/handlers. Promote dompurify to a direct
dependency (was transitive via jspdf).
2026-07-29 10:06:18 -04:00
cproudlock
ced356882c net: strip the ephemeral source port from the forwarded client IP
IIS ARR sets X-Forwarded-For to clientip:port, and the port changes every
connection. Left in, the audit log showed IP:PORT, the dashboard IP fallback
never matched a stored (portless) DashboardDefault.ipaddress, and login rate
limiting keyed per-connection instead of per-host. Add an IPv6-safe
clientip.client_ip / strip_port helper and use it in the audit log, the
dashboard resolver, and the login rate-limit key.
2026-07-29 10:06:18 -04:00
cproudlock
8dce622392 knowledgebase: include the topic (application name) in list search
The KB list search matched only shortdescription + keywords, so searching a
topic (e.g. "Spotfire", the Application name) surfaced just the one article
whose title/keywords contained the word, not the others tied to it by topic.
Match the topic too via an appid IN (apps named like the term) subquery - used
instead of a join so it does not collide with the sort=topic join, and articles
with no app still match on title/keywords.
2026-07-29 10:06:18 -04:00
cproudlock
c5ee164565 ui: format application knowledge-base card (was a wall of text)
Some checks failed
CI / backend (push) Failing after 1m53s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
The .kb-* classes had no styles, so the KB entries rendered as bare inline
spans - shortdescription and keywords (both up to 500 chars) wrapped and mashed
into one block. Style each entry as a bordered clickable card: description as
the link title clamped to 2 lines, keywords split on whitespace into small
muted chips below.
2026-07-29 09:26:58 -04:00
cproudlock
b63690996a fix: location dropdowns rendered blank (wrong field) + require printer model
Some checks failed
CI / backend (push) Failing after 1m52s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 8s
The location option label read l.location, but the Location.to_dict() field is
locationname, so every option rendered blank - the dropdown looked empty and
"massive" (a long list of blank rows). Fixed across all five affected forms:
printers, computers, network devices, network device form, and the subnets
location filter. Other .location uses (printer-driver URL, search-result label,
report bylocation key) are legitimately different fields, left alone.

Also require a model on the printer form: asterisk + required attr, plus a JS
guard in savePrinter (the native required is skipped while the select is
disabled with no vendor picked) that points the user at the vendor first.
2026-07-29 09:07:43 -04:00
cproudlock
b9d0cfac6a ui: stop audit-log columns clipping + widen ge-enforce report modal
Some checks failed
CI / backend (push) Failing after 1m53s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
AuditLogs: the scoped table-layout:fixed + width:100% forced the table to fit
the settings pane, so cells ellipsis-clipped (Timestamp/User/IP fell off) rather
than scrolling. Drop it so columns size to content and the container scrolls
horizontally (global .table-container is overflow-x:auto). Only the free-form
Name/ID cell stays bounded (320px + title tooltip) so one long value cannot blow
the table width out.

EnforcementReports: the per-entry detail modal capped at 640px, too narrow for
the 5-column table. Widen to min(1000px, 92vw) and let the Message column wrap
instead of forcing horizontal scroll inside the modal.
2026-07-29 08:55:47 -04:00
cproudlock
174c6c0b9a slides: gate management on slides.manage permission (grantable to non-admin curator)
Some checks failed
CI / backend (push) Failing after 1m54s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 10s
CI / migrations-mysql (push) Failing after 8s
The lobby-display and screensaver slide manager was admin-only. Add a shared
slides.manage permission so a curator can manage both surfaces without full
admin. Admins keep access via the require_permission admin bypass.

Backend:
- plugins/slides/api/routes.py: all 5 management routes require slides.manage
- plugins/slides/plugin.py: declare it via get_permissions(); nav item carries
  the permission so the frontend can gate visibility
- shopdb/core/api/auth.py: login response now returns the user's permissions
  (matches /me) so the frontend authStore has them on fresh login

Frontend:
- stores/auth.js: hasPermission(name) getter (admin true, else granted list)
- router/index.js: guard supports requiresPermission
- views/AppLayout.vue: hide nav items whose permission the user lacks
- plugins/slides/frontend/routes.js: slide manager gated requiresPermission

Tests: no-perm user 403, curator role with the perm 200 (+ login advertises
it), admin 200 via bypass.

Deploy: run `flask seed permissions` to create the row, then grant it to a
role in Settings > Users & Roles.
2026-07-29 08:41:11 -04:00
cproudlock
7a7c7f37d5 search: multi-word queries match by word, not the exact phrase
Global search did a single ilike('%CSF Roles%'), so any query with more than one
word required the exact contiguous phrase and usually returned nothing. Add
_word_match: split the query into words and AND them (OR across the searched
columns per word), so 'CSF Roles' matches a record with both words in any field,
any order. Applied across every domain (assets, applications, KB, employees
[selfhosted + external HR], notifications, hostnames, IP, custom fields,
vendor/model/type). External HR path uses a parameterized per-word LIKE.
2026-07-29 07:43:31 -04:00
cproudlock
b22701a444 geenforce display dispatcher: resolve role from the server by FQDN, fall back to display-type.txt
The dispatcher now derives its FQDN (F<BIOS serial>.<domain>) and asks
/api/dashboarddefaults/display-role for its role/path, so changing a display's
type/location in Settings > Dashboard Defaults takes effect with no reimage. If
there is no serial, no server mapping, or the lookup fails, it falls back to the
local display-type.txt map (offline-safe). VM-verified both paths.
2026-07-29 07:34:21 -04:00
cproudlock
e237cc2c05 dashboarddefaults: pick a kiosk from a dropdown (auto-fill FQDN) + Location wording
Adds GET /api/computers/display-kiosks - the displays that reported in (Kiosk
type), each with its derived FQDN (F<serial>.<domain>). The Dashboard Defaults
form gets a kiosk dropdown that fills the FQDN so admins pick a display instead
of typing an IP; IP stays an optional manual field. Table shows FQDN or IP.
'Business Unit' label -> 'Location' on this page + the settings nav.
2026-07-29 07:29:34 -04:00
cproudlock
3ba808028c dashboarddefaults: key display mappings by stable FQDN (from BIOS serial), IP fallback
A display's DHCP IP can change; its FQDN (F<serial>.<domain>, domain from the
display_fqdn_domain setting) is stable and the collector already reports the
serial. Add a nullable unique fqdn column (varchar191 so the index fits utf8mb4
without innodb_large_prefix), make ipaddress nullable, and require fqdn OR ip.
visitor-location + display-role resolve by FQDN first, then IP; create/update
accept fqdn. Core migration 7d31, verified up/down/idempotent on MySQL 5.6.
'Business unit' wording -> 'location' in the validation messages.
2026-07-29 07:20:52 -04:00
cproudlock
b539e36096 test: assert display seed keeps exactly two inline payloads across rebuilds 2026-07-28 18:56:29 -04:00
cproudlock
89a1617103 geenforce: fix re-publish FK crash on MySQL (stale entries in draft rebuild)
replace_scope_draft deleted old draft entries with per-object db.session.delete
but left the deleted objects in scope.entries. On a re-publish a caller
(seed_display_scope) then matched a stale deleted entry via next() and
store_inline_payload attached a payload to its dead entryid, failing the
manifestpayloads->manifestentries FK on MySQL (1452); SQLite does not enforce
it so the idempotency test passed. Clear the collection via the delete-orphan
cascade instead, and flush pending inserts before the bulk payload delete so its
autoflush cannot interleave a half-built insert. Verified publish + re-publish
x3 on MySQL 5.6.
2026-07-28 18:55:17 -04:00
cproudlock
9a2d0ccebb dashboard: resolve employee names from directory/user, GE monogram photo fallback, kiosk sweep + label
- notifications shopfloor feed: resolve the employee name live when the stored
  value is a bare SSO (WJ notifications imported as SSOs, never converted), for
  both single and split-per-employee cards
- employee name resolver: after a directory miss, fall back to the shopdb User
  account (firstname/lastname, keyed by SSO username) so users from other
  locations still show a name
- shopfloor dashboard: employee photo falls back to the GE monogram (own asset,
  independent of the site_logo setting) with a loop-guarded onerror; recognition
  + recert tiles both covered
- shopfloor dashboard: 'All Business Units' filter label -> 'All Locations'
- geenforce display dispatcher: startup sweep also matches the imaging
  installers' 'GE Aerospace Dashboard/Lobby' shortcuts by name
2026-07-28 18:21:47 -04:00
cproudlock
3a8df166cf geenforce: broaden kiosk startup sweep to match single-dash -kiosk and shopdb-URL launchers
The prior sweep only matched '--kiosk'; the imaging installers (Inno
GEAerospaceDashboardSetup / lobby) create Startup shortcuts with single-dash
'-kiosk' pointing at /shopdb/shopfloor-dashboard, so they survived. Match any
msedge/chrome Startup .lnk whose args contain -kiosk (one or two dashes) OR a
shopdb kiosk URL (tsgwp00525 / /shopdb/ / shopfloor-dashboard). Unrelated
Startup items are left untouched (VM-verified).
2026-07-28 18:09:12 -04:00
cproudlock
5712f72ccf geenforce: fix http-payload path doubling + sweep stale kiosk startup shortcuts
- Resolve-ShopdbPayloads wrote an absolute local path into the entry, and the
  engine resolves it as Join-Path InstallerRoot <field>, doubling it
  (C:\...\payloads\C:\...\payloads\<sha>.ps1 -> PS1 not found). Write the leaf
  filename instead; the runner already sets InstallerRoot to that payloads dir.
- display dispatcher now removes leftover kiosk launchers from prior installs
  (any Startup .lnk that runs Edge --kiosk, plus .url to a shopdb kiosk page),
  not just its own, so two kiosks do not fight.
2026-07-28 17:49:47 -04:00
cproudlock
4c0cc672a2 geenforce: harden allowlist + fix share-less kiosk client and display scope
- allowlist auth uses remote_addr, not the spoofable first X-Forwarded-For hop
  (adds _trusted_client_ip + a regression test); rate-limit path unchanged
- client psm1: fix Set-StrictMode crashes reading absent keys in Get-ShopdbConfig
  (token-less mode) and Resolve-ShopdbPayloads (no-payload entries); validate
  the manifest response is JSON before overwriting the last-known-good cache
- runner: pass the engine its required -InstallerRoot/-LogFile; create the log
  directory so enforce logging is not silently lost on a fresh kiosk
- display scope: dispatcher writes an all-users Startup shortcut instead of
  Start-Process (SYSTEM cannot show a window in session 0), resolves the base
  URL from HKLM, and adds an always-on power/no-lock entry; tests updated for
  the 6-entry scope
2026-07-28 17:09:21 -04:00
cproudlock
f533af82cd export: --dist builds both /ops and /shopdb frontend bases
Two instances run on the box (dev /ops + prod /shopdb); each needs its own
base-path build. Build both on every --dist so prod never ships a stale
frontend. tools/ is excluded from publication, so this is dev-tooling only.
2026-07-28 08:28:42 -04:00
cproudlock
67de46dfb9 geenforce: import-share recognizes the display scope folder
discover_share only matched 'common' and 'gea-shopfloor-*', so a display/
manifest.json on the share was silently skipped and 'flask geenforce publish
display' failed with 'No scope display/runtime'. The display scope is a
first-class HTTPS-pull target (kiosks fetch pctype=display), so accept it.
2026-07-27 14:54:13 -04:00
cproudlock
d9080a59ca geenforce: move settings into the Settings rail
The client IP allowlist config was a tab inside the GE-Enforce section; move
it to the Settings rail via get_settings_cards (matches printedparts / zabbix /
dell). Route relocated from /geenforce/settings to /settings/geenforce; the
in-section Settings tab is removed. Card: Settings > GE-Enforce.
2026-07-27 14:44:26 -04:00
cproudlock
2d675720b7 geenforce client: make ApiToken optional for IP-allowlisted kiosks
Get-ShopdbConfig required both BaseUrl AND ApiToken, so a token-less kiosk
(authorized by the server's IP allowlist) got a null config and never ran.
Now BaseUrl alone is a valid config; X-API-Key is sent only when a token is
present (New-ShopdbAuthHeaders), so token-authorized sites are unchanged and
vaulted-network sites need no per-PC token.
2026-07-27 14:17:51 -04:00
cproudlock
0860aa85c5 geenforce: IP allowlist for client endpoints + admin Settings tab
Fleet PCs on a trusted (vaulted) network can now reach the GE-Enforce client
endpoints (manifest, payload, report) without a per-PC token: the auth path
accepts a valid geenforce.fetch/report token OR a source IP in the configured
allowlist (setting geenforce_allowed_cidrs). Fail-closed; an empty allowlist
means the token stays the only path, so existing deployments are unchanged.

Rationale: the client token lives in HKLM on every kiosk, so it does not
defend against a compromised kiosk anyway - network-perimeter trust is the
same practical strength with far less provisioning + no token-rotation churn
on a DB wipe. Documented in-UI that this is perimeter trust, not per-device
identity.

- _ip_allowlisted() (ipaddress, X-Forwarded-For-aware via _client_ip)
- /geenforce/config GET/PUT extended with allowedcidrs, server-validated +
  normalized (bad CIDR -> 400)
- new GE-Enforce > Settings tab (GeEnforceSettings.vue) to edit the allowlist
  in admin, no SQL
- 3 regression tests (allow by IP, reject outside list, empty = token required)
2026-07-27 14:06:40 -04:00
cproudlock
19876a5640 warranty: fix bulk Dell re-check duplicating non-dell warranties
The bulk /sync/dell reuse check only matched an existing warranty when its
provider was exactly 'dell'. Warranties added by hand or via import default to
provider 'manual', so re-check-all did not recognize them and created a brand
new Dell warranty for every asset - duplicating the whole set.

Broaden the reuse match to treat a warranty as Dell by any signal (provider,
matching service tag, or a 'Dell' vendor), and canonicalize the reused row to
provider 'dell' so later re-checks match by provider and never duplicate.
2026-07-24 13:24:36 -04:00
cproudlock
6534590fca docker: air-gapped deploy kit (image bundle + offline compose + runbook)
Some checks failed
CI / backend (push) Successful in 1m51s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 9s
Air-gapped sites cannot pip install / npm ci / docker pull, so a build-at-site
compose (build: .) fails and reports 'service api is not running'. Add a
build-once-ship-image path:

- scripts/build-offline-bundle.ps1: on a connected box, build shopdb-flask +
  pull mysql:8.0, docker save both into one gzipped tarball with a sha256.
- docker-compose.airgap.yml: runs pre-loaded images (image:, never build:),
  drops the ./plugins bind mount (which would mask the image's baked-in plugins
  with an empty host dir and load zero plugins at an image-only site), and adds
  a one-shot migrate service (db upgrade + plugin upgrade-all + seed) that api
  waits on via service_completed_successfully, so 'up -d' brings a working site.
- docs/DEPLOY-AIRGAP.md: full runbook (build, transfer+verify, load+run, admin,
  verify, upgrade, troubleshooting), incl the Zscaler in-build cert caveat.
- .env.example: IMAGE_TAG for the air-gap compose to pin the loaded image tag.
2026-07-23 14:15:34 -04:00
cproudlock
75386d2f51 geenforce: resource-scope binding for fetch tokens (0.15.0)
A geenforce.fetch token can now be pinned to specific manifest scopes so a
fleet-wide key (a display's, delivered by DSC or baked into the image) is not a
skeleton key for the whole content store. NULL binding = unrestricted, so every
existing service token keeps working.

Core:
- ApiToken.resourcescopes column + resourcescopelist property (migration
  7d30_apitoken_resourcescopes; NULL = unrestricted).
- apitokens API create/update accept + persist an optional resourcescopes list
  (a resource-name allowlist; not permission-catalog names).
- New contract helper authorized_service_token(scope): same check as
  service_token_authorized but returns the ApiToken so a plugin can read its
  binding. Contract 0.14.0 -> 0.15.0; also export SupportTeam.

GE-Enforce enforcement:
- get_manifest: a bound token requesting a scope outside its allowlist -> 403.
- get_payload: a bound token may only pull a blob its own scope(s) reference
  (service.blob_referenced_by_scopes); anything else -> 404 (no hash probing).
- Decorator stashes the authorized token on g for the route to read.

Also fixes a pre-existing contract-surface violation: the printers/printedparts
alert helpers imported shopdb.core.models / shopdb.extensions directly; now
via shopdb.api (SupportTeam newly exported). Docs: GE-ENFORCE-DISPLAY.md
provisioning note, PLUGIN-HOOKS.md, CLAUDE.md.

9 new resource-binding tests; full suite 1131 passing.
2026-07-23 09:02:42 -04:00
cproudlock
d0bf37ced7 geenforce: display scope is self-sufficient, no common inheritance
Per decision: displays need none of the fleet-wide common scope's software, so
the gea-shopfloor-display scope carries everything it enforces and does not
inherit common. This avoids repackaging common's SMB-backed payloads for a
share-less display.

- Invert the client common-merge switch: -NoCommon (default-on) becomes
  -IncludeCommon (default OFF). A scope now enforces alone unless opted in.
  The capability stays for a future share-less non-display PC; displays omit it.
- Drop the common SMB-payload audit + inheritance sections from the display
  seed comments and docs (GE-ENFORCE-DISPLAY.md); document self-sufficiency.
- GE-ENFORCE-CLIENT.md: common-scope inheritance is now opt-in.
2026-07-23 08:22:23 -04:00
cproudlock
9d65ef103d geenforce: display-readiness batch (server hardening, PS client wiring, display scope)
Get GE-Enforce closer to running on credential-less Intune/Entra display PCs
that pull manifest + payloads over HTTPS instead of SMB.

Server (plugins/geenforce/api/routes.py):
- Rate-limit + 512MB served-size ceiling on GET /payload/<sha256> (reuses the
  login limiter's cache pattern, config-overridable via GEENFORCE_PAYLOAD_*).
- New tests: payload hardening, manifestblobs model-vs-migration parity, and a
  report-contract test locking the lowercase per-entry report keys.

PS client (plugins/geenforce/client/):
- Fix New-ShopdbReport per-entry key casing to lowercase (name/action/selfhealed/
  exitcode/message) to match what the server reads; the engine emits PascalCase.
- Enforce TLS 1.2 in the network functions.
- Fetch + merge the fleet-wide common scope alongside the pctype scope
  (pctype wins on conflict; -NoCommon opt-out).
- Normalize whatever the engine returns into a well-formed summary.
- Make the empty-cache fail-safe observable: event-log entry + report ping
  instead of a silent exit 0.

Manifest (plugins/geenforce/seed_display_scope.py + docs/GE-ENFORCE-DISPLAY.md):
- Seed a gea-shopfloor-display scope: 4 Edge kiosk drift-heal registry entries
  + 1 data-driven dispatcher (Dashboard/Lobby/3DPrintRoom via display-type.txt).
  Kiosk EXEs stay image-baked; the manifest heals policy/config drift only.
- Documents the common SMB-payload audit (entries needing http/inline before a
  share-less display can inherit common).

Migration registry (shopdb/plugins/alembic_template.py + test):
- Register the pre-existing manifestblobs and the new printersupplyalerts tables
  in PLUGIN_TABLE_OWNERS; update EXPECTED_HEAD_REVISION for geenforce (0002blobs),
  printers (0002supplyalerts), and printedparts (0004txnrev) which had drifted.
2026-07-23 08:16:38 -04:00
cproudlock
b211e817d5 printers: low-toner alerts with configurable thresholds + support-team routing
Poll Zabbix for toner levels on a schedule and email/webhook on a downward
crossing. Warning fires at or below the warning threshold (default 5%),
critical at the critical threshold (default 0%); both thresholds are settings.
State lives in printersupplyalerts so an alert fires once per crossing and
re-arms after a refill.

Recipients mirror the printedparts pattern: plugin-scoped shopdb users +
roles + free-text emails (falling back to the site alert_recipients), and a
chosen support team's webhook (falling back to the site alert_webhook_url).

- PrinterSupplyAlert model + migration printers0002supplyalerts
- alerttier(remaining, warning, critical) + check_supplies poller
- flask printers check-toner-alerts CLI (run via scheduled task/cron)
- printers alert settings + Low-Toner Alerts settings page
- 7 tests: tier boundaries, once-per-crossing + re-arm, toner-only scope,
  custom thresholds, support-team webhook routing
2026-07-22 14:47:59 -04:00
cproudlock
fb188fd302 alerts: per-support-team webhook; printedparts routes low-stock to a chosen team
Support teams gain a webhookurl (migration 7d29 + API + settings-page field), so
a team is a notification target. send_webhook(url=) lets a caller override the
site default with a team's webhook. Printedparts gains a 'alert support team'
setting (printedparts_alert_supportteamid) + selector on its settings page;
low-stock alerts post to that team's webhook, falling back to the site
alert_webhook_url. Email leg unchanged. Same pattern extends to other alerting
plugins (printers low-toner next).
2026-07-22 13:32:00 -04:00
cproudlock
824863a95a settings UI: add Site Base URL + Alert Webhook URL/format fields
The webhook + site_base_url settings existed in the backend but the Email
Settings page had no inputs (hardcoded fields), and the settings composable's
reactive map didn't include the keys so they never loaded. Add the three inputs
(site base URL, webhook URL, webhook format select) and register the keys.
2026-07-22 12:55:50 -04:00
cproudlock
38c7ec347b alerts: Teams webhook fan-out (contract 0.14.0) + printedparts detail revision column
send_webhook(title,text) posts alerts to an optional webhook (Teams Incoming
Webhook / Workflow, or generic JSON) via alert_webhook_url + alert_webhook_format
settings; send_alert fans out to it alongside email; exposed on shopdb.api
(0.13.0->0.14.0, PLUGIN-HOOKS synced); low-stock posts on its custom-recipient
path too. Also: recent-transactions table shows the consumed print-file revision.
2026-07-22 11:01:53 -04:00
cproudlock
d141fef203 printedparts: show revision at kiosk; low-stock email uses gage tag + item link
Kiosk now displays the scanned revision (badge, quantity, and done screens) so
the operator sees which revision they checked out - it was already recorded on
the take, just not shown. Low-stock email now uses the gage lab tag (was the
internal item code) and links to the item page when site_base_url is set (new
core setting, category email; emails have no request context to derive the URL).
2026-07-22 10:12:51 -04:00
cproudlock
baf6862151 printedparts: QR label with gage tag + revision; record taken rev
Label switches from CODE128 to a QR encoding 'TAG|rev' (gage lab tag + latest
print-file revision), so a physical part carries which revision it was printed
from - short payload stays low-version + reliable at 0.5in (margin quiet zone,
EC M, no logo). Item exposes latestrevision; kiosk strips the |rev to resolve
and records the scanned revision on the take (migration 0004 adds
printeditemtransactions.revision) for traceability of which rev was consumed.
Manual entry records a null revision.
2026-07-22 07:52:17 -04:00
cproudlock
e85553e5e7 notifications: show start/end date fields for ALL types
The time fields were hidden for the employee types (recognition, recert); show
them for every type. Backend already honors start/end on create and update and
only auto-fills the end when left blank (recognition = next 8 AM, recert = two
weeks), so nothing server-side changes.
2026-07-21 11:09:19 -04:00
cproudlock
0bb906a37c notifications: let Recognition set start/end dates; geenforce B2 client payload fetch
Recognition edit hid the time fields (grouped with Recertification), so start/end
could not be adjusted even though the backend honors them. Show the time fields
for every type except Recertification (due-date driven); Recognition end still
auto-fills to the next 8 AM reset when blank.

Also GE-Enforce B2 client (HTTPS payload consume): ShopdbEnforceClient.psm1 gains
Get-ShopdbPayload (fetch by sha256, verify, cache) + Resolve-ShopdbPayloads
(rewrite http/inline entries to local staged files so the engine installs from
local, no SMB); Invoke-ShopdbEnforce resolves payloads before running the engine;
importer parses PayloadSource/PayloadSha256/PayloadRef. VM-verified: a SYSTEM
Windows client fetched a payload over HTTP by hash, hash matched.
2026-07-21 10:56:00 -04:00
cproudlock
b00ef72581 geenforce: HTTPS payload delivery (content-addressed blob store + endpoint)
Lets share-less (Intune/local-account) PCs pull installers the manifest
references over HTTPS instead of SMB - the general capability the whole fleet
migrates toward. New ManifestBlob registry (migration 0002) with bytes on disk
at instance/geenforce/payloads/<sha256> (deduped by content); service.store_blob
+ blob_path; client-facing GET /api/geenforce/payload/<sha256> (geenforce.fetch
token, ETag=hash, serves the blob store or an inline DB payload by hash). The
serializer now emits PayloadSource/PayloadSha256/PayloadRef for http/inline
entries only (smb entries round-trip unchanged - parity green). CLI
'flask geenforce add-payload <file>' registers a blob and prints its sha256.
This is the shopdb half (B1); the PS client/engine fetch is B2.
2026-07-21 10:10:59 -04:00
cproudlock
60e2947fc7 displays: single display type with IP-driven role (dashboard/lobby/kiosk)
One 'display' image resolves what it shows from its own IP, like the existing
visitor-location BU mapping. Extend DashboardDefault with displayrole
(dashboard|lobby|partskiosk; migration 7d28, businessunitid now nullable since
only the dashboard role needs one) + a role->path map. New unauthenticated
GET /api/dashboarddefaults/display-role returns {role, path, businessunitid}
for the caller IP. Settings UI gains a Display selector, showing the business
unit only for the dashboard role.
2026-07-21 09:49:14 -04:00
cproudlock
b05fa33278 shopfloor dashboard: Verdana font + full left-edge type-color bar
Switch the dashboard to Verdana (system font on the Windows kiosks - built for
on-screen distance reading, zero bundle) and drop the Archivo package. Restyle
the event card's type-color indicator to span the whole left edge of the card
(old-site style) instead of a small pill.
2026-07-20 16:01:58 -04:00
cproudlock
eb1c46c053 shopfloor dashboard: fix recert-name height so long names don't shift the grid
A wrapped (2-line) name made its tile taller and shifted the whole recert grid.
Reserve a fixed two-line height on .recert-name (line-clamp 2 + ellipsis) so
every tile is the same height regardless of name length.
2026-07-20 15:40:28 -04:00
cproudlock
d39d34b55f shopfloor dashboard: use Archivo display font for TV legibility
Bundle @fontsource-variable/archivo (air-gap safe) and apply it to the shopfloor
dashboard only - a sturdy grotesque built for signage/displays, more legible
from across the shop than Inter. Rest of the app stays on Inter.
2026-07-20 15:19:28 -04:00
cproudlock
0dc4dba265 shopfloor dashboard: bigger text for distance + footer at bottom
Shrink the fit surface to 16:9 1600x900 so the fit-scaler upscales the whole
board ~1.2x (more readable from across the shop) and it fills a 1080p TV exactly.
Footer was position:fixed inside a transformed ancestor so it floated at the
content bottom; make it in-flow with margin-top:auto (flex column) so it pins to
the bottom of the surface = the screen bottom.
2026-07-20 15:16:26 -04:00
cproudlock
482d6d4bfe shopfloor dashboard: fit-to-viewport scaling for TV kiosks
The board was a fixed-pixel layout with an internal overflow-y scroll, so on a
TV (no scrolling) content past the fold was unreachable. Wrap it in a fixed
1920-wide surface and scale it to fill the viewport (ResizeObserver + resize),
so the whole board is visible edge-to-edge at any resolution. Drop the
.dashboard-content max-height/overflow scroll.
2026-07-20 14:49:33 -04:00
cproudlock
a50e5b0ec1 slides: fix manager thumbnail 404 under /ops + single-column order list
SlideManager rendered <img :src=slide.url> raw, so the root-relative
/api/slides/img/... path 404'd under the /ops subpath mount (the /tv display
already wrapped withBase; the manager did not). Wrap the thumbnail in withBase.
Also switch the multi-column grid to a single-column list with order numbers so
the top-to-bottom play order is clear to arrange.
2026-07-20 14:16:12 -04:00
cproudlock
8496441ddc slides: add /screensaver route for the shopfloor slide surface
TVDashboard hardcoded surface=lobby, so only the lobby display was reachable.
Read the surface from route meta (/screensaver -> shopfloor) or a ?surface=
query, defaulting to lobby. Adds a /screensaver toplevel route so the shopfloor
screensaver surface can be displayed on a kiosk.
2026-07-20 13:16:31 -04:00
cproudlock
18f027c951 printedparts kiosk: show gage lab tag (not item code) on the SSO prompt
The badge step card still showed the internal item code; show the gage lab tag
(fallback to item code) to match the label and list. The take still posts the
item code as the stable identifier.
2026-07-20 11:34:12 -04:00
cproudlock
108d4d396c printedparts: kiosk tag prefix + labels use the gage lab tag
Kiosk item lookup shows a fixed "WJ" prefix addon so operators type only the
number off the label. The label page - and the detail "Part Label" button,
renamed from "Bin Label" - now barcodes and prints the gage lab tag, falling
back to the internal item code when a part has no tag assigned.
2026-07-20 11:18:16 -04:00
cproudlock
31139267d1 Fix model images 404 under a subpath mount (withBase)
Model image URLs are root-relative (/api/models/image/...), so on an /ops
subpath deploy the raw <img src> resolved to the server root and 404'd. Wrap
every model-image src in withBase(): machine/printer/PC/network detail heroes,
the models settings preview, and the machine-badge / asset-label print pages.
withBase leaves external http(s)/data URLs untouched.
2026-07-20 11:14:00 -04:00
cproudlock
91fa3b9115 printedparts kiosk: one visible entry input for scanner and keyboard
The bin step hid entry behind a "Type the number" link, and a plugged-in
keyboard could not drive the visible fields. Replace the hidden wedge input
with one visible, always-focused input per step that a wedge scanner, a
physical keyboard/numpad, and the on-screen keypad all feed; Enter submits.
inputmode="none" keeps the OS soft keyboard from popping on a touchscreen.
2026-07-20 11:14:00 -04:00
cproudlock
89b3156ea1 printedparts: show gage lab tag in the catalog list
The list showed the internal auto-minted itemcode; the gage lab works from the
WJRP gage lab tag. Show gagelabtag as the primary identifier, falling back to
itemcode when a row has no tag assigned.
2026-07-20 11:03:32 -04:00
cproudlock
a7ff882e21 Add per-role badge colors
Role badges rendered gray for everything except admin, with no way to tell
roles apart. Add an optional color per role, matching how statuses and types
carry one: new roles.color column (migration 7d27_roles_color), color threaded
through the role API and the user serializer, and a ColorSwatchPicker in the
role editor. Badges use the role's color with contrast-aware text and fall
back to the old admin/gray classes when unset.
2026-07-20 10:05:58 -04:00
cproudlock
0b247ed96f CI: run GitHub Actions on self-hosted arc-runner-set
The org IP allow list blocks GitHub-hosted runner IPs (checkout 403), so
point all jobs at the self-hosted arc-runner-set. Drop the rsync dependency
in build-site.sh (cp + bytecode prune; the ARC runner image has no rsync)
and remove the migrations-mysql job - ARC/Kubernetes has no service
containers, so that MySQL 8 coverage stays on the internal CI.
2026-07-20 10:05:58 -04:00
cproudlock
3c830244f8 CLAUDE.md: refresh state header (1077 tests, 13 plugins, 16 WJF stages, lean-build/ADR-014) 2026-07-19 13:18:52 -04:00
cproudlock
e005d1846a docs: wiki staleness sweep (Fable-orchestrated Opus audit)
Audited all 40 docs/ against the live codebase; fixed factual staleness in 23,
14 were clean. Highlights (all verified against code):
- equipment -> machines (ADR-011 rename) in INSTALL/DEPLOY-WINDOWS-IIS,
  PLUGIN-GUIDE, GE-ENFORCE, ROADMAP.
- Versions refreshed: contract 0.10.0 -> 0.13.0, product 0.5.0 -> 0.7.0, plus
  plugin example core_version pins.
- Bundled set corrected to the current 13 (PLUGINS.md 7 -> 13 rows; DEPLOY
  eleven -> thirteen).
- Per-plugin Alembic chain workflow (ADR-008) replacing stale core-chain steps
  in PLUGIN-QUICKSTART / BACKUP-RESTORE; deploy adds plugin upgrade-all.
- Frontend plugin staging (ADR-010) replacing 'no frontend plugin system yet'
  in PLUGIN-GUIDE; view/route paths repointed to plugins/<name>/frontend/.
- Corrected file paths (MapView.vue, manifest_schema.json), CLI (shelf-list),
  API gating (GET /api/plugins is optional-jwt), WJF 15 -> 16 stages, and
  retired Collector/PC-Types settings pages (ADR-012).
- ge-enforce proposal marked ACCEPTED/built.
2026-07-19 12:54:53 -04:00
cproudlock
49a0206b9f docs: lean-build behavior + nav placement + fix stale prod plugin list
- PLUGINS.md: new 'Lean per-site builds' section (backend/frontend/DB layers,
  manifest-less core frontends always ship, menus gated to staged routes).
- PLUGIN-HOOKS.md: get_navigation_items sidebar placement (position ranges ->
  Assets/Information sections, section override, icon key).
- DEPLOY-WINDOWS-IIS.md: fix stale plugin list (equipment -> machines, complete
  the bundled set), add apply-profile + prune-schema flow.
2026-07-19 12:35:43 -04:00
cproudlock
212165befd Lean build: gate Shopfloor Dashboard on the notifications plugin
Shopfloor Dashboard is a core view but its content is entirely
notificationsApi.getShopfloor() + the calendar (both owned by the notifications
plugin). Without notifications the display is empty, so gate the Displays link
on the notifications/calendar route being staged. On a site without it the link
- and the Displays header when nothing else is present - drops.
2026-07-19 12:25:10 -04:00
cproudlock
a5ba973974 Lean build: gate Displays links by staged route, not plugin-enabled state
The Displays section is hardcoded in AppLayout (not plugin nav). TV Slideshow
(/tv, slides) had no gate at all and Parts Kiosk (/parts-kiosk, printedparts)
was gated on isPluginEnabled - which a registry copied from a full site reports
true even when the plugin was never staged, so both showed on a lean site and
dead-ended blank. Now each Displays link (Shopfloor, TV Slideshow, Parts Kiosk)
is gated by whether its route is registered in this build (router.getRoutes),
and the Displays header hides when none are present. Route existence is the true
'is it in this build' test.
2026-07-19 12:22:15 -04:00
cproudlock
5d86bc86b3 Lean build: hide settings cards whose route was not staged
settingsNav.js hardcodes plugin settings (PC Access Protocols, Machine Types,
VLANs, Employee Directory, ...). A lean per-site build only stages the chosen
plugins' settings routes, so the settings rail showed cards for absent plugins
that dead-ended on a blank page. useSettingsCatalog now filters the catalog to
cards whose target route is registered in this build's router (router.getRoutes),
dropping now-empty groups. Generic - gates every settings card by staged routes
with no per-plugin logic; full builds keep every card. Found testing a live
machines+printers lean site.
2026-07-19 12:17:51 -04:00
cproudlock
d009ac94fb Lean build: always ship core (manifest-less) frontends
A frontend dir under plugins/ with no manifest.json is a CORE feature, not a
per-site plugin - applications is one (backend is shopdb/core/api/applications.py,
nav is advertised as core in dashboard.py). stage-frontend.mjs treated it like
a plugin and dropped it under SITE_PLUGINS, so a lean site showed the core
Applications nav item but had no route for it -> blank page. Now manifest-less
frontends always stage regardless of SITE_PLUGINS; SITE_PLUGINS selection applies
only to real plugins. CI lean-build job asserts ApplicationsList ships in a lean
bundle. Found while testing a live machines+printers lean site.
2026-07-19 12:10:40 -04:00
cproudlock
c386e211df ADR-014 Phase 2: flask plugin prune-schema for lean per-site DBs
A lean site still gets every plugin's tables from the shared core Alembic
baseline. prune-schema drops the tables of plugins not installed on this
site, leaving core + chosen-plugin tables, with no edit to any released
migration (the relocate-into-plugin-baselines alternative would mean
rewriting ~15 released core migrations for a cosmetic gain - see ADR-014).

- shopdb/plugins/cli.py: prune-schema command. Dry-run by default; --yes to
  execute; refuses non-empty tables without --force. Drops by table name (no
  plugin import) so it works on a lean image. MySQL: private AUTOCOMMIT engine
  (db.engine's pooled connections sit idle-in-transaction in a CLI context and
  would deadlock the DROP on a metadata lock). SQLite: db.engine, restoring the
  prior foreign_keys pragma so the StaticPool connection is not left changed.
- tests/test_plugin_prune_schema.py: drop-only-not-installed, full no-op,
  refuse-non-empty, force-drops-non-empty.
- docs/DEPLOY.md: lean provisioning step after upgrade-all.
- ADR-014 ACCEPTED; index updated.

Verified on MySQL: full install then prune = no-op (86 tables); lean install
(machines+printers) then prune drops the other 19 plugin tables; second run
no-op. Full suite 1077 passed.
2026-07-19 11:39:46 -04:00
cproudlock
42ca8d75c3 ADR-014: schema-lean per-site (investigation + idempotent create_plugin_tables)
Cross-plugin FK blocker ADR-013 cited is already resolved: the FKs into
machines were held only by dead legacy tables (machinerelationships,
printerdata, installedapps, communications.machineid) that existing
migrations 7a01/7c01 already drop. No live plugin table hard-FKs another
plugin. Schema-lean is unblocked.

Enabling change: create_plugin_tables now skips already-existing tables
(idempotent) so a plugin anchor can create its tables on a fresh lean
install and no-op on a database that has them from the pre-cutover
baseline. The load-bearing baseline lift is staged as ADR-014 Phase 2.
2026-07-19 00:36:27 -04:00
cproudlock
5861caf78f ADR-013 Phase 5: CI lean-build job (delete-a-plugin guarantee)
New CI job builds a lean site (machines + printers) via build-site.sh and
asserts omitted-plugin code (PartsKiosk, ManifestEditor, USBLabelBatch,
KnowledgeBaseDetail) is absent from the bundle while chosen-plugin code is
present, and that only chosen plugin dirs stage into the backend. Locks the
lean-build guarantee so a future change cannot silently pull an unchosen plugin
into a per-site build.
2026-07-19 00:09:23 -04:00
cproudlock
da3cb37be8 ADR-013 Phase 5: lean per-site builds - build-site.sh + import-guard audit
The lean-build endgame: a site ships carrying only the plugins it chose.

- scripts/build-site.sh: reads a site profile, resolves the hard-dependency
  closure from manifests, builds the frontend with SITE_PLUGINS (stage-frontend
  carries only those plugins), and stages a backend tree of core + only the
  chosen plugin dirs. An unchosen plugin is in neither the bundle nor the tree.
- Core lazy-import guard: `flask seed demo` hard-imported the 5 asset subtype
  models, which would crash a lean build missing any of those plugins. Now
  guarded (a missing model skips its demo section).
- test_lean_build_guards.py: statically asserts NO core (shopdb/core, shopdb/cli)
  import of a plugin is unguarded - a lean build omitting that plugin would
  otherwise crash. 0 unguarded today.

Pilot verified: a lean build (machines + printers) carries only machines +
printers code - PartsKiosk / ManifestEditor / USBLabelBatch / KnowledgeBaseDetail
/ EmployeeDirectory are absent from the bundle, and only machines/printers plugin
dirs stage into the backend. (Sidebar labels for absent plugins remain - the
accepted small plugin-aware core remainder.) Guard test + naming green.
2026-07-19 00:08:35 -04:00
cproudlock
c6a1e07a6c ADR-013 Phase 4: lint plugin-frontend imports (self-contained rule)
The naming/style check now fails a plugin frontend (plugins/<name>/frontend/)
that imports with an escaping ../../ or another plugin's path. Plugin frontends
must reach core only through the @/ alias and otherwise import only their own
tree, so a per-site build can drop a plugin cleanly. All 14 plugin frontends
pass.
2026-07-19 00:03:33 -04:00
cproudlock
592ff49abe ADR-013 Phase 4: extract the 11 plugin routes embedded in core.js
core.js still routed plugin-owned pages directly. Extracted all 11 into the
owning plugin's route file + moved their views into plugins/<name>/frontend/:
- computers: reports/pc-relationships, settings/pctypemapping
- printers: reports/toner, settings/printertypes, settings/zabbix (toner/supply
  monitoring)
- machines: settings/machinetypes
- network: settings/networktypes
- warranty: settings/dellwarranty
- slides: settings/slides (its route file gains a default export; it was
  toplevel-only)
- employees: NEW plugin frontend (employees/:sso + settings/employeedirectory) -
  employees had no route file before; its pages lived only in core.js.

core.js now holds only core routes; all 14 bundled plugins are self-contained
under plugins/<name>/frontend/. Verified live: the extracted Machine Types
settings page renders in the settings rail from the machines plugin frontend.
Build + 58 vitest + naming green.
2026-07-19 00:02:32 -04:00
cproudlock
ebca0b00b0 ADR-013 Phase 4: relocate the remaining 9 plugin frontends (all 13 done)
Relocate warranty, measuringtools, network, printers, usb, notifications,
computers, and slides into plugins/<name>/frontend/. Each plugin's views are
pulled from wherever they lived (own dir, plus the shared views/settings/,
views/reports/, views/print/ dirs, and top-level views) into the plugin's
frontend/views/, and its route file becomes the self-contained routes.js.

Handled the messy cases:
- computers: name mismatch (its views live in views/pcs/) - moved by following
  the route file's own imports, so the dir name did not matter. Its OS/access-
  protocol/PC-type settings views move with it (only computers.js routed them).
- network: NetworkHub's sibling sub-views (NetworkDevicesList, SubnetsBrowse,
  not directly routed) moved too so its `./` imports resolve.
- printers: the qrLogo helper is SHARED with core AssetLabel, so it stays in
  views/print/ and PrinterQR imports it via @/views/print/qrLogo.
- slides: route file is toplevel-only (TVDashboard); SlideManager stays core
  (core.js routes /settings/slides).

frontend/src/views/ now holds only core views; frontend/src/router/routes/ holds
only core.js. All 13 plugins are self-contained under plugins/<name>/frontend/.
Verified live: Network (hub + moved sub-views), Computers (name mismatch),
GE-Enforce (helper), printedparts all render from their staged frontends. Build +
58 vitest + naming green.
2026-07-18 23:56:07 -04:00
cproudlock
23dc9fa379 ADR-013 Phase 4: relocate 4 self-contained plugin frontends
Relocate applications, geenforce, knowledgebase, and machines - each owns only
its own views dir, so a clean move to plugins/<name>/frontend/ (views/ +
routes.js, core imports rewritten to @/). geenforce's entryForm.js helper + its
vitest spec move with it (ManifestEditor imports it as a sibling).

Machinery fixes this batch surfaced:
- routes.gen.js codegen uses namespace imports (import * as p_x). A route file
  without a `toplevel` export is undefined on the namespace instead of a strict-
  ESM missing-binding build error.
- vitest gains a `pretest` stage so plugin-frontend specs (now under
  plugins/<name>/frontend/) run from their staged copy in src/.plugins-staged/.

Verified live: GE-Enforce (the most complex, uses the entryForm sibling helper)
renders fully from its staged frontend. Build + 58 vitest + naming green.
2026-07-18 23:51:23 -04:00
cproudlock
af9a3b190b ADR-013 Phase 4: frontend staging machinery + relocate printedparts; fix router crash
The staging step that makes lean per-site frontend builds possible, plus the
first plugin relocated as the pilot.

- scripts/stage-frontend.mjs: copies each chosen plugin's plugins/<name>/frontend/
  into frontend/src/.plugins-staged/<name>/ and codegens routes.gen.js. Plugin
  selection via SITE_PLUGINS (comma-separated); empty = all plugins that have a
  frontend/ (the full build). Wired as npm predev/prebuild; outputs gitignored.
- Router imports routes.gen.js and merges staged routes with the in-tree
  ./routes/*.js glob - dual-location during the transition.
- printedparts relocated: its 6 views (list/detail/form/kiosk + the settings and
  labels views from the shared dirs) moved into plugins/printedparts/frontend/
  views/, core imports rewritten to the @/ alias; routes.js is the self-contained
  route module. Its old in-tree route file is removed.

Also fixes a crash the previous commit (37c764b) shipped: slides.js exports only
`toplevel` (its child routes live in core.js), so the router's
flatMap(m => m.default) produced an undefined child and threw
"Cannot read properties of undefined (reading 'path')" at load - the whole SPA
went blank. Guarded with `m.default || []`. (The earlier "print pages are blank"
reading was this crash, not page nature.)

Verified live: /machines renders again; the relocated /printedparts list renders
identically from the staged plugin frontend; SITE_PLUGINS=machines excludes
printedparts from routes.gen. Build (via npm, runs stage) + vitest + naming green.
2026-07-18 23:42:43 -04:00
cproudlock
37c764ba8d ADR-013 Phase 4: move hardcoded plugin top-level routes into plugin route files
Core-router surgery (the Phase 4 prerequisite for lean builds): index.js
hardcoded six plugin-owned full-screen routes (parts-kiosk, TV, printer-qr x2,
usb-labels, printedparts-labels), so pruning any of those plugins broke the SPA
build on an unresolvable import. The router now also collects a `toplevel`
named export from each plugin route file (alongside the existing default =
AppLayout children) and spreads it into the top-level routes. Each of the six
routes moved into its owning plugin's route file (printedparts, printers, usb,
slides); index.js keeps only the core print pages that span asset types
(machine-badge, asset-label, asset-label-batch).

index.js now references zero plugin view components. Verified: all six route
paths are present in the built bundle and the moved routes resolve exactly like
the unchanged core print routes. Build + vitest + naming green.
2026-07-18 23:26:17 -04:00
cproudlock
296b6e024b ADR-013 Phase 3: generic map-overlays renderer (Path A)
Wires the ADR-010 get_map_overlays hook into the floor map so a plugin decorates
markers as JSON, no map code. ShopFloorMap fetches /api/pluginui/map-overlays,
then each overlay's endpoint (per-asset [{assetid, color, label}]), joins by
assetid, and draws a ring or badge circleMarker on matching markers plus a
legend entry - all as extra Leaflet layers cleared and redrawn with the markers.

Aligned the measuringtools calibration overlay endpoint to the documented
contract: it now returns {assetid, color, label} (was {calibrationstatus,
statuscolor}) and only decorates due/overdue tools.

Additive + guarded (assetid null check, per-endpoint try/catch, cleanup on
re-render), so the map degrades to no decorations on any failure. Verified: the
overlay endpoint serves the contract shape, the map renders without error, and
the frontend builds. A populated badge needs a site that actually places
measuring tools on its map (this dataset places none). 38 measuringtools/pluginui
tests, 58 vitest, build + naming green.
2026-07-18 23:10:09 -04:00
cproudlock
8a2f984393 ADR-013 Phase 3: search routes via get_asset_presentation, not a hardcoded map
Global-search rows built the plugin detail URL from a hardcoded url_map of
plugin routes in core. Now core prefers a plugin's declared
get_asset_presentation route (ADR-010), substituting the core assetid via the
plugin's by-asset resolver; types that have not declared fall back to the legacy
id-keyed map, so nothing breaks. Measuring tools (which declare the route) link
through it now; machines/PCs/printers/network migrate off the hardcode as they
add a by-asset route + declaration. Presentation map is collected once per
search (cached on flask.g). 2 consumer tests; 26 search tests green.
2026-07-18 22:56:05 -04:00
cproudlock
b669561421 ADR-013 Phase 3: migrate all detail pages to PluginAssetPanels; drop WarrantyPanel
Rolls the generic renderer into the remaining four detail pages (PCDetail,
PrinterDetail, NetworkDeviceDetail, MeasuringToolDetail), replacing the
hand-composed <WarrantyPanel> with <PluginAssetPanels>. The warranty hero badge
(useWarrantyBadge) stays on the pages that show it; MeasuringToolDetail dropped
its now-unused warranty composable usage.

WarrantyPanel.vue is deleted - warranty now renders entirely from its
get_asset_panels JSON declaration through the generic renderer. Verified live on
a PC with a warranty: the card is identical to the old bespoke panel (vendor
title, Expiring Soon status badge with color, servicelevel/ends/tag meta, manage
link) with no warranty-specific frontend code. Build clean, 58 vitest, naming green.
2026-07-18 22:50:27 -04:00
cproudlock
e3c4b90afe ADR-013 Phase 3: generic asset-panels renderer (Path A)
Wires the ADR-010 get_asset_panels hook to a generic frontend renderer so a
plugin adds detail-page UI as JSON, no Vue. This is the Path A foundation that
lets simple plugins ship UI without a frontend build.

- components/PluginAssetPanels.vue + pluginAssetPanels.js: fetches
  /api/pluginui/asset-panels for an asset, then each panel's data endpoint, and
  renders by mode: list (title + status badge + meta lines via a field map),
  keyvalue, table (declared or inferred columns), badge. Pure mapping logic is
  in the .js module and unit tested (9 specs), same pattern as entryForm.js.
- New 'list' render mode with a declarative field map (title/badge/meta),
  documented on the hook in base.py.
- Warranty migrated to it: get_asset_panels now declares a 'list' panel + map
  that reproduces WarrantyPanel's output (vendor title, status badge with color
  + label map, servicelevel/ends/tag meta, manage link) with zero
  warranty-specific frontend code.
- MachineDetail swapped from <WarrantyPanel> to <PluginAssetPanels> (pilot); the
  hero warranty badge is unchanged. Verified end to end: the API serves the list
  panel + map and the warranty rows; the page renders without error.

Rollout of the other 4 detail pages (PCDetail, PrinterDetail, NetworkDeviceDetail,
MeasuringToolDetail) and the map-overlays / asset-presentation renderers are
follow-up Phase 3 commits. 58 vitest, build clean, 1067 backend pass, naming green.
2026-07-18 22:43:54 -04:00
cproudlock
beea6c0c9f ADR-013 Phase 2: guard hash-gates a .py sibling of an init-less dir
Fourth review found the last import-path bypass: the is_dir() branch returned
None for a name whose dir has no __init__.py, without checking a same-name
sibling file. FileFinder loads a file over an init-less namespace dir, so an
attacker could overwrite a signed foo.py with malicious bytes, mkdir an empty
foo/ next to it (PROVENANCE untouched, still verifies), and any import of that
name ran the unverified foo.py - RCE with only plugins/ write access.

Fix: the dir-with-no-__init__.py branch no longer returns early; it falls
through to the leaf .py hash gate and the non-source refuse check. Invariant:
find_spec returns None for a plugins.* name ONLY where FileFinder would also
find nothing on the same __path__.

Everything else was confirmed sound this round: the owned plugins root, exec of
exact verified bytes (never .pyc/.so), the extension/bytecode refusal, plugin.py
read-once, the provenance signature gate, dev-exemption scoping, and #3/#4.
Symlink, suffix-ordering, cache-lifecycle, and loader-internal angles cleared.
2 regression tests (tampered .py + sibling dir; unsigned .py + sibling dir). All
13 bundled plugins still load under enforcement; 1067 pass, naming green.
2026-07-18 22:28:33 -04:00
cproudlock
c59d2dab56 ADR-013 Phase 2: import guard fails closed on non-.py + owns package root
Third review found the meta_path guard leaked exactly where it delegated to the
stdlib import system:

1. Non-.py submodules (CRITICAL). When a name had no dir and no .py, find_spec
   returned None and the stdlib loaded a planted .so (ExtensionFileLoader) or a
   sourceless .pyc unverified - an attacker deletes a signed .py and drops a
   same-named .so with arbitrary init code, run on a normal request via core's
   `from plugins.<name>.models import ...`. The guard now refuses any name for
   which a non-source importable candidate (EXTENSION_SUFFIXES + BYTECODE_
   SUFFIXES) exists on disk; None is reserved for genuinely-absent modules.

2. Top-level plugins/__init__.py (CRITICAL). It is in no plugin's provenance,
   is attacker-writable, and Python runs it before any guarded submodule. The
   guard now owns `plugins`: it execs an EMPTY package body (search points at
   the plugins dir), so an overwritten plugins/__init__.py never runs.

Also: specs are built with spec_from_file_location so loaded modules get
__file__/__path__ (Flask blueprint root paths need it) while the loader still
execs the verified in-memory bytes - never re-reading the file.

Verified end to end: under PLUGIN_REQUIRE_SIGNED with all 13 bundled plugins
stamped, the app boots and loads every plugin through the guard; a tampered
plugin file is refused at load. 4 new guard tests (planted .so, sourceless
.pyc, absent-module defer, neutralized package root). Prior fixes #3/#4
confirmed still sound by the review. 1065 pass, naming green.
2026-07-18 22:12:20 -04:00
cproudlock
dd30ca0c3f ADR-013 Phase 2: verify every plugins.* import via a meta_path guard
A re-review showed the previous "single import choke point" claim was wrong:
`plugins` is a normal importable package, so core request handlers that do
`from plugins.<name>.models import ...` never passed through the loader and ran
unverified - an attacker who dropped a file into plugins/<name>/ got arbitrary
in-process code execution on an ordinary HTTP request (and a planted .pyc ran
from cache). Gating load_plugin_class covered only plugin.py, one path of many.

Fix: importguard.py installs a sys.meta_path finder (under enforcement) that
intercepts EVERY plugins.<name>.* import, verifies the plugin's signed
provenance once, then verifies each module file against it and execs the exact
bytes it hashed - read once, compiled, exec'd, never a .pyc, never a re-opened
file. This closes the submodule bypass and the planted-bytecode read, and the
read-once exec closes the verify-vs-exec TOCTOU on the import path. The import
system, not one method, is the real choke point.

- init_app installs the guard when PLUGIN_REQUIRE_SIGNED, clears it otherwise.
- load_plugin_class now verifies plugin.py from a single read and execs that
  buffer (finding #3 on that file); its submodule imports flow through the guard.
- docs: stamp-bundled must cover every plugin dir present (a disabled plugin's
  module can be imported by core); recommend a read-only plugins/ owned by the
  deploy user as defense in depth (closes the residual migrate-time race an
  attacker with concurrent write could otherwise attempt).

Earlier review's fixes #3 (migrate code paths) and #4 (shelf content binding)
were confirmed sound and are unchanged. 7 import-guard tests (submodule verify,
tamper, unsigned refused, planted .pyc ignored, real import through the guard,
install/uninstall). 1061 pass, naming green.
2026-07-18 21:47:32 -04:00
cproudlock
55a6f1b8d3 ADR-013 Phase 2: fix four bypasses found by adversarial review
An adversarial security review of the Phase 2 trust model found four real
bypasses (two remote-triggerable to in-process code execution). Root cause for
three: the set of bytes verification covered was smaller than the set that
determined execution. Fixes:

1. Bytecode-cache blind spot (CRITICAL). verify_dir excluded __pycache__/.pyc,
   so a planted cache ran while escaping the hash map. verify_dir now flags any
   bytecode as an unexpected file; the loader strips bytecode before verify and
   imports under sys.dont_write_bytecode, so only verified source executes.

2. Unauthenticated verify-at-load bypass (CRITICAL). load_plugin_class imported
   plugin.py with no gate, reachable via discover_available / an anonymous GET
   /api/plugins. The verify+strip gate moved INTO load_plugin_class - the single
   import choke point every path flows through - so an unsigned/tampered plugin
   is never imported. discover_available skips a refused plugin instead of 500.

3. Ungated migration entrypoints (HIGH). downgrade_plugin and get_current_head
   (ScriptDirectory imports version modules) ran plugin code with no check. All
   alembic-invoking methods now pass through _verify_ok (strip + verify) first
   and run under no-bytecode.

4. Revocation/content bypass (HIGH). The signed index bound a filename, not
   content; adopt did not bind the delivered bytes to the resolved version, so
   revoked bytes could be served under a live filename. The index now records a
   per-artifact SHA-256; adopt verifies the on-disk digest and requires the
   artifact's own signed manifest version to equal the resolved version.

Enforcement stays default-off; strip/no-bytecode run only under enforcement, so
the unsigned path is unchanged. 6 regression tests (planted bytecode, the
discover import path, downgrade gate, version-swap). 1054 pass, naming green.
2026-07-18 21:06:27 -04:00
cproudlock
5b19f3b554 ADR-013 Phase 2: enforcement + signed shelf + adopt
Completes the marketplace security model. Verification stops being advisory:
a plugin only loads or migrates when its tree matches a trusted signature, and
plugins are pulled from a signed shelf with anti-rollback and revocation.

Enforcement (default OFF - existing deploys unchanged):
- verification.py PluginVerifier, shared by the loader (verify-at-load, before
  plugin.py is imported) and the migration manager (verify-at-migrate, before
  any DDL). Fail-closed: an unsigned/tampered/wrong-key plugin does not run.
- Gated by PLUGIN_REQUIRE_SIGNED. PLUGIN_DEV_TRUST_DIRS exempts named dirs but
  only under DEBUG/TESTING; production ignores it.
- flask plugin stamp-bundled writes provenance into in-tree plugins so
  verify-at-load applies to bundled plugins too (image build step).
- tier:core manifest guard: uninstall/disable refuse a core-tier plugin.

Shelf (shelf.py):
- Signed shelf-index.json (+ .sig): monotonic serial (a site refuses an older
  index - anti-rollback), revoked list carried across builds, per-entry
  version/tier/core_version for browse. Index is a browse layer only; adopt
  reads security-bearing fields from the verified artifact.
- flask plugin shelf-build / shelf-list / adopt / audit. adopt verifies index +
  artifact (signature + every file hash), unpacks to staging, re-verifies, then
  atomically moves into place and installs+enables the closure. Refuses a
  downgrade without --force-downgrade. Anti-rollback serial stored in
  instance/shelf-state.json.
- config PLUGIN_SHELF_DIR; the app only reads the folder, never speaks a
  network. .env.example + docs/PLUGIN-SIGNING.md document the flow.

22 tests: verifier policy (off / no-keys / signed / tampered / wrong-key /
dev-exempt), verify-at-load + verify-at-migrate integration, tier guard, index
sign/verify + tamper/wrong-key, serial state, revocation, version resolution,
verified atomic unpack + tamper refusal. Live-smoked keygen->pack->shelf-build
->list->adopt->audit + serial guard. 1050 pass, naming green.
2026-07-18 20:44:54 -04:00
cproudlock
86f5f1be68 ADR-013 Phase 1: signed plugin artifacts (pack/validate/keygen)
Packaging + provenance for the plugin marketplace. No runtime behavior change
yet - verification is available on demand; enforcing it at plugin load/migrate
and pulling from a shelf are Phase 2.

- signing.py: ed25519 key pairs + provenance. Provenance is a sorted per-file
  SHA-256 map plus metadata; the detached signature covers the exact
  serialized provenance bytes, so verifying is re-hash files, re-serialize,
  check signature. verify() accepts any of several trusted keys (rotation).
  Uses cryptography (already a dependency).
- packaging.py: pack() builds a signed <name>-<version>.shopdbplugin (zip +
  PROVENANCE.json + PROVENANCE.sig). verify_artifact()/verify_dir() re-hash
  and check the signature, and flag a tampered file, an unexpected file, a
  wrong/absent key - all fail closed.
- CLI: `flask plugin keygen` (publisher key pair), `flask plugin pack <name>
  --key` (validates then signs), and `flask plugin validate` extended to a
  signed artifact by path (--pubkey, else PLUGIN_TRUSTED_KEYS).
- config PLUGIN_TRUSTED_KEYS: os.pathsep-separated public-key PEM paths,
  delivered with the site config, never read from the shelf. .env.example
  documents it.
- docs/PLUGIN-SIGNING.md: curator flow (keygen offline, review, pack, publish,
  pin keys, rotate).

The signature proves an artifact is exactly what a curator signed, not that the
code is safe - human review before signing is the control. 11 tests: sign/verify
round trip, wrong key, provenance excludes noise, serialize determinism, pack +
verify, tamper -> hash mismatch, extra file, no-key fail-closed, verify_dir.
1028 pass, naming green.
2026-07-18 20:21:57 -04:00
cproudlock
d178726687 ADR-013 Phase 0: plugin lifecycle groundwork
Additive, zero-risk-to-running-sites prep for the plugin catalog. No
distribution or lean-build behavior yet; fixes latent bugs and adds the
declarative + validate tooling later phases build on.

Fixes:
- upgrade_all_plugins iterates registry.get_all(); only adopted plugins are
  migrated. Removes the phantom hasattr(registry, 'list_installed') probe
  that always fell through to migrating every folder on disk (unadopted DDL
  ran with full DB rights on every deploy).
- Reverse-dependency checks on uninstall/disable read dependencies from the
  manifest on disk via _installed_dependents, so an installed-but-unloaded or
  disabled dependent is counted. Uninstall blocks on any installed dependent;
  disable blocks on an enabled dependent.
- _sort_by_dependencies detects a dependency cycle (back edge in the DFS) and
  raises PluginDependencyError instead of looping or dropping a plugin.

New:
- flask plugin validate <name>: manifest loads + name match, manifest-schema
  check, core_version admits the framework contract, declared dependencies
  exist on disk. No new dependency (lightweight checker); schema ships in the
  package at shopdb/plugins/manifest_schema.json (docs/ is stripped on
  publish). The check caught that provides is an object, not an array.
- flask plugin apply-profile <file>: declarative install AND enable of a
  chosen plugin set plus its hard-dependency closure, in dependency order,
  idempotent. Replaces the hand-ordered runbook sequences that could enable a
  plugin that was never installed. deploy/site-profile.example.json template.
- Dockerfile header corrected (all 13 catalog plugins, not "eleven core").

10 new lifecycle tests (reverse-deps from disk, cycle detection, upgrade-all
scope, profile closure, schema, all 13 manifests match schema). 1018 pass,
naming green.
2026-07-18 18:08:34 -04:00
cproudlock
3ac5ed2580 Add ADR-013: plugin catalog, curated shelf, lean per-site builds
Design record for distributing optional plugins across GE sites: a small
mandatory core plus a catalog of optional plugins, packaged as signed
versioned artifacts, served from a transport-agnostic read-only shelf (a
SharePoint-synced or sneakernet folder - untrusted either way because every
decision-bearing byte is signed), verified at adopt AND at every load and
migrate. Lean per-site builds stage only chosen plugins into the backend
image and SPA bundle.

Status PROPOSED. Grounds the design in the real loader/contract/migration/
frontend code and records defects to fix along the way (upgrade-all
migrating unadopted folders, enable-without-install, reverse-dep checks
blind to unloaded plugins, missing cycle detection and dependency closure,
hardcoded plugin imports in the SPA router). Honest on scope: the frontend
re-org is the long pole (one core-router change plus per-plugin relocation),
not a mechanical move. Phased 0-5 with schema-lean and runtime-JS delivery
explicitly deferred.
2026-07-18 17:40:29 -04:00
cproudlock
603872ff76 Add Copilot custom instructions
Repo-level instructions so GitHub Copilot follows the LOCKED naming rules
(lowercase concatenated DB columns, allowed-acronym list, banned shorthand),
the ASCII-only style policy, and the plugin/migration/contract architecture.
Without this Copilot suggests snake_case columns, em-dashes, and
db.create_all(), which the naming hook and CI then reject. Distilled from
CONTRIBUTING.md; that file stays the authority.
2026-07-17 20:56:41 -04:00
cproudlock
6ab1046ef4 Add flask seed demo sample-data command
New dev/eval seeder populates a small, broad dataset so a fresh site has
something on every screen: ~25 assets across machines, computers,
printers, network devices, and measuring tools, plus supporting
vendors/business-units/locations, six 3D-printed parts (two below their
low-stock threshold to exercise the alert), and a few relationships for
the map and relationship cards. Idempotent, keyed on a DEMO- assetnumber
prefix; skips the plugin sections that are not installed.

`flask seed demo-clear` removes exactly what it created: bulk-deletes the
DEMO- assets so the DB-level ON DELETE CASCADE drops each plugin subtype
row (per-object ORM delete would try to NULL the NOT NULL child assetid),
after clearing the demo relationships first. Leaves reference data,
settings, users, and any imported rows untouched.

Documented as an optional step in the dev setup guide.
2026-07-17 20:32:01 -04:00
cproudlock
83141bacb7 Widen settings.description to TEXT; run seeders in CI
The dualpath_single_machine setting description is 257 chars but
settings.description was varchar(255). On strict MySQL 8 an over-length
insert is a hard error 1406 (Data too long), so `flask seed settings`
failed on a fresh install; older/relaxed MySQL truncated silently and
hid it. Widen the column to TEXT (matches value, already TEXT) via core
migration 7d26.

CI only ran `flask db upgrade` + plugin install, never the seeders, so it
missed this. Add a seed step to the migrations-mysql job so a seeded row
that violates a column constraint fails CI on strict MySQL 8 instead of
shipping.
2026-07-17 20:31:48 -04:00
cproudlock
804c066de4 Add cryptography dependency for MySQL 8 auth
Some checks failed
CI / backend (push) Successful in 1m44s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
pymysql needs the cryptography package to speak MySQL 8's default
caching_sha2_password, so 'flask db upgrade' against a stock MySQL 8
failed with 'cryptography package is required'. Make it a real
dependency (dev, prod, CI all connect cleanly) and drop the CI
native-auth workaround that stood in for it.
2026-07-17 19:41:42 -04:00
cproudlock
9deb194580 Standardize on Python 3.13 (matches prod 3.13.7)
Some checks failed
CI / backend (push) Successful in 1m45s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
Prod runs Python 3.13.7, not the originally planned 3.12. Align the
stack: CI both jobs 3.12->3.13, the IIS install runbook and the dev
setup guide to 3.13 (winget Python.Python.3.13). NOTE for whoever
maintains the offline kit: its wheels are still cp312 and must be
regenerated as cp313 before the next air-gapped deploy.
2026-07-17 19:30:19 -04:00
cproudlock
314f339ba9 Dev setup: winget install commands for the Windows toolchain
Some checks failed
CI / backend (push) Successful in 1m45s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
Add a winget block to the prerequisites (Git/Python/Node/VS Code/MySQL
or Docker) so a Windows dev provisions the whole toolchain from one
terminal, with a note that the LTS Node may be newer than CI's 20 and
it does not matter for this SPA (nvm-windows to pin if wanted).
2026-07-17 18:58:06 -04:00
cproudlock
1361dc6004 Lab intro: tag range through lab-stage-17 (last audit finding)
Some checks failed
CI / backend (push) Successful in 1m45s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
2026-07-17 18:15:59 -04:00
cproudlock
efb879d44a Docs audit fixes: kiosk code drift, PowerShell chains, broken links, leaks
Some checks failed
CI / backend (push) Successful in 1m45s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
From the Fable/Opus documentation audit (8 confirmed + verified
lab-drift the run's session limit had cut short):
- HIGH: the lab's kiosk _kiosk_find_item block showed the pre-stage-17
  row-id resolver as current; replace with the shipped gagelabtag /
  numeric-tail resolver, fix the stale 'resolved by row id' prose and
  the 'stage-7 code is corrected' note.
- MED: the badge _external_lookup block used dict-only row access that
  breaks on a tuple cursor; use the tuple-or-dict form shipped. Split
  '&&' command chains (fail in PowerShell 5.1) in the lab.
- LOW/link: the Windows note's [DEVELOPMENT-SETUP] link dropped the .md
  and 404'd in four docs; fix. Correct the stage-6a->16a comment and
  the lab-stage tag range (..16 -> ..17).
- Leaks: drop /home/camp path from ADR-006, the internal gitea host
  from PLUGINS.md.
- Windows: add an mklink junction note for the external-plugin symlink
  dev loop.
- CI: prime root to mysql_native_password so pymysql connects to the
  MySQL 8 service without the cryptography package (and its kit wheel).
2026-07-17 18:05:47 -04:00
cproudlock
aba588cc07 Neutralize internal-host references for publication
Some checks failed
CI / backend (push) Successful in 1m45s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
The CI workflow comment named the internal server, and
PLUGIN-EXTERNAL-REPO carried internal gitea clone URLs (it becomes a
public wiki page). Point both at the GitHub home / a generic CI
mention so the publication scrub gate passes and the wiki does not
expose internal infrastructure.
2026-07-17 15:13:34 -04:00
cproudlock
f77f0a8d90 Add GitHub Actions CI + Windows notes on the developer docs
Some checks failed
CI / naming (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / migrations-mysql (push) Has been cancelled
CI / backend (push) Has been cancelled
GitHub had no CI, so naming/tests/build were unenforced on the public
mirror. Add .github/workflows/ci.yml mirroring the internal pipeline:
backend pytest, the naming gate, frontend vitest+build, and the
migrations-mysql job that proves a fresh flask db upgrade + every
plugin chain on utf8mb4 MySQL 8 is idempotent. Flip the dev-setup CI
note to reflect it. Add an identical Windows/VS Code convention note to
the four developer docs (venv\Scripts vs venv/bin, $env: vs export,
pointer to DEVELOPMENT-SETUP).
2026-07-17 15:12:36 -04:00
cproudlock
0e194c3237 Dev setup: note GitHub has no CI yet, so local checks are the gate there
Some checks failed
CI / backend (push) Successful in 1m46s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
2026-07-17 15:09:16 -04:00
cproudlock
9a60100cb2 Dev setup: correct the hook claim, ship an opt-in committed hook
Some checks failed
CI / naming (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / migrations-mysql (push) Has been cancelled
CI / backend (push) Has been cancelled
The naming check was documented as an auto-running pre-commit hook,
but .git/hooks is never cloned and no installer existed - a fresh
clone had nothing, and the real enforcement is CI. Say that plainly.
Ship .githooks/pre-commit (LF-pinned) so a dev who wants the local
check can opt in with 'git config core.hooksPath .githooks'; CI stays
the backstop that fails the build on a bad name.
2026-07-17 15:08:50 -04:00
cproudlock
0c37ef9057 Dev setup: Windows-first (most devs are on VS Code / Windows)
Some checks failed
CI / backend (push) Successful in 1m45s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
PowerShell commands lead, bash equivalents in comments: venv
Activate.ps1 + execution-policy note, copy/$env:, a PowerShell
plugin-enable loop, and how the bash naming hook runs under Git Bash
(plus the pre-commit hook catching it automatically). The VS Code
Check task gets a Windows variant (venv\Scripts, bash for the .sh).
Pin shell scripts to LF in .gitattributes so a Windows checkout does
not CRLF-corrupt them into 'bad interpreter' failures.
2026-07-17 15:01:49 -04:00
cproudlock
f764c5d3e9 Add development setup guide + shared VS Code config
Some checks failed
CI / backend (push) Successful in 1m45s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
New docs/DEVELOPMENT-SETUP.md: clone-to-first-change onboarding
(Docker fast path, manual venv+Node daily driver, VS Code, the dev
loop, first-change pointer at the plugin lab, troubleshooting). Ship
.vscode/ launch/tasks/extensions so F5 debugs the backend on 5001 and
a task runs both servers; personal settings.json stays ignored. Fix
the README manual path - it ran the backend on the default 5000, but
the frontend dev server proxies to 5001, so nothing loaded; also add
the plugin upgrade-all step and a VS Code pointer.
2026-07-17 14:40:55 -04:00
cproudlock
3a3dff285e printedparts stage 17: gage-lab asset tag + print-files redesign
Some checks failed
CI / backend (push) Successful in 1m45s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
The gage lab assigns real WJRP asset numbers, so identity splits: the
internal itemcode stays auto-minted and a new optional unique
gagelabtag (migration 0003) carries the lab's number - settable on
create/edit, searchable, and resolved by the kiosk for scans and bare
keypad digits against the numeric tail of either identifier
(unique-match only). The print-files table becomes stacked revision
cards - filename with rev/current badges, one meta line, delete pinned
right - ending the horizontal scroll in that column.
2026-07-17 14:01:09 -04:00
cproudlock
6160a5142a Dark mode: dropdown arrow no longer tiles across selects
Some checks failed
CI / backend (push) Successful in 1m45s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
The dark .form-control override used the background shorthand, which
resets a select's background-repeat and position; the dark select rule
then re-added the arrow image without them, tiling it from the top
left. Use background-color in the overrides and restate
no-repeat/position on the select rule.
2026-07-17 13:37:24 -04:00
cproudlock
d75e80ce79 Plugin lab rewritten as the literal type-along walkthrough
Some checks failed
CI / backend (push) Successful in 1m45s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
The milestone workbook becomes a from-scratch guide with the actual
code inline for every core stage: models, the real migration baseline,
read routes and the list page, mutations and minting, the badge
resolver (final mode-aware form), the single-commit ledger invariant,
RBAC gating, both kiosk endpoints with the wedge-input and focus-guard
mechanics, the 1x0.5in label CSS, and the reconcile query. Field
extensions stay summarized against their tags. New section: how to
contribute a plugin through GitHub (branch, stage commits, the three
CI gates, PR expectations, review checklist, and how publication
folds PRs into release commits).
2026-07-17 13:35:02 -04:00
cproudlock
ee80d684d4 Shopfloor feed resolves employee names live when none is stored
Some checks failed
CI / backend (push) Successful in 1m45s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
Photos already resolved through the directory at read time, but names
only came from the stored employeename column - empty after a
shopdb-only import, so recertification/recognition cards showed bare
SSOs. New resolve_employee_display_name in the employees plugin
(mode-aware: self-hosted table or external HR) backs a fallback in
both the single-card and split-per-employee paths; stored names still
win when present.
2026-07-17 13:29:34 -04:00
cproudlock
bc9159742c printedparts lab: post-stage polish addendum and closing lesson
Some checks failed
CI / backend (push) Successful in 1m45s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 7s
2026-07-17 13:24:11 -04:00
cproudlock
e9235de8ec Floor-map previews honor the mount path
Some checks failed
CI / backend (push) Successful in 1m44s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
The uploaded-blueprint thumbnails on the map settings page and the
setup wizard used the raw setting value (/api/settings/map-blueprint/
...), which resolves at the server root and 404s under a subpath
mount - while the map itself resolves through blueprintUrlFor and
worked. Wrap the previews in withBase.
2026-07-17 11:43:06 -04:00
cproudlock
4f3ea2848a GE monogram avatar fallback + per-page document titles
Some checks failed
CI / backend (push) Successful in 1m44s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
CI / migrations-mysql (push) Failing after 7s
Users without a profile photo (and broken photo URLs) show the GE
monogram instead of nothing/initials - sidebar identity, employee
detail hero, and the directory list thumbs; the shopfloor cards
already did this. Document titles become
'<Facility> ShopDB - <Page>' via a router afterEach (facility from
public settings, page label from meta.title or a prettified route
name with spellings for PCs/USB/GE-Enforce/3D Printed Parts/...), so
copied links and browser tabs identify the page.
2026-07-17 11:31:38 -04:00
cproudlock
5625608bd0 Employees: external photo base URL is a setting
Some checks failed
CI / backend (push) Successful in 1m45s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
External HR Picture values are relative paths; the resolver hardcoded
/static/employees/ (which the SPA then mounts under the subpath, e.g.
/ops/static/...), but sites like WJ serve those photos from the
classic EmployeeDBAPP on another URL entirely. New setting
employee_photo_base_url (blank keeps the old behavior; a full URL like
https://host/EmployeeDBAPP/images/ passes through withBase untouched),
declared in the plugin config schema.
2026-07-17 11:16:04 -04:00
cproudlock
0cc205d25e Users: deleting a user clears their API tokens and detaches audit rows
Some checks failed
CI / backend (push) Successful in 1m45s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
Deleting any user who owned an API token or appeared in the audit log
hit the users FK and 500ed - the import's 'importer' account being the
guaranteed case (its PAT plus every audit row the import wrote).
Tokens are revoked outright; audit history is kept but detached
(userid NULL), so the trail survives the account.
2026-07-17 10:54:47 -04:00
cproudlock
bb5308bae0 printedparts: badge resolution honors the employee directory mode
Some checks failed
CI / backend (push) Successful in 1m44s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
The resolver only read the self-hosted directory table, which is empty
at sites running the external HR directory - every kiosk badge fell to
the deny policy. It now branches on employee_directory_mode like the
usb plugin: selfhosted looks up DirectoryEmployee by SSO; external
queries the HR directory via employee_connection, resolving PayNo
badges by their real PayNo column and recovering the employee's SSO.
2026-07-17 10:39:11 -04:00
cproudlock
d297c5b75d IIS runbook: app pool needs Modify on instance/
Some checks failed
CI / backend (push) Successful in 1m44s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
The plugin registry (instance/plugins.json), uploaded logos, floor
plans, item photos, and print files all write under instance/; with
the app pool at read-only, toggling a plugin in Settings surfaces as
an internal error and every upload fails. Grant Modify in step 7.3
and add the troubleshooting row.
2026-07-17 10:34:52 -04:00
cproudlock
deb6dd2162 Ignore the publication clone's _transfer bundle folder
Some checks failed
CI / backend (push) Successful in 1m43s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
CI / migrations-mysql (push) Failing after 7s
2026-07-17 10:01:34 -04:00
cproudlock
02ed88c7c5 Merge printedparts: 3D-printed parts storefront, kiosk, labels, alerts
Some checks failed
CI / backend (push) Successful in 1m44s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
The plugin-lab exemplar built end to end: catalog with photos and
print-file revisions, badge-attributed stock ledger, touch kiosk with
an open decrement-only take endpoint (decision record), 1x0.5in bin
labels, low-stock alerts to users/roles/emails, reports with a
reconcile check, per-plugin migrations 0001+0002, contract 0.13.0
(mailer + User/Role on the plugin surface).
2026-07-17 09:20:43 -04:00
cproudlock
d1357defc4 printedparts: catalog access is printedparts.view-gated
Some checks failed
CI / backend (push) Successful in 1m43s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 8s
Browsing the catalog (item list, detail, file listings) now requires
authentication plus the view permission, and the /printedparts pages
and the label print page require login. Still deliberately open: the
kiosk endpoints per the decision record, the image serve and file
download (img tags and anchor downloads cannot carry a JWT), and the
reports (product-wide jwt-optional convention). Grant
printedparts.view to the roles that should see the catalog.
2026-07-17 09:19:30 -04:00
cproudlock
96e48e0f50 printedparts kiosk: keypad and entry-panel visual polish
Some checks failed
CI / backend (push) Successful in 1m44s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 7s
The keypad becomes a proper terminal pad: fixed 3-column grid of
rounded square buttons with tabular numerals, press feedback, and
muted Clear/backspace actions. Each manual step (item number, SSO,
quantity) shares one card panel - boxed entry display with placeholder
styling, keypad, and a full-width action button.
2026-07-17 09:13:27 -04:00
cproudlock
4dfdb167d5 printedparts stage 16: kiosk touch fixes from first hands-on use
Some checks failed
CI / backend (push) Successful in 1m44s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
The tap-anywhere wedge refocus stole focus from the manual-entry field
the moment it was tapped - the handler now only reclaims focus from
dead space, never from a real control. Manual entry works without a
physical keyboard: badge entry uses the TouchKeypad (an SSO is
digits), and item lookup accepts bare digits resolved by row id - the
digits in a minted code are the id, which also keeps labels printed
under an older prefix scannable after the prefix changes.
2026-07-17 09:04:52 -04:00
cproudlock
aa4bfcd41c printedparts stage 15: print-file revision history + role-based alerts
Some checks failed
CI / backend (push) Successful in 1m44s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 7s
printeditemfiles lands as the plugin's first incremental migration
(0002 on the plugin chain - the ADR-008 payoff). Revisions are
append-only per item: upload assigns the next number, records the
uploader from the JWT, enforces an extension allowlist and a 100 MB
cap; download serves the original filename; a permission-gated delete
covers wrong-file mistakes. The detail page gains the revision table
with a current badge. Unique storedfilename is sized 191 so the index
fits MySQL's 767-byte prefix - the per-plugin chain does not apply the
core env's ROW_FORMAT hook.

Alert recipients gain roles: Role joins the 0.13.0 surface, a role
picker on the settings page, and every active member of the selected
roles is folded into the deduped recipient list.
2026-07-17 08:47:41 -04:00
cproudlock
26b6b6b32f printedparts: Parts Kiosk link in the sidebar Displays section
Beside Shopfloor Dashboard and TV Slideshow, opening in a new tab and
shown only while the plugin is enabled - kiosk-style pages get
launched from the Displays group, not the Information nav.
2026-07-17 08:38:57 -04:00
cproudlock
eab225e1e6 printedparts stage 14: retire/restore in the UI, dashless item codes
Some checks failed
CI / backend (push) Successful in 1m42s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
CI / migrations-mysql (push) Failing after 7s
Retire button with confirmation on the detail page (item leaves the
storefront and the kiosk rejects its code; ledger history and label
survive), Restore on retired items, and an Include-retired list toggle
with a badge. Restore is its own permission-gated POST - the generic
update still cannot flip isactive. New codes mint as WJRP0042 style
without the dash; existing codes are immutable bin labels and keep
their form.
2026-07-17 08:35:31 -04:00
cproudlock
a8a6baf979 printedparts stage 13: pick alert recipients from shopdb users
Some checks failed
CI / backend (push) Successful in 1m42s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
Contract 0.13.0 puts the User model on the plugin surface. The
settings page gains a checkbox picker over the user list; selected
users receive low-stock alerts at their account email, merged and
deduped with the free-text address list, inactive accounts skipped,
site alert_recipients still the fallback when both are empty.
2026-07-17 08:30:04 -04:00
cproudlock
427eb0de8c printedparts stage 12: admin settings page + settings-rail card
Some checks failed
CI / backend (push) Successful in 1m42s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
CI / migrations-mysql (push) Failing after 7s
PrintedPartsSettings edits the four plugin settings (code prefix,
default threshold, kiosk badge policy, alert recipients) through the
core settings API; the route rides the plugin's router file and the
settings shell nests it into the rail; get_settings_cards contributes
the catalog card while the plugin is enabled.
2026-07-17 08:25:33 -04:00
cproudlock
df918ed38f printedparts stage 11: low-stock email alerts on threshold crossing
Some checks failed
CI / backend (push) Successful in 1m43s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
Contract 0.12.0: send_email/send_alert join the plugin surface (the
mailer was core-only), PLUGIN-HOOKS and status docs updated, manifest
pins the new floor. The alert fires inside _ledger_write only when a
decrement CROSSES the item's threshold - one alert per depletion,
rearmed by restocking above - and is best-effort after the commit so
mail trouble can never fail a take. Recipients come from
printedparts_alert_email, falling back to the site alert_recipients.
on_enable re-seeds settings idempotently so existing installs pick up
new keys. Crossing/rearm semantics proven by test.
2026-07-17 08:15:50 -04:00
cproudlock
fc0d48a6a7 printedparts stage 10: closeout - lab guide rewritten from the real build
Some checks failed
CI / backend (push) Successful in 1m42s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
The lab is now a build-along mirroring what actually happened: ten
stages, each with the goal, the divergences, a see-it-work check, and
the errors genuinely hit while building (empty Migration error from a
broken model import, the migration-guard KeyError, the missing Lucide
icon, nested-app-context test writes, Decimal sums, and the authz
sweep catching the deliberately open kiosk take). That last one gets
its explicit EXEMPT_ENDPOINTS entry with a pointer to the decision
record - the net stays, the exception is reviewable. Full suite: 993
backend tests, 49 vitest, frontend build, naming hook, all green.
2026-07-17 08:11:36 -04:00
cproudlock
b68e927ef6 printedparts stage 9: reports - stock w/ reconcile, consumption, by-person
Some checks failed
CI / backend (push) Failing after 1m42s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
Three jwt-optional endpoints with ?format=csv, merged into the reports
hub via get_reports while the plugin is enabled. The stock report's
ledgerdelta column is the reconcile check: 0 for every item whose
stock moved through the ledger, nonzero for anything that bypassed it
(the hand-seeded dev rows demonstrate the catch). MySQL SUM returns
Decimal - cast to int or the delta serializes as a string.
2026-07-17 08:04:09 -04:00
cproudlock
6439d1ccd9 printedparts stage 8: 1x0.5in bin labels
Some checks failed
CI / backend (push) Failing after 1m42s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
CI / migrations-mysql (push) Failing after 7s
New public print view at /print/printedparts-labels following the
plugin-owned USB label precedent: multi-select with per-item copies,
CODE128 of the item code via JsBarcode (a QR at this size is at the
edge of scanner tolerance), one label per page on 1in x 0.5in roll
stock via a new @page size. The Detail page's Bin Label button
preselects its item through ?item=<id>; the list header gains a batch
Print Labels button.
2026-07-17 08:00:35 -04:00
cproudlock
6ed3da1b64 printedparts stage 7: the kiosk - scan bin, scan badge, keypad, take
Some checks failed
CI / backend (push) Failing after 1m43s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
Two open endpoints: an item lookup by scanned code and the take POST -
the product's first unauthenticated write, held to the decision
record's bar (decrement-only, badge-attributed server-side, bounded,
physically rate-limited; justification in the plugin README). The
/parts-kiosk route is a full-screen no-auth view beside /shopfloor: a
hidden always-focused input consumes keyboard-wedge scans for
whichever step is active, TouchKeypad (net-new 3x4 grid) takes the
quantity, and a success screen resets after a few seconds. Manual
type-in fallbacks cover damaged labels. Kiosk test proves open access,
the over-take guard, the badge policy, and cache==ledger afterward.
2026-07-17 07:49:13 -04:00
cproudlock
d6a78a72ff printedparts stage 6: RBAC - declared permissions gate every mutation
Some checks failed
CI / backend (push) Successful in 1m42s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
get_permissions declares view/create/edit/delete/restock (seeded on
install/enable and by flask seed permissions); every write route adds
require_permission on top of jwt_required. New test proves
authentication alone is not authorization: a role-less member gets
403 where an admin succeeds.
2026-07-17 07:42:07 -04:00
cproudlock
6dfc8906c4 printedparts stage 5: the ledger - restock/adjust with badge attribution
Some checks failed
CI / backend (push) Failing after 1m42s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
Badge resolver copied from the USB contract (SSO digits, 0<digits>BZ
PayNo wrap) with names from the employees directory and the
unknown-badge policy setting; deliberately copied rather than
cross-imported so the contract test stays green. Restock and adjust
write the ledger row and move the cached quantity in one commit -
the single-commit invariant every write path must use. Adjust
requires a reason and refuses to drive stock below zero. Detail page
gains Restock/Adjust modals. Seven tests cover minting, the
cache==ledger invariant, badge shapes, policy toggle, and auth.
2026-07-17 07:41:18 -04:00
cproudlock
cb367a38f9 printedparts stage 4: catalog mutations, item photos, detail + form
Some checks failed
CI / backend (push) Failing after 1m40s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
POST/PUT/DELETE for items: create mints the itemcode from the
configured prefix plus the flushed row id, update refuses
quantityonhand (ledger-managed - restock/adjust arrive next stage),
delete soft-retires. The image upload/serve/delete trio replicates the
models.py pattern into instance/printedpartsimages/ with a public GET.
PrintedItemDetail follows the unified detail skeleton (hero photo,
info list, transaction history table); PrintedItemForm covers
create/edit plus photo management on edit.
2026-07-17 07:36:54 -04:00
cproudlock
d1c844d533 printedparts stage 3: read API + list page (first visible win)
Some checks failed
CI / backend (push) Successful in 1m39s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
GET /items (paginated, search across code/name/description/bin,
lowstock filter) and GET /items/<id> with recent transactions, both
open reads. printedpartsApi client, router file repointed at the
renamed views, PrintedItemsList with image thumbs and a red/green
quantity badge against the per-item threshold. Nav entry '3D Parts'
with a new 'box' Lucide icon mapping (the sidebar renders nothing for
unknown icon names - lab gotcha).
2026-07-16 17:10:42 -04:00
cproudlock
f5cfac33b4 printedparts stage 2: models, real 0001 baseline, tables live
Some checks failed
CI / backend (push) Successful in 1m39s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
PrintedItem (catalog: code, name, image, cached quantityonhand,
per-item threshold, bin) and PrintedItemTransaction (the ledger:
signed quantity change attributed to a badge-resolved employee).
Both registered in PLUGIN_TABLE_OWNERS; 0001 is a post-cutover real
baseline. The migration-guard test learns the new expected head.
Routes are a placeholder ping until the next stage - the scaffold's
list route imported the deleted scaffold model, which surfaces as an
empty 'Migration error' because the alembic env imports the models
package.
2026-07-16 16:57:21 -04:00
cproudlock
8dd1fadeca printedparts stage 1: scaffold, no AssetType, manifest per spec
Some checks failed
CI / backend (push) Failing after 1m38s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
CI / migrations-mysql (push) Failing after 8s
flask plugin new output, minus the scaffold's AssetType seeding:
printed parts are quantity-based consumables, not ADR-001 assets.
on_install seeds the three plugin settings instead. Manifest pins
core >=0.11.0, depends on employees (badge name resolution), ships
disabled until a site opts in.
2026-07-16 16:44:54 -04:00
cproudlock
a0b92f7b5e printedparts lab: set expectations up front
Some checks failed
CI / backend (push) Successful in 1m39s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 7s
State that this is a bundled plugin whose frontend and three small core
edits land in this repo, list the three deliberate divergences from the
scaffold before the learner hits them, suggest a per-milestone solution
branch for instructors, and point out the earliest visible win (wire
the bare list page as soon as the GET endpoint works).
2026-07-16 16:37:28 -04:00
cproudlock
6362cef699 printedparts docs: record the open-write kiosk decision; defer the dashboard widget
Some checks failed
CI / backend (push) Successful in 1m38s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 7s
The kiosk take endpoint is the product's first unauthenticated
mutation; spell out the acceptance criteria (decrement-only, badge
attributed, bounded, physically rate-limited) so future open-write
endpoints meet the same bar. The dashboard-widget milestone is marked
optional: get_dashboard_widgets predates the ADR-010 data-only
renderers and needs a core component to render.
2026-07-16 16:35:17 -04:00
cproudlock
d99de002bf printedparts plugin: design proposal + hands-on development lab
Some checks failed
CI / backend (push) Successful in 1m39s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
Design for a 3D-printed-parts storefront: item catalog with images and
quantity on hand, a transaction ledger attributing every take/restock/
adjust to a badge-scanned employee, an unauthenticated touch kiosk
(scan bin barcode, scan badge, keypad quantity), 1x0.5in CODE128 bin
labels, and stock/consumption/by-person reports.

The lab guide walks a developer through building it in seven
checkpointed milestones, reusing the USB badge contract, the
measuringtools migration baseline, the models-image upload trio, and
the open kiosk-endpoint precedents.
2026-07-16 16:33:19 -04:00
cproudlock
eed947b207 Forward real client IPs through waitress trusted-proxy flags
Some checks failed
CI / backend (push) Successful in 1m39s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
The X-Forwarded-For rewrite rule alone is not enough: waitress 2+
strips forwarded headers from untrusted proxies by default, so the app
still saw 127.0.0.1 with the rule active. Trust the loopback proxy and
consume x-forwarded-for on the waitress command line; waitress then
rewrites remote_addr to the real client. Runbook gains the
allowedServerVariables unlock (500.52) and both troubleshooting rows.
2026-07-16 15:27:45 -04:00
cproudlock
f5f67172aa Scope dark-mode form and notification styles to the explicit theme
Some checks failed
CI / backend (push) Successful in 1m39s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
The form-control and notification-tint blocks applied on the OS
prefers-color-scheme alone, so a machine in OS dark mode leaked dark
widget styling into the app's explicit light theme - dropdown options
rendered near-black on black. The theme store always stamps data-theme
at startup, so scope these rules to [data-theme=dark]. Dropdown options
in dark mode use the solid card background instead of the text color.
2026-07-16 15:11:35 -04:00
cproudlock
97f7dfb0de Export script: emit bundle into the pxe-images github transfer folder
Some checks failed
CI / backend (push) Successful in 1m39s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
CI / migrations-mysql (push) Failing after 7s
2026-07-16 14:14:44 -04:00
cproudlock
3ba09ac9f3 Fix bit(1) import coercion and subpath login redirect; add GitHub export script
Some checks failed
CI / backend (push) Successful in 1m40s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 7s
Loader: bool() on pymysql bit(1) bytes is always true - isinstallable
and isshopfloor imported as 1 for every row; route through _truthy_bit.
The employee source DB is now optional (shopdb-only imports).

Frontend: under a subpath mount the 401 interceptor stored the browser
path (mount base included) as the login redirect and the router applied
its base again (/ops/ops). New stripBase() keeps redirects base-free.

tools/export-github.sh automates the publication flow: prune + scrub +
commit into ~/projects/shopdb-flask-pub and emit a transfer bundle.
2026-07-16 14:08:43 -04:00
cproudlock
f16d2289ff Remove stray import_apps.py (superseded by the import API + wjf loader)
Some checks failed
CI / backend (push) Successful in 1m44s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
2026-07-13 19:53:51 -04:00
cproudlock
99cac87d9a README: refresh to current product surface; retire direct-DB migration guide
Some checks failed
CI / backend (push) Successful in 1m41s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
Overview and plugin list cover all twelve plugins (geenforce, network
subnets, measuring tools, warranty, USB); naming examples use living
tables/columns instead of retired pctypes/isvnc; API params match the
implementation (perpage, dir, assettype); import section points at the
IMPORT-API surface and the wjf reference loader. DATA_MIGRATION_GUIDE
is now a pointer stub (its direct-DB approach is superseded).
2026-07-13 19:49:34 -04:00
cproudlock
7d7862f4b5 Neutral wording in import docs; portable tool output paths
Some checks failed
CI / backend (push) Successful in 1m41s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
Import-surface docs and docstrings describe the automation as a
migration script; status-doc references in CHANGELOG/ADR-009/ROADMAP
point at repo files. Screenshot/verify tools write to /tmp/shopdb-shots
(created on import) instead of a machine-specific directory.
2026-07-13 16:23:24 -04:00
cproudlock
6010f01de1 Support subpath IIS deployment as a second install method
Some checks failed
CI / backend (push) Successful in 1m41s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
The app can run as an IIS Application under an existing site
(e.g. https://host/ops/) instead of its own site + port:

- frontend: vite base via VITE_BASE_PATH; router history, axios
  baseURL, and root-absolute asset/route paths resolve through
  utils/basePath.js withBase()
- backend: MOUNT_PATH (env or .env) wraps the app in a WSGI
  middleware that shifts the prefix into SCRIPT_NAME, so one knob
  serves API + SPA under the mount
- docs: INSTALL-WINDOWS-IIS.md section 7b runbook + troubleshooting
  rows; DEPLOY-WINDOWS-IIS.md pointer; commented examples in
  deploy/windows/web.config and .env.example

Root deployment unchanged (MOUNT_PATH unset, base '/'). Also folds
two stray root-absolute callers into the shared plumbing
(MachineForm relationship-types fetch, reports CSV window.open).
2026-07-13 16:11:12 -04:00
cproudlock
69dd6d0abe Warranty form: make Vendor a strict dropdown of the vendor catalog
Some checks failed
CI / backend (push) Successful in 1m39s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
Replace the free-text/datalist vendor field with a plain select populated from
/api/vendors, so a warranty vendor is always one of the site's known vendors. An
existing warranty's vendor is kept selectable even if it is absent from the
catalog, so editing never blanks it.
2026-07-13 15:02:59 -04:00
cproudlock
fba2fa05b4 Warranty form: vendor combobox + clearer "Lookup source" label
Some checks failed
CI / backend (push) Successful in 1m38s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Has been cancelled
- Vendor is now a datalist combobox seeded from the site vendor catalog
  (/api/vendors), so users pick a known vendor instead of retyping it, while
  still allowing free text. Kept as a string, not an FK, to keep the warranty
  plugin decoupled from core.
- Renamed the "Provider" select to "Lookup source" with helper text. It was
  confusingly synonymous with Vendor; it actually means where coverage data
  comes from (manual entry vs a maker's warranty API that supports Refresh).
2026-07-13 15:00:53 -04:00
cproudlock
9e2544a687 wjf loader: import active machines only (isactive=1)
Classic ASP keeps retired machines in the machines table as history (isactive=0);
every other stage already filters isactive=1, but the assets hub, metrology,
locations and the verify count read machines unfiltered, so ~240 retired units
(incl. all G-prefix hostnames and 61 dead metrology PCs that each synthesized a
phantom measuring tool) landed as live assets. Add the isactive=1 filter to
those four queries. Downstream stages resolve via the id crosswalk, so warranties
/comms/relationships/installs for retired machines now drop automatically.
2026-07-13 15:00:45 -04:00
cproudlock
760b00f4d1 Warranties: fix N+1 slowness, filter alignment, and Covers hover
Some checks failed
CI / backend (push) Successful in 1m39s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 7s
- Perf: list_warranties did a db.session.get(Asset) per link per warranty
  (~1.8s for the full list). Eager-load the links and batch-fetch every linked
  asset in one query -> ~0.19s.
- Filters: the "Status" label wrapped its select onto a second line, so the
  dropdown sat above the search box; keep the label inline so they align.
- Covers: each asset chip now shows the asset name (often the hostname/alias) on
  hover, keeping the machine number as the label.
2026-07-13 14:49:36 -04:00
cproudlock
4a74a1a405 WJ import loader: mark setup_complete (a full import is the setup)
Some checks failed
CI / backend (push) Successful in 1m39s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
A freshly-imported DB left setup_complete unset, so the admin was bounced into
the first-run wizard even though the instance is fully populated. The harness now
sets setup_complete=true (it already mints the admin), so an imported instance
goes straight to the app.
2026-07-13 14:41:23 -04:00
cproudlock
9c909af66d WJ import loader: metrology PCs are computers that control a synthesized tool
Some checks failed
CI / backend (push) Successful in 1m39s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
Routing by pctype wrongly swept ~105 metrology PCs (CMM/Genspect/Keyence/Wax)
into measuringtools - but a PC that drives an instrument is still a computer;
the physical CMM/gauge is the tool. Dropped the pctype override: those PCs now
import as computers (measuringtools drops to the 48 real instrument rows).

Classic has no separate tool row for a metrology PC, so they'd be orphaned. New
metrology stage synthesizes a measuring-tool asset per metrology PC (typed by
its pctype: CMM / Form Tracer / Vision System / Genspect) and a Controls
relationship PC -> tool, mirroring what the runtime collector does.

Result: computers 663->751, measuringtools = 48 real + 88 synthesized (each
linked to its controlling PC), 88 new Controls relationships, no orphans.
2026-07-13 14:38:00 -04:00
cproudlock
c9641a3c62 CLAUDE.md: refresh Current State (tests, plugins, migration head, import docs)
Some checks failed
CI / backend (push) Successful in 1m39s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
808->966 tests (+ the migrations-mysql CI job), 11->12 bundled plugins (add
geenforce), Alembic head 7d24->7d25 (32 migrations; env.py sql_mode note for
strict MySQL 8), date to 2026-07-13, and a legacy-import pointer
(IMPORT-API/ADOPTION/PILOT-DEPLOY + the WJ reference loader).
2026-07-13 14:30:00 -04:00
cproudlock
a736ed541e Fix Actions column border: keep td.actions as a table-cell
Some checks failed
CI / backend (push) Successful in 1m38s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 8s
.actions sets display:inline-flex (correct for a button container), but it was
applied directly to <td class="actions">, pulling the cell out of the table row
box so its bottom border rendered ~1px off from the other columns. Override
td.actions back to display:table-cell and space multiple buttons with a margin
instead of the flex gap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 14:23:28 -04:00
cproudlock
58af5afda2 List rows click through to the item; drop the redundant VLANs tab
Some checks failed
CI / backend (push) Successful in 1m39s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Has been cancelled
Row-click: the whole table row now navigates to the item detail (machines, PCs,
printers, network devices, measuring tools, applications), matching the Networks
list. The actions cell is @click.stop so View/Edit/Delete still work
independently; a shared .clickable-row style gives the cursor + hover.

Network hub: drop the VLANs tab - a subnet belongs to a VLAN (each Networks row
already shows its VLAN) so a sibling tab was redundant; VLAN naming stays in
Settings. Hub is now Devices | Networks.

Verified: row-click navigates on machines/pcs/network; hub shows two tabs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 14:21:34 -04:00
cproudlock
8528617037 Network: consolidate into one tabbed hub; subnet devices span all asset types
Some checks failed
CI / backend (push) Successful in 1m39s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
Replaces the two flat "Network Devices" + "View Networks" nav entries with a
single "Network" entry opening a tabbed hub: Devices | Networks | VLANs
(NetworkHub renders the existing device list, the subnet browse, and the VLAN
list; VLANs is now reachable outside Settings). /network -> hub; /networks
redirects to the Networks tab; subnet detail stays at /networks/:id.

Subnet "Devices on this network" now matches ANY asset whose primary IP falls in
the CIDR (PCs, printers, machines, measuring tools - not just network devices),
computed on the core Communication + Asset tables; each row links to its typed
detail (extension id resolved lazily/guarded per plugin). Fixes the empty list -
printers and PCs carry IPs and now appear (e.g. 35 devices on 10.80.92.0/24).

Also: subnet-browse search uses the standard form-control styling; dropped the
redundant per-tab page header.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 14:15:51 -04:00
cproudlock
dd541fba0a Add "View Networks": front-facing subnet browse + detail with attached devices
Some checks failed
CI / backend (push) Successful in 1m38s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
CI / migrations-mysql (push) Failing after 7s
Subnets previously lived only under Settings, easily confused with the Network
Devices asset list. Add a front-facing browse + detail:

- Nav: rename "Network" -> "Network Devices"; add "View Networks" (subnets), both
  under Assets (network plugin get_navigation_items; frontend fallback matched).
- /networks (SubnetsBrowse): all subnets with name / CIDR / type / VLAN / notes,
  searchable, row-click to detail.
- /networks/:id (SubnetDetail): the subnet (CIDR, network address, type, VLAN,
  gateway, notes) plus the network devices whose primary IP falls inside its
  CIDR - get_subnet now computes that membership in Python (a device's IP lives
  in a Communication row, so it is not a plain SQL join).

Verified on the import DB: 37 networks list (real WJ subnets), detail renders.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 14:06:47 -04:00
cproudlock
5f51aa2383 Detail pages: give the relationships card bottom spacing (was merging with Notes)
.relationships-section had card styling but lacked the margin-bottom +
break-inside:avoid that .section-card has, so it sat flush against the Notes
card below it and looked like one merged card (and could split across a multicol
break). Add both to match section-card. Affects every asset detail page.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 13:53:37 -04:00
cproudlock
a6ebec5118 Employee profile: cap USB checkout history with a show-more toggle
Some checks failed
CI / backend (push) Successful in 1m39s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
USB Checkout History rendered every record unbounded (long table for heavy
users), inconsistent with the recognitions list right above it. Add the same
limit (10) + "Show N more" / "Show less" toggle recognitions use. Client-side
only; no API change (per-user history is small at current scale).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 13:49:33 -04:00
cproudlock
6d843e46e1 Applications: show related knowledge-base articles on the app detail
Some checks failed
CI / backend (push) Successful in 1m39s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
Classic ShopDB listed an application's related KB articles on its page; the flask
app detail didn't, even though KB articles carry an appid FK. get_application now
returns a knowledgebase list (KB rows linked by appid; lazy + plugin-guarded so
core stays decoupled), and ApplicationDetail.vue renders a Knowledge Base section
linking each article.

Verified: an app returns its linked KB (e.g. 77 articles) and the section renders.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 13:44:23 -04:00
cproudlock
49d311b56e WJ import loader: map application link + documentation path
The applications stage dropped applicationlink and documentationpath, so the app
detail's "Launch Application" and "Documentation" links were always empty. Map
both from the classic applications table.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 13:44:22 -04:00
cproudlock
b2dea82ac9 Global search: find employees in selfhosted mode, not just external HR DB
_search_employees always queried the external HR database (employee_connection),
so a site running the selfhosted employee directory (the app-owned
directoryemployees table) got zero employee results - searching an SSO or name
found nothing, and there was no way to reach the employee profile. Made it
mode-aware via the employee_directory_mode setting: selfhosted -> query the
DirectoryEmployee table (lazy, plugin-guarded); external -> the HR DB as before.

Verified: SSO 210009518 -> Jeff Pierce -> /employees/210009518; name "Pierce"
-> Pierce Cox, Andy Pierce, Jeff Pierce.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 13:39:18 -04:00
cproudlock
2229db4a70 Notifications: summarize multi-person names in the calendar event title
Some checks failed
CI / backend (push) Successful in 1m39s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
A recognition/training notification for many people prefixed the calendar title
with the entire roster ("Name1, Name2, ... , Name20: description"), burying the
description. to_calendar_event now shows "First Person +N" when more than one
person is listed; single-person titles are unchanged and the detail popup still
shows the full employeename. Fixes the cluttered month grid, especially after
the loader began resolving employee SSOs to names.

31 notification/calendar tests pass; naming green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 13:32:39 -04:00
cproudlock
c4690da262 WJ import loader: resolve notification employee SSOs to names
Some checks failed
CI / backend (push) Successful in 1m38s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
CI / migrations-mysql (push) Failing after 7s
Notifications imported with only employeesso, leaving employeename null - the
model displays "employeename or employeesso", so recognition/training cards
showed a bare SSO instead of a name. Build an SSO -> "First Last" map from the
employee source and populate employeename (comma-separated SSOs -> joined
names). SSOs not in the directory (former/non-WJF) stay null and fall back to
the SSO, as before.

Verified on the import DB: 261 notifications re-imported, names resolved
(Brandon Saltz, Jon Kolkmann, ...).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 13:25:12 -04:00
cproudlock
35690bd169 Migrations: relax session sql_mode so the chain runs on strict MySQL 8
Some checks failed
CI / backend (push) Successful in 1m38s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
CI / migrations-mysql (push) Failing after 8s
Found by test-deploying on a Windows + MySQL 8.0 VM: migration 7a01 seeds the
canonical relationship types with a raw INSERT that omits the NOT-NULL
createddate/modifieddate columns (the ORM supplies those via Python defaults at
runtime, but a raw migration INSERT does not). MySQL 5.x's lax default sql_mode
accepted it; strict MySQL 8 rejects it with 1364 "Field 'createddate' doesn't
have a default value", so a fresh `flask db upgrade` died at 7a01. Dev runs
MySQL 5.6, so this never surfaced locally.

migrations/env.py now sets the migration session sql_mode to
NO_ENGINE_SUBSTITUTION (dropping STRICT_TRANS_TABLES) for the migration run
only - the app's own runtime connections keep their mode. Makes the whole chain
portable across MySQL versions. Guarded for non-MySQL (sqlite tests).

68 migration/smoke tests pass; fresh upgrade to head verified on MySQL.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 13:06:07 -04:00
cproudlock
83a4867d1b Add production pilot runbook (stand up + import + verify + cutover)
Some checks failed
CI / backend (push) Successful in 1m39s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
docs/PILOT-DEPLOY.md ties the generic per-site deploy (DEPLOY.md) to the legacy
import: pre-flight, stand up an empty instance, enable all plugins (incl usb),
load the three classic dumps into scratch DBs, run the WJ loader against the
pilot DB, verify (row-count audit + UI spot-check checklist), a parallel-run
window, cutover, rollback, and post-cutover (backups, photos, GE-Enforce).
Includes the expected import magnitudes from the dev run.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 12:21:40 -04:00
cproudlock
009ac9f9ef Import docs: adoption playbook + superseded-mappers note + loader status
Some checks failed
CI / backend (push) Successful in 1m39s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
Follow-up to the mapper retirement (the new docs missed the prior commit's
staging). Adds docs/IMPORT-ADOPTION.md (two-layer import story + stage/crosswalk
guidance), scripts/migration/README.md (dir superseded, points at the API +
loader), and updates the WJ loader README to complete status.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 12:18:56 -04:00
cproudlock
474f245ae7 Retire stale legacy-import mappers; add adoption playbook
Some checks failed
CI / naming (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / migrations-mysql (push) Has been cancelled
CI / backend (push) Has been cancelled
Cleanup after the reference loader (scripts/site_imports/wjf/) proved out.

Removed the drifted direct-SQL migrators - migrate_assets/communications/
notifications/usb.py, run_migration.py, verify_migration.py, and
scripts/import_from_mysql.py. They targeted a nonexistent equipment table, the
retired Machine model, and columns that no longer exist; nothing imported them.
scripts/migration/README.md now points at the import API + the site loader.
Kept the one-time SQL fixups (fix_legacy_schema.sql, one-offs/).

Added docs/IMPORT-ADOPTION.md: the two-layer import story (stable IMPORT-API
contract + per-site loader), stage-ordering + crosswalk guidance, the
agent-assisted mapping path, and what the WJ loader demonstrates. Updated the
loader README to complete status (all 15 stages, final counts).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 12:18:14 -04:00
cproudlock
af9afc15c5 WJ import loader: printers stage + Controlled-By relationship reversal
printers: printers come from the printers TABLE (not the machines hub), so a
dedicated stage - assetnumber synthesized PRN-{printerid}, IP folded via the
create route, host machineid resolved to a location when it is a LocationOnly
row. Skips inactive. 50 of 56 imported.

relationships: a "Controlled By" edge now flips to the forward Controls
direction (source/target swapped, mapped to the Controls type) instead of
importing a redundant inverse type.

Final fresh full run: 983 assets (933 machines-hub + 50 printers), zero endpoint
errors, all 15 stages.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 12:15:41 -04:00
cproudlock
99ad6c0ebb WJ import loader: tail stages (locations, relationships, subnets, usb, verify)
Some checks failed
CI / backend (push) Successful in 1m38s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
CI / migrations-mysql (push) Failing after 7s
Completes the loader end to end. Verified on a fresh scratch target, zero
endpoint errors:

- locations: the 24 islocationonly rows -> core Locations (crosswalk machineid
  -> locationid).
- relationships: 93 active edges imported (206 of 299 dropped because an
  endpoint became a Location / was skipped / dedup-lost); types folded onto the
  seeded canonical set; dedup on (source,target,type).
- subnets: 37 (full CIDR reconstructed as INET_NTOA(ipstart)+suffix; VLANs
  lookup-or-create by number; 3 duplicate CIDRs first-wins-skipped).
- usb: 18 cmmc devices + 232 check-in/out events, paired with per-device open
  state so unpaired log rows do not 400. Needs the usb plugin enabled + usb
  directory mode selfhosted.
- verify: source-vs-target row-count audit (assets 1167->933 by the skip rules,
  applications 121=121, employees 415=415, KB 342->341).

Full pipeline default runs all 14 stages in order. NOTE: the import target needs
every bundled plugin enabled (usb ships disabled in this dev registry - enable
it before importing).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 12:05:33 -04:00
cproudlock
40a89b360a WJ import loader: route LocationOnly by the islocationonly bit, not machinetypeid
Some checks failed
CI / backend (push) Successful in 1m38s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
CI / migrations-mysql (push) Failing after 8s
Of the 158 machinetypeid=1 rows, only 24 carry the islocationonly bit (real
named areas: DT Office, IT Closet, Materials, ...). The other 134 are active,
modelled shop machines just left untyped - routing all 158 to Locations dropped
those 134 real assets. Route on the bit instead; the 134 untyped rows import as
machines with a null subtype (machinetypeid=1 is not a real machine subtype, so
catalog skips seeding one).

Also process asset routes in richness order (computer > measuringtool > network
> machine) so on a duplicate machinenumber the PC - which carries installs + IP
a bare untyped machine does not - wins first-come.

Result on the scratch target: 933 assets (computer 663, machine 76, network 58,
measuringtool 136), 24 locations (was mis-routing 158), installs 850 (was 653 -
PCs no longer lose their numbers to bare machines), warranties 464, comms 461.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 11:55:25 -04:00
cproudlock
6d79e469fa WJ import loader: dependent-entity stages (comms, apps/installs, warranty, notif, KB)
Some checks failed
CI / backend (push) Successful in 1m38s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
All consume the machineid->assetid crosswalk from the assets hub. Verified
against the scratch target, zero endpoint errors:

- communications: 435 primary IPs folded onto assets. No bulk endpoint exists,
  so this is the plan's documented direct-ORM gap (reads the source
  communications table where comstypeid=1 AND isprimary=1, not machines.ipaddress1
  which is empty).
- applications: supportteams 45, applications 121 (colliding names dedup via the
  unique-appname 409-resolve), appversions 47, installs 653 (machineid ->
  assetid -> computerid; only computer assets take installs).
- warranties: 424 linked, vendor hardcoded Dell (source has none).
- notifications: types 6, notifications 261 (2099 sentinel endtime clamped).
- knowledgebase: 341 (appid resolved through the applications name map).

Inactive rows skipped everywhere per the decisions. Remaining loader stages:
locations (the 158 LocationOnly rows), relationships (301 active edges),
subnets/VLANs, usb (cmmc pairing), verify.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 11:43:02 -04:00
cproudlock
6c4bd20e01 WJ import loader: catalog + assets-hub stages (the machineid->assetid crosswalk)
Some checks failed
CI / backend (push) Successful in 1m39s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
Adds the keystone stages to the reference loader.

catalog: seeds modeltypes (from classic machinetypes + category), the per-plugin
asset subtypes routed by machinetype (machines/network/measuringtool types),
computer subtypes (from pctype), the 5-row controllertypes vendor/model split,
and the models catalog - each with a persisted legacy->new crosswalk. Verified:
modeltypes 31, computertypes 12, models 118.

assets (the hub): fans classic machines out to the right endpoint by
machinetypeid (+ the pctype metrology override), applying the resolved
decisions - assetnumber = machinenumber else hostname, skip 9999, skip duplicate
machinenumbers, LocationOnly/printer/USB routed out. Persists the machineid ->
assetid crosswalk every downstream stage needs. Verified against a fresh scratch
target: 884 assets (computer 623, machine 68, network 58, measuringtool 135),
zero endpoint errors, idempotent re-run (stays 884). Skips: location 158, dup 69,
other 53, 9999 1.

Harness now runs each plugin's idempotent on_install so the AssetType rows exist
(a DB built with plugin upgrade-all instead of a fresh install lacks them, and
the create routes 500 without them). 409-resolve lookups page through per_page.

Remaining stages: communications (primary IP fold - source is the communications
table, not machines.ipaddress1), applications/installs, warranties, notifications,
KB, subnets/VLANs, usb, verify.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 11:37:11 -04:00
cproudlock
b11a6f26d8 WJ legacy-import reference loader: harness + reference/employees stages
Some checks failed
CI / backend (push) Successful in 1m38s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 8s
Layer 2 of the import design (see the memory + scratchpad/IMPORT-PLAN.md): a
SITE-SPECIFIC reference loader that maps WJ's classic-ASP schema onto the
maintained, schema-agnostic IMPORT-API contract. Other sites copy the pattern
against their own source DB; nobody runs this loader as-is.

Harness (scripts/site_imports/wjf/harness.py): builds the app against the
current DATABASE_URL (point it at a throwaway import DB), mints an unscoped
admin PAT in-process, and drives the real import endpoints through the app test
client with Authorization: Bearer + X-Import-Mode - exercising the same
routes/authz/validation an HTTP client would, no running server needed.
Read-only pymysql access to the three scratch source DBs; legacy-id -> new-id
crosswalks persist to JSON so a crashed run resumes and later stages resolve FKs.

Stages implemented + verified idempotent against a fresh scratch target
(shopdb_flask_import): reference (vendors 46, businessunits 13, operatingsystems
11) and employees (directory 415, re-run updated-not-duplicated). Remaining
stages (models, applications, assets hub + crosswalk, dependents, network, usb,
verify) are stubbed with the same shape; README documents the adoption playbook.

idmap.json is generated state (gitignored).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 11:20:58 -04:00
cproudlock
012718f0fd Import-API: preserve app-version timestamps in import mode
Some checks failed
CI / backend (push) Successful in 1m40s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
POST /api/applications/{id}/versions now calls apply_import_timestamps so an
imported version's original dateadded/releasedate survives (was skipped, unlike
the app + install endpoints). No-op outside import mode.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 11:12:03 -04:00
cproudlock
be1ea29403 Import-API hardening: network device IP endpoint + communicationtypes seeder
The legacy-import surface (docs/IMPORT-API.md) is the schema-agnostic contract
every adopting site targets; these close two gaps found while mapping the WJ
classic import.

Network device IP: POST/PUT /api/network now accept an `ipaddress` and
materialize a primary Communication (mirroring the printer route), and GET
(list + detail + create/update result) surface it. Previously a network
device's IP - which lives in the communications table, not on the extension -
had no HTTP import path at all.

communicationtypes seed: `flask seed reference-data` now seeds the eight
canonical communication types (IP/Serial/Network_Interface/USB/Parallel/VNC/
FTP/DNC), which IMPORT-API.md already documents as a prerequisite. The IP type
must exist before any asset import so printer/network routes can attach an IP.
There is no CRUD endpoint for these, so seeding is the only path.

Tests: network create/update IP upsert + GET surfacing + seed creates IP type.
204 targeted tests pass; naming + pyflakes green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 11:11:01 -04:00
cproudlock
1c6c7ba14b DB review fixes: drop redundant indexes + dead column, add CI MySQL-upgrade job
Some checks failed
CI / backend (push) Successful in 1m39s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
CI / migrations-mysql (push) Failing after 8s
From the database review (verdict: sound-with-minor-issues). Applies the
actionable findings.

Redundant indexes: five non-unique secondary indexes duplicated a named idx_*
or a unique index on the same column - ix_communications_assetid,
ix_computers_hostname, ix_networkdevices_hostname, ix_printers_hostname (each
shadowing an idx_*), and idx_usb_serial (shadowing the serialnumber unique
index). Removed the redundant index source from the models (column index=True /
the extra db.Index) and added core migration 7d25 dropping the live duplicates.
The unique ix_*_assetid indexes are kept (they enforce assetid uniqueness).

Dead column: usbcheckouts.machineid was a NOT NULL soft-ref to the retired
machines table storing sentinel 0 (ADR-001). Dropped from the model + the
machineid=0 literal in selfhosted checkout; usb plugin migration 0002 drops it
live (downgrade restores it default 0).

Index: notifications.businessunitid (filtered by the shopfloor feed) was
unindexed; added index=True + notifications migration 0002.

CI: new migrations-mysql job proves the real multi-site deploy path - fresh
`flask db upgrade` + per-plugin install on utf8mb4 MySQL from empty, asserting
table count + charset and a clean second-run no-op. The pytest suite only
exercises SQLite create_all(), so a regression in the Alembic chain on MySQL
would otherwise ship undetected.

Verified: fresh core upgrade on a scratch utf8mb4 MySQL builds clean + no-op on
rerun (redundant indexes absent, unique assetid kept); plugin migrations applied
+ verified on the dev DB (machineid gone, bu index present). 953 backend tests
pass; naming + pyflakes green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 09:29:45 -04:00
cproudlock
9c8b2c9c9e DB review safe-fix: naive-UTC timestamp defaults (drop db.func.now)
Some checks failed
CI / backend (push) Has been cancelled
CI / naming (push) Has been cancelled
CI / frontend (push) Has been cancelled
DB review found four DateTime columns defaulting to db.func.now() (MySQL
session-timezone wall clock) while the rest of the schema stores naive UTC, so
one schema mixed two clocks and to_dict() labelled the local values UTC with a
'Z' suffix. Switch application.dateadded, computers.installeddate,
knowledgebase.lastupdated (default + onupdate), and slides.uploadeddate to the
module-level naive-UTC _utcnow callable already used elsewhere (apitoken.py).
ORM-side default only - no column-type change, no data migration; affects
new/updated rows.

Targeted tests pass (154); naming + pyflakes green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 09:09:29 -04:00
cproudlock
cd02cd20f4 Frontend naming + CSS-variable cleanups (review low)
Some checks failed
CI / backend (push) Has been cancelled
CI / naming (push) Has been cancelled
CI / frontend (push) Has been cancelled
Naming convention (LOCKED): rename the ManifestEditor simulate state sim ->
simulateInputs / simResult -> simulateResult (+ .sim-result CSS class) - 'sim'
was banned standalone shorthand. Rename AssetRelationships props assetId ->
assetid and machineNumber -> machinenumber so a prop holding a DB field value
mirrors it verbatim; updated the five detail-page call sites (:assetid=).

CSS variables: SearchResults per-domain badge palette moved into CSS variables
on the container; the duplicated prefers-color-scheme dark block collapses to a
single set of variable overrides instead of restating all ten selectors.

frontend build green; vitest 49 pass; naming green; search badges + detail
relationships verified rendering with no console errors.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 08:48:50 -04:00
cproudlock
b99da362b5 Docs: refresh ROADMAP to current state (contract 0.11.0, phase 6 done)
Review flagged ROADMAP as six contract versions stale. Set opening version to
0.11.0; mark phase 6 (multi-site distribution) done and name the real last
milestone (legacy import + prod pilot); drop the three completed items
(per-plugin Alembic chains per ADR-008, local font bundling - Inter is bundled
via @fontsource, measuringtools plugin built); reframe the frontend item to the
part that actually remains (external plugin UI packaging - hooks + gating
already shipped via ADR-009/010); add ADR-007..012 to the decision-log pointers.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 08:42:51 -04:00
cproudlock
ea3cca8954 Test coverage: plugin lifecycle, registry, lockout unlock, contract fleet, seed
Fills the review's highest-value coverage gaps in the framework's own core
feature.

New tests/test_plugin_lifecycle.py: manager enable/disable dependency guards
(beta depends on alpha - enable-beta-first refused, disable-alpha-while-beta-on
refused) via synthetic plugins; enable seeds a plugin's RBAC permissions
idempotently; registry disable-survives-reload, corrupt-file recovery, and the
equipment->machines rename migration; `flask seed settings` idempotency.

test_plugin_contract.py: BUNDLED_PLUGINS now covers all bundled plugins incl.
geenforce, measuringtools, warranty (was 9, contradicting the CLAUDE.md "all
bundled satisfy the contract" claim); the structural checks now run against them.

test_authz.py: account lockout auto-unlock path (expired lockeduntil -> correct
password logs in and clears the lock state), previously untested.

All new tests pass; naming green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 08:41:23 -04:00
cproudlock
d49baeb5fa Security closeout: settings public allowlist, audit.view gating, test-user guard
Settings exposure (review medium): GET /api/settings and /api/settings/<key>
now return the full table only to an authenticated principal. Unauthenticated
callers (kiosk dashboards, print pages, login screen, setup router) get just a
public allowlist - categories branding + map plus a named set (site_base_url,
facility_name, printer_hostname_template, contact_email_domain,
servicenow_enabled, setup_complete). A non-public single-key GET returns 404 so
existence is not confirmed. Secrets stay masked in both cases. Closes the
unauthenticated enumeration of smtp_host / employee_db_host / zabbix_url /
servicenow URLs. Allowlist mirrors the keys siteSettings.js + mapConfig.js +
setupState.js read before login.

audit.view (review low): the three audit-read routes (list, entity-history,
stats) were jwt_required only despite a defined-but-unwired audit.view
permission; now gated by it (seeded to admin), so a role-less member or unscoped
PAT can no longer read the cross-user audit trail.

flask seed test-user (review low): refuses outside DEBUG/TESTING - it creates
the well-known admin/admin123; production sites use `flask seed admin`.

Tests: unauthenticated allowlist + authed-full-masked + private-key-404, and
member-403 / admin-200 on audit routes. 336 authz tests pass; naming green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 08:36:19 -04:00
cproudlock
cd353b6432 Review safe-polish: docs accuracy, dead imports, no-emoji, geenforce robustness
Some checks failed
CI / backend (push) Has been cancelled
CI / naming (push) Has been cancelled
CI / frontend (push) Has been cancelled
From the full multi-agent review (0 high, 7 medium, 17 low findings). Applies
the mechanical, low-risk items; design/policy findings left for a decision.

Docs accuracy: CLAUDE.md contract 0.10.0 -> 0.11.0 and both stale Alembic head
citations -> 7d24_customfield_searchable / 31 migrations; Dockerfile bundled-
plugin comment fixed (drop nonexistent "equipment", add machines +
measuringtools, count eleven).

Style/naming (LOCKED rules): remove a CSS-escaped pushpin emoji before location
search results (no-emoji policy); rename ManifestEditor shareRoot -> shareroot
(variable mirrors the API field verbatim).

Dead code: remove confirmed-unused imports across ~20 modules (require_role/
require_permission scaffold residue, stray db/Vendor/Model/current_user/Optional/
error_response); drop unused build_scope import + a stale GEENFORCE_API_KEY
docstring clause in geenforce. Migration files left untouched.

Correctness: geenforce ingest robustness - record_enforcement_report now 400s
on a non-dict counts / non-list results instead of 500; _apply_app_link ignores
a non-numeric appid per its docstring instead of 500. Regression tests added.

Backend query.get sweep finished: auth.py refresh -> db.session.get (last one).

910 backend tests pass; pyflakes clean; naming green; frontend build green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 08:02:43 -04:00
cproudlock
85a6ab8645 Setup wizard: plugin display names + GE-Enforce next-steps pointer
All checks were successful
CI / backend (push) Successful in 1m37s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
Plugins list now carries a displayname (manifest display_name, else the
machine name title-cased). Adds display_name to the four whose title-case
was wrong: GE-Enforce, USB, Measuring Tools, Knowledge Base. The setup
wizard Features step and Settings > Plugins render it, so "Geenforce"/"Usb"
are gone.

Finish step shows a pointer when GE-Enforce is enabled: it still needs a
scoped service token (Settings > API Tokens) and a share export root
(GE-Enforce page) before the fleet uses it - operational config the wizard
does not collect.

frontend build green; naming green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 07:41:32 -04:00
cproudlock
d7b777a7a0 GE-Enforce editor: onboarding field guidance
All checks were successful
CI / backend (push) Successful in 1m38s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
Per-method detection hint: a plain-language line explains what "already
correct" means for the selected detection method (Registry/File/FileVersion/
Hash/MarkerFile/ValueMatches/pnputil/Always), updating live as the author
picks one. Lives in entryForm.js (DETECTION_METHOD_HINTS + detectionMethodHint)
with 4 new vitest cases; the editor renders it under the dropdown.

Also: relative-path hint on Installer/Source (path under the scope payload
folder or an inline payload), an InUseCheck behavior hint, and refresh the
entryForm.js header now that the editor imports these helpers directly.

vitest 49 pass; frontend build green; naming green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 07:31:43 -04:00
cproudlock
456a44104b GE-Enforce polish: DDL-parity guard test, retire Collector PC Types page
All checks were successful
CI / backend (push) Successful in 1m40s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
Add test_geenforce_ddl_parity to lock the manifest models against their
Alembic baseline (catches model/migration drift for a chain that is still
amendable pre-deploy).

Retire the "Collector PC Types" settings card: GE-Enforce scope
computertypeid supersedes the pctypemap editor UI (ADR-012). The collector
still reads pctype_mapping(), so the backend map stays; only the editor
surface is removed, with a deprecation note in pctypemap.py.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 07:25:01 -04:00
cproudlock
4e5b4228c1 GE-Enforce: compliance view, inline payload upload, frontend test harness
Fleet-install compliance for app-linked manifest entries: new service
compliance_for_scope + GET /geenforce/scopes/<id>/compliance count active
ComputerInstalledApp rows by curated appid (null-safe when computers plugin
absent). ManifestEditor gains a compliance panel. Curated appid stays shopdb
metadata and never enters manifest JSON, so behavioral parity is unaffected.

Inline manifest payloads: store_inline_payload (sha256, 1MB cap,
payloadsource='inline') + POST/GET /geenforce/entries/<id>/payload; editor
gains an upload control. Entry payload metadata surfaced in _entry_payload.

Frontend test harness: extract the editor's entry-form logic into pure
entryForm.js (buildEntryPayload, describeEntry, availableEntryTypes, scope
gates, ...) and cover it with 45 vitest tests. ManifestEditor now imports
those helpers, so the tests exercise the shipped code path (no duplication).

908 backend tests pass; vitest 45 pass; frontend build green; naming green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 07:24:51 -04:00
cproudlock
3355436fcd Add curated manifest-entry -> Application link (honest app tracking)
All checks were successful
CI / backend (push) Successful in 1m34s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
The honest replacement for the backed-out auto-seeding: instead of scraping
manifest labels into duplicate Application rows, an entry can be LINKED to an
existing catalog Application, cross-referencing what shopdb already tracks.

- Model: manifestentries.appid (nullable soft ref to core applications; in the
  0001 baseline). It is shopdb METADATA, deliberately NOT a manifest field - it
  never appears in the rendered manifest JSON, so enforcement + parity are
  unaffected (test asserts it stays out of the preview manifest).
- API: _entry_payload returns appid + resolved appname; create/update accept an
  optional appid (validated, unknown id ignored, null unlinks) via _apply_app_link;
  GET /geenforce/applications is the picker source (id + name).
- Editor: a "Tracked application (optional)" select in the entry modal, and the
  entry summary line notes the linked app ("...; tracked: eDNC").
- Foundation for a future desired-vs-observed compliance view.

889 tests green (incl. the link test + parity/migration unaffected); build +
naming green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 06:45:10 -04:00
cproudlock
810687953f ADR-012: GE-Enforce manifest ownership in shopdb (ACCEPTED)
All checks were successful
CI / backend (push) Successful in 1m38s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
Formalizes the design built this session: manifest as shopdb data (wide entries
table + entrytype discriminator, per-plugin Alembic chain), immutable published
snapshots + rollback, behavioral-parity gate, engine-as-source-of-truth filter
mirror, payload integrity separate from detection, observed-state reporting,
service-token auth (contract 0.11.0), client kit + provisioning-agnostic
Install-GEEnforce bootstrap (engine referenced not vendored), Milestone-1
export-to-share + staged cutover, and NO application auto-seeding (curated
linking instead). Indexed in ADR README + CLAUDE.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 06:37:40 -04:00
cproudlock
3b400d8cc4 Fix GE-Enforce client kit under PowerShell 7 (header-array coercion)
All checks were successful
CI / backend (push) Successful in 1m38s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
Found by running the kit under pwsh 7 against the live API: Invoke-WebRequest
returns header values as string ARRAYS in PS7 (scalars in Windows PowerShell
5.1), so X-Manifest-Version came back as @('1') and [int] on it threw - report
build failed. The target scheduled task runs 5.1 (works), but the kit must be
robust under PS7 too (target preinstalls PowerShell 7). Coerce ETag and
X-Manifest-Version with @(...)[0], a clean scalar in both.

Validated end to end on Linux pwsh 7.6.3 against the dev API: fetch (200) ->
cache-304 -> report sent -> landed received=true/status=ok. All 4 client scripts
parse clean; PSScriptAnalyzer shows only cosmetic warnings (Write-Host in a CLI,
intentional log-guard catch).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 06:23:56 -04:00
cproudlock
8dceb8812f Add GE-Enforce agent deployment: Install-GEEnforce.ps1 + deploy doc
All checks were successful
CI / backend (push) Successful in 1m36s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
Closes the "how do sites actually deploy GE-Enforce" gap (esp. OOBE-ppkg sites
without a PXE/WinPE step). Site-neutral + imaging-path independent.

- plugins/geenforce/client/Install-GEEnforce.ps1: a bootstrap that writes the
  PC's identity (C:\Enrollment\pc-type.txt is what determines the PC type; plus
  machine-number/cmm version/cmm id/site-config as needed), sets the shopdb
  BaseUrl + token in HKLM:\SOFTWARE\GE\ShopDB, deploys the client kit, optionally
  copies the engine from -EngineSource, and registers the SYSTEM scheduled task
  (at logon + every N min). Idempotent; fails loud (installer, not the fail-safe
  runtime). Engine is REFERENCED not vendored - it belongs to the GE-Enforce
  framework; the script warns if absent but still labels the PC.
- docs/GE-ENFORCE-DEPLOY.md: the deploy contract - the three things a PC needs
  (client, identity, credential), the identity table (what determines PC type,
  no auto-detection - the provisioner supplies it; shopdb cannot set it at
  imaging), and how to invoke per path (PXE step, OOBE ppkg via
  ProvisioningCommands, Intune, manual), the engine boundary, and verification.
- Cross-linked from docs/GE-ENFORCE.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 23:02:54 -04:00
cproudlock
6e7d51de21 GE-Enforce editor: phase-aware form, scope summary, narrower entries table
All checks were successful
CI / backend (push) Successful in 1m37s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
- Phase-aware editing: a preinstall scope now only offers MSI/EXE types and
  Registry/File detection (the preinstall runner silently skips the rest), and
  the preinstall flags show only for a preinstall scope - so an author cannot
  pick an option that would do nothing.
- Per-scope summary line ('Installs PC-DMIS 2016, ...; 4 entries; runs after
  common') under the scope header.
- Entries table: dropped the redundant Detection + Filters columns (the
  plain-English entry line already conveys them), fixed-layout with sized button
  columns and stacked Up/Down - no more horizontal scroll.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 22:57:06 -04:00
cproudlock
e6ed47c533 Fix 7 factual errors in GE-Enforce doc (Fable fact-check vs real source)
All checks were successful
CI / backend (push) Successful in 1m39s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
Verified claim-by-claim against the engine/dispatcher/preinstall runner/manifests.
Corrections (3 were ship-blocking):

- Timeline (ship-blocking): identity files (pc-type/machine-number/cmm version/
  site-config) are written in WinPE at the PXE menu BEFORE the image boots, not
  during a post-imaging 'enrollment' step; preinstall already reads them. Added a
  step [0]; enrollment is now only Intune + Azure DSC credential provisioning.
- _CmmVersion (ship-blocking): a CMM bay with NO resolved version gets ALL
  PC-DMIS versions (legacy install-all), not none.
- machine-number 9999 (ship-blocking): the enforcement engine does not
  special-case 9999; it is a placeholder that won't match real bay gates (the
  9999-skip is status-write-back only).
- Preinstall runner implements only MSI/EXE + Registry/File detection, not the
  full matrix (that is runtime-only).
- Runtime processes up to three scopes: common, type, then optional type-subtype.
- pc-subtype.txt is legacy (no longer written at imaging since 2026-05-04).
- The collector ComputerType mapping lives at Settings > Collector PC Types, not
  the geenforce scope (scope computertypeid is a local reference field).
- FileVersion is a raw string compare; 4-part is convention, not engine-enforced.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 22:47:59 -04:00
cproudlock
ab0c9a454b Make GE-Enforce editor more legible: plain-English entries + how-it-works panel
All checks were successful
CI / backend (push) Successful in 1m37s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
- Each manifest entry now shows a one-line intent summary under its name
  (e.g. 'Installs eDNC; reinstalls if not detected; 2 PC types') instead of only
  the raw Type/Detection/Filters columns - turns jargon into what it actually does.
- A collapsible 'How this works' panel at the top gives the mental model in four
  sentences (desired state + self-heal, entries + detection, targeting, doc link).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 22:45:50 -04:00
cproudlock
655fe1087f Add GE-Enforce guide: concepts, shopdb plugin, imaging-time integration
All checks were successful
CI / backend (push) Successful in 1m38s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
docs/GE-ENFORCE.md - operator-facing guide grounded in the real engine +
manifests (Fable-verified analysis): how GE-Enforce works (preinstall vs runtime
phases, the enforce loop, manifest scopes/entries, self-heal detection, gates,
enrollment), WHEN it installs/takes over in the imaging timeline (preinstall at
imaging -> GE-Enforce laid down -> enrollment provisions creds -> runtime
enforcement from first logon), how the shopdb plugin manages it (Manifests
authoring + contextual targeting + simulate + publish/rollback + Export to Share
Milestone 1, Enforcement Reports), day-to-day IT tasks, and a reference index.
Complements GE-ENFORCE-CLIENT.md (client contract) and the proposal (plan).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 22:36:18 -04:00
cproudlock
378af32c1d Rebuild GE-Enforce editor: themed (Fable redesign) + semantics-aware targeting
All checks were successful
CI / backend (push) Successful in 1m43s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
Replace the generic field-dump editor with the Fable-coordinated redesign
(stock .section-card / .settings-grid / .setting-row / .table-container /
badge / btn primitives, opaque global .modal on --bg-card-solid, proper
modal-header/body/footer), then layer the manifest semantics on top:

- CONTEXTUAL targeting: an entry only shows the gates its scope actually uses,
  instead of every gate on every entry. Data-driven + scope-aware -
  common/preinstall (fleet-wide) show PC types; CMM shows the version gate;
  a scope whose entries use machine numbers (collections) shows those; hostnames
  show when used. A short explainer states why (a per-type manifest already runs
  only on its own type), and "Show all targeting options" reveals everything.
  Hidden gates keep their value on save (no data loss).
- Preserves all functionality: type-switched payload blocks, detection reveal,
  structured InUseCheck rows (name/ExePath/timeout), LogFile, preinstall
  checkboxes, simulate, publish/versions/rollback, export. Entry modal widened
  to min(1100px,96vw).

EnforcementReports adopts the themed redesign (filters, card > table-container,
badge-mapped statuses, global modal) with the error handling kept.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 22:15:48 -04:00
cproudlock
3ac41c1556 Back out app auto-seeding; fix report-status + PCTypesStrict bugs (manifest review)
All checks were successful
CI / backend (push) Successful in 1m45s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
A deep Fable review of the real manifest corpus (READ-ONLY reference) showed the
manifests are an ENFORCEMENT PROGRAM, not an application inventory, and that
auto-seeding the Applications catalog from entry Type + Name was wrong:

- The catalog ALREADY tracks these apps from the classic-shopdb migration, with
  version histories (PC - DMIS, UDC x11 versions, eMX / eDNC, CLM, CSF, Oracle
  Database, FormTracePak). Seeding from manifest labels created DUPLICATES under
  different names (PC-DMIS 2016 vs PC - DMIS; eDNC (bundles NTLARS) vs eMX / eDNC;
  OpenText HostExplorer ShopFloor vs CSF). It also misclassified config drops
  (eMxInfo.txt) as apps and could never match a PC's reported ARP name.
So the seed-applications command + service are removed. Properly linking
manifest entries to the EXISTING catalog is a curated feature, not label-scraping.

Two REAL bugs the review found are fixed and kept:
- Report status (R4): every healthy cycle runs Always/no-detection scripts the
  engine counts as "installed", so keying self-heal off installed>0 marked the
  common scope selfhealed forever and made 'ok' unreachable. Status now derives
  from explicit per-entry self-heal flags only; the stored flag no longer infers
  from action=='installed'; the client kit doc reflects it.
- PCTypesStrict (R5): the runtime engine has no strict handling (preinstall
  runner only). filters.matches_pctype now applies strict only when phase ==
  'preinstall'; simulate + parity thread the scope phase through; the strict test
  uses a preinstall scope.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 21:56:18 -04:00
cproudlock
fc1f56fec3 Widen GE-Enforce entry modal to min(1100px,96vw), 3-column form grid
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 21:26:54 -04:00
cproudlock
796007bcca Track manifest apps in the Applications catalog; make CMM gate contextual
All checks were successful
CI / backend (push) Successful in 1m43s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
seed-applications: a flask geenforce seed-applications command + service that
reads the imaging-PC-type manifests and creates a core Application for every
installer entry (MSI/EXE/CMD/BAT), so shopdb tracks what GE-Enforce actually
deploys. Idempotent, deduped by appname; File/Registry/PS1/INF config entries
are skipped. Run against the West Jefferson reference: 27 apps tracked (PC-DMIS
2016/2019/2026, eDNC, Oracle Client, Adobe Reader, HostExplorer, the VC++ redist
matrix, Keyence VR-6000, PowerShell, Display Kiosk, ...). 2 tests.

Editor: the CMM version gate (_CmmVersion) now only shows for CMM scopes - it is
metrology-specific, so a printer/common entry form no longer carries the
irrelevant field.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 21:21:49 -04:00
cproudlock
d9d40297b6 Fix transparent + cramped GE-Enforce modals
--bg-card is translucent (rgba .4) in dark mode - a glass effect for cards on
the page, but a floating modal rendered over content showed the page through it.
Modals now use --bg-card-solid (the opaque surface the shared Modal.vue uses),
with a border + shadow, and are wider (entry editor min(920px,94vw)) so the full
field set fits.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 21:19:06 -04:00
cproudlock
24e38c6e75 Remove orphaned settings-dir GE-Enforce views (moved to geenforce/ section)
All checks were successful
CI / backend (push) Successful in 1m43s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
The earlier move copied instead of moving, leaving unreachable duplicates under
views/settings/. The routed copies live in views/geenforce/.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 21:14:00 -04:00
cproudlock
7089a61b50 Move GE-Enforce to its own top-level section; fix overflow + input theming
GE-Enforce is a large operational surface (manifest authoring + fleet
compliance), not a setting, and it was squished in the settings two-pane shell.
Promote it to a dedicated full-width top-level section:

- New sidebar entry "GE-Enforce" (plugin get_navigation_items, shield icon,
  auto-gated to the enabled plugin) instead of two Settings > Integrations cards.
- Tabbed shell GeEnforceLayout.vue (Manifests | Enforcement Reports) with
  full-width children under AppLayout, not the narrow settings rail.
- Views moved settings/ -> geenforce/ (ManifestEditor.vue, EnforcementReports.vue).

Theming + overflow fixes (the "chaotic / cut off / different inputs" report):
- Inputs/selects/textareas now match the stock settings look (border, radius,
  --bg, focus color) instead of browser defaults.
- No horizontal overflow: editor grid uses minmax(0,1fr) + min-width:0 on
  children, collapses to one column under 1000px; entry table and reports table
  scroll inside their own overflow-x containers; detail actions wrap.

Verified at 1280px: no page overflow, detail pane + tables fit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 21:12:57 -04:00
cproudlock
0dcd186820 Fix defects found in session review of GE-Enforce plugin
All checks were successful
CI / backend (push) Successful in 1m43s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
Consolidated fixes from a three-dimension adversarial review.

Data-loss (HIGH): the manifest entry editor stripped fields the form did not
expose, because PUT /entries is a full reset-then-apply. The form now captures
everything - InUseCheck processes as structured name/ExePath/timeout rows (not
just names), LogFile, and the three preinstall flags as checkboxes; the dead
payload-source control (never wired) is removed. New regression test proves an
edit preserves ExePath/timeout/LogFile/PreEnrollment/PCTypesStrict.

Update-entry crash (found by that regression test): replacing an entry's
one-to-one InUseCheck (unique entryid) collided with the old row mid-flush ->
IntegrityError -> 400. update_entry now frees the old InUseCheck (delete+flush)
before populate re-inserts it.

Export truncation (MEDIUM): export_scope_to_share used a plain truncating open,
so a failed/partial write left the live on-share manifest (every PC reads it)
empty. Now writes a temp file in the same dir and os.replace() atomically.

Report dedup case bug (MEDIUM, confirmed by scratch test): the iscurrent demote
matched hostname case-sensitively while the read path uses ilike, so a PC
reporting different casing left two iscurrent rows and double-counted. Demote is
now case-insensitive; regression test added.

Simulator fidelity (MEDIUM): PCTypesStrict was captured but ignored by the
filter mirror, so the simulator wrongly matched a collections-only strict entry
to a nocollections PC via the shared Standard alias group. matches_pctype now
honors PCTypesStrict (disables alias expansion); test added.

Hardening: removed the dead/unscoped GEENFORCE_API_KEY env fallback (never wired
into config; tokens are the only path); create/update entry return 400 on a
duplicate Name instead of 500; parity now asserts scope-level Version/Site; a
new test guards real-manifest field lengths against column limits (the DB-free
parity harness can't see truncation); error handling added to the previously
unguarded editor + reports API calls.

Full suite green; naming + frontend build green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 20:46:09 -04:00
cproudlock
d894f054ac Add GE-Enforce P4 client kit: fetch + report + shadow mode (reference)
All checks were successful
CI / backend (push) Successful in 1m42s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
Client-side integration kit for sourcing manifests from shopdb and reporting
results back. Site-neutral reference a site adapts into its GE-Enforce.ps1; the
live dispatcher and engine are NOT touched (they are read-only reference under
projects/pxe). Only the manifest JSON source moves from a share file to shopdb,
plus a result report.

- plugins/geenforce/client/ShopdbEnforceClient.psm1: Sync-ShopdbManifest (GET
  with ETag -> local cache; falls back to last-known-good when shopdb is
  unreachable so a PC is never left unmanaged), Compare-ShopdbShadow (behavioral
  diff vs the on-share manifest), Send-ShopdbReport / New-ShopdbReport (best-
  effort POST /report), Get-ShopdbConfig (BaseUrl + token from
  HKLM:\SOFTWARE\GE\ShopDB).
- plugins/geenforce/client/Invoke-ShopdbEnforce.ps1: orchestrator. Fetches,
  optionally shadow-compares (installs from the share, only logs the diff), runs
  the unchanged engine, and reports. Fail-safe: any error exits 0.
- docs/GE-ENFORCE-CLIENT.md: the fetch + report contracts, config, cache/fail-
  safe behavior, the staged shadow -> read-cutover -> payload-migration runbook,
  and TLS/payload-integrity notes.

The report JSON shape matches the POST /api/geenforce/report contract already
covered by the reporting tests. Nothing here runs the live client; shadow mode
and cutover stay a site decision after Milestone 1 sign-off.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 18:30:08 -04:00
cproudlock
adc5b7f69e Add GE-Enforce export-to-share + fleet-compliance UI (Milestone 1 UX)
All checks were successful
CI / backend (push) Successful in 1m43s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 9s
Rounds out the Milestone 1 admin experience: author + publish in shopdb, push
to the share by a button, and see what the fleet actually did.

Export to share:
- GET/PUT /api/geenforce/config stores the on-share export root (Setting
  geenforce_share_root); POST /scopes/<id>/export-share writes the current
  published JSON to <shareroot>/<scope>/manifest.json (preinstall.json for the
  preinstall phase), backing up the existing file to _meta/history first.
  geenforce.publish gated. The engine and PCs are untouched - this is the safe
  Milestone 1 push whose rollback is restoring the history backup.
- Editor: a share-root config row + an "Export to Share" button per scope.
- 3 tests (config roundtrip, export writes the file, second export backs up).

Fleet-compliance UI (Settings > Enforcement Reports):
- New page over GET /reports + /reports/<id>: latest report per PC with
  received (applied vs latest published version), status (ok/selfhealed/failed),
  and install/skip/fail counts; row detail shows per-entry outcomes with
  self-heal flags, exit codes, and messages. Hostname/PC-type filters.
- ADR-010 settings card + ADR-009 plugin-gated route.

Full suite 883 green; frontend build + naming green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 18:06:33 -04:00
cproudlock
7eb6cebeb5 Add GE-Enforce P3 manifest editor UI (Settings > Imaging PC Types)
All checks were successful
CI / backend (push) Successful in 1m41s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
The imaging-PC-type manifest editor, contributed as an ADR-010 settings card
(Integrations group) and an ADR-009 plugin-gated route
(/settings/imagingpctypes, hidden when geenforce is disabled).

- Scope list: every imaging PC type with phase, entry count, and published
  version (or "unpublished"). New PC Type button.
- Scope detail: ComputerType/MeasuringToolType mapping + description; Publish,
  Versions (with per-version Roll Back), Preview (draft JSON), Delete.
- Entry table: ordered with Move Up/Down (the ordering contract, not drag),
  Name/Type/Detection/Filters, Edit/Delete. Add Entry opens a typed modal whose
  fields switch on entry Type (MSI/EXE/... vs PS1 vs File vs Registry), with a
  detection block, comma-separated targeting filters, CMM version gate, payload
  source, and an Advanced disclosure for the inert ApplyMode/UpdateWindow and
  InUseCheck. RegValue is typed by RegType (DWord/QWord -> number).
- Simulator: "what would a PC get" - enter a machine profile, see which entries
  apply and which filter excluded the rest. Verified live: CMM version 2019 ->
  applies 2019 + untagged, filters 2016/2026 by _CmmVersion.

Uses the P2 admin API; JWT+admin gated. Frontend build + naming green; full
backend suite 876 green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 17:22:49 -04:00
cproudlock
d157502d5b Add GE-Enforce P2 admin CRUD API: scopes, entries, reorder, simulate, publish
Full HTTP admin surface behind the manifest editor (geenforce.manage for edits,
geenforce.publish for shipping):

- Scopes: POST/GET/PUT/DELETE /scopes[/<id>] (create imaging PC types, edit the
  ComputerType/MeasuringToolType mapping + metadata, delete).
- Entries: POST /scopes/<id>/entries, PUT/DELETE /entries/<id>. Payloads use the
  manifest Applications[] shape; populate_entry (refactored out of build_entry)
  updates an entry in place, resetting omitted fields and replacing children.
- Reorder: PUT /scopes/<id>/entries/reorder enforces the ordering contract
  (body must list exactly the scope's entry ids).
- Simulate: GET /scopes/<id>/simulate?pctype&subtype&hostname&machinenumber&
  cmmversion returns which entries apply and which filter excluded the rest,
  reusing the engine-mirror filters. The "what would this PC get" tool.
- Publish lifecycle: POST /scopes/<id>/publish (records publishedby from JWT),
  GET /scopes/<id>/versions, GET .../versions/<n> (frozen manifest),
  POST /scopes/<id>/rollback.

Entry type validated against ENTRY_TYPES; 8 CRUD tests. JWT+permission gated so
the authz sweep covers them.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 17:13:27 -04:00
cproudlock
6dc31c6149 Add GE-Enforce observed-state reporting: receipt + self-heal from PCs
All checks were successful
CI / backend (push) Successful in 1m37s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
PCs now report enforcement results back to shopdb, closing the desired-vs-observed
loop.

- POST /api/geenforce/report (geenforce.report service token): each cycle a PC
  posts the published version it applied, install/skip/fail/filtered counts, and
  per-entry outcomes.
- Two tables: manifestenforcementreports (latest-per-host + history: applied
  version, enforcer version, counts, derived status ok/selfhealed/failed) and
  manifestenforcementresults (per entry: action installed/skipped/failed,
  selfhealed flag, exit code, warning/error message).
- RECEIVED: reports carry the applied version; the admin view derives
  receivedlatest by comparing it to the scope's current published version, so
  the fleet view shows which PCs picked up an update.
- SELF-HEAL: per-entry action captures drift correction (installed when it
  should already be present) vs skipped (already good) vs failed, with messages.
- Admin reads: GET /reports (fleet compliance rollup) and GET /reports/<id>
  (per-entry detail). New geenforce.report permission.
- Tables added to the (undeployed) 0001 baseline; geenforce.post_report is a
  service-token endpoint so it is exempt from the JWT authz sweep, like the
  collector blueprint. 8 reporting tests; full suite green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 17:04:07 -04:00
cproudlock
d85b33bd68 Build GE-Enforce manifest-store plugin (P0/P1): model, importer, parity gate
All checks were successful
CI / backend (push) Successful in 1m36s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
First execution phases of docs/proposals/ge-enforce-plugin.md. The GE-Enforce
manifest becomes shopdb data.

P0 scaffold: new geenforce plugin (api_prefix /api/geenforce, default_enabled
false, core_version >=0.7.0). Registered in PLUGIN_TABLE_OWNERS (ADR-008); its
0001 baseline really creates the tables.

P1a model: one wide manifestentries table + entrytype discriminator (not STI,
not JSON blob), manifestscopes (UNIQUE scopename+phase), the three multi-value
filter child tables, inusechecks + processes, immutable manifestpublishedversions
(frozen rendered JSON), manifestpayloads (inline, capped), pctypealiases
(mirror of the engine lib's alias graph). regvalue stored as its raw JSON
literal so DWord typing survives.

P1c importer + exporter: parse common + gea-shopfloor-* + preinstall.json into
draft rows and rebuild the JSON verbatim from rows in sortorder.

P1d parity harness (GATE A): filters.py mirrors the engine's four filter
functions + alias graph; parity.py proves import+export is behaviorally lossless
(field-identical + same-entries-fire across 18 machine-profile fixtures) WITHOUT
byte-diffing. Verified PASS against all 11 real reference manifests (64 entries)
and a synthetic site-neutral fixture covering every type/filter (the CI gate).

First slice (gea-shopfloor-cmm shape): service layer (import/publish/rollback/
export-to-share), CLI (parity, import-share, publish, export-share), and the
client endpoint GET /api/geenforce/manifest serving the current published
snapshot (never the draft) with ETag/304. Split permissions
geenforce.manage/publish/fetch. Tests prove import->publish->serve, draft edits
never change served bytes, publish+rollback, and auth (401 unauth/wrong-scope).

Contract 0.11.0: added service_token_authorized(scope) to shopdb.api so plugin
service endpoints authorize a scoped managed token without importing core token
internals. Documented in PLUGIN-HOOKS.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 16:53:18 -04:00
cproudlock
cf2f9c308e Add manifest revision history + draft audit trail to GE-Enforce plan
All checks were successful
CI / backend (push) Successful in 1m32s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
- Every publish is a permanent immutable revision (manifestpublishedversions),
  kept indefinitely; optional retention policy (keep last M / prune older than N)
  deferred, default keep-everything.
- Draft edits are not versioned (working copy overwrites), so field-level "who
  changed what between publishes" rides the existing core audit system - no new
  table, shows in the Audit Logs UI IT already uses.
- Runbook: History tab for published versions + roll back; Audit Logs for draft
  edit provenance.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 16:24:28 -04:00
cproudlock
c88d673d7b Fold Fable execution review into GE-Enforce plan: simpler, IT-manageable
All checks were successful
CI / backend (push) Successful in 1m25s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
Add an execution plan and simplify the design for average-site-IT operability
(the governing constraint from the review):

Model simplifications:
- One wide manifestentries table with an entrytype discriminator, not SQLAlchemy
  STI subclasses and not a JSON blob. ~64 total entries fleet-wide make sparse
  columns free and keep rows readable in plain SQL.
- Published snapshots freeze the rendered JSON document in a single manifestjson
  column; drop the row-mirrored manifestpublishedentries family. Immutability is
  structural, rollback is a one-flag flip, diff is a text diff.
- New manifestpayloads table for inline bytes with a ~1 MB app cap.
- regvalue stores the raw JSON literal (DWord typing); applymode/updatewindow
  flagged inert-in-engine so the UI labels them.

Execution plan (section 13):
- Phases P0-P6 with gates; parity harness spec (two checks, IT-readable output,
  ~16-18 machine-profile fixtures); first vertical slice through
  gea-shopfloor-cmm; ranked fail-fast risks.
- Milestone 1 = author + publish in shopdb, export to the share by a button,
  engine/dispatcher/PCs unchanged. Real pain relief at zero client risk, with a
  rollback IT already knows (restore the _meta/history backup).
- Export-to-share promoted to a first-class feature and permanent break-glass.
- Split permission geenforce.manage (edit) vs geenforce.publish (ship).
- Move Up/Down instead of drag-and-drop; a "what would this PC get" simulator
  endpoint + UI; three-increment editor build.
- Two-source pctypemap transition window; scope-inventory reconciliation
  (gea-shopfloor-display has no share dir).
- Plain-English IT day-to-day runbook proving the design is manageable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 16:15:41 -04:00
cproudlock
9dd2aa3cc6 Revise GE-Enforce plugin plan after review: parity, integrity, snapshots
All checks were successful
CI / backend (push) Successful in 1m25s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 8s
Fold six review findings into docs/proposals/ge-enforce-plugin.md:

- Parity gate is behavioral equivalence, not byte-identity. Re-serialized JSON
  differs in key order/whitespace/_comment formatting, so a raw diff never
  converges; the test is same ordered entry set with identical detection/
  targeting/action per entry.
- Dedicated payloadsha256 column, independent of detectionmethod. DetectionValue
  is a SHA256 only for detectionmethod=Hash; MSIs with Registry/FileVersion
  detection carry no payload hash, so an HTTP/inline fetch would otherwise run
  unverified bytes. Client verifies fetched bytes against payloadsha256.
- Immutable published snapshots (manifestpublishedversions). Editing touches a
  draft only; publish freezes a snapshot; the client is always served the latest
  published snapshot, never the live draft; rollback republishes a prior
  snapshot (the post-cutover safety net once the on-share JSON is retired).
- Scope uniqueness is (scopename, phase), not scopename alone; preinstall is one
  flat scope gated internally by PCTypes, not per-pctype scopes.
- Alias graph: engine lib stays the single source of truth, shopdb only mirrors
  it for validation; do not invert to engine-fetches-from-shopdb.
- Desired-vs-observed needs a new collector field (the installedVersions status
  map), not existing data; flagged as a dependency.

Plus TLS trust for the SYSTEM-context client and importer skips .bak variants.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 15:34:40 -04:00
cproudlock
1672e349e5 Collector auto-links measuring tools for metrology PCs; settings rail cleanup
All checks were successful
CI / backend (push) Successful in 1m25s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
Metrology PCs (CMM, Keyence, Genspect, wax-and-trace imaging pc-types) drive
an attached measuring instrument. The PC itself stays a shopfloor PC, but the
collector now models the instrument:

- New METROLOGY_TOOL_MAP (pctypemap.py) maps those pc-types to a
  MeasuringToolType (CMM, Vision System, Genspect, Form Tracer).
- ComputersPlugin._sync_measuringtool_link creates the MeasuringTool asset
  once and a directional PC->tool "controls" relationship, tagged
  collector:measuringtool. Idempotent (re-push reuses, no duplicate asset) and
  self-archiving (a PC re-imaged to a non-metrology type deactivates the link
  but keeps the asset and any calibration history). Mirrors the printer-link
  pattern. The MeasuringToolType is created on demand if not seeded.
- 4 tests: create+link, idempotent re-push, non-metrology skip, repurpose
  archives. Non-metrology PCs never warn about a missing controls type.

Settings rail cleanup:
- Collapsible groups so the 13-group rail fits without scrolling (1511px ->
  488px). The group containing the current page expands; the rest collapse.
  CSS-drawn caret (ASCII source, no Unicode). Empty groups never render, in
  both the rail and the landing page.
- Measuring Tools group placed with the other asset groups (right after
  Machines) instead of appended last; empty placeholder positions the
  plugin-contributed cards.
- Operating Systems moved from PCs to General Reference: OS is cross-asset
  (PCs, machines, measuring tools, network devices all run one).

Plus docs/proposals/ge-enforce-plugin.md: a planning doc for refactoring
GE-Enforce/DSC into a shopdb plugin (manifest as shopdb data, payloads on
SMB/HTTP/inline), grounded in the real manifest schema.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-12 15:29:08 -04:00
cproudlock
f6dcaef4c0 Persist list pagination and search in the URL
All checks were successful
CI / backend (push) Successful in 1m25s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
List pages kept the current page in local state, so clicking into an
asset and hitting Back remounted the list at page 1. A shared
useListQuery composable now mirrors the page (and search term) into the
URL query via router.replace across all 18 list pages, so Back restores
the page you were on and lists are deep-linkable. Page 1 with no search
stays a bare path; changing a filter resets to page 1; unrelated query
keys are preserved.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 14:19:50 -04:00
cproudlock
b2e1827e5d Normalize the asset detail card layout across all five pages
All checks were successful
CI / backend (push) Successful in 1m25s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
One canonical card order applied to machines, PCs, printers, network,
and measuring tools: Identity -> type-specific -> status -> Location &
Organization -> domain -> Custom Fields -> Warranty -> Relationships ->
Notes -> audit footer, with a documenting comment on each page so they
stop drifting. The location card is Location & Organization everywhere;
the network detail page is rebuilt into the family (Asset Information
folded into Identity, Record Info retired for the standard audit
footer, a Location & Organization card added). Fixed two latent bugs
found in the process: printer detail had no audit footer, and the
network Record Info read datecreated/datemodified (not in the payload)
so its timestamps rendered blank - the footer now uses the correct
fields.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 14:13:15 -04:00
cproudlock
275224822e Add collector PC->printer links and searchable custom fields
All checks were successful
CI / backend (push) Successful in 1m24s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
Collector: the computers collector schema gains defaultprinter and
printers; apply_collector_payload resolves each reported identifier to
a printer asset (windowsname/hostname/sharename/assetnumber/IP,
first-hit case-insensitive) and idempotently syncs relationships -
defaultprinter (directional) for the default, connectedto for the
rest. Collector-created rows are tagged so a re-report archives dropped
links while manual relationships are never touched; unresolved
identifiers warn instead of failing. Both PC and printer detail pages
show the links via the shared relationships card (no frontend change).
GE-Enforce Win32_Printer collection snippet documented.

Searchable custom fields: a per-field searchable flag (migration 7d24);
global search matches custom-field values on flagged active fields and
routes each hit to the asset detail page, reusing the existing
(type,id) dedupe and search_<type>_enabled domain filter. Searchable
toggle on the Custom Fields settings page.

822 tests pass; both verified live.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 14:01:20 -04:00
cproudlock
9daf1578a7 Balance the two-column card layout on detail pages
All checks were successful
CI / backend (push) Successful in 1m24s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 7s
The content grid hand-assigned cards to a fixed left/right split, so a
PC or machine with many tall cards (installed apps, warranty,
relationships) piled them all on one side. Switched .content-grid to a
balanced CSS multicolumn flow (display:contents flattens the wrappers
so no markup changes), with break-inside:avoid keeping cards intact.
Cards now distribute by height and the columns stay even on every asset
detail page.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 13:41:18 -04:00
cproudlock
d187a2c535 Fix PC installed-applications rendering; minor UI cleanups
All checks were successful
CI / backend (push) Successful in 1m24s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
The PC detail Installed Applications section 500d and vanished on any
real PC: ComputerInstalledApp had no to_dict, so the endpoint errored
and the v-if hid the section. Added the serializer (curated version
wins over the raw collected string, app name + description included)
and aligned PCDetail to the flat payload; regression test added.

Also: employee detail skips its USB panels when the usb plugin is
disabled (was firing 404s), and the shopfloor kiosk header is now
light-on-dark for readability.

810 tests pass; PC 259 installed apps verified live.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 13:37:58 -04:00
cproudlock
68b6949f1c Print a single label at a chosen ULINE sheet cell
All checks were successful
CI / backend (push) Successful in 1m24s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
The single-label page gains an Output toggle (standalone vs place on a
ULINE 6-up sheet) with a 2x3 cell picker, so one label can be printed
into the correct physical position on a partially-used sheet - the
single-label equivalent of the batch page start-cell offset. Batch page
unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 13:09:55 -04:00
cproudlock
cced6fe96e Release 0.7.0
All checks were successful
CI / backend (push) Successful in 1m23s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
Cuts the large post-0.6.0 pile as a pinnable release: the machines
rename (ADR-011), the API import surface, personal/scoped/collector
API tokens and the get_permissions plugin hook, the ADR-010 frontend
hook contract, per-plugin migrations, model/employee photos, the
dualpath single-machine toggle and relationship propagation, support
teams, email sending, and the shared asset label generator. Plugin
contract moved 0.6.0 -> 0.10.0 over this range (distinct series per
ADR-007).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 13:00:58 -04:00
cproudlock
55b42fed00 Fix sidebar footer overflow from the change-password link
All checks were successful
CI / backend (push) Successful in 1m23s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
Adding the Change Password action turned the user menu into a
full-width-button row that overflowed the fixed sidebar. The footer is
now an identity row (avatar + truncating name) above a compact
icon-button actions row (Password / Logout), both within bounds.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 12:28:08 -04:00
cproudlock
4b20c2eb56 Add batch ULINE label printing for all asset types
All checks were successful
CI / backend (push) Successful in 1m24s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
A multi-select batch sheet at /print/asset-label-batch/<type> lays
selected assets onto ULINE label pages (6-up 3x3 or dense 72-up mini),
with code-type and encode toggles matching the single label, plus a
start-cell offset for partial sheets. Print Labels buttons on all five
asset list pages. The per-type config and encode resolution are
extracted to a shared print/assetLabel.js used by both the single and
batch views. Measuring-tool batches encode each tool inspection-
operation code (decode-verified 0615).

808 tests pass.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 12:08:22 -04:00
cproudlock
a846587f39 Add email sending (service + 3 flows) and a general asset label generator
All checks were successful
CI / backend (push) Successful in 1m23s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
Email: a stdlib SMTP mailer (settings-first config, graceful no-op when
unconfigured), a test-email endpoint wired to the Email settings page,
forced first-login password change (users.mustchangepassword, migration
7d23, /change-password flow), new-user welcome mail, and on-demand
report/alert delivery (POST /api/reports/email + Email Report buttons)
with an external-cron-with-a-scoped-PAT path documented for automation.
All tests patch smtplib - no network.

Labels: a shared /print/asset-label/<type>/<id> view any asset detail
page opens - card or plain style, QR or barcode, configurable encoding.
Per-type qr_target_* templates plus label_default_style/codetype/encodes
settings on the Printing page. Measuring-tool labels default to encoding
their inspection-operation code (derived from the location name, e.g.
0615), so every tool in an area shares the area code - verified by
decoding the rendered QR. Machine labels default to the machine number;
blank-serial handled gracefully.

808 tests pass; both features verified live.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 11:58:30 -04:00
cproudlock
7d309aabeb Final-pass polish: support contact UX, audit tooltip, doc refresh
All checks were successful
CI / backend (push) Successful in 1m21s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
Support teams: contact management moved from a row expander to a modal
(Contacts (N) button per team); application detail Support card and the
modal show Email (mailto) and Teams chat buttons for contacts with an
SSO, derived as sso@ + a new contact_email_domain site setting
(default geaerospace.com, blank hides the buttons).

Audit log: hovering a user SSO shows the full name, resolved
best-effort from the employee directory in either mode.

Docs/hygiene from a standards review: CLAUDE.md active-state,
CONTRACT-STABILITY.md and README brought to contract 0.10.0 / 11
plugins / migration head 7d22; get_asset_panels endpoint path fixed in
the hook docstring; leftover debug console.logs removed.

781 tests pass; contacts modal, action-button hrefs, and the audit
tooltip verified live.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 11:22:01 -04:00
cproudlock
1c5128a5c8 Fit the audit-log table; drop the collector token preset button
All checks were successful
CI / backend (push) Successful in 1m20s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
Audit Logs now uses a fixed table layout with tuned column widths:
every column fits the card without horizontal scrolling, long entity
names truncate with a hover tooltip, timestamps show a compact
no-seconds form (full value on hover), and the 9-digit user SSO renders
fully.

Removed the Collector service token quick-preset from the create modal
per user preference - the collector.ingest checkbox in the scope grid
is the path now.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 09:57:02 -04:00
cproudlock
12da0429b0 Preserve destination on session-expiry login; compact the tokens table
All checks were successful
CI / backend (push) Successful in 1m20s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
The 401 interceptor (session expired) redirected to bare /login,
losing the page the user was on - the router guard already preserved
it but the expiry path bypassed the guard. The interceptor now carries
?redirect= like the guard does, and Login returns there.

The API tokens tables overflowed the card: the token column no longer
repeats the shopdb_pat_ prefix per row (short prefix shown, full form
on hover) - the token itself was already never displayed after
creation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 09:33:02 -04:00
cproudlock
7dfbe7bf8a Add the get_permissions plugin hook (contract 0.10.0)
All checks were successful
CI / backend (push) Successful in 1m20s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
Plugins declare their own RBAC permissions instead of core accumulating
them: 36 permissions moved out of the core catalog into the 9 owning
plugins (core keeps the 19 its own blueprints enforce). The catalog is
resolved dynamically (core + enabled plugins) and feeds the roles grid,
the token scope picker and ceiling, and flask seed permissions;
installing or enabling a plugin seeds its permissions automatically. A
disabled plugin drops out of the assignable catalog while existing role
links keep working. New plugins - bundled or external - now bring their
permissions with zero core edits.

781 tests pass; live-verified with a machines.edit-scoped token.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 09:29:55 -04:00
cproudlock
12175169e4 Accept managed collector service tokens on the collector API
All checks were successful
CI / backend (push) Successful in 1m21s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
A token scoped to the new collector.ingest permission is a collector
service token: the collector endpoints accept it via X-API-Key or
Bearer alongside the env fleet keys (which remain the fallback), giving
the fleet credential rotation, revocation, and last-used visibility
from the API Tokens page. Containment holds both ways: a collector
token authorizes nothing else, and no other credential gains collector
access. Shared token validation refactored out of the auth shim; a
Collector service token quick-preset in the create modal; integration
guide documents minting, rotation via site-config.json, and the
service-identity pattern.

765 tests pass; live acceptance matrix verified.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 09:13:33 -04:00
cproudlock
848a8fb34f Add optional permission scopes to API tokens
All checks were successful
CI / backend (push) Successful in 1m19s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
A token may carry a scopes list: it then grants only those permissions,
intersected with what the owner holds at use time, with the admin role
bypass suspended and role-gated routes denied - a scoped token from an
admin account is genuinely limited. Scope ceiling enforced at
create/update too (only permissions the owner holds; 400 lists
violations) and the picker only offers what you hold. Token management
itself now requires the new apitokens.create permission (admin by
default, grantable via roles). Unscoped tokens keep the exact prior
act-as-owner behavior; imports need an unscoped admin token.
Migration 7d22.

756 tests pass; live-verified scoped 201/403 matrix.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 08:58:31 -04:00
cproudlock
688ff6646d Add the missing changelog entry for the integration-gap fixes
All checks were successful
CI / backend (push) Successful in 1m16s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 08:33:47 -04:00
cproudlock
da86b3ae0c Add personal API tokens; wire measuring tools into remaining surfaces
Some checks failed
CI / naming (push) Has been cancelled
CI / frontend (push) Has been cancelled
CI / backend (push) Has been cancelled
API tokens: any user mints named, optionally-expiring tokens
(shopdb_pat_..., sha256-stored, secret shown once) at Settings > API
Tokens; a before-request shim swaps a valid PAT for a request-scoped
JWT of its owner, so the entire existing auth/authz/import-mode stack
works unchanged and revoked/expired tokens 401 cleanly. Built for
long-running scripts - the legacy import no longer dies when a login
JWT expires. Migration 7d21_apitokens; create/revoke audit-logged.

Audited integration gaps fixed: Asset.to_dict serializes measuring
tools (typedata + pluginid - relationship links to tools resolve); map
subtype filter/colors and MapEditor include them; dashboard totals
count them; warranty links use a new by-asset route; the measuringtools
ADR-010 hooks are real (corrected presentation token, implemented
map-overlay endpoint); the login avatar resolves through the
employee-photo helper.

737 tests pass; naming green; frontend builds; both features verified
live end-to-end.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 08:33:02 -04:00
cproudlock
64a5abdb08 Wire measuring tools into identifiers and global search
All checks were successful
CI / backend (push) Successful in 1m14s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
The measuringtools plugin was missing from two cross-cutting surfaces:
the asset-identifier matrix (no measuring_tool column or per-type
keys - gauge lab reference is their primary identifier) and global
search (results fell to a generic URL and gaugelabreference was never
searched). Measuring tools now have identifier toggles, gated
gauge-lab and maintenance-reference fields on their form and detail,
a search domain toggle, gage-tag search, and proper labels, routes,
and filter chips in search results.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 08:11:39 -04:00
cproudlock
31f5e07294 Keep the settings rail alive across child navigation
All checks were successful
CI / backend (push) Successful in 1m13s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
Keying the outer router-view on route.path made every settings click
remount the whole settings shell, resetting the rail scroll to the
top. The key now treats /settings/* as one unit so the shell (and its
scroll position) persists while child pages swap; detail-to-detail
remounts elsewhere are unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 07:56:39 -04:00
cproudlock
9fc5aca63b Group Site and Facility settings by concept
All checks were successful
CI / backend (push) Successful in 1m14s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
Fields rendered in arbitrary key order (dualpath toggle first,
facility name buried, setup flag mid-list). Now grouped with headers:
Identity, Behavior, Naming and Patterns, Data Sources, System - with
unknown future keys falling into Other at the end.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 07:53:45 -04:00
cproudlock
1e93b3d570 Dissolve System Settings into individual settings pages
Some checks failed
CI / backend (push) Successful in 1m13s
CI / naming (push) Successful in 1s
CI / frontend (push) Has been cancelled
The monolithic tab page competed with the settings rail as a second
navigation system, and its Integrations tab was a dumping ground. Each
section is now its own routed rail page (ServiceNow, Zabbix Supplies,
Dell Warranty, Collector PC Types, Branding, Floor Map, Printing and
Labels, Email/SMTP, Audit, Authentication, Asset Identifiers, Global
Search), thin over a shared useSystemSettings composable, grouped
logically in the rail with system groups clustered last. Old
/settings/system?tab= URLs redirect to the right page.

Also fixes the post-login redirect: the auth guard now remembers the
intended destination and Login returns there (same-site paths only).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 07:52:21 -04:00
cproudlock
5393846b8d Polish the Site and Facility settings page
All checks were successful
CI / backend (push) Successful in 1m12s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
Booleans render as checkbox toggles, the employee/USB directory-mode
settings as selfhosted/external dropdowns, and every field has a label
and help text (raw keys and type-true/false text boxes are gone).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 07:14:06 -04:00
cproudlock
7bde765f1a Render vendor-model photos on asset detail heroes
All checks were successful
CI / backend (push) Successful in 1m14s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
Computer and Printer payloads now surface the linked model imageurl the
way machines already did, and the machine/PC/printer detail heroes
render the photo when present (network devices and measuring tools
have no model link, so nothing to surface). Absent images render
nothing rather than a broken icon.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 06:39:11 -04:00
cproudlock
b130ef43f3 Add the dualpath-as-single-machine site toggle
All checks were successful
CI / backend (push) Successful in 1m13s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
Most facilities consider a Dualpath pair one physical dual-bay machine.
New site setting dualpath_single_machine (default on): the machines
list, dashboard counts, machines-by-type report, and floor map collapse
each pair to its primary bay (lower assetnumber), with combined
2007 / 2008 labels; pagination totals stay honest. Detail pages remain
per-bay and always show a dual-bay sibling banner linking the partner.
Pair resolution lives in core services and joins the plugin contract
surface (0.8.0 -> 0.9.0).

On the WJ dataset: 31 pairs collapse, machine counts 262 -> 231, map
470 assets. Toggle verified live in both states, left on.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 06:23:39 -04:00
cproudlock
5a192f3100 Sync System Settings tabs with the URL query
All checks were successful
CI / backend (push) Successful in 1m12s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
activeTab read ?tab= only once at mount, so settings-rail links that
change just the query (Branding, Floor Map) updated the URL without
switching the panel. The query param is now the source of truth: a
watcher applies rail/back/forward navigation and tab clicks write the
query via router.replace.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 06:12:58 -04:00
cproudlock
4fd110a33d Wire relationship directionality and dualpath controls propagation
All checks were successful
CI / backend (push) Successful in 1m13s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
Symmetric relationship types (isdirectional flag, migration 7d19) show
one entry per peer on the relationships card - a Dualpath pair no
longer lists its partner twice - and directional types read naturally
instead of Outgoing/Incoming. Deleting a collapsed entry removes every
underlying direction row.

Propagation is now real (migration 7d20): relationship types declare
propagation-through pairs in relationshiptypepropagations (M:N,
replacing the never-consumed single column); creating a controls link
on either bay of a Dualpath pair auto-creates it on the partner,
mirrored across both endpoints because live data stores controls as
bay -> PC. flask relationships propagate backfills existing data (29
rows fanned out on the WJ dataset, idempotent).

This also completes the tree that commit 1d21bf0 accidentally split
(core/models/__init__ imported RelationshipTypePropagation ahead of the
file that defines it), returning CI to green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-12 06:07:36 -04:00
cproudlock
1d21bf0206 Add photo management for models and employees; fix stale detail navigation
Some checks failed
CI / backend (push) Failing after 9s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
Model photos: upload/replace/delete on /api/models/<id>/image (admin),
stored under instance/modelimages/ with a public serve route; thumbnail
plus Upload/Replace/Remove controls in the Models settings modal; the
URL field remains as a manual alternative.

Employee photos, mode-aware: self-hosted directory employees get
upload/replace/delete (photo-<sso> under instance/employeephotos/,
employees plugin migration 0002); external directory mode passes the
HR-supplied picture URL through read-only (writes 409). One resolver
feeds both consumers - the shopfloor recognition/recert kiosk cards and
the employee detail hero - in either mode.

Navigation fix: router-view is keyed on route path, so following a
relationship link between two assets of the same type (machine ->
dualpath machine) reloads the page instead of showing stale content;
query-only URL changes still avoid a remount.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 21:00:37 -04:00
cproudlock
7dae281993 Add application support teams with contacts
All checks were successful
CI / backend (push) Successful in 1m7s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 8s
Replaces the legacy supportteams/appowners pair: supportteams
(teamname unique, teamurl ServiceNow link) + supportteamcontacts
(multiple named contacts with SSO per team, the people you reach out
to), applications.supportteamid intact. Migration 7d18 migrates each
legacy team owner into a contact, drops appowners, and has a validated
downgrade. New /api/supportteams CRUD (admin writes, import-mode
timestamps, teamname lookup), Support card on application detail,
contacts column on the list, and a settings management page.
IMPORT-API.md mapping updated to the concrete endpoints.

658 tests pass; live dev migration applied (24 teams / 24 contacts);
fresh-install and downgrade round-trips verified on scratch DBs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 20:29:12 -04:00
cproudlock
46e50c07ff Add the API import surface for legacy migrations (contract 0.8.0)
All checks were successful
CI / backend (push) Successful in 1m4s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
Goal: an LLM or script can migrate an entire legacy database using only
the HTTP API - original history preserved, safely re-runnable.

- X-Import-Mode header (admin only): create/update endpoints across 15
  timestamped entity types accept original createddate/modifieddate;
  helper exposed via shopdb.api (contract 0.7.0 -> 0.8.0).
- Exact-match natural-key lookup filters on 13 list endpoints for the
  lookup-then-upsert recipe.
- Selfhosted USB checkout/checkin accept backdated event times in
  import mode.
- docs/IMPORT-API.md: operator manual grounded in the real legacy
  schema - order of operations, full table-by-table mapping including
  the machines fan-out, idempotent Python importer with dry-run, parity
  checks, and decided dispositions for unmigrated tables (DNC config
  stays live-fed via the collector; supportteams/appowners map to the
  upcoming supportteams model).

635 tests pass; naming green; frontend untouched.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 20:10:46 -04:00
cproudlock
f8e5109255 Pin selfhosted directory modes in the authz sweep
All checks were successful
CI / backend (push) Successful in 1m2s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 7s
The sweep exercises usb/employees handlers, which default to EXTERNAL
directory mode when no setting row exists - green on the dev box where
the external MySQL databases happen to exist, red in CI where they do
not. Seed selfhosted mode in an autouse fixture so the guard is
deterministic everywhere. Verified by running the suite with the
external DB hosts pointed at an unreachable address.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 19:56:24 -04:00
cproudlock
24f67d6ac5 Accept + implement ADR-010 frontend plugin hooks (contract 0.7.0)
Some checks failed
CI / backend (push) Failing after 1m2s
CI / naming (push) Successful in 1s
CI / frontend (push) Successful in 7s
Four data-only hooks on BasePlugin (get_settings_cards,
get_asset_panels, get_map_overlays, get_asset_presentation) with a
GET-only /api/pluginui consumer surface copying the dashboard-widgets
semantics. Pilots: warranty declares its asset panel; measuringtools
supplies its settings card, presentation, and calibration overlay -
the last hardcoded settings-nav entry is now hook-sourced. Generic
renderers for panels/overlays/presentation deferred per the ADR's
incremental adoption plan (documented in CONTRACT-STABILITY.md).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 19:38:32 -04:00
cproudlock
9eed3745fc Add parametrized authz sweep guard over all mutating routes
Every non-exempt POST/PUT/PATCH/DELETE must 403 a role-less member
and pass authz for admin; exemptions (auth, collector, setup wizard,
kiosk click-through, admin-or-self user update) are documented in
the test. Any future unguarded mutation fails CI as its own case.
Sweep confirmed existing gating complete: zero routes needed fixes;
lockout already implemented.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 19:35:57 -04:00
527 changed files with 72482 additions and 7418 deletions

View File

@@ -49,6 +49,10 @@ MYSQL_ROOT_PASSWORD=CHANGE_ME_ROOT_PASSWORD
MYSQL_PASSWORD=CHANGE_ME_APP_PASSWORD
MYSQL_PORT=3306
API_PORT=5001
# Air-gapped deploy only (docker-compose.airgap.yml): the loaded image tag,
# which MUST match what build-offline-bundle.ps1 -Version produced. Ignored by
# the connected build template (docker-compose.yml builds from source).
IMAGE_TAG=0.7.0
# ---- Zabbix integration (optional, for printer supply monitoring) ----
ZABBIX_URL=
@@ -61,6 +65,25 @@ ZABBIX_TOKEN=
# COLLECTOR_API_KEY=
# COLLECTOR_API_KEY_COMPUTERS=
# ---- Trusted plugin publisher keys (ADR-013, optional) ----
# Public-key PEM paths (OS path separator: ':' on Linux, ';' on Windows) used
# to verify signed plugin artifacts. Delivered with this config, NEVER from the
# plugin shelf. Empty on a site that does not adopt marketplace plugins.
# PLUGIN_TRUSTED_KEYS=/etc/shopdb/keys/curator.pub
#
# Enforce signatures: a plugin only loads/migrates if its tree matches a
# trusted signature. Default off. Turn on only after stamping plugins
# (flask plugin stamp-bundled) and pinning keys above.
# PLUGIN_REQUIRE_SIGNED=false
#
# Dev-only: directories whose UNSIGNED plugins are trusted, honored ONLY under
# DEBUG/TESTING (external-repo/symlink dev). Production ignores this.
# PLUGIN_DEV_TRUST_DIRS=/home/dev/my-plugin-repo
#
# Read-only folder the app pulls plugin artifacts from (a SharePoint-synced or
# copied shelf). The app reads this folder; it never speaks any network.
# PLUGIN_SHELF_DIR=/srv/shopdb/plugin-shelf
# ---- Employee directory database (optional, read-only) ----
# Separate HR/employee lookup DB consumed by the notifications plugin and the
# public shopfloor kiosks. Leave unset if the feature is not used; there is no
@@ -79,3 +102,10 @@ ZABBIX_TOKEN=
# CMMC_USB_DB_USER=
# CMMC_USB_DB_PASSWORD=
# CMMC_USB_DB_NAME=cmmc_usb
# ---- Subpath deployment (optional) ----
# Serve the app under a URL prefix instead of the server root, e.g. as an IIS
# Application at /ops under an existing site. The frontend must be rebuilt with
# the matching base: VITE_BASE_PATH=/ops/ npm run build. Leave unset when the
# app owns its own site/port (the default). See docs/INSTALL-WINDOWS-IIS.md.
# MOUNT_PATH=/ops

5
.gitattributes vendored
View File

@@ -1 +1,6 @@
.env filter=git-crypt diff=git-crypt
# Shell scripts must stay LF so Git Bash on Windows can run them
*.sh text eol=lf
scripts/check-naming-and-style.sh text eol=lf
.githooks/pre-commit text eol=lf

View File

@@ -5,11 +5,19 @@
# for the host distro). The backend job uses the system python3 in a venv
# instead. setup-node works because node is resolved differently.
#
# Three jobs run on push and pull_request:
# backend - pytest (tests use in-memory SQLite via TestingConfig, so no
# database service is needed).
# naming - the CONTRIBUTING.md naming/style gate.
# frontend - Vue build.
# Jobs run on push and pull_request:
# backend - pytest (tests use in-memory SQLite via TestingConfig, so no
# database service is needed).
# naming - the CONTRIBUTING.md naming/style gate.
# frontend - Vue build.
# migrations-mysql - proves the REAL multi-site deploy path: a fresh
# `flask db upgrade` + per-plugin install on utf8mb4 MySQL
# from empty, idempotent on a second run. The pytest suite
# only exercises SQLite create_all(), so without this a
# regression in the Alembic chain on MySQL would ship
# undetected. Needs a runner that supports service
# containers; if yours does not, run these steps against a
# host MySQL instead.
name: CI
@@ -54,3 +62,60 @@ jobs:
npm ci
npm run build
working-directory: frontend
migrations-mysql:
runs-on: ubuntu-latest
services:
mysql:
image: mysql:8.0
env:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: shopdb_ci
ports:
- 3306:3306
options: >-
--health-cmd="mysqladmin ping -h localhost -uroot -proot"
--health-interval=5s --health-timeout=5s --health-retries=20
env:
DATABASE_URL: mysql+pymysql://root:root@127.0.0.1:3306/shopdb_ci?charset=utf8mb4
SECRET_KEY: ci-secret
JWT_SECRET_KEY: ci-jwt-secret
steps:
- name: Check out
uses: actions/checkout@v4
- name: Install dependencies
run: |
python3 -m venv .venv
.venv/bin/pip install --upgrade pip
.venv/bin/pip install -r requirements.txt
- name: Force utf8mb4 on the CI database
run: |
mysql -h 127.0.0.1 -uroot -proot -e \
"ALTER DATABASE shopdb_ci CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;"
- name: Fresh core upgrade from empty
run: .venv/bin/flask db upgrade
- name: Install every bundled plugin (runs its chain)
run: |
for p in computers employees geenforce knowledgebase machines \
measuringtools network notifications printers slides usb warranty; do
.venv/bin/flask plugin install "$p"
done
- name: Assert schema built + utf8mb4, and a second upgrade is a no-op
run: |
.venv/bin/python - <<'PY'
from shopdb import create_app
from shopdb.extensions import db
from sqlalchemy import text
app = create_app()
with app.app_context():
insp = db.inspect(db.engine)
tables = insp.get_table_names()
assert len(tables) >= 70, f'only {len(tables)} tables built'
row = db.session.execute(text(
"SELECT default_character_set_name FROM information_schema.schemata "
"WHERE schema_name = 'shopdb_ci'")).first()
assert row[0] == 'utf8mb4', f'charset is {row[0]}, not utf8mb4'
print(f'OK: {len(tables)} tables, charset {row[0]}')
PY
- name: Second core upgrade must be a clean no-op
run: .venv/bin/flask db upgrade

5
.githooks/pre-commit Executable file
View File

@@ -0,0 +1,5 @@
#!/usr/bin/env bash
# Committed pre-commit hook. Activate per clone with:
# git config core.hooksPath .githooks
# Runs the naming/style gate; blocks the commit on failure.
exec bash "$(git rev-parse --show-toplevel)/scripts/check-naming-and-style.sh"

87
.github/copilot-instructions.md vendored Normal file
View File

@@ -0,0 +1,87 @@
# Copilot instructions for shopdb-flask
Follow these when suggesting code. They are enforced by a naming/style hook and
by CI (`.github/workflows/ci.yml`) - suggestions that break them fail the build.
`CONTRIBUTING.md` is the full authority; this is the short version.
## Naming (LOCKED - the hook rejects violations)
- **DB tables**: lowercase, concatenated, plural. No underscores, no dashes.
`machines`, `networkdevices`, `businessunits` - NOT `machine_types`, `BusinessUnits`.
- **DB columns**: lowercase, concatenated, singular. No underscores.
`machineid`, `lastzabbixsync`, `isactive` - NOT `machine_id`, `last_zabbix_sync`.
- **Foreign keys**: referenced table (singular) + `id`: `locationid`, `vendorid`.
- **Booleans**: `is`/`has` prefix: `isactive`, `isshopfloor`.
- **Index names**: `idx_<table>_<column>` (underscores allowed here only).
### Python
- A variable, attribute, function, or dict key that holds a DB value MUST match
the column name exactly - do NOT convert to snake_case.
Column `machineid` -> `Machine.machineid`, `{"machineid": 1}`, local `machineid`.
- Pure code that does NOT mirror a DB field uses normal snake_case (PEP 8):
`loop_count`, `current_user`, `validate_input()`.
- Classes: PascalCase, spelled out (`NetworkDevice`, `AssetType`).
### JavaScript / Vue
- A JS variable holding an API field value matches the API key exactly - do NOT
camelCase it. API `{"machineid": 1}` -> `response.machineid`, never `machineId`.
- Components: PascalCase (`AssetDetail.vue`). CSS classes: lowercase-with-dashes.
### API
- Endpoints: lowercase plural nouns, no underscores/dashes: `/api/networkdevices`.
- Query params + response keys match column names: `?locationid=5`,
`{"machineid": 1, "lastzabbixsync": "..."}`.
### Allowed acronyms only
Universal: id url api http https json jwt sql os ip dns csv pdf cors ttl uuid
html css orm. Domain: cmm cnc pc usb vnc winrm ssh ssl tls tcp udp smtp ldap
vlan sso dnc focas clm mtconnect. Anything else: spell it out.
### Banned shorthand
Never use `cfg ctx mgr req res env util helper` or `db` as a standalone variable
name. Spell out: `config context manager request response environment utilities`.
`_bp` is fine only as a suffix with a meaningful prefix (`printers_bp`).
## Style (ASCII only)
- NO emojis anywhere - code, comments, strings, UI.
- NO em-dashes, en-dashes, Unicode arrows, or smart quotes. Plain ASCII only.
- Comments default to NONE. Add one only when the WHY is non-obvious; keep inline
`#`/`//` comments terse. Docstrings stay normal English.
- Dark theme is the default; keep UI functional and professional.
## Architecture (do not violate)
- **Plugins are the product.** Plugin code lives in `plugins/<name>/{models,api,services,schemas}/`
with a `manifest.json` (single source of truth: name, version, dependencies,
api_prefix) and a `BasePlugin` subclass in `plugin.py`.
- **Plugins never import core internals.** Use the contract surface `shopdb.api`
(e.g. `from shopdb.api import db, Asset, success_response`). Adding to that
surface is a contract-version bump + a `docs/PLUGIN-HOOKS` update in the same PR.
- **Migrations, never `db.create_all()`.** The core Alembic chain is in
`migrations/versions/`; each plugin owns its own chain under
`plugins/<name>/migrations/`. New schema = a new migration with an idempotent
guard and a real downgrade. Migrations must run clean on strict MySQL 8.
- **Asset model is the platform contract.** Physical things are an `Asset` plus a
plugin subtype row linked by `assetid` (FK, `ON DELETE CASCADE`). Consumables
with quantities are standalone tables, not assets.
- **Ledger pattern**: a cached `quantityonhand` moves in the SAME commit as the
signed transaction row it reflects.
## Before you finish a change
Run the three gates (CI runs the same):
```
python -m pytest tests/ -q
cd frontend && npx vitest run && npm run build && cd ..
bash scripts/check-naming-and-style.sh
```
Commits: short present-tense subject, body says WHY. No AI/tool attribution in
commit messages, code comments, or docs.

106
.github/workflows/ci.yml vendored Normal file
View File

@@ -0,0 +1,106 @@
# CI for shopdb-flask on GitHub Actions.
#
# Mirrors the internal CI pipeline. Four jobs on push + pull_request:
# backend - pytest (SQLite via TestingConfig, no DB service needed)
# naming - the CONTRIBUTING.md naming/style gate
# frontend - vitest + Vue build
# migrations-mysql - the REAL multi-site deploy path: fresh flask db upgrade
# + every plugin's chain on utf8mb4 MySQL 8, idempotent on
# a second run. The pytest suite only exercises SQLite
# create_all(), so this is what catches an Alembic
# regression on MySQL before it ships.
name: CI
# Jobs run on the org's self-hosted "arc-runner-set" (enterprise
# ge-aerospace-runner-group, Linux). GitHub-hosted runners are blocked by the
# org IP allow list (hosted Azure runner IPs are not allow-listed -> checkout
# 403), so ubuntu-latest cannot be used here. arc-runner-set checks out from an
# internal allow-listed IP and, being Linux, still supports service containers.
on:
push:
pull_request:
jobs:
backend:
runs-on: arc-runner-set
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.14'
cache: pip
- run: pip install -r requirements-dev.txt
- run: python -m pytest -q
naming:
runs-on: arc-runner-set
steps:
- uses: actions/checkout@v4
- run: bash scripts/check-naming-and-style.sh
frontend:
runs-on: arc-runner-set
defaults:
run:
working-directory: frontend
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'
cache: npm
cache-dependency-path: frontend/package-lock.json
- run: npm ci
- run: npx vitest run
- run: npm run build
lean-build:
# ADR-013 Phase 5: prove a per-site build carries only its chosen plugins.
# Builds a lean site (machines + printers) and asserts an omitted plugin's
# code is absent from the bundle - the delete-a-plugin guarantee in CI.
runs-on: arc-runner-set
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '20'
cache: npm
cache-dependency-path: frontend/package-lock.json
- run: cd frontend && npm ci
- name: Build lean site (machines + printers)
run: |
printf '{ "site": "ci-lean", "plugins": ["machines", "printers"] }' \
> /tmp/lean-profile.json
bash scripts/build-site.sh /tmp/lean-profile.json /tmp/leansite
- name: Assert omitted plugin code is absent, chosen present
run: |
assets=/tmp/leansite/frontend-dist/assets
for code in PartsKiosk ManifestEditor USBLabelBatch KnowledgeBaseDetail; do
if grep -rqoh "$code" "$assets"/*.js; then
echo "FAIL: omitted-plugin code '$code' leaked into the lean bundle"
exit 1
fi
done
for code in MachineDetail PrinterDetail; do
grep -rqoh "$code" "$assets"/*.js || {
echo "FAIL: chosen-plugin code '$code' missing from the lean bundle"
exit 1; }
done
# Core frontends (no manifest, e.g. applications) must ship in EVERY
# build regardless of SITE_PLUGINS, or a lean site loses a core page.
grep -rqoh "ApplicationsList" "$assets"/*.js || {
echo "FAIL: core page 'ApplicationsList' missing from the lean bundle"
exit 1; }
test -d /tmp/leansite/plugins/machines
test ! -d /tmp/leansite/plugins/printedparts
echo "lean build verified: only chosen plugins present"
# NOTE: the MySQL-8 migration/seed job (fresh `flask db upgrade` + every
# plugin chain + strict-mode seeders on a real MySQL 8) runs on the internal
# CI server, which supports service containers. The org's arc-runner-set is
# Kubernetes/ARC without docker-in-docker, so GitHub Actions service
# containers ("services: mysql") are unavailable here ("Job Container is
# required"). That coverage stays on the internal CI rather than being
# duplicated on GitHub.

10
.gitignore vendored
View File

@@ -28,7 +28,11 @@ env/
# IDE
.idea/
.vscode/
.vscode/*
# Share the team's launch/tasks/extensions; keep personal settings out.
!.vscode/launch.json
!.vscode/tasks.json
!.vscode/extensions.json
*.swp
*.swo
*~
@@ -75,3 +79,7 @@ secrets.yml
*_secret
*_secrets
credentials.json
scripts/site_imports/wjf/idmap.json
# work-PC publication clone: bundle drop folder for the transfer pipeline
_transfer/

9
.vscode/extensions.json vendored Normal file
View File

@@ -0,0 +1,9 @@
{
"recommendations": [
"ms-python.python",
"ms-python.vscode-pylance",
"Vue.volar",
"dbaeumer.vscode-eslint",
"ms-azuretools.vscode-docker"
]
}

30
.vscode/launch.json vendored Normal file
View File

@@ -0,0 +1,30 @@
{
"version": "0.2.0",
"configurations": [
{
"name": "Flask API (:5001)",
"type": "debugpy",
"request": "launch",
"module": "flask",
"cwd": "${workspaceFolder}",
"env": {
"FLASK_APP": "shopdb",
"FLASK_ENV": "development",
"FLASK_DEBUG": "1"
},
"args": ["run", "--port", "5001", "--no-reload"],
"jinja": true,
"justMyCode": false,
"console": "integratedTerminal"
},
{
"name": "Pytest (current file)",
"type": "debugpy",
"request": "launch",
"module": "pytest",
"cwd": "${workspaceFolder}",
"args": ["${file}", "-v"],
"console": "integratedTerminal"
}
]
}

46
.vscode/tasks.json vendored Normal file
View File

@@ -0,0 +1,46 @@
{
"version": "2.0.0",
"tasks": [
{
"label": "Backend: flask run (:5001)",
"type": "shell",
"command": "${workspaceFolder}/venv/bin/flask run --port 5001",
"options": {
"cwd": "${workspaceFolder}",
"env": { "FLASK_APP": "shopdb" }
},
"windows": {
"command": "${workspaceFolder}\\venv\\Scripts\\flask.exe run --port 5001"
},
"isBackground": true,
"problemMatcher": []
},
{
"label": "Frontend: npm run dev (:5173)",
"type": "shell",
"command": "npm run dev",
"options": { "cwd": "${workspaceFolder}/frontend" },
"isBackground": true,
"problemMatcher": []
},
{
"label": "Dev site (backend + frontend)",
"dependsOn": [
"Backend: flask run (:5001)",
"Frontend: npm run dev (:5173)"
],
"dependsOrder": "parallel",
"problemMatcher": []
},
{
"label": "Check: naming + tests + build",
"type": "shell",
"command": "bash scripts/check-naming-and-style.sh && venv/bin/python -m pytest tests/ -q && cd frontend && npx vitest run && npm run build",
"windows": {
"command": "bash scripts/check-naming-and-style.sh && venv\\Scripts\\python -m pytest tests/ -q && cd frontend && npx vitest run && npm run build"
},
"options": { "cwd": "${workspaceFolder}" },
"problemMatcher": []
}
]
}

View File

@@ -10,6 +10,472 @@ ADR-007 and ADR-002.
## [Unreleased]
## [0.8.0] - 2026-08-05
First release to carry the Windows installer. Everything below shipped after
v0.7.0 was tagged, and a complete install was exercised end to end on Windows
Server 2019 before this release was cut.
### Fixed - installer, from a real Server 2019 install
- `packaging` was imported by the plugin loader but declared nowhere. It reached
development and CI only as a dependency of pytest, so the whole suite passed
while a virtual environment built from `requirements.txt` alone - which is
exactly what the installer builds - could not import the application at all.
`tests/test_runtime_dependencies.py` now fails on any runtime import that is
not a declared dependency.
- IIS returned 500.52 before the application was ever launched. `web.config`
declared `<allowedServerVariables>` for the X-Forwarded-For rule, and that
section ships `overrideModeDefault="Deny"`, so the whole file was rejected.
The installer now permits the single variable at server level instead of
unlocking the section for every site on the machine, and repairs a
`web.config` an earlier build had made unusable.
- The config unlock ran before the application it unlocks existed, so the scoped
form could never succeed on a first install and every install silently fell
back to granting handler delegation server-wide.
- The stage 5 smoke test discarded the status code and error page it had already
received, reporting "site did not return 200" for a fault IIS had named. It
now records both, plus the tail of the application log.
- Database dumps were readable by every authenticated user: a directory created
under ProgramData inherits `Users:RX`, and the owner-only ACL was applied only
when the installer itself created it.
- The uninstaller ran the 32-bit PowerShell, which cannot see IIS, so the site,
pool and application survived a "successful" uninstall pointing at a deleted
directory.
- Uninstall matched applications by alias alone and would remove an unrelated
application of the same name under another site.
- Wizard input reached a command line unchecked: unvalidated ports, a drive-root
path that escaped its own quote, and a password written as ANSI but read back
as UTF-8, which reported a correct non-ASCII password as wrong.
- Plugin deregistration could never succeed - it omitted `--yes` against a
command that prompts - while the plugin directory was deleted regardless.
- Preflight rows were drawn past the bottom of the panel and silently vanished;
the failures loop had no cap at all.
### Changed - installer behaviour
- "Is this a re-run of my install?" is answered from a durable install record
rather than inferred from the state of the machine. Nothing on a server says
who created its database tables, so a retry after a failed first install was
taken for an upgrade of a working system: it demanded a mandatory backup of a
database its own failed attempt had written, then refused to prune tables it
had created minutes earlier. During unfinished first provisioning the backup
is advisory and prune may force; on an established install both are unchanged.
- The installer offers every bundled plugin, so one build serves any site
instead of one build per plugin profile.
### Added - operator documentation and diagnostics
- `docs/UPDATES-WINDOWS.md` - what operators should expect from future updates,
bug fixes and security releases, including downtime, what is preserved, and
the effect on other sites sharing the same IIS server.
- `docs/RELEASING-WINDOWS.md` - how to build and release, and the two known gaps.
- `deploy/windows/shopdb-diagnose.py` - collects what IIS answers, the config
lock state, the application logs and the ACLs in one pass, scrubbing secrets
before writing anything.
### Added - the installer itself
- Air-gapped Windows installer (`deploy/windows/installer/`). One `.exe` per
site, built from that site's plugin profile, containing Python, the wheels,
the SPA, the IIS modules and optionally MySQL. Operator docs:
`docs/INSTALL-WINDOWS.md` and `docs/OPERATE-WINDOWS.md`, both shipped onto the
server. `docs/INSTALL-WINDOWS-IIS.md` and `docs/DEPLOY-WINDOWS-IIS.md` are now
reference-only, for hand-built servers.
- `bundle-lock.json`: an exact sha256 + size record of the installer's
third-party payload (wheels, Python installer, IIS MSIs). Verified as set
equality by both builders and again on the server before anything runs; there
is no install-time override. Regenerate with `refresh-bundle-lock.ps1`.
- CycloneDX SBOM (`sbom.cdx.json`) generated on every build from
`requirements.txt` and `package-lock.json`, covering both ecosystems, staged
into the application tree so an air-gapped server can answer "do we carry this
component" locally: `shopdb-admin.ps1 verify -Path <name>`.
- `shopdb-admin.ps1`, the operator console: status, start/stop/restart, logs,
health check, backup, plugins, verify. `check -Json` emits secret-free
structured state for pasting into a support ticket or an AI assistant.
- `build-installer.ps1`, the whole build natively on Windows, so a work PC needs
no Bash. Shares `scripts/resolve_plugin_closure.py` with `build-site.sh`.
- URL Rewrite is bundled and the wizard asks where client IPs come from
(`-ClientIpSource direct|proxy`). Without the rule IIS sends no
`X-Forwarded-For` at all and every client reads as 127.0.0.1.
### Changed
- `requirements.txt` and `requirements-dev.txt` are compiled `--universal
--generate-hashes`. Installs run under `pip --require-hashes`, so a wheel whose
sha256 is not listed is refused. The dev lockfile is constrained to the
production pins; the two had drifted.
- Python 3.14 across the Dockerfile, CI, `web.config` and the docs, which
previously declared four different versions.
- The naming/style gate covers Markdown, JSON and YAML, not just code.
### Fixed
- `plugins/employees` imported `shopdb.core.models` directly, failing the
contract-surface test on `main` since the dashboard employee-name resolver
landed.
- `scripts/build-site.sh` copied all of `deploy/` into its own output directory,
which recursed when the output was staged inside it - the documented Windows
build could not complete.
- Plugin baseline migrations inherited the MySQL server's default charset: the
utf8mb4 compiler hook lived in `migrations/env.py` and so covered the core
chain only. It is now `shopdb/utils/mysql_charset.py`, imported by both.
`flask db-utils preflight` reports the database's default charset.
### Added
- Configurable site timezone: a `site_timezone` site setting (default
`America/New_York`, public-readable), editable in Settings > Site >
Localization. Notification start/end times are entered and displayed in this
zone, and daily-reset notification expiry is computed in it. A shared
`frontend/src/utils/datetime.js` (Intl-based, DST-safe) does the conversion.
### Changed
- Asset detail pages (machines, PCs, printers, network devices, measuring
tools) now share one canonical card skeleton: Identity -> type-specific ->
status -> Location & Organization -> domain -> Custom Fields -> Warranty ->
Relationships -> Notes -> audit footer. The location card reads "Location &
Organization" on every page. The network device page was rebuilt into the
family (its "Asset Information" folded into Identity, "Record Info" converted
to the standard audit footer). Printer Notes moved out of mid-page to
second-to-last and the printer gained an audit footer. Template reordering
only; no data or API changes.
### Fixed
- Notification start/end times were off by the timezone offset (a 2:34 PM entry
displayed as 6:34 PM). Times are now stored UTC and shown/entered in the site
timezone; the calendar keys all-day events off the site-local day.
- Kiosk displays showed a white screen on login: a legacy 32-bit kiosk
installer's autostart kept relaunching Edge at a now-dead URL. The install's
HKLM Run value was WOW64-redirected into `SOFTWARE\Wow6432Node\...\Run` and
survived earlier cleanup. The `gea-shopfloor-display` dispatcher now purges
the legacy autostarts every enforce cycle across both registry views, all
user hives, Run/RunOnce/policy-Run, and every Startup folder.
- List pages keep the current page (and search term) in the URL query, so
paging to page 9, opening an item, and hitting browser Back returns to page 9
instead of resetting to page 1. Applies to all 18 list views via a shared
useListQuery composable; page 1 with no search stays a bare path.
- PC detail Installed Applications no longer 500s and silently disappears on
real PCs (ComputerInstalledApp had no to_dict); the section renders app
name, version, and description again.
- Employee detail skips its USB panels when the usb plugin is disabled (no
more 404 console noise).
- Shopfloor kiosk header text is readable (light on the dark navy header).
### Added (continued)
- Collector-driven PC -> printer relationships. The computers collector schema
gained optional `defaultprinter` (string) and `printers` (array of strings)
fields carrying Win32_Printer identifiers. On ingest each identifier resolves
to a printer asset (by windows name / share / hostname / asset number-name or
a communications IP) and the PC is linked to it: the default via a
`defaultprinter` relationship, the rest via `connectedto`. The links render in
the shared Relationships card on both the PC and printer detail pages. The
sync is idempotent and archives collector-created links to printers no longer
reported (tagged `assetrelationships.label = 'collector:printers'`, so
manually-created links are never touched); unresolved identifiers become
response warnings, never failures. The collector response carries
`printerlinkcount` and a `printerlinks` list. See docs/COLLECTOR-INTEGRATION.md.
- Searchable custom fields. Each custom-field definition gained a `searchable`
flag (Settings > Custom Fields). When on, that field's stored values are
matched by global search and a hit routes to the owning asset's detail page.
The asset's `search_<type>_enabled` domain toggle still applies, and matches
dedupe against built-in-field asset hits so an asset appears once. Inactive or
non-searchable fields are never matched.
- Single-label sheet-position printing. The single asset-label page
(`/print/asset-label/:assettype/:id`) gained an Output control that toggles
between the standalone label (unchanged default) and placing that one label at
a chosen cell (1-6, via a 2x3 grid picker) of a ULINE 6-up sheet, leaving the
other five cells blank. This prints a single label onto the correct physical
spot of a partially-used sheet instead of wasting a fresh sheet, mirroring the
legacy shopdb behavior and complementing the batch page's start-cell offset.
The ULINE 6-up cell layout and dimensions are replicated from
`AssetLabelBatch.vue` (left untouched); encode resolution stays shared via
`assetLabel.js`.
## [0.7.0] - 2026-07-12
### Added
- Email sending. A stdlib-only mail service (`shopdb/utils/mailer.py`;
`smtplib`/`ssl`/`email`) reads the existing `email` SMTP settings
settings-first with an `SMTP_*` env fallback, sends multipart HTML+text, and
is a graceful no-op (logs a warning, returns False) when email is disabled or
the host is unset. The SMTP password is never logged. Three flows use it:
(1) New-user welcome + forced first-login password change. Admin-created users
(POST `/api/users`) are flagged `mustchangepassword` (new `users` column,
migration `7d23_user_mustchangepassword`, default false) and sent a
best-effort welcome email with the facility name, username, temporary
password, and sign-in link; the account is created even if mail fails
(response carries a `warning`). Login returns `mustchangepassword`; the
frontend forces the user through a new `/change-password` view (POST
`/api/auth/change-password`, jwt-guarded) before the app, and changing the
password clears the flag and resets lockout counters. A self-service "Change
password" entry is also available from the user menu.
(2) Test email. POST `/api/settings/test-email` (settings.edit) sends a probe
and surfaces any SMTP error with the password scrubbed; wires up the Email /
SMTP settings page "Send Test Email" button.
(3) On-demand alert/report delivery. POST `/api/reports/email`
(reports.export) mails `{subject, columns, rows}` as an HTML table to a
supplied recipient or the site `alert_recipients`; an "Email report" button on
the Warranty and Toner report pages posts the loaded rows. There is no
scheduler: automation is an external cron hitting the endpoint with a scoped
API token (PAT). Documented in `docs/CONFIG.md`.
- Shared asset label/code generator: a single `/print/asset-label/<assettype>/<id>`
page (public, like the other `/print/*` routes) that any asset detail page
opens via a "Print Label" button (machines, computers, printers, network
devices, measuring tools). A no-print controls panel toggles the layout
(`card` badge vs `plain` code-only), the code type (QR vs CODE128 barcode),
and what the code encodes: the asset page link, asset number, serial number,
a per-type custom target template, or - for measuring tools by default - the
tool's inspection location code so every tool at one operation shares one
code (e.g. `0615`). QR codes reuse the shared logo-overlay renderer. New
`printing` settings seed and surface on the Printing & Labels settings page:
`qr_target_machine`, `qr_target_computer`, `qr_target_network_device`,
`qr_target_measuring_tool`; `label_default_style` (default `card`) and
`label_default_codetype` (default `qr`); and a per-asset-type default for what
the code encodes, `label_default_encodes_<type>` (machines default to their
machine number, measuring tools to their inspection location code, the rest to
a page link), all overridable on the label page itself. When the chosen field
has no value (e.g. serial number on an asset with none), the label states so
instead of rendering an empty code. Asset payloads now carry a derived
`locationcode` (leading token of the resolved own/inherited location name).
- Batch label sheets: a "Print Labels" button on each asset list page opens
`/print/asset-label-batch/<assettype>`, a multi-select sheet that lays the
chosen assets onto ULINE label pages (6-up 3 in x 3 in, or a dense 72-up
mini-label format), with a start-cell offset to reuse partial sheets. Shares
the same code-type and encode settings/defaults as the single label, so a
batch of measuring tools encodes each tool's inspection location code just
like the single label does. Restores the ULINE batch printing the legacy
shopdb had, generalized across all asset types.
- Support-team contact UX: the settings Support Teams page now manages each
team's contacts in a per-team "Contacts (N)" modal (name, SSO, active, plus
Add/Edit/Delete) instead of an inline row expander, and the application
detail Support card renders Email (`mailto:`) and Microsoft Teams chat
(`teams.microsoft.com/l/chat`) action buttons for every contact that has an
SSO. Both link targets derive as `sso@<domain>` from a new `site` setting
`contact_email_domain` (default `geaerospace.com`; blank hides the buttons),
surfaced in Site & Facility settings under Naming & Patterns and read on the
frontend via `getContactEmailDomain()`.
- Plugin `get_permissions` hook (contract 0.10.0) so a plugin declares the RBAC
permissions its own routes enforce, instead of core accumulating every
plugin's permissions in `Permission.PERMISSIONS` (plugin-is-the-product). The
core catalog (`Permission.CORE_PERMISSIONS`) now holds only genuinely core
sets (assets, applications, reports, settings, users, audit, apitokens,
collector); the 36 permissions for machines, computers, printers, network,
knowledgebase, notifications, usb, warranty, and measuringtools moved into
each owning plugin's hook. New core helper `full_permission_catalog()` merges
core plus every ENABLED plugin's permissions and backs all three consumers:
`flask seed permissions`, the role grid (`GET /api/users/permissions`), and
API-token scope validation (`ApiToken.unknown_scope_names`). Plugin install
and enable seed the plugin's own permissions idempotently. A disabled plugin
drops out of the catalog (no new scope grants or role assignments), but its
existing `Permission` rows and role links persist so current roles keep
working. Docs: `docs/PLUGIN-HOOKS.md` new section, `docs/PLUGIN-GUIDE.md`
permissions walkthrough rewritten to the hook, `docs/PLUGIN-QUICKSTART.md`
hooks table row.
- Personal API tokens (PATs) so scripts and integrations authenticate without
the hourly-expiring login JWT (immediate consumer: long legacy-import runs
that die when the JWT expires mid-run). New core `apitokens` table + migration
`7d21_apitokens` (stores only the sha256 hash of each secret; the full secret
`shopdb_pat_<40 hex>` is shown ONCE at creation). New core blueprint
`/api/apitokens` (list own / admin `?all=true`; create; rename or deactivate;
revoke). A `Bearer shopdb_pat_...` header is recognized before any JWT decode
by a before_request shim that mints a request-scoped JWT for the token's
owner, so the entire existing auth+authz stack (jwt_required,
require_permission, require_role, import mode, current_user) authenticates the
PAT as its owner with zero decorator changes; an invalid, revoked, or expired
PAT gets a clean 401. `lastusedat` is stamped on use (throttled to at most one
write per 60s). Any authenticated user manages their own tokens; admins may
list or revoke anyone's. New Settings > API Tokens page (`ApiTokensList.vue`)
with a create modal that reveals the secret once (copy button) and an admin
All Tokens section. Docs: `docs/IMPORT-API.md` and `docs/CONFIG.md` updated to
recommend a PAT for imports. Core feature; no plugin-contract change.
- Optional permission scopes on personal API tokens. A token MAY carry a scopes
list (permission names, migration `7d22_apitokens_scopes` adds the nullable
`apitokens.scopes` JSON column); NULL keeps the original behavior (acts as its
owner). A scoped token grants ONLY the listed permissions, intersected with
what the owner actually holds at use time, and SUSPENDS the admin-role bypass,
so a scoped token minted by an admin is genuinely limited: it is denied on
role-gated (`require_role`) endpoints and gets no import mode. The shim mints
the request JWT with a `patscopes` claim that `require_permission`,
`require_role`, and `import_mode_active` read; normal login JWTs carry no such
claim and are unaffected (zero regression). Scopes are validated at write time
against the token OWNER's permissions (the scope ceiling - a token can never
grant more than its owner holds; when an admin edits another user's token the
ceiling is that owner's permissions), rejecting unknown or unheld names 400.
Minting/managing tokens now requires the new `apitokens.create` permission
(category `apitokens`; admins hold it by default, grantable via the roles UI)
rather than being open to any authenticated user. The Settings > API Tokens
create/edit modals gain a "Restrict permissions" section (a category-grouped
checkbox grid limited to the permissions the creator holds) and the token
lists show a full-access / N-permissions access chip.
- Managed collector service tokens: the collector ingest API
(`/api/collector/<plugin>` + the legacy `/pc` `/apps` `/heartbeat` `/bulk`
`/status` endpoints) now ALSO accepts a managed API token scoped to the new
`collector.ingest` permission (category `collector`), on top of the existing
`COLLECTOR_API_KEY[_<PLUGIN>]` env keys (which stay supported as a
bootstrap/legacy fallback - nothing breaks). The token may be presented in
`X-API-Key` (as GE-Enforce sends today) OR as an `Authorization: Bearer`
token; both transports validate the PAT the same way the login shim does
(hash lookup, active, unexpired, active owner) via a shared
`resolve_api_token` helper refactored out of `apitoken_auth.py`, require
`collector.ingest` in the token's scope list AND that the owner holds it, and
stamp `lastusedat` (same 60s throttle). A token scoped to ONLY
`collector.ingest` is a collector service token: it authorizes the collector
API and NOTHING else - the existing scoped-token machinery denies it on every
permission- and role-gated route and on import mode, so a leaked collector
token cannot touch the regular API. Recommended flow (documented): an admin
mints the scoped token (the scope suspends the admin bypass, containing it);
rotate by minting a new one, deploying via `site-config.json`, watching
`lastusedat`, then revoking the old. The Settings > API Tokens create modal
gains a "Collector service token" quick-preset (pre-selects only
`collector.ingest`). Docs: `docs/COLLECTOR-INTEGRATION.md` (new "Managed
collector tokens" section) and `docs/CONFIG.md`. Core feature; no
plugin-contract change.
- Vendor-model photos on asset detail heroes: computers and printers now
surface the linked model's `imageurl` in their extension payloads (the
field machines already exposed), and the machine, PC, printer, network
device, and measuring tool detail pages render the photo in the hero card
when present (hidden cleanly when absent). Network devices and measuring
tools have no model link yet, so their heroes stay photo-less until one
is added.
- Dualpath "single machine" site toggle (`dualpath_single_machine`, default
on). A Dualpath relationship pair is one physical dual-bay machine (single
controller, bay-selector switch); when on, the machines list, dashboard and
machines-by-type counts, and the floor map collapse each pair to one entry
(the lower natural-sort assetnumber is PRIMARY; the SECONDARY bay is hidden)
and show a combined `2007 / 2008` label. The data model is unchanged (both
bay records always exist); detail pages stay per-bay and always show a
sibling-bay banner regardless of the toggle. Contract surface (plugin
contract bumped 0.8.0 -> 0.9.0, additive): new `shopdb.api` helpers
`resolve_dualpath_pairs` and `dualpath_single_machine_enabled`, consumed by
the machines plugin to collapse pairs contract-purely.
- Relationship propagation, wired and data-driven: relationship types
declare propagation-through pairs (relationshiptypepropagations M:N,
replacing the never-consumed single column); creating a controls link on
one Dualpath bay auto-creates it on the partner bay, and
`flask relationships propagate` backfills existing data.
- Employee photos, mode-aware: self-hosted directory employees support
upload/replace/delete (admin), served publicly for kiosk cards; external
directory mode passes the HR-supplied picture URL through read-only. One
resolver feeds the shopfloor recognition/recert cards and the employee
detail hero in either mode.
- Vendor-model photo management. New admin-gated core endpoints
`POST /api/models/<modelid>/image` (multipart `file`, png/jpg/jpeg/gif/webp/svg,
one image per model, replace semantics) and
`DELETE /api/models/<modelid>/image`, plus the public
`GET /api/models/image/<filename>` serve route. Uploads land in
`instance/modelimages/` (survives upgrades, backed up with the rest of
`instance/`) and set `models.imageurl` to the served URL; the manual Image URL
field still accepts external URLs and the shipped `/images/models/*` assets
(upload is additive). Delete only removes files we own under the instance dir.
The Models settings page grows a thumbnail, Upload/Replace, and Remove
controls in the edit modal. Asset hero images (e.g. the machine badge) read
`imageurl` unchanged, so uploaded photos render with no consumer changes.
- Application support teams with contacts, replacing the legacy
supportteams/appowners pair. New core `supportteamcontacts` table (multiple
named contacts per team, ordered by `sortorder`); `supportteams` keeps
`teamname` (now unique) and `teamurl` (a ServiceNow group deep link) and
sheds the single-owner `appownerid` FK. New core blueprint at
`/api/supportteams` (team + nested contact CRUD, admin-gated; `?teamname`
exact-match lookup for import; delete a team 409s while any application still
references it). Migration `7d18_supportteamcontacts` migrates each legacy
team's app owner into one contact. Application payloads now flatten
`supportteamname`, `teamurl`, and the team's active `contacts`; a Support
card on the application detail page and a new `settings/supportteams`
management page render them.
- Import mode: a complete, idempotent HTTP migration surface so a migration script
can import the classic ASP shopdb through the API alone (no direct DB writes).
- Contract surface (plugin contract bumped 0.7.0 -> 0.8.0, additive): new
`shopdb.api` helpers `apply_import_timestamps`, `import_mode_active`,
`parse_import_datetime` in `shopdb/utils/import_mode.py`. When the caller is
an admin AND sends header `X-Import-Mode: true`, create/update endpoints
accept optional `createddate` / `modifieddate` (ISO or legacy
`YYYY-MM-DD HH:MM:SS`, naive-UTC) and preserve them instead of stamping now.
Non-admin or missing header: the fields are ignored exactly as before.
Wired into every timestamped import target: assets (all five type plugins),
vendors, models, modeltypes, businessunits, locations, operating systems,
applications, knowledge base, USB devices, and asset relationships.
- Natural-key exact-match lookup filters for the documented
lookup-then-upsert idempotency recipe: `assetnumber` on all five asset
plugin list endpoints; `vendor`, `modelnumber`, `modeltype`,
`businessunit`, `locationname`, `osname`/`osversion`, `appname`,
knowledge base `linkurl`/`shortdescription`, warranty `servicetag`/`vendor`,
and notification `ticketnumber`.
- Backdated event history: in import mode the selfhosted USB checkout/checkin
endpoints accept optional `checkouttime` / `checkintime` overrides so
migrated `usbcheckouts` rows keep their real event times.
- New operator manual `docs/IMPORT-API.md` grounded in the real `prodscratch`
legacy schema: order of operations, a full table-by-table mapping, honest
no-target list with dispositions, a worked idempotent Python importer, and
row-count parity checks.
### Changed
- System Settings is no longer one tabbed page. The inner tab bar is gone and
each section is its own routed settings page reached through the settings
rail: ServiceNow, Zabbix Supplies, Dell Warranty, and Collector PC Types
(the former Integrations dumping ground, now split three-plus ways), plus
Branding, Floor Map, Printing & Labels, Email / SMTP, Authentication,
Audit & Logging, Asset Identifiers, and Global Search. The rail regroups
these under Site & Facility, Integrations, Communication, Search & Identity,
and Access & Security. Shared load/save/upload plumbing moved into a
`useSystemSettings` composable so the pages stay thin. Old bookmarks keep
working: `/settings/system` and every `/settings/system?tab=<key>` redirect
to the matching new page.
### Fixed
- Audit log: hovering a user's SSO now shows their full name (best-effort,
resolved from the employee directory in either mode).
- Refreshed the internal status docs to match the code (project active
state, CONTRACT-STABILITY.md and README plugin list at contract 0.10.0),
corrected the get_asset_panels endpoint path in the hook docstring, and
removed leftover debug console.log lines.
- Measuring tools wired into the remaining cross-cutting surfaces an audit
found them missing from: generic asset serialization (typedata + pluginid,
which also fixes relationship-card links to tools), map subtype
filtering/coloring and the MapEditor filter, dashboard totals, warranty
asset links (via a new by-asset detail route), and the two ADR-010 hook
declarations (presentation route token corrected; the calibration
map-overlay endpoint now actually exists). The login avatar also resolves
through the employee-photo helper, so self-hosted uploads show.
- Measuring tools are now wired into the asset-identifier matrix and global
search. The Settings identifier matrix gains a Measuring Tool column and the
gauge-lab and maintenance reference inputs/rows on the measuring-tool form
and detail pages honor those per-type toggles (a maintenance-reference field
was added, matching the other asset types). Global search gains a Measuring
Tools domain toggle and filter chip, routes measuring-tool hits to
`/measuringtools/<id>` (previously the generic `/assets/<id>` fallback), and
matches on `gaugelabreference` so a gage-tag lookup resolves the tool.
- Site & Facility settings page renders booleans as toggles and the
directory-mode settings as dropdowns, with labels and help text for every
field (no more raw keys or type-true/false boxes).
- System Settings tabs follow the URL: clicking a settings-rail link that
only changes the ?tab= query (Branding, Floor Map) now switches the right
panel, tab clicks update the URL, and browser back/forward restore tabs.
- Following a relationship link between two assets of the same type now loads
the destination page instead of stale content (router-view keyed on path;
query-only URL changes still avoid a remount).
- Asset relationships card no longer lists a symmetric peer twice. Relationship
types gain `relationshiptypes.isdirectional` (migration
`7d19_relationshiptype_directional`; seeded false for the connection-like
types Dualpath, connectedto, Cluster Member, Serial Cable, Direct Ethernet,
USB, WiFi, true for controls/Controlled By/Backup For/Master-Slave/partof/
defaultprinter). The card now collapses every stored direction row of a
symmetric type into one direction-blind "Connected" entry per peer (deleting
it removes all collapsed rows), while directional types drop the
Outgoing/Incoming headers for inline `Type -> peer` / `<- Type from peer`
phrasing. The type CRUD and the per-asset relationships endpoint carry
`isdirectional`; the Relationship Types settings page gains a Directional
toggle.
## [0.6.0] - 2026-07-11
### Added
@@ -104,7 +570,7 @@ letting other GE Aerospace sites stand up their own self-hosted instance
- Multi-stage Docker build that compiles the Vue frontend and ships
`frontend/dist`, which Flask serves.
- Documentation overhaul: new CONFIG, UPGRADE, and BACKUP-RESTORE guides;
reconciled README, DEPLOY, CLAUDE, and ROADMAP.
reconciled README, DEPLOY, status docs, and ROADMAP.
- ADR-007 (product versioning and releases), CHANGELOG, and best-effort
Gitea Actions CI (backend tests, naming/style gate, frontend build).
@@ -137,5 +603,8 @@ letting other GE Aerospace sites stand up their own self-hosted instance
integration that passed the key as a query parameter. See
`docs/COLLECTOR-INTEGRATION.md`.
[Unreleased]: https://gitea.proudtech.net/ge-aerospace/shopdb-flask/compare/v0.5.0...HEAD
[Unreleased]: https://gitea.proudtech.net/ge-aerospace/shopdb-flask/compare/v0.8.0...HEAD
[0.8.0]: https://gitea.proudtech.net/ge-aerospace/shopdb-flask/compare/v0.7.0...v0.8.0
[0.7.0]: https://gitea.proudtech.net/ge-aerospace/shopdb-flask/compare/v0.6.0...v0.7.0
[0.6.0]: https://gitea.proudtech.net/ge-aerospace/shopdb-flask/compare/v0.5.0...v0.6.0
[0.5.0]: https://gitea.proudtech.net/ge-aerospace/shopdb-flask/releases/tag/v0.5.0

View File

@@ -16,14 +16,22 @@ Architecture decisions live in `docs/adr/`. Read those before making schema or c
- ADR-004: Deployment topology (per-site instances, not multi-tenant) - ACCEPTED
- ADR-005: Equipment vs measuringtools plugin scope - ACCEPTED
- ADR-006: Plugin collector contract pattern - ACCEPTED
- ADR-007: Product versioning and releases - ACCEPTED
- ADR-008: Plugin migration ownership (per-plugin chains) - ACCEPTED
- ADR-009: Frontend plugin route gating - ACCEPTED
- ADR-010: Frontend plugin hook contract - ACCEPTED
- ADR-011: Machines rename + modeltypes retyping - ACCEPTED
- ADR-012: GE-Enforce manifest ownership in shopdb - ACCEPTED
- ADR-013: Plugin catalog, curated shelf, and lean per-site builds - PROPOSED
- ADR-014: Schema-lean per-site builds (retire cross-plugin FKs, prune not-installed plugin tables) - ACCEPTED
## Coding convention
`CONTRIBUTING.md` defines naming rules (DB tables, columns, Python, JS, Vue, API). Pre-commit hook at `scripts/check-naming-and-style.sh` enforces them. Read `CONTRIBUTING.md` before naming any new identifier.
## Current state (as of 2026-07-10)
## Current state (as of 2026-07-13)
Refactor phases 0-5 landed; phase 6 (multi-site distribution readiness) in progress.
Refactor phases 0-5 landed; phase 6 (multi-site distribution readiness) largely complete; the last big milestone is the legacy-data import + a production pilot.
### Phases done
@@ -36,10 +44,15 @@ Refactor phases 0-5 landed; phase 6 (multi-site distribution readiness) in progr
### Active state
- 340 tests passing, naming/style check green, Gitea Actions CI (backend + naming + frontend build)
- `__contract_version__` at 0.6.0 (product `__version__` 0.5.0 - distinct series, ADR-007)
- 11 bundled plugins all satisfy contract: computers, employees, knowledgebase, machines, measuringtools, network, notifications, printers, slides, usb, warranty
- Single core Alembic chain: baseline `68b3947ae14f` -> head `7d16_directoryemployees` (23 migrations). A fresh site runs `flask db upgrade` from empty; it is reproducible and idempotent.
- 1159 tests, naming/style check green, Gitea Actions CI (backend + naming + frontend build + a lean-build job + a migrations-mysql job that runs the real fresh upgrade on utf8mb4 MySQL 8)
- GE-Enforce HTTPS cutover: the displays/kiosks cohort now fetches manifest + inline payloads entirely over HTTPS (share-less); the `gea-shopfloor-display` scope is authored in code (`plugins/geenforce/seed_display_scope.py`) and published via `seed_display_scope(publish=True)`. Other fleet PC types still enforce from the SMB share and only report. See `docs/geenforce-api-cutover.md`.
- `__contract_version__` at 0.15.0 (0.12.0 mailer, 0.13.0 User/Role, 0.14.0 send_webhook, 0.15.0 authorized_service_token) (product `__version__` 0.7.0, tags v0.5.0/v0.6.0/v0.7.0 - distinct series, ADR-007)
- 13 bundled plugins all satisfy contract: computers, employees, geenforce, knowledgebase, machines, measuringtools, network, notifications, printedparts, printers, slides, usb, warranty
- Core Alembic chain: baseline `68b3947ae14f` -> head `7d26_settings_description_text` (33 core migrations). Each plugin owns its own chain (ADR-008); deploy runs `flask db upgrade` then `flask plugin upgrade-all`. Reproducible + idempotent from empty (env.py relaxes session sql_mode so the chain runs on strict MySQL 8).
- Lean per-site builds (ADR-013 + ADR-014): `scripts/build-site.sh` (backend) + `SITE_PLUGINS` via `scripts/stage-frontend.mjs` (frontend) ship only chosen plugins; `flask plugin prune-schema` drops non-installed plugins' tables at provisioning. Sidebar nav / settings / Displays all gate on staged routes. Manifest-less `plugins/<name>/frontend/` dirs (e.g. `applications`) are core and always ship.
- Windows sites install from a single air-gapped installer `.exe` built per site from its plugin profile (`deploy/windows/installer/`, built by `build-installer.sh` or `build-installer.ps1`). Operator docs: `docs/INSTALL-WINDOWS.md` + `docs/OPERATE-WINDOWS.md` - these are canonical for a NEW site. `docs/INSTALL-WINDOWS-IIS.md` and `docs/DEPLOY-WINDOWS-IIS.md` are the MANUAL procedure, kept for hand-built servers only. The installer verifies its third-party payload against `bundle-lock.json` and installs wheels with `pip --require-hashes`; every build stages a CycloneDX SBOM (`sbom.cdx.json`) onto the server.
- Legacy import: `docs/IMPORT-API.md` is the schema-agnostic import contract; `docs/IMPORT-ADOPTION.md` + `docs/PILOT-DEPLOY.md` cover adopting a site; `scripts/site_imports/wjf/` is the West Jefferson reference loader (all 16 stages, validated end-to-end including on a Windows + MySQL 8 VM).
- API is migration-complete: an admin PAT + docs/IMPORT-API.md let a script import the whole legacy DB (X-Import-Mode preserves timestamps).
- Pre-1.0 framework; sister sites should pin tight `core_version` ranges until contract reaches 1.0
### Deferred
@@ -47,8 +60,9 @@ Refactor phases 0-5 landed; phase 6 (multi-site distribution readiness) in progr
- Equipment data migration (one-shot script for legacy ASP shopdb -> assets). Per ADR-001, only `category='Equipment' AND machinenumber IS NOT NULL` migrates. Skill `migrating-asset-schema` documents the pattern; the actual one-shot script lives in `scripts/migration/` when run.
- Printers retirement: legacy `PrinterData` model + frontend changes. Coordinated with the equipment data migration.
- (DONE 2026-07-11) `measuringtools` plugin (ADR-005) is built and bundled; docs/PLUGIN-GUIDE.md narrates its construction as the plugin tutorial.
- Frontend hook contract for asset-detail, map markers, search results
- Alembic per-plugin migration chains (the framework supports them; bundled plugins haven't moved off `db.create_all()` yet)
- (DONE) Frontend plugin hook contract (ADR-010): get_settings_cards / get_asset_panels / get_map_overlays / get_asset_presentation shipped; generic renderers for panels/overlays land incrementally.
- (DONE) Per-plugin Alembic chains (ADR-008): every bundled plugin carries its own chain; no plugin uses db.create_all().
- Legacy ASP data import against the renamed schema (unblocked; run via docs/IMPORT-API.md) + a production pilot deployment.
## Quick start
@@ -126,4 +140,4 @@ Each plugin must have:
- `migrations/FIX_LOCATIONONLY_EQUIPMENT_TYPES.md` - LocationOnly equipment type fix
- `migrations/PRODUCTION_MIGRATION_GUIDE.md` - production import methods
- `migrations/rename_underscore_columns.sql` - one-time rename of snake_case columns to lowercase concatenated (per CONTRIBUTING.md)
- `migrations/versions/` - the core Alembic chain (baseline `68b3947ae14f` -> head `7d16_directoryemployees`). Run `flask db upgrade` to apply.
- `migrations/versions/` - the core Alembic chain (baseline `68b3947ae14f` -> head `7d26_settings_description_text`). Run `flask db upgrade` to apply.

View File

@@ -2,9 +2,12 @@
#
# One image, one site. Per ADR-004, each adopting facility runs its own
# stack with its own DB, secrets, and enabled-plugin list. This image
# bundles all ten core plugins (computers, employees, equipment,
# knowledgebase, network, notifications, printers, slides, usb, warranty);
# install them at runtime with `flask plugin install <name>`.
# bundles all 13 catalog plugins (computers, employees, geenforce,
# knowledgebase, machines, measuringtools, network, notifications,
# printedparts, printers, slides, usb, warranty); a site installs + enables
# the ones it wants with `flask plugin install <name>` (or, declaratively,
# `flask plugin apply-profile <profile.json>`). Per ADR-013 a future lean
# build stages only the chosen plugin directories into this image.
#
# The frontend is built in a first stage and its dist output is copied into
# the final image so Flask can serve the SPA (register_frontend_routes in
@@ -29,7 +32,7 @@ RUN npm run build
# Output lands in /build/dist (Vite default), copied into the final stage below.
# ---- Stage 2: Python application image ----
FROM python:3.12-slim AS base
FROM python:3.14-slim AS base
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1 \

121
README.md
View File

@@ -7,16 +7,22 @@ A modern rewrite of the classic ASP/VBScript ShopDB application using Flask (Pyt
ShopDB tracks and manages:
- **Machines** - CNC equipment, CMMs, inspection systems, etc.
- **PCs** - Shopfloor computers, engineering workstations
- **Printers** - Network printers with Zabbix integration
- **Applications** - Software deployed across the shop floor
- **Printers** - Network printers with Zabbix supply integration
- **Network devices** - Switches, routers, and the subnet browser
- **Measuring tools** - Gage-lab instruments with calibration tracking
- **Applications** - Software deployed across the shop floor, with per-PC install tracking
- **Employees** - Directory, recognition and training notifications
- **Warranties** - Coverage records with Dell warranty lookups
- **USB devices** - CMMC check-in/out tracking
- **Knowledge Base** - Documentation and troubleshooting guides
- **GE-Enforce manifests** - Imaging/software manifest editing and fleet compliance
## Tech Stack
**Backend:**
- Python 3.x with Flask
- Python 3.14 with Flask
- SQLAlchemy ORM
- MySQL 5.6+ database
- MySQL 5.7+ database (5.6 works with extra utf8mb4 config; see docs/DEPLOY.md)
- JWT authentication
- Plugin architecture for extensibility
@@ -30,26 +36,26 @@ ShopDB tracks and manages:
```
shopdb-flask/
├── shopdb/ # Flask application
├── core/
├── api/ # REST API endpoints
├── models/ # SQLAlchemy models
├── schemas/ # Validation schemas
└── services/ # Business logic
├── plugins/ # Plugin system
└── utils/ # Shared utilities
├── frontend/ # Vue 3 application
├── src/
├── api/ # API client
├── components/ # Reusable components
├── views/ # Page components
├── router/ # Route definitions
└── stores/ # Pinia stores
└── public/ # Static assets
├── plugins/ # Bundled and external plugins
├── migrations/ # Alembic migration chain (flask db upgrade)
├── scripts/ # Import and utility scripts
└── tests/ # Test suite
+-- shopdb/ # Flask application
| +-- core/
| | +-- api/ # REST API endpoints
| | +-- models/ # SQLAlchemy models
| | +-- schemas/ # Validation schemas
| | `-- services/ # Business logic
| +-- plugins/ # Plugin system
| `-- utils/ # Shared utilities
+-- frontend/ # Vue 3 application
| +-- src/
| | +-- api/ # API client
| | +-- components/ # Reusable components
| | +-- views/ # Page components
| | +-- router/ # Route definitions
| | `-- stores/ # Pinia stores
| `-- public/ # Static assets
+-- plugins/ # Bundled and external plugins
+-- migrations/ # Alembic migration chain (flask db upgrade)
+-- scripts/ # Import and utility scripts
`-- tests/ # Test suite
```
## Naming Conventions
@@ -59,13 +65,13 @@ To maintain consistency with the legacy ShopDB database and codebase, the follow
### Database
- **Table names:** Lowercase, single word, no underscores or dashes
- Examples: `machines`, `pctypes`, `machinetypes`, `businessunits`
- Examples: `assets`, `computers`, `printers`, `businessunits`
- **Column names:** Lowercase, single word, no underscores or dashes
- Examples: `machineid`, `machinenumber`, `pctypeid`, `isactive`, `createddate`
- Examples: `assetid`, `assetnumber`, `hostname`, `isactive`, `createddate`
- **Foreign keys:** Referenced table name + `id`
- Examples: `locationid`, `vendorid`, `modelnumberid`, `pctypeid`
- Examples: `locationid`, `vendorid`, `modelnumberid`, `computertypeid`
- **Boolean columns:** Prefixed with `is` or `has`
- Examples: `isactive`, `isshopfloor`, `isvnc`, `iswinrm`, `islicenced`
- Examples: `isactive`, `isshopfloor`, `iscolor`, `isdhcp`, `islicenced`
### Code
@@ -77,9 +83,9 @@ To maintain consistency with the legacy ShopDB database and codebase, the follow
### API
- **Endpoints:** Lowercase, plural nouns
- Examples: `/api/machines`, `/api/pctypes`, `/api/locations`
- Examples: `/api/machines`, `/api/computers`, `/api/locations`
- **Query parameters:** Lowercase, single word
- Examples: `?type=pc`, `?locationid=5`, `?isactive=true`
- Examples: `?locationid=5`, `?isactive=true`, `?assettype=computer`
## Style Guidelines
@@ -92,7 +98,7 @@ To maintain consistency with the legacy ShopDB database and codebase, the follow
### Prerequisites
- Python 3.8+
- Python 3.14
- Node.js 18+
- MySQL 5.7+ (5.6 works with extra utf8mb4 config; see docs/DEPLOY.md)
@@ -102,7 +108,7 @@ SQLite). Do not run dev or production against SQLite.
### Distribution
The application is distributed internally through the GE Aerospace Gitea. Clone
The application is distributed through the internal GE Aerospace git server. Clone
it from there; there is no public package or image registry.
### Fast path (Docker)
@@ -142,11 +148,13 @@ pip install -r requirements.txt
cp .env.example .env
# Edit .env with your database credentials and secrets.
export FLASK_APP=shopdb
flask db upgrade
flask plugin upgrade-all # per-plugin schema (ADR-008)
flask seed permissions
flask seed settings
flask seed reference-data
flask run
flask run --port 5001 # MUST be 5001 - the frontend dev server proxies here
# Frontend (separate terminal)
cd frontend
@@ -156,17 +164,25 @@ npm run build # production build into frontend/dist (served by Flask)
```
Complete first-run setup at `/setup`, or run `flask seed admin` for a headless
admin account.
admin account. The repo ships VS Code config in `.vscode/` (F5 debugs the
backend; a "Dev site" task runs both servers). A fuller day-one walkthrough,
including VS Code and troubleshooting, is the DEVELOPMENT-SETUP page in the
project wiki.
To import data from the legacy ShopDB MySQL database (one-time, see
`migrations/DATA_MIGRATION_GUIDE.md`):
To import a site's legacy data, use the HTTP import surface: an admin API
token plus [docs/IMPORT-API.md](docs/IMPORT-API.md) drive the whole migration
through documented endpoints (`X-Import-Mode` preserves original timestamps).
`scripts/site_imports/wjf/` is the West Jefferson reference loader.
```bash
python scripts/import_from_mysql.py
```
### Which deployment route
For the full per-site deployment runbook see [docs/DEPLOY.md](docs/DEPLOY.md);
for every environment variable and Setting key see [docs/CONFIG.md](docs/CONFIG.md).
| Target | Use |
|---|---|
| **Windows Server + IIS** (how sister sites run) | **[docs/INSTALL-WINDOWS.md](docs/INSTALL-WINDOWS.md)** - one installer `.exe`, offline, no manual IIS work. Day 2: [docs/OPERATE-WINDOWS.md](docs/OPERATE-WINDOWS.md) |
| Linux / Docker, air-gapped | [docs/DEPLOY-AIRGAP.md](docs/DEPLOY-AIRGAP.md) |
| Linux / Docker, connected | [docs/DEPLOY.md](docs/DEPLOY.md) |
For every environment variable and Setting key see [docs/CONFIG.md](docs/CONFIG.md).
## Configuration
@@ -192,30 +208,37 @@ The REST API follows standard conventions:
| PUT | `/api/machines/:id` | Update machine |
| DELETE | `/api/machines/:id` | Soft delete machine |
Each asset plugin exposes the same CRUD pattern on its own prefix
(`/api/computers`, `/api/printers`, `/api/network`, `/api/measuringtools`),
and cross-cutting asset endpoints live under `/api/assets`.
Query parameters for list endpoints:
- `page` - Page number (default: 1)
- `per_page` - Items per page (default: 25)
- `perpage` - Items per page
- `sort` - Sort field
- `order` - Sort direction (asc/desc)
- `dir` - Sort direction (asc/desc)
- `search` - Search term
- `type` - Filter by asset type (computer, printer, machine, network_device)
- `assettype` - Filter by asset type (computer, printer, machine, networkdevice, measuringtool)
## Plugin System
ShopDB supports plugins for extending functionality. See `CONTRIBUTING.md` for plugin development guidelines.
The image bundles ten plugins; only the ones a site installs are loaded:
The image bundles thirteen plugins; only the ones a site installs are loaded:
- **computers** - Shopfloor PCs and workstations
- **computers** - Shopfloor PCs and workstations, collector fleet ingest
- **employees** - Employee directory
- **machine** - CNC, CMM, and other shop-floor machines
- **geenforce** - GE-Enforce imaging/software manifests and fleet compliance
- **machines** - CNC, CMM, and other shop-floor machines
- **measuringtools** - Gage-lab instruments with calibration tracking
- **knowledgebase** - Documentation and troubleshooting guides
- **network** - Network devices
- **network** - Network devices and subnets
- **notifications** - Shopfloor notifications and recognition feed
- **printedparts** - 3D-printed part catalogue, kiosk issue tracking and stock alerts
- **printers** - Extended printer management with Zabbix integration
- **slides** - TV/kiosk slideshows
- **usb** - CMMC USB check-in/out tracking
- **warranty** - Dell warranty lookups
- **warranty** - Warranty records with Dell lookups
## Legacy Migration

View File

@@ -0,0 +1,20 @@
{
"site": "universal",
"plugins": [
"computers",
"employees",
"geenforce",
"knowledgebase",
"machines",
"measuringtools",
"network",
"notifications",
"printedparts",
"printers",
"slides",
"usb",
"warranty"
],
"locked": [],
"_comment": "The profile the released Windows installer is built from. Every bundled plugin that carries a manifest, so ONE exe serves any site: the wizard offers all of them and the operator ticks what that site uses. Plugins left unticked are never installed, and 'flask plugin prune-schema' drops their tables at provisioning (ADR-014). 'applications' is deliberately absent - it is manifest-less core and always ships. Build with: deploy/windows/installer/build-installer.sh deploy/site-profile-universal.json <repo>. Use site-profile.example.json instead only when a site genuinely needs a lean build; see ADR-013."
}

View File

@@ -0,0 +1,11 @@
{
"site": "example-site",
"plugins": [
"machines",
"computers",
"printers",
"network"
],
"locked": [],
"_comment": "Declarative plugin selection for a site (ADR-013). Apply with: flask plugin apply-profile deploy/site-profile.example.json. 'plugins' is the set this site wants; their hard dependencies are pulled in automatically and everything installs + enables in dependency order (idempotent). Naming a plugin not on disk fails loudly. 'locked' is reserved for a future guard against removing a site-mandated plugin. apply-profile never removes plugins absent from the list - removal stays an explicit flask plugin uninstall. Run flask plugin upgrade-all after applying, then restart."
}

6
deploy/windows/installer/.gitignore vendored Normal file
View File

@@ -0,0 +1,6 @@
# Build output and staged payload - regenerable, and ~220MB.
bundle/
Output/
# Generated by build-installer.sh from the staged bundle.
plugins.iss
version.iss

View File

@@ -0,0 +1,247 @@
# Windows installer
Builds a single self-contained `.exe` that installs ShopDB-Flask on an
**air-gapped** Windows Server. Nothing here ever touches the network at install
time: Python, the wheels, the SPA and (optionally) MySQL all ship inside it.
Lives with the application on purpose. The installer depends on app internals -
`flask plugin` verbs, `site-profile.json`, `MOUNT_PATH`, the plugin registry -
so a separate repo would drift out of step with the thing it installs.
## Files
| File | What it is |
|---|---|
| `shopdb-preflight.ps1` | Stage 1. Read-only. Changes nothing, reports what this server is missing. |
| `shopdb-install.ps1` | Stages 0 and 2-5 plus `uninstall`. All the actual work. |
| `shopdb-admin.ps1` | Operator console installed alongside the app: status, restart, logs, backup, plugins. |
| `ShopDBFlask.iss` | Inno Setup wizard. A thin wrapper - it collects input and runs the stages. |
| `build-installer.sh` | Stages the bundle from a site profile. |
| `make-branding.py` | Generates wizard artwork and the icon from `frontend/public/*.svg`. |
| `*.bmp`, `shopdb.ico` | Generated artwork, committed so a Windows build box needs no Python. |
| `build-installer.ps1` | The same build, natively on Windows. No Bash needed. |
| `bundle-lock.json` | The exact third-party payload this installer ships. Reviewed by commit. |
| `bundle-lock.ps1` | Creates and checks that lock. Also runs on the target server. |
| `refresh-bundle-lock.ps1` | Regenerates the lock, after showing what changed. |
| `verify_bundle_lock.py` | The same check for the Bash builder, so Linux needs no pwsh. |
## Building
Two builders, same result. Use whichever machine you are on - `build-installer.ps1`
does the whole job natively so a Windows work PC needs no Bash.
```bash
# Linux / WSL
./build-installer.sh ../../site-profile.example.json
```
```powershell
# Windows
.\build-installer.ps1 -Profile ..\..\site-profile.example.json
```
Both stage the app tree, build the SPA twice, write `plugins.iss`, copy the
installer scripts **from this directory**, and then verify the third-party
payload against `bundle-lock.json`. They exit non-zero if it does not match.
The payload itself is added by hand, and no longer needs Windows to produce:
```
bundle/wheels/ pip download -r requirements.txt --only-binary=:all: \
--platform win_amd64 --python-version 314 \
--implementation cp --abi cp314 -d wheels
bundle/python/ python-3.14.x-amd64.exe
bundle/httpplatformhandler/ httpPlatformHandler_amd64.msi
bundle/urlrewrite/ rewrite_amd64.msi (client-IP rule; see below)
bundle/vcredist/ VC_redist.x64.exe (MySQL will not install without it)
bundle/mysql/ mysql-8.4.x-winx64.msi (bundled-database option only)
bundle/mysqlclient/ mysql.exe, mysqldump.exe + libcrypto/libssl (backups)
```
Then compile on Windows:
```
iscc ShopDBFlask.iss
```
**Inno Setup 6.6.0 or newer.** The wizard uses the built-in `windows11` custom
style, which earlier versions reject. The script fails at compile time with that
sentence rather than with a bare "WizardStyle is invalid".
The wheelhouse is **cp314-locked**. A different Python minor version means a
different wheelhouse; the installer will not use a Python it did not install.
## What is pinned, and where
Three layers, because no one of them covers the whole problem.
| Layer | Covers | Enforced |
|---|---|---|
| `requirements.txt` sha256 per package | every wheel is genuinely what upstream published | `pip --require-hashes` at install; aborts on mismatch |
| `bundle-lock.json` | the EXACT payload: wheels, Python installer, MSIs | both builders, and again on the server before anything runs |
| git | the application tree | code review |
`--require-hashes` alone is not enough. pip lists every artifact of a pinned
version - `cffi 2.1.0` has 100 hashes - so it proves the wheel is genuine, not
that it is the wheel this bundle was built and tested with. It also ignores
extra files in the wheelhouse, and says nothing about the Python installer or
the MSIs, all of which run as SYSTEM on the target server.
So `bundle-lock.json` records an exact file set with a sha256 and a byte size
each, and verification is **set equality**: a missing file, an unexpected extra
file, or changed content all fail. There is no install-time override.
The lockfile is `--universal`, so one file serves Linux (dev, Docker, CI) and the
Windows wheelhouse. A Linux-only resolve had silently omitted `colorama`, a
win32-only dependency of `click` - which in hash-checking mode is a hard error
rather than a quiet omission.
### Verifying the installer itself
`bundle-lock.json` is **inside** the thing it describes, so it proves the payload
was not altered between build and install - not that the `.exe` you received is
the one that was built. That needs something out of band. Two options, in order
of preference:
1. **Authenticode-sign the `.exe`** with a GE code-signing certificate. Windows
then shows a real publisher instead of "Unknown", which is also what stops an
operator learning to click through the SmartScreen warning.
2. **Publish a sha256 per release** through a different channel than the file
itself, and have the receiving site check it:
`Get-FileHash ShopDBFlask_Installer_*.exe -Algorithm SHA256`
Neither is wired up yet. Until one is, an installer is only as trustworthy as
the share it arrived on.
### Changing what ships
```powershell
.\refresh-bundle-lock.ps1 # show what changed, write nothing
.\refresh-bundle-lock.ps1 -Yes # write it
```
Then **commit `bundle-lock.json`**. That commit is the review - it is the only
place a change to what runs as SYSTEM on a customer's server becomes visible to
a human. `refresh-bundle-lock.ps1` refuses to overwrite an existing lock until
you have seen the diff, for that reason.
To stage a bundle before its lock exists: `ALLOW_UNLOCKED=1` (Bash) or
`-AllowUnlocked` (PowerShell). Bundles built that way must not be shipped.
Verifying a live server, months later and offline:
```powershell
shopdb-admin.ps1 verify
```
## Servers this installer did not build
It is built for greenfield: its own Python, its own venv, its own IIS objects.
Its upgrade path assumes the thing being upgraded came out of a previous run.
Two guards keep it from damaging a server that was deployed by hand.
**Python minor version.** An existing venv is reused, which is right for a repair
or an upgrade. It is wrong when the venv belongs to a different Python - the
wheelhouse is tagged for one minor version, so pip would die at the first
compiled package, *after* Python was installed and the app tree replaced. The
installer compares the two up front and stops with both version numbers.
**IIS objects.** Switching deployment method removes the other method's artifact,
which is correct when the installer owns both and dangerous when it does not: a
wrong `-MountAlias` would delete a live mount with no prompt. It now refuses
unless there is a version stamp proving it made the install, or you pass
`-AdoptExisting`. The refusal lists exactly what it would have removed.
An existing `web.config` is never overwritten in either case.
For the West Jefferson production server specifically, this is a **migration, not
an upgrade** - prod runs Python 3.13 against a hand-built deployment, so it needs
a deliberate window, a database backup, and web.config reconciled by hand.
## Bill of materials
Every build stages a CycloneDX 1.6 SBOM at `sbom.cdx.json`, inside the
application tree, so it installs onto the server with the app. Both ecosystems,
in one document:
- **Python** - every pin in `requirements.txt`, with the sha256 the installer
enforces. Environment markers are ignored: a `sys_platform == 'win32'`
dependency still installs on the target.
- **npm** - every package in `frontend/package-lock.json`. Build-only packages
are marked `scope: excluded` rather than dropped, so "not here" is
distinguishable from "not looked for".
It ships to the server because an air-gapped site cannot be scanned from
anywhere else. When a CVE lands, the answer is already on the box:
```powershell
shopdb-admin.ps1 verify # counts, and which bundle this is
shopdb-admin.ps1 verify -Path leaflet # is that component here, at what version
```
Generated by `scripts/generate_sbom.py` from files that are already pinned and
committed, so it is a translation rather than a scan - no network, no extra
toolchain on the build box, and byte-identical output for the same inputs. It is
deliberately not in `bundle-lock.json`: its provenance is git, not the payload.
## Client IP addresses
IIS does not set `X-Forwarded-For` on its own, and HttpPlatformHandler connects
from loopback. Without a rule, **every client reads as 127.0.0.1** - so the
GE-Enforce IP allowlist, the dashboard's visitor-location lookup and per-host
login rate limiting all stop working, silently.
The wizard asks, because the two answers are mutually exclusive:
- **Clients connect directly** (`-ClientIpSource direct`, the default) - installs
URL Rewrite from the bundle and sets `X-Forwarded-For` from `REMOTE_ADDR`.
Overwriting the header is what stops a client spoofing its own.
- **A proxy sits in front** (`-ClientIpSource proxy`) - leaves the rule off.
Behind ARR or a load balancer `REMOTE_ADDR` is the *proxy*, so applying the
rule would discard the real client IP.
An existing `web.config` is never overwritten - it is the one file on a server
that legitimately carries hand-edits. The installer reports what it found
instead.
## Deployment methods
Chosen in the wizard, and the bundle carries a SPA build for each because Vite
compiles the base path in - it cannot be switched at install time.
- **Its own site** on a port (default 8090).
- **Subpath** under an existing site, e.g. `http://<server-fqdn>/shopdb/`. Needs
no new DNS record. Three things must agree - the IIS application alias,
`MOUNT_PATH` in `.env`, and the SPA's build-time base - so the alias is fixed
per bundle (`SUBPATH_ALIAS`, default `shopdb`) and the installer refuses if the
bundle's build does not match what was asked for.
Switching between methods removes the other one's IIS artifact and reconciles
`MOUNT_PATH` and `CORS_ORIGINS`, so a server never ends up with both.
## Upgrades
Run a newer installer over an existing install. It:
- backs the database up first, **verifies** the dump is complete, and refuses to
migrate if it cannot;
- restores from that backup if migrations fail, and reports honestly that DDL
the failed migration committed cannot be undone;
- refuses to run a bundle older than what is installed;
- keeps `.env` unless new credentials are supplied, and copies it aside first;
- stops the app pool before replacing files, then starts it again.
Whether a run is an upgrade is decided by probing the **target database**, not by
whether the app directory exists - a rebuilt server pointed at an existing
database is an upgrade, and treating it as fresh would drop tables.
## Testing notes
Verified end to end on Windows Server 2025 against both a bundled MySQL 8.4 LTS and
an existing MySQL 5.6: fresh install, upgrade, re-run idempotency, failure and
rollback, uninstall, and both deployment methods including switching between
them.
**Not yet verified:** a fully air-gapped run with the network disabled at the
hypervisor, and any load in a real browser (all HTTP checks so far used curl,
which sends no `Origin` header - so `CORS_ORIGINS` is untested in anger).

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,301 @@
<#
.SYNOPSIS
Stage a lean per-site installer bundle on Windows, and verify its third-party
payload against bundle-lock.json.
.DESCRIPTION
The Windows equivalent of build-installer.sh, for a work PC with no Bash. It
does the whole job natively - it does NOT shell out to build-site.sh - so the
only tools needed are Python, Node and (to compile) Inno Setup.
What it does, in order:
1. Resolves the site's plugin closure from its profile.
2. Builds the SPA twice, because Vite compiles the base path in and it
therefore cannot be chosen at install time: once for /<alias> (subpath
deployment) and once for / (its own site).
3. Stages the backend tree: core, the chosen plugins only, and the runtime
files a deployable tree needs.
4. Writes plugins.iss so the wizard's plugin page matches the payload.
5. Copies the installer scripts from THIS directory.
6. Verifies wheels\, python\ and the MSIs against bundle-lock.json, and
FAILS if the bundle is not exactly what the lock describes.
Step 6 is the point of the script. A bundle that does not match its lock is
not shipped, and "the wheelhouse was missing" is caught here rather than by
an operator halfway through installing on a server with no network.
.PARAMETER Profile
Path to the site profile (see deploy\site-profile.example.json).
.PARAMETER RepoRoot
The repository. Defaults to four levels up from this script, which is correct
for a normal checkout.
.PARAMETER SkipFrontend
Reuse the SPA builds already staged in the bundle. For iterating on the
installer itself, where two Vite builds per run is most of the wall clock.
Never use it for a bundle you intend to ship.
.PARAMETER AllowUnlocked
Stage the bundle and report payload problems WITHOUT failing. For assembling
a bundle before its lock exists. A bundle built this way must not be shipped;
run refresh-bundle-lock.ps1, commit the lock, then build again without this.
.EXAMPLE
.\build-installer.ps1 -Profile ..\..\site-profile.example.json
.\build-installer.ps1 -Profile C:\sites\wjf.json -SkipFrontend
#>
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)] [string] $Profile,
[string] $RepoRoot = (Resolve-Path (Join-Path $PSScriptRoot '..\..\..')).Path,
[string] $BundleRoot = (Join-Path $PSScriptRoot 'bundle'),
[string] $SubpathAlias = 'shopdb',
[switch] $SkipFrontend,
[switch] $AllowUnlocked
)
$ErrorActionPreference = 'Stop'
. (Join-Path $PSScriptRoot 'bundle-lock.ps1')
function Say { param($m, $c = 'Gray') Write-Host $m -ForegroundColor $c }
function Step { param($m) Write-Host ''; Write-Host "==> $m" -ForegroundColor Cyan }
function Die { param($m, $fix = '') Write-Host ''; Write-Host " $m" -ForegroundColor Red
if ($fix) { Write-Host " $fix" -ForegroundColor Yellow }; exit 1 }
function Invoke-Tool {
# Runs a build tool and stops on a non-zero exit. npm writes progress to
# stderr on SUCCESS, so stderr alone must never be treated as failure.
param([string] $Exe, [string[]] $Arguments, [string] $WorkDir, [string] $What)
Push-Location $WorkDir
try {
$prev = $ErrorActionPreference
$ErrorActionPreference = 'Continue'
& $Exe @Arguments 2>&1 | ForEach-Object { Say " $_" 'DarkGray' }
$code = $LASTEXITCODE
$ErrorActionPreference = $prev
if ($code -ne 0) { Die "$What failed (exit $code)" }
} finally { Pop-Location }
}
if (-not (Test-Path $Profile)) { Die "profile not found: $Profile" }
if (-not (Test-Path $RepoRoot)) { Die "repo not found: $RepoRoot" }
$Profile = (Resolve-Path $Profile).Path
$RepoRoot = (Resolve-Path $RepoRoot).Path
$AppOut = Join-Path $BundleRoot 'app'
$python = (Get-Command python -ErrorAction SilentlyContinue)
if (-not $python) { $python = Get-Command py -ErrorAction SilentlyContinue }
if (-not $python) { Die 'Python not found on PATH' 'Install Python 3.14 and re-open the shell.' }
$npm = Get-Command npm.cmd -ErrorAction SilentlyContinue
if (-not $npm -and -not $SkipFrontend) { Die 'npm not found on PATH' 'Install Node, or pass -SkipFrontend to reuse the staged SPA.' }
Say ''
Say " repo : $RepoRoot"
Say " profile : $Profile"
Say " bundle : $BundleRoot"
# --- 1. plugin closure ------------------------------------------------------
# Same resolver the Linux builder uses, so both produce the same set from one
# profile instead of two implementations of the closure rules.
Step 'Resolving plugin closure'
$closure = (& $python.Source (Join-Path $RepoRoot 'scripts\resolve_plugin_closure.py') $Profile $RepoRoot)
if ($LASTEXITCODE -ne 0 -or -not $closure) { Die 'could not resolve the plugin closure from the profile' }
$closure = $closure.Trim()
Say " $closure" 'White'
# --- 2. frontend ------------------------------------------------------------
$frontend = Join-Path $RepoRoot 'frontend'
$subStaged = Join-Path $BundleRoot 'spa-subpath'
$rootStaged= Join-Path $BundleRoot 'spa-root'
if ($SkipFrontend) {
if (-not (Test-Path $subStaged) -or -not (Test-Path $rootStaged)) {
Die '-SkipFrontend was passed but no SPA build is staged' 'Run once without it.'
}
Say ''
Say ' Reusing the staged SPA builds (-SkipFrontend). NOT shippable if the frontend changed.' 'Yellow'
} else {
Step "Building the SPA for /$SubpathAlias/"
$env:SITE_PLUGINS = $closure
$env:VITE_BASE_PATH = "/$SubpathAlias/"
try { Invoke-Tool $npm.Source @('run','build','--silent') $frontend 'subpath frontend build' }
finally { Remove-Item Env:\VITE_BASE_PATH -ErrorAction SilentlyContinue }
Remove-Item $subStaged -Recurse -Force -ErrorAction SilentlyContinue
Copy-Item (Join-Path $frontend 'dist') $subStaged -Recurse -Force
# The alias is fixed at BUILD time; the installer reads this and refuses to
# publish under a different one rather than serving a page that cannot load
# its own assets.
Set-Content -Path (Join-Path $subStaged '.alias') -Value $SubpathAlias -Encoding ASCII
# Root build LAST, so frontend\dist is left in the state a developer expects.
Step 'Building the SPA for /'
try { Invoke-Tool $npm.Source @('run','build','--silent') $frontend 'root frontend build' }
finally { Remove-Item Env:\SITE_PLUGINS -ErrorAction SilentlyContinue }
Remove-Item $rootStaged -Recurse -Force -ErrorAction SilentlyContinue
Copy-Item (Join-Path $frontend 'dist') $rootStaged -Recurse -Force
}
# --- 3. backend tree --------------------------------------------------------
Step "Staging the application tree"
Remove-Item $AppOut -Recurse -Force -ErrorAction SilentlyContinue
New-Item -ItemType Directory -Path (Join-Path $AppOut 'plugins') -Force | Out-Null
Copy-Item (Join-Path $RepoRoot 'shopdb') $AppOut -Recurse -Force
foreach ($name in $closure.Split(',')) {
$src = Join-Path $RepoRoot ('plugins\' + $name.Trim())
if (-not (Test-Path $src)) { Die "plugin in the closure is not on disk: $name" }
Copy-Item $src (Join-Path $AppOut 'plugins') -Recurse -Force
}
# Runtime files a deployable tree needs beyond the Python packages. Without
# these the tree imports but cannot be run or migrated.
foreach ($f in @('wsgi.py', 'requirements.txt')) {
Copy-Item (Join-Path $RepoRoot $f) $AppOut -Force
}
Copy-Item (Join-Path $RepoRoot 'migrations') $AppOut -Recurse -Force
# ONLY web.config, not all of deploy\. The bundle is staged INSIDE deploy\, so
# copying the whole tree would recurse into its own output, and the rest of
# deploy\ is installer source that has no business on an application server.
# shopdb-install.ps1 reads it from exactly this path.
$cfgSrc = Join-Path $RepoRoot 'deploy\windows\web.config'
if (Test-Path $cfgSrc) {
New-Item -ItemType Directory -Path (Join-Path $AppOut 'deploy\windows') -Force | Out-Null
Copy-Item $cfgSrc (Join-Path $AppOut 'deploy\windows') -Force
}
# A CycloneDX SBOM of everything this tree depends on, Python and npm together.
# Staged INTO the tree so it installs onto the server with the application: an
# air-gapped site cannot be scanned remotely, so the only way to answer "are we
# exposed to this CVE, and where" is for the answer to be sitting on the box.
Step 'Generating SBOM'
& $python.Source (Join-Path $RepoRoot 'scripts\generate_sbom.py') $RepoRoot `
-o (Join-Path $AppOut 'sbom.cdx.json') | ForEach-Object { Say " $_" 'White' }
if ($LASTEXITCODE -ne 0) { Die 'SBOM generation failed' }
# Docs the running site serves, plus the runbooks an air-gapped server has no
# other way to reach. Without openapi.json and llms.txt the self-hosted /api/docs
# page is broken on every installed server.
Step 'Staging docs'
$docsOut = Join-Path $AppOut 'docs'
New-Item -ItemType Directory -Path $docsOut -Force | Out-Null
foreach ($doc in @('openapi.json', 'llms.txt', 'api-inventory.json',
'INSTALL-WINDOWS.md', 'OPERATE-WINDOWS.md',
'BACKUP-RESTORE.md', 'UPGRADE.md')) {
$src = Join-Path $RepoRoot ('docs\' + $doc)
if (Test-Path $src) { Copy-Item $src $docsOut -Force; Say " $doc" }
}
# Stage the profile INTO the tree: `flask plugin apply-profile` at provisioning
# reads the same profile the tree was staged from, so the installed plugin set
# and the shipped plugin code cannot drift.
Copy-Item $Profile (Join-Path $AppOut 'site-profile.json') -Force
# The installer expects frontend\dist and frontend\dist-subpath.
$feOut = Join-Path $AppOut 'frontend'
New-Item -ItemType Directory -Path $feOut -Force | Out-Null
Copy-Item $rootStaged (Join-Path $feOut 'dist') -Recurse -Force
Copy-Item $subStaged (Join-Path $feOut 'dist-subpath') -Recurse -Force
Get-ChildItem $AppOut -Recurse -Directory -Filter '__pycache__' -ErrorAction SilentlyContinue |
Remove-Item -Recurse -Force -ErrorAction SilentlyContinue
Get-ChildItem $AppOut -Recurse -File -Filter '*.pyc' -ErrorAction SilentlyContinue |
Remove-Item -Force -ErrorAction SilentlyContinue
# --- 4. plugins.iss ---------------------------------------------------------
Step 'Writing plugins.iss'
$shipped = (Get-ChildItem (Join-Path $AppOut 'plugins') -Directory | Select-Object -ExpandProperty Name) -join ','
$aliasBuilt = ''
$aliasFile = Join-Path $feOut 'dist-subpath\.alias'
if (Test-Path $aliasFile) { $aliasBuilt = (Get-Content $aliasFile -TotalCount 1).Trim() }
@"
; GENERATED by build-installer.ps1 - do not edit.
; The plugins present in bundle\app\plugins at build time.
#define AvailablePlugins "$shipped"
; The alias the subpath SPA was built for, or empty if this bundle has no
; subpath build - in which case the wizard must not offer that option.
#define SubpathAlias "$aliasBuilt"
"@ | Set-Content -Path (Join-Path $PSScriptRoot 'plugins.iss') -Encoding ASCII
Say " $shipped" 'White'
# The product version, read from the code rather than restated in the .iss.
Step 'Writing version.iss'
$initText = Get-Content (Join-Path $RepoRoot 'shopdb\__init__.py') -Raw
if ($initText -notmatch "(?m)^__version__\s*=\s*'([^']+)'") { Die 'could not read __version__ from shopdb\__init__.py' }
$appVersion = $Matches[1]
@"
; GENERATED by build-installer.ps1 from shopdb/__init__.py - do not edit.
#define AppVersion "$appVersion"
"@ | Set-Content -Path (Join-Path $PSScriptRoot 'version.iss') -Encoding ASCII
Say " $appVersion" 'White'
# --- 5. installer scripts ---------------------------------------------------
# From THIS directory, which is the reviewed copy under version control. They
# used to be copied from a downloads folder, so the logic that shipped was not
# the logic that was committed and the build only worked on one machine.
Step 'Copying installer scripts'
foreach ($f in @('shopdb-install.ps1', 'shopdb-preflight.ps1', 'bundle-lock.ps1')) {
$src = Join-Path $PSScriptRoot $f
if (-not (Test-Path $src)) { Die "installer script missing from the repo: $f" }
Copy-Item $src $BundleRoot -Force
Say " $f"
}
# --- 6. payload verification ------------------------------------------------
Step 'Verifying the third-party payload against bundle-lock.json'
$lockPath = Join-Path $PSScriptRoot 'bundle-lock.json'
$lock = Read-BundleLock $lockPath
if (-not $lock) {
$msg = "no bundle-lock.json at $lockPath"
if ($AllowUnlocked) { Say " $msg - continuing because -AllowUnlocked was passed" 'Yellow' }
else {
Die $msg @'
Add the wheels and installers to the bundle, then:
.\refresh-bundle-lock.ps1 (review the list)
.\refresh-bundle-lock.ps1 -Yes (write it)
and commit bundle-lock.json. Pass -AllowUnlocked to stage a bundle without one -
it must not be shipped.
'@
}
} else {
$problems = Test-BundleLock -BundleRoot $BundleRoot -Lock $lock
if ($problems.Count -eq 0) {
Say (" payload matches the lock ({0}, {1})" -f `
(Get-JsonProperty $lock 'pythontag' 'unknown'), (Get-JsonProperty $lock 'platform' 'unknown')) 'Green'
# Ships WITH the bundle: the installer re-checks the payload on the
# target server before running any of it, so tampering between build and
# install is caught too.
Copy-Item $lockPath $BundleRoot -Force
} else {
Say ''
foreach ($p in $problems) { Say " $p" 'Red' }
Say ''
if ($AllowUnlocked) {
Say ' -AllowUnlocked: continuing anyway. DO NOT SHIP this bundle.' 'Yellow'
} else {
Die ("{0} payload problem(s) - the bundle is not what the lock describes" -f $problems.Count) @'
Either the payload is wrong (fix the bundle) or it changed on purpose (run
refresh-bundle-lock.ps1, read the diff, and commit the new lock).
'@
}
}
}
# --- summary ----------------------------------------------------------------
Write-Host ''
Say " Bundle staged at: $BundleRoot" 'White'
foreach ($d in @('app', 'wheels', 'python', 'httpplatformhandler', 'urlrewrite', 'vcredist', 'mysqlclient', 'mysql')) {
$p = Join-Path $BundleRoot $d
if (Test-Path $p) {
$mb = ((Get-ChildItem $p -Recurse -File | Measure-Object Length -Sum).Sum / 1MB)
Say (" {0,-22} {1,8:N1} MB" -f $d, $mb)
} else {
Say (" {0,-22} {1}" -f $d, 'absent') 'DarkGray'
}
}
Write-Host ''
Say ' Compile with: iscc ShopDBFlask.iss' 'White'
Say ' (Inno Setup 6.6.0 or newer - the wizard uses the windows11 custom style.)' 'DarkGray'
Write-Host ''

View File

@@ -0,0 +1,141 @@
#!/bin/bash
# Stage a lean per-site bundle next to ShopDBFlask.iss, ready for Inno Setup.
#
# The bundle is built FOR ONE SITE from its plugin profile (ADR-013): plugins the
# site did not choose are absent from the payload entirely. Build one installer
# per site, not one universal installer.
#
# Usage: build-installer.sh <site-profile.json> [repo-path]
#
# The wheelhouse cannot be built here. Wheels are cp314 win_amd64 and must be
# produced ON Windows with the matching Python:
# pip download -r requirements.txt -d wheels --only-binary=:all:
# Copy that wheels\ directory in before compiling.
set -euo pipefail
HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROFILE="${1:?usage: build-installer.sh <site-profile.json> [repo-path]}"
REPO="${2:-$HOME/projects/shopdb-flask}"
BUNDLE="$HERE/bundle"
[ -f "$PROFILE" ] || { echo "profile not found: $PROFILE"; exit 1; }
[ -d "$REPO" ] || { echo "repo not found: $REPO"; exit 1; }
echo "==> Staging lean app tree from $PROFILE"
rm -rf "$BUNDLE/app"
bash "$REPO/scripts/build-site.sh" "$PROFILE" "$BUNDLE/app"
# build-site.sh emits the SPA as frontend-dist; the installer's web.config and
# static route expect frontend\dist.
if [ -d "$BUNDLE/app/frontend-dist" ]; then
mkdir -p "$BUNDLE/app/frontend"
rm -rf "$BUNDLE/app/frontend/dist"
mv "$BUNDLE/app/frontend-dist" "$BUNDLE/app/frontend/dist"
fi
# The /shopdb-based build, used when the operator picks the subpath deployment.
if [ -d "$BUNDLE/app/frontend-dist-subpath" ]; then
rm -rf "$BUNDLE/app/frontend/dist-subpath"
mv "$BUNDLE/app/frontend-dist-subpath" "$BUNDLE/app/frontend/dist-subpath"
fi
# Tell the .iss which plugins this bundle actually carries, so the wizard's
# plugin page always matches the payload instead of a hand-maintained list.
echo "==> Writing plugins.iss"
PLUGINS=$(ls "$BUNDLE/app/plugins" 2>/dev/null | tr '\n' ',' | sed 's/,$//')
# The subpath SPA is built with its base path compiled in, so whether the wizard
# can OFFER a subpath install is a property of the bundle, not a runtime choice.
SUBPATH_ALIAS_BUILT=""
if [ -f "$BUNDLE/app/frontend/dist-subpath/.alias" ]; then
SUBPATH_ALIAS_BUILT="$(cat "$BUNDLE/app/frontend/dist-subpath/.alias")"
fi
cat > "$HERE/plugins.iss" <<EOF
; GENERATED by build-installer.sh - do not edit.
; The plugins present in bundle\\app\\plugins at build time.
#define AvailablePlugins "$PLUGINS"
; The alias the subpath SPA was built for, or empty if this bundle has no
; subpath build - in which case the wizard must not offer that option.
#define SubpathAlias "$SUBPATH_ALIAS_BUILT"
EOF
echo " $PLUGINS"
# The product version, read from the code rather than restated here. A hardcoded
# AppVersion in the .iss had drifted two minor versions from shopdb/__init__.py.
echo "==> Writing version.iss"
APPVERSION=$(sed -n "s/^__version__ = '\\(.*\\)'/\\1/p" "$REPO/shopdb/__init__.py" | head -1)
[ -n "$APPVERSION" ] || { echo "could not read __version__ from shopdb/__init__.py"; exit 1; }
cat > "$HERE/version.iss" <<EOF
; GENERATED by build-installer.sh from shopdb/__init__.py - do not edit.
#define AppVersion "$APPVERSION"
EOF
echo " $APPVERSION"
# From THIS directory, which is the reviewed copy under version control. These
# used to be copied from $HOME/Downloads, so the installer logic that shipped was
# not the logic that was committed, and the build only worked on one machine.
echo "==> Copying installer scripts"
mkdir -p "$BUNDLE"
for f in shopdb-install.ps1 shopdb-preflight.ps1 bundle-lock.ps1; do
[ -f "$HERE/$f" ] || { echo "installer script missing from the repo: $f"; exit 1; }
cp "$HERE/$f" "$BUNDLE/"
done
echo ""
echo "Bundle staged at: $BUNDLE"
for d in app wheels python httpplatformhandler urlrewrite vcredist mysqlclient mysql; do
if [ -d "$BUNDLE/$d" ]; then
printf ' %-20s %s\n' "$d" "$(du -sh "$BUNDLE/$d" | cut -f1)"
else
printf ' %-20s absent\n' "$d"
fi
done
echo ""
echo " plugins shipped: $(ls "$BUNDLE/app/plugins" 2>/dev/null | tr '\n' ' ')"
# --- payload verification ---------------------------------------------------
# The third-party payload is the part git does not record: the wheels, the Python
# installer and the MSIs that run as SYSTEM on the target server. It must be
# EXACTLY what bundle-lock.json describes - no missing file, no stale extra wheel
# left over from a previous build, no changed content - or this is not a bundle
# anyone reviewed. Previously a missing wheelhouse printed MISSING and the script
# still exited 0, so an empty bundle compiled into a shippable installer and the
# failure surfaced on an air-gapped server with no way to fix it.
#
# ALLOW_UNLOCKED=1 downgrades this to a warning, for assembling a bundle before
# its lock exists. A bundle built that way must not be shipped.
echo ""
echo "==> Verifying the third-party payload against bundle-lock.json"
if python3 "$HERE/verify_bundle_lock.py" "$BUNDLE" "$HERE/bundle-lock.json"; then
echo " payload matches the lock"
# Ships WITH the bundle: the installer re-checks the payload on the target
# server before running any of it, so tampering between build and install is
# caught too.
cp "$HERE/bundle-lock.json" "$BUNDLE/"
elif [ "${ALLOW_UNLOCKED:-0}" = "1" ]; then
echo ""
echo " ALLOW_UNLOCKED=1: continuing anyway. DO NOT SHIP this bundle."
else
echo ""
echo " The bundle is not what the lock describes."
echo ""
echo " Add the missing pieces by hand:"
echo " wheels/ pip download -r requirements.txt --only-binary=:all: \\"
echo " --platform win_amd64 --python-version 314 \\"
echo " --implementation cp --abi cp314 -d wheels"
echo " python/ python-3.14.x-amd64.exe"
echo " httpplatformhandler/ httpPlatformHandler_amd64.msi"
echo " urlrewrite/ rewrite_amd64.msi (client-IP rule; see README)"
echo " vcredist/ VC_redist.x64.exe (MySQL requires it)"
echo " mysql/ mysql-8.4.x-winx64.msi (bundled-database option only)"
echo ""
echo " If the payload changed ON PURPOSE, regenerate and COMMIT the lock:"
echo " pwsh ./refresh-bundle-lock.ps1 # review the diff"
echo " pwsh ./refresh-bundle-lock.ps1 -Yes # write it"
echo ""
echo " To stage a bundle before its lock exists: ALLOW_UNLOCKED=1 $0 ..."
exit 1
fi
echo ""
echo "Then compile on Windows: iscc ShopDBFlask.iss"
echo "(Inno Setup 6.6.0 or newer - the wizard uses the windows11 custom style.)"

View File

@@ -0,0 +1,240 @@
{
"schema": 1,
"generated": "2026-08-04T23:11:22Z",
"pythontag": "cp314",
"platform": "win_amd64",
"note": "Exact third-party payload of the installer bundle. Regenerate with refresh-bundle-lock.ps1 and COMMIT the change as a reviewed dependency bump.",
"payloads": {
"httpplatformhandler": {
"files": {
"httpPlatformHandler_amd64.msi": {
"sha256": "90f8d4905a0ab4f2c95223b3c79e2807a0b74507747d240e43c4302e8db4b5ef",
"size": 557056
}
},
"required": true
},
"python": {
"files": {
"python-3.14.6-amd64.exe": {
"sha256": "14b3e9a710a3fcf0bd9b55ab6b60412bd91227563f813fc49040cabc0209e0bd",
"size": 30774112
}
},
"required": true
},
"vcredist": {
"files": {
"VC_redist.x64.exe": {
"sha256": "cc0ff0eb1dc3f5188ae6300faef32bf5beeba4bdd6e8e445a9184072096b713b",
"size": 25635768
}
},
"required": false
},
"mysql": {
"files": {
"mysql-8.4.6-winx64.msi": {
"sha256": "868885b2e221409f0170729bb30b2b2ce83440a1a4c735d19b3c225b51045762",
"size": 135081984
}
},
"required": false
},
"wheels": {
"files": {
"email_validator-2.3.0-py3-none-any.whl": {
"sha256": "80f13f623413e6b197ae73bb10bf4eb0908faf509ad8362c5edeb0be7fd450b4",
"size": 35604
},
"cachelib-0.13.0-py3-none-any.whl": {
"sha256": "8c8019e53b6302967d4e8329a504acf75e7bc46130291d30188a6e4e58162516",
"size": 20914
},
"flask_caching-2.4.0-py3-none-any.whl": {
"sha256": "d15b8135f055c4f28f6f7dbcf8d36a3de4af1224def975ae6e0b43cbfa684486",
"size": 28727
},
"typing_extensions-4.15.0-py3-none-any.whl": {
"sha256": "f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548",
"size": 44614
},
"greenlet-3.5.0-cp314-cp314-win_amd64.whl": {
"sha256": "3bc59be3945ae9750b9e7d45067d01ae3fe90ea5f9ade99239dabdd6e28a5033",
"size": 239835
},
"charset_normalizer-3.4.7-cp314-cp314-win_amd64.whl": {
"sha256": "92a0a01ead5e668468e952e4238cccd7c537364eb7d851ab144ab6627dbbe12f",
"size": 159634
},
"colorama-0.4.6-py2.py3-none-any.whl": {
"sha256": "4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6",
"size": 25335
},
"Flask_Migrate-4.1.0-py3-none-any.whl": {
"sha256": "24d8051af161782e0743af1b04a152d007bad9772b2bca67b7ec1e8ceeb3910d",
"size": 21237
},
"markupsafe-3.0.3-cp314-cp314-win_amd64.whl": {
"sha256": "bdc919ead48f234740ad807933cdf545180bfbe9342c2bb451556db2ed958581",
"size": 15341
},
"flask_marshmallow-1.5.0-py3-none-any.whl": {
"sha256": "99951c77e5654111ed733811c6dc9310bfb4c3688c78a9e76f80b5ae0b2279a6",
"size": 12161
},
"tzdata-2026.3-py2.py3-none-any.whl": {
"sha256": "dc096730c87af6cab1b171c9d532be840741ff5d459015e7f6947bd7d7e54931",
"size": 348168
},
"python_dotenv-1.2.2-py3-none-any.whl": {
"sha256": "1d8214789a24de455a8b8bd8ae6fe3c6b69a5e3d64aa8a8e5d68e694bbcb285a",
"size": 22101
},
"dnspython-2.8.0-py3-none-any.whl": {
"sha256": "01d9bbc4a2d76bf0db7c1f729812ded6d912bd318d3b1cf81d30c0f845dbf3af",
"size": 331094
},
"pymysql-1.1.3-py3-none-any.whl": {
"sha256": "8164ba62c552f6105f3b11753352d0f16b90d1703ba67d81923d5a8a5d1c5289",
"size": 45356
},
"packaging-26.3-py3-none-any.whl": {
"sha256": "d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c",
"size": 129956
},
"flask_jwt_extended-4.7.3-py2.py3-none-any.whl": {
"sha256": "905ac807b52b5409bc9244dbcca434968c13ca6f9d91bffe7d4cb71e0e6231cb",
"size": 22698
},
"urllib3-2.7.0-py3-none-any.whl": {
"sha256": "9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897",
"size": 131087
},
"marshmallow-4.3.0-py3-none-any.whl": {
"sha256": "46c4fe6984707e3cbd485dfebbf0a59874f58d695aad05c1668d15e8c6e13b46",
"size": 49148
},
"waitress-3.0.2-py3-none-any.whl": {
"sha256": "c56d67fd6e87c2ee598b76abdd4e96cfad1f24cacdea5078d382b1f9d7b5ed2e",
"size": 56232
},
"idna-3.13-py3-none-any.whl": {
"sha256": "892ea0cde124a99ce773decba204c5552b69c3c67ffd5f232eb7696135bc8bb3",
"size": 68629
},
"flask-3.1.3-py3-none-any.whl": {
"sha256": "f4bcbefc124291925f1a26446da31a5178f9483862233b23c0c96a20701f670c",
"size": 103424
},
"werkzeug-3.1.8-py3-none-any.whl": {
"sha256": "63a77fb8892bf28ebc3178683445222aa500e48ebad5ec77b0ad80f8726b1f50",
"size": 226459
},
"flask_sqlalchemy-3.1.1-py3-none-any.whl": {
"sha256": "4ba4be7f419dc72f4efd8802d69974803c37259dd42f3913b0dcf75c9447e0a0",
"size": 25125
},
"marshmallow_sqlalchemy-1.5.0-py3-none-any.whl": {
"sha256": "3865232672f3dd38c4d5e4e85fdedce76904200742c3594948a2d11d0af93258",
"size": 16582
},
"alembic-1.18.4-py3-none-any.whl": {
"sha256": "a5ed4adcf6d8a4cb575f3d759f071b03cd6e5c7618eb796cb52497be25bfe19a",
"size": 263893
},
"tabulate-0.10.0-py3-none-any.whl": {
"sha256": "f0b0622e567335c8fabaaa659f1b33bcb6ddfe2e496071b743aa113f8774f2d3",
"size": 39814
},
"cffi-2.1.0-cp314-cp314-win_amd64.whl": {
"sha256": "1b96bfe2c4bd825681b7d311ad6d9b7280a091f43e8f63da5729638083cd3bfb",
"size": 187937
},
"jinja2-3.1.6-py3-none-any.whl": {
"sha256": "85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67",
"size": 134899
},
"cryptography-50.0.0-cp311-abi3-win_amd64.whl": {
"sha256": "bd1c592e4d5974f0d08d4888e432157adba757c66da0246918e43677fafa2d30",
"size": 3840395
},
"certifi-2026.4.22-py3-none-any.whl": {
"sha256": "3cb2210c8f88ba2318d29b0388d1023c8492ff72ecdde4ebdaddbb13a31b1c4a",
"size": 135707
},
"sqlalchemy-2.0.49-cp314-cp314-win_amd64.whl": {
"sha256": "77641d299179c37b89cf2343ca9972c88bb6eef0d5fc504a2f86afd15cd5adf5",
"size": 2144204
},
"blinker-1.9.0-py3-none-any.whl": {
"sha256": "ba0efaa9080b619ff2f3459d1d500c57bddea4a6b424b60a91141db6fd2f08bc",
"size": 8458
},
"pycparser-3.0-py3-none-any.whl": {
"sha256": "b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992",
"size": 48172
},
"flask_cors-6.0.2-py3-none-any.whl": {
"sha256": "e57544d415dfd7da89a9564e1e3a9e515042df76e12130641ca6f3f2f03b699a",
"size": 13257
},
"requests-2.33.1-py3-none-any.whl": {
"sha256": "4e6d1ef462f3626a1f0a0a9c42dd93c63bad33f9f1c1937509b8c5c8718ab56a",
"size": 64947
},
"pyjwt-2.12.1-py3-none-any.whl": {
"sha256": "28ca37c070cad8ba8cd9790cd940535d40274d22f80ab87f3ac6a713e6e8454c",
"size": 29726
},
"click-8.3.3-py3-none-any.whl": {
"sha256": "a2bf429bb3033c89fa4936ffb35d5cb471e3719e1f3c8a7c3fff0b8314305613",
"size": 110502
},
"mysql_connector_python-9.7.0-cp314-cp314-win_amd64.whl": {
"sha256": "5a5abbc152bc28cb2e64a04605ecd9941eff6b0dc5f9528cb84adb873e9a1e49",
"size": 18197576
},
"mako-1.3.12-py3-none-any.whl": {
"sha256": "8f61569480282dbf557145ce441e4ba888be453c30989f879f0d652e39f53ea9",
"size": 78521
},
"itsdangerous-2.2.0-py3-none-any.whl": {
"sha256": "c6242fc49e35958c8b15141343aa660db5fc54d4f13a1db01a3f5891b98700ef",
"size": 16234
}
},
"required": true
},
"urlrewrite": {
"files": {
"rewrite_amd64_en-US.msi": {
"sha256": "37342ff2f585f263f34f48e9de59eb1051d61015a8e967dbde4075716230a32a",
"size": 6078464
}
},
"required": false
},
"mysqlclient": {
"files": {
"libcrypto-3-x64.dll": {
"sha256": "e84ec61fc1b07a3c899ae39c1dc3f38591cdd310c043a388e7ab40a6828aa297",
"size": 5187216
},
"mysqldump.exe": {
"sha256": "329410b7a8ae3d68d38e6f5b781610770854d3f11b975060e231dae62c20dc46",
"size": 7142528
},
"mysql.exe": {
"sha256": "bd6880253c1853fc17b1e19129d6e5d9f9c86a2dccf2c7c9e6b878a10baa9770",
"size": 7177336
},
"libssl-3-x64.dll": {
"sha256": "e24fcd6c8ec31a14c97f520b5b960ed360c55ddf0935040eff297e511697633e",
"size": 788616
}
},
"required": false
}
}
}

View File

@@ -0,0 +1,261 @@
<#
.SYNOPSIS
Hash manifest for the installer's third-party payload: create it, and check a
bundle against it.
.DESCRIPTION
Dot-source this. It defines two functions and runs nothing on its own:
New-BundleLock hash a staged bundle and return the lock object
Test-BundleLock compare a staged bundle against a lock, return problems
WHAT IT COVERS, and why pip's own hash checking is not enough.
requirements.txt carries a sha256 for every wheel, so pip refuses an artifact
upstream did not publish. Three gaps remain, and all three are what actually
goes wrong with a hand-assembled offline bundle:
1. pip lists EVERY artifact of a pinned version - cffi 2.1.0 alone has 100
hashes. It proves the wheel is genuine, not that it is the wheel this
bundle was built and tested with.
2. pip ignores extra files in the wheelhouse. A stale wheel left behind by
a previous build sits there unnoticed until a resolve picks it up.
3. pip says nothing about the rest of the payload - the Python installer,
the HttpPlatformHandler MSI, URL Rewrite, MySQL. Those are executables
that run as SYSTEM on the target server and were, until this file, the
only unverified thing the installer would run.
So the lock records an exact file set with a sha256 and a byte size each, and
verification is SET EQUALITY: a missing file, an unexpected extra file, or a
changed file all fail. Nothing is skipped and nothing is "close enough".
The app tree is deliberately NOT covered. It is built from the repository on
every run and changes with every commit; hashing it would make the lock churn
constantly and train everyone to regenerate it without reading it. Git is the
record for the app tree. This file is the record for everything that comes
from outside the repository.
.NOTES
Stock Windows PowerShell 5.1, and also runs under pwsh on Linux so the Bash
builder can call the same checker.
#>
# Payload directories under the bundle root. 'required' means the installer
# cannot work without it; the optional ones are per-deployment choices, and an
# absent optional directory is fine. A PRESENT directory is always checked in
# full, optional or not.
$script:BundlePayloads = @(
@{ Name = 'wheels'; Required = $true; What = 'Python wheels for the offline install' }
@{ Name = 'python'; Required = $true; What = 'the Python installer' }
@{ Name = 'httpplatformhandler'; Required = $true; What = 'the IIS module that launches waitress' }
@{ Name = 'urlrewrite'; Required = $false; What = 'IIS URL Rewrite, for the client-IP rule' }
@{ Name = 'mysqlclient'; Required = $false; What = 'mysql/mysqldump, for backups against a remote database' }
@{ Name = 'vcredist'; Required = $false; What = 'the Visual C++ runtime MySQL requires' }
@{ Name = 'mysql'; Required = $false; What = 'MySQL, for the bundled-database option' }
)
function Get-JsonProperty {
# shopdb-install.ps1 runs under Set-StrictMode 2.0, where reading a property
# that does not exist on a PSCustomObject THROWS instead of returning $null.
# A truncated or hand-edited bundle-lock.json would therefore blow up with
# "Property 'payloads' cannot be found" rather than saying what is wrong with
# the lock. Every read of parsed JSON goes through here.
param($Object, [string] $Name, $Default = $null)
if ($null -eq $Object) { return $Default }
$prop = $Object.PSObject.Properties[$Name]
if ($null -eq $prop) { return $Default }
return $prop.Value
}
function Get-FileDigest {
param([string] $Path)
$sha = [System.Security.Cryptography.SHA256]::Create()
$stream = [System.IO.File]::OpenRead($Path)
try { return (-join ($sha.ComputeHash($stream) | ForEach-Object { $_.ToString('x2') })) }
finally { $stream.Dispose(); $sha.Dispose() }
}
function Get-PayloadFiles {
<#
Every file in the directory, keyed by its path RELATIVE to that directory
with forward slashes, so a lock generated on Windows reads the same from
the Bash builder.
The root comes from Get-Item, NOT Resolve-Path, and the prefix is checked
before it is trimmed. Both matter, and a real install proved it:
Inno extracts the bundle under C:\Users\ADMINI~1\AppData\Local\Temp\... -
an 8.3 SHORT path. Resolve-Path kept that short form while Get-ChildItem
returned the long one (Administrator), so the root was five characters
shorter than the prefix it was slicing off. Every relative path came out
mangled - 'wheels/heels/flask.whl' - and the verifier reported all 96 files
as simultaneously missing and unexpected. The payload was fine; the
comparison was not.
Get-Item and Get-ChildItem go through the same provider, so their path
forms agree. The StartsWith guard means that if they ever disagree again
this fails loudly instead of inventing paths.
#>
param([string] $Dir)
$out = @{}
if (-not (Test-Path $Dir)) { return $out }
$rootItem = Get-Item -LiteralPath $Dir
$root = $rootItem.FullName.TrimEnd('\', '/')
foreach ($f in (Get-ChildItem -LiteralPath $rootItem.FullName -Recurse -File)) {
if (-not $f.FullName.StartsWith($root, [System.StringComparison]::OrdinalIgnoreCase)) {
throw ("cannot place '{0}' beneath '{1}' - path forms disagree (8.3 short name?)" -f $f.FullName, $root)
}
$rel = $f.FullName.Substring($root.Length).TrimStart('\', '/').Replace('\', '/')
$out[$rel] = @{ sha256 = (Get-FileDigest $f.FullName); size = $f.Length }
}
return $out
}
function New-BundleLock {
<#
Hash a staged bundle. The caller writes the result to bundle-lock.json;
this returns the object so a caller can diff it against the committed lock
before overwriting anything.
#>
param(
[Parameter(Mandatory = $true)] [string] $BundleRoot,
[string] $PythonTag = 'cp314',
[string] $Platform = 'win_amd64'
)
$payloads = @{}
foreach ($p in $script:BundlePayloads) {
$dir = Join-Path $BundleRoot $p.Name
if (-not (Test-Path $dir)) {
if ($p.Required) { throw ("payload directory is missing: {0} ({1})" -f $p.Name, $p.What) }
continue
}
$files = Get-PayloadFiles $dir
if ($files.Count -eq 0 -and $p.Required) {
throw ("payload directory is empty: {0} ({1})" -f $p.Name, $p.What)
}
$payloads[$p.Name] = @{ required = $p.Required; files = $files }
}
return [ordered]@{
schema = 1
generated = (Get-Date).ToUniversalTime().ToString('yyyy-MM-ddTHH:mm:ssZ')
pythontag = $PythonTag
platform = $Platform
note = 'Exact third-party payload of the installer bundle. Regenerate with refresh-bundle-lock.ps1 and COMMIT the change as a reviewed dependency bump.'
payloads = $payloads
}
}
function Test-BundleLock {
<#
Compare a staged bundle against a lock. Returns an array of problem
strings - EMPTY means the bundle is exactly what the lock describes.
Returning problems rather than throwing is deliberate: an operator fixing a
wheelhouse wants the whole list at once, not one failure per rebuild.
#>
param(
[Parameter(Mandatory = $true)] [string] $BundleRoot,
[Parameter(Mandatory = $true)] $Lock
)
$problems = @()
$payloads = Get-JsonProperty $Lock 'payloads'
# ,@(...) for the same reason as the final return: a bare one-element array
# unrolls to a string, and the caller's .Count then measures the wrong thing.
if ($null -eq $payloads) { return ,@('bundle-lock.json has no "payloads" section') }
foreach ($p in $script:BundlePayloads) {
$name = $p.Name
$dir = Join-Path $BundleRoot $name
$present = Test-Path $dir
$entry = Get-JsonProperty $payloads $name
$locked = ($null -ne $entry)
if (-not $locked) {
# Not in the lock at all. An unlocked directory that exists is a
# payload nobody reviewed, which is exactly what this is here to stop.
if ($present) { $problems += "$name/ is present but is not in bundle-lock.json - regenerate the lock" }
elseif ($p.Required) { $problems += "$name/ is required but is in neither the bundle nor the lock" }
continue
}
if (-not $present) {
if ($p.Required -or (Get-JsonProperty $entry 'required' $false)) {
$problems += "$name/ is in the lock but missing from the bundle ($($p.What))"
}
continue
}
$expected = @{}
$lockedFiles = Get-JsonProperty $entry 'files'
if ($null -ne $lockedFiles) {
foreach ($prop in $lockedFiles.PSObject.Properties) { $expected[$prop.Name] = $prop.Value }
}
$actual = Get-PayloadFiles $dir
# Sorted so the report reads the same way twice, and so it matches the
# order verify_bundle_lock.py produces.
foreach ($rel in ($expected.Keys | Sort-Object)) {
if (-not $actual.ContainsKey($rel)) { $problems += "$name/$rel is in the lock but missing from the bundle"; continue }
if ($actual[$rel].sha256 -ne $expected[$rel].sha256) {
$problems += "$name/$rel does NOT match the lock (expected sha256 $($expected[$rel].sha256.Substring(0,12))..., got $($actual[$rel].sha256.Substring(0,12))...)"
} elseif ([int64] $actual[$rel].size -ne [int64] $expected[$rel].size) {
# Cannot happen for a matching sha256, so it means the lock itself
# was hand-edited. Say so rather than passing it.
$problems += "$name/$rel size disagrees with the lock - the lock has been edited by hand"
}
}
foreach ($rel in ($actual.Keys | Sort-Object)) {
if (-not $expected.ContainsKey($rel)) { $problems += "$name/$rel is in the bundle but NOT in the lock (unexpected extra file)" }
}
}
$problems += Test-WheelhouseCoversRequirements -BundleRoot $BundleRoot
# The comma keeps this an ARRAY through the return. Without it PowerShell
# unrolls an empty result to $null and a single result to a bare string, and
# every caller that asks for .Count then behaves differently depending on how
# many problems there happen to be.
return ,$problems
}
function Test-WheelhouseCoversRequirements {
<#
The lock records what IS in the wheelhouse, not what the application NEEDS.
Without this an incomplete wheelhouse gets locked, blessed, and shipped,
and the install fails on an air-gapped server.
Not hypothetical: assembling the wheelhouse anywhere other than Windows
silently omits colorama, a win32-only dependency of click, because pip
evaluates environment markers against the machine doing the downloading
rather than the machine being targeted. Markers are therefore IGNORED here
- a requirement guarded by sys_platform == 'win32' is precisely the one
that has to be present.
#>
param([Parameter(Mandatory = $true)] [string] $BundleRoot)
$wheels = Join-Path $BundleRoot 'wheels'
$reqs = Join-Path $BundleRoot 'app\requirements.txt'
if (-not (Test-Path $wheels) -or -not (Test-Path $reqs)) { return @() }
$have = @(Get-ChildItem $wheels -File -ErrorAction SilentlyContinue | ForEach-Object { $_.Name.ToLower() })
$problems = @()
$pins = @{}
foreach ($line in (Get-Content $reqs)) {
$trimmed = $line.Trim()
if (-not $trimmed -or $trimmed.StartsWith('#')) { continue }
if ($trimmed -match '^([A-Za-z0-9._-]+)==([^\s;\\]+)') {
# PEP 427 wheel filename form: runs of non-alphanumerics become one _.
$pins[([regex]::Replace($Matches[1], '[^A-Za-z0-9.]+', '_')).ToLower()] = $Matches[2]
}
}
foreach ($name in ($pins.Keys | Sort-Object)) {
$prefix = "$name-$($pins[$name])-"
if (-not ($have | Where-Object { $_.StartsWith($prefix) })) {
$problems += ("wheels/ has no wheel for {0}=={1}, which requirements.txt pins " +
"(a marked-out dependency still installs on Windows)") -f $name, $pins[$name]
}
}
return $problems
}
function Read-BundleLock {
param([Parameter(Mandatory = $true)] [string] $Path)
if (-not (Test-Path $Path)) { return $null }
return (Get-Content $Path -Raw | ConvertFrom-Json)
}

View File

@@ -0,0 +1,159 @@
#!/usr/bin/env python3
"""Generate Inno Setup wizard artwork from the app's own brand assets.
Everything here is derived from frontend/public/*.svg so the installer and the
running application are visibly the same product. Nothing is redrawn by hand.
Inno stretches artwork to fit and does not resample well, so render at the exact
sizes it asks for and supply the 125%/250% variants for high-DPI displays.
WizardImageFile 164x314, 192x386, 384x772
WizardSmallImageFile 55x55, 64x64, 138x138
SetupIconFile .ico with 16/24/32/48/64/128/256
Usage: python3 make-branding.py [output-dir]
"""
import io
import sys
from pathlib import Path
import cairosvg
from PIL import Image, ImageDraw, ImageFont
ASSETS = Path.home() / "projects/shopdb-flask/frontend/public"
OUT = Path(sys.argv[1] if len(sys.argv) > 1 else Path(__file__).parent)
# Sampled from the application's own palette so the installer does not look like
# a different product wearing the same badge.
NAVY = (10, 34, 74) # deep base
BLUE = (16, 74, 150) # GE blue
CYAN = (0, 158, 224) # accent
WHITE = (255, 255, 255)
MUTED = (176, 197, 226)
def render_svg(name, width=None, height=None):
png = cairosvg.svg2png(url=str(ASSETS / name), output_width=width, output_height=height)
return Image.open(io.BytesIO(png)).convert("RGBA")
def recolour(img, colour):
"""Replace RGB while keeping the alpha mask. Source marks are dark-on-light;
on a dark panel they must be inverted or they disappear."""
solid = Image.new("RGBA", img.size, colour + (255,))
solid.putalpha(img.getchannel("A"))
return solid
def font(size, bold=False):
for path in (
f"/usr/share/fonts/truetype/dejavu/DejaVuSans{'-Bold' if bold else ''}.ttf",
f"/usr/share/fonts/truetype/liberation/LiberationSans{'-Bold' if bold else '-Regular'}.ttf",
):
if Path(path).exists():
return ImageFont.truetype(path, size)
return ImageFont.load_default()
def vertical_gradient(size, top, bottom):
w, h = size
img = Image.new("RGB", size)
draw = ImageDraw.Draw(img)
for y in range(h):
t = y / max(1, h - 1)
# Ease the ramp so the middle does not look flat.
t = t * t * (3 - 2 * t)
draw.line(
[(0, y), (w, y)],
fill=tuple(int(top[i] + (bottom[i] - top[i]) * t) for i in range(3)),
)
return img
def banner(w, h):
img = vertical_gradient((w, h), BLUE, NAVY)
draw = ImageDraw.Draw(img)
k = w / 164.0 # scale factor from the 100% design
# Faint diagonal wash: stops the flat area under the text reading as empty.
glow = Image.new("RGBA", (w, h), (0, 0, 0, 0))
gd = ImageDraw.Draw(glow)
gd.polygon([(0, int(h * 0.52)), (w, int(h * 0.30)), (w, h), (0, h)],
fill=(255, 255, 255, 10))
img = Image.alpha_composite(img.convert("RGBA"), glow).convert("RGB")
draw = ImageDraw.Draw(img)
margin = int(22 * k)
# GE Aerospace wordmark at the top, above the product name - the corporate
# mark leads, the product sits under it. (Previously the bare monogram was
# here and the wordmark was stranded at the bottom.)
mark_w = w - (margin * 2)
mark = render_svg("ge-aerospace-logo.svg", mark_w, int(mark_w * 32 / 138))
mark = recolour(mark, WHITE)
img.paste(mark, (margin, int(34 * k)), mark)
# Product name, directly beneath it.
y = int(34 * k) + mark.height + int(30 * k)
draw.text((margin, y), "ShopDB", font=font(int(26 * k), bold=True), fill=WHITE)
y += int(31 * k)
# Hairline rule, then the descriptor. Cheap way to look considered.
draw.rectangle([margin, y, margin + int(30 * k), y + max(1, int(2 * k))], fill=CYAN)
y += int(14 * k)
for line in ("Asset management", "for the shop floor"):
draw.text((margin, y), line, font=font(int(10.5 * k)), fill=MUTED)
y += int(15 * k)
# Accent bar flush to the bottom edge.
bar = max(2, int(4 * k))
draw.rectangle([0, h - bar, w, h], fill=CYAN)
return img
def small(size):
"""Header mark on every page after the welcome page. White plate so it sits
correctly on the wizard's own header, in light or dark mode."""
img = Image.new("RGB", (size, size), WHITE)
m = int(size * 0.80)
mono = recolour(render_svg("ge-monogram.svg", m, m), BLUE)
off = (size - m) // 2
img.paste(mono, (off, off), mono)
return img
def icon(path):
"""Installer icon. Rounded navy tile with the monogram, so it reads at 16px
instead of turning into mush."""
base = 256
img = Image.new("RGBA", (base, base), (0, 0, 0, 0))
d = ImageDraw.Draw(img)
d.rounded_rectangle([0, 0, base - 1, base - 1], radius=int(base * 0.22), fill=BLUE + (255,))
d.rounded_rectangle([0, 0, base - 1, int(base * 0.5)], radius=int(base * 0.22),
fill=(30, 96, 175, 255))
d.rounded_rectangle([0, int(base * 0.3), base - 1, base - 1], radius=int(base * 0.22),
fill=BLUE + (255,))
m = int(base * 0.62)
mono = recolour(render_svg("ge-monogram.svg", m, m), WHITE)
img.paste(mono, ((base - m) // 2, (base - m) // 2), mono)
img.save(path, sizes=[(16, 16), (24, 24), (32, 32), (48, 48), (64, 64),
(128, 128), (256, 256)])
def main():
OUT.mkdir(parents=True, exist_ok=True)
made = []
for w, h, name in [(164, 314, "wizard-image.bmp"),
(192, 386, "wizard-image@125.bmp"),
(384, 772, "wizard-image@250.bmp")]:
banner(w, h).save(OUT / name, "BMP"); made.append((name, f"{w}x{h}"))
for s, name in [(55, "wizard-small.bmp"), (64, "wizard-small@125.bmp"),
(138, "wizard-small@250.bmp")]:
small(s).save(OUT / name, "BMP"); made.append((name, f"{s}x{s}"))
icon(OUT / "shopdb.ico"); made.append(("shopdb.ico", "multi-size"))
for name, dims in made:
print(f" {name:<26} {dims:<10} {(OUT / name).stat().st_size // 1024} KB")
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,110 @@
<#
.SYNOPSIS
Regenerate bundle-lock.json from the staged bundle, after showing what would
change.
.DESCRIPTION
Run this on the machine that assembled the wheelhouse, once the bundle holds
the payload you intend to ship:
bundle\wheels\ the wheels, built with the matching Python
bundle\python\ the Python installer
bundle\httpplatformhandler\ the IIS module MSI
bundle\urlrewrite\ URL Rewrite MSI (optional)
bundle\mysql\ MySQL MSI (optional)
Then COMMIT the resulting bundle-lock.json. That commit is the review: it is
the only place a change to what runs as SYSTEM on a customer's server becomes
visible to a human. A lock regenerated and committed without reading the diff
provides nothing, so this refuses to overwrite an existing lock until you
have seen the change and passed -Yes.
.EXAMPLE
.\refresh-bundle-lock.ps1 # show the diff, write nothing
.\refresh-bundle-lock.ps1 -Yes # write it
.NOTES
Building the wheelhouse itself no longer requires Windows. From any machine:
pip download -r requirements.txt -d wheels --only-binary=:all: `
--platform win_amd64 --python-version 314 --implementation cp --abi cp314
Do it wherever you like; this script records what came out.
#>
[CmdletBinding()]
param(
[string] $BundleRoot = (Join-Path $PSScriptRoot 'bundle'),
[string] $LockPath = (Join-Path $PSScriptRoot 'bundle-lock.json'),
[string] $PythonTag = 'cp314',
[string] $Platform = 'win_amd64',
[switch] $Yes
)
$ErrorActionPreference = 'Stop'
. (Join-Path $PSScriptRoot 'bundle-lock.ps1')
function Say { param($m, $c = 'Gray') Write-Host $m -ForegroundColor $c }
if (-not (Test-Path $BundleRoot)) {
Say "bundle not found: $BundleRoot" 'Red'
Say 'Stage it first with build-installer.ps1 (or build-installer.sh), then add' 'Yellow'
Say 'the wheels and installers by hand.' 'Yellow'
exit 1
}
Say ''
Say " Hashing $BundleRoot" 'Cyan'
$new = New-BundleLock -BundleRoot $BundleRoot -PythonTag $PythonTag -Platform $Platform
foreach ($name in $new.payloads.Keys) {
Say (" {0,-22} {1,4} files" -f $name, $new.payloads[$name].files.Count)
}
$old = Read-BundleLock $LockPath
if (-not $old) {
Say ''
Say ' No existing lock - this will be the first one.' 'Yellow'
} else {
# Diff by file, per payload, so the operator sees exactly which artifacts
# changed rather than "the lock is different".
Say ''
Say ' Changes against the committed lock:' 'Cyan'
$changes = 0
foreach ($name in $new.payloads.Keys) {
$oldFiles = @{}
if ($old.payloads.PSObject.Properties.Name -contains $name) {
foreach ($p in $old.payloads.$name.files.PSObject.Properties) { $oldFiles[$p.Name] = $p.Value.sha256 }
}
$newFiles = $new.payloads[$name].files
foreach ($rel in ($newFiles.Keys | Sort-Object)) {
if (-not $oldFiles.ContainsKey($rel)) { Say " + $name/$rel" 'Green'; $changes++ }
elseif ($oldFiles[$rel] -ne $newFiles[$rel].sha256) { Say " ~ $name/$rel (content changed)" 'Yellow'; $changes++ }
}
foreach ($rel in ($oldFiles.Keys | Sort-Object)) {
if (-not $newFiles.ContainsKey($rel)) { Say " - $name/$rel" 'Red'; $changes++ }
}
}
foreach ($p in $old.payloads.PSObject.Properties.Name) {
if (-not $new.payloads.Contains($p)) { Say " - $p/ (whole payload gone)" 'Red'; $changes++ }
}
if ($changes -eq 0) {
Say ' none - the bundle already matches the lock' 'Green'
exit 0
}
Say ''
Say (" {0} change(s)." -f $changes) 'White'
}
if (-not $Yes) {
Say ''
Say ' Nothing written. Read the list above, then re-run with -Yes.' 'Yellow'
Say ' Commit the resulting bundle-lock.json - that commit IS the review.' 'Yellow'
exit 2
}
# ConvertTo-Json defaults to a depth of 2, which silently flattens the per-file
# entries into "System.Collections.Hashtable" strings and produces a lock that
# verifies against nothing.
$new | ConvertTo-Json -Depth 8 | Set-Content -Path $LockPath -Encoding UTF8
Say ''
Say " Written: $LockPath" 'Green'
Say ' Commit it.' 'Green'

View File

@@ -0,0 +1,865 @@
<#
.SYNOPSIS
Day-to-day control of a ShopDB-Flask installation, on the server itself.
.DESCRIPTION
Installed alongside the application so an operator never has to open IIS
Manager, hunt for a log, or remember an appcmd incantation.
Run it with no arguments for a menu, or pass a command directly:
.\shopdb-admin.ps1 status
.\shopdb-admin.ps1 restart
.\shopdb-admin.ps1 backup D:\backups
Everything here is safe to run at any time EXCEPT backup/restore, which are
called out explicitly.
.NOTES
Written for stock Windows PowerShell 5.1. No modules to install.
#>
[CmdletBinding()]
param(
[ValidateSet('menu','status','start','stop','restart','logs','open','backup',
'check','sessions','plugins','add-plugin','verify','uninstall')]
[string] $Command = 'menu',
[string] $Path = '',
# Machine-readable output for 'check'. The people running this are expected to
# ask an AI assistant for help, and pasting a screenshot of a console into a
# chat window loses most of what matters. One structured blob they can paste
# gives the assistant real state to reason about instead of guesses.
[switch] $Json,
[string] $AppRoot = 'C:\shopdb-flask',
[string] $SiteName = 'shopdb-flask',
[string] $AppPool = 'shopdbflask',
[int] $SitePort = 8090
)
$ErrorActionPreference = 'Continue'
$AppCmd = Join-Path $env:windir 'System32\inetsrv\appcmd.exe'
# --- bitness ---------------------------------------------------------------
# IIS's management COM objects are 64-BIT ONLY. Under the 32-bit PowerShell,
# Import-Module WebAdministration SUCCEEDS but Get-Website then fails with
# Retrieving the COM class factory ... REGDB_E_CLASSNOTREG
# which this script caught and reported as "cannot read IIS" - indistinguishable
# from IIS being absent.
#
# A 32-bit launcher is easy to end up with: Inno Setup is a 32-bit process, so
# anything it starts gets SysWOW64 powershell through WOW64 redirection. Rather
# than fix every launcher, relaunch under the native PowerShell. 'Sysnative' is
# the alias that lets a 32-bit process reach the real System32, and it exists
# ONLY for 32-bit processes - hence the guard.
# Every relaunch below must carry the ORIGINAL arguments through. They used to be
# dropped, so a console started from the Start Menu with -AppRoot D:\shopdb
# relaunched itself with the C:\shopdb-flask default and reported a healthy
# install as missing - and the operator had done nothing wrong.
function Get-ForwardedArgs {
$forward = @('-NoProfile', '-ExecutionPolicy', 'Bypass',
'-File', ('"' + $PSCommandPath + '"'), $Command)
if ($Path) { $forward += @('-Path', ('"' + $Path + '"')) }
if ($AppRoot) { $forward += @('-AppRoot', ('"' + $AppRoot + '"')) }
if ($SiteName) { $forward += @('-SiteName', ('"' + $SiteName + '"')) }
if ($AppPool) { $forward += @('-AppPool', ('"' + $AppPool + '"')) }
if ($SitePort) { $forward += @('-SitePort', $SitePort) }
if ($Json) { $forward += '-Json' }
return $forward
}
if ([Environment]::Is64BitOperatingSystem -and -not [Environment]::Is64BitProcess) {
$native = Join-Path $env:windir 'Sysnative\WindowsPowerShell\v1.0\powershell.exe'
if (Test-Path $native) {
Start-Process -FilePath $native -ArgumentList (Get-ForwardedArgs) -Wait -NoNewWindow
return
}
}
# --- elevation -------------------------------------------------------------
# Reading IIS state needs Administrator: the WebAdministration module and the
# IIS: drive both fail without it. Unelevated, this tool used to report
# "IIS not available / application pool: not installed", which reads as "your
# install is broken" when it actually means "I cannot see it". Relaunch elevated
# instead of reporting a false state.
$identity = [Security.Principal.WindowsIdentity]::GetCurrent()
$principal = New-Object Security.Principal.WindowsPrincipal($identity)
# 'open' just launches a browser. Prompting for administrator to do that trains
# people to click through UAC, and the Start Menu shortcut uses this command.
if ($Command -ne 'open' -and
-not $principal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)) {
Write-Host ''
Write-Host ' Administrator rights are needed to read IIS state.' -ForegroundColor Yellow
Write-Host ' Re-launching elevated - approve the prompt.' -ForegroundColor Yellow
$argList = @('-NoExit') + (Get-ForwardedArgs)
try {
Start-Process -FilePath 'powershell.exe' -ArgumentList $argList -Verb RunAs | Out-Null
} catch {
Write-Host ''
Write-Host ' Elevation was declined.' -ForegroundColor Red
Write-Host ' Right-click the shortcut and choose "Run as administrator".' -ForegroundColor Red
Read-Host ' Press Enter to close' | Out-Null
}
return
}
# --- brand header ----------------------------------------------------------
# Deliberately typographic, NOT ASCII art. The GE monogram is fine cursive
# linework; rendered as block characters at console resolution it reads as noise,
# which looks worse than no mark at all. A clean rule and correct wordmark says
# "considered"; mushy art says the opposite.
#
# Box-drawing characters used here are all in code page 437 (the Windows console
# default), and this file is saved UTF-8 WITH BOM so PowerShell 5.1 reads them
# correctly rather than assuming the ANSI code page.
function Show-Banner {
$rule = ([string][char]0x2500) * 58
Write-Host ''
Write-Host ' GE AEROSPACE' -ForegroundColor Blue
Write-Host (' ' + $rule) -ForegroundColor DarkGray
Write-Host ' ShopDB-Flask' -ForegroundColor White
Write-Host ' Asset management for the shop floor' -ForegroundColor DarkGray
Write-Host ''
}
function Say { param($m, $c = 'Gray') Write-Host $m -ForegroundColor $c }
function Head {
param($m)
Write-Host ''
Write-Host (' ' + $m) -ForegroundColor Cyan
Write-Host (' ' + (([string][char]0x2500) * $m.Length)) -ForegroundColor DarkGray
}
function Get-EnvValue {
param([string] $Key)
$envFile = Join-Path $AppRoot '.env'
if (-not (Test-Path $envFile)) { return '' }
$line = Get-Content $envFile | Where-Object { $_ -like "$Key=*" } | Select-Object -First 1
if ($line) { return $line.Substring($Key.Length + 1) }
return ''
}
function Get-Deployment {
<#
Which way was this installed?
Method A: its own IIS site on $SitePort.
Method B: an IIS Application under an existing site, at /<alias>, reached
on that site's port. MOUNT_PATH in .env is what distinguishes
them - it is the same value wsgi.py uses to mount the app.
Without this the console looked for a SITE that method B never creates and
reported "web site: not installed" on a perfectly healthy server, then
probed the wrong port and said it was not responding.
#>
$mount = (Get-EnvValue 'MOUNT_PATH').Trim()
if (-not $mount) {
return @{ Subpath = $false; Alias = ''; BaseUrl = "http://localhost:$SitePort"; Port = $SitePort }
}
$alias = $mount.Trim('/')
$port = 80
$parent = 'Default Web Site'
try {
Import-Module WebAdministration -ErrorAction Stop
foreach ($site in (Get-Website)) {
$app = Get-WebApplication -Site $site.Name -Name $alias -ErrorAction SilentlyContinue
if ($app) {
$parent = $site.Name
$b = $site.bindings.Collection | Where-Object { $_.protocol -eq 'http' } | Select-Object -First 1
if ($b -and ($b.bindingInformation -match '^[^:]*:(\d+):')) { $port = [int]$Matches[1] }
break
}
}
} catch { }
$base = "http://localhost:$port/$alias"
if ($port -eq 80) { $base = "http://localhost/$alias" }
return @{ Subpath = $true; Alias = $alias; Parent = $parent; BaseUrl = $base; Port = $port }
}
function Get-DbParts {
# Pull host/port/name/user out of DATABASE_URL without printing the password.
$url = Get-EnvValue 'DATABASE_URL'
if ($url -match '://([^:]+):([^@]*)@([^:/]+):(\d+)/([^?]+)') {
# The password is PERCENT-ENCODED in DATABASE_URL (the installer applies
# [uri]::EscapeDataString). SQLAlchemy unescapes it; so must we. Without
# this, every command here fails to authenticate whenever the password
# contains a space, @, %, ! or /, and the console reports the database as
# unreachable on a server where the application is running perfectly.
return @{ User = [uri]::UnescapeDataString($Matches[1])
Pass = [uri]::UnescapeDataString($Matches[2])
Host = $Matches[3]; Port = $Matches[4]; Name = $Matches[5] }
}
return $null
}
function Find-MysqlClient {
$candidates = @(
'C:\MySQL84\bin\mysql.exe',
'C:\Program Files\MySQL\MySQL Server 8.4\bin\mysql.exe',
'C:\Program Files\MySQL\MySQL Server 8.0\bin\mysql.exe',
'C:\mysql56\bin\mysql.exe'
) + @(Get-ChildItem 'C:\Program Files\MySQL' -Filter mysql.exe -Recurse -EA SilentlyContinue |
Select-Object -ExpandProperty FullName)
foreach ($c in $candidates) { if ($c -and (Test-Path $c)) { return $c } }
return ''
}
# --------------------------------------------------------------------------
function Show-Status {
Head 'Status'
$deploy = Get-Deployment
$siteState = 'not installed'
$poolState = 'not installed'
try {
Import-Module WebAdministration -ErrorAction Stop
if ($deploy.Subpath) {
# Method B: there is no site of our own - we are an Application.
$app = Get-WebApplication -Site $deploy.Parent -Name $deploy.Alias -ErrorAction SilentlyContinue
if ($app) {
$parentSite = Get-Website -Name $deploy.Parent -ErrorAction SilentlyContinue
$siteState = ("/{0} under '{1}' ({2})" -f $deploy.Alias, $deploy.Parent,
$(if ($parentSite) { $parentSite.State } else { 'unknown' }))
}
} else {
$site = Get-Website -Name $SiteName -ErrorAction SilentlyContinue
if ($site) { $siteState = $site.State }
}
if (Test-Path "IIS:\AppPools\$AppPool") { $poolState = (Get-Item "IIS:\AppPools\$AppPool").State }
} catch {
# Should be unreachable now that the script self-elevates, but if IIS is
# genuinely absent say THAT, rather than implying ShopDB is missing.
# Name the real cause instead of implying ShopDB is missing.
if (-not (Get-Service W3SVC -ErrorAction SilentlyContinue)) {
$siteState = 'IIS is not installed on this server'
} else {
$siteState = 'could not read IIS - ' + $_.Exception.Message
}
$poolState = $siteState
}
$colour = if (($siteState -eq 'Started') -or ($siteState -like '*Started*')) { 'Green' } else { 'Yellow' }
Say (" published as : {0}" -f $siteState) $colour
Say (" application pool: {0}" -f $poolState) $colour
Say (" installed at : {0}" -f $(if (Test-Path $AppRoot) { $AppRoot } else { 'not found' }))
# Does it actually answer? A "Started" pool proves nothing. Under method B
# this must include the mount path, or it tests a URL that never existed.
$url = $deploy.BaseUrl + '/'
try {
$r = Invoke-WebRequest -Uri $url -UseBasicParsing -TimeoutSec 20
Say (" responding : yes (HTTP {0})" -f $r.StatusCode) 'Green'
} catch {
Say ' responding : NO' 'Red'
Say (" {0}" -f $_.Exception.Message) 'DarkGray'
}
$db = Get-DbParts
if ($db) {
Say (" database : {0} on {1}:{2} as {3}" -f $db.Name, $db.Host, $db.Port, $db.User)
$mysql = Find-MysqlClient
if ($mysql) {
$q = "SELECT COUNT(*) FROM information_schema.tables WHERE table_schema='$($db.Name)';"
$n = $q | & $mysql "-u$($db.User)" "-p$($db.Pass)" "-h$($db.Host)" "-P$($db.Port)" -N 2>$null
if ($LASTEXITCODE -eq 0) { Say (" tables : {0}" -f $n) 'Green' }
else { Say ' tables : could not connect' 'Red' }
}
} else { Say ' database : no .env found' 'Yellow' }
# Is anyone set up yet?
try {
$na = Invoke-WebRequest -Uri ($deploy.BaseUrl + '/api/setup/needs-admin') -UseBasicParsing -TimeoutSec 15
if ($na.Content -match '"needsadmin"\s*:\s*true') {
Say ' first run : NOT SET UP - open the site to create the first administrator' 'Yellow'
} else { Say ' first run : complete' 'Green' }
} catch { }
Say ''
if ($deploy.Subpath) {
$shown = "http://{0}/{1}/login" -f $env:COMPUTERNAME, $deploy.Alias
if ($deploy.Port -ne 80) { $shown = "http://{0}:{1}/{2}/login" -f $env:COMPUTERNAME, $deploy.Port, $deploy.Alias }
} else {
$shown = "http://{0}:{1}/login" -f $env:COMPUTERNAME, $SitePort
}
Say (" open with : {0}" -f $shown) 'White'
}
function Start-App {
Head 'Starting'
$deploy = Get-Deployment
& $AppCmd start apppool /apppool.name:$AppPool 2>&1 | ForEach-Object { Say " $_" }
if (-not $deploy.Subpath) {
& $AppCmd start site /site.name:$SiteName 2>&1 | ForEach-Object { Say " $_" }
} else {
# Method B shares the parent site - starting or stopping THAT would take
# every other application on this server with it.
Say (" (published under '{0}' - only the application pool is ours to control)" -f $deploy.Parent) 'DarkGray'
}
Say ' started' 'Green'
}
function Stop-App {
Head 'Stopping'
$deploy = Get-Deployment
Say ' ShopDB-Flask will be unavailable until you start it again.' 'Yellow'
if (-not $deploy.Subpath) {
& $AppCmd stop site /site.name:$SiteName 2>&1 | ForEach-Object { Say " $_" }
} else {
Say (" (leaving site '{0}' running - stopping it would take down everything else on it)" -f $deploy.Parent) 'DarkGray'
}
& $AppCmd stop apppool /apppool.name:$AppPool 2>&1 | ForEach-Object { Say " $_" }
Say ' stopped' 'Yellow'
}
function Restart-App {
Head 'Restarting'
# A recycle CANNOT start something that is stopped - it is a no-op on a
# stopped pool. Restarting after a failed upgrade, which is exactly when the
# pool is stopped, therefore did nothing and then reported "did not respond"
# in red, as though the application were broken. Start it first when needed.
Import-Module WebAdministration -ErrorAction SilentlyContinue
$started = $true
try {
if (Test-Path "IIS:\AppPools\$AppPool") {
$started = ((Get-Item "IIS:\AppPools\$AppPool").State -eq 'Started')
}
} catch { }
if (-not $started) {
Say ' the application pool is stopped; starting it' 'Yellow'
Start-WebAppPool -Name $AppPool -ErrorAction SilentlyContinue
Start-Sleep -Seconds 3
} else {
# Recycle rather than stop/start: it drains existing requests instead of
# cutting them off, and it is what a config change actually needs.
& $AppCmd recycle apppool /apppool.name:$AppPool 2>&1 | ForEach-Object { Say " $_" }
if ($LASTEXITCODE -ne 0) { Say " recycle reported exit $LASTEXITCODE" 'Yellow' }
Start-Sleep -Seconds 2
}
# A stopped SITE answers nothing however healthy the pool is.
try {
$siteNow = Get-Website -Name $SiteName -ErrorAction SilentlyContinue
if ($siteNow -and $siteNow.State -ne 'Started') {
Say ' the site is stopped; starting it' 'Yellow'
Start-Website -Name $SiteName -ErrorAction SilentlyContinue
Start-Sleep -Seconds 2
}
} catch { }
try {
$r = Invoke-WebRequest -Uri ((Get-Deployment).BaseUrl + '/') -UseBasicParsing -TimeoutSec 30
Say (" back up (HTTP {0})" -f $r.StatusCode) 'Green'
} catch { Say ' did not respond after restart - run: shopdb-admin.ps1 logs' 'Red' }
}
function Show-Logs {
Head 'Recent logs'
$appLog = Join-Path $AppRoot 'logs'
if (Test-Path $appLog) {
$newest = Get-ChildItem "$appLog\*.log" -EA SilentlyContinue |
Sort-Object LastWriteTime -Descending | Select-Object -First 1
if ($newest) {
Say (" application: {0}" -f $newest.FullName) 'White'
Get-Content $newest.FullName -Tail 20 | ForEach-Object { Say " $_" }
} else { Say ' no application log yet' }
}
$inst = 'C:\ProgramData\ShopDB-Flask\logs'
if (Test-Path $inst) {
$newest = Get-ChildItem "$inst\*.log" -EA SilentlyContinue |
Sort-Object LastWriteTime -Descending | Select-Object -First 1
if ($newest) {
Say ''
Say (" install: {0}" -f $newest.FullName) 'White'
Get-Content $newest.FullName -Tail 12 | ForEach-Object { Say " $_" }
}
}
}
function Open-Site {
# The installer records the address it actually published at. Prefer it: this
# command runs unelevated, and .env is ACL'd, so working the address out from
# MOUNT_PATH and IIS may not be possible from here.
$recorded = Join-Path $AppRoot '.installed-url'
if (Test-Path $recorded) {
$u = (Get-Content $recorded -TotalCount 1).Trim()
if ($u) { Start-Process $u; Say " opened $u" 'Green'; return }
}
$deploy = Get-Deployment
if ($deploy.Subpath) {
if ($deploy.Port -eq 80) { $u = "http://{0}/{1}/login" -f $env:COMPUTERNAME, $deploy.Alias }
else { $u = "http://{0}:{1}/{2}/login" -f $env:COMPUTERNAME, $deploy.Port, $deploy.Alias }
} else {
$u = "http://{0}:{1}/login" -f $env:COMPUTERNAME, $SitePort
}
Start-Process $u
Say " opened $u" 'Green'
}
function Backup-Db {
param([string] $Dest)
Head 'Database backup'
$db = Get-DbParts
if (-not $db) { Say ' no .env found - cannot determine the database' 'Red'; return }
$usingDefault = -not $Dest
if (-not $Dest) { $Dest = 'C:\ProgramData\ShopDB-Flask\backups' }
if (-not (Test-Path $Dest)) { New-Item -ItemType Directory -Path $Dest -Force | Out-Null }
# A dump holds every row, including the users table and its password hashes.
# A directory created under ProgramData INHERITS Users:RX, so those hashes
# were readable by every authenticated user on the server whenever this
# command created the directory rather than the installer.
#
# Re-applied on every backup, not only on creation, because this may be
# repairing a directory made by an earlier version.
#
# Only for the default location. A path the operator named is theirs, and
# silently rewriting its ACL is not this command's business - say so instead.
if ($usingDefault) {
& icacls.exe $Dest '/inheritance:r' `
'/grant' 'BUILTIN\Administrators:(OI)(CI)(F)' `
'/grant' 'NT AUTHORITY\SYSTEM:(OI)(CI)(F)' 2>&1 | Out-Null
} else {
Say ' note: this dump contains password hashes - check who can read that directory' 'Yellow'
}
$mysql = Find-MysqlClient
if (-not $mysql) { Say ' mysql client not found' 'Red'; return }
$dump = Join-Path (Split-Path $mysql -Parent) 'mysqldump.exe'
if (-not (Test-Path $dump)) { Say ' mysqldump not found' 'Red'; return }
$file = Join-Path $Dest ("shopdb_flask-{0}.sql" -f (Get-Date -Format 'yyyyMMdd-HHmmss'))
Say (" writing {0}" -f $file)
# Redirect rather than --result-file: keeps it working on 5.6 and 8.0 alike.
# Do NOT pipe mysqldump through the PowerShell pipeline into Out-File.
# PowerShell 5.1 writes a UTF-8 BOM and re-encodes the stream through the
# console code page, producing a dump MySQL refuses to load and mangling any
# non-ASCII data - discovered only when the backup is finally needed.
# Redirect the process's stdout straight to the file instead.
$args = @("-u$($db.User)", "-p$($db.Pass)", "-h$($db.Host)", "-P$($db.Port)",
'--single-transaction', '--routines', '--triggers', $db.Name)
$quoted = $args | ForEach-Object {
if ($_ -match '\s' -and $_ -notmatch '^".*"$') { '"' + $_ + '"' } else { $_ }
}
$err = [System.IO.Path]::GetTempFileName()
try {
$proc = Start-Process -FilePath $dump -ArgumentList $quoted -Wait -PassThru -NoNewWindow `
-RedirectStandardOutput $file -RedirectStandardError $err
if ($proc.ExitCode -ne 0) {
Say (" mysqldump failed (exit {0})" -f $proc.ExitCode) 'Red'
Get-Content $err -Tail 3 -EA SilentlyContinue | ForEach-Object { Say (" " + $_) 'DarkGray' }
return
}
} finally { Remove-Item $err -Force -ErrorAction SilentlyContinue }
# Verify it is complete rather than merely present.
$tail = @(Get-Content $file -Tail 5 -ErrorAction SilentlyContinue)
if ((Test-Path $file) -and ((Get-Item $file).Length -gt 1024) -and ($tail -match 'Dump completed')) {
Say (" done - {0:N1} MB, verified complete" -f ((Get-Item $file).Length / 1MB)) 'Green'
# The dump is NOT everything. Uploaded branding, map blueprints and
# generated files live in instance\ on disk, not in the database, so a
# restore from the .sql alone comes back with no floor map. Saying "all of
# your asset data" was true and misleading at the same time.
$instance = Join-Path $AppRoot 'instance'
if (Test-Path $instance) {
$zip = [System.IO.Path]::ChangeExtension($file, $null) + 'instance.zip'
try {
Add-Type -AssemblyName System.IO.Compression.FileSystem -ErrorAction SilentlyContinue
if (Test-Path $zip) { Remove-Item $zip -Force }
[System.IO.Compression.ZipFile]::CreateFromDirectory($instance, $zip)
Say (" uploaded files: {0:N1} MB -> {1}" -f ((Get-Item $zip).Length / 1MB), (Split-Path $zip -Leaf)) 'Green'
} catch {
Say (" could not archive instance\: {0}" -f $_.Exception.Message) 'Yellow'
Say ' copy it by hand - it holds branding and floor-map images' 'Yellow'
}
}
Say ''
Say ' Store BOTH files off this server. The .sql holds the records; the' 'Yellow'
Say ' .zip holds uploaded branding and floor-map images, which the' 'Yellow'
Say ' database does not. A restore needs both.' 'Yellow'
Say ''
Say ' Contains user password hashes - treat it as sensitive.' 'Yellow'
} else {
Say ' backup is empty or truncated - do NOT rely on it' 'Red'
Remove-Item $file -Force -ErrorAction SilentlyContinue
}
}
function Get-CheckState {
<#
Everything an outside reader needs to reason about this server, gathered
without changing anything. Secrets are NEVER included: the database password
lives in DATABASE_URL and this reports the host, port, name and user only.
#>
$deploy = Get-Deployment
$db = Get-DbParts
$state = [ordered]@{
collected = (Get-Date).ToUniversalTime().ToString('yyyy-MM-ddTHH:mm:ssZ')
computername = $env:COMPUTERNAME
approot = $AppRoot
installed = (Test-Path (Join-Path $AppRoot 'shopdb\__init__.py'))
version = ''
publishedas = if ($deploy.Subpath) { "subpath /$($deploy.Alias) under '$($deploy.Parent)'" } else { "own site '$SiteName' on port $SitePort" }
baseurl = $deploy.BaseUrl
apppool = 'unknown'
poolstate = 'unknown'
sitestate = 'unknown'
responding = $false
httpstatus = 0
database = $null
pythonversion = ''
plugins = @()
sbomcomponents = 0
errors = @()
}
foreach ($pair in @(@('version', '.installed-version'))) {
$f = Join-Path $AppRoot $pair[1]
if (Test-Path $f) { $state[$pair[0]] = (Get-Content $f -TotalCount 1).Trim() }
}
$state.apppool = $AppPool
try {
Import-Module WebAdministration -ErrorAction Stop
if (Test-Path "IIS:\AppPools\$AppPool") { $state.poolstate = (Get-Item "IIS:\AppPools\$AppPool").State.ToString() }
if ($deploy.Subpath) {
$app = Get-WebApplication -Site $deploy.Parent -Name $deploy.Alias -ErrorAction SilentlyContinue
$state.sitestate = if ($app) { 'application present' } else { 'application MISSING' }
} else {
$s = Get-Website -Name $SiteName -ErrorAction SilentlyContinue
$state.sitestate = if ($s) { $s.State.ToString() } else { 'site MISSING' }
}
} catch { $state.errors += "IIS: $($_.Exception.Message)" }
try {
$r = Invoke-WebRequest -Uri ($deploy.BaseUrl + '/') -UseBasicParsing -TimeoutSec 20
$state.responding = $true
$state.httpstatus = [int] $r.StatusCode
} catch { $state.errors += "HTTP: $($_.Exception.Message)" }
if ($db) {
$state.database = [ordered]@{ host = $db.Host; port = $db.Port; name = $db.Name; user = $db.User; reachable = $false }
$mysql = Find-MysqlClient
if ($mysql) {
$q = "SELECT COUNT(*) FROM information_schema.tables WHERE table_schema='$($db.Name)';"
$n = $q | & $mysql "-u$($db.User)" "-p$($db.Pass)" "-h$($db.Host)" "-P$($db.Port)" -N 2>$null
if ($LASTEXITCODE -eq 0) { $state.database.reachable = $true; $state.database.tables = [int] $n }
} else { $state.errors += 'no mysql client found, database not probed' }
} else { $state.errors += 'no .env found' }
$py = Join-Path $AppRoot 'venv\Scripts\python.exe'
if (Test-Path $py) { $state.pythonversion = (& $py -c "import sys; print('%d.%d.%d' % sys.version_info[:3])" 2>$null | Select-Object -First 1) }
$dir = Join-Path $AppRoot 'plugins'
if (Test-Path $dir) {
$state.plugins = @(Get-ChildItem $dir -Directory -EA SilentlyContinue |
Where-Object { Test-Path (Join-Path $_.FullName 'manifest.json') } |
Select-Object -ExpandProperty Name)
}
$sbom = Join-Path $AppRoot 'sbom.cdx.json'
if (Test-Path $sbom) {
try { $state.sbomcomponents = (Get-Content $sbom -Raw | ConvertFrom-Json).components.Count } catch { }
}
return $state
}
function Invoke-Check {
if ($Json) {
# ONLY the JSON goes to stdout, so it can be redirected to a file or piped
# without a banner in the middle of the document.
Get-CheckState | ConvertTo-Json -Depth 6
return
}
Head 'Health check'
$flask = Join-Path $AppRoot 'venv\Scripts\flask.exe'
if (-not (Test-Path $flask)) { Say ' application not installed' 'Red'; return }
Push-Location $AppRoot
$env:FLASK_APP = 'shopdb'
try { & $flask db-utils preflight 2>&1 | ForEach-Object { Say " $_" } }
finally { Pop-Location }
Say ''
Say ' For help from an AI assistant, paste the output of:' 'DarkGray'
Say ' shopdb-admin.ps1 check -Json' 'White'
}
function Show-Sessions {
Head 'Worker processes'
$w = Get-CimInstance Win32_Process -Filter "Name='w3wp.exe'" -EA SilentlyContinue
if (-not $w) { Say ' no IIS worker running (the site starts one on first request)' }
else {
foreach ($p in $w) {
Say (" pid {0} {1:N0} MB started {2}" -f $p.ProcessId,
($p.WorkingSetSize/1MB), $p.CreationDate)
}
}
$py = Get-CimInstance Win32_Process -Filter "Name='python.exe'" -EA SilentlyContinue |
Where-Object { $_.CommandLine -like "*$AppRoot*" }
if ($py) { foreach ($p in $py) { Say (" python pid {0} {1:N0} MB" -f $p.ProcessId, ($p.WorkingSetSize/1MB)) } }
}
# Set by every Invoke-Flask call. Callers MUST test this rather than
# $LASTEXITCODE: when flask.exe is missing no native command runs at all, so
# $LASTEXITCODE keeps whatever value it had from something earlier - which reads
# as success and made a command that did nothing report that it had worked.
$script:LastFlaskExit = 0
function Invoke-Flask {
param([string[]] $Arguments)
$flask = Join-Path $AppRoot 'venv\Scripts\flask.exe'
if (-not (Test-Path $flask)) {
Say ' application not installed' 'Red'
$script:LastFlaskExit = 127
return $null
}
Push-Location $AppRoot
$env:FLASK_APP = 'shopdb'
# The app logs plugin startup to STDERR even on success; with EAP=Stop that
# becomes a terminating error and a healthy command looks like a failure.
$prev = $ErrorActionPreference
$ErrorActionPreference = 'Continue'
try {
& $flask @Arguments 2>&1
$script:LastFlaskExit = $LASTEXITCODE
}
finally { $ErrorActionPreference = $prev; Pop-Location }
}
function Show-Plugins {
Head 'Plugins'
Invoke-Flask @('plugin','list') | ForEach-Object { Say " $_" }
# What is on disk but NOT installed can still be added here. What is absent
# from disk cannot - see the note below.
$dir = Join-Path $AppRoot 'plugins'
if (Test-Path $dir) {
$onDisk = (Get-ChildItem $dir -Directory -EA SilentlyContinue |
Where-Object { Test-Path (Join-Path $_.FullName 'manifest.json') } |
Select-Object -ExpandProperty Name) -join ', '
Say ''
Say (" shipped in this build : {0}" -f $onDisk) 'White'
}
Say ''
Say ' Add one that is shipped : shopdb-admin.ps1 add-plugin -Path <name>' 'White'
Say ''
Say ' A plugin NOT listed above is not on this server at all. This build was' 'DarkGray'
Say ' made for your site''s chosen plugin set, so its code was never shipped.' 'DarkGray'
Say ' Adding one means a new installer built from an updated site profile.' 'DarkGray'
}
function Add-Plugin {
param([string] $Name)
Head 'Add a plugin'
if (-not $Name) { Say ' usage: shopdb-admin.ps1 add-plugin -Path <plugin-name>' 'Yellow'; return }
$dir = Join-Path $AppRoot ('plugins\' + $Name)
if (-not (Test-Path (Join-Path $dir 'manifest.json'))) {
Say (" '{0}' is not present on this server." -f $Name) 'Red'
Say ''
Say ' This build ships only the plugins your site chose. Adding a new one' 'Yellow'
Say ' requires a new installer built from an updated site profile - the' 'Yellow'
Say ' code is not here to install.' 'Yellow'
Say ''
Say ' Run "shopdb-admin.ps1 plugins" to see what IS available.' 'White'
return
}
# apply-profile, not plugin install. Five plugins ship default_enabled=false,
# so `install` alone left them installed-but-disabled: the command printed a
# green success line and the feature did not appear anywhere in the UI.
# apply-profile installs AND enables, and pulls in the dependency closure.
Say (" adding {0}..." -f $Name)
$profilePath = Join-Path $AppRoot 'site-profile.json'
$applied = $false
if (Test-Path $profilePath) {
try {
$profile = Get-Content $profilePath -Raw | ConvertFrom-Json
$wanted = @($profile.plugins)
if ($wanted -notcontains $Name) { $wanted += $Name }
$profile.plugins = $wanted
$profile | ConvertTo-Json -Depth 6 | Set-Content -Path $profilePath -Encoding UTF8
Invoke-Flask @('plugin','apply-profile',$profilePath) | ForEach-Object { Say " $_" }
$applied = ($script:LastFlaskExit -eq 0)
} catch { Say (" could not update site-profile.json: {0}" -f $_.Exception.Message) 'Yellow' }
}
if (-not $applied) {
# No profile on disk, or apply-profile failed: fall back, but enable
# explicitly so the outcome is the same either way.
Invoke-Flask @('plugin','install',$Name) | ForEach-Object { Say " $_" }
$installed = ($script:LastFlaskExit -eq 0)
Invoke-Flask @('plugin','enable',$Name) | ForEach-Object { Say " $_" }
$applied = $installed -and ($script:LastFlaskExit -eq 0)
}
if (-not $applied) {
# Do NOT print the green line on a failure. It used to be unconditional,
# so a failed add reported success and the operator went looking for a
# feature that was never enabled.
Say (" {0} was NOT added - see the output above" -f $Name) 'Red'
return
}
Say ' applying its database migrations...'
Invoke-Flask @('plugin','upgrade-all') | ForEach-Object { Say " $_" }
if ($script:LastFlaskExit -ne 0) {
Say ' migrations FAILED - the feature is installed but its tables are not' 'Red'
Say ' do not use it until this is resolved; restore a backup if needed' 'Red'
return
}
Say ' restarting so its routes register...'
Restart-App
Say (" {0} added" -f $Name) 'Green'
}
function Invoke-Verify {
<#
Prove the installed dependencies are still the ones that shipped.
Two independent records, both written at install time:
requirements.txt a sha256 for every wheel, checked by pip on install
bundle-lock.json the exact third-party payload of the bundle
This re-checks what can still be checked on a live server. It reports; it
changes nothing.
#>
Head 'Verify'
$lock = Join-Path $AppRoot 'bundle-lock.json'
if (Test-Path $lock) {
try {
$l = Get-Content $lock -Raw | ConvertFrom-Json
Say (" installed from : {0} bundle ({1}, {2})" -f $l.generated, $l.pythontag, $l.platform)
} catch { Say ' bundle-lock.json is present but unreadable' 'Yellow' }
} else {
Say ' no bundle-lock.json recorded (installed before payload locking, or by hand)' 'Yellow'
}
# The SBOM travels with the application, because a server on a vaulted
# network cannot be scanned from anywhere else. When a CVE lands, this is
# what answers "is that component here, and at what version" without needing
# the build box, the internet, or anyone's memory.
$sbom = Join-Path $AppRoot 'sbom.cdx.json'
if (Test-Path $sbom) {
try {
$b = Get-Content $sbom -Raw | ConvertFrom-Json
$shipped = @($b.components | Where-Object { $_.scope -eq 'required' }).Count
Say (" components : {0} ({1} shipped), CycloneDX {2}" -f `
$b.components.Count, $shipped, $b.specVersion)
Say (" bill of materials: {0}" -f $sbom) 'DarkGray'
Say ' search it with : shopdb-admin.ps1 verify -Path <name>' 'DarkGray'
} catch { Say ' sbom.cdx.json is present but unreadable' 'Yellow' }
# A named component turns this into the actual question being asked.
# Guarded on $b: an unreadable SBOM leaves it unset, and querying it then
# would report 'none', which reads as "you are not affected".
if ($Path -and $b) {
Say ''
Say (" matches for '{0}':" -f $Path) 'Cyan'
$hits = @($b.components | Where-Object { $_.name -like ('*' + $Path + '*') })
if (-not $hits) { Say ' none - this server does not carry it' 'Green' }
foreach ($h in $hits) {
$tag = if ($h.scope -eq 'required') { 'SHIPPED' } else { 'build only' }
Say (" {0,-40} {1,-14} {2}" -f $h.name, $h.version, $tag) `
$(if ($h.scope -eq 'required') { 'Yellow' } else { 'DarkGray' })
}
}
} else {
Say ' no SBOM recorded (installed before SBOMs shipped, or by hand)' 'Yellow'
}
# pip's own audit. It re-reads the metadata of what is actually installed and
# reports anything missing or version-inconsistent, which is the part that
# can still drift after install - a hand-run `pip install` on the server.
$py = Join-Path $AppRoot 'venv\Scripts\python.exe'
if (-not (Test-Path $py)) { Say ' no venv - application not installed' 'Red'; return }
Say ''
Say ' checking installed packages against requirements.txt...'
$req = Join-Path $AppRoot 'requirements.txt'
if (-not (Test-Path $req)) { Say ' requirements.txt is missing from the install' 'Red'; return }
$wanted = @{}
foreach ($line in (Get-Content $req)) {
if ($line -match '^([A-Za-z0-9._-]+)==([^ \\]+)') { $wanted[$Matches[1].ToLower().Replace('_','-')] = $Matches[2] }
}
$frozen = @{}
foreach ($line in (& $py -m pip freeze 2>$null)) {
if ($line -match '^([A-Za-z0-9._-]+)==(.+)$') { $frozen[$Matches[1].ToLower().Replace('_','-')] = $Matches[2] }
}
$bad = 0
foreach ($name in ($wanted.Keys | Sort-Object)) {
if (-not $frozen.ContainsKey($name)) {
Say (" MISSING {0} {1}" -f $name, $wanted[$name]) 'Red'; $bad++
} elseif ($frozen[$name] -ne $wanted[$name]) {
Say (" DIFFERS {0} {1} installed, {2} expected" -f $name, $frozen[$name], $wanted[$name]) 'Red'; $bad++
}
}
if ($bad -eq 0) {
Say (" all {0} packages match" -f $wanted.Count) 'Green'
} else {
Say ''
Say (" {0} package(s) do not match what shipped." -f $bad) 'Red'
Say ' Something was installed or upgraded on this server by hand. Re-run the' 'Yellow'
Say ' installer to put the shipped set back.' 'Yellow'
}
& $py -m pip check 2>&1 | ForEach-Object { Say " $_" 'DarkGray' }
}
function Show-Uninstall {
Head 'Uninstall'
Say ' Use Settings > Apps > ShopDB-Flask, or Add/Remove Programs.'
Say ''
Say ' That removes the web site, application pool, firewall rule and files.'
Say ' It does NOT drop the database and does NOT uninstall MySQL.' 'Yellow'
Say ' Take a backup first: shopdb-admin.ps1 backup' 'Yellow'
}
function Show-Menu {
$first = $true
while ($true) {
if ($first) { Show-Banner; $first = $false }
Show-Status
Write-Host ''
Write-Host ' 1 Restart the application 6 Back up the database' -ForegroundColor White
Write-Host ' 2 Stop the application 7 Worker processes' -ForegroundColor White
Write-Host ' 3 Start the application 8 Open in browser' -ForegroundColor White
Write-Host ' 4 View recent logs 9 Plugins' -ForegroundColor White
Write-Host ' 5 Health check V Verify this install' -ForegroundColor White
Write-Host ' 0 Exit' -ForegroundColor White
Write-Host ''
$c = Read-Host ' Choose'
switch ($c) {
'1' { Restart-App } '2' { Stop-App } '3' { Start-App }
'4' { Show-Logs } '5' { Invoke-Check } '6' { Backup-Db $Path }
'7' { Show-Sessions } '8' { Open-Site }
'v' { Invoke-Verify } 'V' { Invoke-Verify }
'9' { Show-Plugins
$add = Read-Host ' Name of a shipped plugin to add (Enter to skip)'
if ($add) { Add-Plugin $add } }
'0' { return }
default { Say ' not a choice' 'Yellow' }
}
Write-Host ''
Read-Host ' Press Enter to continue' | Out-Null
Clear-Host
Show-Banner
}
}
switch ($Command) {
'status' { Show-Banner; Show-Status }
'start' { Start-App }
'stop' { Stop-App }
'restart' { Restart-App }
'logs' { Show-Logs }
'open' { Open-Site }
'backup' { Backup-Db $Path }
'check' { Invoke-Check }
'sessions' { Show-Sessions }
'plugins' { Show-Plugins }
'add-plugin'{ Add-Plugin $Path }
'verify' { Invoke-Verify }
'uninstall' { Show-Uninstall }
default { Show-Menu }
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,562 @@
<#
.SYNOPSIS
ShopDB-Flask installer - Stage 1: read-only preflight.
.DESCRIPTION
Discovers everything the installer needs to know about this box and reports
it. Makes NO changes: no installs, no config edits, no service restarts.
Safe to run on a production server.
Written for stock Windows PowerShell 5.1 (Windows Server ships it). No
pwsh-only syntax, no external modules, no network access.
.PARAMETER SitePort
The port the ShopDB site will listen on. Default 8090 (the runbook's example;
the classic ASP site keeps 8080).
.PARAMETER AppRoot
Intended install directory. Default C:\shopdb-flask.
.PARAMETER Json
Emit machine-readable JSON instead of the human report. Later installer
stages consume this.
.EXAMPLE
powershell -ExecutionPolicy Bypass -File .\shopdb-preflight.ps1
powershell -ExecutionPolicy Bypass -File .\shopdb-preflight.ps1 -Json > preflight.json
.NOTES
Exit 0 = no blocking problems. Exit 1 = at least one FAIL.
#>
[CmdletBinding()]
param(
[int] $SitePort = 8090,
[string] $AppRoot = 'C:\shopdb-flask',
# Needed so the port check can tell OUR site apart from a stranger's.
[string] $SiteName = 'shopdb-flask',
[switch] $Json,
# Machine-readable output for a GUI caller: one record per line,
# STATUS|AREA|CHECK|DETAIL|FIX
# The console rendering below aligns columns with padding spaces, which only
# works in a fixed-width font at console width. A GUI must do its own layout,
# so give it DATA and let it decide - do not make it parse a formatted table.
# (-Json exists too, but Inno's Pascal Script has no JSON parser.)
[switch] $Delimited
)
Set-StrictMode -Version 2.0
$ErrorActionPreference = 'Stop'
# --- result collection -------------------------------------------------------
# Every check appends one record. Status is PASS / WARN / FAIL / INFO.
$script:Results = New-Object System.Collections.ArrayList
$script:IisPresent = $false
function Add-Result {
param(
[string] $Area,
[string] $Check,
[ValidateSet('PASS','WARN','FAIL','INFO','SKIP')] [string] $Status,
[string] $Detail,
[string] $Fix = ''
)
$null = $script:Results.Add([PSCustomObject]@{
Area = $Area
Check = $Check
Status = $Status
Detail = $Detail
Fix = $Fix
})
}
# Wrap a check so one failure cannot abort the whole run. On an unfamiliar box
# an unexpected exception is itself a finding, not a crash.
function Invoke-Check {
param([string] $Area, [string] $Check, [scriptblock] $Body)
try { & $Body }
catch {
Add-Result $Area $Check 'WARN' "check could not run: $($_.Exception.Message)" `
'Report this output; the installer needs to handle this box shape.'
}
}
# =============================================================================
# 1. Operator context
# =============================================================================
Invoke-Check 'System' 'Elevation' {
$id = [Security.Principal.WindowsIdentity]::GetCurrent()
$adm = (New-Object Security.Principal.WindowsPrincipal($id)).IsInRole(
[Security.Principal.WindowsBuiltInRole]::Administrator)
if ($adm) { Add-Result 'System' 'Elevation' 'PASS' 'running as Administrator' }
else {
Add-Result 'System' 'Elevation' 'FAIL' 'not elevated' `
'Re-run PowerShell as Administrator. IIS and service changes require it.'
}
}
Invoke-Check 'System' 'Windows version' {
$os = Get-CimInstance Win32_OperatingSystem
$name = $os.Caption
$ver = $os.Version
# ProductType: 1 = workstation, 2 = domain controller, 3 = server
$isServer = ($os.ProductType -ne 1)
$detail = "$name (build $ver), $(if ($isServer) {'Server'} else {'Client'})"
$supported = $false
if ($isServer -and [version]$ver -ge [version]'10.0.17763') { $supported = $true } # 2019+
if (-not $isServer -and [version]$ver -ge [version]'10.0.19045') { $supported = $true } # Win10 22H2+
if ($supported) { Add-Result 'System' 'Windows version' 'PASS' $detail }
else {
Add-Result 'System' 'Windows version' 'FAIL' $detail `
'Supported: Windows Server 2019/2022+, or Windows 10 22H2 / 11 Pro+.'
}
# Client SKUs must be Pro/Enterprise/Education for IIS.
if (-not $isServer -and $name -match 'Home') {
Add-Result 'System' 'Windows edition' 'FAIL' 'Windows Home edition' `
'IIS is not available on Home editions. Pro or higher is required.'
}
}
Invoke-Check 'System' 'Architecture' {
if ([Environment]::Is64BitOperatingSystem) {
Add-Result 'System' 'Architecture' 'PASS' '64-bit'
} else {
Add-Result 'System' 'Architecture' 'FAIL' '32-bit' `
'The bundled Python and wheels are 64-bit (win_amd64) only.'
}
}
Invoke-Check 'System' 'PowerShell version' {
$v = $PSVersionTable.PSVersion
Add-Result 'System' 'PowerShell version' 'INFO' "$v"
if ($v.Major -lt 5) {
Add-Result 'System' 'PowerShell version' 'FAIL' "$v" `
'PowerShell 5.1 or later is required.'
}
}
# =============================================================================
# 2. Disk and ports
# =============================================================================
Invoke-Check 'Disk' 'Free space' {
$drive = (Split-Path -Qualifier $AppRoot)
$d = Get-CimInstance Win32_LogicalDisk -Filter "DeviceID='$drive'"
if ($null -eq $d) {
Add-Result 'Disk' 'Free space' 'FAIL' "drive $drive not found" `
"Choose an -AppRoot on an existing volume."
return
}
$freeGB = [math]::Round($d.FreeSpace / 1GB, 1)
$detail = "$freeGB GB free on $drive"
if ($freeGB -ge 5) { Add-Result 'Disk' 'Free space' 'PASS' $detail }
else { Add-Result 'Disk' 'Free space' 'FAIL' $detail 'At least 5 GB is required.' }
}
Invoke-Check 'Disk' 'AppRoot' {
if (Test-Path $AppRoot) {
$existing = @(Get-ChildItem $AppRoot -Force -ErrorAction SilentlyContinue)
if ($existing.Count -gt 0) {
$hasEnv = Test-Path (Join-Path $AppRoot '.env')
if ($hasEnv) {
# An existing install is the NORMAL state for an upgrade. Reporting
# it as a warning makes a routine update look like a problem.
$ver = ''
$vf = Join-Path $AppRoot '.installed-version'
if (Test-Path $vf) { $ver = ' version ' + (Get-Content $vf -TotalCount 1).Trim() }
Add-Result 'Disk' 'AppRoot' 'INFO' `
"existing ShopDB-Flask install found$ver - it will be upgraded in place, and your settings and database are kept"
} else {
Add-Result 'Disk' 'AppRoot' 'WARN' "$AppRoot exists and is not empty" `
'Confirm this directory is safe to install into.'
}
} else { Add-Result 'Disk' 'AppRoot' 'PASS' "$AppRoot exists and is empty" }
} else { Add-Result 'Disk' 'AppRoot' 'PASS' "$AppRoot does not exist yet" }
}
function Test-PortFree {
param([int] $Port)
# Get-NetTCPConnection is the reliable listener check on Server 2012R2+.
try {
$listening = @(Get-NetTCPConnection -State Listen -LocalPort $Port -ErrorAction SilentlyContinue)
return ($listening.Count -eq 0)
} catch {
# Fall back to a bind attempt if the cmdlet is unavailable.
try {
$l = New-Object System.Net.Sockets.TcpListener([System.Net.IPAddress]::Any, $Port)
$l.Start(); $l.Stop(); return $true
} catch { return $false }
}
}
Invoke-Check 'Network' 'Site port' {
if (Test-PortFree $SitePort) {
Add-Result 'Network' 'Site port' 'PASS' "TCP $SitePort is free"
} else {
$owner = ''
try {
$c = Get-NetTCPConnection -State Listen -LocalPort $SitePort -ErrorAction SilentlyContinue | Select-Object -First 1
if ($c) { $owner = " (pid $($c.OwningProcess): $((Get-Process -Id $c.OwningProcess -ErrorAction SilentlyContinue).ProcessName))" }
} catch { }
# Is the listener OUR OWN site? On a reinstall or upgrade the port is held
# by the very application being upgraded, and blocking on that makes the
# installer refuse to update anything it previously installed.
$ours = $false
try {
Import-Module WebAdministration -ErrorAction SilentlyContinue
$site = Get-Website -Name $SiteName -ErrorAction SilentlyContinue
if ($site) {
foreach ($b in $site.bindings.Collection) {
# ${} is required: "$SitePort:" parses as a DRIVE-qualified variable.
if ($b.bindingInformation -match ":${SitePort}:") { $ours = $true }
}
}
} catch { }
if ($ours) {
Add-Result 'Network' 'Site port' 'INFO' `
"TCP $SitePort is used by the existing $SiteName site - this will be upgraded in place"
} else {
# WARN, not FAIL. This check runs before the operator has reached the
# Address page, so it is testing the DEFAULT port, not necessarily
# the one they intend to use. Blocking here would refuse an install
# over a conflict the very next page lets them resolve.
Add-Result 'Network' 'Site port' 'WARN' "TCP $SitePort is in use$owner" `
"Pick a different port on the Address page later in this wizard, or stop whatever is holding it."
}
}
}
# =============================================================================
# 3. IIS
# =============================================================================
Invoke-Check 'IIS' 'Installed' {
$svc = Get-Service -Name W3SVC -ErrorAction SilentlyContinue
$script:IisPresent = ($null -ne $svc)
if ($null -eq $svc) {
Add-Result 'IIS' 'Installed' 'FAIL' 'W3SVC service not found' `
'Install IIS. Server: Install-WindowsFeature Web-Server -IncludeManagementTools. Client: enable Internet Information Services in Windows Features.'
return
}
Add-Result 'IIS' 'Installed' 'PASS' "W3SVC present, status $($svc.Status)"
if ($svc.Status -ne 'Running') {
Add-Result 'IIS' 'Running' 'WARN' "W3SVC is $($svc.Status)" 'Start-Service W3SVC'
}
}
Invoke-Check 'IIS' 'WebAdministration module' {
$m = Get-Module -ListAvailable -Name WebAdministration
if ($m) { Add-Result 'IIS' 'WebAdministration module' 'PASS' 'available' }
else {
Add-Result 'IIS' 'WebAdministration module' 'FAIL' 'not available' `
'Install the IIS management tools (Web-Mgmt-Console / IIS Management Scripts and Tools).'
}
}
Invoke-Check 'IIS' 'HttpPlatformHandler' {
# The handler registers itself as a global module. Check the module list.
$appcmd = Join-Path $env:windir 'system32\inetsrv\appcmd.exe'
if (-not (Test-Path $appcmd)) {
Add-Result 'IIS' 'HttpPlatformHandler' 'SKIP' 'appcmd.exe not present (IIS not installed)' `
'Re-run this preflight after installing IIS.'
return
}
$modules = & $appcmd list module 2>$null
if ($modules -match 'httpPlatformHandler') {
Add-Result 'IIS' 'HttpPlatformHandler' 'PASS' 'installed'
} else {
# NOT a blocker: the MSI is in the bundle and stage 4 installs it. This
# was a FAIL, which - once the preflight page started blocking on any
# failure - stopped the wizard dead over something the installer was
# about to do by itself, with no way forward but to go and install it by
# hand. Nothing the installer SUPPLIES may be a blocker.
Add-Result 'IIS' 'HttpPlatformHandler' 'INFO' 'not installed yet' `
'The installer installs it from the bundle. No action needed.'
}
}
Invoke-Check 'IIS' 'Locked config sections' {
# The authoritative source is applicationHost.config. `appcmd list config
# /section:X` prints the section CONTENTS, not its lock state, so grepping
# that output silently reports every section as unlocked.
$cfg = Join-Path $env:windir 'system32\inetsrv\config\applicationHost.config'
if (-not (Test-Path $cfg)) {
Add-Result 'IIS' 'Locked config sections' 'SKIP' 'applicationHost.config not found'
return
}
foreach ($name in @('handlers','httpPlatform')) {
$line = Select-String -Path $cfg -Pattern ('<section name="' + $name + '"') |
Select-Object -First 1
if ($null -eq $line) {
# httpPlatform is registered by the HttpPlatformHandler MSI. Before
# that, `appcmd unlock config /section:system.webServer/httpPlatform`
# fails with "Unknown config section".
Add-Result 'IIS' "Section $name" 'SKIP' 'section not registered yet' `
'Install HttpPlatformHandler FIRST; only then can this section be unlocked.'
} elseif ($line.Line -match 'overrideModeDefault="Deny"') {
Add-Result 'IIS' "Section $name" 'WARN' 'locked (overrideModeDefault="Deny")' `
"Installer must run: appcmd unlock config /section:system.webServer/$name (else IIS 500.19)"
} else {
Add-Result 'IIS' "Section $name" 'PASS' 'not locked'
}
}
}
Invoke-Check 'IIS' 'URL Rewrite module' {
$appcmd = Join-Path $env:windir 'system32\inetsrv\appcmd.exe'
if (-not (Test-Path $appcmd)) {
Add-Result 'IIS' 'URL Rewrite module' 'SKIP' 'IIS not installed; cannot check'
return
}
$modules = & $appcmd list module 2>$null
if ($modules -match 'RewriteModule') {
Add-Result 'IIS' 'URL Rewrite module' 'PASS' 'installed (the X-Forwarded-For rule can be enabled)'
} else {
# Not a FAIL: it is only needed for -ClientIpSource direct, and the
# installer carries the MSI and installs it itself. Worth reporting
# because without the rule IIS sends no X-Forwarded-For at all, so every
# client reads as 127.0.0.1 and the GE-Enforce IP allowlist, the
# visitor-location lookup and per-host login rate limiting go quiet.
Add-Result 'IIS' 'URL Rewrite module' 'INFO' 'not installed' `
'The installer installs it from the bundle when -ClientIpSource is direct. Behind a reverse proxy that already sets X-Forwarded-For, use -ClientIpSource proxy and leave it out.'
}
}
Invoke-Check 'IIS' 'Existing sites' {
if (-not $script:IisPresent) {
Add-Result 'IIS' 'Existing sites' 'SKIP' 'IIS not installed'
return
}
try {
Import-Module WebAdministration -ErrorAction Stop
$sites = @(Get-Website)
if ($sites.Count -eq 0) { Add-Result 'IIS' 'Existing sites' 'INFO' 'none' ; return }
$desc = ($sites | ForEach-Object {
$b = ($_.bindings.Collection | ForEach-Object { $_.bindingInformation }) -join ','
"$($_.Name) [$($_.State)] $b"
}) -join '; '
Add-Result 'IIS' 'Existing sites' 'INFO' $desc
# Adoption sites typically run the classic ASP shopdb here already.
if ($desc -match '8080') {
Add-Result 'IIS' 'Classic ASP site' 'INFO' 'a site is bound on 8080 (likely the classic ASP shopdb)' `
'Install ShopDB as a separate site on its own port; do not disturb this one.'
}
} catch {
Add-Result 'IIS' 'Existing sites' 'WARN' "could not enumerate: $($_.Exception.Message)" ''
}
}
# =============================================================================
# 4. MySQL (detect BEFORE offering bundled vs existing)
# =============================================================================
Invoke-Check 'MySQL' 'Service' {
$svcs = @(Get-Service -ErrorAction SilentlyContinue | Where-Object { $_.Name -match '^MySQL' -or $_.DisplayName -match 'MySQL' })
if ($svcs.Count -eq 0) {
Add-Result 'MySQL' 'Service' 'INFO' 'no MySQL service found' `
'Bundled MySQL 8.4 LTS is the appropriate choice on this box.'
return
}
foreach ($s in $svcs) {
Add-Result 'MySQL' 'Service' 'WARN' "$($s.Name) ($($s.DisplayName)) is $($s.Status)" `
'MySQL already present. Default to the EXISTING-server option; installing bundled MySQL will collide on port 3306.'
}
}
Invoke-Check 'MySQL' 'Port 3306' {
if (Test-PortFree 3306) {
Add-Result 'MySQL' 'Port 3306' 'INFO' 'nothing listening on 3306'
} else {
Add-Result 'MySQL' 'Port 3306' 'WARN' 'something is listening on 3306' `
'Bundled MySQL cannot use the default port. Use the existing server, or pick another port.'
}
}
Invoke-Check 'MySQL' 'Backup client' {
# mysqldump is what takes the mandatory pre-upgrade backup. Without it every
# upgrade skips the backup - and skips it AFTER the application pool has been
# stopped and the tree replaced, so the site is down and there is nothing to
# restore from. A site whose database is on another server typically has no
# MySQL client installed here at all, which is exactly the case that needs it.
$names = @('mysqldump.exe')
$found = ''
foreach ($root in @((Join-Path $PSScriptRoot 'mysqlclient'),
(Join-Path $AppRoot 'mysqlclient'),
'C:\MySQL84', 'C:\Program Files\MySQL', 'C:\mysql56\bin',
'C:\Program Files (x86)\MySQL')) {
if (-not (Test-Path $root)) { continue }
$hit = Get-ChildItem $root -Filter $names[0] -Recurse -ErrorAction SilentlyContinue |
Select-Object -First 1
if ($hit) { $found = $hit.FullName; break }
}
if ($found) {
Add-Result 'MySQL' 'Backup client' 'PASS' "mysqldump found ($found)"
} else {
Add-Result 'MySQL' 'Backup client' 'WARN' 'mysqldump not found on this server' `
'Needed for the automatic pre-upgrade backup and for "shopdb-admin.ps1 backup". A first install works without it; upgrades will not be protected. It is not on this server yet - the installer stages its own copy, so this normally resolves itself during installation.'
}
}
Invoke-Check 'MySQL' 'Version and config' {
# Find mysqld.exe via the service binary path; read the version and locate my.ini.
$svc = Get-CimInstance Win32_Service -ErrorAction SilentlyContinue |
Where-Object { $_.PathName -match 'mysqld' } | Select-Object -First 1
if ($null -eq $svc) { return }
$path = $svc.PathName
$exe = ''
if ($path -match '"([^"]+mysqld[^"]*)"') { $exe = $matches[1] }
elseif ($path -match '(\S+mysqld\S*)') { $exe = $matches[1] }
$ver = ''
if ($exe -and (Test-Path $exe)) {
try { $ver = (& $exe --version 2>$null | Out-String).Trim() } catch { }
}
if ($ver) { Add-Result 'MySQL' 'Version' 'INFO' $ver }
# my.ini path is passed as --defaults-file in the service command line.
$ini = ''
if ($path -match '--defaults-file="?([^"]+\.ini)"?') { $ini = $matches[1] }
if ($ini -and (Test-Path $ini)) {
Add-Result 'MySQL' 'Config file' 'INFO' $ini
# MySQL 5.6 needs three flags or `flask db upgrade` dies with error 1071.
$is56 = ($ver -match '\b5\.6\.')
if ($is56) {
$content = Get-Content $ini -Raw
$need = @('innodb_file_per_table','innodb_file_format','innodb_large_prefix')
$missing = @()
foreach ($k in $need) { if ($content -notmatch $k) { $missing += $k } }
if ($missing.Count -eq 0) {
# Present in the FILE is not the same as ACTIVE. MySQL must be
# restarted for them to take effect, and the app's own
# `flask db-utils preflight` queries the live server - trust that.
Add-Result 'MySQL' '5.6 index flags' 'WARN' 'all three present in my.ini' `
'Present in the file only. They take effect after a MySQL RESTART, which interrupts the classic ASP app. Confirm with SHOW VARIABLES or flask db-utils preflight.'
} else {
# WARN, not FAIL. This inspects the LOCAL MySQL, which may not be the
# database the operator is about to install against - a bundled 8.4,
# or a remote server. Blocking the wizard here refused an install
# over a server that had nothing to do with it. Stage 3 runs
# 'flask db-utils preflight' against the database actually chosen,
# which is the check that can genuinely block.
Add-Result 'MySQL' '5.6 index flags' 'WARN' ("missing: " + ($missing -join ', ')) `
"Add to [mysqld] in $ini and restart MySQL, or 'flask db upgrade' fails with error 1071. NOTE: restarting interrupts the classic ASP app."
}
}
}
}
# =============================================================================
# 5. Python (detect, but the installer uses its OWN bundled interpreter)
# =============================================================================
Invoke-Check 'Python' 'On PATH' {
$cmd = Get-Command python -ErrorAction SilentlyContinue
if ($null -eq $cmd) {
Add-Result 'Python' 'On PATH' 'INFO' 'no python on PATH' `
'Expected. The installer supplies its own interpreter.'
return
}
$v = ''
try { $v = (& $cmd.Source --version 2>&1 | Out-String).Trim() } catch { }
$detail = "$v at $($cmd.Source)"
# A per-user install under %LOCALAPPDATA% is unreadable by the IIS app-pool
# identity. That produces a 500 with an empty HttpPlatform log.
if ($cmd.Source -like "$env:LOCALAPPDATA*") {
Add-Result 'Python' 'On PATH' 'WARN' "$detail (PER-USER install)" `
'The IIS app-pool identity cannot read %LOCALAPPDATA%. The installer must install Python for ALL USERS and use absolute paths.'
} elseif ($cmd.Source -like '*WindowsApps*') {
Add-Result 'Python' 'On PATH' 'WARN' "$detail (Microsoft Store)" `
'Store Python misbehaves under service identities. The installer will not use it.'
} else {
Add-Result 'Python' 'On PATH' 'INFO' $detail `
'Not used by the installer, but a manual `flask` command later would resolve to this interpreter.'
}
}
Invoke-Check 'Python' 'Registered installs' {
$found = @()
foreach ($hive in @('HKLM:\SOFTWARE\Python\PythonCore','HKCU:\SOFTWARE\Python\PythonCore')) {
if (Test-Path $hive) {
foreach ($k in Get-ChildItem $hive -ErrorAction SilentlyContinue) {
$ip = Join-Path $k.PSPath 'InstallPath'
if (Test-Path $ip) {
$loc = (Get-ItemProperty $ip -ErrorAction SilentlyContinue).'(default)'
$scope = if ($hive -like 'HKLM*') { 'all-users' } else { 'per-user' }
$found += "$($k.PSChildName) ($scope) $loc"
}
}
}
}
if ($found.Count -eq 0) { Add-Result 'Python' 'Registered installs' 'INFO' 'none' }
else { Add-Result 'Python' 'Registered installs' 'INFO' ($found -join '; ') }
}
# =============================================================================
# Report
# =============================================================================
$fails = @($script:Results | Where-Object { $_.Status -eq 'FAIL' })
$warns = @($script:Results | Where-Object { $_.Status -eq 'WARN' })
$skips = @($script:Results | Where-Object { $_.Status -eq 'SKIP' })
if ($Delimited) {
# Data only. No padding, no colour, no alignment - the caller lays it out.
# Pipes are stripped from field values so the record can be split naively.
foreach ($r in $script:Results) {
$fix = ''
if ($r.Fix) { $fix = $r.Fix }
$fields = @($r.Status, $r.Area, $r.Check, $r.Detail, $fix) | ForEach-Object {
([string]$_) -replace '\|', '/' -replace '\s*\r?\n\s*', ' '
}
Write-Output ($fields -join '|')
}
} elseif ($Json) {
[PSCustomObject]@{
Timestamp = (Get-Date).ToString('s')
Computer = $env:COMPUTERNAME
SitePort = $SitePort
AppRoot = $AppRoot
Failures = $fails.Count
Warnings = $warns.Count
Skipped = $skips.Count
Results = $script:Results
} | ConvertTo-Json -Depth 5
} else {
Write-Host ''
Write-Host 'ShopDB-Flask preflight' -ForegroundColor Cyan
Write-Host (" host {0} site port {1} approot {2}" -f $env:COMPUTERNAME, $SitePort, $AppRoot)
Write-Host ''
$area = ''
foreach ($r in $script:Results) {
if ($r.Area -ne $area) { $area = $r.Area; Write-Host "[$area]" -ForegroundColor White }
$colour = 'Gray'
if ($r.Status -eq 'PASS') { $colour = 'Green' }
if ($r.Status -eq 'WARN') { $colour = 'Yellow' }
if ($r.Status -eq 'FAIL') { $colour = 'Red' }
if ($r.Status -eq 'SKIP') { $colour = 'DarkGray' }
Write-Host (" {0,-5} {1,-28} {2}" -f $r.Status, $r.Check, $r.Detail) -ForegroundColor $colour
if ($r.Fix -and $r.Status -ne 'PASS' -and $r.Status -ne 'INFO' -and $r.Status -ne 'SKIP') {
Write-Host (" -> {0}" -f $r.Fix) -ForegroundColor DarkGray
}
}
Write-Host ''
if ($fails.Count -eq 0) {
Write-Host "No blocking problems. $($warns.Count) warning(s), $($skips.Count) skipped." -ForegroundColor Green
} else {
Write-Host "$($fails.Count) blocking problem(s), $($warns.Count) warning(s), $($skips.Count) skipped." -ForegroundColor Red
}
if ($skips.Count -gt 0) {
Write-Host " Skipped checks were NOT verified. Re-run once their prerequisite is installed." -ForegroundColor DarkGray
}
Write-Host ''
}
if ($fails.Count -gt 0) { exit 1 } else { exit 0 }

Binary file not shown.

After

Width:  |  Height:  |  Size: 52 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 822 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 822 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 822 B

View File

@@ -0,0 +1,130 @@
# Tests the installer's durable install record.
#
# The record answers one question: "is this a re-run of MY install?". Getting it
# wrong is destructive in one direction - answering "first provisioning" for a
# server holding real data lets prune-schema --force loose on it - and a dead
# end in the other, which is what it was built to fix.
#
# Run directly: pwsh -File test-install-state.ps1
# pytest runs it through tests/test_installer_state.py wherever pwsh exists.
Set-StrictMode -Version 2.0
$ErrorActionPreference = 'Stop'
$Here = Split-Path -Parent $MyInvocation.MyCommand.Path
$Installer = Join-Path (Split-Path -Parent $Here) 'shopdb-install.ps1'
if (-not (Test-Path $Installer)) { throw "installer not found at $Installer" }
$Sandbox = Join-Path ([System.IO.Path]::GetTempPath()) ('shopdb-state-' + [System.Guid]::NewGuid().ToString('N'))
# Load ONLY the state block, so the rest of the installer does not run. Bounded
# by the same markers the installer uses, and it fails loudly if either moves
# rather than silently testing nothing.
$lines = Get-Content $Installer
$start = ($lines | Select-String -Pattern '^\$script:StateFile ' | Select-Object -First 1)
$end = ($lines | Select-String -Pattern '^function Invoke-Native' | Select-Object -First 1)
if (-not $start -or -not $end) { throw 'could not locate the install-record block in shopdb-install.ps1' }
$block = $lines[($start.LineNumber - 1)..($end.LineNumber - 2)] -join "`n"
if ($block -notmatch 'function Test-FirstProvisioning') { throw 'extracted block does not contain the state functions' }
# Stand-ins for the surrounding script.
$script:Created = New-Object System.Collections.ArrayList
function Write-Log { param($m, $l = 'INFO') }
function Protect-File { param([string] $Path, [switch] $Directory) }
function Test-SamePath {
param([string] $A, [string] $B)
if (-not $A) { return -not $B }
return ($A.TrimEnd('\', '/').ToLowerInvariant() -eq $B.TrimEnd('\', '/').ToLowerInvariant())
}
if (-not $env:ProgramData) { $env:ProgramData = [System.IO.Path]::GetTempPath() }
Invoke-Expression $block
$script:Failures = 0
function Check {
param([string] $Name, $Expected, $Actual)
if ($Expected -eq $Actual) {
Write-Host (" PASS " + $Name)
} else {
Write-Host (" FAIL {0}: expected [{1}], got [{2}]" -f $Name, $Expected, $Actual)
$script:Failures++
}
}
function Reset-Case {
param([switch] $Stamped)
if (Test-Path $Sandbox) { Remove-Item $Sandbox -Recurse -Force }
New-Item -ItemType Directory -Path $Sandbox -Force | Out-Null
$script:AppRoot = $Sandbox
$script:StateFile = Join-Path $Sandbox 'install-state.json'
$script:InstallState = $null
if ($Stamped) { Set-Content (Join-Path $Sandbox '.installed-version') '0.7.0' }
}
Write-Host 'brand new server'
Reset-Case
Initialize-InstallState
Check 'first provisioning' $true (Test-FirstProvisioning)
Check 'record written' $true (Test-Path $script:StateFile)
Write-Host 'retry of a failed first install'
$script:InstallState = $null
Initialize-InstallState
Check 'still first provisioning' $true (Test-FirstProvisioning)
Write-Host 'genuine upgrade after a completed install'
Complete-InstallState
$script:InstallState = $null
Initialize-InstallState
Check 'no longer first provisioning' $false (Test-FirstProvisioning)
Write-Host 'install predating the record - must take the safe side'
Reset-Case -Stamped
Initialize-InstallState
Check 'treated as established' $false (Test-FirstProvisioning)
Write-Host 'record naming a different directory'
Reset-Case
Initialize-InstallState
$other = Get-Content $script:StateFile -Raw | ConvertFrom-Json
$other.approot = 'C:\somewhere-else'
($other | ConvertTo-Json -Depth 6) | Set-Content $script:StateFile
$script:InstallState = $null
Check 'foreign record rejected' $null (Get-InstallState)
Write-Host 'corrupt record'
Reset-Case
Set-Content $script:StateFile '{ this is not json'
$script:InstallState = $null
Check 'corrupt record rejected' $null (Get-InstallState)
Reset-Case
Set-Content $script:StateFile '{ this is not json'
Initialize-InstallState
Check 'recovers and rewrites' $true ($null -ne $script:InstallState)
Write-Host 'created items survive a crash'
Reset-Case
Initialize-InstallState
Track 'site' 'shopdb-flask'
Track 'apppool' 'shopdbflask'
Track 'site' 'shopdb-flask'
$script:InstallState = $null
Initialize-InstallState
Check 'site remembered' 'shopdb-flask' ((Get-CreatedItems 'site') -join ',')
Check 'apppool remembered' 'shopdbflask' ((Get-CreatedItems 'apppool') -join ',')
# A zero-length array returned from a function unrolls to $null, and $null.Count
# is fatal under StrictMode. This is the check that catches losing the comma.
Check 'unknown kind is an empty array' 0 ((Get-CreatedItems 'firewall')).Count
Write-Host 'no record at all'
$script:InstallState = $null
Check 'empty list, not null' 0 ((Get-CreatedItems 'site')).Count
Check 'not first provisioning' $false (Test-FirstProvisioning)
if (Test-Path $Sandbox) { Remove-Item $Sandbox -Recurse -Force }
if ($script:Failures -gt 0) {
Write-Host ("{0} check(s) failed" -f $script:Failures)
exit 1
}
Write-Host 'all checks passed'
exit 0

View File

@@ -0,0 +1,171 @@
#!/usr/bin/env python3
"""Check a staged installer bundle against bundle-lock.json.
Prints one line per problem and exits non-zero if there are any. Exits 0 only
when the bundle's third-party payload is EXACTLY what the lock describes: no
missing file, no unexpected extra file, no changed content.
Why this exists alongside bundle-lock.ps1, which does the same job:
- bundle-lock.ps1 is canonical. It runs at INSTALL time on the target server,
where PowerShell is the only thing guaranteed to be present - Python is not
installed until stage 2, and verifying the payload after running part of it
would defeat the purpose.
- This file lets the Linux builder (build-installer.sh) do the same check
without adding pwsh as a build dependency.
The two are kept honest by tests/test_bundle_lock.py, which runs BOTH against
the same fixtures and fails if they disagree.
Usage: verify_bundle_lock.py <bundle-root> <bundle-lock.json>
"""
import hashlib
import json
import os
import re
import sys
# Must match $script:BundlePayloads in bundle-lock.ps1.
PAYLOADS = [
('wheels', True, 'Python wheels for the offline install'),
('python', True, 'the Python installer'),
('httpplatformhandler', True, 'the IIS module that launches waitress'),
('urlrewrite', False, 'IIS URL Rewrite, for the client-IP rule'),
('mysqlclient', False, 'mysql/mysqldump, for backups against a remote database'),
('vcredist', False, 'the Visual C++ runtime MySQL requires'),
('mysql', False, 'MySQL, for the bundled-database option'),
]
def digest(path):
sha = hashlib.sha256()
with open(path, 'rb') as fh:
for chunk in iter(lambda: fh.read(1024 * 1024), b''):
sha.update(chunk)
return sha.hexdigest()
def payload_files(directory):
"""Every file under the directory, keyed by forward-slashed relative path."""
found = {}
if not os.path.isdir(directory):
return found
for root, _dirs, files in os.walk(directory):
for name in files:
full = os.path.join(root, name)
rel = os.path.relpath(full, directory).replace(os.sep, '/')
found[rel] = {'sha256': digest(full), 'size': os.path.getsize(full)}
return found
def normalize(name):
"""PEP 427 wheel filename form: runs of non-alphanumerics become one _."""
return re.sub(r'[^A-Za-z0-9.]+', '_', name).lower()
def requirement_pins(requirements_path):
"""Every 'name==version' pinned in a lockfile, including marked-out ones.
Markers are deliberately IGNORED. A requirement guarded by
sys_platform == 'win32' is exactly the case that must be present, because the
target is Windows and the wheelhouse is usually assembled somewhere else.
"""
pins = {}
with open(requirements_path) as fh:
for line in fh:
line = line.strip()
if not line or line.startswith('#'):
continue
match = re.match(r'^([A-Za-z0-9._-]+)==([^\s;\\]+)', line)
if match:
pins[normalize(match.group(1))] = match.group(2)
return pins
def check_wheelhouse_covers_requirements(bundle_root):
"""The lock records what IS in the wheelhouse, not what the app NEEDS.
Without this, an incomplete wheelhouse gets locked and blessed, and the
install fails on an air-gapped server. That is not hypothetical: assembling
the wheelhouse on Linux silently omits colorama, a win32-only dependency of
click, because pip evaluates environment markers against the machine doing
the downloading rather than the machine being targeted.
"""
wheels = os.path.join(bundle_root, 'wheels')
requirements = os.path.join(bundle_root, 'app', 'requirements.txt')
if not os.path.isdir(wheels) or not os.path.exists(requirements):
return []
have = os.listdir(wheels)
problems = []
for name, version in sorted(requirement_pins(requirements).items()):
prefix = '%s-%s-' % (name, version)
if not any(f.lower().startswith(prefix) for f in have):
problems.append(
'wheels/ has no wheel for %s==%s, which requirements.txt pins '
'(a marked-out dependency still installs on Windows)' % (name, version))
return problems
def verify(bundle_root, lock):
problems = []
locked = lock.get('payloads')
if not locked:
return ['bundle-lock.json has no "payloads" section']
for name, required, what in PAYLOADS:
directory = os.path.join(bundle_root, name)
present = os.path.isdir(directory)
if name not in locked:
if present:
problems.append(
'%s/ is present but is not in bundle-lock.json - regenerate the lock' % name)
elif required:
problems.append(
'%s/ is required but is in neither the bundle nor the lock' % name)
continue
if not present:
if required or locked[name].get('required'):
problems.append('%s/ is in the lock but missing from the bundle (%s)' % (name, what))
continue
expected = locked[name].get('files', {})
actual = payload_files(directory)
for rel, want in sorted(expected.items()):
got = actual.get(rel)
if got is None:
problems.append('%s/%s is in the lock but missing from the bundle' % (name, rel))
elif got['sha256'] != want['sha256']:
problems.append(
'%s/%s does NOT match the lock (expected sha256 %s..., got %s...)'
% (name, rel, want['sha256'][:12], got['sha256'][:12]))
elif int(got['size']) != int(want['size']):
# Impossible for a matching sha256, so the lock was hand-edited.
problems.append(
'%s/%s size disagrees with the lock - the lock has been edited by hand'
% (name, rel))
for rel in sorted(actual):
if rel not in expected:
problems.append(
'%s/%s is in the bundle but NOT in the lock (unexpected extra file)'
% (name, rel))
problems.extend(check_wheelhouse_covers_requirements(bundle_root))
return problems
def main():
if len(sys.argv) != 3:
sys.exit('usage: verify_bundle_lock.py <bundle-root> <bundle-lock.json>')
bundle_root, lock_path = sys.argv[1], sys.argv[2]
if not os.path.exists(lock_path):
print('no bundle-lock.json at %s' % lock_path)
return 1
with open(lock_path) as fh:
lock = json.load(fh)
problems = verify(bundle_root, lock)
for problem in problems:
print(problem)
return 1 if problems else 0
if __name__ == '__main__':
sys.exit(main())

Binary file not shown.

After

Width:  |  Height:  |  Size: 151 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 217 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 869 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.1 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 56 KiB

View File

@@ -0,0 +1,422 @@
"""Collect everything needed to diagnose a ShopDB-Flask stage 5 failure.
Stage 5 is the smoke test: the installer asks IIS for the site and expects 200.
When it does not get one, the cause is always in one of four places, and this
script reads all four in one pass so the answer arrives in a single round trip:
1. What IIS actually answers, and with which status code and error page.
2. Whether the config sections httpPlatformHandler needs are unlocked.
3. Whether the app-pool identity can read the app and run its venv.
4. Whether the app itself imports and starts.
Run it ON THE SERVER, as Administrator:
C:\\Python314\\python.exe shopdb-diagnose.py
It writes shopdb-diagnose-<timestamp>.txt next to itself and prints the path.
Send that file back.
SECRETS: the report never contains them. Values from .env (database password,
SECRET_KEY, JWT_SECRET_KEY) are read first, then scrubbed out of every section
of the report before it is written, including command output and tracebacks
that might quote them.
Standard library only, so it runs on the bundled runtime or any system Python.
"""
import os
import re
import socket
import subprocess
import sys
import time
from datetime import datetime
APP_ROOT = os.environ.get('SHOPDB_APPROOT', r'C:\shopdb-flask')
ALIAS = 'shopdb'
TIMEOUT = 25
# Filled from .env, then scrubbed from the whole report.
SECRETS = []
WINDIR = os.environ.get('WINDIR', r'C:\Windows')
# Sysnative gives a 32-bit process the real 64-bit System32. Harmless on 64-bit.
APPCMD_CANDIDATES = [
os.path.join(WINDIR, 'Sysnative', 'inetsrv', 'appcmd.exe'),
os.path.join(WINDIR, 'System32', 'inetsrv', 'appcmd.exe'),
]
def find_appcmd():
for path in APPCMD_CANDIDATES:
if os.path.isfile(path):
return path
return None
class Report(object):
def __init__(self):
self.chunks = []
def head(self, title):
self.chunks.append('\n' + '=' * 72 + '\n' + title + '\n' + '=' * 72)
def line(self, text=''):
self.chunks.append(str(text))
def block(self, title, body):
self.chunks.append('\n--- %s ---' % title)
if body is None or str(body).strip() == '':
self.chunks.append('(no output)')
else:
self.chunks.append(str(body).rstrip())
def text(self):
return '\n'.join(self.chunks) + '\n'
def run(cmd, timeout=60):
"""Run a command, return combined output. Never raises."""
try:
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT, shell=False)
out, _ = proc.communicate(timeout=timeout)
text = out.decode('utf-8', 'replace') if out else ''
return '[exit %s]\n%s' % (proc.returncode, text)
except subprocess.TimeoutExpired:
try:
proc.kill()
except Exception:
pass
return '[TIMED OUT after %ss]' % timeout
except Exception as exc:
return '[could not run: %s]' % exc
def powershell(script, timeout=90):
exe = os.path.join(WINDIR, 'Sysnative', 'WindowsPowerShell', 'v1.0', 'powershell.exe')
if not os.path.isfile(exe):
exe = 'powershell.exe'
return run([exe, '-NoProfile', '-ExecutionPolicy', 'Bypass', '-Command', script],
timeout=timeout)
def load_secrets():
"""Read .env so its secret VALUES can be scrubbed from the report."""
env_path = os.path.join(APP_ROOT, '.env')
found = {}
if not os.path.isfile(env_path):
return found, None
try:
with open(env_path, 'r', encoding='utf-8', errors='replace') as handle:
raw = handle.read()
except Exception as exc:
return found, '[could not read .env: %s]' % exc
for line in raw.splitlines():
line = line.strip()
if not line or line.startswith('#') or '=' not in line:
continue
key, value = line.split('=', 1)
key, value = key.strip(), value.strip().strip('"').strip("'")
found[key] = value
if not value:
continue
upper = key.upper()
if 'SECRET' in upper or 'PASSWORD' in upper or 'TOKEN' in upper or 'KEY' in upper:
SECRETS.append(value)
if upper == 'DATABASE_URL':
# mysql+pymysql://user:PASSWORD@host/db -- the password only.
match = re.match(r'^[^:]+://([^:@/]+):([^@]+)@', value)
if match:
SECRETS.append(match.group(2))
return found, raw
def scrub(text):
"""Remove every known secret value from the report."""
for secret in SECRETS:
if secret and len(secret) >= 4:
text = text.replace(secret, '<REDACTED>')
# Catch a password inside any URL that did not come from .env.
text = re.sub(r'(://[^:@/\s]+:)[^@\s]+(@)', r'\1<REDACTED>\2', text)
return text
def http_probe(url):
"""Fetch a URL, returning status, headers and body even for an error page."""
import urllib.error
import urllib.request
started = time.time()
try:
request = urllib.request.Request(url, headers={'User-Agent': 'shopdb-diagnose'})
with urllib.request.urlopen(request, timeout=TIMEOUT) as response:
body = response.read(4000).decode('utf-8', 'replace')
return {'status': response.status, 'reason': response.reason,
'headers': dict(response.headers), 'body': body,
'seconds': time.time() - started}
except urllib.error.HTTPError as exc:
body = ''
try:
body = exc.read(4000).decode('utf-8', 'replace')
except Exception:
pass
return {'status': exc.code, 'reason': exc.reason,
'headers': dict(exc.headers or {}), 'body': body,
'seconds': time.time() - started}
except Exception as exc:
return {'status': None, 'reason': '%s: %s' % (type(exc).__name__, exc),
'headers': {}, 'body': '', 'seconds': time.time() - started}
def summarise_iis_error(body):
"""Pull the meaningful bits out of an IIS error page."""
if not body:
return None
import html
# style and script blocks first: their contents survive plain tag stripping
# and drag CSS into the summary.
flat = re.sub(r'(?is)<(script|style)[^>]*>.*?</\1>', ' ', body)
flat = re.sub(r'(?s)<!--.*?-->', ' ', flat)
flat = re.sub(r'(?s)<[^>]*>', ' ', flat)
# Any unterminated tag left by the 4000-byte body truncation.
flat = re.sub(r'(?s)<[^>]*$', ' ', flat)
flat = html.unescape(flat)
flat = re.sub(r'\s+', ' ', flat).strip()
hints = []
for pattern in (r'\b\d{3}\.\d+\b', r'0x[0-9a-fA-F]{8}',
r'Error Code[^.]{0,80}', r'Config (?:Error|File)[^.]{0,120}',
r'Requested URL[^.]{0,120}', r'Physical Path[^.]{0,120}'):
for match in re.findall(pattern, flat):
cleaned = re.sub(r'\s+', ' ', match).strip(' :-')
if cleaned and cleaned not in hints:
hints.append(cleaned)
return {'flat': flat[:1200], 'hints': hints}
def main():
report = Report()
stamp = datetime.now().strftime('%Y%m%d-%H%M%S')
env_values, env_raw = load_secrets()
report.line('ShopDB-Flask stage 5 diagnostic')
report.line('generated %s' % datetime.now().strftime('%Y-%m-%d %H:%M:%S'))
report.line('host %s' % socket.gethostname())
report.line('app root %s' % APP_ROOT)
report.line('python %s' % sys.version.replace('\n', ' '))
report.line('process is %d-bit' % (64 if sys.maxsize > 2 ** 32 else 32))
# ---------------------------------------------------------------- 1. HTTP
report.head('1. WHAT IIS ANSWERS')
report.line('This is the single most important section. The status code names')
report.line('the fault: 500.19 = config locked, 502.3/503 = the app did not')
report.line('start, 404 = application or handler mapping missing.')
hostname = socket.gethostname()
targets = [
'http://localhost/%s/' % ALIAS,
'http://127.0.0.1/%s/' % ALIAS,
'http://[::1]/%s/' % ALIAS,
'http://%s/%s/' % (hostname, ALIAS),
'http://localhost/',
]
for url in targets:
result = http_probe(url)
report.line('\n%s' % url)
report.line(' status : %s %s' % (result['status'], result['reason']))
report.line(' time : %.1fs' % result['seconds'])
server = result['headers'].get('Server')
if server:
report.line(' server : %s' % server)
summary = summarise_iis_error(result['body'])
if summary and summary['hints']:
report.line(' hints : %s' % ' | '.join(summary['hints'][:8]))
if summary and summary['flat']:
report.line(' body : %s' % summary['flat'][:600])
# localhost resolving to ::1 first has bitten this install before.
report.block('name resolution for localhost', run(
['nslookup', 'localhost'], timeout=20))
# ------------------------------------------------------------- 2. IIS state
report.head('2. IIS STATE')
appcmd = find_appcmd()
if not appcmd:
report.line('appcmd.exe NOT FOUND - is the IIS role installed?')
else:
report.line('appcmd: %s' % appcmd)
report.block('sites', run([appcmd, 'list', 'sites']))
report.block('applications', run([appcmd, 'list', 'apps']))
report.block('app pools', run([appcmd, 'list', 'apppools']))
report.block('worker processes (empty means nothing is running)',
run([appcmd, 'list', 'wp']))
report.block('modules: httpPlatformHandler present?',
run([appcmd, 'list', 'modules']))
# overrideMode tells us whether the unlock actually took effect.
#
# allowedServerVariables is in this list because it caused a 500.52 that
# the first two sections could not explain: it is Deny by default, so an
# <allowedServerVariables> block in the app's web.config is rejected
# before httpPlatformHandler runs. Checking only the sections we unlock
# would have missed the one we do not.
for section in ('system.webServer/handlers',
'system.webServer/httpPlatform',
'system.webServer/rewrite/allowedServerVariables',
'system.webServer/rewrite/rules'):
report.block('lock state of %s' % section,
run([appcmd, 'list', 'config', '/section:%s' % section,
'/text:*']))
report.block('W3SVC / WAS services', powershell(
"Get-Service W3SVC,WAS | Format-Table Name,Status,StartType -AutoSize | Out-String"))
report.block('listeners on port 80', powershell(
"Get-NetTCPConnection -LocalPort 80 -State Listen -EA SilentlyContinue | "
"Format-Table LocalAddress,LocalPort,OwningProcess -AutoSize | Out-String"))
report.block('app pool detail', powershell(
"Import-Module WebAdministration -EA SilentlyContinue; "
"Get-Item IIS:\\AppPools\\shopdbflask -EA SilentlyContinue | "
"Select-Object name,state,managedRuntimeVersion,enable32BitAppOnWin64,"
"@{n='identity';e={$_.processModel.identityType}} | Format-List | Out-String"))
# --------------------------------------------------------- 3. app + config
report.head('3. APPLICATION AND CONFIG')
web_config = os.path.join(APP_ROOT, 'web.config')
if os.path.isfile(web_config):
try:
with open(web_config, 'r', encoding='utf-8', errors='replace') as handle:
report.block('web.config', handle.read())
except Exception as exc:
report.block('web.config', '[could not read: %s]' % exc)
else:
report.block('web.config', 'MISSING at %s' % web_config)
if env_raw is None:
report.block('.env', 'MISSING at %s' % os.path.join(APP_ROOT, '.env'))
else:
# Keys and non-secret values only. Secret values are scrubbed anyway.
lines = []
for key in sorted(env_values):
upper = key.upper()
secretish = ('SECRET' in upper or 'PASSWORD' in upper
or 'TOKEN' in upper or 'KEY' in upper
or upper == 'DATABASE_URL')
if secretish:
lines.append('%s = <set, %d chars>' % (key, len(env_values[key])))
else:
lines.append('%s = %s' % (key, env_values[key]))
report.block('.env (secret values withheld)', '\n'.join(lines))
venv_python = os.path.join(APP_ROOT, 'venv', 'Scripts', 'python.exe')
report.line('\nvenv python exists: %s' % os.path.isfile(venv_python))
if os.path.isfile(venv_python):
# The exact failure stage 3 used to hit. Proves the app imports.
report.block('venv: import shopdb', run(
[venv_python, '-c',
'import shopdb; print("import OK"); '
'app = shopdb.create_app(); print("create_app OK")'], timeout=120))
report.block('venv: waitress present', run(
[venv_python, '-c', 'import waitress; print(waitress.__version__)'],
timeout=60))
# What httpPlatformHandler is told to launch, and whether it exists.
if os.path.isfile(web_config):
try:
with open(web_config, 'r', encoding='utf-8', errors='replace') as handle:
raw = handle.read()
match = re.search(r'processPath\s*=\s*"([^"]+)"', raw)
args = re.search(r'arguments\s*=\s*"([^"]*)"', raw)
if match:
path = os.path.expandvars(match.group(1))
report.line('\nhttpPlatform processPath : %s' % match.group(1))
report.line(' resolved : %s' % path)
report.line(' exists : %s' % os.path.isfile(path))
if args:
report.line('httpPlatform arguments : %s' % args.group(1))
except Exception as exc:
report.line('[could not parse web.config: %s]' % exc)
# ------------------------------------------------------------- 4. app logs
report.head('4. APPLICATION LOGS')
log_dir = os.path.join(APP_ROOT, 'logs')
if not os.path.isdir(log_dir):
report.line('MISSING: %s' % log_dir)
else:
entries = []
for name in sorted(os.listdir(log_dir)):
full = os.path.join(log_dir, name)
try:
entries.append((os.path.getmtime(full), full, name,
os.path.getsize(full)))
except OSError:
pass
if not entries:
report.line('%s is EMPTY.' % log_dir)
report.line('No stdout log at all means httpPlatformHandler never')
report.line('launched python - look at the pool identity and ACLs.')
for _, full, name, size in sorted(entries, reverse=True)[:5]:
if size == 0:
report.block('%s (0 bytes)' % name,
'EMPTY - python was launched but wrote nothing.')
continue
try:
with open(full, 'r', encoding='utf-8', errors='replace') as handle:
tail = handle.readlines()[-60:]
report.block('%s (%d bytes, last 60 lines)' % (name, size),
''.join(tail))
except Exception as exc:
report.block(name, '[could not read: %s]' % exc)
# ---------------------------------------------------------------- 5. ACLs
report.head('5. PERMISSIONS')
report.line('The pool runs as "IIS AppPool\\shopdbflask". It needs RX on the')
report.line('tree, Modify on logs and instance, and Read on .env.')
for target in (APP_ROOT, log_dir, os.path.join(APP_ROOT, '.env'),
os.path.join(APP_ROOT, 'venv', 'Scripts')):
if os.path.exists(target):
report.block('icacls %s' % target, run(['icacls', target], timeout=40))
# ------------------------------------------------------------ 6. event log
report.head('6. EVENT LOG')
report.block('recent application errors', powershell(
"Get-WinEvent -FilterHashtable @{LogName='Application';"
"StartTime=(Get-Date).AddHours(-6)} -EA SilentlyContinue | "
"Where-Object { $_.ProviderName -match 'HttpPlatform|IIS|W3SVC|WAS|\\.NET' "
"-or $_.LevelDisplayName -eq 'Error' } | Select-Object -First 25 "
"TimeCreated,ProviderName,LevelDisplayName,Message | Format-List | Out-String",
timeout=180))
report.block('system log: WAS / W3SVC', powershell(
"Get-WinEvent -FilterHashtable @{LogName='System';"
"StartTime=(Get-Date).AddHours(-6)} -EA SilentlyContinue | "
"Where-Object { $_.ProviderName -match 'WAS|W3SVC|HTTP' } | "
"Select-Object -First 20 TimeCreated,ProviderName,LevelDisplayName,Message | "
"Format-List | Out-String", timeout=180))
# ------------------------------------------------------------------ write
body = scrub(report.text())
out_path = os.path.join(os.path.dirname(os.path.abspath(__file__)),
'shopdb-diagnose-%s.txt' % stamp)
try:
with open(out_path, 'w', encoding='utf-8') as handle:
handle.write(body)
except Exception:
out_path = os.path.join(os.environ.get('TEMP', r'C:\Windows\Temp'),
'shopdb-diagnose-%s.txt' % stamp)
with open(out_path, 'w', encoding='utf-8') as handle:
handle.write(body)
print('')
print('Report written to:')
print(' %s' % out_path)
print('')
print('%d secret value(s) were scrubbed from it.' % len(SECRETS))
print('Send that file back.')
return 0
if __name__ == '__main__':
sys.exit(main())

View File

@@ -11,7 +11,7 @@
- HttpPlatformHandler IIS module installed
(https://www.iis.net/downloads/microsoft/httpplatformhandler)
- URL Rewrite module installed (only for the optional X-Forwarded-For rule)
- Python 3.12 + a venv at APP_ROOT\venv with requirements.txt + waitress
- Python 3.14 + a venv at APP_ROOT\venv with requirements.txt + waitress
- Secrets live in APP_ROOT\.env (wsgi.py load_dotenv() reads it). Keep them
OUT of this file. Lock .env ACLs to the app pool identity + admins.
@@ -28,7 +28,7 @@
<httpPlatform
processPath="C:\shopdb-flask\venv\Scripts\waitress-serve.exe"
arguments="--port=%HTTP_PLATFORM_PORT% --host=127.0.0.1 --threads=8 wsgi:app"
arguments="--port=%HTTP_PLATFORM_PORT% --host=127.0.0.1 --threads=8 --trusted-proxy=127.0.0.1 --trusted-proxy-headers=x-forwarded-for --trusted-proxy-count=1 wsgi:app"
stdoutLogEnabled="true"
stdoutLogFile="C:\shopdb-flask\logs\httpplatform"
startupTimeLimit="120"
@@ -38,24 +38,57 @@
config (SQL echo, debug, wrong DB URL). Real secrets go in .env. -->
<environmentVariable name="FLASK_ENV" value="production" />
<environmentVariable name="PYTHONPATH" value="C:\shopdb-flask" />
<!-- Subpath method only: when this web.config sits in an IIS
Application (e.g. /ops) under an existing site instead of its own
site, tell the app its mount path. Must match the alias the
Application was created with AND the VITE_BASE_PATH the frontend
was built with ('/ops/'). Omit for the own-site method.
<environmentVariable name="MOUNT_PATH" value="/ops" />
-->
</environmentVariables>
</httpPlatform>
<!--
OPTIONAL: forward the real client IP so audit logs and the kiosk
visitor-location feature (IP -> business unit) see the caller, not the
loopback that HttpPlatformHandler connects from.
Forward the real client IP, so the audit log, the kiosk visitor-location
feature (IP -> business unit), the GE-Enforce IP allowlist and per-host
login rate limiting all see the caller rather than the loopback address
HttpPlatformHandler connects from.
This block is COMMENTED OUT by default because it needs the URL Rewrite
module; with it uncommented but URL Rewrite not installed, IIS returns
HTTP 500.19 ("configuration section not well-formed / cannot be read").
Install URL Rewrite (https://www.iis.net/downloads/microsoft/url-rewrite)
and then uncomment the <rewrite> block below to enable it.
IIS does not set X-Forwarded-For on its own. Without the rule below there
is no such header at all, and every client looks like 127.0.0.1 - so the
allowlist and the visitor-location lookup silently stop working.
ONLY CORRECT WHEN IIS IS DIRECTLY EXPOSED. It overwrites the header with
REMOTE_ADDR, which is what stops a client spoofing its own X-Forwarded-For.
Behind a reverse proxy (ARR, a load balancer) REMOTE_ADDR is the PROXY, so
this rule would destroy the real client IP - there, leave it disabled and
let the proxy set the header.
It ships DISABLED because it needs the URL Rewrite module; enabled without
it, IIS returns HTTP 500.19 ("configuration section not well-formed").
There is deliberately NO <allowedServerVariables> block below. Setting a
server variable requires that variable to be allowed, but the section
system.webServer/rewrite/allowedServerVariables ships with
overrideModeDefault="Deny", so declaring it in an application's own
web.config is refused outright: IIS answered 500.52 with error 0x80070021,
"this configuration section cannot be used at this path", BEFORE it ever
reached httpPlatformHandler - so python was never launched and the stdout
log stayed empty, which looks like an application fault and is not one.
The installer instead allows the single variable at server level, which
grants exactly HTTP_X_FORWARDED_FOR rather than unlocking the section and
letting every site on the machine declare arbitrary server variables.
The installer handles both: -ClientIpSource direct installs URL Rewrite
from the bundle, allows the variable, and enables this; -ClientIpSource
proxy leaves it alone. By hand: install URL Rewrite, run
appcmd set config /section:system.webServer/rewrite/allowedServerVariables ^
/+"[name='HTTP_X_FORWARDED_FOR']" /commit:apphost
then delete the two marker lines below.
-->
<!-- SHOPDB-CLIENTIP-BEGIN
<rewrite>
<allowedServerVariables>
<add name="HTTP_X_FORWARDED_FOR" />
</allowedServerVariables>
<rules>
<rule name="Set X-Forwarded-For" stopProcessing="false">
<match url=".*" />
@@ -66,7 +99,42 @@
</rule>
</rules>
</rewrite>
-->
SHOPDB-CLIENTIP-END -->
</system.webServer>
<!--
Installer downloads: serve /installers/* as IIS static files instead of
forwarding them to Flask. The handler above is path="*", so without this a
request for /installers/Foo.exe goes to waitress, which has no such route
(SPA fallback), and large binaries would stream through a Python thread.
This <location> clears the httpPlatformHandler for that one subpath and puts
the static file handler back, so IIS serves the bytes directly (kernel-mode,
range/resume, no Python thread held).
Requires a physical folder at APP_ROOT\installers (the site's physical path
is APP_ROOT). Drop the installer binaries there, e.g. robocopy them from the
classic wwwroot\installers. The stored installpath 'installers/Foo.exe' then
resolves to <mount>/installers/Foo.exe (e.g. /shopdb/installers/Foo.exe).
.exe/.msi are given an explicit MIME map; if the parent site has a Request
Filtering rule that denies executable extensions, also allow them there.
-->
<location path="installers">
<system.webServer>
<handlers>
<clear />
<add name="StaticFile" path="*" verb="*"
modules="StaticFileModule" resourceType="File"
requireAccess="Read" />
</handlers>
<staticContent>
<remove fileExtension=".exe" />
<mimeMap fileExtension=".exe" mimeType="application/octet-stream" />
<remove fileExtension=".msi" />
<mimeMap fileExtension=".msi" mimeType="application/octet-stream" />
</staticContent>
</system.webServer>
</location>
</configuration>

90
docker-compose.airgap.yml Normal file
View File

@@ -0,0 +1,90 @@
# shopdb-flask AIR-GAPPED single-site stack.
#
# For a site with NO internet. Nothing is built or pulled here: the images are
# built on a connected box (scripts/build-offline-bundle.ps1), shipped as a
# tarball, and `docker load`ed at the site. This file only RUNS pre-loaded
# images. See docs/DEPLOY-AIRGAP.md for the full runbook.
#
# Differences from docker-compose.yml (the connected/build template):
# - api uses `image:` (a loaded image), never `build: .` (build needs the net).
# - NO ./plugins bind mount. The image already carries every plugin baked in;
# binding a host ./plugins (which does not exist at an image-only site) would
# mask the baked plugins with an empty dir and load ZERO plugins.
# - a one-shot `migrate` service runs db upgrade + plugin upgrade-all + seed
# BEFORE api starts, so `up -d` alone brings up a working site (no manual
# `docker compose exec ... flask db upgrade` to forget).
#
# Usage at the site:
# docker load < shopdb-stack-<version>.tar.gz
# cp .env.example .env # then edit: secrets, CORS_ORIGINS, IMAGE_TAG
# docker compose -f docker-compose.airgap.yml up -d
# docker compose -f docker-compose.airgap.yml exec api flask seed admin <user> <email> <password>
# Shared application environment, reused by the migrate one-shot and the api
# service so the two never drift. A YAML anchor, not a container.
x-app-env: &app-env
FLASK_APP: wsgi.py
FLASK_ENV: production
DATABASE_URL: mysql+pymysql://shopdb:${MYSQL_PASSWORD}@db:3306/shopdb_flask?charset=utf8mb4
SECRET_KEY: ${SECRET_KEY:?SECRET_KEY must be set}
JWT_SECRET_KEY: ${JWT_SECRET_KEY:?JWT_SECRET_KEY must be set}
CORS_ORIGINS: ${CORS_ORIGINS:?CORS_ORIGINS must be set}
LOG_LEVEL: ${LOG_LEVEL:-INFO}
ZABBIX_URL: ${ZABBIX_URL:-}
ZABBIX_TOKEN: ${ZABBIX_TOKEN:-}
services:
db:
image: mysql:8.0
command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
restart: unless-stopped
environment:
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:?MYSQL_ROOT_PASSWORD must be set}
MYSQL_DATABASE: shopdb_flask
MYSQL_USER: shopdb
MYSQL_PASSWORD: ${MYSQL_PASSWORD:?MYSQL_PASSWORD must be set}
volumes:
- db_data:/var/lib/mysql
ports:
- "127.0.0.1:${MYSQL_PORT:-3306}:3306"
healthcheck:
test: ["CMD", "mysqladmin", "ping", "-h", "localhost", "-u", "root", "-p${MYSQL_ROOT_PASSWORD}"]
interval: 10s
timeout: 5s
retries: 5
# One-shot schema + seed. Runs to completion and exits; api waits for it.
# Every step is idempotent, so it is safe to run on every `up`.
migrate:
image: shopdb-flask:${IMAGE_TAG:-0.7.0}
restart: "no"
depends_on:
db:
condition: service_healthy
environment:
<<: *app-env
command:
- sh
- -c
- >
flask db upgrade &&
flask plugin upgrade-all &&
flask seed permissions &&
flask seed settings &&
flask seed reference-data
api:
image: shopdb-flask:${IMAGE_TAG:-0.7.0}
restart: unless-stopped
depends_on:
db:
condition: service_healthy
migrate:
condition: service_completed_successfully
environment:
<<: *app-env
ports:
- "${API_PORT:-5001}:5001"
volumes:
db_data:

133
docs/API-REFERENCE.md Normal file
View File

@@ -0,0 +1,133 @@
# API reference (index)
This page is an index and a pointer, not a full specification. It answers three
questions: what endpoints exist, who calls them, and what auth they require. The
detailed request and response shapes live in the live generated docs and in the
per-surface contract docs linked from each table below.
## Live and generated docs
The repo ships hosted, generated API docs. Start there:
- **Interactive spec:** `GET /api/docs` - a self-hosted Redoc page over the
generated OpenAPI spec. The Redoc bundle is vendored under
`shopdb/core/api/staticdocs/`, so it renders fully offline on the air-gapped
prod box (no CDN).
- **Raw spec:** `GET /api/docs/openapi.json` - OpenAPI 3.1, roughly 238 paths and
362 operations. Generated by `scripts/gen_openapi.py` from
`docs/api-inventory.json`; regenerate after any API change.
- **LLM / agent entry point:** `GET /api/docs/llms.txt` - a concise API guide
following the llms.txt convention, plus a read-only MCP server
(`mcp/shopdb_mcp.py`, built with `FastMCP.from_openapi` over the same spec).
The MCP server exposes a curated set of GET endpoints as tools for an agent to
query the asset database over HTTPS with a scoped read token; it never runs on
the prod box. Set it up on a work PC with
`pxe-images/github/setup-mcp.cmd`.
The docs blueprint is `shopdb/core/api/docs.py` (a core blueprint, always
mounted regardless of which plugins are staged into a site build).
Contract docs (linked per table below) hold the deep semantics: field mappings,
idempotency rules, rotation, error envelopes, staged rollout. This page only
routes you to the right one.
---
## 1. Fleet and client contracts (unauthenticated or token)
These are the endpoints the shopfloor PC fleet, kiosks, displays, and printer
installers call. They are consumed by machines, not by the interactive UI, and
they authenticate with a scoped service token or nothing at all.
| Endpoint | Auth | Purpose | Contract doc |
|---|---|---|---|
| `GET /api/geenforce/manifest?pctype=<scope>` | `geenforce.fetch` service token (`X-API-Key` or Bearer PAT) | Serve the current published manifest for a PC-type scope. ETag / 304 supported. | GE-ENFORCE-CLIENT.md, geenforce-api-cutover.md |
| `GET /api/geenforce/payload/<sha256>` | `geenforce.fetch` service token | Serve a payload blob (installer) by content hash so share-less PCs pull over HTTPS instead of SMB. Rate limited and size capped; the sha256 is the integrity guarantee. | GE-ENFORCE-CLIENT.md, geenforce-api-cutover.md |
| `POST /api/geenforce/report` | `geenforce.report` service token | Record one PC's enforcement cycle: applied manifest version plus per-entry self-heal outcomes. | GE-ENFORCE-CLIENT.md, geenforce-api-cutover.md |
| `POST /api/collector/<plugin>` | `X-API-Key` env key or a `collector.ingest` managed token | Generic idempotent inventory upsert; the PC fleet targets `/api/collector/computers`. | COLLECTOR-INTEGRATION.md |
| `POST /api/collector/pc`, `/apps`, `/heartbeat`, `/bulk` | `X-API-Key` or `collector.ingest` token | Legacy computers-only collector paths (predate ADR-006); deprecated in favor of `/api/collector/computers`. | COLLECTOR-INTEGRATION.md |
| `GET /api/collector/status` | `X-API-Key` or `collector.ingest` token | Collector liveness and endpoint list. | COLLECTOR-INTEGRATION.md |
| `GET /api/printers/install-list` | optional JWT (anonymous fleet or logged-in browser) | Flat list of network printers with floor-map positions for the signed installer. `?format=text` returns a pipe-delimited variant. | PRINTER-INSTALLER.md |
| `GET /api/printers/pc-default?machine=NNNN` | optional JWT | The PC's default printer by machine (asset) number, via the `defaultprinter` relationship. `?format=text` supported. | PRINTER-INSTALLER.md |
| `GET /api/printers/install-batch?printerids=1,2,3` | optional JWT | Generate a self-deleting Windows `.bat` that installs the selected printers. | PRINTER-INSTALLER.md |
| `GET /api/dashboarddefaults/display-role?fqdn=<fqdn>` | public (none) | Resolve what a display PC should show (role `dashboard`/`lobby`/`partskiosk`, frontend path, business unit). FQDN-first, IP fallback. | GE-ENFORCE-DISPLAY.md |
| `GET /api/dashboarddefaults/visitor-location?fqdn=<fqdn>` | public (none) | Resolve the business unit for a lobby display by FQDN (IP fallback). | GE-ENFORCE-DISPLAY.md |
The `geenforce.fetch` and `geenforce.report` scopes accept both `X-API-Key` and
`Authorization: Bearer` transports, the same managed-token pattern the collector
uses (see COLLECTOR-INTEGRATION.md for how to mint, deploy, and rotate a scoped
token). A fetch token may be further resource-bound to specific scopes; a bound
token is denied (403 on manifest, 404 on payload) anything outside its scopes.
---
## 2. Import API
The import surface (an admin PAT plus `X-Import-Mode` to preserve legacy
timestamps) lets a script load an entire legacy database through the same
endpoints the UI uses. It is documented in full, per resource, in **IMPORT-API.md**
and is not duplicated here. The dashboarddefaults import fields (FQDN-preferred
keying) are covered there as well.
---
## 3. Core UI API
Everything else is the core UI API: the endpoints the Vue frontend calls. As a
rule these are JWT-authenticated (a login token or a managed Personal Access
Token) and versioned by the plugin contract (`__contract_version__`, currently
0.15.0). Behavior and stability guarantees are in **CONTRACT-STABILITY.md**;
sister sites should pin tight `core_version` ranges until the contract reaches
1.0.
Two auth patterns dominate the reads:
- **Public (no token ever).** The endpoints below are reachable with no
credential at all. This is the surface a firewall or deployment reviewer asks
about, so it is enumerated in full.
- **Optional JWT (`jwt_required(optional=True)`).** Nearly every core and plugin
GET (list / detail / report / dashboard-summary) is optional-auth: it serves
reads anonymously and only requires a JWT for writes. There are well over a
hundred of these; rather than reprint them, enumerate them from the live spec
at `/api/docs` (filter to the `GET` operations). All product reports
(`/api/reports/*` and every plugin `.../report*`) are optional-auth by the same
convention.
Every mutating endpoint (POST / PUT / PATCH / DELETE) requires a JWT and is
gated by `require_role` or `require_permission`; none are public.
### Fully public endpoints (auth = none)
| Endpoint | Purpose |
|---|---|
| `POST /api/auth/login` | Obtain a JWT. |
| `GET /api/setup/needs-admin` | First-run check: does the instance have zero users. |
| `POST /api/setup/create-admin` | First-run only; creates the first admin, then 403s forever. |
| `GET /api/settings/map-blueprint/<filename>` | Serve the floor-map blueprint image. |
| `GET /api/settings/branding/<filename>` | Serve site branding assets (logo, etc.). |
| `GET /api/models/image/<filename>` | Serve a model image. |
| `GET /api/dashboard/navigation` | Public navigation tree. |
| `GET /api/dashboard/health` | Liveness / health probe. |
| `GET /api/plugins/enabled` | List enabled plugins (no claims used). |
| `GET /api/dashboarddefaults/display-role` | Display role resolution (see section 1). |
| `GET /api/dashboarddefaults/visitor-location` | Lobby business-unit resolution (see section 1). |
| `GET /api/employees/search`, `/lookup/<sso>`, `/lookup` | Employee directory lookups (kiosk / sign-in flows). |
| `GET /api/employees/photo/<filename>` | Serve an employee photo. |
| `GET /api/notifications` (and `/types`, `/<id>`, `/active`, `/calendar`, `/dashboard/summary`, `/employee/<sso>`, `/shopfloor`) | Read shop-floor notifications for the public display and kiosks. |
| `GET /api/slides/feed` | Slide feed for the lobby display and screensaver. |
| `GET /api/slides/img/<surface>/<filename>` | Serve a slide image. |
| `GET /api/printedparts/image/<filename>` | Serve a printed-part image. |
| `GET /api/printedparts/kiosk/item/<itemcode>` | Kiosk part lookup (deliberately open; a kiosk carries no JWT). |
| `POST /api/printedparts/kiosk/take` | Kiosk part checkout (deliberately open, per decision record). |
---
## See also
- `docs/adr/README.md` - architecture decision records index.
- `docs/DEPLOY.md` - deployment; the public-endpoint inventory in section 3 above
is the site-exposure surface a deploy reviewer needs.
- `docs/PLUGINS.md` - the plugin catalog.
- Contract docs: GE-ENFORCE-CLIENT.md, geenforce-api-cutover.md,
COLLECTOR-INTEGRATION.md, PRINTER-INSTALLER.md, GE-ENFORCE-DISPLAY.md,
IMPORT-API.md, CONTRACT-STABILITY.md.

View File

@@ -18,6 +18,8 @@ app pointing at floor plans and logos that no longer exist.
|------|----------|-----|
| Database | MySQL `shopdb_flask` | All application data. |
| `instance/branding/` | repo `instance/` dir | Uploaded logos and favicon. |
| `instance/modelimages/` | repo `instance/` dir | Uploaded vendor-model photos. |
| `instance/employeephotos/` | repo `instance/` dir | Uploaded self-hosted employee photos (external mode serves photos from the HR database instead). |
| `instance/` floor plans | repo `instance/` dir | Uploaded map blueprints. |
| `instance/plugins.json` | repo `instance/` dir | Which plugins this site enabled. |
| `.env` | repo root (offline, secured) | Secrets needed to bring the stack back up. Store separately from the data backup, in a secrets manager. |
@@ -94,18 +96,42 @@ CREATE DATABASE shopdb_flask CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
tar xzf instance-2026-07-10.tar.gz # restores ./instance/
```
The docker-compose api container reads `instance/` from the repo working
directory; make sure it is present before starting `api`.
The default `docker-compose.yml` does NOT bind-mount `instance/` into the api
container (its only volume is `- ./plugins:/app/plugins:ro`, and the image never
copies `instance/`), so the container's Flask instance path is an empty
`/app/instance` and a restored host `./instance` is invisible to it. To make the
restored `instance/` visible, add a bind mount to the api service before starting
it:
```yaml
api:
volumes:
- ./plugins:/app/plugins:ro
- ./instance:/app/instance
```
Make sure `./instance` is present on the host before starting `api`.
### Step 4: Bring up the API and reconcile migrations
```bash
docker compose up -d api
docker compose exec api flask db upgrade
docker compose exec api flask plugin upgrade-all
```
For a non-docker deploy:
```bash
flask db upgrade
flask plugin upgrade-all
```
`flask db upgrade` is a safety net: if the dump predates the current code, this
applies any newer migrations. If the dump is at the same version it is a no-op.
applies only the core Alembic chain. `flask plugin upgrade-all` then applies any
newer per-plugin migrations (each bundled plugin owns its own chain, ADR-008);
without it, plugin-owned tables stay un-migrated. If the dump is at the same
version both are no-ops.
### Step 5: Verify
@@ -116,8 +142,40 @@ applies any newer migrations. If the dump is at the same version it is a no-op.
- `curl -s -X POST -H "Content-Type: application/json" -d '{}' http://localhost:5001/api/auth/login | jq .`
should return a `VALIDATION_ERROR`, not a 500.
## Windows sites (installer-built)
On a server installed from the Windows installer, everything above is wrapped by
the operator console. Do not run mysqldump by hand:
```powershell
cd C:\shopdb-flask
.\shopdb-admin.ps1 backup # C:\ProgramData\ShopDB-Flask\backups
.\shopdb-admin.ps1 backup D:\backups
```
The dump is verified complete before it is reported as good; a truncated one is
deleted rather than left to be discovered when it is needed. An upgrade takes its
own backup automatically before touching the schema, and restores from it if a
migration fails.
Two Windows-specific notes:
- The backup directory is locked to Administrators and SYSTEM, because a dump
contains every row including user password hashes. Keep it that way.
- `mysqldump` must be present. It ships with the bundled-database option; a site
using a remote MySQL needs `mysqlclient\` in its installer bundle, or the
pre-upgrade backup is skipped. `shopdb-admin.ps1 check` reports this.
Restoring is the standard `mysql < dump.sql`, then
`.\shopdb-admin.ps1 restart`. Also restore `C:\shopdb-flask\instance\` if you are
rebuilding a server - it holds uploaded branding and map blueprints, which the
database does not.
See [OPERATE-WINDOWS.md](OPERATE-WINDOWS.md).
## See also
- [DEPLOY.md](DEPLOY.md) - first-time deploy
- [INSTALL-WINDOWS.md](INSTALL-WINDOWS.md) - Windows Server install
- [UPGRADE.md](UPGRADE.md) - upgrade procedure (back up first)
- [CONFIG.md](CONFIG.md) - environment variables and Setting keys

View File

@@ -30,6 +30,81 @@ a real caller (the GE-Enforce fleet agent) to it.
HTTP 500 `Collector API key not configured` and rejects every request. An
unconfigured server never silently accepts unauthenticated data.
- A caller that sends the wrong key (or no key) gets HTTP 401 `Invalid API key`.
- In addition to the env keys, a managed API token scoped to `collector.ingest`
is accepted as a collector credential on every collector endpoint. See
"Managed collector tokens" below; env keys remain the fallback.
### Managed collector tokens (recommended)
Alongside the env keys, every collector endpoint (`/api/collector/<plugin>`,
`/pc`, `/apps`, `/heartbeat`, `/bulk`, `/status`) also accepts a **managed API
token** (PAT) scoped to the `collector.ingest` permission. The env keys stay
supported as a bootstrap/legacy fallback - nothing breaks - but a managed token
is the preferred credential because it can be minted, rotated, and revoked from
the UI (Settings > API Tokens) and its use shows up in `lastusedat` and the
audit log.
What makes a token a collector service token: it is scoped to ONLY
`collector.ingest`. That scope authorizes the collector ingest API and NOTHING
else. The existing scoped-token machinery contains it automatically - a scoped
token passes `require_permission` only for its listed permissions and is denied
on every role-gated (`require_role`) endpoint and on import mode, and
`collector.ingest` gates no normal route. So a collector token that leaks cannot
be used to read or write anything through the regular API; it can only submit
collector payloads.
Both wire transports are accepted (send whichever is convenient; GE-Enforce
sends `X-API-Key` today, so that stays ergonomic):
```
POST /api/collector/computers
X-API-Key: shopdb_pat_<40 hex>
```
or
```
POST /api/collector/computers
Authorization: Bearer shopdb_pat_<40 hex>
```
An unscoped PAT, or a PAT scoped to some other permission, is NOT a collector
token and is rejected (401) - only `collector.ingest` in the scope list counts.
A revoked or expired token is rejected (401) on both transports.
#### How to mint one (admin flow)
The simplest contained flow: an **admin** mints the token, scoped to
`collector.ingest`. Because the token is scoped, the admin-role bypass is
suspended for it, so the token is contained to the collector API even though its
owner is an admin - it cannot act with admin authority anywhere.
1. Settings > API Tokens > New Token.
2. Check **Restrict permissions**, then in the permissions grid tick only
**Submit collector payloads (fleet reporting)** (the `collector.ingest`
permission under the Collector category). Name it (e.g. `wj-fleet-collector`),
optionally set an expiry, Create.
3. Copy the `shopdb_pat_...` secret (shown once) and deploy it to the fleet the
same way as the env key: the `collectorApiKey` field in per-site
`site-config.json` (see "Delivering the API key to clients" below). The
client sends it in `X-API-Key` exactly as it sends an env key today - no
client code change.
Service identity (documented, not built): if you prefer a non-admin owner,
create a dedicated low-privilege user (e.g. `svc-collector`) whose role holds
only `collector.ingest`, plus `apitokens.create` if that user is to mint its own
token. The scope ceiling then caps any token it mints at `collector.ingest`.
The admin-minted route above is simpler and equally contained, so it is the
recommended default.
#### Rotation
Managed tokens rotate without a fleet re-image:
1. Mint a new collector token (steps above).
2. Deploy it via `site-config.json` (`collectorApiKey`) - update the one per-site
value.
3. Confirm the new token is in use: watch its `lastusedat` climb in Settings >
API Tokens (and the old token's `lastusedat` go stale).
4. Revoke the old token once traffic has moved. Revocation is immediate.
### Generic endpoint contract: `POST /api/collector/<plugin>`
@@ -135,17 +210,47 @@ a column.
| `modelnumber` | string | `Computer.modelnumberid`, scoped to the vendor when known. Model row auto-created if missing. |
| `osname` | string | `Computer.osid`. Controlled vocab: looked up in `operatingsystems`, NOT auto-created. Unknown value -> warning (row still written, `osid` left unset). |
| `installedsoftware` | array of `{name, version}` | `ComputerInstalledApp` rows for applications shopdb already tracks. Unknown app name -> warning, skipped. |
| `defaultprinter` | string | The default printer's identifier (windows name / share / hostname / port IP). Resolved to a printer asset and linked PC -> printer as a `defaultprinter` relationship. Unresolved -> warning. |
| `printers` | array of strings | All installed network printer identifiers. Each resolves to a printer asset and is linked PC -> printer as a `connectedto` relationship (the default is skipped here since it already links as `defaultprinter`). Unresolved entries -> warning. |
Schema source of truth: `get_collector_schema` in `plugins/computers/plugin.py`.
If you change the payload, change it there and re-check this table.
### PC -> printer relationship sync
When a payload carries `defaultprinter` and/or `printers`, the collector syncs
`AssetRelationship` rows so a PC page shows its printers and a printer page shows
the PCs that use it (both render in the shared Relationships card).
- Resolution: each identifier is matched, first hit wins, against the printer's
`windowsname`, `hostname`, `sharename`, its asset number/name, then any active
printer communications IP. Case-insensitive except the IP (exact). An
identifier that resolves to nothing adds a warning and is skipped; it never
fails the whole push.
- Link types: the default printer links with `defaultprinter` (directional, PC
is the source); every other reported printer links with `connectedto`
(symmetric). A printer that is both default and in `printers` links only as
the default.
- Idempotent: re-reporting the same set creates no duplicate rows (an existing
matching row is reactivated if it was archived, otherwise left as is).
- Stale-link archive: on every push, collector-created links to printers no
longer reported are set inactive. Collector-created rows are tagged in
`assetrelationships.label = 'collector:printers'`; only tagged rows are ever
archived, so links you create by hand in the UI are never touched. A payload
that omits BOTH printer keys leaves all existing printer links untouched
(report an empty `printers: []` to clear the auto links instead).
- Response: the collector response carries `printerlinkcount` and a
`printerlinks` list of `{assetid, relationshiptype}` for the links kept.
### pc-type mapping (configurable per site)
`pctype` (e.g. `gea-shopfloor-cmm`) maps to a shopdb Computer Type through
`pctypemap_<pxetype>` settings (Settings > System > "Collector PC Type Mapping").
Defaults live in `plugins/computers/pctypemap.py` and are seeded on plugin
install; edit per site in the UI. Unmapped pc-types produce a warning, not a
failure.
`pctypemap_<pxetype>` settings. The "Collector PC Types" settings page is
retired (ADR-012): pc-type-to-Computer-Type handling now lives in GE-Enforce
(each imaging PC type is a manifest scope with its own `computertypeid`). The
built-in defaults in `plugins/computers/pctypemap.py` are still seeded on plugin
install and the collector still reads them, so existing enrollment keeps
working. Unmapped pc-types produce a warning, not a failure.
### Classic api.asp field mapping (for porting the PowerShell reporter)
@@ -256,6 +361,13 @@ collector schema, and POSTs with the `X-API-Key` header over TLS 1.2. Every
field name below was checked against `get_collector_schema` in
`plugins/computers/plugin.py`.
The `X-API-Key` value can be EITHER a `COLLECTOR_API_KEY[_COMPUTERS]` env key OR
a managed token scoped to `collector.ingest` (a `shopdb_pat_...` secret; see
"Managed collector tokens"). The script is identical for both - it just carries
whatever `collectorApiKey` the site-config supplies - so switching a site from an
env key to a managed token (and rotating it) is a config change, not a script
change.
```powershell
# Send-ShopdbCollectorReport.ps1
# Reports this PC's identity to shopdb-flask via POST /api/collector/computers.
@@ -406,8 +518,32 @@ function Send-ShopdbCollectorReport {
try { $pcSubType = (Get-Content -LiteralPath 'C:\Enrollment\pc-subtype.txt' -First 1 -ErrorAction Stop).Trim() } catch {}
}
# --- Installed printers (Win32_Printer). The Default flag marks the one
# default printer. We report each printer's port name (an IP or a queue
# host for network printers) and fall back to the share/printer name, which
# the collector resolves flexibly against printer windowsname/hostname/IP. ---
$defaultPrinter = ''
$printerIds = @()
try {
$printers = Get-CimInstance -ClassName Win32_Printer -ErrorAction Stop
foreach ($p in $printers) {
if ($p.Local) { continue } # skip local-only (XPS/PDF/OneNote)
# Prefer the port name (IP or queue host); fall back to ShareName,
# then the printer Name.
$identity = $p.PortName
if (-not $identity) { $identity = $p.ShareName }
if (-not $identity) { $identity = $p.Name }
if (-not $identity) { continue }
$printerIds += $identity
if ($p.Default) { $defaultPrinter = $identity }
}
$printerIds = @($printerIds | Select-Object -Unique)
} catch { Write-CollectorLog "WARN printer read failed: $($_.Exception.Message)" }
# --- Build payload. Field names MUST match get_collector_schema exactly. ---
$payload = @{ hostname = $hostname }
if ($defaultPrinter) { $payload['defaultprinter'] = $defaultPrinter }
if ($printerIds.Count) { $payload['printers'] = $printerIds }
if ($machineNumber) { $payload['machinenumber'] = $machineNumber }
if ($pcType) { $payload['pctype'] = $pcType }
if ($pcSubType) { $payload['pcsubtype'] = $pcSubType }

View File

@@ -59,9 +59,17 @@ limit is approximate across multiple gunicorn workers).
| Variable | Required | Default | Notes |
|----------|----------|---------|-------|
| `COLLECTOR_API_KEY` | No | (empty) | Shared key for `/api/collector/*`. Endpoint fails closed (denies) when unset. Sent as the `X-API-Key` header. |
| `COLLECTOR_API_KEY` | No | (empty) | Shared key for `/api/collector/*`. Endpoint fails closed (denies) when unset and no managed token is presented. Sent as the `X-API-Key` header. |
| `COLLECTOR_API_KEY_<PLUGIN>` | No | (empty) | Per-plugin override, e.g. `COLLECTOR_API_KEY_COMPUTERS`. Checked before the shared key. |
The collector endpoints ALSO accept a managed API token (PAT) scoped to the
`collector.ingest` permission, sent in `X-API-Key` or as an
`Authorization: Bearer` token. Env keys stay supported as a bootstrap/legacy
fallback; a managed token is preferred because it is minted, rotated, and
revoked from Settings > API Tokens with `lastusedat` visibility. A
collector-scoped token is contained to the collector API and nothing else. See
`docs/COLLECTOR-INTEGRATION.md` (Managed collector tokens).
### Zabbix (printer supply monitoring)
| Variable | Required | Default | Notes |
@@ -135,6 +143,16 @@ read back through the API.
| `pc_access_domain` | `device.geaerospace.net` | Domain appended to a PC hostname for remote-access links. Blank = hostname as-is. |
| `employeeid_pattern` | `^\d{9}$` | Regex a search term must match to be treated as an employee id. Invalid regex falls back to the default and never 500s. |
| `printer_hostname_template` | `Printer-{ip}.printer.geaerospace.net` | Printer hostname template. `{ip}` is the dash-separated IP address. |
| `contact_email_domain` | `geaerospace.com` | Email domain appended to a support contact's SSO to build email (`sso@domain`) and Teams-chat links. Blank hides the contact action buttons. |
| `dualpath_single_machine` | `true` | Treat a Dualpath pair (a dual-bay machine with one controller) as a single machine in the machines list, dashboard/report counts, and the floor map (the secondary bay is hidden). The data model always keeps both bay records; detail pages stay per-bay with a sibling banner. `false` lists and counts both bays separately. |
| `site_timezone` | `America/New_York` | IANA timezone for the site. Notification start/end times are entered and displayed in this zone (not the viewer's browser zone), and daily-reset notification expiry (`expirymode=dailytime`) is computed here. Editable in Settings > Site > Localization. Public-readable so kiosks/clients can resolve it. |
Notification times are stored and served in UTC; the frontend converts to
`site_timezone` via `frontend/src/utils/datetime.js` (Intl-based, DST-safe).
Change note: notification times are now timezone-correct (stored UTC, shown in
`site_timezone`); this fixes the prior offset bug where a 2:34 PM entry displayed
as 6:34 PM.
### branding
@@ -160,12 +178,27 @@ them under `instance/branding/`.
| `qr_target_printer` | (empty) | Custom URL template for printer QR labels. Blank = link to the printer page on this instance. Placeholders: `{printerid}`, `{assetid}`, `{assetnumber}`, `{serialnumber}`, `{ip}`, `{hostname}`. |
| `qr_target_usb` | (empty) | Custom URL template for USB label QR codes. Blank = link to the USB device page. Placeholders: `{id}`, `{serialnumber}`, `{alias}`. |
| `usb_label_style` | `barcode` | USB mini-label code style: `barcode` (CODE128 of the serial) or `qr` (QR code linking to the USB QR target). |
| `qr_target_machine` | (empty) | Custom URL template for machine labels. Blank = link to the machine page. Placeholders: `{assetid}`, `{assetnumber}`, `{serialnumber}`, `{name}`, `{pluginid}`. |
| `qr_target_computer` | (empty) | Custom URL template for computer labels. Blank = link to the computer page. Placeholders: `{assetid}`, `{assetnumber}`, `{serialnumber}`, `{name}`, `{pluginid}`. |
| `qr_target_network_device` | (empty) | Custom URL template for network-device labels. Blank = link to the device page. Placeholders: `{assetid}`, `{assetnumber}`, `{serialnumber}`, `{name}`, `{pluginid}`. |
| `qr_target_measuring_tool` | (empty) | Custom URL template for measuring-tool labels. Blank = link to the tool page. Placeholders: `{assetid}`, `{assetnumber}`, `{serialnumber}`, `{name}`, `{pluginid}`, `{locationcode}`, `{locationname}`. |
| `label_default_style` | `card` | Default asset-label layout used when a label first opens: `card` (badge with image and identity) or `plain` (just the code and a caption). |
| `label_default_codetype` | `qr` | Default asset-label code type used when a label first opens: `qr` (QR code) or `barcode` (CODE128). |
| `label_default_encodes_machine` | `assetnumber` | What a machine label encodes by default. |
| `label_default_encodes_computer` | `assetpage` | What a computer label encodes by default. |
| `label_default_encodes_printer` | `assetpage` | What a printer label encodes by default. |
| `label_default_encodes_network_device` | `assetpage` | What a network-device label encodes by default. |
| `label_default_encodes_measuring_tool` | `location` | What a measuring-tool label encodes by default. Values across these five: `assetpage`, `assetnumber`, `serialnumber`, `location` (measuring tools only), or `custom`. Overridable on the label page. |
The shared asset-label generator lives at `/print/asset-label/<assettype>/<id>` (public, like the other `/print/*` pages; `assettype` is one of `machine`, `computer`, `printer`, `network_device`, `measuring_tool`, and `id` is the asset's plugin id). It can encode the asset page link, the asset number, the serial number, a custom `qr_target_<type>` template, or - for measuring tools by default - the asset's inspection location code (the leading token of the location name, e.g. `0615`). A measuring tool with no location falls back to its asset page.
The batch generator at `/print/asset-label-batch/<assettype>` (reached from the "Print Labels" button on each asset list page) lays a multi-selection of one type onto ULINE label sheets: a 6-up 3 in x 3 in format or a dense 72-up mini-label format, with a start-cell offset for reusing partial sheets. It reuses the same code-type and `label_default_encodes_<type>` defaults as the single label.
### map
| Key | Default | Notes |
|-----|---------|-------|
| `map_blueprint_light` | `/static/images/floorplan-placeholder.svg` | Floor-map blueprint (light theme). Re-upload your own in Settings > Map. |
| `map_blueprint_light` | `/static/images/floorplan-placeholder.svg` | Floor-map blueprint (light theme). Re-upload your own in Settings > Floor Map. |
| `map_blueprint_dark` | `/static/images/floorplan-placeholder.svg` | Floor-map blueprint (dark theme). |
| `map_width` | `3300` | Blueprint native width in pixels. |
| `map_height` | `2550` | Blueprint native height in pixels. |
@@ -200,7 +233,48 @@ them under `instance/branding/`.
| `smtp_use_tls` | `true` | Use TLS for the SMTP connection. |
| `smtp_from_address` | (empty) | From address for outgoing email. |
| `smtp_from_name` | `ShopDB` | From name for outgoing email. |
| `alert_recipients` | (empty) | Default alert recipients (comma-separated). |
| `alert_recipients` | (empty) | Default alert/report recipients (comma-separated). |
#### Email flows and delivery model
The mail service (`shopdb/utils/mailer.py`, stdlib `smtplib`/`ssl`/`email`
only) reads the keys above settings-first via the cached settings map, with an
environment-variable fallback (`SMTP_HOST`, `SMTP_PORT`, `SMTP_USERNAME`,
`SMTP_PASSWORD`, `SMTP_USE_TLS`, `SMTP_FROM_ADDRESS`, `SMTP_FROM_NAME`,
`SMTP_ALERT_RECIPIENTS`, `SMTP_ENABLED`) applied only when any `SMTP_*` env var
is present. When `smtp_enabled` is false or `smtp_host` is blank, every send is
a graceful no-op that logs a warning and returns without error, so an
unconfigured site never crashes. The SMTP password is never logged.
Three flows use it:
- Welcome email. When an admin creates a user (POST `/api/users`), the account
is flagged `mustchangepassword` and a best-effort welcome email is sent with
the facility name (`facility_name`), the username, the temporary password,
and the sign-in link (`site_base_url` + `/login`). Mail is best-effort: the
user is created even if the send fails (the response carries a `warning`). On
first login the API returns `mustchangepassword: true`; the frontend forces
the user through `/change-password` (POST `/api/auth/change-password`) before
the app. Changing the password clears the flag and resets lockout counters.
Set `sendwelcome: false` or `mustchangepassword: false` in the create body to
opt out.
- Test email. POST `/api/settings/test-email` (settings.edit) sends a probe to
the supplied `to` (or `alert_recipients`). The Email / SMTP settings page
"Send Test Email" button calls it and shows the result; a real SMTP error is
surfaced with the password scrubbed out.
- Alerts and report delivery (on-demand). POST `/api/reports/email`
(reports.export) takes `{subject, columns, rows, intro?, to?}` and mails the
rows as an HTML table. Recipients default to `alert_recipients` when `to` is
omitted, so the same endpoint serves both report delivery and alerts. Report
pages (Warranty, Toner) carry an "Email report" button that posts the rows
they already loaded.
There is NO scheduler in this app: sending is on-demand. To automate a
recurring send (e.g. a nightly warranty digest), point an external cron job
at `/api/reports/email` using an API token (PAT) scoped to `reports.export`.
See `docs/IMPORT-API.md` for the token model.
### audit
@@ -220,6 +294,25 @@ them under `instance/branding/`.
| `saml_auto_create_users` | `true` | Auto-create users on first SAML login. |
| `saml_admin_group` | (empty) | SAML group name that grants the admin role. |
**Personal API tokens.** Besides login JWTs and SAML, a user may create
personal API tokens (PATs) for scripts and integrations, from Settings > API
Tokens (or `POST /api/apitokens`). A PAT is sent like a JWT
(`Authorization: Bearer shopdb_pat_...`), authenticates as its owning user
across the whole API, and does not carry the hourly `JWT_ACCESS_TOKEN_EXPIRES`
limit (it never expires unless an explicit expiry is set). Only the sha256 hash
is stored; the secret is shown once at creation. This is the recommended
credential for long-running imports (see `docs/IMPORT-API.md`). There is no env
var to configure; PATs are managed entirely through the API/UI.
Creating or managing a PAT requires the `apitokens.create` permission (admins
hold it by default; grant it to other roles from Settings > Users & Roles). By
default a PAT is unscoped and acts with the full authority of its owner. A PAT
may optionally carry a scopes list (a subset of the owner's permissions, capped
at what the owner actually holds): a scoped token grants ONLY those permissions,
intersected with the owner's live permissions at use time, and suspends the
admin bypass, so it is denied on role-gated (admin-only) endpoints and on import
mode. Use an unscoped token for admin-only work and imports.
### identifiers (dynamic)
One boolean key per asset identifier per asset type, keyed
@@ -234,6 +327,20 @@ One boolean key per search domain, keyed `search_<type>_enabled` (default
`true`). Toggles whether a domain appears in global search results. The set is
generated from `SEARCH_DOMAINS` in `shopdb/core/api/settings.py`.
Search terms are matched word-wise: a multi-word query returns rows containing
EVERY word, each word anywhere in the searched fields, in any order ("CSF Roles"
matches a row with "CSF" and "Roles" in different columns). Quoting does not
force a contiguous phrase.
## Custom fields
Site-defined extra attributes per asset type (Settings > Custom Fields, table
`customfields`). Each field has a `searchable` flag (default off). When on, the
field's stored values are matched by global search and a hit routes to the
owning asset's detail page. The asset's `search_<type>_enabled` domain toggle
still applies, so a custom-field hit on a computer only shows when the computer
search domain is enabled. Inactive or non-searchable fields are never matched.
---
## See also

View File

@@ -8,11 +8,11 @@ the live code, not aspiration. The authoritative hook reference is
## Current version
The plugin contract is at **0.6.0**, declared in `shopdb/__init__.py` as
The plugin contract is at **0.13.0**, declared in `shopdb/__init__.py` as
`__contract_version__`. It is pre-1.0, which under semver means any 0.x minor
bump is allowed to break the contract, and this project has used that latitude.
The product release version (`__version__`, currently 0.5.0) is a separate
The product release version (`__version__`, currently 0.7.0) is a separate
series with its own bump rules; see [ADR-007](adr/ADR-007-product-versioning-and-releases.md).
Do not pin against it for compatibility - pin against `__contract_version__`.
@@ -25,8 +25,16 @@ Recorded in the comment block in `shopdb/__init__.py`:
| 0.3.0 | `shopdb.api` expanded to the full plugin import surface (db, cache, model bases, core models, response + pagination helpers, `employee_connection`) so plugins stop importing internal core paths | additive (minor) |
| 0.4.0 | Removed the never-implemented `get_searchable_fields` hook (search is a core concern over the asset model) and wired `get_dashboard_widgets` to a real consumer (`/api/dashboard/widgets`) | pre-1.0 contract reduction |
| 0.6.0 | Added the `get_reports` hook, consumed by `GET /api/reports` to merge plugin report cards into the Reports hub | additive optional hook (minor) |
| 0.7.0 | Added the four ADR-010 frontend-contribution hooks (`get_settings_cards`, `get_asset_panels`, `get_map_overlays`, `get_asset_presentation`), consumed by the `GET /api/pluginui/*` endpoints | additive optional hooks (minor) |
| 0.9.0 | Exposed the dualpath pair-resolution helpers on `shopdb.api` for the machines plugin | additive surface (minor) |
| 0.10.0 | Added the `get_permissions` hook so plugins declare their own RBAC permissions; the catalog is resolved dynamically from core + enabled plugins | additive optional hook (minor) |
| 0.11.0 | Added `service_token_authorized(scope)` to `shopdb.api` so a plugin's unattended endpoints (e.g. the GE-Enforce fetch API) can authorize a scoped managed service token without importing core token internals | additive surface (minor) |
| 0.12.0 | Added the mailer helpers (`send_email`, `send_alert`) to `shopdb.api` | additive surface (minor) |
| 0.13.0 | Added the `User` model to the `shopdb.api` surface | additive surface (minor) |
The source comment block documents 0.3.0, 0.4.0, and 0.6.0. Earlier points
The source comment block documents 0.3.0, 0.4.0, 0.6.0, 0.7.0, 0.9.0, 0.10.0, and 0.11.0 (its
last entry); the current `__contract_version__` 0.13.0 is ahead of the last documented comment
entry. Earlier points
(0.1.x / 0.2.x) predate that recorded rationale; `PluginMeta`'s fallback
`core_version` default of `>=0.2.0,<1.0.0` is the only remaining trace of the
0.2 baseline.
@@ -49,6 +57,8 @@ land with a new or amended ADR.
| `get_navigation_items` | Sidebar menu entries |
| `get_dashboard_widgets` | Dashboard widgets, consumed by `/api/dashboard/widgets` |
| `get_reports` | Report cards, consumed by `/api/reports` (added 0.6.0) |
| `get_permissions` | Plugin RBAC permissions, merged into the catalog for roles and token scopes (added 0.10.0) |
| Frontend-contribution hooks | `get_settings_cards`, `get_asset_panels`, `get_map_overlays`, `get_asset_presentation`, consumed by `/api/pluginui/*` (added 0.7.0, [ADR-010](adr/ADR-010-frontend-plugin-hooks.md)) |
| Collector pair | `get_collector_schema` + `apply_collector_payload` per [ADR-006](adr/ADR-006-collector-contract.md) |
| Settings helpers | `get_setting` / `set_setting`, namespaced to the plugin |
| `get_provisioning_note` | Setup-wizard transparency note for extra tables |
@@ -65,7 +75,7 @@ Known-unstable areas. Building on these means expecting rework.
| Area | Status | Reference |
|------|--------|-----------|
| Frontend hook contract | Not defined yet. There is no server-side hook for asset-detail panels, map markers, or search-result rendering. A plugin that needs custom UI still hand-edits the Vue frontend. This is the single biggest gap. [ADR-010](adr/ADR-010-frontend-plugin-hooks.md) proposes the path: data-only declarative hooks (`get_settings_cards`, `get_asset_panels`, `get_map_overlays`, `get_asset_presentation`) rendered by generic core components, with build-time glob discovery deferred for real components. | [ADR-010](adr/ADR-010-frontend-plugin-hooks.md) (PROPOSED) |
| Frontend renderers (residual) | The four data-only hooks and their `/api/pluginui/*` consumers are settled (0.7.0). The generic core renderers are landing incrementally: the settings-cards rail/landing renderer ships with 0.7.0; the asset-panel, map-overlay, and search-presentation renderers are wired opt-in per the ADR adoption plan. Real component-backed panels (bespoke charts, custom overlays) remain deferred to Option C (build-time glob discovery). | [ADR-010](adr/ADR-010-frontend-plugin-hooks.md) (ACCEPTED) |
| Per-plugin migrations | Brand new. The per-plugin Alembic engine exists and every bundled plugin now carries a chain, but the pattern has one release of production mileage, not years. | [ADR-008](adr/ADR-008-plugin-migration-ownership.md) (2026-07-10) |
| Pip distribution | Deferred to v2. External plugins install by clone / submodule / symlink; there is no entry-point discovery and no automatic update path yet. | [ADR-003](adr/ADR-003-plugin-distribution.md) |

142
docs/CSV-IMPORT.md Normal file
View File

@@ -0,0 +1,142 @@
# Loading a site's data from spreadsheets
For getting a new site's starting data in when you have a spreadsheet rather
than a source database to script against. No developer needed.
If the site *does* have a source system worth reading, the HTTP import API is
the better tool - see [IMPORT-ADOPTION.md](IMPORT-ADOPTION.md).
---
## The short version
```bash
cd C:\shopdb-flask # or your install directory
venv\Scripts\flask csv templates --out csv-templates
```
Fill in the templates. Then:
```bash
venv\Scripts\flask csv import --dir csv-templates
```
That **checks only** and changes nothing. It tells you what it would create and
what is wrong. When you are happy:
```bash
venv\Scripts\flask csv import --dir csv-templates --commit
```
---
## Write names, not numbers
This is the part that makes the difference. Every column that points at another
table accepts the **name** of the thing:
```
assetnumber,name,assettypeid,statusid,locationid
CMM-01,Zeiss Contura,Measuring Tool,Active,Gage Lab
MILL-07,Haas VF-2,Machine,Active,Bay 3
```
`assettypeid` gets `Measuring Tool`. `locationid` gets `Gage Lab`. You never
have to import a file, read back the numbers it generated, and paste them into
the next one.
The column keeps its database name so it matches the rest of the system, but
the value is whatever you actually know. Numeric ids still work if you have
them - useful when re-importing something this system exported.
Names resolve **across files in the same run**, so `assets.csv` can reference a
location that only exists because `locations.csv` was loaded moments earlier.
## Nothing is half-imported
Every row is checked before anything is written. If one row is wrong, nothing is
written at all - you fix the file and run it again. A mistake on line 400 never
leaves 399 rows loaded.
## Running it twice is safe
Each file is matched on a natural key - `assetnumber` for assets, `locationname`
for locations, and so on. Re-importing an edited file **updates** those rows
rather than creating second copies. Correcting a spreadsheet and re-running is
the expected workflow, not a mistake.
## What the errors look like
```
assets: 0 new, 0 updated, 1 problem(s)
line 4, column 'locationid': nothing in locations is named 'Bay 9'
- add it to locations.csv, or import that file first
```
Line, column, value, and what to do. Not a foreign key constraint violation.
---
## What you can import
Fourteen tables, in the order the importer handles them. You only need the ones
you have; skip any file you do not care about.
| Order | File | Matched on |
|---|---|---|
| 1 | `assetstatuses.csv` | `status` |
| 2 | `assettypes.csv` | `assettype` |
| 3 | `locationtypes.csv` | `locationtype` |
| 4 | `modeltypes.csv` | `modeltype` |
| 5 | `computertypes.csv` | `computertype` |
| 6 | `machinetypes.csv` | `machinetype` |
| 7 | `businessunits.csv` | `businessunit` |
| 8 | `locations.csv` | `locationname` |
| 9 | `vendors.csv` | `vendor` |
| 10 | `models.csv` | `modelnumber` |
| 11 | `operatingsystems.csv` | `osname` |
| 12 | `assets.csv` | `assetnumber` |
| 13 | `computers.csv` | `assetid` |
| 14 | `machines.csv` | `assetid` |
`--dir` handles the order for you. Use `--file` with `--table` for one file.
**User accounts are deliberately not importable.** Passwords do not belong in a
spreadsheet, in either direction. Create the first administrator through the
first-run page and the rest in the application.
## The templates are generated, not maintained
`flask csv templates` builds them from the live database schema each time. Every
column offered exists; every required one is marked; every foreign key says
which file it refers to.
This matters because the alternative does not work. A hand-written template set
was tried, and it had invented columns on seven of eleven tables and named a
table that does not exist - while looking entirely plausible. Templates that are
generated cannot drift from the schema, and a test fails the build if they ever
do.
## Editing the files
- **UTF-8**, no BOM. Excel: "CSV UTF-8 (Comma delimited)".
- Lines starting with `#` are ignored, so the notes and the example row in each
template can stay where they are.
- Booleans are `1` or `0`.
- Dates are `YYYY-MM-DD` (`2026-08-04`). `YYYY-MM-DD HH:MM:SS` also works, as do
`DD/MM/YYYY` and `MM/DD/YYYY`.
- Leave a cell **empty** for "no value". Not `NULL`, not `N/A`.
- Quote anything containing a comma: `"Bay 3, North"`.
## If it will not run
**"the 'assets' table does not exist in this database"** - the schema has not
been created. Run `flask db upgrade` first, and check `DATABASE_URL` points at
the site you meant.
**"unknown column(s): ..."** - a column that does not exist, usually from an
older template. Regenerate with `flask csv templates`; the message lists what
the table does accept.
**"required column(s) missing: ..."** - a column that must be present has been
deleted from the header. Regenerate and copy your data across.

149
docs/DEPLOY-AIRGAP.md Normal file
View File

@@ -0,0 +1,149 @@
# Air-gapped Docker deploy
For a site with **no internet**. The site cannot `pip install`, `npm ci`, or
`docker pull`, so nothing is built or pulled there. You build a fully
self-contained image on a **connected** box, ship it as one tarball, and
`docker load` + run it at the site.
This is the counterpart to `docker-compose.yml` (the connected/build template).
The air-gapped stack uses `docker-compose.airgap.yml`, which differs in three
ways that matter:
- **`image:` not `build: .`** - the site runs a loaded image; it never builds.
- **No `./plugins` bind mount** - the image already carries every plugin baked
in. Binding a host `./plugins` (which does not exist at an image-only site)
would mask the baked plugins with an empty dir and load **zero** plugins.
- **A one-shot `migrate` service** runs `db upgrade` + `plugin upgrade-all` +
seed before `api` starts, so `up -d` alone brings up a working site.
---
## 1. Build the bundle (connected box)
On a box with clean access to Docker Hub + PyPI + npm, from the repo root:
```powershell
pwsh scripts/build-offline-bundle.ps1 -Version 0.7.0
```
This builds `shopdb-flask:0.7.0` (frontend + all Python deps baked in), pulls
`mysql:8.0`, and writes:
- `shopdb-stack-0.7.0.tar.gz` - both images in one archive
- `shopdb-stack-0.7.0.tar.gz.sha256` - checksum to verify after transfer
The image is self-contained: **no pip/npm/registry access is needed at the
site.**
### Building behind Zscaler
If the build box is itself behind GE Zscaler, the in-build `pip`/`npm` may fail
with `CERTIFICATE_VERIFY_FAILED` - the build container has its own cert store
and does not trust GE's re-signing root (host `PIP_CERT` / `NODE_EXTRA_CA_CERTS`
do NOT carry into a `docker build`). Easiest: build from a box with clean
internet (home, cloud, or CI). Otherwise the corp root CA must be trusted
**inside** the build (a Dockerfile change to accept the CA - not wired today;
see docs/DEVELOPMENT-SETUP.md section 0b for the all-roots PEM bundle).
---
## 2. Transfer + verify
Carry `shopdb-stack-0.7.0.tar.gz` (+ the `.sha256`), plus
`docker-compose.airgap.yml` and `.env.example`, to the site on approved media.
Verify the archive survived the trip:
```powershell
# PowerShell
(Get-FileHash shopdb-stack-0.7.0.tar.gz -Algorithm SHA256).Hash.ToLower()
# compare against the .sha256 file
```
```bash
# Linux site
sha256sum -c shopdb-stack-0.7.0.tar.gz.sha256
```
---
## 3. Load + run (air-gapped site)
```bash
# 1) Load both images into the local Docker.
docker load -i shopdb-stack-0.7.0.tar.gz
docker image ls | grep -E 'shopdb-flask|mysql' # confirm both present
# 2) Configure the site.
cp .env.example .env
# Edit .env - REQUIRED:
# IMAGE_TAG=0.7.0 # MUST match the loaded image tag
# MYSQL_ROOT_PASSWORD=... # strong, unique
# MYSQL_PASSWORD=... # strong, unique (the app's db user)
# SECRET_KEY=... # 32+ random bytes
# JWT_SECRET_KEY=... # 32+ random bytes, different from SECRET_KEY
# CORS_ORIGINS=https://shopdb.site.example # the site's browser origin(s)
# Optional: API_PORT (default 5001), LOG_LEVEL, ZABBIX_URL/ZABBIX_TOKEN.
# 3) Bring it up. The migrate one-shot runs the schema + seed, then api starts.
docker compose -f docker-compose.airgap.yml up -d
# 4) Watch the one-shot finish (it exits 0 when the schema + seed are done).
docker compose -f docker-compose.airgap.yml logs -f migrate
```
`IMAGE_TAG` in `.env` must equal the loaded tag (`0.7.0` here); otherwise compose
looks for an image that was never loaded and `api`/`migrate` will not start.
---
## 4. Create the first admin
Seeding creates permissions, settings, and reference data, but not a login. Make
one admin (choose the password; it is not automatable):
```bash
docker compose -f docker-compose.airgap.yml exec api \
flask seed admin --username <username> --email <email> --password <password>
# Omit --password to have a strong one generated and printed once.
```
---
## 5. Verify
```bash
docker compose -f docker-compose.airgap.yml ps # db + api "running", migrate "exited (0)"
docker compose -f docker-compose.airgap.yml logs api # gunicorn started, no tracebacks
curl -sf http://localhost:${API_PORT:-5001}/api/dashboard/health # or browse the site origin
```
Then log in at the site origin with the admin created in step 4.
---
## Upgrading to a new version
Build a new bundle on the connected box (`-Version 0.8.0`), transfer, then at the
site:
```bash
docker load -i shopdb-stack-0.8.0.tar.gz
# set IMAGE_TAG=0.8.0 in .env
docker compose -f docker-compose.airgap.yml up -d
```
The `migrate` one-shot re-runs `db upgrade` + `plugin upgrade-all` + seed (all
idempotent) against the existing data before the new `api` starts. Back up first
(see docs/BACKUP-RESTORE.md); the `db_data` volume persists across upgrades.
---
## Troubleshooting
| Symptom | Cause / fix |
| --- | --- |
| `service api is not running` / `manifest ... not found` | The image was not loaded, or `IMAGE_TAG` in `.env` does not match a loaded image. `docker image ls`, fix `IMAGE_TAG`. Also: never use `docker-compose.yml` here - its `build: .` needs internet. |
| Site loads but **no plugins / empty nav** | You used the wrong compose file. `docker-compose.yml` bind-mounts `./plugins` (absent here) over the baked plugins. Use `docker-compose.airgap.yml`. |
| `api` never starts, `migrate` shows an error | Read `logs migrate`. A DB-connection error means `db` is not healthy yet (`logs db`) or `MYSQL_PASSWORD` in `.env` differs from what the `db` volume was first initialised with. A fresh site with a stale `db_data` volume needs the volume removed (`docker compose ... down -v` - DESTROYS data). |
| `SECRET_KEY must be set` (and similar) on `up` | A required `.env` var is empty. Fill every REQUIRED key in step 3. |
| Build fails on the connected box at `pip`/`npm` | Zscaler cert - see "Building behind Zscaler" above. |

View File

@@ -1,5 +1,13 @@
# Deploy shopdb-flask to Windows IIS (MySQL 5.6)
> **Not the route for a new site.** Sister sites install from the Windows
> installer - one `.exe`, no manual IIS work: **[INSTALL-WINDOWS.md](INSTALL-WINDOWS.md)**.
>
> This is the **manual** procedure for the West Jefferson server, which was built
> by hand against its existing MySQL 5.6 and predates the installer. Keep it for
> that box.
Runbook for standing up a single-site instance on the production Windows Server
that already runs the classic ASP shopdb, using IIS + HttpPlatformHandler +
waitress, against the existing MySQL 5.6. This is the test-instance path; keep
@@ -13,13 +21,13 @@ physical path must be `APP_ROOT` (where `wsgi.py` lives).
## 0. Prerequisites on the box
- Python 3.12 (same minor as dev). `py -3.12 --version` to confirm.
- Python 3.14 (same minor as dev and CI). `py -3.14 --version` to confirm.
- IIS with the **HttpPlatformHandler** module:
https://www.iis.net/downloads/microsoft/httpplatformhandler
- **URL Rewrite** module (only for the optional real-client-IP rule).
- Network access to the MySQL 5.6 server.
- If the box is air-gapped, you cannot `pip install` live. On the dev box run
`pip download -r requirements.txt waitress -d wheels\` (on a matching
`pip download -r requirements.txt -d wheels\` (on a matching
Windows/Python target, or use `--platform` wheels), copy `wheels\` over, and
install with `pip install --no-index --find-links wheels\ ...`.
@@ -39,15 +47,14 @@ Ship `frontend/dist` with the code (Node is not needed on the prod box).
```powershell
cd C:\shopdb-flask
py -3.12 -m venv venv
py -3.14 -m venv venv
venv\Scripts\python -m pip install --upgrade pip
venv\Scripts\pip install -r requirements.txt
venv\Scripts\pip install waitress
```
The DB driver is `pymysql` (pure Python) so no C compiler / MySQL client libs
are needed. `waitress` is the WSGI server (installed separately, same as the
Docker image installs gunicorn separately).
are needed. `waitress` is the WSGI server and ships in `requirements.txt`
(unlike gunicorn, which the Docker image installs separately).
## 3. Prepare MySQL 5.6 (the utf8mb4 gotcha)
@@ -106,28 +113,54 @@ $env:FLASK_APP="shopdb"
venv\Scripts\flask db upgrade
venv\Scripts\flask seed reference-data
# Enable the plugins this site tracks (registry lives in the gitignored
# instance/plugins.json, so a fresh box starts with none enabled):
# Install the plugins this site tracks (registry lives in the gitignored
# instance/plugins.json, so a fresh box starts with none installed). Run
# `flask plugin list` to see the current bundled set; the 13 bundled plugins are
# computers, employees, geenforce, knowledgebase, machines, measuringtools,
# network, notifications, printedparts, printers, slides, usb, warranty. Install
# only the ones this site wants:
venv\Scripts\flask plugin list
venv\Scripts\flask plugin install machines
venv\Scripts\flask plugin install printers
venv\Scripts\flask plugin install computers
venv\Scripts\flask plugin install equipment
venv\Scripts\flask plugin install network
venv\Scripts\flask plugin install notifications
venv\Scripts\flask plugin install printers
venv\Scripts\flask plugin install usb
venv\Scripts\flask plugin install knowledgebase
venv\Scripts\flask plugin install slides
venv\Scripts\flask plugin install employees
venv\Scripts\flask plugin upgrade-all
# First admin (password is generated and printed once):
venv\Scripts\flask seed admin --username admin --email admin@yourfacility.example.com
```
Cleaner than a hand list: declare the set once in a site profile and apply it:
```powershell
venv\Scripts\flask plugin apply-profile deploy\site-profile.json # install + enable the chosen set, in dependency order
venv\Scripts\flask plugin upgrade-all
venv\Scripts\flask plugin prune-schema --yes --force # FIRST PROVISIONING ONLY - see the warning below
```
(Alternatively copy the dev box's `instance/plugins.json` to `APP_ROOT\instance\`
to reproduce the exact enabled set, then just run `flask plugin upgrade-all`.)
to reproduce the exact set, then just run `flask plugin upgrade-all`.)
> **`prune-schema --force` is for first provisioning only.** It drops the tables
> of plugins this site did not install *even when they hold rows*. On a site that
> already has data, run `flask plugin prune-schema` with no flags first and read
> what it says it would drop. Re-running with `--force` after a feature has been
> used deletes that feature's records with no prompt and no backup.
## 6. Create the IIS site + web.config
This describes the own-site method (the app gets its own IIS site + port). To
mount the app at a subpath under an existing site instead (e.g.
`https://<host>/ops/` sharing the classic site's binding and cert), see
**docs/INSTALL-WINDOWS-IIS.md section 7b**: same web.config, but the site is a
`New-WebApplication` under the parent, `MOUNT_PATH=/ops` is set (web.config or
`.env`), and the frontend is built with `VITE_BASE_PATH=/ops/`.
1. In IIS Manager, add a new **Site** (separate from the classic ASP site):
- Physical path: `APP_ROOT`
- Binding: a free port or a dedicated hostname (e.g. `https` 443 with the

View File

@@ -7,12 +7,12 @@ shopdb-flask is single-tenant per ADR-004. Each adopting facility runs its own s
- Docker 24+ and Docker Compose v2 (or equivalent container runtime)
- A reverse proxy with TLS termination (nginx, traefik, Caddy, GE corporate LB) -- the framework does not terminate TLS itself
- A MySQL backup destination (offsite recommended)
- Access to the GE Aerospace Gitea or a clone of the repo
- Access to the internal GE Aerospace git server, or a clone of the repo
## Step 1: Clone and configure
```bash
git clone https://gitea.proudtech.net/ge-aerospace/shopdb-flask.git
git clone <internal-git-server>/ge-aerospace/shopdb-flask.git
cd shopdb-flask
cp .env.example .env
```
@@ -73,6 +73,24 @@ any plugin-specific migrations added after the ownership cutover. Both commands
are idempotent, so re-running them is safe. See ADR-008 for why plugin schema
splits into per-plugin chains from the cutover forward.
**Lean sites (ADR-014):** the core chain creates every bundled plugin's tables,
so a site that ships only some plugins still has the others' (empty) tables. To
carry only core + chosen-plugin tables, prune the rest once, at initial
provisioning, after the two commands above:
```bash
docker compose exec api flask plugin prune-schema # dry-run, review
docker compose exec api flask plugin prune-schema --yes --force
```
It drops the tables of every plugin not installed on this site. `--force` is
needed because the core chain seeds a few plugin reference tables (default
access protocols, etc.); at first provisioning those hold only seeded defaults,
before any site data. It refuses to drop a table that holds rows without
`--force`, so it is safe to leave out of routine upgrades - run it only when
provisioning a lean site or after deliberately removing a plugin. Installing a
pruned plugin later recreates its tables automatically.
**Charset:** the schema is utf8mb4 (`utf8mb4_unicode_ci`). The docker-compose `db` service sets `--character-set-server=utf8mb4`, so the auto-created `shopdb_flask` database is utf8mb4. If you point at an external MySQL instead of the bundled container, create the database as utf8mb4 first, or it inherits the server default (often latin1) and the schema silently drifts:
```sql
@@ -98,13 +116,15 @@ docker compose exec api flask seed reference-data
- `seed settings` - writes the default Setting rows (branding, ServiceNow
integration, floor-map placeholders, search toggles, site identity). A site
overrides these later in Settings or the setup wizard.
- `seed reference-data` - creates default `Vendor`, `Location`, `BusinessUnit`,
`OperatingSystem`, `AssetStatus`, `RelationshipType` rows seeded with the
platform contract values (`partof`, `controls`, `connectedto`).
- `seed reference-data` - creates default `ModelType`, `AssetStatus`,
`LocationType`, `CommunicationType`, `OperatingSystem`, `RelationshipType` rows
seeded with the platform contract values (`partof`, `controls`, `connectedto`).
(`Vendor`, `Location`, and `BusinessUnit` are not seeded here; they come from
`seed demo`.)
## Step 5: Pick plugins to enable
The image bundles eleven plugins (computers, employees, knowledgebase, machines, measuringtools, network, notifications, printers, slides, usb, warranty). Only enabled plugins are loaded.
The image bundles thirteen plugins (computers, employees, geenforce, knowledgebase, machines, measuringtools, network, notifications, printedparts, printers, slides, usb, warranty). Only enabled plugins are loaded.
```bash
docker compose exec api flask plugin list

301
docs/DEVELOPMENT-SETUP.md Normal file
View File

@@ -0,0 +1,301 @@
# Development setup: from clone to first change
The goal of this page: a new developer clones the repo and has a working dev
site plus a change they can see in the browser, in one sitting. Reference
material lives elsewhere - naming rules in `CONTRIBUTING.md`, every config
variable in the CONFIG guide, plugin authoring in the PLUGIN docs - this is
just the on-ramp.
**Most developers here are on Windows in VS Code** - commands below are
PowerShell first, with the bash equivalent in a comment where they differ.
Install **Git for Windows** (it ships Git Bash, which VS Code and the git
hooks use to run the shell-based naming check) and **VS Code** with the
extensions this repo recommends (you'll be prompted - section 2c).
Two ways to run it. **Docker** (Docker Desktop on Windows) is the fastest to a
working site. **Manual (venv + Node)** is the daily driver - frontend
hot-reloads, backend restarts on save. Do Docker once to confirm the box is
sane, then use manual for day-to-day work.
---
## 0. Prerequisites
| Need | Version | Check |
| --- | --- | --- |
| Python | 3.14 (64-bit) - matches CI, the container image and the Windows installer wheelhouse | `python --version` |
| Node.js | 18+ | `node --version` |
| MySQL | 8.0 (or Docker, below) | `mysql --version` |
| Git | any recent | `git --version` |
On Windows, install all of them with winget (accept each license, then
reopen the terminal so PATH updates):
```powershell
winget install Git.Git # includes Git Bash (naming hook needs it)
winget install Python.Python.3.14
winget install OpenJS.NodeJS.LTS # LTS; may install v24, fine for this SPA
winget install Microsoft.VisualStudioCode
winget install Oracle.MySQL # or Docker.DockerDesktop for the DB
```
CI runs Node 20; the LTS package may be newer. This Vite/Vue frontend builds
identically across 20-24, so it does not matter. To pin exactly:
`winget install CoreyButler.NVMforWindows` then `nvm install 20; nvm use 20`.
---
## 0b. Corp network (SSL cert) - if you are behind a GE/Zscaler proxy
A proxy that inspects HTTPS (Zscaler on GE PCs) re-signs every connection
with a corporate root CA. `git`, `npm`, `pip`, and Node each keep their own
trust store and do not trust that CA by default, so downloads fail:
| Tool | Symptom |
| --- | --- |
| npm | `UNABLE_TO_GET_ISSUER_CERT_LOCALLY` |
| git | `SSL certificate problem: unable to get local issuer certificate` |
| pip | `SSLError` / `CERTIFICATE_VERIFY_FAILED` |
Fix once - export the corp root CA, point every tool at it. PowerShell
mangles multi-line pastes, so each step below is **one physical line**: paste
it, press Enter, then the next. Do not paste both at once.
```powershell
# 1) Bundle EVERY trusted root into one PEM (one line). Guessing which single cert is the proxy's is fragile; bundling all always includes it.
$sb = New-Object System.Text.StringBuilder; Get-ChildItem Cert:\LocalMachine\Root | ForEach-Object { [void]$sb.AppendLine("-----BEGIN CERTIFICATE-----"); [void]$sb.AppendLine([Convert]::ToBase64String($_.RawData,'InsertLineBreaks')); [void]$sb.AppendLine("-----END CERTIFICATE-----") }; [IO.File]::WriteAllText("$HOME\corp-root-ca.pem", $sb.ToString())
```
Confirm it has many certs (dozens, not 1):
`(Select-String "BEGIN CERTIFICATE" $HOME\corp-root-ca.pem).Count`
```powershell
# 2) Point every tool at it (one line, persistent). NODE_EXTRA_CA_CERTS also fixes Vite / npm run dev.
git config --global http.sslCAInfo "$HOME\corp-root-ca.pem"; npm config set cafile "$HOME\corp-root-ca.pem"; setx NODE_EXTRA_CA_CERTS "$HOME\corp-root-ca.pem"; setx PIP_CERT "$HOME\corp-root-ca.pem"
```
Reopen the terminal so `setx` takes effect. Quick unblock if you cannot
export right now (skips verification - use briefly, then set back):
`npm config set strict-ssl false`, `git config --global http.sslVerify false`.
---
## 1. Get the code
```powershell
git clone https://github.com/ge-aero/shopdb-flask.git
cd shopdb-flask
```
Never work on `main`. Branch for your change:
```powershell
git checkout -b feat/<short-description>
```
---
## 2a. Fast path - Docker (a working site in one command)
```powershell
copy .env.example .env
# Edit .env: set SECRET_KEY, JWT_SECRET_KEY, and the MYSQL_* passwords.
# Generate a secret: python -c "import secrets;print(secrets.token_urlsafe(64))"
docker compose up -d --build # MySQL + the app (frontend built in-image)
# Schema + platform data (idempotent, safe to re-run):
docker compose exec api flask db upgrade
docker compose exec api flask plugin upgrade-all
docker compose exec api flask seed permissions
docker compose exec api flask seed settings
docker compose exec api flask seed reference-data
docker compose exec api flask seed admin --username admin --email you@example.com
docker compose exec api flask seed demo # OPTIONAL: sample data (undo: flask seed demo-clear)
```
The app is on the port the compose file maps (see `docker-compose.yml`). Good
for a smoke test; for active development use the manual path so the frontend
hot-reloads.
---
## 2b. Manual path - venv + Node (the daily driver)
### Database
Either point at an existing MySQL 8, or bring one up with just the db service
from compose:
```powershell
docker compose up -d db # MySQL on 127.0.0.1:3306
```
Create the database + app user (skip if compose already did via env):
```sql
CREATE DATABASE shopdb_flask CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'shopdb'@'%' IDENTIFIED BY 'devpassword';
GRANT ALL PRIVILEGES ON shopdb_flask.* TO 'shopdb'@'%';
FLUSH PRIVILEGES;
```
### Backend
```powershell
python -m venv venv
venv\Scripts\Activate.ps1 # bash/mac: source venv/bin/activate
# If PowerShell blocks the activate script (execution policy), run once:
# Set-ExecutionPolicy -Scope CurrentUser RemoteSigned
pip install -r requirements-dev.txt
copy .env.example .env # bash/mac: cp .env.example .env
# Edit .env - set SECRET_KEY, JWT_SECRET_KEY, and
# DATABASE_URL=mysql+pymysql://shopdb:devpassword@127.0.0.1:3306/shopdb_flask?charset=utf8mb4
# CORS_ORIGINS=http://localhost:5173
$env:FLASK_APP = "shopdb" # bash/mac: export FLASK_APP=shopdb
flask db upgrade # core schema
flask plugin upgrade-all # per-plugin schema (ADR-008)
flask seed permissions
flask seed settings
flask seed reference-data
flask seed admin --username admin --email you@example.com # password printed once
flask seed demo # OPTIONAL: ~25 sample assets across plugins + printed parts (undo: flask seed demo-clear)
```
Enable the plugins you want visible (they install on a fresh box; some ship
disabled). To turn on everything for development:
PowerShell:
```powershell
foreach ($p in "computers","employees","machines","measuringtools","network",
"notifications","printers","slides","usb","warranty",
"knowledgebase","geenforce","printedparts") {
flask plugin install $p; flask plugin enable $p
}
```
(bash/mac: a `for p in ...; do flask plugin install "$p"; ...; done` loop.)
Run the backend ON PORT 5001 - the frontend dev server proxies `/api` and
`/static` there (a bare `flask run` uses 5000 and nothing will load):
```powershell
flask run --port 5001
```
**Leave this running.** `flask run` does not return to a prompt - that is
correct, not a hang. The server holds this terminal until you stop it. Do
NOT press Ctrl+C to move on; that kills the backend. Open the frontend in a
separate terminal (next section) and leave this one alone. Ctrl+C only when
you are done for the day.
### Frontend (a second terminal - leave the backend running)
```powershell
cd frontend
npm install
npm run dev # http://localhost:5173
```
Open http://localhost:5173, log in as `admin` with the printed password.
> Convenience: instead of two terminals you can run both under a process
> manager (pm2, honcho, foreman). Keep the backend on 5001.
---
## 2c. VS Code (turnkey)
The repo ships shared VS Code config in `.vscode/` (personal `settings.json`
stays git-ignored):
- **Recommended extensions** - on first open VS Code offers to install them
(Python + Pylance, Vue Volar, ESLint, Docker). Accept.
- **Run the dev site** - Command Palette > "Tasks: Run Task" >
**Dev site (backend + frontend)** starts both servers in parallel (backend
on 5001, frontend on 5173). Individual tasks exist too.
- **Debug the backend** - the Run panel's **Flask API (:5001)** config runs
the app under the debugger (breakpoints in routes/services, full
stepping); **Pytest (current file)** debugs the open test file.
- **The CI gate** - task **Check: naming + tests + build** runs the same
three checks CI runs, before you commit.
Prerequisite: the venv and `npm install` from 2b must be done first (the
tasks call `venv/` and `frontend/node_modules`).
---
## 3. The development loop
1. Make a change. Backend: `flask run` auto-reloads. Frontend: Vite hot-reloads.
2. Before committing, run the three gates. Easiest: in VS Code, Command
Palette > "Tasks: Run Task" > **Check: naming + tests + build**. By hand
in PowerShell:
```powershell
venv\Scripts\python -m pytest tests/ -q # backend
cd frontend; npx vitest run; npm run build; cd ..
bash scripts/check-naming-and-style.sh # naming - runs via Git Bash
```
There is NO auto-installed git hook - you run these yourself (or the
VS Code task). CI runs all three on every push and pull request
(`.github/workflows/ci.yml` on GitHub Actions; the same gate runs on the
internal server) and fails the build on a bad name, so nothing bad
reaches `main` - running them locally just saves the round trip. The
naming check is a shell script, so that one line needs Git Bash
(installed with Git for Windows).
Want it automatic? The repo ships a hook; enable it once per clone:
```powershell
git config core.hooksPath .githooks
```
Now every `git commit` runs the naming check first (Git for Windows
executes the hook under its bundled bash) and blocks the commit if a name
is wrong. Purely local convenience; CI is the real backstop.
3. Commit in small, working steps. Subject: short, present tense, plain
English; body says WHY. Read `CONTRIBUTING.md` before naming anything - the
naming hook will reject snake_case DB columns, banned shorthand, and
non-ASCII.
Seeing a change in the real app (not just tests) is the bar for "done" -
drive the actual flow in the browser.
---
## 4. Your first change (suggested)
Add a field to an existing list page, or better, build a plugin end to end:
`docs/PLUGIN-LAB-PRINTEDPARTS.md` is a literal type-along that constructs the
3D-printed-parts plugin from scratch, with the finished code on branch
`feat/printedparts-plugin` (tags `lab-stage-01`..`lab-stage-17`) as the
answer key. It touches every hook the framework has.
---
## 5. Contributing back
```powershell
git push -u origin feat/<short-description>
```
Open a Pull Request against `main` on GitHub. Describe what changed, any
plugin hooks implemented, and any contract additions (those need a version
bump + `docs/PLUGIN-HOOKS.md` update in the same PR). See the contributor
section of the plugin lab for the full review checklist.
---
## Common setup problems
| Symptom | Cause / fix |
| --- | --- |
| Frontend loads but every API call fails / CORS error | backend not on 5001 (`flask run --port 5001`), or `CORS_ORIGINS` missing `http://localhost:5173`. |
| App refuses to boot in production config | a required `.env` var (`SECRET_KEY`, `JWT_SECRET_KEY`, `DATABASE_URL`, `CORS_ORIGINS`) missing or a dev default. |
| `flask db upgrade` error 1071 (key too long) | MySQL 5.6 without the `innodb_large_prefix`/Barracuda flags; use MySQL 8 for dev. |
| Nav missing Machines/PCs/... | plugins not installed/enabled (step 2b), or the backend not restarted after enabling. |
| "No time zone found with key America/New_York" | `tzdata` not installed - `pip install -r requirements.txt` includes it. |
| npm/git/pip SSL error (`UNABLE_TO_GET_ISSUER_CERT_LOCALLY`, `unable to get local issuer certificate`) | corp proxy (Zscaler) intercepts HTTPS - point each tool at the corp root CA. See section 0b. |
| Naming hook rejects a commit | you used snake_case on a DB-mirrored field or a banned acronym - see `CONTRIBUTING.md`. |
| Plugin toggle throws an internal error | app cannot write `instance/` (the plugin registry lives there) - fix directory permissions. |

176
docs/GE-ENFORCE-CLIENT.md Normal file
View File

@@ -0,0 +1,176 @@
# GE-Enforce client integration (shopdb manifest source + reporting)
This is the client-side contract for the GE-Enforce manifest-store plugin: how a
PC sources its install manifest from shopdb instead of a share file, and how it
reports its enforcement result back. It pairs with the plugin proposal in
`docs/proposals/ge-enforce-plugin.md`.
The reference kit lives in `plugins/geenforce/client/`:
- `ShopdbEnforceClient.psm1` - fetch (with ETag + last-known-good cache),
shadow compare, and report helpers.
- `Invoke-ShopdbEnforce.ps1` - a reference orchestrator that fetches a manifest,
runs the UNCHANGED engine against it, and reports the result.
These are site-neutral references, not the live dispatcher. A site adapts them
into its GE-Enforce.ps1 flow. The engine (`Install-FromManifest.ps1`),
detection, self-heal, and SMB payload resolution are untouched - only the source
of the manifest JSON moves, plus a result report.
## What does NOT change
- The engine and its four filters, all detection methods, self-heal, marker
files, and SMB payload staging.
- Payload transport for `smb` rows: the client still mounts the share and
resolves `apps/...` paths exactly as today. Only the manifest JSON source moves.
- The fail-safe posture: any error exits 0. A PC is never blocked or broken
because shopdb is unreachable.
## Configuration
Registry (provisioned by Azure DSC, same channel as the SFLD credentials):
```
HKLM:\SOFTWARE\GE\ShopDB
BaseUrl https://shopdb.<site>.geaerospace.net
ApiToken <a geenforce.fetch (+ geenforce.report) managed service token>
```
Mint the token in shopdb: Settings > API Tokens, scopes `geenforce.fetch` and
`geenforce.report`. It is a service token (owner must hold those permissions).
## Fetch contract
```
GET /api/geenforce/manifest?pctype=<scope>[&phase=runtime]
X-API-Key: <token>
If-None-Match: <cached ETag> (optional)
```
- `200` - body is the full published manifest JSON for the scope (fat client:
the engine filters locally, exactly as today). Response headers carry `ETag`
and `X-Manifest-Version`. Cache the body + ETag + version.
- `304` - your cached copy is current; use it.
- `404` - no such scope, or the scope has no published version yet.
- Network failure - enforce from the last-known-good cached manifest (the kit
does this automatically) and log a warning.
The served manifest is always the current PUBLISHED snapshot, never a live draft
being edited in shopdb, so a half-finished edit can never reach a PC.
## Report contract
Each enforcement cycle, POST the result (best-effort; a failed report never
fails the cycle):
```
POST /api/geenforce/report
X-API-Key: <token>
Content-Type: application/json
{
"hostname": "WJCMM01",
"scopename": "gea-shopfloor-cmm",
"appliedversion": 3, // the published version you actually ran
"enforcerversion": "2.6",
"counts": { "installed": 1, "skipped": 3, "failed": 0, "filtered": 2 },
"results": [
{ "name": "PC-DMIS 2019 R2", "action": "installed", "selfhealed": true },
{ "name": "Protect Viewer", "action": "skipped" },
{ "name": "eDNC", "action": "failed", "exitcode": 1603,
"message": "MSI 1603" }
]
}
```
- `appliedversion` lets shopdb show which PCs received the latest manifest
(`receivedlatest` in the fleet view).
- `action` per entry: `installed` (fired - a self-heal when it should already be
present), `skipped` (detected present), `failed`, `filtered`. `selfhealed`
marks a drift correction.
- shopdb keeps the latest report per (hostname, scope, phase) plus history, and
surfaces it under GE-Enforce > Enforcement Reports.
The engine already computes these counts (`installed/skipped/failed/pcFiltered`
at the end of its main loop) and knows each entry's action; shape them into the
`results` list at the call site (`New-ShopdbReport` in the kit takes a summary
with `Installed/Skipped/Failed/Filtered` + a `Results` list).
The engine emits per-entry outcomes in PascalCase (`Name/Action/SelfHealed/
ExitCode/Message`); `New-ShopdbReport` maps every per-entry key down to the
lowercase names above (`name/action/selfhealed/exitcode/message`) before POST,
so the entire wire contract shopdb reads is lowercase. `ConvertTo-ShopdbSummary`
first normalizes whatever the engine returns (a well-formed summary, a bare
return code, `$null`, or several emitted objects) into the count/results shape
`New-ShopdbReport` expects, so a not-yet-compliant engine still produces a valid
report.
## Common-scope inheritance (opt-in, OFF by default)
By default a PC enforces its `-Scope` ALONE. Pass `-IncludeCommon` to also fetch
the fleet-wide `common` scope and merge it on top, mirroring the real
GE-Enforce.ps1 (which applies `common\manifest.json` first, then the pctype's).
When enabled, `Invoke-ShopdbEnforce.ps1` fetches `common` in addition to
`-Scope` and merges it via `Merge-ShopdbManifests`:
- entries are keyed by `Name` (case-insensitive);
- common's unique entries come first, then all pctype entries (common enforces
ahead of the pctype, as on the share);
- on a `Name` conflict the pctype entry wins (its override replaces common's).
Common is fetched over the same fail-safe path (ETag + last-known-good cache).
`-CommonScope <name>` inherits a different fleet scope; a run whose `-Scope`
already is the common scope does not merge itself.
Displays do NOT use this: the `gea-shopfloor-display` scope is self-sufficient,
so the display scheduled task omits `-IncludeCommon`. Common-merge exists for a
future share-less non-display PC that genuinely needs the fleet-wide entries
(which would first require repackaging common's SMB payloads as http/inline).
The three display subtypes (Dashboard, Lobby, 3D Print Room), selected by
`C:\Enrollment\display-type.txt`, carry their shared policy inside the display
scope itself, not via common.
## Fail-safe is observable, not silent
Any error still exits 0 - a bad web app never blocks or breaks a PC. But a fresh
display with an EMPTY cache (first boot, shopdb unreachable or the token
rejected with 401 / a TLS-trust failure) would otherwise enforce nothing
*silently*. When no manifest and no cache are available, the kit:
- writes a Windows Application event-log entry (source `ShopdbEnforce`, event id
1001, type Error) naming the scope and the reason (HTTP status or transport
error), and
- fires a best-effort report ping (counts `failed: 1`, a single
`(manifest-fetch)` result carrying the reason) so the miss surfaces under
GE-Enforce > Enforcement Reports.
The cycle still exits 0; the signal just makes the no-enforcement state visible.
## Cutover (safe, staged)
1. **Configure** the registry values on a canary PC; mint the token.
2. **Shadow mode**: run `Invoke-ShopdbEnforce.ps1 -ShadowMode -ShareManifestPath
<current share manifest>`. It installs from the SHARE (no behavior change),
fetches the shopdb manifest, logs any diff, and reports. Watch for zero diffs
across one PC of every pctype for ~20 cycles.
3. **Read cutover**: drop `-ShadowMode`. The engine now runs against the
shopdb-sourced manifest; payloads still come from the share. Rollback is a
one-line revert to the share-sourced call. Keep exporting manifests from
shopdb to the share (GE-Enforce > Manifests > Export to Share) so the
share stays a break-glass copy.
4. **Payload migration** (optional, later): move small scripts/configs to
`http`/`inline` payloads, verified by `payloadsha256`. Big MSIs stay on SMB.
Do not cut a fleet over before the shadow diffs are clean. Preinstall
(`phase=preinstall`) stays share-sourced until its own cutover is planned - it
runs before enrollment provisions a token.
## Security notes
- The client runs as SYSTEM, so shopdb's TLS certificate must be in the machine
trust store (air-gapped/self-signed sites provision the CA via the same DSC
step as the token).
- `http`/`inline` payloads are verified against `payloadsha256` before running,
independent of how the entry detects install state. This is the real integrity
guarantee and holds even over plain HTTP inside a trusted segment.
- The token is a scoped service token: it can fetch manifests and report, and
nothing else.

141
docs/GE-ENFORCE-DEPLOY.md Normal file
View File

@@ -0,0 +1,141 @@
# Deploying the GE-Enforce agent on a PC
This is the deploy contract: what has to be laid down on a PC so GE-Enforce runs,
and how to do it regardless of imaging path (PXE, OOBE provisioning package,
Intune, or by hand). It complements `docs/GE-ENFORCE.md` (concepts) and
`docs/GE-ENFORCE-CLIENT.md` (the fetch/report contract).
The reference installer is `plugins/geenforce/client/Install-GEEnforce.ps1`. It
is site-neutral: you pass the PC's identity in, it writes the files/registry the
engine reads and registers the enforcement task.
---
## 1. What "deploying GE-Enforce" means
A PC needs three things present before enforcement works. HOW they get there is
up to your imaging path; WHAT they are is fixed:
1. **The GE-Enforce client** - the engine (`Install-FromManifest.ps1`), the
shopdb client kit (`ShopdbEnforceClient.psm1`, `Invoke-ShopdbEnforce.ps1`),
and a scheduled task (at logon + periodic) that runs as SYSTEM.
2. **Identity** in `C:\Enrollment` - so the PC knows what it is (see section 2).
3. **A credential** - the SFLD share credential (for a share-sourced manifest)
and/or the shopdb service token (for the fetch/report client). This is what
gates enforcement actually starting; until it exists the task exits 0 and
retries.
`Install-GEEnforce.ps1` lays down 1 and 2, and can write the shopdb token for 3.
The engine itself is the GE-Enforce framework's, not shopdb's - point the
installer at your copy with `-EngineSource`, or place it under the install root
first (see section 5).
---
## 2. Identity: how a PC determines its PC type (and bay)
There is NO auto-detection. The provisioner supplies the values; the engine only
reads files. This is the core of "set the PC up to know its type."
| Value | Written to | Purpose | Required? |
|---|---|---|---|
| **PC type** | `C:\Enrollment\pc-type.txt` (first line) | picks the manifest scope (`gea-shopfloor-<type>`) | YES |
| Machine (bay) number | `C:\Enrollment\machine-number.txt` (fallback; DNC registry `MachineNo` wins) | per-bay gates | only for bay-gated entries |
| CMM version | `C:\Enrollment\cmm\version.txt` | `_CmmVersion` gating (CMM PCs) | CMM only |
| CMM bay id | `C:\Enrollment\cmm\cmmid.txt` | CMM bay identity | CMM only |
| Share root + site | `C:\Enrollment\site-config.json` | where manifests/payloads live | for share-sourced |
| shopdb URL + token | `HKLM:\SOFTWARE\GE\ShopDB` (BaseUrl, ApiToken) | fetch/report client | for shopdb client |
Valid `pc-type` values are the manifest scope names
(`gea-shopfloor-cmm`, `-collections`, `-nocollections`, `-common`, `-keyence`,
`-genspect`, `-heattreat`, `-partmarker`, `-waxtrace`) or a legacy alias the
engine maps (`Standard`, `CMM`, ...).
**shopdb cannot set the type at imaging** - a PC is not known to shopdb until it
enrolls and reports. If you want the value to come from an asset system, pre-map
asset-tag / hostname -> PC type in your provisioning and feed it to the
installer.
---
## 3. Running it, per imaging path
`Install-GEEnforce.ps1` is the same in every case; only how you invoke it differs.
### PXE / imaging step (identity known at image time)
Run it as an imaging step after the OS lays down, passing the type the operator
selected:
```
powershell -ExecutionPolicy Bypass -File Install-GEEnforce.ps1 `
-PCType gea-shopfloor-cmm -MachineNumber 0615 -CmmVersion 2019 `
-ShareRoot \\server\share\dt\shopfloor -Site "West Jefferson" `
-ShopdbUrl https://shopdb.site.geaerospace.net -ShopdbToken shopdb_pat_xxx `
-EngineSource \\server\share\dt\shopfloor\common
```
### OOBE provisioning package (ppkg)
Sites that apply a ppkg during OOBE (no PXE/WinPE step) embed the installer + the
client kit in the ppkg and run it from a `CommandLine` / `ProvisioningCommands`
action. Supply the PC type from a ppkg variable, a first-boot prompt, or an
asset lookup:
```
powershell -ExecutionPolicy Bypass -File Install-GEEnforce.ps1 -PCType %PCTYPE% ...
```
Timing is forgiving: the scheduled task is fail-safe, so if OOBE finishes before
Intune/DSC provisions the credential, enforcement simply waits and starts once
the credential lands. There is no ordering trap.
### Intune / manual
Same script as a Win32 app / remediation, or run by hand on an existing PC to
retrofit it. `-NoTask` provisions identity + kit without registering the task.
---
## 4. What the installer does (idempotent)
1. Writes the `C:\Enrollment` identity files (section 2).
2. Writes `HKLM:\SOFTWARE\GE\ShopDB` (BaseUrl + token) if provided.
3. Copies the client kit (the two files shipped next to it) to `-InstallRoot`
(default `C:\ProgramData\GE-Enforce`).
4. If `-EngineSource` is given, copies `GE-Enforce.ps1` + `lib\Install-FromManifest.ps1`.
5. Registers the scheduled task (SYSTEM, at logon + every `-IntervalMinutes`) to
run `Invoke-ShopdbEnforce.ps1 -Scope <PCType> -EnginePath <engine>`.
Re-running it updates identity/config and re-registers the task in place.
For PC types that have cut over to HTTPS manifest delivery (currently
displays/kiosks), the full server-side setup, auth model (IP allowlist vs
ApiToken), and per-PC-type cutover playbook live in `geenforce-api-cutover.md`.
This doc covers what gets laid on the PC; that doc covers where the manifest
comes from.
---
## 5. The engine boundary
shopdb ships the **manifest store + client kit + this installer**, not the
GE-Enforce **engine** (`Install-FromManifest.ps1`) or dispatcher - those live in
the GE-Enforce framework. So one of:
- pass `-EngineSource <path>` pointing at a folder that has `GE-Enforce.ps1` and
`lib\Install-FromManifest.ps1` (e.g. your share's `common` dir), or
- place the engine under `<InstallRoot>\lib\Install-FromManifest.ps1` yourself
before enforcement runs.
The installer warns if the engine is missing but still provisions identity so a
PC is at least correctly labelled. Use engine lib >= 2.6 (required for the
`_CmmVersion` gate).
---
## 6. Verify a provisioned PC
- `Get-Content C:\Enrollment\pc-type.txt` -> the expected scope.
- `Get-ItemProperty HKLM:\SOFTWARE\GE\ShopDB` -> BaseUrl + ApiToken set.
- `Get-ScheduledTask GE-Enforce` -> Ready.
- Trigger it once and check the client log
(`C:\Logs\Shopfloor\shopdb-enforce-*.log`), then confirm the PC appears under
**GE-Enforce > Enforcement Reports** in shopdb with the right PC type.

133
docs/GE-ENFORCE-DISPLAY.md Normal file
View File

@@ -0,0 +1,133 @@
# GE-Enforce: the gea-shopfloor-display scope
Displays are the share-less corner of the fleet. They are Entra-joined,
credential-less kiosk PCs that pull their manifest over HTTPS on port 443 and
authenticate with a read-only service PAT scoped `geenforce.fetch`, sent as
`X-API-Key`. They have no SMB share mount. The kiosk engine and the kiosk
browser are baked into the display image, not shipped over HTTPS, so the display
manifest heals POLICY / CONFIG drift only, never EXEs. It is self-sufficient and
does not inherit the fleet-wide `common` scope (see below).
## The display fetch token MUST be resource-bound
The same read-only key ships to every display (delivered by DSC, or baked into
the image), so it must not be a skeleton key for the whole content store. Mint
the display token bound to just this scope, so a leak cannot pull any other
scope's manifest or any blob by hash:
```
POST /api/apitokens
{ "name": "display fetch", "scopes": ["geenforce.fetch"],
"resourcescopes": ["gea-shopfloor-display"] }
```
With `resourcescopes` set, `GET /manifest?pctype=<other>` returns 403 and
`GET /payload/<sha>` returns 404 for any blob the display scope does not
reference. `resourcescopes` NULL (unset) = unrestricted, for back-compat with
existing service tokens. Rotate by minting a new bound token and revoking the
old one (deactivate it server-side); DSC re-delivers, or re-image.
There are three display subtypes, selected by `C:\Enrollment\display-type.txt`:
`Dashboard`, `Lobby`, and `3DPrintRoom`.
## Authoring the scope
The scope is authored programmatically by
`plugins/geenforce/seed_display_scope.py`, which builds a manifest dict and
hands it to `service.replace_scope_draft` (the same call the `import-share` CLI
uses), then attaches the inline dispatcher payload. From a Flask app context:
```python
from plugins.geenforce.seed_display_scope import seed_display_scope
seed_display_scope(publish=True) # publish=False leaves it as a draft
```
`replace_scope_draft` is an idempotent draft rebuild. `publish=True` additionally
freezes an immutable published snapshot (that step is not idempotent: it always
creates a new version).
### What the scope contains
1. Four `Registry` drift-heal entries that re-assert the Microsoft Edge kiosk
relaunch policies set at imaging by `09-Setup-Display.ps1`. Each writes the
value and detects drift with `DetectionMethod = ValueMatches` against the
same path/name, so a display that loses a policy self-heals on the next
enforce cycle with no keyboard or mouse on site:
- `RelaunchNotification = 2` (DWord, Required auto-restart)
- `RelaunchNotificationPeriod = 3600000` (DWord, 1 hour)
- `RelaunchHeadsUpPeriod = 60000` (DWord, 1 minute)
- `RelaunchWindow` (String, JSON, 02:00 start, 120 minute duration)
2. One `PS1` dispatcher, delivered inline over HTTPS. It reads
`C:\Enrollment\display-type.txt` and launches the kiosk target for the
subtype. The subtype -> route map is a data-driven table
(`DISPLAY_TYPE_TARGETS`) at the top of both the seed module and the generated
script, so targets are easy to edit. `DetectionMethod = Always` so it
re-asserts each cycle, but the script is idempotent (it skips relaunch if a
kiosk process is already serving the target URL).
### Role resolution: server first, display-type.txt fallback
The dispatcher first asks the server: `GET
/api/dashboarddefaults/display-role?fqdn=<fqdn>` (public, unauthenticated). A row
in `dashboarddefaults` keyed by the display's FQDN (IP fallback) wins and returns
the role and frontend path directly. Only when the server is unreachable or has
no mapping does the dispatcher fall back to the local `display-type.txt` map
below. To repurpose a display, edit its `dashboarddefaults` row; the change takes
effect on the next enforce cycle.
Fallback map (local file):
| display-type.txt | kiosk route | notes |
| --- | --- | --- |
| `Dashboard` | `/shopfloor` | core ShopfloorDashboard, standalone full-screen |
| `Lobby` | `/tv` | slides plugin TV dashboard (surface `lobby`) |
| `3DPrintRoom` | `/parts-kiosk` | **PLACEHOLDER, TODO-confirm** printedparts parts kiosk route; confirm the real 3D-print-room target with the floor team before publishing to production displays |
### Dashboard-defaults FQDN keying
`dashboarddefaults` rows were historically keyed by IP. Migration
`7d31_dashboarddefault_fqdn` added an `fqdn` column; resolution is now FQDN-first
with IP as fallback (`_resolve_default` in
`shopdb/core/api/dashboarddefaults.py`). FQDNs are stored lowercase. This
survives DHCP churn on kiosk subnets. `POST /api/dashboarddefaults` accepts
`fqdn`, `ipaddress`, `displayrole` (`dashboard`|`lobby`|`partskiosk`),
`businessunitid`, and `description`; `displaypath` is not stored but derived from
the role (`DISPLAY_ROLE_PATHS`). Two public read endpoints consume it:
`/api/dashboarddefaults/display-role` (dispatcher) and
`/api/dashboarddefaults/visitor-location` (lobby business-unit lookup). The
server derives a display's FQDN from its reported BIOS serial as
`F<serial>.<domain>` (`derive_display_fqdn`, domain from the `display_fqdn_domain`
setting); the dispatcher in `plugins/geenforce/seed_display_scope.py` builds the
same FQDN client-side for its lookup.
### Legacy autostart self-heal
The dispatcher also cleans up after the old GE Aerospace Dashboard / Lobby
Display Inno installers, which planted autostarts (a Public-Desktop `.lnk`, an
all-users Startup `.lnk`, and an `HKLM ...\CurrentVersion\Run` value) that
relaunch Edge at now-dead URLs (`/shopfloor-dashboard/`, `/tv-dashboard/`) and
white-screen. The 32-bit installer's Run value was WOW64-redirected into
`Wow6432Node`, which is why it survived earlier cleanup. Every enforce cycle the
dispatcher sweeps both registry views, all loaded user hives, Run/RunOnce/policy
Run keys, and every per-user and common Startup folder, matching by legacy name
and by the old URLs, then kills any old-URL Edge. The kiosk shortcut it writes is
a direct Edge shortcut (no launcher or VBS). The fix ships by re-publishing this
code-authored scope (`seed_display_scope(publish=True)`), not an import-share.
`pxe-images/github/find-legacy-kiosk-autostart.ps1` is a read-only locator for
stragglers.
## Self-sufficient: displays do NOT inherit common
The `gea-shopfloor-display` scope carries everything a display enforces. It does
NOT inherit the fleet-wide `common` scope. Displays run the enforcer with
common-merge off (the client default; common-merge is opt-in via
`Invoke-ShopdbEnforce.ps1 -IncludeCommon`), so `common`'s SMB-backed fleet
entries (Adobe, Oracle, OpenText, Defect Tracker, EventSaver, printer map,
self-update, asset-reporting, ...) never reach a share-less display.
This was a deliberate decision: a display needs none of common's software, and
inheriting common would have forced repackaging every SMB `common` payload as
`http`/`inline` for a share-less box. Keeping the display scope self-sufficient
avoids all of that. If a future non-display share-less PC genuinely needs the
fleet-wide entries, that is what `-IncludeCommon` plus a per-entry SMB->http
payload conversion would be for -- but displays do not use it.

336
docs/GE-ENFORCE.md Normal file
View File

@@ -0,0 +1,336 @@
# GE-Enforce: concepts, the shopdb plugin, and imaging-time integration
This guide explains how GE-Enforce works, how the shopdb `geenforce` plugin
manages it, and when GE-Enforce installs and takes over during the imaging
process. It is written for site IT.
It pairs with two companion docs:
- `docs/GE-ENFORCE-CLIENT.md` - the client fetch/report contract + the reference
PowerShell kit (`plugins/geenforce/client/`).
- `docs/proposals/ge-enforce-plugin.md` - the design/plan and the staged cutover.
The ground truth for behavior is the engine itself
(`Install-FromManifest.ps1`) and the on-share manifests; this guide describes
what they do, it does not replace them.
---
## 1. What GE-Enforce is
GE-Enforce is a **desired-state enforcement** system for shopfloor PCs. Instead
of a one-time install during imaging, it continuously makes each PC match a
declared list of what should be installed - and RE-installs anything that drifts
(uninstalled, corrupted, or overwritten). It is the shopfloor equivalent of a
lightweight, air-gapped-friendly configuration-management agent.
Two things make up the system:
1. **The engine + dispatcher on each PC** - PowerShell that reads a manifest and
enforces it every logon and periodically.
2. **The manifests** - JSON files that declare, per imaging PC type, what to
install / copy / write and how to detect whether it is already correct.
The shopdb `geenforce` plugin adds a third piece: it lets you **author, publish,
and version those manifests in shopdb** (instead of hand-editing JSON on a file
share) and **see what every PC actually did** (fleet compliance reporting).
---
## 2. How GE-Enforce works (the framework)
### 2.1 Two phases
Every shopfloor PC is governed in two distinct phases:
| Phase | When | Runs what | Purpose |
|---|---|---|---|
| **Preinstall** | ONCE, at imaging | `preinstall.json` (via the imaging `00-PreInstall` step) | Day-zero foundation: PowerShell 7, the VC++ redistributable matrix, Oracle Client, Adobe Reader, HostExplorer, serial drivers, etc. "Install once at imaging, no drift correction." |
| **Runtime** | EVERY logon + periodically | `common/manifest.json`, then `gea-shopfloor-<type>/manifest.json`, then an optional `<type>-<subtype>` manifest | Ongoing enforcement + self-heal: app versions, config-file drift, registry drift, per-cycle scripts (asset report, VNC firewall, EventSaver), version-gated installs. |
The two phases share the same entry SHAPE (field names) but are run by different
runners with different capabilities. Preinstall is a one-shot at imaging that
implements only `Type=MSI` and `Type=EXE`, with only `Registry` / `File`
detection (other types/detections are skipped). Runtime is the continuous
enforcement loop and implements the full Type + DetectionMethod matrix below.
### 2.2 The runtime loop, step by step
On each cycle (`GE-Enforce.ps1` on the PC):
1. Read the PC's identity from `C:\Enrollment\` (see 2.4).
2. Look up the SFLD share credential in the registry and **mount the share**
(SYSTEM cannot reach the share as its computer account, so it mounts as the
provisioned SFLD user - `net use W: ...`). If no credential yet, exit 0 and
retry next cycle (Azure DSC has not provisioned it).
3. Run the engine (`Install-FromManifest.ps1`) against `common/manifest.json`,
then `gea-shopfloor-<pctype>/manifest.json`, then a `<pctype>-<subtype>`
manifest if one exists. Common runs first so shared prerequisites (e.g.
Oracle Client) land before type-specific apps that depend on them.
4. Write a status file back to the share (and, in the shopdb model, POST a
report - see 4.3).
Every failure is non-fatal (exit 0) so a network blip or a not-yet-provisioned
credential never blocks or breaks a PC.
### 2.3 The manifest: scopes and entries
A manifest is `{ "Version", "_comment", "Applications": [ entry, ... ] }`. Each
imaging PC type is a **scope** with its own manifest, plus the fleet-wide
`common` scope:
- `common` - runs on EVERY PC type; entries use a `PCTypes` filter to target
subsets (e.g. "EventSaver on collections + heattreat, but not CMM").
- `gea-shopfloor-collections`, `-nocollections`, `-cmm`, `-keyence`, `-common`
(lab/timeclock), `-genspect`, `-heattreat`, `-partmarker`, `-waxtrace` - each
runs only on PCs of that type, so its entries usually do NOT set `PCTypes`
(the manifest already only runs there). Keyence is the exception: it uses
`PCTypes` for hardware SUBTYPE targeting (`keyence-vr6000` vs `keyence-vr3000`).
Each **entry** declares one action. Its `Type` picks the action:
| Type | Action |
|---|---|
| MSI / EXE / CMD / BAT | run an installer with `InstallArgs` |
| PS1 | run a script from the share |
| INF | install a driver via `pnputil` |
| File | copy `Source` -> `Destination` |
| Registry | write a value |
### 2.4 Self-heal via detection
Every entry has a `DetectionMethod` that decides whether the action fires:
| Method | Means "already correct" when... |
|---|---|
| Registry | the key/value exists (optionally equals a value) |
| File | the file exists |
| FileVersion | the file's version string matches exactly (fleet convention is a 4-part string like 6.4.5.0; the engine does a raw string compare, it does not enforce 4 parts) |
| Hash | the file's SHA256 matches (case-insensitive) |
| MarkerFile | a marker file exists (the engine writes it after a clean install) |
| ValueMatches | a registry value equals the entry's target |
| pnputil | a driver matching a pattern is present |
| Always / (none) | fires EVERY cycle (used for per-cycle scripts) |
If detection says "not correct," the action runs. That is the self-heal: delete
`DncMain.exe` and next cycle re-installs eDNC; corrupt a config file whose Hash
no longer matches and next cycle re-copies it. **Entry order is execution
order** - config-restore entries sit AFTER their installer so a mid-cycle vendor
overwrite is healed on the same cycle.
### 2.5 Targeting gates (all ANDed)
An entry can be narrowed by any combination of:
- `PCTypes` - which PC types (alias-aware: old names like `Standard` map to
`collections`/`nocollections`/`common`). Fleet-wide `common` uses this heavily.
- `TargetHostnames` - specific hostnames (supports `*` wildcards).
- `TargetMachineNumbers` - specific bay machine numbers (e.g. Okuma bays).
- `_CmmVersion` - CMM PCs only: a tagged entry applies when it equals the bay's
resolved PC-DMIS version (`C:\Enrollment\cmm\version.txt`). IMPORTANT: if no
version is resolved (file missing/empty - a pre-picker bay), ALL tagged
entries apply (deliberate legacy "install-all" behavior), so such a bay gets
every PC-DMIS version, not none. Requires engine lib >= 2.6.
- `PCTypesStrict` - disables alias expansion (PREINSTALL runner only; the runtime
engine ignores it).
Different PC types have different niche gates: CMM uses a version gate, Keyence a
model subtype, Collections per-bay machine numbers. The shopdb editor shows only
the gates a given scope actually uses (see 4.1).
### 2.6 What the PC needs to know about itself (enrollment)
The runtime engine reads the PC's identity from `C:\Enrollment\`:
- `pc-type.txt` - the imaging PC type (which scope to run). `pc-subtype.txt` is
LEGACY (no longer written at imaging since the 2026-05-04 rename reorg; the
dispatcher still honors it if present on older fleet PCs).
- `machine-number.txt` - the bay number FALLBACK; the eDNC/DNC registry
`MachineNo` value wins if present. `9999` is the imaging placeholder by
convention - the enforcement engine does NOT special-case it; it is simply a
value that won't match a real bay number in a `TargetMachineNumbers` gate.
(The 9999-skip you may see is only in the status write-back, not enforcement.)
- `cmm/version.txt` - CMM bays only: the resolved PC-DMIS version for `_CmmVersion`.
- `site-config.json` - the share root and site settings.
- SFLD credentials at `HKLM:\SOFTWARE\GE\SFLD\Credentials` - provisioned by
Azure DSC after enrollment (this is what gates the runtime phase starting).
Note: all of the identity files above (pc-type, machine-number, cmm version,
site-config) are written in WinPE at the PXE menu, BEFORE the image boots - the
preinstall phase already reads them. What happens post-imaging is only Intune
enrollment + the Azure DSC credential (see the timeline below).
---
## 3. When GE-Enforce installs / takes over (the imaging timeline)
This is the "when to implement it during imaging" question. The order is:
```
[0] WinPE / PXE menu (BEFORE the image boots)
- identity written to C:\Enrollment: pc-type.txt, machine-number.txt,
cmm/version.txt, site-config.json (startnet.cmd). The PC already knows
what it is before Windows starts.
|
v
PXE image applied, Windows boots
|
v
[1] PREINSTALL (00-PreInstall runner runs preinstall.json ONCE)
- reads the step-0 identity files, then installs the foundation:
PowerShell 7, VC++ redists, Oracle Client, Adobe Reader, HostExplorer,
serial drivers, Display kiosk app, ... (things later runtime apps need)
- preinstall implements MSI/EXE + Registry/File detection only
|
v
[2] GE-ENFORCE ITSELF is laid down during imaging
- the dispatcher (GE-Enforce.ps1), the engine lib (Install-FromManifest.ps1),
and a scheduled task (at-logon + every ~5 min + shift windows) are
registered as part of the image / shopfloor setup
|
v
[3] ENROLLMENT (post-imaging)
- Intune / GCCH enrollment, THEN Azure DSC provisions the SFLD share
credential into HKLM:\SOFTWARE\GE\SFLD\Credentials
- (the identity files already exist from step 0 - enrollment adds only the
credential, which is what unblocks runtime)
|
v
[4] FIRST LOGON -> RUNTIME ENFORCEMENT BEGINS
- the scheduled task runs GE-Enforce.ps1: mount share, run common + the
PC-type (+ subtype) manifests, install/self-heal, report
- repeats every logon + periodically forever after
```
Key points on timing:
- **Preinstall (step 1) is the imaging-time install.** Put anything that must
exist before first logon, or that never needs drift correction, here (runtimes,
redistributables, drivers). It runs once and is done.
- **Runtime enforcement (step 4) does not start until enrollment (step 3)
provisions the SFLD credential.** Before that, GE-Enforce exits 0 each cycle
and waits. So a freshly imaged PC that is not yet enrolled is inert, by design.
- **The engine lib version matters.** `_CmmVersion` gating needs lib >= 2.6 on
the PC; deploy the lib before a manifest that uses it.
- Some apps appear in BOTH phases: preinstalled at imaging for day-zero, then
carried by a runtime entry so drift is corrected later (Oracle, UDC, Adobe,
HostExplorer, Defect Tracker).
Rule of thumb: **imaging-time (preinstall) = foundation that must be there or
never drifts; runtime = everything that needs to stay correct over the PC's
life.**
---
## 4. How the shopdb plugin manages this
The `geenforce` plugin turns the manifest from hand-edited JSON on a share into
shopdb data you author, version, publish, and monitor. It lives under the
top-level **GE-Enforce** section (Manifests | Enforcement Reports), not Settings,
because it is a full management surface.
### 4.1 Manifests - authoring (GE-Enforce > Manifests)
- **PC Types (scopes):** each imaging PC type is a row; add/edit/delete. (The
scope's `computertypeid` field is now the imaging-pc-type -> ComputerType
mapping mechanism, per ADR-012. The old Settings > Collector PC Types page is
retired; there is nothing to configure there anymore.)
- **Entries:** an ordered list (Up/Down = the execution-order contract). Add/Edit
opens a typed form: the payload fields switch on `Type` (MSI shows Installer +
InstallArgs, PS1 shows Script + Args, File shows Source + Destination, Registry
shows the Reg* fields), a detection block, an InUseCheck editor, and a
**Targeting** section that shows only the gates the scope uses (CMM shows the
version gate; the common/preinstall scopes show PC types; a scope whose entries
use machine numbers shows those) with a "Show all targeting options" escape
hatch.
- **Simulate ("what would a PC get?"):** enter a PC profile (type, subtype,
hostname, machine number, CMM version) and see which entries apply and why the
rest are filtered - without reading a PowerShell log.
- **Publish / Versions / Roll Back:** editing changes a DRAFT only. Publish
freezes an immutable version; PCs are only ever served the published version;
Roll Back restores an earlier one. History (date, author, note) per version.
### 4.2 Milestone 1 - export to the share (engine unchanged)
Today the enforcement engine still reads manifests from the SFLD share. The
plugin's **Export to Share** button writes the current published manifest to
`<shareroot>/<scope>/manifest.json` (backing up the old file to `_meta/history`
first). So the workflow is:
**author + publish in shopdb -> Export to Share -> the unchanged engine picks it
up next cycle.**
Nothing about the engine, the share layout, or the PCs changes. Rollback is
restoring the `_meta/history` backup (or re-publishing an older version and
re-exporting). This is the safe first milestone: all the authoring benefit, zero
client risk.
Configure the share root once at the top of the Manifests page.
### 4.3 Enforcement Reports - fleet compliance (GE-Enforce > Enforcement Reports)
Each PC reports its enforcement result back to shopdb (see the client kit). The
Reports page shows, per PC:
- **Received** - did the PC apply the latest published version? (applied vs
latest). "behind" means it has not picked up your newest publish yet.
- **Status** - `ok` (nothing needed), `selfhealed` (drift corrected), `failed`.
- **Counts** - installed / skipped / failed, plus per-entry detail (action,
self-heal flag, exit code, message) in the row's Detail view.
This is the observed-state half of the loop: the manifest is what SHOULD be
installed; the report is what each PC ACTUALLY did.
### 4.4 The client side (per PC)
The engine sources the manifest and reports results using the reference kit in
`plugins/geenforce/client/` (`ShopdbEnforceClient.psm1` +
`Invoke-ShopdbEnforce.ps1`), configured from `HKLM:\SOFTWARE\GE\ShopDB`
(BaseUrl + a `geenforce.fetch`/`geenforce.report` service token). See
`docs/GE-ENFORCE-CLIENT.md` for the fetch/report contract, the last-known-good
cache, shadow mode, and the staged cutover from share-sourced to shopdb-sourced
manifests.
The cutover from share-sourced to shopdb-sourced manifests is per PC type. The
**displays/kiosks cohort has cut over**: share-less display PCs fetch their
manifest and payloads entirely over HTTPS (see `docs/geenforce-api-cutover.md`
and `docs/GE-ENFORCE-DISPLAY.md`). All other fleet PC types (cmm, collections,
keyence, genspect, heattreat, partmarker, nocollections, common) still enforce
from the SFLD SMB share via Export to Share (4.2) and only REPORT to shopdb. The
playbook for moving the next PC type is `geenforce-api-cutover.md` section 11.
---
## 5. Day-to-day: common tasks
All in GE-Enforce > Manifests. No PowerShell, no editing JSON on the share.
- **Add an app to a PC type:** open the PC type, Add Entry, pick the Type (the
form adapts), fill the installer + detection + any targeting, place it in order
with Up/Down (config restores go BELOW their installer), Preview, Publish, then
Export to Share.
- **Bump an app version:** drop the new installer in the scope's `apps/` folder
on the share, open the entry, update the Installer filename + the Detection
value (the new version), Publish, Export to Share. PCs self-heal next cycle.
- **Roll back a bad publish:** the PC type's Versions list -> Roll Back to the
last good version -> Export to Share.
- **Canary a risky change:** add the one test PC under Target hostnames (via
"Show all targeting options"), Publish; when happy, remove the filter and
Publish again.
- **Check "did PC Y get app X":** use Simulate with that PC's type / machine
number / CMM version; and check Enforcement Reports for what it actually did.
---
## 6. Reference
- Engine (behavior ground truth): `Install-FromManifest.ps1` (lib >= 2.6).
- Dispatcher: `GE-Enforce.ps1` (mount + run common then type scope).
- Preinstall runner: `00-PreInstall-*` over `preinstall.json` (imaging-time).
- shopdb model + API: `plugins/geenforce/` (models, importer/serializer,
filters mirror, service, routes).
- Behavioral parity gate (proves the shopdb model round-trips the real
manifests): `plugins/geenforce/parity.py` + `flask geenforce parity`.
- Client kit + contract: `plugins/geenforce/client/`, `docs/GE-ENFORCE-CLIENT.md`.
- Agent deployment (per PC, any imaging path): `docs/GE-ENFORCE-DEPLOY.md` +
`plugins/geenforce/client/Install-GEEnforce.ps1`.
- Design + cutover plan: `docs/proposals/ge-enforce-plugin.md`.

101
docs/IMPORT-ADOPTION.md Normal file
View File

@@ -0,0 +1,101 @@
# Importing a site's legacy data
## Two routes in, and which one you want
**If the site has a spreadsheet and no developer**, use the CSV import. It is
the common case, and it needs nothing beyond the templates:
```bash
flask csv templates --out csv-templates # generated from the live schema
# fill them in
flask csv import --dir csv-templates # checks only, changes nothing
flask csv import --dir csv-templates --commit
```
Foreign keys take a NAME, not an id - write `Bay 3`, not `locationid=7`. The
importer resolves them, including across files in the same run, and a name it
cannot find is reported with the line, the column and the value. Nothing is
written unless every row passes, and re-running an edited file updates rows
rather than duplicating them. See [CSV-IMPORT.md](CSV-IMPORT.md).
**If the site has a source database to read from**, and someone able to script
against it, the HTTP import API below is the better tool: it carries the whole
history, preserves original timestamps, and handles relationships the CSV set
does not model.
---
Every adopting site has its own source database - it will not match another
site's schema. So the import is split in two layers:
1. **The import API is the stable contract** (`docs/IMPORT-API.md`). Whatever
your source looks like, you create flask records through the same documented
REST endpoints, authenticated with an admin PAT and the `X-Import-Mode`
header (which preserves legacy timestamps). This layer is the product; it is
schema-agnostic.
2. **A per-site loader is thin glue.** It reads *your* source database and POSTs
to those endpoints. Nobody runs another site's loader - you copy the pattern.
The West Jefferson loader in `scripts/site_imports/wjf/` is reference
implementation #1. Read it alongside this guide.
## The shape of a loader
- `harness.py` - builds the app against the target `DATABASE_URL`, mints an
unscoped admin PAT in-process, and drives the real endpoints through the app
test client with `Authorization: Bearer <pat>` + `X-Import-Mode: true`. This
exercises the same routes/authz/validation an HTTP client would, no running
server needed. It also holds read-only access to the source DB and a JSON
`IdMap` of legacy-id -> new-id crosswalks.
- `run.py` - ordered `stage_*` functions. Each reads a slice of the source,
POSTs it, and records the crosswalk later stages resolve foreign keys against.
Post-import fixups that re-point existing assets (example:
`scripts/reclassify_servers_to_network.py`, servers imported as PCs moved to
network devices in place) belong in the site loader's verify stage, not in the
stable API layer.
### Stage order matters
Reference/lookup tables first (so foreign keys resolve), then the entity hub,
then dependents, then links:
```
reference -> catalog -> assets (persist the source-id -> assetid crosswalk)
-> dependents (installs, warranties, notifications, ...) -> relationships
```
The **crosswalk is the keystone**: capture every legacy id -> new id as you
create rows, and resolve foreign keys through it in later stages. New
autoincrement ids will not match the source's.
## Producing the mapping
You do not have to hand-derive the source -> target mapping. Point the
agent-assisted workflow at a source database plus this API contract and it emits
a per-table mapping (source columns -> endpoint fields, transforms, what is
importable vs out of scope) and a loader skeleton. That is the repeatable
onboarding path.
## Running (against a THROWAWAY import database)
1. Build a fresh target: `flask db upgrade` + `flask plugin upgrade-all` +
`flask seed permissions/settings/reference-data`. Enable every bundled plugin
you need (some ship disabled; a plugin's routes only register when it is
enabled at app start).
2. Load your source dump into a scratch DB the loader can read.
3. Run the loader stages in order, dry-running / spot-checking as you go.
4. Verify: row-count + foreign-key-resolution audit against the source, then a
UI spot-check (log in, eyeball the lists / map / a detail page).
5. Only then point a real instance at the imported database.
## What the WJ loader demonstrates
- Fanning one legacy "machine" table out to the flask asset types
(computer/machine/network/measuring-tool) by a routing rule, with the
duplicate/placeholder/skip decisions applied.
- Synthesizing a natural key when the source lacks one (printers -> `PRN-{id}`).
- Folding a primary IP onto an asset, pairing a check-in/out event log into
checkouts, deduping colliding names, reversing an inverse relationship type.
- The handful of narrow gaps the API cannot cover (e.g. no bulk-communications
endpoint) handled as documented direct-ORM writes.

530
docs/IMPORT-API.md Normal file
View File

@@ -0,0 +1,530 @@
# Import API: migrating the classic ASP shopdb through HTTP alone
This is the operator manual for importing the legacy Classic-ASP shopdb database
(`prodscratch` on the dev MySQL container) into shopdb-flask using ONLY the HTTP
API. No direct writes to the `shopdb_flask` database are needed or wanted: every
row is created through a documented endpoint so authorization, validation,
auditing, and plugin hooks all run exactly as they do for a human operator.
A plain Python script can run the whole migration from this document.
Contents:
1. [Prerequisites](#1-prerequisites)
2. [Order of operations](#2-order-of-operations)
3. [Full table-by-table mapping](#3-full-table-by-table-mapping)
4. [Tables with no target yet](#4-tables-with-no-target-yet)
5. [Idempotency recipe and a worked importer](#5-idempotency-recipe-and-a-worked-importer)
6. [Verification: row-count parity](#6-verification-row-count-parity)
---
## 1. Prerequisites
### Dev URLs
- Flask API: `http://localhost:5001`
- All import calls target `/api/...` on that host.
### Admin token
Every write needs authentication, and import mode additionally needs an admin.
A large import can outlast a login JWT: `access_token` expires after one hour,
so a long run dies mid-import with 401s. Use a **personal API token (PAT)**
instead. A PAT never expires (unless you set an expiry), acts as the user that
created it, and is sent exactly like a JWT. Create one as an admin (via the
Settings > API Tokens page, or the API):
```bash
# Bootstrap: a short login JWT is fine just to mint the long-lived PAT.
JWT=$(curl -s http://localhost:5001/api/auth/login \
-H 'Content-Type: application/json' \
-d '{"username":"<admin>","password":"<password>"}' | jq -r '.data.access_token')
# The full secret (shopdb_pat_...) is returned ONCE. Save it now.
curl -s http://localhost:5001/api/apitokens \
-H "Authorization: Bearer $JWT" \
-H 'Content-Type: application/json' \
-d '{"name":"legacy import runner"}' | jq -r '.data.secret'
```
Send the PAT on every request as `Authorization: Bearer shopdb_pat_...`. It
authenticates the whole import surface (every create/update/delete plus import
mode) as its owning admin, exactly as a login JWT would, but without the hourly
expiry. Revoke it from the same Settings page (or `DELETE /api/apitokens/<id>`)
when the import is done.
Use an **unscoped** token for imports. A token may optionally carry a scopes
list that limits it to specific permissions; a scoped token suspends the admin
bypass and is denied on role-gated endpoints AND on import mode, so it cannot
run an import. Leave the "Restrict permissions" option off (the default) so the
token acts with the full authority of its admin owner. Minting a token itself
requires the `apitokens.create` permission (admins have it by default).
A short-lived login JWT still works for quick one-off calls if you prefer.
### Import mode: the `X-Import-Mode` header
By default the server stamps `createddate`/`modifieddate` to "now" on every
create and update, which would erase a migrated row's real history. To preserve
it, send the request header:
```
X-Import-Mode: true
```
When (and only when) the caller is an admin AND that header is present:
- create/update endpoints on timestamped entities accept optional
`createddate` and `modifieddate` fields in the JSON body and store them
verbatim (naive UTC). Both `2020-01-05T12:00:00` (ISO) and the legacy
`2020-01-05 12:00:00` (MySQL) forms are parsed. A bare `2020-01-05` works too.
- the selfhosted USB checkout/checkin endpoints accept optional `checkouttime`
and `checkintime` overrides so historical events keep their real timestamps.
Without the header, or for a non-admin caller, those fields are silently ignored
and the server behaves exactly as it does normally. This is enforced centrally
by `shopdb/utils/import_mode.py` (`import_mode_active`, `apply_import_timestamps`,
`parse_import_datetime`), exposed on the plugin contract surface `shopdb.api`.
Timestamped entities that honor `createddate`/`modifieddate`: assets (all five
type plugins), vendors, models, modeltypes, businessunits, locations, operating
systems, applications, knowledge base, USB devices, asset relationships.
Entities that carry history in domain fields instead (createddate passthrough is
a no-op there, by design): notifications (`starttime`/`endtime`), warranties
(`startdate`/`enddate`/`lastcheckeddate`). Set those fields directly in the
payload; they are already accepted.
### Reference data seed
Before importing, seed the reference tables that have no CRUD endpoint of their
own (communication types such as IP/Serial/USB, default statuses, canonical
relationship types, permissions, settings):
```bash
flask seed permissions
flask seed settings
flask seed reference-data
```
`communicationtypes` (the target of legacy `comstypes`) is populated here, so
the primary-IP mapping below can resolve `comtype='IP'`.
---
## 2. Order of operations
Import in dependency order so foreign keys always resolve. Each step is a
lookup-then-upsert loop (see section 5); rerunning any step is safe.
1. **Reference / lookup data first**
1. Vendors (`vendors`)
2. Model types (`modeltypes`) - from legacy `machinetypes`
3. Models (`models`) - needs vendors + model types
4. Business units (`businessunits`)
5. Location types, then Locations (`locations/types`, `locations`) - the
legacy LocationOnly machines land here, not as assets
6. Operating systems (`operatingsystems`)
7. Asset statuses (`assets/statuses`) - from legacy `machinestatus`
8. Relationship types (`assets/relationshiptypes`) - from legacy
`relationshiptypes`
9. Notification types (`notifications/types`)
10. Per-plugin subtypes: computer types (from `pctype`), machine types,
printer types, network device types, measuring-tool types
11. Support teams + support-team contacts (`supportteams`,
`supportteams/{id}/contacts` - see section 3.3); import these BEFORE
applications because `applications.supportteamid` points at them
12. Applications (`applications`) and their versions; import legacy `topics`
as applications too (KB links to applications, section 3)
2. **Assets, per type** (each creates the core Asset row plus its extension):
computers, machines, printers, network devices, measuring tools, and USB
devices. Fan out the legacy `machines` table by category (section 3).
3. **Communications**: the primary IP is set through the asset payload's
`ipaddress` field during step 2. There is no bulk-communications endpoint;
see the mapping note.
4. **Relationships** (`assets/relationships`): needs both endpoint assets and
the relationship types to already exist.
5. **Installed applications**: attach apps to computers
(`computers/{id}/apps`), needs computers + applications.
6. **Knowledge base, notifications, warranties, USB checkouts** (including
backdated history).
7. **Custom fields**: for any legacy column with no home in the target schema,
define a custom field for the asset type and store the value per asset.
---
## 3. Full table-by-table mapping
Legend: `->` maps to. Endpoints are relative to `http://localhost:5001`. "NK"
is the natural key used for the idempotent lookup (section 5).
### 3.1 The `machines` hub fans out into the asset plugins
`machines` (885 rows) is the central legacy asset table. Two columns drive the
fan-out: `machinetypeid` (what the asset physically is) and `pctypeid` (a
computer's sub-type). Route each row by `machinetypeid`:
| legacy `machinetypeid` | `machinetypes.machinetype` | target plugin | subtype source |
|---|---|---|---|
| 1 | LocationOnly (also `islocationonly=1`) | core **Locations** (NOT an asset) | `locationtype` |
| 33 | PC | **computers** | `computertype` <- `pctype.typename` via `machines.pctypeid` |
| 20 | Server | **computers** | computer type "Server" |
| 15 | Printer | **printers** | `printertype` |
| 16 Access Point / 17 IDF / 18 Camera / 19 Switch / 46 Firewall | | **network** | `networkdevicetype` |
| 44 | USB Device | **usb** | (usb device) |
| 23 Measuring Machine / 3 CMM / 48 Spline Checker / 8 Eddy Current / 47 Inspection | | **measuringtools** (gage-lab judgment call; ADR-005) | `measuringtooltype` |
| 2,4,5,6,7,9,10,11,12,13,14,21,22,24,25,45 (lathes, mills, welders, grinders, ...) | | **machines** | `machinetype` |
This mapping is a recommended default, not a hard rule; a site may re-route a
`machinetypeid` (for example send CMM to `machines` rather than
`measuringtools`). Decide the routing table once, up front.
Common `machines` columns -> core Asset fields (same for every target plugin):
| legacy column | target field | notes |
|---|---|---|
| `machinenumber` | `assetnumber` | the business identifier / NK |
| `alias` or `hostname` | `name` | layperson label |
| `serialnumber` | `serialnumber` | |
| `machinestatusid` | `statusid` | remap via `machinestatus` -> asset statuses |
| `businessunitid` | `businessunitid` | remap via imported business units |
| `mapleft` | `mapx` | |
| `maptop` | `mapy` | |
| `machinenotes` | `notes` | |
| `dateadded` | `createddate` | import mode only |
| `lastupdated` | `modifieddate` | import mode only |
Per-plugin extension fields:
- **computers** (`POST /api/computers`): `hostname` <- `machines.hostname`,
`osid` <- remapped `machines.osid`, `computertypeid` <- computer type from
`pctype`, `loggedinuser`, `lastboottime`, `vendorid`, `modelnumberid`,
`ipaddress` <- `machines.ipaddress1` (primary IP). NK: `assetnumber`.
- **machines** (`POST /api/machines`): `machinetypeid`, `vendorid`,
`modelnumberid`, `controllervendorid`/`controllermodelid` (from
`controllertypes` remapped to vendors/models), `requiresmanualconfig` <-
`requires_manual_machine_config`, `islocationonly`. NK: `assetnumber`.
- **printers** (`POST /api/printers`): see 3.2 (authoritative source is the
legacy `printers` table).
- **network** (`POST /api/network`): `networkdevicetypeid`, `hostname`,
`vendorid`, `ipaddress` <- `machines.ipaddress1`. NK: `assetnumber`.
- **measuringtools** (`POST /api/measuringtools`): `measuringtooltypeid`,
calibration fields where known. NK: `assetnumber`.
### 3.2 Reference and lookup tables
| legacy table | target endpoint | field mapping | NK |
|---|---|---|---|
| `vendors` | `POST /api/vendors` | `vendor` -> `vendor` | `vendor` |
| `machinetypes` | `POST /api/modeltypes` | `machinetype` -> `modeltype`; set `category` (Equipment/Computer/...) | `modeltype` |
| `models` | `POST /api/models` | `modelnumber`, `vendorid` (remapped), `machinetypeid` -> `modeltypeid`, `notes`, `image` -> `imageurl`, `documentationpath` -> `documentationurl` | `modelnumber` + `vendor` |
`imageurl` imports as a plain URL string (an external URL or a legacy
`/images/models/*` path). Binary photos are not part of the import payload;
upload them after import via `POST /api/models/<modelid>/image` (multipart
`file`), which stores the file under `instance/modelimages/` and rewrites
`imageurl` to the served URL.
| `businessunits` | `POST /api/businessunits` | `businessunit` -> `businessunit` | `businessunit` |
| `operatingsystems` | `POST /api/operatingsystems` | `operatingsystem` -> `osname` | `osname` (+`osversion`) |
| `machinestatus` | `POST /api/assets/statuses` | `machinestatus` -> `status` | `status` |
| `relationshiptypes` | `POST /api/assets/relationshiptypes` | `relationshiptype` -> `relationshiptype`, `description`, `isdirectional` (bool, default true; false = symmetric connection) | `relationshiptype` |
| `notificationtypes` | `POST /api/notifications/types` | `typename`, `typedescription`, `typecolor` | `typename` |
| `pctype` | `POST /api/computers/types` | `typename` -> `computertype`, `description` | `computertype` |
| `subnettypes` | (see subnets) | used as `subnettype` string on subnets | - |
| `subnets` | `POST /api/network/subnets` | `cidr`, `description` -> `name`/`description`, `vlan` -> create VLAN first (`POST /api/network/vlans`) then `vlanid`, `subnettypeid` -> `subnettype` name | `cidr` |
| `dashboarddefaults` | `POST /api/dashboarddefaults` | `fqdn` (preferred key, stored lowercase), `ipaddress` (fallback key), `displayrole` (`dashboard`/`lobby`/`partskiosk`), `businessunitid` (remapped; only the `dashboard` role uses it), `description` | `fqdn`, else `ipaddress` |
| `controllertypes` | remap into `vendors` + `models` | e.g. "Fanuc" -> a Vendor; the controller model -> a Model; then set `controllervendorid`/`controllermodelid` on the machine | - |
| `comstypes` | `communicationtypes` (seeded, no API) | ensure `flask seed reference-data` created IP/Serial/USB/... before importing comms | - |
Resolution at runtime is FQDN-first with IP fallback (migration
`7d31_dashboarddefault_fqdn`); import both when the legacy source has them.
Note on communication types: the classic `comstypes.typename` values
(IP, Serial, Network_Interface, USB, Parallel, VNC, FTP, DNC) correspond to the
seeded `communicationtypes.comtype`. They are created by the reference-data seed,
not imported per-row.
### 3.3 Support teams, applications, topics, installed apps
Support teams and their contacts import BEFORE applications, because
`applications.supportteamid` references a team. The legacy `appowners` table
is folded into contacts: each legacy `supportteams` row carries one
`appownerid`, so import that owner as ONE contact on the team (legacy
`appowner` -> `name`, `sso` -> `sso`).
| legacy table | target endpoint | field mapping | NK |
|---|---|---|---|
| `supportteams` | `POST /api/supportteams` | `teamname`, `teamurl` (ServiceNow group deep link) | `teamname` |
| `appowners` (via each team's `appownerid`) | `POST /api/supportteams/{supportteamid}/contacts` | `appowner` -> `name`, `sso` -> `sso`, `sortorder` (default 0) | (supportteamid, name) |
| `applications` | `POST /api/applications` | `appname`, `appdescription`, `supportteamid` (remap by team `teamname`, GET `/api/supportteams?teamname=...`), `isinstallable`, `applicationnotes`, `installpath`, `applicationlink`, `documentationpath`, `ishidden`, `isprinter`, `islicenced`, `image` | `appname` |
| `appversions` | `POST /api/applications/{appid}/versions` | `version`, `releasedate`, `notes` | `version` (per app) |
| `topics` | `POST /api/applications` | `topics` is a near-clone of `applications` and `knowledgebase.appid` points at it; import each distinct topic as an Application (`appname` = topic name), so KB links resolve against `applications` | `appname` |
| `installedapps` | `POST /api/computers/{computerid}/apps` | body `{appid, appversionid}`; resolve `machineid` -> the imported computer, `appid`/`appversionid` -> imported app + version | (computerid, appid) |
`installedapps` only makes sense for computer-class assets; skip rows whose
`machineid` did not map to a computer.
### 3.4 Communications
| legacy table | target | field mapping | notes |
|---|---|---|---|
| `communications` (comstypeid=1, isprimary) | asset `ipaddress` on create/update | `address` -> `ipaddress` | Sets the primary IP communication for the asset. |
| `communications` (other comstypeids / secondary rows) | none yet | | No bulk-communication create endpoint exists. Import the primary IP only; capture extra interfaces as custom fields, or defer. |
### 3.5 Relationships
| legacy table | target endpoint | field mapping |
|---|---|---|
| `machinerelationships` | `POST /api/assets/relationships` | `machineid` -> `sourceassetid` (the imported asset id), `related_machineid` -> `targetassetid`, `relationshiptypeid` -> `relationshiptypeid` remapped by name, `relationship_notes` -> `notes` |
Suggested legacy-name -> target relationship-type mapping (create these types
first, or map onto the canonical `partof`/`controls`/`connectedto`):
| legacy `relationshiptype` | recommended target |
|---|---|
| Controls | Controls |
| Controlled By | Controls (reverse the source/target) |
| Dualpath | Dualpath (or `connectedto` with label "dualpath") |
| Cluster Member | partof |
| Backup For | Backup For |
| Master-Slave | Controls |
| Contains | partof |
| Stored At | Stored At |
| Connected To | connectedto |
Resolve each machine id to the asset id you got back when you created that
asset (keep a `legacy_machineid -> assetid` map as you import).
### 3.6 Knowledge base, notifications, warranties, USB
| legacy table | target endpoint | field mapping | NK |
|---|---|---|---|
| `knowledgebase` | `POST /api/knowledgebase` | `shortdescription`, `linkurl`, `keywords`, `appid` (remapped to the imported application/topic); `lastupdated` -> `modifieddate` in import mode | `linkurl` (fallback `shortdescription`) |
| `notifications` | `POST /api/notifications` | `notification`, `notificationtypeid` (remapped), `businessunitid` (remapped), `starttime`, `endtime`, `ticketnumber`, `link`, `isshopfloor`, `employeesso`; note legacy `endtime` sentinel `2099-00-03 09:52:32` is invalid - drop or clamp it | `ticketnumber` when set, else append-only |
| `warranties` | `POST /api/warranty` | `warrantyname`/`servicelevel` -> `servicelevel`, `enddate` -> `enddate`, link the covered asset via `assetids: [assetid]`; set `vendor` (required) from the source or "Dell"; `servicetag` if known | `servicetag` + `vendor` |
| `usbcheckouts` | see below | historical checkout/checkin events | - |
USB devices and their history:
1. Create each USB device (legacy `machines` rows with `machinetypeid=44`, or a
dedicated device list) via `POST /api/usb` in selfhosted mode with body
`{device_id: <serial>, device_desc, locker_location}`. NK: `device_id`.
2. Replay each `usbcheckouts` row as a checkout then (if returned) a checkin,
with import-mode backdating:
- `POST /api/usb/{device_id}/checkout` body
`{badge: <sso>, reason: <checkout_reason>, checkouttime: <checkout_time>}`
- if `checkin_time` is set:
`POST /api/usb/{device_id}/checkin` body
`{badge: <sso>, sanitized: <was_wiped>, notes: <checkin_notes>, checkintime: <checkin_time>}`
The `checkouttime`/`checkintime` overrides are honored only in import mode.
Employee directory (people): only self-hosted mode (`employee_directory_mode =
selfhosted`) owns people in this app; import them via the directory bulk-upsert
`POST /api/employees/directory/import` (CSV headers `SSO,First_Name,Last_Name,
Team,Role,Picture`) or per-person `POST /api/employees/directory`. Photos:
- External mode: the photo is a URL/relative path supplied by the HR database
(`Picture` column); it is a read-only pass-through and cannot be uploaded here.
- Self-hosted mode: the `Picture` CSV field is a legacy text label and does not
drive the displayed photo. Upload the real photo after import via
`POST /api/employees/<sso>/photo` (multipart `file`, png/jpg/jpeg/gif/webp),
which stores it under `instance/employeephotos/` and serves it publicly.
### 3.7 Anything unmappable -> custom fields
For a legacy column with no target field (for example `machines.logicmonitorurl`,
`machines.fqdn`, `printers.printerpin`), define a custom field on the asset type
and store the value per asset:
- `POST /api/customfields` body `{assettypeid, label, datatype}` (once per field)
- `PUT /api/customfields/asset/{assetid}` body `{values: {<fieldid>: <value>}}`
Custom-field values are not timestamped, so they carry no history.
---
## 4. Tables with no target yet
These legacy tables have no import target in the current schema. The
dispositions below are DECIDED, not open questions.
### DECIDED: not migrated
- **`dncconfig` and `commconfig`** - intentionally NOT migrated. DNC
communication settings drift constantly, so a one-shot import of stale data
has little value. The plan is a future DNC feature fed live by the GE-Enforce
collector/reporting tool rather than a historical import. When that DNC
support is eventually built, the expected ingestion pattern is one attribute
at a time across the whole facility (for example, sweep every machine's baud
rate in one pass, then ports, and so on) via the GE-Enforce collector, so the
future design should favor per-field fleet-wide updates over per-machine
full-record imports.
### DECIDED: skip (structure only or low value)
- **`compliance`, `compliancescans`** - 0 rows in `prodscratch`. No data to
migrate; a future compliance plugin would own them. Skip.
- **`ednc_installations`, `ednc_logs`** - 0 rows, and they belong to the eDNC
tooling rather than the asset catalog. Skip.
- **`distributiongroups`** (2 rows) - email distribution lists referenced by
`businessunits.distributiongroupid`. No target; skip, or attach as a business
unit custom field if a site needs it.
- **`functionalaccounts`** (7 rows) - service-account concept referenced by
`pctype`/`machinetypes`; no equivalent in the new schema. Skip, or capture as
a computer-type custom field.
- **`skilllevels`** (2 rows) - orphaned lookup (no FK from `machines`). Skip.
---
## 5. Idempotency recipe and a worked importer
The endpoints are NOT upserts. The idempotent unit is a two-step recipe that
composes with import mode:
1. **Look up** the row by its natural key using the exact-match list filter.
2. If found, **PUT** to update it; if not, **POST** to create it.
Each import-relevant list endpoint has an exact-match filter for its natural key
(added for exactly this purpose):
| entity | lookup |
|---|---|
| assets (all 5 plugins) | `GET /api/{plugin}?assetnumber=<n>` |
| vendors | `GET /api/vendors?vendor=<name>` |
| models | `GET /api/models?modelnumber=<m>&vendor=<vendorid>` |
| model types | `GET /api/modeltypes?modeltype=<name>` |
| business units | `GET /api/businessunits?businessunit=<name>` |
| locations | `GET /api/locations?locationname=<name>` |
| operating systems | `GET /api/operatingsystems?osname=<name>` |
| applications | `GET /api/applications?appname=<name>` |
| knowledge base | `GET /api/knowledgebase?linkurl=<url>` |
| warranties | `GET /api/warranty?servicetag=<tag>&vendor=<name>` |
| notifications | `GET /api/notifications?ticketnumber=<t>` |
| USB devices | `GET /api/usb/{device_id}` (exact by id) |
### Worked example
A small, dependency-free importer (`requests`) that authenticates with a PAT
(so a multi-hour run cannot expire mid-import), does the lookup-then-upsert loop
in import mode, supports a `--dry-run` flag, and reports errors without aborting
the whole run:
```python
import argparse
import os
import requests
BASE = "http://localhost:5001"
class ImportClient:
def __init__(self, token=None, dryrun=False):
self.session = requests.Session()
self.dryrun = dryrun
# A personal API token (shopdb_pat_...) does not expire like a login
# JWT, so it survives a long import. See section 1 to mint one.
token = token or os.environ["SHOPDB_TOKEN"]
# X-Import-Mode makes createddate/modifieddate passthrough take effect.
self.session.headers.update({
"Authorization": f"Bearer {token}",
"X-Import-Mode": "true",
})
def lookup(self, path, params):
"""Return the first matching row, or None."""
resp = self.session.get(f"{BASE}{path}", params=params)
resp.raise_for_status()
rows = resp.json().get("data") or []
return rows[0] if rows else None
def upsert(self, path, idfield, lookupparams, payload):
"""Lookup by natural key; PUT if found, else POST. Returns the row."""
existing = self.lookup(path, lookupparams)
if self.dryrun:
verb = "PUT" if existing else "POST"
print(f"[dry-run] {verb} {path} {lookupparams}")
return existing or payload
if existing:
rowid = existing[idfield]
resp = self.session.put(f"{BASE}{path}/{rowid}", json=payload)
else:
resp = self.session.post(f"{BASE}{path}", json=payload)
if resp.status_code >= 400:
# report and keep going; a single bad row must not abort the run
print(f"ERROR {resp.status_code} {path}: {resp.text[:200]}")
return None
return resp.json()["data"]
def import_vendors(client, legacyrows):
for row in legacyrows:
client.upsert(
"/api/vendors",
idfield="vendorid",
lookupparams={"vendor": row["vendor"]},
payload={
"vendor": row["vendor"],
# legacy history preserved because X-Import-Mode is set
"createddate": row.get("dateadded"),
"modifieddate": row.get("lastupdated"),
},
)
if __name__ == "__main__":
parser = argparse.ArgumentParser()
# PAT from the SHOPDB_TOKEN env var, or pass --token explicitly.
parser.add_argument("--token", default=None)
parser.add_argument("--dry-run", action="store_true")
args = parser.parse_args()
client = ImportClient(args.token, dryrun=args.dry_run)
# read legacy rows from prodscratch (read-only) and call the import_* fns
# in the order of section 2, keeping a legacy-id -> new-id map as you go.
```
Keep a `legacy_id -> new_id` dictionary for every entity as you import it; you
need it to remap foreign keys (a machine's `businessunitid`, a checkout's
`machineid`, a relationship's `machineid`/`related_machineid`, and so on).
---
## 6. Verification: row-count parity
After each phase, compare counts. Legacy side (read-only), for example:
```bash
docker exec dev-mysql mysql -uroot -p"$MYSQL_ROOT_PASSWORD" prodscratch \
-e "SELECT COUNT(*) FROM vendors;"
```
New side, via the API pagination metadata (`meta.pagination.total`):
```bash
curl -s "http://localhost:5001/api/vendors?per_page=1" \
-H "Authorization: Bearer <token>" | jq '.meta.pagination.total'
```
Suggested parity checks:
| entity | legacy count | new count |
|---|---|---|
| vendors | `SELECT COUNT(*) FROM vendors` | `GET /api/vendors` total |
| models | `SELECT COUNT(*) FROM models` | `GET /api/models` total |
| business units | `SELECT COUNT(*) FROM businessunits` | `GET /api/businessunits` total |
| applications | `SELECT COUNT(*) FROM applications` | `GET /api/applications?showhidden=true` total |
| knowledge base | `SELECT COUNT(*) FROM knowledgebase WHERE isactive` | `GET /api/knowledgebase` total |
| computers | `SELECT COUNT(*) FROM machines WHERE machinetypeid IN (33,20)` | `GET /api/computers` total |
| machines | `SELECT COUNT(*) FROM machines WHERE machinetypeid IN (2,4,5,6,7,9,10,11,12,13,14,21,22,24,25,45)` | `GET /api/machines` total |
| printers | `SELECT COUNT(*) FROM printers WHERE isactive` | `GET /api/printers` total |
| network devices | `SELECT COUNT(*) FROM machines WHERE machinetypeid IN (16,17,18,19,46)` | `GET /api/network` total |
| USB devices | `SELECT COUNT(*) FROM machines WHERE machinetypeid=44` | `GET /api/usb` total |
| relationships | `SELECT COUNT(*) FROM machinerelationships WHERE isactive` | per-asset `GET /api/assets/{id}/relationships` |
| USB checkouts | `SELECT COUNT(*) FROM usbcheckouts` | `GET /api/usb/checkouts` |
Exact counts will differ where the fan-out routing table (section 3.1) sends a
`machinetypeid` to a different plugin than the example above; adjust the legacy
`WHERE` clause to match the routing you chose. Investigate any gap beyond that.

View File

@@ -1,5 +1,13 @@
# ShopDB - Windows + IIS install runbook
> **Not the route for a new site.** Sister sites install from the Windows
> installer - one `.exe`, no manual IIS work: **[INSTALL-WINDOWS.md](INSTALL-WINDOWS.md)**.
>
> This document is the **manual** procedure, kept for reference and for
> hand-built servers that predate the installer. Note that the installer will not
> adopt a server built this way without `-AdoptExisting`, on purpose.
A step-by-step, **tested** install for a new site on Windows Server / Windows 11
with IIS in front of the Flask app (HttpPlatformHandler -> waitress), backed by
MySQL. This runbook was validated end to end on a win11 + IIS + MySQL 5.6 box.
@@ -13,9 +21,9 @@ lives). Run PowerShell as Administrator.
| Need | Notes |
| --- | --- |
| **Python 3.12** (64-bit) | `python --version` |
| **Python 3.14** (64-bit) | `python --version` |
| **IIS** with **HttpPlatformHandler** | https://www.iis.net/downloads/microsoft/httpplatformhandler (direct MSI: `download.microsoft.com/download/8/1/3/813AC4E6-9203-4F7A-8DD5-F3D54D10C5CD/httpPlatformHandler_amd64.msi`) |
| **MySQL 5.7+/8.0** (or 5.6 with the flags in step 1) | reachable from the app host |
| **MySQL 8.4 LTS** (standard for new installs) | reachable from the app host. 8.0 reached end of life in April 2026 and no longer ships a standalone server MSI. 5.7+ still works on an existing server; 5.6 needs the flags in step 1. |
| URL Rewrite (optional) | only for the real-client-IP rule; skip it and the app still runs |
The app itself pulls in `waitress` and `tzdata` from `requirements.txt` (step 4).
@@ -113,7 +121,7 @@ venv\Scripts\flask seed settings
# enable the plugins this site tracks (registry is empty on a fresh box).
# usb + employees install DISABLED by default - enable them later in the wizard
# if the site wants those (they create extra tables).
foreach ($p in "computers","equipment","network","notifications","printers","knowledgebase","slides","warranty") {
foreach ($p in "computers","machines","network","notifications","printers","knowledgebase","slides","warranty") {
venv\Scripts\flask plugin install $p
}
@@ -129,6 +137,16 @@ venv\Scripts\flask seed admin --username admin --email admin@yourfacility.exampl
## 7. IIS site
Two supported deployment methods:
- **Method A - own site (recommended, default):** the app gets its own IIS
site, port (or hostname), app pool, and venv. Steps 1-5 below.
- **Method B - subpath under an existing site:** the app runs as an IIS
**Application** (e.g. `/ops`) under a site you already have (such as the
classic ASP site or Default Web Site), so it shares that site's binding and
TLS cert: `https://<host>/ops/`. Do steps 1-4 below, then follow **7b**
instead of step 5.
1. Copy `deploy\windows\web.config` to `APP_ROOT\web.config`. If `APP_ROOT` is not
`C:\shopdb-flask`, fix the paths inside it. Create `APP_ROOT\logs`.
2. Create an app pool with **No Managed Code**:
@@ -141,6 +159,8 @@ venv\Scripts\flask seed admin --username admin --email admin@yourfacility.exampl
```powershell
icacls APP_ROOT /grant "IIS AppPool\shopdbflask:(OI)(CI)RX" /T
icacls APP_ROOT\logs /grant "IIS AppPool\shopdbflask:(OI)(CI)M" /T
mkdir APP_ROOT\instance 2>NUL
icacls APP_ROOT\instance /grant "IIS AppPool\shopdbflask:(OI)(CI)M" /T
```
4. **Unlock the handler sections** (locked server-wide by default; without this
IIS returns **HTTP 500.19**):
@@ -159,10 +179,45 @@ IIS launches `waitress-serve --port=%HTTP_PLATFORM_PORT% wsgi:app` per the
web.config and reverse-proxies the site port to it. First request takes ~15s
(the app boots + connects to MySQL).
### 7b. Method B: subpath under an existing site
The mount path must match in **three places**: the IIS Application alias, the
`MOUNT_PATH` the backend sees, and the `VITE_BASE_PATH` the frontend was built
with. `/ops` is the example throughout; any alias works.
1. Rebuild the frontend for the subpath (on the dev box, then copy `dist`):
```bash
cd frontend && VITE_BASE_PATH=/ops/ npm run build # note the trailing slash
```
2. Create the Application under the existing site (instead of `New-Website`):
```powershell
New-WebApplication -Site "Default Web Site" -Name ops -PhysicalPath APP_ROOT -ApplicationPool shopdbflask
```
3. Tell the backend its mount path: in `APP_ROOT\web.config`, uncomment the
`MOUNT_PATH` environment variable (value `/ops`), or set `MOUNT_PATH=/ops`
in `APP_ROOT\.env`. `wsgi.py` then serves everything under the prefix
(requests outside it get a plain 404 naming the mount).
4. Recycle the app pool. The app is at `http(s)://<host>/ops/` and the API at
`/ops/api/...`.
The handler mappings in the app's web.config apply only inside the
Application, so the parent site's own handlers (classic ASP, static files)
are untouched. `CORS_ORIGINS` in `.env` is origin-only (scheme + host + port,
no path), so it is the same for both methods.
> The `X-Forwarded-For` URL Rewrite rule in web.config is **commented out by
> default**. It needs the URL Rewrite module; with it active but the module
> absent, IIS returns 500.19. Install URL Rewrite, then uncomment the
> `<rewrite>` block, to record real client IPs in audit logs.
>
> Two companion requirements, or the app keeps seeing 127.0.0.1:
> `allowedServerVariables` is locked at server level by default (500.52 when
> the block activates) - unlock once with
> `appcmd unlock config -section:system.webServer/rewrite/allowedServerVariables`.
> And waitress 2+ strips X-Forwarded-For from untrusted proxies, so the
> waitress `arguments` line must carry
> `--trusted-proxy=127.0.0.1 --trusted-proxy-headers=x-forwarded-for`
> (the shipped web.config already does).
---
@@ -188,7 +243,12 @@ each gets its own site, app pool, port, and venv.
| --- | --- |
| `flask db upgrade` -> error **1071** | MySQL 5.6 without the step-1 flags (or server not restarted). |
| IIS **500.19** | handler sections not unlocked (step 7.4), or the `<rewrite>` block active without URL Rewrite. |
| IIS **500.52** after enabling the rewrite block | `allowedServerVariables` locked at server level - `appcmd unlock config -section:system.webServer/rewrite/allowedServerVariables`. |
| Audit log shows only **127.0.0.1** with the rewrite block active | waitress strips untrusted proxy headers - `--trusted-proxy=127.0.0.1 --trusted-proxy-headers=x-forwarded-for` missing from the waitress `arguments`. |
| **500** with an empty HttpPlatform log | app-pool identity can't read `APP_ROOT` / run the venv (step 7.3), or `.env` missing/invalid. |
| "internal error" toggling plugins, or uploads fail | app pool cannot WRITE `APP_ROOT\instance` (plugin registry, logos, photos, files live there) - step 7.3 grants it Modify. |
| "No time zone found with key America/New_York" | `tzdata` not installed (`pip install tzdata`). |
| Nav missing Equipment/PCs/... | plugins not installed (step 6 `flask plugin install`), or site not recycled. |
| Nav missing Machines/PCs/... | plugins not installed (step 6 `flask plugin install`), or site not recycled. |
| Method B: blank page / assets 404 under `/ops` | frontend `dist` built without `VITE_BASE_PATH=/ops/` (step 7b.1). |
| Method B: SPA loads but every API call 404s | `MOUNT_PATH` unset or not matching the Application alias (step 7b.3). |
| ConfigError on boot | a required `.env` var missing or left at a dev default. |

216
docs/INSTALL-WINDOWS.md Normal file
View File

@@ -0,0 +1,216 @@
# Install ShopDB-Flask on Windows Server
**This is the route for a new site.** You run one `.exe`, answer a few questions,
and get a working application. Nothing here needs an internet connection, and you
do not need to know IIS, Python or MySQL.
If you are looking after an existing hand-built server, see
[DEPLOY-WINDOWS-IIS.md](DEPLOY-WINDOWS-IIS.md) instead - that is the manual
procedure, and the installer will not adopt a server it did not build.
---
## Before you start
You need **four things**. The installer supplies everything else.
| | What | How to check |
|---|---|---|
| 1 | Windows Server 2019 or newer | `winver` |
| 2 | The **IIS Web Server role** installed | Server Manager -> Manage -> Add Roles -> Web Server (IIS). Or run the PowerShell below. |
| 3 | Administrator rights on the box | Right-click PowerShell -> "Run as administrator" works |
| 4 | A decision about the database - see [Which database?](#which-database) | - |
Installing IIS, if it is missing (this needs no internet):
```powershell
Install-WindowsFeature -Name Web-Server -IncludeManagementTools
```
The installer **checks all of this before it changes anything**, and it will not
let you continue until the check passes. You do not have to get it right first
time.
### Which database?
Two options. Pick before you start, because they ask different questions.
- **Use the bundled MySQL** - the installer puts MySQL 8.4 LTS on this server and
creates the database for you. Choose this when the server has no database
today. Simplest option, nothing to arrange in advance.
- **Use an existing MySQL** - the database already exists somewhere, and you have
a hostname, a database name, a username and a password for it. Choose this if
your site already runs MySQL, or a DBA looks after it.
If you are unsure: if nobody has given you database credentials, you want the
bundled option.
### SQL for your DBA (existing-MySQL option only)
The installer does not create the database or the user - it never needs
administrative rights on your database server. Ask your DBA to run:
```sql
CREATE DATABASE shopdb_flask CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'shopdb'@'%' IDENTIFIED BY '<a password you choose>';
GRANT ALL PRIVILEGES ON shopdb_flask.* TO 'shopdb'@'%';
FLUSH PRIVILEGES;
```
The `utf8mb4` charset matters - the default on older servers is `latin1`, and a
latin1 database mangles any non-ASCII text you store.
---
## Installing
1. Copy the installer `.exe` onto the server. It is one file and needs no
network.
2. **Right-click it -> Run as administrator.** Without this it cannot configure
IIS, and it will tell you so.
3. Work through the wizard. The pages are:
| Page | What it wants | If unsure |
|---|---|---|
| **Server check** | Nothing - it reports what it found | Fix anything red, then "Check again". You cannot continue while something is red, and nothing has been changed yet. |
| **Features** | Which parts of the product this site uses | The defaults are fine. You can add more later; removing needs a new installer. |
| **Database** | Bundled or existing - see above | Bundled |
| **Database details** | Host, port, name, user, password | Only asked for the existing-database option |
| **Address** | How people reach the site | See [Own address or subpath?](#own-address-or-subpath) |
| **Client addresses** | Whether a proxy sits in front | See [Client addresses](#client-addresses) |
| **Location** | Where to install | `C:\shopdb-flask` is fine |
4. The install takes a few minutes. Most of it is Python and the database schema.
5. At the end you get the address to open. **Write it down** - it is also on the
Start Menu as "Open ShopDB-Flask".
### Own address or subpath?
- **Its own address** - `http://yourserver:8090/`. Choose this on a server that
is not already running a website. Simplest.
- **Under this server's existing address** - `http://yourserver/shopdb/`. Choose
this when the server already serves something else and you do not want a second
port or a new DNS name. This is what West Jefferson uses.
You cannot change your mind later without re-running the installer, because the
web interface has the address compiled into it.
### Client addresses
The application records who connects, and some features decide what to show based
on it. The wizard asks one question:
- **Clients connect to this server directly** - the normal answer. Pick this
unless you know otherwise.
- **A proxy or load balancer sits in front** - pick this only if your network
team has told you traffic reaches this server through something else first.
Getting this wrong is not dangerous, but the site will record every visitor as
coming from the server itself, and features that depend on location will not
work. It can be changed later by re-running the installer.
---
## First login
Open the address the installer gave you. With no users in the database yet, the
page offers to **create the first administrator**, then runs a short setup wizard
for site details, features and the floor map.
That first account is a normal administrator account. Use a real password -
this is the account that creates everyone else.
---
## Did it work?
From the Start Menu, open **ShopDB-Flask Console** and pick option 1, or:
```powershell
cd C:\shopdb-flask
.\shopdb-admin.ps1 status
```
You want to see the site started, the pool started, and `responding : yes`.
Day-to-day tasks - restarting, backups, logs, upgrades - are in
[OPERATE-WINDOWS.md](OPERATE-WINDOWS.md).
---
## If the install fails
**The server is left part-configured.** Whatever had been done before the failure
is still there. That is deliberate: it means re-running is able to pick up where
it stopped.
1. Read the error. It names the cause and what to do about it.
2. Fix that, then **run the same installer again**. Re-running is safe - it skips
what is already done and does not touch your database or `.env`.
3. If you would rather start clean, remove it from **Settings -> Apps** first.
The full log is at:
```
C:\ProgramData\ShopDB-Flask\logs\shopdb-install-<date>.log
```
It records every step, including everything that was created. Send this if you
need help.
### Getting help from an AI assistant
These installs are often done with an assistant open in another window. Give it
real state rather than a description:
```powershell
.\shopdb-admin.ps1 check -Json
```
That prints one structured block covering the version, how the site is published,
IIS state, database reachability, Python version, installed features and any
errors. Paste it in. **It contains no passwords.** The install log is also safe
to share - the installer keeps secrets out of it deliberately.
Offline API reference for this server is served at `/api/docs` on the site
itself, and `docs\` in the install directory holds these runbooks.
---
## Upgrading
Run a newer installer over the top. It:
- backs the database up first, **verifies the dump is complete**, and refuses to
continue if it cannot;
- restores from that backup if the schema migration fails;
- refuses to install an **older** build over a newer one;
- keeps your `.env`, your data and your `web.config`.
Nothing else is required. See [UPGRADE.md](UPGRADE.md).
> **Before your first upgrade:** confirm `mysqldump` is available - the console's
> health check reports it. Without it the pre-upgrade backup is skipped, and that
> is the one you would want if a migration went wrong. It ships with the bundled
> database option; for an existing remote database, ask for `mysqlclient\` to be
> included in your installer bundle.
---
## Removing it
**Settings -> Apps -> ShopDB-Flask**, or Add/Remove Programs.
That removes the website, the application pool, the firewall rule and the
application directory. It deliberately **does not** drop the database and does
not uninstall MySQL, so your data survives.
Take a backup first: `.\shopdb-admin.ps1 backup`
---
## Notes for the person who builds the installer
Building a bundle for a site is a separate job, documented in
[../deploy/windows/installer/README.md](../deploy/windows/installer/README.md).
Sites receive a finished `.exe`; they do not build one.

241
docs/OPERATE-WINDOWS.md Normal file
View File

@@ -0,0 +1,241 @@
# Running ShopDB-Flask on Windows Server
Day-to-day operation of a site installed with the Windows installer. If you are
installing for the first time, start with [INSTALL-WINDOWS.md](INSTALL-WINDOWS.md).
Everything here goes through one tool, installed alongside the application:
```
C:\shopdb-flask\shopdb-admin.ps1
```
The Start Menu folder **ShopDB-Flask** has shortcuts for the common tasks. Run it
with no arguments for a menu, or pass a command directly. It needs
Administrator - it will ask, except for `open`.
---
## The commands
| Command | What it does | Safe at any time |
|---|---|---|
| `status` | Is it published, running, responding; database and table count | yes |
| `restart` | Recycles the application pool. **Use this after any config change.** | yes - drains requests rather than cutting them off |
| `stop` / `start` | Takes the site down / brings it back | yes, but `stop` makes it unavailable |
| `logs` | Last lines of the application and install logs | yes |
| `check` | Full health check | yes |
| `check -Json` | The same, machine-readable - see [Getting help](#getting-help) | yes |
| `verify` | Which build this is, and whether what is installed still matches it | yes |
| `sessions` | IIS worker processes and memory | yes |
| `plugins` | Which features are installed, which are available | yes |
| `add-plugin -Path <name>` | Turns on a feature this build ships | changes the site; restarts it |
| `backup [-Path <dir>]` | Writes a verified `.sql` dump | yes, but see below |
| `open` | Opens the site in a browser | yes |
Examples:
```powershell
.\shopdb-admin.ps1 status
.\shopdb-admin.ps1 restart
.\shopdb-admin.ps1 backup D:\backups
.\shopdb-admin.ps1 verify -Path leaflet
```
---
## Backups
```powershell
.\shopdb-admin.ps1 backup
```
Writes to `C:\ProgramData\ShopDB-Flask\backups` unless you pass a directory. The
dump is **verified complete** before it is reported as good - a truncated backup
is deleted rather than left to be discovered later.
Two things to know:
- **The dump contains everything, including user password hashes.** The directory
is locked to Administrators and SYSTEM. Keep it that way, and treat copies as
sensitive.
- **Store it off this server.** A backup on the server does not survive the
server.
An upgrade takes its own backup automatically, before it touches the schema.
Restore, and the Linux/Docker equivalents, are in
[BACKUP-RESTORE.md](BACKUP-RESTORE.md).
---
## Upgrading
Run a newer installer over the top. Nothing else. It backs up first, refuses to
go backwards, and restores if a migration fails. See
[UPGRADE.md](UPGRADE.md).
---
## Adding a feature
```powershell
.\shopdb-admin.ps1 plugins # what is here
.\shopdb-admin.ps1 add-plugin -Path warranty # turn one on
```
Only features **shipped in this build** can be added. Each site's installer is
built for that site's chosen feature set, so a feature nobody asked for is not on
the server at all - adding it means a new installer built from an updated
profile. `plugins` shows you which is which.
---
## When something is wrong
Work down this list.
**1. Is it actually down?**
```powershell
.\shopdb-admin.ps1 status
```
`responding : NO` with the pool `Started` usually means the application failed to
start, not that IIS is broken.
**2. What does it say?**
```powershell
.\shopdb-admin.ps1 logs
```
Application logs are in `C:\shopdb-flask\logs`, install logs in
`C:\ProgramData\ShopDB-Flask\logs`.
**3. Try a restart.** It fixes anything that is a stuck worker, and tells you
immediately if it is not:
```powershell
.\shopdb-admin.ps1 restart
```
**4. Check the database is reachable** - `status` reports the host and whether it
could count tables. A site that starts but shows no data is usually a database
problem, not an application one.
**5. Confirm nothing has drifted:**
```powershell
.\shopdb-admin.ps1 verify
```
This flags packages that no longer match what shipped - which usually means
somebody ran a `pip install` on the server by hand.
---
## Getting help
Give an assistant real state rather than describing the symptom:
```powershell
.\shopdb-admin.ps1 check -Json
```
One structured block: version, how the site is published, IIS and pool state,
whether it responds, database host and reachability, Python version, installed
features, and any errors. **It contains no passwords** and is safe to paste into
a chat window or a ticket.
The install log is also safe to share - secrets are deliberately kept out of it.
Offline reference on the server itself:
- `/api/docs` on the site - the full API reference, self-hosted, no internet.
- `C:\shopdb-flask\docs\` - these runbooks.
- `C:\shopdb-flask\sbom.cdx.json` - every component this build contains.
---
## Answering "are we affected by this vulnerability?"
The server carries its own bill of materials, so this does not need the build box
or an internet connection:
```powershell
.\shopdb-admin.ps1 verify -Path <component-name>
```
It reports whether the component is here, at what version, and whether it
actually **ships** or is only used to build the software. Example:
```
matches for 'leaflet':
leaflet 1.9.4 SHIPPED
```
Nothing found means this server does not carry it.
---
## Adding HTTPS
**The installer publishes over HTTP.** It has no certificate to use and no way to
get one on an air-gapped server, so it does not pretend otherwise. On an internal
network behind the site firewall that is often accepted; confirm it against your
own policy rather than assuming.
If you installed **under an existing site** (the subpath option) and that site
already has a certificate, you are already on HTTPS - nothing to do.
For a site of its own, once you have a certificate in the machine store:
```powershell
Import-Module WebAdministration
# 1. Add the binding. Get the thumbprint from the certificate you imported.
New-WebBinding -Name shopdb-flask -Protocol https -Port 443
$cert = Get-ChildItem Cert:\LocalMachine\My | Where-Object { $_.Subject -like '*yourserver*' }
Get-Item "IIS:\SslBindings\0.0.0.0!443" -EA SilentlyContinue | Remove-Item -EA SilentlyContinue
New-Item "IIS:\SslBindings\0.0.0.0!443" -Value $cert
# 2. Open the port.
New-NetFirewallRule -DisplayName "shopdb-flask 443" -Direction Inbound `
-Protocol TCP -LocalPort 443 -Action Allow
```
Then **update `CORS_ORIGINS` in `C:\shopdb-flask\.env`** to the `https://` address
and restart:
```powershell
.\shopdb-admin.ps1 restart
```
That last step is not optional. `CORS_ORIGINS` is an exact origin match, so a
site reached over `https://` while `.env` still says `http://` loads the page and
then fails every data request - which looks like the application is broken rather
than a configuration mismatch.
## Where things live
| | |
|---|---|
| Application | `C:\shopdb-flask` |
| Configuration and secrets | `C:\shopdb-flask\.env` (locked down - do not loosen) |
| Application logs | `C:\shopdb-flask\logs` |
| Install logs | `C:\ProgramData\ShopDB-Flask\logs` |
| Backups | `C:\ProgramData\ShopDB-Flask\backups` |
| Bill of materials | `C:\shopdb-flask\sbom.cdx.json` |
| Which build this is | `C:\shopdb-flask\.installed-version` |
If the bundled MySQL was installed, its generated root password was written once
to `C:\ProgramData\ShopDB-Flask\mysql-root-password.txt`. **Move it into your
password manager and delete that file.** It cannot be recovered.
## See also
- [UPDATES-WINDOWS.md](UPDATES-WINDOWS.md) - what future updates, bug fixes and
security releases will look like, including downtime and the effect on other
sites on the same IIS server
- [BACKUP-RESTORE.md](BACKUP-RESTORE.md) - what to back up and how to restore
- [INSTALL-WINDOWS.md](INSTALL-WINDOWS.md) - installing a new site

186
docs/PILOT-DEPLOY.md Normal file
View File

@@ -0,0 +1,186 @@
# Production pilot runbook (West Jefferson)
Goal: stand up a real shopdb-flask instance loaded with WJ's classic-ASP data,
run it **in parallel** with the classic app for a validation window, then cut
over. This runbook adds the legacy-data import + verification + cutover on top of
the generic stand-up in [`DEPLOY.md`](DEPLOY.md). Read that first; this only
calls out the pilot-specific steps.
Related: [`IMPORT-ADOPTION.md`](IMPORT-ADOPTION.md) (import model),
[`IMPORT-API.md`](IMPORT-API.md) (the contract), [`BACKUP-RESTORE.md`](BACKUP-RESTORE.md),
`scripts/site_imports/wjf/` (the loader).
---
## 0. Pre-flight checklist
- [ ] Host provisioned (Docker + compose, or a VM with Python 3 + MySQL 8).
- [ ] Three current classic dumps in hand: `shopdb` (main), `cmmc_usb`,
`wjf_employees`. Take fresh dumps at import time - the classic app is live.
- [ ] Target MySQL 8, utf8mb4 (charset is contract, ADR-004). Old MySQL <5.7
needs `innodb_large_prefix=ON` + Barracuda.
- [ ] Decide the pilot URL (e.g. `shopdb-pilot.wjs.geaerospace.net`) - separate
from the classic app; do not reuse its hostname yet.
- [ ] Confirm the import decisions still hold (see the loader README / the
import plan): assetnumber fallback + skip-dups, metrology routing,
cmmc-only USB, warranties = Dell, occurrences parked.
## 1. Stand up the pilot instance
Follow `DEPLOY.md` steps 1-6 against a NEW empty database (name it clearly, e.g.
`shopdb_flask_pilot`):
```bash
flask db upgrade
flask plugin upgrade-all # applies every plugin's chain
flask seed permissions
flask seed settings
flask seed reference-data # seeds communicationtypes (IP) + the rest
```
**Enable every bundled plugin the site tracks - including usb**, which ships
disabled. A plugin's routes only register when it is enabled at app start, and
the importer needs them:
```bash
for p in computers employees machines measuringtools network notifications \
printers slides usb warranty knowledgebase geenforce; do
flask plugin enable "$p"
done
```
Do **not** run the setup wizard yet - the import fills the data the wizard would
otherwise ask you to seed.
## 2. Load the classic data
The loader (`scripts/site_imports/wjf/`) reads the classic dumps and drives the
import API. It is site glue, not product code.
1. Load the three dumps into scratch source DBs the loader can read (strip the
`CREATE DATABASE`/`USE` lines so they land under scratch names, no clobber):
```bash
for pair in "shopdb_src:shopdb_dump.sql" "cmmc_usb_src:cmmc_usb_dump.sql" \
"wjf_employees_src:wjf_employees_dump.sql"; do
db="${pair%%:*}"; f="${pair##*:}"
mysql -h HOST -u root -p -e "CREATE DATABASE $db CHARACTER SET utf8mb4;"
sed -E '/^CREATE DATABASE/d; /^USE `/d' "$f" | mysql -h HOST -u root -p "$db"
done
```
2. Point the loader at the PILOT database and run all stages:
```bash
DATABASE_URL='mysql+pymysql://USER:PW@HOST:3306/shopdb_flask_pilot?charset=utf8mb4' \
venv/bin/python -m scripts.site_imports.wjf.run
```
The 16 stages run in order (reference -> catalog -> assets hub -> locations ->
printers -> dependents -> relationships -> subnets -> usb -> verify). It is
idempotent - a crashed run resumes from `idmap.json`.
3. Reclassify servers into network devices. The classic DB stored servers as
PCs, so the import lands them as `computer` assets. Re-point them in place:
```
DATABASE_URL=... venv/bin/python scripts/reclassify_servers_to_network.py # dry run, prints matches
DATABASE_URL=... venv/bin/python scripts/reclassify_servers_to_network.py --commit # apply
# match on an exact computer type instead of the SVR- name prefix:
... --type "Server" --commit
```
The assetid does not change: communications, relationships, map position, and
audit history carry over. Only the extension row is swapped (computers ->
networkdevices) and the asset type flipped; reclassified devices get the
`Server` networkdevicetype. Run the dry run, eyeball the list, then commit.
Re-running is safe (already-moved assets no longer match).
Expected magnitude (from the WJ dumps used in development - your fresh dumps will
differ slightly):
| entity | count |
|---|---|
| assets [*] | ~983 (computer ~663, machine ~76, network ~58, measuring-tool ~136, printer ~50) |
| locations | ~24 |
| employees | ~415 |
| installs | ~850 |
| primary IPs | ~461 |
| warranties | ~464 |
| notifications | ~261 |
| knowledge base | ~341 |
| relationships | ~93 |
| subnets | ~37 |
| USB devices / events | ~18 / ~232 |
[*] Counts taken AFTER `scripts/reclassify_servers_to_network.py --commit`.
Servers imported as computers are re-pointed to network devices, so the computer
count drops and network rises by the same amount versus a raw import.
> PLACEHOLDER - re-measure before publishing. The computer/network split shown
> in the assets row above still reflects a RAW import (pre-reclassify). Re-run
> the counts on the current prodscratch AFTER the reclassify step above and drop
> in the actual numbers; do not carry these development figures forward as if
> they already account for the reclassify.
The `verify` stage prints a source-vs-target row-count audit; the gaps are the
documented skips (inactive rows, duplicate machinenumbers, LocationOnly, the
9999 placeholder).
## 3. Verify the import
- [ ] Read the `verify` stage output - source vs target counts line up modulo
the documented skips.
- [ ] Create the admin: `flask seed admin --username ... --email ...` (password
printed once). Mark setup done so the app does not force the wizard:
set `setup_complete=true` in settings (or click through the wizard,
skipping the seed steps).
- [ ] UI spot-check (log in): Computers list paginates the full fleet; the Shop
Floor Map plots assets, color-coded by type (positions came from
mapleft/maptop); open a PC detail (installs), a printer (IP + share), an
application (installed-on list), a KB article; check the employee
directory; check a couple of asset relationships.
- [ ] Branding: upload the site logo + floor-plan blueprint under Settings, set
facility name (Settings drive these per `CONFIG.md`).
- [ ] Photos are deferred - employees show initials until a photo batch is run.
## 4. Parallel-run window
- Keep the classic app authoritative during the window. The pilot is read-mostly
for validation; do not dual-write.
- Have a few real users (IT + a floor lead) work the pilot and log gaps.
- Re-import is cheap: fix a loader mapping, drop + rebuild the pilot DB, re-run.
Nothing you do to the pilot touches classic.
- Point the **collector** (GE-Enforce fleet ingest) at the pilot in parallel to
confirm live PC check-ins land (see `COLLECTOR-INTEGRATION.md`), using a
scoped service token.
## 5. Cutover
When the window is clean:
1. Freeze classic writes (announce a short read-only window).
2. Take final fresh dumps; re-run the loader into a clean pilot DB so the
cutover data is current.
3. Verify counts + a fast UI spot-check.
4. Repoint the production hostname/DNS (or the reverse proxy) at the pilot.
5. Retire the classic app to read-only standby (do not delete - keep it as the
rollback for the agreed period).
## 6. Rollback
- Pre-cutover: trivially point back at classic (it never stopped being
authoritative).
- Post-cutover, within the standby window: repoint DNS/proxy back at classic;
investigate; re-cut when fixed. Because the loader is deterministic and the
classic DB is untouched, a re-run reproduces the flask DB exactly.
## 7. Post-cutover
- [ ] Backups on a schedule (`BACKUP-RESTORE.md`) - mysqldump + the `instance/`
dir (uploaded logos, floor plans, tokens).
- [ ] Run the employee-photo batch.
- [ ] GE-Enforce: publish manifests + cut the fleet over to the flask endpoints
when ready (`GE-ENFORCE-DEPLOY.md`) - independent of this pilot.
- [ ] Schedule the deferred data (occurrences, full communications fidelity)
only if a real need appears.

View File

@@ -7,6 +7,12 @@ the framework ships a bundled set, and you drop your own plugin into
`<framework>/plugins/<name>/` by clone, submodule, or symlink. No pip packaging
is required for v1 (pip distribution is deferred to v2 per ADR-003).
> **Windows / VS Code:** command examples use the Linux venv path
> `venv/bin/python`; on Windows use `venv\Scripts\python` and
> `$env:FLASK_APP="shopdb"` (not `export`). Full Windows onboarding:
> [DEVELOPMENT-SETUP](DEVELOPMENT-SETUP.md).
If you have not written a plugin before, start with
[PLUGIN-QUICKSTART.md](PLUGIN-QUICKSTART.md) and the hook reference in
[PLUGIN-HOOKS.md](PLUGIN-HOOKS.md). This document only covers the parts that are
@@ -63,18 +69,23 @@ live. The loader discovers a symlinked directory the same as a real one.
```bash
# 1. Clone the framework and your plugin repo side by side.
git clone https://gitea.proudtech.net/ge-aerospace/shopdb-flask.git
git clone https://gitea.proudtech.net/wjsf/wjsf-shipping.git
git clone https://github.com/ge-aero/shopdb-flask.git
git clone https://github.com/ge-aero/wjsf-shipping.git
# 2. Symlink your repo into the framework's plugins/ directory.
# The link name is the plugin name from your manifest.json.
cd shopdb-flask
ln -s ../../wjsf-shipping plugins/shipping
# (use an absolute path if you prefer: ln -s "$(pwd)/../wjsf-shipping" plugins/shipping)
#
# Windows: use a directory junction instead of ln -s. In an ADMIN prompt
# (or with Developer Mode on) from the shopdb-flask dir:
# mklink /D plugins\shipping ..\..\wjsf-shipping
# The plugin loader treats a junction the same as a real directory.
# 3. Set up the framework as usual.
python3 -m venv venv
venv/bin/pip install -r requirements.txt
venv/bin/pip install -r requirements-dev.txt
# 4. Install (enable) your plugin.
venv/bin/flask plugin install shipping
@@ -101,7 +112,7 @@ admits only the contract minor you tested against, not the whole 0.x line.
The current contract version is declared in `shopdb/__init__.py`:
```python
__contract_version__ = '0.6.0'
__contract_version__ = '0.13.0'
```
Recommended pin in your `manifest.json`, per ADR-002 (pip-style `>=,<` ranges):
@@ -111,7 +122,7 @@ Recommended pin in your `manifest.json`, per ADR-002 (pip-style `>=,<` ranges):
"name": "shipping",
"version": "1.0.0",
"description": "Tracks shipping-station scanners and label printers",
"core_version": ">=0.6.0,<0.7.0",
"core_version": ">=0.13.0,<0.14.0",
"dependencies": []
}
```
@@ -177,14 +188,14 @@ The full script:
# PLUGIN_DIR ($1) path to the plugin directory (holds manifest.json). Required.
# FRAMEWORK_REF ($2) git ref to test against in CI mode. Default: main.
# FRAMEWORK_URL framework git URL for CI mode.
# Default: https://gitea.proudtech.net/ge-aerospace/shopdb-flask.git
# Default: https://github.com/ge-aero/shopdb-flask.git
# LOCAL_FRAMEWORK path to an existing framework checkout. Set it to run offline.
set -eu
PLUGIN_DIR="${PLUGIN_DIR:-${1:-}}"
FRAMEWORK_REF="${FRAMEWORK_REF:-${2:-main}}"
FRAMEWORK_URL="${FRAMEWORK_URL:-https://gitea.proudtech.net/ge-aerospace/shopdb-flask.git}"
FRAMEWORK_URL="${FRAMEWORK_URL:-https://github.com/ge-aero/shopdb-flask.git}"
LOCAL_FRAMEWORK="${LOCAL_FRAMEWORK:-}"
if [ -z "$PLUGIN_DIR" ]; then
@@ -228,7 +239,7 @@ else
python3 -m venv "$WORKDIR/venv"
PYTHON="$WORKDIR/venv/bin/python"
"$PYTHON" -m pip install --upgrade pip >/dev/null
"$PYTHON" -m pip install -r "$FRAMEWORK/requirements.txt"
"$PYTHON" -m pip install -r "$FRAMEWORK/requirements-dev.txt"
fi
echo "==> Linking plugin '$PLUGIN_NAME' into framework plugins/"
@@ -312,14 +323,14 @@ jobs:
runs-on: ubuntu-latest
env:
FRAMEWORK_REF: v0.5.0
FRAMEWORK_URL: https://gitea.proudtech.net/ge-aerospace/shopdb-flask.git
FRAMEWORK_URL: https://github.com/ge-aero/shopdb-flask.git
steps:
- name: Check out the plugin
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.12'
python-version: '3.14'
- name: Fetch the harness from the framework
run: |
git clone --depth 1 --branch "$FRAMEWORK_REF" "$FRAMEWORK_URL" /tmp/framework

View File

@@ -5,6 +5,10 @@ running feature with its own list, detail, form, settings page, and report. It i
the companion to [PLUGIN-QUICKSTART.md](PLUGIN-QUICKSTART.md): the quickstart gets
you moving with `flask plugin new`; this guide explains *why* each piece looks the
way it does by walking the shipped code of the exemplar plugin.
> **Windows / VS Code:** command examples use the Linux venv path
> `venv/bin/python`; on Windows use `venv\Scripts\python` and
> `$env:FLASK_APP="shopdb"` (not `export`). Full Windows onboarding:
> [DEVELOPMENT-SETUP](DEVELOPMENT-SETUP.md).
`measuringtools` was chosen as the exemplar on purpose. It is the first plugin
built after the framework matured (ADR-005 scoped it; ADR-008 changed how plugin
@@ -74,7 +78,7 @@ plugin's identity (ADR-002):
Two fields deserve attention.
`core_version` is a semver range against the framework's `__contract_version__`
(declared in `shopdb/__init__.py`, currently `0.6.0`). The loader refuses to load
(declared in `shopdb/__init__.py`, currently `0.13.0`). The loader refuses to load
a plugin whose range excludes the running framework. We pin `>=0.6.0` because this
plugin uses the `get_reports` hook, which was added to the contract in 0.6.0
(see [PLUGIN-HOOKS.md](PLUGIN-HOOKS.md), "get_reports"). We cap at `<1.0.0` because
@@ -268,7 +272,7 @@ assertion to a frozen `CUTOVER_PLUGINS` list rather than to all discovered plugi
and to expect `measuringtools`'s real baseline revision in the upgrade-all test:
```python
CUTOVER_PLUGINS = ('computers', 'employees', 'equipment', 'knowledgebase',
CUTOVER_PLUGINS = ('computers', 'employees', 'knowledgebase', 'machines',
'network', 'notifications', 'printers', 'slides', 'usb', 'warranty')
@pytest.mark.parametrize('plugin', CUTOVER_PLUGINS) # not all plugins
@@ -277,7 +281,10 @@ def test_anchor_migration_is_noop(plugin):
```
Freezing the list (rather than deriving it) is intentional: a newly discovered
plugin should not silently be treated as a cutover no-op.
plugin should not silently be treated as a cutover no-op. Note that `machines`
(renamed from `equipment`, ADR-011) keeps its original cutover anchor but carries
a `machines0002rename` revision on top of it, so its expected head is that rename
revision rather than the bare anchor (`tests/test_plugin_migrations.py`).
---
@@ -330,20 +337,27 @@ def create_tool():
...
```
The `measuringtools.*` permissions are seeded exactly the way warranty seeds its
own, by adding them to `Permission.PERMISSIONS` in `shopdb/core/models/user.py`:
The `measuringtools.*` permissions belong to the plugin, not to core. The plugin
declares them from the `get_permissions` hook (contract 0.10.0) so core never edits
its catalog to accommodate a plugin:
```python
# Measuring tools
('measuringtools.view', 'View measuring tools', 'measuringtools'),
('measuringtools.create', 'Create measuring tools', 'measuringtools'),
('measuringtools.edit', 'Edit measuring tools', 'measuringtools'),
('measuringtools.delete', 'Delete measuring tools', 'measuringtools'),
class MeasuringToolsPlugin(BasePlugin):
def get_permissions(self):
return [
('measuringtools.view', 'View measuring tools', 'measuringtools'),
('measuringtools.create', 'Create measuring tools', 'measuringtools'),
('measuringtools.edit', 'Edit measuring tools', 'measuringtools'),
('measuringtools.delete', 'Delete measuring tools', 'measuringtools'),
]
```
`flask seed permissions` is idempotent, so re-running it just adds the four new
rows. The `admin` role bypasses every permission check, so an admin can operate the
plugin before anyone grants the granular permissions.
Installing or enabling the plugin seeds these rows automatically, and
`flask seed permissions` (which now seeds core plus every enabled plugin) is
idempotent, so re-running it just adds any missing rows. The `admin` role bypasses
every permission check, so an admin can operate the plugin before anyone grants the
granular permissions. See `get_permissions` in `docs/PLUGIN-HOOKS.md` for the
disabled-plugin edge case.
**Responses use the framework helpers.** `success_response`, `error_response` (with
`ErrorCodes`), and `paginated_response` produce the standard envelope
@@ -488,23 +502,25 @@ the plugin small.
## 9. Frontend integration
There is no frontend plugin system yet (see
[ADR-009](adr/ADR-009-frontend-plugin-gating.md), "Future direction"). A plugin's
Vue routes and views ship in the core bundle. The plugin's job is to add them
correctly and gate them.
A plugin ships its own Vue routes and views under
`plugins/<name>/frontend/` (ADR-010's frontend plugin hook contract). At build
time those files are staged into `frontend/src/.plugins-staged/<name>/` so the
core bundle picks them up; you author them in the plugin tree, not in core
`frontend/src/`. The plugin's job is to add them correctly and gate them.
**Route module with `meta.plugin` gating (ADR-009).** A new file
`frontend/src/router/routes/measuringtools.js` is auto-discovered by the router's
**Route module with `meta.plugin` gating (ADR-009).** The file
`plugins/measuringtools/frontend/routes.js` is staged into
`frontend/src/.plugins-staged/measuringtools/` and auto-discovered by the router's
`import.meta.glob('./routes/*.js')`. Every route carries `meta.plugin =
'measuringtools'`:
```js
export default [
{ path: 'measuringtools', name: 'measuringtools',
component: () => import('../../views/measuringtools/MeasuringToolsList.vue'),
component: () => import('./views/MeasuringToolsList.vue'),
meta: { plugin: 'measuringtools' } },
{ path: 'measuringtools/new', name: 'measuringtool-new',
component: () => import('../../views/measuringtools/MeasuringToolForm.vue'),
component: () => import('./views/MeasuringToolForm.vue'),
meta: { requiresAuth: true, plugin: 'measuringtools' } },
{ path: 'measuringtools/:id', ..., meta: { plugin: 'measuringtools' } },
{ path: 'measuringtools/:id/edit', ..., meta: { requiresAuth: true, plugin: 'measuringtools' } },
@@ -525,22 +541,21 @@ form is `requiresAuth`; the settings subtype page is `requiresAuth + requiresAdm
`update`, `remove`, `calibrationReport`, and a nested `types` CRUD). Do not
reorganize the file; just add the block, mirroring `machinesApi`.
**Views mirror the master templates.** The frontend has master templates the
frontend CLAUDE.md points to (`PrintersList.vue` for lists, `PrinterDetail.vue`
for detail pages). `measuringtools` mirrors the equivalent equipment views:
**Views mirror the master templates.** The frontend has master templates
(`PrintersList.vue` for lists, `PrinterDetail.vue` for detail pages). `measuringtools` mirrors the equivalent equipment views:
- `views/measuringtools/MeasuringToolsList.vue` - table with search, a type filter,
- `plugins/measuringtools/frontend/views/MeasuringToolsList.vue` - table with search, a type filter,
and a calibration-status filter; the status badge uses `utils/colorStyle` with
the color the API derived.
- `views/measuringtools/MeasuringToolDetail.vue` - hero + Identity card +
- `plugins/measuringtools/frontend/views/MeasuringToolDetail.vue` - hero + Identity card +
Calibration card (with the derived badge) + Location card, plus the shared
`CustomFieldsSection` and `WarrantyPanel` (section 10).
- `views/measuringtools/MeasuringToolForm.vue` - asset core fields + type +
- `plugins/measuringtools/frontend/views/MeasuringToolForm.vue` - asset core fields + type +
location + the calibration fields, plus `CustomFieldsInputs`.
- `views/reports/CalibrationReport.vue` - the four buckets (overdue / due soon /
- `plugins/measuringtools/frontend/views/CalibrationReport.vue` - the four buckets (overdue / due soon /
current / unknown), mirroring `WarrantyReport.vue`.
**Settings subtype page.** `views/settings/MeasuringToolTypesList.vue` mirrors
**Settings subtype page.** `plugins/measuringtools/frontend/views/MeasuringToolTypesList.vue` mirrors
`PCTypesList.vue`: add / edit / delete with a `ColorSwatchPicker`. It is linked from
`settingsNav.js` with a "Measuring Tools" card group, so it appears in the settings
rail and landing overview.
@@ -568,7 +583,7 @@ detail page drops in `<CustomFieldsSection :assetid="tool.assetid" />` and the f
drops in `<CustomFieldsInputs :assettypeid="assettypeid" :assetid="currentAssetId" />`,
then calls `customFieldsRef.value.save(assetId)` after the tool saves. The one
subtlety: `CustomFieldsInputs` needs the asset-type id. Rather than hardcode it (the
equipment form hardcodes `EQUIPMENT_ASSETTYPEID = 1`), the measuringtools form
machines form hardcodes `MACHINE_ASSETTYPEID = 1`), the measuringtools form
resolves it dynamically from `GET /api/assets/types`, finding the row whose
`assettype === 'measuring_tool'`. Dynamic lookup is preferred because seeded ids are
not stable across sites.
@@ -666,7 +681,7 @@ When you build a plugin, confirm all of this before you call it done:
- [ ] Imports only from `shopdb.api` and `shopdb.plugins.base` (contract test green).
- [ ] Blueprint: jwt-optional reads, permission-gated writes; framework response and
pagination helpers; audit logs on writes.
- [ ] Permissions added to `Permission.PERMISSIONS`; `flask seed permissions` run.
- [ ] Permissions declared from the `get_permissions` hook; install/enable (or `flask seed permissions`) seeds them.
- [ ] `on_install` seeds the asset type and any reference data, idempotently.
- [ ] Hooks: navigation, reports, models implemented; config schema and collector
implemented or consciously skipped with a reason.

View File

@@ -9,7 +9,7 @@ The contract is locked in [ADR-001](../docs/adr/ADR-001-asset-as-platform-contra
The framework declares its contract version in `shopdb/__init__.py`:
```python
__contract_version__ = '0.6.0'
__contract_version__ = '0.15.0'
```
Each plugin's `manifest.json` declares the range of contract versions it supports:
@@ -174,7 +174,9 @@ isolated in prod, re-raised in dev/test).
### `get_navigation_items() -> List[Dict]`
Returns navigation menu items.
Returns navigation menu items. A plugin owns its own sidebar entry here, so it
appears when the plugin is installed and disappears when it is not (including in
a lean per-site build that omits the plugin).
```python
class ComputersPlugin(BasePlugin):
@@ -187,6 +189,26 @@ class ComputersPlugin(BasePlugin):
}]
```
**Placement.** `position` (int) sets both the sort order and which section the
item lands in - the core sidebar (`AppLayout.vue:buildNavItems`) assigns section
headers by position range:
| `position` | section |
|-----------|---------|
| `< 10` | top, above any header (Dashboard is 0, Map is 4) |
| `10-29` | **Assets** |
| `30-49` | **Information** |
| `>= 50` | trailing, below Information |
Lower number sorts higher within a section. An explicit `'section':
'information'` forces the Information group regardless of position. Only these
two named sections exist; a new section needs a core edit to `buildNavItems`.
The **Displays** group (kiosk/TV links) is hardcoded in `AppLayout.vue`, not
plugin-driven.
`icon` is a string key mapped to a Lucide component core-side (same idea as
`get_settings_cards`); an unknown key renders with no icon.
> Removed in contract 0.4.0: `get_searchable_fields`. Global search
> (`/api/search`) is a core concern that queries the asset model directly and
> already covers every bundled asset type; no plugin ever implemented the hook.
@@ -216,6 +238,139 @@ Consumed by `GET /api/reports`, which merges plugin cards after the static core
reports sorted into category groups by the frontend (disabled plugins are
skipped; a broken plugin is isolated in prod, re-raised in dev/test).
### `get_permissions() -> List`
Returns the RBAC permissions this plugin owns. Added in contract 0.10.0. A
plugin declares the permission names its own routes enforce via
`require_permission`, instead of core accumulating every plugin's permissions in
one catalog (plugin-is-the-product).
Each entry is a `(name, description, category)` tuple, matching the core
permission catalog shape (dicts with those keys are also accepted). Names follow
the naming convention (lowercase dotted, e.g. `machines.edit`).
```python
class MachinesPlugin(BasePlugin):
def get_permissions(self):
return [
('machines.view', 'View machines', 'machines'),
('machines.create', 'Create machines', 'machines'),
('machines.edit', 'Edit machines', 'machines'),
('machines.delete', 'Delete machines', 'machines'),
]
```
Consumed by the core helper `full_permission_catalog()` (core permissions plus
every ENABLED plugin's `get_permissions()`), which backs three consumers:
- `flask seed permissions` seeds the full catalog.
- The role-management grid (`GET /api/users/permissions`) lists it, grouped by
category.
- API-token scope validation (`ApiToken.unknown_scope_names`) accepts a plugin
permission as a scope only while that plugin is enabled.
Plugin install and enable also seed the plugin's own permissions idempotently,
so enabling a fresh plugin creates its `Permission` rows without a separate seed
pass.
Disabled-plugin edge case: a disabled plugin is skipped by the catalog, so its
permissions are no longer offered for new scope grants or new role assignments.
The `Permission` ROWS already in the database are NOT deleted, so roles that
already reference them keep working until an admin edits the role. A broken
plugin is isolated in prod and re-raised in dev/test.
### `get_settings_cards() -> List[Dict]`
Returns settings-catalog card definitions. Added in contract 0.7.0 (ADR-010).
Each card is merged into the settings rail and landing overview without the
plugin hand-editing the core `settingsNav.js` catalog. `icon` is a string key
mapped to a Lucide component core-side, exactly like `get_navigation_items`.
```python
class MeasuringToolsPlugin(BasePlugin):
def get_settings_cards(self):
return [{
'group': 'Measuring Tools', # rail group title (created if new)
'to': '/settings/measuringtooltypes',
'icon': 'ruler', # string key, mapped core-side
'title': 'Measuring Tool Types',
'description': 'Manage measuring-tool subtypes + map colors',
'position': 22, # order within the group
}]
```
Consumed by `GET /api/pluginui/settings-cards`, which merges enabled plugins'
cards into the core catalog (disabled plugins are skipped; a broken plugin is
isolated in prod, re-raised in dev/test).
### `get_asset_panels() -> List[Dict]`
Returns asset-detail extension-panel definitions. Added in contract 0.7.0
(ADR-010). A generic core `AssetPanel` component renders each panel on the
matching detail pages, fetching the panel's `endpoint`. This replaces
hand-composing a plugin panel component into each detail view.
```python
class WarrantyPlugin(BasePlugin):
def get_asset_panels(self):
return [{
'id': 'warranty',
'title': 'Warranty',
'assettypes': ['*'], # detail pages it appears on; ['*'] = all
'endpoint': '/api/warranty/asset/{assetid}',
'render': 'table', # 'keyvalue' | 'table' | 'badge'
'position': 30,
}]
```
Consumed by `GET /api/pluginui/asset-panels?assetid=<id>`, which returns the
panels whose `assettypes` match that asset's type (disabled plugins skipped;
broken plugin isolated in prod, re-raised in dev/test). A panel that needs
bespoke UI (a chart) is out of scope for this data-only hook.
### `get_map_overlays() -> List[Dict]`
Returns shop-floor map overlay/decoration definitions. Added in contract 0.7.0
(ADR-010). The map stays data-driven off asset types + positions; an overlay
adds decoration data (a badge or ring) plus an optional legend entry, with no
plugin-side map code.
```python
class MeasuringToolsPlugin(BasePlugin):
def get_map_overlays(self):
return [{
'id': 'calibration-due',
'label': 'Calibration due', # legend label
'endpoint': '/api/measuringtools/map-overlay', # -> [{assetid, color, label}]
'style': 'badge', # 'badge' | 'ring'
'legend': True,
}]
```
Consumed by `GET /api/pluginui/map-overlays` (disabled plugins skipped; broken
plugin isolated in prod, re-raised in dev/test).
### `get_asset_presentation() -> List[Dict]`
Returns asset-type presentation/routing definitions. Added in contract 0.7.0
(ADR-010). Declares how a plugin-owned asset type renders in global-search rows
and cross-links (which icon, which detail route), so core never hardcodes a
plugin's route or icon.
```python
class MeasuringToolsPlugin(BasePlugin):
def get_asset_presentation(self):
return [{
'assettype': 'measuring_tool', # AssetType.assettype key the plugin owns
'icon': 'ruler',
'label': 'Measuring Tool',
'route': '/measuringtools/{assetid}',
}]
```
Consumed by `GET /api/pluginui/asset-presentation` (disabled plugins skipped;
broken plugin isolated in prod, re-raised in dev/test).
### `get_provisioning_note() -> Optional[Dict]`
Transparency note the setup wizard shows the moment a site checks this plugin
@@ -332,12 +487,40 @@ What `shopdb.api` exposes:
- Model bases: `BaseModel`, `AuditMixin`
- Core models: `Asset`, `AssetType`, `AssetStatus`, `Vendor`, `Model`,
`Communication`, `CommunicationType`, `Location`, `Setting`, `AuditLog`,
`Application`, `AppVersion`, `OperatingSystem`
`Application`, `AppVersion`, `OperatingSystem`, `AssetRelationship`,
`RelationshipType`
- Responses: `success_response`, `error_response`, `paginated_response`,
`ErrorCodes`
- Pagination: `get_pagination_params`, `paginate_query`
- Helpers: `audit_log`, `resolve_asset_position`
- Authorization: `require_permission`, `require_role`,
`service_token_authorized`
(`service_token_authorized(scope)` returns True when the request carries a
managed service token scoped for `scope` whose owner holds that permission -
for unattended plugin endpoints like the GE-Enforce fetch API)
- `authorized_service_token(scope)` (0.15.0) - same check as
`service_token_authorized` but returns the `ApiToken` itself (or None), so a
plugin can honor the token's optional resource binding
(`token.resourcescopelist`: an allowlist of resource names the token may
reach, NULL = unrestricted). GE-Enforce uses it to pin a display's fetch
token to its own manifest scope + that scope's blobs.
- Helpers: `audit_log`, `resolve_asset_position`, `resolve_dualpath_pairs`,
`dualpath_single_machine_enabled`
- Import mode: `apply_import_timestamps`, `import_mode_active`,
`parse_import_datetime`
- Legacy employee directory: `employee_connection`
- CMMC USB check-in/out DB (read-write, used by the usb plugin):
`cmmc_usb_connection`
- `User` / `Role` (0.13.0) - the account and role models, e.g. resolving
alert recipients' emails from selected user ids or role membership
- `SupportTeam` (0.15.0) - the support-team model (carries a `webhookurl`), so
an alerting plugin can route a notification to a chosen team's Teams webhook
- Mailer (0.12.0): `send_email(to, subject, html, text=None)` and
`send_alert(subject, html, text=None)` - settings-first, no-op safe when
email is unconfigured; send_alert targets the site's alert_recipients
- `send_webhook(title, text)` (0.14.0) - POST an alert to the configured
`alert_webhook_url` (Teams Incoming Webhook / Workflow, or generic JSON via
the `alert_webhook_format` setting); best-effort, no-op when unset.
`send_alert` fans out to this automatically alongside email.
```python
from shopdb.api import db, Asset, AssetType, success_response, paginate_query
@@ -387,6 +570,55 @@ position = resolve_asset_position(asset)
See [ADR-001](../docs/adr/ADR-001-asset-as-platform-contract.md) for the position resolution algorithm.
### Dualpath single-machine collapse
A Dualpath relationship pair is one physical dual-bay machine recorded as two
asset rows. When the site setting `dualpath_single_machine` is on (default), the
machines list, dashboard/report counts, and the floor map show the pair as a
single machine (the SECONDARY bay is hidden); the data model always keeps both
rows and detail pages stay per-bay.
```python
from shopdb.api import resolve_dualpath_pairs, dualpath_single_machine_enabled
collapse = resolve_dualpath_pairs()
# collapse.secondaryassetids: set of the non-primary bay asset ids to hide
# collapse.partnerbyasset: {assetid -> {'assetid', 'assetnumber'}} for every
# pair member (primary and secondary), for banners
if dualpath_single_machine_enabled():
# exclude the hidden bays and annotate the visible (primary) bay
...
```
PRIMARY is the pair member with the lower natural-sort assetnumber.
`resolve_dualpath_pairs` ignores the toggle (so a detail-page sibling banner can
show always); gate the collapse itself on `dualpath_single_machine_enabled()`.
### Import mode (legacy timestamp passthrough)
Bulk imports from the classic ASP shopdb need to preserve each row's original
`createddate` / `modifieddate` instead of stamping "now". `apply_import_timestamps`
does this, gated so it never affects normal traffic: it only acts when the
caller is an admin AND sent the `X-Import-Mode: true` request header.
```python
from shopdb.api import apply_import_timestamps
asset = Asset(assetnumber=data['assetnumber'], ...)
db.session.add(asset)
# In import mode, stamp legacy createddate/modifieddate from the payload.
# No-op for normal callers, or when the payload omits the fields.
apply_import_timestamps(asset, data)
db.session.commit()
```
`import_mode_active()` returns the same admin-plus-header predicate, for guarding
other backdated behavior (for example accepting a historical `checkouttime`).
`parse_import_datetime(value)` parses both ISO `2020-01-05T12:00:00` and legacy
`YYYY-MM-DD HH:MM:SS` into naive UTC. See [docs/IMPORT-API.md](IMPORT-API.md) for
the full migration operator manual.
## Removed hooks
The following hooks existed in early drafts and have been removed for v1:

File diff suppressed because it is too large Load Diff

View File

@@ -3,6 +3,12 @@
Build a working shopdb-flask plugin in 30 minutes. This walks through generating, customizing, installing, and testing a plugin from scratch.
For the full hook reference, see [PLUGIN-HOOKS.md](PLUGIN-HOOKS.md).
> **Windows / VS Code:** command examples below use the Linux venv path
> `venv/bin/python`; on Windows use `venv\Scripts\python` and
> `$env:FLASK_APP="shopdb"` (not `export`). Full Windows onboarding:
> [DEVELOPMENT-SETUP](DEVELOPMENT-SETUP.md).
For the architectural decisions behind the contract, see [docs/adr/](../docs/adr/).
## Step 1: Generate the skeleton
@@ -85,13 +91,14 @@ audit_log(action='created', entitytype='Camera', entityid=asset.assetid, entityn
## Step 4: Install the plugin
First add `plugins/cameras/migrations/` with a per-plugin Alembic chain that creates the plugin's tables, and register those tables in `PLUGIN_TABLE_OWNERS` (per ADR-008; the plugin chain owns plugin schema, never the core chain). Then:
```bash
flask plugin install cameras
flask db migrate -m "Add cameras plugin tables"
flask db upgrade
flask plugin upgrade-all
```
`install` runs the plugin's `on_install` hook (which seeds the AssetType row), registers it in the plugin registry, and runs migrations.
`install` runs the plugin's `on_install` hook (which seeds the AssetType row), registers it in the plugin registry, and runs the plugin's own migration chain. `flask db migrate`/`flask db upgrade` is reserved for core tables and must not be used for plugin schema.
## Step 5: Verify it works
@@ -122,6 +129,11 @@ Override hooks on the plugin class as needed. See [PLUGIN-HOOKS.md](PLUGIN-HOOKS
| `get_navigation_items` | Plugin shows up in the sidebar nav |
| `get_dashboard_widgets` | Plugin's dashboard widget appears on the home page |
| `get_reports` | Plugin's report cards appear on the Reports hub |
| `get_settings_cards` | Plugin's card joins the settings rail + landing (no `settingsNav.js` edit) |
| `get_permissions` | Plugin's RBAC permissions join the catalog, seeding, role grid, and token scopes |
| `get_asset_panels` | Plugin panel renders on matching asset-detail pages |
| `get_map_overlays` | Plugin decorates shop-floor map markers + adds a legend entry |
| `get_asset_presentation` | Plugin declares its asset type's search icon + detail route |
| `get_collector_schema` + `apply_collector_payload` | Plugin accepts external pushes at `/api/collector/<name>` |
Each hook has a default that does nothing. Override only what your plugin needs.

141
docs/PLUGIN-SIGNING.md Normal file
View File

@@ -0,0 +1,141 @@
# Plugin signing and packaging (curator guide)
ADR-013 Phase 1. How a plugin becomes a signed, verifiable artifact and how a
site trusts it. The signature proves an artifact is EXACTLY what a curator
reviewed and signed - it does not prove the code is safe. Human review before
signing is the actual safety control; the signature makes that review's verdict
tamper-evident all the way to the point of execution.
Requires the `cryptography` package (already a dependency).
## One-time: create the publisher key pair
```
flask plugin keygen --out ./keys --name curator
```
Writes `keys/curator.key` (PRIVATE) and `keys/curator.pub` (public).
- Keep the `.key` OFFLINE with the curator. It is the only thing that can sign a
trusted artifact. Never put it on the plugin shelf or in the repo.
- Distribute the `.pub` with each site's deployed config and pin it (below).
- Rotation: generate a new pair, pin BOTH public keys on sites for an overlap
window (`verify` accepts any trusted key), then retire the old one.
## Per plugin: review, then pack
1. Review the plugin's source. This is the security gate - read what it does.
2. Validate and package in one step:
```
flask plugin pack printers --key ./keys/curator.key --publisher west-jefferson
```
`pack` refuses to sign a directory that does not validate (manifest schema,
name/dir match, core_version, dependencies on disk). On success it writes
`printers-<version>.shopdbplugin` - a zip of the plugin plus:
- `PROVENANCE.json`: name, version, publisher, created, and a sorted
`{file: sha256}` map of every packaged file.
- `PROVENANCE.sig`: a detached ed25519 signature over the exact
`PROVENANCE.json` bytes.
3. Publish the artifact to the shelf (a SharePoint-synced or copied folder).
Transport is untrusted; the signature is what makes it safe.
## Verify an artifact
```
flask plugin validate dist/printers-1.0.0.shopdbplugin --pubkey ./keys/curator.pub
```
Checks, fail-closed: signature against the trusted key(s), every file's hash,
no unexpected files, manifest schema, and that the plugin's `core_version`
admits this framework's contract version. Any changed byte in any file fails
the hash check; a signature from an untrusted key fails the signature check.
## Pin trusted keys on a site
Set `PLUGIN_TRUSTED_KEYS` to one or more public-key PEM paths, separated by the
OS path separator (`:` on Linux, `;` on Windows), in the site's environment:
```
PLUGIN_TRUSTED_KEYS=/etc/shopdb/keys/curator.pub:/etc/shopdb/keys/curator-next.pub
```
Keys are read only from this deployed config, never from the shelf - a folder an
attacker could write must not also carry the keys that authenticate it. With no
keys set, `validate` on an artifact fails closed (unverifiable).
## Enforce signatures (Phase 2)
By default nothing is enforced - plugins load unsigned, as before. To require
signatures on a site:
1. Stamp the plugins the image ships with, so verify-at-load applies to them
too (run at image build with the site/build key):
```
flask plugin stamp-bundled --key ./keys/curator.key
```
This writes `PROVENANCE.json` + `PROVENANCE.sig` into each in-tree plugin.
2. Pin the public key(s) and turn enforcement on (site config):
```
PLUGIN_TRUSTED_KEYS=/etc/shopdb/keys/curator.pub
PLUGIN_REQUIRE_SIGNED=true
```
Now a plugin only loads or migrates when its tree matches a trusted signature.
Under enforcement a `sys.meta_path` guard verifies EVERY `plugins.<name>.*`
import (not just `plugin.py`) - including the `from plugins.<name>.models import
...` that core request handlers do - against the plugin's signed provenance, and
executes the exact bytes it hashed (never a `.pyc`). An unsigned, tampered, or
wrong-key plugin is refused, fail-closed. `PLUGIN_DEV_TRUST_DIRS` exempts named
directories, but ONLY under DEBUG/TESTING (the external-repo dev workflow);
production ignores it.
Because every `plugins.*` import is verified, `stamp-bundled` must cover EVERY
plugin directory present (its no-argument form does), not only the enabled ones
- core code can import a disabled plugin's module, and an unstamped one would be
refused.
Defense in depth - set filesystem permissions so the app's runtime user CANNOT
write the `plugins/` directory (owned by the deploy user). Import-time
verification closes the "attacker drops a file, a request imports it" path; a
strict read-only `plugins/` also closes the narrow verify-vs-migrate race where
an attacker with concurrent write to `plugins/` swaps a migration script between
the check and alembic re-reading it.
## The shelf and adopt (Phase 2)
A shelf is a read-only folder of artifacts plus a signed index. The app reads
`PLUGIN_SHELF_DIR`; it never talks to SharePoint - a sync (or robocopy/USB)
populates that folder, and the signature makes the transport untrusted and
interchangeable.
Publish (curator, after packing artifacts into the shelf folder):
```
flask plugin shelf-build --dir /srv/shelf --key ./keys/curator.key --serial 3
```
The index carries a monotonic `serial` (a site refuses an index older than the
last it saw) and a `revoked` list (carried forward across builds). Bump
`--serial` on every publish.
On a site:
```
flask plugin shelf-list # browse (verifies index + serial)
flask plugin adopt printers # or printers==1.2.0
flask plugin audit # warn if an installed version is revoked
```
`adopt` verifies the shelf index and the artifact (signature + every file
hash), unpacks into a staging area, re-verifies, then atomically moves it into
place and installs + enables it with its dependency closure. It refuses a
downgrade unless `--force-downgrade`. Run `flask plugin upgrade-all` and restart
afterward so migrations apply and routes register.

View File

@@ -10,11 +10,24 @@ These plugins are in `plugins/` in this repo. Enable per site with `flask plugin
|--------|--------|-------|
| `machines` | Manufacturing machinery: 5-axis mills, lathes, broachers, heat treatment ovens | Manually entered. See [ADR-005](adr/ADR-005-equipment-vs-measuringtools.md). Subtype tables for FOCAS / CLM / MTConnect controller protocols (planned). |
| `computers` | Shop-floor PCs and engineering workstations | Fed by the PXE pipeline collector per [ADR-006](adr/ADR-006-collector-contract.md). |
| `printers` | Network and shop-floor printers | Optional Zabbix integration for supply tracking. Legacy `PrinterData` retiring per ADR-001. |
| `printers` | Network and shop-floor printers | Public installer map page + fleet install contract (`/api/printers/install-list`, `/pc-default`, `/install-batch`; see [PRINTER-INSTALLER.md](PRINTER-INSTALLER.md)). Optional Zabbix integration for supply tracking. Legacy `PrinterData` retiring per ADR-001. |
| `network` | Switches, routers, access points, IDFs as locations | Asset-only; cleanest of the bundled set. |
| `usb` | USB devices issued to shop-floor users | Lightweight checkout / check-in. |
| `notifications` | Shop-floor notifications, recognitions, kiosk feed | Used by `ShopfloorDashboard.vue`. |
| `measuringtools` | Metrology and inspection instruments: calipers, micrometers, thread/bore/height gages, indicators | Per [ADR-005](adr/ADR-005-equipment-vs-measuringtools.md). Calibration lifecycle with derived status. First plugin built on the matured scaffold; its walkthrough is [PLUGIN-GUIDE.md](PLUGIN-GUIDE.md). Ships `default_enabled: false`. |
| `employees` | Read-only employee directory lookup | Backed by a separate HR database. Ships `default_enabled: false`. |
| `geenforce` | GE-Enforce manifest store: imaging PC-type scopes and their install manifests (apps, scripts, files, registry, version gates) | Per [ADR-012](adr/ADR-012-geenforce-manifest-ownership.md). Served to the GE-Enforce client as JSON. Requires GE-Enforce lib >= 2.6 on target PCs. Ships `default_enabled: false`. |
| `knowledgebase` | Knowledge Base articles linking to external resources | Lightweight article store. |
| `printedparts` | 3D-printed parts inventory | Kiosk checkout / check-in. Ships `default_enabled: false`. |
| `slides` | Slides for the lobby display and shop-floor screensaver | Upload / reorder / delete per surface. Management gated on the `slides.manage` permission, grantable to non-admin curators; display routes are public. |
| `warranty` | Asset warranty tracking | Manual entry now, Dell / Lenovo / HP provider lookups later. Derived coverage status with report buckets. |
## Plugin permissions
Plugins may register their own permissions (for example `slides.manage`). Admins
implicitly hold them; grant them to specific roles or users to delegate curation
without admin. Each plugin's registered permissions appear in its `plugin.py`
`get_permissions()`.
## Building your own
@@ -62,11 +75,42 @@ which revisions each plugin has applied in `migrations_applied`.
For sister-site plugins (per [ADR-003](adr/ADR-003-plugin-distribution.md)):
- Plugin lives in its own git repo: `gitea.proudtech.net/<your-site>/<pluginname>`
- Plugin lives in its own git repo: `<git-host>/<your-site>/<pluginname>`
- Adopting site clones or symlinks into their `<repo>/plugins/<name>/`
- Plugin manifest declares `core_version` range matching the framework version they target
- Plugin readme explains: what it tracks, who maintains it, where to file issues
## Lean per-site builds
A site ships only the plugins it chose; a site that never wants printedparts /
usb / network never carries that code (see
[ADR-013](adr/ADR-013-plugin-catalog-and-lean-builds.md) and
[ADR-014](adr/ADR-014-schema-lean-per-site.md)). Three layers make a build lean:
- **Backend code** - `scripts/build-site.sh <profile>` stages `shopdb/core` plus
only the chosen plugins' directories (and their hard-dependency closure). A
plugin a site did not choose is absent from the backend tree.
- **Frontend code** - `SITE_PLUGINS=machines,printers npm run build` (via
`scripts/stage-frontend.mjs`) stages only those plugins' `frontend/` dirs and
codegens the route table. **Exception:** a `plugins/<name>/frontend/` dir with
**no `manifest.json`** is a CORE feature (e.g. `applications`), not a per-site
plugin, and is ALWAYS staged regardless of `SITE_PLUGINS` - otherwise a lean
build would lose a core page.
- **Database** - the shared core Alembic baseline creates every plugin's tables,
so a lean site provisions them and then drops the ones it does not use with
`flask plugin prune-schema` (ADR-014). Run it once at provisioning, after
`flask db upgrade` and `flask plugin upgrade-all`; see
[DEPLOY.md](DEPLOY.md).
**Menus follow the build, not a plugin flag.** The sidebar nav, the settings
rail, and the Displays links all gate on whether the target route was actually
staged into this build (the router's own route table), not on a registry
"enabled" flag. So a lean site never shows a menu entry that dead-ends on a
blank page - an omitted plugin's nav item, settings cards, and kiosk links all
disappear together. Shopfloor Dashboard is a core view but is gated on the
notifications plugin (its only data source), so it drops when notifications is
not in the build.
## Naming policy
Plugin names follow the framework's naming convention (lowercase concatenated, no underscores or dashes; full words preferred over acronyms). See [CONTRIBUTING.md](../CONTRIBUTING.md). Plugin name collisions across sites are not enforced; the convention recommends prefixing site-specific plugins with the site code (e.g., `wjsf-shippingstation`) when there is risk of overlap.

104
docs/PRINTER-INSTALLER.md Normal file
View File

@@ -0,0 +1,104 @@
# Printer installer map and install endpoints
How the shop-floor fleet installs network printers from shopdb-flask, replacing
the classic ASP `apiprinters.asp` / `apipcdefaultprinter.asp` / `installprinter.asp`
contract. Shopfloor 2.0 PCs cannot run unsigned `.bat` maps, so a signed
installer EXE (and the public web map page) drives installs from three endpoints
in the printers plugin.
- Server code: `plugins/printers/api/asset_routes.py`
(`printer_install_list`, `pc_default_printer`, `printer_install_batch`)
- Consumed as a fleet manifest entry: the `common` scope's `printer map`
entry (see `GE-ENFORCE-DISPLAY.md`).
All three endpoints are `@jwt_required(optional=True)`: an anonymous fleet
client works, and a logged-in browser (the public map page) works too.
---
## 1. The public map page
`PrinterInstallerMap` is a public (no-login) frontend page: the floor map with
printer hotspots positioned at each printer's `mapx` / `mapy`. The user clicks
the printers they want, and the page requests an install batch. The PC's default
printer is preselected via `pc-default`.
---
## 2. `GET /api/printers/install-list`
Flat, unpaginated list of active NETWORK printers. A printer counts as network
only if it has a hostname or a non-USB IP; USB-only printers are excluded.
Fields per row:
| Field | Notes |
|---|---|
| `printerid` | Printer id (the token `install-batch` takes). |
| `name` | Asset name, else asset number. |
| `machinenumber` | The asset number. |
| `windowsname` | Standardized Windows printer name. |
| `sharename` | Share / CSF name. |
| `hostname` | Print-queue host. |
| `ipaddress` | Primary IP (falls back to any communication row). |
| `vendorname` | Direct vendor, else the model's vendor. |
| `modelnumber` | Model name. |
| `installpath` | Installer path for this printer (see install-batch). |
| `iscsf` | CSF flag. |
| `locationname` | Location name, if the asset has one. |
| `mapx` / `mapy` | Floor-map hotspot position. |
`?format=text` returns a pipe-delimited line per printer, one printer per line,
with a fixed field order so the Inno / Pascal installer does a `split()` instead
of parsing JSON:
```
printerid|windowsname|vendorname|modelnumber|hostname|ipaddress|mapx|mapy
```
Any pipe or newline inside a value is neutralized to a space so the field count
stays fixed. The web map uses the default JSON.
---
## 3. `GET /api/printers/pc-default?machine=NNNN`
The PC's default printer, by machine (asset) number persisted at PXE enrollment.
Parity with classic `apipcdefaultprinter.asp`: the installer preselects a PC's
default-printer hotspot on the site-map wizard. The link is resolved through the
`defaultprinter` asset relationship (PC asset -> printer asset), so it stays
inside the contract surface (no cross-plugin model import).
Returns `{printerid, windowsname}`, or `{}` when the machine is unknown or has no
active default printer set.
`?format=text` returns one pipe-delimited line (`printerid|windowsname`), or an
EMPTY body when there is no default (so the installer's split yields nothing).
---
## 4. `GET /api/printers/install-batch?printerids=1,2,3`
Returns a self-deleting Windows `.bat` that installs the selected printers,
grouped the same way classic `installprinter.asp` grouped them:
- HP / Xerox: one universal `PrinterInstaller.exe /PRINTER="a,b,c"` call.
- Printers with a `.exe` `installpath`: a PowerShell `Invoke-WebRequest` download
(using the caller's Windows credentials, against the site base URL + the
IIS-served `/installers` folder) followed by running it `/SILENT`.
- No `installpath`, or a non-`.exe` payload (e.g. a `.zip`): listed as a manual
install rather than run blindly.
`printerids` is required, comma-separated; non-numeric tokens are ignored. An
empty / missing list is a validation error.
The install name preference is `windowsname`, else `sharename`, else the asset
name / number.
---
## 5. Fleet wiring
The `common` scope's `printer map` manifest entry (see `GE-ENFORCE-DISPLAY.md`)
lays down the signed installer that consumes these endpoints. The web map page
covers the same install flow for a human at a browser.

56
docs/PROJECT-REVIEW.md Normal file
View File

@@ -0,0 +1,56 @@
# ShopDB Flask - Project Health Review
As of HEAD `ecf4ef6` (2026-07-30), product `__version__ 0.7.0`, contract `__contract_version__ 0.15.0` (verified in `shopdb/__init__.py`).
## 1. Executive Summary
**Overall: healthy engineering, drifting focus.** The stated product vision ("plugin system is the product," the project brief) is delivered: all 7 refactor phases are done, 13 bundled plugins are contract-compliant, the per-plugin migration regime (ADR-008) is live and exercised (geenforce is at `geenforce0002blobs`, proving the post-cutover chain works in anger), and CI enforces naming, contract, and real-MySQL migration idempotency. Test count grew 1077 -> 1159 since the last the project brief snapshot.
The concern is not quality but trajectory. the project brief and ROADMAP.md both name the last big milestone before 1.0 as "legacy-ASP data import + production pilot." The loader is built and VM-validated (16 stages, `scripts/site_imports/wjf/`), but the prod run has not happened, and ~35 of the last 60 commits went to the GE-Enforce HTTPS cutover instead. That work is legitimate and high-value, but it is feature/fleet work on one plugin, and it has accumulated two process debts that violate the project's own discipline: the cutover playbook (`docs/geenforce-api-cutover.md`) is the only dirty file in the repo and is **untracked**, and the `prod-patch-geenforce` robocopy fast-path can leave prod ahead of git.
**Verdict: on track against standards, behind against goals.** The 1.0 gate has four items; only one is arguably done, and the roadmap doc does not know it.
## 2. Standards Compliance
| Rule | Status | Evidence |
|---|---|---|
| Naming (tables/columns/vars, CONTRIBUTING.md) | **MET** | `scripts/check-naming-and-style.sh` present and executable; dedicated `naming` job in the internal CI workflow. One borderline: `asset.py:175` documents a derived API key `location_name` with an underscore - not a DB-mirrored column so likely legal, but worth a glance since "response keys match column names exactly" is the spirit of the rule. |
| Plugin contract (manifest.json, BasePlugin, `shopdb.api` only) | **MET** | 13 `plugins/*/manifest.json` verified; contract test suites in `tests/`; contract bumped correctly to 0.15.0 for the geenforce resource-scope fetch tokens (75386d2), per ADR-002 discipline. |
| Migration ownership (ADR-008) | **MET** | `PLUGIN_TABLE_OWNERS` registry tested by `tests/test_plugin_migrations.py` (`EXPECTED_HEAD_REVISION` lines 47-51); `migrations-mysql` CI job does fresh utf8mb4 MySQL 8 upgrade + all plugin chains + second-upgrade-no-op assertion. The geenforce `0002blobs` revision shows the per-plugin chain is being used as designed, not just anchored. |
| Versioning/release discipline (ADR-007) | **AT RISK** | Tags through v0.7.0 exist and contract bumps are disciplined, but the documentation half of the procedure has drifted - see Gaps 3. |
| ADRs canonical, new priorities get an ADR | **AT RISK** | 14 ADRs present. But lean per-site builds are half-shipped (ADR-014 ACCEPTED and implemented; `default_enabled: false` on 5 plugins) while ADR-013, which defines the catalog/tiers/signed-artifact model those builds imply, is still PROPOSED. The GE-Enforce HTTPS cutover itself - a major architectural shift off the SMB share - lives in an untracked doc, not an ADR or ADR-012 amendment. |
| Style (plain ASCII, no emojis, comment discipline) | **MET** | Enforced by the same pre-commit hook + CI naming job. |
| Everything in git / repo as source of truth | **VIOLATED** | `docs/geenforce-api-cutover.md` untracked (only dirty file, verified `git status`); `prod-patch-geenforce` robocopy path acknowledged in-doc as leaving prod ahead of git. |
## 3. Roadmap Status
**Done:** Phases 0-6 (contract lock through multi-site distribution, tags v0.5.0-v0.7.0). Legacy import machinery complete: `docs/IMPORT-API.md` contract, 16-stage wjf loader VM-validated. Air-gapped deploy kit (6534590).
**1.0 must-haves (ROADMAP.md), honestly scored:**
1. *Asset model fully wired* - **appears DONE but unrecorded.** `Asset.mapx` (`shopdb/core/models/asset.py:121`), `inheritsposition` (`relationship.py:132`), and propagation logic (`relationship.py`, `core/api/assets.py`, `cli/__init__.py`) are all in code. ROADMAP still lists this as outstanding. Verify the ADR-001 contract tests cover it, then strike it.
2. *Equipment data migration one-shot* - **NOT DONE.** `scripts/migration/` contains only `fix_legacy_schema.sql`, `one-offs/`, and a README. No equipment script.
3. *Printers legacy-table cleanup* - **NOT DONE.** Recent printers commits (0d40780..c075658) are installer/feature work, not retirement.
4. *External plugin UI packaging* - **NOT DONE**, and gate criterion 3 (one external plugin built end-to-end) has no evidence.
**In-flight:** GE-Enforce HTTPS cutover dominates (~35/60 recent commits). Per the cutover doc's own section 12: only displays/kiosks are on the API; cmm/collections/keyence/genspect/heattreat/partmarker/common fleet still enforce from the SFLD SMB share; loggedinuser resolution unwired; registry cleanup pending; 3DPrintRoom route is a placeholder. Secondary streams: printers install-batch, applications notes, server reclassification, TV dashboard.
**Pace/scope health:** Velocity is high and test coverage tracks the work (17 of ~29 plugin test files are geenforce). But the project has been at 0.7.0 with "prod pilot is the last big milestone" as the stated goal since mid-July, while shipping ~185 commits of plugin-feature work. That is a real product being used - good - but the 1.0 gate is not moving, and a half-migrated fleet (API for displays, SMB for everything else) is the worst place to pause the cutover.
## 4. Gaps and Risks
1. **Untracked cutover playbook** (`docs/geenforce-api-cutover.md`). The single most valuable in-flight document is one `rm` away from gone, and invisible to any other machine or contributor.
2. **Prod-ahead-of-git debt.** The `prod-patch-geenforce` fast-path means production behavior may not be reproducible from any commit. This directly undermines ADR-012's "engine is source of truth" and the release discipline of ADR-007.
3. **Documentation drift, three concrete instances (all verified):** ROADMAP.md header says contract 0.13.0 (actual 0.15.0); the project brief says 1077 tests (actual 1159 collected) and claims a "lean-build" CI job that does not exist in the internal CI workflow (jobs: backend, naming, frontend, migrations-mysql - lean coverage is folded into pytest via `tests/test_lean_build_guards.py`). Also `.github/workflows/ci.yml` differs from the internal CI workflow - one of them is stale.
4. **Split-brain fleet enforcement.** Displays/kiosks on the API, the rest of the fleet on the SMB share, with staged-but-unpushed manifest fixes elsewhere (MTConnect v1 stranding). Two delivery mechanisms means two failure modes and doubles the audit surface until the cutover finishes.
5. **1.0 gate criterion 4 unproven:** `docs/DEPLOY.md` has not been validated by an actual fresh-host prod deploy. The air-gapped kit exists; the pilot does not.
6. **ADR-013 limbo:** lean builds shipped under ADR-014 while the catalog/signing model that makes external distribution safe remains PROPOSED. Fine short-term, but gate criterion 3 (external plugin) will force the question.
## 5. Prioritized Recommendations
1. **Commit `docs/geenforce-api-cutover.md` today.** Zero-cost, eliminates the worst single-point-of-loss risk.
2. **Reconcile prod-patched geenforce files back into git** and gate or retire the robocopy fast-path. Until prod == some tag, ADR-007 is fiction for this plugin.
3. **One doc-sync pass (30 min):** ROADMAP header to 0.15.0, the project brief test count and CI job list, strike must-have (a) if contract tests confirm the Asset wiring, delete or sync the stale `.github` workflow.
4. **Finish the cutover or park it cleanly.** Either drive the remaining fleet groups onto the API per section 12, or write down the frozen state as an ADR-012 amendment so the split-brain period is a documented decision, not drift.
5. **Schedule the prodscratch import run and prod pilot.** This is the actual 1.0 milestone and everything is built for it; it validates DEPLOY.md (gate 4) for free.
6. **Pair the equipment one-shot migration with printers retirement** (must-haves b and c) - they are coordinated by design; doing them together avoids touching the legacy tables twice.
7. **Decide ADR-013** before building the external-plugin end-to-end proof (gate 3); the geenforce client work is the natural seed for that external plugin.

191
docs/RELEASING-WINDOWS.md Normal file
View File

@@ -0,0 +1,191 @@
# Building and releasing the Windows installer
For whoever builds releases. If you run a server rather than build releases, see
[UPDATES-WINDOWS.md](UPDATES-WINDOWS.md).
Every release is one self-contained `.exe`. It carries the Python runtime, a
hash-checked wheelhouse, MySQL, the IIS modules and the application, so nothing
is fetched from the network at install time. That is the point: sites are
air-gapped.
## What you need
- A checkout of this repository.
- Inno Setup 6.6.0 or newer, on Windows. Compiling needs Windows; staging the
bundle does not.
- PowerShell, for the bundle lock tools.
- `uv`, only when dependencies change.
## The three kinds of change
Almost every release is the first kind.
### 1. Application change: bug fix, feature, no new dependency
```bash
# bump __version__ in shopdb/__init__.py first
./deploy/windows/installer/build-installer.sh deploy/site-profile-universal.json <repo-path>
```
`deploy/site-profile-universal.json` is the profile released builds come from:
every bundled plugin, so one `.exe` serves any site and the operator ticks what
that site uses. Build from a narrower profile only when a site genuinely needs a
lean build (ADR-013). Do not build a release from a profile that is not in the
repository - the build stops being reproducible the moment that file is
somewhere else.
Then on Windows, in `deploy/windows/installer`:
```
iscc ShopDBFlask.iss
```
`build-installer.sh` restages the application, rebuilds the web interface for
the chosen feature set, regenerates `plugins.iss` and `version.iss`, and
re-verifies the third-party payload against `bundle-lock.json`. The lock is
untouched, because nothing third-party changed.
### 2. A dependency is added, removed or moved
The only case that touches the lock.
```bash
# edit requirements.in, then
uv pip compile requirements.in --universal --generate-hashes -o requirements.txt
# fetch the wheel for the target runtime, into the wheelhouse
pip download <name>==<version> -d deploy/windows/installer/bundle/wheels \
--only-binary=:all: --platform win_amd64 --python-version 314 --no-deps
```
Then regenerate and commit the lock:
```powershell
pwsh ./refresh-bundle-lock.ps1 # review the change
pwsh ./refresh-bundle-lock.ps1 -Yes # write it
```
**Commit `bundle-lock.json`. That commit is the review.** The lock records every
third-party file and its SHA-256; the installer refuses to run if the payload it
carries does not match, so an unreviewed substitution cannot reach a server.
Two traps, both of which have already cost a release:
- `--universal` is not optional. A resolve done only for the host platform drops
packages that exist only on Windows, and the install then fails hash checking
on a package with no entry.
- Declare **every** runtime import in `requirements.in`, including ones that
already work locally. `packaging` reached this project only as a test
dependency, so the full suite passed while a production virtual environment,
which has no test dependencies, could not import the application at all.
`tests/test_runtime_dependencies.py` is the gate for this.
### 3. A new plugin
Ordinary plugin work, plus three installer-specific steps:
1. **Stage it.** Add the plugin to the profile you build from. `plugins.iss` is
generated from what is actually in the bundle, so the wizard offers it with
no edit to the installer script.
2. **Decide whether it is ticked by default.** `PluginDefault()` in
`ShopDBFlask.iss` is an exclusion list: a new plugin defaults to **ticked**
unless you name it there. This is the one manual edit.
3. **Register its migrations.** Update `PLUGIN_TABLE_OWNERS` and
`EXPECTED_HEAD_REVISION`, and give the plugin its own migration chain
(ADR-008). Existing sites pick up its tables through `plugin upgrade-all`;
sites that do not tick it have them pruned (ADR-014).
## Version numbers
`build-installer.sh` reads `__version__` from `shopdb/__init__.py` and writes
`version.iss` from it. Do not edit `version.iss`: a hardcoded version in the
installer script had already drifted two minor versions from the code.
Bump the version for every release you hand out. The installer compares versions
and:
- refuses a build **older** than what is installed, because migrations only go
forwards;
- warns and continues on an **equal** version, treating it as a repair.
Equal-version rebuilds are useful while testing and are a poor idea in the
field, since the server cannot then tell you what it is running.
Follow ADR-007 for what each part means.
## Do not hand-edit generated files
Regenerate these; changes are overwritten without warning:
- `version.iss` - from `shopdb/__init__.py`
- `plugins.iss` - from the plugins actually present in the bundle
- `requirements.txt` - from `requirements.in` via `uv pip compile`
- `bundle-lock.json` - via `refresh-bundle-lock.ps1`
`waitress` and `tzdata` were once hand-added to `requirements.txt` and vanished
on the next compile, taking the Windows runtime with them.
## Before you hand a build out
```bash
python -m pytest -q # full suite
./scripts/check-naming-and-style.sh # naming and style
```
The suite includes gates worth knowing about:
- `tests/test_runtime_dependencies.py` - every runtime import is declared
- `tests/test_bundle_lock.py` - the lock covers the payload
- `tests/test_docs_publishable.py` - `docs/` carries no internal references,
because it is published
`build-installer.sh` refuses to finish if the payload does not match the lock.
That check is not advisory; do not work around it.
## Publishing
Alongside the `.exe`, publish:
- a `.sha256` file, so the operator can verify what they received;
- release notes covering what changed and anything needing attention;
- the version in the filename, so a server's build is identifiable on sight.
The compiled installer stages a CycloneDX inventory (`sbom.cdx.json`) onto every
server, which is what answers a security question about a published
vulnerability without anyone guessing.
## Known gaps
**The installer is not code-signed.** Every install shows an unknown-publisher
warning, and the SHA-256 is the only integrity check. This is the significant
remaining gap before wider distribution: a checksum published next to the file
protects against corruption, not against someone who can write to that location.
The decision taken is to wait for a certificate from the organisation's own
certificate authority rather than buy one from a public CA. Every server this
installer runs on is centrally managed, and that CA's root is already trusted on
those machines, so an internally issued Authenticode certificate removes the
warning exactly where it matters. A public certificate would buy trust on
machines this software never reaches.
Until then, publish the SHA-256 through a channel SEPARATE from the installer
itself. A hash sitting beside the file is only as trustworthy as write access to
that location; a hash the operator gets another way means tampering has to
succeed twice.
Wiring it up afterwards is small: Inno has native SignTool support, so a
directive in the script and a signtool configuration on the build machine sign
the installer and its uninstaller. Include a timestamp server, or signatures
stop verifying when the certificate expires.
**Compiling requires Windows.** `build-installer.ps1` exists so the whole
process can run on a Windows workstation. Nothing about it runs in CI, so a
release is a deliberate act by a person.
## See also
- [UPDATES-WINDOWS.md](UPDATES-WINDOWS.md) - what operators should expect
- [INSTALL-WINDOWS.md](INSTALL-WINDOWS.md) - installing a new site
- [OPERATE-WINDOWS.md](OPERATE-WINDOWS.md) - running a site
- `docs/adr/` - ADR-007 versioning, ADR-008 plugin migrations, ADR-013 and
ADR-014 lean per-site builds

View File

@@ -1,6 +1,6 @@
# Roadmap
shopdb-flask is at `__contract_version__ = '0.5.0'` (pre-1.0). This document captures what stands between today and a stable `1.0.0` release. Maintained as scope evolves; supersedes nothing in the ADRs.
shopdb-flask is at `__contract_version__ = '0.13.0'` (pre-1.0; product `__version__ 0.7.0`, tags through v0.7.0). This document captures what stands between today and a stable `1.0.0` release. Maintained as scope evolves; supersedes nothing in the ADRs.
## Phase status
@@ -12,24 +12,22 @@ shopdb-flask is at `__contract_version__ = '0.5.0'` (pre-1.0). This document cap
| 3 - Manifest-first loader, shopdb.api namespace, auto-register blueprints | DONE | `6f085a1` |
| 4 - Plugin scaffolding (`flask plugin new`) | DONE | `8eb9362` |
| 5 - Alembic baseline, per-site deploy, ADRs to docs/adr | DONE | `d4e3ac9` |
| 6 - Multi-site distribution readiness (settings-driven branding/ServiceNow/floor plan, security closeout, docs + Docker frontend build, release engineering) | IN PROGRESS | this phase |
| 6 - Multi-site distribution readiness (settings-driven branding/ServiceNow/floor plan, security closeout, docs + Docker frontend build, release engineering) | DONE | v0.5.0-v0.7.0 |
The last big milestone before 1.0 is the legacy-ASP data import plus a production pilot deployment; the framework work below is what remains after that.
## What's left before tagging 1.0.0
### Must-have
- **Asset model fully wired**. `Asset.mapx, Asset.mapy` columns, `AssetRelationship.inheritsposition`, `AssetRelationship.propagatesthroughid` columns. Models match the locked ADR-001 surface that `resolve_asset_position` already targets.
- **Equipment data migration script** for facilities migrating from legacy ASP shopdb. One-shot script under `scripts/migration/`. See [migrating-asset-schema](../../.claude/skills/migrating-asset-schema.md) for the policy. Per ADR-001, only `category='Equipment' AND machinenumber IS NOT NULL` migrates.
- **Printers retirement**. Legacy `PrinterData` model, `printers_bp` legacy blueprint, and the frontend `PrinterForm.vue` references to `printer.printerdata.*` get removed in lockstep. Coordinated with the equipment migration.
- **Frontend hook contract**. Vue side equivalents for the backend hook system: how plugins expose asset-detail components, map markers, search-result renderers. Requires its own design ADR.
- **Per-plugin Alembic migrations**. The framework supports them via `shopdb/plugins/migrations.py`; bundled plugins still rely on `db.create_all()`. Move each bundled plugin onto its own version chain before sister sites adopt.
- **Asset model fully wired**. `Asset.mapx, Asset.mapy` columns, `AssetRelationship.inheritsposition` column, and the `relationshiptypepropagations` M:N table (`RelationshipTypePropagation` model; propagation lives on `RelationshipType`, not `AssetRelationship`). Models match the locked ADR-001 surface that `resolve_asset_position` already targets.
- **Equipment data migration script** for facilities migrating from legacy ASP shopdb. One-shot script under `scripts/migration/`. Per ADR-001, only `category='Equipment' AND machinenumber IS NOT NULL` migrates.
- **Printers retirement**. The printers plugin already runs on the asset architecture (blueprint `printers_asset_bp`); any remaining legacy printer-table cleanup is coordinated with the equipment migration.
- **External plugin UI packaging**. The Vue-side hook contract ships (ADR-010: get_settings_cards / get_asset_panels / get_map_overlays / get_asset_presentation) and route gating is backend-driven (ADR-009), but plugin routes/views still live in core `frontend/src`. Let an external plugin ship its own Vue bundle so adopters can add UI without editing core.
### Nice-to-have
- **Bundle the Roboto font locally.** `frontend/src/assets/style.css:2` imports Roboto from Google Fonts (`fonts.googleapis.com`). Air-gapped facilities have no route to that host, so the font silently falls back to a system font. Vendor the woff2 files into `frontend/src/assets/` and `@font-face` them locally so every site renders identically offline.
- **Full palette theming.** `brand_primary_color` is settings-driven, but the rest of the CSS palette (surfaces, borders, accents) is still hardcoded in `style.css`. A complete theming pass would expose the palette as CSS variables a site can override, not just the one primary color.
- **Frontend plugin contract.** The backend hook system has no Vue-side equivalent yet (routes/views still ship in core; nav is already backend-driven). See the must-have entry above; this is the design ADR that unblocks external plugins shipping their own UI.
- `measuringtools` plugin built using the scaffold (validates the scaffold under realistic conditions).
- **Full palette theming.** `brand_primary_color` and a few brand colors are settings-driven, but the rest of the CSS palette (surfaces, borders, accents) is still hardcoded in `style.css`. A complete theming pass would expose the palette as CSS variables a site can override, not just the brand colors.
- Frontend scaffolding skill (the backend has `flask plugin new`; the frontend stub is currently manual copy-paste).
- Marketplace listing site (PLUGINS.md is a one-pager; a proper listing with links to sister-site plugins becomes useful when there are more than three external plugins).
- Plugin contract surface diff tooling. Today version bumps are manual judgment; a CI check that diffs the contract surface against the previous tag would catch missed bumps. See ADR-002.
@@ -63,3 +61,9 @@ When a roadmap item gets prioritized, document the why in a new ADR and link fro
- [ADR-004](adr/ADR-004-deployment-topology.md) - Deployment topology (per-site)
- [ADR-005](adr/ADR-005-equipment-vs-measuringtools.md) - Equipment vs measuringtools
- [ADR-006](adr/ADR-006-collector-contract.md) - Collector contract pattern
- [ADR-007](adr/ADR-007-product-versioning-and-releases.md) - Product versioning and releases
- [ADR-008](adr/ADR-008-plugin-migration-ownership.md) - Plugin migration ownership (per-plugin chains)
- [ADR-009](adr/ADR-009-frontend-plugin-gating.md) - Frontend plugin route gating
- [ADR-010](adr/ADR-010-frontend-plugin-hooks.md) - Frontend plugin hook contract
- [ADR-011](adr/ADR-011-machines-rename.md) - Machines rename + modeltypes retyping
- [ADR-012](adr/ADR-012-geenforce-manifest-ownership.md) - GE-Enforce manifest ownership

199
docs/UPDATES-WINDOWS.md Normal file
View File

@@ -0,0 +1,199 @@
# What to expect from updates (Windows sites)
For the people who run a ShopDB-Flask server. It covers how updates arrive, what
they do to your data and your server, how long they take, and what happens to
anything else running on the same machine.
If you are the person building releases, see
[RELEASING-WINDOWS.md](RELEASING-WINDOWS.md).
## Updates arrive as one file
Every release is a single `.exe`, the same kind of file you used to install.
There is no patch, no separate updater, and no download step during the install:
everything the server needs is inside that one file, including the Python
runtime, all library code and the application itself.
To update, run the newer `.exe` on the server as Administrator. That is the
whole procedure.
The installer works out for itself that this is an update rather than a first
install, and skips what is already correct. An update typically takes two to
four minutes, most of which is the database migration.
## What an update changes, and what it leaves alone
Changed:
- The application code and the web interface.
- The database schema, brought forward by migrations.
- The Python runtime and libraries, but only when that release moves them.
Left exactly as they are:
- `.env`, which holds your database connection and secret keys.
- Your data. Updates migrate the schema; they do not reset or reload content.
- `web.config`, if you have edited it. The installer only ever repairs a
specific fault in it that older builds created, and copies the file aside
first when it does.
- Which features are switched on. The feature list opens showing what this site
already has.
- Uploaded files and anything under `instance\`.
Unticking a feature during an update does **not** remove it. Adding is a tick;
removing is a deliberate, separate step. This is so an upgrade can never quietly
delete a feature and its data.
## Downtime
The site is down for the length of the update, so two to four minutes. The
installer stops the application pool before replacing files, because Windows
will not let it overwrite files a running process holds open, and starts it
again afterwards.
There is no reboot. If a release ever needs one, the installer says so rather
than restarting the machine itself.
## Your data is backed up first
Before applying migrations, the installer takes a database backup and checks the
dump is readable. If a migration fails, it restores from that backup and tells
you.
This depends on `mysqldump` being present. Confirm it once, before your first
update:
```powershell
.\shopdb-admin.ps1 check
```
Without it the update still runs, but the pre-update backup is skipped, and that
is precisely the backup you would want if a migration went wrong.
Afterwards:
```powershell
.\shopdb-admin.ps1 status
.\shopdb-admin.ps1 verify
```
## Going backwards is refused
Installing an older build over a newer one is blocked outright. Once migrations
have moved the schema forward, older code cannot read it, and the failures are
difficult to unpick.
To go back you restore a backup taken before the update. Keep the previous
`.exe` until you are satisfied with a release.
## Will an update affect other sites on the same IIS server?
Mostly no, and the exceptions are listed here rather than glossed over.
**Isolated from other sites:**
- The application runs in its own application pool under its own identity, so a
crash or a memory leak cannot reach another site's pool.
- Its configuration lives in its own folder and applies only to its own URL
path. A parent site's own pages, including classic ASP, keep their existing
handlers.
- File permissions are granted on the application folder only.
**Shared, and therefore worth knowing about:**
- **A brief application pool recycle across the server.** Installing the IIS
modules and writing server-level configuration causes IIS to reload its
configuration, which recycles application pools. Requests in flight at that
moment can be dropped, and any session state other sites hold in memory is
lost. It is a few seconds and there is no service outage: IIS itself is never
stopped and no `iisreset` is issued.
- **Two IIS modules are installed machine-wide** the first time:
HttpPlatformHandler and URL Rewrite. Both are standard Microsoft modules. They
do nothing to a site that does not reference them, and if a site already uses
URL Rewrite its rules are untouched.
- **One rewrite server variable is permitted machine-wide**,
`HTTP_X_FORWARDED_FOR`, so the application can see real client addresses
instead of the loopback address. This grants exactly that one variable rather
than opening the section up.
- **A Microsoft C++ runtime** may be installed, which is shared and backwards
compatible.
- **The bundled database option installs MySQL on port 3306.** If this server
already runs MySQL, choose the existing-database option instead. Two servers
will collide on that port. The wizard asks before doing anything.
Removing ShopDB-Flask takes away its own site, application, pool, folder and
firewall rule. It deliberately leaves the shared IIS modules in place, because
another site may have started depending on them.
If your server hosts something critical, schedule updates in a maintenance
window for the pool recycle, not for the application itself.
## Security updates
Two kinds reach you, both as an ordinary `.exe`.
**Application fixes** are built from the source and shipped like any other
release.
**Third-party fixes** cover the Python runtime, the libraries, MySQL and the IIS
modules. These are pinned to exact versions and checked by cryptographic hash at
install time, so a release contains precisely the versions it claims and nothing
substituted. When one of them publishes a fix that affects this application, it
is picked up and a new release is issued.
Each release ships a machine-readable inventory of every third-party component
and its version, installed on the server as `sbom.cdx.json`. If your security
team asks whether you are exposed to a published vulnerability, that file
answers it without anyone guessing.
An update that is only a dependency bump is still worth taking: the version
number moves and the application behaviour does not.
## Check the file before running it
Each release publishes a SHA-256 checksum beside the `.exe`. Verify it:
```powershell
certutil -hashfile ShopDBFlask_Installer_<version>.exe SHA256
```
Compare with the published `.sha256` file.
Windows will warn about an unknown publisher, because the installer is not yet
code-signed. The checksum is the integrity check to rely on today. Get the file
from the agreed location rather than from mail or a message.
## Version numbers
Three parts, for example `0.7.0`:
- The last part changes for bug fixes and security fixes. Nothing you use
behaves differently.
- The middle part changes for new features. Existing features keep working.
- The first part changes for something that needs you to read the notes first.
Before 1.0 the middle number can still bring changes that need attention. Read
the release notes for those.
## If an update fails
The installer stops at the first problem rather than continuing, and says what
failed and what to do. Nothing is left half-applied: either the change is
complete or it is rolled back, and the log records everything either way.
The log is at:
```
C:\ProgramData\ShopDB-Flask\logs\shopdb-install-<timestamp>.log
```
Re-running the same `.exe` is safe and picks up from where it stopped. If it
fails again, send that log with your report; it names the failing step, the
exit code and the relevant output.
## See also
- [OPERATE-WINDOWS.md](OPERATE-WINDOWS.md) - day-to-day running
- [UPGRADE.md](UPGRADE.md) - upgrade notes across all deployment types
- [BACKUP-RESTORE.md](BACKUP-RESTORE.md) - what to back up and how to restore
- [INSTALL-WINDOWS.md](INSTALL-WINDOWS.md) - first-time install

View File

@@ -24,7 +24,7 @@ cp -a instance/ instance-backup-$(date +%F)/
git pull origin main
```
The application is distributed through the internal GE Aerospace Gitea; pull
The application is distributed through the internal GE Aerospace git server; pull
from there. There is no external image registry.
## Step 2: Rebuild
@@ -97,7 +97,7 @@ removes those bundled PNGs and ships a generic placeholder SVG instead.
If your instance's `map_blueprint_light` / `map_blueprint_dark` Settings still
point at `/static/images/sitemap2025-*`, the map will 404 those images after the
upgrade. Re-upload your own floor plan in **Settings > Map**. Uploaded floor
upgrade. Re-upload your own floor plan in **Settings > Floor Map**. Uploaded floor
plans are stored under `instance/` and survive upgrades, so a site that already
uploaded its own plan is unaffected. Only instances still using the old bundled
default need to act.
@@ -105,12 +105,43 @@ default need to act.
To check what your instance points at:
```bash
docker compose exec api flask shell -c "from shopdb.core.models.setting import Setting; print(Setting.query.filter(Setting.key.like('map_blueprint%')).all())"
docker compose exec -T api flask shell <<'PY'
from shopdb.core.models.setting import Setting
print(Setting.query.filter(Setting.key.like('map_blueprint%')).all())
PY
```
## Windows sites (installer-built)
Run a newer installer `.exe` over the existing install. That is the whole
procedure - none of the manual steps above apply.
[UPDATES-WINDOWS.md](UPDATES-WINDOWS.md) is the operator-facing version of this:
downtime, what is and is not touched, security updates, and the effect on other
sites sharing the same IIS server.
It backs the database up first and verifies the dump, applies the core and plugin
migrations, restores from that backup if a migration fails, and refuses to
install an older build over a newer one. Your `.env`, your data and your
`web.config` are kept.
Before the first upgrade, confirm `mysqldump` is available
(`.\shopdb-admin.ps1 check`). Without it the pre-upgrade backup is skipped, which
is the one you would want if a migration went wrong.
Afterwards:
```powershell
.\shopdb-admin.ps1 status
.\shopdb-admin.ps1 verify
```
See [OPERATE-WINDOWS.md](OPERATE-WINDOWS.md).
## See also
- [BACKUP-RESTORE.md](BACKUP-RESTORE.md) - what to back up and how to restore
- [INSTALL-WINDOWS.md](INSTALL-WINDOWS.md) - Windows Server install
- [CONFIG.md](CONFIG.md) - environment variables and Setting keys
- [DEPLOY.md](DEPLOY.md) - first-time deploy runbook
- `CHANGELOG.md` - what changed in each release

View File

@@ -183,7 +183,7 @@ Skipped from migration:
## References
- `shopdb/core/models/asset.py`
- `shopdb/core/models/machine.py` (legacy, deprecated)
- (core `Machine` model retired per this decision; machine data now owned by the machines plugin at `plugins/machines/models/machine.py`)
- `shopdb/plugins/base.py`
- ADR-002 (versioning of the surface)
- ADR-003 (plugin distribution)

View File

@@ -63,16 +63,21 @@ The framework provides:
## Migration strategy (resolved)
Deploys run a single core Alembic chain: `flask db upgrade`. Bundled plugins do
NOT carry their own migration chains - their tables are folded into the core
chain (migration `7c04_fold_plugin_schema`). This was a deliberate resolution of
the Phase 7B footgun where bundled-plugin baselines and the core baseline both
created the same tables, so `flask plugin upgrade-all` would conflict. A fresh
`flask db upgrade` reproduces the live schema exactly (verified on a scratch DB).
Deploys run two commands: `flask db upgrade` then `flask plugin upgrade-all`
(lean/ADR-014 sites add an optional `flask plugin prune-schema` at initial
provisioning). The core Alembic chain applied by `flask db upgrade` creates the
full core AND bundled-plugin schema through the chain head (this includes
migration `7c04_fold_plugin_schema`). But every bundled plugin still carries its
own Alembic chain per ADR-008: `flask plugin upgrade-all` stamps each plugin's
own chain (the `alembic_version_<plugin>` tables) and applies any plugin-specific
migrations added after the ownership cutover. The earlier Phase 7B state that
folded everything into core with no per-plugin chains was superseded by ADR-008's
per-plugin ownership. A fresh `flask db upgrade` reproduces the live core schema
exactly (verified on a scratch DB).
External (out-of-tree) plugins per ADR-003 may still ship their own migrations;
the framework supports per-plugin chains for them. Only the in-tree bundled
plugins are consolidated into core.
External (out-of-tree) plugins per ADR-003 ship their own migrations too; the
framework runs the same per-plugin chain mechanism (ADR-008) for both bundled and
external plugins.
## Open questions

View File

@@ -145,5 +145,5 @@ Reclassification is one-shot, run once, archived. Like the original migration sc
- ADR-001 (Asset is platform contract)
- ADR-002 (versioning of the surface)
- `plugins/equipment/` (current placeholder)
- `plugins/machines/` (equipment plugin, renamed per ADR-011)
- `plugins/computers/` (existing example of plugin pattern)

View File

@@ -145,4 +145,4 @@ Migration path:
- `shopdb/plugins/base.py` (`get_collector_schema` + `apply_collector_payload` hooks)
- ADR-001 (asset model the collectors target)
- ADR-002 (collector schema is part of plugin contract; changes to the hook signature are major bumps)
- The PXE project (`/home/camp/projects/pxe/`) which feeds the computers collector
- The PXE project (the PXE imaging project) which feeds the computers collector

View File

@@ -88,8 +88,8 @@ To cut release `X.Y.Z`:
### Neutral
- CI (`.gitea/workflows/ci.yml`) runs the backend tests, the naming/style
gate, and the frontend build on push and PR. It is best-effort: Gitea
- The internal CI workflow runs the backend tests, the naming/style
gate, and the frontend build on push and PR. It is best-effort: the internal CI
Actions availability on the host is unverified, so the workflow is
config-only until a runner is confirmed.
@@ -112,4 +112,4 @@ To cut release `X.Y.Z`:
- `shopdb/__init__.py` (`__version__`, `__contract_version__`)
- `CHANGELOG.md` (release record)
- `frontend/package.json` (frontend version, kept in lock-step)
- `.gitea/workflows/ci.yml` (CI gate)
- the internal CI workflow (CI gate)

View File

@@ -41,7 +41,9 @@ list. This is the whole of what ships now.
anonymous callers is safe because `GET /api/dashboard/navigation`
already leaks the same enabled/disabled signal, and unauthenticated
kiosk routes (`/tv`) need the answer too. It carries no metadata, so it
reveals strictly less than the admin-gated `GET /api/plugins`.
reveals strictly less than the equally-anonymous but metadata-carrying
`GET /api/plugins` (also `jwt_required(optional=True)`; only the
`PUT /api/plugins/<name>` toggle is admin-gated).
2. **Route tagging.** Every plugin-owned route carries `meta.plugin =
'<pluginname>'`. This covers the per-plugin route modules
@@ -106,7 +108,7 @@ core discovers it, mirroring the backend model. Sketch:
named extension points instead of editing core files: an `iconMap`
registration for nav/asset icons, asset-detail panels, map-marker
renderers, and search-result renderers (the "Frontend hook contract"
already listed as deferred in the project CLAUDE.md). Core exposes a
already listed as deferred in docs/ROADMAP.md). Core exposes a
stable set of shared components (form controls, detail-page shells,
table primitives) as the plugin frontend's only allowed core imports,
the frontend analogue of the `shopdb.api` namespace.

View File

@@ -1,7 +1,8 @@
# ADR-010: Frontend plugin hook contract
- **Status:** PROPOSED
- **Status:** ACCEPTED
- **Date:** 2026-07-11
- **Accepted:** 2026-07-11
- **Deciders:** cproudlock
- **Supersedes:** none
@@ -42,7 +43,7 @@ one would mean forking a core view:
printer detail page, a calibration-status card on a measuring tool. Today
`WarrantyPanel.vue` is composed in by hand-editing each detail view.
4. **Map marker / overlay contributions.** A calibration-due badge on the
shop-floor map (`frontend/src/views/map/MapView.vue`). The map draws type
shop-floor map (`frontend/src/views/MapView.vue`). The map draws type
colors but has no plugin decoration path.
5. **Search-result rendering / routing for plugin asset types.** Global search
returns assets, but core hardcodes how each type renders and where its detail
@@ -112,7 +113,7 @@ not pursued.
## Decision
**PROPOSED:** adopt a hybrid. Add **data-only declarative hooks** (Option B) for
**DECISION:** adopt a hybrid. Add **data-only declarative hooks** (Option B) for
the four presentation surfaces a generic core renderer can serve, and keep
**file-convention glob discovery** (Option C) as the deferred mechanism for the
residual cases where a real component is unavoidable. Do not pursue runtime

View File

@@ -0,0 +1,123 @@
# ADR-012: GE-Enforce manifest ownership in shopdb
- **Status:** ACCEPTED
- **Date:** 2026-07-13
- **Deciders:** cproudlock
- **Relates to:** ADR-002 (plugin contract versioning), ADR-004 (per-site
deployment), ADR-006 (collector contract), ADR-008 (per-plugin Alembic chains)
## Context
GE-Enforce is a desired-state enforcement system for shopfloor PCs: a PowerShell
engine (`Install-FromManifest.ps1`) reads per-PC-type `manifest.json` files off
an SMB share every logon and installs / self-heals what they declare. Authoring
those manifests today means hand-editing JSON on a file share, and there is no
central view of what each PC actually did.
We want shopdb to own the manifests as data (author, version, publish, roll
back) and to observe fleet compliance, while NOT taking on the GE-Enforce engine
itself (which is the GE-Enforce framework's, maintained separately) and NOT
dictating any site's imaging path (per ADR-004, each site is single-tenant with
its own provisioning - PXE at West Jefferson, OOBE provisioning packages at
others).
The manifests are an enforcement PROGRAM, not an application inventory: entry
`Type` is not an app/config discriminator and entry `Name` is a manifest label,
not a Windows ARP DisplayName. Any design that treats them as an app catalog is
wrong.
## Decision
Build a bundled `geenforce` plugin that owns the manifest as shopdb data, with a
client kit and a deployment bootstrap. Specifically:
1. **Data model.** One wide `manifestentries` table with an `entrytype`
discriminator and nullable per-type columns (not SQLAlchemy STI, not a JSON
blob - the fleet is ~64 entries, so sparse columns are free and stay
queryable). Scopes are `manifestscopes`, unique on `(scopename, phase)`;
runtime is per-pctype scopes, preinstall is one flat scope. Multi-value gates
(PCTypes / hostnames / machine numbers) and the nested InUseCheck are child
tables. `sortorder` is the execution-order contract. RegValue is stored as a
raw JSON literal so DWord-vs-string typing survives. Per-plugin Alembic chain
(ADR-008).
2. **Published snapshots.** Editing touches a DRAFT only. Publish freezes the
rendered JSON document into an immutable `manifestpublishedversions` row; the
client is ALWAYS served the current published snapshot, never the draft;
rollback flips `iscurrent` to an older version. Freezing the document (not
row-mirroring) makes immutability structural.
3. **Behavioral-parity gate, not byte-identity.** A DB-free harness
(`parity.py`) imports each real manifest and renders it back, then proves
BEHAVIORAL equivalence (same ordered entries with identical detection /
targeting, and the same entries fire across machine-profile fixtures) - never
byte equality, which re-serialization would never satisfy. This gates any
build that touches the model.
4. **Filter mirror; engine is the single source of truth.** `filters.py`
mirrors the engine's four gate functions and alias graph for the "what would
this PC get" simulator and parity. The engine lib stays authoritative;
shopdb mirrors it (never the reverse). PCTypesStrict is honored only for the
preinstall phase, matching the runners.
5. **Payload integrity is separate from detection.** For `http`/`inline`
payloads a dedicated `payloadsha256` is verified before running - independent
of `DetectionMethod` (DetectionValue is a hash only for `Hash` detection).
`smb` payloads keep the share ACL as their trust boundary. Large binaries
stay on SMB; small config/scripts may move to http/inline later.
6. **Observed-state reporting.** Each PC POSTs its enforcement result;
`manifestenforcementreports` (+ results) records the applied version
(received-latest) and per-entry self-heal / failure. Status derives from
explicit self-heal flags only, never the raw installed count (Always/no-
detection scripts install every cycle without being drift corrections).
7. **Service-token auth.** Client endpoints authorize via managed service
tokens scoped `geenforce.fetch` / `geenforce.report`, through a new
`service_token_authorized(scope)` on the `shopdb.api` contract surface
(contract 0.11.0). Admin CRUD uses `geenforce.manage` / `geenforce.publish`.
8. **Client + deployment, engine referenced not vendored.** shopdb ships the
fetch/report kit (`plugins/geenforce/client/`) and a site-neutral bootstrap
(`Install-GEEnforce.ps1`) that provisions a PC's identity
(`C:\Enrollment\pc-type.txt` etc. - what determines the PC type; there is no
auto-detection, the provisioner supplies it), the shopdb registry config, and
the scheduled task. The GE-Enforce ENGINE is referenced (`-EngineSource`),
not carried by shopdb. Deployment is provisioning-path independent (PXE step,
OOBE ppkg, Intune, manual); the runtime task is fail-safe.
9. **Milestone 1 = export to share; staged cutover.** Until a site cuts its
client over to shopdb-sourced manifests, the plugin publishes and EXPORTS the
manifest to the share (with a `_meta/history` backup, atomic write); the
unchanged engine picks it up. Cutover is staged: shadow mode (fetch from
shopdb AND read the share, log diffs, install from share) then read cutover.
10. **No application auto-seeding.** The core Applications catalog already
tracks these apps (from the classic-shopdb migration) with version
histories; auto-creating Applications from manifest labels produced
duplicates and misclassified config drops. Application linkage, if wanted, is
a curated manifest-entry -> existing-Application link, not label scraping.
## Consequences
- **Positive.** Manifests become validated, versioned, publishable data with
one-click rollback and a fleet-compliance view; desired-state and observed-
state live in one system. The parity gate + published snapshots + separate
payload hash make a fleet-wide-SYSTEM system safe to author. The plugin is
provisioning-agnostic, so any GE Aerospace site can adopt it regardless of
imaging path. Validated end to end: parity green against the real manifests,
and the client kit + installer proven on a Windows VM (PS 5.1) and Linux
pwsh 7.
- **Boundaries / risks.** The engine remains the GE-Enforce framework's, so
shopdb's parity mirror must be kept in sync with the lib (guarded by the parity
fixtures; the plugin pins lib >= 2.6 for `_CmmVersion`). Provisioning writes
the PC identity - shopdb cannot set a PC's type at imaging (a PC is unknown
until it enrolls and reports). Manifest-label vs ARP-name mismatch means the
catalog link, when built, needs a curated alias layer.
- **Deferred.** Desired-vs-observed per-entry compliance (needs a collector
installedVersions field); curated manifest-entry -> Application linking; the
live client cutover (a site operational decision); inline payload upload.
See `docs/proposals/ge-enforce-plugin.md` (design + cutover), `docs/GE-ENFORCE.md`
(concepts + imaging timeline), `docs/GE-ENFORCE-CLIENT.md` (fetch/report
contract), and `docs/GE-ENFORCE-DEPLOY.md` (agent deployment).

View File

@@ -0,0 +1,399 @@
# ADR-013: Plugin Catalog, Curated Shelf, and Lean Per-Site Builds
- Status: PROPOSED
- Date: 2026-07-18
- Deciders: cproudlock
- Relates to: ADR-002 (contract versioning), ADR-003 (plugin distribution), ADR-004 (per-site instances), ADR-008 (per-plugin migrations), ADR-009 (frontend plugin gating), ADR-010 (frontend hook contract)
## Context
Every site today ships identical code. The backend image contains all 13 plugin
directories (Dockerfile COPY at line 55; the header comment listing "eleven core
plugins" is stale), and the SPA compiles every plugin's routes and views via a
static glob (frontend/src/router/index.js:11) plus hardcoded imports. A site's
"chosen set" exists only as runtime enable flags in instance/plugins.json.
Disabled is not absent: a site that never wants printedparts/usb/network still
ships, and can execute, that code.
Distribution per ADR-003 is "drop a directory into plugins/ by hand". There is
no artifact format, no signing, no catalog, no validate gate, and the loader
trusts whatever it finds on disk (loader.py:61-73 discovers any folder with a
plugin.py; loader.py:185-224 loads it; migrations.py:29-62 runs its DDL with
full DB privileges). Plugins run in-process with the full shopdb.api surface
including db, so the only tenable security model on air-gapped GE networks is
curation plus cryptographic provenance, enforced everywhere code can execute,
not sandboxing.
Known defects this ADR also resolves:
- upgrade_all_plugins checks hasattr(registry, 'list_installed') which never
exists (registry has only get_all/get_enabled_plugins), so it always falls
back to migrating every folder on disk, adopted or not (__init__.py:94).
- PILOT-DEPLOY.md enables plugins that were never installed; enable refuses.
There is no declarative "apply this chosen set" operation.
- Reverse-dependency checks on uninstall/disable read only LOADED plugin
instances, so an installed-but-unloaded dependent is invisible.
- Dependency install/enable is check-only; nothing computes a closure, and
install-a-dependent can fail at its own load step because the dependency was
installed disabled (default_enabled=false on employees).
- The dependency sort has no cycle detection.
- Soft couplings (geenforce -> computers, notifications -> employees) are
invisible to the manifest graph.
Frontend reality check (this drove the design below): plugin UI is NOT one
folder per plugin. Routes live in routes/<plugin>.js (slides has none), inside
the shared core.js (computers report, printers toner report, employees detail,
slides settings), and as six hardcoded top-level imports in index.js itself
(/parts-kiosk, /tv, /print/printer-qr x2, /print/usb-labels,
/print/printedparts-labels). View dirs mismatch plugin names (computers ->
views/pcs). Plugin settings cards sit in shared views/settings/
(DellWarrantySettings, ZabbixSettings, SlideManager, EmployeeDirectory,
MeasuringToolTypesList, PrintedPartsSettings), plugin print views in shared
views/print/, and some views span plugins (AssetLabel.vue serves five asset
types; PCDetail imports WarrantyPanel). Any lean-frontend design that only
moves routes/*.js and views/<plugin>/ fails the build the moment a plugin is
pruned. This ADR scopes that work honestly instead of calling it mechanical.
## Decision
### 1. Tiering: mandatory core is the core package; all plugins are catalog-optional
- The mandatory core is the non-plugin shopdb/core/ package (auth, users,
assets, locations, vendors, models, settings, audit, dashboard, search,
reports, plugin management). It already survives every plugin being absent
via hasattr/lazy-import guards. No plugin is promoted into it.
- New optional manifest field `tier: "core" | "optional"`, default "optional".
All 13 existing manifests are unchanged and unchanged in meaning. The
lifecycle gains a guard: uninstall_plugin and disable_plugin refuse a
tier:core plugin (alongside the reverse-dependency checks at
__init__.py:269-278 and :351-360). No plugin ships tier:core initially; the
field and guard exist so a future curation decision is a manifest edit, not a
framework change.
- Per-site mandates live in the site profile (section 5): a `locked` list the
profile applier refuses to remove. This preserves ADR-004 site autonomy: a
wing site can mandate usb without the framework mandating it fleet-wide.
- New manifest field `optional_dependencies: []` (names only, loader-ignored).
Declared for the verified soft couplings: geenforce lists computers
(service.py:32-43 loses app-detection gates without it), notifications lists
employees (routes.py:151-171 loses name/photo enrichment). Catalog listing
and adopt WARN on unmet optional deps; nothing blocks.
- Hard `dependencies` gains optional PEP440 ranges ("employees>=1.1").
validate/adopt honor ranges; the runtime loader keeps name-only semantics
(specifier stripped) so no loader behavior changes. The single existing hard
edge printedparts -> employees stays as-is; whether it can relax to optional
(badges.py has an external HR fallback) is a follow-up product question, not
blocked on this ADR.
- Dependency plumbing fixes: _sort_by_dependencies gains cycle detection
(raise PluginDependencyError on a back edge); reverse-dependency checks read
manifests of ALL installed plugins from disk, not loaded instances.
### 2. Packaging: signed, versioned artifacts
Artifact: `<name>-<version>.shopdbplugin` (a zip of the plugin directory:
manifest.json, plugin.py, api/, models/, migrations/, and frontend/ once
section 6 lands) plus two members generated at pack time:
- `PROVENANCE.json`: plugin name, version, publisher id, build timestamp, and
a sorted map of every packaged file path to its SHA-256. PROVENANCE.json is
not listed in its own map, so there is no circular-hash problem and no zip
canonicalization needed; determinism comes from sorted per-file hashes.
- `PROVENANCE.sig`: detached ed25519 signature over the exact PROVENANCE.json
bytes.
New CLI:
- `flask plugin pack <name> --key <path>` (producer side): runs validate on the
directory, then emits the artifact.
- `flask plugin validate <dir|artifact>` (the missing pre-publish gate),
fail-closed pipeline: signature (artifact mode) -> per-file hashes -> manifest
against a new docs/plugin-manifest.schema.json -> name == directory ->
core_version parses as a specifier and admits the target contract version ->
static import-surface scan reusing tests/test_plugin_contract.py logic ->
alembic versions parse. The schema types the known fields (name, version,
description, dependencies, optional_dependencies, tier, core_version,
api_prefix, display_name, default_enabled, provides, settings) and PERMITS
additional properties, so all 13 existing manifests pass unmodified.
The import-surface scan is documented as a lint, not a security control; it is
trivially bypassed by dynamic import. The security control is human review
before signing (section 4).
### 3. The shelf: a read-only folder, transport-agnostic by design
- One config knob: `PLUGIN_SHELF_DIR`. The app only ever reads this folder. It
never speaks SharePoint, OneDrive, or any network protocol.
- Transport is explicitly out of scope and explicitly untrusted. On networks
that can reach corporate M365, a SharePoint document library sync populates
the folder. On strictly air-gapped floors where no sync agent can run, the
folder is populated by robocopy/USB. Both are equally supported and equally
untrusted, because every decision-bearing byte is signed: swapping transport
changes nothing about the trust model.
- Layout: `<shelf>/<name>/<name>-<version>.shopdbplugin` plus
`shelf-index.json` and `shelf-index.sig`.
- The index is SIGNED with the same publisher key and carries a monotonically
increasing `serial` plus a `revoked` list of name-version pairs. Each site
records the last-seen serial in instance state and refuses an index with a
lower serial (anti-rollback of the catalog itself). The index also carries
per-entry version/tier/core_version so `flask plugin shelf-list` can display
compatibility without unpacking, but the index is a BROWSE layer only:
adopt reads dependencies, tier, and core_version from the signed manifest
inside the verified artifact, never from the index.
- Trusted keys: `PLUGIN_TRUSTED_KEYS` is a list of pinned public keys delivered
out-of-band in the site's deployed config/image. Keys are NEVER read from the
shelf; a folder that can be written by an attacker must not also carry the
keys that authenticate it. Multiple pinned keys allow overlap rotation.
Revocation of an artifact rides the signed index `revoked` list; a
`flask plugin audit` command warns when an installed version appears there.
- Partial-sync robustness: adopt copies the artifact to a temp location,
verifies signature and every file hash there, then unpacks to
plugins/.staging/<name> and renames into place atomically. OneDrive
placeholder stubs, zero-byte files, or an index referencing not-yet-synced
artifacts all fail closed with a clear "artifact not fully synced/verified"
error.
- `flask plugin adopt <name>[==version]`: resolve version from the shelf,
verify, compute the hard-dependency closure from signed manifests, then for
each closure member in topological order: unpack, INSTALL, and ENABLE (not
install-only; the load gate at loader.py:201-206 checks is_enabled, so an
install-only closure with default_enabled=false deps would fail its own
load). Migrations run via the unchanged per-plugin chain (ADR-008). Refuses
to adopt a version lower than the installed one unless
`--force-downgrade` is given interactively. Prints the restart notice.
- Adopt/install/uninstall remain CLI-only. The admin HTTP surface stays a
read-only catalog view plus the existing enable/disable toggle; because
Flask cannot register blueprints after the first request, any adopt or
enable takes full effect only on restart, and the UI says so. There is no
"install button" that pretends otherwise.
### 4. Trust model: verify at adopt AND at every load and migrate
Signing that gates only adoption is bypassable through every other write path
into plugins/ (git clone, symlink, USB drop) and defeated by post-adoption
tampering. Therefore verification is enforced where code executes:
- Adoption leaves PROVENANCE.json and PROVENANCE.sig inside plugins/<name>/
and records publisher + artifact hash in the registry entry.
- load_plugin verifies the signature against PLUGIN_TRUSTED_KEYS and re-hashes
the plugin tree against the provenance file map BEFORE importing plugin.py
(new step ahead of loader.py:185). Missing or invalid provenance is a
fail-closed refusal in production.
- run_plugin_migrations performs the same verification before executing any
revision, so a routine `flask plugin upgrade-all` can never run DDL from an
unverified folder.
- upgrade_all_plugins iterates registry.get_all() (fixing the phantom
list_installed fallback at __init__.py:94), so unadopted on-disk folders are
never migrated as a side effect of deploys.
- Development and the ADR-003 external-repo/symlink workflow (including
scripts/test-external-plugin.sh) are preserved via `PLUGIN_DEV_TRUST_DIRS`,
honored ONLY when DEBUG or TESTING is set. Production ignores it.
- Cost: hashing 13 small plugin trees at boot is milliseconds; accepted.
What signing does NOT claim: a valid signature proves the artifact is exactly
what a curator reviewed and signed, nothing more. Plugins remain in-process
Python with full DB access. The actual safety control is the human review
before signing; the signature makes that review's verdict tamper-evident all
the way to execution.
### 5. Declarative site profiles and lean backend builds
- `site-profile.json` per site (kept in the site's deploy config): site name,
list of chosen plugins, optional `locked` list. `flask plugin apply-profile
<file>` resolves the closure, installs AND enables in dependency order, runs
migrations, reports which changes need a restart. This replaces the
imperative CLI sequences in DEPLOY.md/PILOT-DEPLOY.md and fixes the
enable-without-install bug.
- Lean backend image: `scripts/build-site.sh` reads the profile and stages
only core + chosen plugin directories into the Docker build context
(correcting the Dockerfile COPY and its stale header comment). Discovery
needs no change; it already scans whatever exists.
- Prerequisite the naive version misses: core hardcodes plugin imports.
shopdb/core/api/search.py (~15 sites), reports.py, assets.py, collector.py,
applications.py, auditlogs.py, and shopdb/cli/__init__.py import
plugins.<name>.* lazily. Some already guard ImportError; ALL must, with
graceful degradation, before any site prunes a folder. This is audited and
enforced by a new CI job that deletes one plugin directory and runs the full
test suite (repeated per plugin). Longer term these aggregators should move
to registry-driven contract hooks (get_search_providers/get_report_sources)
so a new catalog plugin can join search/reports without core edits; that is
scoped as follow-up work, not a blocker for lean builds.
- Schema-lean is DEFERRED to its own ADR. The core baseline 68b3947ae14f
unconditionally creates the 10 pre-cutover plugins' tables, and lifting them
into plugin baselines collides with cross-plugin foreign keys (the
computers-owned installedapps table FKs machines.machineid while computers
declares no dependency on machines). Reversing the cutover would either
introduce undeclared hard deps or drop FKs; neither is decided here. A lean
site therefore carries a handful of empty pre-cutover tables. Accepted.
### 6. Frontend delivery: Path C for rich UIs, Path A for simple ones, Path B rejected
Path B (runtime-loaded JS / module federation) is REJECTED: it moves executable
UI delivery from a signed, statically auditable build artifact to runtime
fetching, which is exactly the wrong direction for an air-gapped,
review-then-sign posture, for zero benefit given restarts are already required.
Path A (declarative JSON UI over generic renderers) is COMMITTED and scheduled
EARLY: the three unwired ADR-010 endpoints (pluginui.py asset-panels:62,
map-overlays:88, asset-presentation:100) get generic core renderers, joining
the already-consumed settings-cards. After this, a simple plugin ships JSON-only
UI with zero frontend build involvement. Sequencing this before the relocation
gives every plugin an escape hatch during the migration instead of after it.
Path C (self-contained plugin frontend) is the primary mechanism, scoped
against the real code, not the idealized layout:
- Canonical home: plugins/<name>/frontend/ containing routes.js (the plugin's
complete route array, INCLUDING routes currently embedded in index.js and
core.js), views/, and settings views.
- A pre-Vite staging step (scripts/stage-frontend.mjs, run by build-site.sh
and the dev script) copies the CHOSEN plugins' frontend/ into
frontend/src/.plugins-staged/<name>/ (gitignored) and generates two files
inside the Vite root: routes.gen.js (aggregated plugin routes) and
meta.gen.js (plugin-supplied icon names, title spellings, settings-standalone
flags, replacing the hardcoded iconMap/TITLE_SPELLINGS/SETTINGS_STANDALONE in
AppLayout.vue, settingsCatalog.js, and index.js). This exists because
import.meta.glob requires a static literal inside the project root and
cannot select a per-site subset by itself.
- ONE-TIME core-router surgery, done first and called what it is: the six
hardcoded plugin-view imports in index.js (PartsKiosk, TVDashboard,
PrinterQRBatch/Single, USBLabelBatch, PrintedPartsLabels) and the plugin
routes embedded in core.js move into their owning plugins' routes.js. Without
this, pruning slides/printers/usb/printedparts fails the Vite build on
unresolvable imports; no amount of glob work fixes it.
- Per-plugin relocation PRs (13), each REAL WORK, not a file move: carve routes
out of shared files, move views (handling name mismatches like computers ->
views/pcs), move the plugin's settings views out of shared views/settings/,
and rewrite relative ../../ imports of core shared code to the @/ alias
(relative paths break at the staged depth). A lint rule enforces alias-only
core imports in plugin frontend code from then on.
- Shared plugin-aware code STAYS CORE and ships to every site: AssetLabel.vue
(spans five asset types), views/print helpers (assetLabel.js, qrLogo.js),
MachineBadge.vue, and cross-plugin panels like WarrantyPanel used by
PCDetail. These already null-guard or gate via isPluginEnabled and must keep
degrading when a peer plugin is absent; over time they migrate to ADR-010
asset-panels so the data becomes plugin-supplied. Lean v1 therefore prunes
plugin-EXCLUSIVE code; a small plugin-aware core remainder is accepted and
shrinks as Path A absorbs it.
- Dual-location transition: the staging step unions legacy locations
(routes/*.js glob, views/<plugin>/) with plugins/<name>/frontend/ until each
plugin has moved. The SPA builds green at every commit; each plugin's move is
independently revertable until the legacy glob is removed at the end.
- Nav and settings cards are already server-driven (dashboardApi.navigation,
settings-cards); the remaining hardcoded plugin entries in settingsNav.js
(/settings/zabbix, /settings/dellwarranty) move to those plugins'
get_settings_cards so pruning leaves no dead links.
### 7. Effect on the 13 existing plugins
- Backend: ZERO code changes required. tier/optional_dependencies/provenance
are additive; pack zips the directory as-is; all plugins keep passing
tests/test_plugin_contract.py. Bundled plugins in a site's image get
provenance stamped at build time by pack, so verify-at-load applies to them
identically.
- Frontend: one relocation PR each, of the honest scope above. Until a
plugin's PR lands it keeps working from its legacy location.
- Operationally nothing changes for a site that does nothing: default builds
remain all-plugins, apply-profile is opt-in, and enable/disable semantics
(including the restart requirement) are unchanged.
## Consequences
### Positive
- A real catalog: sites declare their set in site-profile.json and apply it in
one idempotent command; the chosen set drives backend image, SPA bundle, and
runtime state from one source of truth.
- Curated marketplace with end-to-end provenance: review -> sign -> any
transport -> verify at adopt, at load, and at migrate. Transport (SharePoint
sync or sneakernet) is untrusted and interchangeable, which is exactly right
for air-gapped sites.
- Lean per-site builds: unchosen plugins exist in neither the image nor the
bundle, shrinking attack surface and download size.
- Fixes shipped along the way: upgrade-all migrating unadopted folders,
PILOT-DEPLOY install/enable ordering, reverse-dep checks blind to unloaded
plugins, missing cycle detection, missing dependency closure, hardcoded
frontend plugin metadata.
- Path A completion makes simple plugins UI-capable with no build glue, which
is the cheapest possible marketplace onboarding.
### Negative
- Key management is a per-site operational burden: pinned keys delivered
out-of-band, rotation is a config change everywhere. Accepted as the price of
not trusting the distribution folder.
- The frontend re-org is the long pole: one core-router surgery plus 13
non-trivial PRs. It is sequenced to be always-green and per-plugin
revertable, but it is weeks of work, not a rename.
- Schema is not lean: pre-cutover plugin tables still appear at every site
until the deferred baseline re-org ADR.
- Restarts remain required after adopt/enable (Flask blueprint constraint);
the marketplace UX is honest about it rather than working around it.
- Boot adds a signature + tree-hash check per enabled plugin (milliseconds,
but nonzero).
### Risks
- Key compromise or curation failure: a signature proves provenance, not
safety; a compromised pinned key or a rubber-stamp review signs malware that
every gate will happily pass. Mitigations: multi-key pinning with overlap
rotation, signed revocation list with monotonic index serial, and keeping the
signing key offline with the curator. The static import scan is a lint and
must never be presented as a boundary.
- Rollback/downgrade: mitigated three ways: index serial monotonicity, adopt
refusing version downgrades without interactive --force-downgrade, and the
signed revoked list. Residual risk: a site that never syncs a newer index
cannot learn of revocations; `flask plugin audit` at deploy time narrows the
window.
- Version skew across ADR-004 sites: one shelf serves sites at different
contract versions. Adopt checks core_version from the signed manifest against
the site's own __contract_version__ (authoritative); shelf-list shows an
advisory compatibility column from the index. Incompatible artifacts are
listable but not adoptable.
- Partial/placeholder sync files: fail closed on hash verification; the error
message distinguishes "not fully synced" from "tampered" only by wording,
intentionally, since the app cannot tell.
- Dev-trust misuse: PLUGIN_DEV_TRUST_DIRS silently ignored outside
DEBUG/TESTING; a prod config carrying it gets a startup warning.
- Frontend closure drift: plugin views importing cross-plugin components is a
graph the manifest does not model. The lint rule (plugin frontend may import
core @/ paths and its own tree only, never another plugin's) prevents new
edges; existing shared plugin-aware code is explicitly core-owned.
## Implementation phases
- Phase 0, groundwork (small, days): upgrade_all_plugins uses
registry.get_all(); reverse-dep checks read installed manifests from disk;
cycle detection in _sort_by_dependencies; shopdb/plugins/manifest_schema.json +
`flask plugin validate` (directory mode); `flask plugin apply-profile` with
install+enable closure ordering; fix Dockerfile stale comment. All additive,
zero risk to running sites.
- Phase 1, packaging and signing (medium, about a week): PROVENANCE format,
`flask plugin pack`, validate artifact mode, PLUGIN_TRUSTED_KEYS config,
ed25519 signing tooling and curator docs. No runtime behavior change yet.
- Phase 2, shelf and enforcement (medium-large, one to two weeks):
PLUGIN_SHELF_DIR, signed shelf-index with serial + revoked list,
`flask plugin shelf-list` / `adopt` / `audit` with atomic verified unpack;
verify-at-load in load_plugin and verify-at-migrate in
run_plugin_migrations, fail-closed in prod; PLUGIN_DEV_TRUST_DIRS for
dev/test and the external-repo harness; tier:core lifecycle guard;
provenance stamping of bundled plugins at build. This phase completes the
security model; everything after it is delivery optimization.
- Phase 3, Path A completion (medium, one to two weeks): generic renderers for
asset-panels, map-overlays, asset-presentation; migrate settingsNav.js
hardcoded plugin cards to get_settings_cards. Done BEFORE relocation so
JSON-only UI is available during the migration.
- Phase 4, frontend re-org (large, the long pole, several weeks elapsed):
stage-frontend.mjs staging + routes.gen.js/meta.gen.js codegen; ONE core PR
moving the six index.js hardcoded plugin imports and the core.js-embedded
plugin routes into plugin route files; then 13 per-plugin relocation PRs
(views, settings views, name-mismatch dirs, @/ alias rewrite) under the
dual-location union; lint rule for plugin frontend imports. Always-green,
per-plugin revertable.
- Phase 5, lean builds end to end (medium, about a week after Phase 4):
build-site.sh staging backend dirs + frontend staging from site-profile.json;
core lazy-import guard audit finished, enforced by the delete-a-plugin CI
matrix; remove the legacy glob; pilot one real lean site (a location without
printedparts/usb/network) and diff its image and bundle against a full build.
Deferred, each to its own future decision: schema-lean core-baseline re-org
(blocked on the installedapps -> machines FK question), pip/entry-point
distribution (ADR-003 v2), hook-based search/report aggregation contract, and
any revisit of Path B.

View File

@@ -0,0 +1,142 @@
# ADR-014: Schema-lean per-site builds (retire cross-plugin FKs, lift plugin tables)
- Status: ACCEPTED
- Date: 2026-07-19
- Deciders: cproudlock
- Relates to: ADR-008 (per-plugin migration ownership), ADR-013 (plugin catalog + lean per-site builds), ADR-001 (asset model)
## Context
ADR-013 delivered lean per-site builds for plugin CODE (backend tree + frontend
bundle carry only chosen plugins). One residual was left, explicitly deferred:
the DATABASE. The core Alembic baseline (68b3947ae14f) creates EVERY table,
including ~30 plugin-owned tables (PLUGIN_TABLE_OWNERS). Each plugin's own
baseline is a stamp-only no-op (the core chain already made its tables, per
ADR-008). So a lean site that omits a plugin still creates that plugin's tables,
empty and unused.
The deferral cited a blocker: "the computers-owned installedapps table FKs
machines.machineid while computers declares no dependency on machines; reversing
the cutover would introduce undeclared hard deps or drop FKs; neither is
decided." Investigation refined this:
The cross-boundary foreign keys into the machines plugin table are ALL held by
DEAD legacy tables/columns that predate the asset model (ADR-001) and the
per-plugin cutover (ADR-008), and are queried nowhere in the codebase:
- `machinerelationships` (child/parentmachineid -> machines) - superseded by
`assetrelationships`. No model, no query.
- `printerdata` (machineid -> machines) - the pre-cutover printers table,
superseded by the printers plugin. No model, no query.
- `installedapps` (machineid -> machines) - a standalone machine-app link table;
the live relationship is `computerinstalledapps` (FK to computers only). The
standalone table has no model, no query.
- `communications.machineid` (-> machines) - a legacy column on the core
communications table (which is now assetid-based). Not read anywhere.
No LIVE plugin table hard-FKs another plugin's table. computerinstalledapps FKs
only computers.computerid (intra-plugin). So the blocker is dead cruft, not
live design.
## Decision
Two phases, both leaving existing databases correct.
### Phase 1: retire the dead cross-boundary cruft - ALREADY DONE
Investigation found this is already accomplished by existing migrations:
`7a01_adr001_position_contract` and `7c01_drop_legacy_machine` drop
`machinerelationships`, `printerdata`, `installedapps`, and
`communications.machineid` (with its FK). The current schema (verified on the
dev database) has none of them. So the cross-plugin FK blocker ADR-013 cited no
longer exists in the live schema - only in the baseline's transient
create-then-later-drop. No new migration is needed for Phase 1.
Precedent: ADR-001 dropped a cross-plugin FK the same way
(usbcheckouts.machineid -> machines became a soft sentinel).
### Enabling change (executed now): idempotent create_plugin_tables
`shopdb/plugins/alembic_template.py:create_plugin_tables` now skips any table
that already exists (inspects the bind first) instead of raising. This is the
mechanism Phase 2 needs: a plugin anchor can create its tables on a fresh lean
install AND be a safe no-op on an existing database that already has them from
the pre-cutover core baseline. Correct and inert regardless of Phase 2 (no
current caller creates against a populated schema). Verified against the
plugin-migration suite.
### Phase 2 (executed): prune not-installed plugin tables after upgrade
Two mechanisms were weighed to make a lean site's database carry only
core + chosen-plugin tables:
- **Relocate** (rejected): pull every plugin-table create/alter out of the core
chain into the plugin baselines, so the core chain never creates a
not-installed plugin's table. Measurement killed this: plugin tables are
created and altered across ~15 released core migrations (baseline plus 7c04,
7d05, 7d08, 7d13, 7d15, 7d16, 7d17, ...), not just the baseline. Because the
whole core chain runs before any plugin chain, removing a table's create from
core while a later core migration still alters it breaks FULL installs too, so
relocation means surgically rewriting ~15 released migrations - the highest
blast radius in the project - for a purely cosmetic gain (the omitted tables
are empty and the lean CODE build already never loads the plugin).
- **Prune-after-upgrade** (chosen): leave the entire core chain untouched. Add
`flask plugin prune-schema`, which drops the tables of every plugin in
PLUGIN_TABLE_OWNERS that is not installed on this site. Run once at deploy,
after `flask db upgrade` and `flask plugin upgrade-all`. Same end state
(core + chosen tables) with near-zero blast radius: no released migration is
edited, and an existing full site is unaffected because it never runs the
command.
`prune-schema` drops by table name (no plugin-code import), so it works on a
lean image where the omitted plugin's directory is absent. It is a dry-run by
default and refuses to drop a table that holds rows unless `--force`, so a
misfire on a populated site cannot silently delete data. Because the core chain
seeds a few plugin reference tables (e.g. 7d05 inserts default access
protocols), initial lean provisioning uses `--force` - at that point the tables
hold only migration-seeded defaults, before any site data exists.
The idempotent `create_plugin_tables` (enabling change above) is what lets a
lean site later ADD an omitted plugin: its anchor recreates the pruned tables.
Verified end to end on MySQL: fresh full install (86 tables) then prune is a
no-op; fresh lean install (machines + printers) then prune drops the other 19
plugin tables, leaving core + chosen; second prune is a no-op; the non-empty
guard refuses without `--force`. Four SQLite regression tests pin the behavior
(tests/test_plugin_prune_schema.py), running in the backend CI job via the real
CLI runner: drop-only-not-installed, full-site no-op, refuse-non-empty, and
force-drops-non-empty.
## Consequences
### Positive
- A lean site's database contains only core + chosen-plugin tables, with no edit
to any released migration (near-zero blast radius).
- The cross-plugin FK blocker ADR-013 cited is gone (dead cruft, dropped by
existing migrations), so plugin schemas are already FK-independent.
- Adding an omitted plugin to a lean site later just works: the idempotent
anchor recreates its tables.
### Negative / risk
- prune-schema is destructive by nature; the row-count guard + dry-run default +
required `--force` for non-empty tables contain that. It is a deploy-time
provisioning step, not something to run casually on a live populated site.
- A lean fresh install still transiently creates then drops the omitted plugins'
tables (the core chain builds them, prune removes them). Harmless and one-time
at provisioning; the trade for not touching the released baseline.
## Implementation
- Phase 1: nothing to do - the dead cross-boundary FK objects were already
dropped by existing migrations `7a01_adr001_position_contract` and
`7c01_drop_legacy_machine`; verified absent on a fresh full MySQL upgrade.
- Enabling change: `create_plugin_tables` made idempotent
(`shopdb/plugins/alembic_template.py`).
- Phase 2: `flask plugin prune-schema` (`shopdb/plugins/cli.py`), dry-run by
default, `--yes` to execute, `--force` for non-empty tables. Deploy order:
`flask db upgrade` -> `flask plugin upgrade-all` -> `flask plugin prune-schema
--yes --force`. Regression tests in `tests/test_plugin_prune_schema.py` (run in
the backend CI job).

View File

@@ -22,8 +22,11 @@ Each ADR captures a single architectural decision: the context, the decision its
| [007](ADR-007-product-versioning-and-releases.md) | Product versioning and releases | ACCEPTED |
| [008](ADR-008-plugin-migration-ownership.md) | Plugin migration ownership (per-plugin chains) | ACCEPTED |
| [009](ADR-009-frontend-plugin-gating.md) | Frontend plugin route gating | ACCEPTED |
| [010](ADR-010-frontend-plugin-hooks.md) | Frontend plugin hook contract | PROPOSED |
| [010](ADR-010-frontend-plugin-hooks.md) | Frontend plugin hook contract | ACCEPTED |
| [011](ADR-011-machines-rename.md) | Machines rename + modeltypes retyping | ACCEPTED |
| [012](ADR-012-geenforce-manifest-ownership.md) | GE-Enforce manifest ownership in shopdb | ACCEPTED |
| [013](ADR-013-plugin-catalog-and-lean-builds.md) | Plugin catalog, curated shelf, and lean per-site builds | PROPOSED |
| [014](ADR-014-schema-lean-per-site.md) | Schema-lean per-site builds (retire cross-plugin FKs, prune not-installed plugin tables) | ACCEPTED |
## Authoring

2978
docs/api-inventory.json Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,751 @@
# GE-Enforce over HTTPS API: cutover reference
This is the operator/developer reference for moving a shopfloor PC type off the
SMB-share GE-Enforce delivery and onto the shopdb HTTPS API. The first cohort
cut over was the displays/kiosks (`gea-shopfloor-display`): share-less,
Intune/Entra-joined PCs with no SFLD share credentials. This doc captures
everything learned doing that, so extending the cutover to the other pc-types
(`gea-shopfloor-cmm`, `-collections`, `-keyence`, `-genspect`, `-heattreat`,
`-partmarker`, `-nocollections`, `common`) does not require re-learning it.
It pairs with the existing docs (which describe the pieces; this one describes
the CUTOVER):
- `docs/GE-ENFORCE.md` - concepts and the plugin
- `docs/GE-ENFORCE-CLIENT.md` - client fetch/report contract
- `docs/GE-ENFORCE-DEPLOY.md` - what must land on a PC
- `docs/GE-ENFORCE-DISPLAY.md` - the display scope specifics
- the PXE repo (`docs/ge-enforce-v2-architecture.md`) - the SMB world
being cut away from
Contents:
1. [Overview and why](#1-overview-and-why)
2. [Server architecture](#2-server-architecture)
3. [Delivery models: smb vs http/inline payloads](#3-delivery-models-smb-vs-httpinline-payloads)
4. [Authoring a scope](#4-authoring-a-scope)
5. [The on-PC client and engine](#5-the-on-pc-client-and-engine)
6. [Bootstrap for share-less PCs](#6-bootstrap-for-share-less-pcs)
7. [Asset reporting via the collector](#7-asset-reporting-via-the-collector)
8. [HARD-WON GOTCHAS](#8-hard-won-gotchas)
9. [How this was verified](#9-how-this-was-verified)
10. [Deploy](#10-deploy)
11. [PLAYBOOK: extending to a new pc-type](#11-playbook-extending-to-a-new-pc-type)
12. [Open items / TODO](#12-open-items--todo)
---
## 1. Overview and why
GE-Enforce v2 delivers desired-state manifests and installer payloads from the
SFLD SMB share (`\\tsgwp00525.wjs.geaerospace.net\shared\dt\shopfloor\`).
Every PC mounts the share with Azure-DSC-provisioned SFLD credentials, reads
`<scope>\manifest.json`, and runs the engine
(`Install-FromManifest.ps1`). That works for the domain fleet but is a hard
dead end for share-less PCs.
The HTTPS API path replaces the share as the transport, keeping the engine and
its detection/self-heal behavior untouched:
| Concern | SMB share | HTTPS API |
|---------|-----------|-----------|
| Manifest source | `<scope>\manifest.json` on the share | `GET /api/geenforce/manifest?pctype=<scope>` (published snapshot) |
| Payload source | share paths (`apps\...`) | `GET /api/geenforce/payload/<sha256>` (content-addressed) |
| Auth | SFLD share credential (DSC) | `geenforce.fetch` service token OR source-IP allowlist |
| Result visibility | log files on the PC | `POST /api/geenforce/report` -> Enforcement Reports UI |
| Versioning | file overwrite + `_meta/history` | immutable published versions, rollback, ETag |
Which PCs MUST use the API: the share-less ones. Displays/kiosks are
Intune/Entra-joined with no SFLD credentials and no domain trust, so SMB is not
an option at all. The rest of the fleet CAN stay on the share (and currently
does); for them the API is an opt-in migration, not a forced one.
West Jefferson facts used throughout this doc:
| Fact | Value |
|------|-------|
| Prod host | `tsgwp00525.wjs.geaerospace.net` |
| App mount | `/shopdb` (IIS, app dir `C:\inetpub\wwwroot\shopdb`, pool `shopdbflask-prod`) |
| BaseUrl clients use | `https://tsgwp00525.wjs.geaerospace.net/shopdb` |
| Prod DB | `shopdb_flask` (MySQL) |
| Client allowlist CIDRs | `10.134.48.0/23,10.48.249.0/26` (the WJ corp/AESFMA shopfloor subnets) |
| Dev/staging instance | `/ops` mount, DB `shopdb_flask_dev`, pool `shopdbflask` |
---
## 2. Server architecture
All server code is in `plugins/geenforce/` (routes: `plugins/geenforce/api/routes.py`).
### Client-facing endpoints (three)
| Endpoint | Method | Auth | What it does |
|----------|--------|------|--------------|
| `/api/geenforce/manifest?pctype=<scope>&phase=runtime` | GET | `geenforce.fetch` token OR IP allowlist | Serves the CURRENT PUBLISHED manifest snapshot for a scope (never the draft). `ETag: "<scopeid>-v<version>"`, `X-Manifest-Version` header, 304 on `If-None-Match`. |
| `/api/geenforce/payload/<sha256>` | GET | `geenforce.fetch` token OR IP allowlist | Streams a payload blob by content hash: blob store first (`service.blob_path`), then inline `ManifestPayload`. ETag = the hash. Per-IP rate limited (120/min default) and size-capped (512 MB default, 413 above). |
| `/api/geenforce/report` | POST | `geenforce.report` token OR IP allowlist | Records one enforcement cycle via `service.record_enforcement_report`: hostname, scopename, appliedversion, enforcerversion, counts, per-entry results. Upserts the current report per (hostname, scopename, phase); older reports kept as history. |
Plus the collector for asset reporting (section 7): `POST
/api/collector/computers` in `shopdb/core/api/collector.py` - a DIFFERENT auth
domain (`collector.ingest`), NOT covered by the geenforce allowlist.
### Auth model
`_require_service_token(scope)` in `routes.py` is the decorator factory. Two
paths, fail-closed (neither -> 401):
1. A managed service token with the scope (`geenforce.fetch` for
manifest/payload, `geenforce.report` for report), sent as `X-API-Key` or a
Bearer PAT (`authorized_service_token`). A token may carry
`resourcescopelist` bindings: a bound token can only fetch its own scope's
manifest (403 otherwise) and only blobs those scopes' published manifests
reference (`service.blob_referenced_by_scopes`, 404 so hashes cannot be
probed). Displays get a token bound to `gea-shopfloor-display` (see
GE-ENFORCE-DISPLAY.md).
2. The IP allowlist: `_ip_allowlisted()` checks the caller against the setting
`geenforce_allowed_cidrs` (comma-separated CIDRs/IPs, empty = disabled).
Network trust replaces the shared secret for a vaulted fleet. The
allowlisted path has no resource-scope binding (unrestricted).
The token paths use RBAC permissions the plugin registers in
`GeEnforcePlugin.get_permissions()`: `geenforce.manage` (edit),
`geenforce.publish` (ship), `geenforce.fetch` and `geenforce.report`
(client service tokens). Admin CRUD/publish/simulate/compliance routes are JWT
plus `geenforce.manage`/`geenforce.publish`.
### Why the allowlist uses remote_addr (the IIS XFF dependency)
`_trusted_client_ip()` returns `request.remote_addr`, NOT the raw
`X-Forwarded-For` header. Proxies APPEND to X-Forwarded-For, so its first hop
is attacker-controlled: parsing it (as `_client_ip()` does, acceptably, for
rate limiting only) would let any caller send `X-Forwarded-For:
<allowlisted-ip>` and bypass the token entirely.
`remote_addr` is trustworthy only because of a two-piece chain that MUST stay
in place:
1. The IIS URL-Rewrite rule in the `/shopdb` web.config OVERWRITES (not
appends) the inbound `X-Forwarded-For` with `REMOTE_ADDR`, the real TCP
peer.
2. waitress runs with `--trusted-proxy=127.0.0.1
--trusted-proxy-headers=x-forwarded-for`, so it derives `remote_addr` from
that overwritten header only when the request comes from IIS on localhost.
A client that somehow hits waitress directly is not a trusted proxy, so its
`remote_addr` is its own real peer address. Either way, real client IP.
**If the IIS rule is ever removed, the allowlist becomes SPOOFABLE.** This
document previously claimed the opposite; it was wrong, and the reasoning matters.
IIS does not set `X-Forwarded-For` on its own - the rewrite rule is the only
thing that does. Remove the rule and IIS still *forwards* whatever
`X-Forwarded-For` the caller sent. waitress trusts that header because it arrives
from `127.0.0.1`, which is IIS, and sets `remote_addr` from it. So a caller who
sends `X-Forwarded-For: 10.134.48.5` gets `remote_addr = 10.134.48.5`, matches
the allowlist and fetches manifests token-less from anywhere on the network.
The rule is not a nicety that improves logging. It is the control that makes
`remote_addr` trustworthy, and everything downstream - the allowlist, the
dashboard visitor-location lookup, per-host login rate limiting - depends on it.
Three consequences worth stating plainly:
- The Windows installer enables the rule when told IIS faces clients directly
(`-ClientIpSource direct`), installs URL Rewrite from the bundle to make that
possible offline, and its stage-5 check fails if the rule is not live.
- On a hand-built server, verify it: `deploy/windows/web.config` must have the
`<rewrite>` block ACTIVE, not inside the `SHOPDB-CLIENTIP` comment markers.
- Behind a real reverse proxy the rule is the wrong answer, because `REMOTE_ADDR`
is then the proxy. There, the proxy must set `X-Forwarded-For` itself and be
the only thing that can reach IIS. `-ClientIpSource proxy` covers that case.
---
## 3. Delivery models: smb vs http/inline payloads
Every `ManifestEntry` carries a `payloadsource` (`plugins/geenforce/serializer.py`
and `importer.py`):
| PayloadSource | Meaning | Manifest emission |
|---------------|---------|-------------------|
| `smb` (default) | Entry installs from the share exactly as v2 does; the entry's `Installer`/`Script`/`Source` is a share-relative path. | Nothing emitted - share manifests round-trip byte-identical, parity preserved. |
| `http` | Payload lives in the server's content-addressed blob store (`instance/geenforce/payloads/<sha256>`, registry row `ManifestBlob`). For big files (MSIs, EXEs). Upload via `flask geenforce add-payload <file>` or `service.store_blob`. | `PayloadSource`, `PayloadSha256`, `PayloadRef` keys on the entry. |
| `inline` | Payload bytes live IN the DB (`ManifestPayload`, <= 1 MB) - small scripts and configs. Attach via `service.store_inline_payload(entry, filename, contenttype, rawbytes)` or `POST /api/geenforce/entries/<id>/payload`. | Same three keys. |
Both `http` and `inline` are served from the same client URL:
`GET /api/geenforce/payload/<sha256>` (blob store checked first, then inline).
The sha256 IS the integrity contract - the client re-hashes after download.
### How the client stages payloads (Resolve-ShopdbPayloads)
`Resolve-ShopdbPayloads` in `plugins/geenforce/client/ShopdbEnforceClient.psm1`
is the bridge that lets the UNCHANGED engine install share-less:
1. For each entry with `PayloadSha256` and `PayloadSource` http/inline, call
`Get-ShopdbPayload`: download to
`C:\ProgramData\ShopDB\geenforce\payloads\<sha><ext>` (ext from
`PayloadRef`), verify the sha256, keep it as a content-addressed
last-known-good cache (a cache hit only counts if the bytes still hash
right).
2. Rewrite the entry's path field to the LEAF filename of the staged file
(`Split-Path -Leaf`) - NOT the absolute path. Field by Type:
`Installer` for MSI/EXE/CMD/BAT/INF, `Script` for PS1, `Source` for File.
3. Write a sibling `<scope>.resolved.json` manifest and return its path (or
the original path if nothing needed resolving). A payload that cannot be
fetched/verified THROWS - the runner's fail-safe catch decides what happens.
The runner (`Invoke-ShopdbEnforce.ps1`) then sets the engine's
`-InstallerRoot` to that same payloads directory, so the engine's
`Join-Path $InstallerRoot <leaf>` resolves to the staged file. `smb` entries in
a mixed manifest are left untouched and still resolve against the share (a PC
that has it).
---
## 4. Authoring a scope
Two authoring paths, both ending in `service.replace_scope_draft(scopename,
phase, manifest_dict)` (idempotent draft rebuild - published versions are never
touched by a re-import):
### A. import-share: adopt an existing SMB manifest
```
flask geenforce import-share --shareroot <path> [--scope <name>] [--preinstall <path>]
flask geenforce publish <scopename> [--phase runtime] [--notes "..."]
```
`importer.discover_share` walks the share root and ingests
`common/manifest.json`, `display/manifest.json`, and every
`gea-shopfloor-*/manifest.json` (skipping `.bak` variants). Entries come in as
`smb` payloads. Run `flask geenforce parity --shareroot <path>` first (Gate A):
proves import+re-export is behaviorally lossless before anything ships.
### B. authoring in code: seed_display_scope as the template
`plugins/geenforce/seed_display_scope.py` is the reference for a scope that
never existed on the share. Pattern:
- Build the manifest dict in Python (`build_display_manifest()`): four
`Type=Registry` drift-heal entries re-asserting the Edge kiosk relaunch
policies from imaging (`09-Setup-Display.ps1`), one inline PS1 dispatcher,
one inline PS1 always-on script. Registry heals use
`DetectionMethod=ValueMatches` against the same path/name they write, so
drift self-heals; the PS1s use `DetectionMethod=Always` and are idempotent.
- The dispatcher (`Invoke-DisplayKioskDispatch.ps1`, generated by
`build_dispatcher_script()`) reads `C:\Enrollment\display-type.txt`, maps
the subtype through the data-driven `DISPLAY_TYPE_TARGETS` table
(Dashboard -> `/shopfloor`, Lobby -> `/tv`, 3DPrintRoom -> `/parts-kiosk`),
and writes an all-users Startup shortcut (`ShopDB Kiosk.lnk`) launching Edge
`--kiosk` fullscreen at `{BaseUrl}{route}`. It does NOT Start-Process Edge
(see gotchas). Base URL comes from HKLM `BaseUrl`, falling back to the WJ
host.
- `seed_display_scope(publish=False)`: `replace_scope_draft`, flush (entries
need entryids), then `service.store_inline_payload(...)` for each script
entry (sets `payloadsource='inline'`, `payloadsha256`, `payloadref`),
optionally `service.publish_scope(...)`, commit. Draft rebuild is
idempotent; publish always creates a NEW version.
Run it on the server:
```
cd C:\inetpub\wwwroot\shopdb
$env:FLASK_APP = 'shopdb'
'from plugins.geenforce.seed_display_scope import seed_display_scope; print(seed_display_scope(publish=True))' | venv\Scripts\python -m flask shell
```
Expected: `{scopeid, entrycount: 6, entrytypes: [Registry x4, PS1, PS1],
dispatchersha256, alwaysonsha256, publishedversion: N}`.
### Publishing
`service.publish_scope` freezes the draft (rendered by
`serializer.scope_to_json`) into an immutable `ManifestPublishedVersion` and
flips `iscurrent`. Clients only ever see published versions.
`rollback_scope` re-currents an older version. Also available over the API
(`POST /scopes/<id>/publish`, permission `geenforce.publish`) and the
GE-Enforce UI.
### Attaching payloads
- Inline (<= 1 MB): `service.store_inline_payload` in code, or
`POST /api/geenforce/entries/<entryid>/payload` (multipart file).
- Blob (`http`): `flask geenforce add-payload <filepath>` prints the sha256;
set `PayloadSource=http` + `PayloadSha256` (+ `PayloadRef` for the
filename/extension) on the entry.
Publish AFTER attaching - the published JSON is what carries the
`PayloadSha256` the client fetches, and blob access for resource-bound tokens
is checked against the CURRENT published manifest.
---
## 5. The on-PC client and engine
### Config: HKLM:\SOFTWARE\GE\ShopDB
| Value | Used by | Notes |
|-------|---------|-------|
| `BaseUrl` | enforce client + kiosk dispatcher | e.g. `https://tsgwp00525.wjs.geaerospace.net/shopdb`. Required. |
| `ApiToken` | enforce client | `geenforce.fetch` (+ report) PAT. OPTIONAL - a token-less client relies on the IP allowlist (`Get-ShopdbConfig` treats BaseUrl-only as valid). |
| `CollectorKey` | `Report-AssetToShopDB.ps1` | `collector.ingest` PAT. REQUIRED for asset reporting (allowlist does not cover the collector). |
The key's ACL is restricted to SYSTEM + Administrators (the bootstrap does
this) so the kiosk auto-login user cannot read the PATs.
### The pieces on disk (kiosk layout, `C:\ProgramData\GE-Enforce`)
- `Invoke-ShopdbEnforce.ps1` - the runner
- `ShopdbEnforceClient.psm1` - the client module
- `lib\Install-FromManifest.ps1` - the engine (>= 2.6)
- `Report-AssetToShopDB.ps1` - the asset collector
- Cache: `C:\ProgramData\ShopDB\geenforce\` (`<scope>.json`, `.etag`,
`.version`, `payloads\`), logs `C:\Logs\Shopfloor\`
### Scheduled tasks (SYSTEM, RunLevel Highest)
| Task | Runs | Interval |
|------|------|----------|
| `ShopDB GE-Enforce` | `Invoke-ShopdbEnforce.ps1 -Scope <scope> -EnginePath <engine> -BaseUrl <url>` | AtStartup + every 15 min |
| `ShopDB Asset Report` | `Report-AssetToShopDB.ps1` | AtStartup + every 60 min |
Tokens are NOT in the task arguments (visible in task XML) - the scripts read
them from HKLM.
### The runner flow (Invoke-ShopdbEnforce.ps1)
1. `Get-ShopdbConfig` (params override registry). No BaseUrl -> exit 0, retry
next cycle.
2. `Sync-ShopdbManifest -Scope <scope>`: ETag-conditional GET; 200 validates
the JSON before overwriting the cache (a proxy error page served as 200
must not clobber last-known-good); 304 or any network failure -> cached
copy. Nothing at all -> Windows event log entry (source `ShopdbEnforce`,
id 1001) plus a best-effort failure report so it is visible server-side,
then exit 0.
3. `-ShadowMode` (with `-ShareManifestPath`): `Compare-ShopdbShadow` logs
name/order diffs, engine runs against the SHARE (zero behavior change).
This is the first step of every cutover.
4. Cutover mode: optional `-IncludeCommon` merges the fleet `common` scope
(`Merge-ShopdbManifests`: common's unique entries first, pctype wins on
Name conflict). OFF by default - a scope is enforced ALONE and displays are
self-sufficient. Then `Resolve-ShopdbPayloads` stages http/inline payloads
(section 3).
5. Engine call (the integration point):
`& $EnginePath -ManifestPath <resolved> -PCType $Scope -InstallerRoot <payloads dir> -LogFile <log>`.
6. `ConvertTo-ShopdbSummary` normalizes whatever came back (summary object,
array of emitted objects, bare int, $null) into
`@{Installed;Skipped;Failed;Filtered;Results;EnforcerVersion}`, then
`New-ShopdbReport` maps to the lowercase wire contract and
`Send-ShopdbReport` POSTs it. All best-effort; the whole script exits 0 no
matter what (fail-safe: a broken web app never breaks a PC).
### The engine contract (Install-FromManifest.ps1, lib 2.6)
Mandatory params: `-ManifestPath`, `-InstallerRoot`, `-LogFile`; optional
`-PCType`, `-PCSubType`. Entry Types: MSI, EXE, CMD/BAT, PS1, INF, File,
Registry. Detection: Registry, File, FileVersion, Hash, MarkerFile,
ValueMatches, pnputil, Always. Filters: PCTypes (with old/new-name alias
groups), TargetHostnames, TargetMachineNumbers, `_CmmVersion`. Exit 0/1/2
unchanged for the SMB path; NEW in the API cutover: the engine ends with
`Write-Output` of a summary pscustomobject
(`Installed/Skipped/Failed/Filtered/EnforcerVersion/Results`), which is the
only thing on the success stream (logs go via Write-Host), so
`& $EnginePath ...` captures it cleanly.
`SelfHealed` on a result means a REAL drift correction (a detected-missing
entry that got reinstalled). `Always`/no-detection entries install every cycle
by design and are not flagged, so the server-side status derivation
(`service.record_enforcement_report`: failed > selfhealed > ok) stays honest.
### The display dispatcher: server-resolved role, file fallback
For kiosks, per-subtype behavior does not fork the scope: ONE scope
(`gea-shopfloor-display`), one inline dispatcher entry (built by
`plugins/geenforce/seed_display_scope.py`) that resolves what the display should
show at enforce time, in two steps:
1. **Server (authoritative):** `GET
$KioskBaseUrl/api/dashboarddefaults/display-role?fqdn=<own-fqdn>`. This is a
PUBLIC endpoint (no token). The server matches the FQDN against the
`dashboarddefaults` table (IP fallback) and returns `{role, path,
businessunitid, businessunit}`. Roles: `dashboard`, `lobby`, `partskiosk`.
Changing a display's job is now a server-side edit; no touch on the PC.
2. **Fallback (offline, or unmapped):** the local
`C:\Enrollment\display-type.txt` value against the `DISPLAY_TYPE_TARGETS` map
baked into the script. If neither resolves, the dispatcher logs and
configures nothing.
The FQDN is built as `F<BIOS serial>.<domain>` (GE device naming); the domain
comes from HKLM `DisplayFqdnDomain` or the built-in default
(`device.geaerospace.net`). `DetectionMethod = Always`, but the script is
idempotent: it rewrites the all-users Startup shortcut (never Start-Process -
see gotchas) only when the resolved target changed.
### Legacy autostart self-heal
The old GE Aerospace Dashboard / Lobby Display Inno installers planted three
autostarts each: a Public-Desktop `.lnk`, an all-users Startup `.lnk`, and an
`HKLM ...\CurrentVersion\Run` value, all launching Edge at now-dead URLs
(`/shopfloor-dashboard/`, `/tv-dashboard/`) which 404 to a white screen. Because
those installers were 32-bit, the Run value was WOW64-redirected into
`HKLM\SOFTWARE\Wow6432Node\...\Run`, invisible to 64-bit tooling - the reason it
survived earlier cleanup. The dispatcher (`build_dispatcher_script` in
`seed_display_scope.py`) now sweeps, every enforce cycle: both the native and
Wow6432Node registry views, every loaded user hive (HKU), Run + RunOnce +
Policies\Explorer\Run, matching by the legacy value names AND by any value
pointing at the old URLs; plus every per-user and common Startup folder; then
kills any running old-URL Edge. A read-only locator,
`pxe-images/github/find-legacy-kiosk-autostart.ps1`, hunts all these locations
(and Edge startup-URL policy, scheduled tasks, Assigned Access) when a straggler
persists.
The kiosk shortcut is a direct Edge shortcut (no launcher/VBS); the fix ships by
re-publishing the code-authored `gea-shopfloor-display` scope
(`seed_display_scope(publish=True)`), not an import-share.
---
## 6. Bootstrap for share-less PCs
A share-less PC cannot pull its first files from the share, so the bootstrap
itself is downloadable from the web app.
`Install-ShopdbKiosk.ps1` (source of truth:
the imaging share (`shopdb-migration/kiosk-installer/`)) is hosted at
`C:\inetpub\wwwroot\shopdb\installers\kiosk\` and downloadable at
`{BaseUrl}/installers/kiosk/Install-ShopdbKiosk.ps1`. Run elevated on the PC:
```
Set-ExecutionPolicy Bypass -Scope Process -Force
$u = 'https://tsgwp00525.wjs.geaerospace.net/shopdb/installers/kiosk/Install-ShopdbKiosk.ps1'
Invoke-RestMethod $u -OutFile "$env:TEMP\Install-ShopdbKiosk.ps1"
& "$env:TEMP\Install-ShopdbKiosk.ps1" -DisplayType Lobby -CollectorKey 'shopdb_pat_...'
# add -ShopdbToken 'shopdb_pat_...' only if the subnet is NOT allowlisted
```
What it does (idempotent; re-running is also the manual update path):
1. Writes `C:\Enrollment\display-type.txt` (the subtype) and
`C:\Enrollment\pc-type.txt` (the scope, default `gea-shopfloor-display`).
2. Writes HKLM `BaseUrl` [+ `ApiToken`] + `CollectorKey`, then locks the key
ACL to SYSTEM + Administrators.
3. Downloads runner + module + engine + collector from
`{BaseUrl}/installers/kiosk/` over HTTPS (TLS 1.2 forced).
4. Registers the two SYSTEM tasks (section 5).
5. Starts both once so the PC is live immediately.
IIS prerequisite: the `installers\kiosk` web.config MUST carry
`<staticContent>` MIME maps for `.ps1`/`.psm1` (`text/plain`) or IIS 404.3s
the downloads (see gotchas).
Delivery options for the bootstrap itself:
- Imaging-baked: the display image runs it (or lays down the same state) at
imaging time - see `project-display-self-contained`.
- Installer push: Intune/hand-run the one-liner above on an already-deployed
PC. This is how the pilot kiosks were done
(`shopdb-migration/run-on-kiosk-F.txt`).
---
## 7. Asset reporting via the collector
`Report-AssetToShopDB.ps1` (in the kiosk bundle; also deployed in the SMB
`common\` scope for the share fleet) POSTs the PC's identity to:
```
POST {BaseUrl}/api/collector/computers
X-API-Key: <collector.ingest PAT or COLLECTOR_API_KEY env key>
```
Auth (`shopdb/core/api/collector.py`, `_check_collector_auth`): a managed
token scoped `collector.ingest` (Bearer or X-API-Key) OR the
`COLLECTOR_API_KEY`/`COLLECTOR_API_KEY_COMPUTERS` env key. The GE-Enforce IP
allowlist does NOT apply here - the collector always needs a key, read from
HKLM `CollectorKey` (or the manifest entry's `Args -ApiKey`).
Schema (`plugins/computers/plugin.py`, `get_collector_schema`): identity field
`hostname` (required); optional `machinenumber`, `pctype`, `pcsubtype`,
`serialnumber`, `loggedinuser`, `lastboottime`, `lastcheckin`, `ipaddress`,
`vendorname`, `modelnumber`, `osname`, `installedsoftware`, `defaultprinter`,
`printers`. All lowercase concatenated (the project naming convention).
`apply_collector_payload` upserts idempotently by hostname (falls back to
`Asset.assetnumber`), creates the Asset+Computer when missing, maps
`machinenumber` -> `Asset.assetnumber` (imaging placeholder `9999` skipped both
client- and server-side), `pctype` -> ComputerType via the settings mapping,
and creates Vendor/Model/OS rows as needed. Fields not posted are not touched -
a partial read never blanks a good value, so the script only includes fields it
actually resolved.
Client details worth keeping: machine number resolution order is eDNC registry
`MachineNo` (WOW6432Node then native) -> `C:\Enrollment\cmm\cmmid.txt` ->
`C:\Enrollment\machine-number.txt`; the reported `ipaddress` is filtered to
the corp ranges (same two CIDRs as the allowlist) so a machine-LAN controller
NIC never lands in shopdb.
---
## 8. HARD-WON GOTCHAS
Read this section before touching ANY of the moving parts. Every bullet cost
real debugging time. Format: symptom -> cause -> fix.
- **PS crash "property 'X' cannot be found" under Set-StrictMode** ->
the module runs `Set-StrictMode -Version Latest`, and engine
results/summaries arrive as EITHER hashtables or PSCustomObjects with
varying key casing; direct `$obj.Key` access on an absent key throws ->
route every dynamic property read through `Get-ShopdbProperty` (handles
both shapes, case-insensitive, returns $null when absent). Never dot into
parsed JSON or engine output directly.
- **Register-ScheduledTask rejects the repeating trigger** -> passing
`-RepetitionDuration [TimeSpan]::MaxValue` serializes to an out-of-range
Duration the Task Scheduler XML schema rejects -> use
`-RepetitionInterval` ALONE; it defaults to indefinite repetition
(verified Win11 / PS 5.1). See `Register-SystemTask` in
`Install-ShopdbKiosk.ps1`.
- **Engine exits 2 / "InstallerRoot not found"** -> `-InstallerRoot` and
`-LogFile` are MANDATORY engine params and InstallerRoot must EXIST ->
the runner always passes both and pre-creates the payloads dir before the
engine call. Any new caller must do the same.
- **http payload "not found: C:\...\C:\..." (path doubling)** -> the engine
resolves entry paths as `Join-Path $InstallerRoot <field>`; writing the
staged payload's ABSOLUTE path into the entry made the engine double it ->
`Resolve-ShopdbPayloads` writes the LEAF filename only, and the runner sets
`-InstallerRoot` to the payloads cache dir. Keep those two in lockstep.
- **Manifest fetch "works" but parsing fails / cache garbage** -> if the
manifest is served with a non-JSON content type, PowerShell 5.1
`Invoke-WebRequest` `.Content` comes back as a `byte[]` instead of a string
-> the server route returns `mimetype='application/json'` (see
`get_manifest`); any mock server or proxy in the chain must do the same.
The client also validates JSON before overwriting last-known-good.
- **Bootstrap download 404 (HTTP 404.3)** -> IIS refuses to serve unknown
static extensions; `.ps1`/`.psm1` have no default MIME map -> add
`<staticContent><mimeMap fileExtension=".ps1" mimeType="text/plain" />`
(and `.psm1`) in the `installers\kiosk` web.config, with `<remove>` first
if inherited.
- **IP allowlist spoofable / mysteriously not matching** -> raw
`X-Forwarded-For` is attacker-controlled (proxies append; first hop is the
caller's to write) -> `_ip_allowlisted` uses `request.remote_addr` via
`_trusted_client_ip`, which is only correct because the IIS URL-Rewrite
rule OVERWRITES X-Forwarded-For with REMOTE_ADDR and waitress trusts only
127.0.0.1 as proxy. The rule is a hard dependency: never remove it, and
verify the spoof is closed after server changes
(`curl -H "X-Forwarded-For: 10.134.48.10"` from a non-allowlisted host
must get 401).
- **Kiosk browser never appears though the dispatcher "ran fine"** -> the
enforce task runs as SYSTEM in session 0, which has no interactive
desktop; `Start-Process msedge.exe` opens INVISIBLY there -> write an
all-users Startup shortcut
(`C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Startup\ShopDB Kiosk.lnk`)
and let the auto-login user launch it in a visible session. This is why
the dispatcher is shortcut-based.
- **TLS/transport errors on older images ("could not create SSL/TLS secure
channel")** -> Windows PowerShell 5.1 does not reliably negotiate TLS 1.2
by default -> every network helper calls `Set-ShopdbTls`
(`[Net.ServicePointManager]::SecurityProtocol = Tls12`) first; the
bootstrap and collector force it too. Any new script that touches the API
must do the same.
- **Secrets readable by the kiosk user / visible in Task Scheduler** ->
default HKLM\SOFTWARE ACL grants BUILTIN\Users read, and task arguments
are world-readable in the task XML -> restrict the
`HKLM:\SOFTWARE\GE\ShopDB` key ACL to SYSTEM + Administrators (the
bootstrap does), and NEVER put a token in a task's `-Argument` string -
scripts read `ApiToken`/`CollectorKey` from HKLM at run time.
- **Enforcement Reports show 0/0/0 with no per-entry detail** -> the engine
historically returned nothing on the success stream, so the runner had no
counts to report -> the 2.6 engine emits the summary object as its ONLY
`Write-Output` (all logging is Write-Host), and
`ConvertTo-ShopdbSummary` tolerates non-compliant engines by zero-filling.
If reports go 0/0/0 again, the engine on that PC is pre-summary - update
it.
- **Two kiosk browsers fighting / stale kiosk launch after retarget** -> old
installs left their own Startup launchers behind, in several flavors ->
the dispatcher's sweep must match ALL of: single- AND double-dash `-kiosk`
arguments (the regex `-kiosk` matches both), shortcuts whose args carry
shopdb URLs (`tsgwp00525`, `/shopdb/`, the dead `shopfloor-dashboard`
route), the imaging installers' `GE Aerospace Dashboard*` / `GE Aerospace
Lobby*` shortcut names, and `.url` files pointing at the kiosk routes.
Extend the sweep whenever a new launcher naming appears.
---
## 9. How this was verified
Two complementary verification passes; keep BOTH for future cohorts because
they catch different bug classes:
- **Code review** finds logic bugs: the XFF spoof hole, the StrictMode
absent-key crashes, the status-derivation trap (`installed>0` is not
self-heal), the report casing mismatch.
- **VM smoke test** finds integration/OS-version bugs: the
RepetitionDuration serialization rejection, the .ps1 MIME 404, session-0
invisibility, TLS negotiation, byte[] vs string response content - none of
which a read of the code surfaces.
The VM rig:
- The win11 virt-manager VM (see `project-win11-vm` /
`reference-vm-qga-as-system` memory), driven by
the imaging share (`ednc-bins/qga-run.py`) - qemu guest agent
`guest-exec`, which runs PowerShell AS SYSTEM. That matters: the scheduled
tasks run as SYSTEM, so testing as SYSTEM reproduced the session-0 and
profile-less behaviors an interactive test would have masked.
- A mock HTTP server on the host exposing `/api/geenforce/manifest` (served
`application/json`), `/api/geenforce/payload/<sha>`, and capturing the
`/api/geenforce/report` POST body. This exercised the full client loop -
ETag/304, cache fallback (kill the server mid-test), payload hash
verification, resolved-manifest rewrite, engine run, summary -> report
mapping - without touching prod.
- Full pass = bootstrap installer run end to end in the VM, then assert: both
tasks registered, HKLM written + ACLed, manifest cached, payloads staged by
sha, Startup shortcut written, report captured with real counts.
---
## 10. Deploy
TWO independent channels. Confusing them is the classic mistake: the client/
engine/collector/bootstrap are pxe-images SHARE artifacts, NOT deployed by the
git pipeline.
### Channel 1: backend + frontend (the git .cmd pipeline)
Prod is air-gapped from dev; code moves via a git bundle on the share
(`\\172.16.9.9\pxe-images\github\shopdb-flask-pub.bundle`) and the .cmd
scripts in the imaging share (`github/`), run on the work PC:
1. `pull-shopdb-bundle.cmd` - fetch the bundle into the local clone
(ff-only).
2. `update-shopdb-github.cmd` - push the clone to GitHub.
3. `update-dev-server.cmd` - robocopy the clone to `X:` (dev `/ops` tree) +
the `/ops`-base frontend dist. Validate on `/ops` FIRST.
4. `update-prod-server.cmd` - robocopy to `Y:` (`C:\inetpub\wwwroot\shopdb`)
+ the `/shopdb`-base dist (`frontend-dist-subpath-shopdb`). Both scripts
carry instance guards (web.config MOUNT_PATH check) - do not bypass them.
Then RDP: `Restart-WebAppPool shopdbflask-prod`, and if migrations/deps
changed: `flask db upgrade`, `flask plugin upgrade-all`, `flask seed
permissions`, `flask seed settings`.
(The fast path used during the pilot - robocopy just the changed plugin files
from `shopdb-migration\prod-patch-geenforce\` + pool restart, per
`deploy-server-patch.txt` - works, but the same commits must ALSO go through
the bundle pipeline or prod drifts from git.)
### Channel 2: client + engine + collector + bootstrap (the kiosk bundle)
These live at the imaging share (`shopdb-migration/kiosk-installer/`) and
deploy by robocopy from the work PC (Z: = share, Y: = prod app dir):
```
robocopy Z:\shopdb-migration\kiosk-installer Y:\installers\kiosk /E
```
That directory (bundle contents: `Install-ShopdbKiosk.ps1`,
`Invoke-ShopdbEnforce.ps1`, `ShopdbEnforceClient.psm1`,
`lib\Install-FromManifest.ps1`, `Report-AssetToShopDB.ps1`, `web.config` with
the MIME maps) IS the distribution point - PCs download from
`{BaseUrl}/installers/kiosk/`. Reference copies of the client kit also live in
the repo at `plugins/geenforce/client/` and the engine's source of truth is
`the imaging share, common/lib/Install-FromManifest.ps1`; when the engine
or client changes, update the kiosk bundle copy too (nothing syncs it
automatically). PCs pick up new bytes by re-running the bootstrap one-liner.
### Server prerequisites (once per site, all three or token-less clients 401)
1. Publish the scope(s) - `seed_display_scope(publish=True)` or
`flask geenforce publish <scope>`.
2. Seed `geenforce_allowed_cidrs` = `10.134.48.0/23,10.48.249.0/26`
(Settings rail > GE-Enforce Settings, or SQL upsert into `settings`).
3. Mint tokens (Settings > API Tokens, Restrict permissions ON):
`collector.ingest` (required, the kiosk `-CollectorKey`) and
`geenforce.fetch` (fallback for non-allowlisted subnets; resource-bind it
to the scope).
4. Keep the IIS URL-Rewrite XFF-overwrite rule enabled (section 2).
---
## 11. PLAYBOOK: extending to a new pc-type
Checklist for cutting any of the remaining scopes (`gea-shopfloor-cmm`,
`-collections`, `-keyence`, `-genspect`, `-heattreat`, `-partmarker`,
`-nocollections`, `common`) over to the API.
1. **Decide the delivery model.** Does this pc-type keep SMB access? If yes,
the cheap cutover is manifest-over-API + payloads-still-smb (entries stay
`smb`, nothing to upload, the engine resolves share paths as today). Only
a genuinely share-less PC needs http/inline payload conversion. Note the
payload endpoint's 512 MB default ceiling
(`GEENFORCE_PAYLOAD_MAX_BYTES`) before promising huge installers over
HTTPS.
2. **Get the scope into shopdb.** Existing share manifest:
`flask geenforce parity` then `flask geenforce import-share --scope
<name>`. New/reworked scope: author in code following
`seed_display_scope.py` (registry heals with ValueMatches detection,
idempotent Always PS1s, data-driven tables for anything per-subtype).
3. **Convert payloads (share-less only).** Small scripts/configs ->
`store_inline_payload` / the entry payload upload endpoint. Installers ->
`flask geenforce add-payload <file>`, set
`PayloadSource=http` + `PayloadSha256` + `PayloadRef` on the entry.
Remember: `PayloadRef`'s extension decides the staged filename's
extension.
4. **Publish.** New version every publish; clients converge within one
enforce cycle. Verify with
`curl "{BaseUrl}/api/geenforce/manifest?pctype=<scope>"` from an
allowlisted host.
5. **Auth for the PCs.** Subnet already inside
`10.134.48.0/23,10.48.249.0/26` -> token-less, nothing to do. New subnet
-> add its CIDR to `geenforce_allowed_cidrs` (Settings rail validates).
Not network-trustable -> mint a `geenforce.fetch` token resource-bound to
the scope and deliver it to HKLM `ApiToken`.
6. **Bootstrap the client.** Share-attached fleet: adapt the dispatcher /
`Install-GEEnforce.ps1` path (the pilot flow in
`shopdb-migration/kiosk-api-pilot.txt`: shadow first, then flip, then
DISABLE the old share enforce task so the two do not fight). Share-less:
the `Install-ShopdbKiosk.ps1` pattern - generalize `-Scope` and skip the
display-only pieces. Decide `-IncludeCommon`: displays run without it;
a non-display share-less PC that needs the fleet-wide common entries over
HTTPS turns it on AND requires common's entries to be payload-converted
first (an SMB-payload common entry will fail on a share-less PC).
7. **Run SHADOW mode first** on one pilot PC
(`-ShadowMode -ShareManifestPath <share manifest>`): fetch + compare +
report with zero behavior change. Watch the shadow diff log lines and the
Enforcement Reports row before flipping.
8. **Re-read section 8 (gotchas).** Especially: LEAF filenames, StrictMode
property access, SYSTEM/session-0, task trigger serialization, MIME maps
if you host new downloadables.
9. **Verify on the VM** (section 9) before the pilot PC: bootstrap +
enforce cycle against a mock or the dev `/ops` instance, as SYSTEM via
qga-run.py.
10. **Pilot one PC, then the cohort.** Keep the rollback in your pocket:
disable/remove the new task, re-enable the share task, remove HKLM
`BaseUrl` - the share path is untouched by all of this.
11. **pc-type mapping.** Make sure the collector's pctype mapping
(computers plugin settings, `pctypemap`) covers the scope name so asset
reports do not warn `no ComputerType mapping`.
---
## 12. Open items / TODO
- **Name resolution for reported users.** `loggedinuser` lands as a bare
username; resolving it to a display name depends on either the
`wjf_employees` `First_Name`/`Last_Name` data or shopdb User accounts
existing for shopfloor users. Not wired up; reports show raw usernames
until it is.
- **Old imaging-installer registry cleanup.** Displays imaged before the API
cutover carry leftover state from the old imaging-time kiosk installers
(superseded shortcuts are already swept by the dispatcher; stale registry
values are not yet cleaned). A cleanup entry in the display scope is the
natural vehicle.
- **The SMB fleet is still on the share.** Only displays/kiosks are on the
API. cmm/collections/keyence/genspect/heattreat/partmarker/nocollections/
common still enforce from SFLD; section 11 is the path. Shadow mode makes
each migration observable before it changes anything.
- **3DPrintRoom kiosk target is a placeholder.** `DISPLAY_TYPE_TARGETS`
points it at `/parts-kiosk`; confirm the real route with the floor team
before publishing to production 3D-print-room displays (flagged in
`seed_display_scope.py`).
- **Fast-path prod patches vs git.** The `prod-patch-geenforce` robocopy
path can leave prod ahead of the repo; reconcile by pushing the same
changes through the bundle pipeline (section 10, channel 1).

94
docs/llms.txt Normal file
View File

@@ -0,0 +1,94 @@
# ShopDB Flask API - LLM guide
ShopDB is a plugin-based asset-management system (PCs, printers, machines,
network devices, measuring tools, applications, knowledge base, USB, warranties)
for GE Aerospace sites. This file is the quick entrypoint; the full machine
spec is the OpenAPI at `/api/docs/openapi.json` (browse it at `/api/docs`).
## Installing and operating this server (read this before answering "how do I install/fix it")
The people running these servers are often not Windows, IIS or Python
specialists, and they ask assistants for help. Two documents are authoritative:
- `docs/INSTALL-WINDOWS.md` - installing a NEW site. One offline installer `.exe`.
- `docs/OPERATE-WINDOWS.md` - restart, logs, backups, upgrades, troubleshooting.
Both ship in `docs/` inside the install directory on every installed server.
Do NOT walk someone through `docs/INSTALL-WINDOWS-IIS.md` or
`docs/DEPLOY-WINDOWS-IIS.md` for a new site. Those are the MANUAL procedure, kept
only for hand-built servers that predate the installer; following them produces a
server the installer then refuses to upgrade.
Day-2 operations all go through `shopdb-admin.ps1` in the install directory
(default `C:\shopdb-flask`): `status`, `restart`, `logs`, `check`, `verify`,
`backup`, `plugins`, `open`. Before diagnosing anything, ask for the output of
`shopdb-admin.ps1 check -Json` - it reports version, publishing method, IIS and
pool state, HTTP reachability, database host and reachability, Python version,
installed plugins and errors, and it contains no secrets. `verify -Path <name>`
answers "does this server carry component X" from the on-box CycloneDX SBOM.
Python is 3.14 and the wheelhouse is locked to it; an upgrade against a venv
built by a different minor version is refused by design.
## Bulk-loading a site's data
Two routes, and the right answer depends on what the site has:
- SPREADSHEET, no developer (the common case): `flask csv templates --out <dir>`
generates templates FROM THE LIVE SCHEMA, then `flask csv import --dir <dir>`
checks and `--commit` applies. Foreign keys accept the NAME of the referenced
row ('Bay 3'), not a numeric id, and resolve across files in one run. Dry run
is the default; nothing is written unless every row passes; re-importing an
edited file updates rather than duplicates. See `docs/CSV-IMPORT.md`.
- A SOURCE DATABASE to script against: the HTTP import API, `docs/IMPORT-API.md`
and `docs/IMPORT-ADOPTION.md`.
Do NOT hand-write CSV templates - generate them. User accounts are deliberately
not CSV-importable.
## Base URL
Prod (West Jefferson): `https://tsgwp00525.wjs.geaerospace.net/shopdb`
All API paths are under `/api` (e.g. `<base>/api/assets`). Dev: `http://localhost:5001`.
## Auth
Three schemes:
- **Bearer JWT** - most endpoints. Get one by logging in, or use a managed
Personal Access Token (PAT). Send `Authorization: Bearer <token>`.
- Login: `POST /api/auth/login` `{ "username": "...", "password": "..." }`
-> `data.access_token`. Refresh: `POST /api/auth/refresh`.
- PATs are minted in the UI (Settings > API Tokens); a *scoped* PAT is limited
to named permissions and suspends the admin bypass.
- **X-API-Key** - unattended/service endpoints (collector ingest, GE-Enforce
fetch). Send `X-API-Key: <managed-token>`.
- **Public** - some read endpoints (e.g. printer install-list, employee search,
dashboards) need no auth.
Auth level per endpoint is in the OpenAPI `security` field: `bearerAuth`,
`apiKeyAuth`, or none. Admin-only and permission-gated routes both use bearer.
## Response envelope
JSON endpoints return `{ "status": "success", "data": <payload>, "meta": {...} }`.
Errors: `{ "status": "error", "message": "...", "code": "..." }` with an HTTP 4xx/5xx.
Lists include `meta.total` / pagination. A few feed endpoints (screensaver, some
installer text formats) return raw text/JSON without the envelope - noted per route.
## Common recipes
- Search everything: `GET /api/search?q=<term>` (multi-word = AND across words).
- List assets on the map: `GET /api/assets/map`.
- List a type: `GET /api/printers`, `/api/computers`, `/api/machines`,
`/api/network`, `/api/measuringtools` (paginated: `?page=&perpage=`).
- Get one: `GET /api/printers/<id>` etc.
- Create (bearer): `POST /api/printers` `{assetnumber, windowsname, vendorid, ...}`.
- Reports: `GET /api/reports` (list), `GET /api/reports/pc-relationships` (PC<->machine).
- Printer installer data: `GET /api/printers/install-list` (public; add
`?format=text` for a pipe-delimited variant); `GET /api/printers/pc-default?machine=<n>`.
- Collector ingest (X-API-Key): `POST /api/collector/computers`.
- GE-Enforce: `GET /api/geenforce/manifest?pctype=<scope>`,
`GET /api/geenforce/payload/<sha256>`, `POST /api/geenforce/report`.
- Import (admin PAT, preserves timestamps with `X-Import-Mode`): see docs/IMPORT-API.md.
## Conventions
- DB-mirrored params/fields use lowercase concatenated names (no underscores):
`locationid`, `vendorid`, `windowsname` - match them exactly.
- IDs in paths are integers.
- Plugin endpoints live under the plugin's prefix (`/api/<plugin>/...`).
## Full reference
- Machine spec: `GET /api/docs/openapi.json` (OpenAPI 3.1, 362 operations).
- Interactive: `GET /api/docs` (Redoc).
- Human reference: `docs/API-REFERENCE.md`.

6620
docs/openapi.json Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,648 @@
# Proposal: GE-Enforce as a shopdb plugin
Status: ACCEPTED / built - see ADR-012 and plugins/geenforce/.
Author: planning session 2026-07-12.
## 1. What this is
Today GE-Enforce is a PowerShell manifest engine that reads per-PC-type
`manifest.json` files off an SMB share (`\\tsgwp00525.wjs.geaerospace.net\
shared\dt\shopfloor\`). Each logon, a scheduled task running as SYSTEM mounts
the share, reads the manifest for the machine's PC type, and installs or
self-heals apps, files, drivers, registry values, and scripts. A parallel
`preinstall.json` runs the same schema once at imaging.
This proposal turns the *manifest* into shopdb data: the authoritative manifest
lives in the shopdb database, is edited through the shopdb UI (an expansion of
`/settings/pctypemapping`), and is served to clients over HTTP as JSON. The
*payloads* (MSI/EXE/PS1/config bytes) stay on SMB, on HTTP, or both, referenced
by URL/path from the manifest rows. GE-Enforce.ps1 changes from "read a file on
W:" to "GET a manifest from shopdb, then fetch each payload from wherever the
row says."
The result: managing imaging PC types, their apps, scripts, files, registry
rules, and version gates becomes a first-class shopdb feature instead of hand-
edited JSON on a file share.
## 2. Why it fits shopdb
- shopdb already models the fleet (the collector ingests every PC's hostname,
pctype, installed software, versions). Making shopdb *also* own what SHOULD be
installed closes the loop: desired-state (manifest) and observed-state
(collector) live in one system and can be diffed.
- `/settings/pctypemapping` already maps `gea-shopfloor-*` PC types to
`ComputerType`. That page becomes the entry point for full imaging-PC-type
management.
- The plugin contract (per-plugin models, migrations, API prefix, settings
cards, collector hooks) is exactly the shape this needs.
- ADR-004 (per-site instances) matches: each site's shopdb owns each site's
manifest. No multi-tenant complication.
## 3. Grounding: the real manifest schema
Source of truth for these field names (do not invent others):
- Schema: `pxe-images/tsgwp00525-v2/shared/dt/shopfloor/_meta/manifest-schema.json`
- Engine: `pxe-images/common/lib/Install-FromManifest.ps1`
- Dispatcher: `.../shopfloor/common/GE-Enforce.ps1`
- Architecture: `pxe/docs/ge-enforce-v2-architecture.md`
A manifest is `{ "Version": str, "_comment": str, "Applications": [entry, ...] }`.
Only `Name` and `Type` are required per entry.
### Per-entry fields (complete set)
Identity / action:
- `Name` (required, unique, also the status-key `<scope>/<Name>`)
- `Type` (required): one of `MSI EXE CMD BAT PS1 INF File Registry`
- `_comment` (documentation, heavily used in practice)
Type-specific payload references (sparse; depends on Type):
- MSI/EXE/CMD/BAT/INF: `Installer` (relative path) + `InstallArgs`
- PS1: `Script` (relative path, falls back to `Installer`) + `Args`
- File: `Source` (relative) + `Destination` (absolute on-PC path)
- Registry: `RegPath` + `RegName` + `RegValue` + `RegType`
(`RegType` in `String DWord QWord MultiString ExpandString Binary`)
- Optional `LogFile`, `WaitTimeoutSec` (EXE hang kill), `InUseCheck`
Detection (decides whether the action fires / self-heals):
- `DetectionMethod`: one of
`Registry File FileVersion Hash MarkerFile ValueMatches pnputil Always`
- `DetectionPath`, `DetectionName`, `DetectionValue`, `DetectionPattern`
- Note: `DetectionValue` is method-dependent - SHA256 for Hash, a 4-part
version for FileVersion, a registry value for Registry, ignored for
Always/File. Same column, different meaning per method.
- No `DetectionMethod` = always installs.
Targeting filters (all ANDed; each is multi-value):
- `PCTypes` (array; `"*"` = all; alias graph expands old<->new names)
- `PCSubTypes` / subtype via `<pctype>-<subtype>` values
- `TargetHostnames` (array; exact + `-like WJS-*` wildcards)
- `TargetMachineNumbers` (array; per-bay)
- `_CmmVersion` (scalar; per-entry PC-DMIS version gate, needs lib >= 2.6)
Nested:
- `InUseCheck`: `{ Behavior, Processes: [{Name, ExePath, GracefulCloseTimeoutSec}] }`
Behavior in `Defer CloseAndReopen ForceClose ScheduleForReboot`
Parsed-but-inert today (model them, mark inert):
- `ApplyMode` (`Nightly Immediate ImmediateReboot`), `UpdateWindow` (`HH:MM-HH:MM`)
Preinstall-only extras (phase discriminator):
- `PreEnrollment`, `KillAfterDetection`, `PCTypesStrict`, `_pcTypesNote`
### Load-bearing behaviors the model must preserve
1. **Array order IS execution order.** Config-restore entries are deliberately
placed AFTER their vendor installer so a mid-cycle overwrite heals the same
cycle (eMxInfo.txt after eDNC; udc_webserver_settings after UDC). We MUST
store an explicit per-scope `sortorder`, not a set.
2. **PCTypes alias graph** is many-to-many old<->new names resolved by set
intersection, with a `PCTypesStrict` escape hatch. Not a simple FK.
3. **Polymorphic entry by Type** - sparse column set per type. DECISION: one
wide `manifestentries` table with an `entrytype` discriminator column and
nullable per-type columns. NOT SQLAlchemy STI subclasses, NOT a JSON blob.
Justification (section 4): the whole fleet is ~64 entries, so sparse columns
cost nothing; real columns get validated, indexed, field-diffed, joined
against collector data, and read in plain SQL by an IT tech - a JSON blob
hides all of that, and class-per-type STI is expert ceremony for no gain. A
`validate()` that switches on `entrytype` (mirroring the engine's own
`switch ($App.Type)`) is ~40 obvious lines.
4. **Two manifest phases** - runtime (self-heal, per logon) and preinstall
(once at imaging) share the schema. One table with a `phase` discriminator.
## 4. Data model (new `geenforce` plugin)
Per-plugin Alembic chain (ADR-008). Tables (lowercase concatenated per naming
convention). Sizing that shapes every decision here: the real fleet is 10
runtime scopes = 43 entries, plus 1 preinstall manifest = 21 entries, so ~64
rows total. That smallness is why this stays deliberately low-tech (one wide
table, JSON-document snapshots, no row-mirroring) - the design target is an
average site IT tech maintaining it, not a specialist.
- `manifestscopes` - one row per imaging PC type / scope.
- `scopeid` PK
- `scopename` (e.g. `gea-shopfloor-cmm`)
- `phase` enum (`runtime` | `preinstall`)
- UNIQUE (`scopename`, `phase`), NOT `scopename` alone: `common` exists in
runtime, and a scope name can appear in both phases. Note the phases are
shaped differently - runtime is many per-pctype scopes (one manifest file
each), preinstall is ONE flat manifest gated internally by `PCTypes`, so
preinstall is modeled as a single `phase=preinstall` scope, not per-pctype
scopes.
- `computertypeid` FK -> `computertypes` (this REPLACES the thin
`pctypemap_<pxetype>` setting; the mapping becomes a column here).
Runtime-scope only; null for the preinstall scope.
- `measuringtooltypeid` FK -> `measuringtooltypes`, nullable (metrology
scopes: what device this scope implies; keeps imaging + collector agreed,
see section 11).
- `manifestversion` (string, mirrors manifest `Version`)
- `description`, `isactive`
- `iscommon` bool (the `common/` fleet-wide scope)
- `manifestentries` - one row per Applications[] entry (the working/draft copy).
- `entryid` PK, `scopeid` FK
- `sortorder` int (preserves array order; the ordering contract)
- `name`, `entrytype` (MSI/EXE/.../Registry), `comment`
- payload columns (nullable, per type): `installer`, `installargs`,
`scriptpath`, `scriptargs`, `sourcepath`, `destination`,
`regpath`, `regname`, `regvalue`, `regtype`
- `payloadsource` enum (`smb` | `http` | `inline`) + `payloadref`
(see section 5)
- `payloadsha256` - integrity hash of the payload bytes, INDEPENDENT of the
detection method. Mandatory for `http`/`inline` payloads; optional for
`smb`. Do NOT reuse `detectionvalue` for this - `detectionvalue` is a
SHA256 only when `detectionmethod = Hash`; an MSI with `Registry`/
`FileVersion` detection has no payload hash, so an HTTP fetch would
otherwise run unverified bytes (see section 5).
- `regvalue` stores the RAW JSON literal (`1` vs `"1"`) and is emitted
verbatim on export. `RegValue` is untyped in the manifest schema and real
entries carry numbers; the engine string-coerces for `ValueMatches` but
`Set-ItemProperty -Type DWord` cares, so preserve the literal.
- detection columns: `detectionmethod`, `detectionpath`, `detectionname`,
`detectionvalue`, `detectionpattern`
- gates: `cmmversion`, plus child tables for the multi-value filters
- control: `logfile`, `waittimeoutsec`, `applymode`, `updatewindow`
(`applymode`/`updatewindow` are parsed-but-INERT in the engine today; the
UI must label them "not yet enforced" so a tech does not trust a dead gate)
- preinstall flags: `preenrollment`, `killafterdetection`, `pctypesstrict`
- `isactive`
- `manifestpublishedversions` - immutable published snapshots, SIMPLIFIED to
freeze the rendered JSON DOCUMENT in a single `manifestjson` column (drop the
row-mirrored `manifestpublishedentries` family the earlier draft proposed).
The only consumer of a snapshot is the client, and it consumes exactly that
document, so freezing the text makes immutability structural (no UPDATE path),
rollback a one-flag `iscurrent` flip, serving a single-row read, and version
diffing a plain text diff - all things average IT can debug; row-mirroring
would add ~6 shadow tables and a copy routine that can drift. Columns:
`publishedversionid`, `scopeid`, `versionnumber` (1,2,3 per scope),
`manifestjson` (MEDIUMTEXT, verbatim), `publishedat`, `publishedby`,
`iscurrent`, `notes`. Editing `manifestentries` never affects the fleet;
"publish" freezes a new snapshot; the client is ALWAYS served the current
snapshot, never the live draft. Rollback = flip `iscurrent` to an older
version (the post-cutover safety net once the on-share JSON is retired).
Mirrors today's `_meta/history/<date>-<scope>.json` backups, but authoritative.
Revision history: every publish is a permanent, immutable revision kept
indefinitely (snapshots are small JSON text, ~10 scopes - storage is a
non-issue). An OPTIONAL retention policy (keep last M per scope, or prune
older than N months) can be added later; default is keep-everything, off.
- Draft-edit audit trail (field-level history BETWEEN publishes): drafts
(`manifestentries`) are not versioned - editing overwrites the working copy.
To answer "who changed this entry and when" in the window between two
published revisions, log every draft mutation through the EXISTING core audit
system (no new table): on create/update/delete of a scope, entry, or child
row, write an audit record with the actor, timestamp, entry name, and the
changed field(s). This gives per-edit provenance for free and shows up in the
same Audit Logs UI IT already uses; the published snapshots remain the
coarse-grained "what the fleet actually got" record.
- `manifestentrypctypes`, `manifestentryhostnames`, `manifestentrymachinenumbers`
- child rows for the ANDed multi-value filters (one value + a `sortorder` per
row, wildcards stored verbatim as patterns)
- `manifestinusechecks` + `manifestinusecheckprocesses`
- the nested InUseCheck object and its Processes[] child list (leave
`gracefulclosetimeoutsec` nullable; do not bake the engine's default of 10
into the row, emit it only when set)
- `manifestpayloads` - inline payload bytes for `payloadsource = inline`
(`entryid`, `filename`, `contenttype`, `payloadbytes` LONGBLOB, `payloadsha256`,
`uploadedat`). App-enforced size cap ~1 MB; the upload UI rejects larger with
"use SMB for this" so nobody pastes an MSI into the database. Can ship empty
and unused until P6.
- `pctypealiases` - a MIRROR of the old<->new name alias graph from
`Install-FromManifest.ps1:463-475`, for server-side resolve/validate only.
The engine lib stays the single source of truth (see section 10); shopdb
never becomes the authority the client depends on for aliases.
The JSON the client receives is REBUILT from a published snapshot in exact
array order. Parity with the current engine is proven by BEHAVIORAL equivalence,
not byte-identity (see section 9): re-serialized JSON will differ in key order
and whitespace, so the test is that both manifests parse to the same ordered
entry set with the same detection/targeting/action semantics.
## 5. Payloads: SMB and/or HTTP (both supported)
The user asked whether payloads can be SMB and/or HTTP. Yes - per entry:
- `payloadsource = smb`: `payloadref` is the current relative path
(`apps/eDNC_6-4-5.msi`); the client still mounts W: and resolves it against
the scope root exactly as today. The engine is unchanged for these rows (the
mount + scope-root resolution still happen; an HTTP-only site skips the mount
because it has no `smb` rows). This is the default and the migration target
for large binaries (MSIs are hundreds of MB; SMB streaming beats HTTP).
- `payloadsource = http`: `payloadref` is a URL (absolute, or relative to a
configured payload base). The client downloads to a local temp dir, verifies
the Hash/FileVersion detection value, then runs it. Good for small
config/script payloads and for sites with no SMB share.
- `payloadsource = inline`: for small text payloads (a `.ps1`, a config file, a
registry value), the bytes live in shopdb itself and are served in-band. No
external store at all. Best for scripts and File-type config drops.
Manifest generation emits, per entry, whatever the client needs to fetch the
bytes. The engine's existing "stage network EXE to local temp first" logic
(SYSTEM access-denied workaround) generalizes cleanly to HTTP download.
Payload integrity uses the dedicated `payloadsha256` column, NOT `DetectionValue`.
This is the correction to a subtle trap: `DetectionValue` is a SHA256 only when
`DetectionMethod = Hash`. Most binaries detect by `Registry` or `FileVersion`
and carry no payload hash at all, so relying on `DetectionValue` would let an
HTTP/inline-fetched MSI run unverified. Instead, publishing an `http`/`inline`
payload computes and stores `payloadsha256`, and the client verifies the fetched
bytes against it BEFORE running, independent of how the entry detects install
state. `smb` payloads may set it too (defense in depth) but the share ACL is
their primary trust boundary. Detection stays a separate concern: it decides
whether to act; the payload hash decides whether the bytes are trustworthy.
Transport security: the client fetches as SYSTEM, so the shopdb TLS cert must be
trusted machine-wide. Sites with a self-signed or air-gapped shopdb need the CA
in the machine trust store (provisioned by the same Azure DSC step that writes
the token). Plain HTTP is acceptable only inside a trusted segment, and even
then the `payloadsha256` check is what actually guarantees payload integrity.
## 6. API surface (`/api/geenforce/...`)
Two permissions via the plugin's `get_permissions()` hook (split so day-to-day
techs can edit but only a lead ships to the fleet):
- `geenforce.manage` - create/edit/reorder scopes, entries, drafts, payloads.
- `geenforce.publish` - publish, rollback, export-to-share (the fleet-affecting
actions).
Draft editing (`geenforce.manage`):
- `GET/POST /scopes`, `GET/PUT/DELETE /scopes/<id>` - imaging PC types
- `GET/POST /scopes/<id>/entries`, `PUT/DELETE /entries/<id>` - manifest entries
- `PUT /scopes/<id>/entries/reorder` - the ordering contract; Move Up/Down in the
UI (plain buttons + visible `sortorder`), not a drag-and-drop dependency
- `POST /entries/<id>/payload` - upload an inline/http payload (multipart),
compute + store its `payloadsha256` (the integrity hash; NOT `detectionvalue`)
- `GET /scopes/<id>/preview` - the draft JSON a client WOULD receive on next
publish; `GET /scopes/<id>/published` shows the currently-served snapshot
- `GET /scopes/<id>/simulate?pctype=&subtype=&hostname=&machinenumber=&cmmversion=`
- the "what would this PC get" simulator: runs the entry list through the same
filter logic the engine uses and returns which entries apply and why the rest
are filtered out. Reuses the P1 parity harness's filter engine, so it is
nearly free, and it is the single most IT-empowering endpoint - it answers
"why did/didn't app X install on PC Y" without reading a PowerShell log.
Publishing (`geenforce.publish`):
- `POST /scopes/<id>/publish` - freeze the current draft into a new immutable
`manifestpublishedversions` snapshot (this is what the fleet gets)
- `POST /scopes/<id>/rollback/<version>` - mark an older snapshot current
- `POST /scopes/<id>/export-share` (or a `flask geenforce export-share` CLI) -
write the current published JSON to `<shareroot>/<scope>/manifest.json` after
copying the existing file to `_meta/history/<date>-<scope>.json`. This is a
first-class feature, not a footnote: it is the Milestone 1 product (author in
shopdb, engine untouched) and the permanent break-glass path.
Client-facing (gated by a collector-style service token, `geenforce.fetch`
scope, reusing the PAT + `X-API-Key` machinery already built for the collector):
- `GET /manifest?pctype=<scope>&subtype=<s>&hostname=<h>&machinenumber=<n>`
Returns the latest PUBLISHED snapshot for that scope (never the live draft).
The server can pre-apply the PCTypes/hostname/machinenumber/cmmversion filters
(thin client) OR return the full scope and let the engine filter (fat client,
matches today). Start fat: return the scope manifest unchanged so the engine
logic is untouched. Include the snapshot version + an ETag so the client can
cache and no-op when unchanged.
- Payload fetch for `http`/`inline` rows: `GET /payload/<entryid>` streaming the
bytes; the client verifies them against `payloadsha256` from the manifest.
## 7. Frontend: expand `/settings/pctypemapping`
The current page (`PCTypeMappingSettings.vue`, "Collector PC Types") is a read-
only-ish table of `pxetype -> ComputerType` dropdowns. It grows into the imaging-
PC-type manager:
- **Scopes list**: add/rename/delete imaging PC types; each still carries its
`ComputerType` mapping (that column moves from a setting into `manifestscopes`).
A `phase` toggle (runtime vs preinstall). Common scope flagged.
- **Scope detail / manifest editor**: an ordered list of entries with Move
Up/Down buttons and a visible `sortorder` (the ordering contract made visible;
NOT drag-and-drop - a drag library is the kind of dependency that breaks
silently and average IT cannot fix; add drag later if wanted). Each entry is a
typed form - the visible fields switch on `entrytype` (MSI shows
Installer+InstallArgs; PS1 shows Script+Args; File shows Source+Destination;
Registry shows the Reg* quartet), one line of help per detection method.
Filter chips for PCTypes/hostnames/machine numbers. InUseCheck sub-editor.
Payload source selector (smb/http/inline) with upload for the latter two.
`applymode`/`updatewindow` sit behind an "Advanced (not yet enforced by the
engine)" disclosure. Ship the editor in three usable-alone increments: (a)
scope list + entry table, (b) the typed entry form, (c) publish + diff. That
keeps the biggest chunk of the build from ballooning.
- **Simulator ("what would this PC get")**: a small form (pctype, subtype,
hostname, machine number, CMM version) that calls `GET /scopes/<id>/simulate`
and lists which entries apply and why the rest are filtered. The single most
IT-empowering piece of the UI.
- **Draft, preview, publish**: editing changes only the draft; "publish" freezes
an immutable snapshot (see section 4) and is what the fleet then gets. Show the
draft-vs-published diff before publishing. Rollback republishes a prior
snapshot.
- **Desired vs observed (BUILT: observed-state reporting)**: rather than extend
the collector, the plugin has its own reporting path. Each enforcement cycle a
PC POSTs `POST /api/geenforce/report` (geenforce.report service token) with the
published version it applied, the installed/skipped/failed/filtered counts, and
per-entry outcomes. Stored in `manifestenforcementreports` (latest-per-host +
history) and `manifestenforcementresults` (per-entry). Two payoffs fall out:
RECEIVED - `receivedlatest` compares the applied version to the scope's current
published version, so the fleet view shows which PCs picked up an update; and
SELF-HEAL - each entry's action (installed = drift corrected, skipped = already
good, failed) with any warning/error message. Admin reads: `GET /reports`
(fleet compliance) and `GET /reports/<id>` (per-entry detail). This is the
observed half that makes the manifest a closed desired-vs-observed loop.
This is an ADR-010 settings card contributed by the geenforce plugin, so it only
appears when the plugin is enabled.
## 8. Client change (minimal, staged)
`GE-Enforce.ps1` today: mount W:, read `<scope>\manifest.json`, hand to
`Install-FromManifest`. New path: GET the manifest from shopdb, write it to the
same local location the engine reads, then run the engine unchanged. That is the
smallest possible client delta - the engine, detection logic, self-heal, and
SMB payload resolution all stay identical. Only the *source of the JSON* moves
from file to HTTP.
Payloads: `smb` rows need no client change. `http`/`inline` rows need a small
fetch-and-verify helper (download to temp, check SHA256, then the existing
installer action runs against the local copy). The engine already stages network
EXEs to temp, so this is an extension, not a rewrite.
Auth: the client already has SFLD credentials in
`HKLM:\SOFTWARE\GE\SFLD\Credentials`. Add a shopdb service token (a
`geenforce.fetch` PAT) provisioned the same way (Azure DSC writes it to
registry), sent as `X-API-Key`. If shopdb is unreachable, the client falls back
to the last-known-good manifest cached locally (fail-safe: never leave a PC
unmanaged because the web app is down). This mirrors today's "creds missing =
exit 0, retry next cycle" resilience.
## 9. Cutover strategy
The manifest is desired-state that runs as SYSTEM and installs software fleet-
wide. A bad cutover = a fleet-wide mis-install. Stage it:
1. **Import + parity.** Write a one-shot importer that reads the current
on-share manifests (common + every `gea-shopfloor-*` + preinstall.json;
skip `.bak` / `.pre-mtconnect.bak` variants) into the new tables. Then
generate JSON back out and prove BEHAVIORAL equivalence for every scope - do
NOT chase byte-identity. Re-serialized JSON will differ in key order,
whitespace, and `_comment` formatting, so a raw `diff` would never converge.
The correct test: parse both the original and the regenerated manifest,
normalize, and assert the same ordered entry list with identical
detection/targeting/action fields per entry (ideally a small harness that
mimics the engine's filter+detect decisions and confirms the same entries
would fire in the same order on representative machine profiles). That, not
byte equality, is what proves the model is lossless. (Same discipline as the
ADR-001 data migration.)
2. **Shadow mode.** shopdb serves the manifest at a new endpoint; a canary PC
fetches from shopdb but ALSO reads the share, and logs any diff. No install
behavior changes. Run across one of each PC type for a few cycles.
3. **Read cutover, payloads still SMB.** Flip GE-Enforce to source the JSON from
shopdb (payloads stay `smb`). The blast radius is only "where the JSON comes
from"; the bytes and engine are unchanged. Keep the share manifests as the
rollback (revert the dispatcher one-liner).
4. **Payload migration (optional, per entry).** Move small scripts/configs to
`inline`/`http` opportunistically. Leave big MSIs on SMB indefinitely - SMB
is the right transport for them.
5. **Author in shopdb.** Once read-cutover is stable, new manifest edits happen
in the shopdb UI and the on-share JSON is retired (or auto-exported as a
backup for break-glass).
Rollback during cutover (stages 2-4) is a one-line dispatcher revert, because
the engine and payload layout never stop working from the share. AFTER the share
JSON is retired (stage 5), that escape hatch is gone - post-cutover rollback is
republishing a prior `manifestpublishedversions` snapshot (section 4). Both
mechanisms must exist before stage 5, not just the dispatcher revert.
## 10. Risks / open questions
- **The engine is the contract.** Any drift between shopdb's generated JSON and
what `Install-FromManifest.ps1` expects is a fleet-wide install bug. The
byte-identical round-trip test (step 1) is non-negotiable, and the plugin must
pin which engine lib version it targets (>= 2.6 for `_CmmVersion`).
- **PCTypes alias graph** must be kept in sync with
`Install-FromManifest.ps1:463-475`. The engine lib stays the single source of
truth; shopdb only MIRRORS the map for server-side validation. Do NOT invert
this to have the engine fetch aliases from shopdb - that would add exactly the
availability coupling the next bullet warns against. When the lib's alias map
changes, update shopdb's mirror as part of shipping that lib version.
- **Availability coupling.** GE-Enforce currently depends only on SMB. Adding an
HTTP dependency on shopdb means shopdb downtime could stall enforcement -
hence the last-known-good local cache in section 8. Must be built in from day
one, not bolted on. This is also why alias resolution and payloads stay
independent of a live shopdb wherever possible.
- **Transport trust.** The client runs as SYSTEM, so shopdb's TLS cert must be
in the machine trust store (self-signed/air-gapped sites need the CA
provisioned via the same DSC step as the token). `payloadsha256` verification
is the real integrity guarantee and holds even over plain HTTP inside a
trusted segment (section 5).
- **Secrets in payloads.** Some config drops (site-config, credentials) may
contain secrets. `inline` payloads live in the shopdb DB - those must respect
the existing "secrets stay in .env, not the settings table" rule. Likely keep
any secret-bearing payload on SMB with ACLs, never inline.
- **Preinstall runner** is a separate consumer (`00-PreInstall-*` at imaging,
before enrollment). It may not have a shopdb token yet at that point in the
imaging sequence. Preinstall may need to stay share-sourced longer than
runtime, or fetch a bootstrap manifest anonymously over HTTP.
- **This is a big build.** Realistically phased: (P1) model + importer +
behavioral-parity test; (P2) admin API + CRUD + publish/snapshot/rollback;
(P3) frontend editor on /settings/pctypemapping; (P4) client fetch + shadow
mode; (P5) read cutover; (P6) payload migration. P1 is the gating de-risk - if
behavioral parity does not hold, stop. Snapshots (P2) must land before any
client points at shopdb (P4), since serving the live draft to the fleet is
unacceptable.
## 11. Relationship to existing work
- Replaces `plugins/computers/pctypemap.py` (the thin `pctypemap_<pxetype>`
settings) - the pctype -> ComputerType mapping becomes the `computertypeid`
column on `manifestscopes`. Two-source transition window: `pctype_mapping()`
must keep reading the settings until the geenforce plugin is enabled, then
fall back geenforce-table-first / settings-second, and only retire
`seed_pctype_settings` + the settings at Milestone 1 close. Also reconcile the
scope inventory: `pctypemap.py` lists `gea-shopfloor-display` but the share has
no such manifest dir, and the share has a `main/` legacy dir the model ignores
- the importer creates scopes only from what it finds (plus empty scopes for
mapped-but-absent pctypes), and the P1 gate review reconciles the list with
the floor team.
- Also folds in the metrology mapping now living in `pctypemap.py`
(`METROLOGY_TOOL_MAP`). The collector already auto-creates a MeasuringTool
asset and a directional PC->tool `controls` relationship when it sees a
metrology pctype (CMM / Keyence / Genspect / wax-and-trace); the PC stays a
shopfloor PC. A metrology scope in the manifest model should carry the
attached-measuring-tool type alongside its ComputerType so imaging and
collector agree on what device the scope implies.
- Reuses the collector's token machinery (PAT + `X-API-Key` + scopes) for the
client-facing endpoints.
- Reuses `get_permissions()` (contract 0.10.0) for `geenforce.manage` (edit
drafts) / `geenforce.publish` (publish, rollback, export) / `geenforce.fetch`
(the client service token).
- Pairs with the collector: desired-state (this plugin) + observed-state
(collector) enable a fleet compliance view.
## 12. Recommendation
Feasible and a strong architectural fit, but it is a multi-phase build with a
fleet-wide blast radius. The single most important gate is P1: import the real
manifests and prove BEHAVIORAL parity (same entries fire in the same order with
the same detection/targeting), not byte-identity. Do not build the UI or touch a
client until that parity holds. Three things separate a safe build from a
dangerous one and must not be cut: behavioral-parity import (P1), immutable
published snapshots with rollback before any client points at shopdb (P2/P4),
and a dedicated `payloadsha256` for every HTTP/inline payload (section 5). If and
when we proceed, this warrants a new ADR (ADR-012: GE-Enforce manifest
ownership) capturing the desired-state model, the published-snapshot contract,
the SMB/HTTP/inline payload + integrity model, and the fail-safe cache.
## 13. Execution plan (build order, gates, milestones)
Governing constraint: every step must be runnable and maintainable by average
site IT, not just the original developer. Where an earlier draft implied expert
machinery, this section simplifies it (and the model above already reflects
those simplifications: one wide table, JSON-document snapshots, no row-mirroring).
### Phases and gates
- **P0 - Scaffold (S, ~0.5-1 day).** `flask plugin new geenforce`, structure
copied from `plugins/measuringtools/`. Unlike bundled plugins' no-op migration
anchors, this NEW plugin's `0001_geenforce_baseline` actually creates the
tables and registers them in `PLUGIN_TABLE_OWNERS` (ADR-008). Deploy stays the
standard `flask db upgrade` + `flask plugin upgrade-all`. Manifest:
`api_prefix: /api/geenforce`, `default_enabled: false`, tight `core_version`.
- **P1 - Model + importer + parity harness (M, ~1-1.5 wk). THE GATE.** Order
inside: tables -> `flask geenforce import-share` (reads common + every
`gea-shopfloor-*` + preinstall.json, skips `.bak`, idempotent) -> exporter
(rebuilds each scope's JSON from rows in `sortorder`) -> the parity harness
(below). **GATE A:** `flask geenforce parity` prints PASS for all scopes. If
it cannot pass, STOP the project. No API/UI/client work before Gate A.
- **P2 - Publish/snapshot/rollback + admin API + export-to-share (M, ~1.5-2 wk).**
Publish freezes rendered JSON into `manifestpublishedversions`. CRUD per
section 6. Plus `flask geenforce export-share` + an "Export to share" button
that writes each scope's published JSON to the share after backing up the old
file to `_meta/history/`. Engine, dispatcher, share layout, payloads, PCs all
untouched. **GATE B = Milestone 1** (below).
- **P3 - Frontend editor (L, ~2-3 wk; parallel with P4 after P2 API freezes).**
Expand `PCTypeMappingSettings.vue` per section 7, in three shippable
increments; Move Up/Down not drag; the simulator.
- **P4 - Client fetch + shadow mode (M effort + soak time; needs P2, not P3).**
Week-1 spike: a ~20-line PS1 on ONE canary PC proves SYSTEM-context HTTP auth +
TLS trust before any real client change. Then `GE-Enforce.ps1` fetches JSON to
a local cache and hands the file to `Install-FromManifest.ps1` unchanged;
shadow mode installs from the share but logs any diff vs shopdb; ETag +
last-known-good cache from day one. **GATE C:** zero shadow diffs across one PC
of every pctype for >= 20 cycles.
- **P5 - Read cutover (S effort, M calendar).** Per-scope flip, canary first via
`TargetHostnames`. Payloads stay `smb`. Rollback = dispatcher revert; share
export continues as break-glass. **GATE D:** all scopes cut over.
- **P6 - Payload migration (S per entry, optional forever).** Small configs to
`inline` (verified by `payloadsha256`); MSIs stay on SMB. Each entry
independently revertible (flip `payloadsource`).
Hard ordering: P0 -> P1 -> P2 -> rest. **Snapshots (P2) MUST precede any client
pointing at shopdb (P4).** P3 and P4 parallelize. Preinstall stays share-sourced
through at least Milestone 1 (no token pre-enrollment; export writes
`preinstall.json` too, so it is authored-in-shopdb for free with no client risk).
### The P1 parity harness (concrete, IT-re-runnable)
`plugins/geenforce/parity.py` + a CLI, also wrapped as a CI test. Two checks per
scope, output one readable line per scope (`entries N/N identical profiles M/M
same-fire PASS`), exit 0/1, prints the first differing entry/field on fail:
1. **Lossless field check (order-preserving).** Canonicalize each entry to
exactly the fields the engine reads (Name, Type, the payload fields, all
Detection*, the filter arrays, `_CmmVersion`, InUseCheck, preinstall flags);
exclude `_comment` and key order (documentation, not behavior). Compare the
ordered lists position by position.
2. **Same-entries-fire-in-same-order.** Re-implement in ~120 lines of Python the
engine's four filter functions exactly as written in `Install-FromManifest.ps1`
(`Test-PCTypeMatches` incl. the alias groups at lines 463-475, `"*"`, and
`<Type>-<SubType>`; `Test-HostnameMatches` exact + `-like`;
`Test-MachineNumberMatches`; `Test-CmmVersionMatches`). For each machine-
profile fixture, run BOTH manifests through it and assert the identical
ordered list of entry names that pass all filters. Detection itself is not
executed - check 1 already proved detection fields identical, so identical
inputs to detection are guaranteed. This pair proves losslessness without
byte-diffing.
Fixtures (`plugins/geenforce/parityfixtures.json`, ~16-18 profiles): one per
pctype; CMM version variants `2016/2019/2026`/empty; collections machine-number
variants (a credentialed bay, an MTConnect bay, neither); legacy-alias profiles
(`Standard`+`Machine`, `CMM`) to exercise the alias graph both ways; a `WJS-*`
hostname-wildcard profile; preinstall profiles including one that hits
`PCTypesStrict`. Watch-items the harness must handle: empty `Applications: []`
scopes (4 exist), entries with NO `DetectionMethod` (fire every run), and the
`regvalue` literal typing.
### First slice: one vertical through `gea-shopfloor-cmm`
Only 4 entries but hits every hard part - MSI type, Registry detection with and
without a pinned value, nested InUseCheck with Processes[], and the `_CmmVersion`
gate. Tables: scopes, entries, entrypctypes, inusechecks + processes,
publishedversions, pctypealiases. `flask geenforce import-share --scope
gea-shopfloor-cmm`; `flask geenforce publish gea-shopfloor-cmm`; one endpoint
`GET /api/geenforce/manifest?pctype=gea-shopfloor-cmm` serving the published
snapshot (fat-client, ETag, collector-style `X-API-Key`/PAT auth reusing
`shopdb/core/api/collector.py`). **Done =** parity PASS for cmm; the endpoint's
JSON fed to `Install-FromManifest.ps1` on a bench CMM PC logs `4 skipped`
identically to the share manifest; editing a draft does NOT change the served
bytes but publishing does, and rollback restores the prior published bytes;
unauth = 401, wrong-scope = 401.
### Milestone 1 (the recommended first stop)
End of P2 plus the publish/scope-list slice of P3: **manifests are authored and
published in shopdb, exported to the share by a button, and the engine,
dispatcher, share layout, payloads, and every PC are completely unchanged.**
That delivers the real pain relief - validated editing instead of hand-edited
JSON, version history, one-click rollback (republish + re-export), desired-state
data sitting next to collector data - at ZERO client risk, with a rollback any
IT tech already knows (restore the `_meta/history` backup file). Natural point to
write ADR-012 with real experience behind it. P4/P5 (HTTP fetch, cutover) are a
separately green-lit second milestone.
### Ranked risks / fail-fast
1. **Generated-JSON vs engine drift (fleet-wide mis-install).** Parity harness
first; CI re-proves parity against checked-in real manifests on every
exporter change; pin lib >= 2.6.
2. **Serving a half-finished draft.** Structural: client reads only
`iscurrent` snapshots; test asserts a draft edit leaves served bytes
unchanged. Must exist before P4.
3. **Availability coupling.** Last-known-good local cache in the first client
prototype; shadow test blocks shopdb and confirms enforce-from-cache + WARN.
4. **SYSTEM HTTP auth + TLS trust.** The ~20-line canary spike in P4 week 1,
before the real client change. Hours of cost; if it fails, Milestone 1 still
delivers full value.
5. **Alias-graph drift.** Seed pins a lib version; harness legacy-name profiles
fail loudly on divergence; new-lib runbook includes "update the alias seed".
6. **Preinstall has no pre-enrollment token.** Keep share-sourced through
Milestone 1/2; decide later.
7. **Editor scope creep.** Three shippable increments; buttons over drag; reuse
JSON preview.
### IT operability (day-to-day runbook, proving the design is manageable)
All in Settings > Imaging PC Types. No PowerShell, no SQL, no share edits.
- **Add an app to a PC type:** open the PC type, Add Entry, pick Type (fields
adapt), fill installer + detection + targeting, Move Up/Down to order, Preview
(+ simulator), Publish with a note. PCs pick it up next 5-min cycle.
- **Bump a version:** drop the new MSI in the scope's `apps/` on the share,
update the entry's Installer + Detection value, Preview, Publish.
- **Roll back a bad publish:** History -> pick last-good version -> Roll Back
(during Milestone 1 also click Export to Share).
- **Canary a risky change:** add the one test PC under Target Hostnames, Publish;
when happy, remove the filter and Publish again.
- **Check "did PC Y get app X":** the simulator with that PC's type/machine
number/CMM version shows exactly which entries apply and why others are filtered.
- **See revision history / who changed what:** the PC type's History tab lists
every published version (date, author, note) with a Roll Back on each; the
Audit Logs page shows the finer-grained draft edits (who touched which entry
field, when) between publishes.

View File

@@ -0,0 +1,208 @@
# Proposal: printedparts plugin (3D-printed parts storefront + kiosk)
Status: PROPOSED (also serves as the reference design for the plugin-development
lab in `docs/PLUGIN-LAB-PRINTEDPARTS.md`)
## 1. Problem
The 3D-printer engineers stock bins of printed parts (fixtures, clips, covers,
spacers). Anyone on the floor can take parts, so stock silently runs out and
nobody knows who took what or how fast items burn down. They need:
- a catalog ("storefront") of printable items: photo, description, quantity on
hand;
- a barcode label per item (1in x 0.5in) stuck on each bin;
- a touch-screen kiosk: scan the bin barcode, scan your badge, enter how many
you took, submit;
- restock and correction flows for the engineers;
- stock monitoring plus consumption metrics.
## 2. Shape: standalone model plugin, NOT an asset type
These are quantity-based consumables: one row represents a *kind* of part with
a count, not an individually tracked machine. ADR-001 assets are one-row-per-
physical-thing (a PC, a printer). So printedparts follows the
knowledgebase/usb shape - own tables, own blueprint, no AssetType row - and
does NOT join the asset-label TYPE_CONFIG; it ships its own print view the way
USB labels do.
Item identity: `itemcode`, generated `3DP-<zero-padded id>` (prefix
configurable via setting `printedparts_code_prefix`). Short, CODE128-friendly,
human-readable. This is what the bin label encodes.
## 3. Data model (2 tables, LOCKED naming, per-plugin Alembic)
### printeditems
| column | type | notes |
|---|---|---|
| printeditemid | int PK autoincrement | |
| itemcode | varchar(20) unique, indexed | generated on create |
| itemname | varchar(120) NOT NULL | |
| itemdescription | varchar(500) | brief description |
| imageurl | varchar(255) | served upload, models.py pattern |
| quantityonhand | int NOT NULL default 0 | cached; ledger is truth |
| lowstockthreshold | int NOT NULL default 5 | per-item, seeds from setting |
| binlocation | varchar(100) | where the bin lives |
| printnotes | mediumtext | material, print time, slicer file path |
| isactive | tinyint(1) | soft retire |
| createddate / modifieddate | datetime | AuditMixin/BaseModel |
### printeditemtransactions (the ledger - source of truth)
| column | type | notes |
|---|---|---|
| transactionid | int PK | |
| printeditemid | int FK -> printeditems CASCADE, indexed | |
| transactiontype | varchar(10) NOT NULL | take / restock / adjust |
| quantitychange | int NOT NULL | negative for take, signed for adjust |
| employeesso | varchar(20) NOT NULL, indexed | who (badge-resolved) |
| employeename | varchar(120) | resolved at write time (USB pattern) |
| reason | varchar(255) | required for adjust |
| transactiondate | datetime NOT NULL default naive-UTC, indexed | |
Invariants: `quantityonhand` = sum of `quantitychange` (enforced by writing
both in one session/commit; an `adjust` can never drive it below 0 - reject).
Every write records WHO via badge scan; there is no anonymous mutation.
Both tables registered in `PLUGIN_TABLE_OWNERS`
(`shopdb/plugins/alembic_template.py`); migration 0001 is a REAL baseline
(measuringtools pattern - hand-written `op.create_table`, no cross-schema FK
so `create_plugin_tables` would also work, but write the ops explicitly for
the exercise).
## 4. Badge resolution (reuse the USB contract exactly)
Same input shapes as `plugins/usb/api/routes.py`:
- all digits -> SSO;
- `0<digits>BZ` (case-insensitive) -> physical badge wrapping a PayNo;
- resolution to a display name via the employees plugin directory
(`DirectoryEmployee`, selfhosted mode) with graceful "" fallback.
Extract-or-copy decision for the lab: copy the small `_PAYNO_BADGE` regex +
lookup into the plugin (contract-pure, no cross-plugin import of usb).
Resolution happens SERVER-side on the kiosk endpoint - the kiosk client never
supplies a name, only the raw badge string.
Manifest `dependencies: ["employees"]` (name lookup). Badge that resolves to
no employee: configurable policy setting `printedparts_unknown_badge`
(`allow` = record SSO with empty name, `deny` = 422). Default deny.
## 5. API surface (blueprint at /api/printedparts)
Authenticated management (JWT + permission):
| route | method | permission |
|---|---|---|
| `/items` | GET list (search, paginate, lowstock filter) | open read (jwt optional) |
| `/items/<id>` | GET detail + recent transactions | open read |
| `/items` | POST create (mints itemcode) | printedparts.create |
| `/items/<id>` | PUT update | printedparts.edit |
| `/items/<id>` | DELETE soft-retire | printedparts.delete |
| `/items/<id>/image` | POST/DELETE upload/remove | printedparts.edit |
| `/image/<filename>` | GET serve | public (models.py pattern) |
| `/items/<id>/restock` | POST {quantity, badge} | printedparts.restock |
| `/items/<id>/adjust` | POST {quantitychange, reason, badge} | printedparts.restock |
| `/items/<id>/transactions` | GET history, ?format=csv | open read |
Kiosk (unauthenticated, notifications/employees open-endpoint precedent):
| route | method | body |
|---|---|---|
| `/kiosk/item/<itemcode>` | GET | item summary by scanned code |
| `/kiosk/take` | POST | {itemcode, badge, quantity} |
`/kiosk/take` validation: item exists + active; quantity 1..quantityonhand
(clamp/reject configurable? no - reject with clear message, kiosk shows it);
badge resolves per policy. Writes ledger row (negative) + decrements cached
quantity in one commit. Rate of abuse is low (plant floor), but the endpoint
only ever DECREMENTS stock with a recorded badge - it cannot edit the catalog.
Permissions declared via `get_permissions()`: printedparts.view/create/edit/
delete/restock (category `printedparts`), seeded on install/enable.
## 6. Frontend
Management pages (scaffold output, standard layout, master templates
PrintersList/PrinterDetail):
- `PrintedItemsList.vue` - table: image thumb, code, name, qty (red badge when
<= threshold), bin; filters: search, low-stock-only; row click -> detail.
- `PrintedItemDetail.vue` - hero image + fields, transaction history table,
restock/adjust buttons (modal w/ quantity + badge + reason).
- `PrintedItemForm.vue` - create/edit incl. image upload, threshold, bin.
- Router file `router/routes/printedparts.js`, list/detail plugin-gated only,
new/edit + requiresAuth (ADR-009, usb.js precedent).
- Nav via `get_navigation_items()` -> "3D Parts".
Kiosk (net-new, top-level route `/parts-kiosk`, NO requiresAuth, outside
AppLayout - shopfloor precedent):
- Full-screen, 3-step flow: (1) SCAN ITEM - a focused invisible input catches
the keyboard-wedge scan of the bin barcode, shows item card w/ photo + qty;
(2) SCAN BADGE - same wedge input pattern for the badge; (3) QUANTITY - big
touch keypad (0-9, clear, backspace - net-new component
`TouchKeypad.vue`) + TAKE button. Success screen w/ remaining count, auto
reset after a few seconds. All state client-side; one POST at the end.
- Scanner UX rule: keyboard-wedge scanners type the code + Enter. A hidden
always-focused input with @keydown.enter handles both scans; on-screen
prompt tells the user what to scan. Touch fallback: item search + manual
badge entry (small link, for damaged labels).
Labels (own print view, USBLabelBatch precedent):
- `/print/printedparts-labels` public print route.
- NEW physical size: 1in x 0.5in stock -> `@page { size: 1in 0.5in; margin: 0 }`
one label per page (label printers feed roll stock; per-page = per-label).
Layout: CODE128 barcode (JsBarcode, ~0.9in x 0.28in, displayValue false) +
itemcode text under it (~7pt) + optional item name truncated. QR variant
offered but barcode is default at this size (a 0.4in QR is at the edge of
scanner tolerance; CODE128 of `3DP-0042` is comfortable).
- Batch mode: pick items -> one label per page sequence for roll printers;
also a ULINE mini-grid fallback for sheet printers (reuse mini72 pattern).
## 7. Metrics / reports (get_reports hook)
- `printedparts-stock` - current stock levels w/ threshold flags (CSV).
- `printedparts-consumption` - takes per item over a date range (CSV).
- `printedparts-by-person` - takes grouped by employee (CSV).
- Dashboard widget via `get_dashboard_widgets()`: low-stock item count.
- Nice-to-have later: burn-rate (avg takes/week per item + weeks-to-empty
projection) - plain SQL over the ledger, add once basics work.
## 8. Settings (get_settings_cards, category printedparts)
| key | default | purpose |
|---|---|---|
| printedparts_code_prefix | 3DP | itemcode prefix |
| printedparts_default_threshold | 5 | seed for new items |
| printedparts_unknown_badge | deny | kiosk policy for unresolvable badges |
## 9. Manifest
name printedparts, version 0.1.0, api_prefix /api/printedparts,
core_version ">=0.12.0,<1.0.0", dependencies ["employees"],
default_enabled false (site opts in - USB precedent).
## 10. Explicitly out of scope (v1)
- Reservations/approvals, per-item cost, print-queue integration, multi-bin
per item, email low-stock alerts (the reports + dashboard widget cover
monitoring; alerting can ride the existing report-email endpoint later).
## 11. Risks / decisions taken
- Cached quantity vs ledger drift: single-commit writes + a reconcile query in
the stock report (flags items where cache != ledger sum).
### Decision: the kiosk take endpoint is an unauthenticated WRITE
This is the first open mutation in the product - every existing kiosk
endpoint (notifications, employees, shopfloor) is a read, and the closest
write (USB checkout) is JWT + permission gated. Accepted deliberately, on
these grounds, and any future open-write endpoint must meet the same bar:
1. Decrement-only: it can reduce stock of an active item, nothing else - no
catalog edits, no restocks, no reads it does not already expose.
2. Fully attributed: it refuses to act without a badge that resolves per the
site policy; every action lands in the ledger with SSO + name + time.
3. Bounded blast radius: worst case is stock counts driven low, which the
ledger makes visible and reversible (adjust with reason).
4. Physically rate-limited: it exists for a touch screen on the shop floor;
there is nothing to enumerate and nothing returned worth scraping.
- 1x0.5in QR marginal: default to CODE128 barcode.
- Not an Asset: no floor-map plotting or warranty for items. If a site later
wants bins on the floor map, revisit via get_map_overlays (ADR-010).

4
frontend/.gitignore vendored Normal file
View File

@@ -0,0 +1,4 @@
# ADR-013 Phase 4 staged plugin frontends (generated by scripts/stage-frontend.mjs)
src/.plugins-staged/
src/router/routes.gen.js

File diff suppressed because it is too large Load Diff

View File

@@ -1,12 +1,18 @@
{
"name": "shopdb-frontend",
"version": "0.5.0",
"version": "0.8.0",
"private": true,
"type": "module",
"scripts": {
"stage": "node ../scripts/stage-frontend.mjs",
"predev": "npm run stage",
"dev": "vite",
"prebuild": "npm run stage",
"build": "vite build",
"preview": "vite preview"
"preview": "vite preview",
"pretest": "npm run stage",
"test": "vitest run",
"test:watch": "vitest"
},
"dependencies": {
"@fontsource-variable/inter": "^5.2.8",
@@ -14,6 +20,7 @@
"@fullcalendar/daygrid": "^6.1.20",
"@fullcalendar/vue3": "^6.1.20",
"axios": "^1.6.0",
"dompurify": "^3.4.11",
"jsbarcode": "^3.12.3",
"jspdf": "^4.2.1",
"leaflet": "^1.9.4",
@@ -25,6 +32,9 @@
},
"devDependencies": {
"@vitejs/plugin-vue": "^5.2.4",
"vite": "^6.4.1"
"@vue/test-utils": "^2.4.6",
"jsdom": "^25.0.1",
"vite": "^6.4.1",
"vitest": "^2.1.9"
}
}

View File

@@ -1,7 +1,10 @@
import axios from 'axios'
import { withBase, stripBase } from './../utils/basePath'
// BASE_URL ends in '/', so this is '/api' at root or '/ops/api' under a subpath
// mount. Keeps the SPA, its API, and IIS all on the same mount path.
const api = axios.create({
baseURL: '/api',
baseURL: import.meta.env.BASE_URL + 'api',
headers: {
'Content-Type': 'application/json'
}
@@ -28,9 +31,17 @@ api.interceptors.response.use(
const hadToken = localStorage.getItem('token')
localStorage.removeItem('token')
localStorage.removeItem('user')
// Only redirect if user was previously logged in (session expired)
// Only redirect if user was previously logged in (session expired).
// Preserve the destination so login returns the user to this page.
if (hadToken) {
window.location.href = '/login'
const loginPath = withBase('/login')
// Router paths exclude the mount base; strip it or login's
// router.push double-prefixes under a subpath mount.
const here = stripBase(window.location.pathname) + window.location.search
const target = here && here !== '/login'
? loginPath + '?redirect=' + encodeURIComponent(here)
: loginPath
window.location.href = target
}
}
return Promise.reject(error)
@@ -55,6 +66,9 @@ export const authApi = {
return api.post('/auth/refresh', {}, {
headers: { Authorization: `Bearer ${refreshToken}` }
})
},
changePassword(payload) {
return api.post('/auth/change-password', payload)
}
}
@@ -106,6 +120,9 @@ export const computersApi = {
list(params = {}) {
return api.get('/computers', { params })
},
displayKiosks() {
return api.get('/computers/display-kiosks')
},
get(id) {
return api.get(`/computers/${id}`)
},
@@ -301,6 +318,10 @@ export const printersApi = {
dashboardSummary() {
return api.get('/printers/dashboard/summary')
},
// Flat network-printer list (with mapx/mapy) for the installer map.
installList() {
return api.get('/printers/install-list')
},
drivers: {
list(params = {}) {
return api.get('/printers/drivers', { params })
@@ -389,6 +410,15 @@ export const modelsApi = {
},
delete(id) {
return api.delete(`/models/${id}`)
},
uploadImage(id, file) {
// multipart photo upload; backend sets imageurl to the served URL
const form = new FormData()
form.append('file', file)
return api.post(`/models/${id}/image`, form, { headers: { 'Content-Type': 'multipart/form-data' } })
},
removeImage(id) {
return api.delete(`/models/${id}/image`)
}
}
@@ -470,20 +500,36 @@ export const applicationsApi = {
},
updateInstalledApp(machineId, appId, data) {
return api.put(`/applications/machines/${machineId}/${appId}`, data)
}
}
// Support Teams API (teams + nested contacts)
export const supportteamsApi = {
list(params = {}) {
return api.get('/supportteams', { params })
},
// Support teams
getSupportTeams() {
return api.get('/applications/supportteams')
get(id) {
return api.get(`/supportteams/${id}`)
},
createSupportTeam(data) {
return api.post('/applications/supportteams', data)
create(data) {
return api.post('/supportteams', data)
},
// App owners
getAppOwners() {
return api.get('/applications/appowners')
update(id, data) {
return api.put(`/supportteams/${id}`, data)
},
createAppOwner(data) {
return api.post('/applications/appowners', data)
remove(id) {
return api.delete(`/supportteams/${id}`)
},
contacts: {
add(teamId, data) {
return api.post(`/supportteams/${teamId}/contacts`, data)
},
update(teamId, contactId, data) {
return api.put(`/supportteams/${teamId}/contacts/${contactId}`, data)
},
remove(teamId, contactId) {
return api.delete(`/supportteams/${teamId}/contacts/${contactId}`)
}
}
}
@@ -684,6 +730,11 @@ export const reportsApi = {
},
pcRelationships(params = {}) {
return api.get('/reports/pc-relationships', { params })
},
// On-demand report delivery: email the given rows as an HTML table.
// Recipients default to the site Alert Recipients when `to` is omitted.
email(payload) {
return api.post('/reports/email', payload)
}
}
@@ -714,6 +765,15 @@ export const employeesApi = {
},
importCsv(csv) {
return api.post('/employees/directory/import', { csv })
},
// multipart photo upload; backend sets photofilename + returns photourl
uploadPhoto(sso, file) {
const form = new FormData()
form.append('file', file)
return api.post(`/employees/${sso}/photo`, form, { headers: { 'Content-Type': 'multipart/form-data' } })
},
removePhoto(sso) {
return api.delete(`/employees/${sso}/photo`)
}
}
}
@@ -783,6 +843,9 @@ export const settingsApi = {
update(key, value) {
return api.put(`/settings/${key}`, { value })
},
testEmail(to) {
return api.post('/settings/test-email', { to })
},
create(data) {
return api.post('/settings', data)
},
@@ -872,6 +935,24 @@ export const usersApi = {
}
}
// Personal API tokens: authenticate scripts/integrations as a user without
// the hourly-expiring login JWT. The secret is returned ONCE, on create.
export const apitokensApi = {
// all=true (admin) lists everyone's tokens; otherwise just the caller's.
list(params = {}) {
return api.get('/apitokens', { params })
},
create(data) {
return api.post('/apitokens', data)
},
update(id, data) {
return api.put(`/apitokens/${id}`, data)
},
remove(id) {
return api.delete(`/apitokens/${id}`)
}
}
// Network API (devices, subnets, and VLANs)
export const networkApi = {
// Network devices
@@ -1052,3 +1133,61 @@ export const measuringtoolsApi = {
}
}
}
// 3D printed parts (printedparts plugin)
export const printedpartsApi = {
list(params = {}) {
return api.get('/printedparts/items', { params })
},
get(printeditemid) {
return api.get(`/printedparts/items/${printeditemid}`)
},
create(data) {
return api.post('/printedparts/items', data)
},
update(printeditemid, data) {
return api.put(`/printedparts/items/${printeditemid}`, data)
},
remove(printeditemid) {
return api.delete(`/printedparts/items/${printeditemid}`)
},
restore(printeditemid) {
return api.post(`/printedparts/items/${printeditemid}/restore`)
},
uploadImage(printeditemid, file) {
const formData = new FormData()
formData.append('file', file)
return api.post(`/printedparts/items/${printeditemid}/image`, formData, {
headers: { 'Content-Type': 'multipart/form-data' }
})
},
deleteImage(printeditemid) {
return api.delete(`/printedparts/items/${printeditemid}/image`)
},
restock(printeditemid, data) {
return api.post(`/printedparts/items/${printeditemid}/restock`, data)
},
adjust(printeditemid, data) {
return api.post(`/printedparts/items/${printeditemid}/adjust`, data)
},
kioskItem(itemcode) {
return api.get(`/printedparts/kiosk/item/${encodeURIComponent(itemcode)}`)
},
kioskTake(data) {
return api.post('/printedparts/kiosk/take', data)
},
listFiles(printeditemid) {
return api.get(`/printedparts/items/${printeditemid}/files`)
},
uploadFile(printeditemid, file, note) {
const formData = new FormData()
formData.append('file', file)
if (note) formData.append('note', note)
return api.post(`/printedparts/items/${printeditemid}/files`, formData, {
headers: { 'Content-Type': 'multipart/form-data' }
})
},
removeFile(fileid) {
return api.delete(`/printedparts/files/${fileid}`)
}
}

View File

@@ -624,26 +624,30 @@ input[type="radio"] {
cursor: pointer;
}
/* Dark mode form adjustments */
@media (prefers-color-scheme: dark) {
.form-control {
background: var(--bg);
border-color: var(--border);
}
/* Dark mode form adjustments. Scoped to the explicit theme attribute (the
theme store always stamps it at startup) - a bare prefers-color-scheme
query here leaks dark widget styles into light mode on dark-OS machines. */
[data-theme="dark"] .form-control {
/* background-COLOR, not the shorthand: the shorthand resets a select's
background-repeat/position and the dropdown arrow tiles across the box. */
background-color: var(--bg);
border-color: var(--border);
}
.form-control:focus {
background: var(--bg);
box-shadow: 0 0 0 3px rgba(96, 165, 250, 0.2);
}
[data-theme="dark"] .form-control:focus {
background-color: var(--bg);
box-shadow: 0 0 0 3px rgba(96, 165, 250, 0.2);
}
select.form-control {
background-image: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='12' height='12' viewBox='0 0 12 12'%3E%3Cpath fill='%23999' d='M6 8L1 3h10z'/%3E%3C/svg%3E");
background-color: var(--bg);
}
[data-theme="dark"] select.form-control {
background-image: url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='12' height='12' viewBox='0 0 12 12'%3E%3Cpath fill='%23999' d='M6 8L1 3h10z'/%3E%3C/svg%3E");
background-color: var(--bg);
background-repeat: no-repeat;
background-position: right 0.75rem center;
}
select.form-control option {
background: var(--text);
}
[data-theme="dark"] select.form-control option {
background: var(--bg-card-solid);
}
/* Form grid */
@@ -941,8 +945,17 @@ input[type="radio"] {
}
td.actions {
/* A td must stay display:table-cell - the .actions inline-flex above pulls
the cell out of the table's row box, so its bottom border renders ~1px off
from the other cells. Keep it a cell; space multiple buttons with a margin
instead of the flex gap. */
display: table-cell;
vertical-align: middle;
white-space: nowrap;
}
td.actions .btn + .btn {
margin-left: 0.25rem;
}
/* ============================================
DETAIL PAGES (shared styles)
@@ -1051,17 +1064,19 @@ td.actions {
}
/* Content Grid */
/* Balanced two-column card flow. Uses CSS multicol (not a hand-assigned
grid) so cards distribute by height and the columns stay even no matter
how many cards land on either side. display:contents flattens the two
.content-column wrappers so their cards flow directly into the columns,
which keeps the existing markup unchanged. */
.content-grid {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 25px;
column-count: 2;
column-gap: 25px;
margin-bottom: 25px;
}
.content-column {
display: flex;
flex-direction: column;
gap: 25px;
display: contents;
}
/* Section Cards */
@@ -1070,6 +1085,10 @@ td.actions {
border-radius: 0.25rem;
padding: 1.25rem;
box-shadow: 0 2px 10px rgba(0, 0, 0, 0.1);
/* multicol needs per-card spacing (column-gap is horizontal only) and
must not split a card across the column break */
margin-bottom: 25px;
break-inside: avoid;
}
.section-title {
@@ -1083,6 +1102,212 @@ td.actions {
border-bottom: 1px solid var(--border);
}
/* Settings form controls - shared by the individual settings pages
(branding, printing, email, integrations, identifiers, search, map...). */
.setting-group {
border-top: 1px solid var(--border);
padding-top: 1rem;
}
.setting-group:first-of-type {
border-top: none;
padding-top: 0;
}
.setting-group h3 {
margin: 0 0 0.5rem 0;
font-size: 1rem;
color: var(--text);
}
.setting-description {
color: var(--text-light);
font-size: 0.9rem;
margin: 0 0 1rem 0;
line-height: 1.5;
}
.settings-grid {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(180px, 1fr));
gap: 1rem;
margin-bottom: 1rem;
}
.setting-row {
margin-bottom: 1rem;
}
.setting-row.full-width {
grid-column: 1 / -1;
}
.setting-row label {
display: flex;
flex-direction: column;
gap: 0.5rem;
}
.setting-row label span {
color: var(--text);
font-size: 0.9rem;
}
.setting-row input,
.setting-row select {
padding: 0.5rem;
border: 1px solid var(--border);
border-radius: 4px;
background: var(--bg);
color: var(--text);
font-size: 1rem;
max-width: 400px;
}
.setting-row input[type="number"] {
max-width: 120px;
}
.setting-row input:focus,
.setting-row select:focus {
outline: none;
border-color: var(--primary);
}
.input-hint {
color: var(--text-light);
font-size: 0.8rem;
}
.toggle-label {
flex-direction: row !important;
align-items: center;
justify-content: space-between;
max-width: 400px;
}
.toggle-hint {
display: block;
margin-top: -0.5rem;
margin-left: 0;
}
.toggle-btn {
position: relative;
width: 50px;
height: 26px;
border-radius: 13px;
border: none;
background: var(--secondary);
cursor: pointer;
transition: background 0.2s;
}
.toggle-btn.active {
background: var(--success);
}
.toggle-slider {
position: absolute;
top: 3px;
left: 3px;
width: 20px;
height: 20px;
border-radius: 50%;
background: white;
transition: transform 0.2s;
}
.toggle-btn.active .toggle-slider {
transform: translateX(24px);
}
.status-indicator {
display: flex;
align-items: center;
gap: 0.5rem;
padding: 0.75rem;
background: var(--bg);
border-radius: 4px;
font-size: 0.9rem;
color: var(--text-light);
max-width: 400px;
margin-bottom: 1rem;
}
.status-dot {
width: 10px;
height: 10px;
border-radius: 50%;
flex-shrink: 0;
}
.status-dot.inactive { background: var(--secondary); }
.status-dot.warning { background: var(--warning); }
.status-dot.pending { background: var(--primary); }
.status-dot.success { background: var(--success); }
.test-btn {
padding: 0.5rem 1rem;
background: var(--primary);
color: white;
border: none;
border-radius: 4px;
cursor: pointer;
font-size: 0.9rem;
}
.test-btn:hover:not(:disabled) {
background: var(--primary-dark);
}
.test-btn:disabled {
opacity: 0.5;
cursor: not-allowed;
}
.view-logs-link {
display: inline-block;
margin-top: 0.5rem;
color: var(--link);
text-decoration: none;
font-size: 0.9rem;
}
.view-logs-link:hover {
text-decoration: underline;
}
.identifier-matrix {
width: 100%;
border-collapse: collapse;
}
.identifier-matrix th,
.identifier-matrix td {
padding: 0.6rem 0.75rem;
text-align: center;
border-bottom: 1px solid var(--border);
}
.identifier-matrix th:first-child,
.identifier-matrix td.identifier-name {
text-align: left;
}
.identifier-matrix th {
color: var(--text-light);
font-weight: 600;
font-size: 0.9rem;
}
.identifier-matrix .identifier-name {
color: var(--text);
}
.color-input-row {
display: flex;
align-items: center;
gap: 0.75rem;
}
.color-input-row input[type="color"] {
width: 48px;
height: 34px;
padding: 2px;
cursor: pointer;
}
.color-input-row input[type="text"] {
max-width: 160px;
}
.map-upload-row {
display: flex;
align-items: center;
gap: 0.75rem;
margin-top: 0.4rem;
}
.map-thumb {
height: 40px;
border: 1px solid var(--border);
border-radius: 4px;
background: #fff;
}
.map-thumb-dark { background: #222; }
.settings-success {
margin-top: 1rem;
padding: 0.75rem;
background: var(--success);
color: white;
border-radius: 4px;
}
/* Info List */
.info-list {
display: flex;
@@ -1114,6 +1339,24 @@ td.actions {
font-size: 13px;
}
/* Small email/Teams action buttons for support contacts */
.contact-action {
display: inline-block;
margin-left: 0.35rem;
padding: 0.05rem 0.4rem;
font-size: 12px;
line-height: 1.5;
border: 1px solid var(--border);
border-radius: 4px;
color: var(--link);
background: var(--bg);
text-decoration: none;
}
.contact-action:hover {
border-color: var(--primary);
color: var(--primary);
}
/* Feature Tags */
.feature-tag {
display: inline-block;
@@ -1260,7 +1503,7 @@ td.actions {
}
.content-grid {
grid-template-columns: 1fr;
column-count: 1;
}
.audit-footer {
@@ -1487,16 +1730,14 @@ td.actions {
text-decoration: underline;
}
@media (prefers-color-scheme: dark) {
.notification-item.type-incident {
background: rgba(245, 54, 92, 0.1);
}
.notification-item.type-change {
background: rgba(255, 136, 0, 0.1);
}
.notification-item.type-awareness {
background: rgba(4, 185, 98, 0.1);
}
[data-theme="dark"] .notification-item.type-incident {
background: rgba(245, 54, 92, 0.1);
}
[data-theme="dark"] .notification-item.type-change {
background: rgba(255, 136, 0, 0.1);
}
[data-theme="dark"] .notification-item.type-awareness {
background: rgba(4, 185, 98, 0.1);
}
/* Light mode is now default, dark mode via prefers-color-scheme */
@@ -1512,3 +1753,9 @@ td.actions {
color: var(--text-light);
cursor: pointer;
}
/* Clickable list rows: the whole row navigates to the item detail; interactive
cells (the actions column, in-row links) stop propagation so they still work
independently. */
.clickable-row { cursor: pointer; }
.clickable-row:hover td { background: var(--bg); }

View File

@@ -18,30 +18,32 @@
</div>
<template v-else>
<!-- Outgoing relationships (this asset controls/connects to...) -->
<div v-if="outgoing.length > 0" class="relationship-group">
<h4 class="group-title">Outgoing</h4>
<!-- Symmetric connections: one direction-blind entry per peer -->
<div v-if="connectedItems.length > 0" class="relationship-group">
<h4 class="group-title">Connected</h4>
<div class="relationship-list">
<div
v-for="rel in outgoing"
:key="rel.relationshipid"
v-for="item in connectedItems"
:key="item.key"
class="relationship-item"
>
<div class="rel-icon"><component :is="getAssetIcon(rel.targetasset?.assettypename || rel.targetasset?.assettype)" :size="16" /></div>
<div class="rel-icon"><component :is="getAssetIcon(item.peer?.assettypename || item.peer?.assettype)" :size="16" /></div>
<div class="rel-content">
<router-link :to="getAssetRoute(rel.targetasset)" class="rel-name">
{{ rel.targetasset?.name || rel.targetasset?.assetnumber || 'Unknown' }}
</router-link>
<div class="rel-meta">
<span class="badge" :style="colorStyle(colorForType(rel.relationshiptypename))">{{ rel.relationshiptypename }}</span>
<span class="rel-type-badge">{{ rel.targetasset?.assettypename || rel.targetasset?.assettype }}</span>
<div class="rel-line">
<router-link :to="getAssetRoute(item.peer)" class="rel-name">
{{ item.peer?.name || item.peer?.assetnumber || 'Unknown' }}
</router-link>
<span class="badge" :style="colorStyle(colorForType(item.relationshiptypename))">{{ item.relationshiptypename }}</span>
</div>
<div v-if="rel.notes" class="rel-notes">{{ rel.notes }}</div>
<div class="rel-meta">
<span class="rel-type-badge">{{ item.peer?.assettypename || item.peer?.assettype }}</span>
</div>
<div v-if="item.notes" class="rel-notes">{{ item.notes }}</div>
</div>
<button
v-if="authStore.isAuthenticated"
class="btn-icon delete"
@click="deleteRelationship(rel.relationshipid)"
@click="deleteItem(item)"
title="Remove relationship"
>
&times;
@@ -50,30 +52,42 @@
</div>
</div>
<!-- Incoming relationships (...controls/connects to this asset) -->
<div v-if="incoming.length > 0" class="relationship-group">
<h4 class="group-title">Incoming</h4>
<!-- Directional relationships: natural per-row phrasing, no jargon -->
<div v-if="directionalItems.length > 0" class="relationship-group">
<div class="relationship-list">
<div
v-for="rel in incoming"
:key="rel.relationshipid"
v-for="item in directionalItems"
:key="item.key"
class="relationship-item"
>
<div class="rel-icon"><component :is="getAssetIcon(rel.sourceasset?.assettypename || rel.sourceasset?.assettype)" :size="16" /></div>
<div class="rel-icon"><component :is="getAssetIcon(item.peer?.assettypename || item.peer?.assettype)" :size="16" /></div>
<div class="rel-content">
<router-link :to="getAssetRoute(rel.sourceasset)" class="rel-name">
{{ rel.sourceasset?.name || rel.sourceasset?.assetnumber || 'Unknown' }}
</router-link>
<div class="rel-meta">
<span class="badge" :style="colorStyle(colorForType(rel.relationshiptypename))">{{ rel.relationshiptypename }}</span>
<span class="rel-type-badge">{{ rel.sourceasset?.assettypename || rel.sourceasset?.assettype }}</span>
<div class="rel-line">
<template v-if="item.direction === 'outgoing'">
<span class="badge" :style="colorStyle(colorForType(item.relationshiptypename))">{{ item.relationshiptypename }}</span>
<span class="rel-arrow">-&gt;</span>
<router-link :to="getAssetRoute(item.peer)" class="rel-name">
{{ item.peer?.name || item.peer?.assetnumber || 'Unknown' }}
</router-link>
</template>
<template v-else>
<span class="rel-arrow">&lt;-</span>
<span class="badge" :style="colorStyle(colorForType(item.relationshiptypename))">{{ item.relationshiptypename }}</span>
<span class="rel-from">from</span>
<router-link :to="getAssetRoute(item.peer)" class="rel-name">
{{ item.peer?.name || item.peer?.assetnumber || 'Unknown' }}
</router-link>
</template>
</div>
<div v-if="rel.notes" class="rel-notes">{{ rel.notes }}</div>
<div class="rel-meta">
<span class="rel-type-badge">{{ item.peer?.assettypename || item.peer?.assettype }}</span>
</div>
<div v-if="item.notes" class="rel-notes">{{ item.notes }}</div>
</div>
<button
v-if="authStore.isAuthenticated"
class="btn-icon delete"
@click="deleteRelationship(rel.relationshipid)"
@click="deleteItem(item)"
title="Remove relationship"
>
&times;
@@ -182,12 +196,12 @@ import { apiError } from '../utils/apiError'
const toast = useToast()
const props = defineProps({
assetId: {
assetid: {
type: Number,
default: null
},
// Alternative: lookup by machine/asset number
machineNumber: {
machinenumber: {
type: String,
default: null
}
@@ -225,6 +239,71 @@ let searchTimeout = null
const hasRelationships = computed(() => outgoing.value.length > 0 || incoming.value.length > 0)
// Symmetric types collapse to one entry per {peer, type} unordered pair. All
// stored direction rows (both directions, plus any legacy duplicates) fold
// into one displayed entry; its rowIds carries every collapsed relationshipid
// so a delete removes them all.
const connectedItems = computed(() => {
const byPair = new Map()
const rows = [
...outgoing.value.map(rel => ({ rel, peer: rel.targetasset })),
...incoming.value.map(rel => ({ rel, peer: rel.sourceasset })),
]
for (const { rel, peer } of rows) {
if (rel.isdirectional !== false) continue
const selfid = resolvedAssetId.value
const peerid = peer?.assetid
const lo = Math.min(selfid, peerid)
const hi = Math.max(selfid, peerid)
const key = `${rel.relationshiptypeid}:${lo}:${hi}`
const existing = byPair.get(key)
if (existing) {
existing.rowIds.push(rel.relationshipid)
if (!existing.notes && rel.notes) existing.notes = rel.notes
} else {
byPair.set(key, {
key,
rowIds: [rel.relationshipid],
peer,
relationshiptypeid: rel.relationshiptypeid,
relationshiptypename: rel.relationshiptypename,
notes: rel.notes || null,
})
}
}
return Array.from(byPair.values())
})
// Directional types keep one entry per stored row with arrow phrasing.
const directionalItems = computed(() => {
const items = []
for (const rel of outgoing.value) {
if (rel.isdirectional === false) continue
items.push({
key: `out-${rel.relationshipid}`,
rowIds: [rel.relationshipid],
peer: rel.targetasset,
direction: 'outgoing',
relationshiptypeid: rel.relationshiptypeid,
relationshiptypename: rel.relationshiptypename,
notes: rel.notes || null,
})
}
for (const rel of incoming.value) {
if (rel.isdirectional === false) continue
items.push({
key: `in-${rel.relationshipid}`,
rowIds: [rel.relationshipid],
peer: rel.sourceasset,
direction: 'incoming',
relationshiptypeid: rel.relationshiptypeid,
relationshiptypename: rel.relationshiptypename,
notes: rel.notes || null,
})
}
return items
})
const canSave = computed(() => {
return newRel.value.relationshiptypeid && newRel.value.targetAssetId && resolvedAssetId.value
})
@@ -236,14 +315,14 @@ onMounted(async () => {
}
})
watch(() => props.assetId, async () => {
watch(() => props.assetid, async () => {
await resolveAssetId()
if (resolvedAssetId.value) {
await loadRelationships()
}
})
watch(() => props.machineNumber, async () => {
watch(() => props.machinenumber, async () => {
await resolveAssetId()
if (resolvedAssetId.value) {
await loadRelationships()
@@ -251,21 +330,20 @@ watch(() => props.machineNumber, async () => {
})
async function resolveAssetId() {
// If assetId is provided directly, use it
if (props.assetId) {
resolvedAssetId.value = props.assetId
// If assetid is provided directly, use it
if (props.assetid) {
resolvedAssetId.value = props.assetid
lookupFailed.value = false
return
}
// Otherwise, try to look up by machine number
if (props.machineNumber) {
if (props.machinenumber) {
try {
const response = await assetsApi.lookup(props.machineNumber)
const response = await assetsApi.lookup(props.machinenumber)
resolvedAssetId.value = response.data.data?.assetid
lookupFailed.value = !resolvedAssetId.value
} catch (error) {
console.log('Asset lookup failed for:', props.machineNumber)
resolvedAssetId.value = null
lookupFailed.value = true
loading.value = false
@@ -356,11 +434,12 @@ async function saveRelationship() {
}
}
async function deleteRelationship(relationshipId) {
// Delete every collapsed direction row behind a displayed entry.
async function deleteItem(item) {
if (!confirm('Remove this relationship?')) return
try {
await assetsApi.deleteRelationship(relationshipId)
await Promise.all(item.rowIds.map(id => assetsApi.deleteRelationship(id)))
await loadRelationships()
emit('updated')
} catch (error) {
@@ -423,6 +502,11 @@ function getAssetRoute(asset) {
border-radius: 8px;
border: 1px solid var(--border);
padding: 1.25rem;
/* Match .section-card spacing so relationships is its own card with a gap
below (not visually merged with the Notes card) and does not split across
a multicol break. */
margin-bottom: 25px;
break-inside: avoid;
}
.section-header {
@@ -499,6 +583,24 @@ function getAssetRoute(asset) {
text-decoration: underline;
}
.rel-line {
display: flex;
align-items: center;
gap: 0.4rem;
flex-wrap: wrap;
}
.rel-arrow {
font-family: monospace;
font-weight: 600;
color: var(--text-light);
}
.rel-from {
font-size: 0.8rem;
color: var(--text-light);
}
.rel-meta {
display: flex;
align-items: center;

View File

@@ -0,0 +1,53 @@
<template>
<button class="btn btn-secondary" :disabled="sending" @click="emailReport">
{{ sending ? 'Sending...' : 'Email report' }}
</button>
</template>
<script setup>
import { ref } from 'vue'
import { reportsApi } from '../api'
import { useToast } from '../composables/toast'
import { apiError } from '../utils/apiError'
// On-demand report delivery. Emails the given rows as an HTML table to the
// site's Alert Recipients (or an explicit `to`). Automatic/scheduled sending is
// out of scope for this app; point an external cron at POST /api/reports/email
// with an API token to automate.
const props = defineProps({
subject: { type: String, required: true },
columns: { type: Array, required: true },
rows: { type: Array, required: true },
intro: { type: String, default: '' },
to: { type: String, default: '' },
})
const toast = useToast()
const sending = ref(false)
async function emailReport() {
sending.value = true
try {
const payload = {
subject: props.subject,
columns: props.columns,
rows: props.rows,
intro: props.intro,
}
if (props.to) payload.to = props.to
const response = await reportsApi.email(payload)
const result = response.data?.data || {}
if (result.sent) {
toast.success('Report emailed.')
} else if (result.error) {
toast.error(`Report email failed: ${result.error}`)
} else {
toast.info(response.data?.message || 'Email is not configured.')
}
} catch (event) {
toast.error(apiError(event, 'Failed to email report'))
} finally {
sending.value = false
}
}
</script>

Some files were not shown because too many files have changed in this diff Show More