Nine fixes from a review of the installer against its actual audience: DT leads
at sister sites who are not Windows, IIS or Python specialists and who will lean
on an AI assistant to get through it.
TRUTHFULNESS. The preflight was advisory - an operator read 'IIS is not
installed', pressed Next, answered five more pages and the install died partway
through with Python already on the box. The results page now blocks while
anything is failing, repaints on every run instead of latching after the first,
and offers 'Check again' so a fixed problem does not mean starting over. On
failure the wizard said 'Nothing was left running', which is false in every path
because the stages run with -OnFailure never: it now says the server is
part-configured, that re-running is safe, and how to remove it. The final page no
longer reads 'ShopDB-Flask is ready' after a failed install.
SECRETS. The generated MySQL root password went to Write-Host in a process the
wizard runs hidden - so nobody saw it - and stdout is forwarded into the setup
log operators are told to send to support, so it was permanently recorded for
everyone who did not need it. It now goes to an ACL'd file. Database dumps, which
contain every user password hash, landed in a ProgramData directory readable by
every user on the box; the directory is now locked at creation.
UPGRADES ON REMOTE-DATABASE SITES. mysqldump was looked for only under local
MySQL install paths, so a site whose database is on another host silently skipped
every pre-upgrade backup - after stage 2 had already stopped the pool and
replaced the tree. Find-MysqlTool now prefers a client shipped in the bundle,
stage 2 stages it onto the server, preflight reports when it is missing, and
mysqlclient\ is an optional locked payload.
UNINSTALL. A subpath install is an IIS Application, not a site; removing only the
site left the application pointing at a deleted directory, so the parent site -
at West Jefferson, the live classic ASP - served 503 on that path forever while
Add/Remove Programs reported success. Uninstall now reads MOUNT_PATH and removes
the application. The firewall rule was created as "$SiteName $SitePort" and
removed as the literal 'ShopDB-Flask 8090', which matches nothing.
DAY-2 TOOLING. Every shortcut now passes -AppRoot and -SitePort, and the console
forwards them through its own elevation and 32-bit relaunches instead of
discarding them - a non-default directory or port made it report a healthy site
as broken, from a shortcut the installer wrote. 'Open ShopDB-Flask' resolved to a
hardcoded localhost:8090 that was wrong for every subpath install; it now asks
the console, which reads the address the installer recorded, and no longer
demands administrator to open a browser.
SMOKE TEST. The parent-site port lookup filtered for an http binding and
defaulted to 80, so an https-only parent site failed a working install with a red
dialog.
DOCS AND /api/docs. The installer was invisible: nothing in docs/, README.md or
CLAUDE.md mentioned it, so a DT lead or their assistant landed on the manual IIS
runbook and hand-built the very server the installer then refuses to upgrade.
docs/INSTALL-WINDOWS.md and docs/OPERATE-WINDOWS.md are now the canonical route,
the two manual runbooks are bannered as reference-only, README and CLAUDE.md
route by target, and llms.txt tells an assistant which document to follow and to
ask for 'check -Json' before diagnosing. Both ship on the server, along with
openapi.json and llms.txt - without those the self-hosted /api/docs was broken on
every installed box, which matters most to the sites least able to debug it.
Stage 5 now checks it actually serves.
shopdb-admin.ps1 gains 'check -Json': one structured, secret-free block covering
version, publishing method, IIS state, HTTP reachability, database, Python
version, plugins and errors. That is the cheapest useful answer to 'the operator
will ask an LLM' - it works with no infrastructure, which a install-time MCP
server could not.
An air-gapped site cannot be scanned from anywhere else, so when a CVE lands the
only way to answer 'is that component here, and at what version' was to RDP in
and go looking. The frontend was the real blind spot: nothing recorded which
version of leaflet, dompurify, jspdf or html2canvas ends up inside the compiled
SPA.
scripts/generate_sbom.py emits CycloneDX 1.6 covering both ecosystems - every pin
in requirements.txt with the sha256 the installer enforces, and every package in
package-lock.json. Build-only npm packages are marked scope 'excluded' rather
than dropped, so 'not here' stays distinguishable from 'not looked for'.
Dependency edges are real: uv's '# via' comments give the Python graph and
package-lock gives the npm one.
Hand-rolled rather than cyclonedx-py plus cyclonedx-npm because both inputs are
already pinned and committed - this is a format translation, not a scan - and
because the build box may be a work PC with nothing but Python and Node. It is
deterministic by construction: same inputs, byte-identical output, so
regenerating does not churn.
Staged into the application tree by both builders, so it installs onto the
server with the app. shopdb-admin.ps1 verify reports it and searches it by
component name, which is the question actually being asked.
Packages appearing at several depths in package-lock (node_modules/vite and
node_modules/vitest/node_modules/vite) are merged, and a copy reachable outside
the dev tree makes the component count as shipped. Emitting both produced
duplicate bom-refs, which CycloneDX forbids and scanners reject; getting the dev
merge backwards would have hidden a shipped package from a CVE search.
Not covered by bundle-lock.json on purpose: its provenance is git, not the
third-party payload.
The lock records what IS in the wheelhouse, not what the application NEEDS, so
an incomplete wheelhouse was locked, blessed and shipped - and only failed on an
air-gapped server.
That is not hypothetical. Assembling the wheelhouse anywhere other than Windows
silently omits colorama, a win32-only dependency of click, because pip evaluates
environment markers against the machine doing the downloading rather than the
machine being targeted. The bundle built here was short exactly that one wheel.
Both verifiers now cross-check wheels/ against the staged requirements.txt,
ignoring markers, since a requirement guarded by sys_platform == 'win32' is
precisely the one that must be present. Names are normalised to PEP 427 wheel
form, so mysql-connector-python matches mysql_connector_python.
bundle-lock.json is the first real lock: 42 files, cp314/win_amd64 - 39 wheels,
Python 3.14.6, HttpPlatformHandler 1.2 and URL Rewrite. MySQL is absent and
optional; a site choosing the bundled-database option adds it and re-locks.
The naming gate now skips the installer's build output. It contains a staged
copy of the application plus a second SPA build under dist-subpath, which
--exclude-dir=dist does not match, so a staged bundle failed the gate on
vendored minified JS nobody in this repository wrote.
Two fixes to the lean-site build.
The closure resolution moves out of an inline heredoc into
scripts/resolve_plugin_closure.py. The Windows builder needs the same answer,
and a PowerShell reimplementation would have been a second copy of the rules,
free to drift and produce a bundle whose plugin set did not match its profile.
The backend staging step copied all of deploy/ into the output tree. The Windows
installer stages its bundle at deploy/windows/installer/bundle, so that copy
recursed into its own destination and cp aborted with 'cannot copy a directory
into itself' - the documented build could not complete. Only
deploy/windows/web.config is read at install time, so only that is staged; the
rest of deploy/ is installer source and does not belong on an application
server.
Vite compiles the mount path into the bundle, so it cannot be chosen at install
time from a single build - a page served under /shopdb would load and then
request its assets from /assets/, and render nothing.
build-site.sh now produces both:
frontend-dist base / - the app on its own IIS site
frontend-dist-subpath base /<alias> - an IIS Application under an existing
site, e.g. http://<server-fqdn>/shopdb/
SUBPATH_ALIAS (default 'shopdb') is fixed per bundle and written into the staged
build as .alias, so the three places that must agree - the IIS application alias,
MOUNT_PATH in .env, and this compiled base - cannot drift apart. The installer
checks that marker and refuses rather than serving a page that cannot load.
The subpath build runs FIRST and is held in a temp dir: the root build has to be
last so frontend/dist is left in the state a developer expects, and the copy into
$OUT has to happen after the staging step that does rm -rf "$OUT".
build-site.sh staged only shopdb/, the chosen plugins/ and frontend-dist, so the
output could be imported but not run or migrated. The Windows installer had to
assemble wsgi.py, requirements.txt, migrations/ and deploy/ separately, which
meant it could assemble a payload whose plugin set did not match the profile the
tree was staged from.
Stage those runtime files, and copy the profile in as site-profile.json so the
set is self-describing: `flask plugin apply-profile` at provisioning reads the
same profile the tree was built from, so installed plugins and shipped plugin
code cannot drift.
frontend-dist keeps its name; CI reads that path (ci.yml:79).
The closing hint now spells out `prune-schema --yes --force`. ADR-014's prose
says lean provisioning "uses --force", but --force alone only permits dropping
non-empty tables; without --yes the command is a dry run that prints a preview
and exits, so following the ADR literally silently skips the prune.
REQ-D: restore waitress and tzdata to requirements.in. They existed ONLY in the
generated requirements.txt (hand-added in bf9e60e), so the next
`uv pip compile` would have silently removed the WSGI server and the IANA
timezone database from every Windows install.
REQ-E: split production and development requirements. requirements.txt was
installing pytest, pytest-cov, pytest-flask, coverage, iniconfig and pluggy onto
production servers. Verified on a real Windows Server box before this change.
CI, scripts/test-external-plugin.sh and the dev docs now use requirements-dev.txt.
REQ-F: standardise on Python 3.14. The repo declared four different versions
(Dockerfile 3.12, DEPLOY-WINDOWS-IIS 3.12, INSTALL-WINDOWS-IIS 3.13, CI 3.13,
plus README, web.config and PLUGIN-EXTERNAL-REPO). 3.14 is in active bugfix
support until ~Apr 2027 and supported to Oct 2030; 3.13 entered security-only in
Apr 2026. All four compiled dependencies publish win_amd64 wheels for 3.14
(cryptography via an abi3 wheel), verified by building an offline wheelhouse and
installing it on Windows Server 2025.
REQ-G: state MySQL 8.0 as the standard for new installs; 5.7+/5.6 remain
supported on an existing server.
Lockfiles regenerated with uv pip compile. Production deps 44 -> 38.
Generate docs/openapi.json (3.1, 362 operations) from the API inventory via
scripts/gen_openapi.py, and serve it with a self-hosted Redoc bundle at
/api/docs - no CDN, works on the air-gapped box. Also serve docs/llms.txt (a
concise LLM entrypoint) at /api/docs/llms.txt. New core 'docs' blueprint;
staticdocs/ excluded from the naming check (vendored minified JS).
Servers were imported as computers (a PC type) so they show under PCs, not
Network. This one-shot re-points each server's asset in place - assetid is
unchanged, so comms/relationships/map/name/location/audit all carry over; only
the extension row is swapped (computers -> networkdevices), the asset type is
flipped, and the device gets the 'Server' networkdevicetype (created if absent).
Identify servers by their computer type name (--type, default 'Server'). Dry-run
by default; --commit applies. Run on the target instance.
Air-gapped sites cannot pip install / npm ci / docker pull, so a build-at-site
compose (build: .) fails and reports 'service api is not running'. Add a
build-once-ship-image path:
- scripts/build-offline-bundle.ps1: on a connected box, build shopdb-flask +
pull mysql:8.0, docker save both into one gzipped tarball with a sha256.
- docker-compose.airgap.yml: runs pre-loaded images (image:, never build:),
drops the ./plugins bind mount (which would mask the image's baked-in plugins
with an empty host dir and load zero plugins at an image-only site), and adds
a one-shot migrate service (db upgrade + plugin upgrade-all + seed) that api
waits on via service_completed_successfully, so 'up -d' brings a working site.
- docs/DEPLOY-AIRGAP.md: full runbook (build, transfer+verify, load+run, admin,
verify, upgrade, troubleshooting), incl the Zscaler in-build cert caveat.
- .env.example: IMAGE_TAG for the air-gap compose to pin the loaded image tag.
The org IP allow list blocks GitHub-hosted runner IPs (checkout 403), so
point all jobs at the self-hosted arc-runner-set. Drop the rsync dependency
in build-site.sh (cp + bytecode prune; the ARC runner image has no rsync)
and remove the migrations-mysql job - ARC/Kubernetes has no service
containers, so that MySQL 8 coverage stays on the internal CI.
A frontend dir under plugins/ with no manifest.json is a CORE feature, not a
per-site plugin - applications is one (backend is shopdb/core/api/applications.py,
nav is advertised as core in dashboard.py). stage-frontend.mjs treated it like
a plugin and dropped it under SITE_PLUGINS, so a lean site showed the core
Applications nav item but had no route for it -> blank page. Now manifest-less
frontends always stage regardless of SITE_PLUGINS; SITE_PLUGINS selection applies
only to real plugins. CI lean-build job asserts ApplicationsList ships in a lean
bundle. Found while testing a live machines+printers lean site.
The lean-build endgame: a site ships carrying only the plugins it chose.
- scripts/build-site.sh: reads a site profile, resolves the hard-dependency
closure from manifests, builds the frontend with SITE_PLUGINS (stage-frontend
carries only those plugins), and stages a backend tree of core + only the
chosen plugin dirs. An unchosen plugin is in neither the bundle nor the tree.
- Core lazy-import guard: `flask seed demo` hard-imported the 5 asset subtype
models, which would crash a lean build missing any of those plugins. Now
guarded (a missing model skips its demo section).
- test_lean_build_guards.py: statically asserts NO core (shopdb/core, shopdb/cli)
import of a plugin is unguarded - a lean build omitting that plugin would
otherwise crash. 0 unguarded today.
Pilot verified: a lean build (machines + printers) carries only machines +
printers code - PartsKiosk / ManifestEditor / USBLabelBatch / KnowledgeBaseDetail
/ EmployeeDirectory are absent from the bundle, and only machines/printers plugin
dirs stage into the backend. (Sidebar labels for absent plugins remain - the
accepted small plugin-aware core remainder.) Guard test + naming green.
The naming/style check now fails a plugin frontend (plugins/<name>/frontend/)
that imports with an escaping ../../ or another plugin's path. Plugin frontends
must reach core only through the @/ alias and otherwise import only their own
tree, so a per-site build can drop a plugin cleanly. All 14 plugin frontends
pass.
Relocate applications, geenforce, knowledgebase, and machines - each owns only
its own views dir, so a clean move to plugins/<name>/frontend/ (views/ +
routes.js, core imports rewritten to @/). geenforce's entryForm.js helper + its
vitest spec move with it (ManifestEditor imports it as a sibling).
Machinery fixes this batch surfaced:
- routes.gen.js codegen uses namespace imports (import * as p_x). A route file
without a `toplevel` export is undefined on the namespace instead of a strict-
ESM missing-binding build error.
- vitest gains a `pretest` stage so plugin-frontend specs (now under
plugins/<name>/frontend/) run from their staged copy in src/.plugins-staged/.
Verified live: GE-Enforce (the most complex, uses the entryForm sibling helper)
renders fully from its staged frontend. Build + 58 vitest + naming green.
The staging step that makes lean per-site frontend builds possible, plus the
first plugin relocated as the pilot.
- scripts/stage-frontend.mjs: copies each chosen plugin's plugins/<name>/frontend/
into frontend/src/.plugins-staged/<name>/ and codegens routes.gen.js. Plugin
selection via SITE_PLUGINS (comma-separated); empty = all plugins that have a
frontend/ (the full build). Wired as npm predev/prebuild; outputs gitignored.
- Router imports routes.gen.js and merges staged routes with the in-tree
./routes/*.js glob - dual-location during the transition.
- printedparts relocated: its 6 views (list/detail/form/kiosk + the settings and
labels views from the shared dirs) moved into plugins/printedparts/frontend/
views/, core imports rewritten to the @/ alias; routes.js is the self-contained
route module. Its old in-tree route file is removed.
Also fixes a crash the previous commit (37c764b) shipped: slides.js exports only
`toplevel` (its child routes live in core.js), so the router's
flatMap(m => m.default) produced an undefined child and threw
"Cannot read properties of undefined (reading 'path')" at load - the whole SPA
went blank. Guarded with `m.default || []`. (The earlier "print pages are blank"
reading was this crash, not page nature.)
Verified live: /machines renders again; the relocated /printedparts list renders
identically from the staged plugin frontend; SITE_PLUGINS=machines excludes
printedparts from routes.gen. Build (via npm, runs stage) + vitest + naming green.
Loader: bool() on pymysql bit(1) bytes is always true - isinstallable
and isshopfloor imported as 1 for every row; route through _truthy_bit.
The employee source DB is now optional (shopdb-only imports).
Frontend: under a subpath mount the 401 interceptor stored the browser
path (mount base included) as the login redirect and the router applied
its base again (/ops/ops). New stripBase() keeps redirects base-free.
tools/export-github.sh automates the publication flow: prune + scrub +
commit into ~/projects/shopdb-flask-pub and emit a transfer bundle.
Import-surface docs and docstrings describe the automation as a
migration script; status-doc references in CHANGELOG/ADR-009/ROADMAP
point at repo files. Screenshot/verify tools write to /tmp/shopdb-shots
(created on import) instead of a machine-specific directory.
Classic ASP keeps retired machines in the machines table as history (isactive=0);
every other stage already filters isactive=1, but the assets hub, metrology,
locations and the verify count read machines unfiltered, so ~240 retired units
(incl. all G-prefix hostnames and 61 dead metrology PCs that each synthesized a
phantom measuring tool) landed as live assets. Add the isactive=1 filter to
those four queries. Downstream stages resolve via the id crosswalk, so warranties
/comms/relationships/installs for retired machines now drop automatically.
A freshly-imported DB left setup_complete unset, so the admin was bounced into
the first-run wizard even though the instance is fully populated. The harness now
sets setup_complete=true (it already mints the admin), so an imported instance
goes straight to the app.
Routing by pctype wrongly swept ~105 metrology PCs (CMM/Genspect/Keyence/Wax)
into measuringtools - but a PC that drives an instrument is still a computer;
the physical CMM/gauge is the tool. Dropped the pctype override: those PCs now
import as computers (measuringtools drops to the 48 real instrument rows).
Classic has no separate tool row for a metrology PC, so they'd be orphaned. New
metrology stage synthesizes a measuring-tool asset per metrology PC (typed by
its pctype: CMM / Form Tracer / Vision System / Genspect) and a Controls
relationship PC -> tool, mirroring what the runtime collector does.
Result: computers 663->751, measuringtools = 48 real + 88 synthesized (each
linked to its controlling PC), 88 new Controls relationships, no orphans.
The applications stage dropped applicationlink and documentationpath, so the app
detail's "Launch Application" and "Documentation" links were always empty. Map
both from the classic applications table.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Notifications imported with only employeesso, leaving employeename null - the
model displays "employeename or employeesso", so recognition/training cards
showed a bare SSO instead of a name. Build an SSO -> "First Last" map from the
employee source and populate employeename (comma-separated SSOs -> joined
names). SSOs not in the directory (former/non-WJF) stay null and fall back to
the SSO, as before.
Verified on the import DB: 261 notifications re-imported, names resolved
(Brandon Saltz, Jon Kolkmann, ...).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Follow-up to the mapper retirement (the new docs missed the prior commit's
staging). Adds docs/IMPORT-ADOPTION.md (two-layer import story + stage/crosswalk
guidance), scripts/migration/README.md (dir superseded, points at the API +
loader), and updates the WJ loader README to complete status.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Cleanup after the reference loader (scripts/site_imports/wjf/) proved out.
Removed the drifted direct-SQL migrators - migrate_assets/communications/
notifications/usb.py, run_migration.py, verify_migration.py, and
scripts/import_from_mysql.py. They targeted a nonexistent equipment table, the
retired Machine model, and columns that no longer exist; nothing imported them.
scripts/migration/README.md now points at the import API + the site loader.
Kept the one-time SQL fixups (fix_legacy_schema.sql, one-offs/).
Added docs/IMPORT-ADOPTION.md: the two-layer import story (stable IMPORT-API
contract + per-site loader), stage-ordering + crosswalk guidance, the
agent-assisted mapping path, and what the WJ loader demonstrates. Updated the
loader README to complete status (all 15 stages, final counts).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
printers: printers come from the printers TABLE (not the machines hub), so a
dedicated stage - assetnumber synthesized PRN-{printerid}, IP folded via the
create route, host machineid resolved to a location when it is a LocationOnly
row. Skips inactive. 50 of 56 imported.
relationships: a "Controlled By" edge now flips to the forward Controls
direction (source/target swapped, mapped to the Controls type) instead of
importing a redundant inverse type.
Final fresh full run: 983 assets (933 machines-hub + 50 printers), zero endpoint
errors, all 15 stages.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Completes the loader end to end. Verified on a fresh scratch target, zero
endpoint errors:
- locations: the 24 islocationonly rows -> core Locations (crosswalk machineid
-> locationid).
- relationships: 93 active edges imported (206 of 299 dropped because an
endpoint became a Location / was skipped / dedup-lost); types folded onto the
seeded canonical set; dedup on (source,target,type).
- subnets: 37 (full CIDR reconstructed as INET_NTOA(ipstart)+suffix; VLANs
lookup-or-create by number; 3 duplicate CIDRs first-wins-skipped).
- usb: 18 cmmc devices + 232 check-in/out events, paired with per-device open
state so unpaired log rows do not 400. Needs the usb plugin enabled + usb
directory mode selfhosted.
- verify: source-vs-target row-count audit (assets 1167->933 by the skip rules,
applications 121=121, employees 415=415, KB 342->341).
Full pipeline default runs all 14 stages in order. NOTE: the import target needs
every bundled plugin enabled (usb ships disabled in this dev registry - enable
it before importing).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Of the 158 machinetypeid=1 rows, only 24 carry the islocationonly bit (real
named areas: DT Office, IT Closet, Materials, ...). The other 134 are active,
modelled shop machines just left untyped - routing all 158 to Locations dropped
those 134 real assets. Route on the bit instead; the 134 untyped rows import as
machines with a null subtype (machinetypeid=1 is not a real machine subtype, so
catalog skips seeding one).
Also process asset routes in richness order (computer > measuringtool > network
> machine) so on a duplicate machinenumber the PC - which carries installs + IP
a bare untyped machine does not - wins first-come.
Result on the scratch target: 933 assets (computer 663, machine 76, network 58,
measuringtool 136), 24 locations (was mis-routing 158), installs 850 (was 653 -
PCs no longer lose their numbers to bare machines), warranties 464, comms 461.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All consume the machineid->assetid crosswalk from the assets hub. Verified
against the scratch target, zero endpoint errors:
- communications: 435 primary IPs folded onto assets. No bulk endpoint exists,
so this is the plan's documented direct-ORM gap (reads the source
communications table where comstypeid=1 AND isprimary=1, not machines.ipaddress1
which is empty).
- applications: supportteams 45, applications 121 (colliding names dedup via the
unique-appname 409-resolve), appversions 47, installs 653 (machineid ->
assetid -> computerid; only computer assets take installs).
- warranties: 424 linked, vendor hardcoded Dell (source has none).
- notifications: types 6, notifications 261 (2099 sentinel endtime clamped).
- knowledgebase: 341 (appid resolved through the applications name map).
Inactive rows skipped everywhere per the decisions. Remaining loader stages:
locations (the 158 LocationOnly rows), relationships (301 active edges),
subnets/VLANs, usb (cmmc pairing), verify.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the keystone stages to the reference loader.
catalog: seeds modeltypes (from classic machinetypes + category), the per-plugin
asset subtypes routed by machinetype (machines/network/measuringtool types),
computer subtypes (from pctype), the 5-row controllertypes vendor/model split,
and the models catalog - each with a persisted legacy->new crosswalk. Verified:
modeltypes 31, computertypes 12, models 118.
assets (the hub): fans classic machines out to the right endpoint by
machinetypeid (+ the pctype metrology override), applying the resolved
decisions - assetnumber = machinenumber else hostname, skip 9999, skip duplicate
machinenumbers, LocationOnly/printer/USB routed out. Persists the machineid ->
assetid crosswalk every downstream stage needs. Verified against a fresh scratch
target: 884 assets (computer 623, machine 68, network 58, measuringtool 135),
zero endpoint errors, idempotent re-run (stays 884). Skips: location 158, dup 69,
other 53, 9999 1.
Harness now runs each plugin's idempotent on_install so the AssetType rows exist
(a DB built with plugin upgrade-all instead of a fresh install lacks them, and
the create routes 500 without them). 409-resolve lookups page through per_page.
Remaining stages: communications (primary IP fold - source is the communications
table, not machines.ipaddress1), applications/installs, warranties, notifications,
KB, subnets/VLANs, usb, verify.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Layer 2 of the import design (see the memory + scratchpad/IMPORT-PLAN.md): a
SITE-SPECIFIC reference loader that maps WJ's classic-ASP schema onto the
maintained, schema-agnostic IMPORT-API contract. Other sites copy the pattern
against their own source DB; nobody runs this loader as-is.
Harness (scripts/site_imports/wjf/harness.py): builds the app against the
current DATABASE_URL (point it at a throwaway import DB), mints an unscoped
admin PAT in-process, and drives the real import endpoints through the app test
client with Authorization: Bearer + X-Import-Mode - exercising the same
routes/authz/validation an HTTP client would, no running server needed.
Read-only pymysql access to the three scratch source DBs; legacy-id -> new-id
crosswalks persist to JSON so a crashed run resumes and later stages resolve FKs.
Stages implemented + verified idempotent against a fresh scratch target
(shopdb_flask_import): reference (vendors 46, businessunits 13, operatingsystems
11) and employees (directory 415, re-run updated-not-duplicated). Remaining
stages (models, applications, assets hub + crosswalk, dependents, network, usb,
verify) are stubbed with the same shape; README documents the adoption playbook.
idmap.json is generated state (gitignored).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Path.rglob does not descend symlinks, so a symlinked external plugin
(the ADR-003 dev loop) silently escaped the contract-purity scan. The
scanner now resolves plugin dirs before walking, a regression test
plants a symlinked plugin with a real violation and asserts it is
flagged, and the known-limitation notes in the external-repo docs are
lifted.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- flask plugin new now scaffolds the frontend too: List/Detail/Form
views on the global styles, a gated route module (ADR-009), and an
api-client snippet emitted into the plugin dir. Views are written
before the route file so a partially generated plugin cannot 500 the
dev server.
- docs/PLUGIN-EXTERNAL-REPO.md + scripts/test-external-plugin.sh: how a
sister site develops a plugin in its own repo and runs the framework
contract tests in CI against a pinned framework ref (script verified
to fail on a broken core_version pin).
- docs/CONTRACT-STABILITY.md: settled vs churning contract surface and
the provisional 1.0 criteria.
- CLAUDE.md active-state refresh (contract 0.6.0, 11 plugins, 340
tests, measuringtools done).
Known limitation documented: Path.rglob does not descend symlinks, so
the import-surface contract test skips symlinked external plugins.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Make the app distributable to other GE Aerospace sites (one self-hosted
instance per site, ADR-004). GE values remain the shipped defaults; every
site-specific behavior is now a Setting an admin can change in the UI.
Settings-driven site config:
- Branding: site/QR/badge logos, favicon, primary color (upload endpoints
mirror the map-blueprint pattern; new Settings > Branding section).
- ServiceNow: search/incident/change URL templates ({ticket}), ticket
prefixes, enable toggle. Defaults point at the current
geaerospaceqa.service-now.com global search. Disabled = plain-text tickets.
- Employee-id regex (employeeid_pattern), printer hostname template,
QR label targets (qr_target_printer / qr_target_usb, blank = asset page,
else URL template with placeholders), usb_label_style (barcode|qr).
- West Jefferson floor-plan PNGs removed from the tree; generic placeholder
ships as the map default and sites upload their own blueprint.
Security closeout:
- dashboarddefaults writes now require admin.
- Collector: generic error messages (no str(exc) leak); API key accepted
via X-API-Key header only (BREAKING: querystring api_key removed).
- IP-based login rate limiting (AUTH_RATELIMIT_* knobs) atop account lockout.
- Setting.set() creation race fixed (IntegrityError retry).
Release engineering and docs:
- __version__ 0.5.0 (distinct from __contract_version__, ADR-007),
CHANGELOG.md, Gitea Actions CI config, frontend version aligned.
- One wizard-first install story across README/DEPLOY; new CONFIG.md,
UPGRADE.md, BACKUP-RESTORE.md; CLAUDE.md and ROADMAP de-staled.
- Dockerfile multi-stage build now bundles the frontend; compose binds
MySQL to 127.0.0.1; stale database/schema.sql and one-off SQL removed.
Debt and fixes:
- .query.get() -> db.session.get() sweep; datetime.utcnow() removed
(naive-UTC via timezone-aware now); users.py on authz decorators.
- Fixed 4 stale tests (slides feed shape, shopfloor splitperemployee,
plugin contract purity) and the USB label page field mapping (both usb
modes emit the cmmc shape: device_id/device_desc).
- Health endpoint reports the real version.
248 tests pass; naming/style check green; frontend builds; fresh-DB
flask db upgrade + seeds verified; QR targets verified by decoding
rendered codes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Lock the position-resolution columns from ADR-001 in code so
resolve_asset_position's relationship walk activates.
Schema
- Asset.mapleft -> Asset.mapx, Asset.maptop -> Asset.mapy
- Location.mapx / Location.mapy added (fallback for priority 3 of the
ADR-001 resolution chain)
- AssetRelationship.label (free-text nuance per ADR-001)
- AssetRelationship.inheritsposition (bool, server_default true, controls
whether the resolved-position walk follows the edge)
- RelationshipType.propagatesthroughid (self-FK; sibling-propagation rail)
Seeds
- Three canonical ADR-001 relationship types created idempotently:
partof, controls, connectedto
- controls.propagatesthroughid wired to partof (partof + connectedto stay
null per ADR-001 table). Both via Alembic migration AND CLI seed command
so a fresh test fixture and a sister-site deploy both end up correct.
- Legacy connection types (Serial Cable, Direct Ethernet, USB, WiFi,
Dualpath) retained for backward compat with pre-1.0 relationship rows.
Resolver
- shopdb.api.resolve_asset_position now walks inheritsposition=true edges
of type partof (then controls), recursively, depth-capped at 3 with
visited-set cycle protection. Inactive edges + non-inheritable types
are skipped. Falls through to the existing location fallback when the
walk yields nothing.
Tests
- 11 new test_api_namespace cases cover: partof walk, controls-after-
partof ordering, connectedto skipped, inheritsposition=false skipped,
recursion, cycle break, depth-3 cap, self-beats-related, related-beats-
location, inactive-edge skip.
- 111 tests pass. Naming/style check green.
Migration
- migrations/versions/7a01_adr001_position_contract.py:
- alter_column renames on assets (no data loss)
- add_column on locations + relationshiptypes + assetrelationships
- idempotent seed of three ADR types + propagation FK wire-up
- downgrade reverses + best-effort deletion of seeded types that have
no FK refs
Backend rename (mapleft/maptop -> mapx/mapy)
- shopdb/core/api/assets.py
- plugins/{computers,equipment,network,printers}/api/...
- scripts/migration/migrate_assets.py
- Legacy Machine model + machines API + import_from_mysql.py UNCHANGED
(per ADR-001 Machine retires; not part of the asset contract)
Frontend rename
- frontend/src/components/ShopFloorMap.vue
- frontend/src/views/{MapEditor.vue, pcs/{PCDetail,PCForm}.vue,
printers/{PrinterDetail,PrinterForm}.vue,
machines/{MachineDetail,MachineForm}.vue,
network/NetworkDeviceForm.vue}
- Form field labels + v-model bindings + computed flags switched in
lockstep with the backend.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New Plugins:
- USB plugin: Device checkout/checkin with employee lookup, checkout history
- Notifications plugin: Announcements with types, scheduling, shopfloor display
- Network plugin: Network device management with subnets and VLANs
- Equipment and Computers plugins: Asset type separation
Frontend:
- EmployeeSearch component: Reusable employee lookup with autocomplete
- USB views: List, detail, checkout/checkin modals
- Notifications views: List, form with recognition mode
- Network views: Device list, detail, form
- Calendar view with FullCalendar integration
- Shopfloor and TV dashboard views
- Reports index page
- Map editor for asset positioning
- Light/dark mode fixes for map tooltips
Backend:
- Employee search API with external lookup service
- Collector API for PowerShell data collection
- Reports API endpoints
- Slides API for TV dashboard
- Fixed AppVersion model (removed BaseModel inheritance)
- Added checkout_name column to usbcheckouts table
Styling:
- Unified detail page styles
- Improved pagination (page numbers instead of prev/next)
- Dark/light mode theme improvements
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Flask backend with Vue 3 frontend for shop floor machine management.
Includes database schema export for MySQL shopdb_flask database.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
