build-site.sh staged only shopdb/, the chosen plugins/ and frontend-dist, so the
output could be imported but not run or migrated. The Windows installer had to
assemble wsgi.py, requirements.txt, migrations/ and deploy/ separately, which
meant it could assemble a payload whose plugin set did not match the profile the
tree was staged from.
Stage those runtime files, and copy the profile in as site-profile.json so the
set is self-describing: `flask plugin apply-profile` at provisioning reads the
same profile the tree was built from, so installed plugins and shipped plugin
code cannot drift.
frontend-dist keeps its name; CI reads that path (ci.yml:79).
The closing hint now spells out `prune-schema --yes --force`. ADR-014's prose
says lean provisioning "uses --force", but --force alone only permits dropping
non-empty tables; without --yes the command is a dry run that prints a preview
and exits, so following the ADR literally silently skips the prune.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Generate docs/openapi.json (3.1, 362 operations) from the API inventory via
scripts/gen_openapi.py, and serve it with a self-hosted Redoc bundle at
/api/docs - no CDN, works on the air-gapped box. Also serve docs/llms.txt (a
concise LLM entrypoint) at /api/docs/llms.txt. New core 'docs' blueprint;
staticdocs/ excluded from the naming check (vendored minified JS).
Servers were imported as computers (a PC type) so they show under PCs, not
Network. This one-shot re-points each server's asset in place - assetid is
unchanged, so comms/relationships/map/name/location/audit all carry over; only
the extension row is swapped (computers -> networkdevices), the asset type is
flipped, and the device gets the 'Server' networkdevicetype (created if absent).
Identify servers by their computer type name (--type, default 'Server'). Dry-run
by default; --commit applies. Run on the target instance.
Air-gapped sites cannot pip install / npm ci / docker pull, so a build-at-site
compose (build: .) fails and reports 'service api is not running'. Add a
build-once-ship-image path:
- scripts/build-offline-bundle.ps1: on a connected box, build shopdb-flask +
pull mysql:8.0, docker save both into one gzipped tarball with a sha256.
- docker-compose.airgap.yml: runs pre-loaded images (image:, never build:),
drops the ./plugins bind mount (which would mask the image's baked-in plugins
with an empty host dir and load zero plugins at an image-only site), and adds
a one-shot migrate service (db upgrade + plugin upgrade-all + seed) that api
waits on via service_completed_successfully, so 'up -d' brings a working site.
- docs/DEPLOY-AIRGAP.md: full runbook (build, transfer+verify, load+run, admin,
verify, upgrade, troubleshooting), incl the Zscaler in-build cert caveat.
- .env.example: IMAGE_TAG for the air-gap compose to pin the loaded image tag.
The org IP allow list blocks GitHub-hosted runner IPs (checkout 403), so
point all jobs at the self-hosted arc-runner-set. Drop the rsync dependency
in build-site.sh (cp + bytecode prune; the ARC runner image has no rsync)
and remove the migrations-mysql job - ARC/Kubernetes has no service
containers, so that MySQL 8 coverage stays on the internal CI.
A frontend dir under plugins/ with no manifest.json is a CORE feature, not a
per-site plugin - applications is one (backend is shopdb/core/api/applications.py,
nav is advertised as core in dashboard.py). stage-frontend.mjs treated it like
a plugin and dropped it under SITE_PLUGINS, so a lean site showed the core
Applications nav item but had no route for it -> blank page. Now manifest-less
frontends always stage regardless of SITE_PLUGINS; SITE_PLUGINS selection applies
only to real plugins. CI lean-build job asserts ApplicationsList ships in a lean
bundle. Found while testing a live machines+printers lean site.
The lean-build endgame: a site ships carrying only the plugins it chose.
- scripts/build-site.sh: reads a site profile, resolves the hard-dependency
closure from manifests, builds the frontend with SITE_PLUGINS (stage-frontend
carries only those plugins), and stages a backend tree of core + only the
chosen plugin dirs. An unchosen plugin is in neither the bundle nor the tree.
- Core lazy-import guard: `flask seed demo` hard-imported the 5 asset subtype
models, which would crash a lean build missing any of those plugins. Now
guarded (a missing model skips its demo section).
- test_lean_build_guards.py: statically asserts NO core (shopdb/core, shopdb/cli)
import of a plugin is unguarded - a lean build omitting that plugin would
otherwise crash. 0 unguarded today.
Pilot verified: a lean build (machines + printers) carries only machines +
printers code - PartsKiosk / ManifestEditor / USBLabelBatch / KnowledgeBaseDetail
/ EmployeeDirectory are absent from the bundle, and only machines/printers plugin
dirs stage into the backend. (Sidebar labels for absent plugins remain - the
accepted small plugin-aware core remainder.) Guard test + naming green.
The naming/style check now fails a plugin frontend (plugins/<name>/frontend/)
that imports with an escaping ../../ or another plugin's path. Plugin frontends
must reach core only through the @/ alias and otherwise import only their own
tree, so a per-site build can drop a plugin cleanly. All 14 plugin frontends
pass.
Relocate applications, geenforce, knowledgebase, and machines - each owns only
its own views dir, so a clean move to plugins/<name>/frontend/ (views/ +
routes.js, core imports rewritten to @/). geenforce's entryForm.js helper + its
vitest spec move with it (ManifestEditor imports it as a sibling).
Machinery fixes this batch surfaced:
- routes.gen.js codegen uses namespace imports (import * as p_x). A route file
without a `toplevel` export is undefined on the namespace instead of a strict-
ESM missing-binding build error.
- vitest gains a `pretest` stage so plugin-frontend specs (now under
plugins/<name>/frontend/) run from their staged copy in src/.plugins-staged/.
Verified live: GE-Enforce (the most complex, uses the entryForm sibling helper)
renders fully from its staged frontend. Build + 58 vitest + naming green.
The staging step that makes lean per-site frontend builds possible, plus the
first plugin relocated as the pilot.
- scripts/stage-frontend.mjs: copies each chosen plugin's plugins/<name>/frontend/
into frontend/src/.plugins-staged/<name>/ and codegens routes.gen.js. Plugin
selection via SITE_PLUGINS (comma-separated); empty = all plugins that have a
frontend/ (the full build). Wired as npm predev/prebuild; outputs gitignored.
- Router imports routes.gen.js and merges staged routes with the in-tree
./routes/*.js glob - dual-location during the transition.
- printedparts relocated: its 6 views (list/detail/form/kiosk + the settings and
labels views from the shared dirs) moved into plugins/printedparts/frontend/
views/, core imports rewritten to the @/ alias; routes.js is the self-contained
route module. Its old in-tree route file is removed.
Also fixes a crash the previous commit (37c764b) shipped: slides.js exports only
`toplevel` (its child routes live in core.js), so the router's
flatMap(m => m.default) produced an undefined child and threw
"Cannot read properties of undefined (reading 'path')" at load - the whole SPA
went blank. Guarded with `m.default || []`. (The earlier "print pages are blank"
reading was this crash, not page nature.)
Verified live: /machines renders again; the relocated /printedparts list renders
identically from the staged plugin frontend; SITE_PLUGINS=machines excludes
printedparts from routes.gen. Build (via npm, runs stage) + vitest + naming green.
Loader: bool() on pymysql bit(1) bytes is always true - isinstallable
and isshopfloor imported as 1 for every row; route through _truthy_bit.
The employee source DB is now optional (shopdb-only imports).
Frontend: under a subpath mount the 401 interceptor stored the browser
path (mount base included) as the login redirect and the router applied
its base again (/ops/ops). New stripBase() keeps redirects base-free.
tools/export-github.sh automates the publication flow: prune + scrub +
commit into ~/projects/shopdb-flask-pub and emit a transfer bundle.
Import-surface docs and docstrings describe the automation as a
migration script; status-doc references in CHANGELOG/ADR-009/ROADMAP
point at repo files. Screenshot/verify tools write to /tmp/shopdb-shots
(created on import) instead of a machine-specific directory.
Classic ASP keeps retired machines in the machines table as history (isactive=0);
every other stage already filters isactive=1, but the assets hub, metrology,
locations and the verify count read machines unfiltered, so ~240 retired units
(incl. all G-prefix hostnames and 61 dead metrology PCs that each synthesized a
phantom measuring tool) landed as live assets. Add the isactive=1 filter to
those four queries. Downstream stages resolve via the id crosswalk, so warranties
/comms/relationships/installs for retired machines now drop automatically.
A freshly-imported DB left setup_complete unset, so the admin was bounced into
the first-run wizard even though the instance is fully populated. The harness now
sets setup_complete=true (it already mints the admin), so an imported instance
goes straight to the app.
Routing by pctype wrongly swept ~105 metrology PCs (CMM/Genspect/Keyence/Wax)
into measuringtools - but a PC that drives an instrument is still a computer;
the physical CMM/gauge is the tool. Dropped the pctype override: those PCs now
import as computers (measuringtools drops to the 48 real instrument rows).
Classic has no separate tool row for a metrology PC, so they'd be orphaned. New
metrology stage synthesizes a measuring-tool asset per metrology PC (typed by
its pctype: CMM / Form Tracer / Vision System / Genspect) and a Controls
relationship PC -> tool, mirroring what the runtime collector does.
Result: computers 663->751, measuringtools = 48 real + 88 synthesized (each
linked to its controlling PC), 88 new Controls relationships, no orphans.
The applications stage dropped applicationlink and documentationpath, so the app
detail's "Launch Application" and "Documentation" links were always empty. Map
both from the classic applications table.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Notifications imported with only employeesso, leaving employeename null - the
model displays "employeename or employeesso", so recognition/training cards
showed a bare SSO instead of a name. Build an SSO -> "First Last" map from the
employee source and populate employeename (comma-separated SSOs -> joined
names). SSOs not in the directory (former/non-WJF) stay null and fall back to
the SSO, as before.
Verified on the import DB: 261 notifications re-imported, names resolved
(Brandon Saltz, Jon Kolkmann, ...).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Follow-up to the mapper retirement (the new docs missed the prior commit's
staging). Adds docs/IMPORT-ADOPTION.md (two-layer import story + stage/crosswalk
guidance), scripts/migration/README.md (dir superseded, points at the API +
loader), and updates the WJ loader README to complete status.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Cleanup after the reference loader (scripts/site_imports/wjf/) proved out.
Removed the drifted direct-SQL migrators - migrate_assets/communications/
notifications/usb.py, run_migration.py, verify_migration.py, and
scripts/import_from_mysql.py. They targeted a nonexistent equipment table, the
retired Machine model, and columns that no longer exist; nothing imported them.
scripts/migration/README.md now points at the import API + the site loader.
Kept the one-time SQL fixups (fix_legacy_schema.sql, one-offs/).
Added docs/IMPORT-ADOPTION.md: the two-layer import story (stable IMPORT-API
contract + per-site loader), stage-ordering + crosswalk guidance, the
agent-assisted mapping path, and what the WJ loader demonstrates. Updated the
loader README to complete status (all 15 stages, final counts).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
printers: printers come from the printers TABLE (not the machines hub), so a
dedicated stage - assetnumber synthesized PRN-{printerid}, IP folded via the
create route, host machineid resolved to a location when it is a LocationOnly
row. Skips inactive. 50 of 56 imported.
relationships: a "Controlled By" edge now flips to the forward Controls
direction (source/target swapped, mapped to the Controls type) instead of
importing a redundant inverse type.
Final fresh full run: 983 assets (933 machines-hub + 50 printers), zero endpoint
errors, all 15 stages.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Completes the loader end to end. Verified on a fresh scratch target, zero
endpoint errors:
- locations: the 24 islocationonly rows -> core Locations (crosswalk machineid
-> locationid).
- relationships: 93 active edges imported (206 of 299 dropped because an
endpoint became a Location / was skipped / dedup-lost); types folded onto the
seeded canonical set; dedup on (source,target,type).
- subnets: 37 (full CIDR reconstructed as INET_NTOA(ipstart)+suffix; VLANs
lookup-or-create by number; 3 duplicate CIDRs first-wins-skipped).
- usb: 18 cmmc devices + 232 check-in/out events, paired with per-device open
state so unpaired log rows do not 400. Needs the usb plugin enabled + usb
directory mode selfhosted.
- verify: source-vs-target row-count audit (assets 1167->933 by the skip rules,
applications 121=121, employees 415=415, KB 342->341).
Full pipeline default runs all 14 stages in order. NOTE: the import target needs
every bundled plugin enabled (usb ships disabled in this dev registry - enable
it before importing).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Of the 158 machinetypeid=1 rows, only 24 carry the islocationonly bit (real
named areas: DT Office, IT Closet, Materials, ...). The other 134 are active,
modelled shop machines just left untyped - routing all 158 to Locations dropped
those 134 real assets. Route on the bit instead; the 134 untyped rows import as
machines with a null subtype (machinetypeid=1 is not a real machine subtype, so
catalog skips seeding one).
Also process asset routes in richness order (computer > measuringtool > network
> machine) so on a duplicate machinenumber the PC - which carries installs + IP
a bare untyped machine does not - wins first-come.
Result on the scratch target: 933 assets (computer 663, machine 76, network 58,
measuringtool 136), 24 locations (was mis-routing 158), installs 850 (was 653 -
PCs no longer lose their numbers to bare machines), warranties 464, comms 461.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All consume the machineid->assetid crosswalk from the assets hub. Verified
against the scratch target, zero endpoint errors:
- communications: 435 primary IPs folded onto assets. No bulk endpoint exists,
so this is the plan's documented direct-ORM gap (reads the source
communications table where comstypeid=1 AND isprimary=1, not machines.ipaddress1
which is empty).
- applications: supportteams 45, applications 121 (colliding names dedup via the
unique-appname 409-resolve), appversions 47, installs 653 (machineid ->
assetid -> computerid; only computer assets take installs).
- warranties: 424 linked, vendor hardcoded Dell (source has none).
- notifications: types 6, notifications 261 (2099 sentinel endtime clamped).
- knowledgebase: 341 (appid resolved through the applications name map).
Inactive rows skipped everywhere per the decisions. Remaining loader stages:
locations (the 158 LocationOnly rows), relationships (301 active edges),
subnets/VLANs, usb (cmmc pairing), verify.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the keystone stages to the reference loader.
catalog: seeds modeltypes (from classic machinetypes + category), the per-plugin
asset subtypes routed by machinetype (machines/network/measuringtool types),
computer subtypes (from pctype), the 5-row controllertypes vendor/model split,
and the models catalog - each with a persisted legacy->new crosswalk. Verified:
modeltypes 31, computertypes 12, models 118.
assets (the hub): fans classic machines out to the right endpoint by
machinetypeid (+ the pctype metrology override), applying the resolved
decisions - assetnumber = machinenumber else hostname, skip 9999, skip duplicate
machinenumbers, LocationOnly/printer/USB routed out. Persists the machineid ->
assetid crosswalk every downstream stage needs. Verified against a fresh scratch
target: 884 assets (computer 623, machine 68, network 58, measuringtool 135),
zero endpoint errors, idempotent re-run (stays 884). Skips: location 158, dup 69,
other 53, 9999 1.
Harness now runs each plugin's idempotent on_install so the AssetType rows exist
(a DB built with plugin upgrade-all instead of a fresh install lacks them, and
the create routes 500 without them). 409-resolve lookups page through per_page.
Remaining stages: communications (primary IP fold - source is the communications
table, not machines.ipaddress1), applications/installs, warranties, notifications,
KB, subnets/VLANs, usb, verify.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Layer 2 of the import design (see the memory + scratchpad/IMPORT-PLAN.md): a
SITE-SPECIFIC reference loader that maps WJ's classic-ASP schema onto the
maintained, schema-agnostic IMPORT-API contract. Other sites copy the pattern
against their own source DB; nobody runs this loader as-is.
Harness (scripts/site_imports/wjf/harness.py): builds the app against the
current DATABASE_URL (point it at a throwaway import DB), mints an unscoped
admin PAT in-process, and drives the real import endpoints through the app test
client with Authorization: Bearer + X-Import-Mode - exercising the same
routes/authz/validation an HTTP client would, no running server needed.
Read-only pymysql access to the three scratch source DBs; legacy-id -> new-id
crosswalks persist to JSON so a crashed run resumes and later stages resolve FKs.
Stages implemented + verified idempotent against a fresh scratch target
(shopdb_flask_import): reference (vendors 46, businessunits 13, operatingsystems
11) and employees (directory 415, re-run updated-not-duplicated). Remaining
stages (models, applications, assets hub + crosswalk, dependents, network, usb,
verify) are stubbed with the same shape; README documents the adoption playbook.
idmap.json is generated state (gitignored).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Path.rglob does not descend symlinks, so a symlinked external plugin
(the ADR-003 dev loop) silently escaped the contract-purity scan. The
scanner now resolves plugin dirs before walking, a regression test
plants a symlinked plugin with a real violation and asserts it is
flagged, and the known-limitation notes in the external-repo docs are
lifted.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- flask plugin new now scaffolds the frontend too: List/Detail/Form
views on the global styles, a gated route module (ADR-009), and an
api-client snippet emitted into the plugin dir. Views are written
before the route file so a partially generated plugin cannot 500 the
dev server.
- docs/PLUGIN-EXTERNAL-REPO.md + scripts/test-external-plugin.sh: how a
sister site develops a plugin in its own repo and runs the framework
contract tests in CI against a pinned framework ref (script verified
to fail on a broken core_version pin).
- docs/CONTRACT-STABILITY.md: settled vs churning contract surface and
the provisional 1.0 criteria.
- CLAUDE.md active-state refresh (contract 0.6.0, 11 plugins, 340
tests, measuringtools done).
Known limitation documented: Path.rglob does not descend symlinks, so
the import-surface contract test skips symlinked external plugins.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Make the app distributable to other GE Aerospace sites (one self-hosted
instance per site, ADR-004). GE values remain the shipped defaults; every
site-specific behavior is now a Setting an admin can change in the UI.
Settings-driven site config:
- Branding: site/QR/badge logos, favicon, primary color (upload endpoints
mirror the map-blueprint pattern; new Settings > Branding section).
- ServiceNow: search/incident/change URL templates ({ticket}), ticket
prefixes, enable toggle. Defaults point at the current
geaerospaceqa.service-now.com global search. Disabled = plain-text tickets.
- Employee-id regex (employeeid_pattern), printer hostname template,
QR label targets (qr_target_printer / qr_target_usb, blank = asset page,
else URL template with placeholders), usb_label_style (barcode|qr).
- West Jefferson floor-plan PNGs removed from the tree; generic placeholder
ships as the map default and sites upload their own blueprint.
Security closeout:
- dashboarddefaults writes now require admin.
- Collector: generic error messages (no str(exc) leak); API key accepted
via X-API-Key header only (BREAKING: querystring api_key removed).
- IP-based login rate limiting (AUTH_RATELIMIT_* knobs) atop account lockout.
- Setting.set() creation race fixed (IntegrityError retry).
Release engineering and docs:
- __version__ 0.5.0 (distinct from __contract_version__, ADR-007),
CHANGELOG.md, Gitea Actions CI config, frontend version aligned.
- One wizard-first install story across README/DEPLOY; new CONFIG.md,
UPGRADE.md, BACKUP-RESTORE.md; CLAUDE.md and ROADMAP de-staled.
- Dockerfile multi-stage build now bundles the frontend; compose binds
MySQL to 127.0.0.1; stale database/schema.sql and one-off SQL removed.
Debt and fixes:
- .query.get() -> db.session.get() sweep; datetime.utcnow() removed
(naive-UTC via timezone-aware now); users.py on authz decorators.
- Fixed 4 stale tests (slides feed shape, shopfloor splitperemployee,
plugin contract purity) and the USB label page field mapping (both usb
modes emit the cmmc shape: device_id/device_desc).
- Health endpoint reports the real version.
248 tests pass; naming/style check green; frontend builds; fresh-DB
flask db upgrade + seeds verified; QR targets verified by decoding
rendered codes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Lock the position-resolution columns from ADR-001 in code so
resolve_asset_position's relationship walk activates.
Schema
- Asset.mapleft -> Asset.mapx, Asset.maptop -> Asset.mapy
- Location.mapx / Location.mapy added (fallback for priority 3 of the
ADR-001 resolution chain)
- AssetRelationship.label (free-text nuance per ADR-001)
- AssetRelationship.inheritsposition (bool, server_default true, controls
whether the resolved-position walk follows the edge)
- RelationshipType.propagatesthroughid (self-FK; sibling-propagation rail)
Seeds
- Three canonical ADR-001 relationship types created idempotently:
partof, controls, connectedto
- controls.propagatesthroughid wired to partof (partof + connectedto stay
null per ADR-001 table). Both via Alembic migration AND CLI seed command
so a fresh test fixture and a sister-site deploy both end up correct.
- Legacy connection types (Serial Cable, Direct Ethernet, USB, WiFi,
Dualpath) retained for backward compat with pre-1.0 relationship rows.
Resolver
- shopdb.api.resolve_asset_position now walks inheritsposition=true edges
of type partof (then controls), recursively, depth-capped at 3 with
visited-set cycle protection. Inactive edges + non-inheritable types
are skipped. Falls through to the existing location fallback when the
walk yields nothing.
Tests
- 11 new test_api_namespace cases cover: partof walk, controls-after-
partof ordering, connectedto skipped, inheritsposition=false skipped,
recursion, cycle break, depth-3 cap, self-beats-related, related-beats-
location, inactive-edge skip.
- 111 tests pass. Naming/style check green.
Migration
- migrations/versions/7a01_adr001_position_contract.py:
- alter_column renames on assets (no data loss)
- add_column on locations + relationshiptypes + assetrelationships
- idempotent seed of three ADR types + propagation FK wire-up
- downgrade reverses + best-effort deletion of seeded types that have
no FK refs
Backend rename (mapleft/maptop -> mapx/mapy)
- shopdb/core/api/assets.py
- plugins/{computers,equipment,network,printers}/api/...
- scripts/migration/migrate_assets.py
- Legacy Machine model + machines API + import_from_mysql.py UNCHANGED
(per ADR-001 Machine retires; not part of the asset contract)
Frontend rename
- frontend/src/components/ShopFloorMap.vue
- frontend/src/views/{MapEditor.vue, pcs/{PCDetail,PCForm}.vue,
printers/{PrinterDetail,PrinterForm}.vue,
machines/{MachineDetail,MachineForm}.vue,
network/NetworkDeviceForm.vue}
- Form field labels + v-model bindings + computed flags switched in
lockstep with the backend.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New Plugins:
- USB plugin: Device checkout/checkin with employee lookup, checkout history
- Notifications plugin: Announcements with types, scheduling, shopfloor display
- Network plugin: Network device management with subnets and VLANs
- Equipment and Computers plugins: Asset type separation
Frontend:
- EmployeeSearch component: Reusable employee lookup with autocomplete
- USB views: List, detail, checkout/checkin modals
- Notifications views: List, form with recognition mode
- Network views: Device list, detail, form
- Calendar view with FullCalendar integration
- Shopfloor and TV dashboard views
- Reports index page
- Map editor for asset positioning
- Light/dark mode fixes for map tooltips
Backend:
- Employee search API with external lookup service
- Collector API for PowerShell data collection
- Reports API endpoints
- Slides API for TV dashboard
- Fixed AppVersion model (removed BaseModel inheritance)
- Added checkout_name column to usbcheckouts table
Styling:
- Unified detail page styles
- Improved pagination (page numbers instead of prev/next)
- Dark/light mode theme improvements
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Flask backend with Vue 3 frontend for shop floor machine management.
Includes database schema export for MySQL shopdb_flask database.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
