6c4bd20e01
Adds the keystone stages to the reference loader. catalog: seeds modeltypes (from classic machinetypes + category), the per-plugin asset subtypes routed by machinetype (machines/network/measuringtool types), computer subtypes (from pctype), the 5-row controllertypes vendor/model split, and the models catalog - each with a persisted legacy->new crosswalk. Verified: modeltypes 31, computertypes 12, models 118. assets (the hub): fans classic machines out to the right endpoint by machinetypeid (+ the pctype metrology override), applying the resolved decisions - assetnumber = machinenumber else hostname, skip 9999, skip duplicate machinenumbers, LocationOnly/printer/USB routed out. Persists the machineid -> assetid crosswalk every downstream stage needs. Verified against a fresh scratch target: 884 assets (computer 623, machine 68, network 58, measuringtool 135), zero endpoint errors, idempotent re-run (stays 884). Skips: location 158, dup 69, other 53, 9999 1. Harness now runs each plugin's idempotent on_install so the AssetType rows exist (a DB built with plugin upgrade-all instead of a fresh install lacks them, and the create routes 500 without them). 409-resolve lookups page through per_page. Remaining stages: communications (primary IP fold - source is the communications table, not machines.ipaddress1), applications/installs, warranties, notifications, KB, subnets/VLANs, usb, verify. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
WJ classic-ASP import loader (reference implementation)
This is site-specific reference glue, not product code. It maps West
Jefferson's classic-ASP shopdb / cmmc_usb / wjf_employees schema onto the
maintained, schema-agnostic import contract in docs/IMPORT-API.md.
Every adopting site has its own source database. Nobody else runs this loader. Instead, copy the pattern:
- Point the harness at your source DB(s) (edit
harness.Source). - Write per-entity stages that read your tables and POST to the same
docs/IMPORT-API.mdendpoints withAuthorization: Bearer <admin PAT>andX-Import-Mode: true. - Persist legacy-id -> new-id crosswalks (see
harness.IdMap) so later stages resolve foreign keys and a crashed run resumes.
The import API is the stable contract; loaders are per-site. The mapping itself can be produced with the agent-assisted workflow (point Fable/Opus agents at a source DB + this contract -> they emit the mapping + a loader skeleton).
Running (against a THROWAWAY import database)
DATABASE_URL='mysql+pymysql://root:PW@127.0.0.1:3306/shopdb_flask_import?charset=utf8mb4' \
venv/bin/python -m scripts.site_imports.wjf.run --stages reference,employees
Prereqs: a fresh target DB built with flask db upgrade + flask plugin upgrade-all + flask seed permissions/settings/reference-data, and the three
source dumps loaded into scratch DBs (shopdb_src, cmmc_usb_src,
wjf_employees_src). See scratchpad/IMPORT-PLAN.md for the full mapping,
resolved decisions, and remaining stages.
Status
- Implemented + verified idempotent:
reference(vendors, businessunits, operatingsystems),employees(directory bulk upsert; photos deferred). - TODO stages:
models,applications,assets(the hub - fan machines out by type, persist the machineid->assetid crosswalk),dependents(installs, warranties, notifications, KB),network(+ subnets/VLANs),usb(cmmc device + checkinout pairing),verify.
The harness (PAT auth, import-mode, id-map persistence, endpoint error capture)
is proven; the remaining stages are additional stage_* functions in run.py
following the same shape.