b11a6f26d8
Layer 2 of the import design (see the memory + scratchpad/IMPORT-PLAN.md): a SITE-SPECIFIC reference loader that maps WJ's classic-ASP schema onto the maintained, schema-agnostic IMPORT-API contract. Other sites copy the pattern against their own source DB; nobody runs this loader as-is. Harness (scripts/site_imports/wjf/harness.py): builds the app against the current DATABASE_URL (point it at a throwaway import DB), mints an unscoped admin PAT in-process, and drives the real import endpoints through the app test client with Authorization: Bearer + X-Import-Mode - exercising the same routes/authz/validation an HTTP client would, no running server needed. Read-only pymysql access to the three scratch source DBs; legacy-id -> new-id crosswalks persist to JSON so a crashed run resumes and later stages resolve FKs. Stages implemented + verified idempotent against a fresh scratch target (shopdb_flask_import): reference (vendors 46, businessunits 13, operatingsystems 11) and employees (directory 415, re-run updated-not-duplicated). Remaining stages (models, applications, assets hub + crosswalk, dependents, network, usb, verify) are stubbed with the same shape; README documents the adoption playbook. idmap.json is generated state (gitignored). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
WJ classic-ASP import loader (reference implementation)
This is site-specific reference glue, not product code. It maps West
Jefferson's classic-ASP shopdb / cmmc_usb / wjf_employees schema onto the
maintained, schema-agnostic import contract in docs/IMPORT-API.md.
Every adopting site has its own source database. Nobody else runs this loader. Instead, copy the pattern:
- Point the harness at your source DB(s) (edit
harness.Source). - Write per-entity stages that read your tables and POST to the same
docs/IMPORT-API.mdendpoints withAuthorization: Bearer <admin PAT>andX-Import-Mode: true. - Persist legacy-id -> new-id crosswalks (see
harness.IdMap) so later stages resolve foreign keys and a crashed run resumes.
The import API is the stable contract; loaders are per-site. The mapping itself can be produced with the agent-assisted workflow (point Fable/Opus agents at a source DB + this contract -> they emit the mapping + a loader skeleton).
Running (against a THROWAWAY import database)
DATABASE_URL='mysql+pymysql://root:PW@127.0.0.1:3306/shopdb_flask_import?charset=utf8mb4' \
venv/bin/python -m scripts.site_imports.wjf.run --stages reference,employees
Prereqs: a fresh target DB built with flask db upgrade + flask plugin upgrade-all + flask seed permissions/settings/reference-data, and the three
source dumps loaded into scratch DBs (shopdb_src, cmmc_usb_src,
wjf_employees_src). See scratchpad/IMPORT-PLAN.md for the full mapping,
resolved decisions, and remaining stages.
Status
- Implemented + verified idempotent:
reference(vendors, businessunits, operatingsystems),employees(directory bulk upsert; photos deferred). - TODO stages:
models,applications,assets(the hub - fan machines out by type, persist the machineid->assetid crosswalk),dependents(installs, warranties, notifications, KB),network(+ subnets/VLANs),usb(cmmc device + checkinout pairing),verify.
The harness (PAT auth, import-mode, id-map persistence, endpoint error capture)
is proven; the remaining stages are additional stage_* functions in run.py
following the same shape.