The applications stage dropped applicationlink and documentationpath, so the app
detail's "Launch Application" and "Documentation" links were always empty. Map
both from the classic applications table.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Notifications imported with only employeesso, leaving employeename null - the
model displays "employeename or employeesso", so recognition/training cards
showed a bare SSO instead of a name. Build an SSO -> "First Last" map from the
employee source and populate employeename (comma-separated SSOs -> joined
names). SSOs not in the directory (former/non-WJF) stay null and fall back to
the SSO, as before.
Verified on the import DB: 261 notifications re-imported, names resolved
(Brandon Saltz, Jon Kolkmann, ...).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Follow-up to the mapper retirement (the new docs missed the prior commit's
staging). Adds docs/IMPORT-ADOPTION.md (two-layer import story + stage/crosswalk
guidance), scripts/migration/README.md (dir superseded, points at the API +
loader), and updates the WJ loader README to complete status.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
printers: printers come from the printers TABLE (not the machines hub), so a
dedicated stage - assetnumber synthesized PRN-{printerid}, IP folded via the
create route, host machineid resolved to a location when it is a LocationOnly
row. Skips inactive. 50 of 56 imported.
relationships: a "Controlled By" edge now flips to the forward Controls
direction (source/target swapped, mapped to the Controls type) instead of
importing a redundant inverse type.
Final fresh full run: 983 assets (933 machines-hub + 50 printers), zero endpoint
errors, all 15 stages.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Completes the loader end to end. Verified on a fresh scratch target, zero
endpoint errors:
- locations: the 24 islocationonly rows -> core Locations (crosswalk machineid
-> locationid).
- relationships: 93 active edges imported (206 of 299 dropped because an
endpoint became a Location / was skipped / dedup-lost); types folded onto the
seeded canonical set; dedup on (source,target,type).
- subnets: 37 (full CIDR reconstructed as INET_NTOA(ipstart)+suffix; VLANs
lookup-or-create by number; 3 duplicate CIDRs first-wins-skipped).
- usb: 18 cmmc devices + 232 check-in/out events, paired with per-device open
state so unpaired log rows do not 400. Needs the usb plugin enabled + usb
directory mode selfhosted.
- verify: source-vs-target row-count audit (assets 1167->933 by the skip rules,
applications 121=121, employees 415=415, KB 342->341).
Full pipeline default runs all 14 stages in order. NOTE: the import target needs
every bundled plugin enabled (usb ships disabled in this dev registry - enable
it before importing).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Of the 158 machinetypeid=1 rows, only 24 carry the islocationonly bit (real
named areas: DT Office, IT Closet, Materials, ...). The other 134 are active,
modelled shop machines just left untyped - routing all 158 to Locations dropped
those 134 real assets. Route on the bit instead; the 134 untyped rows import as
machines with a null subtype (machinetypeid=1 is not a real machine subtype, so
catalog skips seeding one).
Also process asset routes in richness order (computer > measuringtool > network
> machine) so on a duplicate machinenumber the PC - which carries installs + IP
a bare untyped machine does not - wins first-come.
Result on the scratch target: 933 assets (computer 663, machine 76, network 58,
measuringtool 136), 24 locations (was mis-routing 158), installs 850 (was 653 -
PCs no longer lose their numbers to bare machines), warranties 464, comms 461.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All consume the machineid->assetid crosswalk from the assets hub. Verified
against the scratch target, zero endpoint errors:
- communications: 435 primary IPs folded onto assets. No bulk endpoint exists,
so this is the plan's documented direct-ORM gap (reads the source
communications table where comstypeid=1 AND isprimary=1, not machines.ipaddress1
which is empty).
- applications: supportteams 45, applications 121 (colliding names dedup via the
unique-appname 409-resolve), appversions 47, installs 653 (machineid ->
assetid -> computerid; only computer assets take installs).
- warranties: 424 linked, vendor hardcoded Dell (source has none).
- notifications: types 6, notifications 261 (2099 sentinel endtime clamped).
- knowledgebase: 341 (appid resolved through the applications name map).
Inactive rows skipped everywhere per the decisions. Remaining loader stages:
locations (the 158 LocationOnly rows), relationships (301 active edges),
subnets/VLANs, usb (cmmc pairing), verify.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the keystone stages to the reference loader.
catalog: seeds modeltypes (from classic machinetypes + category), the per-plugin
asset subtypes routed by machinetype (machines/network/measuringtool types),
computer subtypes (from pctype), the 5-row controllertypes vendor/model split,
and the models catalog - each with a persisted legacy->new crosswalk. Verified:
modeltypes 31, computertypes 12, models 118.
assets (the hub): fans classic machines out to the right endpoint by
machinetypeid (+ the pctype metrology override), applying the resolved
decisions - assetnumber = machinenumber else hostname, skip 9999, skip duplicate
machinenumbers, LocationOnly/printer/USB routed out. Persists the machineid ->
assetid crosswalk every downstream stage needs. Verified against a fresh scratch
target: 884 assets (computer 623, machine 68, network 58, measuringtool 135),
zero endpoint errors, idempotent re-run (stays 884). Skips: location 158, dup 69,
other 53, 9999 1.
Harness now runs each plugin's idempotent on_install so the AssetType rows exist
(a DB built with plugin upgrade-all instead of a fresh install lacks them, and
the create routes 500 without them). 409-resolve lookups page through per_page.
Remaining stages: communications (primary IP fold - source is the communications
table, not machines.ipaddress1), applications/installs, warranties, notifications,
KB, subnets/VLANs, usb, verify.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Layer 2 of the import design (see the memory + scratchpad/IMPORT-PLAN.md): a
SITE-SPECIFIC reference loader that maps WJ's classic-ASP schema onto the
maintained, schema-agnostic IMPORT-API contract. Other sites copy the pattern
against their own source DB; nobody runs this loader as-is.
Harness (scripts/site_imports/wjf/harness.py): builds the app against the
current DATABASE_URL (point it at a throwaway import DB), mints an unscoped
admin PAT in-process, and drives the real import endpoints through the app test
client with Authorization: Bearer + X-Import-Mode - exercising the same
routes/authz/validation an HTTP client would, no running server needed.
Read-only pymysql access to the three scratch source DBs; legacy-id -> new-id
crosswalks persist to JSON so a crashed run resumes and later stages resolve FKs.
Stages implemented + verified idempotent against a fresh scratch target
(shopdb_flask_import): reference (vendors 46, businessunits 13, operatingsystems
11) and employees (directory 415, re-run updated-not-duplicated). Remaining
stages (models, applications, assets hub + crosswalk, dependents, network, usb,
verify) are stubbed with the same shape; README documents the adoption playbook.
idmap.json is generated state (gitignored).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
