ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
ecdbe53ef6db6cbc10fc8f8cc940e84428e689bb
shopdb-flask/docs/IMPORT-LEGACY-ASP.md
cproudlock 534f179ee6 Make main the lean product a site receives 
main was 16 days and 259 commits behind the line the product is actually built
on, at contract 0.30.0 against 0.35.0, with no per-PC credential code at all. It
was neither current nor curated - just stale. This brings it forward and, at the
same time, makes it mean something: main is what a receiving site installs, and
nothing else.

WHAT IT CARRIES. Twelve plugins - backups, computers, endpoints, guides,
knowledgebase, machines, network, notifications, printers, slides, tools,
warranty - plus applications, which is manifest-less core and always ships. The
only manifest dependency inside that set is endpoints on computers, and both are
in it, so the closure is self-contained.

WHAT IT DOES NOT. employees, geenforce, measuringtools, printedparts and usb
stay on the site line. So do the Windows installer tree, the VM rehearsal
helpers, the agent tooling config, one site's data loader, the internal audit
and findings records, and the planner spreadsheet. 1,135 tracked files against
1,438.

A REMOVAL THAT BREAKS THE APP IS NOT A REMOVAL, and two nearly shipped:

deploy/site-profile-universal.json still named all seventeen plugins. It is the
file docs/DEPLOY.md tells a site to provision from, so a fresh install would
have failed at `flask plugin apply-profile` with PluginError - the first thing a
receiving site does, broken by the branch meant to serve them. It now names the
twelve that exist, and its comment says the two must agree.

agent/ was never refreshed. Whoever populated this branch copied shopdb/,
plugins/, frontend/ and tests/ from the site line and left the agent tree at its
September state: 22 C# and PowerShell sources missing, including AgentCredentials,
EnrollmentService, AgentHttpFactory and the fleet-alert client, plus one file the
site line had deleted still present. The old tree is internally consistent, so it
compiles, and nothing in CI builds it - every gate was green over a server that
speaks enrollment and per-PC credentials to a bundled agent source that cannot.
Resynced whole.

DECISIONS THIS ENCODES, rather than defers:

- The plugin-authoring lab builds printedparts, whose manifest depends on
  employees, so it could not stay. The lab remains on the site line and main
  ships no build-along tutorial. That is a real loss for an adopting developer
  and is recorded here rather than discovered later.
- Recognition, Training and Recertification notifications are no longer seeded.
  Each splits one notification into a card per employee and renders a photo,
  which needs a plugin this branch does not bundle. Seeding them would offer an
  author a type that cannot resolve anybody. A site that installs employees gets
  them from that plugin's own seed.
- Windows/IIS is documented as the legacy path throughout, and the README leads
  with Docker, because that is what a receiving site gets.

The API description is regenerated for THIS branch - 347 paths and 483
operations, down from 415 and 573 - so the published contract describes what is
here rather than what the site line serves. docs/CONFIG.md lost the two
measuring-tool label keys the code no longer declares, which two lanes had
decided opposite ways.

Suite: 3124 passed, 6 skipped. Naming and style gates pass. The docs gates run
rather than skip, and every index line count is derived from the file it names.
2026-09-19 11:24:40 -04:00

164 lines
9.2 KiB
Markdown
Raw Blame History

<!-- docmeta
purpose: reference
audience: adopter, developer
status: historical
summary: Record of one classic-ASP shopdb migration: the fan-out routing, the judgement calls and what was left behind.
loadwith: IMPORT-API.md
-->
# Migrating the classic-ASP shopdb: the record
**Read this only if your source is the classic-ASP shopdb.** It is one site's
completed migration, kept for the parts a second site would otherwise rederive
and for the parts that still matter after the import is over: how the `machines`
hub was fanned out, which judgement calls were made, and which legacy tables
were deliberately left behind.
It is NOT the product contract. Authentication, `X-Import-Mode`, the seed, the
ordering rule, the idempotent upsert recipe and the verification method are in
[IMPORT-API.md](IMPORT-API.md), which is current and schema-agnostic. Read that
first; this page assumes it. The mechanical column-by-column mapping that the
run was driven from is not reproduced here - the source and target column names
match closely enough to re-derive, and a copy would only drift.
What dates it, and what you must re-check against your own database rather than
trust here: the `machinetypeid` integers below are ids from ONE site's
`machinetypes` lookup, not a shared vocabulary; the row counts are that
database's; and the routing is a recommended default that a site may
legitimately depart from.
---
## The `machines` hub fans out into the asset plugins
`machines` (885 rows) is the central legacy asset table. Two columns drive the
fan-out: `machinetypeid` (what the asset physically is) and `pctypeid` (a
computer's sub-type). Route each row by `machinetypeid`:
| legacy `machinetypeid` | `machinetypes.machinetype` | target plugin | subtype source |
|---|---|---|---|
| 1 | LocationOnly (also `islocationonly=1`) | core **Locations** (NOT an asset) | `locationtype` |
| 33 | PC | **computers** | `computertype` from `pctype.typename` via `machines.pctypeid` |
| 20 | Server | **computers** | computer type "Server" |
| 15 | Printer | **printers** | `printertype` (authoritative source is the legacy `printers` table) |
| 16 Access Point / 17 IDF / 18 Camera / 19 Switch / 46 Firewall | | **network** | `networkdevicetype` |
| 44 | USB Device | **usb** | (usb device) |
| 23 Measuring Machine / 3 CMM / 48 Spline Checker / 8 Eddy Current / 47 Inspection | | **measuringtools** (gage-lab judgment call; ADR-005) | `measuringtooltype` |
| 2,4,5,6,7,9,10,11,12,13,14,21,22,24,25,45 (lathes, mills, welders, grinders, ...) | | **machines** | `machinetype` |
Decide the routing table once, up front: it is what every later count, report
and parity check is measured against. A site may re-route a `machinetypeid` -
sending CMM to `machines` rather than `measuringtools`, for example.
Non-obvious column mappings on the hub, the ones that are not a same-name copy:
- `machinenumber` is the `assetnumber` (the business identifier and the natural
key); `alias` or `hostname` becomes the layperson `name`.
- `mapleft`/`maptop` become `mapx`/`mapy`, which are pixels of ONE level. An
import that omits `levelid` produces markers that are never drawn - see
[IMPORT-API.md](IMPORT-API.md#a-map-position-needs-its-level).
- `machinestatusid`, `businessunitid`, `osid` are remapped through the lookups
imported before the assets.
- `dateadded`/`lastupdated` become `createddate`/`modifieddate`, honored in
import mode only.
- `ipaddress1` is the primary IP, folded onto the asset as `ipaddress`.
## Lookups that are not a straight copy
Most legacy lookup tables map name-to-name onto the obvious endpoint. These do
not:
| legacy table | target | the catch |
|---|---|---|
| `machinetypes` | `POST /api/modeltypes` | becomes the MODEL type vocabulary (ADR-011), and needs a `category` (Equipment/Computer/...) the legacy row does not carry |
| `controllertypes` | `vendors` + `models` | one legacy row splits in two: "Fanuc" becomes a Vendor, the controller model becomes a Model, then set `controllervendorid`/`controllermodelid` on the machine |
| `pctype` | `POST /api/computers/types` | `typename` becomes `computertype`; also drives the computer subtype in the fan-out above |
| `subnets` | `POST /api/network/subnets` | create the VLAN first (`POST /api/network/vlans`), then reference it; `subnettypeid` resolves to a `subnettype` NAME, not an id |
| `dashboarddefaults` | `POST /api/dashboarddefaults` | resolution is FQDN-first with IP fallback, so import both keys when the source has them |
| `comstypes` | `communicationtypes` | seeded, not imported. `flask seed reference-data` creates IP/Serial/USB/... and there is no CRUD endpoint |
| `topics` | `POST /api/applications` | `topics` is a near-clone of `applications` and `knowledgebase.appid` points at it, so each topic imports as an Application or the KB links resolve against nothing |
| `appowners` | `POST /api/supportteams/{id}/contacts` | folded into contacts: each `supportteams` row carries one `appownerid`, which becomes ONE contact on that team |
**Model images** import as a plain URL string only. Binary photos are not part
of the import payload; upload them afterwards with
`POST /api/models/<modelid>/image`. Employee photos work the same way
(`POST /api/employees/<sso>/photo`), and only self-hosted mode owns people at
all - in external mode the photo is a read-only pass-through from the HR
database.
## Relationships
`machinerelationships` becomes `POST /api/assets/relationships`, with
`machineid`/`related_machineid` resolved through the legacy-id crosswalk to
`sourceassetid`/`targetassetid`. The legacy type names do not survive as they
are; this is the mapping that was used:
Controls and Master-Slave become **Controls**; Controlled By becomes **Controls**
with source and target reversed; Cluster Member and Contains become **partof**;
Connected To becomes **connectedto**; Dualpath, Backup For and Stored At keep
their own types. Create the types before the links.
## Data quirks worth knowing about
- **`notifications.endtime` carries the sentinel `2099-00-03 09:52:32`**, which
is not a valid date. Drop or clamp it; a straight copy fails the parse.
- **`installedapps` only makes sense for computer-class assets.** Skip rows
whose `machineid` did not route to a computer.
- **`warranties` needs a `vendor`** that the legacy row may not carry. It is
required on create.
- **USB history replays as events, not rows**: create the device
(`POST /api/usb`), then replay each `usbcheckouts` row as a checkout and, if
it was returned, a checkin, carrying `checkouttime`/`checkintime`. Those
overrides are honored in import mode only.
- Any legacy column with no target field (`machines.logicmonitorurl`,
`machines.fqdn`, `printers.printerpin`) becomes a custom field - the recipe is
step 7 of [IMPORT-API.md](IMPORT-API.md#4-order-of-operations).
---
## Tables with no target
These legacy tables have no import target. The dispositions are DECIDED, not
open questions.
- **`dncconfig` and `commconfig`** - intentionally NOT migrated. DNC
communication settings drift constantly, so a one-shot import of stale data
has little value. The plan is a future DNC feature fed live by the GE-Enforce
collector rather than a historical import. When that is built, the expected
ingestion pattern is one attribute at a time across the whole facility (sweep
every machine's baud rate in one pass, then ports, and so on), so the design
should favor per-field fleet-wide updates over per-machine full-record
imports.
- **`compliance`, `compliancescans`** - 0 rows. A future compliance plugin would
own them. Skip.
- **`ednc_installations`, `ednc_logs`** - 0 rows, and they belong to the eDNC
tooling rather than the asset catalog. Skip.
- **`distributiongroups`** (2 rows) - email distribution lists referenced by
`businessunits.distributiongroupid`. No target; skip, or attach as a business
unit custom field.
- **`functionalaccounts`** (7 rows) - service-account concept referenced by
`pctype`/`machinetypes`; no equivalent in the new schema. Skip, or capture as
a computer-type custom field.
- **`skilllevels`** (2 rows) - orphaned lookup (no FK from `machines`). Skip.
## Row-count parity
The method is section 6 of [IMPORT-API.md](IMPORT-API.md). What is specific to
this source is the legacy-side `WHERE` clause, because the fan-out means one
legacy table feeds several targets. Adjust each to match the routing you chose:
| entity | legacy count | new count |
|---|---|---|
| computers | `machines WHERE machinetypeid IN (33,20)` | `GET /api/computers` total |
| machines | `machines WHERE machinetypeid IN (2,4,5,6,7,9,10,11,12,13,14,21,22,24,25,45)` | `GET /api/machines` total |
| network devices | `machines WHERE machinetypeid IN (16,17,18,19,46)` | `GET /api/network` total |
| USB devices | `machines WHERE machinetypeid=44` | `GET /api/usb` total |
| printers | `printers WHERE isactive` | `GET /api/printers` total |
| applications | `applications` | `GET /api/applications?showhidden=true` total |
| knowledge base | `knowledgebase WHERE isactive` | `GET /api/knowledgebase` total |
| relationships | `machinerelationships WHERE isactive` | per-asset `GET /api/assets/{id}/relationships` |
| USB checkouts | `usbcheckouts` | `GET /api/usb/checkouts` |
Vendors, models, business units and the other straight lookups compare
one-to-one and need no clause.
Reference in New Issue View Git Blame Copy Permalink