shopdb-flask/docs/PILOT-DEPLOY.md

# Production pilot runbook (West Jefferson)

Goal: stand up a real shopdb-flask instance loaded with WJ's classic-ASP data,
run it **in parallel** with the classic app for a validation window, then cut
over. This runbook adds the legacy-data import + verification + cutover on top of
the generic stand-up in [`DEPLOY.md`](DEPLOY.md). Read that first; this only
calls out the pilot-specific steps.

Related: [`IMPORT-ADOPTION.md`](IMPORT-ADOPTION.md) (import model),
[`IMPORT-API.md`](IMPORT-API.md) (the contract), [`BACKUP-RESTORE.md`](BACKUP-RESTORE.md),
`scripts/site_imports/wjf/` (the loader).

---

## 0. Pre-flight checklist

- [ ] Host provisioned (Docker + compose, or a VM with Python 3 + MySQL 8).
- [ ] Three current classic dumps in hand: `shopdb` (main), `cmmc_usb`,
      `wjf_employees`. Take fresh dumps at import time - the classic app is live.
- [ ] Target MySQL 8, utf8mb4 (charset is contract, ADR-004). Old MySQL <5.7
      needs `innodb_large_prefix=ON` + Barracuda.
- [ ] Decide the pilot URL (e.g. `shopdb-pilot.example.net`) - separate
      from the classic app; do not reuse its hostname yet.
- [ ] Confirm the import decisions still hold (see the loader README / the
      import plan): assetnumber fallback + skip-dups, metrology routing,
      cmmc-only USB, warranties = Dell, occurrences parked.

## 1. Stand up the pilot instance

Follow `DEPLOY.md` steps 1-6 against a NEW empty database (name it clearly, e.g.
`shopdb_flask_pilot`):

```bash
flask db upgrade
flask plugin upgrade-all          # applies every plugin's chain
flask seed permissions
flask seed settings
flask seed reference-data          # seeds communicationtypes (IP) + the rest
```

**Enable every bundled plugin the site tracks - including usb**, which ships
disabled. A plugin's routes only register when it is enabled at app start, and
the importer needs them:

```bash
for p in computers employees machines measuringtools network notifications \
         printers slides usb warranty knowledgebase geenforce; do
  flask plugin enable "$p"
done
```

Do **not** run the setup wizard yet - the import fills the data the wizard would
otherwise ask you to seed.

## 2. Load the classic data

The loader (`scripts/site_imports/wjf/`) reads the classic dumps and drives the
import API. It is site glue, not product code.

1. Load the three dumps into scratch source DBs the loader can read (strip the
   `CREATE DATABASE`/`USE` lines so they land under scratch names, no clobber):

   ```bash
   for pair in "shopdb_src:shopdb_dump.sql" "cmmc_usb_src:cmmc_usb_dump.sql" \
               "wjf_employees_src:wjf_employees_dump.sql"; do
     db="${pair%%:*}"; f="${pair##*:}"
     mysql -h HOST -u root -p -e "CREATE DATABASE $db CHARACTER SET utf8mb4;"
     sed -E '/^CREATE DATABASE/d; /^USE `/d' "$f" | mysql -h HOST -u root -p "$db"
   done
   ```

2. Point the loader at the PILOT database and run all stages:

   ```bash
   DATABASE_URL='mysql+pymysql://USER:PW@HOST:3306/shopdb_flask_pilot?charset=utf8mb4' \
     venv/bin/python -m scripts.site_imports.wjf.run
   ```

   The 16 stages run in order (reference -> catalog -> assets hub -> locations ->
   printers -> dependents -> relationships -> subnets -> usb -> verify). It is
   idempotent - a crashed run resumes from `idmap.json`.

3. Reclassify servers into network devices. The classic DB stored servers as
   PCs, so the import lands them as `computer` assets. Re-point them in place:

   ```
   DATABASE_URL=... venv/bin/python scripts/reclassify_servers_to_network.py            # dry run, prints matches
   DATABASE_URL=... venv/bin/python scripts/reclassify_servers_to_network.py --commit   # apply
   # match on an exact computer type instead of the SVR- name prefix:
   ... --type "Server" --commit
   ```

   The assetid does not change: communications, relationships, map position, and
   audit history carry over. Only the extension row is swapped (computers ->
   networkdevices) and the asset type flipped; reclassified devices get the
   `Server` networkdevicetype. Run the dry run, eyeball the list, then commit.
   Re-running is safe (already-moved assets no longer match).

Expected magnitude (from the WJ dumps used in development - your fresh dumps will
differ slightly):

| entity | count |
|---|---|
| assets [*] | ~983 (computer ~663, machine ~76, network ~58, measuring-tool ~136, printer ~50) |
| locations | ~24 |
| employees | ~415 |
| installs | ~850 |
| primary IPs | ~461 |
| warranties | ~464 |
| notifications | ~261 |
| knowledge base | ~341 |
| relationships | ~93 |
| subnets | ~37 |
| USB devices / events | ~18 / ~232 |

[*] Counts taken AFTER `scripts/reclassify_servers_to_network.py --commit`.
Servers imported as computers are re-pointed to network devices, so the computer
count drops and network rises by the same amount versus a raw import.

> PLACEHOLDER - re-measure before publishing. The computer/network split shown
> in the assets row above still reflects a RAW import (pre-reclassify). Re-run
> the counts on the current prodscratch AFTER the reclassify step above and drop
> in the actual numbers; do not carry these development figures forward as if
> they already account for the reclassify.

The `verify` stage prints a source-vs-target row-count audit; the gaps are the
documented skips (inactive rows, duplicate machinenumbers, LocationOnly, the
9999 placeholder).

## 3. Verify the import

- [ ] Read the `verify` stage output - source vs target counts line up modulo
      the documented skips.
- [ ] Create the admin: `flask seed admin --username ... --email ...` (password
      printed once). Mark setup done so the app does not force the wizard:
      set `setup_complete=true` in settings (or click through the wizard,
      skipping the seed steps).
- [ ] UI spot-check (log in): Computers list paginates the full fleet; the Shop
      Floor Map plots assets, color-coded by type (positions came from
      mapleft/maptop); open a PC detail (installs), a printer (IP + share), an
      application (installed-on list), a KB article; check the employee
      directory; check a couple of asset relationships.
- [ ] Branding: upload the site logo + floor-plan blueprint under Settings, set
      facility name (Settings drive these per `CONFIG.md`).
- [ ] Photos are deferred - employees show initials until a photo batch is run.

## 4. Parallel-run window

- Keep the classic app authoritative during the window. The pilot is read-mostly
  for validation; do not dual-write.
- Have a few real users (IT + a floor lead) work the pilot and log gaps.
- Re-import is cheap: fix a loader mapping, drop + rebuild the pilot DB, re-run.
  Nothing you do to the pilot touches classic.
- Point the **collector** (GE-Enforce fleet ingest) at the pilot in parallel to
  confirm live PC check-ins land (see `COLLECTOR-INTEGRATION.md`), using a
  scoped service token.

## 5. Cutover

When the window is clean:

1. Freeze classic writes (announce a short read-only window).
2. Take final fresh dumps; re-run the loader into a clean pilot DB so the
   cutover data is current.
3. Verify counts + a fast UI spot-check.
4. Repoint the production hostname/DNS (or the reverse proxy) at the pilot.
5. Retire the classic app to read-only standby (do not delete - keep it as the
   rollback for the agreed period).

## 6. Rollback

- Pre-cutover: trivially point back at classic (it never stopped being
  authoritative).
- Post-cutover, within the standby window: repoint DNS/proxy back at classic;
  investigate; re-cut when fixed. Because the loader is deterministic and the
  classic DB is untouched, a re-run reproduces the flask DB exactly.

## 7. Post-cutover

- [ ] Backups on a schedule (`BACKUP-RESTORE.md`) - mysqldump + the `instance/`
      dir (uploaded logos, floor plans, tokens).
- [ ] Run the employee-photo batch.
- [ ] GE-Enforce: publish manifests + cut the fleet over to the flask endpoints
      when ready (`GE-ENFORCE-DEPLOY.md`) - independent of this pilot.
- [ ] Schedule the deferred data (occurrences, full communications fidelity)
      only if a real need appears.