Security hardening, multi-site portability, and a documentation and API-surface sweep
This commit is contained in:
188
scripts/site_imports/wjf/PILOT-DEPLOY.md
Normal file
188
scripts/site_imports/wjf/PILOT-DEPLOY.md
Normal file
@@ -0,0 +1,188 @@
|
||||
# Production pilot runbook (West Jefferson)
|
||||
|
||||
Goal: stand up a real shopdb-flask instance loaded with WJ's classic-ASP data,
|
||||
run it **in parallel** with the classic app for a validation window, then cut
|
||||
over. This runbook adds the legacy-data import + verification + cutover on top of
|
||||
the generic stand-up in [`DEPLOY.md`](DEPLOY.md). Read that first; this only
|
||||
calls out the pilot-specific steps.
|
||||
|
||||
Related: [`IMPORT-ADOPTION.md`](IMPORT-ADOPTION.md) (import model),
|
||||
[`IMPORT-API.md`](IMPORT-API.md) (the contract), [`BACKUP-RESTORE.md`](BACKUP-RESTORE.md),
|
||||
`scripts/site_imports/wjf/` (the loader).
|
||||
|
||||
---
|
||||
|
||||
## 0. Pre-flight checklist
|
||||
|
||||
- [ ] Host provisioned (Docker + compose, or a VM with Python 3 + MySQL 8).
|
||||
- [ ] Three current classic dumps in hand: `shopdb` (main), `cmmc_usb`,
|
||||
`wjf_employees`. Take fresh dumps at import time - the classic app is live.
|
||||
- [ ] Target MySQL 8, utf8mb4 (charset is contract, ADR-004). Old MySQL <5.7
|
||||
needs `innodb_large_prefix=ON` + Barracuda.
|
||||
- [ ] Decide the pilot URL (e.g. `shopdb-pilot.example.net`) - separate
|
||||
from the classic app; do not reuse its hostname yet.
|
||||
- [ ] Confirm the import decisions still hold (see the loader README / the
|
||||
import plan): assetnumber fallback + skip-dups, metrology routing,
|
||||
cmmc-only USB, warranties = Dell, occurrences parked.
|
||||
|
||||
## 1. Stand up the pilot instance
|
||||
|
||||
Follow `DEPLOY.md` steps 1-6 against a NEW empty database (name it clearly, e.g.
|
||||
`shopdb_flask_pilot`):
|
||||
|
||||
```bash
|
||||
flask db upgrade
|
||||
flask plugin upgrade-all # applies every plugin's chain
|
||||
flask seed permissions
|
||||
flask seed settings
|
||||
flask seed reference-data # seeds communicationtypes (IP) + the rest
|
||||
```
|
||||
|
||||
**Enable every bundled plugin the site tracks - including usb**, which ships
|
||||
disabled. A plugin's routes only register when it is enabled at app start, and
|
||||
the importer needs them:
|
||||
|
||||
```bash
|
||||
# apply-profile installs AND enables, in dependency order, and runs each
|
||||
# plugin's own migration chain. `plugin enable` alone refuses a plugin that is
|
||||
# not installed yet, so on a fresh database the loop below exited 1 on every
|
||||
# iteration and enabled nothing (ADR-013 records the same defect).
|
||||
flask plugin apply-profile deploy/site-profile-universal.json
|
||||
flask plugin upgrade-all
|
||||
```
|
||||
|
||||
Do **not** run the setup wizard yet - the import fills the data the wizard would
|
||||
otherwise ask you to seed.
|
||||
|
||||
## 2. Load the classic data
|
||||
|
||||
The loader (`scripts/site_imports/wjf/`) reads the classic dumps and drives the
|
||||
import API. It is site glue, not product code.
|
||||
|
||||
1. Load the three dumps into scratch source DBs the loader can read (strip the
|
||||
`CREATE DATABASE`/`USE` lines so they land under scratch names, no clobber):
|
||||
|
||||
```bash
|
||||
for pair in "shopdb_src:shopdb_dump.sql" "cmmc_usb_src:cmmc_usb_dump.sql" \
|
||||
"wjf_employees_src:wjf_employees_dump.sql"; do
|
||||
db="${pair%%:*}"; f="${pair##*:}"
|
||||
mysql -h HOST -u root -p -e "CREATE DATABASE $db CHARACTER SET utf8mb4;"
|
||||
sed -E '/^CREATE DATABASE/d; /^USE `/d' "$f" | mysql -h HOST -u root -p "$db"
|
||||
done
|
||||
```
|
||||
|
||||
2. Point the loader at the PILOT database and run all stages:
|
||||
|
||||
```bash
|
||||
DATABASE_URL='mysql+pymysql://USER:PW@HOST:3306/shopdb_flask_pilot?charset=utf8mb4' \
|
||||
venv/bin/python -m scripts.site_imports.wjf.run
|
||||
```
|
||||
|
||||
The 16 stages run in order (reference -> catalog -> assets hub -> locations ->
|
||||
printers -> dependents -> relationships -> subnets -> usb -> verify). It is
|
||||
idempotent - a crashed run resumes from `idmap.json`.
|
||||
|
||||
3. Reclassify servers into network devices. The classic DB stored servers as
|
||||
PCs, so the import lands them as `computer` assets. Re-point them in place:
|
||||
|
||||
```
|
||||
DATABASE_URL=... venv/bin/python scripts/reclassify_servers_to_network.py # dry run, prints matches
|
||||
DATABASE_URL=... venv/bin/python scripts/reclassify_servers_to_network.py --commit # apply
|
||||
# match on an exact computer type instead of the SVR- name prefix:
|
||||
... --type "Server" --commit
|
||||
```
|
||||
|
||||
The assetid does not change: communications, relationships, map position, and
|
||||
audit history carry over. Only the extension row is swapped (computers ->
|
||||
networkdevices) and the asset type flipped; reclassified devices get the
|
||||
`Server` networkdevicetype. Run the dry run, eyeball the list, then commit.
|
||||
Re-running is safe (already-moved assets no longer match).
|
||||
|
||||
Expected magnitude (from the WJ dumps used in development - your fresh dumps will
|
||||
differ slightly):
|
||||
|
||||
| entity | count |
|
||||
|---|---|
|
||||
| assets [*] | ~983 (computer ~663, machine ~76, network ~58, measuring-tool ~136, printer ~50) |
|
||||
| locations | ~24 |
|
||||
| employees | ~415 |
|
||||
| installs | ~850 |
|
||||
| primary IPs | ~461 |
|
||||
| warranties | ~464 |
|
||||
| notifications | ~261 |
|
||||
| knowledge base | ~341 |
|
||||
| relationships | ~93 |
|
||||
| subnets | ~37 |
|
||||
| USB devices / events | ~18 / ~232 |
|
||||
|
||||
[*] Counts taken AFTER `scripts/reclassify_servers_to_network.py --commit`.
|
||||
Servers imported as computers are re-pointed to network devices, so the computer
|
||||
count drops and network rises by the same amount versus a raw import.
|
||||
|
||||
> PLACEHOLDER - re-measure before publishing. The computer/network split shown
|
||||
> in the assets row above still reflects a RAW import (pre-reclassify). Re-run
|
||||
> the counts on the current prodscratch AFTER the reclassify step above and drop
|
||||
> in the actual numbers; do not carry these development figures forward as if
|
||||
> they already account for the reclassify.
|
||||
|
||||
The `verify` stage prints a source-vs-target row-count audit; the gaps are the
|
||||
documented skips (inactive rows, duplicate machinenumbers, LocationOnly, the
|
||||
9999 placeholder).
|
||||
|
||||
## 3. Verify the import
|
||||
|
||||
- [ ] Read the `verify` stage output - source vs target counts line up modulo
|
||||
the documented skips.
|
||||
- [ ] Create the admin: `flask seed admin --username ... --email ...` (password
|
||||
printed once). Mark setup done so the app does not force the wizard:
|
||||
set `setup_complete=true` in settings (or click through the wizard,
|
||||
skipping the seed steps).
|
||||
- [ ] UI spot-check (log in): Computers list paginates the full fleet; the Shop
|
||||
Floor Map plots assets, color-coded by type (positions came from
|
||||
mapleft/maptop); open a PC detail (installs), a printer (IP + share), an
|
||||
application (installed-on list), a KB article; check the employee
|
||||
directory; check a couple of asset relationships.
|
||||
- [ ] Branding: upload the site logo + floor-plan blueprint under Settings, set
|
||||
facility name (Settings drive these per `CONFIG.md`).
|
||||
- [ ] Photos are deferred - employees show initials until a photo batch is run.
|
||||
|
||||
## 4. Parallel-run window
|
||||
|
||||
- Keep the classic app authoritative during the window. The pilot is read-mostly
|
||||
for validation; do not dual-write.
|
||||
- Have a few real users (IT + a floor lead) work the pilot and log gaps.
|
||||
- Re-import is cheap: fix a loader mapping, drop + rebuild the pilot DB, re-run.
|
||||
Nothing you do to the pilot touches classic.
|
||||
- Point the **collector** (GE-Enforce fleet ingest) at the pilot in parallel to
|
||||
confirm live PC check-ins land (see `COLLECTOR-INTEGRATION.md`), using a
|
||||
scoped service token.
|
||||
|
||||
## 5. Cutover
|
||||
|
||||
When the window is clean:
|
||||
|
||||
1. Freeze classic writes (announce a short read-only window).
|
||||
2. Take final fresh dumps; re-run the loader into a clean pilot DB so the
|
||||
cutover data is current.
|
||||
3. Verify counts + a fast UI spot-check.
|
||||
4. Repoint the production hostname/DNS (or the reverse proxy) at the pilot.
|
||||
5. Retire the classic app to read-only standby (do not delete - keep it as the
|
||||
rollback for the agreed period).
|
||||
|
||||
## 6. Rollback
|
||||
|
||||
- Pre-cutover: trivially point back at classic (it never stopped being
|
||||
authoritative).
|
||||
- Post-cutover, within the standby window: repoint DNS/proxy back at classic;
|
||||
investigate; re-cut when fixed. Because the loader is deterministic and the
|
||||
classic DB is untouched, a re-run reproduces the flask DB exactly.
|
||||
|
||||
## 7. Post-cutover
|
||||
|
||||
- [ ] Backups on a schedule (`BACKUP-RESTORE.md`) - mysqldump + the `instance/`
|
||||
dir (uploaded logos, floor plans, tokens).
|
||||
- [ ] Run the employee-photo batch.
|
||||
- [ ] GE-Enforce: publish manifests + cut the fleet over to the flask endpoints
|
||||
when ready (`GE-ENFORCE-DEPLOY.md`) - independent of this pilot.
|
||||
- [ ] Schedule the deferred data (occurrences, full communications fidelity)
|
||||
only if a real need appears.
|
||||
Reference in New Issue
Block a user