docs: take one site's name, hosts and paths off the public wiki
The publishability gate caught internal tooling names and developer paths but nothing site-specific, so roughly sixty leaks reached the wiki: the site name in ten documents, real fleet hostnames in the collector and GE-Enforce examples, an internal database name through the whole import guide, imaging-share paths, and a maintainer's username as the Deciders line of every ADR and inside a generated curl example. None of it is a security matter on an air-gapped fleet. It matters because these pages are read by engineers at other plants, and a document that names one site throughout reads as that site's notes rather than a product's documentation - which is exactly what it then gets treated as. Examples now use neutral hostnames, the site is "the reference site" where the distinction carries meaning, and ADRs are decided by "ShopDB maintainers". The gate carries all of these patterns, so the next one fails a build. Two documents leave docs/ because they were never written for an outside reader. PROJECT-REVIEW.md is an internal health memo pinned to a commit from July, whose headline finding (an untracked playbook) has since been fixed - it is history, and git holds it. PILOT-DEPLOY.md is one site's own cutover runbook, complete with a "re-measure before publishing" placeholder; it moves next to the loader it belongs to, in scripts/site_imports/wjf/. ADR-015 is AMENDED rather than rewritten. Its enforcement section still said report-only and its backlog still listed hardcodes that are now cleared, which left the record contradicting itself. The amendment says what changed and why the report-only period ended; the original text stays, because what the decision looked like when it was taken is the part worth keeping. Also corrects llms.txt's response envelope, which had errors at the top level and pagination at meta.total. Both are nested one deeper, so anything written against that description read undefined on every error it tried to handle.
This commit is contained in:
cproudlock
186
scripts/site_imports/wjf/PILOT-DEPLOY.md
Normal file
186
scripts/site_imports/wjf/PILOT-DEPLOY.md
Normal file
@@ -0,0 +1,186 @@
|
||||
# Production pilot runbook (West Jefferson)
|
||||
|
||||
Goal: stand up a real shopdb-flask instance loaded with WJ's classic-ASP data,
|
||||
run it **in parallel** with the classic app for a validation window, then cut
|
||||
over. This runbook adds the legacy-data import + verification + cutover on top of
|
||||
the generic stand-up in [`DEPLOY.md`](DEPLOY.md). Read that first; this only
|
||||
calls out the pilot-specific steps.
|
||||
|
||||
Related: [`IMPORT-ADOPTION.md`](IMPORT-ADOPTION.md) (import model),
|
||||
[`IMPORT-API.md`](IMPORT-API.md) (the contract), [`BACKUP-RESTORE.md`](BACKUP-RESTORE.md),
|
||||
`scripts/site_imports/wjf/` (the loader).
|
||||
|
||||
---
|
||||
|
||||
## 0. Pre-flight checklist
|
||||
|
||||
- [ ] Host provisioned (Docker + compose, or a VM with Python 3 + MySQL 8).
|
||||
- [ ] Three current classic dumps in hand: `shopdb` (main), `cmmc_usb`,
|
||||
`wjf_employees`. Take fresh dumps at import time - the classic app is live.
|
||||
- [ ] Target MySQL 8, utf8mb4 (charset is contract, ADR-004). Old MySQL <5.7
|
||||
needs `innodb_large_prefix=ON` + Barracuda.
|
||||
- [ ] Decide the pilot URL (e.g. `shopdb-pilot.example.net`) - separate
|
||||
from the classic app; do not reuse its hostname yet.
|
||||
- [ ] Confirm the import decisions still hold (see the loader README / the
|
||||
import plan): assetnumber fallback + skip-dups, metrology routing,
|
||||
cmmc-only USB, warranties = Dell, occurrences parked.
|
||||
|
||||
## 1. Stand up the pilot instance
|
||||
|
||||
Follow `DEPLOY.md` steps 1-6 against a NEW empty database (name it clearly, e.g.
|
||||
`shopdb_flask_pilot`):
|
||||
|
||||
```bash
|
||||
flask db upgrade
|
||||
flask plugin upgrade-all # applies every plugin's chain
|
||||
flask seed permissions
|
||||
flask seed settings
|
||||
flask seed reference-data # seeds communicationtypes (IP) + the rest
|
||||
```
|
||||
|
||||
**Enable every bundled plugin the site tracks - including usb**, which ships
|
||||
disabled. A plugin's routes only register when it is enabled at app start, and
|
||||
the importer needs them:
|
||||
|
||||
```bash
|
||||
for p in computers employees machines measuringtools network notifications \
|
||||
printers slides usb warranty knowledgebase geenforce; do
|
||||
flask plugin enable "$p"
|
||||
done
|
||||
```
|
||||
|
||||
Do **not** run the setup wizard yet - the import fills the data the wizard would
|
||||
otherwise ask you to seed.
|
||||
|
||||
## 2. Load the classic data
|
||||
|
||||
The loader (`scripts/site_imports/wjf/`) reads the classic dumps and drives the
|
||||
import API. It is site glue, not product code.
|
||||
|
||||
1. Load the three dumps into scratch source DBs the loader can read (strip the
|
||||
`CREATE DATABASE`/`USE` lines so they land under scratch names, no clobber):
|
||||
|
||||
```bash
|
||||
for pair in "shopdb_src:shopdb_dump.sql" "cmmc_usb_src:cmmc_usb_dump.sql" \
|
||||
"wjf_employees_src:wjf_employees_dump.sql"; do
|
||||
db="${pair%%:*}"; f="${pair##*:}"
|
||||
mysql -h HOST -u root -p -e "CREATE DATABASE $db CHARACTER SET utf8mb4;"
|
||||
sed -E '/^CREATE DATABASE/d; /^USE `/d' "$f" | mysql -h HOST -u root -p "$db"
|
||||
done
|
||||
```
|
||||
|
||||
2. Point the loader at the PILOT database and run all stages:
|
||||
|
||||
```bash
|
||||
DATABASE_URL='mysql+pymysql://USER:PW@HOST:3306/shopdb_flask_pilot?charset=utf8mb4' \
|
||||
venv/bin/python -m scripts.site_imports.wjf.run
|
||||
```
|
||||
|
||||
The 16 stages run in order (reference -> catalog -> assets hub -> locations ->
|
||||
printers -> dependents -> relationships -> subnets -> usb -> verify). It is
|
||||
idempotent - a crashed run resumes from `idmap.json`.
|
||||
|
||||
3. Reclassify servers into network devices. The classic DB stored servers as
|
||||
PCs, so the import lands them as `computer` assets. Re-point them in place:
|
||||
|
||||
```
|
||||
DATABASE_URL=... venv/bin/python scripts/reclassify_servers_to_network.py # dry run, prints matches
|
||||
DATABASE_URL=... venv/bin/python scripts/reclassify_servers_to_network.py --commit # apply
|
||||
# match on an exact computer type instead of the SVR- name prefix:
|
||||
... --type "Server" --commit
|
||||
```
|
||||
|
||||
The assetid does not change: communications, relationships, map position, and
|
||||
audit history carry over. Only the extension row is swapped (computers ->
|
||||
networkdevices) and the asset type flipped; reclassified devices get the
|
||||
`Server` networkdevicetype. Run the dry run, eyeball the list, then commit.
|
||||
Re-running is safe (already-moved assets no longer match).
|
||||
|
||||
Expected magnitude (from the WJ dumps used in development - your fresh dumps will
|
||||
differ slightly):
|
||||
|
||||
| entity | count |
|
||||
|---|---|
|
||||
| assets [*] | ~983 (computer ~663, machine ~76, network ~58, measuring-tool ~136, printer ~50) |
|
||||
| locations | ~24 |
|
||||
| employees | ~415 |
|
||||
| installs | ~850 |
|
||||
| primary IPs | ~461 |
|
||||
| warranties | ~464 |
|
||||
| notifications | ~261 |
|
||||
| knowledge base | ~341 |
|
||||
| relationships | ~93 |
|
||||
| subnets | ~37 |
|
||||
| USB devices / events | ~18 / ~232 |
|
||||
|
||||
[*] Counts taken AFTER `scripts/reclassify_servers_to_network.py --commit`.
|
||||
Servers imported as computers are re-pointed to network devices, so the computer
|
||||
count drops and network rises by the same amount versus a raw import.
|
||||
|
||||
> PLACEHOLDER - re-measure before publishing. The computer/network split shown
|
||||
> in the assets row above still reflects a RAW import (pre-reclassify). Re-run
|
||||
> the counts on the current prodscratch AFTER the reclassify step above and drop
|
||||
> in the actual numbers; do not carry these development figures forward as if
|
||||
> they already account for the reclassify.
|
||||
|
||||
The `verify` stage prints a source-vs-target row-count audit; the gaps are the
|
||||
documented skips (inactive rows, duplicate machinenumbers, LocationOnly, the
|
||||
9999 placeholder).
|
||||
|
||||
## 3. Verify the import
|
||||
|
||||
- [ ] Read the `verify` stage output - source vs target counts line up modulo
|
||||
the documented skips.
|
||||
- [ ] Create the admin: `flask seed admin --username ... --email ...` (password
|
||||
printed once). Mark setup done so the app does not force the wizard:
|
||||
set `setup_complete=true` in settings (or click through the wizard,
|
||||
skipping the seed steps).
|
||||
- [ ] UI spot-check (log in): Computers list paginates the full fleet; the Shop
|
||||
Floor Map plots assets, color-coded by type (positions came from
|
||||
mapleft/maptop); open a PC detail (installs), a printer (IP + share), an
|
||||
application (installed-on list), a KB article; check the employee
|
||||
directory; check a couple of asset relationships.
|
||||
- [ ] Branding: upload the site logo + floor-plan blueprint under Settings, set
|
||||
facility name (Settings drive these per `CONFIG.md`).
|
||||
- [ ] Photos are deferred - employees show initials until a photo batch is run.
|
||||
|
||||
## 4. Parallel-run window
|
||||
|
||||
- Keep the classic app authoritative during the window. The pilot is read-mostly
|
||||
for validation; do not dual-write.
|
||||
- Have a few real users (IT + a floor lead) work the pilot and log gaps.
|
||||
- Re-import is cheap: fix a loader mapping, drop + rebuild the pilot DB, re-run.
|
||||
Nothing you do to the pilot touches classic.
|
||||
- Point the **collector** (GE-Enforce fleet ingest) at the pilot in parallel to
|
||||
confirm live PC check-ins land (see `COLLECTOR-INTEGRATION.md`), using a
|
||||
scoped service token.
|
||||
|
||||
## 5. Cutover
|
||||
|
||||
When the window is clean:
|
||||
|
||||
1. Freeze classic writes (announce a short read-only window).
|
||||
2. Take final fresh dumps; re-run the loader into a clean pilot DB so the
|
||||
cutover data is current.
|
||||
3. Verify counts + a fast UI spot-check.
|
||||
4. Repoint the production hostname/DNS (or the reverse proxy) at the pilot.
|
||||
5. Retire the classic app to read-only standby (do not delete - keep it as the
|
||||
rollback for the agreed period).
|
||||
|
||||
## 6. Rollback
|
||||
|
||||
- Pre-cutover: trivially point back at classic (it never stopped being
|
||||
authoritative).
|
||||
- Post-cutover, within the standby window: repoint DNS/proxy back at classic;
|
||||
investigate; re-cut when fixed. Because the loader is deterministic and the
|
||||
classic DB is untouched, a re-run reproduces the flask DB exactly.
|
||||
|
||||
## 7. Post-cutover
|
||||
|
||||
- [ ] Backups on a schedule (`BACKUP-RESTORE.md`) - mysqldump + the `instance/`
|
||||
dir (uploaded logos, floor plans, tokens).
|
||||
- [ ] Run the employee-photo batch.
|
||||
- [ ] GE-Enforce: publish manifests + cut the fleet over to the flask endpoints
|
||||
when ready (`GE-ENFORCE-DEPLOY.md`) - independent of this pilot.
|
||||
- [ ] Schedule the deferred data (occurrences, full communications fidelity)
|
||||
only if a real need appears.
|
||||
Reference in New Issue
Block a user