ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Make main the lean product a site receives

Browse Source 
main was 16 days and 259 commits behind the line the product is actually built
on, at contract 0.30.0 against 0.35.0, with no per-PC credential code at all. It
was neither current nor curated - just stale. This brings it forward and, at the
same time, makes it mean something: main is what a receiving site installs, and
nothing else.

WHAT IT CARRIES. Twelve plugins - backups, computers, endpoints, guides,
knowledgebase, machines, network, notifications, printers, slides, tools,
warranty - plus applications, which is manifest-less core and always ships. The
only manifest dependency inside that set is endpoints on computers, and both are
in it, so the closure is self-contained.

WHAT IT DOES NOT. employees, geenforce, measuringtools, printedparts and usb
stay on the site line. So do the Windows installer tree, the VM rehearsal
helpers, the agent tooling config, one site's data loader, the internal audit
and findings records, and the planner spreadsheet. 1,135 tracked files against
1,438.

A REMOVAL THAT BREAKS THE APP IS NOT A REMOVAL, and two nearly shipped:

deploy/site-profile-universal.json still named all seventeen plugins. It is the
file docs/DEPLOY.md tells a site to provision from, so a fresh install would
have failed at `flask plugin apply-profile` with PluginError - the first thing a
receiving site does, broken by the branch meant to serve them. It now names the
twelve that exist, and its comment says the two must agree.

agent/ was never refreshed. Whoever populated this branch copied shopdb/,
plugins/, frontend/ and tests/ from the site line and left the agent tree at its
September state: 22 C# and PowerShell sources missing, including AgentCredentials,
EnrollmentService, AgentHttpFactory and the fleet-alert client, plus one file the
site line had deleted still present. The old tree is internally consistent, so it
compiles, and nothing in CI builds it - every gate was green over a server that
speaks enrollment and per-PC credentials to a bundled agent source that cannot.
Resynced whole.

DECISIONS THIS ENCODES, rather than defers:

- The plugin-authoring lab builds printedparts, whose manifest depends on
  employees, so it could not stay. The lab remains on the site line and main
  ships no build-along tutorial. That is a real loss for an adopting developer
  and is recorded here rather than discovered later.
- Recognition, Training and Recertification notifications are no longer seeded.
  Each splits one notification into a card per employee and renders a photo,
  which needs a plugin this branch does not bundle. Seeding them would offer an
  author a type that cannot resolve anybody. A site that installs employees gets
  them from that plugin's own seed.
- Windows/IIS is documented as the legacy path throughout, and the README leads
  with Docker, because that is what a receiving site gets.

The API description is regenerated for THIS branch - 347 paths and 483
operations, down from 415 and 573 - so the published contract describes what is
here rather than what the site line serves. docs/CONFIG.md lost the two
measuring-tool label keys the code no longer declares, which two lanes had
decided opposite ways.

Suite: 3124 passed, 6 skipped. Naming and style gates pass. The docs gates run
rather than skip, and every index line count is derived from the file it names.
This commit is contained in:
cproudlock
2026-09-19 11:24:40 -04:00
parent cb3975049c
commit 534f179ee6
675 changed files with 115759 additions and 60239 deletions
Download Patch File Download Diff File Expand all files Collapse all files

219
scripts/site_imports/wjf/PILOT-DEPLOY.md
View File

@@ -1,219 +0,0 @@
# Production pilot runbook (West Jefferson)
Goal: stand up a real shopdb-flask instance loaded with WJ's classic-ASP data,
run it **in parallel** with the classic app for a validation window, then cut
over. This runbook adds the legacy-data import + verification + cutover on top of
the generic stand-up in [`DEPLOY.md`](DEPLOY.md). Read that first; this only
calls out the pilot-specific steps.
Related: [`IMPORT-ADOPTION.md`](IMPORT-ADOPTION.md) (import model),
[`IMPORT-API.md`](IMPORT-API.md) (the contract), [`BACKUP-RESTORE.md`](BACKUP-RESTORE.md),
`scripts/site_imports/wjf/` (the loader).
---
## 0. Pre-flight checklist
- [ ] Host provisioned (Docker + compose, or a VM with Python 3 + MySQL 8).
- [ ] Three current classic dumps in hand: `shopdb` (main), `cmmc_usb`,
`wjf_employees`. Take fresh dumps at import time - the classic app is live.
- [ ] Target MySQL 8, utf8mb4 (charset is contract, ADR-004). Old MySQL <5.7
needs `innodb_large_prefix=ON` + Barracuda.
- [ ] Decide the pilot URL (e.g. `shopdb-pilot.example.net`) - separate
from the classic app; do not reuse its hostname yet.
- [ ] Confirm the import decisions still hold (see the loader README / the
import plan): assetnumber fallback + skip-dups, metrology routing,
cmmc-only USB, warranties = Dell, occurrences parked.
## 1. Stand up the pilot instance
Follow `DEPLOY.md` steps 1-6 against a NEW empty database (name it clearly, e.g.
`shopdb_flask_pilot`):
```bash
flask db upgrade
flask plugin upgrade-all # applies every plugin's chain
flask seed permissions
flask seed settings
flask seed reference-data # seeds communicationtypes (IP) + the rest
```
**Enable every bundled plugin the site tracks - including usb**, which ships
disabled. A plugin's routes only register when it is enabled at app start, and
the importer needs them:
```bash
# apply-profile installs AND enables, in dependency order, and runs each
# plugin's own migration chain. `plugin enable` alone refuses a plugin that is
# not installed yet, so on a fresh database the loop below exited 1 on every
# iteration and enabled nothing (ADR-013 records the same defect).
flask plugin apply-profile deploy/site-profile-universal.json
flask plugin upgrade-all
```
Do **not** run the setup wizard yet - the import fills the data the wizard would
otherwise ask you to seed.
## 2. Load the classic data
The loader (`scripts/site_imports/wjf/`) reads the classic dumps and drives the
import API. It is site glue, not product code.
1. Load the three dumps into scratch source DBs the loader can read (strip the
`CREATE DATABASE`/`USE` lines so they land under scratch names, no clobber):
```bash
for pair in "shopdb_src:shopdb_dump.sql" "cmmc_usb_src:cmmc_usb_dump.sql" \
"wjf_employees_src:wjf_employees_dump.sql"; do
db="${pair%%:*}"; f="${pair##*:}"
mysql -h HOST -u root -p -e "CREATE DATABASE $db CHARACTER SET utf8mb4;"
sed -E '/^CREATE DATABASE/d; /^USE `/d' "$f" | mysql -h HOST -u root -p "$db"
done
```
2. Point the loader at the PILOT database and run all stages:
```bash
DATABASE_URL='mysql+pymysql://USER:PW@HOST:3306/shopdb_flask_pilot?charset=utf8mb4' \
venv/bin/python -m scripts.site_imports.wjf.run
```
The 17 stages run in order (reference -> employees -> catalog -> assets hub
-> locations -> printers -> metrology -> communications -> applications ->
warranties -> notifications -> knowledgebase -> relationships ->
defaultprinters -> subnets -> usb -> verify).
**Pointing it at a different source.** The three source databases are
named logically in the loader and resolved from the environment, so a
restore under another name needs no code edit:
```
SOURCE_DB_MAIN=prodscratch \
SOURCE_DB_USB=cmmc_usb_src \
SOURCE_DB_EMPLOYEES=wjf_employees_src \
DATABASE_URL=... venv/bin/python -m scripts.site_imports.wjf.run
```
Also `SOURCE_DB_HOST`, `SOURCE_DB_PORT`, `SOURCE_DB_USER`,
`SOURCE_DB_PASSWORD`. The run prints every source it redirected, so
"it imported nothing" can be told apart from "it read the wrong database".
**Resume is per target database.** A crashed run resumes from the crosswalk
in `idmap.json`, which maps this site's legacy ids to assetids **in one
specific database**. The loader stamps that database's identity into the
file and refuses a crosswalk built against a different one, because reusing
it attaches warranties and printer assignments to whatever asset now holds
that id - wrong data, no error. If you rebuild the target, delete
`idmap.json` or pass `--idmap` a fresh path.
The exit code means something: non-zero if any endpoint errored, so a run
that logged 200 failures cannot read as a success in a deploy log.
3. Reclassify servers into network devices. The classic DB stored servers as
PCs, so the import lands them as `computer` assets. Re-point them in place:
```
DATABASE_URL=... venv/bin/python scripts/reclassify_servers_to_network.py # dry run, prints matches
DATABASE_URL=... venv/bin/python scripts/reclassify_servers_to_network.py --commit # apply
# match on an exact computer type instead of the SVR- name prefix:
... --type "Server" --commit
```
The assetid does not change: communications, relationships, map position, and
audit history carry over. Only the extension row is swapped (computers ->
networkdevices) and the asset type flipped; reclassified devices get the
`Server` networkdevicetype. Run the dry run, eyeball the list, then commit.
Re-running is safe (already-moved assets no longer match).
Expected magnitude (from the WJ dumps used in development - your fresh dumps will
differ slightly):
| entity | count |
|---|---|
| assets [*] | ~983 (computer ~663, machine ~76, network ~58, measuring-tool ~136, printer ~50) |
| locations | ~24 |
| employees | ~415 |
| installs | ~850 |
| primary IPs | ~461 |
| warranties | ~464 |
| notifications | ~261 |
| knowledge base | ~341 |
| relationships | ~93 |
| subnets | ~37 |
| USB devices / events | ~18 / ~232 |
[*] Counts taken AFTER `scripts/reclassify_servers_to_network.py --commit`.
Servers imported as computers are re-pointed to network devices, so the computer
count drops and network rises by the same amount versus a raw import.
> PLACEHOLDER - re-measure before publishing. The computer/network split shown
> in the assets row above still reflects a RAW import (pre-reclassify). Re-run
> the counts on the current prodscratch AFTER the reclassify step above and drop
> in the actual numbers; do not carry these development figures forward as if
> they already account for the reclassify.
The `verify` stage prints a source-vs-target row-count audit; the gaps are the
documented skips (inactive rows, duplicate machinenumbers, LocationOnly, the
9999 placeholder).
## 3. Verify the import
- [ ] Read the `verify` stage output - source vs target counts line up modulo
the documented skips.
- [ ] Create the admin: `flask seed admin --username ... --email ...` (password
printed once). Mark setup done so the app does not force the wizard:
set `setup_complete=true` in settings (or click through the wizard,
skipping the seed steps).
- [ ] UI spot-check (log in): Computers list paginates the full fleet; the Shop
Floor Map plots assets, color-coded by type (positions came from
mapleft/maptop); open a PC detail (installs), a printer (IP + share), an
application (installed-on list), a KB article; check the employee
directory; check a couple of asset relationships.
- [ ] Branding: upload the site logo + floor-plan blueprint under Settings, set
facility name (Settings drive these per `CONFIG.md`).
- [ ] Photos are deferred - employees show initials until a photo batch is run.
## 4. Parallel-run window
- Keep the classic app authoritative during the window. The pilot is read-mostly
for validation; do not dual-write.
- Have a few real users (IT + a floor lead) work the pilot and log gaps.
- Re-import is cheap: fix a loader mapping, drop + rebuild the pilot DB,
**delete `idmap.json`**, re-run. Nothing you do to the pilot touches classic.
The crosswalk is bound to the database it was built against and the loader
refuses a stale one, but deleting it is the habit to have - a refusal costs
you a restart, and before the guard existed a stale file silently attached
rows to the wrong assets.
- Point the **collector** (GE-Enforce fleet ingest) at the pilot in parallel to
confirm live PC check-ins land (see `COLLECTOR-INTEGRATION.md`), using a
scoped service token.
## 5. Cutover
When the window is clean:
1. Freeze classic writes (announce a short read-only window).
2. Take final fresh dumps; re-run the loader into a clean pilot DB so the
cutover data is current. A clean DB means a fresh `idmap.json` too.
3. Verify counts + a fast UI spot-check.
4. Repoint the production hostname/DNS (or the reverse proxy) at the pilot.
5. Retire the classic app to read-only standby (do not delete - keep it as the
rollback for the agreed period).
## 6. Rollback
- Pre-cutover: trivially point back at classic (it never stopped being
authoritative).
- Post-cutover, within the standby window: repoint DNS/proxy back at classic;
investigate; re-cut when fixed. Because the loader is deterministic and the
classic DB is untouched, a re-run reproduces the flask DB exactly.
## 7. Post-cutover
- [ ] Backups on a schedule (`BACKUP-RESTORE.md`) - mysqldump + the `instance/`
dir (uploaded logos, floor plans, tokens).
- [ ] Run the employee-photo batch.
- [ ] GE-Enforce: publish manifests + cut the fleet over to the flask endpoints
when ready (`GE-ENFORCE-DEPLOY.md`) - independent of this pilot.
- [ ] Schedule the deferred data (occurrences, full communications fidelity)
only if a real need appears.

66
scripts/site_imports/wjf/README.md
View File

@@ -1,66 +0,0 @@
# WJ classic-ASP import loader (reference implementation)
This is **site-specific reference glue**, not product code. It maps West
Jefferson's classic-ASP `shopdb` / `cmmc_usb` / `wjf_employees` schema onto the
maintained, schema-agnostic import contract in the IMPORT-API wiki page.
**Every adopting site has its own source database.** Nobody else runs this
loader. Instead, copy the pattern:
1. Point the harness at your source DB(s) (edit `harness.Source`).
2. Write per-entity stages that read your tables and POST to the same
the IMPORT-API wiki page endpoints with `Authorization: Bearer <admin PAT>` and
`X-Import-Mode: true`.
3. Persist legacy-id -> new-id crosswalks (see `harness.IdMap`) so later stages
resolve foreign keys and a crashed run resumes.
The import API is the stable contract; loaders are per-site. The mapping itself
is the bulk of the work: enumerate the source tables against this contract and
write one stage per entity (this loader is the worked example).
## Running (against a THROWAWAY import database)
```bash
DATABASE_URL='mysql+pymysql://root:PW@127.0.0.1:3306/shopdb_flask_import?charset=utf8mb4' \
venv/bin/python -m scripts.site_imports.wjf.run --stages reference,employees
```
Prereqs: a fresh target DB built with `flask db upgrade` + `flask plugin
upgrade-all` + `flask seed permissions/settings/reference-data`, and the three
source dumps loaded into scratch DBs (`shopdb_src`, `cmmc_usb_src`,
`wjf_employees_src`). See `scratchpad/IMPORT-PLAN.md` for the full mapping,
resolved decisions, and remaining stages.
## Status
All 17 stages built: `reference`, `employees`, `catalog`, `assets` (the hub +
machineid->assetid crosswalk), `locations`, `printers`, `metrology`,
`communications`, `applications` (+ installs), `warranties`, `notifications`,
`knowledgebase`, `relationships`, `defaultprinters`, `subnets`, `usb`, `verify`.
**The counts below are from the 2026-07-13 run and are STALE.** Two routing
changes landed after them - the `isactive=1` filter and the metrology routing
rework - so the asset mix has moved substantially. They are kept only as a
shape, not as a checklist: re-measure from the next full run before quoting
them anywhere, and do not verify a fresh import against them.
Last full run (2026-07-13, superseded): 983 assets (computer 663, machine 76,
network 58, measuring-tool 136, printer 50), 24 locations, 415 employees, 850
installs, 461 primary IPs, 464 warranties, 261 notifications, 341 KB, 93
relationships, 37 subnets, 18 USB devices + 232 events.
Known gaps, none of them blocking a scratch run:
- `warranties`, `notifications` and `knowledgebase` POST without a duplicate
check on the API side, so resuming ACROSS one of those stages doubles its
rows. Resuming at or after them is safe. Re-run into a clean target when in
doubt.
- There is no `--dry-run` and no guard on the target: the harness mints an
admin PAT and sets `setup_complete` in whatever `DATABASE_URL` points at.
Check the target before running.
- The `verify` stage counts target `assets` against source `machines`, which is
apples to oranges - the target also holds printers and synthesized metrology
tools.
Known follow-ups: general asset `locationid` is null (no source column outside
printers); the ~146 duplicate machinenumbers are first-wins-skipped by decision.

0
scripts/site_imports/wjf/__init__.py
View File

306
scripts/site_imports/wjf/harness.py
View File

@@ -1,306 +0,0 @@
"""Import harness for the WJ classic-ASP -> flask reference loader.
This is SITE-SPECIFIC reference glue, NOT product code. It maps West Jefferson's
classic schema onto the maintained, schema-agnostic import contract in
the IMPORT-API wiki page. Other sites copy this pattern against their own source DB;
they do not run this loader.
The harness builds the flask app against the current DATABASE_URL (point it at a
throwaway import database), mints an unscoped admin PAT in-process, and drives
the REAL import endpoints through the app test client with the X-Import-Mode
header - so it exercises the same routes/authz/validation an HTTP client would,
without needing a running server. Legacy-id -> new-id maps persist to JSON so a
crashed run resumes and later stages resolve foreign keys.
"""
import json
import uuid
import os
import pymysql
from shopdb import create_app
from shopdb.extensions import db
def _source_password():
"""Root password for the scratch source DBs: reuse the dev DATABASE_URL in
.env unless SOURCE_DB_PASSWORD overrides."""
if os.environ.get('SOURCE_DB_PASSWORD'):
return os.environ['SOURCE_DB_PASSWORD']
env = os.path.join(os.path.dirname(__file__), '..', '..', '..', '.env')
with open(os.path.abspath(env)) as handle:
for line in handle:
if line.startswith('DATABASE_URL='):
# Not every server runs as root, and 'root:' then simply is not
# there: split()[1] raised IndexError, which read as a crash
# rather than the clear message two lines down.
if 'root:' not in line:
continue
after = line.split('root:', 1)[1]
return after.split('@', 1)[0]
raise RuntimeError('could not resolve source DB password')
class Source:
"""Read-only access to the three source databases.
The stage functions name a source by its LOGICAL name - 'shopdb_src',
'cmmc_usb_src', 'wjf_employees_src' - and this class resolves each to a
real database. That indirection is the point: the names were hardcoded at
29 call sites, so pointing the loader at a differently-named copy (the
prodscratch restore, or a site's own dump) meant editing run.py, which is
not something anybody should do to aim a loader. The logical names stay in
the stages where they read as documentation; the mapping lives here.
Everything is environment-overridable, defaulting to the scratch layout:
SOURCE_DB_MAIN default shopdb_src
SOURCE_DB_USB default cmmc_usb_src
SOURCE_DB_EMPLOYEES default wjf_employees_src
SOURCE_DB_HOST/PORT/USER/PASSWORD
"""
# logical name -> (env var, default real name)
ALIASES = {
'shopdb_src': ('SOURCE_DB_MAIN', 'shopdb_src'),
'cmmc_usb_src': ('SOURCE_DB_USB', 'cmmc_usb_src'),
'wjf_employees_src': ('SOURCE_DB_EMPLOYEES', 'wjf_employees_src'),
}
def __init__(self, host=None, port=None, user=None, password=None):
self.password = password or _source_password()
self.host = host or os.environ.get('SOURCE_DB_HOST', '127.0.0.1')
self.port = int(port or os.environ.get('SOURCE_DB_PORT', 3306))
self.user = user or os.environ.get('SOURCE_DB_USER', 'root')
self._conns = {}
@classmethod
def resolve(cls, name):
"""The real database behind a logical source name."""
env, default = cls.ALIASES.get(name, (None, name))
return os.environ.get(env, default) if env else name
def db(self, name):
real = self.resolve(name)
if real not in self._conns:
self._conns[real] = pymysql.connect(
host=self.host, port=self.port, user=self.user,
password=self.password, database=real, charset='utf8mb4',
cursorclass=pymysql.cursors.DictCursor)
return self._conns[real]
def rows(self, dbname, sql, args=None):
with self.db(dbname).cursor() as cur:
cur.execute(sql, args or ())
return cur.fetchall()
class StaleIdMapError(RuntimeError):
"""The crosswalk on disk was built against a different target database."""
class IdMap:
"""Persisted legacy-id -> new-id crosswalks, one namespace per entity.
BOUND TO ONE TARGET DATABASE. The crosswalk holds this site's legacy ids
mapped to assetids in a specific database, and those assetids mean nothing
anywhere else. Two ways that used to go wrong, both silent:
- Delete idmap.json and re-run: every create 409s, nothing resolves into
the crosswalk, and warranties, installs, relationships and printer
assignments all quietly import as ~0 rows.
- Keep idmap.json across a drop-and-rebuild of the target: the legacy ids
now point at assetids belonging to the OLD database, so warranties and
printer assignments attach to whatever asset happens to hold that id
now. Wrong data, no error, and the runbook's two "drop and rebuild"
steps never said to delete the file.
The bind is a marker row in the target's own settings table, so it dies
with the data: a rebuild produces a new marker, and the mismatch is caught
before a single row is written. The database NAME is not enough on its own,
because a rebuild keeps it.
"""
MARKER_KEY = 'siteimport_targetinstance'
def __init__(self, path, target=None):
self.path = path
self.target = target
self._data = {}
if os.path.exists(path):
with open(path) as handle:
self._data = json.load(handle)
self._check_target()
def _check_target(self):
if not self.target:
return
seen = self._data.get('_target')
if seen is None:
# Written before this check existed. Adopt it rather than refuse:
# refusing would strand a crosswalk that is probably fine, and the
# next run is bound properly.
self._data['_target'] = self.target
self.save()
return
if seen != self.target:
raise StaleIdMapError(
'%s was built against target %s, but this run targets %s.\n'
'Those assetids belong to a different database and using them '
'would attach imported rows to the wrong assets.\n'
'Delete the file to start a fresh crosswalk, or point '
'--idmap at the one for this target.'
% (self.path, seen, self.target))
def put(self, namespace, legacy, new):
self._data.setdefault(namespace, {})[str(legacy)] = new
self.save()
def get(self, namespace, legacy, default=None):
return self._data.get(namespace, {}).get(str(legacy), default)
def has(self, namespace, legacy):
return str(legacy) in self._data.get(namespace, {})
def count(self, namespace):
entry = self._data.get(namespace, {})
return len(entry) if isinstance(entry, dict) else 0
def save(self):
with open(self.path, 'w') as handle:
json.dump(self._data, handle, indent=1)
class Harness:
"""App + admin PAT + import-mode test client + id maps."""
def __init__(self, idmap_path=None):
self.app = create_app()
self.appcontext = self.app.app_context()
self.appcontext.push()
self._silence_sql_logging()
self.client = self.app.test_client()
self._ensure_asset_types()
self.secret = self._mint_admin_pat()
default = os.path.join(os.path.dirname(__file__), 'idmap.json')
self.ids = IdMap(idmap_path or default, target=self._target_instance())
self.source = Source()
self.errors = []
self._defaultlevelid = None
def _target_instance(self):
"""A stable id for THIS target database, created once and stored in it.
Kept in the target's own settings table so it dies when the data does.
A drop-and-rebuild yields a new id, which is what lets IdMap catch a
crosswalk carried over from the previous database - the failure that
silently attaches imported rows to whatever asset now holds that id.
The database name alone would not do: a rebuild keeps the name.
"""
from shopdb.core.models import Setting
existing = Setting.query.filter_by(key=IdMap.MARKER_KEY).first()
if existing and existing.value:
return existing.value
marker = uuid.uuid4().hex
if existing:
existing.value = marker
else:
db.session.add(Setting(
key=IdMap.MARKER_KEY, value=marker, valuetype='string',
category='import',
description='Identifies this database to the site-import '
'crosswalk. Created by the loader; do not copy it '
'between databases.'))
db.session.commit()
return marker
@property
def defaultlevelid(self):
"""The level imported map positions belong to (ADR-017).
The legacy schema predates levels: it has ONE floor plan, so every
mapleft/maptop it carries is a coordinate on this site's default level.
Importing them without a level produces markers the map refuses to draw
- it will not guess a drawing for coordinates that do not name one.
"""
if self._defaultlevelid is None:
from shopdb.core.models import MapLevel
level = MapLevel.default_level()
self._defaultlevelid = level.levelid if level else None
return self._defaultlevelid
def _silence_sql_logging(self):
import logging
logging.getLogger('sqlalchemy.engine').setLevel(logging.WARNING)
def _ensure_asset_types(self):
"""Register each plugin's AssetType + seeded subtypes. on_install (which
does this) is skipped when a DB is built with `flask plugin upgrade-all`
rather than a fresh `install`, so run it idempotently here - the create
routes 500 without the AssetType row."""
manager = self.app.extensions.get('plugin_manager')
if not manager:
return
for _name, plugin in manager.get_all_plugins().items():
try:
plugin.on_install(self.app)
except Exception:
pass
# A full import IS the setup, so mark it complete - otherwise the admin
# is bounced to the first-run wizard on every fresh import DB.
from shopdb.core.models import Setting
Setting.set('setup_complete', 'true', valuetype='boolean', category='site')
db.session.commit()
def _mint_admin_pat(self):
"""Create (or reuse) an admin user and an unscoped admin PAT. Unscoped
because import mode + role gates require full owner authority
(the IMPORT-API wiki page). Returns the clear secret."""
from werkzeug.security import generate_password_hash
from shopdb.core.models import User, Role, ApiToken
admin = User.query.join(User.roles).filter(Role.rolename == 'admin').first()
if not admin:
role = Role.query.filter_by(rolename='admin').first() \
or Role(rolename='admin', description='Administrator')
db.session.add(role)
admin = User(username='importer', email='importer@localhost',
passwordhash=generate_password_hash('x'), isactive=True)
admin.roles.append(role)
db.session.add(admin)
db.session.commit()
secret = ApiToken.generate_secret()
token = ApiToken(userid=admin.userid, name='wjf legacy import',
tokenprefix=ApiToken.prefix_of(secret),
tokenhash=ApiToken.hash_secret(secret), scopes=None)
db.session.add(token)
db.session.commit()
return secret
def _headers(self, import_mode):
# A PAT authenticates as Authorization: Bearer shopdb_pat_... (the
# before_request shim resolves it into a JWT identity). X-API-Key is for
# collector/service tokens, not personal tokens.
headers = {'Authorization': f'Bearer {self.secret}'}
if import_mode:
headers['X-Import-Mode'] = 'true'
return headers
def post(self, path, payload, import_mode=True):
"""POST to an import endpoint. Returns (status, data). 409s are returned
(idempotency handled by the caller), other 4xx/5xx are recorded."""
resp = self.client.post(path, json=payload, headers=self._headers(import_mode))
body = resp.get_json() or {}
data = body.get('data', body)
if resp.status_code >= 400 and resp.status_code != 409:
self.errors.append((path, resp.status_code, payload, data))
return resp.status_code, data
def get(self, path):
resp = self.client.get(path, headers=self._headers(False))
body = resp.get_json() or {}
return resp.status_code, body.get('data', body)
def close(self):
self.ids.save()
self.appcontext.pop()

1032
scripts/site_imports/wjf/run.py
View File

File diff suppressed because it is too large Load Diff