ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Bring the wilm branch into main: endpoints plugin, agent, policies, CSV import

Browse Source 
Ports the corporate-GitHub wilm branch onto main, keeping the two branches
separate. Thirteen upstream commits arrive with their authorship intact; the
review fixes and the portability work below are on top.

What arrives:

  endpoints plugin  - a ShopDB agent reporting endpoint state, telemetry,
                      services and interfaces, with a command channel the agent
                      long-polls. Ingest stays on the generic collector route
                      (ADR-006); the agent key is a UI-managed single-scope
                      token whose revocation is an instant kill switch.
  computers policies - Configuration and Business Policies (RMM-style), scope
                      trees validated at write time, actions checked against
                      the endpoints command allowlist. Feature-gated: every
                      route 404s while the setting is off, so enabling it can
                      never half-expose the surface.
  asset CSV import  - templates, a collector script carrying the site path, and
                      a validate-then-commit import. A plugin can own a virtual
                      column plus a post-write row hook, which is how a PC's
                      machine number becomes a controls link without a
                      hand-built relationships sheet.
  agent/            - the Windows agent and its installer.

Contract 0.21.0 -> 0.24.0, additive throughout: register_import_hook (0.22),
mint/list/revoke_service_token (0.23), collector_authorized (0.24). The stability
table had no rows for any of them and now does.

MySQL 5.6 portability. The branch stored three policy columns as sa.JSON, the
only JSON columns in the product, and 5.6 has no JSON type: the migration failed
with a bare syntax error near the word JSON. They are TEXT now, through a
JSONText type decorator, so all seventeen call sites still read and write dicts.
The convention elsewhere is already TEXT (guides.blocksjson,
geenforce.manifestjson).

Review fixes carried on top:

  - six policy GETs were jwt_required(optional=True), which authenticates a
    caller without checking anything; they require computers.view now, and six
    redundant jwt_required decorators are gone
  - the agent installer took a default BaseUrl and CollectorKey, so a mistyped
    run silently pointed an endpoint at the wrong instance; both are mandatory
    and a non-https BaseUrl throws unless -AllowInsecureHttp is passed
  - a committing CSV import wrote no audit row

Documentation: the 34 new routes are described in docs/api-inventory.json, which
is hand-maintained on purpose, and the generated spec now covers 331 paths and
477 operations across 22 surfaces. DEVELOPMENT-SETUP's 1071 troubleshooting row
said "use MySQL 8 for dev" and now gives the actual fix, including the trap that
5.6 downgrades ROW_FORMAT=DYNAMIC to Compact with a warning instead of an error.

Full suite 2123 passed, vitest 260 passed, all seventeen plugin chains migrate
clean on MySQL 5.6 and the naming, docs, parity and read-surface gates are green.
This commit is contained in:
cproudlock
2026-08-28 16:17:17 -04:00
parent 93ca7587ce
commit 784964c907
149 changed files with 17545 additions and 627 deletions
Download Patch File Download Diff File Expand all files Collapse all files

22
README.md
View File

@@ -22,7 +22,7 @@ ShopDB tracks and manages:
**Backend:**
- Python 3.14 with Flask
- SQLAlchemy ORM
- MySQL 5.7+ database (5.6 works with extra utf8mb4 config; see docs/DEPLOY.md)
- MySQL 5.7+ database (5.6 works with extra utf8mb4 config; see the DEPLOY wiki page)
- JWT authentication
- Plugin architecture for extensibility
@@ -100,7 +100,7 @@ To maintain consistency with the legacy ShopDB database and codebase, the follow
- Python 3.14
- Node.js 18+
- MySQL 5.7+ (5.6 works with extra utf8mb4 config; see docs/DEPLOY.md)
- MySQL 5.7+ (5.6 works with extra utf8mb4 config; see the DEPLOY wiki page)
ShopDB Flask uses MySQL as the canonical database. SQLite is used only for the
test suite (`TestingConfig` in `shopdb/config.py` points at an in-memory
@@ -121,7 +121,7 @@ so a container deploy needs no separate Node build step.
```bash
cp .env.example .env
# Edit .env: set SECRET_KEY, JWT_SECRET_KEY, DATABASE_URL, CORS_ORIGINS,
# and the MYSQL_* passwords (see docs/CONFIG.md for every variable).
# and the MYSQL_* passwords (see the CONFIG wiki page for every variable).
docker compose up -d --build
@@ -172,7 +172,7 @@ including VS Code and troubleshooting, is the DEVELOPMENT-SETUP page in the
project wiki.
To import a site's legacy data, use the HTTP import surface: an admin API
token plus [docs/IMPORT-API.md](docs/IMPORT-API.md) drive the whole migration
token plus the IMPORT-API page in the project wiki drive the whole migration
through documented endpoints (`X-Import-Mode` preserves original timestamps).
`scripts/site_imports/wjf/` is the reference site's loader, kept as a worked
example of the whole sequence.
@@ -181,11 +181,11 @@ example of the whole sequence.
| Target | Use |
|---|---|
| **Windows Server + IIS** (how sister sites run) | **[docs/INSTALL-WINDOWS.md](docs/INSTALL-WINDOWS.md)** - one installer `.exe`, offline, no manual IIS work. Day 2: [docs/OPERATE-WINDOWS.md](docs/OPERATE-WINDOWS.md) |
| Linux / Docker, air-gapped | [docs/DEPLOY-AIRGAP.md](docs/DEPLOY-AIRGAP.md) |
| Linux / Docker, connected | [docs/DEPLOY.md](docs/DEPLOY.md) |
| **Windows Server + IIS** (how sister sites run) | **the INSTALL-WINDOWS page in the project wiki** - one installer `.exe`, offline, no manual IIS work. Day 2: the OPERATE-WINDOWS page in the project wiki |
| Linux / Docker, air-gapped | the DEPLOY-AIRGAP page in the project wiki |
| Linux / Docker, connected | the DEPLOY page in the project wiki |
For every environment variable and Setting key see [docs/CONFIG.md](docs/CONFIG.md).
For every environment variable and Setting key see the CONFIG page in the project wiki.
### What the Windows installer requires
@@ -205,7 +205,7 @@ The binding constraint is the operating system rather than IIS: the payload is
Python 3.14 plus HttpPlatformHandler and URL Rewrite, and the handler itself runs
on older IIS. An older host is untested rather than known-broken.
If you must deploy onto something older, [docs/INSTALL-WINDOWS-IIS.md](docs/INSTALL-WINDOWS-IIS.md)
If you must deploy onto something older, the INSTALL-WINDOWS-IIS page in the project wiki
is the manual procedure and has no OS gate - it even documents MySQL 5.6, which is
that era of machine. The trade is stated there: it produces a server the
installer will not subsequently upgrade.
@@ -248,7 +248,7 @@ Query parameters for list endpoints:
## Where to start
[docs/START-HERE.md](docs/START-HERE.md) routes by what you are here to do -
the START-HERE page in the project wiki routes by what you are here to do -
standing up a site, deploying the shop-floor tools, writing a plugin,
integrating with the API, or diagnosing something. On the published mirror the
same page is the wiki's front door.
@@ -259,7 +259,7 @@ ShopDB supports plugins for extending functionality. See `CONTRIBUTING.md` for p
The image bundles these plugins; only the ones a site installs are loaded. The
current count and each plugin's version and migration head are in
[docs/PROJECT-MAP.md](docs/PROJECT-MAP.md), which is generated:
the PROJECT-MAP page in the project wiki, which is generated:
- **backups** - Machine configuration backups (NTLARS/DNC, CMM, part marker, UDC) with revision history and diffs
- **computers** - Shopfloor PCs and workstations, collector fleet ingest