Plugin framework maturation, reports overhaul, theming, and USB frontend repair
Framework: - Per-plugin Alembic migration chains (ADR-008): every bundled plugin carries its own chain with a stamp-only anchor at the ownership cutover; new plugin schema lands in plugins/<name>/migrations/, never the core chain. Deploys add flask plugin upgrade-all. Fixed a latent bug in the shared alembic template (engine URL resolution) and taught the metadata filter to include FK-referenced core tables. - Frontend plugin route gating (ADR-009): plugin routes carry meta.plugin; a disabled plugin's pages redirect to the dashboard via a cached, fail-open check against the new public GET /api/plugins/enabled. - get_reports() plugin hook (contract 0.5.0 -> 0.6.0): plugins contribute report cards; warranty and toner cards moved off the hardcoded list. Reports: - Hub grouped by category with search; inline reports render at the top, are URL-backed (?report=id, back-button and deep links work), expose their server-side filter params as controls, and export CSV. Warranty and Toner pages gained CSV export. - Deleted the dead legacy Warranty Status report (always-zero buckets from a retired column). Theming and fonts: - Inter (variable) bundled locally via @fontsource, replacing the Google Fonts Roboto import - air-gapped installs now render correctly; tables use tabular numerals. - Optional brand_primary_dark_color, brand_accent_color, brand_sidebar_color settings applied to CSS vars at bootstrap. USB frontend repair (views were reading a dead legacy shape): - List/detail/form and the employee profile USB panels remapped to the real API shape (device_id/device_desc/checkinoutlog); employee panels now use /usb/checkouts endpoints; external-mode /usb/checkouts/active honors the badge filter; dead client methods pruned. Also: warranties list page no longer requires login (matches app convention); collector doc rewritten with a GE-Enforce integration guide and paste-ready PowerShell reporter; ADR index and CHANGELOG updated. Verified: 323 tests pass, naming/style green, frontend builds, plugin migration dry-run green on scratch MySQL. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -1,67 +1,156 @@
|
||||
# Collector integration (PC auto-update)
|
||||
|
||||
How the shopfloor PC fleet pushes inventory into shopdb-flask, replacing the
|
||||
classic ASP `api.asp?action=updateCompleteAsset` path.
|
||||
classic ASP `api.asp?action=updateCompleteAsset` path. The generic collector
|
||||
contract is defined in ADR-006; this doc is the operational reference for wiring
|
||||
a real caller (the GE-Enforce fleet agent) to it.
|
||||
|
||||
## Endpoint
|
||||
- Server code: `shopdb/core/api/collector.py`
|
||||
- Computers schema + upsert: `plugins/computers/plugin.py`
|
||||
(`get_collector_schema` / `apply_collector_payload`)
|
||||
- Contract rationale: `docs/adr/ADR-006-collector-contract.md`
|
||||
|
||||
`POST /api/collector/computers`
|
||||
---
|
||||
|
||||
Auth: API key header `X-API-Key: <key>`, resolved as `COLLECTOR_API_KEY_COMPUTERS`
|
||||
then the shared `COLLECTOR_API_KEY` (ADR-006). Idempotent upsert keyed on
|
||||
`hostname`.
|
||||
## 1. How it works now
|
||||
|
||||
> **Breaking change:** the API key must be sent in the `X-API-Key` header. The
|
||||
> old `?api_key=<key>` querystring fallback has been removed, on every collector
|
||||
> endpoint (`/api/collector/<plugin>`, `/pc`, `/apps`, `/heartbeat`, `/bulk`,
|
||||
> `/status`). Querystring keys leak into access logs and proxy history. Update
|
||||
> any caller still passing `api_key` in the URL to use the header instead.
|
||||
### Auth model (header-only, env keys, fail-closed)
|
||||
|
||||
## Payload (project naming convention: lowercase concatenated)
|
||||
- The API key travels in the `X-API-Key` request header. Nothing else is
|
||||
accepted. The old `?api_key=<key>` querystring fallback has been removed on
|
||||
every collector endpoint (see the breaking-change section below).
|
||||
- Keys come from environment variables (never the database, never a file the app
|
||||
serves):
|
||||
- `COLLECTOR_API_KEY_<PLUGINNAME>` - per-plugin override, uppercased plugin
|
||||
name (e.g. `COLLECTOR_API_KEY_COMPUTERS`).
|
||||
- `COLLECTOR_API_KEY` - shared fallback used when no per-plugin key is set.
|
||||
- Resolution order per request: per-plugin key first, then the shared key
|
||||
(`_plugin_api_key` in `collector.py`).
|
||||
- Fail-closed: if neither variable is set server-side, the endpoint returns
|
||||
HTTP 500 `Collector API key not configured` and rejects every request. An
|
||||
unconfigured server never silently accepts unauthenticated data.
|
||||
- A caller that sends the wrong key (or no key) gets HTTP 401 `Invalid API key`.
|
||||
|
||||
| Field | Meaning | Flask target |
|
||||
|-------|---------|--------------|
|
||||
| `hostname` (required) | identity | `Computer.hostname` |
|
||||
| `machinenumber` | machine number | `Asset.assetnumber` (skips `9999` placeholder, falls back to hostname) |
|
||||
| `pctype` | imaging pc-type | `Computer.computertypeid` via the configurable mapping |
|
||||
| `pcsubtype` | finer class | accepted, not yet stored (warning) |
|
||||
| `serialnumber` | BIOS serial | `Asset.serialnumber` |
|
||||
| `loggedinuser` | current user | `Computer.loggedinuser` |
|
||||
| `lastboottime` | ISO datetime | `Computer.lastboottime` |
|
||||
| `lastcheckin` | ISO datetime | accepted (heartbeat) |
|
||||
| `ipaddress` | primary IP | primary `Communication` |
|
||||
| `vendorname` | manufacturer | `Computer.vendorid` (created if missing) |
|
||||
| `modelnumber` | model | `Computer.modelnumberid` (created if missing) |
|
||||
| `osname` | OS caption | `Computer.osid` (looked up; warned if unknown) |
|
||||
| `installedsoftware` | `[{name, version}]` | `ComputerInstalledApp` (known apps only) |
|
||||
### Generic endpoint contract: `POST /api/collector/<plugin>`
|
||||
|
||||
Response: `{status, action: created|updated, assetid, identityvalue, warnings[]}`.
|
||||
One dynamic route serves every enabled plugin that returns a collector schema.
|
||||
For the PC fleet that is `POST /api/collector/computers`.
|
||||
|
||||
## Source of truth on the PC (current method, may change)
|
||||
Request flow inside `generic_collect`:
|
||||
|
||||
The data already exists at image time and at runtime:
|
||||
1. Look up the plugin's schema. Unknown / disabled plugin -> HTTP 404
|
||||
`No collector registered for plugin <plugin>`.
|
||||
2. Resolve and check the API key (per-plugin then shared, as above).
|
||||
3. Parse the JSON body. Missing or non-object body -> HTTP 400 `No data provided`.
|
||||
4. Identity-field resolution: the schema names an `identityfield`
|
||||
(`hostname` for computers). If that field is missing or blank in the payload,
|
||||
HTTP 400 `<identityfield> is required`.
|
||||
5. Idempotent upsert: the plugin's `apply_collector_payload` finds the existing
|
||||
asset by identity and updates it, or inserts a new one. Same identity on a
|
||||
later submission updates the same row - re-imaging a PC does not duplicate it,
|
||||
and existing asset relationships are preserved.
|
||||
6. Audit logging: every accepted submission writes an `AuditLog` row
|
||||
(`action` = created/updated, entity `Collector`, `details = collector:<plugin>
|
||||
action=<action>`), then commits.
|
||||
|
||||
- **machine number**: registry `HKLM\SOFTWARE\[WOW6432Node\]GE Aircraft Engines\Dnc\General\MachineNo`
|
||||
FIRST (authoritative post Update-MachineNumber; ignore the `9999` placeholder),
|
||||
then `C:\Enrollment\machine-number.txt` as fallback. This is exactly what
|
||||
GE-Enforce.ps1 already does.
|
||||
- **pc-type / pc-subtype**: `C:\Enrollment\pc-type.txt` / `pc-subtype.txt`
|
||||
(the `gea-shopfloor-*` taxonomy).
|
||||
- **serial / vendor / model / os / user / boot**: live WMI on the PC.
|
||||
Response body (HTTP 200), wrapped in the standard envelope
|
||||
`{status, data, message, meta}`, with the collector result under `data`:
|
||||
|
||||
GE-Enforce currently writes a status JSON to the SFLD share rather than POSTing.
|
||||
Whatever transport is used (a relay reading those status files, or a direct POST
|
||||
later), map its field names to the table above.
|
||||
```json
|
||||
{
|
||||
"status": "success",
|
||||
"data": {
|
||||
"status": "ok",
|
||||
"action": "created",
|
||||
"assetid": 12345,
|
||||
"identityvalue": "WJRP2335",
|
||||
"warnings": ["unknown operating system: Microsoft Windows 11 Enterprise 23H2 (build 22631)"]
|
||||
},
|
||||
"message": "computers collector created",
|
||||
"meta": { "timestamp": "...", "requestid": "..." }
|
||||
}
|
||||
```
|
||||
|
||||
## pc-type mapping (configurable)
|
||||
`action` is `created`, `updated`, or `noop`. `warnings` is a list of soft
|
||||
problems (unmapped pc-type, unknown OS, unknown app, un-stored pcsubtype) that
|
||||
did NOT fail the request - the row was still written.
|
||||
|
||||
`pctype` (e.g. `gea-shopfloor-cmm`) is mapped to a flask Computer Type through
|
||||
### Error responses
|
||||
|
||||
Errors use the same envelope with `status: "error"` and the detail under
|
||||
`data.error`:
|
||||
|
||||
```json
|
||||
{ "status": "error", "data": { "error": { "code": "UNAUTHORIZED", "message": "Invalid API key" } }, "meta": { } }
|
||||
```
|
||||
|
||||
The plugin can raise `ValueError` for controlled validation failures; its message
|
||||
is returned verbatim (HTTP 400). Any other exception is caught, logged
|
||||
server-side, the transaction is rolled back, and the caller gets a generic
|
||||
HTTP 500 `Internal error processing collector payload` with no internal detail
|
||||
(by design - check the server log to see what actually failed).
|
||||
|
||||
### Schema discovery: `GET /api/collector/_schemas`
|
||||
|
||||
Returns the collector schema for every enabled plugin. This route is
|
||||
JWT-protected (an interactive/admin token), NOT API-key protected - it is for
|
||||
humans and tooling introspecting what payloads are accepted, not for the
|
||||
headless collectors themselves.
|
||||
|
||||
### Legacy endpoints (computers-only, predate ADR-006)
|
||||
|
||||
These still exist for the older PowerShell callers and all require the same
|
||||
`X-API-Key` header:
|
||||
|
||||
| Endpoint | Purpose |
|
||||
|---|---|
|
||||
| `POST /api/collector/pc` | Update one PC matched by `hostname` (lastboot, user, serial). |
|
||||
| `POST /api/collector/apps` | Update installed apps for a PC (known apps only). |
|
||||
| `POST /api/collector/heartbeat` | Record check-in for one or many hostnames. |
|
||||
| `POST /api/collector/bulk` | Update many PCs in one call. |
|
||||
| `GET /api/collector/status` | Liveness + endpoint list. |
|
||||
|
||||
New integrations should target `POST /api/collector/computers`, not these. Per
|
||||
ADR-006 the legacy `/pc` path is deprecated and slated for removal before v1.0.
|
||||
|
||||
### Computers plugin field mapping
|
||||
|
||||
Payload naming follows the project convention (lowercase concatenated). The
|
||||
identity field is `hostname`. All other fields are optional; an omitted field
|
||||
leaves the existing value untouched (patch-style), so a bare report never blanks
|
||||
a column.
|
||||
|
||||
| Payload field | Type | Server behaviour (`apply_collector_payload`) |
|
||||
|---|---|---|
|
||||
| `hostname` (required) | string | Identity. Matches `Computer.hostname` (case-insensitive), then falls back to `Asset.assetnumber`. New asset created if no match. |
|
||||
| `machinenumber` | string | Business tag -> `Asset.assetnumber`. The placeholder `9999` and empty string are skipped; when skipped a new PC falls back to `assetnumber = hostname`. On an existing PC a real value updates `assetnumber`. |
|
||||
| `pctype` | string | `gea-shopfloor-*` imaging type -> `Computer.computertypeid` via the configurable `pctypemap` settings. Unmapped value -> warning, not error. |
|
||||
| `pcsubtype` | string | Accepted but not stored yet -> warning. |
|
||||
| `serialnumber` | string | `Asset.serialnumber`. |
|
||||
| `loggedinuser` | string | `Computer.loggedinuser`. (`currentuser` is also accepted as a legacy alias.) |
|
||||
| `lastboottime` | ISO-8601 datetime | `Computer.lastboottime`. Unparseable -> warning. |
|
||||
| `lastcheckin` | ISO-8601 datetime | Accepted (heartbeat semantics). `lastreporteddate` is set server-side on every call regardless. |
|
||||
| `ipaddress` | string | Primary `Communication` row (`isprimary=True`, type `IP`). Updated in place or created. |
|
||||
| `vendorname` | string | `Computer.vendorid`. Vendor row auto-created if missing (free vocab). |
|
||||
| `modelnumber` | string | `Computer.modelnumberid`, scoped to the vendor when known. Model row auto-created if missing. |
|
||||
| `osname` | string | `Computer.osid`. Controlled vocab: looked up in `operatingsystems`, NOT auto-created. Unknown value -> warning (row still written, `osid` left unset). |
|
||||
| `installedsoftware` | array of `{name, version}` | `ComputerInstalledApp` rows for applications shopdb already tracks. Unknown app name -> warning, skipped. |
|
||||
|
||||
Schema source of truth: `get_collector_schema` in `plugins/computers/plugin.py`.
|
||||
If you change the payload, change it there and re-check this table.
|
||||
|
||||
### pc-type mapping (configurable per site)
|
||||
|
||||
`pctype` (e.g. `gea-shopfloor-cmm`) maps to a shopdb Computer Type through
|
||||
`pctypemap_<pxetype>` settings (Settings > System > "Collector PC Type Mapping").
|
||||
Defaults live in `plugins/computers/pctypemap.py` and are seeded on plugin
|
||||
install; edit per site in the UI. Unmapped pc-types are recorded as a warning,
|
||||
not an error.
|
||||
install; edit per site in the UI. Unmapped pc-types produce a warning, not a
|
||||
failure.
|
||||
|
||||
## Classic api.asp field mapping (for migrating the PowerShell scripts)
|
||||
### Classic api.asp field mapping (for porting the PowerShell reporter)
|
||||
|
||||
The fleet's classic-ASP reporter posts form fields to
|
||||
`api.asp?action=updateCompleteAsset`. Map them to the collector JSON as follows:
|
||||
|
||||
| Classic `updateCompleteAsset` form field | Collector field |
|
||||
|---|---|
|
||||
@@ -76,5 +165,365 @@ not an error.
|
||||
| `osVersion` | `osname` |
|
||||
| `installedApps` | `installedsoftware` |
|
||||
|
||||
Not carried over (no current home): warranty fields, DNC config, multi-NIC
|
||||
detail beyond the primary IP, VNC/WinRM flags.
|
||||
Not carried over (no current home in the computers schema): warranty fields, DNC
|
||||
config, multi-NIC detail beyond the single primary IP, VNC/WinRM flags. The
|
||||
classic reporter posts a full `networkInterfaces` array; the collector accepts
|
||||
only one `ipaddress`, so pick the corp/routable NIC (see the corp-range gate in
|
||||
the PowerShell below).
|
||||
|
||||
---
|
||||
|
||||
## 2. Breaking change: querystring `api_key` removed
|
||||
|
||||
The API key must now be sent in the `X-API-Key` header. The `?api_key=<key>`
|
||||
querystring form has been removed on every collector endpoint
|
||||
(`/api/collector/<plugin>`, `/pc`, `/apps`, `/heartbeat`, `/bulk`, `/status`).
|
||||
Querystring keys leak into web-server access logs, proxy history, and browser
|
||||
history. Header-only keeps the secret out of those logs.
|
||||
|
||||
Before (no longer works - the key is ignored and the request is rejected 401):
|
||||
|
||||
```
|
||||
POST /api/collector/computers?api_key=SECRET
|
||||
Content-Type: application/json
|
||||
|
||||
{ "hostname": "WJRP2335", "machinenumber": "2335" }
|
||||
```
|
||||
|
||||
After (correct):
|
||||
|
||||
```
|
||||
POST /api/collector/computers
|
||||
X-API-Key: SECRET
|
||||
Content-Type: application/json
|
||||
|
||||
{ "hostname": "WJRP2335", "machinenumber": "2335" }
|
||||
```
|
||||
|
||||
PowerShell before/after:
|
||||
|
||||
```powershell
|
||||
# BEFORE (broken)
|
||||
Invoke-RestMethod -Uri "https://$SiteHost/api/collector/computers?api_key=$key" `
|
||||
-Method Post -Body $json -ContentType 'application/json'
|
||||
|
||||
# AFTER (correct)
|
||||
Invoke-RestMethod -Uri "https://$SiteHost/api/collector/computers" `
|
||||
-Method Post -Body $json -ContentType 'application/json' `
|
||||
-Headers @{ 'X-API-Key' = $key }
|
||||
```
|
||||
|
||||
Any caller still putting `api_key` in the URL must move it to the header.
|
||||
|
||||
---
|
||||
|
||||
## 3. GE-Enforce implementation guide
|
||||
|
||||
GE-Enforce is the SYSTEM-context fleet agent that runs every enforcement cycle on
|
||||
each shopfloor PC (scheduled task, at-logon + periodic). It lives OUTSIDE this
|
||||
repo. The facts below are drawn from the real scripts so the payload matches what
|
||||
the PC already knows:
|
||||
|
||||
- `playbook/shopfloor-setup/common/GE-Enforce.ps1` - the enforcement pass. At the
|
||||
end of each run it writes a status JSON to
|
||||
`<share>\_outputs\logs\<hostname>\status.json` (interim transport; the fleet
|
||||
dashboard reads those files). It logs via `Write-EnforceLog` to
|
||||
`C:\Logs\Shopfloor\enforce-YYYYMMDD.log`. Its machine-number resolution is:
|
||||
registry `HKLM:\SOFTWARE\WOW6432Node\GE Aircraft Engines\Dnc\General` then
|
||||
`HKLM:\SOFTWARE\GE Aircraft Engines\Dnc\General`, property `MachineNo`,
|
||||
skipping the `9999` placeholder, then `C:\Enrollment\machine-number.txt` as
|
||||
fallback. hostname is `[System.Environment]::MachineName` (live NetBIOS name,
|
||||
not `$env:COMPUTERNAME`, which goes stale after a post-image rename).
|
||||
- `playbook/shopfloor-setup/gea-shopfloor-collections/Report-AssetToShopDB.ps1` -
|
||||
the existing HTTP reporter that POSTs to the classic ASP
|
||||
`api.asp?action=updateCompleteAsset`. It runs every cycle as a `Type=PS1`
|
||||
manifest entry (DetectionMethod `Always`) under the SYSTEM task, logs to
|
||||
`C:\Logs\Shopfloor\report-asset-YYYYMMDD.log`, always exits 0, and reads the
|
||||
same machine number (reg, then `cmm\cmmid.txt`, then `machine-number.txt`) plus
|
||||
BIOS serial, OS caption, make/model, logged-in user, and corp-NIC IP from WMI.
|
||||
The function below is the direct-to-flask analogue of this reporter.
|
||||
- `playbook/shopfloor-setup/Shopfloor/lib/Update-MachineNumber.ps1` - keeps the
|
||||
reg `MachineNo` and `C:\Enrollment\machine-number.txt` in sync when a tech
|
||||
reassigns a bay, so both sources agree.
|
||||
|
||||
### Paste-ready reporter
|
||||
|
||||
Drop this in as a `Type=PS1` / DetectionMethod `Always` manifest entry (mirroring
|
||||
`Report-AssetToShopDB.ps1`), or dot-source `Send-ShopdbCollectorReport` from
|
||||
GE-Enforce.ps1 and call it at the end of the pass. It reads the machine number
|
||||
exactly the way GE-Enforce already does, builds a payload matching the computers
|
||||
collector schema, and POSTs with the `X-API-Key` header over TLS 1.2. Every
|
||||
field name below was checked against `get_collector_schema` in
|
||||
`plugins/computers/plugin.py`.
|
||||
|
||||
```powershell
|
||||
# Send-ShopdbCollectorReport.ps1
|
||||
# Reports this PC's identity to shopdb-flask via POST /api/collector/computers.
|
||||
# Runs as SYSTEM from the GE-Enforce cycle. Always exits without throwing;
|
||||
# failures are logged, never fatal.
|
||||
|
||||
function Get-ShopdbCollectorApiKey {
|
||||
# NEVER hardcode the key in this script (it lives on the share). Read it
|
||||
# from a per-site value the SYSTEM/machine account can already reach.
|
||||
# Preferred: a field in the site-config.json GE-Enforce already parses.
|
||||
# Fallback: a machine-local file staged at imaging, ACL'd to SYSTEM.
|
||||
$key = ''
|
||||
$siteCfg = 'C:\Enrollment\site-config.json'
|
||||
if (Test-Path -LiteralPath $siteCfg) {
|
||||
try {
|
||||
$cfg = Get-Content -LiteralPath $siteCfg -Raw | ConvertFrom-Json
|
||||
if ($cfg.collectorApiKey) { $key = "$($cfg.collectorApiKey)".Trim() }
|
||||
} catch {}
|
||||
}
|
||||
if (-not $key) {
|
||||
$keyFile = 'C:\Enrollment\collector-api-key.txt'
|
||||
if (Test-Path -LiteralPath $keyFile) {
|
||||
try { $key = (Get-Content -LiteralPath $keyFile -First 1 -ErrorAction Stop).Trim() } catch {}
|
||||
}
|
||||
}
|
||||
return $key
|
||||
}
|
||||
|
||||
function Get-ShopdbMachineNumber {
|
||||
# Same resolution GE-Enforce.ps1 / Report-AssetToShopDB.ps1 use:
|
||||
# eDNC registry (skip 9999 placeholder) then C:\Enrollment\machine-number.txt.
|
||||
$machineNumber = ''
|
||||
foreach ($rp in @(
|
||||
'HKLM:\SOFTWARE\WOW6432Node\GE Aircraft Engines\Dnc\General',
|
||||
'HKLM:\SOFTWARE\GE Aircraft Engines\Dnc\General'
|
||||
)) {
|
||||
if ($machineNumber) { break }
|
||||
if (Test-Path $rp) {
|
||||
try {
|
||||
$v = (Get-ItemProperty -Path $rp -Name MachineNo -ErrorAction Stop).MachineNo
|
||||
if ($v -and "$v".Trim() -ne '9999') { $machineNumber = "$v".Trim() }
|
||||
} catch {}
|
||||
}
|
||||
}
|
||||
if (-not $machineNumber -and (Test-Path 'C:\Enrollment\machine-number.txt')) {
|
||||
try {
|
||||
$v = (Get-Content 'C:\Enrollment\machine-number.txt' -First 1 -ErrorAction Stop).Trim()
|
||||
if ($v -and $v -ne '9999') { $machineNumber = $v }
|
||||
} catch {}
|
||||
}
|
||||
return $machineNumber
|
||||
}
|
||||
|
||||
function Get-ShopdbCorpIPv4 {
|
||||
# Pick the corp/AESFMA NIC IP. Same allowed-range gate as
|
||||
# Report-AssetToShopDB.ps1 - update the ranges if the site re-VLANs.
|
||||
$allowedRanges = @(
|
||||
@{ Network = '10.134.48.0'; PrefixLen = 23 },
|
||||
@{ Network = '10.48.249.0'; PrefixLen = 26 }
|
||||
)
|
||||
function ConvertTo-Uint32([string]$ip) {
|
||||
$bytes = ([System.Net.IPAddress]::Parse($ip)).GetAddressBytes()
|
||||
[Array]::Reverse($bytes)
|
||||
return [BitConverter]::ToUInt32($bytes, 0)
|
||||
}
|
||||
try {
|
||||
$ips = Get-NetIPAddress -AddressFamily IPv4 -ErrorAction Stop |
|
||||
Where-Object { $_.IPAddress -notmatch '^169\.254' -and $_.IPAddress -ne '127.0.0.1' }
|
||||
foreach ($ipo in $ips) {
|
||||
$ipInt = ConvertTo-Uint32 $ipo.IPAddress
|
||||
foreach ($r in $allowedRanges) {
|
||||
$netInt = ConvertTo-Uint32 $r.Network
|
||||
$mask = [uint32]([math]::Pow(2, 32) - [math]::Pow(2, 32 - $r.PrefixLen))
|
||||
if (($ipInt -band $mask) -eq ($netInt -band $mask)) { return $ipo.IPAddress }
|
||||
}
|
||||
}
|
||||
} catch {}
|
||||
return ''
|
||||
}
|
||||
|
||||
function Send-ShopdbCollectorReport {
|
||||
param(
|
||||
[string]$SiteHost = 'tsgwp00525.wjs.geaerospace.net',
|
||||
[string]$ApiKey = (Get-ShopdbCollectorApiKey),
|
||||
[int]$TimeoutSec = 30,
|
||||
[string]$LogFile = ('C:\Logs\Shopfloor\collector-{0}.log' -f (Get-Date -Format 'yyyyMMdd'))
|
||||
)
|
||||
|
||||
$logDir = Split-Path -Parent $LogFile
|
||||
if (-not (Test-Path $logDir)) {
|
||||
New-Item -ItemType Directory -Path $logDir -Force -ErrorAction SilentlyContinue | Out-Null
|
||||
}
|
||||
function Write-CollectorLog([string]$msg) {
|
||||
$ts = Get-Date -Format 'yyyy-MM-dd HH:mm:ss'
|
||||
"$ts $msg" | Tee-Object -FilePath $LogFile -Append | Out-Null
|
||||
}
|
||||
|
||||
Write-CollectorLog '=== Report to shopdb-flask collector ==='
|
||||
|
||||
if (-not $ApiKey) {
|
||||
Write-CollectorLog 'ERROR no collector API key (site-config.json / collector-api-key.txt) - skipping.'
|
||||
return
|
||||
}
|
||||
|
||||
# PowerShell 5.1 does not always negotiate TLS 1.2 by default.
|
||||
try { [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 } catch {}
|
||||
|
||||
# --- Gather identity (matches the computers collector schema) ---
|
||||
$hostname = [System.Environment]::MachineName
|
||||
if (-not $hostname) { $hostname = $env:COMPUTERNAME }
|
||||
|
||||
$machineNumber = Get-ShopdbMachineNumber
|
||||
$ipAddress = Get-ShopdbCorpIPv4
|
||||
|
||||
$serialNumber = ''
|
||||
try {
|
||||
$serialNumber = "$((Get-CimInstance -ClassName Win32_BIOS -ErrorAction Stop).SerialNumber)".Trim()
|
||||
} catch { Write-CollectorLog "WARN BIOS serial read failed: $($_.Exception.Message)" }
|
||||
|
||||
$vendorName = ''; $modelNumber = ''; $loggedInUser = ''
|
||||
try {
|
||||
$cs = Get-CimInstance -ClassName Win32_ComputerSystem -ErrorAction Stop
|
||||
$vendorName = "$($cs.Manufacturer)".Trim()
|
||||
$modelNumber = "$($cs.Model)".Trim()
|
||||
# UserName is COMPUTERNAME\user or DOMAIN\user; keep the bare username.
|
||||
if ($cs.UserName) { $loggedInUser = ($cs.UserName -split '\\')[-1].Trim() }
|
||||
} catch { Write-CollectorLog "WARN ComputerSystem read failed: $($_.Exception.Message)" }
|
||||
|
||||
$osName = ''; $lastBootTime = ''
|
||||
try {
|
||||
$os = Get-CimInstance -ClassName Win32_OperatingSystem -ErrorAction Stop
|
||||
$osName = "$($os.Caption)".Trim()
|
||||
$dv = ''
|
||||
try { $dv = (Get-ItemProperty 'HKLM:\SOFTWARE\Microsoft\Windows NT\CurrentVersion' -Name DisplayVersion -ErrorAction Stop).DisplayVersion } catch {}
|
||||
if ($dv) { $osName += " $dv" }
|
||||
if ($os.BuildNumber) { $osName += " (build $($os.BuildNumber))" }
|
||||
$osName = $osName.Trim()
|
||||
# ISO-8601 so the server's datetime parse accepts it.
|
||||
try { $lastBootTime = $os.LastBootUpTime.ToUniversalTime().ToString('yyyy-MM-ddTHH:mm:ssZ') } catch {}
|
||||
} catch { Write-CollectorLog "WARN OS read failed: $($_.Exception.Message)" }
|
||||
|
||||
$pcType = ''
|
||||
if (Test-Path -LiteralPath 'C:\Enrollment\pc-type.txt') {
|
||||
try { $pcType = (Get-Content -LiteralPath 'C:\Enrollment\pc-type.txt' -First 1 -ErrorAction Stop).Trim() } catch {}
|
||||
}
|
||||
$pcSubType = ''
|
||||
if (Test-Path -LiteralPath 'C:\Enrollment\pc-subtype.txt') {
|
||||
try { $pcSubType = (Get-Content -LiteralPath 'C:\Enrollment\pc-subtype.txt' -First 1 -ErrorAction Stop).Trim() } catch {}
|
||||
}
|
||||
|
||||
# --- Build payload. Field names MUST match get_collector_schema exactly. ---
|
||||
$payload = @{ hostname = $hostname }
|
||||
if ($machineNumber) { $payload['machinenumber'] = $machineNumber }
|
||||
if ($pcType) { $payload['pctype'] = $pcType }
|
||||
if ($pcSubType) { $payload['pcsubtype'] = $pcSubType }
|
||||
if ($serialNumber) { $payload['serialnumber'] = $serialNumber }
|
||||
if ($loggedInUser) { $payload['loggedinuser'] = $loggedInUser }
|
||||
if ($lastBootTime) { $payload['lastboottime'] = $lastBootTime }
|
||||
if ($ipAddress) { $payload['ipaddress'] = $ipAddress }
|
||||
if ($vendorName) { $payload['vendorname'] = $vendorName }
|
||||
if ($modelNumber) { $payload['modelnumber'] = $modelNumber }
|
||||
if ($osName) { $payload['osname'] = $osName }
|
||||
$payload['lastcheckin'] = (Get-Date).ToUniversalTime().ToString('yyyy-MM-ddTHH:mm:ssZ')
|
||||
# installedsoftware is optional; add an array of @{ name=..; version=.. }
|
||||
# here if the manifest introspection is wired to real Application names.
|
||||
|
||||
$uri = "https://$SiteHost/api/collector/computers"
|
||||
$json = $payload | ConvertTo-Json -Depth 5
|
||||
$headers = @{ 'X-API-Key' = $ApiKey }
|
||||
|
||||
Write-CollectorLog ("POST {0} host={1} machineNo={2} pcType={3} serial={4} ip={5}" -f `
|
||||
$uri, $hostname, $machineNumber, $pcType, $serialNumber, $ipAddress)
|
||||
|
||||
try {
|
||||
$resp = Invoke-RestMethod -Uri $uri -Method Post -Body $json `
|
||||
-ContentType 'application/json' -Headers $headers `
|
||||
-TimeoutSec $TimeoutSec -ErrorAction Stop
|
||||
$d = $resp.data
|
||||
Write-CollectorLog ("OK action={0} assetid={1} identity={2} warnings={3}" -f `
|
||||
$d.action, $d.assetid, $d.identityvalue, ((@($d.warnings)) -join '; '))
|
||||
} catch {
|
||||
$code = $null
|
||||
try { $code = [int]$_.Exception.Response.StatusCode.value__ } catch {}
|
||||
Write-CollectorLog "ERROR POST failed (http=$code): $($_.Exception.Message)"
|
||||
}
|
||||
}
|
||||
|
||||
# Entry point when run as a standalone manifest PS1 entry:
|
||||
Send-ShopdbCollectorReport
|
||||
```
|
||||
|
||||
### Where the call slots into the GE-Enforce run
|
||||
|
||||
Put the POST at the END of the enforcement pass, after the enforcement scopes are
|
||||
processed and the PC identity is settled. Two placements, pick one:
|
||||
|
||||
- Preferred - a sibling manifest entry, mirroring `Report-AssetToShopDB.ps1`:
|
||||
add `Send-ShopdbCollectorReport.ps1` as a `Type=PS1`, DetectionMethod `Always`
|
||||
manifest entry in `common\manifest.json`. It runs every cycle under the SYSTEM
|
||||
task, keeps GE-Enforce.ps1 untouched, and gets its own log file. This matches
|
||||
the established pattern the classic reporter already uses.
|
||||
- Alternative - inline: dot-source the function and call
|
||||
`Send-ShopdbCollectorReport` inside GE-Enforce.ps1's status-write-back `try`
|
||||
block (right after `status.json` is written, near the end of the pass). Use
|
||||
this only if you want the HTTP push to share GE-Enforce's own log and lifecycle.
|
||||
|
||||
Either way, run it ALONGSIDE the existing `status.json` share write (and
|
||||
alongside the classic `Report-AssetToShopDB.ps1`) during rollout - do not remove
|
||||
the share write until the HTTP path is proven.
|
||||
|
||||
### Delivering the API key to clients
|
||||
|
||||
Do NOT hardcode the key in the script - the script itself lives on the SFLD
|
||||
share, so a literal key there is effectively published to everyone with share
|
||||
read. Two viable options:
|
||||
|
||||
1. Per-site value the machine account already reads. GE-Enforce already parses
|
||||
`C:\Enrollment\site-config.json` (its local copy of the per-site config on the
|
||||
share). Add one field, `collectorApiKey`, populated per site. The script reads
|
||||
it with the machine/SYSTEM identity it already runs as. Rotation = update the
|
||||
one config value at the site, no re-image, no script redeploy.
|
||||
2. Baked at imaging into a machine-local file (`C:\Enrollment\collector-api-key.txt`)
|
||||
or an HKLM value, ACL'd to SYSTEM/Administrators, written by the enrollment
|
||||
pipeline. Tighter blast radius (the key never sits on the share at all), but
|
||||
rotation requires touching every PC or re-imaging.
|
||||
|
||||
Recommendation: use option 1 (`collectorApiKey` in the per-site config), matching
|
||||
how GE-Enforce already sources its share paths and mirroring how SFLD credentials
|
||||
are provisioned per site. It keeps the secret out of source control and off the
|
||||
script, sits at the same trust boundary as everything else the SYSTEM agent
|
||||
already reads, and supports rotation without a fleet touch. `Get-ShopdbCollectorApiKey`
|
||||
above already prefers this and falls back to the imaging-baked file, so a site
|
||||
can start on option 1 and tighten to option 2 later with no code change. The
|
||||
server side pairs this with `COLLECTOR_API_KEY_COMPUTERS` (per-plugin) so the PC
|
||||
fleet's key is scoped to the computers collector only.
|
||||
|
||||
### Staged rollout
|
||||
|
||||
1. Keep everything as-is: `status.json` share write + classic
|
||||
`Report-AssetToShopDB.ps1` continue running. Provision `collectorApiKey` per
|
||||
site and set `COLLECTOR_API_KEY_COMPUTERS` on the shopdb-flask server.
|
||||
2. Add `Send-ShopdbCollectorReport` as a new manifest entry on a handful of pilot
|
||||
bays (one per pc-type: collections, cmm, keyence, waxtrace, ...). It logs to
|
||||
`C:\Logs\Shopfloor\collector-YYYYMMDD.log` and cannot break enforcement (it
|
||||
never throws, exits cleanly).
|
||||
3. Verify on the server: pilot PCs appear/update in shopdb-flask, `action` is
|
||||
`created`/`updated`, warnings are understood (map any unmapped `pctype`, add
|
||||
any missing OS strings to the `operatingsystems` vocab). Cross-check the
|
||||
collector log's `OK` lines against the audit log.
|
||||
4. Roll the manifest entry out fleet-wide (it deploys from `common\`, so every
|
||||
pc-type gets it). Continue running both the share write and the HTTP POST.
|
||||
5. Once shopdb-flask is the system of record and stable across a full imaging
|
||||
cycle, retire the classic `Report-AssetToShopDB.ps1` and, if desired, the
|
||||
`status.json` share write.
|
||||
|
||||
---
|
||||
|
||||
## 4. Troubleshooting
|
||||
|
||||
| Symptom | HTTP | Cause / fix |
|
||||
|---|---|---|
|
||||
| `Invalid API key` | 401 | The `X-API-Key` header is missing or wrong. Confirm the client key matches `COLLECTOR_API_KEY_COMPUTERS` (or the shared `COLLECTOR_API_KEY`) on the server. Check the key is being read (`Get-ShopdbCollectorApiKey` returned non-empty). Remember the querystring form no longer works. |
|
||||
| `Collector API key not configured` | 500 | Fail-closed: neither `COLLECTOR_API_KEY_COMPUTERS` nor `COLLECTOR_API_KEY` is set in the server environment. Set one and restart the app. This is a SERVER config gap, not a client problem. |
|
||||
| `No collector registered for plugin computers` | 404 | The computers plugin is disabled or not loaded on that instance. Enable it. |
|
||||
| `hostname is required` / `No data provided` | 400 | Empty body, non-JSON body, or missing identity field. Check `-ContentType 'application/json'` and that `hostname` is set. |
|
||||
| `Internal error processing collector payload` | 500 | Generic by design - the server does not leak the cause to the caller. The real error (DB, unexpected exception) is in the server log (`Collector upsert failed for computers`). Check there. |
|
||||
| `warnings` present but `action` is created/updated | 200 | Soft issues only; the row WAS written. Common: unmapped `pctype` (fix the pctypemap setting), unknown `osname` (add it to the `operatingsystems` vocab), unknown app name, `pcsubtype` not stored. No action needed unless the warning matters to you. |
|
||||
|
||||
Client-side log for the fleet reporter: `C:\Logs\Shopfloor\collector-YYYYMMDD.log`.
|
||||
Server-side: the Flask app log (the collector logs upsert failures and per-plugin
|
||||
schema failures there).
|
||||
|
||||
@@ -148,7 +148,10 @@ them under `instance/branding/`.
|
||||
| `qr_logo` | `/ge-monogram.svg` | Logo composited in printer QR labels. Blank = no overlay. |
|
||||
| `badge_logo` | `/ge-aerospace-logo.svg` | Logo on the equipment badge print page. |
|
||||
| `site_favicon` | (empty) | Browser tab favicon. Blank = shipped `/favicon.svg`. |
|
||||
| `brand_primary_color` | (empty) | Primary brand color as a CSS color value. Blank = built-in theme color. |
|
||||
| `brand_primary_color` | (empty) | Primary brand color as a CSS color value (maps to `--primary`). Blank = built-in theme color. |
|
||||
| `brand_primary_dark_color` | (empty) | Primary hover/active color (maps to `--primary-dark`). Blank = auto-derived by darkening the primary color ~15%. |
|
||||
| `brand_accent_color` | (empty) | Accent color for secondary buttons and badges (maps to `--secondary`). Blank = built-in theme color. |
|
||||
| `brand_sidebar_color` | (empty) | Sidebar background color (maps to `--sidebar-bg`). Blank = built-in theme color. |
|
||||
|
||||
### printing
|
||||
|
||||
|
||||
@@ -62,9 +62,16 @@ If `ProductionConfig.validate()` raises, the container exits with the offending
|
||||
|
||||
```bash
|
||||
docker compose exec api flask db upgrade
|
||||
docker compose exec api flask plugin upgrade-all
|
||||
```
|
||||
|
||||
This applies the baseline migration (creates all tables) and any subsequent migrations. Re-running is idempotent.
|
||||
`flask db upgrade` applies the core Alembic chain: the baseline migration plus
|
||||
every later migration, which together create all core AND bundled-plugin tables
|
||||
through the chain head. `flask plugin upgrade-all` then stamps each bundled
|
||||
plugin's own migration chain (the `alembic_version_<plugin>` tables) and applies
|
||||
any plugin-specific migrations added after the ownership cutover. Both commands
|
||||
are idempotent, so re-running them is safe. See ADR-008 for why plugin schema
|
||||
splits into per-plugin chains from the cutover forward.
|
||||
|
||||
**Charset:** the schema is utf8mb4 (`utf8mb4_unicode_ci`). The docker-compose `db` service sets `--character-set-server=utf8mb4`, so the auto-created `shopdb_flask` database is utf8mb4. If you point at an external MySQL instead of the bundled container, create the database as utf8mb4 first, or it inherits the server default (often latin1) and the schema silently drifts:
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ The contract is locked in [ADR-001](../docs/adr/ADR-001-asset-as-platform-contra
|
||||
The framework declares its contract version in `shopdb/__init__.py`:
|
||||
|
||||
```python
|
||||
__contract_version__ = '0.4.0'
|
||||
__contract_version__ = '0.6.0'
|
||||
```
|
||||
|
||||
Each plugin's `manifest.json` declares the range of contract versions it supports:
|
||||
@@ -192,6 +192,30 @@ class ComputersPlugin(BasePlugin):
|
||||
> already covers every bundled asset type; no plugin ever implemented the hook.
|
||||
> Search honors runtime plugin enable/disable.
|
||||
|
||||
### `get_reports() -> List[Dict]`
|
||||
|
||||
Returns report card definitions for the Reports hub. Added in contract 0.6.0.
|
||||
|
||||
Each entry has `id`, `name`, `description`, `category`, plus EXACTLY ONE of
|
||||
`route` (a frontend path for a dedicated report page) or `endpoint` (an API
|
||||
endpoint the hub renders inline).
|
||||
|
||||
```python
|
||||
class WarrantyPlugin(BasePlugin):
|
||||
def get_reports(self):
|
||||
return [{
|
||||
'id': 'warranty',
|
||||
'name': 'Warranty Report',
|
||||
'description': 'Assets bucketed by coverage: expired, expiring soon, active',
|
||||
'category': 'warranty',
|
||||
'route': '/reports/warranty',
|
||||
}]
|
||||
```
|
||||
|
||||
Consumed by `GET /api/reports`, which merges plugin cards after the static core
|
||||
reports sorted into category groups by the frontend (disabled plugins are
|
||||
skipped; a broken plugin is isolated in prod, re-raised in dev/test).
|
||||
|
||||
### `get_collector_schema() -> Optional[Dict]`
|
||||
|
||||
Declares the JSON Schema for an external collector pushing to `/api/collector/<pluginname>`. See [ADR-006](../docs/adr/ADR-006-collector-contract.md) for the contract.
|
||||
|
||||
@@ -121,6 +121,7 @@ Override hooks on the plugin class as needed. See [PLUGIN-HOOKS.md](PLUGIN-HOOKS
|
||||
|------|------|
|
||||
| `get_navigation_items` | Plugin shows up in the sidebar nav |
|
||||
| `get_dashboard_widgets` | Plugin's dashboard widget appears on the home page |
|
||||
| `get_reports` | Plugin's report cards appear on the Reports hub |
|
||||
| `get_collector_schema` + `apply_collector_payload` | Plugin accepts external pushes at `/api/collector/<name>` |
|
||||
|
||||
Each hook has a default that does nothing. Override only what your plugin needs.
|
||||
@@ -147,6 +148,7 @@ Copy from an existing plugin's view files (e.g., `frontend/src/views/network/`)
|
||||
|
||||
## Next steps
|
||||
|
||||
- [PLUGIN-GUIDE.md](PLUGIN-GUIDE.md) - the full narrative walkthrough of building the `measuringtools` plugin end to end (models, per-plugin migration baseline, authz, hooks, frontend integration, tests). Read this after the quickstart when you want the exemplar that exercises every framework feature.
|
||||
- [PLUGIN-HOOKS.md](PLUGIN-HOOKS.md) for the full hook reference
|
||||
- [CONTRIBUTING.md](../CONTRIBUTING.md) for naming conventions
|
||||
- [docs/adr/ADR-001-asset-as-platform-contract.md](../docs/adr/ADR-001-asset-as-platform-contract.md) for what your plugin can rely on
|
||||
|
||||
@@ -4,7 +4,7 @@ shopdb-flask is a framework. The plugins listed here are the pieces other GE Aer
|
||||
|
||||
## Bundled (ship with the framework)
|
||||
|
||||
These six plugins are in `plugins/` in this repo. Enable per site with `flask plugin install <name>`.
|
||||
These plugins are in `plugins/` in this repo. Enable per site with `flask plugin install <name>`.
|
||||
|
||||
| Plugin | Tracks | Notes |
|
||||
|--------|--------|-------|
|
||||
@@ -14,16 +14,16 @@ These six plugins are in `plugins/` in this repo. Enable per site with `flask pl
|
||||
| `network` | Switches, routers, access points, IDFs as locations | Asset-only; cleanest of the bundled set. |
|
||||
| `usb` | USB devices issued to shop-floor users | Lightweight checkout / check-in. |
|
||||
| `notifications` | Shop-floor notifications, recognitions, kiosk feed | Used by `ShopfloorDashboard.vue`. |
|
||||
|
||||
## Planned (in the roadmap, not yet built)
|
||||
|
||||
| Plugin | Tracks | Status |
|
||||
|--------|--------|--------|
|
||||
| `measuringtools` | Metrology and inspection: CMMs, Keyence vision, surface profilometers, GenSpec | Per [ADR-005](adr/ADR-005-equipment-vs-measuringtools.md). First plugin to be built using `flask plugin new` as the canary for the scaffold. |
|
||||
| `measuringtools` | Metrology and inspection instruments: calipers, micrometers, thread/bore/height gages, indicators | Per [ADR-005](adr/ADR-005-equipment-vs-measuringtools.md). Calibration lifecycle with derived status. First plugin built on the matured scaffold; its walkthrough is [PLUGIN-GUIDE.md](PLUGIN-GUIDE.md). Ships `default_enabled: false`. |
|
||||
|
||||
## Building your own
|
||||
|
||||
See [PLUGIN-QUICKSTART.md](PLUGIN-QUICKSTART.md) for the 30-minute walkthrough. The contract is locked in [ADR-001](adr/ADR-001-asset-as-platform-contract.md) and versioned per [ADR-002](adr/ADR-002-plugin-versioning.md).
|
||||
Two guides:
|
||||
|
||||
- [PLUGIN-QUICKSTART.md](PLUGIN-QUICKSTART.md) - generate, customize, install, and test a plugin in 30 minutes using `flask plugin new`.
|
||||
- [PLUGIN-GUIDE.md](PLUGIN-GUIDE.md) - the full narrative walkthrough of building the `measuringtools` plugin, the exemplar that exercises every current framework feature (models, per-plugin migrations, authz, hooks, frontend integration, tests).
|
||||
|
||||
The contract is locked in [ADR-001](adr/ADR-001-asset-as-platform-contract.md) and versioned per [ADR-002](adr/ADR-002-plugin-versioning.md).
|
||||
|
||||
Quick path:
|
||||
|
||||
@@ -33,6 +33,29 @@ flask plugin new cameras --description "Tracks shop-floor surveillance cameras"
|
||||
flask plugin install cameras
|
||||
```
|
||||
|
||||
## Migrations (per-plugin chains)
|
||||
|
||||
Each plugin that owns tables carries its own Alembic chain under
|
||||
`plugins/<name>/migrations/`, with a per-plugin version table
|
||||
`alembic_version_<name>` independent of the core `alembic_version`. Ownership is
|
||||
split at a fixed cutover (see [ADR-008](adr/ADR-008-plugin-migration-ownership.md)):
|
||||
|
||||
- The core chain (`flask db upgrade`) created every table that existed through
|
||||
its head, including the bundled-plugin tables. Each bundled plugin's `0001`
|
||||
migration is a stamp-only no-op anchor recording that fact.
|
||||
- From the cutover forward, a change to a plugin's schema lands as
|
||||
`plugins/<name>/migrations/versions/000N_*.py`, never in the core chain. The
|
||||
core chain is reserved for core tables.
|
||||
- A plugin built AFTER the cutover (e.g. `measuringtools`) is different: the
|
||||
core chain never created its tables, so its `0001` is a REAL baseline that
|
||||
creates them, not a no-op anchor. See [PLUGIN-GUIDE.md](PLUGIN-GUIDE.md) for
|
||||
the anchor-vs-baseline distinction.
|
||||
|
||||
Deploys and upgrades run `flask db upgrade` then `flask plugin upgrade-all`.
|
||||
`upgrade-all` stamps every plugin anchor and applies any later plugin
|
||||
migrations; it is idempotent. The registry (`instance/plugins.json`) records
|
||||
which revisions each plugin has applied in `migrations_applied`.
|
||||
|
||||
## Distribution conventions
|
||||
|
||||
For sister-site plugins (per [ADR-003](adr/ADR-003-plugin-distribution.md)):
|
||||
|
||||
@@ -51,12 +51,16 @@ cd frontend && npm ci && npm run build && cd ..
|
||||
```bash
|
||||
# Docker:
|
||||
docker compose exec api flask db upgrade
|
||||
docker compose exec api flask plugin upgrade-all
|
||||
# venv:
|
||||
flask db upgrade
|
||||
flask plugin upgrade-all
|
||||
```
|
||||
|
||||
`flask db upgrade` applies any new migrations in the core Alembic chain. It is
|
||||
idempotent; running it when already at head is a no-op.
|
||||
`flask db upgrade` applies any new migrations in the core Alembic chain.
|
||||
`flask plugin upgrade-all` then applies any new per-plugin migrations (each
|
||||
bundled plugin owns its schema going forward - see ADR-008). Both are
|
||||
idempotent; running them when already at head is a no-op.
|
||||
|
||||
## Step 4: Re-seed permissions and settings
|
||||
|
||||
|
||||
123
docs/adr/ADR-008-plugin-migration-ownership.md
Normal file
123
docs/adr/ADR-008-plugin-migration-ownership.md
Normal file
@@ -0,0 +1,123 @@
|
||||
# ADR-008: Plugin migration ownership (per-plugin chains from the cutover)
|
||||
|
||||
- **Status:** ACCEPTED
|
||||
- **Date:** 2026-07-10
|
||||
- **Deciders:** cproudlock
|
||||
- **Supersedes:** the "Migration strategy (resolved)" section of ADR-004
|
||||
|
||||
## Context
|
||||
|
||||
ADR-004 resolved a Phase 7B footgun by folding every bundled plugin's tables
|
||||
into the single core Alembic chain (migration `7c04_fold_plugin_schema`), and
|
||||
later core migrations (`7d04`..`7d16`) kept adding plugin schema directly to the
|
||||
core chain. At the time this was the safe choice: bundled-plugin baselines and
|
||||
the core baseline had both been creating the same tables, so
|
||||
`flask plugin upgrade-all` would collide with `flask db upgrade`.
|
||||
|
||||
That resolution left the per-plugin Alembic engine
|
||||
(`shopdb/plugins/migrations.py`, `shopdb/plugins/alembic_template.py`, the
|
||||
`alembic_version_<plugin>` version tables, and `flask plugin upgrade-all`) fully
|
||||
built but unused for the bundled plugins. The framework is the product (per the
|
||||
project's charter); a plugin that cannot own its own schema is not really a
|
||||
plugin. Sister sites that adopt or fork a plugin need its schema history to
|
||||
travel with the plugin, not be entangled in the host's core chain. Keeping every
|
||||
future plugin table change in the core chain also means the core chain grows
|
||||
without bound and a plugin can never be cleanly removed.
|
||||
|
||||
The blocker ADR-004 worried about (double table creation) only exists while a
|
||||
plugin's own migration tries to CREATE tables the core chain already created.
|
||||
That is avoidable: history is immutable, so the tables already built by the core
|
||||
chain stay owned by the core chain; only NEW schema needs a new home.
|
||||
|
||||
## Decision
|
||||
|
||||
Ownership splits at a fixed cutover, the current core-chain head.
|
||||
|
||||
1. **Core chain owns history through its head.** The core Alembic chain
|
||||
(baseline `68b3947ae14f` .. head `7d16_directoryemployees`) remains
|
||||
authoritative for every table that exists at the cutover, including the
|
||||
bundled-plugin tables it created. Those migrations are immutable and are not
|
||||
rewritten. `flask db upgrade` continues to reproduce the full schema.
|
||||
|
||||
2. **Plugin chains own plugin schema going forward.** From the cutover forward,
|
||||
any change to a plugin's schema lands as
|
||||
`plugins/<name>/migrations/versions/000N_*.py` in that plugin's own chain,
|
||||
never in the core chain. The core chain is reserved for core tables.
|
||||
|
||||
3. **Every table-owning bundled plugin gets a `0001` anchor.** Each such plugin
|
||||
carries a migration chain whose first revision is a stamp-only no-op:
|
||||
`upgrade()` does nothing because the core chain already created the tables.
|
||||
The anchor exists so the plugin chain has a base that
|
||||
`flask plugin upgrade-all` can stamp into the per-plugin version table
|
||||
`alembic_version_<plugin>`. Blueprint-only plugins that own no tables get no
|
||||
chain.
|
||||
|
||||
4. **Deploy and upgrade sequence.** A deploy runs `flask db upgrade`
|
||||
(core chain, creates everything through the head) then
|
||||
`flask plugin upgrade-all` (stamps every plugin anchor and applies any later
|
||||
per-plugin migrations). The same two commands upgrade an existing install;
|
||||
both are idempotent. The registry (`instance/plugins.json`) tracks each
|
||||
plugin's applied revisions in `migrations_applied`.
|
||||
|
||||
5. **Table-ownership registry.** `PLUGIN_TABLE_OWNERS` in
|
||||
`shopdb/plugins/alembic_template.py` is the explicit map of which tables each
|
||||
plugin owns; it is kept in sync with the plugins' `__tablename__` declarations
|
||||
and pinned by `tests/test_plugin_migrations.py`.
|
||||
|
||||
External (out-of-tree) plugins per ADR-003 already shipped their own chains;
|
||||
this ADR brings the bundled plugins onto the same model, so there is one rule
|
||||
for all plugins.
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- A plugin's schema history travels with the plugin. Adopting or forking sites
|
||||
get the plugin's migrations, not a slice of someone else's core chain.
|
||||
- The core chain stops accreting plugin schema; it stays about core tables.
|
||||
- A plugin can be evolved (or, with its own downgrade, removed) independently.
|
||||
- The long-built per-plugin Alembic engine is finally exercised on every deploy,
|
||||
so it cannot silently rot.
|
||||
|
||||
### Negative / cost
|
||||
|
||||
- Two migrate commands per deploy instead of one. Documented in `docs/DEPLOY.md`
|
||||
and `docs/UPGRADE.md`; both are idempotent so the cost is one extra safe call.
|
||||
- A plugin author must now put new tables in the plugin chain and register them
|
||||
in `PLUGIN_TABLE_OWNERS`, rather than autogenerating into the core chain. The
|
||||
`flask plugin new` guidance and this ADR spell that out.
|
||||
- The cutover is a discontinuity: tables created before it are core-owned,
|
||||
tables created after it are plugin-owned. The line is the core-chain head at
|
||||
this ADR's date, recorded here so it is unambiguous.
|
||||
|
||||
### Neutral
|
||||
|
||||
- No schema changes and no data migration: the anchors are no-ops. A fresh
|
||||
install and an existing install converge to the same state.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
1. **Keep everything in the core chain (status quo per ADR-004).** Simplest
|
||||
operationally but defeats the plugin-as-product goal: plugin schema cannot
|
||||
travel, the core chain grows without bound, and plugins can never be cleanly
|
||||
removed. Rejected.
|
||||
2. **Rewrite history so plugin tables move out of the core chain into plugin
|
||||
`0001` CREATE migrations.** Would make each plugin chain self-contained from
|
||||
empty, but breaks the immutability rule, forces every existing site to
|
||||
re-run a rewritten chain, and re-introduces the exact double-creation footgun
|
||||
ADR-004 fixed. Rejected.
|
||||
3. **Anchor that CREATEs tables with `IF NOT EXISTS` guards.** Lets a from-empty
|
||||
install build plugin tables from the plugin chain, but then two chains both
|
||||
claim the same tables and drift can diverge silently. The no-op anchor keeps
|
||||
a single authoritative creator (the core chain) for cutover-era tables.
|
||||
Rejected.
|
||||
|
||||
## References
|
||||
|
||||
- ADR-003 (plugin distribution; external plugins already ship chains)
|
||||
- ADR-004 (deployment topology; this ADR supersedes its migration-strategy note)
|
||||
- `shopdb/plugins/alembic_template.py` (`PLUGIN_TABLE_OWNERS`, shared env runner)
|
||||
- `shopdb/plugins/migrations.py`, `shopdb/plugins/cli.py` (`upgrade-all`)
|
||||
- `plugins/<name>/migrations/` (per-plugin chains and `0001` anchors)
|
||||
- `tests/test_plugin_migrations.py` (ownership + chain + idempotency guards)
|
||||
- `docs/DEPLOY.md`, `docs/UPGRADE.md`, `docs/PLUGINS.md` (deploy sequence)
|
||||
146
docs/adr/ADR-009-frontend-plugin-gating.md
Normal file
146
docs/adr/ADR-009-frontend-plugin-gating.md
Normal file
@@ -0,0 +1,146 @@
|
||||
# ADR-009: Frontend plugin gating
|
||||
|
||||
- **Status:** ACCEPTED
|
||||
- **Date:** 2026-07-10
|
||||
- **Deciders:** cproudlock
|
||||
- **Supersedes:** none
|
||||
|
||||
## Context
|
||||
|
||||
The backend plugin system (ADR-002, ADR-003) lets an operator disable a
|
||||
plugin. Disabling it unregisters the plugin's API blueprint, drops its
|
||||
rows from global search (see `test_search_disabled`), and removes its
|
||||
entry from `get_navigation_items()`, so the disabled feature's sidebar
|
||||
link disappears.
|
||||
|
||||
The frontend told a different story. Every plugin's Vue routes and views
|
||||
ship inside the core bundle. `frontend/src/router/index.js` auto-discovers
|
||||
them with `import.meta.glob('./routes/*.js')`, so a route like `/usb` or
|
||||
`/printers/3` is registered regardless of whether the owning backend
|
||||
plugin is enabled. A user who typed the URL, followed a stale bookmark,
|
||||
or clicked a cross-link reached a page whose API calls all 404, landing
|
||||
on a broken shell instead of a clean redirect. The navigation was already
|
||||
data-driven; direct-URL reachability was the gap.
|
||||
|
||||
There is no frontend plugin system yet. The product is packaged as one
|
||||
core Flask app plus backend-only plugins; the Vue app is monolithic and
|
||||
build-time static. Any gating has to live in core frontend code because
|
||||
that is the only place the plugin routes exist.
|
||||
|
||||
## Decision
|
||||
|
||||
### Step 1 (this ADR, implemented): route-level gating
|
||||
|
||||
Plugin-owned frontend routes are gated against the backend's enabled-plugin
|
||||
list. This is the whole of what ships now.
|
||||
|
||||
1. **Enabled-list endpoint.** A new `GET /api/plugins/enabled`
|
||||
(`jwt_required(optional=True)`) returns a flat JSON array of enabled
|
||||
plugin name strings and nothing else. It is a cheap registry read
|
||||
(`registry.get_enabled_plugins()`), no database access. Exposing it to
|
||||
anonymous callers is safe because `GET /api/dashboard/navigation`
|
||||
already leaks the same enabled/disabled signal, and unauthenticated
|
||||
kiosk routes (`/tv`) need the answer too. It carries no metadata, so it
|
||||
reveals strictly less than the admin-gated `GET /api/plugins`.
|
||||
|
||||
2. **Route tagging.** Every plugin-owned route carries `meta.plugin =
|
||||
'<pluginname>'`. This covers the per-plugin route modules
|
||||
(`routes/computers.js`, `routes/equipment.js`, ...) and the
|
||||
plugin-owned routes that physically live in core files: the
|
||||
PC-relationships and toner reports, the slide manager and `/tv`
|
||||
dashboard (slides), the printer-QR and USB-label print pages, and the
|
||||
employee-detail page. Genuinely core routes (dashboard, search, map,
|
||||
applications, reference-data settings) stay untagged and are never
|
||||
gated.
|
||||
|
||||
3. **Cached fetch, fail-open.** A composable
|
||||
(`composables/enabledPlugins.js`) fetches the list exactly once behind
|
||||
a cached promise. If the fetch fails or returns a non-array, the code
|
||||
fails **open**: every plugin is treated as enabled. A transient API
|
||||
error must never brick navigation. The cost is that a disabled
|
||||
plugin's page is briefly reachable during an outage, which is
|
||||
acceptable because its API calls would fail anyway and the next
|
||||
successful fetch closes the gap.
|
||||
|
||||
4. **Router guard.** `router.beforeEach` awaits the cached fetch when the
|
||||
target route has `meta.plugin`. If that plugin is not enabled it
|
||||
redirects to `/` and raises an info toast. The endpoint is
|
||||
jwt-optional, so the guard works for both authenticated pages and the
|
||||
unauthenticated `/tv` kiosk route.
|
||||
|
||||
This is intentionally a thin layer. It does not change the plugin
|
||||
contract surface, so `__contract_version__` does not move: adding a core
|
||||
HTTP endpoint and tagging core-shipped routes are not plugin-contract
|
||||
changes. Plugins still ship no frontend code of their own.
|
||||
|
||||
### Non-goals for step 1
|
||||
|
||||
- No build-time or runtime loading of plugin-authored Vue code.
|
||||
- No per-permission or per-role route gating (that stays with the
|
||||
existing `requiresAuth` / `requiresAdmin` meta flags).
|
||||
- No removal of disabled routes from the route table; they remain
|
||||
registered and are intercepted by the guard. Keeping them registered
|
||||
avoids a rebuild when a plugin is toggled and keeps the redirect path
|
||||
simple.
|
||||
|
||||
## Future direction (PROPOSED, not implemented)
|
||||
|
||||
Step 1 gates routes that core already owns. The longer-term goal is a
|
||||
real frontend-plugin contract where a plugin ships its own frontend and
|
||||
core discovers it, mirroring the backend model. Sketch:
|
||||
|
||||
1. **Plugin-owned frontend tree.** Each plugin gains a
|
||||
`plugins/<name>/frontend/` directory holding its route module, views,
|
||||
and any plugin-specific components. Core stops carrying `views/usb`,
|
||||
`views/printers`, and so on.
|
||||
|
||||
2. **Build-time discovery.** The Vite build discovers plugin frontends
|
||||
with a glob over `plugins/*/frontend/routes.js` (analogous to today's
|
||||
`import.meta.glob('./routes/*.js')`), so a plugin's presence in the
|
||||
tree is what puts its routes in the bundle. Combined with the step-1
|
||||
enabled-list gate, a plugin that is absent from the build ships no
|
||||
code and a plugin that is present-but-disabled is route-gated at
|
||||
runtime.
|
||||
|
||||
3. **Shared component + registration contract.** Plugins register into
|
||||
named extension points instead of editing core files: an `iconMap`
|
||||
registration for nav/asset icons, asset-detail panels, map-marker
|
||||
renderers, and search-result renderers (the "Frontend hook contract"
|
||||
already listed as deferred in the project CLAUDE.md). Core exposes a
|
||||
stable set of shared components (form controls, detail-page shells,
|
||||
table primitives) as the plugin frontend's only allowed core imports,
|
||||
the frontend analogue of the `shopdb.api` namespace.
|
||||
|
||||
4. **Versioned frontend contract.** The shared-component and
|
||||
registration surface would be versioned the same way the backend
|
||||
contract is (ADR-002), so a plugin frontend can declare the core
|
||||
frontend range it needs.
|
||||
|
||||
### Tradeoffs of the future direction
|
||||
|
||||
- **Pro:** true plugin self-containment; a site can drop in or remove a
|
||||
plugin (frontend and backend together) without patching core; smaller
|
||||
core; clearer ownership.
|
||||
- **Con:** significant build-system work (per-plugin Vite entry
|
||||
discovery, code-splitting, dev-server HMR across the plugin tree); a
|
||||
new versioned frontend contract to maintain and document; a migration
|
||||
that moves ten plugins' worth of views out of core; risk of a leaky
|
||||
shared-component surface becoming an accidental contract. The payoff
|
||||
only matters once external (out-of-tree) plugins with their own
|
||||
frontends are a real requirement. Until then, step 1's route gating
|
||||
delivers the user-visible correctness (no reachable dead pages) at a
|
||||
fraction of the cost.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Disabling a backend plugin now makes its frontend routes redirect to
|
||||
the dashboard instead of loading a broken shell. Behavior matches the
|
||||
already-dynamic navigation.
|
||||
- One extra lightweight request at app boot (`GET /api/plugins/enabled`),
|
||||
cached for the session.
|
||||
- Fail-open means gating is a UX guardrail, not a security control. It is
|
||||
not a substitute for backend authorization: the API still enforces auth
|
||||
and the disabled plugin's endpoints are simply unregistered. Never rely
|
||||
on route gating to protect data.
|
||||
- The future frontend-plugin contract remains open work; this ADR records
|
||||
the direction and its cost so a later decision can pick it up.
|
||||
@@ -20,6 +20,8 @@ Each ADR captures a single architectural decision: the context, the decision its
|
||||
| [005](ADR-005-equipment-vs-measuringtools.md) | Equipment vs measuringtools plugin scope | ACCEPTED |
|
||||
| [006](ADR-006-collector-contract.md) | Plugin collector contract pattern | ACCEPTED |
|
||||
| [007](ADR-007-product-versioning-and-releases.md) | Product versioning and releases | ACCEPTED |
|
||||
| [008](ADR-008-plugin-migration-ownership.md) | Plugin migration ownership (per-plugin chains) | ACCEPTED |
|
||||
| [009](ADR-009-frontend-plugin-gating.md) | Frontend plugin route gating | ACCEPTED |
|
||||
|
||||
## Authoring
|
||||
|
||||
|
||||
Reference in New Issue
Block a user