The scanner has been reporting the same count for weeks, which is what a rule
that only prints becomes. It now FAILS the build, and it looks where the leaks
actually were: PowerShell, the installer, the seeds, generated JSON, the
frontend - case-insensitively, across plugins, shopdb, scripts, deploy, tools.
A line that is deliberate declares itself with an ADR-015-OK marker and a
reason, so the claim is visible in review instead of tolerated in silence.
What it found, fixed here:
- The shadow client wrote one site's ShopDB URL into HKLM whenever the registry
disagreed. At the site it was written for that reads as healing drift;
anywhere else it overwrites the site's own address on every enforce cycle,
and the site cannot win because the cycle repeats. The bay's value now wins,
an explicit -BaseUrl seeds it, and with neither there is nothing honest to
write, so it says so and skips.
- The kiosk dispatcher fell back to one plant's host when HKLM was unset, so a
kiosk elsewhere quietly opened a server it has no business reaching. The
fallback is now this site's site_base_url, baked in at seed time, and the
dispatcher refuses rather than guessing when neither is set. Its legacy
shortcut matcher derives the host from that URL instead of naming one.
- The OpenAPI generator hardcoded a production hostname into every spec it
generated, which then published to a public wiki. The relative mount is the
only server it can honestly name; a site passes its own by environment.
- Placeholders and examples in the UI and the client help offered real internal
subnets and a real production URL. They now use documentation ranges.
Both publication gates - the export scrub and the docs publishability test -
carry the site patterns, which neither did. One plant's hostname, FQDN and
internal networks are out of the documentation and the generated specs.
Comments naming the reference site are reworded rather than deleted: the
reasoning is worth keeping, the plant name is not what makes it true.
Deleting a relationship is soft, so the row survives with isactive False, and
three things read them without knowing that.
Re-adding a deleted link answered 409 "this relationship already exists" about a
link the page no longer shows, and there was no way forward from the UI at all -
the row cannot simply be inserted again, since the triple is unique.
Reactivating IS the create for an inactive row.
The inverse guard blocked on a deleted inverse, which made "remove the existing
one first" - the instruction in its own message - fail to unblock anything.
fix-controls-direction retired the reversed row whenever a correctly-directed
one existed, without checking whether that one was itself deleted. So it removed
the only live link and reported a successful clean-up. It now reactivates the
row pointing the right way before retiring the one pointing the wrong way.
These are the commands the docs tell an operator to run against production.
An SVG is an XML document that may carry a script, and it is an accepted image
type because floor-plan maps and branding genuinely want vector. Loaded through
an img tag that script never runs, so the tiles and maps were never the risk.
Opening the file's own URL is - and the application image route is public, so
that URL needs no session.
Every route that serves an upload now goes through one helper that sends
Content-Security-Policy: default-src 'none'; sandbox, and nosniff. Seven routes
across core and five plugins, so a new one added later starts from the same
place rather than repeating the reasoning. Banning the format instead would
have cost the maps their only sensible one.
The app also sent no security headers at all. It now sets nosniff,
frame-ancestors self (as X-Frame-Options too, for the display bays' browsers)
and a referrer policy. Deliberately NOT a page-wide CSP: this serves an SPA with
inline styles, so a real script-src policy is a change worth making with the
frontend in front of you, and a permissive header claiming one would be worse
than having none.
Contract 0.19.0. send_upload is on the shopdb.api surface, because a plugin
serving user-supplied bytes should not have to remember these headers. The same
bump records that get_dashboard_widgets has taken data and shape rather than a
component name since the dashboard was rebuilt - that shipped without a bump,
while BasePlugin and PLUGIN-HOOKS.md both still documented the shape nothing
renders, which is how five plugins came to declare widgets pointing at
components nobody had written.
Core's Alembic env got connection.commit() when the stamp bug was found; the
per-plugin template did not. MySQL commits DDL implicitly, which flushes
everything queued before it including the previous migration's version stamp,
and the LAST migration of a run has no DDL after it - so its stamp rolled back
at close while its schema change survived. flask plugin upgrade-all then exited
0 having silently re-run that migration, and re-ran it again on every deploy
after. Invisible for exactly as long as every plugin head happened to be
idempotent.
Two were not.
backups 0003 cleared lastseenat for EVERY row, which is correct once and
destroys evidence on each repeat. It is now scoped to the backfill's actual
signature, COALESCE(collectedat, createdat) - the expression 0002 wrote - plus a
date bound. Both conditions are needed. Matching on collectedat alone misses
every row whose collectedat is NULL, so precisely the rows carrying the most
invented value would have kept it forever; and value equality is not a signature
on MySQL, where db.DateTime is second-precision and the collector writes both
stamps in one statement, so a genuinely fresh revision would read as a backfill
and be wiped. SQLite keeps microseconds, which is why no test could show it.
geenforce 0003 added a column unconditionally, so it failed on a fresh database
built from the models and on any re-run. Guarded like network0003prefix.
Prod grew rows saying both "PC controls 2005" and "2005 controls PC", and a CMM
PC showing "<- controls from CMM4" beside its own outgoing link. Only one
direction can be true: a PC drives a machine, never the reverse.
Nothing stopped it. The duplicate check was keyed on (source, target, type), so
the inverse inserted cleanly, and the Add Relationship dialog offers an incoming
direction that writes exactly that. The legacy import stores controls the wrong
way round as well. Directional creates now refuse the reverse with a 409 naming
the row that already holds it, and refuse self-links, which render as a
duplicate on the asset's own page and mean nothing. Symmetric types are exempt:
Dualpath stores both directions on purpose and the card collapses them. The
propagation fan-out got the same guard so a rail meant to spread one direction
across sibling bays cannot manufacture a pair.
fix-controls-direction only matched source assettype 'machine', so every
measuring_tool, printer and network_device row it was written to clean survived
it - which is why running it would never have fixed the CMM. It now matches any
non-computer controlled BY a computer.
New `flask relationships audit` reports what is already stored: reciprocal
pairs, self-links, and PCs controlling several assets of one type. Read-only,
and it prints each row's label because that usually names the writer outright -
collector:* means this code made it, anything else means a person or the import
did. That distinction decides the fix for duplicate device assets, which is not
in this commit: the collector keys idempotency on its own label, so a device
somebody created by hand is invisible to it and it mints another, and the
adoption rule needs the audit run against prod before it can be written.
Two false positives were found writing it, against the dev database, and both
would have made the report useless. A self-link is its own inverse, so it was
counted as a reciprocal pair AND printed twice. And Dualpath siblings looked
like duplicate devices - a dual-bay machine is one physical machine with one
controller and controls is propagated to both bays deliberately. That was 30 of
32 findings, consecutive bay numbers pair by pair.
Core called the roles dashboard / lobby / partskiosk. The kiosks call them
Dashboard / Lobby / 3DPrintRoom, which are the literal contents of
C:\Enrollment\display-type.txt, read by the GE-Enforce dispatcher to pick a
target. Two vocabularies for three kiosks, each with its own copy of the same
route map.
That is not cosmetic. A display reporting its own type sends what its file
says, so it could report a role core would not accept, and core could store
'partskiosk', a value no dispatcher would ever match. The enforcement report
column would have shown one vocabulary from the device and the other from the
DashboardDefault fallback, in the same column.
The machine's file wins, because that is what a person edits. DISPLAY_ROLE_PATHS
takes the kiosk spelling and the display scope now uses that dict rather than
holding a second one, so the two cannot drift again. normalize_display_role
resolves any casing and the retired 'partskiosk' forward; the dispatcher already
matched its map case-insensitively and the server now agrees with it.
Nothing is turned away over a capital: the API accepts any spelling and stores
the canonical one, displaypath resolves through the normalizer so rows written
before this keep working, and the settings dropdown canonicalises on open so an
old value does not render as a blank select.
A reported subtype is normalised on the way in, but an UNRECOGNISED one is kept
verbatim. That is a kiosk with a typo in its file or a role nobody declared, and
both are worth seeing in the fleet table rather than blanked or guessed at.
Contract bumped for the added names. DashboardDefault is finally listed in
__all__ too - 0.17.0 put it on the surface and never exported it.
The enforcement reports needed to name what a display IS, and reached
straight into shopdb.core.models.dashboarddefault to do it. Plugins may
only touch core through shopdb.api, and the contract test said so.
The role belongs on the surface rather than behind it: it lives in core,
no plugin owns it, and a plugin reporting on displays has no other way to
resolve it. Added there and the version bumped, which the docs test pins.
Adding an application meant typing an image FILENAME and trusting someone had
dropped the file into the frontend's own directory by hand, and typing an
install path from memory. Both are uploads now, following the model-image trio
that models and part photos already use.
The two differ deliberately. The image is public, because application tiles
render before anything is authenticated. The installer is not: it is licensed
vendor software, an open URL would publish it to anything that can reach the
site, and it is always sent as an attachment rather than rendered.
Installers are capped at 500MB and the size is measured by seeking the stream
rather than trusting Content-Length, which a chunked upload does not send and a
client can understate. Anything larger belongs on the share, and the error says
so rather than just refusing.
Files are chosen before a new application exists, so they are held and uploaded
once there is an id to attach them to. A failed upload leaves the saved record
alone and reports, rather than losing what saved fine.
Removing an installer only clears installpath when it pointed at the upload - a
share path was typed by a person and is not ours to wipe. The detail page reads
both shapes, since entries from the classic site hold a bare filename that is
still served from /images/applications/.
"All assets 704" sat beside "all assets in use 737", and both were correct
about different populations. The totals summed five specific asset types and
subtracted dual-bay secondaries; the status counts took every asset row of any
type with no collapse, so USB devices and hidden secondary bays inflated one
side of a comparison the layout invites. Status is now counted over exactly the
same assets the totals describe.
Warranty rows fell back to asset.name when the covered asset had no hostname,
and an asset's name is usually the MACHINE's descriptive name - which is how a
column meant to identify a PC ended up showing a machine. Hostname, else the
asset number, never the name. The machine number loses its label too: the row
is hostname, machine, state, and "machine 3015" spends a word on what position
already conveys.
Printer names now carry the floor-plan preview on hover, the same
LocationMapTooltip the printer's own page uses - a location name tells you the
room, the map tells you where to walk. Declared as map.maphover on the card, so
any card with coordinates gets it; a row without them shows a plain link rather
than being dropped.
Cards are four across rather than five. At five columns a row holding a
hostname, a machine number and a state truncates on exactly the rows that
matter. auto-fit, so two cards fill the width instead of leaving empty tracks.
Not covered by a test: the count fix. I started one and it was interrupted, and
I have not gone back for it - the assertion worth having is that in-use can
never exceed the total.
Product version to 0.9.0, frontend in lock-step, Unreleased notes moved into a
dated section per ADR-007. The plugin contract stays at 0.16.0: it moved after
v0.8.1 and is already recorded in this release's notes, and nothing since
touched the contract surface.
A minor rather than a patch: collector behaviour changed in ways an integrator
must know about. A reported machine number no longer becomes the PC's asset
number, it builds a controls link instead; a second PC claiming a machine is
treated as a claim rather than a handover; and a backup revision chain is now
per source PC rather than per asset.
A PC that drives a device which is its own asset had been implemented twice.
METROLOGY_TOOL_MAP covered CMM, Keyence, Genspect and wax-trace, minting a
measuring_tool. A separate path keyed on one hardcoded pc-type minted a Part
Marker machine and filed it under its operation. Both create a device, link the
PC with controls, and archive that link when the PC is re-imaged: one mechanism
with different nouns, written out twice because the second case arrived later.
That is the same trap as the site literals in ADR-015 - a pattern implemented
per instance rather than declared - and it has a known next occurrence. Part
markers already share operation numbers, and any site with two marking lasers
or two wax-trace units on one number needs identical treatment.
One SUBORDINATE_DEVICE_MAP now declares asset type, type name, naming suffix,
whether the device files partof the operation, and the relationship label. The
labels are unchanged per case on purpose: those values are in the production
database and only rows carrying them are archived by a collector push. A site
overrides or adds an entry through subordinatedevice_<pctype> settings, per
ADR-015, so the next case needs no code. A malformed override falls back to the
default rather than failing the push, because a bad setting must not stop a bay
reporting its inventory.
metrology_tool_for stays as a shim over the same map: filters.py and the older
tests read it, and unifying must not change what it returns. A test pins that.
Also adds flask relationships check-shared-machines, which finds the next 0615
rather than waiting for someone to notice duplicate backups. Several devices
legitimately sharing a number and two PCs mis-numbered at imaging look the same
from outside; the difference is whether child assets exist, so that is what it
reports. Read-only.
Adds a kind-pluggable backups plugin. Configuration captured from a PC is
filed against the MACHINE it controls, with a revision history and download
back to the native format.
NTLARS/DNC is the first kind. Settings live in the controlling PC's registry
but describe the machine, so revisions attach to the machine's asset and carry
no foreign key to the PC: history survives a PC being replaced or deleted, and
sourcehostname records the handover.
Storage splits by kind. Parseable kinds store a dialect-neutral JSON
projection in ShopDB and re-render on download; opaque vendor formats (part
marker and similar) keep their bytes on the SFLD share with ShopDB holding
metadata and the UNC pointer.
Two .reg dialects exist in the wild: NTLARS's own Save... export omits the
WOW6432Node path segment, scripted exports include it. Parsing strips whichever
root matched, so a stored revision commits to neither and download offers both
(NTLARS Load... by default, WOW6432Node for direct reg import). Getting this
backwards is silent, so the dedup hash deliberately excludes sourcedialect and
both dialects of one config dedup to a single revision.
Dedup is load-bearing: the collector runs every GE-Enforce cycle across the
fleet, so a revision is inserted only when the content hash differs from that
asset's latest for that kind.
A freshly imaged PC opens NTLARS with a blank General tab. Recording that would
make an empty config the newest revision exactly when someone needs the last
good one, so a blank MachineNo is rejected rather than accepted as a change.
Two of the 320 known-good backups on the share already have that shape.
DNC Info card summarises the latest revision on the machine page: General
(Cnc, NcIF, HostType), eFocas, Serial, NTSHR when populated (only 18 of 147
machines), and MARK when the machine is a marker. MARK is gated on Cnc=MARKER
or the ShopDB machine type, not on the MARK key having content: MARK carries
serial defaults on 145 of 147 machines and DncPatterns reads YES on 103
including ordinary lathes, so neither identifies a marker.
The info card is owned by the kind (BackupKind.infopanel/buildinfo) and served
by a generic endpoint, so the expected successor to DNC ships its own card by
adding a class rather than changing the plugin or the panel wiring.
Also: schedule and retention settings with a prune that never drops the newest
or the oldest revision, and scripts/import_ntlars_backups.py to seed history
from the existing per-machine .reg files (144 of 147 resolve to assets).
Codec verified against all 320 real backups: round-trips clean through both
dialects. Bay-side generation verified on Windows against reg.exe export.
Three defects, all found on printedparts_label_prefix, all one root cause:
nothing in the framework knew that setting existed.
The parts kiosk runs logged out. An unauthenticated read of a setting is
limited to an allowlist, the key was not on it, so the kiosk got a 404 and
fell back to no prefix. An admin previewing the same page while logged in saw
the prefix, which is why it looked like it worked.
The same setting also looked like it would not save. The row did not exist on
a site that installed the plugin before the setting was added, so the first
save created it - under the placeholder category the settings API uses for
keys it does not recognise, where the plugin's settings page, which lists by
category, could no longer see it. The value was in the database the whole
time.
And the row was missing in the first place because seeding ran from
on_install / on_enable, which fire only on a state transition. Neither runs
again on an upgrade, so a setting added in a later plugin version never
reached a site that installed an earlier one. The comment claiming enable ran
every upgrade cycle was simply wrong.
A plugin now declares the settings it owns in get_settings_defaults(): key,
default, type, category, description, and whether a logged-out page may read
it. The framework seeds declared keys at install, at enable, and on every
flask plugin upgrade-all; files a first-time write under the declared
category; re-homes any row left in the placeholder category, value untouched;
and answers an anonymous read for keys marked public. Core carries no list of
any plugin's keys.
Contract 0.16.0 (additive optional hook). printedparts and printers move to
the hook and floor their core_version at 0.16.0. The dev database had two rows
in the misfiled state (printedparts_alert_email, employee_db_host); the first
repairs itself on the next upgrade pass.
The installer on the share was stamped 0.8.0 and contained sixteen commits that
were not in the v0.8.0 tag. A build that misreports its own version is exactly
what the version check in docs/RELEASING-WINDOWS.md exists to prevent, and it
would have left two sites unable to say what they were running.
Everything in 0.8.1 landed after v0.8.0 was tagged this morning, driven by two
sites entering real data for the first time: the blank-code 500, the two-slide
display that never rotated, modals discarding a part-filled form, filters
returning an empty page, model photos that could not be saved, and the shared
equipment catalog that lets a new site start with vendors, models and printer
supply part numbers already present.
CHANGELOG gains a 0.8.1 section, and the OpenAPI document follows __version__
rather than being restated.
`flask seed reference-data` wrote a dozen generic model types and no vendors or
models at all, so adopting this platform began by retyping a catalog another
site had already spent a year building. That is the largest single obstacle to
standing a new facility up.
scripts/export_catalog.py dumps the catalog from a live instance to
shopdb/data/catalog.json, and `flask seed catalog` loads it. What travels:
vendors 53, models 128, modelsupplies 146, modeltypes 35, machinetypes 21,
computertypes 10, printertypes 9, networkdevicetypes 5, locationtypes 11,
operatingsystems 14, measuringtooltypes 8, notificationtypes 3,
accessprotocols 3
The 146 printer supplies are the most useful part after the models themselves:
every toner, drum and maintenance kit with its part number, colour, capacity
tier and page yield, already matched to the right model, instead of somebody
reading them off spent cartridges.
IDEMPOTENT and ADDITIVE. Records match on a natural key - a vendor by name, a
model by vendor plus model number, a supply by model plus part number - so a
second run adds nothing, and it never updates or deletes: a site that corrected
a description or pointed a model at its own photo keeps its version.
Catalog only. No assets, locations, employees, business units or anything with a
serial number: nobody wants one plant's machines appearing at another. Vendor
contact details are excluded too, since a rep's name and number belong to
whoever holds that relationship. supportteams, printerdrivers and customfields
are site-specific and deliberately absent.
Models and supplies reference their vendor by NAME rather than id, because ids
differ between databases and an id-keyed catalog would silently attach part
numbers to the wrong printer.
The installer offers it as a tick-box on a new "Starter data" page, defaulting
to on, passing -SeedCatalog to stage 3. Offered rather than assumed: a site that
machines nothing does not want 21 machine types cluttering its dropdowns.
Verified by loading into an empty database and running twice: every group
populated on the first pass, "Catalog already present, nothing to add" on the
second.
A server whose migrations or seeds never finished does not fail politely. Most
pages answer 500 and settings endpoints answer 404 for keys that were never
created, which reads as a broken application rather than an unfinished install.
One site spent a morning being debugged that way.
`shopdb-admin.ps1 repair` runs what stage 3 of the installer runs: db upgrade,
plugin upgrade-all, and the three seeds. Every step is idempotent, so running it
on a healthy server changes nothing, and each step runs independently so one
failure does not silently skip the rest.
`check` now says so before anyone has to infer it:
THIS SERVER IS NOT FULLY PROVISIONED
- seed data is missing (permissions, settings or reference data)
Most pages will answer 500 until this is fixed. Run:
shopdb-admin.ps1 repair
That needs a real test to sit on, so `flask db-utils seed-state` reports each
seed group and exits non-zero when any is missing. Verified by emptying the
settings table inside a transaction: MISSING, exit 1, rollback clean. Without it
the console check would have looked reassuring while testing nothing - an older
build with no such command reports UNKNOWN rather than healthy, for the same
reason.
A site reported "internal server error" adding a second business unit. It was
reproducible: create one with a blank code, create another with a blank code,
500.
A column that is unique and nullable accepts any number of NULLs - that is what
makes "optional but unique" work - and exactly ONE empty string. The form sent
'', so the first blank code saved and every one after it collided with it. The
field showed no asterisk because it genuinely is optional; the database just
behaved as though it were not.
This is not specific to business units. A dozen columns across core and the
plugins are unique and nullable - asset numbers, hostnames, item codes, subnet
names, gage-lab tags - and each was one blank form away from the same 500.
Fixing them an endpoint at a time would have left the next to be found by a
user, so a before_flush listener normalises blank to NULL on any unique nullable
text column. Listening on Session rather than on individual mappers covers
plugin models imported later, and avoids mapper-event semantics that differ
between SQLAlchemy versions.
A genuine duplicate is now a 409 with a readable message rather than a bare 500
with a traceback in the log: reusing a code that is taken is the caller's
mistake, not a server fault.
Verified against the development database: three business units with blank codes
all save, the blank stores as NULL, and a real duplicate code returns 409.
The Windows installer has never shipped under a version: v0.7.0 was tagged
before any of it existed, so every build handed out so far stamped a server with
0.7.0. Two servers running different builds were indistinguishable, and the
installer logged each upgrade as "same version already installed" rather than
recording what changed. This cuts the release that fixes that.
0.8.0 rather than a patch: the air-gapped installer is a new capability, and
pre-1.0 semantic versioning puts that in the minor slot (ADR-007).
CHANGELOG gains a 0.8.0 section covering the twelve defects a real Windows
Server 2019 install surfaced, the move from inferring "is this a re-run of my
install?" to recording it, and the operator documentation.
deploy/site-profile-universal.json is now in the repository. Released builds
were being produced from a profile in a temporary directory, so the next release
could not have been reproduced once that file was cleaned up.
docs/RELEASING-WINDOWS.md points at the committed profile and says why.
scripts/gen_openapi.py reads __version__ out of shopdb/__init__.py instead of
restating it. Its hardcoded copy had already drifted a release behind, which is
the same mistake that once shipped an installer stamped with the wrong version.
Adopting a site means getting its asset register in. The HTTP import API suits a
site with a source system and someone to script against it; a sister site with a
spreadsheet and no developer needs something else, and that is the common case.
FOREIGN KEYS TAKE NAMES. This is the whole design. A CSV row has to say where an
asset is, and the database stores locationid, an integer. Requiring the number
means importing locations, reading back the generated ids and pasting them into
the asset sheet - a workflow nobody finishes. Every foreign key here accepts
either a numeric id or the referenced row's name:
assetnumber,assettypeid,statusid,locationid
CMM-01,Measuring Tool,Active,Gage Lab
The column keeps its database name, per CONTRIBUTING.md; the value is whatever
the operator actually knows. Names resolve across files in one run, so
assets.csv can reference a location that only exists because locations.csv was
read moments earlier. A name that does not resolve is reported with its line,
column and value, not as a foreign key violation from three layers down.
Dry run is the default, and writes go into the transaction either way - the
rollback is what makes it a dry run. Skipping the writes instead made every
cross-file reference fail, which is the one thing a folder-wide check exists to
verify. Validation covers every row before anything is written, so a typo on
line 400 cannot leave 399 rows imported. Files are matched on a natural key, so
correcting a spreadsheet and re-running updates rather than duplicates.
TEMPLATES ARE GENERATED, NOT MAINTAINED. "flask csv templates" builds them from
the live schema, annotated with required/optional and which file each foreign
key refers to. The prompt for this was a hand-written template set that had
invented columns on seven of eleven tables and named a table that does not
exist, while looking entirely plausible - and described an import mechanism
(a Data Import page, a flask import-csv command) that had never existed. A test
fails the build if a generated template ever offers a column the schema lacks.
User accounts are deliberately not importable: passwords do not belong in a
spreadsheet in either direction.
Verified end to end against MySQL 5.6 - a folder dry run catching one bad
reference, the fix, the commit, and a re-run reporting updates rather than
inserts. 16 tests.
CLIENT IP / SPOOFABILITY. docs/geenforce-api-cutover.md claimed that removing the
IIS rewrite rule made the allowlist fail closed and that it does NOT become
spoofable. The opposite is true. IIS never sets X-Forwarded-For on its own; the
rule is the only thing that does. Remove it and IIS still forwards whatever
X-Forwarded-For the CALLER sent, waitress trusts it because it arrives from
127.0.0.1, and remote_addr becomes attacker-controlled - so a token-less caller
can fetch manifests from anywhere on the network. The document and the
_trusted_client_ip docstring now say so, waitress runs with
--trusted-proxy-count=1, and stage 5 checks the rule is actually live rather than
assuming it. The wizard question is rephrased to something an operator can verify
with their network team instead of guessing at.
NON-ASCII. The style gate only ever checked .py/.vue/.js/.ts, so documentation
accumulated em-dashes, arrows and box-drawing characters against this repo's own
convention - including in files added this week. Cleaned, and the gate now uses
INCLUDES_ALL so Markdown, JSON and YAML are covered.
PLUGIN DEFAULTS. The wizard pre-ticked measuringtools and printedparts, both of
which ship default_enabled=false, so every site taking the defaults installed and
enabled them against their manifests. Inno has no JSON parser so the list must be
hardcoded, but tests/test_installer_defaults.py now fails when it drifts.
UPGRADES. The payload copy merges, so a plugin dropped from a site's profile kept
its code forever - which defeats a lean build and leaves core's optional-import
guards succeeding for a plugin the site no longer has. Stale plugin directories
are now deregistered and removed before the copy.
add-plugin used 'plugin install', which for the five default_enabled=false
plugins left them installed but DISABLED - and printed a green success line
anyway. It now goes through apply-profile, and the success line is gated on the
exit code. Invoke-Flask records its own exit status, because $LASTEXITCODE keeps
a stale value when flask.exe is missing and no native command runs.
CHARSET. The utf8mb4 compiler hook lived inline in migrations/env.py, so it
covered the CORE chain only: plugin baselines inherited the server default, which
on a latin1 server means two charsets in one database. It is now
shopdb/utils/mysql_charset.py, imported by both, and preflight reports the
database's default charset.
BACKUP HONESTY. The dump was described as 'all of your asset data'. Uploaded
branding and floor-map images live in instance\ on disk, not in the database, so
a restore from the .sql alone comes back with no map. backup now archives
instance\ alongside it and says both are needed.
VERSIONING. AppVersion was hardcoded at 0.9.0 while the product, the frontend and
the newest tag said 0.7.0 - and 0.9.0 collides with a retired contract version.
Both builders now generate version.iss from shopdb/__init__.py.
Smaller: rollback overwrites .env before deleting it, as uninstall already did;
appcmd unlocks are scoped to this site's location rather than server-wide, with
the wide unlock as a fallback; DEVELOPMENT-SETUP says Python 3.14; the README
plugin list gains printedparts; prune-schema --force is documented as
first-provisioning-only; HTTPS is documented as not-the-default with the steps to
add it; the DBA SQL is on the wizard's database page; the features page says
unticking does not remove an installed feature; and the installer README states
that bundle-lock cannot vouch for the exe itself - that needs signing or an
out-of-band hash, neither of which is wired up.
Two independent ways a restart leaves the SPA unable to navigate, both of
which look identical to a user - a click that does nothing.
1. The router awaits loadEnabledPlugins() to gate plugin routes. An app-pool
restart leaves that request hanging (IIS queues it while the worker starts)
and axios sets no timeout, so the navigation never resolves. Worse, the
promise is cached, so every later navigation awaited the same dead request
and stayed frozen long after the backend recovered. Bound the wait and fail
open on expiry, and drop the cached promise when an attempt times out or
fails so the next navigation retries. The setup-state probe in the guard
gets the same bound (it already fails open, defaulting to "complete").
2. A deploy replaces the content-hashed chunk files, so a tab open across it
asks for chunks that no longer exist and the dynamic import rejects with
nothing handling it. Reload once on a chunk-load error, via router.onError
and Vite's preloadError, guarded by a sessionStorage flag against a reload
loop and cleared on the next successful navigation.
Also stop index.html being cached: it names the hashed chunks, so a stale copy
points at files the deploy already deleted. It now revalidates while the
hashed assets under assets/ cache for a year.
Every asset list shows a Type column (and printers a Model, machines and
network a Vendor), but the search filters only looked at the asset number,
name, serial and hostname. Searching a type returned zero rows: 'Part Washer'
on machines, 'Standard' on PCs, 'Thermal' on printers.
Extend the search on machines, computers, printers, network devices,
measuring tools and the unified asset list to cover the type name plus the
vendor/model where the list shows them. Joins are outer joins so an asset
missing a type or vendor still matches on its own fields; the core list uses
a correlated EXISTS instead, since its type-name filter already joins
AssetType.
Notification start/end times displayed and stored wrong by the tz offset
(a 2:34 PM entry showed 6:34 PM). Two stacked bugs: to_dict emitted stored
UTC as naive ISO (no offset) so the browser read it as local, and the form
filled the datetime-local input from toISOString() (UTC).
Fix and generalize to a configurable site timezone (multi-site):
- New setting site_timezone (default America/New_York), public, editable in
Settings > Site > Localization (common-zone dropdown).
- Backend tags datetimes UTC (_utc_iso); parse normalizes to naive UTC
(_parse_utc); daily-reset expiry uses the site zone (_next_site_time);
calendar allDay events key off the site-local day (_site_date).
- Shared frontend util datetime.js (Intl-based, DST-safe) converts between a
UTC instant and a site-zone wall clock. Notification form, list, and
calendar all render/enter in the site zone.
Generate docs/openapi.json (3.1, 362 operations) from the API inventory via
scripts/gen_openapi.py, and serve it with a self-hosted Redoc bundle at
/api/docs - no CDN, works on the air-gapped box. Also serve docs/llms.txt (a
concise LLM entrypoint) at /api/docs/llms.txt. New core 'docs' blueprint;
staticdocs/ excluded from the naming check (vendored minified JS).
Prod had 331 relationships, 268 computers, 204 machines, but the report came
back empty. The query only matched computer(source) -> machine(target), while
the import stores the general machinerelationships as machine(source) ->
PC(target) (only the synthetic measuring-tool links are PC -> tool). So the real
shop-floor edges never matched.
Make the query direction-agnostic (UNION of both orientations); a PC-runs-machine
report is conceptually undirected. Also drop the comtypeid=1 filter so the IP is
taken from the primary communication regardless of its type.
Test: a machine(source) -> PC(target) edge now appears in the report.
IIS ARR sets X-Forwarded-For to clientip:port, and the port changes every
connection. Left in, the audit log showed IP:PORT, the dashboard IP fallback
never matched a stored (portless) DashboardDefault.ipaddress, and login rate
limiting keyed per-connection instead of per-host. Add an IPv6-safe
clientip.client_ip / strip_port helper and use it in the audit log, the
dashboard resolver, and the login rate-limit key.
The lobby-display and screensaver slide manager was admin-only. Add a shared
slides.manage permission so a curator can manage both surfaces without full
admin. Admins keep access via the require_permission admin bypass.
Backend:
- plugins/slides/api/routes.py: all 5 management routes require slides.manage
- plugins/slides/plugin.py: declare it via get_permissions(); nav item carries
the permission so the frontend can gate visibility
- shopdb/core/api/auth.py: login response now returns the user's permissions
(matches /me) so the frontend authStore has them on fresh login
Frontend:
- stores/auth.js: hasPermission(name) getter (admin true, else granted list)
- router/index.js: guard supports requiresPermission
- views/AppLayout.vue: hide nav items whose permission the user lacks
- plugins/slides/frontend/routes.js: slide manager gated requiresPermission
Tests: no-perm user 403, curator role with the perm 200 (+ login advertises
it), admin 200 via bypass.
Deploy: run `flask seed permissions` to create the row, then grant it to a
role in Settings > Users & Roles.
Global search did a single ilike('%CSF Roles%'), so any query with more than one
word required the exact contiguous phrase and usually returned nothing. Add
_word_match: split the query into words and AND them (OR across the searched
columns per word), so 'CSF Roles' matches a record with both words in any field,
any order. Applied across every domain (assets, applications, KB, employees
[selfhosted + external HR], notifications, hostnames, IP, custom fields,
vendor/model/type). External HR path uses a parameterized per-word LIKE.
A display's DHCP IP can change; its FQDN (F<serial>.<domain>, domain from the
display_fqdn_domain setting) is stable and the collector already reports the
serial. Add a nullable unique fqdn column (varchar191 so the index fits utf8mb4
without innodb_large_prefix), make ipaddress nullable, and require fqdn OR ip.
visitor-location + display-role resolve by FQDN first, then IP; create/update
accept fqdn. Core migration 7d31, verified up/down/idempotent on MySQL 5.6.
'Business unit' wording -> 'location' in the validation messages.
A geenforce.fetch token can now be pinned to specific manifest scopes so a
fleet-wide key (a display's, delivered by DSC or baked into the image) is not a
skeleton key for the whole content store. NULL binding = unrestricted, so every
existing service token keeps working.
Core:
- ApiToken.resourcescopes column + resourcescopelist property (migration
7d30_apitoken_resourcescopes; NULL = unrestricted).
- apitokens API create/update accept + persist an optional resourcescopes list
(a resource-name allowlist; not permission-catalog names).
- New contract helper authorized_service_token(scope): same check as
service_token_authorized but returns the ApiToken so a plugin can read its
binding. Contract 0.14.0 -> 0.15.0; also export SupportTeam.
GE-Enforce enforcement:
- get_manifest: a bound token requesting a scope outside its allowlist -> 403.
- get_payload: a bound token may only pull a blob its own scope(s) reference
(service.blob_referenced_by_scopes); anything else -> 404 (no hash probing).
- Decorator stashes the authorized token on g for the route to read.
Also fixes a pre-existing contract-surface violation: the printers/printedparts
alert helpers imported shopdb.core.models / shopdb.extensions directly; now
via shopdb.api (SupportTeam newly exported). Docs: GE-ENFORCE-DISPLAY.md
provisioning note, PLUGIN-HOOKS.md, CLAUDE.md.
9 new resource-binding tests; full suite 1131 passing.
Get GE-Enforce closer to running on credential-less Intune/Entra display PCs
that pull manifest + payloads over HTTPS instead of SMB.
Server (plugins/geenforce/api/routes.py):
- Rate-limit + 512MB served-size ceiling on GET /payload/<sha256> (reuses the
login limiter's cache pattern, config-overridable via GEENFORCE_PAYLOAD_*).
- New tests: payload hardening, manifestblobs model-vs-migration parity, and a
report-contract test locking the lowercase per-entry report keys.
PS client (plugins/geenforce/client/):
- Fix New-ShopdbReport per-entry key casing to lowercase (name/action/selfhealed/
exitcode/message) to match what the server reads; the engine emits PascalCase.
- Enforce TLS 1.2 in the network functions.
- Fetch + merge the fleet-wide common scope alongside the pctype scope
(pctype wins on conflict; -NoCommon opt-out).
- Normalize whatever the engine returns into a well-formed summary.
- Make the empty-cache fail-safe observable: event-log entry + report ping
instead of a silent exit 0.
Manifest (plugins/geenforce/seed_display_scope.py + docs/GE-ENFORCE-DISPLAY.md):
- Seed a gea-shopfloor-display scope: 4 Edge kiosk drift-heal registry entries
+ 1 data-driven dispatcher (Dashboard/Lobby/3DPrintRoom via display-type.txt).
Kiosk EXEs stay image-baked; the manifest heals policy/config drift only.
- Documents the common SMB-payload audit (entries needing http/inline before a
share-less display can inherit common).
Migration registry (shopdb/plugins/alembic_template.py + test):
- Register the pre-existing manifestblobs and the new printersupplyalerts tables
in PLUGIN_TABLE_OWNERS; update EXPECTED_HEAD_REVISION for geenforce (0002blobs),
printers (0002supplyalerts), and printedparts (0004txnrev) which had drifted.
Support teams gain a webhookurl (migration 7d29 + API + settings-page field), so
a team is a notification target. send_webhook(url=) lets a caller override the
site default with a team's webhook. Printedparts gains a 'alert support team'
setting (printedparts_alert_supportteamid) + selector on its settings page;
low-stock alerts post to that team's webhook, falling back to the site
alert_webhook_url. Email leg unchanged. Same pattern extends to other alerting
plugins (printers low-toner next).
send_webhook(title,text) posts alerts to an optional webhook (Teams Incoming
Webhook / Workflow, or generic JSON) via alert_webhook_url + alert_webhook_format
settings; send_alert fans out to it alongside email; exposed on shopdb.api
(0.13.0->0.14.0, PLUGIN-HOOKS synced); low-stock posts on its custom-recipient
path too. Also: recent-transactions table shows the consumed print-file revision.
Kiosk now displays the scanned revision (badge, quantity, and done screens) so
the operator sees which revision they checked out - it was already recorded on
the take, just not shown. Low-stock email now uses the gage lab tag (was the
internal item code) and links to the item page when site_base_url is set (new
core setting, category email; emails have no request context to derive the URL).
One 'display' image resolves what it shows from its own IP, like the existing
visitor-location BU mapping. Extend DashboardDefault with displayrole
(dashboard|lobby|partskiosk; migration 7d28, businessunitid now nullable since
only the dashboard role needs one) + a role->path map. New unauthenticated
GET /api/dashboarddefaults/display-role returns {role, path, businessunitid}
for the caller IP. Settings UI gains a Display selector, showing the business
unit only for the dashboard role.
Role badges rendered gray for everything except admin, with no way to tell
roles apart. Add an optional color per role, matching how statuses and types
carry one: new roles.color column (migration 7d27_roles_color), color threaded
through the role API and the user serializer, and a ColorSwatchPicker in the
role editor. Badges use the role's color with contrast-aware text and fall
back to the old admin/gray classes when unset.
A lean site still gets every plugin's tables from the shared core Alembic
baseline. prune-schema drops the tables of plugins not installed on this
site, leaving core + chosen-plugin tables, with no edit to any released
migration (the relocate-into-plugin-baselines alternative would mean
rewriting ~15 released core migrations for a cosmetic gain - see ADR-014).
- shopdb/plugins/cli.py: prune-schema command. Dry-run by default; --yes to
execute; refuses non-empty tables without --force. Drops by table name (no
plugin import) so it works on a lean image. MySQL: private AUTOCOMMIT engine
(db.engine's pooled connections sit idle-in-transaction in a CLI context and
would deadlock the DROP on a metadata lock). SQLite: db.engine, restoring the
prior foreign_keys pragma so the StaticPool connection is not left changed.
- tests/test_plugin_prune_schema.py: drop-only-not-installed, full no-op,
refuse-non-empty, force-drops-non-empty.
- docs/DEPLOY.md: lean provisioning step after upgrade-all.
- ADR-014 ACCEPTED; index updated.
Verified on MySQL: full install then prune = no-op (86 tables); lean install
(machines+printers) then prune drops the other 19 plugin tables; second run
no-op. Full suite 1077 passed.
Cross-plugin FK blocker ADR-013 cited is already resolved: the FKs into
machines were held only by dead legacy tables (machinerelationships,
printerdata, installedapps, communications.machineid) that existing
migrations 7a01/7c01 already drop. No live plugin table hard-FKs another
plugin. Schema-lean is unblocked.
Enabling change: create_plugin_tables now skips already-existing tables
(idempotent) so a plugin anchor can create its tables on a fresh lean
install and no-op on a database that has them from the pre-cutover
baseline. The load-bearing baseline lift is staged as ADR-014 Phase 2.
The lean-build endgame: a site ships carrying only the plugins it chose.
- scripts/build-site.sh: reads a site profile, resolves the hard-dependency
closure from manifests, builds the frontend with SITE_PLUGINS (stage-frontend
carries only those plugins), and stages a backend tree of core + only the
chosen plugin dirs. An unchosen plugin is in neither the bundle nor the tree.
- Core lazy-import guard: `flask seed demo` hard-imported the 5 asset subtype
models, which would crash a lean build missing any of those plugins. Now
guarded (a missing model skips its demo section).
- test_lean_build_guards.py: statically asserts NO core (shopdb/core, shopdb/cli)
import of a plugin is unguarded - a lean build omitting that plugin would
otherwise crash. 0 unguarded today.
Pilot verified: a lean build (machines + printers) carries only machines +
printers code - PartsKiosk / ManifestEditor / USBLabelBatch / KnowledgeBaseDetail
/ EmployeeDirectory are absent from the bundle, and only machines/printers plugin
dirs stage into the backend. (Sidebar labels for absent plugins remain - the
accepted small plugin-aware core remainder.) Guard test + naming green.
Global-search rows built the plugin detail URL from a hardcoded url_map of
plugin routes in core. Now core prefers a plugin's declared
get_asset_presentation route (ADR-010), substituting the core assetid via the
plugin's by-asset resolver; types that have not declared fall back to the legacy
id-keyed map, so nothing breaks. Measuring tools (which declare the route) link
through it now; machines/PCs/printers/network migrate off the hardcode as they
add a by-asset route + declaration. Presentation map is collected once per
search (cached on flask.g). 2 consumer tests; 26 search tests green.
Wires the ADR-010 get_asset_panels hook to a generic frontend renderer so a
plugin adds detail-page UI as JSON, no Vue. This is the Path A foundation that
lets simple plugins ship UI without a frontend build.
- components/PluginAssetPanels.vue + pluginAssetPanels.js: fetches
/api/pluginui/asset-panels for an asset, then each panel's data endpoint, and
renders by mode: list (title + status badge + meta lines via a field map),
keyvalue, table (declared or inferred columns), badge. Pure mapping logic is
in the .js module and unit tested (9 specs), same pattern as entryForm.js.
- New 'list' render mode with a declarative field map (title/badge/meta),
documented on the hook in base.py.
- Warranty migrated to it: get_asset_panels now declares a 'list' panel + map
that reproduces WarrantyPanel's output (vendor title, status badge with color
+ label map, servicelevel/ends/tag meta, manage link) with zero
warranty-specific frontend code.
- MachineDetail swapped from <WarrantyPanel> to <PluginAssetPanels> (pilot); the
hero warranty badge is unchanged. Verified end to end: the API serves the list
panel + map and the warranty rows; the page renders without error.
Rollout of the other 4 detail pages (PCDetail, PrinterDetail, NetworkDeviceDetail,
MeasuringToolDetail) and the map-overlays / asset-presentation renderers are
follow-up Phase 3 commits. 58 vitest, build clean, 1067 backend pass, naming green.
Fourth review found the last import-path bypass: the is_dir() branch returned
None for a name whose dir has no __init__.py, without checking a same-name
sibling file. FileFinder loads a file over an init-less namespace dir, so an
attacker could overwrite a signed foo.py with malicious bytes, mkdir an empty
foo/ next to it (PROVENANCE untouched, still verifies), and any import of that
name ran the unverified foo.py - RCE with only plugins/ write access.
Fix: the dir-with-no-__init__.py branch no longer returns early; it falls
through to the leaf .py hash gate and the non-source refuse check. Invariant:
find_spec returns None for a plugins.* name ONLY where FileFinder would also
find nothing on the same __path__.
Everything else was confirmed sound this round: the owned plugins root, exec of
exact verified bytes (never .pyc/.so), the extension/bytecode refusal, plugin.py
read-once, the provenance signature gate, dev-exemption scoping, and #3/#4.
Symlink, suffix-ordering, cache-lifecycle, and loader-internal angles cleared.
2 regression tests (tampered .py + sibling dir; unsigned .py + sibling dir). All
13 bundled plugins still load under enforcement; 1067 pass, naming green.
Third review found the meta_path guard leaked exactly where it delegated to the
stdlib import system:
1. Non-.py submodules (CRITICAL). When a name had no dir and no .py, find_spec
returned None and the stdlib loaded a planted .so (ExtensionFileLoader) or a
sourceless .pyc unverified - an attacker deletes a signed .py and drops a
same-named .so with arbitrary init code, run on a normal request via core's
`from plugins.<name>.models import ...`. The guard now refuses any name for
which a non-source importable candidate (EXTENSION_SUFFIXES + BYTECODE_
SUFFIXES) exists on disk; None is reserved for genuinely-absent modules.
2. Top-level plugins/__init__.py (CRITICAL). It is in no plugin's provenance,
is attacker-writable, and Python runs it before any guarded submodule. The
guard now owns `plugins`: it execs an EMPTY package body (search points at
the plugins dir), so an overwritten plugins/__init__.py never runs.
Also: specs are built with spec_from_file_location so loaded modules get
__file__/__path__ (Flask blueprint root paths need it) while the loader still
execs the verified in-memory bytes - never re-reading the file.
Verified end to end: under PLUGIN_REQUIRE_SIGNED with all 13 bundled plugins
stamped, the app boots and loads every plugin through the guard; a tampered
plugin file is refused at load. 4 new guard tests (planted .so, sourceless
.pyc, absent-module defer, neutralized package root). Prior fixes #3/#4
confirmed still sound by the review. 1065 pass, naming green.
A re-review showed the previous "single import choke point" claim was wrong:
`plugins` is a normal importable package, so core request handlers that do
`from plugins.<name>.models import ...` never passed through the loader and ran
unverified - an attacker who dropped a file into plugins/<name>/ got arbitrary
in-process code execution on an ordinary HTTP request (and a planted .pyc ran
from cache). Gating load_plugin_class covered only plugin.py, one path of many.
Fix: importguard.py installs a sys.meta_path finder (under enforcement) that
intercepts EVERY plugins.<name>.* import, verifies the plugin's signed
provenance once, then verifies each module file against it and execs the exact
bytes it hashed - read once, compiled, exec'd, never a .pyc, never a re-opened
file. This closes the submodule bypass and the planted-bytecode read, and the
read-once exec closes the verify-vs-exec TOCTOU on the import path. The import
system, not one method, is the real choke point.
- init_app installs the guard when PLUGIN_REQUIRE_SIGNED, clears it otherwise.
- load_plugin_class now verifies plugin.py from a single read and execs that
buffer (finding #3 on that file); its submodule imports flow through the guard.
- docs: stamp-bundled must cover every plugin dir present (a disabled plugin's
module can be imported by core); recommend a read-only plugins/ owned by the
deploy user as defense in depth (closes the residual migrate-time race an
attacker with concurrent write could otherwise attempt).
Earlier review's fixes#3 (migrate code paths) and #4 (shelf content binding)
were confirmed sound and are unchanged. 7 import-guard tests (submodule verify,
tamper, unsigned refused, planted .pyc ignored, real import through the guard,
install/uninstall). 1061 pass, naming green.
An adversarial security review of the Phase 2 trust model found four real
bypasses (two remote-triggerable to in-process code execution). Root cause for
three: the set of bytes verification covered was smaller than the set that
determined execution. Fixes:
1. Bytecode-cache blind spot (CRITICAL). verify_dir excluded __pycache__/.pyc,
so a planted cache ran while escaping the hash map. verify_dir now flags any
bytecode as an unexpected file; the loader strips bytecode before verify and
imports under sys.dont_write_bytecode, so only verified source executes.
2. Unauthenticated verify-at-load bypass (CRITICAL). load_plugin_class imported
plugin.py with no gate, reachable via discover_available / an anonymous GET
/api/plugins. The verify+strip gate moved INTO load_plugin_class - the single
import choke point every path flows through - so an unsigned/tampered plugin
is never imported. discover_available skips a refused plugin instead of 500.
3. Ungated migration entrypoints (HIGH). downgrade_plugin and get_current_head
(ScriptDirectory imports version modules) ran plugin code with no check. All
alembic-invoking methods now pass through _verify_ok (strip + verify) first
and run under no-bytecode.
4. Revocation/content bypass (HIGH). The signed index bound a filename, not
content; adopt did not bind the delivered bytes to the resolved version, so
revoked bytes could be served under a live filename. The index now records a
per-artifact SHA-256; adopt verifies the on-disk digest and requires the
artifact's own signed manifest version to equal the resolved version.
Enforcement stays default-off; strip/no-bytecode run only under enforcement, so
the unsigned path is unchanged. 6 regression tests (planted bytecode, the
discover import path, downgrade gate, version-swap). 1054 pass, naming green.
Completes the marketplace security model. Verification stops being advisory:
a plugin only loads or migrates when its tree matches a trusted signature, and
plugins are pulled from a signed shelf with anti-rollback and revocation.
Enforcement (default OFF - existing deploys unchanged):
- verification.py PluginVerifier, shared by the loader (verify-at-load, before
plugin.py is imported) and the migration manager (verify-at-migrate, before
any DDL). Fail-closed: an unsigned/tampered/wrong-key plugin does not run.
- Gated by PLUGIN_REQUIRE_SIGNED. PLUGIN_DEV_TRUST_DIRS exempts named dirs but
only under DEBUG/TESTING; production ignores it.
- flask plugin stamp-bundled writes provenance into in-tree plugins so
verify-at-load applies to bundled plugins too (image build step).
- tier:core manifest guard: uninstall/disable refuse a core-tier plugin.
Shelf (shelf.py):
- Signed shelf-index.json (+ .sig): monotonic serial (a site refuses an older
index - anti-rollback), revoked list carried across builds, per-entry
version/tier/core_version for browse. Index is a browse layer only; adopt
reads security-bearing fields from the verified artifact.
- flask plugin shelf-build / shelf-list / adopt / audit. adopt verifies index +
artifact (signature + every file hash), unpacks to staging, re-verifies, then
atomically moves into place and installs+enables the closure. Refuses a
downgrade without --force-downgrade. Anti-rollback serial stored in
instance/shelf-state.json.
- config PLUGIN_SHELF_DIR; the app only reads the folder, never speaks a
network. .env.example + docs/PLUGIN-SIGNING.md document the flow.
22 tests: verifier policy (off / no-keys / signed / tampered / wrong-key /
dev-exempt), verify-at-load + verify-at-migrate integration, tier guard, index
sign/verify + tamper/wrong-key, serial state, revocation, version resolution,
verified atomic unpack + tamper refusal. Live-smoked keygen->pack->shelf-build
->list->adopt->audit + serial guard. 1050 pass, naming green.
Packaging + provenance for the plugin marketplace. No runtime behavior change
yet - verification is available on demand; enforcing it at plugin load/migrate
and pulling from a shelf are Phase 2.
- signing.py: ed25519 key pairs + provenance. Provenance is a sorted per-file
SHA-256 map plus metadata; the detached signature covers the exact
serialized provenance bytes, so verifying is re-hash files, re-serialize,
check signature. verify() accepts any of several trusted keys (rotation).
Uses cryptography (already a dependency).
- packaging.py: pack() builds a signed <name>-<version>.shopdbplugin (zip +
PROVENANCE.json + PROVENANCE.sig). verify_artifact()/verify_dir() re-hash
and check the signature, and flag a tampered file, an unexpected file, a
wrong/absent key - all fail closed.
- CLI: `flask plugin keygen` (publisher key pair), `flask plugin pack <name>
--key` (validates then signs), and `flask plugin validate` extended to a
signed artifact by path (--pubkey, else PLUGIN_TRUSTED_KEYS).
- config PLUGIN_TRUSTED_KEYS: os.pathsep-separated public-key PEM paths,
delivered with the site config, never read from the shelf. .env.example
documents it.
- docs/PLUGIN-SIGNING.md: curator flow (keygen offline, review, pack, publish,
pin keys, rotate).
The signature proves an artifact is exactly what a curator signed, not that the
code is safe - human review before signing is the control. 11 tests: sign/verify
round trip, wrong key, provenance excludes noise, serialize determinism, pack +
verify, tamper -> hash mismatch, extra file, no-key fail-closed, verify_dir.
1028 pass, naming green.
Additive, zero-risk-to-running-sites prep for the plugin catalog. No
distribution or lean-build behavior yet; fixes latent bugs and adds the
declarative + validate tooling later phases build on.
Fixes:
- upgrade_all_plugins iterates registry.get_all(); only adopted plugins are
migrated. Removes the phantom hasattr(registry, 'list_installed') probe
that always fell through to migrating every folder on disk (unadopted DDL
ran with full DB rights on every deploy).
- Reverse-dependency checks on uninstall/disable read dependencies from the
manifest on disk via _installed_dependents, so an installed-but-unloaded or
disabled dependent is counted. Uninstall blocks on any installed dependent;
disable blocks on an enabled dependent.
- _sort_by_dependencies detects a dependency cycle (back edge in the DFS) and
raises PluginDependencyError instead of looping or dropping a plugin.
New:
- flask plugin validate <name>: manifest loads + name match, manifest-schema
check, core_version admits the framework contract, declared dependencies
exist on disk. No new dependency (lightweight checker); schema ships in the
package at shopdb/plugins/manifest_schema.json (docs/ is stripped on
publish). The check caught that provides is an object, not an array.
- flask plugin apply-profile <file>: declarative install AND enable of a
chosen plugin set plus its hard-dependency closure, in dependency order,
idempotent. Replaces the hand-ordered runbook sequences that could enable a
plugin that was never installed. deploy/site-profile.example.json template.
- Dockerfile header corrected (all 13 catalog plugins, not "eleven core").
10 new lifecycle tests (reverse-deps from disk, cycle detection, upgrade-all
scope, profile closure, schema, all 13 manifests match schema). 1018 pass,
naming green.
New dev/eval seeder populates a small, broad dataset so a fresh site has
something on every screen: ~25 assets across machines, computers,
printers, network devices, and measuring tools, plus supporting
vendors/business-units/locations, six 3D-printed parts (two below their
low-stock threshold to exercise the alert), and a few relationships for
the map and relationship cards. Idempotent, keyed on a DEMO- assetnumber
prefix; skips the plugin sections that are not installed.
`flask seed demo-clear` removes exactly what it created: bulk-deletes the
DEMO- assets so the DB-level ON DELETE CASCADE drops each plugin subtype
row (per-object ORM delete would try to NULL the NOT NULL child assetid),
after clearing the demo relationships first. Leaves reference data,
settings, users, and any imported rows untouched.
Documented as an optional step in the dev setup guide.
The dualpath_single_machine setting description is 257 chars but
settings.description was varchar(255). On strict MySQL 8 an over-length
insert is a hard error 1406 (Data too long), so `flask seed settings`
failed on a fresh install; older/relaxed MySQL truncated silently and
hid it. Widen the column to TEXT (matches value, already TEXT) via core
migration 7d26.
CI only ran `flask db upgrade` + plugin install, never the seeders, so it
missed this. Add a seed step to the migrations-mysql job so a seeded row
that violates a column constraint fails CI on strict MySQL 8 instead of
shipping.
