cbdfcbabb774a1d84aff97bc0c3f62d2622e05fb
35 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
cproudlock
|
cbdfcbabb7 |
One badge vocabulary, and images a browser can actually load
Two operator-visible defects and three doc corrections, closing the tail of the 2026-09-03 run. THE BADGE. The two kiosks read the same physical badges and disagreed. The parts kiosk understood both shapes a scanner emits - the bare SSO and the PayNo wrapper 0<digits>BZ - while the USB kiosk compared the raw string to an SSO, so the wrapper resolved to nobody. An operator whose keyboard-wedge scanner emits it could take a 3D-printed part and could not check out a drive, with the same badge, at two cabinets a few metres apart. The shape now lives in shopdb/utils/badges.py on the contract surface, because a plugin may not import another plugin and because a badge is a property of the site rather than of a cabinet. Both kiosks use it, and both store the DIGITS rather than the string that arrived: one row per spelling makes a custody history that cannot be joined to itself. That closed a subtler hole in the same place. str.isdigit() is true for Arabic-Indic and other decimal forms, and int() parses them, so a badge written in those characters was accepted, matched a REAL employee, and was then stored as an SSO string nothing else matches. ASCII digits only. An empty scan and an unreadable one now get different answers, because they are different problems for the person at the cabinet. THE IMAGES. /api/applications/image/<filename> and /api/models/image/<filename> required a permission, and a browser fetching <img src="..."> sends no Authorization header - so the gate did not protect the file, it guaranteed the picture never appeared. Both are ungated now, like the four sibling image routes (branding logo, map blueprint, employee photo, part picture), and registered in PUBLIC_ENDPOINTS with that reasoning. /api/printers/lookup?fqdn= never matched anything: both arguments were compared against Communication.ipaddress, so a hostname lookup 404d on printers whose hostname was on the record. An FQDN now matches the hostname, with a fallback to the short name. The slide manager gained the per-slide hold time. The PATCH route and the feed have always carried `seconds`; nothing called it, so the one documented per-slide control was curl-only. Docs: API-REFERENCE listed the employee lookups and the whole notifications surface as unauthenticated when they answer 401, and called both kiosks "deliberately open" when they are off by default and refuse anything without the site's device token. The installer download says applications.download, which is what the code requires. api-inventory.json said printers/lookup was jwt-optional. Fourteen tests. The badge rule verified by planting the old per-kiosk behaviour: three fail, one in each kiosk. Scope: product |
||
|
cproudlock
|
13e992d863 |
Ask what the caller may do, not what its owner may do
Wave 4, first half. A personal API token scoped to endpoints.command alone could queue system_restart, run_script and update_app against a shop-floor PC. require_permission has always suspended the admin bypass for a scoped token, and tests pin that. What nothing covered was a route checking a SECOND permission by hand: the endpoints command route asked current_user.haspermission(elevated), which reads the token's OWNER. An admin minting a narrow token for a script therefore handed out remote restart, script execution and app deployment - the exact bypass require_permission exists to prevent, reintroduced one hand-rolled check at a time. caller_holds is now scope-aware, so the answer is about the caller rather than the person behind it: the owner must hold the permission AND, when the request carries a scoped token, the scope list must name it. An unscoped token is the person and is unchanged; a session is unchanged; guests still answer from the guest role. Both hand-rolled checks in the tree now use it - the endpoints elevated gate and the secure-send tools.secrets check, which had the same shape and the same consequence at a smaller scale. Two tests, one proving the refusal and one proving an unscoped token still passes (a fix that quietly broke every service token would be worse than the bug). Verified by planting the owner-based check back: the first fails. Reported by the fleet-agent lane of the 2026-09-03 test run and confirmed by an independent verifier that minted its own token and queued its own commands. Scope: product |
||
|
cproudlock
|
debef39a18 |
Answer a nonsense filter with a 400, and sweep every list route for the rest
Wave 3. Six findings, one cause: `int(request.args.get('typeid'))` with nothing
catching the ValueError. The caller sent a bad argument and got
500 INTERNAL_ERROR plus a traceback in the log, which says the server is broken
when the truth is that 'abc' is not a number.
shopdb/utils/queryargs.intarg reads it and raises QueryArgError, which an
app-level handler answers as a 400 naming the argument. App-level on purpose: a
route should not have to remember, and forgetting was the entire defect. The
policy differs from get_pagination_params, which falls back to a default for a
bad page number - a nonsense page is harmless and the caller still wants their
list, but silently ignoring a nonsense FILTER returns rows they believe they
excluded.
The gate matters more than the six fixes. tests/test_core/test_bad_query_args.py
sweeps all 131 registered no-parameter GET routes under /api/ with twelve
argument shapes and fails on any 5xx. It found sites the test run had not
reported - statusid and locationid on /api/assets, the second asset-list route,
limit on the employee lookup and the printer supply history - because the
pattern is how a list filter gets written rather than a property of any one
route. Verified non-vacuous by planting the old `int(...)` back into
/api/machines: four tests fail.
Four usb routes are skipped by exact path, with the reason: they reach the
EXTERNAL cmmc_usb database, so their 500 in a test environment is a missing
database rather than an argument fault. That is the complete set - every usb
GET route calling cmmc_usb_connection - not a convenient subset.
One correction worth recording: the first pass rewrote every
`if x := request.args.get(...)` in the touched files, including text arguments,
so a search for 'ZZNOTHINGMATCHES' briefly became a 400. Caught by
test_list_search_identifiers. Only id-shaped arguments are integers.
Scope: product
|
||
|
cproudlock
|
12392a361e |
Refuse a value the database cannot store, instead of quietly changing it
Wave 2 of the remediation plan. Eight findings from the feature test run were the same defect twice: a write that reports success while storing something other than what was sent. The first half is a date parser that swallowed its own error. Two plugins carried an identical private _parse_date returning None on ValueError, so an unparseable date and a deliberately cleared one produced the same write. PUT /api/warranty/<id> with enddate '30-06-2027' answered 200 "Warranty updated" and stored NULL, dropping the record out of every expiry surface on the strength of a typo; the measuring-tool calibration date did the same and flipped the tool to 'unknown', out of the calibration report and the map overlay. shopdb/utils/datevalues.py now holds one strict parser on the contract surface: an explicitly empty value clears the field, anything that is not a date is refused with a 400 naming the field, and those are different intents with different answers. The vendor-lookup paths keep a lenient wrapper on purpose - a provider changing its date shape must not fail the refresh of an otherwise good record, and no person typed that value. The second half is why the truncation findings were possible at all. MySQL was running non-strict: sql_mode on the 5.6 server is NO_ENGINE_SUBSTITUTION with no STRICT_TRANS_TABLES, so it accepted writes it could not store and adjusted them. A 257-character username became its first 100 characters (and the next distinct one then collided with it); an over-long asset number was cut at 50; a malformed releasedate became '0000-00-00' and permanently broke every read of that application. All with a 201. The mode is now set on the CONNECTION rather than left to the server, so a site whose my.cnf disagrees still gets it. That trades silent corruption for a DataError, which uncaught is a 500 - and a 500 says the server is broken when the truth is that the value does not fit. An error handler maps it to a 400 naming the column where MySQL names one. Verified by hand against MySQL 5.6: POST /api/users with a 257-character username now answers 400 "That value is too long for username." and stores nothing, where it previously stored a truncated row and answered 201. Twelve tests. The date guard is verified by planting: restore the swallowing parser and four of them fail. The DataError tests raise the error directly because the suite runs on SQLite, which does not enforce VARCHAR length at all - there is no value that would trigger it there, which is precisely why this went unnoticed. Scope: product |
||
|
cproudlock
|
ccd8221adf |
Key the client address on the connection, not a header the caller sends
client_ip() read the first X-Forwarded-For hop unconditionally. The
login rate limit and the account lockout are keyed on that value and the
audit log records it, so rotating the header per request bought an
unbounded budget and an untraceable trail.
This walked around TRUSTED_PROXY_COUNT, which exists to decide exactly
this. The app leaves ProxyFix off by default and says why, ten lines of
comment away: ProxyFix trusts X-Forwarded-*, a client can send those
itself, and the login limiter is keyed on the address. Then client_ip
read the header anyway, so the stated protection never applied.
Resolution is now from remote_addr, which ProxyFix has already filled
from the header wherever a proxy is genuinely configured. Both cases end
at the same trustworthy source, and the port still comes off because ARR
forwards clientip:port and ProxyFix passes it through verbatim.
geenforce and tools had each hand-rolled the same mistake for their own
limiters. Rather than duplicate security-relevant logic a third time,
client_ip and strip_port join the shopdb.api surface (contract 0.28.0).
geenforce's _trusted_client_ip, which already refused the raw header for
its IP allowlist, is untouched and was the model for this.
The stability table was also missing rows for 0.26.0 and 0.27.0.
INSTALL-WINDOWS-IIS.md never mentioned TRUSTED_PROXY_COUNT, so a site
installed from it runs behind IIS with the setting at 0: every request
looks like it came from IIS, the whole site shares one rate-limit
bucket, and the audit log records the proxy for every action. It now
says so, and says not to set it when nothing is in front.
Three test modules simulated separate callers by setting
X-Forwarded-For, which is the header a real attacker rotates to escape
the limit. One of them, test_client_ip_prefers_forwarded_first_hop_
stripped, asserted the defect outright. The suite could not have caught
this. They now drive REMOTE_ADDR, with a regression guard that forging
the header buys nothing.
Scope: product
(cherry picked from commit
|
||
|
cproudlock
|
54f0857e13 |
Split the fleet roster from per-endpoint reads, and stop offering controls nobody can use
Twelve endpoints routes shared one permission, which made a reasonable policy impossible to express. A site that wants signed-out visitors to see live status on a PC page had to hand them the entire fleet - including every logged-in username on the floor - because the per-asset panels and the fleet-wide list were gated on the same string. endpoints.fleet now gates the list and the three dashboard routes. endpoints.view keeps the per-asset reads: status, services, processes, interfaces, events, command history. The name is load-bearing. guest_grantable_permissions() opens any '.view' or '.read' permission to the guest role, so calling this one 'endpoints.fleet' rather than 'endpoints.fleetview' is what keeps the roster off the public surface while per-asset telemetry stays grantable. Verified both ways. Also: the services panel emitted Start/Stop row actions to every reader, because row_actions_for() never consulted the caller. Clicking was always refused server-side, so nothing was exploitable, but showing a read-only viewer controls over a shop-floor PC is the wrong signal - and it stops being cosmetic the moment a site grants that panel to guests. That needed a contract addition rather than a local check: caller_holds() on shopdb.api answers whether THIS caller holds a permission without refusing the request, which is the complement to require_permission for a surface that serves everyone but should only offer what the caller can act on. Guests are included, since a signed-out visitor holds whatever the guest role was granted. Reaching into shopdb.utils.authz instead would repeat the ADR-002 violation the contract gate caught earlier in this branch. Contract 0.24.0 -> 0.25.0. DEPLOYMENT: endpoints.fleet is a new permission. Run 'flask seed permissions' and grant it to the roles that should keep seeing the Fleet page. Until then that page is admin-only, since admins bypass every check - a safe direction, but not an automatic one. Full suite 2124 passed, naming and contract gates green, map and spec regenerated. |
||
|
cproudlock
|
784964c907 |
Bring the wilm branch into main: endpoints plugin, agent, policies, CSV import
Ports the corporate-GitHub wilm branch onto main, keeping the two branches
separate. Thirteen upstream commits arrive with their authorship intact; the
review fixes and the portability work below are on top.
What arrives:
endpoints plugin - a ShopDB agent reporting endpoint state, telemetry,
services and interfaces, with a command channel the agent
long-polls. Ingest stays on the generic collector route
(ADR-006); the agent key is a UI-managed single-scope
token whose revocation is an instant kill switch.
computers policies - Configuration and Business Policies (RMM-style), scope
trees validated at write time, actions checked against
the endpoints command allowlist. Feature-gated: every
route 404s while the setting is off, so enabling it can
never half-expose the surface.
asset CSV import - templates, a collector script carrying the site path, and
a validate-then-commit import. A plugin can own a virtual
column plus a post-write row hook, which is how a PC's
machine number becomes a controls link without a
hand-built relationships sheet.
agent/ - the Windows agent and its installer.
Contract 0.21.0 -> 0.24.0, additive throughout: register_import_hook (0.22),
mint/list/revoke_service_token (0.23), collector_authorized (0.24). The stability
table had no rows for any of them and now does.
MySQL 5.6 portability. The branch stored three policy columns as sa.JSON, the
only JSON columns in the product, and 5.6 has no JSON type: the migration failed
with a bare syntax error near the word JSON. They are TEXT now, through a
JSONText type decorator, so all seventeen call sites still read and write dicts.
The convention elsewhere is already TEXT (guides.blocksjson,
geenforce.manifestjson).
Review fixes carried on top:
- six policy GETs were jwt_required(optional=True), which authenticates a
caller without checking anything; they require computers.view now, and six
redundant jwt_required decorators are gone
- the agent installer took a default BaseUrl and CollectorKey, so a mistyped
run silently pointed an endpoint at the wrong instance; both are mandatory
and a non-https BaseUrl throws unless -AllowInsecureHttp is passed
- a committing CSV import wrote no audit row
Documentation: the 34 new routes are described in docs/api-inventory.json, which
is hand-maintained on purpose, and the generated spec now covers 331 paths and
477 operations across 22 surfaces. DEVELOPMENT-SETUP's 1071 troubleshooting row
said "use MySQL 8 for dev" and now gives the actual fix, including the trap that
5.6 downgrades ROW_FORMAT=DYNAMIC to Compact with a warning instead of an error.
Full suite 2123 passed, vitest 260 passed, all seventeen plugin chains migrate
clean on MySQL 5.6 and the naming, docs, parity and read-surface gates are green.
|
||
|
cproudlock
|
afce24379c |
Reading anything needs a permission
A sweep of the URL map found 165 GET routes that answered anybody, with no token at all: the whole asset inventory, every machine, PC, printer and network device, the USB checkout history and who is holding which drive, the staff directory with its photographs, every core report. Guest access had been argued about one feature-flag at a time while the front door stood open. Each of those now carries the read permission it belongs to. That also makes the guest role mean something: a site can open exactly the reads it chooses, and nothing follows by accident. Thirty-three endpoints stay public, and each is named with its reason rather than left to be inferred: the app shell and its bootstrap, the settings allowlist (the handler already filters an anonymous caller down to branding and a few keys), images fetched by <img src> which carries no Authorization header, what a display kiosk renders before anybody signs in, the parts and USB kiosks, the client scripts that run as SYSTEM before a user session exists, and a secure-send link whose unguessable token IS the authorisation. tests/test_core/test_public_read_surface.py is the standing gate, the companion to the mutating-route sweep that has guarded writes for months. A new public read fails CI until somebody adds it to that list in front of a reviewer. Four further reads need a login but no permission, because the handler decides from who you are - your own user record, your own API tokens, the role catalogue the grid renders - and the same test proves each still refuses an anonymous caller. Verified not vacuous by ungating one route and watching the sweep name it. The employees plugin gained employees.view; the directory is names, SSOs, photographs and reporting lines, and it had no permission of any kind. Around ninety tests asserted payload shapes against endpoints that answered anybody. They are about what the answer says, not about who may ask, so they are signed in now rather than the gate being loosened. The ones that assert an anonymous caller is refused were left exactly as they were. |
||
|
cproudlock
|
151d7c4f44 |
Anyone not signed in holds the guest role
Whether somebody with no account may see something was being answered one feature at a time: a securesend_allow_guests setting, an enable flag per kiosk, and very nearly a third for guides - each with its own settings page, its own vocabulary and its own hand-written branch inside a route. Every one of them answered a permissions question, and this app already administers permissions per role. So a caller with no token is not nobody: they hold the seeded `guest` role. require_permission consults it when there is no identity, which is the entire mechanism. The role is seeded holding NOTHING, so an upgrade opens no door; a site opens one by ticking a box in the same grid it uses for every other role, and closes it the same way. Revocation is read on every request rather than cached, because taking guest access away is what somebody does the moment they realise what they opened, and it has to bite immediately. The redundant @jwt_required() above require_permission is gone from 184 routes. It was belt-and-braces before; now it is actively wrong, because it refuses a request with no token before the permission check can have an opinion, so a grant would appear to do nothing. What a guest may hold is bounded. Any .view or .read permission is grantable. A WRITE has to be blessed by whoever owns it, through the new get_guest_grantable_permissions hook (contract 0.21.0), and Secure Send is the only case that exists: sending a password to a vendor is the one write this product means to offer somebody who never signs in. Everything else is refused by the API and greyed out in the grid rather than warned about - a checkbox that quietly makes deleting an asset anonymous is not a warning anybody reads twice. A guest write records no creator: the trail says when and from where, never who, and that is the reason for the bar. GE-Enforce and Slides gained read permissions so they can be opened to guests without handing out the manage right. Their read routes accept either through require_any_permission, so nothing an existing manager could reach has moved. The navigation endpoint now returns only what the caller can actually reach. Without that a signed-out visitor gets a sidebar of the whole app where every link bounces off the login page, which reads as a broken site rather than as a closed door. A nav item may name several permissions; holding any one is enough, which is what GE-Enforce and Slides need now that a viewer and a manager hold different strings. Two things this found on the way: current_user is a LocalProxy, so `current_user is None` is False for a request with nobody behind it - harmless under a required JWT, an AttributeError the moment verification became optional. And an image pasted into a guide before its first save was never attached to it, so it was visible to editors and invisible to every reader; saving now attaches it, and `flask guides fix-images` repairs what is stored. The guides guest setting written earlier today is gone. Granting guides.view to the guest role replaces it, which is the point. |
||
|
cproudlock
|
3324dbd91e |
Buildings and levels for the floor map, and make every identifier searchable
The map was one picture of one floor. A second floor was added, the blueprint changed size, and machines moved, so a position now records WHICH DRAWING its coordinates belong to. Buildings and levels (ADR-017). Each level owns its blueprint per theme and its own native pixel size; assets.mapx/mapy are pixels of assets.levelid, not of the site. A position whose level is unknown renders "level unknown" and is never drawn on the default level, because a marker on the wrong floor plan looks entirely correct while pointing at the wrong place. Repositioning in bulk: filter by unplaced, needs-review or level, search, place, confirm. Landmark recalibration solves the transform PER AXIS from landmark pairs and never from image dimensions - the canvas grew taller without rescaling, so a dimension-derived scale would stretch Y by 1.57 and be wrong everywhere. It defaults to a dry run, reports what would land off the drawing, snapshots before applying, and clears mapverifiedat because a transform is a guess awaiting review. Snapshots restore, including the level and the review state, and a restore snapshots first so an undo is undoable. Search: gaugelabreference was matched only for measuring tools and maintenancereference was matched nowhere at all, for any asset type, while Settings happily offers both identifiers on machines and PCs. A tag an operator is told to record has to be findable or it is a write-only field. USB devices and printed items were unreachable from search entirely - neither is an asset, so the generic asset search could not see them and no searcher existed; they now match on serial, asset tag, label, bin code and gage-lab tag, honouring isactive, with Settings toggles and result labels to match. The retired-application rule was half a rule: GET /api/knowledgebase hid articles whose topic application is retired while global search still returned them and printed the retired application as the subject. A filter is only real if every path that reaches the row applies it. Contract to 0.20.0 (additive): Asset gained levelid and mapverifiedat, Location gained levelid, and resolve_asset_position returns the levelid belonging to whichever source supplied the coordinates. The five plugins that write a map position are re-pinned. The install-list text format gained levelid as a NINTH field, appended, because the shipped Pascal installer reads fields 0-7 by index. That installer still compiles in one drawing's dimensions and bundles one blueprint, so its map is accurate for the default level only; /api/maplevels is deliberately unauthenticated so it can read both at runtime once rebuilt. Recorded in PRINTER-INSTALLER.md section 6 along with the other known gaps. Migration 7d33 converts an existing single-map site into one building and one default level carrying the old map_* settings, then assigns every placed asset and location to it. Nothing moves on screen. Old settings rows are kept so a rollback still finds them. Verified end to end on MySQL 5.6 from a production-shaped database. |
||
|
cproudlock
|
035419fa51 |
ADR-015: stop shipping one site's values, and make the rule a gate
The scanner has been reporting the same count for weeks, which is what a rule that only prints becomes. It now FAILS the build, and it looks where the leaks actually were: PowerShell, the installer, the seeds, generated JSON, the frontend - case-insensitively, across plugins, shopdb, scripts, deploy, tools. A line that is deliberate declares itself with an ADR-015-OK marker and a reason, so the claim is visible in review instead of tolerated in silence. What it found, fixed here: - The shadow client wrote one site's ShopDB URL into HKLM whenever the registry disagreed. At the site it was written for that reads as healing drift; anywhere else it overwrites the site's own address on every enforce cycle, and the site cannot win because the cycle repeats. The bay's value now wins, an explicit -BaseUrl seeds it, and with neither there is nothing honest to write, so it says so and skips. - The kiosk dispatcher fell back to one plant's host when HKLM was unset, so a kiosk elsewhere quietly opened a server it has no business reaching. The fallback is now this site's site_base_url, baked in at seed time, and the dispatcher refuses rather than guessing when neither is set. Its legacy shortcut matcher derives the host from that URL instead of naming one. - The OpenAPI generator hardcoded a production hostname into every spec it generated, which then published to a public wiki. The relative mount is the only server it can honestly name; a site passes its own by environment. - Placeholders and examples in the UI and the client help offered real internal subnets and a real production URL. They now use documentation ranges. Both publication gates - the export scrub and the docs publishability test - carry the site patterns, which neither did. One plant's hostname, FQDN and internal networks are out of the documentation and the generated specs. Comments naming the reference site are reworded rather than deleted: the reasoning is worth keeping, the plant name is not what makes it true. |
||
|
cproudlock
|
c7dffce81e |
Serve an uploaded file as data, not as a document that can run
An SVG is an XML document that may carry a script, and it is an accepted image type because floor-plan maps and branding genuinely want vector. Loaded through an img tag that script never runs, so the tiles and maps were never the risk. Opening the file's own URL is - and the application image route is public, so that URL needs no session. Every route that serves an upload now goes through one helper that sends Content-Security-Policy: default-src 'none'; sandbox, and nosniff. Seven routes across core and five plugins, so a new one added later starts from the same place rather than repeating the reasoning. Banning the format instead would have cost the maps their only sensible one. The app also sent no security headers at all. It now sets nosniff, frame-ancestors self (as X-Frame-Options too, for the display bays' browsers) and a referrer policy. Deliberately NOT a page-wide CSP: this serves an SPA with inline styles, so a real script-src policy is a change worth making with the frontend in front of you, and a permissive header claiming one would be worse than having none. Contract 0.19.0. send_upload is on the shopdb.api surface, because a plugin serving user-supplied bytes should not have to remember these headers. The same bump records that get_dashboard_widgets has taken data and shape rather than a component name since the dashboard was rebuilt - that shipped without a bump, while BasePlugin and PLUGIN-HOOKS.md both still documented the shape nothing renders, which is how five plugins came to declare widgets pointing at components nobody had written. |
||
|
cproudlock
|
705dd771bd |
Store a blank optional unique field as NULL, and answer a duplicate with 409
A site reported "internal server error" adding a second business unit. It was reproducible: create one with a blank code, create another with a blank code, 500. A column that is unique and nullable accepts any number of NULLs - that is what makes "optional but unique" work - and exactly ONE empty string. The form sent '', so the first blank code saved and every one after it collided with it. The field showed no asterisk because it genuinely is optional; the database just behaved as though it were not. This is not specific to business units. A dozen columns across core and the plugins are unique and nullable - asset numbers, hostnames, item codes, subnet names, gage-lab tags - and each was one blank form away from the same 500. Fixing them an endpoint at a time would have left the next to be found by a user, so a before_flush listener normalises blank to NULL on any unique nullable text column. Listening on Session rather than on individual mappers covers plugin models imported later, and avoids mapper-event semantics that differ between SQLAlchemy versions. A genuine duplicate is now a 409 with a readable message rather than a bare 500 with a traceback in the log: reusing a code that is taken is the caller's mistake, not a server fault. Verified against the development database: three business units with blank codes all save, the blank stores as NULL, and a real duplicate code returns 409. |
||
|
cproudlock
|
2c415a1712 |
fix(installer): correct a false security claim, and clear the should-fix list
CLIENT IP / SPOOFABILITY. docs/geenforce-api-cutover.md claimed that removing the IIS rewrite rule made the allowlist fail closed and that it does NOT become spoofable. The opposite is true. IIS never sets X-Forwarded-For on its own; the rule is the only thing that does. Remove it and IIS still forwards whatever X-Forwarded-For the CALLER sent, waitress trusts it because it arrives from 127.0.0.1, and remote_addr becomes attacker-controlled - so a token-less caller can fetch manifests from anywhere on the network. The document and the _trusted_client_ip docstring now say so, waitress runs with --trusted-proxy-count=1, and stage 5 checks the rule is actually live rather than assuming it. The wizard question is rephrased to something an operator can verify with their network team instead of guessing at. NON-ASCII. The style gate only ever checked .py/.vue/.js/.ts, so documentation accumulated em-dashes, arrows and box-drawing characters against this repo's own convention - including in files added this week. Cleaned, and the gate now uses INCLUDES_ALL so Markdown, JSON and YAML are covered. PLUGIN DEFAULTS. The wizard pre-ticked measuringtools and printedparts, both of which ship default_enabled=false, so every site taking the defaults installed and enabled them against their manifests. Inno has no JSON parser so the list must be hardcoded, but tests/test_installer_defaults.py now fails when it drifts. UPGRADES. The payload copy merges, so a plugin dropped from a site's profile kept its code forever - which defeats a lean build and leaves core's optional-import guards succeeding for a plugin the site no longer has. Stale plugin directories are now deregistered and removed before the copy. add-plugin used 'plugin install', which for the five default_enabled=false plugins left them installed but DISABLED - and printed a green success line anyway. It now goes through apply-profile, and the success line is gated on the exit code. Invoke-Flask records its own exit status, because $LASTEXITCODE keeps a stale value when flask.exe is missing and no native command runs. CHARSET. The utf8mb4 compiler hook lived inline in migrations/env.py, so it covered the CORE chain only: plugin baselines inherited the server default, which on a latin1 server means two charsets in one database. It is now shopdb/utils/mysql_charset.py, imported by both, and preflight reports the database's default charset. BACKUP HONESTY. The dump was described as 'all of your asset data'. Uploaded branding and floor-map images live in instance\ on disk, not in the database, so a restore from the .sql alone comes back with no map. backup now archives instance\ alongside it and says both are needed. VERSIONING. AppVersion was hardcoded at 0.9.0 while the product, the frontend and the newest tag said 0.7.0 - and 0.9.0 collides with a retired contract version. Both builders now generate version.iss from shopdb/__init__.py. Smaller: rollback overwrites .env before deleting it, as uninstall already did; appcmd unlocks are scoped to this site's location rather than server-wide, with the wide unlock as a fallback; DEVELOPMENT-SETUP says Python 3.14; the README plugin list gains printedparts; prune-schema --force is documented as first-provisioning-only; HTTPS is documented as not-the-default with the steps to add it; the DBA SQL is on the wizard's database page; the features page says unticking does not remove an installed feature; and the installer README states that bundle-lock cannot vouch for the exe itself - that needs signing or an out-of-band hash, neither of which is wired up. |
||
|
cproudlock
|
ced356882c |
net: strip the ephemeral source port from the forwarded client IP
IIS ARR sets X-Forwarded-For to clientip:port, and the port changes every connection. Left in, the audit log showed IP:PORT, the dashboard IP fallback never matched a stored (portless) DashboardDefault.ipaddress, and login rate limiting keyed per-connection instead of per-host. Add an IPv6-safe clientip.client_ip / strip_port helper and use it in the audit log, the dashboard resolver, and the login rate-limit key. |
||
|
cproudlock
|
75386d2f51 |
geenforce: resource-scope binding for fetch tokens (0.15.0)
A geenforce.fetch token can now be pinned to specific manifest scopes so a fleet-wide key (a display's, delivered by DSC or baked into the image) is not a skeleton key for the whole content store. NULL binding = unrestricted, so every existing service token keeps working. Core: - ApiToken.resourcescopes column + resourcescopelist property (migration 7d30_apitoken_resourcescopes; NULL = unrestricted). - apitokens API create/update accept + persist an optional resourcescopes list (a resource-name allowlist; not permission-catalog names). - New contract helper authorized_service_token(scope): same check as service_token_authorized but returns the ApiToken so a plugin can read its binding. Contract 0.14.0 -> 0.15.0; also export SupportTeam. GE-Enforce enforcement: - get_manifest: a bound token requesting a scope outside its allowlist -> 403. - get_payload: a bound token may only pull a blob its own scope(s) reference (service.blob_referenced_by_scopes); anything else -> 404 (no hash probing). - Decorator stashes the authorized token on g for the route to read. Also fixes a pre-existing contract-surface violation: the printers/printedparts alert helpers imported shopdb.core.models / shopdb.extensions directly; now via shopdb.api (SupportTeam newly exported). Docs: GE-ENFORCE-DISPLAY.md provisioning note, PLUGIN-HOOKS.md, CLAUDE.md. 9 new resource-binding tests; full suite 1131 passing. |
||
|
cproudlock
|
fb188fd302 |
alerts: per-support-team webhook; printedparts routes low-stock to a chosen team
Support teams gain a webhookurl (migration 7d29 + API + settings-page field), so a team is a notification target. send_webhook(url=) lets a caller override the site default with a team's webhook. Printedparts gains a 'alert support team' setting (printedparts_alert_supportteamid) + selector on its settings page; low-stock alerts post to that team's webhook, falling back to the site alert_webhook_url. Email leg unchanged. Same pattern extends to other alerting plugins (printers low-toner next). |
||
|
cproudlock
|
38c7ec347b |
alerts: Teams webhook fan-out (contract 0.14.0) + printedparts detail revision column
send_webhook(title,text) posts alerts to an optional webhook (Teams Incoming Webhook / Workflow, or generic JSON) via alert_webhook_url + alert_webhook_format settings; send_alert fans out to it alongside email; exposed on shopdb.api (0.13.0->0.14.0, PLUGIN-HOOKS synced); low-stock posts on its custom-recipient path too. Also: recent-transactions table shows the consumed print-file revision. |
||
|
cproudlock
|
7d7862f4b5 |
Neutral wording in import docs; portable tool output paths
Import-surface docs and docstrings describe the automation as a migration script; status-doc references in CHANGELOG/ADR-009/ROADMAP point at repo files. Screenshot/verify tools write to /tmp/shopdb-shots (created on import) instead of a machine-specific directory. |
||
|
cproudlock
|
cd353b6432 |
Review safe-polish: docs accuracy, dead imports, no-emoji, geenforce robustness
From the full multi-agent review (0 high, 7 medium, 17 low findings). Applies the mechanical, low-risk items; design/policy findings left for a decision. Docs accuracy: CLAUDE.md contract 0.10.0 -> 0.11.0 and both stale Alembic head citations -> 7d24_customfield_searchable / 31 migrations; Dockerfile bundled- plugin comment fixed (drop nonexistent "equipment", add machines + measuringtools, count eleven). Style/naming (LOCKED rules): remove a CSS-escaped pushpin emoji before location search results (no-emoji policy); rename ManifestEditor shareRoot -> shareroot (variable mirrors the API field verbatim). Dead code: remove confirmed-unused imports across ~20 modules (require_role/ require_permission scaffold residue, stray db/Vendor/Model/current_user/Optional/ error_response); drop unused build_scope import + a stale GEENFORCE_API_KEY docstring clause in geenforce. Migration files left untouched. Correctness: geenforce ingest robustness - record_enforcement_report now 400s on a non-dict counts / non-list results instead of 500; _apply_app_link ignores a non-numeric appid per its docstring instead of 500. Regression tests added. Backend query.get sweep finished: auth.py refresh -> db.session.get (last one). 910 backend tests pass; pyflakes clean; naming green; frontend build green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
cproudlock
|
d85b33bd68 |
Build GE-Enforce manifest-store plugin (P0/P1): model, importer, parity gate
First execution phases of docs/proposals/ge-enforce-plugin.md. The GE-Enforce manifest becomes shopdb data. P0 scaffold: new geenforce plugin (api_prefix /api/geenforce, default_enabled false, core_version >=0.7.0). Registered in PLUGIN_TABLE_OWNERS (ADR-008); its 0001 baseline really creates the tables. P1a model: one wide manifestentries table + entrytype discriminator (not STI, not JSON blob), manifestscopes (UNIQUE scopename+phase), the three multi-value filter child tables, inusechecks + processes, immutable manifestpublishedversions (frozen rendered JSON), manifestpayloads (inline, capped), pctypealiases (mirror of the engine lib's alias graph). regvalue stored as its raw JSON literal so DWord typing survives. P1c importer + exporter: parse common + gea-shopfloor-* + preinstall.json into draft rows and rebuild the JSON verbatim from rows in sortorder. P1d parity harness (GATE A): filters.py mirrors the engine's four filter functions + alias graph; parity.py proves import+export is behaviorally lossless (field-identical + same-entries-fire across 18 machine-profile fixtures) WITHOUT byte-diffing. Verified PASS against all 11 real reference manifests (64 entries) and a synthetic site-neutral fixture covering every type/filter (the CI gate). First slice (gea-shopfloor-cmm shape): service layer (import/publish/rollback/ export-to-share), CLI (parity, import-share, publish, export-share), and the client endpoint GET /api/geenforce/manifest serving the current published snapshot (never the draft) with ETag/304. Split permissions geenforce.manage/publish/fetch. Tests prove import->publish->serve, draft edits never change served bytes, publish+rollback, and auth (401 unauth/wrong-scope). Contract 0.11.0: added service_token_authorized(scope) to shopdb.api so plugin service endpoints authorize a scoped managed token without importing core token internals. Documented in PLUGIN-HOOKS.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
cproudlock
|
a846587f39 |
Add email sending (service + 3 flows) and a general asset label generator
Email: a stdlib SMTP mailer (settings-first config, graceful no-op when unconfigured), a test-email endpoint wired to the Email settings page, forced first-login password change (users.mustchangepassword, migration 7d23, /change-password flow), new-user welcome mail, and on-demand report/alert delivery (POST /api/reports/email + Email Report buttons) with an external-cron-with-a-scoped-PAT path documented for automation. All tests patch smtplib - no network. Labels: a shared /print/asset-label/<type>/<id> view any asset detail page opens - card or plain style, QR or barcode, configurable encoding. Per-type qr_target_* templates plus label_default_style/codetype/encodes settings on the Printing page. Measuring-tool labels default to encoding their inspection-operation code (derived from the location name, e.g. 0615), so every tool in an area shares the area code - verified by decoding the rendered QR. Machine labels default to the machine number; blank-serial handled gracefully. 808 tests pass; both features verified live. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
cproudlock
|
12175169e4 |
Accept managed collector service tokens on the collector API
A token scoped to the new collector.ingest permission is a collector service token: the collector endpoints accept it via X-API-Key or Bearer alongside the env fleet keys (which remain the fallback), giving the fleet credential rotation, revocation, and last-used visibility from the API Tokens page. Containment holds both ways: a collector token authorizes nothing else, and no other credential gains collector access. Shared token validation refactored out of the auth shim; a Collector service token quick-preset in the create modal; integration guide documents minting, rotation via site-config.json, and the service-identity pattern. 765 tests pass; live acceptance matrix verified. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
cproudlock
|
848a8fb34f |
Add optional permission scopes to API tokens
A token may carry a scopes list: it then grants only those permissions, intersected with what the owner holds at use time, with the admin role bypass suspended and role-gated routes denied - a scoped token from an admin account is genuinely limited. Scope ceiling enforced at create/update too (only permissions the owner holds; 400 lists violations) and the picker only offers what you hold. Token management itself now requires the new apitokens.create permission (admin by default, grantable via roles). Unscoped tokens keep the exact prior act-as-owner behavior; imports need an unscoped admin token. Migration 7d22. 756 tests pass; live-verified scoped 201/403 matrix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
cproudlock
|
da86b3ae0c |
Add personal API tokens; wire measuring tools into remaining surfaces
API tokens: any user mints named, optionally-expiring tokens (shopdb_pat_..., sha256-stored, secret shown once) at Settings > API Tokens; a before-request shim swaps a valid PAT for a request-scoped JWT of its owner, so the entire existing auth/authz/import-mode stack works unchanged and revoked/expired tokens 401 cleanly. Built for long-running scripts - the legacy import no longer dies when a login JWT expires. Migration 7d21_apitokens; create/revoke audit-logged. Audited integration gaps fixed: Asset.to_dict serializes measuring tools (typedata + pluginid - relationship links to tools resolve); map subtype filter/colors and MapEditor include them; dashboard totals count them; warranty links use a new by-asset route; the measuringtools ADR-010 hooks are real (corrected presentation token, implemented map-overlay endpoint); the login avatar resolves through the employee-photo helper. 737 tests pass; naming green; frontend builds; both features verified live end-to-end. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
cproudlock
|
46e50c07ff |
Add the API import surface for legacy migrations (contract 0.8.0)
Goal: an LLM or script can migrate an entire legacy database using only the HTTP API - original history preserved, safely re-runnable. - X-Import-Mode header (admin only): create/update endpoints across 15 timestamped entity types accept original createddate/modifieddate; helper exposed via shopdb.api (contract 0.7.0 -> 0.8.0). - Exact-match natural-key lookup filters on 13 list endpoints for the lookup-then-upsert recipe. - Selfhosted USB checkout/checkin accept backdated event times in import mode. - docs/IMPORT-API.md: operator manual grounded in the real legacy schema - order of operations, full table-by-table mapping including the machines fan-out, idempotent Python importer with dry-run, parity checks, and decided dispositions for unmigrated tables (DNC config stays live-fed via the collector; supportteams/appowners map to the upcoming supportteams model). 635 tests pass; naming green; frontend untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
cproudlock
|
b8c22244a1 |
Multi-site distribution readiness: settings-driven site config, security closeout, release engineering, v0.5.0
Make the app distributable to other GE Aerospace sites (one self-hosted
instance per site, ADR-004). GE values remain the shipped defaults; every
site-specific behavior is now a Setting an admin can change in the UI.
Settings-driven site config:
- Branding: site/QR/badge logos, favicon, primary color (upload endpoints
mirror the map-blueprint pattern; new Settings > Branding section).
- ServiceNow: search/incident/change URL templates ({ticket}), ticket
prefixes, enable toggle. Defaults point at the current
geaerospaceqa.service-now.com global search. Disabled = plain-text tickets.
- Employee-id regex (employeeid_pattern), printer hostname template,
QR label targets (qr_target_printer / qr_target_usb, blank = asset page,
else URL template with placeholders), usb_label_style (barcode|qr).
- West Jefferson floor-plan PNGs removed from the tree; generic placeholder
ships as the map default and sites upload their own blueprint.
Security closeout:
- dashboarddefaults writes now require admin.
- Collector: generic error messages (no str(exc) leak); API key accepted
via X-API-Key header only (BREAKING: querystring api_key removed).
- IP-based login rate limiting (AUTH_RATELIMIT_* knobs) atop account lockout.
- Setting.set() creation race fixed (IntegrityError retry).
Release engineering and docs:
- __version__ 0.5.0 (distinct from __contract_version__, ADR-007),
CHANGELOG.md, Gitea Actions CI config, frontend version aligned.
- One wizard-first install story across README/DEPLOY; new CONFIG.md,
UPGRADE.md, BACKUP-RESTORE.md; CLAUDE.md and ROADMAP de-staled.
- Dockerfile multi-stage build now bundles the frontend; compose binds
MySQL to 127.0.0.1; stale database/schema.sql and one-off SQL removed.
Debt and fixes:
- .query.get() -> db.session.get() sweep; datetime.utcnow() removed
(naive-UTC via timezone-aware now); users.py on authz decorators.
- Fixed 4 stale tests (slides feed shape, shopfloor splitperemployee,
plugin contract purity) and the USB label page field mapping (both usb
modes emit the cmmc shape: device_id/device_desc).
- Health endpoint reports the real version.
248 tests pass; naming/style check green; frontend builds; fresh-DB
flask db upgrade + seeds verified; QR targets verified by decoding
rendered codes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
cproudlock
|
65b0197b2b |
Document external-DB schema contracts for employees + USB plugins
Every site's HR directory and USB check-in/out databases may use a different schema, so document exactly what each plugin queries and how to adapt. - plugins/employees/README.md: required employees table columns (SSO, First_Name, Last_Name, Team, Role, Picture), the queries run, photo handling, and a CREATE VIEW recipe to map a different site schema without code changes. - plugins/usb/README.md: cmmc_usb devices / checkinoutlog / users columns, read-write ops, the employee-directory dependency, and a view recipe. - USB plugin gains get_config_schema() (cmmc_usb_db_host/name/user + password); cmmc_usb_connection reads host/name/user settings-first (env fallback), the password stays env-only - matching the employees plugin. - Config-field help points at the READMEs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
cproudlock
|
087ece0f8c |
Setup wizard P2: per-plugin config schema + settings-first creds
- Plugin contract gains get_config_schema(); the plugins list API returns it. Employees plugin declares its directory-DB fields (host/name/user + password). - employee_connection reads host/name/user settings-first (env fallback); the password stays env-only. - Setup wizard Features step renders each enabled plugin's config: non-secret fields save to settings; secrets are never stored - the wizard emits .env lines to paste. Fixed the plugins-list data path (data.plugins). - Settings PUT now upserts (creates the row on first write) so plugin-config keys can be saved without pre-seeding. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
cproudlock
|
78a0ee8d83 |
Add custom fields + warranty plugin, rework settings into two-pane shell
Feature work from the 2026-07 session: Settings IA - Replace the flat 27-card settings hub with a persistent two-pane shell (SettingsLayout.vue): grouped, searchable left rail + content pane. - Nest all settings/* routes under the shell via router post-processing; shared nav catalog in settingsNav.js. Group by asset class (PCs, Printers, Equipment, Network) so per-type settings stop scattering. Custom fields (core) - customfields + customfieldvalues tables (migration 7d14), CRUD API at /api/customfields, per-asset value get/save. - Settings management page + reusable CustomFieldsSection (detail) and CustomFieldsInputs (form) wired into all four asset types. Warranty (new plugin) - plugins/warranty: warranties + warrantyassets (migration 7d15), derived coverage status, provider abstraction (manual now; Dell/Lenovo/HP stubs). - API CRUD + per-asset panel + report buckets; WarrantyPanel on all four detail pages; Warranties management page; Warranty report + Reports card. - Seed warranty.* permissions. Printer drivers - printerdrivers table (migration 7d13) linked to printer models; drivers now surface on the matching printer's detail page. Other - PCDetail rebalanced (Network + Status + Warranty + custom fields on the right). - Rename PCs list "Features" column to "Remote Access"; fix badge hover underline. - Drop equipment islocationonly field. - Centralize asset-type label/route maps into utils/assetTypes.js. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
cproudlock
|
b516b9b771 |
Security: mask settings secrets, remove hardcoded employee-DB creds
- GET /settings now masks password/token values (were returned in plaintext to anonymous callers); sending the mask back on update is a no-op so the real secret is never clobbered. - Move the employee-directory DB credentials out of source into env-backed config (shopdb.utils.employee_db); employees + notification recognition use the shared helper. Employee lookups stop leaking exception strings. - Fix low-supplies report using loc.location instead of loc.locationname. Employee/notification read endpoints stay unauthenticated by design (public shopfloor kiosk displays consume them). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
cproudlock
|
5bfd25f977 |
Phase 6: simplify-pass policy, plugins listing, roadmap to 1.0
End-of-pipeline cleanup. No structural changes; documents what is done, what is left, and the discipline for future cleanup commits. Skill: - simplifying-python: end-of-session cleanup discipline. Inspired by Anthropic's open-sourced code-simplifier. Targets duplication, dead code, voodoo constants, stale comments, misnamed identifiers, unnecessary abstraction. Anti-targets: architecture, public API surface, schema, sweeping rewrites. Behavior preservation enforced by running pytest before and after. Simplify pass on shopdb/utils/responses.py: - error_response docstring claimed the error info lives at top level `error`. Implementation puts it under `data.error` (consistent with the success envelope). Implementation is correct; docstring updated. - paginated_response docstring used snake_case keys (`per_page`, `total_pages`, etc). Implementation uses lowercase concatenated per CONTRIBUTING.md. Docstring updated to match. Documentation: - docs/PLUGINS.md: bundled plugins (six, with what they track and caveats per ADR), planned plugins (measuringtools as the scaffold canary per ADR-005), distribution conventions for sister-site plugins per ADR-003, naming policy. - docs/ROADMAP.md: phase status table (0-5 done, 6 in progress), must-have work for 1.0 (Asset.mapx/mapy, equipment migration, printers retirement, frontend hook contract, per-plugin Alembic chains), nice-to-have (measuringtools plugin, frontend scaffolding, marketplace listing, surface-diff tooling), deferred (multi-tenancy, pip-installable plugins, event bus). Defines what 1.0.0 means as a contract. Test count unchanged: 101 passing. Naming/style check green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
cproudlock
|
ce7259c17a |
Phase 0: lock platform contract, naming convention, and style enforcement
Establishes the framework's foundation as a multi-site adoptable platform. ADRs (migrations/adr/): - ADR-001 (ACCEPTED): Asset is the platform contract; Machine retires. Three relationship types (partof, controls, connectedto) with free-text label, position-resolution chain (asset > related > location), hierarchical locations, sibling-bay propagation. - ADR-002 (ACCEPTED): Plugin contract semver via __contract_version__. - ADR-003 (ACCEPTED): Hybrid plugin distribution (in-tree bundled + filesystem-based external). - ADR-004 (ACCEPTED): Per-site instances, not multi-tenant. - ADR-005 (ACCEPTED): Equipment plugin (manufacturing) split from measuringtools plugin (metrology). Subtype-table pattern for protocol data (FOCAS, CLM, MTConnect). - ADR-006 (ACCEPTED): Plugin collector contract via get_collector_schema hook with API-key auth and identity-based upsert. Naming convention v1 (CONTRIBUTING.md): - DB tables/columns: lowercase concatenated, no underscores or dashes - DB-mirrored Python/JS variables match column names exactly; pure code follows host-language convention (PEP 8 / camelCase) - Closed acronym allowlist (universal + shop-floor domain), banned shorthand list with suffix exception (printers_bp etc allowed) - Plain ASCII everywhere: chat, docs, comments, string literals Style enforcement (scripts/check-naming-and-style.sh): - Pre-commit-runnable check script: non-ASCII, banned shorthand, snake_case DB names, snake_case API params in frontend - Fixes 14 violations across 11 files (Unicode arrows, snake_case params, ctx -> canvasContext, res -> response, req -> request_obj) Project state (CLAUDE.md, README.md, frontend/CLAUDE.md): - De-staled CLAUDE.md to reflect actual current state - README unifies DB story (MySQL canonical, SQLite test-only) - frontend/CLAUDE.md points at root convention Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
cproudlock
|
83caaa7b7c |
Add print badges, pagination, route splitting, JWT auth fixes, and list page alignment
- Fix equipment badge barcode not rendering (loading race condition) - Fix printer QR code not rendering on initial load (same race condition) - Add model image to equipment badge via imageurl from Model table - Fix white-on-white machine number text on badge, tighten barcode spacing - Add PaginationBar component used across all list pages - Split monolithic router into per-plugin route modules - Fix 25 GET API endpoints returning 401 (jwt_required -> optional=True) - Align list page columns across Equipment, PCs, and Network pages - Add print views: EquipmentBadge, PrinterQRSingle, PrinterQRBatch, USBLabelBatch - Add PC Relationships report, migration docs, and CLAUDE.md project guide - Various plugin model, API, and frontend refinements Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
cproudlock
|
30dd65674d |
Initial commit: Shop Database Flask Application
Flask backend with Vue 3 frontend for shop floor machine management. Includes database schema export for MySQL shopdb_flask database. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |