2feb569686aee682b004e7576343ad3ecf0a28d5
33 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
cproudlock
|
ccd8221adf |
Key the client address on the connection, not a header the caller sends
client_ip() read the first X-Forwarded-For hop unconditionally. The
login rate limit and the account lockout are keyed on that value and the
audit log records it, so rotating the header per request bought an
unbounded budget and an untraceable trail.
This walked around TRUSTED_PROXY_COUNT, which exists to decide exactly
this. The app leaves ProxyFix off by default and says why, ten lines of
comment away: ProxyFix trusts X-Forwarded-*, a client can send those
itself, and the login limiter is keyed on the address. Then client_ip
read the header anyway, so the stated protection never applied.
Resolution is now from remote_addr, which ProxyFix has already filled
from the header wherever a proxy is genuinely configured. Both cases end
at the same trustworthy source, and the port still comes off because ARR
forwards clientip:port and ProxyFix passes it through verbatim.
geenforce and tools had each hand-rolled the same mistake for their own
limiters. Rather than duplicate security-relevant logic a third time,
client_ip and strip_port join the shopdb.api surface (contract 0.28.0).
geenforce's _trusted_client_ip, which already refused the raw header for
its IP allowlist, is untouched and was the model for this.
The stability table was also missing rows for 0.26.0 and 0.27.0.
INSTALL-WINDOWS-IIS.md never mentioned TRUSTED_PROXY_COUNT, so a site
installed from it runs behind IIS with the setting at 0: every request
looks like it came from IIS, the whole site shares one rate-limit
bucket, and the audit log records the proxy for every action. It now
says so, and says not to set it when nothing is in front.
Three test modules simulated separate callers by setting
X-Forwarded-For, which is the header a real attacker rotates to escape
the limit. One of them, test_client_ip_prefers_forwarded_first_hop_
stripped, asserted the defect outright. The suite could not have caught
this. They now drive REMOTE_ADDR, with a regression guard that forging
the header buys nothing.
Scope: product
(cherry picked from commit
|
||
|
cproudlock
|
784964c907 |
Bring the wilm branch into main: endpoints plugin, agent, policies, CSV import
Ports the corporate-GitHub wilm branch onto main, keeping the two branches
separate. Thirteen upstream commits arrive with their authorship intact; the
review fixes and the portability work below are on top.
What arrives:
endpoints plugin - a ShopDB agent reporting endpoint state, telemetry,
services and interfaces, with a command channel the agent
long-polls. Ingest stays on the generic collector route
(ADR-006); the agent key is a UI-managed single-scope
token whose revocation is an instant kill switch.
computers policies - Configuration and Business Policies (RMM-style), scope
trees validated at write time, actions checked against
the endpoints command allowlist. Feature-gated: every
route 404s while the setting is off, so enabling it can
never half-expose the surface.
asset CSV import - templates, a collector script carrying the site path, and
a validate-then-commit import. A plugin can own a virtual
column plus a post-write row hook, which is how a PC's
machine number becomes a controls link without a
hand-built relationships sheet.
agent/ - the Windows agent and its installer.
Contract 0.21.0 -> 0.24.0, additive throughout: register_import_hook (0.22),
mint/list/revoke_service_token (0.23), collector_authorized (0.24). The stability
table had no rows for any of them and now does.
MySQL 5.6 portability. The branch stored three policy columns as sa.JSON, the
only JSON columns in the product, and 5.6 has no JSON type: the migration failed
with a bare syntax error near the word JSON. They are TEXT now, through a
JSONText type decorator, so all seventeen call sites still read and write dicts.
The convention elsewhere is already TEXT (guides.blocksjson,
geenforce.manifestjson).
Review fixes carried on top:
- six policy GETs were jwt_required(optional=True), which authenticates a
caller without checking anything; they require computers.view now, and six
redundant jwt_required decorators are gone
- the agent installer took a default BaseUrl and CollectorKey, so a mistyped
run silently pointed an endpoint at the wrong instance; both are mandatory
and a non-https BaseUrl throws unless -AllowInsecureHttp is passed
- a committing CSV import wrote no audit row
Documentation: the 34 new routes are described in docs/api-inventory.json, which
is hand-maintained on purpose, and the generated spec now covers 331 paths and
477 operations across 22 surfaces. DEVELOPMENT-SETUP's 1071 troubleshooting row
said "use MySQL 8 for dev" and now gives the actual fix, including the trap that
5.6 downgrades ROW_FORMAT=DYNAMIC to Compact with a warning instead of an error.
Full suite 2123 passed, vitest 260 passed, all seventeen plugin chains migrate
clean on MySQL 5.6 and the naming, docs, parity and read-surface gates are green.
|
||
|
cproudlock
|
151d7c4f44 |
Anyone not signed in holds the guest role
Whether somebody with no account may see something was being answered one feature at a time: a securesend_allow_guests setting, an enable flag per kiosk, and very nearly a third for guides - each with its own settings page, its own vocabulary and its own hand-written branch inside a route. Every one of them answered a permissions question, and this app already administers permissions per role. So a caller with no token is not nobody: they hold the seeded `guest` role. require_permission consults it when there is no identity, which is the entire mechanism. The role is seeded holding NOTHING, so an upgrade opens no door; a site opens one by ticking a box in the same grid it uses for every other role, and closes it the same way. Revocation is read on every request rather than cached, because taking guest access away is what somebody does the moment they realise what they opened, and it has to bite immediately. The redundant @jwt_required() above require_permission is gone from 184 routes. It was belt-and-braces before; now it is actively wrong, because it refuses a request with no token before the permission check can have an opinion, so a grant would appear to do nothing. What a guest may hold is bounded. Any .view or .read permission is grantable. A WRITE has to be blessed by whoever owns it, through the new get_guest_grantable_permissions hook (contract 0.21.0), and Secure Send is the only case that exists: sending a password to a vendor is the one write this product means to offer somebody who never signs in. Everything else is refused by the API and greyed out in the grid rather than warned about - a checkbox that quietly makes deleting an asset anonymous is not a warning anybody reads twice. A guest write records no creator: the trail says when and from where, never who, and that is the reason for the bar. GE-Enforce and Slides gained read permissions so they can be opened to guests without handing out the manage right. Their read routes accept either through require_any_permission, so nothing an existing manager could reach has moved. The navigation endpoint now returns only what the caller can actually reach. Without that a signed-out visitor gets a sidebar of the whole app where every link bounces off the login page, which reads as a broken site rather than as a closed door. A nav item may name several permissions; holding any one is enough, which is what GE-Enforce and Slides need now that a viewer and a manager hold different strings. Two things this found on the way: current_user is a LocalProxy, so `current_user is None` is False for a request with nobody behind it - harmless under a required JWT, an AttributeError the moment verification became optional. And an image pasted into a guide before its first save was never attached to it, so it was visible to editors and invisible to every reader; saving now attaches it, and `flask guides fix-images` repairs what is stored. The guides guest setting written earlier today is gone. Granting guides.view to the guest role replaces it, which is the point. |
||
|
cproudlock
|
28af92aaec |
An unmeasurable backup must not hide a working one
CI caught this: judging a host by its stalest kind treated a never-confirmed revision (lastseenat NULL) as the worst case, so it won. The verdict for a NULL confirmation is tri-state None, which renders as NO BADGE - so a host with a healthy ntlars backup and one backfilled row showed nothing at all. Hiding a working backup is the one thing that badge must never do. A NULL confirmation is not stale, it is unmeasurable. The stalest kind is now chosen among kinds that can actually be judged, which is the rule staleness.py already applies: a chain becomes measurable the first time its PC posts and says nothing until then. If every kind is unmeasurable the host still reports which kind it has, with a None verdict and no badge. The test that caught it pinned the old newest-wins behaviour, which part markers made wrong; it now pins stalest-measurable-wins and keeps its NULL clause, which was right all along and is what found the bug. Also worth recording how it was missed: the change was verified against test_geenforce_reporting.py and test_backups.py, and this assertion lives in test_geenforce_publish_gate.py, which was never run. |
||
|
cproudlock
|
ee208808f4 |
Judge a PC's backup by its stalest kind, not its newest
Until part markers, a PC reported exactly one backup kind and "newest revision wins" was the same thing as the truth. A marker bay reports two: ntlars for the machine's DNC settings, partmarker for the controller's patterns. With newest-wins, the daily ntlars post keeps the badge green while the marker backup has been failing for a month. The failure shows only on that marker's asset page, which is not where anyone looks to find out something stopped - and "is this still being checked" is the one question this badge exists to answer. So a host is green only when everything it backs up is current, and the badge names the kind that is not. Freshest revision per kind first, because lastseenat only ever moves on the newest revision of a chain; judging every row would report a healthy, frequently changing config as permanently stale. Known consequence, and the right direction to err: a device retired from a PC leaves its last revision behind, so that host stays stale until the revision is pruned. It complains about a backup that genuinely is not running rather than staying quiet about one. The dashboard card needed no change - it already lists every stale chain independently, so a stale partmarker chain was never hidden there. |
||
|
cproudlock
|
f34b9ca710 |
Carry the level everywhere a position is drawn, and gate it per occurrence
The hover mini-map said "This asset has a position (2835, 1410) but no level" for every asset in the product. When 0.11.0 gave LocationMapTooltip a levelid prop, NONE of its seven call sites were taught to pass one - printer, machine and PC detail pages, the toner report, enforcement reports, the warranty chip and the dashboard cards - so the component correctly reported a missing level and the preview never drew. Two payloads behind those views also emitted mapx/mapy with no level: the toner report and the enforcement report. The map PDF export had the ORIGINAL bug still in it: it plotted every filtered asset onto the sheet, so exporting the ground floor printed second-floor markers on it. Worse than on screen, because nobody can correct a sheet once it has been printed and carried onto the floor. It now exports only the level being viewed. The legacy import loader sent mapleft/maptop with no level at three call sites. That loader is the one still to run against production, and every marker it created would have been undrawable. It now resolves the site's default level - the legacy schema predates levels and has one floor plan, so that is what its coordinates mean. THE GATE MISSED ALL OF THIS because it asked whether a FILE mentions 'levelid', not whether each position does: one module emitted 'mapx' six times and 'levelid' once and passed. It now checks per occurrence, covers scripts/ as well as shopdb/ and plugins/, and fails any Vue file that binds tooltip coordinates without :levelid. Both new rules were confirmed to fail the build against planted violations before being relied on. Printer QR labels: the asset number is no longer printed. A label now reads name (8201-HPLaserJetPro), QR, FQDN, then IP. The name falls back to the assetnumber because that is where sites actually keep it - every printer here has an empty name field, so preferring the Windows queue name alone would have printed a blank line on every label. |
||
|
cproudlock
|
3324dbd91e |
Buildings and levels for the floor map, and make every identifier searchable
The map was one picture of one floor. A second floor was added, the blueprint changed size, and machines moved, so a position now records WHICH DRAWING its coordinates belong to. Buildings and levels (ADR-017). Each level owns its blueprint per theme and its own native pixel size; assets.mapx/mapy are pixels of assets.levelid, not of the site. A position whose level is unknown renders "level unknown" and is never drawn on the default level, because a marker on the wrong floor plan looks entirely correct while pointing at the wrong place. Repositioning in bulk: filter by unplaced, needs-review or level, search, place, confirm. Landmark recalibration solves the transform PER AXIS from landmark pairs and never from image dimensions - the canvas grew taller without rescaling, so a dimension-derived scale would stretch Y by 1.57 and be wrong everywhere. It defaults to a dry run, reports what would land off the drawing, snapshots before applying, and clears mapverifiedat because a transform is a guess awaiting review. Snapshots restore, including the level and the review state, and a restore snapshots first so an undo is undoable. Search: gaugelabreference was matched only for measuring tools and maintenancereference was matched nowhere at all, for any asset type, while Settings happily offers both identifiers on machines and PCs. A tag an operator is told to record has to be findable or it is a write-only field. USB devices and printed items were unreachable from search entirely - neither is an asset, so the generic asset search could not see them and no searcher existed; they now match on serial, asset tag, label, bin code and gage-lab tag, honouring isactive, with Settings toggles and result labels to match. The retired-application rule was half a rule: GET /api/knowledgebase hid articles whose topic application is retired while global search still returned them and printed the retired application as the subject. A filter is only real if every path that reaches the row applies it. Contract to 0.20.0 (additive): Asset gained levelid and mapverifiedat, Location gained levelid, and resolve_asset_position returns the levelid belonging to whichever source supplied the coordinates. The five plugins that write a map position are re-pinned. The install-list text format gained levelid as a NINTH field, appended, because the shipped Pascal installer reads fields 0-7 by index. That installer still compiles in one drawing's dimensions and bundles one blueprint, so its map is accurate for the default level only; /api/maplevels is deliberately unauthenticated so it can read both at runtime once rebuilt. Recorded in PRINTER-INSTALLER.md section 6 along with the other known gaps. Migration 7d33 converts an existing single-map site into one building and one default level carrying the old map_* settings, then assigns every placed asset and location to it. Nothing moves on screen. Old settings rows are kept so a rollback still finds them. Verified end to end on MySQL 5.6 from a production-shaped database. |
||
|
cproudlock
|
838932a72d |
geenforce: gate publishing on the library version, not on the manifest's own
The publish gate exists because a minor version bump that NARROWS behaviour is not backward compatible: _CmmVersion arrived in lib 2.6, and an older lib does not know the field, so every gated entry looks unfiltered and it installs every PC-DMIS version it cannot detect, on every CMM, in one cycle. It was comparing the fleet's reported library versions against manifestversion. That is the manifest's own 'Version' field. For a share-imported manifest the two numbering schemes happen to coincide; for a scope authored in code they do not, and seed_display_scope writes '2.0' - which every kiosk exceeds. So the gate passed on the scope that most needed it. A scope now declares minlibversion. Unset, the requirement is DERIVED from what the manifest actually uses, so a scope written before this column existed is still judged on its contents rather than on a number that says nothing about the library. Only features that narrow behaviour belong in that table; an addition an old lib ignores harmlessly needs no floor. manifestversion remains the last fallback, which preserves what share-imported manifests already relied on. |
||
|
cproudlock
|
4bded210e4 |
geenforce: the backup hover names both dates, because they mean different things
"Checked 13 Aug 1:20 PM. Verified the backup taken 12 Aug 5:20 PM is still current." Two facts, and one date could not carry both. lastseenat moves on every successful confirmation and proves the check is still running. collectedat moves only when the configuration actually CHANGED and says what is stored. Showing lastseenat alone hid what is in the backup; showing collectedat alone read as "last backup was a month ago" on a machine that is perfectly healthy, because a stable config writes no new revision. The hover now states the check and the capture separately and says outright that the second being older is the normal case. The stale wording gets the same treatment: it names the check that stopped AND the newest copy that exists, which is the thing someone recovering from it actually needs. collectedat is exposed as backupcollectedat. Both stay tooltip-only - the badge is still just the kind and a colour, so nothing here changes what a new backup kind has to do to inherit it. |
||
|
cproudlock
|
1d7191c2d3 |
geenforce: the fleet table links where it says, and judges backups instead of dating them
Two fixes to the same table, in the same regions of the same files. ASSET LINK POINTED AT THE WRONG RECORD. The Asset chip linked /machines/<assetid>, but /machines/:id keys on machineid - the plugin extension id - as MachineDetail itself does everywhere. So the link landed on whichever machine happened to carry that number: a wrong page that looks right, which is worse than a 404. Same for /measuringtools/. The API now returns machinepluginid / toolpluginid beside the asset ids and the view links on those. Both lookups are import-guarded, and with no plugin id the number renders as plain text rather than a link that misleads. AssetRelationships already resolved this correctly; this brings the reports table in line. BACKUP COLUMN READ AS NEGLECT. It showed a raw date, and a revision is only written when the config CHANGES - dedup means a machine stable for months has a months-old newest revision and is perfectly healthy. The column already used lastseenat, the last time the collector CONFIRMED the config, but a bare timestamp says "nothing has happened since", which at the default 24h collection interval IS the healthy steady state. It made a working system look stalled and made the reader do arithmetic against a setting they would have to go and find. It now returns backupok and shows a badge naming the kind, green when confirmed recently, red when not, with the date and an explanation in the hover. backupok is tri-state on purpose: null means no revision at all, and renders as NO badge rather than a green one, because "never seen" must not read as healthy. The threshold is the backups plugin's own backups_staledays, read through its service so there is one definition of stale rather than a second drifting here. Nothing in the badge is kind-specific, so a backup kind added later inherits it by existing. docs/BACKUP-KINDS.md records that, the BackupKind contract, and why the rule is time-based rather than per-kind. |
||
|
cproudlock
|
1e884dc02a |
geenforce: judge silence on both clocks, not just the server's
The fleet table has two time columns and staleness only looked at one. A PC can go quiet in either direction: silent to the server, or still posting while its own cycle has stopped advancing. Only the first was caught, so a machine whose Last check-in had not moved since the previous morning still showed 'ok'. The two are tested independently rather than by taking the newer of them. receivedat is the server's own record and cannot be argued with; lastcheckin is asserted by the client, so a wrong clock there must not be able to vouch for a PC the server has not actually heard from. Checking them separately means a future-dated lastcheckin cannot mask real silence. The tooltip now names both times, since which one stopped says what kind of failure it is. |
||
|
cproudlock
|
962979d483 |
geenforce: a PC that has gone quiet stops reading as healthy
A report records how ONE cycle went. Nothing ages it, so a PC that stops reporting keeps the status of its last good cycle: switch a machine off after a clean run and it shows 'ok' indefinitely. One had been offline more than a day and still read 'ok'. Silence is a different fact from the last cycle's outcome, so it is computed separately rather than by rewriting the stored status. receivedat - the server's own clock, not anything a client asserts - is compared against geenforce_reportstaleminutes, default 30, which is roughly two missed cycles at the usual cadence. Set it to 0 to turn the check off. In the table 'stale' takes the badge, because a status from a machine that has not spoken since is not evidence of anything. What it last reported stays in the tooltip with the time it was heard. The stored status is untouched: it is still a true record of that cycle, just not proof the PC is alive. A site whose scope enforces less often than the threshold will read stale while healthy, which is what the setting is for. |
||
|
cproudlock
|
84bf5d04ed |
geenforce: let a kiosk say what it is instead of guessing
A display knows whether it is a Dashboard, a Lobby screen or the 3D print room - the dispatcher reads C:\Enrollment\display-type.txt to choose which page to open. It never told shopdb, so the fleet table inferred it from the DashboardDefault fqdn mapping, which is empty unless somebody added a row per kiosk. The column was blank for every display. The client now reads that file and reports it, the report stores it, and the API prefers the reported value with the old mapping left as a fallback for hosts still on an older client. Reported by the device beats inferred from a lookup table, the same way enforcerversion already works. A PC with no display-type.txt reports nothing rather than something invented, and an empty string lands as NULL. Two guards had to learn about it. The DDL parity check read only the 0001 baseline, so a column added by a later revision looked like drift even though its migration existed; it now runs the whole chain, which is what 'do the models match what the migrations build' means. 0002 added a whole table rather than a column, which is why this is the first time it bit. |
||
|
cproudlock
|
f1f573862d |
geenforce: order backup revisions in Python, not in MySQL
The reports table 500'd on every load: the backup lookup ordered with ORDER BY lastseenat DESC NULLS LAST, which SQLite accepts and MySQL rejects outright. Every test passed and the real database refused the query - the tests run on SQLite, so the dialect difference was invisible. Sorting in Python removes the dependency for nothing: the rows are one per host per kind. The regression test pins which revision wins, including that one never confirmed does not, and says why the sort lives here so it does not get helpfully moved back into SQL. |
||
|
cproudlock
|
52eb10f5ca |
contract 0.17.0: expose DashboardDefault to plugins
The enforcement reports needed to name what a display IS, and reached straight into shopdb.core.models.dashboarddefault to do it. Plugins may only touch core through shopdb.api, and the contract test said so. The role belongs on the surface rather than behind it: it lives in core, no plugin owns it, and a plugin reporting on displays has no other way to resolve it. Added there and the version bumped, which the docs test pins. |
||
|
cproudlock
|
598c2c98bc |
geenforce: reports say what the PC is, where, and whether it is backed up
A report row carried a hostname and some counts. Everything an operator wants next hangs off the asset behind that host, and none of it was there. Host now links to its PC page. Beside it, what the PC IS or DRIVES: the machine number for a bay or part-marker PC, the measuring tool for a tool PC, the role for a display - each linking to that asset's own page. A map pin appears only when the asset has coordinates and hovers to the same floor-plan preview the asset pages use; an icon that opened an empty map would be worse than no icon. The backup column is LAST CONFIRMED, not last changed. Dedup means an unchanged config writes no revision, so a machine stable for six months has a six-month-old newest revision and is perfectly healthy - the one to worry about is the machine whose backup stopped running. It reads lastseenat and names the kind rather than assuming ntlars, since udc/file kinds on the share are coming. Resolution is bulk, never per row: this table shows the whole fleet, so a lookup inside the loop would be one query per PC. It reads the collector's existing 'controls' relationship rather than re-deriving which machine a PC drives - that same resolution living in two places is what put a wrong subtype filter on the map. Every plugin it touches is optional, so each lookup is ImportError-guarded and a lean build renders the table without those columns. A host ShopDB has no asset for still shows: the enforcement result is real even when the inventory is behind. |
||
|
cproudlock
|
9e34fafce5 |
geenforce: refuse to publish a manifest the fleet's lib cannot read
The engine treats a minor-newer manifest as backward compatible and carries on. That holds for additions which WIDEN behaviour - an old lib skips a Type it does not know - and inverts for one that NARROWS it. _CmmVersion arrived in lib 2.6 as a minor bump, so a PC on 2.5 does not recognise the field, reads every gated entry as unfiltered, and installs every PC-DMIS version it cannot detect, on every CMM, within one cycle. The share runbook already says push the lib first. A runbook is not a control, and the failure is silent, fleet-wide and about five minutes fast. ShopDB already had the evidence and was not using it: every enforcement report carries the enforcer version, and publish_scope had no gate at all. It now compares the scope's manifest version against the versions PCs actually report for that scope and refuses when any is behind, naming the hosts. force=True for someone who knows why. A report with no or an unreadable version counts as behind - that field arrived with the summary-emitting engine, so its absence IS an old lib, and treating unknown as safe is precisely how this fails open. A scope nobody has reported for still publishes, or a fresh site could never publish anything. Versions compare numerically, since as text '2.10' sorts below '2.9'. Also exposed as a preflight endpoint so the UI can warn before someone clicks publish, and as a 409 with the offending hosts rather than a 500. |
||
|
cproudlock
|
8b50e6fe2a |
geenforce: first dashboard card, and the widget contract it proves
Wave one of the dashboard proposal, built as a vertical slice so the contract is proven by something real before the other five cards follow. GET /api/geenforce/dashboard/failures lists entries that FAILED on their PC's most recent enforcement cycle. Per ENTRY, not per report: "three PCs failed" is a number, while "Install OpenText failed with exit 1603 on WJSF1234" is something a person can act on. Only current reports count, so a failure that has since been fixed clears itself instead of needing dismissing. Hostnames resolve to computerids in one query so each row links to the PC, and a PC shopdb does not know still appears - the failure is real even when the inventory is behind, and that is the bay most likely to be misconfigured. The data has been there all along. The only way to see any of it was to open one PC's report modal, one PC at a time. The widget declaration is the contract change. The old shape named a Vue component per widget, which cannot survive a lean build where a plugin's component may never be staged into the bundle - which is exactly why five plugins declare widgets pointing at components nobody ever wrote. This declares data, a generic renderer, a permission and a link template, the way ADR-010 already does for asset panels. A test asserts no 'component' key, so the old shape cannot creep back. empty: hide is part of the contract, not decoration. A card reporting "nothing wrong" daily teaches people to stop reading the page, which is how a fleet log reached 3,234 lines with 17 that mattered. Frontend rendering comes next; the endpoint and declaration stand alone and change nothing that exists. |
||
|
cproudlock
|
2c415a1712 |
fix(installer): correct a false security claim, and clear the should-fix list
CLIENT IP / SPOOFABILITY. docs/geenforce-api-cutover.md claimed that removing the IIS rewrite rule made the allowlist fail closed and that it does NOT become spoofable. The opposite is true. IIS never sets X-Forwarded-For on its own; the rule is the only thing that does. Remove it and IIS still forwards whatever X-Forwarded-For the CALLER sent, waitress trusts it because it arrives from 127.0.0.1, and remote_addr becomes attacker-controlled - so a token-less caller can fetch manifests from anywhere on the network. The document and the _trusted_client_ip docstring now say so, waitress runs with --trusted-proxy-count=1, and stage 5 checks the rule is actually live rather than assuming it. The wizard question is rephrased to something an operator can verify with their network team instead of guessing at. NON-ASCII. The style gate only ever checked .py/.vue/.js/.ts, so documentation accumulated em-dashes, arrows and box-drawing characters against this repo's own convention - including in files added this week. Cleaned, and the gate now uses INCLUDES_ALL so Markdown, JSON and YAML are covered. PLUGIN DEFAULTS. The wizard pre-ticked measuringtools and printedparts, both of which ship default_enabled=false, so every site taking the defaults installed and enabled them against their manifests. Inno has no JSON parser so the list must be hardcoded, but tests/test_installer_defaults.py now fails when it drifts. UPGRADES. The payload copy merges, so a plugin dropped from a site's profile kept its code forever - which defeats a lean build and leaves core's optional-import guards succeeding for a plugin the site no longer has. Stale plugin directories are now deregistered and removed before the copy. add-plugin used 'plugin install', which for the five default_enabled=false plugins left them installed but DISABLED - and printed a green success line anyway. It now goes through apply-profile, and the success line is gated on the exit code. Invoke-Flask records its own exit status, because $LASTEXITCODE keeps a stale value when flask.exe is missing and no native command runs. CHARSET. The utf8mb4 compiler hook lived inline in migrations/env.py, so it covered the CORE chain only: plugin baselines inherited the server default, which on a latin1 server means two charsets in one database. It is now shopdb/utils/mysql_charset.py, imported by both, and preflight reports the database's default charset. BACKUP HONESTY. The dump was described as 'all of your asset data'. Uploaded branding and floor-map images live in instance\ on disk, not in the database, so a restore from the .sql alone comes back with no map. backup now archives instance\ alongside it and says both are needed. VERSIONING. AppVersion was hardcoded at 0.9.0 while the product, the frontend and the newest tag said 0.7.0 - and 0.9.0 collides with a retired contract version. Both builders now generate version.iss from shopdb/__init__.py. Smaller: rollback overwrites .env before deleting it, as uninstall already did; appcmd unlocks are scoped to this site's location rather than server-wide, with the wide unlock as a fallback; DEVELOPMENT-SETUP says Python 3.14; the README plugin list gains printedparts; prune-schema --force is documented as first-provisioning-only; HTTPS is documented as not-the-default with the steps to add it; the DBA SQL is on the wizard's database page; the features page says unticking does not remove an installed feature; and the installer README states that bundle-lock cannot vouch for the exe itself - that needs signing or an out-of-band hash, neither of which is wired up. |
||
|
cproudlock
|
4c0cc672a2 |
geenforce: harden allowlist + fix share-less kiosk client and display scope
- allowlist auth uses remote_addr, not the spoofable first X-Forwarded-For hop (adds _trusted_client_ip + a regression test); rate-limit path unchanged - client psm1: fix Set-StrictMode crashes reading absent keys in Get-ShopdbConfig (token-less mode) and Resolve-ShopdbPayloads (no-payload entries); validate the manifest response is JSON before overwriting the last-known-good cache - runner: pass the engine its required -InstallerRoot/-LogFile; create the log directory so enforce logging is not silently lost on a fresh kiosk - display scope: dispatcher writes an all-users Startup shortcut instead of Start-Process (SYSTEM cannot show a window in session 0), resolves the base URL from HKLM, and adds an always-on power/no-lock entry; tests updated for the 6-entry scope |
||
|
cproudlock
|
0860aa85c5 |
geenforce: IP allowlist for client endpoints + admin Settings tab
Fleet PCs on a trusted (vaulted) network can now reach the GE-Enforce client endpoints (manifest, payload, report) without a per-PC token: the auth path accepts a valid geenforce.fetch/report token OR a source IP in the configured allowlist (setting geenforce_allowed_cidrs). Fail-closed; an empty allowlist means the token stays the only path, so existing deployments are unchanged. Rationale: the client token lives in HKLM on every kiosk, so it does not defend against a compromised kiosk anyway - network-perimeter trust is the same practical strength with far less provisioning + no token-rotation churn on a DB wipe. Documented in-UI that this is perimeter trust, not per-device identity. - _ip_allowlisted() (ipaddress, X-Forwarded-For-aware via _client_ip) - /geenforce/config GET/PUT extended with allowedcidrs, server-validated + normalized (bad CIDR -> 400) - new GE-Enforce > Settings tab (GeEnforceSettings.vue) to edit the allowlist in admin, no SQL - 3 regression tests (allow by IP, reject outside list, empty = token required) |
||
|
cproudlock
|
75386d2f51 |
geenforce: resource-scope binding for fetch tokens (0.15.0)
A geenforce.fetch token can now be pinned to specific manifest scopes so a fleet-wide key (a display's, delivered by DSC or baked into the image) is not a skeleton key for the whole content store. NULL binding = unrestricted, so every existing service token keeps working. Core: - ApiToken.resourcescopes column + resourcescopelist property (migration 7d30_apitoken_resourcescopes; NULL = unrestricted). - apitokens API create/update accept + persist an optional resourcescopes list (a resource-name allowlist; not permission-catalog names). - New contract helper authorized_service_token(scope): same check as service_token_authorized but returns the ApiToken so a plugin can read its binding. Contract 0.14.0 -> 0.15.0; also export SupportTeam. GE-Enforce enforcement: - get_manifest: a bound token requesting a scope outside its allowlist -> 403. - get_payload: a bound token may only pull a blob its own scope(s) reference (service.blob_referenced_by_scopes); anything else -> 404 (no hash probing). - Decorator stashes the authorized token on g for the route to read. Also fixes a pre-existing contract-surface violation: the printers/printedparts alert helpers imported shopdb.core.models / shopdb.extensions directly; now via shopdb.api (SupportTeam newly exported). Docs: GE-ENFORCE-DISPLAY.md provisioning note, PLUGIN-HOOKS.md, CLAUDE.md. 9 new resource-binding tests; full suite 1131 passing. |
||
|
cproudlock
|
9d65ef103d |
geenforce: display-readiness batch (server hardening, PS client wiring, display scope)
Get GE-Enforce closer to running on credential-less Intune/Entra display PCs that pull manifest + payloads over HTTPS instead of SMB. Server (plugins/geenforce/api/routes.py): - Rate-limit + 512MB served-size ceiling on GET /payload/<sha256> (reuses the login limiter's cache pattern, config-overridable via GEENFORCE_PAYLOAD_*). - New tests: payload hardening, manifestblobs model-vs-migration parity, and a report-contract test locking the lowercase per-entry report keys. PS client (plugins/geenforce/client/): - Fix New-ShopdbReport per-entry key casing to lowercase (name/action/selfhealed/ exitcode/message) to match what the server reads; the engine emits PascalCase. - Enforce TLS 1.2 in the network functions. - Fetch + merge the fleet-wide common scope alongside the pctype scope (pctype wins on conflict; -NoCommon opt-out). - Normalize whatever the engine returns into a well-formed summary. - Make the empty-cache fail-safe observable: event-log entry + report ping instead of a silent exit 0. Manifest (plugins/geenforce/seed_display_scope.py + docs/GE-ENFORCE-DISPLAY.md): - Seed a gea-shopfloor-display scope: 4 Edge kiosk drift-heal registry entries + 1 data-driven dispatcher (Dashboard/Lobby/3DPrintRoom via display-type.txt). Kiosk EXEs stay image-baked; the manifest heals policy/config drift only. - Documents the common SMB-payload audit (entries needing http/inline before a share-less display can inherit common). Migration registry (shopdb/plugins/alembic_template.py + test): - Register the pre-existing manifestblobs and the new printersupplyalerts tables in PLUGIN_TABLE_OWNERS; update EXPECTED_HEAD_REVISION for geenforce (0002blobs), printers (0002supplyalerts), and printedparts (0004txnrev) which had drifted. |
||
|
cproudlock
|
b00ef72581 |
geenforce: HTTPS payload delivery (content-addressed blob store + endpoint)
Lets share-less (Intune/local-account) PCs pull installers the manifest references over HTTPS instead of SMB - the general capability the whole fleet migrates toward. New ManifestBlob registry (migration 0002) with bytes on disk at instance/geenforce/payloads/<sha256> (deduped by content); service.store_blob + blob_path; client-facing GET /api/geenforce/payload/<sha256> (geenforce.fetch token, ETag=hash, serves the blob store or an inline DB payload by hash). The serializer now emits PayloadSource/PayloadSha256/PayloadRef for http/inline entries only (smb entries round-trip unchanged - parity green). CLI 'flask geenforce add-payload <file>' registers a blob and prints its sha256. This is the shopdb half (B1); the PS client/engine fetch is B2. |
||
|
cproudlock
|
cd353b6432 |
Review safe-polish: docs accuracy, dead imports, no-emoji, geenforce robustness
From the full multi-agent review (0 high, 7 medium, 17 low findings). Applies the mechanical, low-risk items; design/policy findings left for a decision. Docs accuracy: CLAUDE.md contract 0.10.0 -> 0.11.0 and both stale Alembic head citations -> 7d24_customfield_searchable / 31 migrations; Dockerfile bundled- plugin comment fixed (drop nonexistent "equipment", add machines + measuringtools, count eleven). Style/naming (LOCKED rules): remove a CSS-escaped pushpin emoji before location search results (no-emoji policy); rename ManifestEditor shareRoot -> shareroot (variable mirrors the API field verbatim). Dead code: remove confirmed-unused imports across ~20 modules (require_role/ require_permission scaffold residue, stray db/Vendor/Model/current_user/Optional/ error_response); drop unused build_scope import + a stale GEENFORCE_API_KEY docstring clause in geenforce. Migration files left untouched. Correctness: geenforce ingest robustness - record_enforcement_report now 400s on a non-dict counts / non-list results instead of 500; _apply_app_link ignores a non-numeric appid per its docstring instead of 500. Regression tests added. Backend query.get sweep finished: auth.py refresh -> db.session.get (last one). 910 backend tests pass; pyflakes clean; naming green; frontend build green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
cproudlock
|
4e5b4228c1 |
GE-Enforce: compliance view, inline payload upload, frontend test harness
Fleet-install compliance for app-linked manifest entries: new service compliance_for_scope + GET /geenforce/scopes/<id>/compliance count active ComputerInstalledApp rows by curated appid (null-safe when computers plugin absent). ManifestEditor gains a compliance panel. Curated appid stays shopdb metadata and never enters manifest JSON, so behavioral parity is unaffected. Inline manifest payloads: store_inline_payload (sha256, 1MB cap, payloadsource='inline') + POST/GET /geenforce/entries/<id>/payload; editor gains an upload control. Entry payload metadata surfaced in _entry_payload. Frontend test harness: extract the editor's entry-form logic into pure entryForm.js (buildEntryPayload, describeEntry, availableEntryTypes, scope gates, ...) and cover it with 45 vitest tests. ManifestEditor now imports those helpers, so the tests exercise the shipped code path (no duplication). 908 backend tests pass; vitest 45 pass; frontend build green; naming green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
cproudlock
|
3355436fcd |
Add curated manifest-entry -> Application link (honest app tracking)
The honest replacement for the backed-out auto-seeding: instead of scraping
manifest labels into duplicate Application rows, an entry can be LINKED to an
existing catalog Application, cross-referencing what shopdb already tracks.
- Model: manifestentries.appid (nullable soft ref to core applications; in the
0001 baseline). It is shopdb METADATA, deliberately NOT a manifest field - it
never appears in the rendered manifest JSON, so enforcement + parity are
unaffected (test asserts it stays out of the preview manifest).
- API: _entry_payload returns appid + resolved appname; create/update accept an
optional appid (validated, unknown id ignored, null unlinks) via _apply_app_link;
GET /geenforce/applications is the picker source (id + name).
- Editor: a "Tracked application (optional)" select in the entry modal, and the
entry summary line notes the linked app ("...; tracked: eDNC").
- Foundation for a future desired-vs-observed compliance view.
889 tests green (incl. the link test + parity/migration unaffected); build +
naming green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
cproudlock
|
3ac41c1556 |
Back out app auto-seeding; fix report-status + PCTypesStrict bugs (manifest review)
A deep Fable review of the real manifest corpus (READ-ONLY reference) showed the manifests are an ENFORCEMENT PROGRAM, not an application inventory, and that auto-seeding the Applications catalog from entry Type + Name was wrong: - The catalog ALREADY tracks these apps from the classic-shopdb migration, with version histories (PC - DMIS, UDC x11 versions, eMX / eDNC, CLM, CSF, Oracle Database, FormTracePak). Seeding from manifest labels created DUPLICATES under different names (PC-DMIS 2016 vs PC - DMIS; eDNC (bundles NTLARS) vs eMX / eDNC; OpenText HostExplorer ShopFloor vs CSF). It also misclassified config drops (eMxInfo.txt) as apps and could never match a PC's reported ARP name. So the seed-applications command + service are removed. Properly linking manifest entries to the EXISTING catalog is a curated feature, not label-scraping. Two REAL bugs the review found are fixed and kept: - Report status (R4): every healthy cycle runs Always/no-detection scripts the engine counts as "installed", so keying self-heal off installed>0 marked the common scope selfhealed forever and made 'ok' unreachable. Status now derives from explicit per-entry self-heal flags only; the stored flag no longer infers from action=='installed'; the client kit doc reflects it. - PCTypesStrict (R5): the runtime engine has no strict handling (preinstall runner only). filters.matches_pctype now applies strict only when phase == 'preinstall'; simulate + parity thread the scope phase through; the strict test uses a preinstall scope. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
cproudlock
|
0dcd186820 |
Fix defects found in session review of GE-Enforce plugin
Consolidated fixes from a three-dimension adversarial review. Data-loss (HIGH): the manifest entry editor stripped fields the form did not expose, because PUT /entries is a full reset-then-apply. The form now captures everything - InUseCheck processes as structured name/ExePath/timeout rows (not just names), LogFile, and the three preinstall flags as checkboxes; the dead payload-source control (never wired) is removed. New regression test proves an edit preserves ExePath/timeout/LogFile/PreEnrollment/PCTypesStrict. Update-entry crash (found by that regression test): replacing an entry's one-to-one InUseCheck (unique entryid) collided with the old row mid-flush -> IntegrityError -> 400. update_entry now frees the old InUseCheck (delete+flush) before populate re-inserts it. Export truncation (MEDIUM): export_scope_to_share used a plain truncating open, so a failed/partial write left the live on-share manifest (every PC reads it) empty. Now writes a temp file in the same dir and os.replace() atomically. Report dedup case bug (MEDIUM, confirmed by scratch test): the iscurrent demote matched hostname case-sensitively while the read path uses ilike, so a PC reporting different casing left two iscurrent rows and double-counted. Demote is now case-insensitive; regression test added. Simulator fidelity (MEDIUM): PCTypesStrict was captured but ignored by the filter mirror, so the simulator wrongly matched a collections-only strict entry to a nocollections PC via the shared Standard alias group. matches_pctype now honors PCTypesStrict (disables alias expansion); test added. Hardening: removed the dead/unscoped GEENFORCE_API_KEY env fallback (never wired into config; tokens are the only path); create/update entry return 400 on a duplicate Name instead of 500; parity now asserts scope-level Version/Site; a new test guards real-manifest field lengths against column limits (the DB-free parity harness can't see truncation); error handling added to the previously unguarded editor + reports API calls. Full suite green; naming + frontend build green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
cproudlock
|
adc5b7f69e |
Add GE-Enforce export-to-share + fleet-compliance UI (Milestone 1 UX)
Rounds out the Milestone 1 admin experience: author + publish in shopdb, push to the share by a button, and see what the fleet actually did. Export to share: - GET/PUT /api/geenforce/config stores the on-share export root (Setting geenforce_share_root); POST /scopes/<id>/export-share writes the current published JSON to <shareroot>/<scope>/manifest.json (preinstall.json for the preinstall phase), backing up the existing file to _meta/history first. geenforce.publish gated. The engine and PCs are untouched - this is the safe Milestone 1 push whose rollback is restoring the history backup. - Editor: a share-root config row + an "Export to Share" button per scope. - 3 tests (config roundtrip, export writes the file, second export backs up). Fleet-compliance UI (Settings > Enforcement Reports): - New page over GET /reports + /reports/<id>: latest report per PC with received (applied vs latest published version), status (ok/selfhealed/failed), and install/skip/fail counts; row detail shows per-entry outcomes with self-heal flags, exit codes, and messages. Hostname/PC-type filters. - ADR-010 settings card + ADR-009 plugin-gated route. Full suite 883 green; frontend build + naming green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
cproudlock
|
d157502d5b |
Add GE-Enforce P2 admin CRUD API: scopes, entries, reorder, simulate, publish
Full HTTP admin surface behind the manifest editor (geenforce.manage for edits, geenforce.publish for shipping): - Scopes: POST/GET/PUT/DELETE /scopes[/<id>] (create imaging PC types, edit the ComputerType/MeasuringToolType mapping + metadata, delete). - Entries: POST /scopes/<id>/entries, PUT/DELETE /entries/<id>. Payloads use the manifest Applications[] shape; populate_entry (refactored out of build_entry) updates an entry in place, resetting omitted fields and replacing children. - Reorder: PUT /scopes/<id>/entries/reorder enforces the ordering contract (body must list exactly the scope's entry ids). - Simulate: GET /scopes/<id>/simulate?pctype&subtype&hostname&machinenumber& cmmversion returns which entries apply and which filter excluded the rest, reusing the engine-mirror filters. The "what would this PC get" tool. - Publish lifecycle: POST /scopes/<id>/publish (records publishedby from JWT), GET /scopes/<id>/versions, GET .../versions/<n> (frozen manifest), POST /scopes/<id>/rollback. Entry type validated against ENTRY_TYPES; 8 CRUD tests. JWT+permission gated so the authz sweep covers them. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
cproudlock
|
6dc31c6149 |
Add GE-Enforce observed-state reporting: receipt + self-heal from PCs
PCs now report enforcement results back to shopdb, closing the desired-vs-observed loop. - POST /api/geenforce/report (geenforce.report service token): each cycle a PC posts the published version it applied, install/skip/fail/filtered counts, and per-entry outcomes. - Two tables: manifestenforcementreports (latest-per-host + history: applied version, enforcer version, counts, derived status ok/selfhealed/failed) and manifestenforcementresults (per entry: action installed/skipped/failed, selfhealed flag, exit code, warning/error message). - RECEIVED: reports carry the applied version; the admin view derives receivedlatest by comparing it to the scope's current published version, so the fleet view shows which PCs picked up an update. - SELF-HEAL: per-entry action captures drift correction (installed when it should already be present) vs skipped (already good) vs failed, with messages. - Admin reads: GET /reports (fleet compliance rollup) and GET /reports/<id> (per-entry detail). New geenforce.report permission. - Tables added to the (undeployed) 0001 baseline; geenforce.post_report is a service-token endpoint so it is exempt from the JWT authz sweep, like the collector blueprint. 8 reporting tests; full suite green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
cproudlock
|
d85b33bd68 |
Build GE-Enforce manifest-store plugin (P0/P1): model, importer, parity gate
First execution phases of docs/proposals/ge-enforce-plugin.md. The GE-Enforce manifest becomes shopdb data. P0 scaffold: new geenforce plugin (api_prefix /api/geenforce, default_enabled false, core_version >=0.7.0). Registered in PLUGIN_TABLE_OWNERS (ADR-008); its 0001 baseline really creates the tables. P1a model: one wide manifestentries table + entrytype discriminator (not STI, not JSON blob), manifestscopes (UNIQUE scopename+phase), the three multi-value filter child tables, inusechecks + processes, immutable manifestpublishedversions (frozen rendered JSON), manifestpayloads (inline, capped), pctypealiases (mirror of the engine lib's alias graph). regvalue stored as its raw JSON literal so DWord typing survives. P1c importer + exporter: parse common + gea-shopfloor-* + preinstall.json into draft rows and rebuild the JSON verbatim from rows in sortorder. P1d parity harness (GATE A): filters.py mirrors the engine's four filter functions + alias graph; parity.py proves import+export is behaviorally lossless (field-identical + same-entries-fire across 18 machine-profile fixtures) WITHOUT byte-diffing. Verified PASS against all 11 real reference manifests (64 entries) and a synthetic site-neutral fixture covering every type/filter (the CI gate). First slice (gea-shopfloor-cmm shape): service layer (import/publish/rollback/ export-to-share), CLI (parity, import-share, publish, export-share), and the client endpoint GET /api/geenforce/manifest serving the current published snapshot (never the draft) with ETag/304. Split permissions geenforce.manage/publish/fetch. Tests prove import->publish->serve, draft edits never change served bytes, publish+rollback, and auth (401 unauth/wrong-scope). Contract 0.11.0: added service_token_authorized(scope) to shopdb.api so plugin service endpoints authorize a scoped managed token without importing core token internals. Documented in PLUGIN-HOOKS.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |