f59aca5a1b5dc23fc3eae38af6e0a3962ae958e3
172 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
cproudlock
|
e60d6b74d8 |
Search: a domain that FAILED no longer looks like one that found nothing
Every domain catches broadly and returns no rows so one broken table cannot take the whole search box down. That part is right. On its own, though, it makes a DOWN domain indistinguishable from an EMPTY one: the technician sees no results, is offered a spelling correction, and concludes the record does not exist. It does - the search just could not look. Failed domains are now named in the response and stated above the results. The suggestion is deliberately still offered on a degraded run. Suppressing it was tried first and is worse: a chronically unreachable domain, an external HR directory say, would silently disable spelling help for every search for ever with nothing saying why. Naming the failure is the honest fix; withholding a second useful signal is not. Guides search also moves to the published revision. It correctly limited itself to published GUIDES, but matched and displayed title/summary/tags/blocksjson from the Guide row - which is the WORKING copy. A published guide with unpublished edits matched on wording that only exists in the draft, and the reader opened it to find the word absent. |
||
|
cproudlock
|
9d96006eae |
Audit log retention: a way to purge, off by default
The audit log is append-only and nothing ever removed a row, so it grows for as long as a site runs. This site reached tens of thousands of pages, which is past the point of being useful: nobody pages through that to find anything. Retention is OPT-IN and the default keeps everything. An upgrade that silently started deleting a site's audit history would be indefensible, so audit_retentiondays defaults to 0 and nothing is removed until somebody sets a number. The Site & Facility card shows the log's size and exactly how many entries a purge would delete BEFORE anything is pressed. Deleting happens in committed batches. That is not an optimisation: one DELETE over a million rows builds a single enormous transaction and holds locks across the table while it runs, which on this server means blocking the collectors posting inventory. The API caps its batch count so the request cannot outlive a proxy timeout, and a run that stops early reports how much is left rather than claiming it finished - a purge reporting success over 900k remaining rows would be the same silent half-failure this codebase keeps getting bitten by. `flask audit stats` and `flask audit purge` for the server, the latter reporting by default and deleting only with --apply, with no batch cap because it is not behind a proxy and a purge that stops halfway leaves the operator guessing. audit.purge is a separate permission from audit.view. Reading the log and destroying it are different acts, and plenty of people need only the first. Chunking selects a page of ids and deletes those rather than using DELETE ... LIMIT, which is MySQL-only while the suite runs on SQLite. |
||
|
cproudlock
|
3c68ac2850 |
ADR-015: the site-literal check was matching nothing, and had been all along
The patterns are assembled from fragments so the published script does not
contain the strings it greps for. They used the apostrophe-escape form
('a'\''b'), which EMBEDS a literal apostrophe, instead of the concatenation form
('a''b'), which joins them. Every pattern therefore carried a stray quote in the
middle and could never match a line of real code, so the check passed everything
from the day it was written.
Verified the only way that means anything: plant a violation, watch it fail,
revert. Fixed, it immediately found 25 real hits.
Among them, in seven places, the default ServiceNow URLs - one organisation's QA
tenant, shipped with the integration ENABLED. Any site that had not configured
ServiceNow still rendered ticket links, pointing its technicians at a tenant
they cannot reach; an adopting site would have been pointed into somebody else's
instance entirely. There is no sensible cross-site default for "which ServiceNow
is yours", so the URLs ship blank and the integration ships off, matching the
Zabbix block that got this right.
A named SaaS tenant is as site-specific as a hostname, so it joins the patterns.
Colocated .spec.js files are excluded the same way tests/ already is: sample data
naming a facility is not a shipped literal.
|
||
|
cproudlock
|
03094e93b2 |
Let Guides be switched off in Settings > Search like every other domain
Global search has per-domain toggles (search_<type>_enabled), enforced server-side and edited from Settings. Guides answered the search box but had no toggle, so it was the one domain nobody could turn off - and the omission would only have surfaced when somebody went looking for the switch and could not find it. Missing key still means enabled, so nothing changes until somebody decides otherwise. |
||
|
cproudlock
|
2c016de59e |
Guides answer the search box, and proofing catches what spellcheck cannot
A guide was invisible to global search. Core search has a hardcoded block per domain rather than a hook, so a new plugin stays unreachable until someone adds one - which is the same shape as the gap this test file already documents for USB devices and printed parts: records with a list page, labels in the world, and no way to find them from the search box. Guides are searched by title, summary, tags AND body. The body matters more here than elsewhere: people look for a procedure by something written halfway down it, not by a title they never read. PUBLISHED ONLY - a draft is deliberately invisible to readers, and search is exactly where that rule leaks if nobody writes it down. The test proves it with a draft sharing a tag and a body word with a published guide, so a missing filter would show up as a ride-along. PROOFING catches what a spellchecker cannot, because none of these are spelling: a smart quote in a command so it will not run, a checklist item with no text so a step is silently missing, a heading left with nothing under it after a reorganise, an image with no alt text, a TODO published by accident. Warnings, never blocks - somebody publishing at 14:40 with one empty bullet should be told, not stopped, and a procedure that exists imperfectly beats one nobody wrote. The smart-character issue reports once per block with a one-click fix, rather than once per offending character: an opening and a closing quote are one problem, and listing both makes the panel look like it is nagging. The character table itself is written as escapes. The style gate forbids non-ASCII in source, and a file whose job is detecting smart quotes cannot contain any - which the gate caught, correctly. |
||
|
cproudlock
|
38d1e367fc |
Guides: write a procedure in ShopDB, not beside it
The knowledgebase plugin is a link registry - a description, a URL and some keywords pointing at a document held somewhere else. That leaves the documents themselves as HTML files on a share: no search, no history, no way to know whether the copy pinned to a noticeboard is current, and no way for a technician to tick off where they are. A guide is stored as an ordered list of typed blocks rather than a blob of HTML, because three of the four things this feature exists for depend on it. A tickable step needs a stable id to be remembered; dark mode needs semantic markup rather than pasted colours; and print needs to know where a step begins so it does not split one across a page. Free HTML gives none of that and lets pasted styling fight the theme. Inline text is a restricted Markdown subset - bold, italic, inline code, links, line breaks - escaped BEFORE formatting, so nothing an author pasted can reach the DOM as markup. No fonts, sizes or colours: emphasis is semantic and the theme decides how it looks. That is what makes a guide written next year match one written today, and dark mode work without an author thinking about it. DRAFTS AND VERSIONS. The guide row is the working copy; every save that changes something writes an immutable revision; and a separate pointer says which revision READERS get. Without that pointer, editing a live procedure changes it under a technician who is halfway through following it. Publishing promotes an existing revision rather than snapshotting, so a publish with unsaved edits cannot ship a version nobody reviewed. Restoring writes a new revision with old content, so the record of what people were told in between survives. Saves dedup on a content hash covering title, summary, tags and blocks, so an autosave that changed nothing writes no version - otherwise the history fills with identical entries and hides the three edits that mattered. Progress lives in the browser, not the database: shopfloor PCs share a login, so per-user ticks would show one technician's progress to the next person on that bay, which reads as "already done". Keyed by guide and version, because step 4 of version 3 is not step 4 of version 2. Five permissions, and the split between edit and publish is the point: a technician can write a procedure up while somebody else decides it is fit for the floor. create-without-edit works too - an author may edit their own guide until it is published, after which other people are following it. |
||
|
cproudlock
|
23784b8199 |
Ship one name per vendor, and stop shipping a test row
Canonical names chosen by the site: Dell Inc., HP, Zebra. The rule is the same in each case - keep the spelling that holds the entries - it just lands on a different form per company. Dell Inc. carries 310 references and 33 models against 1 apiece for Dell and DellInc.; HP carries 51 and 24 while HP Inc. has none; Zebra carries 7 and 2 while Zebra Technologies has none. So HP Inc. and Zebra Technologies are gone from the shipped catalog. Both were empty rows that only ever offered a second way to spell a vendor already present, and neither exists in this site's database at all - they would have arrived as duplicates at a NEW site and nowhere else. Also removed UnscopedLiveVendor, which is a test artifact. It reached the catalog because export_catalog.py reads the live development database, so a row left behind by a lab run was published as reference data and has shipped to every site seeded since. Vendors 51 to 48. Verified nothing referenced any of the three before removing them, and the catalog still has no duplicates by any spelling. |
||
|
cproudlock
|
6106f4c5f4 |
Stop shipping duplicate vendors, models and operating systems
The seed catalog is what a new site inherits, so its duplicates propagate. Removed the four space-stripped twins it carried: vendor 'DellInc.', model 'OptiPlex7080', and operating systems 'MicrosoftWindows10Enterprise' and 'Windows10', each of which had a correctly spaced original already present. 'OptiPlex5000' was NOT a duplicate - no spaced twin existed - so it was renamed to 'OptiPlex 5000' rather than dropped, matching every sibling in the list. Its vendor moved to 'Dell Inc.', which this site treats as the one canonical Dell. Vendors 53 to 51, models 128 to 127, operating systems 14 to 12. Verified afterwards: no exact duplicate rows, no near-duplicates, no orphaned model-to-vendor or supply-to-model reference. |
||
|
cproudlock
|
d51127841d |
Merge catalog rows that are the same name typed two ways
The legacy data carried space-stripped twins: 'DellInc.' beside 'Dell Inc.', 'Windows10' beside 'Windows 10', 'OptiPlex7080' beside 'OptiPlex 7080'. They rode into the shipped seed catalog, so every site that ran `seed catalog` inherited them, and they stay invisible until somebody opens a vendor picker and finds the same company listed twice. Not a code fault. The collector strips only leading and trailing whitespace, matches with ilike, and treats the OS list as lookup-only, so it can neither have created these nor be creating more. They arrived with the classic-ASP import, which holds the same three pairs today. `flask catalog dedupe` merges only whitespace-and-punctuation twins, because those are one name typed two ways. 'Dell' and 'Dell Inc.' normalise apart and are left alone: that is a naming choice and only a person can make it. The surviving row is the most referenced, tie-broken toward the spaced spelling, which is the one somebody pressed the space bar for. `flask catalog merge` makes that judgment call explicitly, and refuses to invent a target that does not exist, so a typo cannot move rows into a new vendor. `flask catalog rename` handles the case neither can reach: a stripped name with NO twin, where there is nothing to merge into. All three report and change nothing unless --apply is passed, and dedupe is idempotent so it is safe in a deploy script. References are discovered from the LIVE schema rather than listed in code. There are 15 foreign keys into these three tables across 7 tables, and machines.controllervendorid and printerdrivers.modelnumberid are not ones anybody would enumerate by hand. Every reference is repointed before the row is deleted, so nothing is orphaned, and a plugin adding a table is covered without editing this command. |
||
|
cproudlock
|
7b79a444b8 |
Scaffold a plugin's frontend into the plugin, so it can be pruned
`flask plugin new` wrote views to frontend/src/views/<name>/ and a route file to frontend/src/router/routes/<name>.js. Every one of the 16 bundled plugins keeps its frontend in plugins/<name>/frontend/ instead, staged into .plugins-staged by stage-frontend.mjs and aggregated into routes.gen.js. It was not visibly broken, which is why it lasted: the router still globs ./routes/*.js during the Phase 4 dual-location transition, so a scaffolded plugin's pages loaded. What a new plugin author silently lost was ADR-013 lean builds. Pruning works off the staged tree, so a frontend in the core tree cannot be excluded from a site build, and deleting the plugin strands its views behind. The templates for the correct layout already existed; only the output paths were wrong. Also fixed the imports they carry: core imports now use the @ alias (@/api, @/components/...), which resolves both before and after staging, where the previous ../../api was correct only in the old location. Sibling views stay relative and travel with the route file. frontend_dir is gone from scaffold_plugin. There is no second output tree to point at, and a plugin developed in its own repo now gets the same frontend as a bundled one rather than being skipped. Proven end to end: scaffolded a throwaway plugin, confirmed it stages, builds real chunks, and that SITE_PLUGINS=network prunes it, which is what the old layout could not do. |
||
|
cproudlock
|
eecb0595dd |
Open the record a search result found, not the list it sits in
A NOTICE result linked to /notifications, so clicking the row search had already identified dropped the reader on the list page to find it again by hand. The id was in the payload the whole time; only the URL ignored it. `/notifications/<id>` is the existing notification-detail route. The subnet searcher had the same defect two functions away: an IP search resolves which subnet the address falls inside, then linked to /network, the hub. It now links to /networks/<id>, the subnet detail route. Every other search result type already carried its id. These were the only two that did not. |
||
|
cproudlock
|
b96637e1a1 |
Let a baseline meet a table that is already there
Reported: `flask plugin upgrade-all` failed the tools plugin with "table securesendsecrets already exists". Every post-cutover baseline emitted a plain op.create_table. That is correct only while a table's existence and its plugin's Alembic stamp can never disagree, and they can. Anything that builds tables from the models - db.create_all(), which `flask db-utils create-all` calls - creates them and stamps nothing. A run that creates a table and then fails before its stamp commits leaves the same split, and re-running is the obvious thing to try. In both cases upgrade-all meets existing tables with an unstamped chain and the create is fatal. Nothing caught it because the migration test DROPS the post-cutover tables db.create_all() built before running upgrade-all. That arranges the one state where an unguarded create works, and never the state the report came from. The new test is the same sequence with the drop removed for the plugin under test. create_table_if_missing and create_index_if_missing are in alembic_template.py next to create_plugin_tables, which had the idempotency rule right and turns out to be called by nobody - its docstring claimed every 0001 used it. Fixed the docstring rather than rewiring the baselines onto it, because the reason they hand-write their DDL still stands: the helper's per-plugin MetaData cannot resolve a foreign key to a core table at CreateTable-compile time. INDEXES ARE MATCHED ON COLUMNS, NOT NAME, and that distinction matters. The two creators disagree about names - db.create_all() takes SQLAlchemy's default convention (ix_securesendsecrets_token) and a baseline spells it the way this repo's naming rules require (ixsecuresendsecretstoken). A name-keyed guard would have called the index missing and added a second one over the same column, which MySQL permits with a warning, so every affected site would have quietly grown duplicates. The test asserts no table ends up with two indexes over the same columns. Applied to all nine table-creating plugin migrations, not just the tools baseline that was reported: the four later migrations that create tables have the same exposure, and geenforce, measuringtools, printedparts and backups are the same defect waiting for a different site. WHAT THIS DOES NOT FIX. It makes a baseline idempotent, not a whole chain. A longer chain can still meet an ALTER whose column is already present - printedparts adding gagelabtag to a table create_all built at head is the case that showed up while testing. Guarding every op one at a time is the wrong answer to that; a database built from the models is already AT head and its chains want stamping rather than replaying, and there is no `flask plugin stamp` today. Left alone deliberately rather than half-built, and the new test is scoped to single-baseline plugins so it does not pretend otherwise. |
||
|
cproudlock
|
fc46bab72d |
Say that the subpath belongs in site_base_url, because it does
Answering "did I set it wrong": no. The setting was right and the description was misleading, which is what sent the last two commits chasing the wrong fix. It said "Public base URL of this site (scheme + host), e.g. https://shopdb.example.net". That is true only of a site mounted at the root. Both existing consumers append the in-app route to this value and add nothing themselves: `${await getSiteBaseUrl()}${config.detailPath(detailId)}` asset labels `${baseUrl}${defaultPath}` QR codes so a site served at /shopdb must carry that here or every QR code it prints 404s. The email copy of the same setting has always documented it correctly, as https://host/ops. The description now says so, and the comment above the defaults records why - including that a subpath left out produces links that fail rather than anything that looks like a configuration error. Existing rows keep the old text until a site reseeds; the value is unaffected. |
||
|
cproudlock
|
d4b49b91b4 |
Move secure send into Tech Tools, where somebody would look for it
|
||
|
cproudlock
|
98458672eb |
Hand somebody a password without leaving it in a chat window
The habit this replaces is pasting a credential into Teams, where it stays until someone deletes the conversation. Tabled on 2026-08-21 pending a decision about scope; the decision was text only, so this is the half that survives it. BACKEND ONLY. There is no page yet, so nothing here is reachable by a person. TEXT, NOT DOCUMENTS, and that shapes everything. Dropping files removed the export-control question entirely - a password is not technical data - along with blob storage, size and MIME handling, and the need to authenticate the recipient so a document fetch could be pinned to a name. TWO DOORS. Creating needs a login AND securesend.create: anonymous creation would be an untraceable way to move data out of a system that otherwise records who did what, and the creator is the only person these rows can ever name. Reading needs neither, because the recipient usually has no account here and the link is handed over on a call - the unguessable token IS the authorisation, which is why it is 128 bits of randomness and why the row is spent on first read. REDEEMING IS A POST, and that is the detail that would have broken this in practice. Teams, Outlook and Safe Links prefetch GET urls to build previews and scan for malware, so a GET that redeemed would let a preview window consume a secret before its recipient ever saw it. The GET route that remains reports only whether a link is still live, so the read page can say "opening this will destroy it" without spending the only view to find out. THE SIZE LIMIT IS THE WHOLE ENFORCEMENT. The key lives in the URL fragment, which browsers never transmit, so the server holds ciphertext it cannot read and cannot tell a password from a page of a drawing. MAX_CIPHERTEXT is therefore not a tuning knob: raising it quietly turns this into the document transfer it was scoped not to be. The refusal says what the limit is FOR, because somebody hitting it is usually trying to send a file and the answer is a different tool. Line counts, file signatures and refusing a pasted file belong in the browser, guide an honest person, and are bypassable by anyone willing to POST directly. NOT ENUMERABLE. Missing, expired and spent all answer identically - telling them apart would confirm to anyone holding a guessed token whether it ever named anything. A spent or expired row is deleted on the way past rather than waiting for a sweeper, because it has no reason to keep its ciphertext. Its own plugin rather than a Tech Tool: that plugin is browser-only by design, and this needs storage, expiry and a view counter. Not an asset extension either, so no assetid and no asset type - the scaffold assumes one and it was removed. FOUND BY ITS OWN TESTS: expiryhours=0 returned 201. `int(x or DEFAULT)` treats an explicit zero as absent, so "expire immediately" silently became "expire in a day". A caller who sends a number is now told when it is not allowed rather than quietly given a different one. BEFORE ANYONE RELIES ON THIS, test that your link rewriter preserves URL fragments: create a secret, send the link to yourself in Teams, click it from the message, and check the key is still after the #. If Safe Links strips it the secret is unrecoverable and the server cannot help, which is the point of the design. The fallback is passphrase mode, worth building only if that test says so. Three product gates caught what a new plugin has to declare, and each was right to: the route-parity gate wanted the endpoints described in api-inventory.json, the lean-build guard wanted the plugin listed in the universal profile the released installer builds from, and the authz sweep wanted the public redeem route named in EXEMPT_ENDPOINTS with a reason. That last one matters most - a mutating route that answers an unauthenticated caller is a decision somebody has to write down, not a default, and reading a secret is a mutation because it spends it. |
||
|
cproudlock
|
5001aedcf9 |
Offer a spelling on every list, not just the global search
Correction only helped on the global search page, and people search from the list they are already on. Extending it turned out to be a shape question rather than a volume one: thirty routes take a search parameter across a dozen files, so carrying a suggestion in each of their responses is a large change today and one more thing every future plugin author has to remember. So the suggestion moved to ITS OWN ENDPOINT, /api/search/suggest, which any page can call after rendering no rows. A page that has not adopted it shows nothing, which is exactly what it showed before - nothing breaks by omission. The plumbing lives in useListQuery, which already owned the search term, so a list needs three lines: take `suggestion` and `reportCount` from the composable, call reportCount(rows.length) after a fetch, and drop SearchSuggestion into the empty state it already has. A list that never calls reportCount never offers a suggestion. Wired: global search, machines, printers, PCs, network devices, measuring tools, knowledge base, vendors. NOT wired, deliberately: the type and reference lists (machine types, PC types, VLANs, subnets, operating systems and the rest), which are small controlled vocabularies nobody typo-searches, and USB, whose empty state has a different shape and wants doing by hand rather than by pattern. TRAP FOUND WHILE WIRING IT, and left commented in every page: applying a suggestion by calling setSearch alone updates the box and the URL and does NOT reload the list. setSearch only syncs the URL, and the watcher that would reload is suppressed because search.value already holds the new term - the same trap the global search page documents in performSearch. Each page calls its own load function directly. The composable guards a stale answer arriving after a newer search was typed, never offers back the word that was typed, and swallows its own errors: a search that found nothing is already the answer, and failing to improve on it is not worth an error in front of anyone. The route-parity gate caught the new endpoint being served without an entry in docs/api-inventory.json, which is hand-written on purpose; added, and the spec regenerated from it (283 paths, 418 operations). That regeneration also carries the openapi version to 0.12.0, left over from the release. |
||
|
cproudlock
|
1593ee8204 |
Offer a spelling when a search finds nothing
The corrector landed in
|
||
|
cproudlock
|
fa0f6d7ebc |
Record what nobody can find, and a spelling corrector that refuses to guess
Two halves of one question - how to handle a misspelled search - deliberately kept separate, because only one of them should be turned on today. WHAT NOBODY CAN FIND IS NOT RECORDED ANYWHERE. A search returning zero results is the only evidence of the gap between what people look for and what is there, and it vanished. Logged now at INFO with a stable prefix, so a week of it greps into a list. That list is what should decide whether correction is worth wiring in: multi-word matching and cross-field matching both changed in the last day, so a good share of what used to fail may already be found. The failures left over might be typos, or vocabulary nobody has entered, or records that genuinely do not exist - and each wants a different answer. THE CORRECTOR IS BUILT AND NOT WIRED IN. shopdb/core/services/spellfix.py, with tests, ready to attach in about ten lines once there is evidence about what to attach it to. NOT SOUNDEX, which was the obvious candidate. MySQL has it and SQLite does not, and the suite runs on SQLite while production runs MySQL - so a SOUNDEX() in a query is either an error in every test or a production path no test executes. It is also wrong for this data: soundex is English-name phonetics, four characters wide, and it DISCARDS DIGITS, so CSF16 and CSF17 hash identically. Half of what people search here is an identifier. So: character distance in Python, same behaviour on both dialects. THE VOCABULARY IS THE DATA. No dictionary holds Genspect, Telesis, Keyence or wax-trace. Terms come from the columns being searched, which also means a vendor added this morning is correctable this morning. TWO GUARDS, and they are the point rather than a detail. Digits must match EXACTLY: CSF16 to CSF17 is one edit and a different bay, so anything carrying digits is either right or not correctable - while cfs16 to csf16 still works, because the guard is on the digits and not on identifiers wholesale. And the first character must match, since typos land mid-word far more often than on the first key, which costs almost no recall. Distance is Damerau-Levenshtein so a transposition costs one edit rather than two - Keyecne for Keyence is the commonest error there is, and plain Levenshtein pushes it past the threshold on short words. Allowed distance scales with length. A tie returns NOTHING: two equally good candidates means there is no answer, and offering either implies a confidence that is not there. It suggests; it never rewrites. Silently searching for something else is how somebody orders the wrong cartridge. 13 tests, weighted toward the refusals, because those are the cases where being wrong costs something. |
||
|
cproudlock
|
d0eeaa08d5 |
Find a knowledge base article by words from different fields
An article's searchable text lives in three places: its title, its keywords,
and the name of its topic. Nobody typing a search knows or cares which word
came from which, so 'CMM Community' means "the article tagged community, under
the CMM topic". Both search paths returned nothing for it.
The plugin's own listing, which is what the KB page calls, built a single
ilike('%CMM Community%'). That needs the whole phrase contiguous in ONE field,
and no article has it: 'CMM' is only in the topic's name, 'Community' only in
the keywords. Global search already split the query into words but only ever
looked at title and keywords, so a word that only the topic could satisfy
failed there too - the same bug wearing a different face.
Every word must now be found somewhere across all three fields, in any order.
_word_match gains an `extra` hook for a word that a RELATED row satisfies
rather than a column of this table, which is how the topic joins in without
colliding with the sort=='topic' join.
The retired-topic rule is unchanged and pinned on both paths: matching more
words is not a way past it.
Verified against the 235-article dev library, where all three of these
returned nothing before: 'Fieldglass Jefferson' (title word plus topic word),
'outage notification' (two keywords, not adjacent), 'compucom Jefferson'.
The same phrase-only pattern is repeated in about ten other list endpoints
(computers, machines, printers, network, printedparts, measuringtools, usb,
notifications). Left alone here - fixing them properly means promoting the
word-match helper onto the shopdb.api contract surface rather than copying it
per plugin, which is a contract bump.
|
||
|
cproudlock
|
d60ed602a1 |
Stop three ways the collector and the forms wrote things nobody asked for
A review of last week's device-identity work found these; two were writing bad data and one was reproduced against a live server before being fixed. AN UPDATE COULD BLANK AN ASSET NUMBER, on all six asset update paths. Create validates it and the column is NOT NULL, but the conflict check only runs when the value DIFFERS, and '' collides with nothing - so an empty assetnumber went straight through to a required column. This is the likely source of the assets found with no number: a form that loaded blank and was then saved. MACHINEFORM COULD LOAD BLANK AND LET YOU SAVE IT. One try/catch wrapped eight reference loads AND the machine fetch, so a single transient failure among them - one page of listAll() timing out during a collector cycle is enough - rejected the whole block and rendered a fully editable EDIT form with every field empty, the error banner far below next to Save. Typing an asset number and saving then wrote the blanks over a real machine. The record now loads in its own try, and a failure shows the reason INSTEAD of the form: an empty edit form is indistinguishable from a record whose fields are genuinely empty. NAMING A DEVICE THAT DID NOT RESOLVE STILL MINTED A TWIN. Both device paths warned "not linked" and then fell through to mint <HOST>-PARTMARKER or <HOST>-CMM - the hostname-derived twin the resolution order exists to prevent. The warning was true about the typo'd number and false about the twin. Naming a device is a commitment: if the name does not resolve, or resolves to the wrong kind of thing, link nothing and say so. Silence still means "work it out", so a bay with no file keeps the reuse-then-mint behaviour it always had. TWO PCS COULD BOTH HOLD ONE DEVICE, ACTIVELY, WITH NO WARNING. Verified against a live server: report as one host, then as another naming the same marker, and both controls rows stayed active. Neither device path had ever looked at who else held the target - only at links whose source was THIS PC - so a replaced PC kept its link forever and an asset-id.txt copied to a second bay claimed the device silently. It now reuses the machine link's rule rather than inventing a second one: an incumbent that has gone quiet past the claim window or been moved off In Use has yielded and is archived, never deleted; a live incumbent keeps the device and the challenger is recorded dormant. The swap test asserted the old behaviour and now asserts the new one, split in two: a live incumbent keeps it, and handover completes once the incumbent yields. Two other tests were passing while their names lied - the unknown-device one checked only that the typo'd asset was not created, not that nothing was linked, and it passed while a twin was minted beside it. |
||
|
cproudlock
|
adb30c8875 |
Release 0.12.0
Printers become a property of the bay, and the container stops throwing away the site's own files every time it is updated. Printer assignment is now end to end. The assignment belongs to the MACHINE and reaches whichever PC controls it, so a reimaged or swapped bay comes back with its printers and nothing had to be saved off the old box. One picker serves both forms rather than two implementations of the same override. A relationship type can declare itself singular, so setting a second default REPLACES the first instead of silently losing to the older row. Alongside the assigned half there is now an observed half: a bay reports what it actually has, kept strictly apart from what it is told to have, because a drifted bay's own state becoming its desired state would make every configuration error permanent. The client script corrects a drifted queue in place rather than only installing a missing one, and a driver rollout can be spread across waves so 300 bays do not pull 30 GB through one five-minute window. DOCKER SITES SHOULD READ THIS PARAGRAPH. The compose stack never persisted the instance directory, so `docker compose build api && up -d api` - the update path the docs themselves gave - discarded plugins.json and every upload with it. It is a volume now, and DEPLOY.md carries the one-time rescue for a stack that predates it. The image also could not be built at all: the frontend stage never copied the plugin staging script its own prebuild hook runs, so every build since that script landed failed. Air-gapped sites need a fresh offline bundle, because the stack moves to MySQL 8.4 LTS and an existing tarball carries only the 8.0 image. The Windows installer already bundled MySQL 8.4, so this closes a gap between the two halves rather than moving anyone. Adds a topology migration guide: IIS to Docker, Docker to a new host, and back. The data moves cleanly; what costs time is that the server address is baked into GE-Enforce manifests, the generated collector script, the printer client scripts and printed QR codes. 1736 tests green. Contract stays at 0.20.0 - nothing under shopdb/api changed. |
||
|
cproudlock
|
2d09fa3201 |
Collect what bays actually have, separately from what they are told to have
ShopDB knew what a bay SHOULD have and nothing about what it DOES. Adding the observed half makes a rollout a review instead of a typing exercise: the floor reports itself in, you look, and you adopt. The collection uses the mechanism that already exists rather than a new one. POST /api/collector/printers dispatches to the printers plugin's apply_collector_payload, the same ADR-006 hook the computers and backups plugins implement. New client script, new plugin-owned table, no new transport and no new credential. OBSERVED AND ASSIGNED STAY APART, and that is the point rather than a detail. A collector report can never write an assignment row: _reconcile_edges is the only function that writes usesprinter/defaultprinter, it has two call sites, and both are authenticated routes a human calls. If a drifted bay's own state were allowed to become what it is told to install, every configuration error would become permanent the next time that PC checked in. Seeding an assignment from observed state is explicit - POST /assignments/seed-from-observed - because a rollout adopts many machines at once. It routes through the same _reconcile_edges as the editor, so there is one write path with two doors, and a queue matching no known printer is REFUSED rather than guessed into an assignment. That last rule is the lesson from the measuring tools: adopting on a weak key produced 43 duplicate instruments. Two fixes on top of what the agents built. The replace deleted a host's previous rows by exact case-folded name while the read path treats a short name and its FQDN as one machine, so a PC that changed spelling appeared to hold every queue twice - which reads as drift that is not there. And the client sent 'reportedat' where the declared schema said 'observedat'. Also here: the legacy loader now imports machines.printerid, the classic system's record of each machine's default printer, which it silently dropped - the production import would have lost every one. And Set-ShopdbPrinters.ps1 finally registers the per-user logon task, staging Apply-ShopdbDefaultPrinter.ps1 to C:\ProgramData first because the share it lives on is mounted only during the enforcement cycle and the task runs at logon when it is gone. VALIDATED ON WINDOWS 11 (build 26200), not just on Linux pwsh, which parses these scripts happily and executes none of the spooler branches. The reporter: posts a correct payload with the X-API-Key header; resolves BaseUrl and CollectorKey from HKLM when given no arguments; suppresses the virtual queues by port; resolves port addresses; and reads the CONSOLE USER's default out of HKU rather than SYSTEM's own, which is a different and usually wrong answer. Two results matter more than the rest. With the spooler stopped, both the cmdlet and the CIM path fail and the script posts NOTHING - verified against a capture server that recorded zero requests, where an empty list would instead have erased that host's observed rows and read as a bay that lost its printers. A genuinely empty host still posts [], because that is a real and different fact. The logon task registers as the Users group at Limited, and falls back to the well-known SID S-1-5-32-545 when the group name will not resolve, as it will not on localised Windows. It was then run with the source directory RENAMED AWAY, to stand in for the share being unmounted, and it still moved the user's default - which is the whole reason the script is staged to C:\ProgramData rather than run from where it lives. The guarantees against damage were re-checked rather than assumed: an empty assignment changes nothing, an unreachable server changes nothing, -WhatIfOnly leaves no queue, no task, no staged file and no registry value behind, and a drifted queue is repointed IN PLACE with Set-Printer so whoever has it as their default keeps it. Not covered by any of this: the driver-staging path, which needs a real vendor package rather than the class drivers a VM ships with. |
||
|
cproudlock
|
e2c45d33bc |
One printer picker for machines and PCs, and one default per asset
The assignment belongs to the MACHINE, and until now there was no way to set it except the generic relationships card or the API - the form for the thing the feature is about did not exist. MachineForm now carries the picker, and PCForm uses the SAME component rather than its own copy: the PC's set overrides the machine's, and two implementations of that would drift, with the two ends of an override disagreeing being exactly the bug nobody would spot. The shared picker also fixes what PCForm did on save. It wrote row at a time through the generic relationship endpoints, which is a non-atomic reconcile: an HTTP failure part way left a PC half-assigned with nothing recording what was meant. It now calls the reconcile endpoint, which validates the default before writing anything. A relationship type can now say it allows one active row per asset (relationshiptypes.issingular, migration 7d34), and defaultprinter says it. Cardinality belongs to the type rather than the printers plugin: core's create path is where every hand-made link passes, and the next type meaning "exactly one" gets the rule for free. Setting a second default REPLACES the first instead of refusing, because "make this the default" means that - and a card answering 409 would leave the user hunting for the old row. Without it the schema was happy to hold two defaults: the unique constraint is (source, target, type), so two different targets are two valid rows, and the resolver takes the OLDEST - the new default silently lost. Proven by disabling the new rule and watching the tests fail. FOUND WHILE TESTING IN A BROWSER, and it was not mine: MachineForm read .data.data off computersApi.listAll(), which resolves to the ARRAY - fetchAllPages has already unwrapped every page. The whole parallel load threw into the catch, so every dropdown on the machine edit form came up empty and the machine's own values never loaded. A build cannot see this; only opening the page can. GET /api/printers/assignments/for-asset/<id> returns an asset's OWN assignment, without inheritance, because the editor must show what this asset's rows say - otherwise a machine's printers appear ticked on the PC that inherits them and unticking one silently creates an override. |
||
|
cproudlock
|
72b3904f71 |
Release 0.11.3
The last of the buildings-and-levels bugs, and the shop-floor clients brought into the product. Anyone on 0.11.0 through 0.11.2 should take this. Every hover mini-map in the product reported "this asset has a position but no level" - the levelid prop added in 0.11.0 was passed by none of its seven call sites - and the map PDF printed markers from every floor onto one sheet, which nobody can correct once it is carried onto the floor. The legacy import loader, still to run against production, created markers with no level at all. The gate that should have caught all three asked whether a FILE mentions levelid rather than whether each position does. It now checks per occurrence. Also: printers can be assigned to a MACHINE and reach whichever PC controls it, so a reimaged bay reinstalls its own printers with nothing saved off the old PC; printerdrivers can name a vendor, so two rows cover 41 of 44 printers instead of twenty-one near-duplicates; and the collector reporter and EventSaver now live in the repository with no site baked into either, the reporter generated per site on request. The client scripts were validated on Windows 11 against a live ShopDB, not only by the suite: a bay with no rows of its own created both queues from its machine's assignment, bound them to the right universal drivers, and set the per-user default. The version and the changelog are the release; the detail is in the entry. |
||
|
cproudlock
|
0dc0ac13c8 |
Assign printers to a machine, and let the PC that drives it inherit them
Printers belong to the bay, not to the box currently driving it. The assignment
goes on the MACHINE asset and reaches whichever PC controls it, so a reimaged or
swapped PC comes back with the right printers and nothing had to be saved off the
old one. The asset register is the backup.
New relationship type usesprinter ("this printer is installed here"), beside the
existing defaultprinter ("which of them is the default"), both seeded and both
given a propagation rail through controls. The rails are consumed at READ time
only: the create-time fan-out skips directional through-types, and controls is
directional, so assigning a printer to a machine does not copy rows onto its PC.
That is what keeps own-beats-inherited possible.
Resolution for a PC is its OWN rows if it has any, otherwise one hop out along
controls to the machines it drives. Whole set at a time, not merged: a PC with
its own assignment is overriding the bay deliberately, and the UI has to say so
or a tech "fixing" a bay by editing the PC will shadow the machine's record and
wonder why they keep disagreeing.
GET /api/printers/for-host/<hostname> is what the convergence client asks every
cycle. Resolved by hostname because the collector upserts PCs by hostname and an
office PC has no machine number. An unknown host, a site without the computers
plugin, and nothing assigned all return an empty set - that is the client's
designed no-op and it must stay indistinguishable from "assigned nothing".
PUT /api/printers/assignments/for-asset/<id> reconciles the whole set in one
call. The endpoint was specified, documented and asserted by three tests, and
never written - the verification pass caught that, with four failures. It
validates the default BEFORE any write, so a rejected request changes nothing;
soft-deletes rows that went away; and REACTIVATES soft-deleted rows rather than
inserting, because the unique constraint spans inactive rows and a blind insert
after an unassign raises IntegrityError on MySQL while passing on SQLite.
One default per asset, enforced here because the schema cannot: the constraint is
(source, target, type), which accepts two different defaults quite happily. Two
active defaults are still reachable through the generic relationships endpoint,
where the oldest silently wins - recorded in the proposal as the next thing to
close.
printerdrivers gains drivername: the exact string the INF declares, which
Add-PrinterDriver matches on and nothing else. Deriving it by parsing INFs on
hundreds of bays is fragile; a human confirming it once is not.
|
||
|
cproudlock
|
2083029ff2 |
Generate the collector script per site, and bring EventSaver into the repo
A site adopting ShopDB had to be handed two files and told what to edit in them. Both are now the product's, and one of them the server writes for you. GET /api/computers/client-script (admin) returns Report-AssetToShopDB.ps1 with this site's values already in it: site_base_url becomes the -ApiUrl default and the new computers_routableranges setting becomes -AllowedRanges. Only the PARAMETER DEFAULTS are substituted - the copy in plugins/computers/client/ stays runnable, so there is no second version to drift from the first - and everything stamped stays overridable by argument or registry, because a bay may need to differ from its site. Settings > Computers > Asset reporter edits the ranges, downloads the script and shows its SHA-256. The collector key is deliberately not stamped in, and a test fails if it ever is. That file lands on every shop-floor PC, and a token spread across hundreds of bays cannot be rotated quietly; it stays in the registry, provisioned per ADOPTING-AT-ANOTHER-SITE.md. The routable ranges are the last thing that was hardcoded in that script. They are now a setting, so West Jefferson's two CIDRs move out of source code and into that site's own configuration - which is what ADR-015 asks for - and a site that sets nothing still works, because the script falls back to the NIC carrying the default route. EventSaver joins it in plugins/slides/client/, source only: EventSaver.cs and EventSaver.ini, no compiled .scr - a binary is a release asset, like the installer exe. The share path that was compiled into Config.Folder is gone. It used to be the fallback when the ini was missing, which silently pointed a new site at the reference site's file server; it is now empty, and failing visibly beats displaying another site's slides. Verified by compiling the edited source in the Windows VM with the in-box csc.exe: 15,872 bytes, exit 0. Also: the DSC example in the adoption guide gains a CollectorRanges resource and stops passing -ApiUrl to a script that already reads BaseUrl from the registry the same example writes, and the guide points at the generated download instead of hand-editing a URL. The contract test caught the endpoint importing shopdb directly for the version string, which ADR-002 forbids a plugin from doing. The product and contract versions are in app.config now, which a plugin reads through current_app. Adds docs/proposals/printer-assignment.md: assign printers to a PC in ShopDB and let the bay install them, with what the fleet data says about drivers - HP and Xerox cover 41 of 44 printers with universal drivers, there are no Brother printers at all despite 208 files of Brother inkjet drivers in the installer, and printerdrivers holds one row pointing at a per-model folder instead of a universal driver. |
||
|
cproudlock
|
8bde89c47e |
Release 0.11.2
Bug fixes for the buildings-and-levels work in 0.11.0, every one of them found by using the feature rather than by the suite. Anyone running 0.11.0 or 0.11.1 with more than one level should take this release: the map drew markers from every level on whichever floor plan happened to be showing, which is the failure ADR-017 exists to prevent, and it was in the one component that draws the map. The viewer also had no way to choose a level, and the editor never accepted a click at all - its handler was bound only if the map was already a picker at mount, and the editor opens with nothing selected. Also: a level's native size could not be set (the settings page had no height field), the same search could return different rows because fourteen searchers truncated without an ORDER BY, and both map pages carried a scrollbar from subtracting the wrong page chrome from the viewport height. No schema change, and the plugin contract stays at 0.20.0. The version and the changelog are the release; the detail is in the entry. |
||
|
cproudlock
|
aa6db94179 |
Fix the levels viewer: markers were drawn on whichever plan was showing
Five defects, four of them mine from 0.11.0, found by using the feature. THE SERIOUS ONE: ShopFloorMap never checked a marker's level. It skipped null coordinates and drew everything else on whatever blueprint was displayed, so level 1 markers appeared on level 2 - the exact failure ADR-017 exists to prevent, in the one component that draws the map. The build gate did not catch it because that rule checks files EMITTING mapx, not the component consuming it. Markers are now filtered to the drawn level, and a position with no level is omitted rather than approximated. The map page had no way to choose a level at all. It read currentlevelid only to title the PDF, so a second floor was unreachable from the viewer that most people use. Adds a level selector (hidden when a site has one level), passes it to the map, and switches the drawing, the bounds, the coordinate space and the markers together - swapping the image without the bounds would place every marker against the wrong scale. Searching the map now follows results across levels: a search whose matches are all on another floor showed an empty map while the filter counted them. Floor map settings had NO height input - only width - so a level's native size could not be set even while empty, which is the one time it is editable. Both fields are there now, and size is editable on a level that has markers, because refusing it blocked the case the feature was built for: a new blueprint of new dimensions on a floor already full of markers. It confirms first and points at landmark recalibration. Search results differed between the sidebar box and the results-page box: - 14 of 16 searchers truncated with .limit() and no ORDER BY, so the database could return a DIFFERENT subset of matching rows for the same query. Every searcher now ends in a total order (display key plus primary key). - Searching a term already in the URL was a duplicate navigation the router aborts, so the route watcher never fired and the button did nothing. The sidebar never hit this, because it always navigates from another page - which is why the two boxes appeared to disagree. Also removes a scrollbar from both map pages. They subtracted 2rem and 40px from 100vh for the page chrome, which is really 90px of padding on .main-content, so each overflowed by the difference. The padding is now a CSS variable both the layout and the pages read. Measured in the browser before and after: 1058 vs a 1000px viewport, now 1000. |
||
|
cproudlock
|
dd503be4ba |
Release 0.11.1
A patch release. The pagination-cap fixes, and the documentation that missed the 0.11.0 installer by a few minutes. docs/UPGRADE.md is one of the documents the Windows installer puts on the server, and its buildings-and-levels section was committed after 0.11.0 was tagged. So 0.11.0 shipped the page an operator reads WHILE upgrading without the guidance about that upgrade - which matters most on an air-gapped box, where there is no other way to reach it. The installer now also ships FLOOR-MAP.md for the same reason the Windows runbooks ship. No application code changed on the server side and the plugin contract stays at 0.20.0. The version and the changelog are the release; the detail is in the entry. |
||
|
cproudlock
|
24647aa8e8 |
Release 0.11.0
The floor map became a set of drawings instead of one picture, and search now reaches what the forms have been collecting all along. A site can hold more than one building and a building more than one level, and every map position records which drawing its coordinates belong to. An existing single-map site upgrades into one building and one default level carrying the blueprint it already had, and nothing moves on screen. Landmark recalibration moves a level's markers onto a redrawn plan, solved per axis from the landmarks and never from the image dimensions, dry run by default, snapshotted before it applies. Two identifiers the UI collects were findable in almost no way and two plugins' records in no way at all: gaugelabreference matched only measuring tools, maintenancereference matched nothing anywhere, and USB devices and printed items were absent from search entirely because neither is an asset. All four are searchable now, from the global bar and from every list's own search box. Contract 0.20.0 is additive: a plugin that writes a map position must write its level with it. The version and the changelog are the release; the detail is in the entry. |
||
|
cproudlock
|
3324dbd91e |
Buildings and levels for the floor map, and make every identifier searchable
The map was one picture of one floor. A second floor was added, the blueprint changed size, and machines moved, so a position now records WHICH DRAWING its coordinates belong to. Buildings and levels (ADR-017). Each level owns its blueprint per theme and its own native pixel size; assets.mapx/mapy are pixels of assets.levelid, not of the site. A position whose level is unknown renders "level unknown" and is never drawn on the default level, because a marker on the wrong floor plan looks entirely correct while pointing at the wrong place. Repositioning in bulk: filter by unplaced, needs-review or level, search, place, confirm. Landmark recalibration solves the transform PER AXIS from landmark pairs and never from image dimensions - the canvas grew taller without rescaling, so a dimension-derived scale would stretch Y by 1.57 and be wrong everywhere. It defaults to a dry run, reports what would land off the drawing, snapshots before applying, and clears mapverifiedat because a transform is a guess awaiting review. Snapshots restore, including the level and the review state, and a restore snapshots first so an undo is undoable. Search: gaugelabreference was matched only for measuring tools and maintenancereference was matched nowhere at all, for any asset type, while Settings happily offers both identifiers on machines and PCs. A tag an operator is told to record has to be findable or it is a write-only field. USB devices and printed items were unreachable from search entirely - neither is an asset, so the generic asset search could not see them and no searcher existed; they now match on serial, asset tag, label, bin code and gage-lab tag, honouring isactive, with Settings toggles and result labels to match. The retired-application rule was half a rule: GET /api/knowledgebase hid articles whose topic application is retired while global search still returned them and printed the retired application as the subject. A filter is only real if every path that reaches the row applies it. Contract to 0.20.0 (additive): Asset gained levelid and mapverifiedat, Location gained levelid, and resolve_asset_position returns the levelid belonging to whichever source supplied the coordinates. The five plugins that write a map position are re-pinned. The install-list text format gained levelid as a NINTH field, appended, because the shipped Pascal installer reads fields 0-7 by index. That installer still compiles in one drawing's dimensions and bundles one blueprint, so its map is accurate for the default level only; /api/maplevels is deliberately unauthenticated so it can read both at runtime once rebuilt. Recorded in PRINTER-INSTALLER.md section 6 along with the other known gaps. Migration 7d33 converts an existing single-map site into one building and one default level carrying the old map_* settings, then assigns every placed asset and location to it. Nothing moves on screen. Old settings rows are kept so a rollback still finds them. Verified end to end on MySQL 5.6 from a production-shaped database. |
||
|
cproudlock
|
7d9a54ca0f |
Release 0.10.0
A privilege-escalation fix in the Windows installer, the multi-site portability work, and a documentation and API surface that a second site can rely on. The installer created C:\shopdb-flask, C:\Python314 and C:\MySQL84 without breaking inheritance from the drive root, whose default DACL grants Authenticated Users an inherit-only Modify. Two files inside those roots are executed by someone else - shopdb-admin.ps1 self-elevates, and instance config.py is loaded unconditionally by the app - so any logged-on user could replace either and wait. Verified fixed as SYSTEM in the Windows 11 VM against the shipped functions. Contract 0.19.0 is breaking: get_dashboard_widgets returns data and shape, not a component name. The change itself shipped earlier without a bump, with the contract docs still teaching the dead shape; both are corrected. The version and the changelog are the release; the detail is in the entry. |
||
|
cproudlock
|
035419fa51 |
ADR-015: stop shipping one site's values, and make the rule a gate
The scanner has been reporting the same count for weeks, which is what a rule that only prints becomes. It now FAILS the build, and it looks where the leaks actually were: PowerShell, the installer, the seeds, generated JSON, the frontend - case-insensitively, across plugins, shopdb, scripts, deploy, tools. A line that is deliberate declares itself with an ADR-015-OK marker and a reason, so the claim is visible in review instead of tolerated in silence. What it found, fixed here: - The shadow client wrote one site's ShopDB URL into HKLM whenever the registry disagreed. At the site it was written for that reads as healing drift; anywhere else it overwrites the site's own address on every enforce cycle, and the site cannot win because the cycle repeats. The bay's value now wins, an explicit -BaseUrl seeds it, and with neither there is nothing honest to write, so it says so and skips. - The kiosk dispatcher fell back to one plant's host when HKLM was unset, so a kiosk elsewhere quietly opened a server it has no business reaching. The fallback is now this site's site_base_url, baked in at seed time, and the dispatcher refuses rather than guessing when neither is set. Its legacy shortcut matcher derives the host from that URL instead of naming one. - The OpenAPI generator hardcoded a production hostname into every spec it generated, which then published to a public wiki. The relative mount is the only server it can honestly name; a site passes its own by environment. - Placeholders and examples in the UI and the client help offered real internal subnets and a real production URL. They now use documentation ranges. Both publication gates - the export scrub and the docs publishability test - carry the site patterns, which neither did. One plant's hostname, FQDN and internal networks are out of the documentation and the generated specs. Comments naming the reference site are reworded rather than deleted: the reasoning is worth keeping, the plant name is not what makes it true. |
||
|
cproudlock
|
2df5028883 |
relationships: the cleanup tools stop acting on links that were deleted
Deleting a relationship is soft, so the row survives with isactive False, and three things read them without knowing that. Re-adding a deleted link answered 409 "this relationship already exists" about a link the page no longer shows, and there was no way forward from the UI at all - the row cannot simply be inserted again, since the triple is unique. Reactivating IS the create for an inactive row. The inverse guard blocked on a deleted inverse, which made "remove the existing one first" - the instruction in its own message - fail to unblock anything. fix-controls-direction retired the reversed row whenever a correctly-directed one existed, without checking whether that one was itself deleted. So it removed the only live link and reported a successful clean-up. It now reactivates the row pointing the right way before retiring the one pointing the wrong way. These are the commands the docs tell an operator to run against production. |
||
|
cproudlock
|
c7dffce81e |
Serve an uploaded file as data, not as a document that can run
An SVG is an XML document that may carry a script, and it is an accepted image type because floor-plan maps and branding genuinely want vector. Loaded through an img tag that script never runs, so the tiles and maps were never the risk. Opening the file's own URL is - and the application image route is public, so that URL needs no session. Every route that serves an upload now goes through one helper that sends Content-Security-Policy: default-src 'none'; sandbox, and nosniff. Seven routes across core and five plugins, so a new one added later starts from the same place rather than repeating the reasoning. Banning the format instead would have cost the maps their only sensible one. The app also sent no security headers at all. It now sets nosniff, frame-ancestors self (as X-Frame-Options too, for the display bays' browsers) and a referrer policy. Deliberately NOT a page-wide CSP: this serves an SPA with inline styles, so a real script-src policy is a change worth making with the frontend in front of you, and a permissive header claiming one would be worse than having none. Contract 0.19.0. send_upload is on the shopdb.api surface, because a plugin serving user-supplied bytes should not have to remember these headers. The same bump records that get_dashboard_widgets has taken data and shape rather than a component name since the dashboard was rebuilt - that shipped without a bump, while BasePlugin and PLUGIN-HOOKS.md both still documented the shape nothing renders, which is how five plugins came to declare widgets pointing at components nobody had written. |
||
|
cproudlock
|
38deefe619 |
migrations: commit the plugin chains too, and bound what re-ran
Core's Alembic env got connection.commit() when the stamp bug was found; the per-plugin template did not. MySQL commits DDL implicitly, which flushes everything queued before it including the previous migration's version stamp, and the LAST migration of a run has no DDL after it - so its stamp rolled back at close while its schema change survived. flask plugin upgrade-all then exited 0 having silently re-run that migration, and re-ran it again on every deploy after. Invisible for exactly as long as every plugin head happened to be idempotent. Two were not. backups 0003 cleared lastseenat for EVERY row, which is correct once and destroys evidence on each repeat. It is now scoped to the backfill's actual signature, COALESCE(collectedat, createdat) - the expression 0002 wrote - plus a date bound. Both conditions are needed. Matching on collectedat alone misses every row whose collectedat is NULL, so precisely the rows carrying the most invented value would have kept it forever; and value equality is not a signature on MySQL, where db.DateTime is second-precision and the collector writes both stamps in one statement, so a genuinely fresh revision would read as a backfill and be wiped. SQLite keeps microseconds, which is why no test could show it. geenforce 0003 added a column unconditionally, so it failed on a fresh database built from the models and on any re-run. Guarded like network0003prefix. |
||
|
cproudlock
|
e67fe47fe2 |
relationships: refuse links that cannot both be true, and report the ones already stored
Prod grew rows saying both "PC controls 2005" and "2005 controls PC", and a CMM PC showing "<- controls from CMM4" beside its own outgoing link. Only one direction can be true: a PC drives a machine, never the reverse. Nothing stopped it. The duplicate check was keyed on (source, target, type), so the inverse inserted cleanly, and the Add Relationship dialog offers an incoming direction that writes exactly that. The legacy import stores controls the wrong way round as well. Directional creates now refuse the reverse with a 409 naming the row that already holds it, and refuse self-links, which render as a duplicate on the asset's own page and mean nothing. Symmetric types are exempt: Dualpath stores both directions on purpose and the card collapses them. The propagation fan-out got the same guard so a rail meant to spread one direction across sibling bays cannot manufacture a pair. fix-controls-direction only matched source assettype 'machine', so every measuring_tool, printer and network_device row it was written to clean survived it - which is why running it would never have fixed the CMM. It now matches any non-computer controlled BY a computer. New `flask relationships audit` reports what is already stored: reciprocal pairs, self-links, and PCs controlling several assets of one type. Read-only, and it prints each row's label because that usually names the writer outright - collector:* means this code made it, anything else means a person or the import did. That distinction decides the fix for duplicate device assets, which is not in this commit: the collector keys idempotency on its own label, so a device somebody created by hand is invisible to it and it mints another, and the adoption rule needs the audit run against prod before it can be written. Two false positives were found writing it, against the dev database, and both would have made the report useless. A self-link is its own inverse, so it was counted as a reciprocal pair AND printed twice. And Dualpath siblings looked like duplicate devices - a dual-bay machine is one physical machine with one controller and controls is propagated to both bays deliberately. That was 30 of 32 findings, consecutive bay numbers pair by pair. |
||
|
cproudlock
|
20a95013ad |
contract 0.18.0: one name per display role, the kiosk's own
Core called the roles dashboard / lobby / partskiosk. The kiosks call them Dashboard / Lobby / 3DPrintRoom, which are the literal contents of C:\Enrollment\display-type.txt, read by the GE-Enforce dispatcher to pick a target. Two vocabularies for three kiosks, each with its own copy of the same route map. That is not cosmetic. A display reporting its own type sends what its file says, so it could report a role core would not accept, and core could store 'partskiosk', a value no dispatcher would ever match. The enforcement report column would have shown one vocabulary from the device and the other from the DashboardDefault fallback, in the same column. The machine's file wins, because that is what a person edits. DISPLAY_ROLE_PATHS takes the kiosk spelling and the display scope now uses that dict rather than holding a second one, so the two cannot drift again. normalize_display_role resolves any casing and the retired 'partskiosk' forward; the dispatcher already matched its map case-insensitively and the server now agrees with it. Nothing is turned away over a capital: the API accepts any spelling and stores the canonical one, displaypath resolves through the normalizer so rows written before this keep working, and the settings dropdown canonicalises on open so an old value does not render as a blank select. A reported subtype is normalised on the way in, but an UNRECOGNISED one is kept verbatim. That is a kiosk with a typo in its file or a role nobody declared, and both are worth seeing in the fleet table rather than blanked or guessed at. Contract bumped for the added names. DashboardDefault is finally listed in __all__ too - 0.17.0 put it on the surface and never exported it. |
||
|
cproudlock
|
52eb10f5ca |
contract 0.17.0: expose DashboardDefault to plugins
The enforcement reports needed to name what a display IS, and reached straight into shopdb.core.models.dashboarddefault to do it. Plugins may only touch core through shopdb.api, and the contract test said so. The role belongs on the surface rather than behind it: it lives in core, no plugin owns it, and a plugin reporting on displays has no other way to resolve it. Added there and the version bumped, which the docs test pins. |
||
|
cproudlock
|
c28b02e45b |
Upload an application's image and installer instead of typing paths
Adding an application meant typing an image FILENAME and trusting someone had dropped the file into the frontend's own directory by hand, and typing an install path from memory. Both are uploads now, following the model-image trio that models and part photos already use. The two differ deliberately. The image is public, because application tiles render before anything is authenticated. The installer is not: it is licensed vendor software, an open URL would publish it to anything that can reach the site, and it is always sent as an attachment rather than rendered. Installers are capped at 500MB and the size is measured by seeking the stream rather than trusting Content-Length, which a chunked upload does not send and a client can understate. Anything larger belongs on the share, and the error says so rather than just refusing. Files are chosen before a new application exists, so they are held and uploaded once there is an id to attach them to. A failed upload leaves the saved record alone and reports, rather than losing what saved fine. Removing an installer only clears installpath when it pointed at the upload - a share path was typed by a person and is not ours to wipe. The detail page reads both shapes, since entries from the classic site hold a bare filename that is still served from /images/applications/. |
||
|
cproudlock
|
94d8d6c9b6 |
dashboard: numbers that agree, a map on hover, wider cards
"All assets 704" sat beside "all assets in use 737", and both were correct about different populations. The totals summed five specific asset types and subtracted dual-bay secondaries; the status counts took every asset row of any type with no collapse, so USB devices and hidden secondary bays inflated one side of a comparison the layout invites. Status is now counted over exactly the same assets the totals describe. Warranty rows fell back to asset.name when the covered asset had no hostname, and an asset's name is usually the MACHINE's descriptive name - which is how a column meant to identify a PC ended up showing a machine. Hostname, else the asset number, never the name. The machine number loses its label too: the row is hostname, machine, state, and "machine 3015" spends a word on what position already conveys. Printer names now carry the floor-plan preview on hover, the same LocationMapTooltip the printer's own page uses - a location name tells you the room, the map tells you where to walk. Declared as map.maphover on the card, so any card with coordinates gets it; a row without them shows a plain link rather than being dropped. Cards are four across rather than five. At five columns a row holding a hostname, a machine number and a state truncates on exactly the rows that matter. auto-fit, so two cards fill the width instead of leaving empty tracks. Not covered by a test: the count fix. I started one and it was interrupted, and I have not gone back for it - the assertion worth having is that in-use can never exceed the total. |
||
|
cproudlock
|
105345fb3d |
Release 0.9.0
Product version to 0.9.0, frontend in lock-step, Unreleased notes moved into a dated section per ADR-007. The plugin contract stays at 0.16.0: it moved after v0.8.1 and is already recorded in this release's notes, and nothing since touched the contract surface. A minor rather than a patch: collector behaviour changed in ways an integrator must know about. A reported machine number no longer becomes the PC's asset number, it builds a controls link instead; a second PC claiming a machine is treated as a claim rather than a handover; and a backup revision chain is now per source PC rather than per asset. |
||
|
cproudlock
|
c90ebcbc7c |
computers: declare subordinate devices instead of coding each one
A PC that drives a device which is its own asset had been implemented twice. METROLOGY_TOOL_MAP covered CMM, Keyence, Genspect and wax-trace, minting a measuring_tool. A separate path keyed on one hardcoded pc-type minted a Part Marker machine and filed it under its operation. Both create a device, link the PC with controls, and archive that link when the PC is re-imaged: one mechanism with different nouns, written out twice because the second case arrived later. That is the same trap as the site literals in ADR-015 - a pattern implemented per instance rather than declared - and it has a known next occurrence. Part markers already share operation numbers, and any site with two marking lasers or two wax-trace units on one number needs identical treatment. One SUBORDINATE_DEVICE_MAP now declares asset type, type name, naming suffix, whether the device files partof the operation, and the relationship label. The labels are unchanged per case on purpose: those values are in the production database and only rows carrying them are archived by a collector push. A site overrides or adds an entry through subordinatedevice_<pctype> settings, per ADR-015, so the next case needs no code. A malformed override falls back to the default rather than failing the push, because a bad setting must not stop a bay reporting its inventory. metrology_tool_for stays as a shim over the same map: filters.py and the older tests read it, and unifying must not change what it returns. A test pins that. Also adds flask relationships check-shared-machines, which finds the next 0615 rather than waiting for someone to notice duplicate backups. Several devices legitimately sharing a number and two PCs mis-numbered at imaging look the same from outside; the difference is whether child assets exist, so that is what it reports. Read-only. |
||
|
cproudlock
|
fca775c737 |
backups plugin: per-asset config backups with revision history
Adds a kind-pluggable backups plugin. Configuration captured from a PC is filed against the MACHINE it controls, with a revision history and download back to the native format. NTLARS/DNC is the first kind. Settings live in the controlling PC's registry but describe the machine, so revisions attach to the machine's asset and carry no foreign key to the PC: history survives a PC being replaced or deleted, and sourcehostname records the handover. Storage splits by kind. Parseable kinds store a dialect-neutral JSON projection in ShopDB and re-render on download; opaque vendor formats (part marker and similar) keep their bytes on the SFLD share with ShopDB holding metadata and the UNC pointer. Two .reg dialects exist in the wild: NTLARS's own Save... export omits the WOW6432Node path segment, scripted exports include it. Parsing strips whichever root matched, so a stored revision commits to neither and download offers both (NTLARS Load... by default, WOW6432Node for direct reg import). Getting this backwards is silent, so the dedup hash deliberately excludes sourcedialect and both dialects of one config dedup to a single revision. Dedup is load-bearing: the collector runs every GE-Enforce cycle across the fleet, so a revision is inserted only when the content hash differs from that asset's latest for that kind. A freshly imaged PC opens NTLARS with a blank General tab. Recording that would make an empty config the newest revision exactly when someone needs the last good one, so a blank MachineNo is rejected rather than accepted as a change. Two of the 320 known-good backups on the share already have that shape. DNC Info card summarises the latest revision on the machine page: General (Cnc, NcIF, HostType), eFocas, Serial, NTSHR when populated (only 18 of 147 machines), and MARK when the machine is a marker. MARK is gated on Cnc=MARKER or the ShopDB machine type, not on the MARK key having content: MARK carries serial defaults on 145 of 147 machines and DncPatterns reads YES on 103 including ordinary lathes, so neither identifies a marker. The info card is owned by the kind (BackupKind.infopanel/buildinfo) and served by a generic endpoint, so the expected successor to DNC ships its own card by adding a class rather than changing the plugin or the panel wiring. Also: schedule and retention settings with a prune that never drops the newest or the oldest revision, and scripts/import_ntlars_backups.py to seed history from the existing per-machine .reg files (144 of 147 resolve to assets). Codec verified against all 320 real backups: round-trips clean through both dialects. Bay-side generation verified on Windows against reg.exe export. |
||
|
cproudlock
|
593dd46525 |
Show the kiosk label prefix, and let a plugin declare the settings it owns
Three defects, all found on printedparts_label_prefix, all one root cause: nothing in the framework knew that setting existed. The parts kiosk runs logged out. An unauthenticated read of a setting is limited to an allowlist, the key was not on it, so the kiosk got a 404 and fell back to no prefix. An admin previewing the same page while logged in saw the prefix, which is why it looked like it worked. The same setting also looked like it would not save. The row did not exist on a site that installed the plugin before the setting was added, so the first save created it - under the placeholder category the settings API uses for keys it does not recognise, where the plugin's settings page, which lists by category, could no longer see it. The value was in the database the whole time. And the row was missing in the first place because seeding ran from on_install / on_enable, which fire only on a state transition. Neither runs again on an upgrade, so a setting added in a later plugin version never reached a site that installed an earlier one. The comment claiming enable ran every upgrade cycle was simply wrong. A plugin now declares the settings it owns in get_settings_defaults(): key, default, type, category, description, and whether a logged-out page may read it. The framework seeds declared keys at install, at enable, and on every flask plugin upgrade-all; files a first-time write under the declared category; re-homes any row left in the placeholder category, value untouched; and answers an anonymous read for keys marked public. Core carries no list of any plugin's keys. Contract 0.16.0 (additive optional hook). printedparts and printers move to the hook and floor their core_version at 0.16.0. The dev database had two rows in the misfiled state (printedparts_alert_email, employee_db_host); the first repairs itself on the next upgrade pass. |
||
|
cproudlock
|
959db2922b |
Release 0.8.1
The installer on the share was stamped 0.8.0 and contained sixteen commits that were not in the v0.8.0 tag. A build that misreports its own version is exactly what the version check in docs/RELEASING-WINDOWS.md exists to prevent, and it would have left two sites unable to say what they were running. Everything in 0.8.1 landed after v0.8.0 was tagged this morning, driven by two sites entering real data for the first time: the blank-code 500, the two-slide display that never rotated, modals discarding a part-filled form, filters returning an empty page, model photos that could not be saved, and the shared equipment catalog that lets a new site start with vendors, models and printer supply part numbers already present. CHANGELOG gains a 0.8.1 section, and the OpenAPI document follows __version__ rather than being restated. |
||
|
cproudlock
|
367bc56a6d |
Ship the equipment catalog so a new site does not start empty
`flask seed reference-data` wrote a dozen generic model types and no vendors or models at all, so adopting this platform began by retyping a catalog another site had already spent a year building. That is the largest single obstacle to standing a new facility up. scripts/export_catalog.py dumps the catalog from a live instance to shopdb/data/catalog.json, and `flask seed catalog` loads it. What travels: vendors 53, models 128, modelsupplies 146, modeltypes 35, machinetypes 21, computertypes 10, printertypes 9, networkdevicetypes 5, locationtypes 11, operatingsystems 14, measuringtooltypes 8, notificationtypes 3, accessprotocols 3 The 146 printer supplies are the most useful part after the models themselves: every toner, drum and maintenance kit with its part number, colour, capacity tier and page yield, already matched to the right model, instead of somebody reading them off spent cartridges. IDEMPOTENT and ADDITIVE. Records match on a natural key - a vendor by name, a model by vendor plus model number, a supply by model plus part number - so a second run adds nothing, and it never updates or deletes: a site that corrected a description or pointed a model at its own photo keeps its version. Catalog only. No assets, locations, employees, business units or anything with a serial number: nobody wants one plant's machines appearing at another. Vendor contact details are excluded too, since a rep's name and number belong to whoever holds that relationship. supportteams, printerdrivers and customfields are site-specific and deliberately absent. Models and supplies reference their vendor by NAME rather than id, because ids differ between databases and an id-keyed catalog would silently attach part numbers to the wrong printer. The installer offers it as a tick-box on a new "Starter data" page, defaulting to on, passing -SeedCatalog to stage 3. Offered rather than assumed: a site that machines nothing does not want 21 machine types cluttering its dropdowns. Verified by loading into an empty database and running twice: every group populated on the first pass, "Catalog already present, nothing to add" on the second. |
||
|
cproudlock
|
ead5bd8f58 |
Give the console a repair verb, and something real to check
A server whose migrations or seeds never finished does not fail politely. Most
pages answer 500 and settings endpoints answer 404 for keys that were never
created, which reads as a broken application rather than an unfinished install.
One site spent a morning being debugged that way.
`shopdb-admin.ps1 repair` runs what stage 3 of the installer runs: db upgrade,
plugin upgrade-all, and the three seeds. Every step is idempotent, so running it
on a healthy server changes nothing, and each step runs independently so one
failure does not silently skip the rest.
`check` now says so before anyone has to infer it:
THIS SERVER IS NOT FULLY PROVISIONED
- seed data is missing (permissions, settings or reference data)
Most pages will answer 500 until this is fixed. Run:
shopdb-admin.ps1 repair
That needs a real test to sit on, so `flask db-utils seed-state` reports each
seed group and exits non-zero when any is missing. Verified by emptying the
settings table inside a transaction: MISSING, exit 1, rollback clean. Without it
the console check would have looked reassuring while testing nothing - an older
build with no such command reports UNKNOWN rather than healthy, for the same
reason.
|
||
|
cproudlock
|
705dd771bd |
Store a blank optional unique field as NULL, and answer a duplicate with 409
A site reported "internal server error" adding a second business unit. It was reproducible: create one with a blank code, create another with a blank code, 500. A column that is unique and nullable accepts any number of NULLs - that is what makes "optional but unique" work - and exactly ONE empty string. The form sent '', so the first blank code saved and every one after it collided with it. The field showed no asterisk because it genuinely is optional; the database just behaved as though it were not. This is not specific to business units. A dozen columns across core and the plugins are unique and nullable - asset numbers, hostnames, item codes, subnet names, gage-lab tags - and each was one blank form away from the same 500. Fixing them an endpoint at a time would have left the next to be found by a user, so a before_flush listener normalises blank to NULL on any unique nullable text column. Listening on Session rather than on individual mappers covers plugin models imported later, and avoids mapper-event semantics that differ between SQLAlchemy versions. A genuine duplicate is now a 409 with a readable message rather than a bare 500 with a traceback in the log: reusing a code that is taken is the caller's mistake, not a server fault. Verified against the development database: three business units with blank codes all save, the blank stores as NULL, and a real duplicate code returns 409. |
||
|
cproudlock
|
89e880afc3 |
Release 0.8.0
The Windows installer has never shipped under a version: v0.7.0 was tagged before any of it existed, so every build handed out so far stamped a server with 0.7.0. Two servers running different builds were indistinguishable, and the installer logged each upgrade as "same version already installed" rather than recording what changed. This cuts the release that fixes that. 0.8.0 rather than a patch: the air-gapped installer is a new capability, and pre-1.0 semantic versioning puts that in the minor slot (ADR-007). CHANGELOG gains a 0.8.0 section covering the twelve defects a real Windows Server 2019 install surfaced, the move from inferring "is this a re-run of my install?" to recording it, and the operator documentation. deploy/site-profile-universal.json is now in the repository. Released builds were being produced from a profile in a temporary directory, so the next release could not have been reproduced once that file was cleaned up. docs/RELEASING-WINDOWS.md points at the committed profile and says why. scripts/gen_openapi.py reads __version__ out of shopdb/__init__.py instead of restating it. Its hardcoded copy had already drifted a release behind, which is the same mistake that once shipped an installer stamped with the wrong version. |