The lean-build job proved the delete-a-plugin guarantee with two plugins and four hand-picked marker components. It now builds a real twelve-plugin set, and the assertions follow a rule instead of a hunch: one component from EVERY omitted plugin must be absent, one from EVERY chosen plugin must be present, and the backend tree is checked both ways. An omitted set that grows without its check growing is how a leak ships. CHANGELOG had not been touched across the whole range, so the largest security-behaviour change in the product - per-PC endpoint credentials, the shared-key refusal for enrolled PCs, server pinning, agent 1.20.1 and tray 1.7.1 - had no release record. An [Unreleased] section now carries it, written from the commits rather than from memory. CLAUDE.md and the agent quick-reference stated that ADR-016 bans storing credentials in the registry. It does not, and this fleet depends on HKLM for its collector key and server pins: the next person following that text removes them and silently breaks reporting and pinning on every un-enrolled PC. Corrected, along with two other statements that were false on this branch - the database is MySQL, not PostgreSQL, and column names are lowercase concatenated, not snake_case, which is the opposite of what the gate enforces. The wiki drafts are updated against the code as it stands, each old finding given a verdict with a file and line rather than left as an assertion from a week ago, and a new page records the ADR conformance result - including that the ADR set itself is not on this branch, so nothing here enforces it except the naming and style script. They are also scrubbed of the things that must not reach a published wiki: the site name, the internal git server, developer home paths and the assistant tooling. They now pass the repository's own publishability patterns, which is what the scrub is for.
9.2 KiB
ShopDB Quick Reference
Fast lookup for architectural decisions and contracts.
Critical ADRs (in wiki)
- ADR-001: Asset as platform contract - Core asset model, BaseAsset, composition
- ADR-004: Deployment topology - Air-gapped sites, bundle workflow
- ADR-006: Collector contract - Endpoint agent protocol
- ADR-007: Product versioning - Semantic versioning, release process
- ADR-008: Plugin migration ownership - PLUGIN_TABLE_OWNERS, migration chains
- ADR-013: Plugin catalog - Lean builds, plugin manifest
- ADR-015: Site-specific configuration - Where site literals belong
- ADR-016: Credential delivery to the fleet - cmdkey credentials for bays, targeting and rotation. ACCEPTED but NOT built. It says nothing about device tokens or kiosk auth, and nothing forbids the registry: it names registry provisioning as a recommended first step
Import Surface (stable API)
Every name below is in shopdb/api/__init__.py __all__. That list is the
surface; a name that is not in it is not importable from shopdb.api, whatever
else in the tree defines it.
from shopdb.api import (
# Database
db,
# Core models
Asset, AssetType, AssetStatus, AssetRelationship,
Location, BusinessUnit, Model, OperatingSystem, Vendor,
User, Role, Setting,
# Auth / authz
require_permission, require_role, require_any_permission,
caller_holds, caller_is_role,
client_ip, ip_in_cidrs, request_ip_allowlisted,
# Responses and paging
success_response, error_response, ErrorCodes,
get_pagination_params, paginate_query, paginated_response,
# Utilities
audit_log, cache,
normalise_badge, badge_kind, BadgeShapeError,
parse_date, parse_date_fields, DateValueError,
intarg, intargs, QueryArgError,
resolve_asset_position,
)
NOT on this surface. This block used to name all six, so the first import a new agent copied raised ImportError:
BasePlugin,PluginMeta-from shopdb.plugins.base import ..., which is what everyplugins/*/plugin.pyin the tree does.Building,MapLevel-shopdb.core.models.maplevel. An asset's position normally comes back throughresolve_asset_position, which IS exported.Employee- no such model. The directory lives in a read-only external database reached throughemployee_connection, and the plugin's own row type isDirectoryEmployeeinplugins/employees/models/.badges,datevalues,queryargs- module names undershopdb/utils/, not exported. Import the functions listed above instead.
Plugin Structure
plugins/{name}/
|-- plugin.py # BasePlugin subclass, manifest
|-- models/ # SQLAlchemy models
| +-- {name}.py
|-- api/ # Flask routes
| +-- routes.py
|-- frontend/ # Vue components
| +-- views/
| |-- {Name}List.vue
| |-- {Name}Detail.vue
| +-- {Name}Form.vue
|-- migrations/ # Alembic migrations
| +-- versions/
+-- services/ # Business logic (optional)
Database Naming
- Tables:
machines,mappeddrives(lowercase concatenated, plural) - Columns:
machineid,lastzabbixsync,isactive(lowercase concatenated, singular). NOT snake_case - Indexes:
idx_machines_locationid. The only place an underscore is allowed - Enforced:
scripts/check-naming-and-style.sh, on model attributes AND on the column names a migration creates
Branch Model
wj-> prod (WJ shop floor)dev-> ops (monitoring/pilot)main-> public mirror (GitHub only)wilm-> upstream (benjamin.baker's changes)
Common Commands
# Run tests
pytest -v
# Check style
scripts/check-naming-and-style.sh
# Create migration
cd plugins/{name}
alembic revision -m "description"
# Dev server
docker compose up
# Bundle for dev PC
# (on work PC) ~/Documents/github/pull-shopdb-bundle.cmd
# Fetch wilm changes
# (on work PC) ~/Documents/github/fetch-wilm-branch.cmd
Test Patterns
def test_feature(client, db_session):
# Setup
item = Model(name="test")
db_session.add(item)
db_session.commit()
# Execute
resp = client.get('/api/plugin/endpoint')
# Assert
assert resp.status_code == 200
assert resp.json['name'] == 'test'
Site Data Rules (ADR-015)
[OK] Allowed:
shopdb/core/settings/buildings.pyshopdb/core/seed/*.sqlscripts/site_imports/*.py
[FAIL] Forbidden:
- Hardcoded in plugin code
- Embedded in migrations
- Configuration files
Migration Ownership (ADR-008)
Each plugin owns its chain. Coordinate cross-plugin FK changes via PLUGIN_TABLE_OWNERS:
PLUGIN_TABLE_OWNERS = {
'asset': 'core',
'employee': 'employees',
'pc': 'computers',
'endpoint': 'endpoints',
# ...
}
Kiosk Access
Not ADR-016 - that ADR is about cmdkey credentials for bays and is not built. A kiosk route admits a caller on EITHER ground, both gated on the kiosk's own enable flag:
This is plugins/usb/api/routes.py as it stands, not a paraphrase:
from shopdb.api import Setting, client_ip, ip_in_cidrs
KIOSK_NEVER_A_CLIENT = '127.0.0.0/8, ::1/128'
KIOSK_TRUSTED_PROXIES = 'usb_kiosk_trusted_proxies'
def _kiosk_client_ip():
return client_ip(request)
def _kiosk_address_is_infrastructure(address):
if ip_in_cidrs(address, KIOSK_NEVER_A_CLIENT):
return True
return ip_in_cidrs(address, Setting.get(KIOSK_TRUSTED_PROXIES))
def _kiosk_ip_allowlisted():
address = _kiosk_client_ip()
if _kiosk_address_is_infrastructure(address):
return False
return ip_in_cidrs(address, Setting.get('usb_kiosk_allowed_cidrs'))
def _kiosk_open():
if not _kiosk_enabled(): # the site has to turn it on first
return False
return _kiosk_token_presented() or _kiosk_ip_allowlisted()
THIS IS NOT request_ip_allowlisted(request, cidrs). That helper is exactly
ip_in_cidrs(client_ip(request_obj), rawcidrs) and nothing else
(shopdb/utils/ipallowlist.py), so it has no infrastructure refusal. The USB
kiosk refuses an address that belongs to the path rather than to a caller BEFORE
it consults the allowlist, and that refusal is the whole point: see the topology
below. Writing the helper here instead of the real shape is how the two got
confused.
- The allowlists are per kiosk (
usb_kiosk_allowed_cidrs,printedparts_kiosk_allowed_cidrs) and ship EMPTY, so a site that enters no ranges stays token-only. ip_in_cidrsfails closed on an empty setting, an unparseable caller address and a malformed entry. Never copy a CIDR parser into a plugin.- A refused caller gets 404, not 403: these routes do not advertise themselves.
- The two kiosks HAVE DIVERGED.
plugins/printedparts/api/routes.py:556still callsrequest_ip_allowlisted(request, Setting.get('printedparts_kiosk_allowed_cidrs'))with no infrastructure check, so a site that allowlists loopback to work around the topology below opens that kiosk to every caller. Read the plugin you are editing; do not assume the other one matches.
Why the infrastructure refusal exists (the shipped Windows path)
Trace it, do not guess. IIS hands the request to HttpPlatformHandler, which
connects to waitress on 127.0.0.1 with
--trusted-proxy=127.0.0.1 --trusted-proxy-headers=x-forwarded-for --trusted-proxy-count=1 (deploy/windows/web.config). waitress, seeing the
trusted peer, overwrites environ['REMOTE_ADDR'] from the LAST X-Forwarded-For
hop. client_ip() then returns strip_port(request.remote_addr)
(shopdb/utils/clientip.py) - it never reads the header itself.
The catch: IIS does not set X-Forwarded-For on its own, and the rewrite rule
that would is commented out in web.config - it SHIPS DISABLED because it needs
the URL Rewrite module. So on a default install there is no header, waitress has
nothing to promote, and every caller arrives as 127.0.0.1. TRUSTED_PROXY_COUNT
defaults to 0 (shopdb/config.py), so ProxyFix is not installed either, and it
would not help: waitress already did that job.
Consequences a kiosk change has to respect:
- Until someone enables the rule (
shopdb-install.ps1 -ClientIpSource directinstalls URL Rewrite, allowsHTTP_X_FORWARDED_FORat server level and uncomments it), a CIDR allowlist admits nobody and the kiosk is token-only. That is the fail-closed direction and it is correct. - Allowlisting 127.0.0.0/8 to "fix" that would admit everyone, which is why
KIOSK_NEVER_A_CLIENTrefuses it whatever the setting says. -ClientIpSource proxyleaves the rule off ON PURPOSE: behind ARR or a load balancer, REMOTE_ADDR is the proxy and the rule would destroy the real client address. There the upstream proxy sets the header.
Fleet Credentials
- Agent and GE-Enforce reporters run as SYSTEM, so there is nobody to prompt.
The collector key, base URL and server pins come from
HKLM\SOFTWARE\GE\ShopDB. That hive is the design, not a violation. - An enrolled PC's own per-PC credential is DPAPI-protected in a SYSTEM-only
directory and sent as
X-Endpoint-Credential; the shared fleet key still works for a PC that has not enrolled and is refused for one that has.
Wiki Location
Full docs: ~/Documents/github/shopdb-flask.wiki/
All ADRs, deployment guides, plugin development docs live there.