ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Make main the lean product a site receives

Browse Source 
main was 16 days and 259 commits behind the line the product is actually built
on, at contract 0.30.0 against 0.35.0, with no per-PC credential code at all. It
was neither current nor curated - just stale. This brings it forward and, at the
same time, makes it mean something: main is what a receiving site installs, and
nothing else.

WHAT IT CARRIES. Twelve plugins - backups, computers, endpoints, guides,
knowledgebase, machines, network, notifications, printers, slides, tools,
warranty - plus applications, which is manifest-less core and always ships. The
only manifest dependency inside that set is endpoints on computers, and both are
in it, so the closure is self-contained.

WHAT IT DOES NOT. employees, geenforce, measuringtools, printedparts and usb
stay on the site line. So do the Windows installer tree, the VM rehearsal
helpers, the agent tooling config, one site's data loader, the internal audit
and findings records, and the planner spreadsheet. 1,135 tracked files against
1,438.

A REMOVAL THAT BREAKS THE APP IS NOT A REMOVAL, and two nearly shipped:

deploy/site-profile-universal.json still named all seventeen plugins. It is the
file docs/DEPLOY.md tells a site to provision from, so a fresh install would
have failed at `flask plugin apply-profile` with PluginError - the first thing a
receiving site does, broken by the branch meant to serve them. It now names the
twelve that exist, and its comment says the two must agree.

agent/ was never refreshed. Whoever populated this branch copied shopdb/,
plugins/, frontend/ and tests/ from the site line and left the agent tree at its
September state: 22 C# and PowerShell sources missing, including AgentCredentials,
EnrollmentService, AgentHttpFactory and the fleet-alert client, plus one file the
site line had deleted still present. The old tree is internally consistent, so it
compiles, and nothing in CI builds it - every gate was green over a server that
speaks enrollment and per-PC credentials to a bundled agent source that cannot.
Resynced whole.

DECISIONS THIS ENCODES, rather than defers:

- The plugin-authoring lab builds printedparts, whose manifest depends on
  employees, so it could not stay. The lab remains on the site line and main
  ships no build-along tutorial. That is a real loss for an adopting developer
  and is recorded here rather than discovered later.
- Recognition, Training and Recertification notifications are no longer seeded.
  Each splits one notification into a card per employee and renders a photo,
  which needs a plugin this branch does not bundle. Seeding them would offer an
  author a type that cannot resolve anybody. A site that installs employees gets
  them from that plugin's own seed.
- Windows/IIS is documented as the legacy path throughout, and the README leads
  with Docker, because that is what a receiving site gets.

The API description is regenerated for THIS branch - 347 paths and 483
operations, down from 415 and 573 - so the published contract describes what is
here rather than what the site line serves. docs/CONFIG.md lost the two
measuring-tool label keys the code no longer declares, which two lanes had
decided opposite ways.

Suite: 3124 passed, 6 skipped. Naming and style gates pass. The docs gates run
rather than skip, and every index line count is derived from the file it names.
This commit is contained in:
cproudlock
2026-09-19 11:24:40 -04:00
parent cb3975049c
commit 534f179ee6
675 changed files with 115759 additions and 60239 deletions
Download Patch File Download Diff File Expand all files Collapse all files

88
shopdb/__init__.py
View File

@@ -33,9 +33,9 @@ from .plugins import plugin_manager
# which backs seeding, the role grid, and API-token scope validation. Additive
# optional hook, minor bump.
# 0.11.0: added service_token_authorized(scope) to shopdb.api so a plugin's
# unattended endpoints (e.g. the GE-Enforce fetch API) can authorize a scoped
# managed service token without importing core token internals. Additive name
# on the import surface, minor bump.
# unattended endpoints (a fetch route a fleet agent polls, say) can authorize a
# scoped managed service token without importing core token internals. Additive
# name on the import surface, minor bump.
# 0.16.0: added the get_settings_defaults hook so a plugin declares the Setting
# rows it owns (key, value, type, category, description, public). The framework
# seeds them at install, at enable, and on `flask plugin upgrade-all`, files a
@@ -46,8 +46,8 @@ from .plugins import plugin_manager
# beside DashboardDefault (which 0.17.0 imported but never listed in __all__).
# The role vocabulary is now the kiosk's own - Dashboard / Lobby / 3DPrintRoom,
# the literal values of C:\Enrollment\display-type.txt - so a plugin holding its
# own copy of that map (geenforce did) can read core's instead of drifting from
# it. Additive names on the import surface, minor bump.
# own copy of that map can read core's instead of drifting from it. Additive
# names on the import surface, minor bump.
# 0.19.0: get_dashboard_widgets returns DATA AND SHAPE, not a component name.
# The old shape ('name' + 'component' + 'size') named a Vue component per
# widget, which cannot survive a lean build - a plugin's component may never be
@@ -110,7 +110,46 @@ from .plugins import plugin_manager
# internals. The journal records the primitive mutations a bulk import makes
# (update a cell, insert a row) tied to a batch id, and a generic undo replays
# them in reverse. Additive names on the surface, minor bump.
__contract_version__ = '0.30.0'
#
# 0.31.0 widens the asset-panel vocabulary a plugin may declare: two more
# tab groups (serviceprocess, reports), a datetime column format, a map body
# key, and a tabs section that renders as a table. All optional, so a plugin
# that declares none of them is unaffected. Additive, minor bump.
# 0.32.0 adds caller_is_role to the authorization surface: the body-answering
# counterpart to require_role, for a route that serves everyone but treats one
# class of caller differently (admin break-glass override). Scoped personal
# tokens are always refused by it, matching require_role, so a plugin cannot
# accidentally let a scoped token inherit role authority from its owner.
# Additive name on the surface, minor bump.
# 0.33.0 adds per-PC agent credentials to the collector contract. BasePlugin
# gains two optional hooks, resolve_agent_credential(secret) and
# agent_shared_key_refused(hostname), and shopdb.api exports
# resolve_agent_credential, agent_shared_key_refused and collector_ingest. A
# collector schema may declare "acceptsagentcredential": true; the generic
# collector then accepts X-Endpoint-Credential for it, refuses the shared key
# for an enrolled PC (401 SHARED_KEY_REFUSED), and maps a PermissionError from
# apply_collector_payload to 403 ASSET_MISMATCH. All optional, so a plugin that
# declares none of it is unaffected. Additive, minor bump.
# 0.34.0 adds BusinessUnit to the core model surface. It was the one thing an
# asset carries that a plugin could not name: the endpoints fleet-alert scope
# resolves a business unit per poll, and with no contract name for it the route
# had dropped to db.text('SELECT businessunit FROM businessunits ...') against a
# core table. That workaround respects the letter of ADR-002 and defeats its
# point - a raw query is hidden coupling to the core schema that no contract
# bump can catch, and it is the kind of thing that gets copied. Exposing the
# model closes the gap instead of leaving the workaround as the worked example.
# Additive name on the surface, minor bump.
# 0.35.0 adds ip_in_cidrs and request_ip_allowlisted to shopdb.api: network
# position as an admission check for routes that run without a login. The two
# kiosks needed the identical gate - a device token suits an unattended screen
# provisioned once, and suits nothing about somebody opening the same page on
# their own PC - and a CIDR parser copied into two plugins is exactly where a
# fail-open creeps in later. Both fail closed on an empty setting, an
# unparseable caller address and a malformed entry. Caller trustworthiness is a
# deployment property: they read remote_addr, never raw X-Forwarded-For, so the
# IIS rewrite rule is what makes them meaningful. Additive names on the surface,
# minor bump.
__contract_version__ = '0.35.0'
# Product release version (see ADR-007). The product version and the
# plugin-contract version above are distinct series with independent
@@ -199,11 +238,46 @@ def create_app(config_name: str = None) -> Flask:
# Initialize plugin manager
with app.app_context():
# On first run (no users exist), tell the operator a setup token is
# active and how to read it. The token gates POST /api/setup/create-admin
# to prevent a network race during the install window (API starts
# serving before the admin exists in every documented deploy path). The
# secret itself is never logged: logs are shipped and retained far more
# widely than the .env it came from.
from shopdb.core.models import User
try:
if User.query.count() == 0:
token = app.config.get('SETUP_TOKEN') or os.environ.get('SETUP_TOKEN')
if token:
app.logger.warning('=' * 60)
app.logger.warning('FIRST RUN: No users exist.')
app.logger.warning('A setup token is active (SETUP_TOKEN). '
'Read it with: flask seed show-setup-token')
app.logger.warning('Create admin: POST /api/setup/create-admin')
app.logger.warning(' with header: X-Setup-Token: <token>')
app.logger.warning('OR use CLI: docker exec shopdb-api flask seed admin')
app.logger.warning('=' * 60)
else:
app.logger.warning('FIRST RUN: No users exist, no SETUP_TOKEN set.')
app.logger.warning('Use CLI: docker exec shopdb-api flask seed admin')
except Exception:
pass # DB not ready yet, skip
plugin_manager.init_app(app, db)
# Register core blueprints
register_blueprints(app)
# Finish any restore that is waiting on its schema upgrade. The backup
# sidecar has no Flask, so it loads the dump and leaves a sentinel; this
# container owns the app context that can run the migrations. Started here
# rather than left to a browser poll, because a restore must complete
# whether or not anyone is still watching it.
#
# Skipped unless a backups store actually exists, which keeps it off every
# deployment that does not run the backup overlay - and out of the test
# suite, which builds this app thousands of times.
# NOTE: wilm's backup implementation doesn't require initialization here
# Personal API token auth shim: recognize `Bearer shopdb_pat_...` before
# any JWT decode and mint a request-scoped JWT for the token's owner.
from .utils.apitoken_auth import install_apitoken_auth
@@ -292,12 +366,14 @@ CORE_BLUEPRINT_NAMES = (
'collector',
'settings',
'auditlogs',
'employees',
'users',
'customfields',
'setup',
'pluginui',
'apitokens',
'docs',
'shopdbbackups',
)

118
shopdb/api/__init__.py
View File

@@ -32,6 +32,7 @@ from shopdb.core.models import (
Asset,
AssetType,
AssetStatus,
BusinessUnit,
Vendor,
Model,
Communication,
@@ -70,11 +71,11 @@ from shopdb.utils.pagination import get_pagination_params, paginate_query
# Authorization decorators for gating plugin write routes
from shopdb.utils.authz import (require_permission, require_any_permission,
require_role, caller_holds)
require_role, caller_holds, caller_is_role)
# Service-token authorization for unattended plugin endpoints (collector,
# GE-Enforce fetch, ...): checks a scoped managed token without exposing token
# internals.
# Service-token authorization for unattended plugin endpoints (the collector
# ingest API, a plugin's own machine-to-machine fetch route, ...): checks a
# scoped managed token without exposing token internals.
from shopdb.utils.apitoken_auth import (
service_token_authorized, authorized_service_token,
mint_service_token, list_service_tokens, revoke_service_tokens,
@@ -99,6 +100,82 @@ def collector_authorized(pluginname: str):
from shopdb.core.api.collector import _check_collector_auth, _plugin_api_key
return _check_collector_auth(_plugin_api_key(pluginname), pluginname)
def _enabled_plugins():
from flask import current_app
manager = current_app.extensions.get('plugin_manager')
if not manager:
return []
return [plugin for name, plugin in manager.get_all_plugins().items()
if manager.registry.is_enabled(name)]
def resolve_agent_credential(secret: str) -> Optional[int]:
"""Asset id for an active per-PC agent credential, or None.
Asks every enabled plugin's resolve_agent_credential hook and returns the
first non-None answer.
"""
if not secret:
return None
for plugin in _enabled_plugins():
assetid = plugin.resolve_agent_credential(secret)
if assetid is not None:
return assetid
return None
def agent_shared_key_refused(hostname: str) -> bool:
"""True when any enabled plugin refuses the shared collector key for hostname."""
return any(plugin.agent_shared_key_refused(hostname)
for plugin in _enabled_plugins())
def collector_ingest(pluginname: str):
"""Run the generic collector ingest for a plugin on the current request.
Lets a plugin serve the same ingest under its own prefix (the endpoints
agent posts to /api/endpoints/collect) without importing core collector
internals. Auth, validation, audit and response are exactly those of
POST /api/collector/<pluginname>.
"""
from shopdb.core.api.collector import generic_collect
return generic_collect(pluginname)
def application_package_location():
"""Where application installers live, and the URL prefix that serves them.
Returns (directory, urlprefix). A plugin that registers an installable
payload (the endpoint agent bundle, for example) must put the file exactly
where the applications API expects it and build installpath the same way,
or update_app cannot resolve the download. Reconstructing either by hand
lets the two drift silently, so both come from one place.
Imported lazily for the same reason as collector_authorized above:
shopdb.core.api imports back into shopdb.api, so a module-level import
would be circular at load time.
"""
from shopdb.core.api.applications import (
_apppackage_dir, APP_PACKAGE_URL_PREFIX)
return _apppackage_dir(), APP_PACKAGE_URL_PREFIX
def script_body_location():
"""Where script bodies live, and the URL prefix that serves them.
Returns (directory, urlprefix). Same reasoning as
application_package_location: a plugin that seeds a script into the
library must write the body where the download route reads it from, and
reconstructing that path by hand lets the two drift apart silently.
Imported lazily to avoid the shopdb.core.api -> shopdb.api cycle.
"""
from shopdb.core.api.scripts import _script_dir, SCRIPT_URL_PREFIX
return _script_dir(), SCRIPT_URL_PREFIX
# Serving an uploaded file safely. A plugin that serves user-supplied bytes must
# not have to remember the headers that keep an SVG or an HTML file from running
# as script in this origin - see shopdb/utils/uploads.py.
@@ -110,6 +187,11 @@ from shopdb.utils.uploads import send_upload, harden_upload_response
# nothing. See shopdb/utils/clientip.py.
from shopdb.utils.clientip import client_ip, strip_port
# Network-position allowlists for the unauthenticated kiosk routes. On the
# contract surface because two plugins need the identical gate and a
# copy-pasted CIDR parser is exactly where a fail-open creeps in.
from shopdb.utils.ipallowlist import ip_in_cidrs, request_ip_allowlisted
# Strict date parsing. On the contract surface because every plugin that stores
# a date needs it and the two that rolled their own both swallowed the error.
from shopdb.utils.queryargs import QueryArgError, intarg, intargs
@@ -145,13 +227,17 @@ from shopdb.core.services.csvimport import (
RowWarning,
)
# Legacy employee directory lookup (read-only) used by notifications
# External HR employee directory lookup (read-only) used by core global search
# and core audit-log name resolution.
from shopdb.utils.employee_db import employee_connection
# The two resolvers built on top of it. On the contract surface because a plugin
# that shows a person - a notification aimed at somebody, a card on a wall
# screen - must render the same name and the same photo core does, and a plugin
# may not reach into shopdb.core.
from shopdb.core.api.employees import (
resolve_employee_display_name, resolve_employee_photo_url)
from shopdb.utils.mailer import send_email, send_alert, send_webhook
# CMMC USB check-in/out database (read-write) used by the usb plugin
from shopdb.utils.cmmc_usb_db import cmmc_usb_connection
# Reversible bulk-import journal. A plugin that drives a bulk import (a site
# asset adoption, an integration sync) records the primitive changes it makes so
@@ -341,6 +427,7 @@ __all__ = [
'Asset',
'AssetType',
'AssetStatus',
'BusinessUnit',
'Vendor',
'Model',
'Communication',
@@ -380,6 +467,7 @@ __all__ = [
'require_permission',
'require_any_permission',
'caller_holds',
'caller_is_role',
'require_role',
'service_token_authorized',
'authorized_service_token',
@@ -387,23 +475,31 @@ __all__ = [
'list_service_tokens',
'revoke_service_tokens',
'collector_authorized',
'resolve_agent_credential',
'agent_shared_key_refused',
'collector_ingest',
'application_package_location',
'script_body_location',
'SupportTeam',
# Serving uploads
'send_upload',
'harden_upload_response',
'client_ip',
'strip_port',
'ip_in_cidrs',
'request_ip_allowlisted',
# Import-mode helpers
'apply_import_timestamps',
'import_mode_active',
'parse_import_datetime',
# Legacy employee directory
# External HR employee directory (read-only) and the two resolvers a
# plugin uses to show a person the same way core does.
'employee_connection',
'resolve_employee_display_name',
'resolve_employee_photo_url',
'send_email',
'send_alert',
'send_webhook',
'User',
'Role',
# CMMC USB check-in/out database
'cmmc_usb_connection',
]

128
shopdb/cli/__init__.py
View File

@@ -199,6 +199,32 @@ def seed_cli():
pass
@seed_cli.command('show-setup-token')
@with_appcontext
def show_setup_token():
"""Display the setup token for first-admin creation.
The token gates POST /api/setup/create-admin so a network race cannot claim
the instance during the install window. Startup only logs that a token is
active (never the secret itself); this command is where it is read.
"""
import os
from flask import current_app
from shopdb.core.models import User
if User.query.count() > 0:
click.echo(click.style("Setup already complete (users exist).", fg='green'))
return
token = current_app.config.get('SETUP_TOKEN') or os.environ.get('SETUP_TOKEN')
if token:
click.echo(click.style("Setup token: ", fg='cyan') + token)
click.echo("Use: curl -X POST .../api/setup/create-admin -H 'X-Setup-Token: <token>' ...")
else:
click.echo(click.style("No SETUP_TOKEN configured.", fg='yellow'))
click.echo("Use 'flask seed admin' from inside the container (no token needed).")
@seed_cli.command('catalog')
@click.option('--file', 'path', default=None,
help='catalog JSON to load (default shopdb/data/catalog.json)')
@@ -317,8 +343,7 @@ def seed_catalog(path, dry_run):
# Small plugin vocabularies, loaded through the registry so a lean build
# missing that plugin skips them instead of failing to import.
for key, model_table, column in (('measuringtooltypes', 'measuringtooltypes', 'name'),
('notificationtypes', 'notificationtypes', 'typename'),
for key, model_table, column in (('notificationtypes', 'notificationtypes', 'typename'),
('accessprotocols', 'accessprotocols', 'name')):
model = by_table.get(model_table)
if model is not None:
@@ -713,9 +738,9 @@ def fix_controls_direction():
computer and whose source is not. If the flipped row already exists, the
reversed duplicate is deactivated instead. Idempotent.
Covers machines, measuring tools, printers and network devices. It used to
match machines ONLY, so a CMM PC kept showing "<- controls from CMM4"
alongside its own outgoing link and no amount of running this fixed it.
Covers every non-computer asset type, not machines alone. It used to match
machines ONLY, so a CMM PC kept showing "<- controls from CMM4" alongside
its own outgoing link and no amount of running this fixed it.
"""
from sqlalchemy.orm import aliased
from shopdb.extensions import db
@@ -743,7 +768,7 @@ def fix_controls_direction():
AssetRelationship.isactive == True,
# ANY non-computer controlled BY a computer, not just machines. The
# original filter said assettype == 'machine', which left every
# measuring_tool -> computer row untouched - a CMM PC showing
# non-machine -> computer row untouched - a CMM PC showing
# "<- controls from CMM4" beside its own outgoing link, which is
# exactly the shape this command exists to clean. Printers and
# network devices reach shopdb the same way and had the same gap.
@@ -784,7 +809,7 @@ def fix_controls_direction():
@seed_cli.command('permissions')
@with_appcontext
def seed_permissions():
"""Seed predefined permissions, and the guest role that can hold them."""
"""Seed predefined permissions, the guest role, and the prebuilt scopes."""
from shopdb.extensions import db
from shopdb.core.models import Permission, Role
@@ -793,11 +818,23 @@ def seed_permissions():
# on the upgrade it already runs. It holds nothing until somebody grants
# it something, so seeding it changes no behaviour.
guest = Role.seedguest()
# AFTER Permission.seed(), which is load-bearing: seedscopes only links
# permissions that exist as rows, so seeding the catalog first is what
# lets a newly installed plugin's permissions reach the scopes in the
# same command rather than on the next run.
scopes = Role.seedscopes()
db.session.commit()
click.echo(click.style(f"{created} permissions created.", fg='green'))
if guest:
click.echo(click.style(
"guest role created (holds no permissions).", fg='green'))
for rolename in scopes['created']:
click.echo(click.style(f"{rolename} role created.", fg='green'))
for rolename, count in sorted(scopes['granted'].items()):
click.echo(click.style(
f"{rolename}: {count} permission(s) granted.", fg='green'))
if not scopes['created'] and not scopes['granted']:
click.echo("Prebuilt scopes already up to date.")
@seed_cli.command('settings')
@@ -834,9 +871,9 @@ def seed_demo(force):
"""Seed a small, broad sample dataset for a dev/eval site.
Populates a handful of rows across every asset-based plugin (machines,
computers, printers, network devices, measuring tools) plus 3D-printed
parts, with supporting vendors/business-units/locations and a few
relationships, so every screen has something to show. Run AFTER
computers, printers, network devices), with supporting
vendors/business-units/locations and a few relationships, so every screen
has something to show. Run AFTER
`flask seed reference-data` and after the plugins are installed. Idempotent:
all rows are keyed on the DEMO- prefix and skipped if already present.
@@ -926,7 +963,6 @@ def seed_demo(force):
Computer = _subtype_model('computers', 'Computer')
Printer = _subtype_model('printers', 'Printer')
NetworkDevice = _subtype_model('network', 'NetworkDevice')
MeasuringTool = _subtype_model('measuringtools', 'MeasuringTool')
machines = [
('MILL-01', 'Haas VF-2 Mill', 'In Use', 'Cell A', 'Machining'),
@@ -972,42 +1008,6 @@ def seed_demo(force):
make_asset('network_device', num, name, NetworkDevice, status=st,
location=loc, unit='IT')
tools = [
('CMM-01', 'Zeiss CMM', 'In Use', 'QA Lab'),
('GAGE-01', 'Height Gage', 'In Use', 'QA Lab'),
('GAGE-02', 'Bore Gage', 'In Use', 'QA Lab'),
('MIC-01', 'Digital Micrometer', 'In Use', 'Cell A'),
('CAL-01', 'Digital Caliper', 'Inventory', 'QA Lab'),
]
for num, name, st, loc in tools:
make_asset('measuring_tool', num, name, MeasuringTool, status=st,
location=loc, unit='Inspection')
# 3D-printed parts are not assets - own table. A couple sit below their
# low-stock threshold on purpose so the low-stock alert has something to fire.
printedparts_made = 0
try:
from plugins.printedparts.models import PrintedItem
parts = [
# itemname, itemcode, gagelabtag, qty, threshold, bin
('Fixture Bracket', 'PP0001', 'WJRP10021', 12, 4, 'A1'),
('Gage Holder', 'PP0002', 'WJRP10022', 3, 5, 'A2'),
('Cable Clip', 'PP0003', None, 40, 10, 'B1'),
('Sensor Mount', 'PP0004', 'WJRP10023', 2, 6, 'B2'),
('Label Guide', 'PP0005', None, 25, 8, 'C1'),
('Knob Cover', 'PP0006', None, 0, 3, 'C2'),
]
for name, code, tag, qty, thr, binloc in parts:
if PrintedItem.query.filter_by(itemcode=code).first():
continue
db.session.add(PrintedItem(
itemname=name, itemcode=code, gagelabtag=tag,
quantityonhand=qty, lowstockthreshold=thr, binlocation=binloc,
itemdescription=f'Sample 3D-printed part: {name}.'))
printedparts_made += 1
except ImportError:
pass # printedparts plugin not installed - skip
db.session.flush()
# A few relationships so the map + relationship cards are not empty.
@@ -1048,7 +1048,7 @@ def seed_demo(force):
db.session.commit()
click.echo(click.style(
f"Demo data seeded: {made['assets']} assets, "
f"{printedparts_made} printed parts, {rels_made} relationships "
f"{rels_made} relationships "
f"({made['skipped']} already existed).", fg='green'))
click.echo("Remove later with: flask seed demo-clear")
@@ -1057,31 +1057,22 @@ def seed_demo(force):
@click.option('--yes', is_flag=True, help='Skip the confirmation prompt.')
@with_appcontext
def seed_demo_clear(yes):
"""Delete everything `flask seed demo` created (DEMO- assets + sample parts).
"""Delete everything `flask seed demo` created (the DEMO- assets).
Only touches rows the demo seeder made: assets with the DEMO- prefix (their
plugin subtype rows cascade) and the PP000x sample printed parts. Leaves
reference data, settings, users, and any real/imported rows alone.
Only touches rows the demo seeder made: assets with the DEMO- prefix, whose
plugin subtype rows cascade. Leaves reference data, settings, users, and any
real/imported rows alone.
"""
from shopdb.extensions import db
from shopdb.core.models import Asset
demo_ids = [a.assetid for a in Asset.query.filter(
Asset.assetnumber.like(f'{DEMO_PREFIX}%')).all()]
try:
from plugins.printedparts.models import PrintedItem
parts_count = PrintedItem.query.filter(
PrintedItem.itemcode.like('PP000%')).count()
except ImportError:
parts_count = 0
if not demo_ids and not parts_count:
if not demo_ids:
click.echo(click.style("No demo data found.", fg='yellow'))
return
if not yes:
click.confirm(
f"Delete {len(demo_ids)} demo assets and "
f"{parts_count} sample parts?", abort=True)
click.confirm(f"Delete {len(demo_ids)} demo assets?", abort=True)
if demo_ids:
# Drop the demo relationships first - assetrelationships has no cascade
@@ -1098,16 +1089,9 @@ def seed_demo_clear(yes):
Asset.query.filter(Asset.assetid.in_(demo_ids)).delete(
synchronize_session=False)
parts_deleted = 0
if parts_count:
parts_deleted = PrintedItem.query.filter(
PrintedItem.itemcode.like('PP000%')).delete(
synchronize_session=False)
db.session.commit()
click.echo(click.style(
f"Removed {len(demo_ids)} demo assets and "
f"{parts_deleted} sample parts.", fg='green'))
f"Removed {len(demo_ids)} demo assets.", fg='green'))
@click.group('csv')
@@ -1372,7 +1356,7 @@ def audit_relationships():
its own page and is never meaningful.
3. DUPLICATE DEVICES. One PC controlling several assets of the same type -
three measuring tools for one physical CMM, say. The collector keys its
three machine rows for one physical CMM, say. The collector keys its
idempotency on its own label, so a device somebody created by hand, or
the legacy import created, is invisible to it and it mints another one on
every fresh PC.

92
shopdb/config.py
View File

@@ -10,6 +10,18 @@ class ConfigError(Exception):
"""Raised when required configuration is missing or unsafe."""
# Connection pool per worker process. Env-driven so a site can size it against
# its own MySQL max_connections without editing code; see the arithmetic in
# docker-compose.yml next to the worker count.
#
# Deliberately NOT named SQLALCHEMY_POOL_SIZE and friends: those were
# Flask-SQLAlchemy 2.x config keys, removed in 3.x, and reusing the names would
# suggest the extension still reads them. It reads SQLALCHEMY_ENGINE_OPTIONS.
DB_POOL_SIZE = int(os.environ.get('DB_POOL_SIZE', 10))
DB_MAX_OVERFLOW = int(os.environ.get('DB_MAX_OVERFLOW', 10))
DB_POOL_TIMEOUT = int(os.environ.get('DB_POOL_TIMEOUT', 5))
def _required_env(varname):
"""Read an env var; raise ConfigError if missing or empty."""
value = os.environ.get(varname)
@@ -32,9 +44,32 @@ class Config:
'mysql+pymysql://root:password@localhost:3306/shopdb_flask',
)
SQLALCHEMY_TRACK_MODIFICATIONS = False
# Connection pool, sized on purpose rather than left to SQLAlchemy.
#
# The defaults (pool_size 5, max_overflow 10, pool_timeout 30) were never
# chosen: they simply applied, and nothing in the repo related them to the
# gunicorn worker count or to the MySQL max_connections ceiling. The
# arithmetic that ties the three together is written next to the worker
# count in docker-compose.yml.
#
# Under gevent a pooled connection is held for the duration of whatever the
# greenlet is doing, so a request that sits inside a slow external HTTP call
# WITH THE SESSION OPEN pins a connection for that whole call. Hundreds of
# greenlets can enter such a path at once and drain the pool, at which point
# every other request blocks on checkout. Hence the fail-fast timeout: 5
# seconds, not 30. A pool that is empty for five seconds is not going to be
# fine in thirty, and a fast 500 is diagnosable where a 30-second hang looks
# like a dead database.
#
# See PLUGIN-DEVELOPMENT / the plugin docs: a plugin must not hold a DB
# session across external I/O.
SQLALCHEMY_ENGINE_OPTIONS = {
'pool_pre_ping': True,
'pool_recycle': 300,
'pool_size': DB_POOL_SIZE,
'max_overflow': DB_MAX_OVERFLOW,
'pool_timeout': DB_POOL_TIMEOUT,
# Pin the client connection to utf8mb4 so reads/writes match the
# utf8mb4 schema regardless of the server default charset. TestingConfig
# (SQLite) replaces this whole dict, so the charset arg never reaches it.
@@ -130,15 +165,6 @@ class Config:
EMPLOYEE_DB_PASSWORD = os.environ.get('EMPLOYEE_DB_PASSWORD', '')
EMPLOYEE_DB_NAME = os.environ.get('EMPLOYEE_DB_NAME', 'wjf_employees')
# Read-write CMMC USB check-in/out database (separate from the app DB).
# Credentials come from the environment; never hardcode them in source.
# No safe default for the password: unset means empty, the connection
# fails loud rather than silently trying a guessed credential.
CMMC_USB_DB_HOST = os.environ.get('CMMC_USB_DB_HOST', 'localhost')
CMMC_USB_DB_USER = os.environ.get('CMMC_USB_DB_USER', '')
CMMC_USB_DB_PASSWORD = os.environ.get('CMMC_USB_DB_PASSWORD', '')
CMMC_USB_DB_NAME = os.environ.get('CMMC_USB_DB_NAME', 'cmmc_usb')
CACHE_TYPE = 'SimpleCache'
CACHE_DEFAULT_TIMEOUT = 600
@@ -150,6 +176,22 @@ class Config:
AUTH_RATELIMIT_WINDOW_SECONDS = int(
os.environ.get('AUTH_RATELIMIT_WINDOW_SECONDS', 300))
# Swagger UI "try it" page at /api/docs/try, beside the read-only Redoc
# page at /api/docs. One spec, two renderers.
#
# Off unless a site turns it on, and that is the point rather than caution:
# Swagger UI's Execute button fires a REAL request at THIS server from the
# operator's browser, carrying whatever session that browser already holds.
# On a production box that is a one-click way to POST against live fleet
# data from a page anyone can open. Redoc, which cannot send anything, stays
# the default and is always on.
#
# When off the routes are never bound at all, so /api/docs/try is Flask's
# own 404 and the Redoc page does not mention it. See
# shopdb/core/api/docs.py.
API_DOCS_TRYIT_ENABLED = os.environ.get(
'API_DOCS_TRYIT_ENABLED', 'false').lower() == 'true'
DEFAULT_PAGE_SIZE = 20
MAX_PAGE_SIZE = 100
@@ -163,11 +205,14 @@ class Config:
# three different routes, which makes it a property of the app rather than
# of any handler.
#
# 128 MB clears the largest thing anyone legitimately sends - the agent
# binary upload caps itself at 64 MB - with room for the multipart envelope.
# 256 MB clears the largest thing anyone legitimately sends - the agent
# binary upload caps itself at 128 MB - with room for the multipart
# envelope. This must stay STRICTLY ABOVE the agent cap: it bounds the
# whole request (file + boundaries + form fields), so setting the two
# equal would reject a file sitting exactly at the agent limit.
# Over this, Werkzeug answers 413 without reading the body.
MAX_CONTENT_LENGTH = int(
os.environ.get('MAX_CONTENT_LENGTH', 128 * 1024 * 1024))
os.environ.get('MAX_CONTENT_LENGTH', 256 * 1024 * 1024))
class DevelopmentConfig(Config):
@@ -176,6 +221,12 @@ class DevelopmentConfig(Config):
DEBUG = True
SQLALCHEMY_ECHO = True
# On in development: firing the request is the whole reason a developer
# opens the API docs. Still read from the environment, so a shared dev
# instance can turn it back off without editing this file.
API_DOCS_TRYIT_ENABLED = os.environ.get(
'API_DOCS_TRYIT_ENABLED', 'true').lower() == 'true'
SQLALCHEMY_DATABASE_URI = os.environ.get(
'DATABASE_URL',
'mysql+pymysql://root:changeme@127.0.0.1:3306/shopdb_flask',
@@ -189,6 +240,23 @@ class TestingConfig(Config):
# Off by default so login-heavy fixtures do not trip the limiter; tests
# that exercise rate limiting flip it on via app.config override.
AUTH_RATELIMIT_ENABLED = False
# Same reason: the enrollment suites post to /enroll dozens of times, and
# the limiter counts in a process-wide cache that outlives a test, so with
# it on by default a budget spent in one module surfaces as an unrelated
# failure in another - and which modules share a worker depends on the
# xdist distribution, so the failure moves around. The tests that are about
# the limiter turn it on themselves.
ENDPOINTS_ENROLL_RATELIMIT_ENABLED = False
# Off, and stated here rather than inherited, because a test suite that
# silently ran with it on would be testing a configuration no site ships.
# The session-wide app in tests/conftest.py is what
# tests/test_core/test_public_read_surface.py sweeps for ungated GET
# routes, and the try-it page plus its two vendored assets are
# deliberately unauthenticated. Keeping the default here means that sweep
# keeps describing what production actually exposes; the tests that ARE
# about the page build their own app with the flag on
# (tests/test_core/test_api_docs_tryit.py).
API_DOCS_TRYIT_ENABLED = False
SQLALCHEMY_DATABASE_URI = 'sqlite:///:memory:'
SQLALCHEMY_ENGINE_OPTIONS = {
'connect_args': {'check_same_thread': False},

4
shopdb/core/api/__init__.py
View File

@@ -21,12 +21,14 @@ from .reports import reports_bp
from .collector import collector_bp
from .settings import settings_bp
from .auditlogs import auditlogs_bp
from .employees import employees_bp
from .users import users_bp
from .customfields import customfields_bp
from .setup import setup_bp
from .pluginui import pluginui_bp
from .apitokens import apitokens_bp
from .docs import docs_bp
from .shopdbbackups import shopdbbackups_bp
__all__ = [
'auth_bp',
@@ -50,10 +52,12 @@ __all__ = [
'collector_bp',
'settings_bp',
'auditlogs_bp',
'employees_bp',
'users_bp',
'customfields_bp',
'setup_bp',
'pluginui_bp',
'apitokens_bp',
'docs_bp',
'shopdbbackups_bp',
]

6
shopdb/core/api/apitokens.py
View File

@@ -49,9 +49,9 @@ def _validate_resourcescopes(resourcescopes):
"""Validate a resourcescopes payload. Return (list_or_none, error_response).
resourcescopes is a DIFFERENT axis from scopes: it pins the token to a set
of resource names (today geenforce manifest scope names, e.g.
gea-shopfloor-display), not permission names, so there is no catalog to
check against - a plugin owns the meaning. None/absent = unrestricted.
of resource names a plugin defines, not permission names, so there is no
catalog to check against - a plugin owns the meaning. None/absent =
unrestricted.
"""
if resourcescopes is None:
return None, None

466
shopdb/core/api/applications.py
View File

@@ -4,6 +4,7 @@ import glob
import hashlib
import re
import os
import shutil
from flask import Blueprint, request, current_app
from flask_jwt_extended import jwt_required
@@ -36,11 +37,147 @@ APP_PACKAGE_EXTENSIONS = {'.exe', '.msi', '.msp', '.zip', '.7z', '.cab', '.iso',
'.appx', '.msix', '.ps1', '.bat', '.txt', '.pdf'}
APP_PACKAGE_URL_PREFIX = '/api/applications/package/'
def _requires_policy_ack(app, data):
"""True when this save would newly generate an install-enforcing policy.
Ticking "Required on all PCs" does not just flag a report row: it creates a
policy that installs this application on every matched endpoint. That is a
real change to real machines, so the caller has to acknowledge it
explicitly. Enforced here rather than only in the form because the checkbox
is reachable through the API too, and a confirmation dialog that can be
skipped by calling the endpoint directly is not a safeguard.
Only NEWLY ticking it needs acknowledgement. Unticking, or saving an app
that was already required, does not.
"""
if not data.get('isrequired'):
return False
if app is not None and app.isrequired:
return False
return not data.get('acknowledgepolicy')
POLICY_ACK_MESSAGE = (
'Marking this application as required on all PCs creates a configuration '
'policy that installs it on every non-server endpoint it matches, once that '
'policy is added to an active business policy. It is not added to one '
'automatically. Confirm to create the policy.'
)
# Returned alongside policyresult='created'. The generated policy is not
# attached to any BusinessPolicy, and the resolver only applies policies reached
# through one, so saying "this will install everywhere" would be untrue.
POLICY_CREATED_NOTE = (
'A configuration policy was created for this application, but it is not '
'attached to any business policy and installs nothing until an admin adds '
'it to one.'
)
# Deleting an application is irreversible and takes its versions and uploaded
# installer with it, so the caller has to type this rather than click once.
# Checked server-side as well as in the dialog: the endpoint is reachable
# directly, and a confirmation that only exists in the UI is not a safeguard.
DELETE_CONFIRM_TOKEN = 'OK'
DELETE_CONFIRM_MESSAGE = (
'Deleting an application permanently removes it, its versions, and its '
'uploaded installer. This cannot be undone. Type OK to confirm.'
)
def _delete_confirmed(data):
"""True when the caller typed the confirmation token exactly.
Compared case-sensitively after stripping surrounding whitespace. Accepting
"ok" would defeat the point of asking for deliberate input.
"""
if not isinstance(data, dict):
return False
return str(data.get('confirm', '')).strip() == DELETE_CONFIRM_TOKEN
def _validate_monitorable(data, existing=None):
"""Return an error message if monitorable is set without its two fields.
A process monitor needs an image name to watch for and a path to launch when
it is missing; without both, ticking the box would create a monitor that
cannot do anything. Enforced here rather than as a DB constraint so the
columns can stay nullable for rows that predate the feature.
`existing` lets a PATCH-style update validate the MERGED result: a payload
that only toggles the checkbox is still checked against values already
stored on the row.
"""
def merged(field):
if field in data:
return (data.get(field) or '').strip()
return ((getattr(existing, field, None) or '').strip()
if existing is not None else '')
if 'ismonitorable' in data:
monitorable = data.get('ismonitorable')
else:
monitorable = getattr(existing, 'ismonitorable', False)
if not monitorable:
return None
if not merged('monitorlaunchpath'):
return 'A monitorable application needs a launch path'
if not merged('monitorprocessname'):
return 'A monitorable application needs a process name'
return None
def _policy_payload(payload, policyresult):
"""Attach the policy sync outcome (and the not-attached note) to a response."""
if policyresult in ('created', 'removed', 'kept'):
payload['policyresult'] = policyresult
if policyresult == 'created':
payload['policynote'] = POLICY_CREATED_NOTE
return payload
def _deactivate_policies_for_app(app_id):
"""Deactivate remaining policies that reference a deleted appid, if available."""
try:
from plugins.computers.services.apppolicy import deactivate_policies_for_app
except ImportError:
return []
return deactivate_policies_for_app(app_id)
def _sync_required_app_policy(app):
"""Sync the auto-generated "required on all PCs" policy, if available.
The policy engine lives in the computers PLUGIN, which may be absent or
disabled, so this is a guarded import: core must not hard-depend on a
plugin. When the plugin is not installed the checkbox keeps its original
compliance-report meaning and nothing else happens.
"""
try:
from plugins.computers.services.apppolicy import sync_required_app_policy
except ImportError:
return None
try:
return sync_required_app_policy(app)
except Exception:
# A policy-generation failure must not lose the user's application edit.
current_app.logger.exception(
'Failed to sync the required-app policy for appid=%s', app.appid)
return None
# A real installer runs to hundreds of megabytes; an ISO runs to gigabytes and
# does not belong in an instance directory. 500MB is the line: enough for the
# msi/exe this is for, small enough that the disk cannot vanish behind it.
MAX_PACKAGE_BYTES = 500 * 1024 * 1024
# How much of an installer to read when looking for its version resource. The
# PE resource directory is usually within the first few MB; reading the whole
# file would mean holding a 500MB upload in memory to learn a version string.
PE_VERSION_SCAN_BYTES = 32 * 1024 * 1024
def _appimage_dir():
return os.path.join(current_app.instance_path, 'applicationimages')
@@ -64,6 +201,46 @@ def _sha256_of(stream):
return digest.hexdigest()
def _detect_installer_version(stream):
"""Read the FileVersion out of an uploaded PE installer, or None.
Deliberately reuses the endpoints plugin's PE reader rather than parsing the
format again here. A version detected at upload has to match what the agent
reports from the installed machine, and two separate implementations of the
same structure walk would drift apart and make an endpoint look
out-of-compliance for no real reason.
FileVersion only, never ProductVersion: ProductVersion may carry semver
metadata (1.5.0+abc123) which is not a version number we can compare or use
in a path. Returns None for MSI and archives, which are not PE files at all
and simply have no version to read.
"""
try:
from plugins.endpoints.api.agentstore import read_pe_file_version
except ImportError:
return None
position = stream.tell()
try:
# The version resource lives in the PE header near the start, but the
# resource directory it points into can sit much deeper. Read a bounded
# prefix so a 500MB installer is not pulled into memory whole.
data = stream.read(PE_VERSION_SCAN_BYTES)
finally:
stream.seek(position)
# The parser documents itself as never raising, but it walks offsets read
# out of an attacker-supplied file. A version is a convenience; it must not
# cost the operator the upload of a 500MB installer if that guarantee ever
# slips.
try:
return read_pe_file_version(data)
except Exception:
current_app.logger.warning('Version detection failed for an installer',
exc_info=True)
return None
def _replace_upload(directory, stem, ext, upload):
"""Save one file per application, replacing any prior extension.
@@ -212,6 +389,13 @@ def create_application():
http_code=409
)
monitor_error = _validate_monitorable(data)
if monitor_error:
return error_response(ErrorCodes.VALIDATION_ERROR, monitor_error)
if _requires_policy_ack(None, data):
return error_response(ErrorCodes.VALIDATION_ERROR, POLICY_ACK_MESSAGE)
app = Application(
appname=data['appname'],
appdescription=data.get('appdescription'),
@@ -225,6 +409,9 @@ def create_application():
isprinter=data.get('isprinter', False),
islicenced=data.get('islicenced', False),
isrequired=data.get('isrequired', False),
ismonitorable=data.get('ismonitorable', False),
monitorlaunchpath=data.get('monitorlaunchpath'),
monitorprocessname=data.get('monitorprocessname'),
image=data.get('image')
)
@@ -234,9 +421,13 @@ def create_application():
AuditLog.log('created', 'Application', entityid=app.appid, entityname=app.appname)
policyresult = _sync_required_app_policy(app)
db.session.commit()
return success_response(app.to_dict(), message='Application created', http_code=201)
payload = _policy_payload(app.to_dict(), policyresult)
return success_response(payload, message='Application created', http_code=201)
@applications_bp.route('/<int:app_id>', methods=['PUT'])
@@ -260,12 +451,34 @@ def update_application(app_id: int):
http_code=409
)
monitor_error = _validate_monitorable(data, existing=app)
if monitor_error:
return error_response(ErrorCodes.VALIDATION_ERROR, monitor_error)
if _requires_policy_ack(app, data):
return error_response(ErrorCodes.VALIDATION_ERROR, POLICY_ACK_MESSAGE)
fields = [
'appname', 'appdescription', 'supportteamid', 'isinstallable',
'applicationnotes', 'installpath', 'applicationlink', 'documentationpath',
'ishidden', 'isprinter', 'islicenced', 'isrequired', 'image', 'isactive'
'ishidden', 'isprinter', 'islicenced', 'isrequired', 'image', 'isactive',
'ismonitorable', 'monitorlaunchpath', 'monitorprocessname'
]
# Only touch policy when the checkbox itself moved. Syncing on every save
# would let an unrelated edit resurrect a policy an admin had deleted.
#
# Unticking is synced BEFORE any field is applied, so a PUT that renames and
# unticks in one go withdraws the policy that belongs to the app as it was,
# not one looked up under the new name.
policyresult = None
unticking = ('isrequired' in data and bool(app.isrequired)
and not data['isrequired'])
if unticking:
app.isrequired = False
policyresult = _sync_required_app_policy(app)
app.isrequired = True
changes = {}
for key in fields:
if key in data:
@@ -279,27 +492,210 @@ def update_application(app_id: int):
AuditLog.log('updated', 'Application', entityid=app.appid,
entityname=app.appname, changes=changes)
if 'isrequired' in changes and not unticking:
policyresult = _sync_required_app_policy(app)
elif 'appname' in changes and app.isrequired:
# A rename of a required app keeps its generated policy's name current.
_sync_required_app_policy(app)
apply_import_timestamps(app, data)
db.session.commit()
return success_response(app.to_dict(), message='Application updated')
payload = _policy_payload(app.to_dict(), policyresult)
return success_response(payload, message='Application updated')
def _remove_package_files(app_id: int):
"""Delete the stored installer(s) for an application, leaving the image.
Split out from _remove_upload_files because revert-to-record removes the
executables but keeps the icon: the record stays in the catalogue and a
record with no image is a worse record. Delete calls both.
Best-effort by design: a missing or unreadable file must not abort the
operation and strand the row. Returns the filenames actually removed so the
audit entry records what went.
"""
removed = []
packagedir = os.path.join(_apppackage_dir(), str(app_id))
for path in glob.glob(os.path.join(packagedir, '*')):
removed.append(os.path.basename(path))
if os.path.isdir(packagedir):
shutil.rmtree(packagedir, ignore_errors=True)
if os.path.isdir(packagedir):
current_app.logger.warning(
'Could not remove %s for appid=%s', packagedir, app_id)
return removed
def _remove_upload_files(app_id: int):
"""Delete the stored installer and image for an application.
Used by delete, which takes everything. Revert calls
_remove_package_files() instead so the image survives.
"""
removed = []
# The image is still one flat file per application; the installer now lives
# in a per-application directory so it can keep its vendor filename.
pattern = os.path.join(_appimage_dir(),
secure_filename(f'application-{app_id}') + '.*')
for path in glob.glob(pattern):
try:
os.remove(path)
removed.append(os.path.basename(path))
except OSError:
current_app.logger.warning(
'Could not remove %s while deleting appid=%s', path, app_id)
return removed + _remove_package_files(app_id)
def _clear_application_references(app_id: int):
"""Detach rows that point at this application before it is deleted.
The FKs differ in intent, so they are not treated the same:
- appversions.appid is NOT NULL and the versions are meaningless without
the application, so they go.
- computerinstalledapps.appid is NOT NULL; these are inventory links, so
the link goes while the computer stays.
- knowledgebase.appid is nullable and the article is somebody's writing,
so it is unlinked rather than deleted.
"""
AppVersion.query.filter_by(appid=app_id).delete(synchronize_session=False)
models = _computer_models()
if models is not None:
_, ComputerInstalledApp = models
ComputerInstalledApp.query.filter_by(appid=app_id).delete(
synchronize_session=False)
try:
from plugins.knowledgebase.models import KnowledgeBase
except ImportError:
return
KnowledgeBase.query.filter_by(appid=app_id).update(
{'appid': None}, synchronize_session=False)
@applications_bp.route('/<int:app_id>', methods=['DELETE'])
@require_permission('applications.delete')
def delete_application(app_id: int):
"""Delete (deactivate) an application."""
"""Permanently delete an application, its versions, and its uploads.
This is a hard delete. The previous soft delete only set isactive=False,
which left the name reserved forever: creating an application with the same
name failed with "already exists" against a row no screen would show.
"""
app = db.session.get(Application, app_id)
if not app:
return error_response(ErrorCodes.NOT_FOUND, 'Application not found', http_code=404)
app.isactive = False
if not _delete_confirmed(request.get_json(silent=True)):
return error_response(ErrorCodes.VALIDATION_ERROR, DELETE_CONFIRM_MESSAGE)
AuditLog.log('deleted', 'Application', entityid=app.appid, entityname=app.appname)
# Withdraw the generated policy first. A policy that outlived its
# application would keep installing something that no longer exists.
appname = app.appname
app.isrequired = False
policyresult = _sync_required_app_policy(app)
# 'kept' means a customised generated policy survived the sync, and other
# hand-authored policies may name this appid too. Every one of them would
# keep trying to install an application that no longer exists, so they are
# deactivated (not deleted - they are someone's work) and reported.
deactivated = _deactivate_policies_for_app(app_id)
_clear_application_references(app_id)
removed = _remove_upload_files(app_id)
db.session.delete(app)
# The audit row is the only record left that this application existed, so
# it names the files that went with it.
auditchanges = {}
if removed:
auditchanges['removedfiles'] = removed
if deactivated:
auditchanges['deactivatedpolicies'] = deactivated
AuditLog.log('deleted', 'Application', entityid=app_id, entityname=appname,
changes=auditchanges or None)
db.session.commit()
return success_response(message='Application deleted')
payload = {}
if policyresult in ('removed', 'kept'):
payload['policyresult'] = policyresult
if deactivated:
payload['deactivatedpolicies'] = deactivated
message = 'Application deleted'
if deactivated:
message += ('; deactivated {} policy(ies) that referenced it: {}'
.format(len(deactivated), ', '.join(deactivated)))
return success_response(payload or None, message=message)
REVERT_CONFIRM_MESSAGE = (
'Remove all policy associations and uploaded executables? This action '
"cannot be reversed. Type OK to proceed"
)
@applications_bp.route('/<int:app_id>/revert', methods=['POST'])
@require_permission('applications.edit')
def revert_application_to_record(app_id: int):
"""Strip an application back to a catalogue record.
The middle ground between editing and deleting. Deleting loses the name,
the description and the knowledgebase links, which is too much when the
intent is only to stop deploying something and re-upload a clean installer.
This withdraws the generated policy, drops the version rows and removes the
uploaded files, while the application itself stays in the catalogue.
Inventory links are deliberately kept: computerinstalledapps records that a
machine really does have this software on it, and that stays true whether
or not we still ship an installer for it. Deleting those would make the
fleet look clean when it is not.
Gated on applications.edit rather than applications.delete: the record
survives, and a user trusted to change an application is trusted to stop
deploying it.
"""
app = db.session.get(Application, app_id)
if not app:
return error_response(ErrorCodes.NOT_FOUND, 'Application not found',
http_code=404)
if not _delete_confirmed(request.get_json(silent=True)):
return error_response(ErrorCodes.VALIDATION_ERROR, REVERT_CONFIRM_MESSAGE)
# Withdraw the generated policy first, for the same reason delete does: a
# policy left behind would keep pushing an installer we just removed.
app.isrequired = False
_sync_required_app_policy(app)
AppVersion.query.filter_by(appid=app_id).delete(synchronize_session=False)
# Executables only. The image is not an executable and the warning does not
# offer to remove it - the record survives this operation, and stripping its
# icon would degrade the thing we are explicitly keeping.
removed = _remove_package_files(app_id)
# Only clear an installpath we own. A UNC path typed by an operator points
# at their share, not our upload directory, and is not ours to wipe.
if (app.installpath or '').startswith(APP_PACKAGE_URL_PREFIX):
app.installpath = None
app.installersha256 = None
AuditLog.log('updated', 'Application', entityid=app_id,
entityname=app.appname,
changes={'revertedtorecord': {'new': True},
'removedfiles': removed})
db.session.commit()
return success_response(app.to_dict(), message='Application reverted to record')
# ---- Versions ----
@@ -613,14 +1009,43 @@ def delete_application_image(app_id: int):
return success_response(app.to_dict(), message='Application image removed')
def _store_package(app_id: int, upload):
"""Save an installer under its own vendor filename, one per application.
Keeping the shipped name matters at import: "DeviceLoggerSetup.msi" is
recognisable as the thing already in the catalogue, whereas the old
application-<id>.msi told the operator nothing and made a re-import look
like new software, which is how duplicate applications and versions got
created in the first place.
Files are namespaced by application id so two applications shipping a
generic "setup.exe" cannot overwrite each other. Any previous installer for
this application is removed first, so replacing one does not accumulate.
Returns (storedname, originalname).
"""
directory = os.path.join(_apppackage_dir(), str(app_id))
if os.path.isdir(directory):
shutil.rmtree(directory, ignore_errors=True)
os.makedirs(directory, exist_ok=True)
original = os.path.basename(upload.filename or '')
# secure_filename can reduce a name to nothing (all-unicode, all-dots), and
# an empty name would save as a bare directory entry.
stored = secure_filename(original) or 'installer{}'.format(
os.path.splitext(original)[1].lower())
upload.save(os.path.join(directory, stored))
return stored, original
@applications_bp.route('/<int:app_id>/package', methods=['POST'])
@require_permission('applications.edit')
def upload_application_package(app_id: int):
"""Upload (or replace) the installer for an application.
multipart/form-data: file=<installer>. The original filename is kept in
installpath so the download arrives named the way the vendor shipped it,
and the stored copy is application-<id><ext>.
multipart/form-data: file=<installer>. The installer is stored under the
filename the vendor shipped, inside a per-application directory.
"""
app = db.session.get(Application, app_id)
if not app:
@@ -650,8 +1075,11 @@ def upload_application_package(app_id: int):
'on the share and link to them with Install Path.'.format(
size / 1048576, MAX_PACKAGE_BYTES / 1048576))
filename = _replace_upload(_apppackage_dir(), f'application-{app_id}', ext, upload)
app.installpath = '{}{}'.format(APP_PACKAGE_URL_PREFIX, filename)
# Read the version before saving: the stream is consumed by save().
detectedversion = _detect_installer_version(upload.stream)
filename, _original = _store_package(app_id, upload)
app.installpath = '{}{}/{}'.format(APP_PACKAGE_URL_PREFIX, app_id, filename)
# The endpoint installer push pins bytes by digest; compute it once here so
# the server, not the agent, decides what the correct payload is. Stamp it
@@ -682,6 +1110,10 @@ def upload_application_package(app_id: int):
return success_response(
{**app.to_dict(), 'uploadedfilename': upload.filename, 'bytes': size,
'sha256': sha256,
# None means "no version resource" - the form must ask the operator
# rather than silently saving an installer with no version, which is
# how duplicate version rows appeared.
'detectedversion': detectedversion,
'appversionid': stamped_version.appversionid if stamped_version else None},
message='Installer uploaded')
@@ -698,6 +1130,10 @@ def serve_application_package(filename):
Gated on applications.download, not applications.view: a guest may be
granted the metadata permission, and "an open URL would publish it" is just
as true of a URL any anonymous visitor can reach.
filename is "<appid>/<vendorname>" now that installers keep the name they
shipped under. send_from_directory refuses to escape the base directory, so
the appid segment cannot be walked out of with "..".
"""
return send_upload(_apppackage_dir(), filename, as_attachment=True)
@@ -712,12 +1148,12 @@ def delete_application_package(app_id: int):
f'Application with ID {app_id} not found',
http_code=404)
for old in glob.glob(os.path.join(_apppackage_dir(),
secure_filename(f'application-{app_id}') + '.*')):
os.remove(old)
shutil.rmtree(os.path.join(_apppackage_dir(), str(app_id)),
ignore_errors=True)
# Only clear the path if it pointed at the upload; a site's own share path
# was typed by a person and is not ours to wipe.
if (app.installpath or '').startswith(APP_PACKAGE_URL_PREFIX):
app.installpath = None
app.installersha256 = None
db.session.commit()
return success_response(app.to_dict(), message='Installer removed')

27
shopdb/core/api/assets.py
View File

@@ -997,14 +997,6 @@ def get_assets_map():
)
except (ImportError, AttributeError):
pass
try:
from plugins.measuringtools.models import MeasuringTool
eager_options.append(
subqueryload(Asset.measuringtool)
.joinedload(MeasuringTool.measuringtooltype)
)
except (ImportError, AttributeError):
pass
query = Asset.query.options(*eager_options).filter(
Asset.isactive == True,
@@ -1041,7 +1033,7 @@ def get_assets_map():
# Filter by subtype (depends on asset type) - case-insensitive matching
if (subtype_id := intarg('subtype')) is not None:
# Normalize the underscore DB form (measuring_tool, network_device) to
# Normalize the underscore DB form (network_device) to
# the space form the branches below compare against.
asset_type_lower = (
selected_assettype.lower().replace('_', ' ') if selected_assettype else '')
@@ -1077,15 +1069,6 @@ def get_assets_map():
)
except ImportError:
pass
elif asset_type_lower == 'measuring tool':
try:
from plugins.measuringtools.models import MeasuringTool
query = query.join(
MeasuringTool, MeasuringTool.assetid == Asset.assetid).filter(
MeasuringTool.measuringtooltypeid == subtype_id
)
except ImportError:
pass
# Filter by business unit
if (bu_id := intarg('businessunitid')) is not None:
@@ -1224,14 +1207,6 @@ def get_assets_map():
except ImportError:
subtypes['Printer'] = []
try:
from plugins.measuringtools.models import MeasuringToolType
measuringtool_types = MeasuringToolType.query.filter(
MeasuringToolType.isactive == True).order_by(MeasuringToolType.name).all()
subtypes['Measuring Tool'] = [{'id': mt.measuringtooltypeid, 'name': mt.name, 'color': mt.color} for mt in measuringtool_types]
except ImportError:
subtypes['Measuring Tool'] = []
return success_response({
'assets': data,
'total': len(data),

23
shopdb/core/api/auditlogs.py
View File

@@ -13,27 +13,16 @@ auditlogs_bp = Blueprint('auditlogs', __name__)
def _resolve_full_names(ssos):
"""Best-effort SSO -> full name map for audit rows. Mode-aware, guarded,
degrades to {} so the list still renders if the directory is unreachable."""
"""Best-effort SSO -> full name map for audit rows. Guarded, degrades to {}
so the list still renders if the directory is unreachable."""
wanted = {s for s in ssos if s and str(s).isdigit()}
if not wanted:
return {}
names = {}
# Self-hosted directory first (no external dependency).
try:
from plugins.employees.models import DirectoryEmployee
rows = DirectoryEmployee.query.filter(
DirectoryEmployee.sso.in_(wanted)).all()
for emp in rows:
full = f'{(emp.firstname or "").strip()} {(emp.lastname or "").strip()}'.strip()
if full:
names[str(emp.sso)] = full
except Exception:
pass
missing = wanted - set(names)
if not missing:
return names
# External HR directory for anything still unresolved.
missing = set(wanted)
# External HR directory. Best effort: a site with EMPLOYEE_DB_* unset, or a
# directory that is down, gets bare SSOs rather than an error page over a
# log it can still read.
try:
from shopdb.utils.employee_db import employee_connection
conn = employee_connection()

145
shopdb/core/api/collector.py
View File

@@ -23,6 +23,48 @@ collector_bp = Blueprint('collector', __name__)
# identity - no require_permission on these routes).
COLLECTOR_SCOPE = 'collector.ingest'
# Per-PC agent credential (contract 0.33.0). Accepted only by collector schemas
# declaring acceptsagentcredential.
AGENT_CREDENTIAL_HEADER = 'X-Endpoint-Credential'
# How a collector request authorized, recorded on g so a route can tell a
# managed service token from the shared env key after the check has passed.
AUTH_MANAGED_TOKEN = 'token'
AUTH_SHARED_KEY = 'sharedkey'
# The four legacy routes below (/pc, /apps, /heartbeat, /bulk) predate both the
# generic collector contract and per-PC credentials. They take a hostname from
# the body and rewrite that PC's logged-in user, boot time, serial number and
# installed-app inventory, so the shared key is an any-host write key on them.
# plugin.endpoints.sharedkeyallowed does not reach here - it gates the endpoints
# collector, not core - so turning it off leaves these open. This setting is the
# core counterpart. Default false = today's behaviour, unchanged.
LEGACY_SHARED_KEY_SETTING = 'collector.legacysharedkeyrefused'
# Declared here so the wording stays with the code that enforces it.
# build_default_settings (shopdb/core/api/settings.py) has to include this dict
# for the row to be seeded; until it does, the key is created on demand through
# the settings API and the absent-row default below is what applies.
LEGACY_SHARED_KEY_SETTING_DEFAULT = {
'key': LEGACY_SHARED_KEY_SETTING,
'value': 'false',
'valuetype': 'boolean',
'category': 'collector',
'description': 'Refuse the shared collector key on the legacy core routes '
'POST /api/collector/pc, /apps, /heartbeat and /bulk. Those '
'routes name their target PC in the body, so any holder of '
'the shared key can rewrite logged-in user, last boot time, '
'serial number and installed apps for ANY hostname. Turning '
'this on closes them to the shared key; a collector-scoped '
'managed token still works. Turn it on only after checking '
'the "Legacy collector route used with the shared key" '
'warnings in the log for a while: whatever still calls them '
'stops working the moment it is on. The shipped reporters '
'post to /api/collector/computers and '
'/api/collector/printers and are NOT affected. This should '
'follow the move to per-PC credentials, not precede it.',
}
def _computer_models():
"""Lazily import the computers plugin models.
@@ -94,8 +136,8 @@ def _token_is_collector(token, pluginname=None):
def _collector_managed_token(api_key):
"""Find the managed token backing this collector request, or None.
Two transports are accepted (GE-Enforce sends X-API-Key today; Bearer is
the standard PAT wire form):
Two transports are accepted (a fleet reporter that can only set a fixed
header sends X-API-Key; Bearer is the standard PAT wire form):
X-API-Key: the before_request PAT shim ignores this header, so resolve the
secret here.
Bearer: the shim already resolved+swapped it (the raw Bearer secret is
@@ -127,13 +169,17 @@ def _check_collector_auth(expected_key, pluginname=None):
api_key = request.headers.get('X-API-Key')
g.collectorauth = None
token = _collector_managed_token(api_key)
if token is not None and _token_is_collector(token, pluginname):
touch_apitoken_lastused(token)
g.collectorauth = AUTH_MANAGED_TOKEN
return None
if expected_key and api_key == expected_key \
and not _env_fallback_disabled(pluginname):
g.collectorauth = AUTH_SHARED_KEY
return None
# Fail-closed only on pure server misconfiguration: no env key AND the
@@ -160,6 +206,54 @@ def require_api_key(f):
return decorated
def _legacy_shared_key_refused():
"""True when the site has closed the legacy routes to the shared key.
Absent setting = false = the routes stay open, which is what every site
does today. Read tolerantly because the row can be created by hand through
the settings API, where it may land with valuetype string.
"""
from shopdb.core.models.setting import Setting
value = Setting.get(LEGACY_SHARED_KEY_SETTING, default=False)
if isinstance(value, str):
return value.strip().lower() in ('true', '1', 'yes')
return bool(value)
def require_api_key_legacy(f):
"""require_api_key plus the legacy-route shared-key gate.
These routes take their target hostname from the body, so the shared key is
an any-host write key on them and no per-PC credential narrows it. A site
that has finished the move to per-PC credentials can shut it off here
without waiting for the routes to be retired.
Every shared-key use is logged whether the gate is on or off, so a site can
read the warnings and learn what still depends on the key BEFORE flipping
it - otherwise turning it on is a guess.
"""
from shopdb.utils.clientip import client_ip
@wraps(f)
def decorated(*args, **kwargs):
error = _check_collector_auth(current_app.config.get('COLLECTOR_API_KEY'))
if error is not None:
return error
if getattr(g, 'collectorauth', None) == AUTH_SHARED_KEY:
if _legacy_shared_key_refused():
return error_response(
'SHARED_KEY_REFUSED',
'The shared key is not accepted on the legacy collector '
'routes; use a collector-scoped token',
http_code=401)
current_app.logger.warning(
'Legacy collector route used with the shared key: %s from %s',
request.path, client_ip(request))
return f(*args, **kwargs)
return decorated
def _find_pc(hostname):
"""Find a computer by hostname, falling back to its asset number.
@@ -241,11 +335,28 @@ def generic_collect(pluginname):
plugin, schema = plugins[pluginname]
# Per-plugin env key (fallback) OR a collector-scoped managed token. Header
# only (querystring fallback dropped, see require_api_key).
error = _check_collector_auth(_plugin_api_key(pluginname), pluginname)
if error is not None:
return error
# A schema that accepts agent credentials takes X-Endpoint-Credential and
# derives the calling asset from it. Otherwise: per-plugin env key or a
# collector-scoped managed token, header only (see require_api_key).
from shopdb.api import resolve_agent_credential, agent_shared_key_refused
acceptscredential = bool(schema.get('acceptsagentcredential'))
g.agentauth = None
g.agentassetid = None
secret = (request.headers.get(AGENT_CREDENTIAL_HEADER) or '').strip()
if secret and acceptscredential:
assetid = resolve_agent_credential(secret)
if assetid is None:
return error_response('CREDENTIAL_INVALID',
'Endpoint credential is unknown or revoked; '
'enroll again', http_code=401)
g.agentauth = 'credential'
g.agentassetid = assetid
else:
error = _check_collector_auth(_plugin_api_key(pluginname), pluginname)
if error is not None:
return error
g.agentauth = 'sharedkey'
payload = request.get_json(silent=True)
if not payload or not isinstance(payload, dict):
@@ -257,6 +368,12 @@ def generic_collect(pluginname):
return error_response(ErrorCodes.VALIDATION_ERROR,
f'{identityfield} is required')
if (acceptscredential and g.agentauth == 'sharedkey'
and agent_shared_key_refused(identityvalue)):
return error_response('SHARED_KEY_REFUSED',
'The shared key is not accepted for this '
'endpoint; use its credential', http_code=401)
try:
outcome = plugin.apply_collector_payload(payload)
except NotImplementedError:
@@ -264,6 +381,12 @@ def generic_collect(pluginname):
ErrorCodes.INTERNAL_ERROR,
f'Plugin {pluginname} does not implement apply_collector_payload',
http_code=500)
except PermissionError as exc:
# The payload names a different asset than the agent credential.
db.session.rollback()
return error_response('ASSET_MISMATCH', str(exc) or
'The payload names a different endpoint than '
'the credential', http_code=403)
except ValueError as exc:
# ValueError is the plugin's controlled validation signal; its message
# is safe to return to the caller.
@@ -314,7 +437,7 @@ def collector_schemas():
@collector_bp.route('/pc', methods=['POST'])
@require_api_key
@require_api_key_legacy
def update_pc_info():
"""Update one PC from the collection script (matched by hostname)."""
data = request.get_json()
@@ -352,7 +475,7 @@ def update_pc_info():
@collector_bp.route('/apps', methods=['POST'])
@require_api_key
@require_api_key_legacy
def update_installed_apps():
"""Update installed applications for a PC (matched by hostname)."""
data = request.get_json()
@@ -424,7 +547,7 @@ def update_installed_apps():
@collector_bp.route('/heartbeat', methods=['POST'])
@require_api_key
@require_api_key_legacy
def pc_heartbeat():
"""Record PC online status / heartbeat (single or batch)."""
data = request.get_json()
@@ -459,7 +582,7 @@ def pc_heartbeat():
@collector_bp.route('/bulk', methods=['POST'])
@require_api_key
@require_api_key_legacy
def bulk_update():
"""Bulk update multiple PCs at once."""
data = request.get_json()

11
shopdb/core/api/dashboard.py
View File

@@ -16,7 +16,6 @@ _TYPE_CATEGORY = {
'computer': 'PC',
'printer': 'Printer',
'network_device': 'Network',
'measuring_tool': 'Measuring Tool',
}
@@ -51,15 +50,14 @@ def _inactive_excluded():
.count())
COUNTEDTYPES = ('machine', 'computer', 'network_device', 'printer',
'measuring_tool')
COUNTEDTYPES = ('machine', 'computer', 'network_device', 'printer')
def _countedassets():
"""Base query for the assets the dashboard totals describe.
The status tiles and the type tiles MUST count the same population. They
did not: the totals summed five specific types and subtracted dual-bay
did not: the totals summed the COUNTEDTYPES above and subtracted dual-bay
secondaries, while the status counts took every asset row of any type with
no collapse. So "all assets 704" sat beside "in use 737", and both were
correct about different things - which makes them worse than either alone.
@@ -90,8 +88,7 @@ def get_dashboard():
pc_count = _count_by_type('computer')
network_count = _count_by_type('network_device')
printer_count = _count_by_type('printer')
measuringtool_count = _count_by_type('measuring_tool')
total = machine_count + pc_count + network_count + printer_count + measuringtool_count
total = machine_count + pc_count + network_count + printer_count
# Count by status over the SAME population as the totals above, so the
# tiles can be read against each other.
@@ -113,7 +110,6 @@ def get_dashboard():
'totalpc': pc_count,
'totalnetwork': network_count,
'totalprinter': printer_count,
'totalmeasuringtool': measuringtool_count,
'activeassets': status_dict.get('In Use', 0),
'inrepair': status_dict.get('In Repair', 0),
# What the totals actually cover, so the page can say so rather than
@@ -138,7 +134,6 @@ def get_dashboard():
'pcs': pc_count,
'networkdevices': network_count,
'printers': printer_count,
'measuringtools': measuringtool_count,
'total': total
},
'bystatus': status_dict,

153
shopdb/core/api/docs.py
View File

@@ -4,11 +4,21 @@ Served at /api/docs (relative to the mount). The Redoc bundle is vendored in
staticdocs/ so this works fully offline on the air-gapped prod box - no CDN.
The spec is docs/openapi.json in the repo (regenerate with
scripts/gen_openapi.py after API changes).
Two renderers over ONE spec. Redoc is read-only by design and is the default,
because what is published is a contract. Swagger UI at /api/docs/try adds the
one thing Redoc will not do: fire a real request from the page, against this
server, with whatever session the browser is already carrying. That is its
whole value and the whole reason it is off unless a site turns it on - see
API_DOCS_TRYIT_ENABLED in shopdb/config.py.
"""
import json
import os
from flask import Blueprint, Response, send_file, url_for
from markupsafe import escape
from flask import Blueprint, Response, current_app, send_file, url_for
docs_bp = Blueprint('docs', __name__)
@@ -19,21 +29,71 @@ _SPEC = os.path.join(_REPO_ROOT, 'docs', 'openapi.json')
_LLMS = os.path.join(_REPO_ROOT, 'docs', 'llms.txt')
_REDOC = os.path.join(_HERE, 'staticdocs', 'redoc.standalone.js')
# Vendored from swagger-ui-dist 5.33.0. TWO files out of the ~30 in that dist,
# because those are the two the page below actually loads:
# swagger-ui.css - the stylesheet. Its images are inline data: URIs,
# so it pulls nothing else. The trailing
# sourceMappingURL comment was stripped: the .map
# is not vendored, and left in place it turns every
# devtools open into a 404 on a box with no
# internet to fall back on.
# swagger-ui-bundle.min.js - the renderer plus the core presets, self-
# contained. NOT vendored beside it:
# swagger-ui-standalone-preset.js (the topbar with
# the "load any spec URL" box - this page serves
# one spec on purpose), the ES bundles (module
# builds of the same code), the .map files (~10 MB
# of source maps that only devtools reads),
# oauth2-redirect.html (no OAuth flow here), and
# the favicons.
# The upstream name is swagger-ui-bundle.js; the vendored copy carries .min.js,
# which is what it is - a minified single-line build. That suffix is also what
# scripts/check-naming-and-style.sh already filters vendored bundles by, so the
# style gate does not report thousands of hits in code nobody here wrote.
# swagger-ui.LICENSE.txt is the Apache-2.0 text that ships with the dist. It is
# never served; it is here because redistributing the bundle requires it.
_SWAGGER_CSS = os.path.join(_HERE, 'staticdocs', 'swagger-ui.css')
_SWAGGER_JS = os.path.join(_HERE, 'staticdocs', 'swagger-ui-bundle.min.js')
_TRYIT_ENDPOINT = 'docs.swagger_tryit'
def _tryit_is_mounted():
"""True when the try-it page exists on THIS app.
Reads the url map rather than the config flag on purpose: the routes are
bound once, at blueprint registration, so a config value changed after boot
would have the Redoc page advertise a link that answers 404.
"""
return _TRYIT_ENDPOINT in current_app.view_functions
@docs_bp.route('/', strict_slashes=False)
def docs_index():
"""Redoc page. url_for keeps the asset URLs correct under any mount."""
spec_url = url_for('docs.openapi_spec')
redoc_url = url_for('docs.redoc_js')
# Mentioned only when it is actually there. An unconditional note would be
# an advert for a 404 on every box that ships the shipped default.
tryit_note = ''
if _tryit_is_mounted():
tryit_note = (
'<p style="font:14px/1.4 system-ui,sans-serif;margin:0;'
'padding:10px 16px;background:#eef2f7;border-bottom:1px solid #cbd5e1">'
'This page is read-only. To send a real request against this '
'server, use <a href="%s">the try-it view</a>.</p>'
% url_for(_TRYIT_ENDPOINT)
)
page = (
'<!doctype html><html><head>'
'<title>ShopDB API</title>'
'<meta charset="utf-8">'
'<meta name="viewport" content="width=device-width, initial-scale=1">'
'<style>body{margin:0;padding:0}</style></head><body>'
'%s'
'<redoc spec-url="%s"></redoc>'
'<script src="%s"></script>'
'</body></html>' % (spec_url, redoc_url)
'</body></html>' % (tryit_note, spec_url, redoc_url)
)
return Response(page, mimetype='text/html')
@@ -59,3 +119,92 @@ def llms_txt():
if not os.path.isfile(_LLMS):
return Response('llms.txt not found', status=404, mimetype='text/plain')
return send_file(_LLMS, mimetype='text/plain')
def swagger_tryit():
"""Swagger UI over the SAME spec the Redoc page renders.
Every URL on the page comes from url_for, so the spec and both assets stay
same-origin and correct under a mount path (prod runs under /shopdb).
"""
spec_url = url_for('docs.openapi_spec')
css_url = url_for('docs.swagger_css')
js_url = url_for('docs.swagger_js')
redoc_page_url = url_for('docs.docs_index')
page = (
'<!doctype html><html><head>'
'<title>ShopDB API - try it</title>'
'<meta charset="utf-8">'
'<meta name="viewport" content="width=device-width, initial-scale=1">'
'<link rel="stylesheet" href="%(css)s">'
'<style>body{margin:0;padding:0}'
'.tryit-warn{font:14px/1.4 system-ui,sans-serif;background:#7f1d1d;'
'color:#fff;margin:0;padding:10px 16px}'
'.tryit-warn a{color:#fff}</style>'
'</head><body>'
'<p class="tryit-warn"><b>Execute sends a real request to this '
'server</b>, as whoever is signed in on this browser. A write '
'operation changes live data. The read-only contract is at '
'<a href="%(redoc)s">/api/docs</a>.</p>'
'<div id="swagger-ui"></div>'
'<script src="%(js)s"></script>'
'<script>\n'
'window.ui = SwaggerUIBundle({\n'
# json.dumps, NOT a bare "%s". SCRIPT_NAME is proxy-controlled and
# ProxyFix honours X-Forwarded-Prefix, so a mount path carrying a quote
# closed this string literal and executed what followed: SCRIPT_NAME of
# /a"+alert(1)+" rendered url: "/a"+alert(1)+"/api/docs/openapi.json".
# Escaping it as JSON is what makes the value data rather than code.
' url: %(spec)s,\n'
' dom_id: "#swagger-ui",\n'
' presets: [SwaggerUIBundle.presets.apis],\n'
' layout: "BaseLayout",\n'
' deepLinking: true,\n'
' filter: true,\n'
# 573 operations. Expanded by default, the page spends many seconds
# laying out before it answers a scroll.
' docExpansion: "none",\n'
' withCredentials: true,\n'
# THE air-gap setting. Left unset, Swagger UI renders a "valid" badge
# by handing this spec to the public swagger validator - an outbound
# request that cannot succeed here, and that would ship the internal
# API surface off the box if it ever could.
' validatorUrl: null\n'
'});\n'
'</script>'
'</body></html>'
% {'css': escape(css_url), 'js': escape(js_url),
'spec': json.dumps(spec_url),
'redoc': escape(redoc_page_url)}
)
return Response(page, mimetype='text/html')
def swagger_css():
"""Vendored Swagger UI stylesheet (offline)."""
return send_file(_SWAGGER_CSS, mimetype='text/css')
def swagger_js():
"""Vendored Swagger UI bundle (offline)."""
return send_file(_SWAGGER_JS, mimetype='application/javascript')
@docs_bp.record_once
def _mount_tryit(state):
"""Bind the try-it routes only when the flag is on for this app.
Binding at registration rather than checking the flag inside the view, so
that when it is off the routes DO NOT EXIST: the URL gets Flask's own 404
and confirms nothing, and the unauthenticated-GET sweep in
tests/test_core/test_public_read_surface.py never has to make an exception
for three routes that are only meant to be reachable on a dev box.
"""
if not state.app.config.get('API_DOCS_TRYIT_ENABLED'):
return
state.add_url_rule('/try', endpoint='swagger_tryit',
view_func=swagger_tryit, strict_slashes=False)
state.add_url_rule('/swagger-ui.css', endpoint='swagger_css',
view_func=swagger_css)
state.add_url_rule('/swagger-ui-bundle.min.js', endpoint='swagger_js',
view_func=swagger_js)

278
shopdb/core/api/employees.py Normal file
View File

@@ -0,0 +1,278 @@
"""Read-only employee directory API, backed by the external HR database.
WHY this is core and not a plugin. ShopDB never owns the staff directory: the
rows live in a separate HR database a site points at with EMPLOYEE_DB_*, and
this app only reads them. Several core features already depend on that read -
global search serves an `employee` domain, the audit log resolves an SSO to a
name, the login store fills a display name - and a notification aimed at a
person needs a way to pick that person. A read this many core surfaces depend
on cannot be optional, so it lives here.
Read-only on purpose: no create, update, delete or photo upload. Those belong
to whatever system owns the directory. The only writeable thing in the
neighbourhood is the `employee_photo_base_url` setting, which says where a
relative Picture path resolves.
Every route is gated on `employees.view` (declared in Permission.CORE_PERMISSIONS)
and returns only the columns in _FIELDS.
"""
import logging
from flask import Blueprint, request, current_app
# Core imports core directly, NOT through shopdb.api: the contract surface
# re-exports the two resolvers below, so importing it back from here would be a
# cycle that only works by accident of line order.
from shopdb.core.models import Setting
from shopdb.utils.responses import (success_response, error_response,
ErrorCodes)
from shopdb.utils.authz import require_permission
from shopdb.utils.employee_db import employee_connection
from shopdb.utils.queryargs import intarg
logger = logging.getLogger(__name__)
employees_bp = Blueprint('employees', __name__)
# Columns safe to expose. Anything the HR table holds beyond these stays there.
_FIELDS = 'SSO, First_Name, Last_Name, Team, Role, Picture'
# Where a bare relative Picture path resolves when employee_photo_base_url is
# unset. A site whose photos live on another host sets that setting to a full
# URL instead.
EMPLOYEE_PHOTO_STATIC_PREFIX = '/static/employees/'
def _external_photo_url(picture):
"""Turn an HR Picture value into a usable URL, or None.
Absolute URLs and already-rooted paths pass through untouched; a bare
relative path (e.g. 'Support/210009518.png') resolves under the configured
base.
"""
if not picture:
return None
text = str(picture).strip()
if not text:
return None
if text.startswith(('http://', 'https://', '/')):
return text
base = (Setting.get('employee_photo_base_url') or '').strip() \
or EMPLOYEE_PHOTO_STATIC_PREFIX
return base.rstrip('/') + '/' + text.lstrip('/')
def _hr_picture(sso):
"""Raw Picture value for an SSO. None on any miss."""
try:
conn = employee_connection()
with conn.cursor() as cur:
cur.execute('SELECT Picture FROM employees WHERE SSO = %s', (int(sso),))
row = cur.fetchone()
conn.close()
return row.get('Picture') if row else None
except Exception:
return None
def resolve_employee_display_name(sso):
"""Display name ("First Last") for an SSO. None on any miss.
Used as a live fallback where only an SSO was stored - the shopfloor feed
does this for a notification imported without an employee name.
"""
if sso is None or not str(sso).isdigit():
return None
try:
conn = employee_connection()
with conn.cursor() as cur:
cur.execute(
'SELECT First_Name, Last_Name FROM employees WHERE SSO = %s',
(int(sso),))
row = cur.fetchone()
conn.close()
if row:
first = row.get('First_Name') or ''
last = row.get('Last_Name') or ''
name = f'{first.strip()} {last.strip()}'.strip()
if name:
return name
except Exception:
# Logged, not swallowed. This is the branch that fails on a live site -
# an unreachable HR host, a rotated credential, a renamed column - and a
# bare pass made all of those look identical to "no such employee": a
# bare SSO where a name should be, with nothing written down anywhere.
current_app.logger.exception(
'Employee directory lookup failed for SSO %s', sso)
# Fallback: a shopdb login account for this SSO (somebody the directory does
# not carry). Their name lives on the User record; usernames are the SSO.
try:
from shopdb.api import User
user = User.query.filter_by(username=str(sso)).first()
if user:
name = f'{(user.firstname or "").strip()} {(user.lastname or "").strip()}'.strip()
if name:
return name
except Exception:
current_app.logger.exception(
'User-account name fallback failed for SSO %s', sso)
return None
def resolve_employee_photo_url(sso, external_picture=None):
"""Display photo URL for an SSO, or None.
Pass external_picture when the caller already has the row, to avoid a
second query.
"""
if sso is None or not str(sso).isdigit():
return None
picture = external_picture if external_picture is not None else _hr_picture(sso)
return _external_photo_url(picture)
def _with_photo_url(employee):
"""Add the resolved photourl to an employee row dict."""
employee['photourl'] = resolve_employee_photo_url(
employee.get('SSO'), employee.get('Picture'))
return employee
@employees_bp.route('/search', methods=['GET'])
@require_permission('employees.view')
def search_employees():
"""Search the directory by name or SSO.
Query parameters:
- q: search query, at least 2 characters
- limit: max results (default 10, capped at 50)
"""
query = request.args.get('q', '').strip()
limit = min(intarg('limit', 10), 50)
if len(query) < 2:
return error_response(
ErrorCodes.VALIDATION_ERROR,
'Search query must be at least 2 characters'
)
# A multi-word query is a FULL NAME. Each word used to be matched against
# first name or last name on its own, so "Dana Reyes" found nobody: no
# single column contains that string. Somebody typing the whole name - what
# a person does before pressing Enter, rather than picking from the list -
# got an empty result and their entry became free text linked to nobody.
words = query.split()
try:
conn = employee_connection()
with conn.cursor() as cur:
if len(words) > 1:
clauses = ' AND '.join(
['(First_Name LIKE %s OR Last_Name LIKE %s)'] * len(words))
params = []
for word in words:
params.extend([f'%{word}%', f'%{word}%'])
params.append(limit)
cur.execute(f'''
SELECT {_FIELDS}
FROM employees
WHERE {clauses}
ORDER BY Last_Name, First_Name
LIMIT %s
''', tuple(params))
else:
cur.execute(f'''
SELECT {_FIELDS}
FROM employees
WHERE First_Name LIKE %s
OR Last_Name LIKE %s
OR CAST(SSO AS CHAR) LIKE %s
ORDER BY Last_Name, First_Name
LIMIT %s
''', (f'%{query}%', f'%{query}%', f'%{query}%', limit))
employees = cur.fetchall()
conn.close()
return success_response([_with_photo_url(e) for e in employees])
except Exception:
logger.exception('Employee search failed')
return error_response(
ErrorCodes.INTERNAL_ERROR,
'Employee lookup failed',
http_code=500
)
@employees_bp.route('/lookup/<sso>', methods=['GET'])
@require_permission('employees.view')
def lookup_employee(sso):
"""Look up a single employee by SSO."""
if not sso.isdigit():
return error_response(ErrorCodes.VALIDATION_ERROR, 'SSO must be numeric')
try:
conn = employee_connection()
with conn.cursor() as cur:
cur.execute(
f'SELECT {_FIELDS} FROM employees WHERE SSO = %s', (int(sso),))
employee = cur.fetchone()
conn.close()
if not employee:
return error_response(
ErrorCodes.NOT_FOUND,
f'Employee with SSO {sso} not found',
http_code=404
)
return success_response(_with_photo_url(employee))
except Exception:
logger.exception('Employee lookup failed for SSO %s', sso)
return error_response(
ErrorCodes.INTERNAL_ERROR,
'Employee lookup failed',
http_code=500
)
@employees_bp.route('/lookup', methods=['GET'])
@require_permission('employees.view')
def lookup_employees():
"""Look up several employees at once.
Query parameters:
- sso: comma-separated list of SSOs
"""
sso_list = request.args.get('sso', '')
ssos = [s.strip() for s in sso_list.split(',') if s.strip().isdigit()]
if not ssos:
return error_response(
ErrorCodes.VALIDATION_ERROR,
'At least one valid SSO is required'
)
try:
conn = employee_connection()
with conn.cursor() as cur:
placeholders = ','.join(['%s'] * len(ssos))
cur.execute(
f'SELECT {_FIELDS} FROM employees WHERE SSO IN ({placeholders})',
[int(s) for s in ssos]
)
employees = cur.fetchall()
conn.close()
employees = [_with_photo_url(e) for e in employees]
names = ', '.join(
f"{(e['First_Name'] or '').strip()} {(e['Last_Name'] or '').strip()}".strip()
for e in employees
)
return success_response({'employees': employees, 'names': names})
except Exception:
logger.exception('Employee multi-lookup failed')
return error_response(
ErrorCodes.INTERNAL_ERROR,
'Employee lookup failed',
http_code=500
)

27
shopdb/core/api/models.py
View File

@@ -2,6 +2,7 @@
import glob
import os
from urllib.parse import urlparse
from flask import Blueprint, request, current_app
from flask_jwt_extended import jwt_required
@@ -39,6 +40,16 @@ def _modelimage_dir():
return os.path.join(current_app.instance_path, 'modelimages')
def _validate_url_scheme(url):
"""Validate that URL uses safe scheme (prevents XSS via javascript: URIs)."""
if not url:
return True
parsed = urlparse(url)
if parsed.scheme and parsed.scheme not in ['http', 'https']:
raise ValueError(f"Invalid URL scheme '{parsed.scheme}'. Only http:// and https:// are allowed.")
return True
@models_bp.route('', methods=['GET'])
@require_permission('assets.view')
def list_models():
@@ -127,6 +138,13 @@ def create_model():
http_code=409
)
# Validate URL scheme to prevent XSS
try:
_validate_url_scheme(data.get('documentationurl'))
_validate_url_scheme(data.get('imageurl'))
except ValueError as e:
return error_response(ErrorCodes.VALIDATION_ERROR, str(e))
m = Model(
modelnumber=data['modelnumber'],
vendorid=data.get('vendorid'),
@@ -162,6 +180,15 @@ def update_model(model_id: int):
if not data:
return error_response(ErrorCodes.VALIDATION_ERROR, 'No data provided')
# Validate URL schemes to prevent XSS
try:
if 'documentationurl' in data:
_validate_url_scheme(data['documentationurl'])
if 'imageurl' in data:
_validate_url_scheme(data['imageurl'])
except ValueError as e:
return error_response(ErrorCodes.VALIDATION_ERROR, str(e))
for key in ['modelnumber', 'vendorid', 'modeltypeid', 'description', 'imageurl', 'documentationurl', 'notes', 'isactive']:
if key in data:
setattr(m, key, data[key])

53
shopdb/core/api/plugins.py
View File

@@ -3,6 +3,7 @@
from flask import Blueprint, current_app, request
from flask_jwt_extended import jwt_required
from shopdb.plugins.base import renderable_settings_routes
from shopdb.utils.responses import success_response, error_response, ErrorCodes
from shopdb.utils.authz import require_role
@@ -42,6 +43,58 @@ def list_enabled_plugins():
return success_response(sorted(names))
@plugins_bp.route('/settings-routes', methods=['GET'])
@require_permission('settings.view')
def get_plugin_settings_routes():
"""Return settings routes contributed by all enabled plugins.
Plugins can register routes to add tabs/pages to settings hubs like
Model Catalog, Asset Config, etc. Frontend router merges these dynamically.
Gated on settings.view, the same permission list_plugins carries. It was
optional-auth and listed as a public read, on the stated reasoning that
unauthenticated pages (kiosks, TV displays) register plugin tabs before
login. Nothing does: the only caller in the frontend is
views/settings/ModelCatalogHub.vue, which sits behind requiresAuth and
requiresAdmin, and no kiosk route touches it. Open, it handed an anonymous
caller the enabled plugin list, the settings hub layout and the internal Vue
component paths - free reconnaissance about which modules a site runs. The
login page never calls this, so gating it costs nothing pre-login.
A route naming a hub that has no code to draw plugin tabs is DROPPED here,
loudly. Returning it would put an entry in a response that nothing reads:
the plugin author sees a correct declaration and an API that accepts it, and
the tab never appears anywhere. Dropping it costs the same missing tab and
leaves a line in the log that says which plugin, which hub, and what the
supported set is. See SETTINGS_HUBS_WITH_PLUGIN_TABS in shopdb/plugins/base.
Returns: { routes: [{hub, path, name, tab, component}, ...] }
"""
pm = current_app.extensions.get('plugin_manager')
if not pm:
return success_response({'routes': []})
routes = []
for plugin_name, plugin in pm.get_all_plugins().items():
# Only include routes from enabled plugins
if not pm.registry.is_enabled(plugin_name):
continue
if not hasattr(plugin, 'get_settings_routes'):
continue
# One broken plugin must not take every settings hub down with it.
try:
plugin_routes = plugin.get_settings_routes()
except Exception:
current_app.logger.exception(
'get_settings_routes failed for plugin %s; skipping it',
plugin_name)
continue
routes.extend(renderable_settings_routes(plugin_name, plugin_routes,
current_app.logger))
return success_response({'routes': routes})
@plugins_bp.route('/<name>', methods=['PUT'])
@jwt_required()
@require_role('admin')

110
shopdb/core/api/pluginui.py
View File

@@ -25,7 +25,14 @@ pluginui_bp = Blueprint('pluginui', __name__)
#
# 'details' is first and is the fallback: it holds what an asset IS, which is
# what a panel declaring nothing almost always contributes.
ASSET_PANEL_GROUPS = ('details', 'monitors', 'software', 'activity')
#
# This tuple is a THIRD copy of the tab vocabulary - RENDERED_GROUPS in
# PluginAssetPanels.vue and the tab list in PCDetail.vue are the other two.
# Adding a group means touching all three: a value missing here is rewritten to
# 'details' before the client ever sees it, so the panels quietly render in the
# wrong tab and the intended tab never appears. No error, no log line.
ASSET_PANEL_GROUPS = ('details', 'monitors', 'serviceprocess', 'reports',
'software', 'activity')
def _collect(hookname):
@@ -124,3 +131,104 @@ def asset_presentation():
"""
entries = _collect('get_asset_presentation')
return success_response(entries)
@pluginui_bp.route('/asset-type-descriptors', methods=['GET'])
@require_permission('assets.view')
def asset_type_descriptors():
"""Merge enabled plugins' asset-type subtype descriptors.
The Asset Type Hub uses these to render generic CRUD interfaces for
each plugin's subtype table (printertypes, machinetypes, etc.) without
importing plugin models or views.
Each descriptor maps to ONE asset type (from assettypes table) and
tells core:
- Which API endpoint handles the subtype CRUD
- Field names for ID, label, description, color
- Required permission
- Optional plugin-specific tabs (e.g., 'supplies' for printers)
Returns: List[Dict] sorted by assettype name
"""
pm = current_app.extensions.get('plugin_manager')
if not pm:
return success_response([])
descriptors = []
for name, plugin in pm.get_all_plugins().items():
if not pm.registry.is_enabled(name):
continue
try:
desc = plugin.get_asset_type_descriptor()
if desc is None:
continue
desc['plugin'] = name
descriptors.append(desc)
except Exception:
if current_app.config.get('DEBUG') or current_app.config.get('TESTING'):
raise
current_app.logger.exception('Plugin %s get_asset_type_descriptor failed', name)
# Validation: each descriptor must have required keys
required_keys = {'assettype', 'subtype_endpoint', 'subtype_id_field',
'subtype_label_field', 'subtype_description_field',
'subtype_color_field', 'edit_permission'}
validated = []
for desc in descriptors:
missing = required_keys - set(desc.keys())
if missing:
current_app.logger.error(
'Plugin %s descriptor missing keys: %s',
desc.get('plugin', 'unknown'), missing
)
if current_app.config.get('DEBUG') or current_app.config.get('TESTING'):
raise ValueError(f"Descriptor missing keys: {missing}")
continue
validated.append(desc)
validated.sort(key=lambda d: d.get('assettype', ''))
return success_response(validated)
@pluginui_bp.route('/core-type-descriptors', methods=['GET'])
@require_permission('assets.view')
def core_type_descriptors():
"""Return type descriptors for core-owned non-asset types.
These are types managed by core (not plugins) that fit the same
management pattern as asset subtypes: location types, notification types, etc.
Returns same descriptor format as asset-type-descriptors, with optional
'custom_component' field for types that need specialized UI.
"""
descriptors = [
{
'category': 'location',
'type_name': 'Location Type',
'type_name_plural': 'Location Types',
'subtype_endpoint': '/locations/types',
'subtype_id_field': 'locationtypeid',
'subtype_label_field': 'locationtype',
'subtype_description_field': 'description',
'subtype_color_field': 'color',
'optional_tabs': [],
'edit_permission': 'admin',
'group': 'other',
},
{
'category': 'notification',
'type_name': 'Notification Type',
'type_name_plural': 'Notification Types',
'subtype_endpoint': '/notifications/types',
'subtype_id_field': 'notificationtypeid',
'subtype_label_field': 'typename',
'subtype_description_field': 'typedescription',
'subtype_color_field': 'typecolor',
'optional_tabs': [],
'edit_permission': 'admin',
'group': 'other',
'custom_component': 'NotificationTypesTab',
},
]
return success_response(descriptors)

172
shopdb/core/api/scripts.py
View File

@@ -6,16 +6,33 @@ its body by digest on push (the same model as application installer delivery).
The body never crosses the command wire and is never pre-staged on the endpoint;
the agent downloads it with its collector credential and verifies sha256.
Gated on the applications.* permissions - scripts are software delivery like
apps, so they ride the same operator grant rather than a separate permission.
Gated on endpoints.runscript, the same standalone grant that authorizes pushing
a script. These are NOT applications: an application installer is a vendor
artefact, while a script here is arbitrary PowerShell this site wrote that the
agent runs as SYSTEM. Gating library management on applications.* meant anyone
with the software-delivery grant could rewrite the bytes of a script a trusted
operator later pushes, which is a privilege escalation wearing a content-
management hat. The write and the push now require the same trust.
This TIGHTENS access, and there is deliberately NO backfill migration. The
operator scope grants applications.create/edit (they are edit verbs), so
operator-scope holders could reach this library before and cannot now. Do not
"fix" that by granting endpoints.runscript to applications.edit holders on
upgrade: it is in SCOPE_NEVER_GRANT precisely because it is a per-user checkbox
decision, and the same grant also authorizes the remote PUSH, so a backfill
would hand every operator SYSTEM code execution on the fleet. Admins are
unaffected via the role bypass; anyone else who genuinely needs the library
gets the checkbox in the user editor.
"""
import os
import re
from flask import Blueprint, request, current_app
from shopdb.extensions import db
from shopdb.core.models import Script, RUNCONTEXTS, RUNCONTEXT_SYSTEM, AuditLog
from shopdb.core.models import (Script, RUNCONTEXTS, RUNCONTEXT_SYSTEM,
ERRORMODES, ERRORMODE_DEFAULT, AuditLog)
from shopdb.utils.responses import (
success_response,
error_response,
@@ -32,6 +49,11 @@ SCRIPT_EXTENSIONS = {'.ps1'}
SCRIPT_URL_PREFIX = '/api/scripts/body/'
# A delivery script is small; cap well under the installer limit.
MAX_SCRIPT_BYTES = 1 * 1024 * 1024
# Owned by the endpoints plugin, which is where the risk is described in full
# ("ARBITRARY CODE, executes as SYSTEM"). Core names it as a string rather than
# importing it so this module stays importable with the plugin absent - the
# grant simply cannot be held by anyone if nothing seeds it.
SCRIPT_PERMISSION = 'endpoints.runscript'
def _script_dir():
@@ -42,7 +64,7 @@ scripts_bp = Blueprint('scripts', __name__)
@scripts_bp.route('', methods=['GET'])
@require_permission('applications.view')
@require_permission(SCRIPT_PERMISSION)
def list_scripts():
"""List scripts (active by default)."""
query = Script.query
@@ -55,7 +77,7 @@ def list_scripts():
@scripts_bp.route('/<int:script_id>', methods=['GET'])
@require_permission('applications.view')
@require_permission(SCRIPT_PERMISSION)
def get_script(script_id: int):
script = db.session.get(Script, script_id)
if not script:
@@ -73,8 +95,92 @@ def _validate_runcontext(value):
return value, None
def _validate_errormode(value):
"""Normalize/validate an errormode, defaulting to permissive when absent.
Defaults to ERRORMODE_DEFAULT rather than strict on purpose. Strict promotes
PowerShell's non-terminating errors to terminating ones, so a body that was
written to tolerate a failing cleanup or to probe with a cmdlet that errors
when the target is absent would begin aborting partway through. An absent
field therefore means "what this script already did", never a silent upgrade.
"""
if value is None:
return ERRORMODE_DEFAULT, None
value = str(value).strip().lower()
if value not in ERRORMODES:
return None, 'errormode must be one of: {}'.format(', '.join(sorted(ERRORMODES)))
return value, None
def _validate_captureoutput(value):
"""Normalize the "send output back on success" flag. Returns (value, error).
Absent means False: a script authored before this field existed never
returned its successful output to the server, and reading an omitted field
as True would start persisting output from bodies nobody reviewed with that
question in mind. Strings are accepted because the field arrives from a
form control that may send 'true'/'false' rather than a JSON boolean.
"""
if value is None:
return False, None
if isinstance(value, bool):
return value, None
normalized = str(value).strip().lower()
if normalized in ('true', '1', 'yes'):
return True, None
if normalized in ('false', '0', 'no', ''):
return False, None
return None, 'captureoutput must be true or false'
def _validate_workingdirectory(value):
"""Normalize the optional folder a script runs in. Returns (value, error).
Empty means "the agent's staging directory", which is the behaviour every
script had before this field existed. The purpose is to let a script sit
beside a corporate tool already installed on the endpoint and call it by
relative path.
The checks are shape checks, NOT a privilege boundary - a script body is
already arbitrary SYSTEM PowerShell and can change its own directory. They
exist so a typo fails here with an explanation instead of on the endpoint:
- must be an absolute local path (a drive letter), because a relative one
is meaningless without knowing what it would be relative TO
- no UNC. The stated use case is a tool already on this machine; a UNC
working directory resolves as the MACHINE ACCOUNT against a file server,
which moves where the binaries come from and is a different trust story
than "Intune put this here"
- no traversal segment, which only ever obscures what the path really is
"""
if value is None:
return None, None
value = str(value).strip().strip('"')
if not value:
return None, None
if value.startswith('\\\\') or value.startswith('//'):
return None, ('workingdirectory must be a local path on the endpoint, '
'not a UNC share')
normalized = value.replace('/', '\\')
if not re.match(r'^[A-Za-z]:\\', normalized):
return None, ('workingdirectory must be an absolute path including a '
r'drive letter, for example C:\Program Files\Sysinternals')
if '..' in normalized.split('\\'):
return None, 'workingdirectory must not contain a ".." segment'
if len(normalized) > 255:
return None, 'workingdirectory must be 255 characters or fewer'
# Trailing separator is noise, EXCEPT at a drive root: "C:\" trimmed to
# "C:" is drive-RELATIVE on Windows and would resolve to wherever the
# process happened to be on that drive, which is the opposite of pinning a
# folder. Keep the root form intact.
trimmed = normalized.rstrip('\\')
if len(trimmed) < 3:
trimmed = normalized[:3]
return trimmed, None
@scripts_bp.route('', methods=['POST'])
@require_permission('applications.create')
@require_permission(SCRIPT_PERMISSION)
def create_script():
"""Create a script catalog entry (body uploaded separately)."""
data = request.get_json(silent=True) or {}
@@ -85,6 +191,15 @@ def create_script():
return error_response(ErrorCodes.VALIDATION_ERROR,
f'A script named {name!r} already exists')
runcontext, err = _validate_runcontext(data.get('runcontext'))
if err:
return error_response(ErrorCodes.VALIDATION_ERROR, err)
workingdirectory, err = _validate_workingdirectory(data.get('workingdirectory'))
if err:
return error_response(ErrorCodes.VALIDATION_ERROR, err)
errormode, err = _validate_errormode(data.get('errormode'))
if err:
return error_response(ErrorCodes.VALIDATION_ERROR, err)
captureoutput, err = _validate_captureoutput(data.get('captureoutput'))
if err:
return error_response(ErrorCodes.VALIDATION_ERROR, err)
@@ -92,18 +207,24 @@ def create_script():
scriptname=name,
scriptdescription=data.get('scriptdescription'),
runcontext=runcontext,
workingdirectory=workingdirectory,
errormode=errormode,
captureoutput=captureoutput,
)
db.session.add(script)
db.session.commit()
AuditLog.log('created', 'Script', entityid=script.scriptid,
entityname=name,
changes={'runcontext': {'new': runcontext}})
changes={'runcontext': {'new': runcontext},
'workingdirectory': {'new': workingdirectory},
'errormode': {'new': errormode},
'captureoutput': {'new': captureoutput}})
db.session.commit()
return success_response(script.to_dict(), message='Script created', http_code=201)
@scripts_bp.route('/<int:script_id>', methods=['PUT'])
@require_permission('applications.edit')
@require_permission(SCRIPT_PERMISSION)
def update_script(script_id: int):
script = db.session.get(Script, script_id)
if not script:
@@ -127,6 +248,29 @@ def update_script(script_id: int):
if err:
return error_response(ErrorCodes.VALIDATION_ERROR, err)
script.runcontext = runcontext
if 'workingdirectory' in data:
workingdirectory, err = _validate_workingdirectory(
data.get('workingdirectory'))
if err:
return error_response(ErrorCodes.VALIDATION_ERROR, err)
script.workingdirectory = workingdirectory
if 'errormode' in data:
errormode, err = _validate_errormode(data.get('errormode'))
if err:
return error_response(ErrorCodes.VALIDATION_ERROR, err)
script.errormode = errormode
# Audited on change: this decides whether a successful run's output is
# persisted server-side, so who turned it on is worth being able to answer.
if 'captureoutput' in data:
captureoutput, err = _validate_captureoutput(data.get('captureoutput'))
if err:
return error_response(ErrorCodes.VALIDATION_ERROR, err)
if captureoutput != script.captureoutput:
AuditLog.log('updated', 'Script', entityid=script.scriptid,
entityname=script.scriptname,
changes={'captureoutput': {'old': script.captureoutput,
'new': captureoutput}})
script.captureoutput = captureoutput
if 'isactive' in data:
script.isactive = bool(data.get('isactive'))
@@ -135,7 +279,7 @@ def update_script(script_id: int):
@scripts_bp.route('/<int:script_id>', methods=['DELETE'])
@require_permission('applications.delete')
@require_permission(SCRIPT_PERMISSION)
def delete_script(script_id: int):
script = db.session.get(Script, script_id)
if not script:
@@ -149,7 +293,7 @@ def delete_script(script_id: int):
@scripts_bp.route('/<int:script_id>/body', methods=['POST'])
@require_permission('applications.edit')
@require_permission(SCRIPT_PERMISSION)
def upload_script_body(script_id: int):
"""Upload (or replace) the .ps1 body for a script.
@@ -196,15 +340,15 @@ def upload_script_body(script_id: int):
@scripts_bp.route('/body/<path:filename>', methods=['GET'])
@require_permission('applications.download')
@require_permission(SCRIPT_PERMISSION)
def download_script_body(filename: str):
"""Operator-facing script body download (JWT/RBAC).
The endpoint agent uses the collector-authenticated plugin route instead;
this one is for humans reviewing what a script does.
Gated on applications.download, not applications.view: this serves the
SYSTEM-context PowerShell a site pushes to its bays, and a guest may be
granted the metadata permission.
Same grant as the rest of the library: this serves the SYSTEM-context
PowerShell a site pushes to its bays, so reading it is not a lesser act
than listing it.
"""
return send_upload(_script_dir(), filename, as_attachment=True)

176
shopdb/core/api/search.py
View File

@@ -212,8 +212,6 @@ def _get_asset_result(asset, query, relevance=None):
plugin_id = asset.network_device.networkdeviceid
elif asset_type_name == 'printer' and hasattr(asset, 'printer') and asset.printer:
plugin_id = asset.printer.printerid
elif asset_type_name == 'measuring_tool' and hasattr(asset, 'measuringtool') and asset.measuringtool:
plugin_id = asset.measuringtool.measuringtoolid
# Prefer a plugin's declared get_asset_presentation route (ADR-010): core
# stops hardcoding the plugin's route as each plugin declares one. Those
@@ -229,7 +227,6 @@ def _get_asset_result(asset, query, relevance=None):
'computer': f"/pcs/{plugin_id}",
'network_device': f"/network/{plugin_id}",
'printer': f"/printers/{plugin_id}",
'measuring_tool': f"/measuringtools/{plugin_id}",
}
url = url_map.get(asset_type_name, f"/assets/{asset.assetid}")
result_id = plugin_id
@@ -420,24 +417,13 @@ def _search_knowledgebase(query, search_term):
def _employee_rows(query):
"""Employee rows matching search_term, from the SELFHOSTED directory table
or the external HR DB per the employee_directory_mode setting. Returns dicts
shaped like the external query (SSO/First_Name/Last_Name/Team/Role)."""
settings = get_cached_settings()
mode = (settings.get('employee_directory_mode') or 'external').lower()
if mode == 'selfhosted':
try:
from plugins.employees.models import DirectoryEmployee
except ImportError:
return []
rows = DirectoryEmployee.query.filter(
_word_match(query, DirectoryEmployee.firstname,
DirectoryEmployee.lastname,
db.cast(DirectoryEmployee.sso, db.String))
).order_by(DirectoryEmployee.lastname, DirectoryEmployee.firstname).limit(10).all()
return [{'SSO': e.sso, 'First_Name': e.firstname or '',
'Last_Name': e.lastname or '', 'Team': e.team, 'Role': e.role}
for e in rows]
"""Employee rows matching search_term, read from the external HR database.
There is one mode, not two. A self-hosted directory table used to be the
alternative; it was never core, and with no plugin left to own that table
the branch could only ever return nothing. Returns dicts shaped like the
external query (SSO/First_Name/Last_Name/Team/Role).
"""
# External HR DB: shared env-backed connection helper; never hardcode creds.
# Word-AND so "First Last" matches (each word in any name/SSO field), not just
# the exact contiguous phrase. Parameterized - one LIKE triple per word.
@@ -461,7 +447,7 @@ def _employee_rows(query):
def _search_employees(query, search_term):
"""Search employees (selfhosted directory or external HR DB, per mode)."""
"""Search employees, read from the external HR database."""
results = []
try:
for emp in _employee_rows(query):
@@ -545,144 +531,7 @@ def _search_assets(query, search_term):
return results
def _search_measuringtools(query, search_term):
"""Search measuring tools, including the gaugelabreference identifier.
The generic asset search already matches number/name/serial across all asset
types; this gated searcher adds gaugelabreference (the gage tag the gage lab
searches by) and drops out when the measuringtools plugin is disabled.
"""
results = []
try:
_require_enabled('measuringtools')
from plugins.measuringtools.models import MeasuringTool
assets = db.session.query(Asset).join(
MeasuringTool, MeasuringTool.assetid == Asset.assetid
).options(
joinedload(Asset.assettype),
joinedload(Asset.location),
).filter(
Asset.isactive == True,
_word_match(query, Asset.assetnumber, Asset.name,
Asset.serialnumber, Asset.gaugelabreference)
).order_by(Asset.assetnumber, Asset.assetid).limit(15).all()
for asset in assets:
relevance = 15
if asset.assetnumber and query.lower() == asset.assetnumber.lower():
relevance = 100
elif asset.gaugelabreference and query.lower() == asset.gaugelabreference.lower():
relevance = 90
elif asset.serialnumber and query.lower() == asset.serialnumber.lower():
relevance = 85
elif asset.name and query.lower() in asset.name.lower():
relevance = 50
results.append(_get_asset_result(asset, query, relevance))
except ImportError:
pass # measuringtools plugin absent or disabled
except Exception as e:
_notefailure("Measuring tool", e)
return results
def _search_usbdevices(query, search_term):
"""Search USB devices by serial, asset tag and product name.
A USB device is NOT an asset - it lives in the usb plugin's own table - so
the generic asset search cannot see it and these records were unreachable
from search entirely. Serial number is the field people actually have in
hand: it is what is printed on the stick they are holding.
currentusername is deliberately NOT searched. It records who holds the
device, and making search a way to list what a named person has checked out
is a different feature from finding a device.
"""
results = []
try:
_require_enabled('usb')
from plugins.usb.models import USBDevice
devices = USBDevice.query.filter(
USBDevice.isactive == True,
_word_match(query, USBDevice.serialnumber, USBDevice.assetnumber,
USBDevice.label, USBDevice.productname)
).order_by(USBDevice.serialnumber,
USBDevice.usbdeviceid).limit(10).all()
for device in devices:
relevance = 20
if query.lower() == (device.serialnumber or '').lower():
relevance = 100
elif query.lower() == (device.assetnumber or '').lower():
relevance = 90
elif query.lower() == (device.label or '').lower():
relevance = 85
elif query.lower() in (device.label or '').lower():
relevance = 50
elif query.lower() in (device.productname or '').lower():
relevance = 40
results.append({
'type': 'usb_device',
'id': device.usbdeviceid,
'title': device.label or device.productname or device.serialnumber,
'subtitle': device.assetnumber or device.serialnumber,
'url': f'/usb/{device.usbdeviceid}',
'relevance': relevance,
})
except ImportError:
pass # usb plugin absent or disabled
except Exception as e:
_notefailure("USB device", e)
return results
def _search_printeditems(query, search_term):
"""Search printed items by bin code, gage-lab tag, name and description.
Printed items are their own records, not assets, so the generic asset search
never covered them. itemcode (the bin label, e.g. 3DP-0042) and gagelabtag
are both unique and both printed on physical labels, which makes them the
likeliest thing anyone types into search.
"""
results = []
try:
_require_enabled('printedparts')
from plugins.printedparts.models import PrintedItem
items = PrintedItem.query.filter(
PrintedItem.isactive == True,
_word_match(query, PrintedItem.itemcode, PrintedItem.gagelabtag,
PrintedItem.itemname, PrintedItem.itemdescription)
).order_by(PrintedItem.itemcode,
PrintedItem.printeditemid).limit(10).all()
for item in items:
relevance = 20
if query.lower() == (item.itemcode or '').lower():
relevance = 100
elif query.lower() == (item.gagelabtag or '').lower():
relevance = 95
elif query.lower() == (item.itemname or '').lower():
relevance = 90
elif query.lower() in (item.itemname or '').lower():
relevance = 50
subtitle = item.itemcode or item.gagelabtag
if item.binlocation:
subtitle = f'{subtitle} - {item.binlocation}' if subtitle else item.binlocation
results.append({
'type': 'printed_item',
'id': item.printeditemid,
'title': item.itemname,
'subtitle': subtitle,
'url': f'/printedparts/{item.printeditemid}',
'relevance': relevance,
})
except ImportError:
pass # printedparts plugin absent or disabled
except Exception as e:
_notefailure("Printed item", e)
return results
def _search_customfields(query, search_term):
@@ -1255,9 +1104,9 @@ def global_search():
# Every domain answers only to a caller who holds ITS permission.
#
# The route is gated on assets.view alone, and that used to be the whole
# check: one permission then queried twelve domains, so it read out of the
# staff directory, the USB checkout history, the notification log and the
# network inventory to anybody who could look up an asset number. The
# check: one permission then queried every domain at once, so it read out of
# the staff directory, the notification log and the network inventory to
# anybody who could look up an asset number. The
# SEARCH_DOMAINS settings are feature toggles a site sets, not authorization,
# and were never doing this job.
#
@@ -1274,9 +1123,6 @@ def global_search():
('kb.view', _search_knowledgebase),
('guides.view', _search_guides),
('employees.view', _search_employees),
('measuringtools.view', _search_measuringtools),
('usb.view', _search_usbdevices),
('printedparts.view', _search_printeditems),
('notifications.view', _search_notifications),
('computers.view', _search_hostnames),
]

232
shopdb/core/api/settings.py
View File

@@ -127,8 +127,7 @@ IDENTIFIER_LABELS = {
'maintenancereference': 'Maintenance Reference',
'fqdn': 'FQDN / hostname',
}
IDENTIFIER_ASSETTYPES = ['machine', 'computer', 'printer', 'network_device',
'measuring_tool']
IDENTIFIER_ASSETTYPES = ['machine', 'computer', 'printer', 'network_device']
# Global-search result types that can be toggled on/off independently of whether
# the owning plugin is enabled. Keys match the `type` field on search results;
@@ -143,11 +142,8 @@ SEARCH_DOMAINS = {
'computer': 'PCs',
'printer': 'Printers',
'network_device': 'Network Devices',
'measuring_tool': 'Measuring Tools',
'notification': 'Notifications',
'subnet': 'Subnets',
'usb_device': 'USB Devices',
'printed_item': 'Printed Items',
}
def _declared_default(key: str) -> dict:
@@ -161,14 +157,50 @@ def _declared_default(key: str) -> dict:
return _plugin_declared_settings().get(key, {})
def _is_secret(key: str) -> bool:
return 'password' in key or 'token' in key or 'secret' in key
def _is_secret(key: str, valuetype=None) -> bool:
"""True when a key's value must never leave the API in plaintext.
The test is the key NAME, which is a heuristic, so it is narrowed by the
stored type when the caller knows it. A boolean, integer or float setting
cannot hold a secret, and masking one breaks the control that renders it:
plugin.endpoints.bootstrapquerytoken is a true/false switch whose name
contains 'token', and the settings page read back '********' instead of
its state, so the checkbox could not show whether it was on.
valuetype omitted means "judge by name alone", which is the safe direction:
an unknown type is treated as a string.
"""
named = 'password' in key or 'token' in key or 'secret' in key
if not named:
return False
return valuetype is None or str(valuetype).lower() == 'string'
def _inferred_valuetype(value) -> str:
"""Valuetype for a key nobody declares, from the JSON type sent.
Only reached on the upsert path in update_setting. JSON already carries the
distinction the row has to keep, so honour it: true/false is a boolean row,
a number is integer or float, everything else is a string. Guessing from
the TEXT (e.g. the string 'true') is deliberately not done - a genuine
string setting whose value happens to read 'true' would then be typed
boolean and coerced out from under its reader.
bool is checked before int because bool subclasses int in Python.
"""
if isinstance(value, bool):
return 'boolean'
if isinstance(value, int):
return 'integer'
if isinstance(value, float):
return 'float'
return 'string'
def _serialize_setting(setting):
"""Serialize a setting, masking secret values so they never leave the API."""
data = setting.to_dict()
if _is_secret(setting.key):
if _is_secret(setting.key, setting.valuetype):
data['value'] = SECRET_MASK if setting.value else ''
return data
@@ -795,7 +827,14 @@ def update_setting(key: str):
setting = Setting(
key=key,
value='',
valuetype=declared.get('valuetype', 'string'),
# An undeclared key used to be born valuetype='string'
# unconditionally, and that was a trap: a caller PUTting JSON true
# got a string row, get_typed_value then handed every reader the
# text 'true', and a reader doing bool(...) or `is True` silently
# disagreed with the UI. Infer from the JSON type the client sent
# instead, so a boolean key is born boolean.
valuetype=declared.get('valuetype') or _inferred_valuetype(
data.get('value')),
category=declared.get('category', 'plugin'),
description=declared.get('description'),
)
@@ -808,7 +847,7 @@ def update_setting(key: str):
# A secret submitted as the mask placeholder means "leave unchanged" - the
# client only ever received the mask, so don't overwrite the real secret.
if _is_secret(key) and value == SECRET_MASK:
if _is_secret(key, setting.valuetype) and value == SECRET_MASK:
return success_response(_serialize_setting(setting), message='Setting unchanged')
# The declared valuetype is a promise about what every reader will get, and
@@ -846,7 +885,7 @@ def update_setting(key: str):
setting.value = str(value) if value is not None else None
# Audit log (mask sensitive values)
is_sensitive = _is_secret(key)
is_sensitive = _is_secret(key, setting.valuetype)
AuditLog.log('updated', 'Setting', entityname=key, changes={
'value': {
'old': '***' if is_sensitive else old_value,
@@ -915,10 +954,13 @@ def create_setting():
else:
value_str = str(value) if value is not None else None
# Same reasoning as the upsert in update_setting: a caller that omits
# valuetype and sends JSON true must not get a string row that every
# boolean reader then misreads.
setting = Setting(
key=data['key'],
value=value_str,
valuetype=data.get('valuetype', 'string'),
valuetype=data.get('valuetype') or _inferred_valuetype(value),
category=data.get('category', 'general'),
description=data.get('description')
)
@@ -1025,18 +1067,14 @@ def build_default_settings():
'description': "IANA timezone for this site (e.g. America/New_York, America/Chicago). Notification start/end times and daily-reset expiry are computed in this zone."
},
{
'key': 'employee_directory_mode',
'value': 'selfhosted',
# Where a relative Picture path from the external HR directory
# resolves. Blank = this site's /static/employees/. A site whose
# photos live on another host points this at that host instead.
'key': 'employee_photo_base_url',
'value': '',
'valuetype': 'string',
'category': 'site',
'description': "Employee directory source: 'selfhosted' (tables in this app, default) or 'external' (a separate HR database)"
},
{
'key': 'usb_directory_mode',
'value': 'selfhosted',
'valuetype': 'string',
'category': 'site',
'description': "USB check-in/out source: 'selfhosted' (tables in this app, default) or 'external' (a separate cmmc_usb database)"
'description': 'Base URL for employee photos supplied as relative paths by the HR directory, e.g. https://host/EmployeeDBAPP/images/. Blank = this site\'s /static/employees/.'
},
{
'key': 'site_base_url',
@@ -1183,20 +1221,6 @@ def build_default_settings():
'category': 'printing',
'description': 'Custom URL template for printer QR labels. Blank = link to the printer page. Placeholders: {printerid}, {assetid}, {assetnumber}, {serialnumber}, {ip}, {hostname}.'
},
{
'key': 'qr_target_usb',
'value': '',
'valuetype': 'string',
'category': 'printing',
'description': 'Custom URL template for USB label QR codes. Blank = link to the USB device page. Placeholders: {id}, {serialnumber}, {alias}.'
},
{
'key': 'usb_label_style',
'value': 'barcode',
'valuetype': 'string',
'category': 'printing',
'description': "USB mini-label code style: 'barcode' (CODE128 of the serial number) or 'qr' (QR code linking to the QR target)."
},
{
'key': 'qr_target_machine',
'value': '',
@@ -1218,13 +1242,6 @@ def build_default_settings():
'category': 'printing',
'description': 'Custom URL template for network-device labels. Blank = link to the device page. Placeholders: {assetid}, {assetnumber}, {serialnumber}, {name}, {pluginid}.'
},
{
'key': 'qr_target_measuring_tool',
'value': '',
'valuetype': 'string',
'category': 'printing',
'description': 'Custom URL template for measuring-tool labels. Blank = link to the tool page. Placeholders: {assetid}, {assetnumber}, {serialnumber}, {name}, {pluginid}, {locationcode}, {locationname}.'
},
{
'key': 'label_default_style',
'value': 'card',
@@ -1242,15 +1259,15 @@ def build_default_settings():
]
# Per-asset-type default for what a label's code encodes. Machines default
# to their machine number (assetnumber), measuring tools to their inspection
# location code, everything else to a link to the asset page. Values:
# assetpage | assetnumber | serialnumber | location | custom.
# to their machine number (assetnumber), everything else to a link to the
# asset page. Values: assetpage | assetnumber | serialnumber | location |
# custom - 'location' stays a legal value because a plugin may add an asset
# type whose label is read off a shelf, not off the asset.
labelencodesdefaults = {
'machine': 'assetnumber',
'computer': 'assetpage',
'printer': 'assetpage',
'network_device': 'assetpage',
'measuring_tool': 'location',
}
printingdefaults += [
{
@@ -1259,16 +1276,23 @@ def build_default_settings():
'valuetype': 'string',
'category': 'printing',
'description': f'What a {assettype} label encodes by default: assetpage, '
'assetnumber, serialnumber'
+ (', location' if assettype == 'measuring_tool' else '')
+ ', or custom.',
'assetnumber, serialnumber, or custom.',
}
for assettype, value in labelencodesdefaults.items()
]
# Collector pc-type -> ComputerType mapping is computers-plugin domain;
# the plugin seeds pctypemap_<pxetype> settings on install.
# Collector authentication. The wording lives with the code that enforces
# it (shopdb/core/api/collector.py); seeding it here is what makes the row
# exist, and the row is what the settings UI can render a control for.
# Imported inside the function because core.api.__init__ imports collector
# and settings in the same package - a module-level import would couple
# their import order for no gain.
from shopdb.core.api.collector import LEGACY_SHARED_KEY_SETTING_DEFAULT
defaults = sitedefaults + brandingdefaults + printingdefaults + identifierdefaults + searchdefaults + mapdefaults + [
LEGACY_SHARED_KEY_SETTING_DEFAULT,
# ServiceNow ticket links. Each instance points these at its own
# ServiceNow tenant; blank/disabled renders tickets as plain text.
#
@@ -1320,6 +1344,21 @@ def build_default_settings():
'DeviceSettingsMenuBlade/~/overview/aadDeviceId/{deviceid}'
)
},
{
# ADR-015-OK: the DEFAULT is empty, so no cloud is assumed. Same
# reasoning as entra_device_url - the LAPS blade lives in the same
# tenant-specific portal, so the host cannot be a product fact.
'key': 'laps_device_url',
'value': '',
'valuetype': 'string',
'category': 'integrations',
'description': (
'Azure LAPS (local admin password) URL template. {objectid} is '
'the device object id. Example (US Gov): https://portal.azure.us/'
'?feature.msaljs=true#view/Microsoft_AAD_Devices/'
'DeviceDetailsMenuBlade/~/LocalAdminPassword/objectId/{objectid}'
)
},
{
'key': 'servicenow_ticket_prefixes',
'value': 'GEINC,GECHG,GERIT,GESCT',
@@ -1488,6 +1527,99 @@ def build_default_settings():
'Incoming Webhook), adaptivecard (Teams Workflow), '
'or json (generic {title,text})'
},
# Backup destination settings
#
# These are the ground wiring for where the one-shot backup container
# ships each set. 'local' keeps sets on the mounted backups volume and
# needs no credentials. 'smb' and 'aws' are declared so the UI and the
# backup runner can be configured now; the client libraries (an SMB
# stack, boto3/awscli) are deliberately NOT pulled in yet - those land
# behind the security review that vets third-party deps against
# external exploit. Until then a non-local destination is accepted and
# persisted but the runner treats it as not-yet-armed.
#
# Secrets follow the existing convention: any key containing
# 'password'/'secret'/'token' is masked by _is_secret on the way out
# and preserved when the client submits the mask back unchanged.
{
'key': 'backup_destination_kind',
'value': 'local',
'valuetype': 'string',
'category': 'backup',
'description': 'Where backup sets are shipped: local, smb, or aws. '
'Non-local destinations are wiring-only until the '
'security review clears their client libraries.'
},
{
'key': 'backup_smb_host',
'value': '',
'valuetype': 'string',
'category': 'backup',
'description': 'SMB/CIFS server hostname or IP for the backup share'
},
{
'key': 'backup_smb_share',
'value': '',
'valuetype': 'string',
'category': 'backup',
'description': 'SMB share name (the top-level export, no path)'
},
{
'key': 'backup_smb_path',
'value': '',
'valuetype': 'string',
'category': 'backup',
'description': 'Subpath within the SMB share to write sets into'
},
{
'key': 'backup_smb_username',
'value': '',
'valuetype': 'string',
'category': 'backup',
'description': 'SMB authentication username (DOMAIN\\user allowed)'
},
{
'key': 'backup_smb_password',
'value': '',
'valuetype': 'string',
'category': 'backup',
'description': 'SMB authentication password'
},
{
'key': 'backup_aws_region',
'value': '',
'valuetype': 'string',
'category': 'backup',
'description': 'AWS region for the S3 backup bucket (e.g. us-east-1)'
},
{
'key': 'backup_aws_bucket',
'value': '',
'valuetype': 'string',
'category': 'backup',
'description': 'S3 bucket name that holds backup sets'
},
{
'key': 'backup_aws_prefix',
'value': '',
'valuetype': 'string',
'category': 'backup',
'description': 'Optional key prefix within the S3 bucket'
},
{
'key': 'backup_aws_accesskeyid',
'value': '',
'valuetype': 'string',
'category': 'backup',
'description': 'AWS access key id for the backup bucket'
},
{
'key': 'backup_aws_secretaccesskey',
'value': '',
'valuetype': 'string',
'category': 'backup',
'description': 'AWS secret access key for the backup bucket'
},
# Audit log retention lives at 'audit_retentiondays' (category site),
# defined above and read by shopdb/core/services/auditretention.py. A
# second key 'audit_retention_days' was seeded here with a different

62
shopdb/core/api/setup.py
View File

@@ -5,6 +5,10 @@ server work: seeding common starter data and marking setup finished. Site,
plugin, and map configuration reuse the existing settings / plugins APIs.
"""
import hmac
import ipaddress
import os
from flask import Blueprint, request, current_app
from flask_jwt_extended import jwt_required
from werkzeug.security import generate_password_hash
@@ -29,6 +33,45 @@ STARTER_VENDORS = [
]
# Minimum length for the bootstrap administrator password. 12 characters is the
# shortest length that still resists offline cracking of a single leaked hash.
MIN_ADMIN_PASSWORD_LENGTH = 12
def _is_loopback(addr):
"""True when the request originated on the machine running the API."""
if not addr:
return False
try:
return ipaddress.ip_address(addr.split('%')[0]).is_loopback
except ValueError:
return False
def _bootstrap_allowed():
"""Proof of operator control for first-admin creation.
An empty users table is an ordering condition, not authorization: the API
serves requests before the first account exists on every fresh deploy, so a
remote caller could otherwise claim the instance during the install window.
Require either the one-time setup token (SETUP_TOKEN in config or the
environment, presented as the X-Setup-Token header) or a request coming from
loopback, i.e. the host the operator is installing on.
"""
token = str(current_app.config.get('SETUP_TOKEN')
or os.environ.get('SETUP_TOKEN') or '').strip()
if token:
supplied = (request.headers.get('X-Setup-Token') or '').strip()
return bool(supplied) and hmac.compare_digest(supplied, token)
return _is_loopback(request.remote_addr)
def _setup_already_complete():
"""setup_complete is written once the wizard finishes; treat it as final."""
row = Setting.query.filter_by(key='setup_complete').first()
return bool(row) and str(row.value).strip().lower() in ('true', '1', 'yes')
@setup_bp.route('/needs-admin', methods=['GET'])
def needs_admin():
"""True when no users exist yet - the app's very first run. Public so the
@@ -38,11 +81,18 @@ def needs_admin():
@setup_bp.route('/create-admin', methods=['POST'])
def create_admin():
"""Create the first admin. Only works while there are zero users, so it
can't be abused after setup. No auth (there is no one to auth as yet)."""
if User.query.count() > 0:
"""Create the first admin. Only works while there are zero users, setup has
not been marked complete, and the caller proves operator control (setup
token or a loopback request)."""
if User.query.count() > 0 or _setup_already_complete():
return error_response(ErrorCodes.FORBIDDEN,
'An account already exists; sign in instead.', http_code=403)
if not _bootstrap_allowed():
return error_response(
ErrorCodes.FORBIDDEN,
'First-admin creation requires the setup token (X-Setup-Token) '
'or a request from the server itself.',
http_code=403)
data = request.get_json() or {}
username = (data.get('username') or '').strip()
email = (data.get('email') or '').strip()
@@ -50,6 +100,10 @@ def create_admin():
if not (username and email and password):
return error_response(ErrorCodes.VALIDATION_ERROR,
'username, email and password are required')
if len(password) < MIN_ADMIN_PASSWORD_LENGTH:
return error_response(
ErrorCodes.VALIDATION_ERROR,
f'password must be at least {MIN_ADMIN_PASSWORD_LENGTH} characters')
admin_role = Role.query.filter_by(rolename='admin').first()
if not admin_role:
@@ -61,6 +115,8 @@ def create_admin():
user.roles.append(admin_role)
db.session.add(user)
db.session.commit()
current_app.logger.warning('First administrator %r created from %s',
username, request.remote_addr)
return success_response({'username': username}, message='Admin created', http_code=201)

745
shopdb/core/api/shopdbbackups.py Normal file
View File

@@ -0,0 +1,745 @@
"""Backups API routes.
Admin-only surface over the site's backup sets. The heavy lifting (mysqldump,
restic snapshot of the instance tree, retention prune, integrity check) runs in
the one-shot backup container defined in docker-compose.backup.yml; this
blueprint is the read-mostly control plane the UI talks to.
Two recovery products stay firewalled, per the design notes:
- code rollback lives in git (tags/releases), NOT here
- data restore lives here, gated on schema compatibility via each set's manifest
Every backup set carries a manifest.json stamped with the core + plugin Alembic
revisions and the running image tag at capture time. A restore is a CHECKED
operation: current code must be at least the set's schema revision, so we never
load rows that reference a column the live schema has not applied yet.
"""
import json
import os
import subprocess
from datetime import datetime, timezone
from flask import Blueprint, current_app, request
from shopdb.core.models import AuditLog
from shopdb.core.models.setting import Setting
from shopdb.utils.responses import success_response, error_response, ErrorCodes
from shopdb.utils.authz import require_role
shopdbbackups_bp = Blueprint('shopdbbackups', __name__)
# Supported backup destination kinds. 'local' needs no credentials; 'smb' and
# 'aws' are ground-wired here so operators can configure them now, but their
# client libraries are held back until the security review clears them, so the
# runner treats a non-local destination as configured-but-not-yet-armed.
DESTINATION_KINDS = ('local', 'smb', 'aws')
# The setting keys that make up each destination's config, grouped by kind so
# the GET can surface only the relevant fields and the PUT can validate what it
# is allowed to write. Secret keys (password/secret/token in the name) are
# masked by the shared settings serializer, same convention as SMTP/SAML.
DESTINATION_FIELDS = {
'local': [],
'smb': [
'backup_smb_host',
'backup_smb_share',
'backup_smb_path',
'backup_smb_username',
'backup_smb_password',
],
'aws': [
'backup_aws_region',
'backup_aws_bucket',
'backup_aws_prefix',
'backup_aws_accesskeyid',
'backup_aws_secretaccesskey',
],
}
# Keys whose value must never leave the API in the clear. Mirrors the shared
# settings convention (_is_secret) so the two surfaces agree on what is secret.
_SECRET_TOKENS = ('password', 'secret', 'token')
SECRET_MASK = '********'
def _is_secret_key(key):
return any(token in key for token in _SECRET_TOKENS)
# Where the backup container drops each set and its manifest. Mounted read-only
# for the api container so the UI can enumerate sets without being able to
# mutate them; writes happen only from the backup one-shot.
BACKUPS_ROOT_ENV = 'SHOPDB_BACKUPS_ROOT'
DEFAULT_BACKUPS_ROOT = '/backups'
# Post-restore upgrade sentinels. The backup sidecar drops .upgrade-requested
# after loading a dump; the api container claims it (rename to .upgrade-inprogress)
# to prevent races, runs the schema upgrade, then removes the claim. A failure
# parks the sentinel as .upgrade-failed so operators can inspect before re-arming.
UPGRADE_SENTINEL = '.upgrade-requested'
UPGRADE_CLAIM = '.upgrade-inprogress'
UPGRADE_FAILED = '.upgrade-failed'
# Retention the backup container enforces; surfaced read-only so the UI can show
# "day 1 of 5 retained" without re-deriving the policy.
RETENTION = {
'keepdaily': 5,
'keepweekly': 4,
'keepmonthly': 6,
}
def _backups_root():
return os.environ.get(BACKUPS_ROOT_ENV, DEFAULT_BACKUPS_ROOT)
def _read_json(path):
"""Load a JSON file, or None when it is absent or unreadable."""
try:
with open(path, 'r', encoding='utf-8') as handle:
return json.load(handle)
except (OSError, ValueError):
return None
def _read_status(kind):
"""Return the live status dict for a run kind, or None when idle."""
root = _backups_root()
return _read_json(os.path.join(root, '.' + kind + '-status.json'))
def _read_history(kind):
"""Return the recorded run history for a kind, newest first."""
root = _backups_root()
entries = _read_json(os.path.join(root, 'history', kind + 's.json'))
if not isinstance(entries, list):
return []
return list(reversed(entries))
def _write_status(kind, phase, setname, message):
"""Seed a live status file so the UI shows movement the instant a run is
queued, before the watcher has picked up the sentinel."""
root = _backups_root()
stamp = datetime.now(timezone.utc).replace(tzinfo=None).isoformat() + 'Z'
payload = {
'kind': kind,
'phase': phase,
'setname': setname,
'message': message,
'updatedat': stamp,
}
try:
path = os.path.join(root, '.' + kind + '-status.json')
with open(path, 'w', encoding='utf-8') as handle:
json.dump(payload, handle)
except OSError:
pass
def _append_history(kind, outcome, setname, message):
"""Append one run record to the shared history the UI reads, newest last.
Mirrors status.sh's history_append so a finalize that completes in the api
(not the sidecar) still lands in the same history file the restore page
shows. Best-effort: history is a convenience log, never a gate.
"""
root = _backups_root()
stamp = datetime.now(timezone.utc).replace(tzinfo=None).isoformat() + 'Z'
entry = {
'kind': kind,
'outcome': outcome,
'setname': setname,
'message': message,
'at': stamp,
}
path = os.path.join(root, 'history', kind + 's.json')
try:
os.makedirs(os.path.dirname(path), exist_ok=True)
entries = _read_json(path)
if not isinstance(entries, list):
entries = []
entries.append(entry)
with open(path, 'w', encoding='utf-8') as handle:
json.dump(entries, handle)
except OSError:
pass
def _claim_upgrade():
"""Atomically claim a pending upgrade to prevent races.
Returns the setname if this call won the claim, None otherwise. Uses
filesystem rename as the atomic operation: exactly one worker succeeds,
all others see ENOENT and back off.
"""
root = _backups_root()
sentinel = os.path.join(root, UPGRADE_SENTINEL)
claim = os.path.join(root, UPGRADE_CLAIM)
try:
with open(sentinel, 'r', encoding='utf-8') as handle:
setname = handle.read().strip()
except OSError:
return None
try:
os.rename(sentinel, claim)
return setname
except OSError:
return None
def _finalize_claimed(setname):
"""Run the post-restore schema upgrade and clear the claim.
Returns True on success, False on failure. A failure parks the sentinel
as .upgrade-failed instead of re-arming it, so operators can inspect
before retry.
"""
if not setname:
return True
root = _backups_root()
claim = os.path.join(root, UPGRADE_CLAIM)
failed_sentinel = os.path.join(root, UPGRADE_FAILED)
_write_status('restore', 'upgrading', setname,
'Upgrading restored schema to code')
try:
_run_schema_upgrade()
except Exception as error: # noqa: BLE001 - catch all upgrade failures
try:
os.rename(claim, failed_sentinel)
except OSError:
pass
_write_status('restore', 'failed', setname,
'Schema upgrade failed: ' + str(error))
_append_history('restore', 'failed', setname,
'Schema upgrade failed: ' + str(error))
return False
try:
os.remove(claim)
except OSError:
pass
_write_status('restore', 'done', setname, 'Restore complete')
_append_history('restore', 'success', setname, 'Restore complete')
return True
def _run_schema_upgrade():
"""Bring the just-restored database up to the running code, in-process.
Runs the same two idempotent steps every deploy runs: the core Alembic
upgrade to head, then each plugin chain via the plugin manager. Both are
no-ops when the data already matches the code head, so this is safe to call
even when the restored dump was already current. Uses the live app context
and its configured engine, so no subprocess or extra Flask CLI is needed.
"""
from flask_migrate import upgrade as alembic_upgrade
alembic_upgrade()
manager = current_app.extensions.get('plugin_manager')
if manager is None:
raise RuntimeError('Plugin manager not initialized; cannot upgrade plugins')
results = manager.upgrade_all_plugins()
failed = {name: status for name, status in (results or {}).items()
if status not in ('ok', 'no-migrations')}
if failed:
raise RuntimeError('Plugin upgrade failed: ' + json.dumps(failed))
def _read_manifest(setdir):
"""Load a set's manifest.json, or None when absent/unreadable.
A set without a readable manifest is not restorable (we cannot check its
schema), so callers treat None as "unverified" rather than trusting it.
"""
path = os.path.join(setdir, 'manifest.json')
try:
with open(path, 'r', encoding='utf-8') as handle:
return json.load(handle)
except (OSError, ValueError):
return None
def _current_schema_revisions():
"""Return the live Alembic head revisions (core + plugins) for this build.
Compared against a set's stamped revisions to decide restore compatibility.
Imported lazily so a missing alembic at request time degrades to an empty
map (everything reads as "unknown") instead of 500ing the list endpoint.
"""
try:
from alembic.config import Config
from alembic.script import ScriptDirectory
except ImportError:
return {}
root = current_app.root_path
migrations = os.path.join(os.path.dirname(root), 'migrations')
ini = os.path.join(migrations, 'alembic.ini')
if not os.path.isfile(ini):
return {}
try:
config = Config(ini)
# This is a Flask-Migrate project: alembic.ini carries no
# script_location (it is set at runtime), so ScriptDirectory.from_config
# would raise and leave every set reading as incompatible. Point it at
# the migrations directory - the same versions tree manifest.py scans.
config.set_main_option('script_location', migrations)
script = ScriptDirectory.from_config(config)
return {'core': list(script.get_heads())}
except Exception:
return {}
def _known_core_revisions():
"""Return every core Alembic revision this build knows, heads and ancestors.
Restore compatibility is now data-driven: a set restores when the running
code KNOWS the dump's data revision (so `flask db upgrade` can roll it
forward to head). Heads alone cannot answer that - a dump stamped at an
older-but-valid revision is perfectly restorable - so we walk the whole
script history. Lazy import + broad except keeps a missing/broken alembic
from 500ing the list endpoint (returns empty = everything reads unknown).
"""
try:
from alembic.config import Config
from alembic.script import ScriptDirectory
except ImportError:
return set()
root = current_app.root_path
migrations = os.path.join(os.path.dirname(root), 'migrations')
ini = os.path.join(migrations, 'alembic.ini')
if not os.path.isfile(ini):
return set()
try:
config = Config(ini)
config.set_main_option('script_location', migrations)
script = ScriptDirectory.from_config(config)
return {rev.revision for rev in script.walk_revisions()}
except Exception:
return set()
def _known_plugin_revisions():
"""Return known plugin revisions for each installed plugin.
Returns a dict mapping plugin name to set of known revision IDs. Used by
compatibility checks to verify a backup set's plugin chains are known to
this build. Plugins that fail to load or have no migrations are omitted.
"""
manager = current_app.extensions.get('plugin_manager')
if not manager:
return {}
try:
from alembic.config import Config
from alembic.script import ScriptDirectory
except ImportError:
return {}
known = {}
for plugin_name in manager.registry.get_all():
try:
migrations_dir = manager.migration_manager.get_migrations_dir(plugin_name)
if not migrations_dir:
continue
# Point alembic at this plugin's migrations directory
config = Config()
config.set_main_option('script_location', str(migrations_dir))
script = ScriptDirectory.from_config(config)
# Walk all revisions (heads and ancestors) for this plugin
revisions = {rev.revision for rev in script.walk_revisions()}
if revisions:
known[plugin_name] = revisions
except Exception:
# Skip plugins that fail to load or have malformed migrations
continue
return known
def _is_compatible(manifest, current):
"""Decide whether a set restores onto THIS build.
Data-driven gate (the fix for the FBJXBDB4 outage): the authoritative
signal is the set's `datarev` - the actual `alembic_version` of the dumped
DATABASE, not the code head that captured it. A restore is safe when the
running code KNOWS that revision (it is somewhere in this build's migration
history), because then the post-restore finalize can run `flask db upgrade`
to roll the loaded data forward to head. If the code does not know the
revision, the dump is from a NEWER schema than this build and would 500 on
restored rows, so we block until the code is upgraded.
Also checks plugin revisions when present: a set from a newer plugin chain
is refused even if the core revision is known, because plugin table schema
could be ahead of what this build understands.
Legacy sets predate `datarev` and `dataplugins`; fall back to the old
head-subset check on `schemarev` so they stay listable and restorable.
"""
if not manifest:
return False
# Check if manifest has datarev key (even if empty)
if 'datarev' in manifest:
datarev = manifest['datarev']
# Empty string means backup.sh couldn't read alembic_version (error case)
# Treat as incompatible - don't fall back to the legacy check this was
# introduced to replace
if not datarev:
return False
# Check core revision is known
known = _known_core_revisions()
if not known:
return False
if datarev not in known:
return False
# Check plugin revisions if present
dataplugins = manifest.get('dataplugins')
if dataplugins:
known_plugins = _known_plugin_revisions()
for plugin_name, plugin_rev in dataplugins.items():
# Refuse if this build doesn't have the plugin at all
if plugin_name not in known_plugins:
return False
# Refuse if the plugin exists but this revision is unknown
if plugin_rev not in known_plugins[plugin_name]:
return False
return True
# Legacy: fall back to schemarev head-subset check for pre-datarev sets
setrevs = set(manifest.get('schemarev', {}).get('core', []))
currevs = set(current.get('core', []))
if not setrevs or not currevs:
return False
# Compatible when the current heads cover every head the set was taken at.
return setrevs.issubset(currevs)
# Reserved entries under the backups root that are not backup sets: the run
# history store and any dotfile sentinels/status files the watcher drops.
_NON_SET_ENTRIES = ('history',)
def _list_sets():
"""Enumerate backup sets newest-first, each summarized for the list view."""
root = _backups_root()
if not os.path.isdir(root):
return []
current = _current_schema_revisions()
sets = []
for name in sorted(os.listdir(root), reverse=True):
if name in _NON_SET_ENTRIES or name.startswith('.'):
continue
setdir = os.path.join(root, name)
if not os.path.isdir(setdir):
continue
manifest = _read_manifest(setdir)
dbdump = os.path.join(setdir, 'db.sql.gz')
sizebytes = 0
for base, _dirs, files in os.walk(setdir):
for filename in files:
try:
sizebytes += os.path.getsize(os.path.join(base, filename))
except OSError:
pass
sets.append({
'snapshotid': name,
'takenat': (manifest or {}).get('takenat'),
'sizebytes': sizebytes,
'schemarev': (manifest or {}).get('schemarev', {}),
'imagetag': (manifest or {}).get('imagetag'),
'ischeckpassed': bool((manifest or {}).get('ischeckpassed')),
'hasmanifest': manifest is not None,
'hasdbdump': os.path.isfile(dbdump),
'iscompatible': _is_compatible(manifest, current),
})
return sets
@shopdbbackups_bp.route('', methods=['GET'])
@require_role('admin')
def list_backups():
"""List every retained backup set, newest first, with a compatibility flag."""
sets = _list_sets()
laststgood = next((s for s in sets if s['ischeckpassed']), None)
meta = {
'retention': RETENTION,
'lastgoodat': (laststgood or {}).get('takenat'),
'count': len(sets),
}
return success_response(data=sets, meta=meta)
@shopdbbackups_bp.route('/<snapshotid>', methods=['GET'])
@require_role('admin')
def get_backup(snapshotid):
"""Return one set's full manifest plus a resolved compatibility verdict."""
root = _backups_root()
setdir = os.path.join(root, os.path.basename(snapshotid))
if not os.path.isdir(setdir):
return error_response(ErrorCodes.NOT_FOUND, 'Backup set not found',
http_code=404)
manifest = _read_manifest(setdir)
current = _current_schema_revisions()
return success_response(data={
'snapshotid': os.path.basename(snapshotid),
'manifest': manifest,
'currentschemarev': current,
'iscompatible': _is_compatible(manifest, current),
'hasmanifest': manifest is not None,
})
@shopdbbackups_bp.route('', methods=['POST'])
@require_role('admin')
def trigger_backup():
"""Kick off an on-demand backup run.
The api container cannot run docker; it writes a sentinel the host-side
scheduler (or an operator running `docker compose run --rm backup`) picks
up. Returning isrunning=false with a clear message keeps the UI honest
rather than pretending the run started inside this process.
"""
root = _backups_root()
try:
os.makedirs(root, exist_ok=True)
sentinel = os.path.join(root, '.backup-requested')
stamp = datetime.now(timezone.utc).replace(tzinfo=None).isoformat() + 'Z'
with open(sentinel, 'w', encoding='utf-8') as handle:
handle.write(stamp)
except OSError as error:
return error_response(ErrorCodes.INTERNAL_ERROR,
'Could not queue a backup: ' + str(error),
http_code=500)
_write_status('backup', 'queued', '', 'Backup queued')
AuditLog.log('requested', 'Backup', entityname=stamp)
return success_response(
data={'isrunning': False, 'isqueued': True, 'requestedat': stamp},
message='Backup queued. The backup service runs it on its next pass.')
@shopdbbackups_bp.route('/<snapshotid>/restore', methods=['POST'])
@require_role('admin')
def restore_backup(snapshotid):
"""Queue a restore of a set, gated on schema compatibility.
We refuse outright when the set is incompatible (current code older than the
set's schema) so an operator cannot foot-gun a schema mismatch from the UI.
The actual restore is performed by scripts/backup/restore.sh in the backup
container; here we validate and record intent.
"""
root = _backups_root()
setdir = os.path.join(root, os.path.basename(snapshotid))
if not os.path.isdir(setdir):
return error_response(ErrorCodes.NOT_FOUND, 'Backup set not found',
http_code=404)
manifest = _read_manifest(setdir)
current = _current_schema_revisions()
if not _is_compatible(manifest, current):
return error_response(
ErrorCodes.CONFLICT,
'This backup was taken on a newer schema than the running code. '
'Upgrade the code to at least the set\'s revision, run '
'`flask db upgrade` and `flask plugin upgrade-all`, then retry.',
details={'currentschemarev': current,
'setschemarev': (manifest or {}).get('schemarev', {})},
http_code=409)
try:
sentinel = os.path.join(root, '.restore-requested')
stamp = datetime.now(timezone.utc).replace(tzinfo=None).isoformat() + 'Z'
with open(sentinel, 'w', encoding='utf-8') as handle:
handle.write(os.path.basename(snapshotid))
except OSError as error:
return error_response(ErrorCodes.INTERNAL_ERROR,
'Could not queue the restore: ' + str(error),
http_code=500)
_write_status('restore', 'queued', os.path.basename(snapshotid),
'Restore queued')
AuditLog.log('restore-requested', 'Backup',
entityname=os.path.basename(snapshotid))
return success_response(
data={'isqueued': True, 'snapshotid': os.path.basename(snapshotid)},
message='Restore queued and validated against the live schema.')
@shopdbbackups_bp.route('/finalize-restore', methods=['POST'])
@require_role('admin')
def finalize_restore():
"""Run the post-restore schema upgrade in the app context.
The restore itself runs in the Flask-less backup sidecar, which loads the
dump and then drops an .upgrade-requested sentinel because it cannot run
'flask db upgrade' itself. This endpoint is the api-side half: it brings the
just-restored DATA up to the running CODE with the exact idempotent commands
the deploy path uses, then stamps the final 'done'. Without it a restored
dump that sits at an older revision leaves every newer-migration table
throwing 1054 Unknown column.
Supports re-arming a failed upgrade: if .upgrade-failed exists, moves it back
to .upgrade-requested so the operator can retry after fixing the issue.
Idempotent and safe to call when nothing is pending: if there is no sentinel
it reports there was nothing to finalize. Uses an atomic claim mechanism to
prevent races when multiple workers poll this endpoint simultaneously.
"""
root = _backups_root()
# Check if backup store is configured
if not os.path.isdir(root):
return error_response(ErrorCodes.NOT_FOUND,
'Backup store not configured',
http_code=404)
# Support re-arming a failed upgrade
failed_sentinel = os.path.join(root, UPGRADE_FAILED)
if os.path.isfile(failed_sentinel):
try:
with open(failed_sentinel, 'r', encoding='utf-8') as handle:
setname = handle.read().strip()
# Re-arm by moving failed back to requested
sentinel = os.path.join(root, UPGRADE_SENTINEL)
os.rename(failed_sentinel, sentinel)
return success_response(
data={'retried': True, 'snapshotid': setname},
message='Failed upgrade re-armed for retry.')
except OSError as error:
return error_response(ErrorCodes.INTERNAL_ERROR,
'Could not re-arm failed upgrade: ' + str(error),
http_code=500)
# Check if there's a pending upgrade sentinel (but don't claim it)
# The watcher/background worker will claim and run it
sentinel = os.path.join(root, UPGRADE_SENTINEL)
if os.path.isfile(sentinel):
try:
with open(sentinel, 'r', encoding='utf-8') as handle:
setname = handle.read().strip()
return success_response(
data={'queued': True, 'snapshotid': setname},
message='Schema upgrade queued.')
except OSError:
pass
return success_response(
data={'queued': False},
message='No restore is awaiting schema upgrade.')
@shopdbbackups_bp.route('/status', methods=['GET'])
@require_role('admin')
def get_status():
"""Return the live status of the most recent backup and restore runs.
The watcher writes these files as it moves through each phase; the UI polls
this endpoint to drive the restore progress spinner and phase label. Either
value is None when that kind of run has never been requested.
"""
return success_response(data={
'backup': _read_status('backup'),
'restore': _read_status('restore'),
})
@shopdbbackups_bp.route('/history', methods=['GET'])
@require_role('admin')
def get_history():
"""Return the recorded history of backup and restore runs, newest first.
These are operational run logs written by the backup container onto the
shared volume, not the audit trail (AuditLog records the requested intent).
"""
return success_response(data={
'backups': _read_history('backup'),
'restores': _read_history('restore'),
})
@shopdbbackups_bp.route('/settings', methods=['GET'])
@require_role('admin')
def get_backup_settings():
"""Return destination + retention config for the settings pane.
The destination kind and its (non-secret) fields come from the settings
store so an operator can configure where backups ship without editing env
or redeploying. Secret fields (SMB password, AWS secret key) are never
returned in the clear: we send a mask when a value is set and an empty
string when it is not, matching the shared settings API convention.
'smb' and 'aws' are configurable now but their client libraries are held
back pending the security review, so we flag those destinations as armed:
false to tell the UI the config is stored but not yet acting.
"""
kind = Setting.get('backup_destination_kind', 'local')
if kind not in DESTINATION_KINDS:
kind = 'local'
fields = {}
for key in DESTINATION_FIELDS.get(kind, []):
raw = Setting.get(key, '')
if _is_secret_key(key):
fields[key] = SECRET_MASK if raw else ''
else:
fields[key] = raw or ''
return success_response(data={
'destinationkind': kind,
'destinationkinds': list(DESTINATION_KINDS),
'fields': fields,
'isarmed': kind == 'local',
'retention': RETENTION,
'backupsroot': _backups_root(),
})
@shopdbbackups_bp.route('/settings', methods=['PUT'])
@require_role('admin')
def update_backup_settings():
"""Persist the backup destination kind and its credential/config fields.
Only keys that belong to the chosen destination are accepted, so a stray
payload cannot write arbitrary settings rows through this surface. A secret
field submitted as the mask placeholder means "leave unchanged" - the
client only ever received the mask, so we never overwrite a real secret
with it.
"""
data = request.get_json(silent=True) or {}
kind = data.get('destinationkind', Setting.get('backup_destination_kind', 'local'))
if kind not in DESTINATION_KINDS:
return error_response(
ErrorCodes.VALIDATION_ERROR,
'destinationkind must be one of: ' + ', '.join(DESTINATION_KINDS))
Setting.set('backup_destination_kind', kind, valuetype='string',
category='backup')
incoming = data.get('fields', {}) or {}
allowed = set(DESTINATION_FIELDS.get(kind, []))
for key, value in incoming.items():
if key not in allowed:
continue
if _is_secret_key(key) and value == SECRET_MASK:
continue
Setting.set(key, value if value is not None else '',
valuetype='string', category='backup')
AuditLog.log('updated', 'BackupSettings', entityname=kind, changes={
'destinationkind': kind,
})
return get_backup_settings()

2
shopdb/core/api/staticdocs/redoc.standalone.js
View File

File diff suppressed because one or more lines are too long

2
shopdb/core/api/staticdocs/swagger-ui-bundle.min.js vendored Normal file
View File

File diff suppressed because one or more lines are too long

202
shopdb/core/api/staticdocs/swagger-ui.LICENSE.txt Normal file
View File

@@ -0,0 +1,202 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

2
shopdb/core/api/staticdocs/swagger-ui.css Normal file
View File

File diff suppressed because one or more lines are too long

28
shopdb/core/api/supportteams.py
View File

@@ -1,5 +1,7 @@
"""Support Teams API endpoints - teams and their nested contacts."""
from urllib.parse import urlparse
from flask import Blueprint, request
from flask_jwt_extended import jwt_required
@@ -19,6 +21,16 @@ from shopdb.utils.authz import require_permission
supportteams_bp = Blueprint('supportteams', __name__)
def _validate_url_scheme(url):
"""Validate that URL uses safe scheme (prevents XSS via javascript: URIs)."""
if not url:
return True
parsed = urlparse(url)
if parsed.scheme and parsed.scheme not in ['http', 'https']:
raise ValueError(f"Invalid URL scheme '{parsed.scheme}'. Only http:// and https:// are allowed.")
return True
@supportteams_bp.route('', methods=['GET'])
@require_permission('assets.view')
def list_support_teams():
@@ -62,6 +74,13 @@ def create_support_team():
f"Support team '{data['teamname']}' already exists",
http_code=409)
# Validate URL schemes to prevent XSS
try:
_validate_url_scheme(data.get('teamurl'))
_validate_url_scheme(data.get('webhookurl'))
except ValueError as e:
return error_response(ErrorCodes.VALIDATION_ERROR, str(e))
team = SupportTeam(
teamname=data['teamname'],
teamurl=data.get('teamurl'),
@@ -101,6 +120,15 @@ def update_support_team(team_id: int):
f"Support team '{data['teamname']}' already exists",
http_code=409)
# Validate URL schemes to prevent XSS
try:
if 'teamurl' in data:
_validate_url_scheme(data['teamurl'])
if 'webhookurl' in data:
_validate_url_scheme(data['webhookurl'])
except ValueError as e:
return error_response(ErrorCodes.VALIDATION_ERROR, str(e))
for key in ['teamname', 'teamurl', 'webhookurl', 'isactive']:
if key in data:
setattr(team, key, data[key])

158
shopdb/core/api/users.py
View File

@@ -7,13 +7,82 @@ from werkzeug.security import generate_password_hash
from shopdb.extensions import db
from shopdb.core.models import (
User, Role, Permission, AuditLog, full_permission_catalog,
guest_grantable_permissions)
guest_grantable_permissions, SCOPE_ENDPOINT_ACTIONS,
SCOPES_DENIED_ENDPOINT_ACTIONS)
from shopdb.utils.responses import success_response, error_response, ErrorCodes
from shopdb.utils.authz import require_role
users_bp = Blueprint('users', __name__)
def _check_scope_grants(rolenames, permissionnames):
"""Refuse direct grants a prebuilt scope is defined as never holding.
Returns an error_response, or None when the combination is allowed.
This is the server-side half of the scope dropdown, and it is the half that
is actually load-bearing. The UI greys out the endpoint action checkboxes
for a reporter, but a greyed-out checkbox is a suggestion: the same account
can be created with a restart grant by anyone willing to POST JSON. Reporter
was specified as holding no action capability EVEN IF expressly permitted,
and only a check on the write path can make that true.
Enforced here rather than in User.haspermission() on purpose. A subtractive
ceiling in the read predicate would sit in the hot path of every
permission-gated route in the app and change the meaning of a grant
everywhere, to fix a combination that should simply never have been stored.
Refusing the write is the narrow version: bad state cannot get in, and the
thousands of existing reads keep their current semantics.
Note this deliberately does NOT restrict the admin role, which bypasses
permission checks by rule, nor any site-created custom role. It constrains
only the scopes that were seeded with a documented promise about what they
hold.
"""
denied_scopes = SCOPES_DENIED_ENDPOINT_ACTIONS & set(rolenames)
if not denied_scopes:
return None
conflicts = sorted(set(permissionnames) & set(SCOPE_ENDPOINT_ACTIONS))
if not conflicts:
return None
scope = sorted(denied_scopes)[0]
return error_response(
ErrorCodes.VALIDATION_ERROR,
f"The {scope} scope cannot hold endpoint actions "
f"({', '.join(conflicts)}). Assign the technician or operator scope "
f"if this user needs to act on endpoints.")
def _resolve_directpermissions(names):
"""Resolve permission names to rows, rejecting any that do not exist.
Returns (permissions, error_response). A None permissions value with no
error means the caller did not mention direct grants at all, which is not
the same as sending an empty list - that clears them.
Unknown names are refused rather than dropped. Silently ignoring a typo
would report success while granting nothing, and the admin would believe
the capability was handed out.
"""
if names is None:
return None, None
if not isinstance(names, list):
return None, error_response(
ErrorCodes.VALIDATION_ERROR,
'directpermissions must be a list of permission names')
if not names:
return [], None
wanted = {str(n) for n in names}
permissions = Permission.query.filter(Permission.name.in_(wanted)).all()
missing = sorted(wanted - {p.name for p in permissions})
if missing:
return None, error_response(
ErrorCodes.VALIDATION_ERROR,
'Unknown permissions: ' + ', '.join(missing))
return permissions, None
@users_bp.route('', methods=['GET'])
@jwt_required()
@require_role('admin')
@@ -61,6 +130,28 @@ def create_user():
if User.query.filter_by(email=data['email']).first():
return error_response(ErrorCodes.CONFLICT, 'Email already exists', http_code=409)
# Resolve and VALIDATE roles and grants before touching the User object.
#
# Ordering is deliberate. Assigning user.roles first put the relationship
# into the session's autoflush path while the User itself was still
# unattached, which SQLAlchemy warned about ("Object of type <User> not in
# session, add operation along 'Role.users' won't proceed"). Nothing broke,
# because the refusal returns before db.session.add(). But it left a live
# trap: move or add a query below this point and the autoflush becomes a
# real write of a half-built user, surfacing later as an orphan row nobody
# can explain. Validate everything, then build.
role_ids = data.get('roles', [])
roles = Role.query.filter(Role.roleid.in_(role_ids)).all() if role_ids else []
permissions, err = _resolve_directpermissions(data.get('directpermissions'))
if err:
return err
err = _check_scope_grants(
[r.rolename for r in roles],
[p.name for p in (permissions or [])])
if err:
return err
# Admin-created accounts are forced to change the password on first login
# unless the admin explicitly opts out.
mustchange = data.get('mustchangepassword', True)
@@ -75,12 +166,13 @@ def create_user():
mustchangepassword=bool(mustchange)
)
# Assign roles
role_ids = data.get('roles', [])
if role_ids:
roles = Role.query.filter(Role.roleid.in_(role_ids)).all()
if roles:
user.roles = roles
# Direct per-user permission grants, held alongside (not instead of) roles.
if permissions is not None:
user.directpermissions = permissions
db.session.add(user)
# Audit log
@@ -181,6 +273,39 @@ def update_user(userid: int):
changes['roles'] = {'old': old_roles, 'new': new_roles}
user.roles = roles
# Direct grants are audited by NAME rather than id: the point of the
# audit row is to be readable years later, when a permission id means
# nothing to the person reading it.
if 'directpermissions' in data:
permissions, err = _resolve_directpermissions(
data['directpermissions'])
if err:
return err
old_direct = user.getdirectpermissionnames()
new_direct = sorted(p.name for p in (permissions or []))
if old_direct != new_direct:
changes['directpermissions'] = {
'old': old_direct, 'new': new_direct}
user.directpermissions = permissions or []
# Validate the COMBINATION, after both assignments and against the
# user's effective state rather than only what this request mentioned.
# A payload may change roles without mentioning grants, or grants
# without mentioning roles; checking only the supplied half would let a
# request that demotes an operator to reporter keep the restart grant
# they already held, which is exactly the state the scope promises is
# impossible. Reading back off `user` covers all three shapes.
#
# Safe to check after mutating because nothing has committed yet - the
# early return leaves the session to be rolled down by the request
# teardown, so a refused combination is never persisted.
err = _check_scope_grants(
[r.rolename for r in user.roles],
user.getdirectpermissionnames())
if err:
db.session.rollback()
return err
# Unlock user
if data.get('unlock'):
user.lockeduntil = None
@@ -237,6 +362,21 @@ def delete_user(userid: int):
# revoke their API tokens outright, and DETACH their audit history
# (userid -> NULL) - the log rows themselves are kept, entityname and
# details still tell the story.
#
# What a departing user leaves behind is deliberately NOT undone. Policies
# carry no FK to users, so configuration and business policies they authored
# keep working; the audit rows keep username, which is denormalized on
# auditlogs for exactly this reason. Leaving is not a reason to unpick
# someone's work.
#
# TODO (future): offer to EXPORT this user's audit history at the moment of
# removal, and on a significant scope reduction too. Detaching costs the
# userid link, so after this runs their trail is only reachable by matching
# the username string - which is fragile if the name is ever reused. An
# export taken at the point of change preserves the joined-up record while
# it still exists. Deactivating (isactive=False) instead of deleting keeps
# the userid link intact and is the better choice when the account may ever
# need to be audited in depth.
from shopdb.core.models import ApiToken
ApiToken.query.filter_by(userid=userid).delete(synchronize_session=False)
AuditLog.query.filter_by(userid=userid).update(
@@ -452,6 +592,14 @@ def user_to_dict(user: User) -> dict:
'lastlogindate': user.lastlogindate.isoformat() + 'Z' if user.lastlogindate else None,
'failedlogins': user.failedlogins,
'roles': [{'roleid': r.roleid, 'rolename': r.rolename, 'color': r.color} for r in user.roles],
# Split deliberately. directpermissions is what is STORED on the user and
# is what the editor's checkboxes bind to; permissions is the EFFECTIVE
# set including everything the roles supply, which is what a reader needs
# to answer "can this person do X". A checkbox bound to the effective set
# would appear ticked for a capability the user holds through a role, and
# unticking it would silently do nothing.
'directpermissions': user.getdirectpermissionnames(),
'permissions': sorted(user.getpermissions()),
'createddate': user.createddate.isoformat() + 'Z' if user.createddate else None,
'modifieddate': user.modifieddate.isoformat() + 'Z' if user.modifieddate else None
}

15
shopdb/core/models/__init__.py
View File

@@ -14,10 +14,13 @@ from .operatingsystem import OperatingSystem
from .relationship import AssetRelationship, RelationshipType, RelationshipTypePropagation
from .communication import Communication, CommunicationType
from .user import (User, Role, Permission, full_permission_catalog,
guest_grantable_permissions)
guest_grantable_permissions, scope_permission_names,
PREBUILT_SCOPES, SCOPE_ENDPOINT_ACTIONS,
SCOPE_NEVER_GRANT, SCOPES_DENIED_ENDPOINT_ACTIONS)
from .application import Application, AppVersion
from .script_library import (Script, RUNCONTEXTS, RUNCONTEXT_SYSTEM,
RUNCONTEXT_USER)
RUNCONTEXT_USER, ERRORMODES, ERRORMODE_DEFAULT,
ERRORMODE_STRICT)
from .supportteam import SupportTeam, SupportTeamContact
from .setting import Setting
from .auditlog import AuditLog
@@ -63,6 +66,11 @@ __all__ = [
'Permission',
'full_permission_catalog',
'guest_grantable_permissions',
'scope_permission_names',
'PREBUILT_SCOPES',
'SCOPE_ENDPOINT_ACTIONS',
'SCOPE_NEVER_GRANT',
'SCOPES_DENIED_ENDPOINT_ACTIONS',
# Applications
'Application',
'AppVersion',
@@ -71,6 +79,9 @@ __all__ = [
'RUNCONTEXTS',
'RUNCONTEXT_SYSTEM',
'RUNCONTEXT_USER',
'ERRORMODES',
'ERRORMODE_DEFAULT',
'ERRORMODE_STRICT',
# Support teams
'SupportTeam',
'SupportTeamContact',

8
shopdb/core/models/apitoken.py
View File

@@ -54,10 +54,10 @@ class ApiToken(BaseModel):
# owner holds, and suspends the admin bypass. See scopelist below.
scopes = db.Column(db.Text, nullable=True)
# JSON array of RESOURCE-scope names (a different axis from `scopes`, which
# is what the token may DO). Today these are geenforce manifest scope names
# (e.g. gea-shopfloor-display): a resource-bound geenforce.fetch token may
# only pull those scopes' manifests and only blobs those manifests
# reference. NULL = unrestricted (any resource), for back-compat. See
# is what the token may DO). The names are a plugin's own, not core's: a
# plugin that serves per-fleet bundles, say, pins a token to the bundle
# names it may pull, so the permission says "may fetch" and this says "may
# fetch THESE". NULL = unrestricted (any resource), for back-compat. See
# resourcescopelist below.
resourcescopes = db.Column(db.Text, nullable=True)

14
shopdb/core/models/application.py
View File

@@ -33,6 +33,20 @@ class Application(BaseModel):
db.Boolean, default=False,
comment='Required on all PCs (drives the software-compliance report)'
)
# Monitorable: this app can be watched by the endpoint agent as a process
# monitor. A monitor is useless without something to launch and something to
# look for, so both fields below are REQUIRED whenever this is set - enforced
# in the applications API rather than as a DB constraint, because the columns
# must stay nullable for the rows that existed before this shipped.
ismonitorable = db.Column(
db.Boolean, default=False,
comment='Can be watched by the endpoint agent (needs launch path + process name)'
)
# Full path used to START the app when a monitor finds it not running.
monitorlaunchpath = db.Column(db.String(512))
# Windows image name to watch for, e.g. "notepad.exe". This is what the agent
# matches against the running process list.
monitorprocessname = db.Column(db.String(255))
image = db.Column(db.String(255))
# Digest of the app-level uploaded installer (installpath). Lets the
# endpoint update_app push resolve an install source by app alone, with no

5
shopdb/core/models/asset.py
View File

@@ -253,8 +253,6 @@ class Asset(BaseModel, SoftDeleteMixin, AuditMixin):
result['pluginid'] = self.network_device.networkdeviceid
elif hasattr(self, 'printer') and self.printer:
result['pluginid'] = self.printer.printerid
elif hasattr(self, 'measuringtool') and self.measuringtool:
result['pluginid'] = self.measuringtool.measuringtoolid
# What this asset OWNS, before any inheritance is merged in below.
#
@@ -321,7 +319,4 @@ class Asset(BaseModel, SoftDeleteMixin, AuditMixin):
# Check for printer extension
if hasattr(self, 'printer') and self.printer:
return self.printer.to_dict()
# Check for measuring-tool extension
if hasattr(self, 'measuringtool') and self.measuringtool:
return self.measuringtool.to_dict()
return None

13
shopdb/core/models/dashboarddefault.py
View File

@@ -11,11 +11,14 @@ from .base import BaseModel
# Display role -> the frontend kiosk path it maps to. THE one place a role is
# named, and the reason the names look like this: these are the literal values a
# person types into C:\Enrollment\display-type.txt on the kiosk itself, which the
# GE-Enforce dispatcher reads to pick a target. Two vocabularies used to exist -
# core said 'partskiosk' where the kiosk said '3DPrintRoom' - so a display could
# report a role core could not store and core could store a role no kiosk would
# ever match. The machine's own file wins, because that is what a person edits.
# person types into C:\Enrollment\display-type.txt on the display PC itself,
# which the launcher on that PC reads to pick a target. Two vocabularies used to
# exist - core said 'partskiosk' where the display said '3DPrintRoom' - so a
# display could report a role core could not store and core could store a role
# no display would ever match. The machine's own file wins, because that is what
# a person edits. The vocabulary is deliberately not trimmed to the pages this
# build happens to serve: a display reports what its file says, and core must be
# able to store that answer whether or not a plugin renders the page.
DISPLAY_ROLE_PATHS = {
'Dashboard': '/shopfloor',
'Lobby': '/tv',

50
shopdb/core/models/script_library.py
View File

@@ -6,6 +6,8 @@ body never lives on the endpoint ahead of time and never crosses the command
wire; the agent downloads it with its collector credential and verifies sha256.
"""
import sqlalchemy as sa
from shopdb.extensions import db
from .base import BaseModel
@@ -16,6 +18,26 @@ RUNCONTEXT_SYSTEM = 'system'
RUNCONTEXT_USER = 'user'
RUNCONTEXTS = (RUNCONTEXT_SYSTEM, RUNCONTEXT_USER)
# How the agent treats a PowerShell NON-TERMINATING error.
#
# DEFAULT is what PowerShell does on its own: the error is written, the script
# keeps going, and powershell.exe still exits 0. That is why a script could
# report succeeded while doing none of its work - the exit code answers "did
# the interpreter reach the end", not "did the script do its job".
#
# STRICT prepends $ErrorActionPreference = 'Stop' so the first error terminates
# and the exit code becomes non-zero, which makes the reported verdict mean
# what an operator assumes it means.
#
# Default is DEFAULT, deliberately. Strict changes execution semantics: a body
# that tolerates a failing Remove-Item, probes with Get-LocalUser, or loops past
# a bad item would start aborting partway. Existing scripts were authored under
# the permissive rule, so this is opt-in per script AFTER the body has been read
# rather than a library-wide switch flipped on code nobody re-checked.
ERRORMODE_DEFAULT = 'default'
ERRORMODE_STRICT = 'strict'
ERRORMODES = (ERRORMODE_DEFAULT, ERRORMODE_STRICT)
class Script(BaseModel):
"""ShopDB-hosted PowerShell script the endpoint agent runs on push."""
@@ -32,6 +54,34 @@ class Script(BaseModel):
# 'system' or 'user' (see RUNCONTEXTS). Per-script so a drive-mapping script
# can target the user session while a machine-wide script runs as SYSTEM.
runcontext = db.Column(db.String(16), default=RUNCONTEXT_SYSTEM, nullable=False)
# Optional folder to run the script IN. Empty means the agent's staging
# directory, which is the right default for a self-contained script and the
# wrong one for a script that shells out to a corporate tool delivered by
# Intune (Sysinternals, vendor CLIs) and referenced by relative path.
#
# This grants NOTHING. The body is already arbitrary PowerShell the agent
# runs as SYSTEM, and it could Set-Location anywhere itself; the security
# boundary is endpoints.runscript plus the digest-pinned library, not the
# working directory. Recorded here so this is not later mistaken for a
# privilege surface and gated harder than it needs to be.
workingdirectory = db.Column(db.String(255))
# 'default' or 'strict' (see ERRORMODES). Opt-in per script because strict
# turns non-terminating errors fatal and would change how existing bodies
# run. Not nullable with a default so a row written before this column
# existed reads as the permissive behavior it was authored under.
errormode = db.Column(db.String(16), default=ERRORMODE_DEFAULT, nullable=False,
server_default=ERRORMODE_DEFAULT)
# Whether a SUCCESSFUL run sends its output back to the server. Failures
# always do: a failed run has to be diagnosable. Success is opt-in because
# script output carries whatever the body printed - hostnames, paths, a
# token an author echoed while debugging - and a run nobody needs to
# diagnose is not worth that exposure by default. The endpoint log keeps the
# full copy either way, so this only controls what crosses to the server.
#
# Off by default so existing scripts keep the behavior they were authored
# under, same rule as errormode above.
captureoutput = db.Column(db.Boolean, default=False, nullable=False,
server_default=sa.false())
def to_dict(self):
result = super().to_dict()

270
shopdb/core/models/user.py
View File

@@ -19,6 +19,25 @@ rolepermissions = db.Table(
db.Column('permissionid', db.Integer, db.ForeignKey('permissions.permissionid'), primary_key=True)
)
# Association table for DIRECT per-user permissions (many-to-many).
#
# A grant here is an ordinary permission, identical in power to one held through
# a role - it counts toward the API token scope ceiling and everything else. It
# exists because a site of 5-7 operators should be able to give one person one
# capability without minting a role for them, which would otherwise turn the
# role list into a mixture of job titles and single-capability flags.
#
# Anything that answers "what may this user do" MUST fold this in. There are two
# such surfaces and they are computed separately: haspermission() (the predicate
# authorization uses) and getpermissions() (the list baked into JWT claims and
# read by the frontend authStore). Teaching only one of them makes the server and
# the UI disagree silently.
userpermissions = db.Table(
'userpermissions',
db.Column('userid', db.Integer, db.ForeignKey('users.userid'), primary_key=True),
db.Column('permissionid', db.Integer, db.ForeignKey('permissions.permissionid'), primary_key=True)
)
class Permission(db.Model):
"""
@@ -35,7 +54,7 @@ class Permission(db.Model):
# Core permission catalog. This is CORE ONLY - permissions for a plugin
# domain (machines, computers, printers, network, knowledgebase,
# notifications, usb, warranty, measuringtools) live in that plugin's
# notifications, warranty) live in that plugin's
# BasePlugin.get_permissions() hook, not here (plugin-is-the-product). The
# merged core+plugins view is full_permission_catalog() below.
CORE_PERMISSIONS = [
@@ -79,6 +98,13 @@ class Permission(db.Model):
# perms, and this perm gates no other route). See collector.py.
('collector.ingest', 'Submit collector payloads (fleet reporting)',
'collector'),
# Employee directory. This one is CORE even though nothing here owns an
# employee table: global search serves an employee domain straight out
# of the external HR database (Config.EMPLOYEE_DB_*, see
# shopdb/utils/employee_db.py), and search.py gates that domain on this
# permission. Leave it out and the gate can never be satisfied, so a
# working feature is unreachable.
('employees.view', 'View the employee directory', 'employees'),
]
def __repr__(self):
@@ -179,6 +205,157 @@ CORE_GUEST_GRANTABLE = {
}
# ---------------------------------------------------------------------------
# Prebuilt user scopes
# ---------------------------------------------------------------------------
#
# Why these exist: before them the only roles were `admin` and `guest`, so a new
# user was given nothing, held no permissions, and the sidebar correctly hid
# every link. There was no rung between "runs the site" and "sees nothing".
#
# The four scopes, as specified:
# admin - unchanged. Bypasses every check in haspermission(); NOT seeded
# here and NOT described by a grant list, because it holds
# everything by rule rather than by grant.
# operator - full record access except deletion, which stays admin-only.
# technician - read-only on records. Cannot create or edit.
# reporter - reads everything and exports reports. No action capability.
#
# DERIVED, not hardcoded, and this is the opposite choice to
# CORE_GUEST_GRANTABLE above. That set is declared name-by-name because it
# governs what an ANONYMOUS visitor may reach, where the bar is "could this be
# public forever" and a wrong answer leaks to the internet. These scopes govern
# authenticated internal staff, and the catalog is plugin-driven: roughly sixty
# permissions across fifteen plugins, growing whenever a plugin is added. A
# hardcoded list would silently omit every future plugin, so a Technician would
# lose access to new modules by default and the failure would look like a
# permissions bug rather than a stale list.
#
# So scopes are computed from the catalog by verb, minus an explicit exclusion
# set. Adding a plugin extends the scopes automatically; keeping something OUT
# of them is the thing that requires a deliberate code change.
# Verbs that mutate records. Split by which scope may hold them.
_EDIT_VERBS = ('create', 'edit', 'restock', 'manage', 'publish')
_DESTRUCTIVE_VERBS = ('delete', 'purge')
# Never granted by ANY prebuilt scope, regardless of verb. Each is either an
# administrative capability, a machine-to-machine credential, or a payload that
# a scope-holder should not get merely for being staff.
SCOPE_NEVER_GRANT = {
# Administrative surface. A scope is not a way to become an admin.
'settings.view', 'settings.edit',
# users.view is deliberately ABSENT: every scope may see who exists, so an
# operator can tell whether an account is already there before asking an
# admin to make one. Creating, editing, and deleting accounts stay
# admin-only, so seeing the directory grants no power over it.
'users.create', 'users.edit', 'users.delete',
'audit.view', 'audit.purge',
'apitokens.create',
# Service-token permissions: these authorize machines, not people. Granting
# one to a human scope would let a person impersonate the collector.
'collector.ingest',
# Bytes, not metadata: SYSTEM-context PowerShell bodies and vendor
# installers. Deliberately separate from applications.view for exactly this
# reason (see the CORE_PERMISSIONS comment), so scopes must not undo that.
'applications.download',
# The big four endpoint actions. These are per-user direct grants chosen by
# checkbox in the user editor, NOT role grants - that is the whole point of
# the Phase A2 split. A scope that granted them would take the choice away
# from the person creating the account.
'endpoints.restart',
'endpoints.runscript',
'endpoints.updateapp',
'endpoints.corptools',
# Operator report screenshots are an uncontrolled capture of the operator's
# screen and may carry shopfloor content, so who sees them is a per-account
# decision rather than something a scope confers.
'endpoints.operatorreports',
# Fleet-wide and agent-distribution capabilities, admin-only by nature.
'endpoints.fleet',
'endpoints.agentpackage',
'endpoints.agentupdate',
}
# Endpoint actions offered as per-user checkboxes in the user editor. Held here
# rather than in the endpoints plugin because the USER EDITOR is what needs the
# list, and core cannot import a plugin. Kept in sync by
# test_scope_endpoint_actions_match_the_plugin.
SCOPE_ENDPOINT_ACTIONS = (
'endpoints.restart',
'endpoints.runscript',
'endpoints.updateapp',
'endpoints.corptools',
'endpoints.operatorreports',
)
def _permission_verb(name):
"""The verb half of a dotted permission name ('assets.edit' -> 'edit')."""
return name.rsplit('.', 1)[-1] if '.' in name else ''
def scope_permission_names(scope, catalog=None):
"""Permission names a prebuilt scope grants, derived from the catalog.
Returns a sorted list so seeding and tests are deterministic.
reporter - every non-excluded permission whose verb is neither a mutation
nor a destruction, plus reports.export. The export is a read
that produces a file, not an action against a machine, and the
user asked for it by name.
technician - the same reads. Identical to reporter at ROLE level: the two
differ by what the user editor offers alongside the role, where
a technician may be given endpoint action checkboxes and a
reporter may not.
operator - the reads plus the editing verbs, still never the destructive
ones. Deletion stays admin-only, per the user's instruction.
"""
if catalog is None:
catalog = full_permission_catalog()
names = {entry[0] for entry in normalize_permission_entries(catalog)}
names -= SCOPE_NEVER_GRANT
readable = {
name for name in names
if _permission_verb(name) not in _EDIT_VERBS
and _permission_verb(name) not in _DESTRUCTIVE_VERBS
}
if scope in ('reporter', 'technician'):
return sorted(readable)
if scope == 'operator':
return sorted(
readable | {
name for name in names
if _permission_verb(name) in _EDIT_VERBS
})
raise ValueError(f'Unknown scope: {scope!r}')
# The seeded scopes, in the order the user editor should offer them. `admin` is
# absent on purpose: it already exists, is created by `seed admin`, and holds
# everything by bypass rather than by grant.
PREBUILT_SCOPES = (
('operator',
'Full access to records except deletion, which stays with admins. '
'Endpoint actions are granted per user by checkbox.',
'#0D6EFD'),
('technician',
'Read-only on records: cannot create, edit, or delete, and cannot author '
'policy. Endpoint actions are granted per user by checkbox.',
'#0DCAF0'),
('reporter',
'Views everything and exports reports. Holds no action capability and '
'cannot be granted one.',
'#6F42C1'),
)
# Scopes that may NOT hold the per-user endpoint action grants, enforced
# server-side in the user API. Reporter's defining property is that it holds no
# action capability EVEN when someone tries to grant one directly.
SCOPES_DENIED_ENDPOINT_ACTIONS = frozenset({'reporter'})
def guest_grantable_permissions():
"""Which permissions the guest role may be granted.
@@ -297,6 +474,60 @@ class Role(BaseModel):
))
return True
@classmethod
def seedscopes(cls):
"""Create the prebuilt scope roles and sync their grants.
Returns {'created': [rolename...], 'granted': {rolename: n}}.
Idempotent, and ADDITIVE on re-run. A second run creates nothing and
adds only grants the role is missing; it never REMOVES a permission from
an existing role.
That asymmetry is deliberate. Re-running has to add, because the grant
lists are derived from a plugin-driven catalog: install a plugin, re-run
the seed, and Technicians should gain read access to the new module
without anyone editing a list. But it must not remove, because the role
grid is an administrative surface and a site may have deliberately given
Operator something extra. A revoking seed would silently undo that
decision on the next deploy, which is a worse failure than a role
holding one grant more than the default - the first is invisible, the
second is on screen in the role editor.
Consequence worth stating plainly: this cannot be used to TIGHTEN a
scope after the fact. If a permission is ever added to
SCOPE_NEVER_GRANT, roles already holding it keep it, and revoking is a
separate deliberate migration.
"""
catalog = full_permission_catalog()
result = {'created': [], 'granted': {}}
for rolename, description, color in PREBUILT_SCOPES:
role = cls.query.filter_by(rolename=rolename).first()
if role is None:
role = cls(rolename=rolename, description=description,
color=color)
db.session.add(role)
db.session.flush()
result['created'].append(rolename)
wanted = set(scope_permission_names(rolename, catalog=catalog))
held = {p.name for p in role.permissions}
missing = wanted - held
if not missing:
continue
# Only link permissions that actually exist as rows. A name in the
# catalog with no Permission row means `seed permissions` has not
# run for that plugin yet; the next run picks it up.
rows = Permission.query.filter(Permission.name.in_(missing)).all()
for permission in rows:
role.permissions.append(permission)
if rows:
result['granted'][rolename] = len(rows)
return result
def getpermissionnames(self) -> list:
"""Get list of permission names."""
if self.rolename == 'admin':
@@ -332,6 +563,14 @@ class User(BaseModel):
backref=db.backref('users', lazy='dynamic')
)
# Permissions granted to this user directly, outside any role. See the
# userpermissions table comment for why these exist and what must fold them.
directpermissions = db.relationship(
'Permission',
secondary=userpermissions,
lazy='joined'
)
def __repr__(self):
return f"<User {self.username}>"
@@ -347,18 +586,43 @@ class User(BaseModel):
return any(r.rolename == rolename for r in self.roles)
def haspermission(self, permission_name: str) -> bool:
"""Check if user has a specific permission through any role."""
"""Check if user has a permission, through any role or directly.
A direct grant is not a lesser kind of permission: it is honoured here
exactly like a role grant, which is what keeps the API token scope
ceiling correct without any change to the ceiling itself. Both ceiling
checks (mint time in core/api/apitokens.py and per-request in
utils/authz.py) call this method.
"""
# Admin role has all permissions
if self.hasrole('admin'):
return True
if any(p.name == permission_name for p in self.directpermissions):
return True
return any(r.haspermission(permission_name) for r in self.roles)
def getdirectpermissionnames(self) -> list:
"""Permission names granted to this user directly, ignoring roles.
The user editor needs the DIRECT grants alone so a checkbox reflects
what is stored on the user rather than what a role happens to supply;
getpermissions() deliberately cannot answer that because it folds both.
"""
return sorted(p.name for p in self.directpermissions)
def getpermissions(self) -> list:
"""Get list of all permission names from all roles."""
"""Get list of all permission names, from roles and direct grants.
This is the LIST surface: it feeds JWT claims and the frontend
authStore. It must agree with haspermission(), the predicate surface, or
the UI hides controls the server would allow (or worse, shows ones it
would refuse).
"""
if self.hasrole('admin'):
return [p[0] for p in full_permission_catalog()]
perms = set()
for role in self.roles:
perms.update(role.getpermissionnames())
perms.update(p.name for p in self.directpermissions)
return list(perms)

2
shopdb/core/services/csvimport.py
View File

@@ -16,7 +16,7 @@ So every foreign key here accepts EITHER a numeric id OR the referenced row's
name, and names are resolved at import time:
assetnumber,assettypeid,locationid,businessunitid
CMM-01,Measuring Tool,Building 1 Bay 3,Quality
MILL-01,Machine,Building 1 Bay 3,Machining
The column keeps its database name - the convention in CONTRIBUTING.md - while
the VALUE is whatever the operator actually knows. A name that does not resolve

69
shopdb/data/catalog.json
View File

@@ -3437,75 +3437,6 @@
"marketingname": "C7000 / C7100 Waste Cartridge"
}
],
"measuringtooltypes": [
{
"name": "Bore Gage",
"description": "Bore / hole diameter gage",
"color": "#7934f3"
},
{
"name": "Caliper",
"description": "Vernier / digital caliper",
"color": "#14abef"
},
{
"name": "CMM"
},
{
"name": "Eddy Current"
},
{
"name": "Form Tracer"
},
{
"name": "Gage Block Set",
"description": "Reference gage block set",
"color": "#e83e8c"
},
{
"name": "Genspect"
},
{
"name": "Height Gage",
"description": "Height / vertical measuring gage",
"color": "#ffc107"
},
{
"name": "Indicator",
"description": "Dial / test indicator",
"color": "#11cdef"
},
{
"name": "Inspection"
},
{
"name": "Measuring Machine"
},
{
"name": "Micrometer",
"description": "Outside / inside / depth micrometer",
"color": "#2dce89"
},
{
"name": "Other",
"description": "Other measuring tool",
"color": "#6c757d"
},
{
"name": "Spline Checker"
},
{
"name": "Thread Gage",
"description": "Go / no-go thread gage",
"color": "#fb6340"
},
{
"name": "Vision System"
},
{
"name": "Wax Trace"
}
],
"notificationtypes": [
{
"typename": "Awareness",

91
shopdb/plugins/__init__.py
View File

@@ -4,8 +4,9 @@ from pathlib import Path
from typing import Dict, List, Optional
from flask import Flask
import logging
import re
from .base import BasePlugin, PluginMeta
from .base import BasePlugin, PluginMeta, renderable_settings_routes
from .registry import PluginRegistry, PluginState
from .loader import PluginLoader
from .migrations import PluginMigrationManager
@@ -102,6 +103,22 @@ class PluginManager:
for name, plugin in plugins.items():
self._register_plugin_components(plugin)
self._check_settings_routes(name, plugin)
def _check_settings_routes(self, name: str, plugin: BasePlugin) -> None:
"""Say at boot, once per plugin, if a declared settings tab is dead.
Only some settings hubs draw plugin tabs (see
SETTINGS_HUBS_WITH_PLUGIN_TABS). A route for any other hub is accepted
by the hook and returned by the API and then rendered by nobody, which
looks from every side like the plugin is at fault. The API drops such a
route; this makes the same finding at startup, so it is in the log of a
site that never opens the settings page."""
try:
renderable_settings_routes(name, plugin.get_settings_routes(),
logger)
except Exception:
logger.exception("get_settings_routes failed for %s", name)
def upgrade_all_plugins(self) -> Dict[str, str]:
"""Run pending Alembic migrations for every discovered plugin.
@@ -119,6 +136,13 @@ class PluginManager:
# upgrade is the only moment left to seed them. Runs even with no
# migration manager - it is independent of the Alembic chains.
self.sync_all_plugin_settings()
# Permissions need the same treatment for the same reason. A permission
# added in a later plugin version would otherwise exist only after a
# manual `flask seed permissions`, and until someone ran it the route
# guarding on that name would refuse every caller - including admins on
# sites that gate by permission rather than role. Deploys run
# upgrade-all, so this is the moment the new name has to appear.
self.sync_all_plugin_permissions()
if not self.migration_manager:
return results
# Only ADOPTED plugins (those in the registry) get migrated. A plugin
@@ -158,6 +182,51 @@ class PluginManager:
touched += 1
return touched
def sync_all_plugin_permissions(self) -> int:
"""Seed every enabled plugin's declared permissions, then relink scopes.
Returns the number of plugins whose permissions were seeded. Best-effort
per plugin: one bad plugin must not abort a deploy's upgrade pass.
The scope relink is the load-bearing half. Creating the Permission row
only makes the name exist; the prebuilt scopes derive their grants from
the catalog, so a permission seeded without a following
Role.seedscopes() belongs to nobody but admin. The symptom is a plugin
that installs cleanly and whose UI is invisible to every non-admin until
someone re-runs the seed by hand. Ordering matters for the same reason
it does in `flask seed permissions`: seedscopes only links rows that
already exist, so it runs last.
"""
touched = 0
for name in list(self.registry.get_all().keys()):
if not self.registry.is_enabled(name):
continue
try:
plugin = self.loader.load_plugin(name, self._app, self._db)
except Exception:
logger.exception(
"Could not load %s to sync its permissions", name)
continue
if plugin:
self._seed_plugin_permissions(plugin)
touched += 1
# Additive: this can hand a scope a newly declared permission, never
# take one away, so a site's deliberate role edits survive the deploy.
from shopdb.core.models import Role
try:
with self._app.app_context():
result = Role.seedscopes()
self._db.session.commit()
granted = sum((result or {}).get('granted', {}).values())
if granted:
logger.info(
"Linked %d new scope grant(s) after plugin upgrade",
granted)
except Exception:
logger.exception("Could not relink prebuilt scopes after upgrade")
return touched
def get_declared_plugin_settings(self) -> Dict[str, dict]:
"""Return {key: declared entry} across all loaded plugins.
@@ -468,11 +537,25 @@ class PluginManager:
return True
def enable_plugin(self, name: str) -> bool:
"""Enable a disabled plugin."""
if not self.registry.is_installed(name):
logger.error(f"Plugin {name} is not installed")
"""Enable a disabled plugin. Auto-installs if available but not installed."""
# Security: Validate plugin name to prevent path traversal
# Only allow alphanumeric, underscore, and hyphen (matches existing plugins)
if not re.match(r'^[a-zA-Z0-9_-]+$', name):
logger.error(f"Invalid plugin name format: {name}")
return False
# Reject reserved names that could be used for directory traversal
if name in ('.', '..', ''):
logger.error(f"Invalid plugin name: {name}")
return False
# Auto-install if available but not installed (UX: enabling should just work)
if not self.registry.is_installed(name):
logger.info(f"Plugin {name} not installed, installing first")
if not self.install_plugin(name):
logger.error(f"Failed to install plugin {name}")
return False
if self.registry.is_enabled(name):
logger.info(f"Plugin {name} is already enabled")
return True

56
shopdb/plugins/alembic_template.py
View File

@@ -1,8 +1,8 @@
"""Shared Alembic env.py logic for bundled plugins.
Every bundled plugin that owns tables (computers, employees, knowledgebase,
machines, network, notifications, printers, slides, usb, warranty) has a
`migrations/env.py` that does the minimum:
Every bundled plugin that owns tables (backups, computers, endpoints, guides,
knowledgebase, machines, network, notifications, printers, slides, tools,
warranty) has a `migrations/env.py` that does the minimum:
import os
os.environ['PLUGIN_NAME'] = 'computers'
@@ -23,6 +23,13 @@ the core chain.
Plugin tables must be importable via `plugins.<name>.models`. Plugins
register their `__tablename__` set in PLUGIN_TABLE_OWNERS below so the
filter is explicit (avoids depending on import-side-effect global state).
ORPHAN_TABLE_OWNERS below is the other half of that map, and it exists because
the core chain creates tables that no bundled plugin owns any more. Those rows
are not a plugin's, so they must not be in PLUGIN_TABLE_OWNERS - nothing can
import their models and no chain owns them - but they still land on every site,
and `flask plugin prune-schema` only knows how to drop a table it can attribute
to an owner. Drop the entry and the table becomes undroppable.
"""
from __future__ import annotations
@@ -51,35 +58,48 @@ PLUGIN_TABLE_OWNERS: dict[str, Iterable[str]] = {
'computers': ('computertypes', 'computers', 'computerinstalledapps',
'accessprotocols', 'computeraccess',
'configurationpolicies', 'configurationpolicyactions',
'businesspolicies', 'businesspolicyitems'),
'employees': ('directoryemployees',),
'endpoints': ('endpointstates', 'endpointevents', 'endpointcommands', 'endpointapplications', 'endpointmonitors', 'operatorreports'),
'geenforce': ('manifestscopes', 'manifestentries', 'manifestentrypctypes',
'manifestentryhostnames', 'manifestentrymachinenumbers',
'manifestinusechecks', 'manifestinusecheckprocesses',
'manifestpublishedversions', 'manifestpayloads',
'manifestblobs',
'manifestenforcementreports', 'manifestenforcementresults',
'pctypealiases'),
'businesspolicies', 'businesspolicyitems',
'policyactionholds'),
'endpoints': ('endpointstates', 'endpointevents', 'endpointcommands',
'endpointapplications', 'endpointmonitors',
'operatorreports', 'endpointhealthstates',
'endpointhealthtransitions', 'endpointmonitorimpacts',
'endpointlocks', 'endpointlogcollections',
'endpointcredentials', 'endpointenrollrequests',
'endpointmigrationwaves', 'endpointbootstraptokens'),
'guides': ('guides', 'guiderevisions', 'guideimages'),
'knowledgebase': ('knowledgebase',),
'machines': ('machinetypes', 'machines'),
'measuringtools': ('measuringtooltypes', 'measuringtools'),
'network': ('networkdevicetypes', 'networkdevices', 'vlans', 'subnets'),
'notifications': ('notificationtypes', 'notifications'),
'printedparts': ('printeditems', 'printeditemtransactions',
'printeditemfiles'),
'notifications': ('notificationtypes', 'notifications', 'notificationacknowledgements'),
'printers': ('printertypes', 'printers', 'modelsupplies', 'printerdrivers',
'printersupplyalerts', 'printerobservedqueues'),
'slides': ('tvslides',),
# Tech Tools was schemaless until secure send: every other tool in it
# runs in the browser and owns nothing.
'tools': ('securesendsecrets',),
'usb': ('usbdevicetypes', 'usbdevices', 'usbcheckouts'),
'warranty': ('warranties', 'warrantyassets'),
}
# Tables the CORE Alembic chain creates that NO bundled plugin owns.
#
# They are not dead weight by accident: the core baseline (68b3947ae14f) creates
# usbdevicetypes/usbdevices/usbcheckouts and 7d16_directoryemployees creates
# directoryemployees, and neither migration can be removed - the whole plugin
# chain set anchors on that head, so editing it breaks `flask db upgrade` on a
# fresh database. The tables therefore arrive at every site whether or not
# anything reads them.
#
# Keyed by the name of the plugin that used to own them, because that is the
# name an operator recognises in prune-schema output. A key here is NOT a plugin
# this build ships, and nothing imports models for it.
ORPHAN_TABLE_OWNERS: dict[str, Iterable[str]] = {
'employees': ('directoryemployees',),
'usb': ('usbdevicetypes', 'usbdevices', 'usbcheckouts'),
}
def _get_plugin_metadata(plugin_name: str) -> MetaData:
"""Import the plugin's models and return a MetaData containing only its
declared tables (filtered via PLUGIN_TABLE_OWNERS)."""

176
shopdb/plugins/base.py
View File

@@ -1,4 +1,17 @@
"""Base plugin class that all plugins must inherit from."""
"""Base plugin class that all plugins must inherit from.
One rule that is not expressible as a method signature, so it lives here where
a plugin author reads it: A PLUGIN MUST NOT HOLD A DB SESSION ACROSS EXTERNAL
I/O. Finish with the database, then make the HTTP call / run the subprocess /
touch the share, then open a fresh session to write the result. Under gevent a
slow external call does not block its worker, but it does pin one of that
worker's pooled database connections for its whole duration, and enough
concurrent greenlets on such a path drain the pool and stall every other
request in the process - the whole site, every plugin, with the CPU idle and
MySQL healthy. See CONTRIBUTING.md ("A plugin must not hold a DB session across
external I/O") for the shape to write, and docker-compose.yml for how the pool
relates to the worker count and to MySQL max_connections.
"""
from abc import ABC, abstractmethod
from typing import List, Dict, Optional, Type
@@ -6,6 +19,41 @@ from dataclasses import dataclass, field
from flask import Flask, Blueprint
# Settings hubs whose Vue page actually draws plugin-contributed tabs: it asks
# /api/plugins/settings-routes, keeps the routes whose `hub` matches its own id,
# and registers them with router.addRoute(). Every other hub renders its own
# tabs and nothing else, so a route naming one would be returned by the API and
# never drawn - a tab the author declared correctly that simply does not exist
# for the user. Hubs are added here only when their .vue file gains that code;
# tests/test_core/test_plugin_settings_hubs.py fails if the two drift apart.
SETTINGS_HUBS_WITH_PLUGIN_TABS = ('model-catalog',)
def renderable_settings_routes(plugin_name: str, routes: Optional[List[Dict]],
logger) -> List[Dict]:
"""The subset of a plugin's declared settings routes that a hub can draw.
Anything naming a hub outside SETTINGS_HUBS_WITH_PLUGIN_TABS is dropped and
logged as an error naming the plugin, the hub and the supported set. The
declaration is not a crash - the plugin's API, models and pages are all
fine - so this refuses the dead tab rather than the plugin. The hard stop
lives in the test suite, where the author still has a chance to fix it.
"""
kept = []
for route in routes or []:
hub = route.get('hub')
if hub in SETTINGS_HUBS_WITH_PLUGIN_TABS:
kept.append(route)
continue
logger.error(
"Plugin %s declares settings route '%s' for hub '%s'. No settings "
"hub page registers plugin tabs for that hub, so the tab would "
"never render; dropping it. Hubs that support plugin tabs: %s",
plugin_name, route.get('name'), hub,
', '.join(SETTINGS_HUBS_WITH_PLUGIN_TABS))
return kept
@dataclass
class PluginMeta:
"""Plugin metadata container."""
@@ -16,7 +64,8 @@ class PluginMeta:
author: str = ""
dependencies: List[str] = field(default_factory=list)
# Default to the current pre-1.0 contract range; tighten when 1.0 lands
core_version: str = ">=0.2.0,<1.0.0"
# 0.20.0: added get_asset_type_descriptor hook (additive, backward-compatible)
core_version: str = ">=0.20.0,<1.0.0"
api_prefix: str = None
def __post_init__(self):
@@ -78,9 +127,9 @@ class BasePlugin(ABC):
"""Transparency note shown when a site enables this plugin.
Return None for plugins that need no special setup. For plugins that
create extra tables (e.g. a self-hosted directory or USB tables), return:
create extra tables (a revision history, an events log), return:
{
'tables': ['directoryemployees', ...], # created in the shopdb DB
'tables': ['backuprevisions', ...], # created in the shopdb DB
'note': 'Plain-language what/why.',
'docs': 'plugins/<name>/README.md', # where the schema lives
}
@@ -128,6 +177,56 @@ class BasePlugin(ABC):
"""
return []
def get_settings_routes(self) -> List[Dict]:
"""Frontend routes this plugin adds to settings pages.
Returns a list of route configurations for the plugin's settings pages.
Each route is a dict with:
hub - settings hub to attach to; ONLY a hub named in
SETTINGS_HUBS_WITH_PLUGIN_TABS (top of this module) draws
plugin tabs. A route for any other hub is dropped by
/api/plugins/settings-routes with an error in the log.
path - route path relative to hub (e.g. 'operatingsystems')
name - unique route name (e.g. 'model-catalog-os')
tab - tab label shown in hub UI (e.g. 'Operating Systems')
component- path to the Vue component AS IT EXISTS IN THE BUILD, i.e.
relative to frontend/src, which for a plugin means
'.plugins-staged/<name>/views/<Component>.vue' - the copy
scripts/stage-frontend.mjs makes of plugins/<name>/frontend
Example:
return [{
'hub': 'model-catalog',
'path': 'operatingsystems',
'name': 'model-catalog-os',
'tab': 'Operating Systems',
'component': '.plugins-staged/computers/views/OperatingSystemsList.vue'
}]
Three constraints the shape of this hook does not show, so read them
before you rely on it (ADR-019, "Constraints as implemented"):
1. Declare a hub only if it is in SETTINGS_HUBS_WITH_PLUGIN_TABS. Today
that is 'model-catalog' alone. The other hubs have no code to draw
your tab, so the route is accepted and dropped.
2. Route the page yourself as well, in plugins/<name>/frontend/
routes.js, and treat THAT as the real route. The hub interpolates
`component` into a dynamic import with no static file extension,
which Vite will not rewrite, so the tab loads under `npm run dev`
and 404s in a production build. A page reachable only through the
tab is a page nobody can open at a site. Your frontend also has to
be staged into the build (SITE_PLUGINS decides that), so enabling a
plugin whose UI was never staged needs a frontend rebuild.
3. The tab is not a link you can hand out. The hub registers the route
when it mounts, so the URL resolves only for someone who has already
been through the parent hub in that page load. Do not link to it
from an email, a doc, or another page, and do not expect a refresh
on it to land anywhere.
Return [] (default) if the plugin adds no settings pages.
"""
return []
def get_settings_defaults(self) -> List[Dict]:
"""Declare the Setting rows this plugin owns.
@@ -211,6 +310,27 @@ class BasePlugin(ABC):
f"implement apply_collector_payload"
)
def resolve_agent_credential(self, secret: str) -> Optional[int]:
"""Asset id for an active per-PC agent credential, else None.
Optional (contract 0.33.0). A plugin that issues endpoint credentials
answers here so the generic collector can accept them on any schema
declaring 'acceptsagentcredential'. Called via
shopdb.api.resolve_agent_credential, which asks enabled plugins and
takes the first non-None answer.
"""
return None
def agent_shared_key_refused(self, hostname: str) -> bool:
"""True when the shared collector key may not report for this hostname.
Optional (contract 0.33.0). True when the shared key is switched off or
the hostname's asset holds an active per-PC credential. Called via
shopdb.api.agent_shared_key_refused, which is True if any enabled
plugin says so.
"""
return False
def on_install(self, app: Flask) -> None:
"""Called when plugin is installed via CLI."""
pass
@@ -418,6 +538,11 @@ class BasePlugin(ABC):
Consumed by GET /api/pluginui/map-overlays. Disabled plugins are
skipped; a broken plugin is isolated in prod, re-raised in dev/test.
No bundled plugin implements this, so the dict above IS the whole
specification - there is no shipped example to copy. Core's side is
exercised against a stub in tests/test_core/test_pluginui.py, which is
also the shortest worked example of a declaration.
"""
return []
@@ -438,5 +563,48 @@ class BasePlugin(ABC):
Consumed by GET /api/pluginui/asset-presentation. Disabled plugins are
skipped; a broken plugin is isolated in prod, re-raised in dev/test.
No bundled plugin implements this either; same note as get_map_overlays
above. With nothing declaring one, the route returns [] and core falls
back to its legacy id-keyed route map in shopdb/core/api/search.py.
"""
return []
def get_asset_type_descriptor(self) -> Optional[Dict]:
"""Return subtype management descriptor for this plugin's asset type.
Declares how to CRUD the plugin's subtype table (printertypes,
machinetypes, etc.) so core can render a generic CRUD UI without
importing the plugin's models or views. Return None if the plugin
has no subtype table.
Contract (all keys required if returning non-None):
{
'assettype': str, # matches assettypes.assettype
'subtype_endpoint': str, # API path WITHOUT /api prefix (e.g., '/printers/types')
'subtype_id_field': str, # PK field name (e.g., 'printertypeid')
'subtype_label_field': str, # Display name field (e.g., 'printertype')
'subtype_description_field': str, # Description field
'subtype_color_field': str, # Color field for map markers
'optional_tabs': List[str], # Plugin-specific tabs (e.g., ['supplies'])
'edit_permission': str, # Required permission (e.g., 'printers.manage')
}
The endpoint must support:
- GET (list) ?active=true/false for pagination
- GET /{id} for single record
- POST (create)
- PUT /{id} (update)
- DELETE /{id}
Payload fields:
- {subtype_label_field}: required string
- {subtype_description_field}: optional text
- {subtype_color_field}: optional string (CSS color)
- isactive: optional boolean (default True)
Consumed by GET /api/pluginui/asset-type-descriptors. Disabled
plugins are skipped; a broken plugin is isolated in prod, re-raised
in dev/test.
"""
return None

19
shopdb/plugins/cli.py
View File

@@ -903,9 +903,10 @@ def prune_schema(yes: bool, force: bool):
Schema-lean per-site DBs: the shared core Alembic baseline creates every
plugin's tables, so a lean site that omits a plugin still carries that
plugin's (empty) tables. This drops the tables of every plugin in
PLUGIN_TABLE_OWNERS that is not installed here, leaving core + chosen-plugin
tables only. Run once at deploy AFTER `flask db upgrade` and
`flask plugin upgrade-all`.
PLUGIN_TABLE_OWNERS that is not installed here, plus the
ORPHAN_TABLE_OWNERS tables the core chain creates for no bundled plugin at
all, leaving core + chosen-plugin tables only. Run once at deploy AFTER
`flask db upgrade` and `flask plugin upgrade-all`.
Dry-run by default; pass --yes to execute. Drops by table name (no plugin
code import) so it works on a lean image where the omitted plugin's
@@ -915,7 +916,8 @@ def prune_schema(yes: bool, force: bool):
from sqlalchemy import create_engine, inspect, text
from shopdb.extensions import db
from shopdb.plugins.alembic_template import PLUGIN_TABLE_OWNERS
from shopdb.plugins.alembic_template import (
ORPHAN_TABLE_OWNERS, PLUGIN_TABLE_OWNERS)
pm = current_app.extensions.get('plugin_manager')
if not pm:
@@ -928,8 +930,15 @@ def prune_schema(yes: bool, force: bool):
# A table is a prune candidate when its owning plugin is not installed here
# AND the table actually exists in this database.
#
# ORPHAN_TABLE_OWNERS is folded in here and nowhere else: those tables have
# no plugin left to install, so they are never in `installed` and are always
# candidates. Without them a lean site carries them for ever, because the
# core chain that creates them cannot be edited.
candidates = dict(PLUGIN_TABLE_OWNERS)
candidates.update(ORPHAN_TABLE_OWNERS)
victims = []
for plugin, tables in PLUGIN_TABLE_OWNERS.items():
for plugin, tables in candidates.items():
if plugin in installed:
continue
for tablename in tables:

23
shopdb/plugins/loader.py
View File

@@ -84,6 +84,29 @@ class PluginLoader:
return self._manifests[name]
manifest_path = self.plugins_dir / name / 'manifest.json'
# Security: Ensure resolved path stays within plugins_dir boundary
# Defense in depth against path traversal, even if name was validated
resolved_manifest = manifest_path.resolve()
resolved_plugins = self.plugins_dir.resolve()
# Python 3.9+ has is_relative_to, fallback for older versions
try:
if not resolved_manifest.is_relative_to(resolved_plugins):
raise PluginNotFoundError(
f'Plugin path escapes plugins directory: {name}',
plugin_name=name,
)
except AttributeError:
# Python < 3.9 fallback using relative_to
try:
resolved_manifest.relative_to(resolved_plugins)
except ValueError:
raise PluginNotFoundError(
f'Plugin path escapes plugins directory: {name}',
plugin_name=name,
)
if not manifest_path.exists():
raise PluginNotFoundError(
f'Plugin {name} has no manifest.json at {manifest_path}',

4
shopdb/plugins/manifest_schema.json
View File

@@ -29,7 +29,7 @@
},
"dependencies": {
"type": "array",
"description": "Hard dependencies: plugin names that must be installed and enabled first. A name may carry an optional PEP440 range (e.g. 'employees>=1.1'); the runtime loader uses name-only semantics.",
"description": "Hard dependencies: plugin names that must be installed and enabled first. A name may carry an optional PEP440 range (e.g. 'computers>=1.1'); the runtime loader uses name-only semantics.",
"items": { "type": "string" }
},
"optional_dependencies": {
@@ -48,7 +48,7 @@
},
"api_prefix": {
"type": "string",
"description": "URL prefix for the plugin blueprint, e.g. '/api/printedparts'. Must be unique across enabled plugins."
"description": "URL prefix for the plugin blueprint, e.g. '/api/warranty'. Must be unique across enabled plugins."
},
"default_enabled": {
"type": "boolean",

5
shopdb/utils/apitoken_auth.py
View File

@@ -190,8 +190,9 @@ def service_token_authorized(scope):
Touches lastusedat on success.
The single contract-surface entry point for unattended SERVICE tokens
(collector.ingest, geenforce.fetch, ...), so plugins authorize a service
token without reaching into core token internals. Returns False on any
(collector.ingest and any permission a plugin declares for a machine caller
of its own), so plugins authorize a service token without reaching into core
token internals. Returns False on any
miss; the caller returns its own 401.
"""
return authorized_service_token(scope) is not None

36
shopdb/utils/authz.py
View File

@@ -125,6 +125,32 @@ def caller_holds(permission_name: str) -> bool:
return True
def caller_is_role(rolename: str) -> bool:
"""Whether THIS caller holds a role, without refusing the request.
The body-answering counterpart to require_role, for a route that serves
everyone but treats one class of caller differently - break-glass override
being the case that prompted it.
A scoped personal token is ALWAYS refused here, matching require_role: scopes
gate individual permissions, not roles. This is the invariant that keeps a
leaked scoped agent token from inheriting override authority just because the
token's owner happens to be an admin. A scoped PAT is a narrower caller than
its owner, including when that owner is an admin.
Guests hold no roles, so an unauthenticated caller is always False.
"""
user = currentuser()
if user is None:
return False
try:
if get_jwt().get('patscopes') is not None:
return False
return user.hasrole(rolename)
except Exception:
return False
def require_permission(permission_name: str):
"""Gate a route behind a single permission. Admin role bypasses.
@@ -182,11 +208,11 @@ def require_permission(permission_name: str):
def require_any_permission(*permission_names):
"""Gate a route behind ANY ONE of several permissions. Admin bypasses.
For a read that two different rights should reach. GE-Enforce and Slides
are the cases: a GET was gated on the MANAGE permission, because managing
was the only thing anybody did there. Splitting a read permission out would
have locked out every existing manager, so a read now accepts either, and
the read permission is the one a site can hand to a guest.
For a read that two different rights should reach. Slides is the case: a
GET was gated on the MANAGE permission, because managing was the only thing
anybody did there. Splitting a read permission out would have locked out
every existing manager, so a read now accepts either, and the read
permission is the one a site can hand to a guest.
A scoped PAT must carry one of the names, and its owner must hold that same
one - the intersection rule of require_permission, applied per name.

18
shopdb/utils/badges.py
View File

@@ -1,11 +1,11 @@
"""What a scanned badge looks like, in one place.
Two kiosks read the same physical badges and disagreed about them. The parts
kiosk understood both shapes a scanner emits; the USB kiosk took the string
raw and compared it to an SSO, so the wrapped form was simply "not recognized".
An operator whose keyboard-wedge scanner emits `0270015376BZ` could take a
3D-printed part and could not check out a drive - with the same badge, at two
cabinets a few metres apart.
Two callers read the same physical badges and disagreed about them. One
understood both shapes a scanner emits; the other took the string raw and
compared it to an SSO, so the wrapped form was simply "not recognized". The
same operator, with the same badge, was recognised at one screen and not at the
next. Anything that turns a scan into a person uses this module, so there is
one answer to the question.
The shapes:
@@ -17,9 +17,9 @@ of NON-ASCII digits is not, however much it looks like one: Arabic-Indic digits
pass Python's str.isdigit(), and a badge written in them was accepted, stored as
the employee's SSO, and joined to nothing afterwards.
This lives in core rather than in either plugin because a plugin may not import
another plugin, and because a badge is a property of the site, not of a
cabinet.
This lives in core rather than in a plugin because a plugin may not import
another plugin, and because a badge is a property of the site. Its consumer
today is the notifications plugin, which resolves an employee by SSO.
"""
import re

38
shopdb/utils/cmmc_usb_db.py
View File

@@ -1,38 +0,0 @@
"""Connection helper for the CMMC USB check-in/out database.
Credentials are pulled from app config (env-backed, see Config.CMMC_USB_DB_*),
never hardcoded. Used by the USB plugin to track device check-in/out against a
separate MySQL database (cmmc_usb), mirroring the read-only employee-directory
pattern in employee_db.py. Unlike the employee DB this one is read-write, so
callers commit their own writes and close the connection in a finally block.
"""
import pymysql
from flask import current_app
def _setting_or_config(setting_key, config_key):
"""Non-secret config: a saved Setting wins, else the env-backed app config.
Lets the setup wizard edit host/name/user without touching .env, while the
password stays env-only.
"""
from shopdb.core.models import Setting
try:
row = Setting.query.filter_by(key=setting_key).first()
if row and row.value:
return row.value
except Exception:
pass
return current_app.config[config_key]
def cmmc_usb_connection():
"""Open a pymysql connection to the cmmc_usb DB."""
return pymysql.connect(
host=_setting_or_config('cmmc_usb_db_host', 'CMMC_USB_DB_HOST'),
user=_setting_or_config('cmmc_usb_db_user', 'CMMC_USB_DB_USER'),
password=current_app.config['CMMC_USB_DB_PASSWORD'],
database=_setting_or_config('cmmc_usb_db_name', 'CMMC_USB_DB_NAME'),
cursorclass=pymysql.cursors.DictCursor,
)

3
shopdb/utils/import_mode.py
View File

@@ -5,7 +5,8 @@ needs two things the normal API withholds:
1. Preserve each row's original createddate/modifieddate instead of stamping
"now" on insert/update.
2. Backdate event history such as USB checkouts to when they really happened.
2. Backdate event history - a notification acknowledgement, an endpoint
event - to when it really happened rather than to the import run.
Both are gated. The caller must BOTH be an admin AND send the request header
`X-Import-Mode: true`. Outside import mode every helper here is a no-op, so

64
shopdb/utils/ipallowlist.py Normal file
View File

@@ -0,0 +1,64 @@
"""Network-position allowlists for unauthenticated kiosk routes.
A kiosk route writes without a login. The device token (see the kiosk views)
covers unattended screens an administrator provisions once, but it does not fit
the other caller: somebody opening the kiosk page on their own PC, who never
logs in and whose browser nobody provisioned. Per-browser provisioning would
mean touching every PC on the floor and losing it again on the next reimage.
What those callers have in common is where they are on the network.
Fails closed, in every direction that matters:
- an empty or absent setting matches nothing (the feature is off),
- an unparseable caller address matches nothing,
- a malformed entry in the list is skipped rather than widening the match.
SPOOF RESISTANCE IS A DEPLOYMENT PROPERTY, NOT A PROPERTY OF THIS CODE.
The caller address comes from client_ip(), which reads remote_addr and never
the raw X-Forwarded-For header - reading that header directly would let any
caller send 'X-Forwarded-For: <allowlisted-ip>' and walk straight through.
remote_addr is only the real client because of what sits in front of the app:
behind IIS the URL-Rewrite rule overwrites X-Forwarded-For with the real TCP
peer and ProxyFix resolves it into remote_addr (TRUSTED_PROXY_COUNT). Remove
that rule and IIS forwards whatever the caller sent; waitress trusts it because
it arrives from 127.0.0.1, remote_addr becomes attacker-controlled, and every
allowlist built on this becomes bypassable from anywhere on the network. It
does NOT fail closed in that case. The rewrite rule is the control, and every
allowlist a plugin builds on this helper inherits that dependency.
Callers store plain CIDR strings rather than references to subnet records: a
site that does not run the network plugin still has to be able to configure
this, and a stored allowlist must not break when an inventory row is deleted.
"""
import ipaddress
from shopdb.utils.clientip import client_ip
def ip_in_cidrs(address, rawcidrs):
"""True when `address` falls inside any CIDR in the comma-separated list.
Pure and side-effect free so it can be tested without a request context.
"""
raw = (rawcidrs or '').strip()
if not raw:
return False
try:
caller = ipaddress.ip_address((address or '').strip())
except ValueError:
return False
for part in raw.split(','):
part = part.strip()
if not part:
continue
try:
if caller in ipaddress.ip_network(part, strict=False):
return True
except ValueError:
continue
return False
def request_ip_allowlisted(request_obj, rawcidrs):
"""True when the request's client address is inside the allowlist."""
return ip_in_cidrs(client_ip(request_obj), rawcidrs)