Split shipped monitor bundle into per-monitor policies
The product shipped one Configuration Policy carrying every shopfloor app monitor, wrapped in a shipped Site Policy that applied it to every endpoint. That bundled two decisions a site cannot separate: which monitors exist, and which endpoints get them. A site that wanted eDNC on one PC type and MTConnect on another had to delete the shipped policy and rebuild both halves. Ship one Configuration Policy per monitor with no PC type attribution, and stop shipping a Site Policy at all. Composition is a site decision; the product now supplies parts, not an assembled opinion. Shipped Configuration Policies become editable and deletable. Provenance stays a source tag rather than a lock, which is safe because seeding is insert-if- absent and no reconciling seeder exists, so a site edit is never stomped. The stale comments claiming shipped rows were read-only and reconciled described behavior the code never had.
This commit is contained in:
benjamin.baker
@@ -19,7 +19,7 @@ from shopdb.api import (db, AuditLog, success_response, error_response,
|
||||
from .routes import computers_bp
|
||||
from ..models import (ConfigurationPolicy, ConfigurationPolicyAction,
|
||||
BusinessPolicy, BusinessPolicyItem, POLICY_SOURCE_LOCAL,
|
||||
POLICY_SOURCE_AUTOAPP,
|
||||
POLICY_SOURCE_AUTOAPP, POLICY_SOURCE_SHIPPED,
|
||||
ACTION_MODE_DESIRED, ACTION_MODE_GATE)
|
||||
from ..policyflag import policy_flag_enabled
|
||||
from ..services.apppolicy import disown_if_generated
|
||||
@@ -262,8 +262,12 @@ def update_configuration_policy(policy_id: int):
|
||||
'Configuration policy not found', http_code=404)
|
||||
# An autoapp row is generated from an Application's "Required on all PCs"
|
||||
# checkbox and IS editable: editing it is precisely how an admin takes
|
||||
# ownership. Genuinely external sources (shipped/managed) stay read-only.
|
||||
if policy.source not in (POLICY_SOURCE_LOCAL, POLICY_SOURCE_AUTOAPP):
|
||||
# ownership. A shipped row is editable for the same reason - the product
|
||||
# ships a starting point, not an opinion about one site's floor - and keeps
|
||||
# source='shipped' so provenance survives the edit. Genuinely external
|
||||
# sources (a future managed owner) stay read-only.
|
||||
if policy.source not in (POLICY_SOURCE_LOCAL, POLICY_SOURCE_AUTOAPP,
|
||||
POLICY_SOURCE_SHIPPED):
|
||||
return error_response(ErrorCodes.CONFLICT,
|
||||
'Policy is managed externally and is read-only',
|
||||
http_code=409)
|
||||
@@ -326,9 +330,11 @@ def delete_configuration_policy(policy_id: int):
|
||||
if not policy:
|
||||
return error_response(ErrorCodes.NOT_FOUND,
|
||||
'Configuration policy not found', http_code=404)
|
||||
# As with update: a generated row is the site's to remove, an externally
|
||||
# managed one is not.
|
||||
if policy.source not in (POLICY_SOURCE_LOCAL, POLICY_SOURCE_AUTOAPP):
|
||||
# As with update: a generated or shipped row is the site's to remove, an
|
||||
# externally managed one is not. A site that does not run MTConnect should
|
||||
# be able to delete its monitor policy outright rather than carry it.
|
||||
if policy.source not in (POLICY_SOURCE_LOCAL, POLICY_SOURCE_AUTOAPP,
|
||||
POLICY_SOURCE_SHIPPED):
|
||||
return error_response(ErrorCodes.CONFLICT,
|
||||
'Policy is managed externally and is read-only',
|
||||
http_code=409)
|
||||
|
||||
@@ -27,10 +27,21 @@ from .jsontext import JSONText
|
||||
# owner. Kept a plain string, not an enum column, so adding a source is data.
|
||||
POLICY_SOURCE_LOCAL = 'local'
|
||||
|
||||
# Product-owned rows we ship (the shopfloor app monitor bundle). Same read-only
|
||||
# treatment as any non-local source: the CRUD API refuses to edit or delete
|
||||
# them, the seeder reconciles them. Distinct from 'local' so a site's own
|
||||
# policies are never confused with what the product shipped.
|
||||
# Rows the product seeds (the shopfloor monitor policies, one per monitor).
|
||||
# Distinct from 'local' so a site's own policies are never confused with what
|
||||
# the product shipped, and so the UI can badge the origin.
|
||||
#
|
||||
# A shipped row is EDITABLE and DELETABLE, unlike a genuinely external source.
|
||||
# The product ships a starting point, not an opinion about one site's floor: a
|
||||
# plant that runs no MTConnect must be able to drop that policy. source stays
|
||||
# 'shipped' through an edit, so provenance is a fact about where the row came
|
||||
# from rather than a claim about who controls it now.
|
||||
#
|
||||
# Nothing reconciles these rows. The seeding migrations are insert-if-absent
|
||||
# (they SELECT by name and return early if it exists), so an upgrade cannot
|
||||
# stomp a site's edits. That property is load-bearing now that the rows are
|
||||
# editable - do not replace it with a reconciling seeder without adding a
|
||||
# dirty flag first.
|
||||
POLICY_SOURCE_SHIPPED = 'shipped'
|
||||
|
||||
# Rows generated FOR US by another surface rather than authored by hand: today
|
||||
|
||||
@@ -13,8 +13,9 @@ so the catalog matches what a deployed agent already watches:
|
||||
process agent.exe -> MTConnect
|
||||
service MTConnect UDC Agent -> MTConnect
|
||||
|
||||
Shipped rows are product-owned (source='shipped'): the CRUD API refuses to edit
|
||||
or delete them. Launch paths mirror the agent's launchpaths block; collector and
|
||||
Shipped rows (source='shipped') keep their kind and displayname - that pair is
|
||||
the identity the monitor policies match on - but a site may edit their paths and
|
||||
active flag. Launch paths mirror the agent's launchpaths block; collector and
|
||||
log paths are left null until site paths are supplied.
|
||||
|
||||
Idempotent: creation checks the live schema first so a db.create_all() site
|
||||
|
||||
@@ -1,12 +1,22 @@
|
||||
"""Ship the shopfloor app monitors as a read-only Site Policy.
|
||||
"""Ship the shopfloor app monitors as a Site Policy bundle.
|
||||
|
||||
SUPERSEDED BY 0019. This migration is kept because installs have stamped it, but
|
||||
what it seeds no longer survives an upgrade: 0019 splits this single bundle into
|
||||
one Configuration Policy per monitor and REMOVES the shipped Site Policy, so
|
||||
sites compose their own. Read this for history; do not extend it.
|
||||
|
||||
Two claims below were true when written and are now false, corrected here rather
|
||||
than left to mislead: the seeded rows are NOT read-only (0019 admits 'shipped'
|
||||
at the policy CRUD guards), and no seeder reconciles them - the idempotency
|
||||
check at the top of upgrade() returns early if the policy exists, which is what
|
||||
keeps a site's edits safe across upgrades.
|
||||
|
||||
Wraps the four shipped endpointmonitors (eDNC, Universal Data Collector,
|
||||
MTConnect process, MTConnect UDC Agent service) in a product-owned policy bundle
|
||||
so the agent stops carrying them as hardcoded appsettings.json defaults and reads
|
||||
its watch-list from the server instead.
|
||||
MTConnect process, MTConnect UDC Agent service) in a policy bundle so the agent
|
||||
stops carrying them as hardcoded appsettings.json defaults and reads its
|
||||
watch-list from the server instead.
|
||||
|
||||
Seeds two computers-plugin rows, both source='shipped' (read-only in the UI, the
|
||||
CRUD API refuses to edit or delete them):
|
||||
Seeds two computers-plugin rows, both source='shipped':
|
||||
|
||||
ConfigurationPolicy "Shopfloor App Monitors"
|
||||
scope {"type":"true"} (every endpoint), one action per shipped monitor,
|
||||
|
||||
@@ -0,0 +1,318 @@
|
||||
"""Split the shipped monitor bundle into one policy per monitor.
|
||||
|
||||
0007 seeded a single Configuration Policy "Shopfloor App Monitors" carrying all
|
||||
four shipped monitor actions, wrapped in a shipped Site (business) Policy of the
|
||||
same name. That bundle was all-or-nothing: a site that runs eDNC but no
|
||||
MTConnect could not detach the half it does not have, because the whole row was
|
||||
product-owned and read-only.
|
||||
|
||||
This migration replaces it with one Configuration Policy per shipped monitor:
|
||||
|
||||
Monitor: eDNC (process)
|
||||
Monitor: Universal Data Collector (process)
|
||||
Monitor: MTConnect (process)
|
||||
Monitor: MTConnect (service)
|
||||
|
||||
Each carries exactly one 'monitor:<endpointmonitorid>' action, mode='gate',
|
||||
scope {"type":"true"} - no PC type attribution. The name carries the kind
|
||||
because displayname alone is not unique: MTConnect ships as both a process and
|
||||
a service, and configurationpolicies.name is UNIQUE.
|
||||
|
||||
source stays 'shipped'. Provenance is still true - the product did ship these -
|
||||
and the UI keeps showing where they came from. What changes is that the CRUD
|
||||
guards now admit 'shipped' for edit/delete, so a site can retune or drop one
|
||||
without losing the record of its origin.
|
||||
|
||||
The shipped SITE policy is REMOVED and not replaced. Composition is a site
|
||||
decision: which monitors apply to which endpoints, and at what priority, is
|
||||
exactly the judgement a plant makes about its own floor. Shipping one meant
|
||||
every install inherited "apply all four to every endpoint" as a starting point
|
||||
that could not be edited. Sites now build their own from the per-monitor pieces.
|
||||
|
||||
Removing it is the behaviour change to be aware of on upgrade: monitors that
|
||||
were attached through that bundle stop being attached until a site policy
|
||||
composes them. It is deliberate and it is the point of the split - see the
|
||||
"off by default" direction in the 2026-09-08 plan - but it is not silent, since
|
||||
the per-monitor policies are listed and ready to compose.
|
||||
|
||||
Idempotent on the policy names, so a re-run neither duplicates the split rows
|
||||
nor resurrects the bundle. Guards on the policy tables being present and no-ops
|
||||
if they are not, matching 0007.
|
||||
|
||||
Revision ID: endpoints0019splitmonitorpolicies
|
||||
Revises: endpoints0018endpointlocks
|
||||
"""
|
||||
from alembic import op
|
||||
|
||||
import sqlalchemy as sa
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
|
||||
revision = 'endpoints0019splitmonitorpolicies'
|
||||
down_revision = 'endpoints0018endpointlocks'
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
|
||||
BUNDLE_NAME = 'Shopfloor App Monitors'
|
||||
|
||||
# The shipped monitors, by identity (kind, displayname) - never by hardcoded id,
|
||||
# so this survives id drift across installs exactly as 0007 does.
|
||||
_SHIPPED_MONITORS = (
|
||||
('process', 'eDNC'),
|
||||
('process', 'Universal Data Collector'),
|
||||
('process', 'MTConnect'),
|
||||
('service', 'MTConnect'),
|
||||
)
|
||||
|
||||
_REQUIRED = {'endpointmonitors', 'configurationpolicies',
|
||||
'configurationpolicyactions', 'businesspolicies',
|
||||
'businesspolicyitems'}
|
||||
|
||||
|
||||
def _tables(bind):
|
||||
from sqlalchemy import inspect
|
||||
|
||||
return set(inspect(bind).get_table_names())
|
||||
|
||||
|
||||
def _policy_name(kind, displayname):
|
||||
return f'Monitor: {displayname} ({kind})'
|
||||
|
||||
|
||||
def _policy_description(kind, displayname):
|
||||
noun = 'process' if kind == 'process' else 'service'
|
||||
return (f'Product-shipped watch-list entry for the {displayname} {noun}. '
|
||||
f'Attach it to a Site Policy to apply it to endpoints.')
|
||||
|
||||
|
||||
def _config_policy_id(bind, name):
|
||||
return bind.execute(
|
||||
sa.text('SELECT configurationpolicyid FROM configurationpolicies '
|
||||
'WHERE name = :name'),
|
||||
{'name': name}).scalar()
|
||||
|
||||
|
||||
def _drop_config_policy(bind, name):
|
||||
"""Delete a configuration policy, its actions, and any items citing it."""
|
||||
policyid = _config_policy_id(bind, name)
|
||||
if policyid is None:
|
||||
return
|
||||
# Clear children explicitly rather than lean on ON DELETE CASCADE, which
|
||||
# sqlite does not enforce by default.
|
||||
bind.execute(
|
||||
sa.text('DELETE FROM businesspolicyitems '
|
||||
'WHERE configurationpolicyid = :id'), {'id': policyid})
|
||||
bind.execute(
|
||||
sa.text('DELETE FROM configurationpolicyactions '
|
||||
'WHERE configurationpolicyid = :id'), {'id': policyid})
|
||||
bind.execute(
|
||||
sa.text('DELETE FROM configurationpolicies '
|
||||
'WHERE configurationpolicyid = :id'), {'id': policyid})
|
||||
|
||||
|
||||
def _drop_business_policy(bind, name):
|
||||
policyid = bind.execute(
|
||||
sa.text('SELECT businesspolicyid FROM businesspolicies '
|
||||
'WHERE name = :name AND source = :source'),
|
||||
{'name': name, 'source': 'shipped'}).scalar()
|
||||
if policyid is None:
|
||||
return
|
||||
bind.execute(
|
||||
sa.text('DELETE FROM businesspolicyitems WHERE businesspolicyid = :id'),
|
||||
{'id': policyid})
|
||||
bind.execute(
|
||||
sa.text('DELETE FROM businesspolicies WHERE businesspolicyid = :id'),
|
||||
{'id': policyid})
|
||||
|
||||
|
||||
def _monitor_id(bind, kind, displayname):
|
||||
return bind.execute(
|
||||
sa.text('SELECT endpointmonitorid FROM endpointmonitors '
|
||||
'WHERE kind = :kind AND displayname = :displayname'),
|
||||
{'kind': kind, 'displayname': displayname}).scalar()
|
||||
|
||||
|
||||
def upgrade():
|
||||
bind = op.get_bind()
|
||||
if not _REQUIRED.issubset(_tables(bind)):
|
||||
return
|
||||
|
||||
now = datetime.utcnow()
|
||||
|
||||
config_policies = sa.table(
|
||||
'configurationpolicies',
|
||||
sa.column('name', sa.String),
|
||||
sa.column('description', sa.Text),
|
||||
sa.column('scopejson', sa.Text),
|
||||
sa.column('source', sa.String),
|
||||
sa.column('createddate', sa.DateTime),
|
||||
sa.column('modifieddate', sa.DateTime),
|
||||
sa.column('isactive', sa.Boolean),
|
||||
)
|
||||
actions = sa.table(
|
||||
'configurationpolicyactions',
|
||||
sa.column('configurationpolicyid', sa.Integer),
|
||||
sa.column('action', sa.String),
|
||||
sa.column('mode', sa.String),
|
||||
sa.column('desiredstate', sa.Text),
|
||||
sa.column('params', sa.Text),
|
||||
sa.column('intervalminutes', sa.Integer),
|
||||
sa.column('sortorder', sa.Integer),
|
||||
sa.column('createddate', sa.DateTime),
|
||||
sa.column('modifieddate', sa.DateTime),
|
||||
sa.column('isactive', sa.Boolean),
|
||||
)
|
||||
|
||||
for kind, displayname in _SHIPPED_MONITORS:
|
||||
name = _policy_name(kind, displayname)
|
||||
if _config_policy_id(bind, name) is not None:
|
||||
continue
|
||||
monitorid = _monitor_id(bind, kind, displayname)
|
||||
if monitorid is None:
|
||||
# A site that never had this monitor gets no policy for it.
|
||||
continue
|
||||
|
||||
op.bulk_insert(config_policies, [{
|
||||
'name': name,
|
||||
'description': _policy_description(kind, displayname),
|
||||
'scopejson': '{"type": "true"}',
|
||||
'source': 'shipped',
|
||||
'createddate': now,
|
||||
'modifieddate': now,
|
||||
'isactive': True,
|
||||
}])
|
||||
op.bulk_insert(actions, [{
|
||||
'configurationpolicyid': _config_policy_id(bind, name),
|
||||
'action': f'monitor:{monitorid}',
|
||||
'mode': 'gate',
|
||||
'desiredstate': None,
|
||||
'params': None,
|
||||
'intervalminutes': None,
|
||||
'sortorder': 0,
|
||||
'createddate': now,
|
||||
'modifieddate': now,
|
||||
'isactive': True,
|
||||
}])
|
||||
|
||||
# Retire the monolith. The business policy goes first so its item rows are
|
||||
# gone before the configuration policy it cited is removed.
|
||||
_drop_business_policy(bind, BUNDLE_NAME)
|
||||
_drop_config_policy(bind, BUNDLE_NAME)
|
||||
|
||||
|
||||
def downgrade():
|
||||
"""Reinstate the single shipped bundle and drop the per-monitor policies.
|
||||
|
||||
Rebuilt from the same monitor identities rather than from anything the
|
||||
upgrade stashed, so it lands in the state 0007 produces regardless of what
|
||||
a site did to the split rows in between.
|
||||
"""
|
||||
bind = op.get_bind()
|
||||
if not _REQUIRED.issubset(_tables(bind)):
|
||||
return
|
||||
|
||||
now = datetime.utcnow()
|
||||
|
||||
monitor_ids = []
|
||||
for kind, displayname in _SHIPPED_MONITORS:
|
||||
monitorid = _monitor_id(bind, kind, displayname)
|
||||
if monitorid is not None:
|
||||
monitor_ids.append(monitorid)
|
||||
|
||||
for kind, displayname in _SHIPPED_MONITORS:
|
||||
_drop_config_policy(bind, _policy_name(kind, displayname))
|
||||
|
||||
if not monitor_ids or _config_policy_id(bind, BUNDLE_NAME) is not None:
|
||||
return
|
||||
|
||||
config_policies = sa.table(
|
||||
'configurationpolicies',
|
||||
sa.column('name', sa.String),
|
||||
sa.column('description', sa.Text),
|
||||
sa.column('scopejson', sa.Text),
|
||||
sa.column('source', sa.String),
|
||||
sa.column('createddate', sa.DateTime),
|
||||
sa.column('modifieddate', sa.DateTime),
|
||||
sa.column('isactive', sa.Boolean),
|
||||
)
|
||||
op.bulk_insert(config_policies, [{
|
||||
'name': BUNDLE_NAME,
|
||||
'description': 'Product-shipped watch-list for the shopfloor apps '
|
||||
'(eDNC, Universal Data Collector, MTConnect). Read-only.',
|
||||
'scopejson': '{"type": "true"}',
|
||||
'source': 'shipped',
|
||||
'createddate': now,
|
||||
'modifieddate': now,
|
||||
'isactive': True,
|
||||
}])
|
||||
config_policy_id = _config_policy_id(bind, BUNDLE_NAME)
|
||||
|
||||
actions = sa.table(
|
||||
'configurationpolicyactions',
|
||||
sa.column('configurationpolicyid', sa.Integer),
|
||||
sa.column('action', sa.String),
|
||||
sa.column('mode', sa.String),
|
||||
sa.column('desiredstate', sa.Text),
|
||||
sa.column('params', sa.Text),
|
||||
sa.column('intervalminutes', sa.Integer),
|
||||
sa.column('sortorder', sa.Integer),
|
||||
sa.column('createddate', sa.DateTime),
|
||||
sa.column('modifieddate', sa.DateTime),
|
||||
sa.column('isactive', sa.Boolean),
|
||||
)
|
||||
op.bulk_insert(actions, [{
|
||||
'configurationpolicyid': config_policy_id,
|
||||
'action': f'monitor:{monitorid}',
|
||||
'mode': 'gate',
|
||||
'desiredstate': None,
|
||||
'params': None,
|
||||
'intervalminutes': None,
|
||||
'sortorder': index,
|
||||
'createddate': now,
|
||||
'modifieddate': now,
|
||||
'isactive': True,
|
||||
} for index, monitorid in enumerate(monitor_ids)])
|
||||
|
||||
business_policies = sa.table(
|
||||
'businesspolicies',
|
||||
sa.column('name', sa.String),
|
||||
sa.column('description', sa.Text),
|
||||
sa.column('priority', sa.Integer),
|
||||
sa.column('source', sa.String),
|
||||
sa.column('createddate', sa.DateTime),
|
||||
sa.column('modifieddate', sa.DateTime),
|
||||
sa.column('isactive', sa.Boolean),
|
||||
)
|
||||
op.bulk_insert(business_policies, [{
|
||||
'name': BUNDLE_NAME,
|
||||
'description': 'Product-shipped Site Policy that applies the shopfloor '
|
||||
'app monitors to every endpoint. Read-only.',
|
||||
'priority': 100,
|
||||
'source': 'shipped',
|
||||
'createddate': now,
|
||||
'modifieddate': now,
|
||||
'isactive': True,
|
||||
}])
|
||||
business_policy_id = bind.execute(
|
||||
sa.text('SELECT businesspolicyid FROM businesspolicies '
|
||||
'WHERE name = :name'), {'name': BUNDLE_NAME}).scalar()
|
||||
|
||||
items = sa.table(
|
||||
'businesspolicyitems',
|
||||
sa.column('businesspolicyid', sa.Integer),
|
||||
sa.column('configurationpolicyid', sa.Integer),
|
||||
sa.column('sortorder', sa.Integer),
|
||||
sa.column('createddate', sa.DateTime),
|
||||
sa.column('modifieddate', sa.DateTime),
|
||||
sa.column('isactive', sa.Boolean),
|
||||
)
|
||||
op.bulk_insert(items, [{
|
||||
'businesspolicyid': business_policy_id,
|
||||
'configurationpolicyid': config_policy_id,
|
||||
'sortorder': 0,
|
||||
'createddate': now,
|
||||
'modifieddate': now,
|
||||
'isactive': True,
|
||||
}])
|
||||
@@ -113,7 +113,9 @@ EXPECTED_HEAD_REVISION['warranty'] = 'warranty0002proof'
|
||||
# and endpointmonitorimpacts.
|
||||
# 0018 adds endpointlocks: the admin quarantine refusing actions from non-admins
|
||||
# on one endpoint, a caller-authority axis separate from the policy hold.
|
||||
EXPECTED_HEAD_REVISION['endpoints'] = 'endpoints0018endpointlocks'
|
||||
# 0019 splits the 0007 bundle into one shipped Configuration Policy per monitor
|
||||
# and drops the shipped Business Policy, so sites compose their own.
|
||||
EXPECTED_HEAD_REVISION['endpoints'] = 'endpoints0019splitmonitorpolicies'
|
||||
|
||||
# Plugins built after the cutover: their 0001 baseline really creates tables the
|
||||
# core chain never owned.
|
||||
|
||||
@@ -1,141 +0,0 @@
|
||||
"""The shipped shopfloor app monitors ride a read-only Site Policy.
|
||||
|
||||
Migration endpoints0007shippedpolicy offloads the eDNC/UDC/MTConnect monitors
|
||||
from the agent's hardcoded appsettings.json into a product-owned policy bundle:
|
||||
a shipped ConfigurationPolicy "Shopfloor App Monitors" (global scope, one
|
||||
desired-state monitor action per shipped monitor) wrapped in a shipped
|
||||
BusinessPolicy "Shopfloor App Monitors".
|
||||
|
||||
Run the endpoints chain against a fresh engine, alongside the computers policy
|
||||
tables the bundle writes into, and assert the bundle exists, is source='shipped'
|
||||
(read-only), and attaches every shipped monitor as a desired/running:true action.
|
||||
"""
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from sqlalchemy import create_engine, inspect, text
|
||||
from alembic.migration import MigrationContext
|
||||
from alembic.operations import Operations
|
||||
|
||||
|
||||
_ROOT = Path(__file__).resolve().parent.parent.parent
|
||||
_ENDPOINTS_VERSIONS = _ROOT / 'plugins' / 'endpoints' / 'migrations' / 'versions'
|
||||
_COMPUTERS_VERSIONS = _ROOT / 'plugins' / 'computers' / 'migrations' / 'versions'
|
||||
|
||||
CONFIG_POLICY_NAME = 'Shopfloor App Monitors'
|
||||
BUSINESS_POLICY_NAME = 'Shopfloor App Monitors'
|
||||
|
||||
|
||||
def _load(path, prefix):
|
||||
spec = importlib.util.spec_from_file_location(prefix + path.stem, str(path))
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
def _run_bundle_migrations(connection):
|
||||
"""Build the policy tables (computers 0004), the monitor catalog (endpoints
|
||||
0006) and the shipped bundle (endpoints 0007) on one engine, in order.
|
||||
|
||||
The core baseline creates every plugin's tables before per-plugin chains run,
|
||||
so on a real deploy the computers policy tables already exist when endpoints
|
||||
0007 runs. Here we stand up exactly the three migrations the bundle depends
|
||||
on - the earlier endpoints revisions carry FKs to core tables that a bare
|
||||
engine does not have, and are irrelevant to what 0007 seeds.
|
||||
"""
|
||||
operations = Operations(MigrationContext.configure(connection))
|
||||
operations._install_proxy()
|
||||
try:
|
||||
_load(_COMPUTERS_VERSIONS / '0004_policies.py',
|
||||
'computers_mig_').upgrade()
|
||||
_load(_ENDPOINTS_VERSIONS / '0006_endpoint_monitors.py',
|
||||
'endpoints_mig_0006_').upgrade()
|
||||
_load(_ENDPOINTS_VERSIONS / '0007_shipped_monitor_policy.py',
|
||||
'endpoints_mig_0007_').upgrade()
|
||||
finally:
|
||||
operations._remove_proxy()
|
||||
connection.commit()
|
||||
|
||||
|
||||
def test_shipped_bundle_seeds_read_only_site_policy():
|
||||
engine = create_engine('sqlite://')
|
||||
with engine.connect() as connection:
|
||||
_run_bundle_migrations(connection)
|
||||
|
||||
config = connection.execute(text(
|
||||
'SELECT configurationpolicyid, source, scopejson '
|
||||
'FROM configurationpolicies WHERE name = :name'),
|
||||
{'name': CONFIG_POLICY_NAME}).first()
|
||||
assert config is not None, 'shipped Configuration Policy was not seeded'
|
||||
config_id, config_source, scopejson = config
|
||||
assert config_source == 'shipped'
|
||||
assert json.loads(scopejson) == {'type': 'true'}
|
||||
|
||||
business = connection.execute(text(
|
||||
'SELECT businesspolicyid, source FROM businesspolicies '
|
||||
'WHERE name = :name'), {'name': BUSINESS_POLICY_NAME}).first()
|
||||
assert business is not None, 'shipped Business Policy was not seeded'
|
||||
business_id, business_source = business
|
||||
assert business_source == 'shipped'
|
||||
|
||||
item = connection.execute(text(
|
||||
'SELECT configurationpolicyid FROM businesspolicyitems '
|
||||
'WHERE businesspolicyid = :id'), {'id': business_id}).first()
|
||||
assert item is not None and item[0] == config_id
|
||||
|
||||
|
||||
def test_shipped_bundle_attaches_every_shipped_monitor():
|
||||
engine = create_engine('sqlite://')
|
||||
with engine.connect() as connection:
|
||||
_run_bundle_migrations(connection)
|
||||
|
||||
monitor_ids = {row[0] for row in connection.execute(text(
|
||||
'SELECT endpointmonitorid FROM endpointmonitors '
|
||||
"WHERE source = 'shipped'"))}
|
||||
assert monitor_ids, 'no shipped monitors to attach'
|
||||
|
||||
config_id = connection.execute(text(
|
||||
'SELECT configurationpolicyid FROM configurationpolicies '
|
||||
'WHERE name = :name'), {'name': CONFIG_POLICY_NAME}).scalar()
|
||||
|
||||
actions = connection.execute(text(
|
||||
'SELECT action, mode, desiredstate FROM configurationpolicyactions '
|
||||
'WHERE configurationpolicyid = :id'), {'id': config_id}).all()
|
||||
|
||||
attached_ids = set()
|
||||
for action, mode, desiredstate in actions:
|
||||
assert action.startswith('monitor:')
|
||||
# Gate, not desired: the shipped bundle reports state and exposes
|
||||
# Start/Stop in the UI, but never auto-starts a process. A desired
|
||||
# running state is a deliberate per-monitor opt-in, not shipped.
|
||||
assert mode == 'gate'
|
||||
assert desiredstate is None
|
||||
attached_ids.add(int(action[len('monitor:'):]))
|
||||
|
||||
assert attached_ids == monitor_ids
|
||||
|
||||
|
||||
def test_shipped_bundle_is_idempotent():
|
||||
engine = create_engine('sqlite://')
|
||||
with engine.connect() as connection:
|
||||
_run_bundle_migrations(connection)
|
||||
# Re-run only the shipped-bundle migration; it must not duplicate rows.
|
||||
operations = Operations(MigrationContext.configure(connection))
|
||||
operations._install_proxy()
|
||||
try:
|
||||
_load(_ENDPOINTS_VERSIONS / '0007_shipped_monitor_policy.py',
|
||||
'endpoints_mig_rerun_').upgrade()
|
||||
finally:
|
||||
operations._remove_proxy()
|
||||
connection.commit()
|
||||
|
||||
config_count = connection.execute(text(
|
||||
'SELECT COUNT(*) FROM configurationpolicies WHERE name = :name'),
|
||||
{'name': CONFIG_POLICY_NAME}).scalar()
|
||||
business_count = connection.execute(text(
|
||||
'SELECT COUNT(*) FROM businesspolicies WHERE name = :name'),
|
||||
{'name': BUSINESS_POLICY_NAME}).scalar()
|
||||
assert config_count == 1
|
||||
assert business_count == 1
|
||||
224
tests/test_plugins/test_endpoints_split_monitor_policies.py
Normal file
224
tests/test_plugins/test_endpoints_split_monitor_policies.py
Normal file
@@ -0,0 +1,224 @@
|
||||
"""The shipped shopfloor monitors ride one Configuration Policy per monitor.
|
||||
|
||||
0007 seeded a single "Shopfloor App Monitors" bundle wrapped in a shipped Site
|
||||
Policy. 0019 replaced that with one policy per monitor and removed the shipped
|
||||
Site Policy, so sites compose their own.
|
||||
|
||||
Run the endpoints chain against a fresh engine, alongside the computers policy
|
||||
tables the migrations write into, and assert the post-0019 shape: a policy per
|
||||
shipped monitor, each carrying exactly one gate-mode monitor action with no PC
|
||||
type attribution, and no shipped Site Policy left behind.
|
||||
|
||||
The idempotency test is the load-bearing one. Nothing reconciles shipped rows,
|
||||
so insert-if-absent is the only thing protecting a site's edits from an upgrade
|
||||
that re-runs a seeding migration - see POLICY_SOURCE_SHIPPED in
|
||||
plugins/computers/models/policy.py.
|
||||
"""
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from sqlalchemy import create_engine, text
|
||||
from alembic.migration import MigrationContext
|
||||
from alembic.operations import Operations
|
||||
|
||||
|
||||
_ROOT = Path(__file__).resolve().parent.parent.parent
|
||||
_ENDPOINTS_VERSIONS = _ROOT / 'plugins' / 'endpoints' / 'migrations' / 'versions'
|
||||
_COMPUTERS_VERSIONS = _ROOT / 'plugins' / 'computers' / 'migrations' / 'versions'
|
||||
|
||||
BUNDLE_NAME = 'Shopfloor App Monitors'
|
||||
SPLIT_MIGRATION = '0019_split_shipped_monitor_policies.py'
|
||||
|
||||
_SHIPPED_MONITORS = (
|
||||
('process', 'eDNC'),
|
||||
('process', 'Universal Data Collector'),
|
||||
('process', 'MTConnect'),
|
||||
('service', 'MTConnect'),
|
||||
)
|
||||
|
||||
|
||||
def _policy_name(kind, displayname):
|
||||
return f'Monitor: {displayname} ({kind})'
|
||||
|
||||
|
||||
def _load(path, prefix):
|
||||
spec = importlib.util.spec_from_file_location(prefix + path.stem, str(path))
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
def _run(connection, *loaded):
|
||||
operations = Operations(MigrationContext.configure(connection))
|
||||
operations._install_proxy()
|
||||
try:
|
||||
for module in loaded:
|
||||
module.upgrade()
|
||||
finally:
|
||||
operations._remove_proxy()
|
||||
connection.commit()
|
||||
|
||||
|
||||
def _run_bundle_migrations(connection):
|
||||
"""Build the policy tables, monitor catalog, bundle and split, in order.
|
||||
|
||||
The core baseline creates every plugin's tables before per-plugin chains
|
||||
run, so on a real deploy the computers policy tables already exist. Here we
|
||||
stand up exactly the migrations the bundle depends on - the earlier
|
||||
endpoints revisions carry FKs to core tables a bare engine does not have.
|
||||
"""
|
||||
_run(
|
||||
connection,
|
||||
_load(_COMPUTERS_VERSIONS / '0004_policies.py', 'computers_mig_'),
|
||||
_load(_ENDPOINTS_VERSIONS / '0006_endpoint_monitors.py',
|
||||
'endpoints_mig_0006_'),
|
||||
_load(_ENDPOINTS_VERSIONS / '0007_shipped_monitor_policy.py',
|
||||
'endpoints_mig_0007_'),
|
||||
_load(_ENDPOINTS_VERSIONS / SPLIT_MIGRATION, 'endpoints_mig_0019_'),
|
||||
)
|
||||
|
||||
|
||||
def _monitor_id(connection, kind, displayname):
|
||||
return connection.execute(text(
|
||||
'SELECT endpointmonitorid FROM endpointmonitors '
|
||||
'WHERE kind = :kind AND displayname = :displayname'),
|
||||
{'kind': kind, 'displayname': displayname}).scalar()
|
||||
|
||||
|
||||
def test_split_seeds_one_policy_per_shipped_monitor():
|
||||
engine = create_engine('sqlite://')
|
||||
with engine.connect() as connection:
|
||||
_run_bundle_migrations(connection)
|
||||
|
||||
for kind, displayname in _SHIPPED_MONITORS:
|
||||
name = _policy_name(kind, displayname)
|
||||
row = connection.execute(text(
|
||||
'SELECT configurationpolicyid, source, scopejson '
|
||||
'FROM configurationpolicies WHERE name = :name'),
|
||||
{'name': name}).first()
|
||||
assert row is not None, f'{name} was not seeded'
|
||||
policyid, source, scopejson = row
|
||||
assert source == 'shipped'
|
||||
# No PC type attribution: the policy applies wherever a site
|
||||
# composes it, and carries no opinion about which types match.
|
||||
assert json.loads(scopejson) == {'type': 'true'}
|
||||
|
||||
actions = connection.execute(text(
|
||||
'SELECT action, mode, desiredstate '
|
||||
'FROM configurationpolicyactions '
|
||||
'WHERE configurationpolicyid = :id'), {'id': policyid}).all()
|
||||
assert len(actions) == 1, f'{name} should carry exactly one action'
|
||||
action, mode, desiredstate = actions[0]
|
||||
# Gate, not desired: a monitor reports state and exposes Start/Stop
|
||||
# but never auto-starts a process. A desired running state is a
|
||||
# deliberate per-monitor opt-in, not something the product ships on.
|
||||
assert mode == 'gate'
|
||||
assert desiredstate is None
|
||||
expected = _monitor_id(connection, kind, displayname)
|
||||
assert action == f'monitor:{expected}'
|
||||
|
||||
|
||||
def test_split_removes_the_shipped_site_policy():
|
||||
engine = create_engine('sqlite://')
|
||||
with engine.connect() as connection:
|
||||
_run_bundle_migrations(connection)
|
||||
|
||||
business = connection.execute(text(
|
||||
'SELECT COUNT(*) FROM businesspolicies WHERE name = :name'),
|
||||
{'name': BUNDLE_NAME}).scalar()
|
||||
assert business == 0, 'the shipped Site Policy must not survive 0019'
|
||||
|
||||
config = connection.execute(text(
|
||||
'SELECT COUNT(*) FROM configurationpolicies WHERE name = :name'),
|
||||
{'name': BUNDLE_NAME}).scalar()
|
||||
assert config == 0, 'the bundled Configuration Policy must be retired'
|
||||
|
||||
items = connection.execute(text(
|
||||
'SELECT COUNT(*) FROM businesspolicyitems')).scalar()
|
||||
assert items == 0
|
||||
|
||||
|
||||
def test_no_shipped_business_policy_remains():
|
||||
"""Sites build their own composition; the product ships none."""
|
||||
engine = create_engine('sqlite://')
|
||||
with engine.connect() as connection:
|
||||
_run_bundle_migrations(connection)
|
||||
|
||||
shipped = connection.execute(text(
|
||||
"SELECT COUNT(*) FROM businesspolicies WHERE source = 'shipped'"
|
||||
)).scalar()
|
||||
assert shipped == 0
|
||||
|
||||
|
||||
def test_split_is_idempotent_and_preserves_site_edits():
|
||||
"""A re-run must not duplicate rows nor stomp an edit.
|
||||
|
||||
Nothing reconciles shipped rows, so this is the whole protection: the
|
||||
migration returns early when the policy name is already present. If that
|
||||
ever becomes a reconciling seeder, this test is what fails.
|
||||
"""
|
||||
engine = create_engine('sqlite://')
|
||||
with engine.connect() as connection:
|
||||
_run_bundle_migrations(connection)
|
||||
|
||||
edited = _policy_name('process', 'eDNC')
|
||||
connection.execute(text(
|
||||
'UPDATE configurationpolicies SET description = :description '
|
||||
'WHERE name = :name'),
|
||||
{'description': 'site edited', 'name': edited})
|
||||
connection.commit()
|
||||
|
||||
_run(connection, _load(_ENDPOINTS_VERSIONS / SPLIT_MIGRATION,
|
||||
'endpoints_mig_rerun_'))
|
||||
|
||||
for kind, displayname in _SHIPPED_MONITORS:
|
||||
count = connection.execute(text(
|
||||
'SELECT COUNT(*) FROM configurationpolicies '
|
||||
'WHERE name = :name'),
|
||||
{'name': _policy_name(kind, displayname)}).scalar()
|
||||
assert count == 1
|
||||
|
||||
description = connection.execute(text(
|
||||
'SELECT description FROM configurationpolicies WHERE name = :name'),
|
||||
{'name': edited}).scalar()
|
||||
assert description == 'site edited', 'a re-run stomped a site edit'
|
||||
|
||||
|
||||
def test_downgrade_restores_the_bundle():
|
||||
engine = create_engine('sqlite://')
|
||||
with engine.connect() as connection:
|
||||
_run_bundle_migrations(connection)
|
||||
|
||||
split = _load(_ENDPOINTS_VERSIONS / SPLIT_MIGRATION,
|
||||
'endpoints_mig_down_')
|
||||
operations = Operations(MigrationContext.configure(connection))
|
||||
operations._install_proxy()
|
||||
try:
|
||||
split.downgrade()
|
||||
finally:
|
||||
operations._remove_proxy()
|
||||
connection.commit()
|
||||
|
||||
config_id = connection.execute(text(
|
||||
'SELECT configurationpolicyid FROM configurationpolicies '
|
||||
'WHERE name = :name'), {'name': BUNDLE_NAME}).scalar()
|
||||
assert config_id is not None, 'downgrade did not restore the bundle'
|
||||
|
||||
business_id = connection.execute(text(
|
||||
'SELECT businesspolicyid FROM businesspolicies WHERE name = :name'),
|
||||
{'name': BUNDLE_NAME}).scalar()
|
||||
assert business_id is not None
|
||||
|
||||
item = connection.execute(text(
|
||||
'SELECT configurationpolicyid FROM businesspolicyitems '
|
||||
'WHERE businesspolicyid = :id'), {'id': business_id}).first()
|
||||
assert item is not None and item[0] == config_id
|
||||
|
||||
for kind, displayname in _SHIPPED_MONITORS:
|
||||
count = connection.execute(text(
|
||||
'SELECT COUNT(*) FROM configurationpolicies '
|
||||
'WHERE name = :name'),
|
||||
{'name': _policy_name(kind, displayname)}).scalar()
|
||||
assert count == 0, 'downgrade left the split policies behind'
|
||||
Reference in New Issue
Block a user