ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

collector: adopt an instrument before minting one, and stop the 500 loops

Browse Source 
Minting derived a measuring tool's asset number from the HOSTNAME, so a
permanent instrument inherited the identity of whichever PC drove it that week:
replace the PC and either the number lies or a second tool appears for the same
physical unit. And because idempotency was keyed on the collector's own label,
it could not see a tool it had not created - on prod that left 43 legacy MT-####
tools shadowed by minted <HOST>-CMM twins, three records deep in places.

Resolution order is now most-stable-identity-first: the instrument named by
measuringtool-id.txt, then a prior collector link, then a tool this PC already
controls that somebody else created, then the reported machine number, and only
then mint. What minting produces should be read as a placeholder until a real
identifier is recorded.

Three separate 500 loops came out of the same mistake, looking a relationship up
by LABEL when assetrelationships is unique on (source, target, type):

- On a CMM the instrument IS the reported bay, so the machine sync has already
  made a row for that exact triple - and it finds its own rows by that label.
  Relabelling hid the link, so the next cycle built a second row for the same
  triple and MySQL rejected it: 200 once, then 500 forever. The machine link is
  now recognised and left alone; adoption only needs the identity.
- A part-marker PC hit it twice over, once on its partof row and once because
  the marker's asset number is derived from the PC and could already be taken.
  Both are get-or-create on the triple now, and an existing asset of that number
  is adopted rather than duplicated.

A named instrument also supersedes a minted twin properly: the stale link is
archived by TARGET, not by object identity, which is what left a PC reading as
the controller of two instruments.

Reported identifiers are matched exactly rather than with ilike. They arrive
from a text file on a shopfloor PC, and ilike reads _ and % as wildcards, so
MT-600_ adopted MT-6001 and a bare % adopted whatever active asset came first.
A named id that is not a measuring tool is refused with a warning rather than
linked as one.
This commit is contained in:
cproudlock
2026-08-14 13:46:40 -04:00
parent 38deefe619
commit d830dd49a9
3 changed files with 592 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files

313
plugins/computers/plugin.py
View File

@@ -106,6 +106,13 @@ class ComputersPlugin(BasePlugin):
'fields': {
'hostname': {'type': 'string', 'required': True},
'machinenumber': {'type': 'string'},
# The instrument this PC drives, named by the enrollment file
# measuringtool-id.txt. SEPARATE from machinenumber on purpose:
# "which bay is this" and "which instrument is this" are
# different facts, and machinenumber is what GE-Enforce
# TargetMachineNumbers gates on - repointing it at a tool would
# silently stop every bay-gated manifest entry matching.
'measuringtoolid': {'type': 'string'},
'pctype': {'type': 'string'},
'pcsubtype': {'type': 'string'},
'serialnumber': {'type': 'string'},
@@ -187,7 +194,7 @@ class ComputersPlugin(BasePlugin):
"""Whether the PC-to-machine alerts may be sent.
Off by default, deliberately. A site may legitimately run several PCs
on one machine number - part markers do at West Jefferson - and there
on one machine number - part markers do at the reference site - and there
both the handover and the contested case fire on normal, correct data,
which is noise rather than news. Turn it on at a site where a machine
number means exactly one PC.
@@ -370,7 +377,9 @@ class ComputersPlugin(BasePlugin):
# Measuring-tool sync: metrology PCs (CMM/Keyence/Genspect/wax-trace)
# get an attached MeasuringTool asset auto-created and linked.
measuringtoollinks = self._sync_measuringtool_link(
comp.asset, pctype, hostname, warnings)
comp.asset, pctype, hostname, warnings,
measuringtoolid=payload.get('measuringtoolid'),
machinenumber=machinenumber)
db.session.commit()
return {
@@ -619,9 +628,7 @@ class ComputersPlugin(BasePlugin):
'run flask seed reference-data')
return []
machine = Asset.query.filter(
Asset.assetnumber.ilike(machinenumber),
Asset.isactive.is_(True)).first()
machine = self._asset_by_number(machinenumber)
if not machine:
# Reported a machine ShopDB does not know. Warn rather than invent
# an asset: a mistyped number would create a machine nobody can
@@ -791,22 +798,48 @@ class ComputersPlugin(BasePlugin):
db.session.flush()
hostname = comp.hostname
markerasset = Asset(
assetnumber='{}-{}'.format(
pcasset.assetnumber or hostname, spec['suffix']),
name='{} ({})'.format(spec['typename'], hostname),
assettypeid=coretype.assettypeid,
statusid=1)
db.session.add(markerasset)
db.session.flush()
db.session.add(Machine(assetid=markerasset.assetid,
machinetypeid=devicetype.machinetypeid))
db.session.add(AssetRelationship(
assetnumber = '{}-{}'.format(
pcasset.assetnumber or hostname, spec['suffix'])
# ADOPT AN EXISTING ASSET OF THAT NUMBER before minting. The number
# is derived from the PC, so it is entirely predictable and may
# already exist - left by an earlier run whose link was archived, or
# created by a person. assetnumber is unique, so inserting a second
# one raised and the whole report 500'd, on every cycle, forever.
#
# Deliberately NOT filtered on isactive: an inactive asset still
# holds the number, and it is the constraint that decides this.
markerasset = Asset.query.filter(
db.func.lower(Asset.assetnumber) == assetnumber.lower()).first()
if markerasset is None:
markerasset = Asset(
assetnumber=assetnumber,
name='{} ({})'.format(spec['typename'], hostname),
assettypeid=coretype.assettypeid,
statusid=1)
db.session.add(markerasset)
db.session.flush()
# The extension row may be missing on an adopted asset, and the link
# may already exist under another label - both get-or-create.
if not Machine.query.filter_by(assetid=markerasset.assetid).first():
db.session.add(Machine(assetid=markerasset.assetid,
machinetypeid=devicetype.machinetypeid))
controlrow = AssetRelationship.query.filter_by(
sourceassetid=pcasset.assetid,
targetassetid=markerasset.assetid,
relationshiptypeid=controls.relationshiptypeid,
label=label,
isactive=True))
relationshiptypeid=controls.relationshiptypeid).first()
if controlrow is not None:
controlrow.isactive = True
if not controlrow.label:
controlrow.label = label
else:
db.session.add(AssetRelationship(
sourceassetid=pcasset.assetid,
targetassetid=markerasset.assetid,
relationshiptypeid=controls.relationshiptypeid,
label=label,
isactive=True))
# One marker per PC: archive any other collector marker link.
for rel in existing:
@@ -847,9 +880,7 @@ class ComputersPlugin(BasePlugin):
'run flask seed reference-data')
return None
operation = Asset.query.filter(
Asset.assetnumber.ilike(machinenumber),
Asset.isactive.is_(True)).first()
operation = self._asset_by_number(machinenumber)
if not operation:
warnings.append(
'no asset for machine number {!r}; marker not filed under an '
@@ -865,19 +896,34 @@ class ComputersPlugin(BasePlugin):
machinenumber))
return None
links = AssetRelationship.query.filter(
AssetRelationship.sourceassetid == markerasset.assetid,
AssetRelationship.relationshiptypeid == partof.relationshiptypeid,
AssetRelationship.label == label,
).all()
found = None
for rel in links:
if rel.targetassetid == operation.assetid:
rel.isactive = True
found = rel
else:
# Our own rows first: a marker that moved to another operation has its
# old membership archived, never deleted, so where it used to live stays
# answerable.
for rel in AssetRelationship.query.filter(
AssetRelationship.sourceassetid == markerasset.assetid,
AssetRelationship.relationshiptypeid == partof.relationshiptypeid,
AssetRelationship.label == label).all():
if rel.targetassetid != operation.assetid and rel.isactive:
rel.isactive = False
if found is None:
# Then get-or-create KEYED ON THE UNIQUE TRIPLE, not on the label.
# assetrelationships is unique on (source, target, type), so a row made
# by hand or by the legacy import - carrying a different label, or none
# at all - was invisible to the label-filtered lookup this used to do,
# and the insert then violated that constraint. The PC reported 200 on
# the cycle that minted the marker and 500 on every cycle after.
found = AssetRelationship.query.filter_by(
sourceassetid=markerasset.assetid,
targetassetid=operation.assetid,
relationshiptypeid=partof.relationshiptypeid).first()
if found is not None:
found.isactive = True
if not found.label:
# Unlabelled means the import or a person made it. Stamping ours
# is what makes the next cycle reuse it. A row carrying somebody
# else's label is left as theirs.
found.label = label
else:
db.session.add(AssetRelationship(
sourceassetid=markerasset.assetid,
targetassetid=operation.assetid,
@@ -1022,18 +1068,39 @@ class ComputersPlugin(BasePlugin):
'PC-superseded alert failed for machine %s',
getattr(machine, 'assetnumber', '?'))
def _sync_measuringtool_link(self, pcasset, pctype, hostname, warnings):
"""Auto-create + link the MeasuringTool a metrology PC drives.
def _sync_measuringtool_link(self, pcasset, pctype, hostname, warnings,
measuringtoolid=None, machinenumber=None):
"""Link the MeasuringTool a metrology PC drives. ADOPT before minting.
A CMM / Keyence / Genspect / wax-and-trace imaging pc-type means the
shopfloor PC controls an attached measuring instrument. This creates
that instrument once as a MeasuringTool asset and a directional
PC->tool 'controls' relationship, tagged MEASURINGTOOL_LINK_ORIGIN so
it is idempotent and self-archiving. The PC's own ComputerType is left
alone (it stays a shopfloor PC). A non-metrology pc-type archives any
collector-created tool link (e.g. a PC re-imaged to another type) but
never deletes the tool asset, which may carry calibration history.
Returns the desired-link list.
A CMM / Keyence / Genspect / wax-and-trace pc-type means the PC controls
an attached instrument. This links it with a directional PC->tool
'controls' relationship tagged MEASURINGTOOL_LINK_ORIGIN.
RESOLUTION ORDER, most stable identity first:
1. measuringtoolid - the instrument named by the enrollment file
measuringtool-id.txt. Explicit, survives a PC swap, and it is what
a Keyence or Genspect bay has instead of a bay id.
2. a prior collector link on this PC (reactivate + retype).
3. an EXISTING measuring tool this PC already controls that the
collector did not create - a legacy MT-#### row, or one somebody
made by hand. Adopting stamps the label so it is ours from then on.
4. the reported machine number, when it resolves to a measuring tool.
This is the CMM case: cmmid.txt already reports CMM4.
5. mint, only when none of the above found anything.
WHY THIS ORDER. Minting derives an asset number from the HOSTNAME, so a
permanent instrument inherited the identity of whichever PC drove it that
week - replace the PC and either the number lies or a second tool appears
for the same physical unit. It also could not see a tool it had not
created itself, so on prod 43 legacy MT-#### tools were shadowed by
minted <HOST>-CMM twins, three records deep in places. Minting is now the
last resort, and what it produces should be treated as a placeholder
until a real identifier is recorded.
A non-metrology pc-type archives any collector-created tool link (a PC
re-imaged to another type) but never deletes the tool asset, which may
carry calibration history. Returns the desired-link list.
"""
from shopdb.api import AssetRelationship, RelationshipType, Asset
from .pctypemap import metrology_tool_for
@@ -1073,11 +1140,75 @@ class ComputersPlugin(BasePlugin):
typename, typedescription = tool_spec
tooltype = self._ensure_measuringtool_type(typename, typedescription)
# Reuse any prior collector link (reactivate + retype) before creating,
# so a re-metrology PC never duplicates the tool asset.
# --- 1. an explicitly named instrument wins over everything ---------
adopted = None
named = (measuringtoolid or '').strip()
if named:
candidate = self._asset_by_number(named)
if candidate is None:
# Warn rather than invent: a typo in the file must not mint a
# phantom instrument that nobody can account for.
warnings.append(
'no asset for measuring tool {!r}; not linked'.format(named))
elif candidate.assetid not in self._measuringtool_assetids():
# The name resolved, but not to an instrument. measuringtool-id
# .txt holding a machine number would otherwise link the PC to
# that MACHINE under a measuring-tool label - a link that reads
# as an instrument everywhere downstream, on a row the machine
# sync also owns. Refuse and say which asset it hit.
warnings.append(
'asset {!r} is not a measuring tool; not linked'.format(
candidate.assetnumber))
else:
adopted = candidate
# --- 2. a prior collector link (reactivate + retype) -----------------
reuse = next((rel for rel in existing if rel.isactive), None) \
or (existing[0] if existing else None)
if reuse:
# --- 3/4. adopt a tool this PC already controls that we did not make --
# Ordered after the collector's own link so a settled PC keeps what it
# has, and before minting so an existing instrument is never twinned.
if adopted is None and reuse is None:
adopted = self._adoptable_measuringtool(
pcasset, controls, machinenumber)
if adopted is not None:
# CLAIM the existing link rather than adding a second one. A row for
# (pc, tool, controls) usually already exists - that is how the tool
# was found - and assetrelationships is unique on exactly that
# triple, so inserting would raise.
claimed = AssetRelationship.query.filter_by(
sourceassetid=pcasset.assetid,
targetassetid=adopted.assetid,
relationshiptypeid=controls.relationshiptypeid).first()
if claimed is not None and claimed.label == MACHINE_LINK_ORIGIN:
# LEAVE IT ALONE. On a CMM the instrument IS the reported bay,
# so the machine sync has already made the row for this exact
# triple - and it finds its own rows BY that label. Relabelling
# hid the link from it, so the next cycle built a second row for
# the same triple and MySQL rejected it: the PC reported 200,
# then 500 forever after. The row's lifecycle (including staying
# dormant while a bay is contested) belongs to the machine sync;
# adoption only needs the identity, which it has.
pass
elif claimed is not None:
# Relabelling is what adoption MEANS for any other row: the link
# is ours from here, so a later cycle reuses it instead of
# minting.
claimed.label = MEASURINGTOOL_LINK_ORIGIN
claimed.isactive = True
else:
db.session.add(AssetRelationship(
sourceassetid=pcasset.assetid,
targetassetid=adopted.assetid,
relationshiptypeid=controls.relationshiptypeid,
label=MEASURINGTOOL_LINK_ORIGIN))
if adopted.measuringtool and tooltype:
adopted.measuringtool.measuringtooltypeid = \
tooltype.measuringtooltypeid
targetid = adopted.assetid
elif reuse:
reuse.isactive = True
toolasset = db.session.get(Asset, reuse.targetassetid)
if toolasset and toolasset.measuringtool and tooltype:
@@ -1111,14 +1242,96 @@ class ComputersPlugin(BasePlugin):
label=MEASURINGTOOL_LINK_ORIGIN))
targetid = toolasset.assetid
# Only one tool link is desired; archive any other collector rows.
# Only one tool link is desired; archive any other collector row.
#
# Keyed on the TARGET, not on object identity. `rel is not reuse` spared
# the row reuse pointed at even when adoption had just resolved a
# different instrument - so a bay whose measuringtool-id.txt named the
# real tool kept its minted twin link alive alongside the adopted one,
# and the PC read as controlling two instruments. That is the exact
# duplicate this whole adoption path exists to remove.
for rel in existing:
if rel is not reuse and rel.isactive:
if rel.targetassetid != targetid and rel.isactive:
rel.isactive = False
return [{'assetid': targetid, 'relationshiptype': 'controls',
'measuringtooltype': typename}]
@staticmethod
def _asset_by_number(assetnumber):
"""The active asset with exactly this asset number, case-insensitively.
NOT `ilike`. Every identifier resolved here arrives from a file on a
shopfloor PC - cmmid.txt, measuringtool-id.txt, the reported machine
number - and ilike reads `_` and `%` in that value as WILDCARDS. A
reported '%' matched whatever active asset happened to come first, and
'MT-600_' matched MT-6001: an identifier silently resolving to an asset
it does not name, then linked to as though it did.
"""
from shopdb.api import Asset
number = (assetnumber or '').strip()
if not number:
return None
return Asset.query.filter(
db.func.lower(Asset.assetnumber) == number.lower(),
Asset.isactive.is_(True)).first()
def _measuringtool_assetids(self):
"""Asset ids that really are measuring tools.
Empty when the measuringtools plugin is not installed, which makes both
callers fall through rather than guess: an asset number alone does not
say what kind of thing it names.
"""
try:
from plugins.measuringtools.models import MeasuringTool
except ImportError:
return set()
return {row.assetid for row in
MeasuringTool.query.with_entities(MeasuringTool.assetid)}
def _adoptable_measuringtool(self, pcasset, controls, machinenumber):
"""A measuring tool this PC already controls that we did not create.
Two sources, in order:
a. any ACTIVE controls link from this PC to a measuring_tool asset
that carries a different label - the legacy import's MT-#### rows,
or one somebody added by hand. These were invisible to the
collector, which keyed idempotency on its OWN label and so minted a
twin for the same physical instrument on every fresh PC.
b. the reported machine number, when it names a measuring tool. That
is the CMM case: cmmid.txt reports CMM4, which already resolves to
a real asset through the machine-link path.
Returns the Asset to adopt, or None. Never creates anything.
"""
from shopdb.api import AssetRelationship, Asset
toolassetids = self._measuringtool_assetids()
if not toolassetids:
return None
# a. already linked, by someone else
for rel in (AssetRelationship.query
.filter(AssetRelationship.sourceassetid == pcasset.assetid,
AssetRelationship.relationshiptypeid
== controls.relationshiptypeid,
AssetRelationship.isactive.is_(True))
.order_by(AssetRelationship.relationshipid.asc()).all()):
if rel.label == MEASURINGTOOL_LINK_ORIGIN:
continue
if rel.targetassetid in toolassetids:
return db.session.get(Asset, rel.targetassetid)
# b. the reported machine number naming a tool
number = (machinenumber or '').strip()
if number:
candidate = self._asset_by_number(number)
if candidate is not None and candidate.assetid in toolassetids:
return candidate
return None
def _ensure_measuringtool_type(self, name, description):
"""Find or create a MeasuringToolType (metrology types are not in the
measuringtools starter seed)."""