Import the wilm agent store and WINC adoption work, with four fixes
Eleven commits from the sister site: a reversible Maximo/WINC machine-adoption import, a versioned agent binary store with upload and rollback, remote-BuildKit CI, tray version reporting, and UI fixes. Applied as a patch again - the published mirror rewrites hashes, so the branches have unrelated histories and git refuses to merge them. Reviewed by three agents before merging. Four things fixed here. THE PUBLISH WORKFLOW WAS DEAD. The new remote-BuildKit step in publish-images.yml sits at column 0 instead of indented under steps:, so the whole file is invalid YAML and GitHub would refuse it. Their entire GHCR distribution channel - which Start-ShopDBDev.ps1 and the compose files consume - was broken at the source and clearly never run. Indented; all five workflows now parse. THE IMPORT'S APPROVAL CHECK COULD NOT FAIL. commit() built the plan and the approval packet from the same upload and then asked the writer whether they matched, so packet_matches_plan compared a hash against itself. The documented contract - "a human approves a specific, frozen plan, and the writer can prove nothing drifted" - did not exist in the web path: an admin could preview file A and commit file B, gated only by the needs-create group NAMES they ticked. The preview's planhash is now required at commit and checked against a freshly rebuilt plan. Their tests committed without previewing, which is what let this through; they go through the real flow now, and three new tests pin it - all confirmed to fail with the check stubbed out. endpoints.agentpackage QUIETLY GREW. It used to mean "wrap the staged exe in a ZIP"; with the upload routes it also means "choose the executable every future package installs and runs as SYSTEM". Before this, staging needed filesystem access to the server. The permission description still said only "generate and download a package", so an admin granting it could not see what they were granting. Reworded. NOTHING RECORDED WHAT WAS STAGED, OR BY WHOM. The manifest held version, size and timestamp; re-uploading a version replaces the bytes in place under the same label. A sha256 of the stored file is now recorded, and the upload is audit-logged with it - after an incident there was previously no answer to who put that binary on the fleet. Also documented the nine new routes in docs/api-inventory.json (that gate cannot run on the mirror, so it falls to us at promotion time again), regenerated the spec and the project map, and kept our own docs where they conflicted: docs/ is scrubbed from the mirror, so their DEV-BUILD-TEST.md still had the dev-be channel mapping we corrected, and their PLUGIN-HOOKS.md was a new 45-line file rather than our 737-line canonical one. Their journal section is appended to ours. Contract version comes across as 0.30.0 for a genuinely additive surface. Suite 2369, frontend 324, gate green. Scope: product
This commit is contained in:
cproudlock
154
shopdb/core/services/wincimport.py
Normal file
154
shopdb/core/services/wincimport.py
Normal file
@@ -0,0 +1,154 @@
|
||||
"""WINC machine-adoption import service (preview / commit / undo / list).
|
||||
|
||||
Web-facing wrapper over the validated WINC CLI helpers. The CLI in
|
||||
scripts/site_imports/winc drives the same analyzer, writer, and journal; this
|
||||
module adapts them to a Flask request: it works on the request-scoped
|
||||
db.session connection (so the route owns the transaction and commit) and returns
|
||||
plain JSON-serialisable dicts for the API.
|
||||
|
||||
WINC is NOT a source adapter like Azure/Thor. It never creates assets - it
|
||||
updates existing machines only, requires acknowledgment before creating business
|
||||
units, and records every change to the reversible import journal. That contract
|
||||
lives here and in the writer, not in the generic canonical-CSV importer.
|
||||
"""
|
||||
|
||||
from shopdb.extensions import db
|
||||
from scripts.site_imports import journal
|
||||
from scripts.site_imports.approval import build_packet
|
||||
from scripts.site_imports.winc import writer as winc_writer
|
||||
from scripts.site_imports.winc.dry_run import analyze_rows, parse_export_text
|
||||
|
||||
|
||||
class WincImportError(Exception):
|
||||
"""An uploaded export could not be analyzed or applied as WINC."""
|
||||
|
||||
|
||||
def _connection():
|
||||
return db.session.connection()
|
||||
|
||||
|
||||
def _plan_and_packet(export_text):
|
||||
"""Parse an uploaded export, build the plan and the approval packet."""
|
||||
rows = parse_export_text(export_text)
|
||||
if not rows:
|
||||
raise WincImportError('the uploaded file had no rows')
|
||||
if 'assetnum' not in rows[0]:
|
||||
raise WincImportError(
|
||||
'this does not look like a WINC asset export (no assetnum column)')
|
||||
|
||||
plan, summary, _lookups = analyze_rows(_connection(), rows)
|
||||
packet = build_packet('winc-machine-adoption', plan, summary,
|
||||
winc_writer._PACKET_GROUPS)
|
||||
return plan, summary, packet
|
||||
|
||||
|
||||
def _report(summary, packet):
|
||||
"""Shape the packet + summary into the JSON the UI renders."""
|
||||
return {
|
||||
'kind': packet['kind'],
|
||||
'planhash': packet['planhash'],
|
||||
'rowcount': packet['rowcount'],
|
||||
'summary': dict(summary),
|
||||
'needscreate': {group: dict(items)
|
||||
for group, items in packet.get('needscreate', {}).items()},
|
||||
'unresolved': {group: dict(items)
|
||||
for group, items in packet.get('unresolved', {}).items()},
|
||||
'blocking': packet.get('blocking', False),
|
||||
}
|
||||
|
||||
|
||||
def preview(export_text):
|
||||
"""Read-only: analyze the export and return the report the UI shows.
|
||||
|
||||
Writes nothing. The whole point is to surface what a commit WOULD do -
|
||||
counts, the business units it would create (needs acknowledgment), and any
|
||||
unresolved values that block the import.
|
||||
"""
|
||||
_plan, summary, packet = _plan_and_packet(export_text)
|
||||
report = _report(summary, packet)
|
||||
report['committed'] = False
|
||||
return report
|
||||
|
||||
|
||||
def commit(export_text, acknowledge=None, createdby=None, planhash=None):
|
||||
"""Apply the plan reversibly inside the request transaction.
|
||||
|
||||
Rebuilds the plan from the same upload, re-checks it against a freshly built
|
||||
packet (drift, blocking, acknowledgment) and writes through the journal. The
|
||||
caller owns the commit/rollback of db.session. Returns the report plus the
|
||||
batchid on success.
|
||||
|
||||
planhash is the one the PREVIEW returned. It is what makes the approval mean
|
||||
anything: the packet handed to the writer is built from this same upload, so
|
||||
the writer's drift check compared a hash against itself and could never fail.
|
||||
An admin previewing file A and committing file B - or the same file after the
|
||||
locations or aliases moved underneath it - got whatever the second plan said,
|
||||
gated only by the needs-create group NAMES they happened to tick.
|
||||
|
||||
Passing it is optional so the CLI path, which builds and applies in one go,
|
||||
is unaffected. The web route requires it.
|
||||
|
||||
Raises WincImportError (mapped to a 400 by the route) when the plan is
|
||||
blocking, a needs-create group was not acknowledged, or the plan no longer
|
||||
matches the one that was approved.
|
||||
"""
|
||||
plan, summary, packet = _plan_and_packet(export_text)
|
||||
if planhash is not None and packet['planhash'] != planhash:
|
||||
raise WincImportError(
|
||||
'This plan is not the one that was previewed - the upload or the '
|
||||
'data behind it changed. Preview again and re-check what it would '
|
||||
'create before committing.')
|
||||
try:
|
||||
batchid = winc_writer.apply_plan(
|
||||
_connection(), plan, packet,
|
||||
acknowledge=acknowledge, createdby=createdby)
|
||||
except winc_writer.WriterError as exc:
|
||||
raise WincImportError(str(exc))
|
||||
|
||||
report = _report(summary, packet)
|
||||
report['committed'] = True
|
||||
report['batchid'] = batchid
|
||||
return report
|
||||
|
||||
|
||||
def undo(batchid):
|
||||
"""Reverse an applied batch. Caller owns the transaction. Returns a dict."""
|
||||
result = journal.undo_batch(_connection(), batchid,
|
||||
references=winc_writer._UNDO_REFERENCES)
|
||||
return {
|
||||
'batchid': result.batchid,
|
||||
'reverted': result.reverted,
|
||||
'conflicts': [
|
||||
{'operation': conflict.operation, 'table': conflict.table,
|
||||
'targetid': conflict.targetid, 'column': conflict.column,
|
||||
'reason': conflict.reason}
|
||||
for conflict in result.conflicts
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def list_batches(limit=50):
|
||||
"""Applied/undone WINC batches, newest first, for the UI's undo list."""
|
||||
from sqlalchemy import text
|
||||
|
||||
def _iso(value):
|
||||
# PyMySQL hands back datetime objects; SQLite (tests) hands back the
|
||||
# stored ISO string. Normalise both to an ISO string with a Z suffix.
|
||||
if value is None:
|
||||
return None
|
||||
if hasattr(value, 'isoformat'):
|
||||
return value.isoformat() + 'Z'
|
||||
return str(value)
|
||||
|
||||
rows = _connection().execute(text(
|
||||
'SELECT batchid, kind, rowcount, createdat, createdby, status, '
|
||||
'undoneat FROM importbatches WHERE kind = :kind '
|
||||
'ORDER BY createdat DESC LIMIT :limit'),
|
||||
{'kind': 'winc-machine-adoption', 'limit': limit}).fetchall()
|
||||
return [
|
||||
{'batchid': r[0], 'kind': r[1], 'rowcount': r[2],
|
||||
'createdat': _iso(r[3]),
|
||||
'createdby': r[4], 'status': r[5],
|
||||
'undoneat': _iso(r[6])}
|
||||
for r in rows
|
||||
]
|
||||
Reference in New Issue
Block a user