ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Let a share-backed backup say what is in it

Browse Source 
A 'share' kind keeps its bytes on the file share and gives ShopDB only a hash
and a UNC path. That is right for an opaque vendor file, but it means the app
cannot answer the question a tech actually arrives with - what does this part
marker hold, and which pattern disappeared last month. Reading the share from
the app server is not an option: the download routes refuse to mount it on
purpose, because a permissions slip there becomes an unexplained empty panel.

So the collector now accepts an optional manifest alongside the hash: the
index, not the content. Names, sizes, per-file hashes, and the marking system
each pattern was authored for. Advisory throughout - a client that sends none
behaves exactly as before, and dedup still keys on contenthash.

The marker asset gains a Marker Contents card built from it: counts by type,
the controller family, and a diff against the previous snapshot in the same
source chain. Removed files are named rather than counted, because "1 removed"
only tells someone to go and look. Files edited in place are named too - the
file set can be identical while the content hash moved, and reporting no
change there would contradict the reason the revision exists.

The manifest is deliberately NOT covered by contenthash, which attests the
bytes on the share. A client that starts describing itself therefore posts an
unchanged hash, so a no-op backfills a manifest the row does not yet have -
otherwise the card would appear only on markers that had recently changed, and
stay hidden on every healthy one. The same no-op refreshes sharepath, so a
snapshot that moved on the share stops being pointed at where it used to be.

Manifests are validated at ingest rather than at render: a malformed one would
otherwise become the latest revision and fail every view of the asset page
until the content next changed, outliving the client bug that caused it.

sharedir() for the marker kind is overridden to the layout the site actually
uses, so what ShopDB displays and validates matches what the collector writes.
This commit is contained in:
cproudlock
2026-08-25 13:06:49 -04:00
parent 9d61e52ca7
commit 512cc9be76
6 changed files with 712 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

65
plugins/backups/plugin.py
View File

@@ -263,6 +263,15 @@ class BackupsPlugin(BasePlugin):
'type': 'string',
'description': "Full UNC path. Required for 'share' kinds.",
},
'manifest': {
'type': 'object',
'description': "Optional description of a 'share' kind's "
"contents - what is in the snapshot, not "
"the bytes themselves. Lets an asset panel "
"list a snapshot and diff it against the "
"previous one without the app server "
"mounting the share.",
},
'sourcefilename': {'type': 'string'},
'sourcehostname': {
'type': 'string',
@@ -325,6 +334,42 @@ class BackupsPlugin(BasePlugin):
"contenthash and sharepath are required for kind '{}'".format(
kindkey))
# A share kind may also describe itself. The bytes stay on the
# share - that part is unchanged - but a manifest of what the
# snapshot CONTAINS is small, structured, and the only way an asset
# panel can list a part marker's patterns or say which one
# disappeared last month. The alternative is the app server
# mounting SFLD, which the read routes deliberately refuse to do.
#
# It is advisory: dedup still keys on contenthash, so a client that
# sends no manifest behaves exactly as before.
manifest = payload.get('manifest')
if manifest is not None:
# Validated at the door, not at render time. A shallow check
# here would let a malformed manifest become the LATEST
# revision, and every view of the asset panel would then fail
# until the content next changed - a client bug turning into a
# broken page that outlives it.
if not isinstance(manifest, dict):
raise ValueError('manifest must be an object')
files = manifest.get('files')
if files is not None:
if not isinstance(files, list):
raise ValueError('manifest files must be a list')
for entry in files:
if not isinstance(entry, dict):
raise ValueError(
'each manifest file must be an object')
if not isinstance(entry.get('name'), str) or not entry['name']:
raise ValueError(
'each manifest file needs a name')
size = entry.get('bytes')
if size is not None and not isinstance(size, int):
raise ValueError(
'manifest file {!r} has a non-integer size'.format(
entry['name']))
projection = manifest
# Dedup is against the LATEST row of THIS SOURCE's chain, not the
# asset's. Several PCs legitimately share one machine number here - the
# part markers on 0613, 0615 and WJPRT do, and their configs genuinely
@@ -356,6 +401,26 @@ class BackupsPlugin(BasePlugin):
# backup that died six months ago. Cheap: one column on a row that
# already exists, no new history.
latest.lastseenat = datetime.utcnow()
# BACKFILL, on a no-op. The contenthash attests the BYTES, not the
# manifest, so a client that starts describing itself posts the same
# hash as before and would otherwise no-op forever - leaving the
# index unstored on exactly the markers that are healthy and
# unchanging, which is most of them. A revision that already has a
# manifest is left alone; this only fills a hole.
#
# Honest by construction: the hash proves the content is identical,
# so indexing it now describes the same snapshot. lastseenat
# semantics are untouched.
if projection is not None and latest.payload is None:
latest.payload = projection
# Same reasoning for the pointer: a snapshot that moved on the share
# (a changed layout, a renamed asset directory) reports a new
# sharepath with unchanged bytes, and the stored path would
# otherwise go on pointing at a directory nobody writes to.
if sharepath and latest.sharepath != sharepath:
latest.sharepath = sharepath
db.session.flush()
return {
'action': 'noop',