Guides: write a procedure in ShopDB, not beside it

The knowledgebase plugin is a link registry - a description, a URL and some
keywords pointing at a document held somewhere else. That leaves the documents
themselves as HTML files on a share: no search, no history, no way to know
whether the copy pinned to a noticeboard is current, and no way for a technician
to tick off where they are.

A guide is stored as an ordered list of typed blocks rather than a blob of
HTML, because three of the four things this feature exists for depend on it. A
tickable step needs a stable id to be remembered; dark mode needs semantic
markup rather than pasted colours; and print needs to know where a step begins
so it does not split one across a page. Free HTML gives none of that and lets
pasted styling fight the theme.

Inline text is a restricted Markdown subset - bold, italic, inline code, links,
line breaks - escaped BEFORE formatting, so nothing an author pasted can reach
the DOM as markup. No fonts, sizes or colours: emphasis is semantic and the
theme decides how it looks. That is what makes a guide written next year match
one written today, and dark mode work without an author thinking about it.

DRAFTS AND VERSIONS. The guide row is the working copy; every save that changes
something writes an immutable revision; and a separate pointer says which
revision READERS get. Without that pointer, editing a live procedure changes it
under a technician who is halfway through following it. Publishing promotes an
existing revision rather than snapshotting, so a publish with unsaved edits
cannot ship a version nobody reviewed. Restoring writes a new revision with old
content, so the record of what people were told in between survives.

Saves dedup on a content hash covering title, summary, tags and blocks, so an
autosave that changed nothing writes no version - otherwise the history fills
with identical entries and hides the three edits that mattered.

Progress lives in the browser, not the database: shopfloor PCs share a login, so
per-user ticks would show one technician's progress to the next person on that
bay, which reads as "already done". Keyed by guide and version, because step 4
of version 3 is not step 4 of version 2.

Five permissions, and the split between edit and publish is the point: a
technician can write a procedure up while somebody else decides it is fit for
the floor. create-without-edit works too - an author may edit their own guide
until it is published, after which other people are following it.

plugins/guides/services/authoring.py

@@ -0,0 +1,134 @@
+"""Saving, revisioning and publishing a guide.
+
+Kept out of the API layer so the rules are testable without a request, and so
+there is exactly ONE path that writes a guide. A second write path is how a
+system ends up with revisions that skipped the dedup, or a published pointer
+that points at nothing.
+"""
+
+from datetime import datetime
+
+from shopdb.api import db
+
+from ..models import Guide, GuideRevision
+from .blocks import validate, contenthash
+
+
+def _utcnow():
+    return datetime.utcnow()
+
+
+def save(guide, title=None, summary=None, tags=None, blocks=None, note=None,
+         userid=None):
+    """Apply an edit to the working copy, writing a revision if it changed.
+
+    Returns (guide, revision_or_None). None means the save was a no-op: the
+    content hashed identically to the last revision, so nothing was recorded.
+    That is the normal outcome of an autosave where the author changed nothing,
+    and it is what keeps the history readable.
+    """
+    if title is not None:
+        guide.title = title.strip()
+    if summary is not None:
+        guide.summary = summary.strip() or None
+    if tags is not None:
+        # Normalised once, here: trimmed, empties dropped, order preserved.
+        # Tags are compared case-insensitively when hashing, but stored as the
+        # author typed them - "PXE" should not become "pxe" on screen.
+        guide.tags = ', '.join(t.strip() for t in tags.split(',') if t.strip()) or None
+    if blocks is not None:
+        guide.blocks = validate(blocks)
+
+    incoming = contenthash(guide.title, guide.summary, guide.tags, guide.blocks)
+
+    latest = (GuideRevision.query
+              .filter_by(guideid=guide.guideid)
+              .order_by(GuideRevision.revision.desc())
+              .first())
+    if latest is not None and latest.contenthash == incoming:
+        # Nothing a reader would notice has changed. Touch nothing else: an
+        # updatedat that moves on every autosave makes "last edited" a lie.
+        return guide, None
+
+    guide.revision = (latest.revision + 1) if latest else 1
+    guide.updatedby = userid
+    guide.updatedat = _utcnow()
+
+    revision = GuideRevision(
+        guideid=guide.guideid,
+        revision=guide.revision,
+        title=guide.title,
+        summary=guide.summary,
+        tags=guide.tags,
+        blocksjson=guide.blocksjson,
+        contenthash=incoming,
+        note=(note or '').strip() or None,
+        createdby=userid,
+    )
+    db.session.add(revision)
+    db.session.flush()
+    return guide, revision
+
+
+def publish(guide, userid=None):
+    """Point readers at the current working copy.
+
+    Publishing does NOT snapshot: it promotes the latest existing revision. If
+    it snapshotted, a publish with unsaved edits would create a version nobody
+    reviewed - which is exactly the accident this workflow exists to prevent.
+    """
+    latest = (GuideRevision.query
+              .filter_by(guideid=guide.guideid)
+              .order_by(GuideRevision.revision.desc())
+              .first())
+    if latest is None:
+        raise ValueError('nothing to publish: this guide has no saved content')
+
+    guide.publishedrevisionid = latest.guiderevisionid
+    guide.status = 'published'
+    guide.publishedat = _utcnow()
+    guide.updatedby = userid
+    db.session.flush()
+    return latest
+
+
+def unpublish(guide, userid=None):
+    """Withdraw a guide from readers without deleting anything.
+
+    For a procedure that turns out to be wrong. The revisions stay, so it can be
+    corrected and published again, and so there is still a record of what people
+    were being told in the meantime.
+    """
+    guide.publishedrevisionid = None
+    guide.status = 'draft'
+    guide.updatedby = userid
+    db.session.flush()
+    return guide
+
+
+def readable(guide):
+    """The revision a READER should be served, or None.
+
+    Never the working copy: a published guide shows what was published, even
+    while somebody is editing the next version. Editors get the working copy
+    through the editor, which is a different route on purpose.
+    """
+    if not guide.publishedrevisionid:
+        return None
+    return db.session.get(GuideRevision, guide.publishedrevisionid)
+
+
+def restore(guide, revision, userid=None):
+    """Copy an old revision back onto the working copy.
+
+    Deliberately NOT a rollback of history: restoring writes a NEW revision with
+    the old content. The record of what was published between then and now stays
+    intact, which is the difference between a history and a rewrite.
+    """
+    return save(guide,
+                title=revision.title,
+                summary=revision.summary or '',
+                tags=revision.tags or '',
+                blocks=revision.blocks,
+                note='Restored from version {}'.format(revision.revision),
+                userid=userid)

plugins/guides/services/blocks.py

@@ -0,0 +1,152 @@
+"""Block validation.
+
+The editor is the only thing that writes blocks today, so this is not defending
+against a hostile author - it is defending against a BAD SAVE. A block list that
+is wrong in a way nobody notices at save time becomes a guide that will not
+render weeks later, when the person who wrote it has moved on and the only
+evidence is a blank page.
+
+So: validate at the door, name the block that is wrong, and refuse the save.
+The alternative - storing whatever arrives and coping at render time - was tried
+in the backups plugin with a manifest and produced exactly the failure this
+avoids: a client bug turning into a broken page that outlived it.
+
+WHAT IS NOT VALIDATED HERE: the inline text. It is a restricted Markdown subset
+rendered through a sanitiser in the browser, which is where a renderer belongs.
+Storing text verbatim keeps what the author typed recoverable even if the
+rendering rules change later.
+"""
+
+import hashlib
+import json
+
+from ..models import BLOCKTYPES
+
+# A guide is a procedure, not a document repository. This cap is generous for
+# anything a person will read on a shop floor and small enough that a runaway
+# paste cannot fill a MEDIUMTEXT column.
+MAXBLOCKSBYTES = 2 * 1024 * 1024
+MAXBLOCKS = 500
+MAXTEXT = 20000
+
+
+class BlockError(ValueError):
+    """Raised with a message naming the offending block, for the API to relay."""
+
+
+def _fail(index, message):
+    raise BlockError('block {}: {}'.format(index + 1, message))
+
+
+def validate(blocks):
+    """Check a block list, returning it unchanged, or raise BlockError.
+
+    Kept pure - no database, no request - so the rules are testable directly and
+    the same function can be reused by an importer later.
+    """
+    if not isinstance(blocks, list):
+        raise BlockError('blocks must be a list')
+    if len(blocks) > MAXBLOCKS:
+        raise BlockError('a guide is limited to {} blocks'.format(MAXBLOCKS))
+
+    seen = set()
+    for index, block in enumerate(blocks):
+        if not isinstance(block, dict):
+            _fail(index, 'must be an object')
+
+        blockid = block.get('id')
+        if not isinstance(blockid, str) or not blockid.strip():
+            # Ids are what make a checklist item rememberable between visits.
+            # A block without one silently loses its tick the next time the
+            # guide is opened, which reads as "the checkboxes do not work".
+            _fail(index, 'needs a stable id')
+        if blockid in seen:
+            _fail(index, 'duplicate id {!r} - ticks would follow the wrong item'
+                  .format(blockid))
+        seen.add(blockid)
+
+        blocktype = block.get('type')
+        if blocktype not in BLOCKTYPES:
+            _fail(index, 'unknown type {!r}. Known types: {}'.format(
+                blocktype, ', '.join(BLOCKTYPES)))
+
+        text = block.get('text')
+        if text is not None:
+            if not isinstance(text, str):
+                _fail(index, 'text must be a string')
+            if len(text) > MAXTEXT:
+                _fail(index, 'text is longer than {} characters'.format(MAXTEXT))
+
+        items = block.get('items')
+        if items is not None:
+            if not isinstance(items, list):
+                _fail(index, 'items must be a list')
+            for item in items:
+                # Checklist items carry their own id for the same reason blocks
+                # do: that id is the thing a tick is stored against.
+                if isinstance(item, dict):
+                    if not isinstance(item.get('id'), str) or not item['id'].strip():
+                        _fail(index, 'every item needs an id')
+                    if not isinstance(item.get('text', ''), str):
+                        _fail(index, 'item text must be a string')
+                elif not isinstance(item, str):
+                    _fail(index, 'items must be strings or objects')
+
+    encoded = json.dumps(blocks, separators=(',', ':'))
+    if len(encoded.encode('utf-8')) > MAXBLOCKSBYTES:
+        raise BlockError('this guide is larger than {} MB'.format(
+            MAXBLOCKSBYTES // (1024 * 1024)))
+    return blocks
+
+
+def plaintext(blocks):
+    """Flatten a guide to searchable text.
+
+    Search should find a guide by something written halfway down it, not only by
+    its title - that is how people actually look for a procedure ("the one about
+    the drive letter"). Cheap because it runs on save, not on every query.
+    """
+    parts = []
+    for block in blocks or []:
+        if not isinstance(block, dict):
+            continue
+        if isinstance(block.get('text'), str):
+            parts.append(block['text'])
+        for item in block.get('items') or []:
+            if isinstance(item, str):
+                parts.append(item)
+            elif isinstance(item, dict) and isinstance(item.get('text'), str):
+                parts.append(item['text'])
+        for row in block.get('rows') or []:
+            if isinstance(row, list):
+                parts.extend(cell for cell in row if isinstance(cell, str))
+    return '\n'.join(parts)
+
+
+def checklistitemcount(blocks):
+    """How many tickable items a guide has, for the list and the progress bar."""
+    total = 0
+    for block in blocks or []:
+        if isinstance(block, dict) and block.get('type') == 'checklist':
+            total += len(block.get('items') or [])
+    return total
+
+
+def contenthash(title, summary, tags, blocks):
+    """sha256 over everything a reader would notice changing.
+
+    Title, summary and tags are included, not just the blocks: renaming a guide
+    or retagging it is a change worth a revision, and hashing only the body
+    would silently drop it from the history.
+
+    Canonical by construction - the blocks are re-serialized with sorted keys
+    and no whitespace, so the same content saved twice hashes the same however
+    the editor happened to order its JSON that day.
+    """
+    canonical = json.dumps({
+        'title': (title or '').strip(),
+        'summary': (summary or '').strip(),
+        'tags': sorted(t.strip().lower() for t in (tags or '').split(',') if t.strip()),
+        'blocks': blocks or [],
+    }, sort_keys=True, separators=(',', ':'))
+    return hashlib.sha256(canonical.encode('utf-8')).hexdigest()