Files
shopdb-flask/migrations/versions/7d51_endpoint_logcollections.py
cproudlock 534f179ee6 Make main the lean product a site receives
main was 16 days and 259 commits behind the line the product is actually built
on, at contract 0.30.0 against 0.35.0, with no per-PC credential code at all. It
was neither current nor curated - just stale. This brings it forward and, at the
same time, makes it mean something: main is what a receiving site installs, and
nothing else.

WHAT IT CARRIES. Twelve plugins - backups, computers, endpoints, guides,
knowledgebase, machines, network, notifications, printers, slides, tools,
warranty - plus applications, which is manifest-less core and always ships. The
only manifest dependency inside that set is endpoints on computers, and both are
in it, so the closure is self-contained.

WHAT IT DOES NOT. employees, geenforce, measuringtools, printedparts and usb
stay on the site line. So do the Windows installer tree, the VM rehearsal
helpers, the agent tooling config, one site's data loader, the internal audit
and findings records, and the planner spreadsheet. 1,135 tracked files against
1,438.

A REMOVAL THAT BREAKS THE APP IS NOT A REMOVAL, and two nearly shipped:

deploy/site-profile-universal.json still named all seventeen plugins. It is the
file docs/DEPLOY.md tells a site to provision from, so a fresh install would
have failed at `flask plugin apply-profile` with PluginError - the first thing a
receiving site does, broken by the branch meant to serve them. It now names the
twelve that exist, and its comment says the two must agree.

agent/ was never refreshed. Whoever populated this branch copied shopdb/,
plugins/, frontend/ and tests/ from the site line and left the agent tree at its
September state: 22 C# and PowerShell sources missing, including AgentCredentials,
EnrollmentService, AgentHttpFactory and the fleet-alert client, plus one file the
site line had deleted still present. The old tree is internally consistent, so it
compiles, and nothing in CI builds it - every gate was green over a server that
speaks enrollment and per-PC credentials to a bundled agent source that cannot.
Resynced whole.

DECISIONS THIS ENCODES, rather than defers:

- The plugin-authoring lab builds printedparts, whose manifest depends on
  employees, so it could not stay. The lab remains on the site line and main
  ships no build-along tutorial. That is a real loss for an adopting developer
  and is recorded here rather than discovered later.
- Recognition, Training and Recertification notifications are no longer seeded.
  Each splits one notification into a card per employee and renders a photo,
  which needs a plugin this branch does not bundle. Seeding them would offer an
  author a type that cannot resolve anybody. A site that installs employees gets
  them from that plugin's own seed.
- Windows/IIS is documented as the legacy path throughout, and the README leads
  with Docker, because that is what a receiving site gets.

The API description is regenerated for THIS branch - 347 paths and 483
operations, down from 415 and 573 - so the published contract describes what is
here rather than what the site line serves. docs/CONFIG.md lost the two
measuring-tool label keys the code no longer declares, which two lanes had
decided opposite ways.

Suite: 3124 passed, 6 skipped. Naming and style gates pass. The docs gates run
rather than skip, and every index line count is derived from the file it names.
2026-09-19 11:24:40 -04:00

93 lines
4.5 KiB
Python

"""Add endpointlogcollections
Backs the manual Log Collector tab: one row per plaintext log file an operator
collected from an endpoint by path.
Collected content gets its own table rather than living in
endpointcommands.resultjson for three reasons. It is bulky, and resultjson is
read by every command-history list that only wants a status. It needs its own
retention rule, which a command row cannot carry without deleting the audit
record of the command itself. And the download route has to address a single
collection by id and serve it as a file, which is awkward against a JSON column.
Retention is PER ASSET (20, enforced on insert in the result route), not global.
The history renders on a single machine's detail page, so a global cap would let
one busy endpoint quietly evict the history of another - the panel would be
unreliable exactly where it is read.
endpointcommandid is SET NULL rather than CASCADE: commands are reaped and
expired over time, and a collected log an operator is still reading must not
vanish because its originating command row was cleaned up. assetid stays CASCADE
because a log has no meaning once the machine is gone.
filesizebytes is BIGINT while contentbytes is INTEGER on purpose - the file on
disk may be gigabytes, while what we store is a bounded tail.
Revision ID: 7d51_endpoint_logcollections
Revises: 7d50_script_captureoutput
Create Date: 2026-09-09
"""
from alembic import op
import sqlalchemy as sa
revision = '7d51_endpoint_logcollections'
down_revision = '7d50_script_captureoutput'
branch_labels = None
depends_on = None
def upgrade():
op.create_table(
'endpointlogcollections',
sa.Column('endpointlogcollectionid', sa.Integer(), nullable=False),
sa.Column('assetid', sa.Integer(), nullable=False),
sa.Column('endpointcommandid', sa.Integer(), nullable=True),
sa.Column('sourcepath', sa.String(length=512), nullable=False,
comment='Path as the AGENT resolved it, which may differ '
'from what the operator typed once environment '
'variables are expanded endpoint-side.'),
sa.Column('content', sa.Text(length=16777215), nullable=True),
sa.Column('filesizebytes', sa.BigInteger(), nullable=True,
comment='Size of the file on disk. Differs from contentbytes '
'when the stored copy is a truncated tail.'),
sa.Column('contentbytes', sa.Integer(), nullable=True),
sa.Column('linecount', sa.Integer(), nullable=True),
sa.Column('istruncated', sa.Boolean(), nullable=False,
server_default=sa.false(),
comment='True when content is only the tail of a larger file.'),
sa.Column('filemodifiedat', sa.DateTime(), nullable=True),
sa.Column('collectedbyuserid', sa.Integer(), nullable=True),
sa.Column('collectedbyusername', sa.String(length=100), nullable=True,
comment='Username snapshot; the id alone becomes unreadable '
'if the user is later removed.'),
sa.Column('collectedat', sa.DateTime(), nullable=False),
sa.ForeignKeyConstraint(['assetid'], ['assets.assetid'],
ondelete='CASCADE'),
sa.ForeignKeyConstraint(['endpointcommandid'],
['endpointcommands.endpointcommandid'],
ondelete='SET NULL'),
sa.ForeignKeyConstraint(['collectedbyuserid'], ['users.userid'],
ondelete='SET NULL'),
sa.PrimaryKeyConstraint('endpointlogcollectionid'),
)
op.create_index('ix_endpointlogcollections_assetid',
'endpointlogcollections', ['assetid'])
op.create_index('ix_endpointlogcollections_collectedat',
'endpointlogcollections', ['collectedat'])
# Serves both hot paths: listing an asset's history newest-first, and
# finding that asset's oldest rows to prune on insert.
op.create_index('idx_endpointlogcollections_assetcollected',
'endpointlogcollections', ['assetid', 'collectedat'])
def downgrade():
op.drop_index('idx_endpointlogcollections_assetcollected',
table_name='endpointlogcollections')
op.drop_index('ix_endpointlogcollections_collectedat',
table_name='endpointlogcollections')
op.drop_index('ix_endpointlogcollections_assetid',
table_name='endpointlogcollections')
op.drop_table('endpointlogcollections')