e685852c2e
A fresh install could not build its schema. Core revision 7d51_endpoint_logcollections creates endpointlogcollections with a foreign key onto endpointcommands, but PLUGIN_TABLE_OWNERS lists both tables as owned by the endpoints plugin, and only the plugin's 0002 revision creates endpointcommands. Core migrations run before any plugin chain, and a plugin chain only runs once that plugin is enabled, which cannot happen before the schema exists. So there is no ordering in which the core revision can succeed on an empty database. MySQL rejects it with 1824, 'Failed to open the referenced table endpointcommands'. That is the first command the README gives an adopting site, on the branch that exists for sites to adopt. Nothing caught it because the suite runs on sqlite:///:memory: and SQLite accepts a foreign key onto a table that does not exist. The failure is MySQL-only and therefore invisible to every test. Fix: - endpoints revision 0025 creates the table in the plugin chain, after 0002 has created its foreign key target. Guarded, so it is a no-op on sites that already applied the core revision. - 7d51 skips when endpointcommands is absent, which is exactly the fresh install case 0025 then covers. Its downgrade is guarded to match. - EXPECTED_HEAD_REVISION records the new endpoints head. Verified against real MySQL 8.4, not SQLite: on an empty database the README sequence now completes, 7d51 stamps without creating anything, and enabling the endpoints plugin builds endpointlogcollections with endpointlogcollections_ibfk_2 pointing at endpointcommands.
111 lines
5.4 KiB
Python
111 lines
5.4 KiB
Python
"""Add endpointlogcollections
|
|
|
|
Backs the manual Log Collector tab: one row per plaintext log file an operator
|
|
collected from an endpoint by path.
|
|
|
|
Collected content gets its own table rather than living in
|
|
endpointcommands.resultjson for three reasons. It is bulky, and resultjson is
|
|
read by every command-history list that only wants a status. It needs its own
|
|
retention rule, which a command row cannot carry without deleting the audit
|
|
record of the command itself. And the download route has to address a single
|
|
collection by id and serve it as a file, which is awkward against a JSON column.
|
|
|
|
Retention is PER ASSET (20, enforced on insert in the result route), not global.
|
|
The history renders on a single machine's detail page, so a global cap would let
|
|
one busy endpoint quietly evict the history of another - the panel would be
|
|
unreliable exactly where it is read.
|
|
|
|
endpointcommandid is SET NULL rather than CASCADE: commands are reaped and
|
|
expired over time, and a collected log an operator is still reading must not
|
|
vanish because its originating command row was cleaned up. assetid stays CASCADE
|
|
because a log has no meaning once the machine is gone.
|
|
|
|
filesizebytes is BIGINT while contentbytes is INTEGER on purpose - the file on
|
|
disk may be gigabytes, while what we store is a bounded tail.
|
|
|
|
Revision ID: 7d51_endpoint_logcollections
|
|
Revises: 7d50_script_captureoutput
|
|
Create Date: 2026-09-09
|
|
|
|
"""
|
|
from alembic import op
|
|
import sqlalchemy as sa
|
|
|
|
|
|
revision = '7d51_endpoint_logcollections'
|
|
down_revision = '7d50_script_captureoutput'
|
|
branch_labels = None
|
|
depends_on = None
|
|
|
|
|
|
def upgrade():
|
|
# GUARD: this table belongs to the endpoints PLUGIN, not to core.
|
|
# PLUGIN_TABLE_OWNERS has always said so, and the foreign key below points
|
|
# at endpointcommands, which only the plugin's 0002 revision creates. Core
|
|
# runs first and a plugin chain only runs once the plugin is enabled, so on
|
|
# a fresh database endpointcommands does not exist yet and MySQL rejects
|
|
# the create with 1824. Skip here and let endpoints0025logcollections build
|
|
# it in the chain that owns it.
|
|
#
|
|
# Sites that already applied this revision have the table, so they are
|
|
# unaffected either way: the plugin revision is guarded and no-ops there.
|
|
if not sa.inspect(op.get_bind()).has_table('endpointcommands'):
|
|
return
|
|
|
|
op.create_table(
|
|
'endpointlogcollections',
|
|
sa.Column('endpointlogcollectionid', sa.Integer(), nullable=False),
|
|
sa.Column('assetid', sa.Integer(), nullable=False),
|
|
sa.Column('endpointcommandid', sa.Integer(), nullable=True),
|
|
sa.Column('sourcepath', sa.String(length=512), nullable=False,
|
|
comment='Path as the AGENT resolved it, which may differ '
|
|
'from what the operator typed once environment '
|
|
'variables are expanded endpoint-side.'),
|
|
sa.Column('content', sa.Text(length=16777215), nullable=True),
|
|
sa.Column('filesizebytes', sa.BigInteger(), nullable=True,
|
|
comment='Size of the file on disk. Differs from contentbytes '
|
|
'when the stored copy is a truncated tail.'),
|
|
sa.Column('contentbytes', sa.Integer(), nullable=True),
|
|
sa.Column('linecount', sa.Integer(), nullable=True),
|
|
sa.Column('istruncated', sa.Boolean(), nullable=False,
|
|
server_default=sa.false(),
|
|
comment='True when content is only the tail of a larger file.'),
|
|
sa.Column('filemodifiedat', sa.DateTime(), nullable=True),
|
|
sa.Column('collectedbyuserid', sa.Integer(), nullable=True),
|
|
sa.Column('collectedbyusername', sa.String(length=100), nullable=True,
|
|
comment='Username snapshot; the id alone becomes unreadable '
|
|
'if the user is later removed.'),
|
|
sa.Column('collectedat', sa.DateTime(), nullable=False),
|
|
sa.ForeignKeyConstraint(['assetid'], ['assets.assetid'],
|
|
ondelete='CASCADE'),
|
|
sa.ForeignKeyConstraint(['endpointcommandid'],
|
|
['endpointcommands.endpointcommandid'],
|
|
ondelete='SET NULL'),
|
|
sa.ForeignKeyConstraint(['collectedbyuserid'], ['users.userid'],
|
|
ondelete='SET NULL'),
|
|
sa.PrimaryKeyConstraint('endpointlogcollectionid'),
|
|
)
|
|
op.create_index('ix_endpointlogcollections_assetid',
|
|
'endpointlogcollections', ['assetid'])
|
|
op.create_index('ix_endpointlogcollections_collectedat',
|
|
'endpointlogcollections', ['collectedat'])
|
|
# Serves both hot paths: listing an asset's history newest-first, and
|
|
# finding that asset's oldest rows to prune on insert.
|
|
op.create_index('idx_endpointlogcollections_assetcollected',
|
|
'endpointlogcollections', ['assetid', 'collectedat'])
|
|
|
|
|
|
def downgrade():
|
|
# Mirror of the upgrade guard: on a fresh install this revision created
|
|
# nothing, so there is nothing here to drop.
|
|
if not sa.inspect(op.get_bind()).has_table('endpointlogcollections'):
|
|
return
|
|
|
|
op.drop_index('idx_endpointlogcollections_assetcollected',
|
|
table_name='endpointlogcollections')
|
|
op.drop_index('ix_endpointlogcollections_collectedat',
|
|
table_name='endpointlogcollections')
|
|
op.drop_index('ix_endpointlogcollections_assetid',
|
|
table_name='endpointlogcollections')
|
|
op.drop_table('endpointlogcollections')
|