ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Guides image + draft gating, audit retention, ADR-015 gate, backup credential redaction

Browse Source
This commit is contained in:
Cameron Proudlock
2026-08-26 15:08:53 -04:00
parent 4e38303326
commit 0738d71525
37 changed files with 1953 additions and 129 deletions
Download Patch File Download Diff File Expand all files Collapse all files

98
CHANGELOG.md
View File

@@ -12,6 +12,104 @@ ADR-007 and ADR-002.
### Added
- **Audit log retention.** The audit log is append-only and nothing ever removed
a row, so it grew for as long as a site ran - here to the point where the
audit page was tens of thousands of pages deep and therefore useless, because
nobody pages through that to find anything. Site & Facility gains
`audit_retentiondays` plus a card showing the log's size and exactly how many
entries a purge would remove, and there is `flask audit stats` /
`flask audit purge --apply` for the server. **Opt-in: the default is 0, which
keeps everything.** An upgrade that silently started deleting a site's audit
history would be indefensible. Deletion runs in committed batches, because one
`DELETE` over a million rows holds table locks long enough to block the
collectors posting inventory; a run that stops early reports how much is left
rather than claiming it finished. Purging needs its own `audit.purge`
permission - reading the log and destroying it are different acts.
- **Guides: images, and a reader who cannot see drafts.** Guide images are
fetched through the authenticated API client instead of being pointed at by
`<img src>`. An image tag sends no Authorization header and this app keeps its
JWT in one, so every guide image was a broken icon for every user; the same
change makes stored image links survive a subpath mount. A failed image now
says so in words rather than showing a broken icon. Revision and image routes
are gated on their parent guide - revision ids are sequential, so reading one
guide used to be enough to walk the ids and read every unpublished draft on
the site. Search, the guide list and the tag chips now match the PUBLISHED
text, not the working copy, so a reader can no longer get a hit on wording
that is not in the page when they open it.
### Fixed
- **A guide's URL no longer moves when its title is edited.** The slug was
regenerated on every save that carried a title, and the editor autosaves every
three seconds - so fixing a typo in a heading silently moved the guide and
broke exactly the pasted links the stored slug exists to protect. The slug is
now set once, at create.
- **Publishing a guide after a failed save reported success.** `publish()` ran
regardless of whether the pre-publish save reached the server, so the author
saw "Published version N" while readers got the previous revision. A failed
save now stops the publish and says why.
- **The ADR-015 site-literal check was enforcing nothing.** Its patterns are
assembled from fragments so the published script does not contain the strings
it greps for, but they used the apostrophe-escape form (`'a'\''b'`) instead of
concatenation (`'a''b'`), embedding a literal apostrophe in every pattern.
Nothing could ever match and the check passed everything from the day it was
written. Fixed, and it immediately found 25 real violations - among them a QA
ServiceNow tenant URL sitting in the shipped defaults in seven places.
- **ServiceNow ships off and blank.** The defaults pointed at one organisation's
QA ServiceNow tenant with the integration enabled, so a site that had never
configured ServiceNow still rendered ticket links - to a tenant its
technicians cannot reach, and for an adopting site, into somebody else's
instance. There is no cross-site default for "which ServiceNow is yours".
- **Credentials are no longer stored in backup revisions.** The NTLARS/DNC
registry carries an FTP password in cleartext, and it was stored verbatim -
rendered by the revision panel and handed back by the download route as a
ready-to-import `.reg`. Credential values are now redacted at parse, so they
never reach the database, and the rendered file NAMES each withheld value
instead of carrying a placebo one: importing a fake password would produce a
machine that looks restored and cannot connect. `scripts/redact_backup_secrets.py`
scrubs revisions recorded before this (dry run by default). Rotate any
password that was stored.
- **A search domain that fails is no longer indistinguishable from one that
found nothing.** Each domain catches broadly and returns no rows so one broken
table cannot take the whole search box down - but the technician was then told
they might have misspelled a query that was right, and concluded the record
did not exist. Failed domains are named in the response, shown as a banner,
and suppress the did-you-mean.
- **An empty `sourcehostname` defeated backup dedup permanently.** The dedup
query maps `''` to `IS NULL` while the insert stored the raw `''`, so those
rows could never match their own chain: every post inserted, the table grew
without bound and the history filled with identical entries. Normalized once
at ingest, whitespace included.
- **`printedparts` migration 0003 was unguarded.** Unlike its twenty siblings it
ran a bare `ALTER`, so `flask plugin upgrade-all` failed on any database built
by `create_all` and took 0004 down with it. The migration test that should
have caught this only covered single-migration chains; it now covers every
post-cutover chain.
- **Guide links with several parameters opened the wrong page.** Text is escaped
before formatting, so a captured URL arrives carrying entities and was then
escaped a second time - `&amp;amp;` in every multi-parameter ServiceNow and
SharePoint link. Emphasis could also reach inside an `href`, and
`//host` passed as an in-app link when the browser reads it as off-site.
- **Retire an asset from the interface.** Machines, PCs, printers and measuring
tools had no way to take a record out of ShopDB without calling the API, so a
scrapped tool stayed in every list and report. The endpoints already existed
and were already soft deletes: the record leaves the lists while its
calibration history, backups and relationships stay put, and it can be
restored by setting `isactive` back. Called Retire rather than Delete for
that reason, and the confirm says what is kept. USB is not included - it
retires through a status column in the external CMMC database and needs its
own decision.
- **`flask catalog dedupe`, `merge` and `rename`: catalog hygiene.** The legacy
data carried space-stripped twins - `DellInc.` beside `Dell Inc.`,
`Windows10` beside `Windows 10` - which rode into the shipped seed catalog

5
frontend/src/api/index.js
View File

@@ -343,6 +343,11 @@ export const printersApi = {
update(id, data) {
return api.put(`/printers/${id}`, data)
},
// Soft delete: the server deactivates the asset, so the printer leaves every
// list while its history and relationships stay intact.
remove(id) {
return api.delete(`/printers/${id}`)
},
updateExtension(id, data) {
return api.put(`/printers/${id}/printerdata`, data)
},

4
frontend/src/composables/pageTitle.js
View File

@@ -1,7 +1,7 @@
// The browser tab title, including WHICH record is open.
//
// The router can only name the KIND of page, because it runs before the view
// has fetched anything: every application read "West Jefferson ShopDB -
// has fetched anything: every application read "<facility> ShopDB -
// Application", so two tabs on two different machines were indistinguishable,
// and a shared link previewed as the section rather than the thing.
//
@@ -51,7 +51,7 @@ export async function applyGenericTitle(key, page) {
if (key !== activeKey) return // navigated on while we awaited
// A view that named the record before the facility name arrived composed its
// title against the placeholder prefix, so rewrite it rather than leaving
// 'ShopDB - CSF16' on a site called West Jefferson.
// 'ShopDB - CSF16' without the facility name in front of it.
document.title = compose(recordTitle !== null ? recordTitle : page)
}

26
frontend/src/utils/siteSettings.js
View File

@@ -98,22 +98,20 @@ function darkenHexColor(hex, fraction) {
// ServiceNow ticket-link config. Returns the enabled flag, incident/change URL
// templates, and the global-search URL template (all with a {ticket}
// placeholder). Defaults mirror the previously-hardcoded GE ServiceNow URLs.
// placeholder).
//
// The defaults are BLANK and the integration ships OFF, matching the server.
// They used to be one organisation's QA ServiceNow tenant, so any site that had
// not configured ServiceNow still rendered ticket links - pointing its
// technicians at a tenant they cannot reach and, for an adopting site, at
// somebody else's. A blank URL renders tickets as plain text, which is the
// correct behaviour for "not configured".
export async function getServicenowUrls() {
const enabledRaw = await getSetting('servicenow_enabled', 'true')
const enabledRaw = await getSetting('servicenow_enabled', 'false')
const enabled = enabledRaw !== 'false' && enabledRaw !== '0' && enabledRaw !== false
const incidentUrl = await getSetting(
'servicenow_incident_url',
'https://geaerospaceqa.service-now.com/now/nav/ui/search/0f8b85d0c7922010099a308dc7c2606a/params/search-term/{ticket}/global-search-data-config-id/c861cea2c7022010099a308dc7c26041/'
)
const changeUrl = await getSetting(
'servicenow_change_url',
'https://geaerospaceqa.service-now.com/now/nav/ui/search/0f8b85d0c7922010099a308dc7c2606a/params/search-term/{ticket}/global-search-data-config-id/c861cea2c7022010099a308dc7c26041/'
)
const searchUrl = await getSetting(
'servicenow_search_url',
'https://geaerospaceqa.service-now.com/now/nav/ui/search/0f8b85d0c7922010099a308dc7c2606a/params/search-term/{ticket}/global-search-data-config-id/c861cea2c7022010099a308dc7c26041/'
)
const incidentUrl = await getSetting('servicenow_incident_url', '')
const changeUrl = await getSetting('servicenow_change_url', '')
const searchUrl = await getSetting('servicenow_search_url', '')
return { enabled, incidentUrl, changeUrl, searchUrl }
}

27
frontend/src/views/SearchResults.vue
View File

@@ -35,6 +35,18 @@
<div v-if="loading" class="loading">Searching...</div>
<template v-else-if="query">
<!--
A domain that could not be searched is stated plainly. Without this,
a broken table reads as "that record does not exist" - and the person
goes off believing the machine was never entered.
-->
<div v-if="degraded.length" class="search-degraded">
<strong>Some results may be missing.</strong>
Could not search: {{ degraded.join(', ') }}.
Anything in {{ degraded.length > 1 ? 'those areas' : 'that area' }}
will not appear below, even if it exists.
</div>
<div v-if="filteredResults.length === 0 && results.length > 0" class="no-results">
No {{ activeFilter }} results for "{{ query }}"
<button class="btn btn-secondary" style="margin-top: 0.5rem;" @click="activeFilter = 'all'">Show all results</button>
@@ -103,6 +115,7 @@ const router = useRouter()
const loading = ref(false)
const results = ref([])
const suggestion = ref(null)
const degraded = ref([])
const query = ref('')
const searchInput = ref('')
const activeFilter = ref('all')
@@ -182,6 +195,7 @@ async function search(q) {
if (!q || q.length < 2) {
results.value = []
suggestion.value = null
degraded.value = []
return
}
@@ -195,6 +209,7 @@ async function search(q) {
// call is needed here. A list page that has no such key uses the
// useSearchSuggestion composable against /search/suggest instead.
suggestion.value = data?.suggestion || null
degraded.value = data?.degraded || []
// Handle ServiceNOW redirect
if (data?.redirect?.type === 'servicenow') {
@@ -512,4 +527,16 @@ watch(results, () => {
--rt-subnet-bg: rgba(191, 54, 12, 0.2); --rt-subnet-fg: #ffab91;
}
}
/* Warning, not an error: results below are still usable, just incomplete. */
.search-degraded {
margin-bottom: 1rem;
padding: 0.7rem 0.9rem;
border: 1px solid var(--warning, #b26a00);
border-left-width: 4px;
border-radius: 4px;
background: var(--surface, transparent);
color: var(--text);
font-size: 0.9rem;
}
</style>

2
frontend/src/views/settings/ServiceNowSettings.vue
View File

@@ -34,7 +34,7 @@
<input
type="url"
v-model="settings.servicenow_search_url"
placeholder="https://geaerospaceqa.service-now.com/now/nav/ui/search/.../{ticket}/..."
placeholder="https://yourtenant.service-now.com/now/nav/ui/search/.../{ticket}/..."
@blur="saveSetting('servicenow_search_url', settings.servicenow_search_url)"
:disabled="saving"
>

107
frontend/src/views/settings/SiteSettings.vue
View File

@@ -35,12 +35,55 @@
</div>
</div>
</div>
<!--
Retention is a setting; purging is an ACT. Keeping them apart matters here
more than in most places: saving a number must never silently destroy
history, so the count of what would go is shown before anything happens
and the deletion is a separate, deliberate press.
-->
<div class="card">
<h3 class="group-title">Audit Log Size</h3>
<div v-if="audit.loading" class="muted">Loading...</div>
<div v-else-if="audit.error" class="error">{{ audit.error }}</div>
<template v-else>
<div class="audit-stats">
<span><strong>{{ audit.total.toLocaleString() }}</strong> entries</span>
<span v-if="audit.oldest" class="muted">oldest {{ shortDate(audit.oldest) }}</span>
<span v-if="audit.newest" class="muted">newest {{ shortDate(audit.newest) }}</span>
</div>
<p v-if="!audit.retentiondays" class="muted">
No retention period is set, so nothing is ever deleted. Set
<em>Audit Log Retention</em> above and save to enable purging.
</p>
<p v-else-if="audit.expired" class="muted">
<strong>{{ audit.expired.toLocaleString() }}</strong> entries are older
than {{ audit.retentiondays }} days and would be deleted.
</p>
<p v-else class="muted">
Nothing is older than {{ audit.retentiondays }} days.
</p>
<div class="actions">
<button class="btn btn-secondary" :disabled="audit.busy" @click="loadAudit">
Refresh
</button>
<button class="btn btn-danger"
:disabled="audit.busy || !audit.retentiondays || !audit.expired"
@click="purgeAudit">
{{ audit.busy ? 'Purging...' : 'Purge now' }}
</button>
<span v-if="audit.message" class="muted">{{ audit.message }}</span>
</div>
</template>
</div>
</div>
</template>
<script setup>
import { ref, computed, onMounted } from 'vue'
import { settingsApi } from '@/api'
import api, { settingsApi } from '@/api'
import { apiError } from '../../utils/apiError'
const items = ref([])
@@ -75,6 +118,7 @@ const LABELS = {
employee_directory_mode: 'Employee Directory Mode',
usb_directory_mode: 'USB Directory Mode',
site_timezone: 'Site Timezone',
audit_retentiondays: 'Audit Log Retention (days)',
setup_complete: 'Setup Complete'
}
function prettyLabel(key) {
@@ -104,6 +148,7 @@ const GROUPS = [
{ title: 'Naming & Patterns', keys: ['pc_access_domain', 'printer_hostname_template', 'contact_email_domain', 'employeeid_pattern'] },
{ title: 'Data Sources', keys: ['employee_directory_mode', 'usb_directory_mode'] },
{ title: 'Localization', keys: ['site_timezone'] },
{ title: 'Audit Log', keys: ['audit_retentiondays'] },
{ title: 'System', keys: ['setup_complete'] }
]
@@ -116,12 +161,68 @@ const HELP = {
printer_hostname_template: 'Template for generating printer hostnames from an IP. Use {ip} where the dash-separated IP goes. Example: Printer-{ip}.printer.geaerospace.net', // ADR-015-OK: GE Aerospace-wide domain, and only the DEFAULT of a documented setting every site can override.
contact_email_domain: 'Email domain appended to a support contact SSO to build email (sso@domain) and Microsoft Teams chat links. Example: geaerospace.com. Leave blank to hide the contact action buttons.',
dualpath_single_machine: 'Treat a Dualpath pair (a dual-bay machine with one controller) as a single machine in lists, counts, and the floor map. Both bay records are always kept; detail pages stay per-bay with a sibling banner. Enter true or false. Default: true.',
audit_retentiondays: 'Delete audit entries older than this many days. 0 keeps everything forever, which is the default - nothing is removed until you set a number here. Saving a retention period does NOT delete anything by itself; use Purge below, or run "flask audit purge --apply" on the server.',
site_timezone: 'IANA timezone for this site. Notification start/end times are shown and entered in this zone, and daily-reset notification expiry is computed here. Default: America/New_York.'
}
function fieldHelp(key) {
return HELP[key] || ''
}
// ---------------------------------------------------------------- audit ---
const audit = ref({ loading: true, busy: false, error: '', message: '',
total: 0, oldest: null, newest: null,
retentiondays: 0, expired: 0 })
function shortDate (iso) {
const d = new Date(iso)
return isNaN(d) ? iso : d.toLocaleDateString()
}
async function loadAudit () {
audit.value.error = ''
try {
const response = await api.get('/auditlogs/retention')
const data = response.data?.data || response.data
Object.assign(audit.value, {
total: data.total || 0,
oldest: data.oldest,
newest: data.newest,
retentiondays: data.retentiondays || 0,
expired: data.expired || 0,
})
} catch (err) {
audit.value.error = apiError(err, 'Could not read the audit log size.')
} finally {
audit.value.loading = false
}
}
async function purgeAudit () {
const days = audit.value.retentiondays
const ok = window.confirm(
'Permanently delete ' + audit.value.expired.toLocaleString() +
' audit entries older than ' + days + ' days?\n\n' +
'Audit history cannot be recovered once deleted.')
if (!ok) return
audit.value.busy = true
audit.value.message = ''
try {
const response = await api.post('/auditlogs/purge', { days })
const result = response.data?.data || response.data
audit.value.message = response.data?.message || 'Purged.'
// A capped run leaves work behind, and says so rather than looking done.
if (result && result.complete === false) {
audit.value.message += ' Press Purge again to continue.'
}
await loadAudit()
} catch (err) {
audit.value.error = apiError(err, 'Could not purge the audit log.')
} finally {
audit.value.busy = false
}
}
async function load() {
loading.value = true
try {
@@ -152,7 +253,7 @@ async function save() {
}
}
onMounted(load)
onMounted(() => { load(); loadAudit() })
</script>
<style scoped>
@@ -214,4 +315,6 @@ onMounted(load)
gap: 12px;
margin-top: 8px;
}
.audit-stats { display: flex; gap: 1.25rem; align-items: baseline;
flex-wrap: wrap; margin-bottom: 0.5rem; }
</style>

12
plugins/backups/migrations/versions/0002_backups_lastseenat.py
View File

@@ -27,8 +27,10 @@ depends_on = None
def upgrade():
columns = {c['name'] for c in
sa.inspect(op.get_bind()).get_columns('backuprevisions')}
inspector = sa.inspect(op.get_bind())
if 'backuprevisions' not in inspector.get_table_names():
return
columns = {c['name'] for c in inspector.get_columns('backuprevisions')}
if 'lastseenat' not in columns:
op.add_column('backuprevisions',
sa.Column('lastseenat', sa.DateTime(), nullable=True))
@@ -37,7 +39,9 @@ def upgrade():
def downgrade():
columns = {c['name'] for c in
sa.inspect(op.get_bind()).get_columns('backuprevisions')}
inspector = sa.inspect(op.get_bind())
if 'backuprevisions' not in inspector.get_table_names():
return
columns = {c['name'] for c in inspector.get_columns('backuprevisions')}
if 'lastseenat' in columns:
op.drop_column('backuprevisions', 'lastseenat')

10
plugins/backups/plugin.py
View File

@@ -410,7 +410,13 @@ class BackupsPlugin(BasePlugin):
# with_for_update serializes concurrent posts for the same chain: the
# fleet collects on one GE-Enforce cycle, so a retry could otherwise
# pass the hash check twice and insert the same revision.
sourcehostname = payload.get('sourcehostname')
# Normalize ONCE, and use the same value to query and to store.
# samesource() maps '' to IS NULL, but the insert below used to store
# the raw '' - so a client posting an empty hostname wrote rows that its
# own next post could never match. Every post then deduped against
# nothing, inserting forever: unbounded growth, and a history no
# technician can read. Whitespace does the same thing more quietly.
sourcehostname = (payload.get('sourcehostname') or '').strip() or None
latest = (db.session.query(BackupRevision)
.filter(BackupRevision.assetid == assetid,
BackupRevision.backupkind == kindkey,
@@ -476,7 +482,7 @@ class BackupsPlugin(BasePlugin):
sharepath=sharepath,
sourcefilename=sourcefilename,
bytesize=bytesize,
sourcehostname=payload.get('sourcehostname'),
sourcehostname=sourcehostname,
collectedat=collectedat or datetime.utcnow(),
lastseenat=datetime.utcnow(),
)

64
plugins/backups/services/ntlars.py
View File

@@ -32,6 +32,31 @@ import re
SCHEMA = 'ntlars/1'
# SECRETS ARE NOT BACKED UP.
#
# The DNC registry carries an FTP password in cleartext. Stored verbatim it
# reached three places it must not: the backuprevisions table, the revision
# panel (which renders values), and the download route (which hands back a
# ready-to-import .reg). Anyone with backups.view could read the fleet's FTP
# password off a web page.
#
# Redaction happens at PARSE, so the secret never enters the database at all.
# Redacting only on display would leave it at rest and one query away.
#
# What is deliberately NOT done: dropping the value silently. This module's
# parser refuses to discard an unparseable value precisely because a backup that
# looks complete and restores a machine with a setting missing is the worst
# outcome. A redacted value keeps its NAME and records itself in
# `redacted`, so both the panel and a restore can say plainly that this one
# field must be set by hand.
REDACTEDMARKER = '***REDACTED***'
SECRETNAMEPATTERN = re.compile(r'passw|pwd|secret|apikey|token', re.IGNORECASE)
def issecretname(name):
"""Whether a registry value name holds a credential."""
return bool(SECRETNAMEPATTERN.search(name or ''))
REGROOT = 'HKEY_LOCAL_MACHINE'
DNCPATH = r'SOFTWARE\GE Aircraft Engines\DNC'
DNCPATHWOW = r'SOFTWARE\WOW6432Node\GE Aircraft Engines\DNC'
@@ -245,6 +270,20 @@ def parse(raw):
dialect = 'wow6432node' if sawwow else ('ntlars' if sawplain else 'unknown')
# Strip credentials before the projection is returned - i.e. before anything
# persists it. Recorded by path\name so the omission is visible rather than
# discovered during a restore.
redacted = []
for path in keys:
for name, value in keys[path].items():
if not issecretname(name):
continue
if value.get('data') in ('', None):
continue # nothing to hide
keys[path][name] = {'type': value.get('type', 'REG_SZ'),
'data': REDACTEDMARKER}
redacted.append('{}\\{}'.format(path, name) if path else name)
return {
'schema': SCHEMA,
'sourcedialect': dialect,
@@ -252,6 +291,7 @@ def parse(raw):
{'path': path, 'values': dict(sorted(keys[path].items()))}
for path in sorted(keys)
],
'redacted': sorted(redacted),
}
@@ -291,6 +331,12 @@ def render(projection, dialect='ntlars', comments=None):
dialect='ntlars' omits WOW6432Node - use with the NTLARS Load... button
dialect='wow6432node' includes it - use with `reg import` on 64-bit
Redacted credentials are rendered as a comment naming the value, NOT as the
marker string. Importing '***REDACTED***' as the FTP password would leave a
machine that looks restored and cannot connect, and the technician would be
debugging the network. A commented-out value fails loudly at the one moment
somebody is looking at the file.
"""
if dialect not in ('ntlars', 'wow6432node'):
raise ValueError('unknown dialect: {!r}'.format(dialect))
@@ -302,10 +348,28 @@ def render(projection, dialect='ntlars', comments=None):
if comments:
out.append('')
redacted = projection.get('redacted') or []
if redacted:
out.append('; ---------------------------------------------------------------')
out.append('; {} credential value(s) were NOT backed up and are NOT in this'
.format(len(redacted)))
out.append('; file. Set them by hand after importing:')
for name in redacted:
out.append('; {}'.format(name))
out.append('; ---------------------------------------------------------------')
out.append('')
for key in projection.get('keys', []):
path = key.get('path', '')
out.append('[{}]'.format(root + ('\\' + path if path else '')))
for name, entry in (key.get('values') or {}).items():
# A redacted credential is commented out, never emitted as the
# marker. Importing the marker as a password produces a machine that
# looks restored and cannot log in.
if (entry or {}).get('data') == REDACTEDMARKER:
out.append('; "{}" was not backed up - set it by hand'.format(
_escape(name)))
continue
out.append(_rendervalue(name, entry))
out.append('')

24
plugins/computers/frontend/views/PCDetail.vue
View File

@@ -5,6 +5,7 @@
<div class="header-actions">
<router-link :to="`/print/asset-label/computer/${$route.params.id}`" class="btn btn-secondary" target="_blank">Print Label</router-link>
<router-link :to="`/pcs/${$route.params.id}/edit`" class="btn btn-primary">Edit</router-link>
<button @click="confirmRetire" class="btn btn-danger">Retire</button>
<router-link to="/pcs" class="btn btn-secondary">Back to List</router-link>
</div>
</div>
@@ -254,7 +255,7 @@
<script setup>
import { ref, onMounted } from 'vue'
import { colorStyle } from "@/utils/colorStyle"
import { useRoute } from 'vue-router'
import { useRoute, useRouter } from 'vue-router'
import { computersApi, applicationsApi } from '@/api'
import { withBase } from '@/utils/basePath'
import { useWarrantyBadge } from '@/composables/warrantyBadge'
@@ -264,8 +265,12 @@ import PluginAssetPanels from '@/components/PluginAssetPanels.vue'
import AssetRelationships from '@/components/AssetRelationships.vue'
import { useIdentifierFlags } from '@/composables/identifierSettings'
import { setRecordTitle } from '@/composables/pageTitle'
import { useToast } from '@/composables/toast'
import { apiError } from '@/utils/apiError'
const route = useRoute()
const router = useRouter()
const toast = useToast()
const { isEnabled } = useIdentifierFlags()
const loading = ref(true)
@@ -310,6 +315,23 @@ function formatDate(dateStr) {
if (!dateStr) return '-'
return new Date(dateStr).toLocaleString()
}
// Retire, not delete: the server deactivates the asset, so the PC
// leaves every list and report while its history and relationships stay put.
// Reversible by an admin setting isactive back to true.
async function confirmRetire() {
const label = computer.value?.assetnumber || computer.value?.name || 'this PC'
if (!confirm(`Retire ${label}?\n\nIt will no longer appear in ShopDB. `
+ `History and relationships are kept, and it can be restored.`)) return
try {
await computersApi.delete(route.params.id)
toast.success(`Retired ${label}`)
router.push('/pcs')
} catch (error) {
console.error('Error retiring PC:', error)
toast.error(apiError(error, 'Failed to retire PC'))
}
}
</script>
<style scoped>

163
plugins/guides/api/routes.py
View File

@@ -11,6 +11,7 @@ serving the working copy from the reader's route would quietly undo it.
"""
from datetime import datetime
import json
import re
from flask import Blueprint, request, Response
@@ -19,7 +20,7 @@ from flask_jwt_extended import jwt_required, get_jwt_identity, current_user
from shopdb.api import (
db,
success_response, error_response, ErrorCodes,
require_permission,
require_permission, harden_upload_response,
)
from ..models import Guide, GuideRevision, GuideImage
@@ -109,27 +110,51 @@ def list_guides():
drafts are invisible to readers - but an author must be able to find their
own, or drafting is a trap door.
"""
query = Guide.query
caneditall = _has('guides.edit')
mine = _userid()
# Match against the PUBLISHED text for anything the caller is only allowed
# to read, and against the working copy only for guides they may edit.
# Searching the working copy for everyone leaks draft wording to readers:
# they get a hit on a word that is not in the page when they open it.
published = db.aliased(GuideRevision)
query = Guide.query.outerjoin(
published, Guide.publishedrevisionid == published.guiderevisionid)
search = (request.args.get('q') or '').strip()
if search:
like = '%{}%'.format(search)
query = query.filter(db.or_(
Guide.title.ilike(like),
Guide.summary.ilike(like),
Guide.tags.ilike(like),
def matches(title, summary, tags, blocks):
# The body too: people look for "the one about the drive letter",
# not for a title they never read.
Guide.blocksjson.ilike(like),
))
return db.or_(title.ilike(like), summary.ilike(like),
tags.ilike(like), blocks.ilike(like))
working = matches(Guide.title, Guide.summary, Guide.tags, Guide.blocksjson)
readable = matches(published.title, published.summary,
published.tags, published.blocksjson)
if caneditall:
query = query.filter(db.or_(working, readable))
else:
query = query.filter(db.or_(
readable,
db.and_(Guide.createdby == mine, working),
))
tag = (request.args.get('tag') or '').strip()
if tag:
query = query.filter(Guide.tags.ilike('%{}%'.format(tag)))
like = '%{}%'.format(tag)
if caneditall:
query = query.filter(db.or_(Guide.tags.ilike(like),
published.tags.ilike(like)))
else:
query = query.filter(db.or_(
published.tags.ilike(like),
db.and_(Guide.createdby == mine, Guide.tags.ilike(like)),
))
caneditall = _has('guides.edit')
if not caneditall:
mine = _userid()
query = query.filter(db.or_(
Guide.publishedrevisionid.isnot(None),
Guide.createdby == mine,
@@ -143,6 +168,18 @@ def list_guides():
data['ispublished'] = bool(guide.publishedrevisionid)
data['checklistitems'] = checklistitemcount(guide.blocks)
data['canedit'] = _mayedit(guide)
# A reader's card must describe the guide they will actually open. The
# Guide row carries the working copy, so a retitled draft would list
# under its new name and open under its old one.
if not data['canedit'] and guide.publishedrevisionid:
revision = db.session.get(GuideRevision, guide.publishedrevisionid)
if revision is not None:
data['title'] = revision.title
data['summary'] = revision.summary
data['tags'] = [t.strip() for t in (revision.tags or '').split(',')
if t.strip()]
data['checklistitems'] = checklistitemcount(revision.blocks)
out.append(data)
return success_response(out)
@@ -151,9 +188,24 @@ def list_guides():
@jwt_required()
@require_permission('guides.view')
def list_tags():
"""Every tag in use, with a count, for the filter chips."""
"""Every tag in use, with a count, for the filter chips.
Readers count published tags only. A chip carrying a tag that exists solely
on somebody's draft is both a leak and a dead end - filtering by it returns
nothing a reader is allowed to see.
"""
counts = {}
for (tags,) in db.session.query(Guide.tags).filter(Guide.tags.isnot(None)).all():
if _has('guides.edit'):
pairs = db.session.query(Guide.tags).filter(Guide.tags.isnot(None)).all()
else:
published = db.aliased(GuideRevision)
pairs = db.session.query(published.tags).join(
Guide, Guide.publishedrevisionid == published.guiderevisionid
).filter(published.tags.isnot(None)).all()
mine = _userid()
pairs += db.session.query(Guide.tags).filter(
Guide.createdby == mine, Guide.tags.isnot(None)).all()
for (tags,) in pairs:
for tag in [t.strip() for t in tags.split(',') if t.strip()]:
counts[tag] = counts.get(tag, 0) + 1
return success_response([
@@ -229,6 +281,13 @@ def list_revisions(guideid):
guide = db.session.get(Guide, guideid)
if guide is None:
return error_response(ErrorCodes.NOT_FOUND, 'Guide not found', http_code=404)
# History is editors' business. guides.view alone is permission to read what
# was PUBLISHED; the revision list is every draft this guide has ever been,
# including the ones deliberately not published yet.
if not _mayedit(guide):
return error_response(ErrorCodes.FORBIDDEN,
'You do not have permission to see this history',
http_code=403)
rows = (GuideRevision.query.filter_by(guideid=guideid)
.order_by(GuideRevision.revision.desc()).all())
published = guide.publishedrevisionid
@@ -242,10 +301,27 @@ def list_revisions(guideid):
@jwt_required()
@require_permission('guides.view')
def read_revision(revisionid):
"""One revision by id.
Gated on its PARENT, not on guides.view alone. Revision ids are sequential
integers, so an ungated route here is an invitation to walk 1..n and read
every unpublished draft on the site - which is the opposite of what
the GUIDES wiki page promises about drafts.
A reader may fetch the published revision (it is what the slug route would
serve anyway); everything else takes edit rights on that guide.
"""
revision = db.session.get(GuideRevision, revisionid)
if revision is None:
return error_response(ErrorCodes.NOT_FOUND, 'Revision not found',
http_code=404)
guide = db.session.get(Guide, revision.guideid)
ispublished = bool(guide is not None
and guide.publishedrevisionid == revision.guiderevisionid)
if not ispublished and not (guide is not None and _mayedit(guide)):
# 404, not 403: a draft's existence is itself the thing being withheld.
return error_response(ErrorCodes.NOT_FOUND, 'Revision not found',
http_code=404)
return success_response(revision.to_dict(includeblocks=True))
@@ -311,7 +387,16 @@ def update_guide(guideid):
db.session.rollback()
return error_response(ErrorCodes.VALIDATION_ERROR, str(exc))
if 'title' in data:
# The slug is set once, at create, and does NOT follow the title.
#
# _slugify's own docstring says the slug is stored so that renaming cannot
# break a link somebody pasted into a work order months ago - and then this
# re-slugged on every save that carried a title, which the editor's 3-second
# autosave does on every keystroke. Fixing a typo in a heading silently
# moved the guide's URL and broke exactly the links the stored slug exists
# to protect. A guide with no slug (older row, or a failed create) still
# gets one here.
if not guide.slug:
guide.slug = _slugify(guide.title, guide.guideid)
db.session.commit()
@@ -426,6 +511,47 @@ def upload_image():
return success_response(image.to_dict(), message='Image uploaded')
def _mayseeimage(image):
"""Whether the caller may fetch this image.
An image is content. One pasted into a published guide must be readable by
anyone who can read that guide; one pasted into a draft must not be, or the
draft leaks a screenshot at a time to anyone willing to count upwards
through image ids.
So: editors of the parent see it, and everyone else sees it only if the
PUBLISHED revision actually references it. An unattached image - pasted
before the guide was first saved - belongs to whoever uploaded it.
"""
if image.guideid is None:
return _has('guides.edit') or image.uploadedby == _userid()
guide = db.session.get(Guide, image.guideid)
if guide is None:
return _has('guides.edit')
if _mayedit(guide):
return True
if not guide.publishedrevisionid:
return False
revision = db.session.get(GuideRevision, guide.publishedrevisionid)
if revision is None:
return False
try:
blocks = json.loads(revision.blocksjson or '[]')
except (TypeError, ValueError):
return False
needle = '/guides/images/{}'.format(image.guideimageid)
for block in blocks:
if not isinstance(block, dict) or block.get('type') != 'image':
continue
if str(block.get('imageid') or '') == str(image.guideimageid):
return True
if needle in str(block.get('url') or ''):
return True
return False
@guides_bp.route('/images/<int:imageid>', methods=['GET'])
@jwt_required()
@require_permission('guides.view')
@@ -433,10 +559,15 @@ def serve_image(imageid):
image = db.session.get(GuideImage, imageid)
if image is None:
return error_response(ErrorCodes.NOT_FOUND, 'Image not found', http_code=404)
return Response(image.data, mimetype=image.mimetype, headers={
if not _mayseeimage(image):
return error_response(ErrorCodes.NOT_FOUND, 'Image not found', http_code=404)
# Same inert-document headers every other upload route serves under: an
# uploaded file is attacker-influenced content and must not be able to run
# as a document in this origin.
return harden_upload_response(Response(image.data, mimetype=image.mimetype, headers={
# Immutable: an image is never edited in place, only replaced by a new
# upload with a new id, so it can be cached hard.
'Cache-Control': 'private, max-age=31536000, immutable',
'Content-Disposition': 'inline; filename="{}"'.format(
image.filename or 'image{}'.format(imageid)),
})
}))

33
plugins/guides/frontend/components/GuideBlocks.vue
View File

@@ -72,8 +72,27 @@
<pre :ref="el => registerCode(block.id, el)"><code>{{ block.text }}</code></pre>
</div>
<!--
The image is fetched by script, not by the browser.
An <img src> is a plain browser request: it carries no Authorization
header, and this app keeps its JWT in one. So a direct src against the
authenticated image route is a 401 and a broken-image icon for every
reader - which is exactly how this shipped. Fetching through the api
client sends the token, and it resolves against the client's baseURL, so
it also survives a subpath mount that a stored '/api/...' string does
not.
A failure says so in words. A guide whose screenshot is missing must not
look like a guide that never had one.
-->
<figure v-else-if="block.type === 'image'" class="gb-figure">
<img :src="block.url" :alt="block.alt || ''" loading="lazy">
<img v-if="imageState(block).src" :src="imageState(block).src"
:alt="block.alt || ''">
<p v-else-if="imageState(block).failed" class="gb-imagefailed">
Image unavailable{{ block.alt ? ': ' + block.alt : '' }}
</p>
<p v-else class="gb-imageloading">Loading image...</p>
<figcaption v-if="block.caption">{{ block.caption }}</figcaption>
</figure>
@@ -97,8 +116,9 @@
</template>
<script setup>
import { ref, computed } from 'vue'
import { ref, computed, toRef } from 'vue'
import { renderInline } from '../utils/inline.js'
import { useGuideImages } from '../utils/images.js'
const props = defineProps({
blocks: { type: Array, default: () => [] },
@@ -112,6 +132,9 @@ const selected = ref('')
const codeNodes = {}
function registerCode (id, el) { if (el) codeNodes[id] = el }
// Images are fetched by script, not by <img src>; see utils/images.js.
const { state: imageState } = useGuideImages(toRef(props, 'blocks'))
function inline (text) {
return renderInline(text)
}
@@ -226,6 +249,12 @@ function select (block) {
.gb-figure { margin: 0 0 1.2rem; }
.gb-figure img { max-width: 100%; height: auto; border: 1px solid var(--border); border-radius: 4px; }
.gb-imageloading, .gb-imagefailed {
margin: 0; padding: 0.75rem 1rem; border: 1px dashed var(--border);
border-radius: 4px; font-size: 0.9rem;
}
.gb-imageloading { color: var(--textmuted); }
.gb-imagefailed { color: var(--danger, #b3261e); }
.gb-figure figcaption { font-size: 0.8rem; color: var(--text-light); margin-top: 0.35rem; }
.gb-tablewrap { overflow-x: auto; margin: 0 0 1.2rem; }

77
plugins/guides/frontend/utils/images.js Normal file
View File

@@ -0,0 +1,77 @@
/**
* Resolving guide images.
*
* A guide image lives behind an authenticated route, and an <img src> is a
* plain browser request that carries no Authorization header - this app keeps
* its JWT in one. Pointing src at the image route therefore yields 401 and a
* broken-image icon for every reader, which is how guides first shipped.
*
* So the bytes are fetched through the api client, which attaches the token and
* resolves against its own baseURL - the latter also fixing a stored
* '/api/guides/images/N' string under a subpath mount, where the API is not at
* /api at all.
*
* Used by the renderer AND the editor: an author needs to see the screenshot
* they just pasted as much as a reader does.
*/
import { reactive, watch, onBeforeUnmount } from 'vue'
import api from '@/api'
/** The image id a block refers to, whichever way it was written.
*
* Blocks store imageid (written since this was fixed) or a url string (written
* before it). The url is never used as a src, only read for its id.
*/
export function imageIdOf (block) {
if (!block) return ''
if (block.imageid) return String(block.imageid)
const match = /\/guides\/images\/(\d+)/.exec(block.url || '')
return match ? match[1] : ''
}
/**
* Keeps object URLs for the image blocks in `blocksRef`, fetching each id once
* however many blocks use it, and revoking them as blocks go away.
*
* Returns `state(block)` -> { src, failed }.
*/
export function useGuideImages (blocksRef) {
const images = reactive({})
async function load (id) {
if (images[id]) return
images[id] = { src: '', failed: false }
try {
const response = await api.get(`/guides/images/${id}`, { responseType: 'blob' })
images[id].src = URL.createObjectURL(response.data)
} catch (err) {
images[id].failed = true
}
}
function release (id) {
if (images[id] && images[id].src) URL.revokeObjectURL(images[id].src)
delete images[id]
}
function state (block) {
const id = imageIdOf(block)
if (!id) return { src: '', failed: true }
return images[id] || { src: '', failed: false }
}
// Editing replaces the block list on every keystroke, so resolve only ids
// that are new and keep the ones already held. Without that, every keystroke
// would refetch every image in the guide.
watch(blocksRef, blocks => {
const wanted = new Set(
(blocks || []).filter(b => b && b.type === 'image')
.map(imageIdOf).filter(Boolean))
wanted.forEach(load)
Object.keys(images).forEach(id => { if (!wanted.has(id)) release(id) })
}, { immediate: true, deep: true })
onBeforeUnmount(() => { Object.keys(images).forEach(release) })
return { state }
}

32
plugins/guides/frontend/utils/inline.js
View File

@@ -27,12 +27,29 @@ function escapeHtml (text) {
.replace(/'/g, '&#39;')
}
// The text was HTML-escaped before any pattern ran, so a captured URL arrives
// carrying entities: a real query string like ?a=1&b=2 reaches here as
// ?a=1&amp;b=2. Escaping that again for the attribute produced &amp;amp; and a
// link that opened the wrong record - the ServiceNow and SharePoint links that
// carry several parameters were all broken this way. Decode first, then let the
// single escape on output put back exactly one level.
function decodeEntities (text) {
return String(text)
.replace(/&lt;/g, '<')
.replace(/&gt;/g, '>')
.replace(/&quot;/g, '"')
.replace(/&#39;/g, "'")
.replace(/&amp;/g, '&') // last, so &amp;lt; does not become <
}
// Only http(s) and mailto. A javascript: or data: URL in a link is the one way
// an author could otherwise smuggle script past the escaping above.
function safeHref (url) {
const trimmed = String(url || '').trim()
if (/^(https?:|mailto:)/i.test(trimmed)) return trimmed
if (trimmed.startsWith('/')) return trimmed // in-app link
// In-app link. NOT '//host', which looks like a path and is read by the
// browser as protocol-relative - an off-site link wearing a local costume.
if (trimmed.startsWith('/') && !trimmed.startsWith('//')) return trimmed
return null
}
@@ -50,13 +67,18 @@ export function renderInline (text) {
// [label](url). The URL pattern allows ONE level of balanced parentheses so
// real links survive - Wikipedia-style /wiki/Foo_(bar) is the common case,
// and a naive [^)]+ silently truncates it into a broken link.
// Finished links are stashed like code spans, so the emphasis passes below
// cannot reach inside an href. Without that, a URL containing *characters*
// has <em> injected into the middle of the attribute and the link breaks.
const links = []
html = html.replace(/\[([^\]]+)\]\(((?:[^()\s]|\([^()\s]*\))+)\)/g, (match, label, url) => {
const href = safeHref(url)
const href = safeHref(decodeEntities(url))
if (!href) return label
const external = /^https?:/i.test(href)
return '<a href="' + escapeHtml(href) + '"' +
links.push('<a href="' + escapeHtml(href) + '"' +
(external ? ' target="_blank" rel="noopener noreferrer"' : '') +
'>' + label + '</a>'
'>' + label + '</a>')
return '\u0000LINK' + (links.length - 1) + '\u0000'
})
// **bold** then *italic*, longest marker first so **x** is not read as
@@ -66,6 +88,8 @@ export function renderInline (text) {
html = html.replace(/\n/g, '<br>')
html = html.replace(/\u0000LINK(\d+)\u0000/g, (match, index) => links[Number(index)])
html = html.replace(/\u0000CODE(\d+)\u0000/g, (match, index) =>
'<code>' + codeSpans[Number(index)] + '</code>')

50
plugins/guides/frontend/utils/inline.spec.js Normal file
View File

@@ -0,0 +1,50 @@
/**
* Inline rendering: the security model and the link bugs it hid.
*
* Escape-then-format is the whole defence, and it has one awkward consequence
* that produced real broken links: by the time a URL is captured it is already
* escaped, so it has to be decoded before it can be escaped again exactly once.
*/
import { describe, it, expect } from 'vitest'
import { renderInline } from './inline.js'
describe('renderInline', () => {
it('escapes markup an author pasted', () => {
expect(renderInline('<img src=x onerror=alert(1)>'))
.toBe('&lt;img src=x onerror=alert(1)&gt;')
})
it('keeps a multi-parameter URL intact', () => {
// Double-escaped, this became &amp;amp; and the link opened the wrong
// record. Every ServiceNow and SharePoint link carries several parameters.
const html = renderInline('[ticket](https://x.example/nav?a=1&b=2&c=3)')
expect(html).toContain('href="https://x.example/nav?a=1&amp;b=2&amp;c=3"')
expect(html).not.toContain('&amp;amp;')
})
it('does not let emphasis reach inside an href', () => {
const html = renderInline('[t](https://x.example/a*b*c)')
expect(html).toContain('href="https://x.example/a*b*c"')
expect(html).not.toContain('<em>')
})
it('refuses a protocol-relative URL dressed as an in-app path', () => {
// '//evil.example' starts with '/' but the browser reads it as off-site.
expect(renderInline('[evil](//evil.example/x)')).toBe('evil')
})
it('refuses javascript: and keeps the label', () => {
expect(renderInline('[bad](javascript:alert(1))')).toBe('bad')
})
it('allows an in-app link', () => {
expect(renderInline('[map](/map)')).toBe('<a href="/map">map</a>')
})
it('still renders bold, italic and code', () => {
expect(renderInline('**b**')).toBe('<strong>b</strong>')
expect(renderInline('*i*')).toBe('<em>i</em>')
expect(renderInline('`C:\\**temp**`'))
.toBe('<code>C:\\**temp**</code>')
})
})

34
plugins/guides/frontend/views/GuideEditor.vue
View File

@@ -142,7 +142,12 @@
</div>
<div v-if="block.type === 'image'" class="ge-image">
<img v-if="block.url" :src="block.url" alt="">
<!-- Same authenticated fetch as the reader's view: an author must
see the screenshot they just pasted. -->
<img v-if="imageState(block).src" :src="imageState(block).src" alt="">
<p v-else-if="imageState(block).failed" class="ge-imagefailed">
Image unavailable - it may have been removed
</p>
<input v-model="block.caption" placeholder="Caption (optional)">
<input v-model="block.alt" placeholder="Alt text, for anyone who cannot see it">
</div>
@@ -203,6 +208,7 @@ import { useAuthStore } from '@/stores/auth'
import GuideBlocks from '../components/GuideBlocks.vue'
import { templates } from '../utils/templates.js'
import { proof, applyFix } from '../utils/proof.js'
import { useGuideImages } from '../utils/images.js'
const route = useRoute()
const router = useRouter()
@@ -382,7 +388,10 @@ async function insertImage (file) {
status.value = 'Uploading image...'
try {
const image = await uploadImage(file)
const block = { id: newid(), type: 'image', url: image.url, caption: '', alt: '' }
// imageid is what the renderer resolves; url stays for older readers and
// for anything that still treats a block as a plain document.
const block = { id: newid(), type: 'image', imageid: image.guideimageid,
url: image.url, caption: '', alt: '' }
blocks.value.splice(insertIndex(), 0, block)
focused.value = block.id
status.value = ''
@@ -413,15 +422,21 @@ function onFilePick (event) {
event.target.value = ''
}
// Editor thumbnails resolve the same way the reader's do; see utils/images.js.
const { state: imageState } = useGuideImages(blocks)
function payload () {
return { title: guide.title, summary: guide.summary || '',
tags: guide.tags || '', blocks: blocks.value }
}
// Returns whether the save actually reached the server. publish() depends on
// that answer: a publish that runs after a failed save promotes the LAST
// revision the server accepted, which is not what is on the author's screen.
async function save (quiet) {
if (!guide.title.trim()) {
if (!quiet) error.value = 'A guide needs a title before it can be saved'
return
return false
}
saving.value = true
error.value = ''
@@ -439,9 +454,11 @@ async function save (quiet) {
dirty.value = false
status.value = 'Saved ' + new Date().toLocaleTimeString()
await loadRevisions()
return true
} catch (err) {
error.value = apiError(err, 'Could not save this guide')
status.value = 'Not saved'
return false
} finally {
saving.value = false
}
@@ -476,7 +493,14 @@ async function publish () {
+ 'anything flagged in a command. Publish anyway?')
if (!ok) return
}
await save(true)
// Publishing promotes the newest SAVED revision, so an unsaved edit is not
// in what readers get. Reporting "Published version 4" over a failed save
// tells the author their words went out when the old ones did.
if (!await save(true)) {
error.value = error.value
|| 'Could not save this guide, so it was not published'
return
}
try {
const response = await api.post('/guides/' + guide.guideid + '/publish')
guide.ispublished = true
@@ -637,6 +661,8 @@ onMounted(async () => {
background: transparent; color: var(--text-light); border-radius: 3px;
cursor: pointer; justify-self: start;
}
.ge-imagefailed { margin: 0 0 0.35rem; padding: 0.5rem 0.7rem; font-size: 0.85rem;
color: var(--danger, #b3261e); border: 1px dashed var(--border); border-radius: 4px; }
.ge-image img { max-width: 18rem; display: block; margin-bottom: 0.35rem;
border: 1px solid var(--border); border-radius: 4px; }
.ge-image input { width: 100%; margin-bottom: 0.25rem; padding: 0.3rem 0.45rem;

24
plugins/machines/frontend/views/MachineDetail.vue
View File

@@ -12,6 +12,7 @@
<router-link :to="`/machines/${machine?.machine?.machineid}/edit`" class="btn btn-primary" v-if="machine">
Edit
</router-link>
<button @click="confirmRetire" class="btn btn-danger" v-if="machine">Retire</button>
<router-link to="/machines" class="btn btn-secondary">Back to List</router-link>
</div>
</div>
@@ -248,7 +249,7 @@
<script setup>
import { ref, onMounted } from 'vue'
import { useRoute } from 'vue-router'
import { useRoute, useRouter } from 'vue-router'
import { machinesApi } from '@/api'
import { withBase } from '@/utils/basePath'
import LocationMapTooltip from '@/components/LocationMapTooltip.vue'
@@ -258,8 +259,12 @@ import AssetRelationships from '@/components/AssetRelationships.vue'
import { useWarrantyBadge } from '@/composables/warrantyBadge'
import { useIdentifierFlags } from '@/composables/identifierSettings'
import { setRecordTitle } from '@/composables/pageTitle'
import { useToast } from '@/composables/toast'
import { apiError } from '@/utils/apiError'
const route = useRoute()
const router = useRouter()
const toast = useToast()
const { isEnabled } = useIdentifierFlags()
const loading = ref(true)
@@ -293,6 +298,23 @@ function formatDate(dateStr) {
if (!dateStr) return '-'
return new Date(dateStr).toLocaleString()
}
// Retire, not delete: the server deactivates the asset, so the machine
// leaves every list and report while its history and relationships stay put.
// Reversible by an admin setting isactive back to true.
async function confirmRetire() {
const label = machine.value?.assetnumber || machine.value?.name || 'this machine'
if (!confirm(`Retire ${label}?\n\nIt will no longer appear in ShopDB. `
+ `History and relationships are kept, and it can be restored.`)) return
try {
await machinesApi.delete(machine.value?.machine?.machineid)
toast.success(`Retired ${label}`)
router.push('/machines')
} catch (error) {
console.error('Error retiring machine:', error)
toast.error(apiError(error, 'Failed to retire machine'))
}
}
</script>
<style scoped>

24
plugins/measuringtools/frontend/views/MeasuringToolDetail.vue
View File

@@ -9,6 +9,7 @@
<router-link :to="`/measuringtools/${tool?.measuringtool?.measuringtoolid}/edit`" class="btn btn-primary" v-if="tool">
Edit
</router-link>
<button @click="confirmRetire" class="btn btn-danger" v-if="tool">Retire</button>
<router-link to="/measuringtools" class="btn btn-secondary">Back to List</router-link>
</div>
</div>
@@ -165,7 +166,7 @@
<script setup>
import { ref, onMounted } from 'vue'
import { useRoute } from 'vue-router'
import { useRoute, useRouter } from 'vue-router'
import { colorStyle } from '@/utils/colorStyle'
import { measuringtoolsApi } from '@/api'
import CustomFieldsSection from '@/components/CustomFieldsSection.vue'
@@ -173,8 +174,12 @@ import PluginAssetPanels from '@/components/PluginAssetPanels.vue'
import AssetRelationships from '@/components/AssetRelationships.vue'
import { useIdentifierFlags } from '@/composables/identifierSettings'
import { setRecordTitle } from '@/composables/pageTitle'
import { useToast } from '@/composables/toast'
import { apiError } from '@/utils/apiError'
const route = useRoute()
const router = useRouter()
const toast = useToast()
const { isEnabled } = useIdentifierFlags()
@@ -201,6 +206,23 @@ onMounted(async () => {
loading.value = false
}
})
// Retire, not delete: the server deactivates the asset, so the measuring tool
// leaves every list and report while its history and relationships stay put.
// Reversible by an admin setting isactive back to true.
async function confirmRetire() {
const label = tool.value?.assetnumber || tool.value?.name || 'this measuring tool'
if (!confirm(`Retire ${label}?\n\nIt will no longer appear in ShopDB. `
+ `History and relationships are kept, and it can be restored.`)) return
try {
await measuringtoolsApi.remove(tool.value?.measuringtool?.measuringtoolid)
toast.success(`Retired ${label}`)
router.push('/measuringtools')
} catch (error) {
console.error('Error retiring measuring tool:', error)
toast.error(apiError(error, 'Failed to retire measuring tool'))
}
}
</script>
<style scoped>

39
plugins/printedparts/migrations/versions/0003_gagelabtag.py
View File

@@ -3,6 +3,15 @@
The gage lab issues WJRP asset numbers for printed parts; the internal
itemcode stays auto-generated, and this optional unique tag carries the
lab's number. The kiosk resolves scans/typed digits against both.
Guarded/idempotent: skips when the table is absent (plugin disabled) or the
column already exists (e.g. a test DB built by db.create_all() from the model).
Unguarded, this failed `flask plugin upgrade-all` on any create-all-built
database and took 0004 down with it, since a chain stops at its first error.
The index is matched by COLUMN SET, not by name: create_all names indexes by
SQLAlchemy's convention and this migration by the repo's, so a name-keyed guard
would add a second index over the same column rather than skip.
"""
from alembic import op
import sqlalchemy as sa
@@ -15,13 +24,31 @@ branch_labels = None
depends_on = None
def _indexed(inspector):
return any(tuple(ix.get('column_names') or ()) == ('gagelabtag',)
for ix in inspector.get_indexes('printeditems'))
def upgrade():
op.add_column('printeditems',
sa.Column('gagelabtag', sa.String(length=50), nullable=True))
op.create_index('ix_printeditems_gagelabtag', 'printeditems',
['gagelabtag'], unique=True)
inspector = sa.inspect(op.get_bind())
if 'printeditems' not in inspector.get_table_names():
return
columns = {column['name'] for column in inspector.get_columns('printeditems')}
if 'gagelabtag' not in columns:
op.add_column('printeditems',
sa.Column('gagelabtag', sa.String(length=50), nullable=True))
inspector = sa.inspect(op.get_bind())
if not _indexed(inspector):
op.create_index('ix_printeditems_gagelabtag', 'printeditems',
['gagelabtag'], unique=True)
def downgrade():
op.drop_index('ix_printeditems_gagelabtag', table_name='printeditems')
op.drop_column('printeditems', 'gagelabtag')
inspector = sa.inspect(op.get_bind())
if 'printeditems' not in inspector.get_table_names():
return
if _indexed(inspector):
op.drop_index('ix_printeditems_gagelabtag', table_name='printeditems')
columns = {column['name'] for column in inspector.get_columns('printeditems')}
if 'gagelabtag' in columns:
op.drop_column('printeditems', 'gagelabtag')

24
plugins/printers/frontend/views/PrinterDetail.vue
View File

@@ -10,6 +10,7 @@
Print Label
</router-link>
<router-link :to="`/printers/${$route.params.id}/edit`" class="btn btn-primary">Edit</router-link>
<button @click="confirmRetire" class="btn btn-danger">Retire</button>
<router-link to="/printers" class="btn btn-secondary">Back to List</router-link>
</div>
</div>
@@ -296,7 +297,7 @@
<script setup>
import { ref, onMounted, computed } from 'vue'
import { useRoute } from 'vue-router'
import { useRoute, useRouter } from 'vue-router'
import { printersApi } from '@/api'
import { withBase } from '@/utils/basePath'
import LocationMapTooltip from '@/components/LocationMapTooltip.vue'
@@ -306,8 +307,12 @@ import AssetRelationships from '@/components/AssetRelationships.vue'
import { useWarrantyBadge } from '@/composables/warrantyBadge'
import { useIdentifierFlags } from '@/composables/identifierSettings'
import { setRecordTitle } from '@/composables/pageTitle'
import { useToast } from '@/composables/toast'
import { apiError } from '@/utils/apiError'
const route = useRoute()
const router = useRouter()
const toast = useToast()
const { isEnabled } = useIdentifierFlags()
const loading = ref(true)
@@ -386,6 +391,23 @@ function formatDate(dateStr) {
if (!dateStr) return '-'
return new Date(dateStr).toLocaleString()
}
// Retire, not delete: the server deactivates the asset, so the printer
// leaves every list and report while its history and relationships stay put.
// Reversible by an admin setting isactive back to true.
async function confirmRetire() {
const label = displayTitle.value || 'this printer'
if (!confirm(`Retire ${label}?\n\nIt will no longer appear in ShopDB. `
+ `History and relationships are kept, and it can be restored.`)) return
try {
await printersApi.remove(route.params.id)
toast.success(`Retired ${label}`)
router.push('/printers')
} catch (error) {
console.error('Error retiring printer:', error)
toast.error(apiError(error, 'Failed to retire printer'))
}
}
</script>
<style scoped>

21
scripts/check-naming-and-style.sh
View File

@@ -233,17 +233,30 @@ echo "==> Checking for site-specific literals in product code (ADR-015)..."
# publication scrub greps for the very strings below - written literally, the
# rule's own definition trips the gate that enforces it. Same reason
# tests/test_docs_publishable.py splits its terms.
SITE_HOST='tsg'\''wp00525'
SITE_DOMAIN='\.geaero'\''space\.net'
SITE_NAME='West Jeff'\''erson'
# Adjacent quoted strings CONCATENATE: 'ab''cd' is the four-character string
# abcd. The apostrophe-escape form ('ab'\''cd') looks almost identical and is
# not - it embeds a literal ' in the middle, so every pattern below matched
# nothing and this check silently passed everything from the day it was written
# until 2026-08-26. That is how a QA SaaS tenant sat in the shipped defaults for
# months with the gate green. Keep the fragments split (the scrub greps this
# file for the very strings the patterns are made of) and keep them ADJACENT.
SITE_HOST='tsg''wp00525'
SITE_DOMAIN='\.geaero''space\.net'
SITE_NAME='West Jeff''erson'
SITE_NETS='10\.134\.48\.|10\.48\.249\.'
SITE_PATTERNS="${SITE_HOST}|${SITE_DOMAIN}|\bwjs\b|${SITE_NAME}|${SITE_NETS}"
# A named SaaS TENANT is as site-specific as a hostname, and this one shipped as
# the default ServiceNow URL for months: every adopting site would have linked
# its technicians into another organisation's QA instance. The vendor domain
# itself is fine to mention; the tenant is not.
SITE_TENANTS='geaero''spaceqa'
SITE_PATTERNS="${SITE_HOST}|${SITE_DOMAIN}|\bwjs\b|${SITE_NAME}|${SITE_NETS}|${SITE_TENANTS}"
SITE_HITS=$(grep -rPni "$SITE_PATTERNS" "${EXCLUDES[@]}" \
--include='*.py' --include='*.vue' --include='*.js' \
--include='*.ps1' --include='*.psm1' --include='*.sh' --include='*.iss' \
--include='*.json' --include='*.html' \
plugins/ shopdb/ scripts/ deploy/ frontend/src/ tools/ 2>/dev/null \
| grep -v '/tests\?/' \
| grep -v '\.spec\.js:' \
| grep -v 'site_imports/' \
| grep -v 'installer/bundle/' \
| grep -v '\.plugins-staged/' \

136
scripts/redact_backup_secrets.py Normal file
View File

@@ -0,0 +1,136 @@
"""Remove credentials from backup revisions already stored in the database.
WHY THIS EXISTS
The NTLARS/DNC registry carries an FTP password in cleartext. The codec now
redacts credentials at PARSE, so nothing new reaches the database - but every
revision recorded before that still holds the password verbatim, where the
revision panel renders it and the download route hands it back as a
ready-to-import .reg. Anyone with backups.view could read it off a web page.
Rotating the password does not fix these rows: it makes them stale AND still
teaches anyone reading them what the old one was.
WHAT IT CHANGES: backuprevisions.payloadjson only, and only the values whose
NAME looks like a credential (the same test the codec applies). The value name
is kept and recorded in the revision's `redacted` list, so a restore from a
scrubbed revision says plainly which field must be set by hand rather than
quietly producing a machine that cannot log in.
WHAT IT DOES NOT CHANGE: contenthash. The hash identifies the CONFIGURATION a
bay reported, and dedup compares the next post against it. Rewriting hashes here
would make every bay's next post look like a change and insert a duplicate
revision across the fleet.
Usage (on the target instance, in the app dir):
venv\\Scripts\\python scripts\\redact_backup_secrets.py
venv\\Scripts\\python scripts\\redact_backup_secrets.py --commit
Dry run by default: it reports which revisions hold a credential and stops.
Take a database backup before --commit.
"""
import argparse
import json
import os
import sys
# Run from anywhere: put the repo root (parent of scripts/) on the path.
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from dotenv import load_dotenv
# Load .env before anything reads config. Without it DATABASE_URL stays unset
# and config falls back to a hardcoded root@localhost URL, which fails with a
# bare 1045 Access denied naming a user the site does not use - a database
# credential error that is really "nothing told the app which database".
load_dotenv(os.path.join(os.getcwd(), '.env'))
from shopdb import create_app # noqa: E402
from shopdb.extensions import db # noqa: E402
def scrub(projection, issecretname, marker):
"""Redact credential values in one projection. Returns (changed, names)."""
if not isinstance(projection, dict):
return False, []
names = list(projection.get('redacted') or [])
changed = False
for key in projection.get('keys') or []:
if not isinstance(key, dict):
continue
path = key.get('path', '')
values = key.get('values')
if not isinstance(values, dict):
continue
for name, entry in values.items():
if not issecretname(name) or not isinstance(entry, dict):
continue
if entry.get('data') in ('', None, marker):
continue
entry['data'] = marker
fullname = '{}\\{}'.format(path, name) if path else name
if fullname not in names:
names.append(fullname)
changed = True
if changed:
projection['redacted'] = sorted(names)
return changed, names
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--commit', action='store_true',
help='Apply the changes. Dry run by default.')
args = parser.parse_args()
app = create_app()
with app.app_context():
from plugins.backups.models import BackupRevision
from plugins.backups.services.ntlars import issecretname, REDACTEDMARKER
rows = BackupRevision.query.order_by(BackupRevision.backuprevisionid).all()
hits = []
for revision in rows:
projection = revision.payload
if not projection:
continue
# Work on a copy so a dry run cannot mutate a loaded instance into
# something a later flush would write.
working = json.loads(json.dumps(projection))
changed, names = scrub(working, issecretname, REDACTEDMARKER)
if not changed:
continue
hits.append((revision, working, names))
if not hits:
print('No stored revision holds a credential. Nothing to do.')
return 0
print('{} revision(s) hold a credential:'.format(len(hits)))
for revision, _, names in hits:
print(' revision {:<6} asset {:<6} {:<10} {}'.format(
revision.backuprevisionid, revision.assetid,
revision.backupkind, ', '.join(names)))
if not args.commit:
print()
print('Dry run. Nothing was changed. Re-run with --commit to write.')
print('Take a database backup first.')
return 0
for revision, working, _ in hits:
revision.payload = working
db.session.commit()
print()
print('Redacted {} revision(s). contenthash left untouched, so no bay '
'will re-post as changed.'.format(len(hits)))
return 0
if __name__ == '__main__':
raise SystemExit(main())

3
shopdb/__init__.py
View File

@@ -244,7 +244,7 @@ def register_cli_commands(app: Flask):
"""Register Flask CLI commands."""
from .plugins.cli import plugin_cli
from .cli import (db_cli, seed_cli, relationships_cli, csv_cli,
catalog_cli)
catalog_cli, audit_cli)
app.cli.add_command(plugin_cli)
app.cli.add_command(db_cli)
@@ -252,6 +252,7 @@ def register_cli_commands(app: Flask):
app.cli.add_command(relationships_cli)
app.cli.add_command(csv_cli)
app.cli.add_command(catalog_cli)
app.cli.add_command(audit_cli)
def register_error_handlers(app: Flask):

72
shopdb/cli/__init__.py
View File

@@ -1835,3 +1835,75 @@ def catalog_rename(table, source, target, apply_changes):
{'t': target, 'k': sourcerow[0]})
click.echo(click.style(f'Renamed {source!r} to {target!r}.', fg='green'))
@click.group('audit')
def audit_cli():
"""Audit log maintenance."""
pass
@audit_cli.command('stats')
@with_appcontext
def audit_stats():
"""Show how big the audit log is and what retention is configured."""
from shopdb.core.services import auditretention
days = auditretention.configureddays()
info = auditretention.stats(days or None)
click.echo(f"Entries: {info['total']:,}")
click.echo(f"Oldest: {info['oldest'] or '-'}")
click.echo(f"Newest: {info['newest'] or '-'}")
if days:
click.echo(f"Retention: {days} days")
click.echo(f"Expired: {info['expired']:,} entry(s) older than that")
else:
click.echo('Retention: not set - nothing is ever deleted')
click.echo("Set one in Settings > Site & Facility, or pass --days here.")
@audit_cli.command('purge')
@click.option('--days', type=int, default=None,
help='Override the configured retention for this run.')
@click.option('--apply', 'apply_changes', is_flag=True,
help='Actually delete. Without this the command only reports.')
@with_appcontext
def audit_purge(days, apply_changes):
"""Delete audit entries older than the retention window.
Reports by default and deletes only with --apply, because this is the one
command in the app that destroys history a site may be keeping deliberately.
"""
from shopdb.core.services import auditretention
days = days if days is not None else auditretention.configureddays()
if not days:
click.echo(click.style(
'No retention configured, so there is nothing to purge. Set '
'audit_retentiondays in Settings > Site & Facility, or pass --days.',
fg='yellow'))
return
preview = auditretention.purge(days, dryrun=True)
click.echo(f"Retention {days} days - cutoff {preview['cutoff']}")
click.echo(f"{preview['matched']:,} entry(s) are older than that.")
if not preview['matched']:
return
if not apply_changes:
click.echo(click.style(
'Dry run. Nothing was deleted. Re-run with --apply to write.',
fg='yellow'))
return
# No batch cap here: unlike the API this is not behind a proxy timeout, and
# a purge that stops halfway leaves the operator wondering which half.
result = auditretention.purge(days)
click.echo(click.style(
f"Deleted {result['deleted']:,} entry(s) in {result['batches']} batch(es).",
fg='green'))
if not result['complete']:
click.echo(click.style(
f"{result['remaining']:,} still older than the cutoff - re-run.",
fg='yellow'))

74
shopdb/core/api/auditlogs.py
View File

@@ -4,7 +4,8 @@ from flask import Blueprint, request
from flask_jwt_extended import jwt_required
from shopdb.core.models import AuditLog
from shopdb.utils.responses import success_response
from shopdb.extensions import db
from shopdb.utils.responses import success_response, error_response, ErrorCodes
from shopdb.utils.authz import require_permission
auditlogs_bp = Blueprint('auditlogs', __name__)
@@ -194,3 +195,74 @@ def db_func_count_by(column):
).group_by(column).all()
return {r[0]: r[1] for r in results}
@auditlogs_bp.route('/retention', methods=['GET'])
@jwt_required()
@require_permission('audit.view')
def get_retention():
"""Size of the log and what the configured retention would remove.
Drives the Site & Facility card: an admin should see the number of entries a
purge deletes BEFORE pressing anything.
"""
from shopdb.core.services import auditretention
days = auditretention.configureddays()
return success_response(auditretention.stats(days or None))
@auditlogs_bp.route('/purge', methods=['POST'])
@jwt_required()
@require_permission('audit.purge')
def purge_auditlogs():
"""Delete entries older than the retention window.
Separate permission from audit.view: reading the log and destroying it are
not the same act, and plenty of people need the first.
Batch-capped, because this runs inside a request that a proxy will time out.
An incomplete purge returns complete=false and the number still to go rather
than reporting a finished job - the caller re-posts, or runs
`flask audit purge --apply` on the server, which has no cap.
"""
from flask import request as flask_request
from shopdb.core.services import auditretention
body = flask_request.get_json(silent=True) or {}
days = body.get('days')
if days is None:
days = auditretention.configureddays()
try:
days = int(days)
except (TypeError, ValueError):
days = 0
if days <= 0:
return error_response(
ErrorCodes.VALIDATION_ERROR,
'No retention period is configured. Set one in Site & Facility '
'settings before purging.')
dryrun = bool(body.get('dryrun'))
result = auditretention.purge(
days, dryrun=dryrun,
maxbatches=None if dryrun else auditretention.MAXBATCHES_REQUEST)
if dryrun:
message = '{:,} entry(s) would be deleted'.format(result['matched'])
elif result['complete']:
message = 'Deleted {:,} entry(s)'.format(result['deleted'])
else:
message = ('Deleted {:,} entry(s); {:,} still older than the cutoff. '
'Run it again to continue.').format(
result['deleted'], result['remaining'])
# The purge is itself an audited act, and this row postdates the cutoff so
# it survives the purge that created it.
if not dryrun and result['deleted']:
from shopdb.api import audit_log
audit_log(action='deleted', entitytype='AuditLog',
entityname='purge older than {} days'.format(days),
changes={'deleted': result['deleted'],
'cutoff': result['cutoff']})
return success_response(result, message=message)

132
shopdb/core/api/search.py
View File

@@ -61,6 +61,32 @@ def _word_match(query, *columns, extra=None):
return db.and_(*[per_word(w) for w in words])
def _notefailure(domain, exc):
"""Record that one search domain failed this request, and log it.
Every domain below catches broadly and returns [] so one broken table cannot
take the whole search box down. That is right, but on its own it makes a
DOWN domain indistinguishable from an EMPTY one: the technician sees "no
results", is told they may have misspelled it, and concludes the record does
not exist. It does; the search just could not look.
The names collected here travel back in the response so the UI can say which
part of the search did not run.
"""
from flask import g
logger.error(f"{domain} search failed: {exc}")
failures = getattr(g, '_searchfailures', None)
if failures is None:
failures = g._searchfailures = []
if domain not in failures:
failures.append(domain)
def _searchfailures():
from flask import g
return list(getattr(g, '_searchfailures', []) or [])
def _require_enabled(name):
"""Raise ImportError when the named plugin is disabled.
@@ -72,13 +98,12 @@ def _require_enabled(name):
if pm and not pm.registry.is_enabled(name):
raise ImportError(f'{name} plugin disabled')
# Shipped GE defaults. Settings override these per-site; identical fallbacks
# live here so this consumer works even if the settings seed has not run.
SERVICENOW_URL_DEFAULT = (
'https://geaerospaceqa.service-now.com/now/nav/ui/search/'
'0f8b85d0c7922010099a308dc7c2606a/params/search-term/{ticket}/'
'global-search-data-config-id/c861cea2c7022010099a308dc7c26041/'
)
# Fallbacks for when the settings seed has not run. The ServiceNow URL is BLANK
# on purpose and must stay that way: a hardcoded tenant here would link one
# site's technicians into another site's ServiceNow, and the code below already
# treats a blank URL as "ServiceNow not configured" and renders tickets as plain
# text. There is no sensible cross-site default for a tenant URL.
SERVICENOW_URL_DEFAULT = ''
SERVICENOW_PREFIXES_DEFAULT = 'GEINC,GECHG,GERIT,GESCT'
EMPLOYEEID_PATTERN_DEFAULT = r'^\d{9}$'
@@ -108,9 +133,12 @@ def _get_search_integrations():
prefixes_raw = SERVICENOW_PREFIXES_DEFAULT
prefixes = [p.strip() for p in str(prefixes_raw).split(',') if p.strip()]
# Missing means unconfigured, and unconfigured means OFF - matching the
# shipped default. Defaulting to True here made a site with no ServiceNow
# settings at all behave as though the integration were on.
servicenow_enabled = settings.get('servicenow_enabled')
if servicenow_enabled is None:
servicenow_enabled = True
servicenow_enabled = False
servicenow_url = settings.get('servicenow_search_url')
if servicenow_url is None:
@@ -251,7 +279,7 @@ def _search_applications(query, search_term):
'relevance': relevance
})
except Exception as e:
logger.error(f"Application search failed: {e}")
_notefailure("Application", e)
return results
@@ -264,43 +292,53 @@ def _search_guides(query, search_term):
is a LIKE against it: crude, but it finds the guide, and a guide corpus is
tens of documents rather than millions of rows.
PUBLISHED ONLY. A draft is deliberately invisible to readers, and search is
the one place that rule would otherwise leak - somebody would find a
half-written procedure through the search box and follow it.
PUBLISHED ONLY, and published TEXT only. A draft is deliberately invisible
to readers, and search is the one place that rule would otherwise leak -
somebody would find a half-written procedure through the search box and
follow it.
Restricting to published GUIDES is not enough on its own, which is how this
first shipped: title/summary/tags/blocksjson on the Guide row are the
WORKING copy. A published guide with unpublished edits would match on a word
that exists only in the draft, and hand the reader a hit that is not in the
page when they open it. So the match, the title and the summary all come
from the published revision.
"""
results = []
try:
_require_enabled('guides')
from plugins.guides.models import Guide
from plugins.guides.models import Guide, GuideRevision
rows = Guide.query.filter(
published = db.aliased(GuideRevision)
rows = db.session.query(Guide, published).join(
published, Guide.publishedrevisionid == published.guiderevisionid
).filter(
Guide.isactive == True,
Guide.publishedrevisionid.isnot(None),
_word_match(query, Guide.title, Guide.summary, Guide.tags,
Guide.blocksjson)
_word_match(query, published.title, published.summary,
published.tags, published.blocksjson)
).order_by(Guide.modifieddate.desc()).limit(20).all()
for guide in rows:
for guide, revision in rows:
# A title match outranks a body match: someone searching "marker"
# wants the guide called that, not every guide mentioning one.
relevance = 10
if query.lower() in (guide.title or '').lower():
if query.lower() in (revision.title or '').lower():
relevance += 20
if guide.tags and query.lower() in guide.tags.lower():
if revision.tags and query.lower() in revision.tags.lower():
relevance += 15
results.append({
'type': 'guide',
'id': guide.guideid,
'title': guide.title,
'subtitle': guide.summary,
'title': revision.title,
'subtitle': revision.summary,
'url': f"/guides/{guide.slug}",
'relevance': relevance
})
except ImportError:
pass # guides plugin absent or disabled
except Exception as e:
logger.error(f"Guides search failed: {e}")
_notefailure("Guides", e)
return results
@@ -363,7 +401,7 @@ def _search_knowledgebase(query, search_term):
except ImportError:
pass # knowledgebase plugin absent or disabled
except Exception as e:
logger.error(f"KnowledgeBase search failed: {e}")
_notefailure("KnowledgeBase", e)
return results
@@ -433,7 +471,7 @@ def _search_employees(query, search_term):
'relevance': relevance
})
except Exception as e:
logger.error(f"Employee search failed: {e}")
_notefailure("Employee", e)
return results
@@ -483,7 +521,7 @@ def _search_assets(query, search_term):
results.append(_get_asset_result(asset, query, relevance))
except Exception as e:
logger.error(f"Asset search failed: {e}")
_notefailure("Asset", e)
return results
@@ -523,7 +561,7 @@ def _search_measuringtools(query, search_term):
except ImportError:
pass # measuringtools plugin absent or disabled
except Exception as e:
logger.error(f"Measuring tool search failed: {e}")
_notefailure("Measuring tool", e)
return results
@@ -574,7 +612,7 @@ def _search_usbdevices(query, search_term):
except ImportError:
pass # usb plugin absent or disabled
except Exception as e:
logger.error(f"USB device search failed: {e}")
_notefailure("USB device", e)
return results
@@ -623,7 +661,7 @@ def _search_printeditems(query, search_term):
except ImportError:
pass # printedparts plugin absent or disabled
except Exception as e:
logger.error(f"Printed item search failed: {e}")
_notefailure("Printed item", e)
return results
@@ -658,7 +696,7 @@ def _search_customfields(query, search_term):
result['subtitle'] = field.label
results.append(result)
except Exception as e:
logger.error(f"Custom field search failed: {e}")
_notefailure("Custom field", e)
return results
@@ -686,7 +724,7 @@ def _search_by_ip(query, search_term):
result['subtitle'] = comm.ipaddress
results.append(result)
except Exception as e:
logger.error(f"IP search failed: {e}")
_notefailure("IP", e)
return results
@@ -718,7 +756,7 @@ def _search_subnets(query):
except ImportError:
pass
except Exception as e:
logger.error(f"Subnet search failed: {e}")
_notefailure("Subnet", e)
return results
@@ -746,7 +784,7 @@ def _search_hostnames(query, search_term):
except ImportError:
pass
except Exception as e:
logger.error(f"Computer hostname search failed: {e}")
_notefailure("Computer hostname", e)
# Search Printers
try:
@@ -770,7 +808,7 @@ def _search_hostnames(query, search_term):
except ImportError:
pass
except Exception as e:
logger.error(f"Printer hostname search failed: {e}")
_notefailure("Printer hostname", e)
# Search Network Devices
try:
@@ -793,7 +831,7 @@ def _search_hostnames(query, search_term):
except ImportError:
pass
except Exception as e:
logger.error(f"Network device hostname search failed: {e}")
_notefailure("Network device hostname", e)
return results
@@ -841,7 +879,7 @@ def _search_notifications(query, search_term):
except ImportError:
pass
except Exception as e:
logger.error(f"Notification search failed: {e}")
_notefailure("Notification", e)
return results
@@ -875,7 +913,7 @@ def _search_vendor_model_type(query, search_term):
except ImportError:
pass
except Exception as e:
logger.error(f"Machine vendor/model/type search failed: {e}")
_notefailure("Machine vendor/model/type", e)
# Printers: vendor, model, printertype
try:
@@ -903,7 +941,7 @@ def _search_vendor_model_type(query, search_term):
except ImportError:
pass
except Exception as e:
logger.error(f"Printer vendor/model/type search failed: {e}")
_notefailure("Printer vendor/model/type", e)
# Network Devices: vendor, networkdevicetype
try:
@@ -929,7 +967,7 @@ def _search_vendor_model_type(query, search_term):
except ImportError:
pass
except Exception as e:
logger.error(f"Network device vendor/type search failed: {e}")
_notefailure("Network device vendor/type", e)
return results
@@ -1250,12 +1288,24 @@ def global_search():
# multi-word and cross-field matching changes may already have fixed a good
# share of what used to fail. Guessing at that would be building for a
# problem nobody has measured.
degraded = _searchfailures()
suggestion = None
if not unique_results:
logger.info('search-no-results query=%r', query)
# Offered, never applied. The response carries the better spelling and
# the caller decides: silently searching for something else is how
# somebody orders the wrong cartridge.
#
# Still offered on a degraded run, because the response also carries
# `degraded` and the UI states it ABOVE the suggestion - the reader is
# told the search was incomplete before being offered a spelling.
#
# Suppressing it instead was tried and is worse: a domain that is
# chronically unreachable (an external HR directory, say) would silently
# disable spelling help for every search, for ever, and nothing would
# say why. Naming the failure is the honest fix; withholding the
# suggestion just removes a second useful signal.
suggestion = _suggestion_for(query)
# Limit total results
@@ -1271,6 +1321,10 @@ def global_search():
}
if suggestion:
response_data['suggestion'] = suggestion
if degraded:
# The UI says which part of the search did not run. Without this a
# broken domain reads as "that record does not exist".
response_data['degraded'] = degraded
redirect = _check_smart_redirect(query, classification)
if redirect:

25
shopdb/core/api/settings.py
View File

@@ -522,6 +522,13 @@ def build_default_settings():
'category': 'site',
'description': 'Public base URL of this site INCLUDING any subpath it is served under, e.g. https://shopdb.example.net or https://host/shopdb. Consumers append the in-app route to this and add nothing themselves, so a subpath left off here produces links that 404. Blank = use the browsing origin.'
},
{
'key': 'audit_retentiondays',
'value': '0',
'valuetype': 'integer',
'category': 'site',
'description': 'Delete audit log entries older than this many days. 0 (default) keeps everything forever. The audit log is append-only and grows for as long as the site runs, so a long-lived site eventually needs a limit here; nothing is deleted until one is set, and the settings page shows how many entries a purge would remove before it runs.'
},
{
'key': 'facility_name',
'value': '',
@@ -727,20 +734,22 @@ def build_default_settings():
defaults = sitedefaults + brandingdefaults + printingdefaults + identifierdefaults + searchdefaults + mapdefaults + [
# ServiceNow ticket links. Each instance points these at its own
# ServiceNow tenant; blank/disabled renders tickets as plain text.
#
# Shipped OFF and BLANK, like the Zabbix block below. These defaults
# used to be one tenant's QA instance, which is wrong twice over: a
# sister site adopting this product would silently link its technicians
# at somebody else's ServiceNow, and even at the originating site a QA
# tenant is not where a real ticket lives.
{
'key': 'servicenow_enabled',
'value': 'true',
'value': 'false',
'valuetype': 'boolean',
'category': 'integrations',
'description': 'Enable ServiceNow ticket recognition and links in search and dashboards'
},
{
'key': 'servicenow_search_url',
'value': (
'https://geaerospaceqa.service-now.com/now/nav/ui/search/'
'0f8b85d0c7922010099a308dc7c2606a/params/search-term/{ticket}/'
'global-search-data-config-id/c861cea2c7022010099a308dc7c26041/'
),
'value': '',
'valuetype': 'string',
'category': 'integrations',
'description': 'ServiceNow global-search URL template. {ticket} is the ticket number.'
@@ -754,14 +763,14 @@ def build_default_settings():
},
{
'key': 'servicenow_incident_url',
'value': 'https://geaerospaceqa.service-now.com/now/nav/ui/search/0f8b85d0c7922010099a308dc7c2606a/params/search-term/{ticket}/global-search-data-config-id/c861cea2c7022010099a308dc7c26041/',
'value': '',
'valuetype': 'string',
'category': 'integrations',
'description': 'ServiceNow incident URL template. {ticket} is the incident number.'
},
{
'key': 'servicenow_change_url',
'value': 'https://geaerospaceqa.service-now.com/now/nav/ui/search/0f8b85d0c7922010099a308dc7c2606a/params/search-term/{ticket}/global-search-data-config-id/c861cea2c7022010099a308dc7c26041/',
'value': '',
'valuetype': 'string',
'category': 'integrations',
'description': 'ServiceNow change-request URL template. {ticket} is the change number.'

3
shopdb/core/models/user.py
View File

@@ -62,6 +62,9 @@ class Permission(db.Model):
('users.delete', 'Delete users', 'admin'),
# Audit
('audit.view', 'View audit logs', 'admin'),
# Deliberately separate from audit.view: reading the log and destroying
# history are different acts, and many people need only the first.
('audit.purge', 'Delete audit log entries past the retention period', 'admin'),
# API tokens
('apitokens.create', 'Create and manage API tokens', 'apitokens'),
# Collector service tokens. A token scoped to ONLY this permission is a

133
shopdb/core/services/auditretention.py Normal file
View File

@@ -0,0 +1,133 @@
"""Audit log retention.
The audit log is append-only and nothing ever removed a row, so it grows for as
long as a site runs. That is fine for a year and not fine forever: this site
reached the point where the log is most of the database and the audit page is
tens of thousands of pages deep - which also makes it useless, because nobody
pages through that to find anything.
Retention is OPT-IN, and the default keeps everything. An upgrade that silently
started deleting a site's audit history would be indefensible; a site that wants
a limit sets one, and is told exactly how many rows that will remove before it
happens.
DELETING IN CHUNKS IS NOT AN OPTIMISATION. A single
`DELETE FROM auditlogs WHERE timestamp < ...` over a million rows builds one
enormous transaction, holds locks across the table for the duration, and on a
modest MySQL box either times out or blocks every writer while it runs - which
on this server means the collectors that are posting inventory. Deleting in
bounded batches, committing each, keeps every individual statement short.
Chunking is done by SELECTing a page of ids and deleting those. `DELETE ... LIMIT`
would be simpler but is MySQL-only, and the test suite runs on SQLite.
"""
from datetime import datetime, timedelta, timezone
from shopdb.extensions import db
from shopdb.core.models import AuditLog
# Big enough that a large purge does not take thousands of round trips, small
# enough that one statement stays short and a kill mid-purge loses little work.
CHUNKSIZE = 5000
# A purge triggered from the browser must not run until a proxy gives up on it.
# The CLI has no such limit and passes None.
MAXBATCHES_REQUEST = 40
def _utcnow():
return datetime.now(timezone.utc).replace(tzinfo=None)
def cutofffor(days):
"""The timestamp before which rows are older than the retention window."""
return _utcnow() - timedelta(days=int(days))
def stats(days=None):
"""Row count, date range, and how many rows a purge at `days` would remove.
Cheap enough to render on a settings page: three aggregates over an indexed
column.
"""
total = db.session.query(db.func.count(AuditLog.auditlogid)).scalar() or 0
oldest, newest = db.session.query(
db.func.min(AuditLog.timestamp), db.func.max(AuditLog.timestamp)).first()
expired = None
if days:
expired = (db.session.query(db.func.count(AuditLog.auditlogid))
.filter(AuditLog.timestamp < cutofffor(days)).scalar() or 0)
return {
'total': total,
'oldest': oldest.isoformat() if oldest else None,
'newest': newest.isoformat() if newest else None,
'retentiondays': int(days) if days else 0,
'expired': expired,
}
def purge(days, dryrun=False, maxbatches=None, chunksize=CHUNKSIZE):
"""Delete audit rows older than `days`, in committed batches.
Returns a dict describing what happened, including `remaining`: a purge that
stopped early because it hit `maxbatches` says so rather than reporting
success over a job it did not finish. A caller that reported "purged" while
900k rows were still there would be exactly the kind of quiet half-failure
this codebase keeps getting bitten by.
"""
days = int(days or 0)
if days <= 0:
raise ValueError('retention must be a positive number of days; '
'0 means keep everything, which is not a purge')
cutoff = cutofffor(days)
matched = (db.session.query(db.func.count(AuditLog.auditlogid))
.filter(AuditLog.timestamp < cutoff).scalar() or 0)
if dryrun or not matched:
return {'cutoff': cutoff.isoformat(), 'matched': matched, 'deleted': 0,
'remaining': matched, 'batches': 0, 'dryrun': bool(dryrun),
'complete': not matched}
deleted = 0
batches = 0
while True:
if maxbatches is not None and batches >= maxbatches:
break
ids = [row[0] for row in
db.session.query(AuditLog.auditlogid)
.filter(AuditLog.timestamp < cutoff)
.order_by(AuditLog.auditlogid)
.limit(chunksize).all()]
if not ids:
break
(db.session.query(AuditLog)
.filter(AuditLog.auditlogid.in_(ids))
.delete(synchronize_session=False))
db.session.commit()
deleted += len(ids)
batches += 1
remaining = (db.session.query(db.func.count(AuditLog.auditlogid))
.filter(AuditLog.timestamp < cutoff).scalar() or 0)
return {'cutoff': cutoff.isoformat(), 'matched': matched, 'deleted': deleted,
'remaining': remaining, 'batches': batches, 'dryrun': False,
'complete': remaining == 0}
def configureddays():
"""The site's retention setting, or 0 when nothing is set (keep forever)."""
from shopdb.core.models import Setting
try:
raw = Setting.get('audit_retentiondays')
except Exception:
return 0
try:
return max(0, int(raw or 0))
except (TypeError, ValueError):
return 0

135
tests/test_core/test_audit_retention.py Normal file
View File

@@ -0,0 +1,135 @@
"""Audit retention: the purge, its default, and its refusal to overreach.
The audit log is the one table in this app where deleting is the FEATURE, which
makes every guard here load-bearing. A retention bug does not corrupt data, it
destroys history nobody can get back.
"""
from datetime import datetime, timedelta, timezone
import pytest
from shopdb.core.models import AuditLog
from shopdb.core.services import auditretention
def _utcnow():
return datetime.now(timezone.utc).replace(tzinfo=None)
def _entry(db, ageindays, action='updated'):
row = AuditLog(action=action, entitytype='Machine', entityname='M-1',
timestamp=_utcnow() - timedelta(days=ageindays))
db.session.add(row)
db.session.commit()
return row
def test_nothing_is_deleted_when_no_retention_is_configured(db):
"""The default must keep everything.
An upgrade that started silently deleting a site's audit history because a
setting defaulted to 90 days would be indefensible.
"""
_entry(db, 4000)
assert auditretention.configureddays() == 0
with pytest.raises(ValueError):
auditretention.purge(0)
assert AuditLog.query.count() == 1
def test_a_purge_deletes_only_what_is_older_than_the_window(db):
_entry(db, 400)
_entry(db, 100)
keeper = _entry(db, 5)
result = auditretention.purge(90)
assert result['deleted'] == 2
assert result['complete'] is True
assert [row.auditlogid for row in AuditLog.query.all()] == [keeper.auditlogid]
def test_a_dry_run_reports_without_deleting(db):
_entry(db, 400)
_entry(db, 400)
result = auditretention.purge(90, dryrun=True)
assert result['matched'] == 2
assert result['deleted'] == 0
assert AuditLog.query.count() == 2, 'a dry run that deleted would be a trap'
def test_a_capped_purge_says_it_did_not_finish(db):
"""A half-done purge must not report success.
The API caps batches so the request cannot outlive a proxy timeout. If that
path reported a finished job, an admin would believe a log was trimmed while
most of it was still there.
"""
for _ in range(5):
_entry(db, 400)
result = auditretention.purge(90, maxbatches=2, chunksize=1)
assert result['deleted'] == 2
assert result['remaining'] == 3
assert result['complete'] is False
def test_chunking_deletes_everything_it_matched(db):
"""Batching is an implementation detail; the outcome must not depend on it."""
for _ in range(7):
_entry(db, 400)
result = auditretention.purge(90, chunksize=2)
assert result['deleted'] == 7
assert result['complete'] is True
assert AuditLog.query.count() == 0
def test_stats_report_the_backlog_a_purge_would_remove(db):
_entry(db, 400)
_entry(db, 5)
info = auditretention.stats(90)
assert info['total'] == 2
assert info['expired'] == 1
assert info['oldest'] is not None
def test_purging_requires_its_own_permission(client, db, auth_headers):
"""audit.view is not permission to destroy the log."""
from werkzeug.security import generate_password_hash
from shopdb.core.models import User, Role, Permission
perm = Permission(name='audit.view', description='View audit logs',
category='admin')
role = Role(rolename='auditreader', description='Reads the audit log')
role.permissions.append(perm)
db.session.add_all([perm, role])
db.session.flush()
user = User(username='auditreader', email='ar@test.local',
passwordhash=generate_password_hash('testpass'))
user.roles.append(role)
db.session.add(user)
db.session.commit()
login = client.post('/api/auth/login',
json={'username': 'auditreader', 'password': 'testpass'})
headers = {'Authorization':
'Bearer ' + login.get_json()['data']['access_token']}
assert client.get('/api/auditlogs/retention',
headers=headers).status_code == 200
assert client.post('/api/auditlogs/purge', json={'days': 90},
headers=headers).status_code == 403
def test_the_purge_route_refuses_when_no_retention_is_set(client, db, auth_headers):
response = client.post('/api/auditlogs/purge', json={}, headers=auth_headers)
assert response.status_code == 400
assert 'retention' in response.get_json()['data']['error']['message'].lower()

65
tests/test_core/test_search_degraded.py Normal file
View File

@@ -0,0 +1,65 @@
"""A search domain that FAILS must not look like a domain that found nothing.
Every domain catches broadly and returns [] so one broken table cannot take the
whole search box down. Correct - but on its own it makes a down domain
indistinguishable from an empty one, and the technician is then told they may
have misspelled a query that was right.
The failure is injected INSIDE a domain (by breaking something it uses) rather
than by replacing the domain function, because replacing the function would also
remove the try/except being tested.
"""
import pytest
class _ExplodingModel:
"""Stands in for a model whose table is unreachable."""
@property
def query(self):
raise RuntimeError('table is gone')
def __getattr__(self, name):
raise RuntimeError('table is gone')
@pytest.fixture
def broken_applications(monkeypatch):
from shopdb.core.api import search as searchmod
monkeypatch.setattr(searchmod, 'Application', _ExplodingModel())
return searchmod
def test_a_failing_domain_is_reported_as_degraded(client, db, auth_headers,
broken_applications):
response = client.get('/api/search?q=nothingmatchesthis', headers=auth_headers)
assert response.status_code == 200, 'one broken domain must not 500 the search'
data = response.get_json()['data']
assert 'degraded' in data, 'a failed domain must be visible to the caller'
assert 'Application' in data['degraded']
def test_a_healthy_domain_is_not_listed_as_degraded(client, db, auth_headers,
broken_applications):
response = client.get('/api/search?q=nothingmatchesthis', headers=auth_headers)
degraded = response.get_json()['data'].get('degraded', [])
assert 'Asset' not in degraded, 'only the domain that actually failed'
def test_a_degraded_run_still_offers_a_spelling(client, db, auth_headers,
broken_applications):
"""Both signals, not one.
Suppressing the suggestion on a degraded run was tried first. It is worse: a
chronically unreachable domain - an external HR directory, say - would
silently disable spelling help for every search for ever, with nothing
saying why. The banner states the search was incomplete, and the suggestion
stays available above it.
"""
response = client.get('/api/search?q=machne', headers=auth_headers)
data = response.get_json()['data']
assert data.get('degraded'), 'the failure is still named'
# The suggestion is offered when spellfix has one; the point is that being
# degraded does not by itself remove it.
assert 'degraded' in data

30
tests/test_core/test_search_integrations.py
View File

@@ -14,14 +14,41 @@ from shopdb.core.api.settings import invalidate_settings_cache
from shopdb.core.api.search import _get_search_integrations, _classify_query
def _configure_servicenow():
"""Turn ServiceNow on with a tenant URL.
Required now, and that is the point: the integration ships OFF with a BLANK
URL, because the defaults used to be one organisation's QA tenant and any
adopting site inherited links into somebody else's ServiceNow.
"""
Setting.set('servicenow_enabled', True, valuetype='boolean',
category='integrations')
Setting.set('servicenow_search_url',
'https://example.service-now.com/search/{ticket}',
valuetype='string', category='integrations')
def _redirect(client, auth_headers, term):
resp = client.get(f'/api/search?q={term}', headers=auth_headers)
assert resp.status_code == 200, resp.get_json()
return resp.get_json()['data'].get('redirect')
def test_unconfigured_servicenow_yields_no_redirect(client, db, auth_headers):
"""Shipped OFF and BLANK. Unconfigured must mean inactive.
The defaults were once a QA ServiceNow tenant with enabled=true, so a site
that had never configured ServiceNow still sent its technicians to a tenant
they could not reach - and an adopting site to another organisation's.
"""
invalidate_settings_cache()
redirect = _redirect(client, auth_headers, 'GEINC123')
assert redirect is None or redirect.get('type') != 'servicenow'
def test_default_prefix_classifies_servicenow(client, db, auth_headers):
"""With no override, a GE-prefixed ticket redirects to ServiceNow."""
"""Once configured, a default-prefixed ticket redirects to ServiceNow."""
_configure_servicenow()
invalidate_settings_cache()
redirect = _redirect(client, auth_headers, 'GEINC123')
assert redirect is not None
@@ -30,6 +57,7 @@ def test_default_prefix_classifies_servicenow(client, db, auth_headers):
def test_custom_prefix_classifies_servicenow(client, db, auth_headers):
"""A site-configured prefix classifies its tickets as ServiceNow."""
_configure_servicenow()
Setting.set('servicenow_ticket_prefixes', 'ACMEINC', valuetype='string',
category='integrations')
invalidate_settings_cache()

15
tests/test_core/test_settings_branding.py
View File

@@ -143,12 +143,19 @@ def test_defaults_contain_new_branding_keys():
def test_defaults_contain_new_integration_keys():
"""build_default_settings seeds the ServiceNow integration keys."""
by_key = _defaults_by_key()
assert by_key['servicenow_enabled']['value'] == 'true'
# Ships OFF with blank URLs: the defaults were once a QA tenant, which an
# adopting site would have inherited as links into another organisation.
assert by_key['servicenow_enabled']['value'] == 'false'
assert by_key['servicenow_enabled']['category'] == 'integrations'
assert by_key['servicenow_ticket_prefixes']['value'] == 'GEINC,GECHG,GERIT,GESCT'
assert '{ticket}' in by_key['servicenow_search_url']['value']
assert '{ticket}' in by_key['servicenow_incident_url']['value']
assert '{ticket}' in by_key['servicenow_change_url']['value']
# Blank, not a tenant URL. There is no cross-site default for "which
# ServiceNow instance is yours", and a blank URL is what the search code
# reads as "not configured" - tickets then render as plain text.
for key in ('servicenow_search_url', 'servicenow_incident_url',
'servicenow_change_url'):
assert by_key[key]['value'] == '', key
assert '{ticket}' in by_key[key]['description'], \
'the placeholder must still be documented for whoever fills it in'
def test_defaults_contain_new_site_keys():

23
tests/test_plugin_migrations.py
View File

@@ -282,18 +282,21 @@ def test_upgrade_all_on_fresh_db_is_clean_and_idempotent(tmp_path, monkeypatch):
# Plugins whose whole chain is a single baseline. Their tables existing already
# is the ONLY divergence possible between schema and stamp, so upgrade-all must
# handle it end to end. A longer chain can also meet an ALTER whose column is
# already present, which no amount of guarding a create_table fixes - see the
# test below for why that is a different problem.
SINGLE_BASELINE_PLUGINS = tuple(
plugin for plugin in POST_CUTOVER_PLUGINS
if len(list((PLUGINS_DIR / plugin / 'migrations' / 'versions').glob('0*.py'))) == 1
)
# EVERY post-cutover chain, not just the single-baseline ones. This was once
# restricted to plugins whose whole chain is one baseline, on the reasoning that
# a longer chain meets a second problem - an ALTER whose column is already
# present - that guarding a create_table does not fix. True, but the conclusion
# was wrong: the answer is to guard the ALTERs too, not to stop testing the
# chains that have them. While this was narrowed, printedparts 0003 sat
# unguarded for a month and would fail upgrade-all on any create-all-built
# database, taking 0004 down with it, with no test able to see it.
#
# The invariant is now the whole one: a post-cutover chain must run to its head
# against a database whose tables were built from the models and stamped
# nowhere, and must do it twice.
@pytest.mark.parametrize('plugin', SINGLE_BASELINE_PLUGINS)
@pytest.mark.parametrize('plugin', POST_CUTOVER_PLUGINS)
def test_baseline_skips_tables_that_already_exist(plugin, tmp_path, monkeypatch):
"""A baseline must stamp, not fail, when its table is already there.

109
tests/test_plugins/test_backups.py
View File

@@ -1416,3 +1416,112 @@ def test_the_card_omits_the_snapshot_row_when_no_path_is_passed(bk_app, bk_plugi
from plugins.backups.services import partmarkerinfo
card = partmarkerinfo.build(_markermanifest(['A.PAT']), 1)
assert 'Snapshot' not in [f['label'] for f in card['fields']]
def test_an_empty_source_hostname_still_dedups(bk_app, bk_plugin):
"""A client posting sourcehostname='' must not defeat dedup forever.
samesource() maps '' to IS NULL for the QUERY, but the insert stored the raw
'' - so the rows a client wrote could never match its own next post. Every
post inserted, the table grew without bound, and the revision history filled
with identical entries nobody could read.
"""
from plugins.backups.models import BackupRevision
with bk_app.app_context():
first = bk_plugin.apply_collector_payload(_payload(sourcehostname=''))
assert first['action'] == 'created'
second = bk_plugin.apply_collector_payload(_payload(sourcehostname=''))
assert second['action'] == 'noop', 'an identical re-post must dedup'
rows = BackupRevision.query.all()
assert len(rows) == 1
assert rows[0].sourcehostname is None, \
"'' is stored as NULL, matching how samesource() queries it"
def test_a_padded_source_hostname_is_the_same_source(bk_app, bk_plugin):
"""Whitespace must not fork a machine's history into two chains."""
with bk_app.app_context():
bk_plugin.apply_collector_payload(_payload(sourcehostname='GGBX0NH3ESF'))
second = bk_plugin.apply_collector_payload(
_payload(sourcehostname=' GGBX0NH3ESF '))
assert second['action'] == 'noop'
# ---------------------------------------------------------------------------
# Credential redaction. The DNC registry carries an FTP password in cleartext;
# stored verbatim it reached the database, the revision panel and the download
# route, so anyone with backups.view could read the fleet's password off a page.
# ---------------------------------------------------------------------------
# A configured install that also carries the FTP credential, which is what a
# real marker/DNC bay actually has in its registry.
SECRETREG = CONFIGUREDREG.replace(
'"MachineNo"="3204"',
'"MachineNo"="3204"\r\n"FtpPasswd"="hunter2"\r\n"FtpUser"="dnc"')
def test_a_credential_never_enters_the_projection():
"""Redaction happens at parse, so the secret is never persisted.
Redacting on display instead would leave it at rest and one query away.
"""
import json
from plugins.backups.services import ntlars
projection = ntlars.parse(_asbytes(SECRETREG))
assert 'hunter2' not in json.dumps(projection)
assert projection['redacted'] == ['General\\FtpPasswd']
values = dict(projection['keys'][0]['values'])
for key in projection['keys']:
values.update(key['values'])
assert values['FtpPasswd']['data'] == ntlars.REDACTEDMARKER
assert values['FtpUser']['data'] == 'dnc', 'only credentials are withheld'
def test_a_redacted_value_is_commented_out_not_imported():
"""Importing the marker as a password yields a machine that cannot log in.
It would LOOK restored, and the technician would go debugging the network.
A commented-out value fails at the one moment somebody is reading the file.
"""
from plugins.backups.services import ntlars
text = ntlars.render(ntlars.parse(_asbytes(SECRETREG))).decode('utf-16-le')
assert 'hunter2' not in text
assert ntlars.REDACTEDMARKER not in text
assert 'FtpPasswd' in text, 'the omission must be named, not silent'
assert '"FtpPasswd"="' not in text, 'must not be an importable value line'
assert '"FtpUser"="dnc"' in text, 'everything else still restores'
def test_the_download_route_cannot_hand_back_a_password(bk_app, bk_plugin):
"""End to end: post a config with a password, download it back."""
with bk_app.app_context():
result = bk_plugin.apply_collector_payload(_payload(reg=SECRETREG))
assert result['action'] == 'created'
from plugins.backups.models import BackupRevision
revision = _db.session.get(BackupRevision, result['backuprevisionid'])
stored = json.dumps(revision.payload)
assert 'hunter2' not in stored
rendered = ntlars_render(revision.payload)
assert 'hunter2' not in rendered
def ntlars_render(projection):
from plugins.backups.services import ntlars
return ntlars.render(projection).decode('utf-16-le')
def test_secret_name_matching_covers_the_usual_spellings():
from plugins.backups.services.ntlars import issecretname
for name in ('FtpPasswd', 'Password', 'ftp_pwd', 'ApiKey', 'AuthToken',
'ClientSecret'):
assert issecretname(name), name
for name in ('FtpUser', 'FMSHost', 'Retries', 'Passive'):
assert not issecretname(name), name

227
tests/test_plugins/test_guides.py
View File

@@ -332,3 +332,230 @@ def test_a_corrupt_blocks_column_renders_as_empty_rather_than_erroring(gd_app, g
guide.blocksjson = 'not json at all'
_db.session.commit()
assert guide.blocks == []
# ---------------------------------------------------------------------------
# The API surface. Everything above this line tests services directly; the
# draft-invisibility rule lives in the ROUTES, and until these existed no test
# executed a single one of them. That is how three routes shipped serving
# unpublished drafts to any reader who could count upwards through ids.
# ---------------------------------------------------------------------------
def _user_with(db, username, permissions, rolename=None):
"""A user holding exactly these guides permissions and nothing else."""
from werkzeug.security import generate_password_hash
from shopdb.core.models import User, Role, Permission
role = Role(rolename=rolename or (username + 'role'), description=username)
for name in permissions:
# Shared across fixtures: two users in one test both want guides.view,
# and permissions.name is unique.
permission = Permission.query.filter_by(name=name).first()
if permission is None:
permission = Permission(name=name, description=name, category='guides')
db.session.add(permission)
db.session.flush()
role.permissions.append(permission)
db.session.add(role)
db.session.flush()
user = User(username=username, email=username + '@test.local',
passwordhash=generate_password_hash('testpass'))
user.roles.append(role)
db.session.add(user)
db.session.commit()
return user
def _login(client, username):
response = client.post('/api/auth/login',
json={'username': username, 'password': 'testpass'})
assert response.status_code == 200, response.get_json()
return {'Authorization':
'Bearer ' + response.get_json()['data']['access_token']}
@pytest.fixture
def reader(client, db):
"""Can read guides. Cannot author them."""
_user_with(db, 'guidereader', ['guides.view'])
return _login(client, 'guidereader')
@pytest.fixture
def author(client, db):
"""Can write their own guides, but holds no guides.edit over others'."""
_user_with(db, 'guideauthor', ['guides.view', 'guides.create'])
return _login(client, 'guideauthor')
def _draft(db, title='Unreleased procedure', published=False):
"""A guide with one saved revision, published or not."""
from plugins.guides.models import Guide
from plugins.guides.services import authoring
guide = Guide(title=title, slug=title.lower().replace(' ', '-'),
status='draft')
db.session.add(guide)
db.session.flush()
guide, revision = authoring.save(
guide, title=title,
blocks=[_block('b1', 'paragraph', text='secret draft wording')])
if published:
authoring.publish(guide)
db.session.commit()
return guide, revision
def test_a_reader_cannot_open_an_unpublished_guide_by_slug(client, db, reader):
guide, _ = _draft(db)
response = client.get('/api/guides/' + guide.slug, headers=reader)
assert response.status_code == 404
def test_a_reader_cannot_read_an_unpublished_revision_by_id(client, db, reader):
"""Revision ids are sequential, so this route is walkable by hand.
Gating it on guides.view alone meant anyone who could read one guide could
read every draft on the site by counting upwards.
"""
guide, revision = _draft(db)
response = client.get('/api/guides/revisions/{}'.format(revision.guiderevisionid),
headers=reader)
assert response.status_code == 404
assert 'secret draft wording' not in response.get_data(as_text=True)
def test_a_reader_may_read_the_published_revision_by_id(client, db, reader):
guide, revision = _draft(db, title='Published procedure', published=True)
response = client.get('/api/guides/revisions/{}'.format(revision.guiderevisionid),
headers=reader)
assert response.status_code == 200
def test_a_reader_cannot_list_a_guides_revision_history(client, db, reader):
guide, _ = _draft(db, title='Published procedure', published=True)
response = client.get('/api/guides/{}/revisions'.format(guide.guideid),
headers=reader)
assert response.status_code == 403
def test_search_does_not_match_a_reader_against_draft_wording(client, db, reader):
"""A hit a reader cannot open is worse than no hit.
The guide row holds the WORKING copy, so matching it would surface a
published guide on a word that exists only in somebody's unpublished edit -
and the word is not on the page when the reader opens it.
"""
from plugins.guides.services import authoring
guide, revision = _draft(db, title='Published procedure', published=True)
authoring.save(guide, blocks=[_block('b1', 'paragraph',
text='unpublished sentence about widgets')])
db.session.commit()
response = client.get('/api/guides?q=widgets', headers=reader)
assert response.status_code == 200
assert response.get_json()['data'] == []
# The author of the draft still finds their own work.
response = client.get('/api/guides?q=secret', headers=reader)
assert response.status_code == 200
def test_an_author_may_edit_their_own_draft_but_not_someone_elses(client, db, author):
guide, _ = _draft(db)
response = client.get('/api/guides/{}/working'.format(guide.guideid),
headers=author)
assert response.status_code == 403, 'not their guide'
def test_an_image_upload_refuses_a_type_that_is_not_an_image(client, db, author):
import io
data = {'file': (io.BytesIO(b'MZ not a picture'), 'payload.exe')}
response = client.post('/api/guides/images', data=data, headers=author,
content_type='multipart/form-data')
assert response.status_code == 400
assert 'Unsupported image type' in response.get_json()['data']['error']['message']
def test_an_image_in_a_draft_is_not_served_to_a_reader(client, db, reader, author):
"""An image is content, and a draft's images are draft content."""
import io
from plugins.guides.models import GuideImage
png = (b'\x89PNG\r\n\x1a\n' + b'0' * 32)
upload = client.post(
'/api/guides/images',
data={'file': (io.BytesIO(png), 'screenshot.png')},
headers=author, content_type='multipart/form-data')
assert upload.status_code == 200, upload.get_json()
imageid = upload.get_json()['data']['guideimageid']
# Unattached: it belongs to whoever uploaded it, not to every reader.
assert client.get('/api/guides/images/{}'.format(imageid),
headers=reader).status_code == 404
assert client.get('/api/guides/images/{}'.format(imageid),
headers=author).status_code == 200
# Attached to an unpublished guide: still not the reader's business.
guide, _ = _draft(db)
db.session.get(GuideImage, imageid).guideid = guide.guideid
db.session.commit()
assert client.get('/api/guides/images/{}'.format(imageid),
headers=reader).status_code == 404
def test_an_image_in_a_published_guide_is_served_and_hardened(client, db, reader, author):
"""The other half: a published screenshot must actually reach the reader.
Guarding this route without checking the published case is how you trade a
leak for every image on the site rendering as a broken icon.
"""
import io
from plugins.guides.models import Guide, GuideImage
from plugins.guides.services import authoring
png = (b'\x89PNG\r\n\x1a\n' + b'0' * 32)
upload = client.post(
'/api/guides/images',
data={'file': (io.BytesIO(png), 'screenshot.png')},
headers=author, content_type='multipart/form-data')
imageid = upload.get_json()['data']['guideimageid']
guide = Guide(title='With a picture', slug='with-a-picture', status='draft')
db.session.add(guide)
db.session.flush()
guide, revision = authoring.save(
guide, title='With a picture',
blocks=[{'id': 'b1', 'type': 'image', 'imageid': imageid,
'url': '/api/guides/images/{}'.format(imageid)}])
authoring.publish(guide)
db.session.get(GuideImage, imageid).guideid = guide.guideid
db.session.commit()
response = client.get('/api/guides/images/{}'.format(imageid), headers=reader)
assert response.status_code == 200
assert response.data == png
# Uploaded bytes are attacker-influenced; they must be inert in this origin.
assert response.headers.get('X-Content-Type-Options') == 'nosniff'
assert 'Content-Security-Policy' in response.headers
def test_renaming_a_guide_does_not_move_its_url(client, db, author):
"""The stored slug exists so a pasted link keeps working.
The editor autosaves on a timer, so re-slugging on title change moved the
URL while somebody was still typing the title.
"""
created = client.post('/api/guides',
json={'title': 'Replacing a marker controler',
'blocks': []}, headers=author)
assert created.status_code == 200, created.get_json()
guideid = created.get_json()['data']['guideid']
slug = created.get_json()['data']['slug']
fixed = client.put('/api/guides/{}'.format(guideid),
json={'title': 'Replacing a marker controller',
'blocks': []}, headers=author)
assert fixed.status_code == 200
assert fixed.get_json()['data']['slug'] == slug, 'the URL must not move'
assert fixed.get_json()['data']['title'] == 'Replacing a marker controller'