ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Collector: ingest GE-Enforce/enrollment data with configurable pc-type mapping

Browse Source 
Extends the computers collector so it can replace the classic api.asp
updateCompleteAsset path that the shopfloor PC fleet uses to auto-update data.

Collector schema (project naming) now accepts the GE-Enforce/enrollment shape:
machinenumber, pctype, pcsubtype, serialnumber, loggedinuser, lastboottime,
lastcheckin, ipaddress, vendorname, modelnumber, osname, installedsoftware.
- machinenumber -> Asset.assetnumber (skips the 9999 imaging placeholder, falls
  back to hostname), on create and update.
- pctype -> ComputerType via a configurable mapping (see below).
- vendor/model created if missing (free vocab); OS looked up (controlled, warns
  if unknown); pcsubtype accepted but not yet stored (warning).
- Dropped per scope: VNC/WinRM flags, warranty, DNC config, multi-NIC.

Configurable pc-type mapping (the gea-shopfloor-* imaging taxonomy ->
ComputerType): defaults + resolution live in plugins/computers/pctypemap.py
(plugin domain, contract-pure - reads Setting via shopdb.api); overrides stored
as pctypemap_<pxetype> settings, seeded on plugin install, edited in Settings >
System > "Collector PC Type Mapping" (new UI section).

Migration doc: docs/COLLECTOR-INTEGRATION.md maps classic api.asp fields +
GE-Enforce status fields to the collector schema, documents machine-number
sourcing (registry MachineNo first, then C:\Enrollment\machine-number.txt) and
that the transport is interim.

Tests: complete-asset payload maps machinenumber/pctype/vendor/model/os; 9999
placeholder falls back to hostname. 186 tests pass, naming green, app boots,
mapping UI verified.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
cproudlock
2026-06-26 21:35:23 -04:00
parent d20682fd06
commit 10ed83e14c
6 changed files with 332 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

74
docs/COLLECTOR-INTEGRATION.md Normal file
View File

@@ -0,0 +1,74 @@
# Collector integration (PC auto-update)
How the shopfloor PC fleet pushes inventory into shopdb-flask, replacing the
classic ASP `api.asp?action=updateCompleteAsset` path.
## Endpoint
`POST /api/collector/computers`
Auth: API key header `X-API-Key: <key>`, resolved as `COLLECTOR_API_KEY_COMPUTERS`
then the shared `COLLECTOR_API_KEY` (ADR-006). Idempotent upsert keyed on
`hostname`.
## Payload (project naming convention: lowercase concatenated)
| Field | Meaning | Flask target |
|-------|---------|--------------|
| `hostname` (required) | identity | `Computer.hostname` |
| `machinenumber` | machine number | `Asset.assetnumber` (skips `9999` placeholder, falls back to hostname) |
| `pctype` | imaging pc-type | `Computer.computertypeid` via the configurable mapping |
| `pcsubtype` | finer class | accepted, not yet stored (warning) |
| `serialnumber` | BIOS serial | `Asset.serialnumber` |
| `loggedinuser` | current user | `Computer.loggedinuser` |
| `lastboottime` | ISO datetime | `Computer.lastboottime` |
| `lastcheckin` | ISO datetime | accepted (heartbeat) |
| `ipaddress` | primary IP | primary `Communication` |
| `vendorname` | manufacturer | `Computer.vendorid` (created if missing) |
| `modelnumber` | model | `Computer.modelnumberid` (created if missing) |
| `osname` | OS caption | `Computer.osid` (looked up; warned if unknown) |
| `installedsoftware` | `[{name, version}]` | `ComputerInstalledApp` (known apps only) |
Response: `{status, action: created|updated, assetid, identityvalue, warnings[]}`.
## Source of truth on the PC (current method, may change)
The data already exists at image time and at runtime:
- **machine number**: registry `HKLM\SOFTWARE\[WOW6432Node\]GE Aircraft Engines\Dnc\General\MachineNo`
FIRST (authoritative post Update-MachineNumber; ignore the `9999` placeholder),
then `C:\Enrollment\machine-number.txt` as fallback. This is exactly what
GE-Enforce.ps1 already does.
- **pc-type / pc-subtype**: `C:\Enrollment\pc-type.txt` / `pc-subtype.txt`
(the `gea-shopfloor-*` taxonomy).
- **serial / vendor / model / os / user / boot**: live WMI on the PC.
GE-Enforce currently writes a status JSON to the SFLD share rather than POSTing.
Whatever transport is used (a relay reading those status files, or a direct POST
later), map its field names to the table above.
## pc-type mapping (configurable)
`pctype` (e.g. `gea-shopfloor-cmm`) is mapped to a flask Computer Type through
`pctypemap_<pxetype>` settings (Settings > System > "Collector PC Type Mapping").
Defaults live in `plugins/computers/pctypemap.py` and are seeded on plugin
install; edit per site in the UI. Unmapped pc-types are recorded as a warning,
not an error.
## Classic api.asp field mapping (for migrating the PowerShell scripts)
| Classic `updateCompleteAsset` form field | Collector field |
|---|---|
| `hostname` | `hostname` |
| `machineNo` | `machinenumber` |
| `pcType` | `pctype` |
| `serialNumber` | `serialnumber` |
| `loggedInUser` | `loggedinuser` |
| `lastBootUpTime` / `lastBootTime` | `lastboottime` |
| `manufacturer` | `vendorname` |
| `model` | `modelnumber` |
| `osVersion` | `osname` |
| `installedApps` | `installedsoftware` |
Not carried over (no current home): warranty fields, DNC config, multi-NIC
detail beyond the primary IP, VNC/WinRM flags.

75
frontend/src/views/settings/SystemSettings.vue
View File

@@ -440,6 +440,41 @@
</div>
</div>
</div>
<!-- Collector PC Type Mapping Section -->
<div class="section-card" v-if="pcTypeMappings.length">
<h2 class="section-title">Collector PC Type Mapping</h2>
<div class="setting-group">
<p class="setting-description">
When the collector ingests a PC, its imaging pc-type (from
C:\Enrollment\pc-type.txt) is mapped to one of your Computer Types.
Adjust the mapping per site.
</p>
<div class="table-container">
<table class="identifier-matrix">
<thead>
<tr><th>Imaging pc-type</th><th>Computer Type</th></tr>
</thead>
<tbody>
<tr v-for="row in pcTypeMappings" :key="row.pxetype">
<td class="identifier-name">{{ row.pxetype }}</td>
<td>
<select
:value="row.computertype"
@change="changePcTypeMapping(row.pxetype, $event.target.value)"
:disabled="saving"
>
<option v-for="ct in computerTypes" :key="ct" :value="ct">{{ ct }}</option>
</select>
</td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
</div>
<div v-if="error" class="error-message">{{ error }}</div>
@@ -448,7 +483,7 @@
<script setup>
import { ref, reactive, onMounted, computed } from 'vue'
import { settingsApi } from '../../api'
import { settingsApi, computersApi } from '../../api'
import { setIdentifierFlag } from '../../composables/identifierSettings'
const settings = reactive({
@@ -522,6 +557,10 @@ function searchValue(domainKey) {
return key in searchMatrix ? searchMatrix[key] : true
}
// Collector pc-type -> ComputerType mapping (pctypemap_<pxetype> settings).
const pcTypeMappings = ref([]) // [{ pxetype, computertype }]
const computerTypes = ref([]) // ComputerType names for the dropdown
const loading = ref(true)
const saving = ref(false)
const testingEmail = ref(false)
@@ -579,6 +618,7 @@ async function loadSettings() {
loading.value = true
const { data } = await settingsApi.list()
const pctypeRows = []
for (const setting of data.data) {
if (setting.key in settings) {
settings[setting.key] = setting.value
@@ -586,8 +626,22 @@ async function loadSettings() {
identifierMatrix[setting.key] = setting.value !== false
} else if (/^search_.+_enabled$/.test(setting.key)) {
searchMatrix[setting.key] = setting.value !== false
} else if (setting.key.startsWith('pctypemap_')) {
pctypeRows.push({
pxetype: setting.key.slice('pctypemap_'.length),
computertype: setting.value
})
}
}
pcTypeMappings.value = pctypeRows.sort((a, b) => a.pxetype.localeCompare(b.pxetype))
// Computer type options for the mapping dropdown.
try {
const typesResponse = await computersApi.types.list({ perpage: 100 })
computerTypes.value = (typesResponse.data.data || []).map(t => t.computertype)
} catch (typesError) {
console.error('Failed to load computer types', typesError)
}
} catch (e) {
error.value = 'Failed to load settings'
console.error(e)
@@ -596,6 +650,25 @@ async function loadSettings() {
}
}
async function changePcTypeMapping(pxetype, computertype) {
const key = `pctypemap_${pxetype}`
try {
saving.value = true
error.value = ''
success.value = ''
await settingsApi.update(key, computertype)
const row = pcTypeMappings.value.find(r => r.pxetype === pxetype)
if (row) row.computertype = computertype
success.value = 'Setting saved'
setTimeout(() => { success.value = '' }, 2000)
} catch (e) {
error.value = e.response?.data?.message || 'Failed to save setting'
console.error(e)
} finally {
saving.value = false
}
}
async function toggleSetting(key) {
const newValue = !settings[key]
await saveSetting(key, newValue)

46
plugins/computers/pctypemap.py Normal file
View File

@@ -0,0 +1,46 @@
"""PXE/GE-Enforce pc-type -> ComputerType mapping (computers-plugin domain).
The imaging pipeline writes C:\\Enrollment\\pc-type.txt (the gea-shopfloor-*
taxonomy). The collector maps that to a flask ComputerType. Defaults live here;
per-site overrides live in core Settings (pctypemap_<pxetype>, category
'pctypemapping') and are editable in the Settings UI. This indirection means
the imaging taxonomy can change without code edits.
"""
from shopdb.api import Setting
# pxe pc-type -> flask ComputerType name. Defaults; overridable via settings.
DEFAULT_PCTYPE_MAP = {
'gea-shopfloor-collections': 'Shopfloor PC',
'gea-shopfloor-nocollections': 'Shopfloor PC',
'gea-shopfloor-common': 'Shopfloor PC',
'gea-shopfloor-keyence': 'Shopfloor PC',
'gea-shopfloor-cmm': 'CMM PC',
'gea-shopfloor-genspect': 'Shopfloor PC',
'gea-shopfloor-heattreat': 'Shopfloor PC',
'gea-shopfloor-waxtrace': 'Shopfloor PC',
'gea-shopfloor-display': 'Kiosk',
'gea-shopfloor-partmarker': 'Shopfloor PC',
}
_SETTING_PREFIX = 'pctypemap_'
_SETTING_CATEGORY = 'pctypemapping'
def pctype_mapping():
"""Return the live pctype -> ComputerType-name map (settings over defaults)."""
mapping = dict(DEFAULT_PCTYPE_MAP)
for setting in Setting.query.filter_by(category=_SETTING_CATEGORY).all():
if setting.key.startswith(_SETTING_PREFIX):
mapping[setting.key[len(_SETTING_PREFIX):]] = setting.get_typed_value()
return mapping
def seed_pctype_settings():
"""Seed the pctypemap_<pxetype> settings from defaults (idempotent)."""
for pxetype, computertype in DEFAULT_PCTYPE_MAP.items():
key = f'{_SETTING_PREFIX}{pxetype}'
if not Setting.query.filter_by(key=key).first():
Setting.set(key, computertype, valuetype='string',
category=_SETTING_CATEGORY,
description=f'Collector maps pc-type {pxetype} to this ComputerType')

93
plugins/computers/plugin.py
View File

@@ -67,15 +67,27 @@ class ComputersPlugin(BasePlugin):
# -- ADR-006 collector contract -----------------------------------------
def get_collector_schema(self) -> Optional[Dict]:
"""Schema for the PC collector payload (matched by hostname)."""
"""Schema for the PC collector payload (matched by hostname).
Aligns with the GE-Enforce status shape (transport may change), using
the project naming convention (lowercase concatenated). The caller maps
its own field names to these.
"""
return {
'identityfield': 'hostname',
'fields': {
'hostname': {'type': 'string', 'required': True},
'machinenumber': {'type': 'string'},
'pctype': {'type': 'string'},
'pcsubtype': {'type': 'string'},
'serialnumber': {'type': 'string'},
'currentuser': {'type': 'string'},
'loggedinuser': {'type': 'string'},
'lastboottime': {'type': 'string', 'format': 'date-time'},
'lastcheckin': {'type': 'string', 'format': 'date-time'},
'ipaddress': {'type': 'string'},
'vendorname': {'type': 'string'},
'modelnumber': {'type': 'string'},
'osname': {'type': 'string'},
'installedsoftware': {
'type': 'array',
'items': {'name': 'string', 'version': 'string'},
@@ -86,13 +98,23 @@ class ComputersPlugin(BasePlugin):
def apply_collector_payload(self, payload: Dict) -> Dict:
"""Idempotent upsert of a PC from a collector payload (by hostname)."""
from datetime import datetime
from shopdb.api import Asset, Application, Communication, CommunicationType
from shopdb.api import (
Asset, Application, Communication, CommunicationType,
Vendor, Model, OperatingSystem,
)
from .pctypemap import pctype_mapping
warnings = []
hostname = (payload.get('hostname') or '').strip()
if not hostname:
raise ValueError('hostname is required')
# Machine number is the business identifier (Asset.assetnumber). Skip
# the imaging-time placeholder '9999' and fall back to hostname.
machinenumber = (payload.get('machinenumber') or '').strip()
if machinenumber in ('', '9999'):
machinenumber = None
comp = Computer.query.filter(Computer.hostname.ilike(hostname)).first()
if not comp:
comp = (Computer.query.join(Asset, Asset.assetid == Computer.assetid)
@@ -103,14 +125,16 @@ class ComputersPlugin(BasePlugin):
atype = AssetType.query.filter_by(assettype='computer').first()
# statusid=1 is the first seeded asset status ("In Use"); a
# collector-discovered PC is by definition in use.
asset = Asset(assetnumber=hostname, assettypeid=atype.assettypeid,
statusid=1)
asset = Asset(assetnumber=machinenumber or hostname,
assettypeid=atype.assettypeid, statusid=1)
db.session.add(asset)
db.session.flush()
comp = Computer(assetid=asset.assetid, hostname=hostname)
db.session.add(comp)
db.session.flush()
action = 'created'
elif machinenumber and comp.asset:
comp.asset.assetnumber = machinenumber
comp.lastreporteddate = datetime.utcnow()
if payload.get('lastboottime'):
@@ -119,11 +143,63 @@ class ComputersPlugin(BasePlugin):
payload['lastboottime'].replace('Z', '+00:00'))
except (ValueError, AttributeError):
warnings.append('lastboottime not parseable')
if payload.get('currentuser'):
comp.loggedinuser = payload['currentuser']
loggedinuser = payload.get('loggedinuser') or payload.get('currentuser')
if loggedinuser:
comp.loggedinuser = loggedinuser
if payload.get('serialnumber') and comp.asset:
comp.asset.serialnumber = payload['serialnumber']
# pc-type -> ComputerType via the configurable settings mapping.
pctype = (payload.get('pctype') or '').strip()
if pctype:
from .models import ComputerType
mapped = pctype_mapping().get(pctype)
if not mapped:
warnings.append(f'no ComputerType mapping for pctype: {pctype}')
else:
ctype = ComputerType.query.filter_by(computertype=mapped).first()
if ctype:
comp.computertypeid = ctype.computertypeid
else:
warnings.append(f'mapped ComputerType not found: {mapped}')
# Vendor / model are free vocab - create if missing.
vendorname = (payload.get('vendorname') or '').strip()
vendor = None
if vendorname:
vendor = Vendor.query.filter(Vendor.vendor.ilike(vendorname)).first()
if not vendor:
vendor = Vendor(vendor=vendorname)
db.session.add(vendor)
db.session.flush()
comp.vendorid = vendor.vendorid
modelnumber = (payload.get('modelnumber') or '').strip()
if modelnumber:
model_query = Model.query.filter(Model.modelnumber.ilike(modelnumber))
if vendor:
model_query = model_query.filter(Model.vendorid == vendor.vendorid)
model = model_query.first()
if not model:
model = Model(modelnumber=modelnumber,
vendorid=vendor.vendorid if vendor else None)
db.session.add(model)
db.session.flush()
comp.modelnumberid = model.modelnumberid
# OS is a controlled vocab - look up only, warn if unknown.
osname = (payload.get('osname') or '').strip()
if osname:
os_row = OperatingSystem.query.filter(
OperatingSystem.osname.ilike(osname)).first()
if os_row:
comp.osid = os_row.osid
else:
warnings.append(f'unknown operating system: {osname}')
if payload.get('pcsubtype'):
warnings.append('pcsubtype received but not stored (no model field)')
if payload.get('ipaddress'):
ip_comtype = CommunicationType.query.filter_by(comtype='IP').first()
primary = Communication.query.filter_by(
@@ -167,6 +243,9 @@ class ComputersPlugin(BasePlugin):
with app.app_context():
self._ensure_asset_type()
self._ensure_computer_types()
from .pctypemap import seed_pctype_settings
seed_pctype_settings()
db.session.commit()
logger.info("Computers plugin installed")
def _ensure_asset_type(self) -> None:

3
shopdb/core/api/settings.py
View File

@@ -43,7 +43,6 @@ SEARCH_DOMAINS = {
'subnet': 'Subnets',
}
def _is_secret(key: str) -> bool:
return 'password' in key or 'token' in key or 'secret' in key
@@ -210,6 +209,8 @@ def build_default_settings():
for key, label in SEARCH_DOMAINS.items()
]
# Collector pc-type -> ComputerType mapping is computers-plugin domain;
# the plugin seeds pctypemap_<pxetype> settings on install.
defaults = identifierdefaults + searchdefaults + [
# Zabbix integration
{

50
tests/test_core/test_collector_contract.py
View File

@@ -115,3 +115,53 @@ def test_per_plugin_key_overrides_shared(client, db, app, computer_assettype):
finally:
app.config.pop('COLLECTOR_API_KEY_COMPUTERS', None)
app.config['COLLECTOR_API_KEY'] = None
def test_complete_asset_payload_maps_enrollment_data(client, db, collector_key,
computer_assettype):
"""A GE-Enforce-shaped payload maps machinenumber/pctype/vendor/model/os."""
from shopdb.core.models import OperatingSystem
from plugins.computers.models import Computer, ComputerType
# Seed the controlled-vocab rows the collector looks up.
db.session.add(ComputerType(computertype='Shopfloor PC'))
db.session.add(OperatingSystem(osname='Windows 11'))
db.session.commit()
payload = {
'hostname': 'WJSF1234',
'machinenumber': '0615',
'pctype': 'gea-shopfloor-collections',
'serialnumber': 'SN-ENROLL',
'vendorname': 'Dell',
'modelnumber': 'OptiPlex 7090',
'osname': 'Windows 11',
}
resp = client.post('/api/collector/computers', json=payload,
headers={'X-API-Key': KEY})
assert resp.status_code == 200, resp.get_json()
assert resp.get_json()['data']['action'] == 'created'
with client.application.app_context():
comp = Computer.query.filter(Computer.hostname.ilike('WJSF1234')).first()
assert comp is not None
assert comp.asset.assetnumber == '0615' # machinenumber -> assetnumber
assert comp.asset.serialnumber == 'SN-ENROLL'
assert comp.computertype.computertype == 'Shopfloor PC' # pctype mapped
assert comp.vendor.vendor == 'Dell' # created
assert comp.model.modelnumber == 'OptiPlex 7090' # created
assert comp.operatingsystem.osname == 'Windows 11'
def test_placeholder_machinenumber_9999_falls_back_to_hostname(client, db,
collector_key,
computer_assettype):
"""The 9999 imaging placeholder is ignored; assetnumber falls back to host."""
resp = client.post('/api/collector/computers',
json={'hostname': 'WJSF9999', 'machinenumber': '9999'},
headers={'X-API-Key': KEY})
assert resp.status_code == 200
from plugins.computers.models import Computer
with client.application.app_context():
comp = Computer.query.filter(Computer.hostname.ilike('WJSF9999')).first()
assert comp.asset.assetnumber == 'WJSF9999'