Files
shopdb-flask/.github/GE-ENFORCE-COLLECTOR-FIX.html
cproudlock 1164cfb287 Add the enrollment page, move scripts to per-PC credentials, and document the rollout
- Settings > Endpoints > Enrollment. Migration tab: readiness counts and
  progress, each PC's state with the reasons it is blocked, "Upgrade
  selected" and "Upgrade next 25", and live wave progress. Credentials tab:
  enrolled PCs with revoke, pending requests with approve and reject, and the
  shared key and auto-enroll switches. Turning the shared key off asks for
  confirmation and shows how many PCs are not enrolled.
- The server certificate page shows the SPKI pin to put in the GE-Enforce
  pin entry.
- The agent upgrade script and the asset and printer reporters send the PC's
  credential when it has one (over https, only to the issuing server), fall
  back to the shared key, check server pins, and log refusals clearly.
- New Set-ShopDBServerPins.ps1 (GE-Enforce entry) and Get-ShopDBServerPin.ps1.
- AGENT-CONTRACT.md documents per-PC credentials; the rollout runbook covers
  phases, gates, rollback, pin and wave entries, and every place the shared
  key lives and how to clear it at cutover.
- Removed the 09-14 certificate design notes and dated session reports, and
  two VM notes that carried a live collector key and advised plain http.
- Corrected the GE-Enforce collector note, which claimed a plain registry
  token is protected by ACL.
2026-09-15 16:07:06 -04:00

752 lines
25 KiB
HTML

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>GE-Enforce Collector Fix - API Key Deployment via Intune</title>
<style>
body {
font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
line-height: 1.6;
max-width: 1200px;
margin: 0 auto;
padding: 20px;
background: #f5f5f5;
}
.container {
background: white;
padding: 30px;
border-radius: 8px;
box-shadow: 0 2px 4px rgba(0,0,0,0.1);
}
h1 {
color: #003087;
border-bottom: 3px solid #003087;
padding-bottom: 10px;
}
h2 {
color: #005eb8;
margin-top: 30px;
border-left: 4px solid #005eb8;
padding-left: 15px;
}
h3 {
color: #0077c8;
}
.alert {
padding: 15px;
margin: 20px 0;
border-radius: 4px;
border-left: 4px solid;
}
.alert-danger {
background: #f8d7da;
border-color: #dc3545;
color: #721c24;
}
.alert-warning {
background: #fff3cd;
border-color: #ffc107;
color: #856404;
}
.alert-success {
background: #d4edda;
border-color: #28a745;
color: #155724;
}
.alert-info {
background: #d1ecf1;
border-color: #17a2b8;
color: #0c5460;
}
pre {
background: #2b2b2b;
color: #f8f8f2;
padding: 15px;
border-radius: 4px;
overflow-x: auto;
font-family: 'Consolas', 'Monaco', monospace;
font-size: 14px;
}
code {
background: #f4f4f4;
padding: 2px 6px;
border-radius: 3px;
font-family: 'Consolas', 'Monaco', monospace;
color: #c7254e;
}
pre code {
background: transparent;
color: #f8f8f2;
padding: 0;
}
.step {
background: #f8f9fa;
border-left: 4px solid #007bff;
padding: 15px;
margin: 15px 0;
border-radius: 4px;
}
.step-number {
display: inline-block;
background: #007bff;
color: white;
width: 30px;
height: 30px;
line-height: 30px;
text-align: center;
border-radius: 50%;
margin-right: 10px;
font-weight: bold;
}
table {
width: 100%;
border-collapse: collapse;
margin: 20px 0;
}
th, td {
border: 1px solid #ddd;
padding: 12px;
text-align: left;
}
th {
background: #003087;
color: white;
}
tr:nth-child(even) {
background: #f9f9f9;
}
.badge {
display: inline-block;
padding: 4px 8px;
border-radius: 3px;
font-size: 12px;
font-weight: bold;
}
.badge-primary {
background: #007bff;
color: white;
}
.badge-success {
background: #28a745;
color: white;
}
.badge-warning {
background: #ffc107;
color: black;
}
.toc {
background: #f8f9fa;
border: 1px solid #dee2e6;
padding: 20px;
border-radius: 4px;
margin: 20px 0;
}
.toc ul {
list-style: none;
padding-left: 20px;
}
.toc a {
color: #007bff;
text-decoration: none;
}
.toc a:hover {
text-decoration: underline;
}
@media print {
body {
background: white;
}
.container {
box-shadow: none;
}
pre {
border: 1px solid #ccc;
}
}
</style>
</head>
<body>
<div class="container">
<h1>GE-Enforce Collector API Key Fix</h1>
<p><strong>Issue Date:</strong> Identified September 14, 2026</p>
<p><strong>Last Check-in:</strong> August 28, 2026 at 9:55 PM</p>
<p><strong>Affected Systems:</strong> All GE-Enforce client machines</p>
<div class="alert alert-danger">
<strong>CRITICAL:</strong> GE-Enforce profiles stopped checking in on August 28, 2026.
Collector API token was likely revoked. Immediate action required to restore service.
</div>
<!-- Table of Contents -->
<div class="toc">
<h3>Table of Contents</h3>
<ul>
<li><a href="#problem">1. Problem Summary</a></li>
<li><a href="#diagnosis">2. Diagnosis</a></li>
<li><a href="#quick-fix">3. Quick Fix (Database)</a></li>
<li><a href="#intune-deployment">4. Intune DSC Deployment</a>
<ul>
<li><a href="#option1">Option 1: PowerShell Script (Recommended)</a></li>
<li><a href="#option2">Option 2: Configuration Profile</a></li>
<li><a href="#option3">Option 3: Full DSC Configuration</a></li>
</ul>
</li>
<li><a href="#token-generation">5. Generate New API Token</a></li>
<li><a href="#testing">6. Testing</a></li>
<li><a href="#monitoring">7. Monitoring</a></li>
<li><a href="#rollback">8. Rollback Plan</a></li>
</ul>
</div>
<!-- Problem Summary -->
<h2 id="problem">1. Problem Summary</h2>
<h3>What Happened</h3>
<p>On <strong>August 28, 2026</strong>, a major code deployment introduced a new collector authentication system
that supports per-plugin managed tokens. As part of this change:</p>
<ul>
<li>Plugins can now disable the <code>.env</code> API key fallback via database setting</li>
<li>GE-Enforce collector token was likely revoked during deployment</li>
<li>All GE-Enforce clients stopped checking in after 9:55 PM on August 28</li>
</ul>
<h3>Technical Details</h3>
<p><strong>Code Change:</strong> Commit <code>80bb7b5ba1</code> - "Merge wilm: endpoints plugin, agent, policies"</p>
<p><strong>Affected File:</strong> <code>shopdb/core/api/collector.py</code></p>
<div class="step">
<strong>Before August 28:</strong>
<pre><code>if api_key == expected_key:
return None # Authorized</code></pre>
</div>
<div class="step">
<strong>After August 28:</strong>
<pre><code>if expected_key and api_key == expected_key \
and not _env_fallback_disabled(pluginname): # NEW CHECK
return None # Authorized</code></pre>
</div>
<!-- Diagnosis -->
<h2 id="diagnosis">2. Diagnosis</h2>
<h3>Check Production Database</h3>
<pre><code>-- Check if .env fallback is disabled
SELECT * FROM settings
WHERE key = 'plugin.geenforce.envkeyfallback';
-- Expected result if disabled: value = 'false'
-- Expected result if enabled: no rows or value = 'true'
-- Check last check-in
SELECT MAX(receivedat) as last_checkin,
COUNT(DISTINCT hostname) as unique_hosts
FROM manifestenforcementreports
WHERE iscurrent = 1;
-- Check managed tokens
SELECT * FROM apitokens
WHERE scope LIKE '%geenforce%'
ORDER BY createdat DESC;</code></pre>
<h3>Check IIS Logs</h3>
<p>Look for collector requests failing:</p>
<pre><code>POST /api/collector/geenforce
Status: 401 Unauthorized
Error: "Invalid API key" or "Token revoked"</code></pre>
<!-- Quick Fix -->
<h2 id="quick-fix">3. Quick Fix (Database)</h2>
<div class="alert alert-warning">
<strong>Emergency Fix:</strong> Re-enable .env API key fallback to restore service immediately.
</div>
<div class="step">
<span class="step-number">1</span>
<strong>Connect to production database</strong>
</div>
<div class="step">
<span class="step-number">2</span>
<strong>Re-enable .env fallback</strong>
<pre><code>-- Option A: Delete the setting (defaults to enabled)
DELETE FROM settings
WHERE key = 'plugin.geenforce.envkeyfallback';
-- Option B: Explicitly set to true
INSERT INTO settings (key, value, valuetype)
VALUES ('plugin.geenforce.envkeyfallback', 'true', 'boolean')
ON DUPLICATE KEY UPDATE value = 'true';</code></pre>
</div>
<div class="step">
<span class="step-number">3</span>
<strong>Verify clients start checking in</strong>
<pre><code>-- Wait 15-30 minutes, then check
SELECT MAX(receivedat) as last_checkin
FROM manifestenforcementreports
WHERE receivedat > NOW() - INTERVAL 1 HOUR;</code></pre>
</div>
<div class="alert alert-info">
<strong>Note:</strong> This assumes your <code>.env</code> file has a valid <code>COLLECTOR_API_KEY</code>
and all GE-Enforce clients are configured with that key.
</div>
<!-- Intune Deployment -->
<h2 id="intune-deployment">4. Intune DSC Deployment</h2>
<p>Deploy the API key to all GE-Enforce client machines via Intune.</p>
<h3>GE-Enforce Registry Configuration</h3>
<table>
<tr>
<th>Registry Path</th>
<th>Value Name</th>
<th>Value Type</th>
<th>Purpose</th>
</tr>
<tr>
<td><code>HKLM:\SOFTWARE\GE\ShopDB</code></td>
<td><code>BaseUrl</code></td>
<td>REG_SZ</td>
<td>ShopDB server URL</td>
</tr>
<tr>
<td><code>HKLM:\SOFTWARE\GE\ShopDB</code></td>
<td><code>ApiToken</code></td>
<td>REG_SZ</td>
<td>Collector API key</td>
</tr>
</table>
<!-- Option 1 -->
<h3 id="option1">Option 1: PowerShell Script <span class="badge badge-success">RECOMMENDED</span></h3>
<p><strong>Deployment Path:</strong> Intune Portal > Devices > Scripts > Add Windows 10 script</p>
<div class="step">
<span class="step-number">1</span>
<strong>Create PowerShell script</strong>
<p>Save as <code>Deploy-GEEnforceAPIKey.ps1</code>:</p>
</div>
<pre><code>&lt;#
.SYNOPSIS
Deploy GE-Enforce collector API key via Intune
.DESCRIPTION
Writes the ShopDB API token to registry for GE-Enforce client.
Run as SYSTEM, no user context needed.
.NOTES
Deploy via: Intune > Devices > Scripts > Add
Run as: SYSTEM
Platform: Windows 10/11
#>
$ErrorActionPreference = 'Stop'
# ============================================
# CONFIGURATION - UPDATE THESE VALUES
# ============================================
$ShopdbBaseUrl = "https://shopdb.your-company.com"
$ShopdbApiToken = "shopdb_pat_YOUR_NEW_TOKEN_HERE" # Replace with actual token
# ============================================
# REGISTRY CONFIGURATION
# ============================================
$RegPath = 'HKLM:\SOFTWARE\GE\ShopDB'
try {
Write-Output "Starting GE-Enforce API key deployment..."
# Create registry key if it doesn't exist
if (-not (Test-Path $RegPath)) {
New-Item -Path $RegPath -Force | Out-Null
Write-Output "Created registry path: $RegPath"
}
# Set BaseUrl
Set-ItemProperty -Path $RegPath -Name 'BaseUrl' -Value $ShopdbBaseUrl -Type String
Write-Output "Set BaseUrl: $ShopdbBaseUrl"
# Set ApiToken (REG_SZ - plain text). HKLM\SOFTWARE\GE\ShopDB inherits
# read for BUILTIN\Users, so any logged-on user can read this value unless
# an ACL is applied to the key. Per-PC credentials remove the need for a
# fleet secret here: see plugins/endpoints/docs/PER-PC-CREDENTIALS-ROLLOUT.md
Set-ItemProperty -Path $RegPath -Name 'ApiToken' -Value $ShopdbApiToken -Type String
# Verify token was written
$savedToken = Get-ItemProperty -Path $RegPath -Name 'ApiToken' -ErrorAction Stop
if ($savedToken.ApiToken -eq $ShopdbApiToken) {
Write-Output "Set ApiToken: [REDACTED - first 20 chars: $($ShopdbApiToken.Substring(0, 20))...]"
Write-Output "SUCCESS: GE-Enforce API configuration deployed"
exit 0
} else {
throw "Token verification failed"
}
} catch {
Write-Error "FAILED: $_"
Write-Output "Registry path: $RegPath"
Write-Output "Error details: $($_.Exception.Message)"
exit 1
}</code></pre>
<div class="step">
<span class="step-number">2</span>
<strong>Configure Intune deployment</strong>
<ul>
<li><strong>Name:</strong> Deploy GE-Enforce Collector API Key</li>
<li><strong>Description:</strong> Updates ShopDB API token for GE-Enforce clients</li>
<li><strong>Script location:</strong> Upload <code>Deploy-GEEnforceAPIKey.ps1</code></li>
<li><strong>Run this script using logged-on credentials:</strong> No (run as SYSTEM)</li>
<li><strong>Enforce script signature check:</strong> No</li>
<li><strong>Run script in 64-bit PowerShell:</strong> Yes</li>
</ul>
</div>
<div class="step">
<span class="step-number">3</span>
<strong>Assign to devices</strong>
<ul>
<li><strong>Pilot Group:</strong> Deploy to test group first (5-10 machines)</li>
<li><strong>Production:</strong> All devices with GE-Enforce installed</li>
<li><strong>Schedule:</strong> Run once (not recurring)</li>
</ul>
</div>
<!-- Option 2 -->
<h3 id="option2">Option 2: Configuration Profile <span class="badge badge-primary">ENTERPRISE</span></h3>
<p><strong>Deployment Path:</strong> Intune Portal > Devices > Configuration profiles > Create > Settings catalog</p>
<div class="step">
<span class="step-number">1</span>
<strong>Create Settings Catalog profile</strong>
<ul>
<li><strong>Name:</strong> GE-Enforce Collector Configuration</li>
<li><strong>Platform:</strong> Windows 10 and later</li>
</ul>
</div>
<div class="step">
<span class="step-number">2</span>
<strong>Add Settings</strong>
<p><strong>Setting 1: BaseUrl</strong></p>
<ul>
<li>Category: Administrative Templates > Registry</li>
<li>Key Path: <code>HKLM\SOFTWARE\GE\ShopDB</code></li>
<li>Value Name: <code>BaseUrl</code></li>
<li>Value Type: String</li>
<li>Value Data: <code>https://shopdb.your-company.com</code></li>
</ul>
<p><strong>Setting 2: ApiToken</strong></p>
<ul>
<li>Category: Administrative Templates > Registry</li>
<li>Key Path: <code>HKLM\SOFTWARE\GE\ShopDB</code></li>
<li>Value Name: <code>ApiToken</code></li>
<li>Value Type: String</li>
<li>Value Data: <code>shopdb_pat_YOUR_TOKEN_HERE</code></li>
</ul>
</div>
<div class="step">
<span class="step-number">3</span>
<strong>Assign and deploy</strong>
</div>
<!-- Option 3 -->
<h3 id="option3">Option 3: Full DSC Configuration <span class="badge badge-primary">ADVANCED</span></h3>
<p>For Azure Automation DSC environments:</p>
<pre><code>Configuration GEEnforceCollector {
param(
[Parameter(Mandatory)]
[string]$BaseUrl,
[Parameter(Mandatory)]
[string]$ApiToken
)
Import-DscResource -ModuleName 'PSDesiredStateConfiguration'
Node 'localhost' {
Registry ShopDBBaseUrl {
Key = 'HKLM:\SOFTWARE\GE\ShopDB'
ValueName = 'BaseUrl'
ValueData = $BaseUrl
ValueType = 'String'
Ensure = 'Present'
}
Registry ShopDBApiToken {
Key = 'HKLM:\SOFTWARE\GE\ShopDB'
ValueName = 'ApiToken'
ValueData = $ApiToken
ValueType = 'String'
Ensure = 'Present'
DependsOn = '[Registry]ShopDBBaseUrl'
}
Script VerifyGEEnforce {
GetScript = { @{ Result = Test-Path 'HKLM:\SOFTWARE\GE\ShopDB' } }
TestScript = {
$token = Get-ItemProperty -Path 'HKLM:\SOFTWARE\GE\ShopDB' -Name 'ApiToken' -ErrorAction SilentlyContinue
return ($null -ne $token)
}
SetScript = { Write-Verbose "Registry already configured by dependencies" }
DependsOn = '[Registry]ShopDBApiToken'
}
}
}
# Compile configuration
GEEnforceCollector -BaseUrl "https://shopdb.your-company.com" `
-ApiToken "shopdb_pat_YOUR_TOKEN_HERE" `
-OutputPath "C:\DSC\GEEnforce"
# Upload to Azure Automation and assign to nodes</code></pre>
<!-- Token Generation -->
<h2 id="token-generation">5. Generate New API Token</h2>
<h3>Option A: Use Existing .env Key <span class="badge badge-success">QUICKEST</span></h3>
<p>On your production IIS server, check <code>.env</code> file:</p>
<pre><code># Look for:
COLLECTOR_API_KEY=shopdb_pat_abc123def456...
# Or plugin-specific:
COLLECTOR_API_KEY_GEENFORCE=shopdb_pat_xyz789...</code></pre>
<p>Use this token in your Intune deployment script.</p>
<h3>Option B: Generate New Managed Token <span class="badge badge-primary">RECOMMENDED</span></h3>
<div class="step">
<span class="step-number">1</span>
<strong>Connect to production server</strong>
</div>
<div class="step">
<span class="step-number">2</span>
<strong>Generate token via Flask CLI</strong>
<pre><code># SSH/RDP to IIS server
cd C:\path\to\shopdb-flask
# Generate new token
flask token create collector.geenforce "GE-Enforce Collector - Intune" `
--scope "collector.ingest.geenforce" `
--userid 1
# Output will be:
# Token created: shopdb_pat_abc123def456...
# Copy this token for Intune deployment</code></pre>
</div>
<div class="step">
<span class="step-number">3</span>
<strong>Verify token in database</strong>
<pre><code>SELECT tokenprefix, scope, description, isactive, createdat
FROM apitokens
WHERE scope LIKE '%geenforce%'
ORDER BY createdat DESC
LIMIT 1;</code></pre>
</div>
<!-- Testing -->
<h2 id="testing">6. Testing</h2>
<h3>Manual Registry Test (Before Intune Deployment)</h3>
<p>Test on a single pilot machine first:</p>
<pre><code># Run as Administrator on a test machine
Set-ItemProperty -Path 'HKLM:\SOFTWARE\GE\ShopDB' `
-Name 'BaseUrl' -Value 'https://shopdb.your-company.com'
Set-ItemProperty -Path 'HKLM:\SOFTWARE\GE\ShopDB' `
-Name 'ApiToken' -Value 'shopdb_pat_YOUR_TOKEN_HERE'
# Verify registry
Get-ItemProperty -Path 'HKLM:\SOFTWARE\GE\ShopDB'
# Run GE-Enforce manually to test
& "C:\Program Files\GE-Enforce\Invoke-ShopdbEnforce.ps1"
# Check for errors in output</code></pre>
<h3>Verify Check-in on Server</h3>
<pre><code>-- Wait 15-30 minutes after deployment, then check database
SELECT hostname, scopename, status, lastcheckin, receivedat
FROM manifestenforcementreports
WHERE receivedat > NOW() - INTERVAL 1 HOUR
ORDER BY receivedat DESC;</code></pre>
<h3>Intune Deployment Status</h3>
<p>Monitor deployment in Intune portal:</p>
<ul>
<li><strong>Devices > Scripts</strong> - Check device status</li>
<li><strong>Look for:</strong> Green checkmark = Success, Red X = Failed</li>
<li><strong>View logs:</strong> Click device > View output</li>
</ul>
<!-- Monitoring -->
<h2 id="monitoring">7. Monitoring</h2>
<h3>Create Alert for Missing Check-ins</h3>
<pre><code>-- Run this query on a schedule (every 6 hours)
-- Alert if any results returned
SELECT
'geenforce' as collector,
MAX(receivedat) as last_checkin,
TIMESTAMPDIFF(HOUR, MAX(receivedat), NOW()) as hours_since,
COUNT(DISTINCT hostname) as total_hosts
FROM manifestenforcementreports
WHERE iscurrent = 1
HAVING hours_since > 24;
-- Expected: 0 rows (all clients checking in)
-- Alert if: > 24 hours since last check-in</code></pre>
<h3>Dashboard Query</h3>
<pre><code>-- GE-Enforce fleet health dashboard
SELECT
DATE(receivedat) as report_date,
COUNT(DISTINCT hostname) as unique_hosts,
COUNT(*) as total_reports,
SUM(CASE WHEN status = 'success' THEN 1 ELSE 0 END) as successful,
SUM(CASE WHEN status = 'failed' THEN 1 ELSE 0 END) as failed
FROM manifestenforcementreports
WHERE receivedat > NOW() - INTERVAL 7 DAY
GROUP BY DATE(receivedat)
ORDER BY report_date DESC;</code></pre>
<!-- Rollback -->
<h2 id="rollback">8. Rollback Plan</h2>
<div class="alert alert-warning">
<strong>If deployment fails:</strong> Follow these steps to restore service.
</div>
<h3>Step 1: Re-enable .env Fallback (Immediate)</h3>
<pre><code>DELETE FROM settings
WHERE key = 'plugin.geenforce.envkeyfallback';</code></pre>
<h3>Step 2: Deploy Old Token via Intune</h3>
<p>Re-deploy the original API key from <code>.env</code> file using same Intune script.</p>
<h3>Step 3: Verify Recovery</h3>
<pre><code>SELECT MAX(receivedat) as last_checkin
FROM manifestenforcementreports
WHERE receivedat > NOW() - INTERVAL 1 HOUR;</code></pre>
<h3>Step 4: Review Logs</h3>
<p>Check Intune device output logs for errors:</p>
<ul>
<li>Access denied to registry</li>
<li>Token format invalid</li>
<li>Script execution failure</li>
</ul>
<!-- Summary -->
<h2>Summary Checklist</h2>
<table>
<tr>
<th>Step</th>
<th>Action</th>
<th>Owner</th>
<th>Status</th>
</tr>
<tr>
<td>1</td>
<td>Verify problem in production database</td>
<td>Admin</td>
<td>☐</td>
</tr>
<tr>
<td>2</td>
<td>Re-enable .env fallback (emergency fix)</td>
<td>Admin</td>
<td>☐</td>
</tr>
<tr>
<td>3</td>
<td>Generate new API token (or get from .env)</td>
<td>Admin</td>
<td>☐</td>
</tr>
<tr>
<td>4</td>
<td>Test manually on 1 pilot machine</td>
<td>Engineer</td>
<td>☐</td>
</tr>
<tr>
<td>5</td>
<td>Create Intune PowerShell script</td>
<td>Engineer</td>
<td>☐</td>
</tr>
<tr>
<td>6</td>
<td>Deploy to pilot group (5-10 machines)</td>
<td>Admin</td>
<td>☐</td>
</tr>
<tr>
<td>7</td>
<td>Verify pilot check-ins in database</td>
<td>Engineer</td>
<td>☐</td>
</tr>
<tr>
<td>8</td>
<td>Deploy to all GE-Enforce clients</td>
<td>Admin</td>
<td>☐</td>
</tr>
<tr>
<td>9</td>
<td>Monitor check-ins for 24 hours</td>
<td>Engineer</td>
<td>☐</td>
</tr>
<tr>
<td>10</td>
<td>Create monitoring alert</td>
<td>Engineer</td>
<td>☐</td>
</tr>
</table>
<div class="alert alert-success">
<strong>Success Criteria:</strong> All GE-Enforce clients checking in within 1 hour of deployment,
no 401 errors in IIS logs, monitoring alert configured.
</div>
<hr style="margin: 40px 0;">
<p><small>
<strong>Document Version:</strong> 1.0<br>
<strong>Created:</strong> September 14, 2026<br>
<strong>Repository:</strong> <code>flask-shopdb/.github/</code><br>
<strong>Related Changes:</strong> Commit <code>80bb7b5ba1</code> (August 28, 2026)
</small></p>
</div>
</body>
</html>