geenforce: shadow runs inside the enforce cycle, not on its own clock

Test bays reported 0 installed / 0 skipped / 0 failed and nothing was being
processed. The share is a TRANSIENT mount: GE-Enforce.ps1 maps it for the length
of its cycle and unmounts at the end - mounted 12:10:02, unmounted 12:10:27 on
FB9TP7V3. The shadow task ran on its own 15-minute schedule, so it woke with the
drive gone, handed the engine a path that no longer resolved, and the summary
zero-filled. A silent nothing, indistinguishable from a healthy no-op, which is
why it read as "not processing" rather than as a failure.

Register-ShopdbShadow.ps1 is replaced by Invoke-ShopdbShadow.ps1, which runs AS
a manifest entry and invokes the runner directly. The share is mounted because
the enforce cycle is what called us; there is one cadence instead of two; and
there is no task to register, drift, or heal. It also unregisters the superseded
task, so bays that already carry it clean themselves up rather than keeping a
run that cannot see the share and overwrites the real result with 0/0/0.

The runner gains the Test-Path guard that should have been there: an unreachable
-ShareManifestPath now logs, writes an event, and reports a real failure naming
the reason, instead of running the engine against nothing. That silent zero is
what made this expensive to find.

Verified on the win11 VM with a substituted drive removed between runs - the
same call gives real counts while mapped and the guard when not. The earlier
task self-heal is moot now; it was a fix at the wrong layer, and testing against
a local path is what hid a mount I had already noted was dynamic.

plugins/geenforce/client/Invoke-ShopdbEnforce.ps1

@@ -131,6 +131,29 @@ try {
     if ($ShadowMode -and $ShareManifestPath) {
         # Shadow: install from the share exactly as today (no payload resolve,
         # no common merge - the share already carries its own common scope).
+        #
+        # GUARDED, because the share is a TRANSIENT mount. GE-Enforce.ps1 maps it
+        # for the length of its cycle and unmounts at the end, so a caller that
+        # runs on its own schedule finds the drive gone. Handing the engine a
+        # path on a dead drive returned nothing usable and the summary zero-filled
+        # to 0 installed / 0 skipped / 0 failed - a silent nothing that reads
+        # exactly like a healthy no-op. Shadow must run INSIDE the enforce cycle;
+        # this makes the alternative loud instead of invisible.
+        if (-not (Test-Path -LiteralPath $ShareManifestPath)) {
+            $reason = "share manifest not reachable at $ShareManifestPath " +
+                      '(is the share still mounted? shadow must run inside the ' +
+                      'enforce cycle)'
+            Write-Log $reason 'WARN'
+            Write-ShopdbEventLog -Message ("GE-Enforce shadow could not reach the share manifest for scope '$Scope': $reason") -EntryType 'Error' -EventId 1002
+            try {
+                $failReport = New-ShopdbReport -Scope $Scope -AppliedVersion 0 -Summary @{
+                    Installed = 0; Skipped = 0; Failed = 1; Filtered = 0; EnforcerVersion = '2.6'
+                    Results = @(@{ Name = '(share-manifest)'; Action = 'failed'; Message = $reason })
+                }
+                Send-ShopdbReport -Config $config -Report $failReport | Out-Null
+            } catch {}
+            exit 0
+        }
         $manifestToRun = $ShareManifestPath
     } else {
         # Optional common-scope inheritance (OFF by default; displays are

plugins/geenforce/client/Invoke-ShopdbShadow.ps1

@@ -0,0 +1,85 @@
+# Invoke-ShopdbShadow.ps1 -- run one shopdb SHADOW cycle, from the manifest.
+#
+# Shadow = fetch the shopdb manifest, diff it against the share manifest, report
+# the cycle to shopdb. The engine still installs FROM THE SHARE, so behaviour is
+# unchanged. It is the observable step before any cutover.
+#
+# WHY THIS RUNS AS A MANIFEST ENTRY AND NOT A SCHEDULED TASK
+#
+# The first version registered a separate 15-minute task. That cannot work:
+# GE-Enforce.ps1 mounts the SFLD share for the length of its own cycle and
+# unmounts it at the end (mounted 12:10:02, unmounted 12:10:27). A task on its
+# own schedule therefore wakes up with the drive gone, hands the engine a path
+# that no longer resolves, and reports 0 installed / 0 skipped / 0 failed - a
+# silent nothing indistinguishable from a healthy no-op. It also meant two
+# cadences that could drift apart, and a registration that had to be healed.
+#
+# Running here removes all of it: the share is mounted because the enforce cycle
+# is what invoked us, the cadence is the fleet's own, and there is no task.
+#
+# Runs as SYSTEM under GE-Enforce, from an entry gated to the test bays.
+# Fail-safe: exits 0 on every path - shadowing must never stop a bay enforcing.
+
+$ErrorActionPreference = 'Continue'
+
+$InstallDir = 'C:\Program Files\GE\Shopfloor'
+$BaseUrl    = 'https://tsgwp00525.wjs.geaerospace.net/shopdb'
+$LegacyTask = 'ShopDB GE-Enforce (shadow)'
+
+function Write-ShadowLog {
+    # To a file as well as the host: the engine records only "ps1: <path>" and
+    # an exit code for a PS1 entry, so Write-Host reaches nothing, and with the
+    # fail-safe exit 0 a silent early-out looks exactly like success.
+    param([string]$Message)
+    $line = "[{0}] [shadow] {1}" -f (Get-Date -Format 'yyyy-MM-dd HH:mm:ss'), $Message
+    Write-Host $line
+    try {
+        $dir = 'C:\Logs\Shopfloor'
+        if (-not (Test-Path $dir)) { New-Item -ItemType Directory -Path $dir -Force | Out-Null }
+        Add-Content -LiteralPath (Join-Path $dir ('shadow-{0}.log' -f (Get-Date -Format yyyyMMdd))) -Value $line
+    } catch { }
+}
+
+try {
+    # Retire the scheduled task the earlier version left behind. Bays that got
+    # it would otherwise keep firing a run that cannot see the share, reporting
+    # 0/0/0 over the top of the real result from this one.
+    $stale = Get-ScheduledTask -TaskName $LegacyTask -ErrorAction SilentlyContinue
+    if ($stale) {
+        Unregister-ScheduledTask -TaskName $LegacyTask -Confirm:$false -ErrorAction SilentlyContinue
+        Write-ShadowLog "removed the superseded '$LegacyTask' task (it ran outside the share mount)."
+    }
+
+    # Scope is this script's own directory name, never hardcoded: the same file
+    # ships from more than one scope and a wrong value would shadow the wrong
+    # manifest silently.
+    $scopeDir = Split-Path -Parent $PSScriptRoot
+    $scope = Split-Path -Leaf $scopeDir
+    $shareManifest = Join-Path $scopeDir 'manifest.json'
+
+    $runner = Join-Path $InstallDir 'Invoke-ShopdbEnforce.ps1'
+    $engine = Join-Path $InstallDir 'lib\Install-FromManifest.ps1'
+    foreach ($required in @($runner, $engine, $shareManifest)) {
+        if (-not (Test-Path -LiteralPath $required)) {
+            Write-ShadowLog "MISSING $required - skipping this cycle."
+            exit 0
+        }
+    }
+
+    # BaseUrl only when it differs, so a hand-set value is not churned.
+    $regPath = 'HKLM:\SOFTWARE\GE\ShopDB'
+    if (-not (Test-Path $regPath)) { New-Item -Path $regPath -Force | Out-Null }
+    if ((Get-ItemProperty -Path $regPath -Name BaseUrl -ErrorAction SilentlyContinue).BaseUrl -ne $BaseUrl) {
+        Set-ItemProperty -Path $regPath -Name BaseUrl -Value $BaseUrl
+        Write-ShadowLog "BaseUrl set to $BaseUrl"
+    }
+
+    Write-ShadowLog "shadowing $scope against $shareManifest"
+    & $runner -Scope $scope -EnginePath $engine -ShadowMode -ShareManifestPath $shareManifest
+    Write-ShadowLog "cycle complete (runner exit $LASTEXITCODE)."
+    exit 0
+}
+catch {
+    Write-ShadowLog "FAILED: $_"
+    exit 0
+}

plugins/geenforce/client/Register-ShopdbShadow.ps1

@@ -1,129 +0,0 @@
-# Register-ShopdbShadow.ps1 -- put this bay into shopdb SHADOW mode.
-#
-# Shadow mode = fetch the shopdb manifest, diff it against the share manifest,
-# report the cycle to shopdb, and install FROM THE SHARE exactly as today. Zero
-# behaviour change. It is the observable step before any cutover.
-#
-# Runs as SYSTEM under GE-Enforce, from a manifest entry gated to one hostname.
-# Idempotent: re-registers the task each cycle so drift self-heals, and writes
-# BaseUrl only when it differs.
-#
-# WHY the share manifest is read through the mounted drive: GE-Enforce.ps1
-# mounts the SFLD share with SFLD credentials before invoking the engine, and
-# this script runs inside that window. SYSTEM has no standing access to the UNC
-# path, so the mounted drive is the only path that resolves. The drive letter is
-# not fixed, so it is derived from where this script is running rather than
-# hardcoded.
-
-$ErrorActionPreference = 'Continue'
-
-$TaskName   = 'ShopDB GE-Enforce (shadow)'
-$InstallDir = 'C:\Program Files\GE\Shopfloor'
-$BaseUrl    = 'https://tsgwp00525.wjs.geaerospace.net/shopdb'
-# Scope is NOT hardcoded: this script is shipped by more than one scope
-# (collections and nocollections today), and a wrong value here would shadow the
-# wrong manifest silently. It runs from <drive>:\<scope>\shopdb-client, so the
-# directory it sits under IS the scope name - the same derivation used below for
-# the share manifest, so the two cannot disagree.
-$Scope      = Split-Path -Leaf (Split-Path -Parent $PSScriptRoot)
-
-function Write-ShadowLog {
-    # Write-Host ALONE is not enough: the engine records only "ps1: <path>" and
-    # the exit code for a PS1 entry, so nothing this script says reaches the
-    # enforce log. Combined with the fail-safe `exit 0` on every path, a silent
-    # early-out was indistinguishable from success - which is exactly how the
-    # never-firing task went unnoticed. Write to a file as well so the next
-    # failure is answerable from disk.
-    param([string]$Message)
-    $line = "[{0}] [shadow-setup] {1}" -f (Get-Date -Format 'yyyy-MM-dd HH:mm:ss'), $Message
-    Write-Host $line
-    try {
-        $dir = 'C:\Logs\Shopfloor'
-        if (-not (Test-Path $dir)) { New-Item -ItemType Directory -Path $dir -Force | Out-Null }
-        Add-Content -LiteralPath (Join-Path $dir ('shadow-setup-{0}.log' -f (Get-Date -Format yyyyMMdd))) -Value $line
-    } catch { }
-}
-
-try {
-    # --- BaseUrl (the client reads this; no token needed on an allowlisted subnet)
-    $regPath = 'HKLM:\SOFTWARE\GE\ShopDB'
-    if (-not (Test-Path $regPath)) { New-Item -Path $regPath -Force | Out-Null }
-    $current = (Get-ItemProperty -Path $regPath -Name BaseUrl -ErrorAction SilentlyContinue).BaseUrl
-    if ($current -ne $BaseUrl) {
-        Set-ItemProperty -Path $regPath -Name BaseUrl -Value $BaseUrl
-        Write-ShadowLog "BaseUrl set to $BaseUrl"
-    }
-
-    $runner = Join-Path $InstallDir 'Invoke-ShopdbEnforce.ps1'
-    $engine = Join-Path $InstallDir 'lib\Install-FromManifest.ps1'
-    foreach ($p in @($runner, $engine)) {
-        if (-not (Test-Path -LiteralPath $p)) {
-            Write-ShadowLog "MISSING $p - the File entries have not landed yet; will retry next cycle."
-            exit 0   # fail-safe: never break the bay, the entry re-runs
-        }
-    }
-
-    # --- the share manifest this scope is enforced from, via the mounted drive.
-    # $PSScriptRoot is <drive>:\<scope>\shopdb-client, so its grandparent is the
-    # scope dir. Deriving it keeps this correct whatever letter GE-Enforce mounted.
-    $scopeDir = Split-Path -Parent $PSScriptRoot
-    $shareManifest = Join-Path $scopeDir 'manifest.json'
-    if (-not (Test-Path -LiteralPath $shareManifest)) {
-        Write-ShadowLog "share manifest not found at $shareManifest - not registering."
-        exit 0
-    }
-
-    $arguments = '-NoProfile -ExecutionPolicy Bypass -File "{0}" -Scope "{1}" -EnginePath "{2}" -ShadowMode -ShareManifestPath "{3}"' `
-        -f $runner, $Scope, $engine, $shareManifest
-
-    # ONLY register when it is missing or its arguments changed.
-    #
-    # Registering unconditionally is what stopped this working the first time.
-    # A `-Once -At (Get-Date)` trigger does NOT fire immediately: the first run
-    # is start-boundary PLUS the repetition interval, so 15 minutes out. This
-    # entry is DetectionMethod=Always and runs every enforce cycle, 5 minutes
-    # apart, and each Register-ScheduledTask -Force reset the start boundary to
-    # "now" - pushing the first run back to +15 before the previous +15 could
-    # elapse. 5 < 15, so the task sat at Ready with LastTaskResult 267011
-    # (SCHED_S_TASK_HAS_NOT_RUN) indefinitely. Measured on the win11 VM.
-    $existing = Get-ScheduledTask -TaskName $TaskName -ErrorAction SilentlyContinue
-    if ($existing) {
-        $currentArgs = ($existing.Actions | Select-Object -First 1).Arguments
-        if ($currentArgs -eq $arguments) {
-            # Correct arguments, but a bay provisioned by the BROKEN version of
-            # this script carries a task that was reset every cycle and so has
-            # never run. Leaving it alone would strand exactly the bays that hit
-            # the bug. Kick it once; after that LastRunTime is set and this is a
-            # no-op forever.
-            $info = $existing | Get-ScheduledTaskInfo
-            $neverran = ($null -eq $info.LastRunTime) -or
-                        ($info.LastRunTime -lt (Get-Date '2000-01-01'))
-            if ($neverran) {
-                Write-ShadowLog ("task exists but has never run (LastTaskResult $($info.LastTaskResult)) - starting it once.")
-                Start-ScheduledTask -TaskName $TaskName -ErrorAction SilentlyContinue
-            }
-            return   # schedule is correct; do not reset the start boundary
-        }
-        Write-ShadowLog 'task arguments changed - re-registering.'
-    }
-
-    $action = New-ScheduledTaskAction -Execute 'powershell.exe' -Argument $arguments
-    # RepetitionInterval ALONE - passing RepetitionDuration serializes to a
-    # Duration the Task Scheduler schema rejects.
-    $trigger = New-ScheduledTaskTrigger -Once -At (Get-Date) -RepetitionInterval (New-TimeSpan -Minutes 15)
-    $principal = New-ScheduledTaskPrincipal -UserId 'NT AUTHORITY\SYSTEM' -RunLevel Highest
-    $settings = New-ScheduledTaskSettingsSet -AllowStartIfOnBatteries -DontStopIfGoingOnBatteries -StartWhenAvailable
-
-    Register-ScheduledTask -TaskName $TaskName -Action $action -Trigger $trigger `
-        -Principal $principal -Settings $settings -Force | Out-Null
-    # Kick it once rather than waiting out the first 15-minute interval, so a
-    # freshly provisioned bay reports on this cycle instead of the next.
-    Start-ScheduledTask -TaskName $TaskName -ErrorAction SilentlyContinue
-    Write-ShadowLog "registered '$TaskName' (every 15 min), shadowing $shareManifest"
-    exit 0
-}
-catch {
-    # Fail-safe: a broken setup script must never stop the bay enforcing.
-    Write-ShadowLog "FAILED: $_"
-    exit 0
-}