fix(installer): clear the retry path, which is the path everyone is actually on

Five defects from the Windows-defect review, each confirmed against the code
before changing it. Four of the five only fire on a RE-RUN - and after eight
attempts a re-run is the normal case, not an edge case, which is exactly why
they survived.

Invoke-Native, three defects in one function:
  - Any non-zero exit was failure. 3010 and 1641 mean "done, reboot required",
    and the VC++ redistributable returns 3010 on a server with a pending file
    rename - an ordinary state on a freshly patched box. It is now an accepted
    outcome for the installers that can report it, logged as a warning so the
    operator knows a reboot is owed.
  - -Wait blocks inside Start-Process until the child exits, so the -TimeoutSec
    block below it could never run. Every timeout on every MSI was decorative.
    The wait is now bounded here, followed by a parameterless WaitForExit so the
    redirected output is flushed before it is read.
  - The Python bootstrapper ran /quiet with no /norestart, free to reboot the
    server mid-install.

Stage 0 refused to run when a MySQL service existed - including the MySQL84 it
had registered itself. Every bundled-database retry dead-ended while the wizard
promised that re-running was safe. A foreign MySQL still blocks; ours is started
if stopped, and the create-the-server block is skipped. It also no longer tries
to bootstrap through a root account whose password it set on the previous run:
with the handoff present there is nothing to do, and without it there is no safe
automatic recovery, so it says what to do instead of guessing.

Stage 4's appcmd unlock used '2>&1' under $ErrorActionPreference = 'Stop', which
turns any appcmd stderr into a terminating error - so the exit-code test and the
server-wide fallback, the whole reason the block exists, were unreachable, and
the stage aborted after Python, the venv, the schema and the ACLs had been
changed.

Stage 3 ran prune-schema and treated its refusal as a failure. Refusing is the
designed outcome when a table holds rows, signalled with SystemExit(1), so
Invoke-Native killed the stage and the reporting written to explain the refusal
was unreachable. Core migration 7d05 seeds access protocols owned by the
computers plugin, so any profile omitting computers hit this on every retry.

The preflight's MySQL 5.6 index-flag check is a warning, not a blocker. It
inspects the LOCAL MySQL, which may not be the database being installed against;
stage 3 checks the one actually chosen. Same class as the HttpPlatformHandler
blocker fixed earlier.

deploy/windows/installer/shopdb-install.ps1

@@ -155,6 +155,8 @@ try { if (-not (Test-Path $script:LogDir)) { New-Item -ItemType Directory -Path
 catch { $script:LogDir = $env:TEMP }
 $script:LogPath = Join-Path $script:LogDir ("shopdb-install-{0}.log" -f (Get-Date -Format 'yyyyMMdd-HHmmss'))
 $script:Created = New-Object System.Collections.ArrayList   # for rollback
+$script:RebootPending = $false
+$script:MysqlAlreadyOurs = $false
 # Was this server already stamped by a previous run of THIS installer, as observed
 # BEFORE stage 2 writes its own stamp? Stage 4 needs the answer to know whether the
 # IIS objects it is about to reconcile are its own. $null means nothing has looked
@@ -194,8 +196,12 @@ function Invoke-Native {
     #     $ErrorActionPreference='Stop' turns that into a terminating error.
     #  3. Argument VALUES must never be logged: a DATABASE_URL or password would
     #     land in the log we invite operators to send to support.
+    #  4. Some installers succeed with a NON-ZERO code. 3010 and 1641 both mean
+    #     "done, reboot required" - the VC++ redistributable returns 3010 on a
+    #     server with a pending file rename, which is an ordinary state on a
+    #     freshly patched box. Treating it as failure aborts a correct install.
     param([string] $Exe, [string[]] $Arguments, [string] $What, [string] $StdinFile = '',
-          [int] $TimeoutSec = 0)
+          [int] $TimeoutSec = 0, [int[]] $OkExit = @(0))
     Write-Log ("  exec {0} ({1} args)" -f (Split-Path $Exe -Leaf), $Arguments.Count)
     # Start-Process -ArgumentList joins the array with spaces WITHOUT quoting, so
     # any argument containing whitespace gets split into several. That is how
@@ -212,22 +218,27 @@ function Invoke-Native {
     $so = [System.IO.Path]::GetTempFileName()
     $se = [System.IO.Path]::GetTempFileName()
     try {
+        # NO -Wait when a timeout is wanted. -Wait blocks inside Start-Process
+        # until the child exits, so by the time the timeout check below ran the
+        # process had ALWAYS exited and the whole block was dead code - every
+        # -TimeoutSec on every MSI call was doing nothing. Start without -Wait
+        # and do the waiting here, where it can be bounded.
+        $common = @{ FilePath = $Exe; ArgumentList = $quoted; PassThru = $true
+                     NoNewWindow = $true
+                     RedirectStandardOutput = $so; RedirectStandardError = $se }
         if ($StdinFile) {
             # Feeding SQL as a file on stdin, not as -e "source <path>": the mysql
             # client reads that path as a DATABASE NAME and fails with
             # ERROR 1049 Unknown database '<path>'.
-            $p = Start-Process -FilePath $Exe -ArgumentList $quoted -Wait -PassThru `
+            $common['RedirectStandardInput'] = $StdinFile
-                     -NoNewWindow -RedirectStandardInput $StdinFile `
-                     -RedirectStandardOutput $so -RedirectStandardError $se
-        } else {
-            $p = Start-Process -FilePath $Exe -ArgumentList $quoted -Wait -PassThru `
-                     -NoNewWindow -RedirectStandardOutput $so -RedirectStandardError $se
         }
+        if ($TimeoutSec -le 0) { $common['Wait'] = $true }
+        $p = Start-Process @common
+
         # A hung child must not hang the install. msiexec in particular deadlocks
         # when a second Windows Installer transaction holds the global mutex: two
-        # msiexec processes sat there for 40 minutes with no output and no error,
+        # msiexec processes sat there for 40 minutes with no output and no error.
-        # and -Wait alone gives you no way out.
+        if ($TimeoutSec -gt 0) {
-        if ($TimeoutSec -gt 0 -and -not $p.HasExited) {
             if (-not $p.WaitForExit($TimeoutSec * 1000)) {
                 try { $p.Kill() } catch { }
                 Fail "$What timed out after $TimeoutSec seconds" @'
@@ -236,14 +247,25 @@ transaction held the global mutex: check for stray msiexec processes
 (Get-Process msiexec), end them, then re-run this stage.
 '@
             }
+            # The parameterless overload waits for the redirected streams to be
+            # flushed and closed. Without it the log files can still be being
+            # written when they are read below, losing the tail of a failure.
+            $p.WaitForExit()
         }
         $out = @()
         if (Test-Path $so) { $out += Get-Content $so -ErrorAction SilentlyContinue }
         if (Test-Path $se) { $out += Get-Content $se -ErrorAction SilentlyContinue }
-        if ($p.ExitCode -ne 0) {
+        if ($OkExit -notcontains $p.ExitCode) {
             $out | Select-Object -Last 8 | ForEach-Object { Write-Log "    $_" 'FAIL' }
             Fail "$What failed (exit $($p.ExitCode))"
         }
+        if ($p.ExitCode -ne 0) {
+            # Accepted, but say so: 3010/1641 mean the server needs a reboot to
+            # finish, and an operator who never sees that wonders later why
+            # something only half works.
+            Write-Log ("$What returned {0} - completed, but this server needs a reboot to finish" -f $p.ExitCode) 'WARN'
+            $script:RebootPending = $true
+        }
         return $out
     }
     finally { Remove-Item $so, $se -Force -ErrorAction SilentlyContinue }
@@ -659,14 +681,33 @@ bundle, or skip stage 0 and use the site's existing server (the common case).
     # Refuse to stack a second server on an existing one. Two servers fighting
     # over 3306, or a silent takeover of the classic ASP app's database, is far
     # worse than stopping here.
-    $existing = Get-Service -Name 'MySQL*' -ErrorAction SilentlyContinue
+    # OUR service is not "someone else's MySQL". This refused to run when the
-    if ($existing) {
+    # only service present was the MySQL84 that a previous stage 0 registered -
-        Fail ("MySQL service already present: {0}" -f (($existing | ForEach-Object Name) -join ', ')) @'
+    # so every retry of a part-completed bundled install dead-ended, while the
+    # wizard told the operator that re-running was safe.
+    $existing = @(Get-Service -Name 'MySQL*' -ErrorAction SilentlyContinue)
+    $foreign  = @($existing | Where-Object { $_.Name -ne $MysqlService })
+    if ($foreign.Count -gt 0) {
+        Fail ("MySQL service already present: {0}" -f (($foreign | ForEach-Object Name) -join ', ')) @'
 This box already runs MySQL. Do NOT install the bundled server on top of it:
 it would collide on port 3306 and could disrupt the existing application.
 Skip stage 0 and point the installer at the existing server with -DbHost.
 '@
     }
+    $ours = @($existing | Where-Object { $_.Name -eq $MysqlService })
+    if ($ours.Count -gt 0) {
+        # Ours, from an earlier run. Bring it up if it is down and let the rest of
+        # the stage run: the bootstrap SQL is idempotent (CREATE ... IF NOT EXISTS
+        # plus ALTER USER), so it re-asserts the password and the grants and
+        # leaves .dbpass agreeing with the server.
+        Write-Log "$MysqlService is already registered by a previous run" 'OK'
+        if ($ours[0].Status -ne 'Running') {
+            Write-Log "starting $MysqlService"
+            Start-Service -Name $MysqlService -ErrorAction SilentlyContinue
+            Start-Sleep -Seconds 3
+        }
+        $script:MysqlAlreadyOurs = $true
+    }
     $inUse = Get-NetTCPConnection -LocalPort $DbPort -State Listen -ErrorAction SilentlyContinue
     if ($inUse) { Fail "port $DbPort is already in use" 'Free the port, or use the existing server.' }
 
@@ -715,7 +756,7 @@ Microsoft Visual C++ 2015-2022 Redistributable (x64) on this server by hand.
             Write-Log "installing $($vc.Name) (MySQL requires it)"
             if (-not $WhatIfOnly) {
                 Invoke-Native $vc.FullName @('/install','/quiet','/norestart') `
-                    'Visual C++ runtime' -TimeoutSec 600
+                    'Visual C++ runtime' -TimeoutSec 600 -OkExit 0,3010,1641
                 Write-Log 'Visual C++ runtime installed' 'OK'
             }
         }
@@ -730,7 +771,7 @@ Microsoft Visual C++ 2015-2022 Redistributable (x64) on this server by hand.
         Write-Log "MSI log: $msiLog"
         Invoke-Native 'msiexec.exe' @('/i', $msi.FullName, '/quiet', '/norestart',
                                       '/l*v', $msiLog,
-                                      "INSTALLDIR=$MysqlRoot") 'MySQL MSI' -TimeoutSec 900
+                                      "INSTALLDIR=$MysqlRoot") 'MySQL MSI' -TimeoutSec 900 -OkExit 0,3010
     }
     # An installer exit code of 0 does NOT mean it did what you asked - the Python
     # bootstrapper returned 0 while installing to the wrong directory. Verify.
@@ -759,6 +800,14 @@ log-error=$MysqlDataDir\mysql-error.log
     Write-Log "wrote $MysqlIni"
     Track 'mysql-files' $MysqlIni
 
+    # Everything from here to the bootstrap creates the server. Skip it wholesale
+    # when the service is already ours: the datadir check would refuse a
+    # populated directory, --initialize-insecure would refuse it too, and
+    # --install would fail on an existing service name.
+    if ($script:MysqlAlreadyOurs) {
+        Write-Log 'server already exists from a previous run; skipping create' 'OK'
+    } else {
+
     # --- data directory -----------------------------------------------------
     # --initialize-insecure REFUSES a non-empty datadir, so a previous failed
     # attempt must be cleared before retrying.
@@ -788,9 +837,32 @@ data, STOP: you are about to destroy a database.
     Set-Service -Name $MysqlService -StartupType Automatic
     Write-Log "service $MysqlService running" 'OK'
 
+    }   # end of the create-the-server block
+
     # --- bootstrap ----------------------------------------------------------
     # --initialize-insecure leaves root with an EMPTY password, so this must run
     # immediately. Generated, never supplied; root is shown once and not stored.
+    #
+    # On a server this stage already built, root HAS a password, so the
+    # --skip-password connection below cannot authenticate. The database, the
+    # user and the handoff all exist from that run, so there is nothing to do -
+    # skip rather than fail. Without the handoff there is no safe automatic
+    # recovery, so say what to do instead of guessing.
+    if ($script:MysqlAlreadyOurs) {
+        if (Test-Path (Join-Path $AppRoot '.dbpass')) {
+            Write-Log 'database, user and password handoff already exist; skipping bootstrap' 'OK'
+            Write-Log 'stage 0 complete' 'OK'
+            return
+        }
+        Fail 'MySQL is installed by a previous run but its password handoff is gone' @'
+The bundled server exists and root already has a generated password, so this
+stage cannot re-create the application user, and .dbpass is not there to reuse.
+
+Either point the installer at this server with -DbHost 127.0.0.1 and the
+password from C:\shopdb-flask\.env, or remove the MySQL84 service and its data
+directory to install cleanly.
+'@
+    }
     $rootPass = New-Secret -Bytes 24
     $appPass  = New-Secret -Bytes 24
     # Written OUTSIDE %TEMP% and ACL'd BEFORE the secrets go in - a file created
@@ -907,10 +979,12 @@ function Invoke-Stage2 {
     } else {
         Write-Log "installing Python (all users) from $($pyInstaller.Name)"
         if (-not $WhatIfOnly) {
+            # /norestart matters: without it the bootstrapper is free to reboot
+            # the server on its own, mid-install, with the wizard still running.
             Invoke-Native $pyInstaller.FullName @(
-                '/quiet','InstallAllUsers=1','PrependPath=0','Include_launcher=1',
+                '/quiet','/norestart','InstallAllUsers=1','PrependPath=0','Include_launcher=1',
                 'Include_test=0','AssociateFiles=0',"TargetDir=$pyTarget"
-            ) 'Python install'
+            ) 'Python install' -TimeoutSec 900 -OkExit 0,3010,1641
             Track 'python' $pyTarget
         }
     }
@@ -1485,12 +1559,23 @@ build to get a matching pair, then send the install log to support.
         # which is the behaviour we want when data exists.
         if (-not $script:DbWasEmpty) {
             Write-Log 'flask plugin prune-schema --yes   (no --force: refuses to drop tables holding data)'
-            $pruneOut = Invoke-Native $Flask @('plugin','prune-schema','--yes') 'prune not-installed plugin tables'
+            # -OkExit 0,1: refusing is the DESIGNED outcome here, and the CLI
-            $skipped = @($pruneOut | Where-Object { $_ -match 'refus|not empty|rows' })
+            # signals it with SystemExit(1). Without this, Invoke-Native failed
+            # the stage the moment prune declined to drop a table holding rows -
+            # and the reporting below, which exists precisely to explain that,
+            # could never run. Core migration 7d05 seeds access protocols owned
+            # by the computers plugin, so any profile omitting computers hit it
+            # on every retry.
+            $pruneOut = Invoke-Native $Flask @('plugin','prune-schema','--yes') `
+                            'prune not-installed plugin tables' -OkExit 0,1
+            $skipped = @($pruneOut | Where-Object { $_ -match 'REFUSING|rows$|not empty' })
             if ($skipped.Count -gt 0) {
                 Write-Log 'some plugin tables were kept because they hold data:' 'WARN'
                 $skipped | ForEach-Object { Write-Log "    $_" 'WARN' }
                 Write-Log '  drop them by hand only if you are certain that data is not needed' 'WARN'
+            } elseif ($pruneOut -match 'not initialized') {
+                # The other reason this CLI exits 1, and it is a genuine failure.
+                Fail 'the plugin manager did not initialise' 'Check the application log; the app could not start.'
             }
         } else {
             # Provably-empty database only: core migrations seed a few plugin
@@ -1533,7 +1618,7 @@ the module by hand, then re-run stage 4.
         Write-Log "installing $($hphMsi.Name)"
         if (-not $WhatIfOnly) {
             Invoke-Native 'msiexec.exe' @('/i', $hphMsi.FullName, '/quiet', '/norestart') `
-                'HttpPlatformHandler MSI' -TimeoutSec 600
+                'HttpPlatformHandler MSI' -TimeoutSec 600 -OkExit 0,3010
             if (-not (Test-Path $hphDll)) {
                 Fail 'the HttpPlatformHandler MSI reported success but the module is missing' `
                      'An installer exit code of 0 does not prove it did what was asked.'
@@ -1565,7 +1650,7 @@ has to come from the bundle either way.
             Write-Log "installing $($rwMsi.Name)"
             if (-not $WhatIfOnly) {
                 Invoke-Native 'msiexec.exe' @('/i', $rwMsi.FullName, '/quiet', '/norestart') `
-                    'URL Rewrite MSI' -TimeoutSec 600
+                    'URL Rewrite MSI' -TimeoutSec 600 -OkExit 0,3010
                 if (-not (Test-Path $rewriteDll)) {
                     Fail 'the URL Rewrite MSI reported success but the module is missing' `
                          'An installer exit code of 0 does not prove it did what was asked.'
@@ -1659,12 +1744,25 @@ has to come from the bundle either way.
     foreach ($section in @('system.webServer/handlers','system.webServer/httpPlatform')) {
         if ($WhatIfOnly) { Write-Log "would unlock $section for $ourLocation"; continue }
         Write-Log "unlocking $section for $ourLocation"
-        $scoped = & $appcmd unlock config "$ourLocation" /section:$section 2>&1
+        # EAP must drop to Continue around these. appcmd writes to stderr on
-        $scoped | ForEach-Object { Write-Log "    $_" }
+        # ordinary refusals, and '2>&1' with $ErrorActionPreference = 'Stop'
-        if ($LASTEXITCODE -ne 0) {
+        # turns that into a TERMINATING NativeCommandError - so the exit-code
-            Write-Log "  scoped unlock refused; unlocking $section server-wide" 'WARN'
+        # test and the server-wide fallback below, which is the entire point of
-            & $appcmd unlock config /section:$section 2>&1 | ForEach-Object { Write-Log "    $_" }
+        # the block, were unreachable, and stage 4 aborted after Python, the
-        }
+        # venv, the schema and the ACLs had all been changed.
+        $prevEap = $ErrorActionPreference
+        $ErrorActionPreference = 'Continue'
+        try {
+            $scoped = & $appcmd unlock config "$ourLocation" /section:$section 2>&1
+            $scoped | ForEach-Object { Write-Log "    $_" }
+            if ($LASTEXITCODE -ne 0) {
+                Write-Log "  scoped unlock refused; unlocking $section server-wide" 'WARN'
+                & $appcmd unlock config /section:$section 2>&1 | ForEach-Object { Write-Log "    $_" }
+                if ($LASTEXITCODE -ne 0) {
+                    Write-Log "  server-wide unlock also refused - IIS may return 500.19" 'WARN'
+                }
+            }
+        } finally { $ErrorActionPreference = $prevEap }
     }
 
     # Switching method must not leave BOTH deployments in place: two entry points

deploy/windows/installer/shopdb-preflight.ps1

@@ -439,7 +439,13 @@ Invoke-Check 'MySQL' 'Version and config' {
                 Add-Result 'MySQL' '5.6 index flags' 'WARN' 'all three present in my.ini' `
                     'Present in the file only. They take effect after a MySQL RESTART, which interrupts the classic ASP app. Confirm with SHOW VARIABLES or flask db-utils preflight.'
             } else {
-                Add-Result 'MySQL' '5.6 index flags' 'FAIL' ("missing: " + ($missing -join ', ')) `
+                # WARN, not FAIL. This inspects the LOCAL MySQL, which may not be the
+            # database the operator is about to install against - a bundled 8.4,
+            # or a remote server. Blocking the wizard here refused an install
+            # over a server that had nothing to do with it. Stage 3 runs
+            # 'flask db-utils preflight' against the database actually chosen,
+            # which is the check that can genuinely block.
+            Add-Result 'MySQL' '5.6 index flags' 'WARN' ("missing: " + ($missing -join ', ')) `
                     "Add to [mysqld] in $ini and restart MySQL, or 'flask db upgrade' fails with error 1071. NOTE: restarting interrupts the classic ASP app."
             }
         }