diff --git a/README.md b/README.md index 9dac23b..071da42 100644 --- a/README.md +++ b/README.md @@ -1,271 +1,272 @@ -# GE Aerospace PXE Boot Server - -Automated, air-gapped PXE boot server for deploying GE Aerospace Windows images. Built on Ubuntu 24.04 Server with zero-touch provisioning via autoinstall and Ansible. - -## Overview - -This project provides a complete, repeatable build process for a PXE boot server that serves Windows PE images to client machines on an isolated network. Everything runs offline after initial setup — no internet required on the target server. - -### Boot Chain - -``` -Client PXE boot (UEFI Secure Boot) - -> iPXE (TFTP, Broadcom-signed for Secure Boot) - -> iPXE boot menu (HTTP, port 4433) - -> User selects boot option: - ├── Windows PE -> wimboot -> boot.wim -> startnet.cmd -> Samba share -> Image deployment - ├── Clonezilla -> vmlinuz/initrd -> Disk cloning/imaging - ├── Blancco -> Native kernel + initramfs -> NIST 800-88 drive erasure - └── Memtest86+ -> Memory diagnostics -``` - -### Services - -| Service | Port | Purpose | -|-------------|-----------|------------------------------------------| -| dnsmasq | 67/udp | DHCP (172.16.9.10-100, 12h lease) | -| dnsmasq | 69/udp | TFTP (serves ipxe.efi) | -| Apache | 80/tcp | HTTP (wimboot, WinPE boot files, proxy) | -| Apache | 4433/tcp | iPXE boot script (GetPxeScript.aspx) | -| Samba | 445/tcp | Deployment content + Clonezilla + Blancco| -| Flask Webapp| 9009/tcp | Web management interface | - -### Network - -- **PXE server IP:** `172.16.9.1/24` -- **DHCP range:** `172.16.9.10` - `172.16.9.100` -- **Firewall:** UFW deny-by-default, only service ports open (22, 67, 69, 80, 445, 4433, 9009) - -## Quick Start - -### Prerequisites - -**On your workstation (internet-connected):** -- Ubuntu 24.04 (or Linux Mint / similar) for downloading packages -- Ubuntu Server 24.04 ISO -- GE Aerospace Media Creator LITE (for WinPE images) -- USB drive >= 8 GB (32+ GB if bundling WinPE images) - -**GE Access Packages (MyAccess portal):** -- EPM Rufus Exception Request -- EPM DT Functions -- DLP - Encrypted Removable (USB) Long Term Access - -### Step 1: Download Offline Packages - -```bash -./scripts/download-packages.sh -./scripts/fetch-ipxe.sh -``` - -`fetch-ipxe.sh` downloads the Secure Boot iPXE chain into `boot-files/`, pinned by sha256: the iPXE release -tarball (its snponly build becomes `ipxe.efi`) and iPXE's dual-signed shim (Microsoft UEFI CA 2011 + 2023), -which every UEFI client boots as `ipxe-shim.efi`. Keep the old Broadcom-signed `boot-files/ipxe.efi` too: the -playbook installs it as `ipxe-bcm.efi`, the rollback boot file. - -Downloads all .deb packages and Python wheels for offline installation (~140 MB of debs, ~20 MB of wheels). - -### Step 2: Prepare Boot Tools (optional) - -```bash -./scripts/prepare-boot-tools.sh /path/to/blancco.iso /path/to/clonezilla.zip /path/to/memtest.bin -``` - -Extracts and configures boot tool files (Blancco, Clonezilla, Memtest86+). Automatically patches Blancco's config.img to auto-save erasure reports to the PXE server's Samba share. - -### Step 3: Build the USB - -```bash -sudo ./scripts/build-usb.sh /dev/sdX /path/to/ubuntu-24.04-live-server-amd64.iso -``` - -Creates a bootable USB with two partitions: -- **Partition 1:** Ubuntu Server installer -- **Partition 2:** CIDATA (autoinstall config, offline .debs, pip wheels, Ansible playbook, webapp, boot tools) - -### Step 4: Install on Target Server - -1. Insert USB into the target machine -2. Press F12 and boot from USB -3. Ubuntu auto-installs with no interaction -4. After reboot, the first-boot script: - - Installs all offline .deb packages - - Runs the Ansible playbook (configures dnsmasq, Apache, Samba, UFW, webapp) - - Configures static IP `172.16.9.1/24` -5. Move the server's wired NIC to the isolated PXE switch - -### Step 5: Access the Web Interface - -Open `http://172.16.9.1:9009` from any machine on the isolated network. - -## Web Management Interface - -The Flask webapp (port 9009) provides a browser-based management UI: - -- **Dashboard** — Service status overview, disk usage, connected DHCP clients -- **Image Import** — Import WinPE deployment images from USB drives -- **Unattend Editor** — Edit Windows unattend.xml files per image type (XML syntax highlighting) -- **startnet.cmd Editor** — Modify the startnet.cmd inside boot.wim without Windows (uses wimtools) -- **Clonezilla Backups** — Upload, download, and manage disk backup images -- **Blancco Reports** — View, download, and manage drive erasure reports (auto-collected via Samba) -- **Image Config** — Per-image configuration editor (drivers, OS packages, hardware models) -- **Enrollment** — Upload, download, and manage GCCH bulk enrollment packages -- **Audit Log** — Activity history for all write operations (imports, edits, deletes) - -### Image Types Supported - -| Image Type | Domain | Description | -|----------------------|-----------------|-------------------------| -| gea-standard | geaerospace.com | Standard desktop | -| gea-engineer | geaerospace.com | Engineering desktop | -| gea-shopfloor | geaerospace.com | Shop floor kiosk | -| ge-standard | ge.com | Standard desktop | -| ge-engineer | ge.com | Engineering desktop | -| ge-shopfloor-lockdown| ge.com | Shop floor (locked) | -| ge-shopfloor-mce | ge.com | Shop floor (MCE) | - -## Project Structure - -``` -pxe-server/ -├── autoinstall/ -│ ├── user-data # Cloud-init autoinstall config + first-boot script -│ └── meta-data # Cloud-init metadata (required, empty) -├── playbook/ -│ ├── pxe_server_setup.yml # Ansible playbook: all server configuration -│ ├── inventory.ini # Ansible inventory -│ ├── startnet.cmd # WinPE startup script (injected into boot.wim) -│ ├── blancco-init.sh # Custom initramfs for Blancco PXE boot -│ ├── blancco-preferences.xml # Blancco auto-report preferences -│ ├── check-bios.cmd # Pre-imaging BIOS update checker -│ ├── FlatUnattendW10-shopfloor.xml # Shopfloor unattend.xml -│ └── shopfloor-setup/ # PC type setup scripts (Standard, CMM, Display, etc.) -├── webapp/ -│ ├── app.py # Flask application (~1600 lines) -│ ├── requirements.txt # Python deps (flask, lxml) -│ ├── static/ -│ │ ├── ge-aerospace-logo.svg # GE Aerospace branding -│ │ ├── favicon.ico # Browser favicon -│ │ ├── app.js # Frontend JavaScript -│ │ ├── bootstrap.min.css # Bootstrap 5 (bundled offline) -│ │ ├── bootstrap.bundle.min.js -│ │ ├── bootstrap-icons.min.css -│ │ └── fonts/ # Icon fonts (woff/woff2) -│ └── templates/ -│ ├── base.html # Layout with GE branding and sidebar nav -│ ├── dashboard.html # Service status and overview -│ ├── import.html # USB image import wizard -│ ├── unattend_editor.html # XML editor for unattend files -│ ├── startnet_editor.html # startnet.cmd WIM editor -│ ├── image_config.html # Per-image driver and hardware config -│ ├── enrollment.html # GCCH bulk enrollment packages -│ ├── backups.html # Clonezilla backup management -│ ├── reports.html # Blancco erasure reports -│ └── audit.html # Activity audit log -├── docs/ -│ └── shopfloor-display-imaging-guide.md # End-user imaging guide -├── boot-tools/ # Extracted boot tool files (gitignored) -│ ├── blancco/ # Blancco Drive Eraser -│ ├── clonezilla/ # Clonezilla Live -│ └── memtest/ # Memtest86+ -├── boot-files/ # WinPE boot files (boot.wim, wimboot, ipxe.efi, BCD) -├── offline-packages/ # .deb files (gitignored, built by download-packages.sh) -├── pip-wheels/ # Python wheels (gitignored, built by download-packages.sh) -├── enrollment/ # PPKGs and run-enrollment.ps1 (gitignored) -├── bios-staging/ # Dell BIOS update binaries (gitignored) -├── scripts/ # Build, deploy, and helper scripts -│ ├── build-usb.sh # Builds the installer USB (2-partition) -│ ├── build-proxmox-iso.sh # Builds self-contained Proxmox installer ISO -│ ├── prepare-boot-tools.sh # Extracts and patches boot tool files -│ ├── download-packages.sh # Downloads offline .debs + pip wheels -│ ├── download-drivers.py # Downloads Dell drivers directly from dell.com -│ ├── deploy-bios.sh # Pushes BIOS updates to enrollment share -│ ├── pull-bios.sh # Pulls BIOS binaries from upstream cache -│ ├── sync_hardware_models.py # Syncs hardware model configs across images -│ ├── Upload-Image.ps1 # Windows: upload MCL cache to PXE via SMB -│ └── Download-Drivers.ps1 # Windows: download hardware drivers from GE CDN -├── config/ # Site-specific configuration overrides -├── startnet-template.cmd # startnet.cmd template (synced with playbook copy) -├── README.md # This file -└── SETUP.md # Detailed setup guide -``` - -## Proxmox Deployment - -A single ISO can be built for deploying the PXE server in a Proxmox VM: - -### Build the ISO - -```bash -# Prerequisites (on build workstation) -sudo apt install xorriso p7zip-full - -# Build the installer ISO -./scripts/build-proxmox-iso.sh /path/to/ubuntu-24.04-live-server-amd64.iso -``` - -This creates `pxe-server-proxmox.iso` containing the Ubuntu installer, autoinstall config, all offline packages, the Ansible playbook, webapp, and boot tools. - -### Deploy on Proxmox - -1. Upload `pxe-server-proxmox.iso` to Proxmox storage (Datacenter -> Storage -> ISO Images) -2. Create a new VM: - - **OS:** Linux 6.x kernel - - **BIOS:** OVMF (UEFI) or SeaBIOS - - **Memory:** 4096 MB - - **CPU:** 2+ cores - - **Disk:** 40+ GB (VirtIO SCSI) - - **Network:** Bridge connected to your isolated PXE network -3. Attach the ISO as CD-ROM and start the VM -4. Ubuntu auto-installs with zero interaction (~10-15 minutes) -5. After reboot, first-boot configures all PXE services automatically -6. Access the web interface at `http://172.16.9.1:9009` - -### Import WinPE Images - -After the server is running, import deployment images via the web interface at `http://172.16.9.1:9009/import` or by mounting a USB drive with WinPE content. - -## Samba Shares - -| Share | Path | Purpose | -|-----------------|---------------------------|--------------------------------| -| winpeapps | /srv/samba/winpeapps | WinPE deployment images | -| clonezilla | /srv/samba/clonezilla | Clonezilla disk backup images | -| blancco-reports | /srv/samba/blancco-reports| Blancco erasure reports (auto) | -| enrollment | /srv/samba/enrollment | GCCH bulk enrollment packages | -| image-upload | /home/pxe/image-upload | Image upload staging area | - -All shares use guest access (no authentication) for ease of use on the isolated network. - -## Blancco Drive Erasure - -Blancco Drive Eraser 7.15.1 boots via a native Ubuntu kernel with a custom initramfs (`blancco-init.sh`) that downloads and mounts the Blancco rootfs over HTTP. XML erasure reports are automatically saved to the PXE server's Samba share (`blancco-reports`). The server supports BMC cloud licensing for Blancco activation over WiFi. - -Reports are viewable and downloadable from the web interface at `http://172.16.9.1:9009/reports`. - -## Notes - -- Run `download-packages.sh` before building USB — it downloads all offline `.deb` packages including wimtools (needed for startnet.cmd editing) -- The webapp uses session-based CSRF tokens on all POST forms and API endpoints - -## Commit History - -| Commit | Description | -|---------|--------------------------------------------------------------------| -| 5791bd1 | Initial project setup: automated PXE server provisioning | -| cee4ecd | Add web management UI, offline packages, WinPE consolidation | -| f614596 | Fix unattend.xml path to match actual image structure | -| e7313c2 | Add multi-boot PXE menu, Clonezilla backups, GE Aerospace branding| -| 89b5834 | Add wimtools and startnet.cmd editor for boot.wim modification | -| 05dbb7e | Add Blancco erasure reports Samba share and webapp viewer | -| ef75839 | Auto-patch Blancco config.img for network report storage | -| 92c9b0f | Fix review findings: offline assets, security, audit logging | -| 725c8f4 | Change webapp to port 9009, add test VM script | -| f3a384f | Add Proxmox ISO builder, CSRF protection, boot-files integration | -| 1a5c4f7 | Eliminate USB requirement for WinPE PXE boot | -| dd2fec5 | Blancco PXE boot via Ubuntu kernel switch_root | -| 86660a0 | Remove UEFI HTTP Boot config from dnsmasq | -| 6d0e6ee | BIOS check fix, parallel downloads, shopfloor hardening | -| 7616549 | Shopfloor PC type system, webapp enhancements | -| b7cd097 | Blancco 7.15.1 upgrade: native kernel boot, BMC cloud licensing | -| 163e58a | Fix dnsmasq reboot cron: use /etc/cron.d/ instead of crontab | +# GE Aerospace PXE Boot Server + +Automated, air-gapped PXE boot server for deploying GE Aerospace Windows images. Built on Ubuntu 24.04 Server with zero-touch provisioning via autoinstall and Ansible. + +## Overview + +This project provides a complete, repeatable build process for a PXE boot server that serves Windows PE images to client machines on an isolated network. Everything runs offline after initial setup — no internet required on the target server. + +### Boot Chain + +``` +Client PXE boot (UEFI Secure Boot) + -> iPXE (TFTP, Broadcom-signed for Secure Boot) + -> iPXE boot menu (HTTP, port 4433) + -> User selects boot option: + ├── Windows PE -> wimboot -> boot.wim -> startnet.cmd -> Samba share -> Image deployment + ├── Clonezilla -> vmlinuz/initrd -> Disk cloning/imaging + ├── Blancco -> Native kernel + initramfs -> NIST 800-88 drive erasure + └── Memtest86+ -> Memory diagnostics +``` + +### Services + +| Service | Port | Purpose | +|-------------|-----------|------------------------------------------| +| dnsmasq | 67/udp | DHCP (172.16.9.10-100, 12h lease) | +| dnsmasq | 69/udp | TFTP (serves ipxe.efi) | +| Apache | 80/tcp | HTTP (wimboot, WinPE boot files, proxy) | +| Apache | 4433/tcp | iPXE boot script (GetPxeScript.aspx) | +| Samba | 445/tcp | Deployment content + Clonezilla + Blancco| +| Flask Webapp| 9009/tcp | Web management interface | + +### Network + +- **PXE server IP:** `172.16.9.1/24` +- **DHCP range:** `172.16.9.10` - `172.16.9.100` +- **Firewall:** UFW deny-by-default, only service ports open (22, 67, 69, 80, 445, 4433, 9009) + +## Quick Start + +### Prerequisites + +**On your workstation (internet-connected):** +- Ubuntu 24.04 (or Linux Mint / similar) for downloading packages +- Ubuntu Server 24.04 ISO +- GE Aerospace Media Creator LITE (for WinPE images) +- USB drive >= 8 GB (32+ GB if bundling WinPE images) + +**GE Access Packages (MyAccess portal):** +- EPM Rufus Exception Request +- EPM DT Functions +- DLP - Encrypted Removable (USB) Long Term Access + +### Step 1: Download Offline Packages + +```bash +./scripts/download-packages.sh --iso /path/to/ubuntu-24.04.x-live-server-amd64.iso +./scripts/check-updates.py # expect: PASSED, 0 security updates +./scripts/fetch-ipxe.sh +``` + +`fetch-ipxe.sh` downloads the Secure Boot iPXE chain into `boot-files/`, pinned by sha256: the iPXE release +tarball (its snponly build becomes `ipxe.efi`) and iPXE's dual-signed shim (Microsoft UEFI CA 2011 + 2023), +which every UEFI client boots as `ipxe-shim.efi`. Keep the old Broadcom-signed `boot-files/ipxe.efi` too: the +playbook installs it as `ipxe-bcm.efi`, the rollback boot file. + +Downloads all .deb packages and Python wheels for offline installation (~140 MB of debs, ~20 MB of wheels). + +### Step 2: Prepare Boot Tools (optional) + +```bash +./scripts/prepare-boot-tools.sh /path/to/blancco.iso /path/to/clonezilla.zip /path/to/memtest.bin +``` + +Extracts and configures boot tool files (Blancco, Clonezilla, Memtest86+). Automatically patches Blancco's config.img to auto-save erasure reports to the PXE server's Samba share. + +### Step 3: Build the USB + +```bash +sudo ./scripts/build-usb.sh /dev/sdX /path/to/ubuntu-24.04-live-server-amd64.iso +``` + +Creates a bootable USB with two partitions: +- **Partition 1:** Ubuntu Server installer +- **Partition 2:** CIDATA (autoinstall config, offline .debs, pip wheels, Ansible playbook, webapp, boot tools) + +### Step 4: Install on Target Server + +1. Insert USB into the target machine +2. Press F12 and boot from USB +3. Ubuntu auto-installs with no interaction +4. After reboot, the first-boot script: + - Installs all offline .deb packages + - Runs the Ansible playbook (configures dnsmasq, Apache, Samba, UFW, webapp) + - Configures static IP `172.16.9.1/24` +5. Move the server's wired NIC to the isolated PXE switch + +### Step 5: Access the Web Interface + +Open `http://172.16.9.1:9009` from any machine on the isolated network. + +## Web Management Interface + +The Flask webapp (port 9009) provides a browser-based management UI: + +- **Dashboard** — Service status overview, disk usage, connected DHCP clients +- **Image Import** — Import WinPE deployment images from USB drives +- **Unattend Editor** — Edit Windows unattend.xml files per image type (XML syntax highlighting) +- **startnet.cmd Editor** — Modify the startnet.cmd inside boot.wim without Windows (uses wimtools) +- **Clonezilla Backups** — Upload, download, and manage disk backup images +- **Blancco Reports** — View, download, and manage drive erasure reports (auto-collected via Samba) +- **Image Config** — Per-image configuration editor (drivers, OS packages, hardware models) +- **Enrollment** — Upload, download, and manage GCCH bulk enrollment packages +- **Audit Log** — Activity history for all write operations (imports, edits, deletes) + +### Image Types Supported + +| Image Type | Domain | Description | +|----------------------|-----------------|-------------------------| +| gea-standard | geaerospace.com | Standard desktop | +| gea-engineer | geaerospace.com | Engineering desktop | +| gea-shopfloor | geaerospace.com | Shop floor kiosk | +| ge-standard | ge.com | Standard desktop | +| ge-engineer | ge.com | Engineering desktop | +| ge-shopfloor-lockdown| ge.com | Shop floor (locked) | +| ge-shopfloor-mce | ge.com | Shop floor (MCE) | + +## Project Structure + +``` +pxe-server/ +├── autoinstall/ +│ ├── user-data # Cloud-init autoinstall config + first-boot script +│ └── meta-data # Cloud-init metadata (required, empty) +├── playbook/ +│ ├── pxe_server_setup.yml # Ansible playbook: all server configuration +│ ├── inventory.ini # Ansible inventory +│ ├── startnet.cmd # WinPE startup script (injected into boot.wim) +│ ├── blancco-init.sh # Custom initramfs for Blancco PXE boot +│ ├── blancco-preferences.xml # Blancco auto-report preferences +│ ├── check-bios.cmd # Pre-imaging BIOS update checker +│ ├── FlatUnattendW10-shopfloor.xml # Shopfloor unattend.xml +│ └── shopfloor-setup/ # PC type setup scripts (Standard, CMM, Display, etc.) +├── webapp/ +│ ├── app.py # Flask application (~1600 lines) +│ ├── requirements.txt # Python deps (flask, lxml) +│ ├── static/ +│ │ ├── ge-aerospace-logo.svg # GE Aerospace branding +│ │ ├── favicon.ico # Browser favicon +│ │ ├── app.js # Frontend JavaScript +│ │ ├── bootstrap.min.css # Bootstrap 5 (bundled offline) +│ │ ├── bootstrap.bundle.min.js +│ │ ├── bootstrap-icons.min.css +│ │ └── fonts/ # Icon fonts (woff/woff2) +│ └── templates/ +│ ├── base.html # Layout with GE branding and sidebar nav +│ ├── dashboard.html # Service status and overview +│ ├── import.html # USB image import wizard +│ ├── unattend_editor.html # XML editor for unattend files +│ ├── startnet_editor.html # startnet.cmd WIM editor +│ ├── image_config.html # Per-image driver and hardware config +│ ├── enrollment.html # GCCH bulk enrollment packages +│ ├── backups.html # Clonezilla backup management +│ ├── reports.html # Blancco erasure reports +│ └── audit.html # Activity audit log +├── docs/ +│ └── shopfloor-display-imaging-guide.md # End-user imaging guide +├── boot-tools/ # Extracted boot tool files (gitignored) +│ ├── blancco/ # Blancco Drive Eraser +│ ├── clonezilla/ # Clonezilla Live +│ └── memtest/ # Memtest86+ +├── boot-files/ # WinPE boot files (boot.wim, wimboot, ipxe.efi, BCD) +├── offline-packages/ # .deb files (gitignored, built by download-packages.sh) +├── pip-wheels/ # Python wheels (gitignored, built by download-packages.sh) +├── enrollment/ # PPKGs and run-enrollment.ps1 (gitignored) +├── bios-staging/ # Dell BIOS update binaries (gitignored) +├── scripts/ # Build, deploy, and helper scripts +│ ├── build-usb.sh # Builds the installer USB (2-partition) +│ ├── build-proxmox-iso.sh # Builds self-contained Proxmox installer ISO +│ ├── prepare-boot-tools.sh # Extracts and patches boot tool files +│ ├── download-packages.sh # Downloads offline .debs + pip wheels +│ ├── download-drivers.py # Downloads Dell drivers directly from dell.com +│ ├── deploy-bios.sh # Pushes BIOS updates to enrollment share +│ ├── pull-bios.sh # Pulls BIOS binaries from upstream cache +│ ├── sync_hardware_models.py # Syncs hardware model configs across images +│ ├── Upload-Image.ps1 # Windows: upload MCL cache to PXE via SMB +│ └── Download-Drivers.ps1 # Windows: download hardware drivers from GE CDN +├── config/ # Site-specific configuration overrides +├── startnet-template.cmd # startnet.cmd template (synced with playbook copy) +├── README.md # This file +└── SETUP.md # Detailed setup guide +``` + +## Proxmox Deployment + +A single ISO can be built for deploying the PXE server in a Proxmox VM: + +### Build the ISO + +```bash +# Prerequisites (on build workstation) +sudo apt install xorriso p7zip-full + +# Build the installer ISO +./scripts/build-proxmox-iso.sh /path/to/ubuntu-24.04-live-server-amd64.iso +``` + +This creates `pxe-server-proxmox.iso` containing the Ubuntu installer, autoinstall config, all offline packages, the Ansible playbook, webapp, and boot tools. + +### Deploy on Proxmox + +1. Upload `pxe-server-proxmox.iso` to Proxmox storage (Datacenter -> Storage -> ISO Images) +2. Create a new VM: + - **OS:** Linux 6.x kernel + - **BIOS:** OVMF (UEFI) or SeaBIOS + - **Memory:** 4096 MB + - **CPU:** 2+ cores + - **Disk:** 40+ GB (VirtIO SCSI) + - **Network:** Bridge connected to your isolated PXE network +3. Attach the ISO as CD-ROM and start the VM +4. Ubuntu auto-installs with zero interaction (~10-15 minutes) +5. After reboot, first-boot configures all PXE services automatically +6. Access the web interface at `http://172.16.9.1:9009` + +### Import WinPE Images + +After the server is running, import deployment images via the web interface at `http://172.16.9.1:9009/import` or by mounting a USB drive with WinPE content. + +## Samba Shares + +| Share | Path | Purpose | +|-----------------|---------------------------|--------------------------------| +| winpeapps | /srv/samba/winpeapps | WinPE deployment images | +| clonezilla | /srv/samba/clonezilla | Clonezilla disk backup images | +| blancco-reports | /srv/samba/blancco-reports| Blancco erasure reports (auto) | +| enrollment | /srv/samba/enrollment | GCCH bulk enrollment packages | +| image-upload | /home/pxe/image-upload | Image upload staging area | + +All shares use guest access (no authentication) for ease of use on the isolated network. + +## Blancco Drive Erasure + +Blancco Drive Eraser 7.15.1 boots via a native Ubuntu kernel with a custom initramfs (`blancco-init.sh`) that downloads and mounts the Blancco rootfs over HTTP. XML erasure reports are automatically saved to the PXE server's Samba share (`blancco-reports`). The server supports BMC cloud licensing for Blancco activation over WiFi. + +Reports are viewable and downloadable from the web interface at `http://172.16.9.1:9009/reports`. + +## Notes + +- Run `download-packages.sh --iso ` before building the USB: it builds the offline apt repo (every update for the ISO's base plus the explicit packages, wimtools included) that the installer patches and installs from +- The webapp uses session-based CSRF tokens on all POST forms and API endpoints + +## Commit History + +| Commit | Description | +|---------|--------------------------------------------------------------------| +| 5791bd1 | Initial project setup: automated PXE server provisioning | +| cee4ecd | Add web management UI, offline packages, WinPE consolidation | +| f614596 | Fix unattend.xml path to match actual image structure | +| e7313c2 | Add multi-boot PXE menu, Clonezilla backups, GE Aerospace branding| +| 89b5834 | Add wimtools and startnet.cmd editor for boot.wim modification | +| 05dbb7e | Add Blancco erasure reports Samba share and webapp viewer | +| ef75839 | Auto-patch Blancco config.img for network report storage | +| 92c9b0f | Fix review findings: offline assets, security, audit logging | +| 725c8f4 | Change webapp to port 9009, add test VM script | +| f3a384f | Add Proxmox ISO builder, CSRF protection, boot-files integration | +| 1a5c4f7 | Eliminate USB requirement for WinPE PXE boot | +| dd2fec5 | Blancco PXE boot via Ubuntu kernel switch_root | +| 86660a0 | Remove UEFI HTTP Boot config from dnsmasq | +| 6d0e6ee | BIOS check fix, parallel downloads, shopfloor hardening | +| 7616549 | Shopfloor PC type system, webapp enhancements | +| b7cd097 | Blancco 7.15.1 upgrade: native kernel boot, BMC cloud licensing | +| 163e58a | Fix dnsmasq reboot cron: use /etc/cron.d/ instead of crontab | diff --git a/SETUP.md b/SETUP.md index c15f656..11963fb 100644 --- a/SETUP.md +++ b/SETUP.md @@ -1,251 +1,265 @@ -# PXE Server Setup Guide - -Automated build process for deploying an Ubuntu-based PXE boot server that hosts GE Aerospace Windows PE images. The entire setup is air-gapped — no internet required on the target machine. - -## Architecture - -``` -Client PXE boot - -> Broadcom signed iPXE (Secure Boot) - -> iPXE boot menu (HTTP, port 4433) - ├── Windows PE -> wimboot -> boot.wim -> startnet.cmd -> Samba -> Image deployment - ├── Clonezilla -> vmlinuz/initrd -> Disk cloning/imaging - ├── Blancco -> Native kernel + initramfs -> NIST 800-88 drive erasure (auto-reports) - └── Memtest86+ -> Memory diagnostics -``` - -### Services on the PXE Server - -| Service | Port | Purpose | -|-------------|-----------|------------------------------------------| -| dnsmasq | 67/udp | DHCP (172.16.9.10-100) | -| dnsmasq | 69/udp | TFTP (serves ipxe.efi) | -| Apache | 80/tcp | HTTP (wimboot, WinPE boot files, proxy) | -| Apache | 4433/tcp | iPXE boot script (GetPxeScript.aspx) | -| Samba | 445/tcp | Deployment content + backup + reports | -| Flask Webapp| 9009/tcp | Web management interface | - -## Prerequisites - -### Hardware -- Server or PC with >= 8 GB RAM, >= 250 GB disk, one wired NIC -- USB thumb drive >= 8 GB (32+ GB if bundling WinPE images) - -### Software (on your workstation) -- Ubuntu Server 24.04 ISO — https://ubuntu.com/download/server -- Linux workstation (Ubuntu/Mint) for running download and build scripts -- GE Aerospace Media Creator LITE (for caching WinPE images) - -### GE Access Packages -- EPM Rufus Exception Request -- EPM DT Functions -- DLP - Encrypted Removable (USB) Long Term Access - -## Setup Process - -### Step 1: Download Offline Packages (one-time, requires internet) - -```bash -./scripts/download-packages.sh -./scripts/fetch-ipxe.sh -``` - -`fetch-ipxe.sh` downloads the Secure Boot iPXE chain into `boot-files/`, pinned by sha256: the iPXE release -tarball (its snponly build becomes `ipxe.efi`) and iPXE's dual-signed shim (Microsoft UEFI CA 2011 + 2023), -which every UEFI client boots as `ipxe-shim.efi`. Keep the old Broadcom-signed `boot-files/ipxe.efi` too: the -playbook installs it as `ipxe-bcm.efi`, the rollback boot file. - -Downloads all .deb packages (ansible, dnsmasq, apache2, samba, wimtools, etc.) into `offline-packages/` and Python wheels (flask, lxml) into `pip-wheels/`. Approximately 252 packages (~140 MB) + 8 Python wheels. - -**Packages included:** -- **Server:** dnsmasq, apache2, samba, ufw, cron -- **Automation:** ansible -- **Tools:** wimtools, unzip, p7zip-full -- **Python:** python3-pip, python3-venv -- **Network:** network-manager, wpasupplicant, wireless-tools, linux-firmware - -### Step 2: Prepare Boot Tools (optional) - -```bash -./scripts/prepare-boot-tools.sh /path/to/blancco.iso /path/to/clonezilla.zip /path/to/memtest.bin -``` - -Extracts boot files for Blancco, Clonezilla, and Memtest86+ into the `boot-tools/` directory. Automatically patches Blancco's `config.img` to auto-save erasure reports to the PXE server's Samba share. - -### Step 3: Build the USB - -```bash -# Basic — server only (import WinPE images later) -sudo ./scripts/build-usb.sh /dev/sdX /path/to/ubuntu-24.04-live-server-amd64.iso - -# With WinPE images bundled (single USB, larger drive needed) -sudo ./scripts/build-usb.sh /dev/sdX /path/to/ubuntu-24.04.iso /path/to/winpe-images - -# Same layout into a disk-image file, no root (dd it to a stick later, or -# boot it in a VM: see tests/vm/README.md). SIZE is e.g. 16G or "auto". -./scripts/build-usb.sh --image pxe-usb.img auto /path/to/ubuntu-24.04.iso -``` - -Options: `--payload-root DIR` takes offline-packages/, pip-wheels/, -boot-files/, boot-tools/, enrollment/, bios-staging/ and drivers-staging/ -from another checkout; `--skip-enrollment`, `--skip-bios`, -`--skip-drivers`, `--skip-boot-tools` leave those out; `--yes` skips the -prompt. The ISO volume label and El Torito load sizes are read from the -ISO, so any 24.04.x point release works. - -This creates a bootable USB with: -- **Partition 1:** Ubuntu Server installer -- **Partition 2:** CIDATA (autoinstall config, offline .debs, pip wheels, Ansible playbook, webapp, boot tools) - -### Step 4: Boot the Target Machine - -1. Insert the USB into the target machine -2. Press F12 (or vendor boot key) and select the USB -3. Ubuntu auto-installs — no interaction needed -4. After reboot, the first-boot script installs all .deb packages and runs the Ansible playbook -5. PXE services (dnsmasq, Apache, Samba) are configured automatically -6. Flask webapp starts on port 9009 - -### Step 5: Connect to Isolated Network - -Move the server's wired NIC to the isolated switch for PXE clients. - -### Step 6: Import WinPE Content (if not bundled in Step 3) - -**Option A:** Use the web interface at `http://172.16.9.1:9009` to import from USB. - -**Option B:** Manual copy: -```bash -sudo mkdir -p /mnt/usb2 -sudo mount /dev/sdb2 /mnt/usb2 -sudo cp -r /mnt/usb2/. /srv/samba/winpeapps/gea-standard -sudo umount /mnt/usb2 -``` - -## Web Management Interface - -Access at `http://172.16.9.1:9009` from any machine on the isolated network. - -| Page | URL Path | Purpose | -|-------------------|-------------|-----------------------------------------------| -| Dashboard | / | Service status, disk usage, DHCP clients | -| Image Import | /import | Import WinPE images from USB drives | -| Unattend Editor | /unattend | Edit Windows unattend.xml per image type | -| startnet.cmd | /startnet | Edit startnet.cmd inside boot.wim (wimtools) | -| Clonezilla Backups| /backups | Upload/download/manage disk backup images | -| Blancco Reports | /reports | View/download drive erasure reports | -| Image Config | /images/*/config | Per-image driver, OS, and hardware config | -| Enrollment | /enrollment | GCCH bulk enrollment package management | -| Audit Log | /audit | Activity history for all write operations | - -## Verification - -1. Connect a test workstation to the isolated switch -2. Set Network Boot (PXE) as first boot in BIOS/UEFI -3. Boot — the client should pull an IP from 10.9.100.x -4. iPXE loads, fetches the boot script from port 4433 -5. Select an option from the boot menu: - - **Windows PE**: Boots via wimboot + boot.wim, maps Samba shares, begins deployment - - **Clonezilla**: Boots Clonezilla Live for disk imaging - - **Blancco**: Boots Drive Eraser, auto-saves reports to server - - **Memtest86+**: Runs memory diagnostics - -## Testing with KVM - -```bash -# Download Ubuntu ISO -wget -O ~/Downloads/ubuntu-24.04.3-live-server-amd64.iso \ - https://releases.ubuntu.com/noble/ubuntu-24.04.3-live-server-amd64.iso - -# Launch test VM -sudo ./test-vm.sh ~/Downloads/ubuntu-24.04.3-live-server-amd64.iso - -# Watch progress (Ctrl+] to detach) -sudo virsh console pxe-test - -# After install: ssh pxe@172.16.9.1 / http://172.16.9.1:9009 - -# Clean up -sudo ./test-vm.sh --destroy -``` - -## Project Structure - -``` -pxe-server/ -├── autoinstall/ -│ ├── user-data # Cloud-init autoinstall + first-boot script -│ └── meta-data # Cloud-init metadata (required, empty) -├── playbook/ -│ ├── pxe_server_setup.yml # Ansible: dnsmasq, Apache, Samba, iPXE, UFW, webapp -│ ├── inventory.ini # Ansible inventory -│ ├── startnet.cmd # WinPE startup script (injected into boot.wim) -│ ├── blancco-init.sh # Custom initramfs for Blancco PXE boot -│ ├── blancco-preferences.xml # Blancco auto-report preferences -│ ├── check-bios.cmd # Pre-imaging BIOS update checker -│ ├── FlatUnattendW10-shopfloor.xml # Shopfloor unattend.xml -│ └── shopfloor-setup/ # PC type setup scripts (Standard, CMM, Display, etc.) -├── webapp/ -│ ├── app.py # Flask application (~1600 lines) -│ ├── requirements.txt # Python deps (flask, lxml) -│ ├── static/ # CSS, JS, fonts, logo (all bundled offline) -│ └── templates/ # Jinja2 HTML templates (10 pages) -├── docs/ -│ └── shopfloor-display-imaging-guide.md # End-user imaging guide -├── boot-tools/ # Extracted boot files (gitignored, built by prepare-boot-tools.sh) -│ ├── blancco/ # Blancco Drive Eraser -│ ├── clonezilla/ # Clonezilla Live -│ └── memtest/ # Memtest86+ -├── boot-files/ # WinPE boot files (boot.wim, wimboot, ipxe.efi, BCD) -├── offline-packages/ # .deb files (gitignored, built by download-packages.sh) -├── pip-wheels/ # Python wheels (gitignored, built by download-packages.sh) -├── enrollment/ # PPKGs and run-enrollment.ps1 (gitignored) -├── bios-staging/ # Dell BIOS update binaries (gitignored) -├── scripts/ # Build, deploy, and helper scripts -│ ├── build-usb.sh # Builds the 2-partition installer USB -│ ├── build-proxmox-iso.sh # Builds self-contained Proxmox installer ISO -│ ├── prepare-boot-tools.sh # Extracts/patches boot tools from ISOs -│ ├── download-packages.sh # Downloads all offline packages -│ ├── download-drivers.py # Downloads Dell drivers from dell.com -│ ├── deploy-bios.sh # Pushes BIOS updates to enrollment share -│ ├── pull-bios.sh # Pulls BIOS binaries from upstream cache -│ ├── sync_hardware_models.py # Syncs hardware model configs across images -│ ├── Upload-Image.ps1 # Windows: upload MCL cache to PXE via SMB -│ └── Download-Drivers.ps1 # Windows: download hardware drivers from GE CDN -├── config/ # Site-specific configuration overrides -├── startnet-template.cmd # startnet.cmd template (synced with playbook copy) -├── README.md # Project overview -└── SETUP.md # Detailed setup guide (this file) -``` - -## Image Types - -| Image Type | Domain | Description | -|----------------------|-----------------|-------------------------| -| gea-standard | geaerospace.com | Standard desktop | -| gea-engineer | geaerospace.com | Engineering desktop | -| gea-shopfloor | geaerospace.com | Shop floor kiosk | -| ge-standard | ge.com | Standard desktop | -| ge-engineer | ge.com | Engineering desktop | -| ge-shopfloor-lockdown| ge.com | Shop floor (locked) | -| ge-shopfloor-mce | ge.com | Shop floor (MCE) | - -## Network Configuration - -- PXE server static IP: `172.16.9.1/24` -- DHCP range: `172.16.9.10` - `172.16.9.100` -- Lease time: 12 hours -- DNS: `8.8.8.8` (passed to clients, not used by server) -- Firewall: UFW deny-by-default, allow 67/udp 69/udp 80/tcp 445/tcp 4433/tcp 9009/tcp - -## Samba Shares - -| Share | Path | Purpose | -|-----------------|----------------------------|-------------------------------| -| winpeapps | /srv/samba/winpeapps | WinPE deployment images | -| clonezilla | /srv/samba/clonezilla | Clonezilla disk backup images | -| blancco-reports | /srv/samba/blancco-reports | Blancco erasure reports (auto)| -| enrollment | /srv/samba/enrollment | GCCH bulk enrollment packages | -| image-upload | /home/pxe/image-upload | Image upload staging area | - -All shares use guest access for the isolated network. +# PXE Server Setup Guide + +Automated build process for deploying an Ubuntu-based PXE boot server that hosts GE Aerospace Windows PE images. The entire setup is air-gapped — no internet required on the target machine. + +## Architecture + +``` +Client PXE boot + -> Broadcom signed iPXE (Secure Boot) + -> iPXE boot menu (HTTP, port 4433) + ├── Windows PE -> wimboot -> boot.wim -> startnet.cmd -> Samba -> Image deployment + ├── Clonezilla -> vmlinuz/initrd -> Disk cloning/imaging + ├── Blancco -> Native kernel + initramfs -> NIST 800-88 drive erasure (auto-reports) + └── Memtest86+ -> Memory diagnostics +``` + +### Services on the PXE Server + +| Service | Port | Purpose | +|-------------|-----------|------------------------------------------| +| dnsmasq | 67/udp | DHCP (172.16.9.10-100) | +| dnsmasq | 69/udp | TFTP (serves ipxe.efi) | +| Apache | 80/tcp | HTTP (wimboot, WinPE boot files, proxy) | +| Apache | 4433/tcp | iPXE boot script (GetPxeScript.aspx) | +| Samba | 445/tcp | Deployment content + backup + reports | +| Flask Webapp| 9009/tcp | Web management interface | + +## Prerequisites + +### Hardware +- Server or PC with >= 8 GB RAM, >= 250 GB disk, one wired NIC +- USB thumb drive >= 8 GB (32+ GB if bundling WinPE images) + +### Software (on your workstation) +- Ubuntu Server 24.04 ISO — https://ubuntu.com/download/server +- Linux workstation (Ubuntu/Mint) for running download and build scripts +- GE Aerospace Media Creator LITE (for caching WinPE images) + +### GE Access Packages +- EPM Rufus Exception Request +- EPM DT Functions +- DLP - Encrypted Removable (USB) Long Term Access + +## Setup Process + +### Step 1: Download Offline Packages (one-time, requires internet) + +```bash +./scripts/download-packages.sh --iso /path/to/ubuntu-24.04.x-live-server-amd64.iso +./scripts/check-updates.py # expect: PASSED, 0 security updates +./scripts/fetch-ipxe.sh +``` + +`fetch-ipxe.sh` downloads the Secure Boot iPXE chain into `boot-files/`, pinned by sha256: the iPXE release +tarball (its snponly build becomes `ipxe.efi`) and iPXE's dual-signed shim (Microsoft UEFI CA 2011 + 2023), +which every UEFI client boots as `ipxe-shim.efi`. Keep the old Broadcom-signed `boot-files/ipxe.efi` too: the +playbook installs it as `ipxe-bcm.efi`, the rollback boot file. + +`download-packages.sh` needs docker (no sudo) and the SAME server ISO that +`build-usb.sh` will use. The install is offline (autoinstall uses a dead +mirror on purpose), so the server's base is exactly that ISO's; the script +unpacks the ISO's install image into a docker image and resolves there: +every update of an installed package, the explicit package list +(ansible, dnsmasq, apache2, samba, wimtools, zip, the HWE kernel, ...) and +the newest version of what the installer adds from the ISO's pool (kernel, +grub, openssh-server). `offline-packages/` becomes an apt repository +(debs + `Packages` + `explicit.list`), checked offline in a fresh container +of the base before the script finishes. On the server, autoinstall's +late-commands run `scripts/offline-apt-install.sh` on it: one apt +transaction from the local repo only, any error fails the install (log: +`/var/log/offline-apt-install.log`). The webapp's pinned, hashed wheels go +to `pip-wheels/`. About 500 debs (~1.6 GB) + 8 wheels. + +**Packages included:** +- **Server:** dnsmasq, apache2, samba, ufw, cron +- **Automation:** ansible +- **Tools:** wimtools, unzip, p7zip-full +- **Python:** python3-pip, python3-venv +- **Network:** network-manager, wpasupplicant, wireless-tools, linux-firmware + +### Step 2: Prepare Boot Tools (optional) + +```bash +./scripts/prepare-boot-tools.sh /path/to/blancco.iso /path/to/clonezilla.zip /path/to/memtest.bin +``` + +Extracts boot files for Blancco, Clonezilla, and Memtest86+ into the `boot-tools/` directory. Automatically patches Blancco's `config.img` to auto-save erasure reports to the PXE server's Samba share. + +### Step 3: Build the USB + +```bash +# Basic — server only (import WinPE images later) +sudo ./scripts/build-usb.sh /dev/sdX /path/to/ubuntu-24.04-live-server-amd64.iso + +# With WinPE images bundled (single USB, larger drive needed) +sudo ./scripts/build-usb.sh /dev/sdX /path/to/ubuntu-24.04.iso /path/to/winpe-images + +# Same layout into a disk-image file, no root (dd it to a stick later, or +# boot it in a VM: see tests/vm/README.md). SIZE is e.g. 16G or "auto". +./scripts/build-usb.sh --image pxe-usb.img auto /path/to/ubuntu-24.04.iso +``` + +Options: `--payload-root DIR` takes offline-packages/, pip-wheels/, +boot-files/, boot-tools/, enrollment/, bios-staging/ and drivers-staging/ +from another checkout; `--skip-enrollment`, `--skip-bios`, +`--skip-drivers`, `--skip-boot-tools` leave those out; `--yes` skips the +prompt. The ISO volume label and El Torito load sizes are read from the +ISO, so any 24.04.x point release works. + +This creates a bootable USB with: +- **Partition 1:** Ubuntu Server installer +- **Partition 2:** CIDATA (autoinstall config, offline .debs, pip wheels, Ansible playbook, webapp, boot tools) + +### Step 4: Boot the Target Machine + +1. Insert the USB into the target machine +2. Press F12 (or vendor boot key) and select the USB +3. Ubuntu auto-installs — no interaction needed +4. After reboot, the first-boot script installs all .deb packages and runs the Ansible playbook +5. PXE services (dnsmasq, Apache, Samba) are configured automatically +6. Flask webapp starts on port 9009 + +### Step 5: Connect to Isolated Network + +Move the server's wired NIC to the isolated switch for PXE clients. + +### Step 6: Import WinPE Content (if not bundled in Step 3) + +**Option A:** Use the web interface at `http://172.16.9.1:9009` to import from USB. + +**Option B:** Manual copy: +```bash +sudo mkdir -p /mnt/usb2 +sudo mount /dev/sdb2 /mnt/usb2 +sudo cp -r /mnt/usb2/. /srv/samba/winpeapps/gea-standard +sudo umount /mnt/usb2 +``` + +## Web Management Interface + +Access at `http://172.16.9.1:9009` from any machine on the isolated network. + +| Page | URL Path | Purpose | +|-------------------|-------------|-----------------------------------------------| +| Dashboard | / | Service status, disk usage, DHCP clients | +| Image Import | /import | Import WinPE images from USB drives | +| Unattend Editor | /unattend | Edit Windows unattend.xml per image type | +| startnet.cmd | /startnet | Edit startnet.cmd inside boot.wim (wimtools) | +| Clonezilla Backups| /backups | Upload/download/manage disk backup images | +| Blancco Reports | /reports | View/download drive erasure reports | +| Image Config | /images/*/config | Per-image driver, OS, and hardware config | +| Enrollment | /enrollment | GCCH bulk enrollment package management | +| Audit Log | /audit | Activity history for all write operations | + +## Verification + +1. Connect a test workstation to the isolated switch +2. Set Network Boot (PXE) as first boot in BIOS/UEFI +3. Boot — the client should pull an IP from 10.9.100.x +4. iPXE loads, fetches the boot script from port 4433 +5. Select an option from the boot menu: + - **Windows PE**: Boots via wimboot + boot.wim, maps Samba shares, begins deployment + - **Clonezilla**: Boots Clonezilla Live for disk imaging + - **Blancco**: Boots Drive Eraser, auto-saves reports to server + - **Memtest86+**: Runs memory diagnostics + +## Testing with KVM + +```bash +# Download Ubuntu ISO +wget -O ~/Downloads/ubuntu-24.04.3-live-server-amd64.iso \ + https://releases.ubuntu.com/noble/ubuntu-24.04.3-live-server-amd64.iso + +# Launch test VM +sudo ./test-vm.sh ~/Downloads/ubuntu-24.04.3-live-server-amd64.iso + +# Watch progress (Ctrl+] to detach) +sudo virsh console pxe-test + +# After install: ssh pxe@172.16.9.1 / http://172.16.9.1:9009 + +# Clean up +sudo ./test-vm.sh --destroy +``` + +## Project Structure + +``` +pxe-server/ +├── autoinstall/ +│ ├── user-data # Cloud-init autoinstall + first-boot script +│ └── meta-data # Cloud-init metadata (required, empty) +├── playbook/ +│ ├── pxe_server_setup.yml # Ansible: dnsmasq, Apache, Samba, iPXE, UFW, webapp +│ ├── inventory.ini # Ansible inventory +│ ├── startnet.cmd # WinPE startup script (injected into boot.wim) +│ ├── blancco-init.sh # Custom initramfs for Blancco PXE boot +│ ├── blancco-preferences.xml # Blancco auto-report preferences +│ ├── check-bios.cmd # Pre-imaging BIOS update checker +│ ├── FlatUnattendW10-shopfloor.xml # Shopfloor unattend.xml +│ └── shopfloor-setup/ # PC type setup scripts (Standard, CMM, Display, etc.) +├── webapp/ +│ ├── app.py # Flask application (~1600 lines) +│ ├── requirements.txt # Python deps (flask, lxml) +│ ├── static/ # CSS, JS, fonts, logo (all bundled offline) +│ └── templates/ # Jinja2 HTML templates (10 pages) +├── docs/ +│ └── shopfloor-display-imaging-guide.md # End-user imaging guide +├── boot-tools/ # Extracted boot files (gitignored, built by prepare-boot-tools.sh) +│ ├── blancco/ # Blancco Drive Eraser +│ ├── clonezilla/ # Clonezilla Live +│ └── memtest/ # Memtest86+ +├── boot-files/ # WinPE boot files (boot.wim, wimboot, ipxe.efi, BCD) +├── offline-packages/ # .deb files (gitignored, built by download-packages.sh) +├── pip-wheels/ # Python wheels (gitignored, built by download-packages.sh) +├── enrollment/ # PPKGs and run-enrollment.ps1 (gitignored) +├── bios-staging/ # Dell BIOS update binaries (gitignored) +├── scripts/ # Build, deploy, and helper scripts +│ ├── build-usb.sh # Builds the 2-partition installer USB +│ ├── build-proxmox-iso.sh # Builds self-contained Proxmox installer ISO +│ ├── prepare-boot-tools.sh # Extracts/patches boot tools from ISOs +│ ├── download-packages.sh # Downloads all offline packages +│ ├── download-drivers.py # Downloads Dell drivers from dell.com +│ ├── deploy-bios.sh # Pushes BIOS updates to enrollment share +│ ├── pull-bios.sh # Pulls BIOS binaries from upstream cache +│ ├── sync_hardware_models.py # Syncs hardware model configs across images +│ ├── Upload-Image.ps1 # Windows: upload MCL cache to PXE via SMB +│ └── Download-Drivers.ps1 # Windows: download hardware drivers from GE CDN +├── config/ # Site-specific configuration overrides +├── startnet-template.cmd # startnet.cmd template (synced with playbook copy) +├── README.md # Project overview +└── SETUP.md # Detailed setup guide (this file) +``` + +## Image Types + +| Image Type | Domain | Description | +|----------------------|-----------------|-------------------------| +| gea-standard | geaerospace.com | Standard desktop | +| gea-engineer | geaerospace.com | Engineering desktop | +| gea-shopfloor | geaerospace.com | Shop floor kiosk | +| ge-standard | ge.com | Standard desktop | +| ge-engineer | ge.com | Engineering desktop | +| ge-shopfloor-lockdown| ge.com | Shop floor (locked) | +| ge-shopfloor-mce | ge.com | Shop floor (MCE) | + +## Network Configuration + +- PXE server static IP: `172.16.9.1/24` +- DHCP range: `172.16.9.10` - `172.16.9.100` +- Lease time: 12 hours +- DNS: `8.8.8.8` (passed to clients, not used by server) +- Firewall: UFW deny-by-default, allow 67/udp 69/udp 80/tcp 445/tcp 4433/tcp 9009/tcp + +## Samba Shares + +| Share | Path | Purpose | +|-----------------|----------------------------|-------------------------------| +| winpeapps | /srv/samba/winpeapps | WinPE deployment images | +| clonezilla | /srv/samba/clonezilla | Clonezilla disk backup images | +| blancco-reports | /srv/samba/blancco-reports | Blancco erasure reports (auto)| +| enrollment | /srv/samba/enrollment | GCCH bulk enrollment packages | +| image-upload | /home/pxe/image-upload | Image upload staging area | + +All shares use guest access for the isolated network. diff --git a/autoinstall/user-data b/autoinstall/user-data index 8f15adf..1dbd766 100644 --- a/autoinstall/user-data +++ b/autoinstall/user-data @@ -27,11 +27,19 @@ autoinstall: dhcp6: false optional: true - # The corp NIC may now have a lease, but the install stays offline-capable: - # no geoip lookup, and if the Ubuntu mirror is unreachable (Zscaler, - # no route) install from the ISO instead of failing. + # The install is always offline, even when the corp NIC has a lease and + # internet: the mirror points at a closed local port, so subiquity's + # mirror check fails and it installs from the ISO alone (no "security + # updates" step from the archive). The base is then exactly the ISO's, + # which is what download-packages.sh resolved the offline set against; + # late-commands patch it from CIDATA. The real archive sources are + # written back at the end of late-commands. apt: geoip: false + preserve_sources_list: false + primary: + - arches: [default] + uri: "http://127.0.0.1:9/ubuntu" fallback: offline-install # Storage configuration @@ -56,23 +64,25 @@ autoinstall: # Installer-stage late commands late-commands: - # Step 1: Install deb packages from CIDATA USB + # Step 1: Upgrade and install from the offline set on CIDATA + # (packages/ is an apt repo: debs + Packages + explicit.list). Any apt + # error fails the install; the log is /var/log/offline-apt-install.log. - | - curtin in-target --target=/target -- bash -c ' + curtin in-target --target=/target -- bash -euc ' mkdir -p /mnt/cidata CIDATA_DEV=$(blkid -L CIDATA) - if [ -n "$CIDATA_DEV" ]; then - mount "$CIDATA_DEV" /mnt/cidata - if compgen -G "/mnt/cidata/packages/*.deb" > /dev/null; then - cp /mnt/cidata/packages/*.deb /tmp/ - dpkg -i /tmp/*.deb 2>/dev/null || true - dpkg -i /tmp/*.deb 2>/dev/null || true - dpkg --configure -a 2>/dev/null || true - if command -v nmcli >/dev/null; then - systemctl enable NetworkManager - fi - fi - umount /mnt/cidata + mount -o ro "$CIDATA_DEV" /mnt/cidata + set +e + bash /mnt/cidata/offline-apt-install.sh /mnt/cidata/packages 2>&1 | tee /var/log/offline-apt-install.log + rc=${PIPESTATUS[0]} + set -e + umount /mnt/cidata + if [ "$rc" -ne 0 ]; then + echo "offline package install FAILED (rc=$rc), see /var/log/offline-apt-install.log" >&2 + exit "$rc" + fi + if command -v nmcli >/dev/null; then + systemctl enable NetworkManager fi ' @@ -98,14 +108,10 @@ autoinstall: if [ -n "$CIDATA_DEV" ]; then mkdir -p /mnt/usb mount "$CIDATA_DEV" /mnt/usb - if compgen -G "/mnt/usb/packages/*.deb" > /dev/null; then - dpkg -i /mnt/usb/packages/*.deb 2>/dev/null || true - dpkg -i /mnt/usb/packages/*.deb 2>/dev/null || true - dpkg --configure -a 2>/dev/null || true - fi if [ -f /mnt/usb/playbook/pxe_server_setup.yml ]; then cd /mnt/usb/playbook ansible-playbook -i localhost, -c local pxe_server_setup.yml + cd / fi umount /mnt/usb fi @@ -125,6 +131,25 @@ autoinstall: chmod +x /etc/rc.local ' + # Step 5: Real archive sources for later online maintenance (the + # install used a dead mirror on purpose, see apt: above). + - | + rm -f /target/etc/apt/sources.list.d/*.list /target/etc/apt/sources.list.d/*.curtin.orig + echo "# Ubuntu sources have moved to /etc/apt/sources.list.d/ubuntu.sources" > /target/etc/apt/sources.list + cat > /target/etc/apt/sources.list.d/ubuntu.sources <<"EOF" + Types: deb + URIs: http://archive.ubuntu.com/ubuntu/ + Suites: noble noble-updates noble-backports + Components: main restricted universe multiverse + Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg + + Types: deb + URIs: http://security.ubuntu.com/ubuntu/ + Suites: noble-security + Components: main restricted universe multiverse + Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg + EOF + user-data: disable_root: false diff --git a/scripts/build-usb.sh b/scripts/build-usb.sh index b69b627..872333c 100755 --- a/scripts/build-usb.sh +++ b/scripts/build-usb.sh @@ -170,6 +170,17 @@ else fi fi +# The installer patches and installs from packages/ as an apt repo +# (autoinstall late-commands -> offline-apt-install.sh): it needs the index +# and the explicit list that download-packages.sh writes with the debs. +for f in "$OFFLINE_PKG_DIR/Packages" "$OFFLINE_PKG_DIR/explicit.list" "$REPO_ROOT/scripts/offline-apt-install.sh"; do + if [ ! -f "$f" ]; then + echo "ERROR: $f not found. Build the offline set with" + echo " scripts/download-packages.sh --iso $ISO_PATH $OFFLINE_PKG_DIR" + exit 1 + fi +done + if [ -n "$SITE_VARS" ] && [ ! -f "$SITE_VARS" ]; then echo "ERROR: --site-vars file not found: $SITE_VARS" exit 1 @@ -487,16 +498,17 @@ echo "[5/6] Copying autoinstall config, packages, and playbook to CIDATA..." # Copy cloud-init files t_cp "" "$AUTOINSTALL_DIR/user-data" "$AUTOINSTALL_DIR/meta-data" -# Copy offline .deb packages into packages/ subdirectory +# Copy the offline package set (an apt repo: debs + Packages + explicit.list) +# into packages/, and the script the installer runs on it to the root. t_mkdir packages +t_cp packages "$OFFLINE_PKG_DIR/Packages" "$OFFLINE_PKG_DIR/explicit.list" DEB_COUNT=0 -if [ -d "$OFFLINE_PKG_DIR" ] && compgen -G "$OFFLINE_PKG_DIR/*.deb" >/dev/null; then +if compgen -G "$OFFLINE_PKG_DIR/*.deb" >/dev/null; then t_cp packages "$OFFLINE_PKG_DIR"/*.deb DEB_COUNT=$(find "$OFFLINE_PKG_DIR" -maxdepth 1 -name '*.deb' | wc -l) -else - echo " WARNING: no .deb packages in $OFFLINE_PKG_DIR (run download-packages.sh)" fi -echo " Copied $DEB_COUNT .deb packages to packages/" +t_cp "" "$REPO_ROOT/scripts/offline-apt-install.sh" +echo " Copied $DEB_COUNT .deb packages + Packages + explicit.list to packages/" # Copy playbook directory t_cp "" "$PLAYBOOK_DIR" diff --git a/scripts/download-packages.sh b/scripts/download-packages.sh index 0fa2068..c8d6a00 100755 --- a/scripts/download-packages.sh +++ b/scripts/download-packages.sh @@ -1,63 +1,48 @@ #!/bin/bash # -# download-packages.sh - Download all .deb packages needed for offline PXE server setup +# download-packages.sh - Build the offline package set for the PXE server # -# The PXE server installs Ubuntu 24.04 (Noble), so all packages MUST come from the -# 24.04 archive. If this script is run on a non-24.04 host (e.g. Zorin 17 / 22.04), -# it auto-spawns an Ubuntu 24.04 docker container to do the download. +# The server installs from the Ubuntu 24.04 server ISO with no network +# (autoinstall/user-data), so the installed base is exactly the ISO's +# "ubuntu-server" image plus what the installer adds from the ISO's pool/. +# This script resolves against THAT base, not against whatever the build +# host has: +# +# 1. The ISO's default install source (squashfs layers) is unpacked into +# a docker image, pxe-iso-base: (cached). +# 2. In a container of that image, with the Ubuntu archive (noble, +# noble-updates, noble-security) as the source, apt downloads: +# - every update of an installed package (dist-upgrade closure) +# - the explicit package list below, with its dependencies +# - the newest version of every package in the ISO's pool/ (what the +# installer may add: kernel, bootloader, openssh-server...) and +# the dependencies those versions need +# and writes Packages (an apt index) and explicit.list next to the +# debs, so the set is an apt repository. +# 3. A second container of the base image, with NO network, resolves the +# install exactly as the server will (offline-apt-install.sh +# --simulate). An unsatisfiable set fails the build here. +# 4. The webapp's pinned, hashed wheels go to pip-wheels/. +# +# On the server, autoinstall's late-commands run +# offline-apt-install.sh /mnt/cidata/packages +# which upgrades every installed package that the set has a newer version +# of and installs explicit.list, all from the local repo only. +# +# Needs docker (no sudo; the user must be in the docker group), xorriso and +# internet access to the Ubuntu archive and PyPI. # # Usage: -# ./download-packages.sh [output_directory] +# ./download-packages.sh --iso ubuntu-24.04.x-live-server-amd64.iso [output_directory] +# ISO=... ./download-packages.sh [output_directory] # -# Default output: ./offline-packages/ +# Default output: ./offline-packages/ (its old .debs are removed first). +# Wheels go to PIP_DIR (default: /pip-wheels). set -euo pipefail -OUT_DIR="${1:-./offline-packages}" -OUT_DIR_ABS="$(cd "$(dirname "$OUT_DIR")" 2>/dev/null && pwd)/$(basename "$OUT_DIR")" - -# Detect host Ubuntu codename. Run inside the container if not Noble (24.04). -HOST_CODENAME="$(. /etc/os-release && echo "${UBUNTU_CODENAME:-${VERSION_CODENAME:-}}")" - -# USE_DOCKER=1 forces the container even on a noble host: no sudo needed, -# and the host's own apt state is left alone. -if [ "${IN_DOCKER:-}" != "1" ] && { [ "$HOST_CODENAME" != "noble" ] || [ "${USE_DOCKER:-}" = "1" ]; }; then - echo "Host is '$HOST_CODENAME'; downloading inside an ubuntu:24.04 container." - echo "Re-running inside ubuntu:24.04 docker container..." - echo "" - - if ! command -v docker >/dev/null; then - echo "ERROR: docker not installed. Install docker or run on a real Ubuntu 24.04 host." - exit 1 - fi - - SCRIPT_PATH="$(readlink -f "$0")" - REPO_DIR="$(cd "$(dirname "$SCRIPT_PATH")"/.. && pwd)" - mkdir -p "$OUT_DIR_ABS" - # Wheels go to their own mount so PIP_DIR is honoured inside the container - # (the in-container default would be the repo's pip-wheels/). - PIP_DIR_HOST="$(mkdir -p "${PIP_DIR:-$REPO_DIR/pip-wheels}" && cd "${PIP_DIR:-$REPO_DIR/pip-wheels}" && pwd)" - - docker run --rm -i \ - -v "$REPO_DIR:/repo" \ - -v "$OUT_DIR_ABS:/out" \ - -v "$PIP_DIR_HOST:/pipout" \ - -e IN_DOCKER=1 \ - -e PIP_DIR=/pipout \ - -e HOST_UID="$(id -u)" -e HOST_GID="$(id -g)" \ - -e BLANCCO_KVER="${BLANCCO_KVER:-}" \ - -w /repo \ - ubuntu:24.04 \ - bash -c "apt-get update -qq && apt-get install -y --no-install-recommends sudo python3-pip python3-setuptools python3-wheel ca-certificates >/dev/null && /repo/scripts/download-packages.sh /out; rc=\$?; chown -R \$HOST_UID:\$HOST_GID /out /pipout; exit \$rc" - - echo "" - echo "============================================" - echo "Container build complete. Files in: $OUT_DIR_ABS" - echo "============================================" - exit 0 -fi - -mkdir -p "$OUT_DIR" +SCRIPT_PATH="$(readlink -f "$0")" +REPO_DIR="$(cd "$(dirname "$SCRIPT_PATH")"/.. && pwd)" # Kernel the Blancco UEFI path boots (playbook var blancco_kver; keep the two # in step). The playbook builds vmlinuz-ubuntu-70, kexec-initrd-70.img and @@ -108,75 +93,241 @@ AUTOINSTALL_PACKAGES=( ALL_PACKAGES=("${PLAYBOOK_PACKAGES[@]}" "${AUTOINSTALL_PACKAGES[@]}") +# --------------------------------------------------------------------------- +# Inside the base container (internal; started by the host part below) +# --------------------------------------------------------------------------- +if [ "${IN_BASE:-}" = "1" ]; then + export DEBIAN_FRONTEND=noninteractive + OUT=/out + ARCH=/tmp/arch + mkdir -p "$ARCH/partial" + + # The ISO's sources point at the archive (and maybe the cdrom); use the + # archive pockets only. + rm -f /etc/apt/sources.list /etc/apt/sources.list.d/* + cat >/etc/apt/sources.list.d/ubuntu.sources <<'SRC' +Types: deb +URIs: http://archive.ubuntu.com/ubuntu/ +Suites: noble noble-updates +Components: main restricted universe multiverse +Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg + +Types: deb +URIs: http://security.ubuntu.com/ubuntu/ +Suites: noble-security +Components: main restricted universe multiverse +Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg +SRC + echo "[base] apt-get update" + apt-get update -qq + + GET=(apt-get -o "Dir::Cache::archives=$ARCH" -o APT::Sandbox::User=root) + + # 1+2: dist-upgrade closure plus the explicit list, as one transaction + # (the same one offline-apt-install.sh runs on the server). + DU=$(apt-get -s dist-upgrade) + if grep -q 'kept back' <<<"$DU"; then + echo "ERROR: dist-upgrade keeps packages back on the ISO base:" >&2 + echo "$DU" >&2 + exit 1 + fi + mapfile -t UPGRADES < <(awk '/^Inst /{print $2}' <<<"$DU") + echo "[base] ${#UPGRADES[@]} installed packages have updates" + if ! SIM=$(apt-get -s install "${ALL_PACKAGES[@]}" "${UPGRADES[@]}" 2>&1); then + echo "ERROR: apt could not resolve the package list on the ISO base:" >&2 + grep -E "^(E|W):| : Depends" <<<"$SIM" >&2 || echo "$SIM" >&2 + exit 1 + fi + echo "[base] downloading the install + upgrade transaction" + "${GET[@]}" -y -q --download-only install "${ALL_PACKAGES[@]}" "${UPGRADES[@]}" >/tmp/dl.log 2>&1 \ + || { echo "ERROR: download failed:" >&2; tail -30 /tmp/dl.log >&2; exit 1; } + + # 3: newest version of every pool package the installer may add, with + # the dependencies that version needs on the base. Versioned kernel ABI + # packages (linux-*-6.8.0-71*) are reached through their metapackages. + POOL_EXTRA=() + SKIPPED=() + while read -r name; do + [ -n "$name" ] || continue + case "$name" in + linux-*-[0-9]*.[0-9]*.[0-9]*-[0-9]*) continue ;; + # Driver packages for ubuntu-drivers (NVIDIA): never on this box. + *nvidia*) continue ;; + esac + if PS=$(apt-get -s install "$name" 2>&1); then + mapfile -t -O "${#POOL_EXTRA[@]}" POOL_EXTRA < <(awk '/^Inst /{print $2}' <<<"$PS") + else + SKIPPED+=("$name") + fi + done < /meta/pool.txt + mapfile -t POOL_EXTRA < <(printf '%s\n' "${POOL_EXTRA[@]}" | sort -u) + echo "[base] pool packages: $(wc -l /tmp/pool-dl.log 2>&1) \ + || { echo "ERROR: pool download failed:" >&2; tail -30 /tmp/pool-dl.log >&2; exit 1; } + + # Several versions of one package (download + pool): keep the newest. + find "$OUT" -maxdepth 1 \( -name '*.deb' -o -name 'Packages*' -o -name explicit.list \) -delete + cp "$ARCH"/*.deb "$OUT"/ + python3 - "$OUT" <<'PY' +import glob, os, subprocess, sys +best = {} +for p in glob.glob(os.path.join(sys.argv[1], "*.deb")): + n, v, a = os.path.basename(p)[:-4].split("_") + v = v.replace("%3a", ":") + k = (n, a) + if k in best: + old_v, old_p = best[k] + if subprocess.run(["dpkg", "--compare-versions", v, "gt", old_v]).returncode == 0: + os.remove(old_p); best[k] = (v, p) + else: + os.remove(p) + else: + best[k] = (v, p) +PY + printf '%s\n' "${ALL_PACKAGES[@]}" >"$OUT/explicit.list" + # Uncompressed index only: apt's file: method takes it as is. + (cd "$OUT" && apt-ftparchive packages . >Packages 2>/dev/null) + echo "[base] $(find "$OUT" -maxdepth 1 -name '*.deb' | wc -l) debs, $(du -sh "$OUT" | cut -f1)" + chown -R "${HOST_UID:-0}:${HOST_GID:-0}" "$OUT" + exit 0 +fi + +# --------------------------------------------------------------------------- +# Host part +# --------------------------------------------------------------------------- +ISO="${ISO:-}" +POSITIONAL=() +while [ $# -gt 0 ]; do + case "$1" in + --iso) ISO="$2"; shift 2 ;; + -h|--help) sed -n '2,45p' "$SCRIPT_PATH"; exit 0 ;; + *) POSITIONAL+=("$1"); shift ;; + esac +done +OUT_DIR="${POSITIONAL[0]:-./offline-packages}" +mkdir -p "$OUT_DIR" +OUT_DIR_ABS="$(cd "$OUT_DIR" && pwd)" +PIP_DIR_HOST="${PIP_DIR:-$REPO_DIR/pip-wheels}" +mkdir -p "$PIP_DIR_HOST" +PIP_DIR_HOST="$(cd "$PIP_DIR_HOST" && pwd)" + +[ -n "$ISO" ] || { echo "ERROR: give the server ISO: --iso ubuntu-24.04.x-live-server-amd64.iso (the same one build-usb.sh uses)" >&2; exit 1; } +[ -f "$ISO" ] || { echo "ERROR: ISO not found: $ISO" >&2; exit 1; } +for t in docker xorriso; do + command -v "$t" >/dev/null || { echo "ERROR: $t not installed" >&2; exit 1; } +done + +META=$(mktemp -d "${TMPDIR:-/tmp}/dlpkg.XXXXXX") +trap 'rm -rf "$META"' EXIT + echo "============================================" -echo "Offline Package Downloader (Ubuntu 24.04 noble)" +echo "Offline package set for the PXE server" echo "============================================" -echo "Output directory: $OUT_DIR" -echo "" -echo "Packages to resolve:" -printf ' - %s\n' "${ALL_PACKAGES[@]}" +echo "ISO : $ISO" +echo "Output : $OUT_DIR_ABS" +echo "Wheels : $PIP_DIR_HOST" echo "" -# Update package cache -echo "[1/4] Updating package cache..." -sudo apt-get update -qq +# --- 1. base image from the ISO's default install source --- +VOLID=$(xorriso -indev "$ISO" -pvd_info 2>/dev/null | sed -n 's/^Volume Id *: *//p' | head -1) +TAG="pxe-iso-base:$(tr 'A-Z ' 'a-z-' <<<"$VOLID" | tr -cd 'a-z0-9.-')" +xorriso -osirrox on -indev "$ISO" -extract /casper/install-sources.yaml "$META/install-sources.yaml" >/dev/null 2>&1 +# Default source: the entry with "default: true"; a layered path a.b.squashfs +# is layer a.squashfs under a.b.squashfs. +SRC_PATH=$(awk '/^- /{d=0} /default: true/{d=1} /^ path:/{p=$2} /^ type:/{ if (d) {print p; exit} }' "$META/install-sources.yaml") +[ -n "$SRC_PATH" ] || { echo "ERROR: no default source in casper/install-sources.yaml" >&2; exit 1; } +LAYERS=() +stem="${SRC_PATH%.squashfs}" +acc="" +IFS=. read -r -a parts <<<"$stem" +for p in "${parts[@]}"; do + acc="${acc:+$acc.}$p" + LAYERS+=("$acc.squashfs") +done +echo "[1/4] Base: $VOLID, source $SRC_PATH (layers: ${LAYERS[*]}) -> $TAG" -# Simulate install to find all dependencies -echo "[2/4] Resolving dependencies..." -SIM=$(apt-get install --simulate "${ALL_PACKAGES[@]}" 2>&1) || { - # e.g. a package name that does not exist in noble: say which. - echo "ERROR: apt could not resolve the package list:" >&2 - echo "$SIM" | grep -E "^(E|W):" >&2 - exit 1 -} -EXPLICIT_DEPS=$(echo "$SIM" | grep "^Inst " | awk '{print $2}') +xorriso -indev "$ISO" -find /pool -type f -name '*.deb' 2>/dev/null \ + | sed "s|.*/||; s|'\$||; s|_.*||" | sort -u >"$META/pool.txt" -# ALSO pull every package that would upgrade in a dist-upgrade. This is -# critical: the Ubuntu ISO ships a point-in-time baseline, but our explicit -# packages (from noble-updates) may depend on *newer* versions of ISO-baseline -# packages (e.g. gnupg 17.4 needs matching gpgv 17.4). Without this, offline -# install fails with dpkg "dependency problems" because transitive version -# bumps aren't captured by --simulate on the explicit list. -UPGRADE_DEPS=$(apt-get dist-upgrade --simulate 2>&1 \ - | grep "^Inst " \ - | awk '{print $2}') +if docker image inspect "$TAG" >/dev/null 2>&1; then + echo " image $TAG exists (docker rmi $TAG to rebuild)" +else + for l in "${LAYERS[@]}"; do + xorriso -osirrox on -indev "$ISO" -extract "/casper/$l" "$META/$l" >/dev/null 2>&1 + done + # Unpack the layers as root in a throwaway container (keeps ownership), + # apply overlayfs whiteouts, stream the tree into docker import. + docker run --rm -i -v "$META:/meta:ro" ubuntu:24.04 bash -euc ' + apt-get update -qq >/dev/null && apt-get install -y -qq squashfs-tools >/dev/null + mkdir /r + first=1 + for l in '"${LAYERS[*]}"'; do + if [ $first = 1 ]; then + unsquashfs -q -no-xattrs -f -d /r "/meta/$l" >/dev/null; first=0 + else + # Whiteouts (char devices) = files deleted in this layer. + # The container may not mknod, so list them, delete those + # paths from the tree and extract the layer without them. + unsquashfs -lls "/meta/$l" | awk '"'"'$1 ~ /^c/ {sub(/^.*squashfs-root/, ""); print}'"'"' >/wo + while read -r w; do rm -rf "/r$w"; done /dev/null + else + unsquashfs -q -no-xattrs -d /l "/meta/$l" >/dev/null + fi + cp -a /l/. /r/ + fi + done + tar -C /r -c .' >"$META/base.tar" + # Import only a complete tree: a failed unpack must not leave an image. + docker import "$META/base.tar" "$TAG" >/dev/null + rm -f "$META/base.tar" + docker run --rm "$TAG" test -x /usr/bin/apt-get \ + || { docker rmi "$TAG" >/dev/null; echo "ERROR: base image $TAG is incomplete" >&2; exit 1; } + echo " built $TAG" +fi +for l in "${LAYERS[@]}"; do rm -f "$META/$l"; done -DEPS=$(printf '%s\n%s\n' "$EXPLICIT_DEPS" "$UPGRADE_DEPS" | sort -u | grep -v '^$') +# --- 2. resolve + download against the base --- +echo "[2/4] Resolving and downloading on the ISO base (this takes a while)..." +docker run --rm \ + -v "$REPO_DIR:/repo:ro" \ + -v "$OUT_DIR_ABS:/out" \ + -v "$META:/meta:ro" \ + -e IN_BASE=1 -e BLANCCO_KVER="$BLANCCO_KVER" \ + -e HOST_UID="$(id -u)" -e HOST_GID="$(id -g)" \ + "$TAG" bash /repo/scripts/download-packages.sh -DEP_COUNT=$(echo "$DEPS" | wc -l) -echo " Found $DEP_COUNT packages (explicit + baseline upgrades)" +# --- 3. offline check: resolve exactly as the server will, no network --- +echo "[3/4] Offline check on a fresh base container (no network)..." +docker run --rm --network none \ + -v "$REPO_DIR:/repo:ro" -v "$OUT_DIR_ABS:/out:ro" \ + "$TAG" bash /repo/scripts/offline-apt-install.sh --simulate /out -# Download all packages -echo "[3/4] Downloading .deb packages to $OUT_DIR..." -cd "$OUT_DIR" -apt-get download $DEPS 2>&1 | tail -5 - -DEB_COUNT=$(ls -1 *.deb 2>/dev/null | wc -l) -TOTAL_SIZE=$(du -sh . | cut -f1) - -echo " $DEB_COUNT packages ($TOTAL_SIZE)" - -# Download pip wheels for Flask webapp (offline install) -echo "[4/4] Downloading Python wheels for webapp..." -# Place pip-wheels next to the script (or /repo when in docker), not next to OUT_DIR -REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")"/.. && pwd)" -PIP_DIR="${PIP_DIR:-$REPO_ROOT/pip-wheels}" +# --- 4. pinned webapp wheels --- +echo "[4/4] Downloading Python wheels for the webapp..." # Exactly the pinned, hashed set in webapp/requirements.txt, for the server's # Python 3.12 on x86_64. The folder is emptied first: the playbook installs # every wheel in it and refuses any wheel that is not pinned. -find "$PIP_DIR" -mindepth 1 -delete 2>/dev/null; mkdir -p "$PIP_DIR" -pip3 download -d "$PIP_DIR" --require-hashes -r "$REPO_ROOT/webapp/requirements.txt" \ - --only-binary=:all: --no-deps --python-version 3.12 --implementation cp --abi cp312 \ - --platform manylinux_2_28_x86_64 --platform manylinux_2_17_x86_64 --platform manylinux2014_x86_64 \ - 2>&1 | tail -5 - -WHL_COUNT=$(ls -1 "$PIP_DIR"/*.whl "$PIP_DIR"/*.tar.gz 2>/dev/null | wc -l) -echo " $WHL_COUNT Python packages downloaded to pip-wheels/" +find "$PIP_DIR_HOST" -mindepth 1 -delete +docker run --rm -v "$REPO_DIR:/repo:ro" -v "$PIP_DIR_HOST:/pipout" \ + -e HOST_UID="$(id -u)" -e HOST_GID="$(id -g)" ubuntu:24.04 bash -euc ' + apt-get update -qq >/dev/null + apt-get install -y -qq --no-install-recommends python3-pip ca-certificates >/dev/null + pip3 download -q -d /pipout --require-hashes -r /repo/webapp/requirements.txt \ + --only-binary=:all: --no-deps --python-version 3.12 --implementation cp --abi cp312 \ + --platform manylinux_2_28_x86_64 --platform manylinux_2_17_x86_64 --platform manylinux2014_x86_64 + chown -R "$HOST_UID:$HOST_GID" /pipout' +DEB_COUNT=$(find "$OUT_DIR_ABS" -maxdepth 1 -name '*.deb' | wc -l) +WHL_COUNT=$(find "$PIP_DIR_HOST" -maxdepth 1 \( -name '*.whl' -o -name '*.tar.gz' \) | wc -l) echo "" echo "============================================" echo "Download complete!" echo "============================================" -echo " .deb packages: $DEB_COUNT ($TOTAL_SIZE) in $OUT_DIR/" -echo " Python wheels: $WHL_COUNT in $PIP_DIR/" +echo " .deb packages: $DEB_COUNT ($(du -sh "$OUT_DIR_ABS" | cut -f1)) in $OUT_DIR_ABS/ (apt repo: Packages, explicit.list)" +echo " Python wheels: $WHL_COUNT in $PIP_DIR_HOST/" +echo " Check them: scripts/check-updates.py --debs $OUT_DIR_ABS" echo "" diff --git a/scripts/offline-apt-install.sh b/scripts/offline-apt-install.sh new file mode 100755 index 0000000..cade271 --- /dev/null +++ b/scripts/offline-apt-install.sh @@ -0,0 +1,85 @@ +#!/bin/bash +# +# offline-apt-install.sh - Install and patch from the offline package set +# +# Uses DIR (offline-packages/ as built by download-packages.sh: the .debs, +# a Packages index and explicit.list) as the ONLY apt source: +# 1. every installed package that DIR has a newer version of is upgraded +# 2. every package in DIR/explicit.list is installed +# in one apt transaction, then checks that nothing is left to upgrade. +# The system's own sources.list and lists are not read or changed. +# +# Any apt failure stops the script with a non-zero exit and apt's message. +# +# Usage: +# offline-apt-install.sh DIR install (as root) +# offline-apt-install.sh --simulate DIR resolve only (apt-get -s) +# +# Runs in the autoinstall late-commands (curtin in-target) and in +# download-packages.sh's offline check of the bundle. + +set -euo pipefail + +SIMULATE=0 +if [ "${1:-}" = "--simulate" ]; then SIMULATE=1; shift; fi +DIR="${1:?usage: $0 [--simulate] DIR}" +DIR="$(cd "$DIR" && pwd)" + +for f in Packages explicit.list; do + [ -f "$DIR/$f" ] || { echo "offline-apt: ERROR: $DIR/$f missing (rebuild with download-packages.sh)" >&2; exit 1; } +done + +WORK=$(mktemp -d) +trap 'rm -rf "$WORK"' EXIT +mkdir -p "$WORK/lists/partial" "$WORK/parts" +echo "deb [trusted=yes] file:$DIR ./" >"$WORK/sources.list" + +APT=(apt-get + -o "Dir::Etc::SourceList=$WORK/sources.list" + -o "Dir::Etc::SourceParts=$WORK/parts" + -o "Dir::State::Lists=$WORK/lists" + -o "APT::Sandbox::User=root" + -o "Dpkg::Options::=--force-confdef" + -o "Dpkg::Options::=--force-confold") +export DEBIAN_FRONTEND=noninteractive + +echo "offline-apt: source $DIR ($(find "$DIR" -maxdepth 1 -name '*.deb' | wc -l) debs)" +"${APT[@]}" update + +mapfile -t EXPLICIT < <(grep -v '^[[:space:]]*\(#\|$\)' "$DIR/explicit.list") +DU=$("${APT[@]}" -s dist-upgrade) +if grep -q 'kept back' <<<"$DU"; then + echo "offline-apt: ERROR: upgrades kept back (missing dependencies in $DIR):" >&2 + echo "$DU" >&2 + exit 1 +fi +mapfile -t UPGRADES < <(awk '/^Inst /{print $2}' <<<"$DU") +echo "offline-apt: ${#UPGRADES[@]} upgrades, ${#EXPLICIT[@]} explicit packages" + +if [ "$SIMULATE" -eq 1 ]; then + OUT=$("${APT[@]}" -s install "${EXPLICIT[@]}" "${UPGRADES[@]}") + echo "offline-apt: simulated: $(grep -c '^Inst ' <<<"$OUT") packages to unpack, $(grep -c '^Conf ' <<<"$OUT") to configure" + exit 0 +fi + +"${APT[@]}" -y install "${EXPLICIT[@]}" "${UPGRADES[@]}" + +AFTER=$("${APT[@]}" -s dist-upgrade) +LEFT=$(awk '/^Inst /{print $2}' <<<"$AFTER") +if grep -q 'kept back' <<<"$AFTER"; then + echo "offline-apt: ERROR: upgrades kept back (missing dependencies in $DIR):" >&2 + echo "$AFTER" >&2 + exit 1 +fi +if [ -n "$LEFT" ]; then + echo "offline-apt: ERROR: still upgradable from $DIR after install:" >&2 + echo "$LEFT" >&2 + exit 1 +fi +BROKEN=$(dpkg --audit || true) +if [ -n "$BROKEN" ]; then + echo "offline-apt: ERROR: dpkg --audit reports:" >&2 + echo "$BROKEN" >&2 + exit 1 +fi +echo "offline-apt: done, nothing left to upgrade from the offline set" diff --git a/tests/vm/README.md b/tests/vm/README.md index 787f902..5e5a91c 100644 --- a/tests/vm/README.md +++ b/tests/vm/README.md @@ -10,7 +10,8 @@ a laptop, without root and without touching a real network. | `pxetest-imaging` | libvirt network, bridge `virbr-pxetest`. No ``, no ``: the host has no address on it and libvirt runs no DHCP there. Only VM taps are on the bridge. | | `default` | libvirt NAT, 192.168.122.0/24. Stands in for the GE/corp network. | | `pxetest-server` | 2 vCPU, 4 GiB, q35 + OVMF Secure Boot (MS keys), 60G thin qcow2 (`vda`, boot 1), the USB image as a USB disk (`sda`, boot 2). | -| `pxetest-client` | 2 vCPU, 2 GiB, OVMF Secure Boot (MS keys), 1G blank disk, network boot first (UEFI IPv4 PXE, virtio-net, OVMF driver, no option ROM). | +| `pxetest-client` | 2 vCPU, 2 GiB, OVMF Secure Boot (MS keys), 1G blank disk, network boot first (UEFI IPv4 PXE, virtio-net, OVMF driver, no option ROM). Proves shim -> iPXE -> menu -> WinPE under Secure Boot; WinPE has no virtio driver, so it stops at "Waiting for network". | +| `pxetest-client-nosb` | 2 vCPU, 2 GiB, OVMF without Secure Boot, e1000e NIC, 64G thin SATA disk. WinPE has in-box drivers for both, so startnet gets a network and maps the shares. | NIC MACs (fixed): @@ -19,6 +20,7 @@ NIC MACs (fixed): | server | GE | `52:54:00:9e:10:01` | `default` NAT | `corp_nic_mac` | | server | imaging | `52:54:00:9e:10:02` | `pxetest-imaging` | `pxe_nic_mac` | | client | - | `52:54:00:9e:10:03` | `pxetest-imaging` | - | +| client-nosb | - | `52:54:00:9e:10:04` | `pxetest-imaging` | - | `build-image.sh` writes `$PXETEST_CACHE/site-vars.yml` with these two MACs (`pxe_nic_mac`, `corp_nic_mac`) and passes it to `build-usb.sh @@ -40,14 +42,19 @@ IPv4 address or a non-tap bridge port. export PXETEST_CACHE=~/.cache/pxetest # ISO + image (about 10 GB) export PXETEST_PAYLOAD_ROOT=/home/camp/projects/pxe # checkout holding offline-packages/, boot-files/, ... + export UBUNTU_ISO_NAME=ubuntu-24.04.3-live-server-amd64.iso # optional; default 24.04.5 - tests/vm/fetch-iso.sh # Ubuntu 24.04.5 live-server ISO, SHA256SUMS-verified + tests/vm/fetch-iso.sh # the live-server ISO, SHA256SUMS-verified + scripts/download-packages.sh --iso "$PXETEST_CACHE/$UBUNTU_ISO_NAME" "$PXETEST_PAYLOAD_ROOT/offline-packages" + scripts/check-updates.py --debs "$PXETEST_PAYLOAD_ROOT/offline-packages" tests/vm/build-image.sh # build-usb.sh --image, upload as volume pxetest-usb.img tests/vm/create.sh # network, volumes, both VMs (prints the MAC pins) tests/vm/start.sh server # autoinstall, reboot, first-boot playbook tests/vm/status.sh --shot /tmp/pxetest-shots # states, leases, screenshots - tests/vm/start.sh client # once the server playbook is done - tests/vm/stop.sh # ACPI shutdown both, keep them defined + tests/vm/start.sh client # once the server playbook is done (Secure Boot client) + tests/vm/stop.sh client + tests/vm/start.sh client2 # Secure Boot off, e1000e: WinPE maps the shares + tests/vm/stop.sh # ACPI shutdown all, keep them defined tests/vm/destroy.sh # remove every pxetest-* object `build-image.sh` skips enrollment/, bios-staging/, drivers-staging/ and @@ -62,5 +69,10 @@ to `/var/log/first-boot.log`. Once the GE NIC takes a DHCP lease on the NAT network, `status.sh` shows its address and `ssh pxe@
` works from the host. -Memory: server 4 GiB + client 2 GiB. Keep other VMs (win11) off while +The offline package set must be built against the SAME ISO the image is +built from (download-packages.sh resolves against that ISO's base). For a +fresh server disk: `tests/vm/destroy.sh`, then build-image.sh, create.sh, +start.sh server. + +Memory: server 4 GiB + one client 2 GiB (start.sh refuses a second client). Keep other VMs (win11) off while both run, and stop them when done. diff --git a/tests/vm/common.sh b/tests/vm/common.sh index 8e2e8ed..143eb45 100755 --- a/tests/vm/common.sh +++ b/tests/vm/common.sh @@ -47,6 +47,11 @@ CLIENT_MEM_MIB="${CLIENT_MEM_MIB:-2048}" SERVER_GE_MAC="52:54:00:9e:10:01" # on 'default' NAT (GE/corp side) SERVER_IMAGING_MAC="52:54:00:9e:10:02" # on pxetest-imaging (imaging side) CLIENT_MAC="52:54:00:9e:10:03" # on pxetest-imaging only +# Second client: Secure Boot OFF, e1000e NIC and SATA disk (in-box WinPE +# drivers), so WinPE gets a network and startnet can map the shares. +CLIENT2_VM="pxetest-client-nosb" +CLIENT2_DISK_VOL="pxetest-client-nosb.qcow2" +CLIENT2_MAC="52:54:00:9e:10:04" # on pxetest-imaging only log() { printf '[pxetest] %s\n' "$*"; } die() { printf '[pxetest] ERROR: %s\n' "$*" >&2; exit 1; } diff --git a/tests/vm/create.sh b/tests/vm/create.sh index 9742528..357aaa7 100755 --- a/tests/vm/create.sh +++ b/tests/vm/create.sh @@ -43,6 +43,10 @@ if ! vol_exists "$SERVER_DISK_VOL"; then virsh vol-create-as "$POOL" "$SERVER_DISK_VOL" "$SERVER_DISK_SIZE" --format qcow2 >/dev/null log "created $SERVER_DISK_VOL ($SERVER_DISK_SIZE thin qcow2)" fi +if ! vol_exists "$CLIENT2_DISK_VOL"; then + virsh vol-create-as "$POOL" "$CLIENT2_DISK_VOL" 64G --format qcow2 >/dev/null + log "created $CLIENT2_DISK_VOL (64G thin, blank)" +fi if ! vol_exists "$CLIENT_DISK_VOL"; then virsh vol-create-as "$POOL" "$CLIENT_DISK_VOL" 1G --format qcow2 >/dev/null log "created $CLIENT_DISK_VOL (1G blank)" @@ -157,8 +161,49 @@ XML log "defined $CLIENT_VM" fi +# --- second client: Secure Boot off, e1000e + SATA (WinPE in-box drivers) --- +if ! dom_exists "$CLIENT2_VM"; then + cat >"$TMP/client2.xml" < + $CLIENT2_VM + $CLIENT_MEM_MIB + 2 + + hvm + + + + + + + + + + + + + + + + + + + + + + + + + + + +XML + virsh define "$TMP/client2.xml" >/dev/null + log "defined $CLIENT2_VM" +fi + # Never attach a pxetest VM anywhere but default NAT / pxetest-imaging. -for vm in "$SERVER_VM" "$CLIENT_VM"; do +for vm in "$SERVER_VM" "$CLIENT_VM" "$CLIENT2_VM"; do while read -r _ type src _ _; do case "$type:$src" in network:"$GE_NET") [ "$vm" = "$SERVER_VM" ] || die "$vm is on $GE_NET" ;; diff --git a/tests/vm/destroy.sh b/tests/vm/destroy.sh index aa7550e..f4cffd3 100755 --- a/tests/vm/destroy.sh +++ b/tests/vm/destroy.sh @@ -5,13 +5,13 @@ # Usage: tests/vm/destroy.sh source "$(dirname "$0")/common.sh" -for vm in "$CLIENT_VM" "$SERVER_VM"; do +for vm in "$CLIENT_VM" "$CLIENT2_VM" "$SERVER_VM"; do dom_exists "$vm" || continue [ "$(dom_state "$vm")" = "shut off" ] || virsh destroy "$vm" >/dev/null virsh undefine "$vm" --nvram >/dev/null log "undefined $vm" done -for v in "$SERVER_DISK_VOL" "$USB_VOL" "$CLIENT_DISK_VOL"; do +for v in "$SERVER_DISK_VOL" "$USB_VOL" "$CLIENT_DISK_VOL" "$CLIENT2_DISK_VOL"; do vol_exists "$v" || continue virsh vol-delete --pool "$POOL" "$v" >/dev/null log "deleted volume $v" diff --git a/tests/vm/start.sh b/tests/vm/start.sh index d63c2ef..c5bb65b 100755 --- a/tests/vm/start.sh +++ b/tests/vm/start.sh @@ -1,16 +1,19 @@ #!/bin/bash # Start pxetest VMs. Idempotent (a running VM is left alone). -# Usage: tests/vm/start.sh [server|client|all] (default: server) -# Start the client only once the server's playbook has finished; both -# together use 6 GiB. +# Usage: tests/vm/start.sh [server|client|client2|all] (default: server) +# client = Secure Boot client (virtio NIC: reaches the iPXE menu and WinPE) +# client2 = Secure Boot OFF client, e1000e NIC (WinPE gets a network) +# Start a client only once the server's playbook has finished, and only one +# client at a time: server + one client = 6 GiB. source "$(dirname "$0")/common.sh" which="${1:-server}" case "$which" in server) vms=("$SERVER_VM") ;; client) vms=("$CLIENT_VM") ;; + client2|nosb) vms=("$CLIENT2_VM") ;; all) vms=("$SERVER_VM" "$CLIENT_VM") ;; - *) die "usage: $0 [server|client|all]" ;; + *) die "usage: $0 [server|client|client2|all]" ;; esac net_exists "$IMAGING_NET" || die "run create.sh first" @@ -19,6 +22,15 @@ assert_imaging_isolated for vm in "${vms[@]}"; do dom_exists "$vm" || die "$vm not defined; run create.sh" + # RAM: one client at a time. + case "$vm" in + "$CLIENT_VM") other="$CLIENT2_VM" ;; + "$CLIENT2_VM") other="$CLIENT_VM" ;; + *) other="" ;; + esac + if [ -n "$other" ] && [ "$(dom_state "$other")" = "running" ]; then + die "$other is running; stop it first (tests/vm/stop.sh) to stay within RAM" + fi if [ "$vm" = "$SERVER_VM" ]; then vol_exists "$USB_VOL" || die "no $USB_VOL; run build-image.sh" fi diff --git a/tests/vm/status.sh b/tests/vm/status.sh index 6bf8925..3480225 100755 --- a/tests/vm/status.sh +++ b/tests/vm/status.sh @@ -12,14 +12,14 @@ echo "network $IMAGING_NET: $n" if net_active "$IMAGING_NET"; then (assert_imaging_isolated && echo " isolation: OK (no forward, no ip, no host IPv4, only vnet ports)") fi -for v in "$SERVER_DISK_VOL" "$USB_VOL" "$CLIENT_DISK_VOL"; do +for v in "$SERVER_DISK_VOL" "$USB_VOL" "$CLIENT_DISK_VOL" "$CLIENT2_DISK_VOL"; do if vol_exists "$v"; then echo "volume $v: $(virsh vol-info --pool "$POOL" "$v" | awk '/Capacity|Allocation/{printf "%s %s %s ", $1, $2, $3}')" else echo "volume $v: absent" fi done -for vm in "$SERVER_VM" "$CLIENT_VM"; do +for vm in "$SERVER_VM" "$CLIENT_VM" "$CLIENT2_VM"; do if dom_exists "$vm"; then echo "domain $vm: $(dom_state "$vm")" virsh domiflist "$vm" | tail -n +3 | grep -v '^$' | sed 's/^/ /' diff --git a/tests/vm/stop.sh b/tests/vm/stop.sh index 9017e4f..0ef0f8b 100755 --- a/tests/vm/stop.sh +++ b/tests/vm/stop.sh @@ -1,15 +1,16 @@ #!/bin/bash # Shut pxetest VMs down (ACPI, then force after a timeout). Keeps them # defined. Idempotent. -# Usage: tests/vm/stop.sh [server|client|all] (default: all) +# Usage: tests/vm/stop.sh [server|client|client2|all] (default: all) source "$(dirname "$0")/common.sh" which="${1:-all}" case "$which" in server) vms=("$SERVER_VM") ;; client) vms=("$CLIENT_VM") ;; - all) vms=("$CLIENT_VM" "$SERVER_VM") ;; - *) die "usage: $0 [server|client|all]" ;; + client2|nosb) vms=("$CLIENT2_VM") ;; + all) vms=("$CLIENT_VM" "$CLIENT2_VM" "$SERVER_VM") ;; + *) die "usage: $0 [server|client|client2|all]" ;; esac for vm in "${vms[@]}"; do dom_exists "$vm" || continue