pxe-server/SETUP.md

# PXE Server Setup Guide

Automated build process for deploying an Ubuntu-based PXE boot server that hosts GE Aerospace Windows PE images. The entire setup is air-gapped — no internet required on the target machine.

## Architecture

```
Client PXE boot
  -> Broadcom signed iPXE (Secure Boot)
  -> iPXE boot menu (HTTP, port 4433)
     ├── Windows PE -> wimboot -> boot.wim -> startnet.cmd -> Samba -> Image deployment
     ├── Clonezilla -> vmlinuz/initrd -> Disk cloning/imaging
     ├── Blancco    -> Native kernel + initramfs -> NIST 800-88 drive erasure (auto-reports)
     (memory test: Dell F12 > Diagnostics)
```

### Services on the PXE Server

| Service     | Port      | Purpose                                  |
|-------------|-----------|------------------------------------------|
| dnsmasq     | 67/udp    | DHCP (172.16.9.10-100)                   |
| dnsmasq     | 69/udp    | TFTP (serves ipxe.efi)                   |
| Apache      | 80/tcp    | HTTP (wimboot, WinPE boot files, proxy)  |
| Apache      | 4433/tcp  | iPXE boot script (GetPxeScript.aspx)     |
| Samba       | 445/tcp   | Deployment content + backup + reports    |
| Flask Webapp| 9009/tcp  | Web management interface                 |

## Prerequisites

### Hardware
- Server or PC with >= 8 GB RAM, >= 250 GB disk, one wired NIC
- USB thumb drive >= 8 GB (32+ GB if bundling WinPE images)

### Software (on your workstation)
- Ubuntu Server 24.04 ISO — https://ubuntu.com/download/server
- Linux workstation (Ubuntu/Mint) for running download and build scripts
- GE Aerospace Media Creator LITE (for caching WinPE images)

### GE Access Packages
- EPM Rufus Exception Request
- EPM DT Functions
- DLP - Encrypted Removable (USB) Long Term Access

## Setup Process

### Step 1: Download Offline Packages (one-time, requires internet)

```bash
./scripts/download-packages.sh --iso /path/to/ubuntu-24.04.x-live-server-amd64.iso
./scripts/check-updates.py          # expect: PASSED, 0 security updates
./scripts/fetch-ipxe.sh
```

`fetch-ipxe.sh` downloads the Secure Boot iPXE chain into `boot-files/`, pinned by sha256: the iPXE release
tarball (its snponly build becomes `ipxe.efi`) and iPXE's dual-signed shim (Microsoft UEFI CA 2011 + 2023),
which every UEFI client boots as `ipxe-shim.efi`. Keep the old Broadcom-signed `boot-files/ipxe.efi` too: the
playbook installs it as `ipxe-bcm.efi`, the rollback boot file.

`download-packages.sh` needs docker (no sudo) and the SAME server ISO that
`build-usb.sh` will use. The install is offline (autoinstall uses a dead
mirror on purpose), so the server's base is exactly that ISO's; the script
unpacks the ISO's install image into a docker image and resolves there:
every update of an installed package, the explicit package list
(ansible, dnsmasq, apache2, samba, wimtools, zip, the HWE kernel, ...) and
the newest version of what the installer adds from the ISO's pool (kernel,
grub, openssh-server). `offline-packages/` becomes an apt repository
(debs + `Packages` + `explicit.list`), checked offline in a fresh container
of the base before the script finishes. On the server, autoinstall's
late-commands run `scripts/offline-apt-install.sh` on it: one apt
transaction from the local repo only, any error fails the install (log:
`/var/log/offline-apt-install.log`). The webapp's pinned, hashed wheels go
to `pip-wheels/`. About 500 debs (~1.6 GB) + 8 wheels.

**Packages included:**
- **Server:** dnsmasq, apache2, samba, ufw, cron
- **Automation:** ansible
- **Tools:** wimtools, unzip, p7zip-full
- **Python:** python3-pip, python3-venv
- **Network:** network-manager, wpasupplicant, wireless-tools, linux-firmware

### Step 2: Prepare Boot Tools (optional)

```bash
./scripts/prepare-boot-tools.sh /path/to/blancco.iso
```

Extracts boot files for Blancco and Clonezilla (64-bit with Debian's signed shim, so it boots with Secure Boot on, plus the last 32-bit release for XP-era PCs) into the `boot-tools/` directory. Automatically patches Blancco's `config.img` to auto-save erasure reports to the PXE server's Samba share.

### Step 2b: Capture site content from a running server

```bash
./scripts/capture-site-content.sh            # dry run: lists what would change
./scripts/capture-site-content.sh --apply    # pull it
```

Pulls what neither the repo nor the MCL image import brings, from a working PXE server (default 172.16.9.1, SSH key login as `pxe`): `_shared/Applications/extra` (Office, Adobe, OpenText, Java, fonts, Wi-Fi, Zscaler), the GE CA certificates and the Engineering Layer payload in each image's `Deploy/Applications`, and the links images keep there, into `site-content/`. It also refreshes `bios-staging/` (BIOS files and `models.txt`), `boot-files/boot.wim` and the Blancco files in `boot-tools/blancco/` from the server. It writes `site-content/SHA256SUMS`, which the playbook checks before restoring anything, and refuses content a FAT32 USB cannot hold. Without it a new server cannot build Standard or Engineer (`build-usb.sh` refuses unless you pass `--skip-site-content`).

The first MCL image import on the new server then seeds `_shared/Applications/BPRT`, adding our overlay scripts from the repo.

### Step 3: Build the USB

```bash
# Basic: server only (import WinPE images later). site-vars.yml holds the
# two NIC MACs (copy playbook/site-vars.example.yml); --site-vars or
# --no-site-vars is required.
sudo ./scripts/build-usb.sh --site-vars site-vars.yml /dev/sdX /path/to/ubuntu-24.04-live-server-amd64.iso

# With WinPE images bundled (single USB, larger drive needed; no file may
# be 4 GiB or more, FAT32)
sudo ./scripts/build-usb.sh --site-vars site-vars.yml /dev/sdX /path/to/ubuntu-24.04.iso /path/to/winpe-images

# Same layout into a disk-image file, no root (dd it to a stick later, or
# boot it in a VM: see tests/vm/README.md; on Proxmox: qm importdisk).
# SIZE is e.g. 16G or "auto".
./scripts/build-usb.sh --site-vars site-vars.yml --image pxe-usb.img auto /path/to/ubuntu-24.04.iso
```

Options: `--payload-root DIR` takes offline-packages/, pip-wheels/,
boot-files/, boot-tools/, enrollment/, bios-staging/ and drivers-staging/
from another checkout; `--skip-enrollment`, `--skip-bios`,
`--skip-drivers`, `--skip-boot-tools` leave those out; `--yes` skips the
prompt. The build refuses payloads that do not match their pins: the iPXE
files against the playbook's `ipxe_*_sha256`, each wheel against its own
package's hashes in `webapp/requirements.txt` (no PyPI fallback), and the
ISO against the volume id `offline-packages/iso-volid` was resolved for. The ISO volume label and El Torito load sizes are read from the
ISO, so any 24.04.x point release works.

This creates a bootable USB with:
- **Partition 1:** Ubuntu Server installer
- **Partition 2:** CIDATA (autoinstall config, offline .debs, pip wheels, Ansible playbook, webapp, boot tools)

### Step 4: Boot the Target Machine

1. Insert the USB into the target machine
2. Press F12 (or vendor boot key) and select the USB
3. Ubuntu auto-installs — no interaction needed
4. After reboot, the first-boot script runs the Ansible playbook (the .debs
   were installed by the installer); it pins the two NICs by MAC
5. PXE services (dnsmasq, Apache, Samba) are configured automatically
6. Flask webapp starts on port 9009
7. If the playbook fails, `/var/lib/pxe-firstboot.failed` and a banner on
   the console and at SSH login point at `/var/log/first-boot.log`; first
   boot retries at the next reboot (5 attempts)

### Step 5: Connect to Isolated Network

Both NICs are cabled before the install: the imaging NIC (`pxe_nic_mac`)
to the isolated switch for PXE clients, the corp NIC (`corp_nic_mac`) to
the GE network. Nothing needs to move after the install.

### Step 6: Import WinPE Content (if not bundled in Step 3)

**Option A:** Use the web interface at `http://172.16.9.1:9009` to import from USB.

**Option B:** Manual copy:
```bash
sudo mkdir -p /mnt/usb2
sudo mount /dev/sdb2 /mnt/usb2
sudo cp -r /mnt/usb2/. /srv/samba/winpeapps/gea-standard
sudo umount /mnt/usb2
```

## Web Management Interface

Access at `http://172.16.9.1:9009` from any machine on the isolated network.

| Page              | URL Path    | Purpose                                       |
|-------------------|-------------|-----------------------------------------------|
| Dashboard         | /           | Service status, disk usage, DHCP clients       |
| Image Import      | /import     | Import WinPE images from USB drives            |
| Unattend Editor   | /unattend   | Edit Windows unattend.xml per image type       |
| startnet.cmd      | /startnet   | Edit startnet.cmd inside boot.wim (wimtools)   |
| Clonezilla Backups| /backups    | Upload/download/manage disk backup images      |
| Blancco Reports   | /reports    | View/download drive erasure reports            |
| Image Config      | /images/*/config | Per-image driver, OS, and hardware config |
| Enrollment        | /enrollment | GCCH bulk enrollment package management         |
| Audit Log         | /audit      | Activity history for all write operations      |

## Verification

1. Connect a test workstation to the isolated switch
2. Set Network Boot (PXE) as first boot in BIOS/UEFI
3. Boot — the client should pull an IP from 10.9.100.x
4. iPXE loads, fetches the boot script from port 4433
5. Select an option from the boot menu:
   - **Windows PE**: Boots via wimboot + boot.wim, maps Samba shares, begins deployment
   - **Clonezilla**: Boots Clonezilla Live for disk imaging
   - **Blancco**: Boots Drive Eraser, auto-saves reports to server
   - **Memory test**: not on the menu; Dell PCs have one built in (F12 at power-on, Diagnostics)

## Testing with KVM

```bash
# Download Ubuntu ISO
wget -O ~/Downloads/ubuntu-24.04.3-live-server-amd64.iso \
  https://releases.ubuntu.com/noble/ubuntu-24.04.3-live-server-amd64.iso

# Launch test VM
sudo ./test-vm.sh ~/Downloads/ubuntu-24.04.3-live-server-amd64.iso

# Watch progress (Ctrl+] to detach)
sudo virsh console pxe-test

# After install: ssh pxe@172.16.9.1 / http://172.16.9.1:9009

# Clean up
sudo ./test-vm.sh --destroy
```

## Project Structure

```
pxe-server/
├── autoinstall/
│   ├── user-data                 # Cloud-init autoinstall + first-boot script
│   └── meta-data                 # Cloud-init metadata (required, empty)
├── playbook/
│   ├── pxe_server_setup.yml      # Ansible: dnsmasq, Apache, Samba, iPXE, UFW, webapp
│   ├── inventory.ini             # Ansible inventory
│   ├── startnet.cmd              # WinPE startup script (injected into boot.wim)
│   ├── blancco-init.sh           # Custom initramfs for Blancco PXE boot
│   ├── blancco-preferences.xml   # Blancco auto-report preferences
│   ├── check-bios.cmd            # Pre-imaging BIOS update checker
│   ├── FlatUnattendW10-shopfloor.xml  # Shopfloor unattend.xml
│   └── shopfloor-setup/          # PC type setup scripts (Standard, CMM, Display, etc.)
├── webapp/
│   ├── app.py                    # Flask application (~1600 lines)
│   ├── requirements.txt          # Python deps (flask, lxml)
│   ├── static/                   # CSS, JS, fonts, logo (all bundled offline)
│   └── templates/                # Jinja2 HTML templates (10 pages)
├── docs/
│   └── shopfloor-display-imaging-guide.md  # End-user imaging guide
├── boot-tools/                   # Extracted boot files (gitignored, built by prepare-boot-tools.sh)
│   ├── blancco/                  # Blancco Drive Eraser
│   └── clonezilla/               # Clonezilla Live
├── boot-files/                   # WinPE boot files (boot.wim, wimboot, ipxe.efi, BCD)
├── offline-packages/             # .deb files (gitignored, built by download-packages.sh)
├── pip-wheels/                   # Python wheels (gitignored, built by download-packages.sh)
├── enrollment/                   # PPKGs and run-enrollment.ps1 (gitignored)
├── bios-staging/                 # Dell BIOS update binaries (gitignored)
├── scripts/                      # Build, deploy, and helper scripts
│   ├── build-usb.sh              # Builds the 2-partition installer USB
│   ├── build-proxmox-iso.sh      # UNSUPPORTED (use build-usb.sh --image)
│   ├── prepare-boot-tools.sh     # Extracts/patches boot tools from ISOs
│   ├── download-packages.sh      # Downloads all offline packages
│   ├── download-drivers.py       # Downloads Dell drivers from dell.com
│   ├── deploy-bios.sh            # Pushes BIOS updates to enrollment share
│   ├── capture-site-content.sh   # Pulls site content + fresh payload from a live server
│   ├── sync_hardware_models.py   # Syncs hardware model configs across images
│   ├── Upload-Image.ps1          # Windows: upload MCL cache to PXE via SMB
│   └── Download-Drivers.ps1      # Windows: download hardware drivers from GE CDN
├── config/                       # Site-specific configuration overrides
├── startnet-template.cmd         # startnet.cmd template (synced with playbook copy)
├── README.md                     # Project overview
└── SETUP.md                      # Detailed setup guide (this file)
```

## Image Types

| Image Type           | Domain          | Description             |
|----------------------|-----------------|-------------------------|
| gea-standard         | geaerospace.com | Standard desktop        |
| gea-engineer         | geaerospace.com | Engineering desktop     |
| gea-shopfloor        | geaerospace.com | Shop floor kiosk        |
| ge-standard          | ge.com          | Standard desktop        |
| ge-engineer          | ge.com          | Engineering desktop     |
| ge-shopfloor-lockdown| ge.com          | Shop floor (locked)     |
| ge-shopfloor-mce     | ge.com          | Shop floor (MCE)        |

## Network Configuration

- PXE server static IP: `172.16.9.1/24`
- DHCP range: `172.16.9.10` - `172.16.9.100`
- Lease time: 12 hours
- DNS: `8.8.8.8` (passed to clients, not used by server)
- Firewall: UFW deny-by-default, allow 67/udp 69/udp 80/tcp 445/tcp 4433/tcp 9009/tcp

## Samba Shares

| Share           | Path                       | Purpose                       |
|-----------------|----------------------------|-------------------------------|
| winpeapps       | /srv/samba/winpeapps       | WinPE deployment images       |
| clonezilla      | /srv/samba/clonezilla      | Clonezilla disk backup images |
| blancco-reports | /srv/samba/blancco-reports | Blancco erasure reports (auto)|
| enrollment      | /srv/samba/enrollment      | GCCH bulk enrollment packages |
| image-upload    | /home/pxe/image-upload     | Image upload staging area     |

All shares use guest access for the isolated network.