cmp170hx_check/README.md

# cmp170hx_check

Single-card checker for **NVIDIA CMP 170HX** cards (PCI IDs `10de:20c2` / `10de:2082`) — run it **one card at a time**: it waits for the card, verifies the PCIe capacitor mod, records PCIe bus health, backs up VBIOS, runs the cmpunlocker verification, and optionally stresses VRAM. Each card gets its own folder + JSON report named by serial, so the folders accumulate as you go through your batch (e.g. 23 cards).

Reference harness (auto-cloned into `harness/` by preflight):

- [170th-Street](https://github.com/amoghmunikote/170th-Street) — community docs, incl. `modifications/pcie-capacitor-mod.md` (the pass criteria: `LnkSta Width x16` after the mod, `x4` when missing)
- [cmpunlocker](https://github.com/amoghmunikote/cmpunlocker) — `verify.sh` is executed **read-only** per card (nothing is installed or patched by this tool)

## Requirements

- Linux x86-64 (tested target: **Ubuntu 26.04**), Python 3.8+
- **sudo privileges** — precondition for `--preflight` package installation and for privileged checks (PCI rescan, link retrain, sysfs ROM, `dmesg`, `verify.sh`)
- `pciutils` (`lspci`, `setpci`), `git` — installed automatically by preflight
- NVIDIA driver with `nvidia-smi` (nvidia-open 610.xx+, see cmpunlocker) for unlock verification — preflight can install it for you (asks first)
- `nvflash` — **bundled in this repo** (`nvflash_5.867_linux/x64/nvflash`), used directly by the script; no system install needed (see [nvflash](#nvflash-bundled-in-this-repo))
- Optional: `vkmark` or `vulkan-tools` for the 5-minute VRAM stress test

## Preflight (setup step)

```bash
python3 check_cards.py --preflight
```

On Ubuntu 26.04 this:

1. **Checks the sudo precondition** — runs `sudo -v` if needed so credentials are cached; if sudo is unavailable it prints the fix and **does not install anything** (exit code 3 if blocking tools are still missing)
2. Installs required packages via `apt-get update && apt-get install`: `git`, `pciutils`, `vulkan-tools`, `vkmark`
3. Offers to install the **NVIDIA driver** (`ubuntu-drivers install`, falling back to the `nvidia-driver` metapackage) when `nvidia-smi` is missing — reboot afterwards
4. Clones the two harness repos into `harness/`
5. Prints a tool status table — the bundled `nvflash` is detected automatically and its path is shown

Skip installation with `--no-install` (no sudo needed at all).

## nvflash (bundled in this repo)

The Linux build of NVIDIA's `nvflash` 5.867 is committed to this repo at
`nvflash_5.867_linux/x64/nvflash` (plus `aarch64`/`ppc64`/`x86` variants) — the script
uses it **directly**, no system-wide installation required.

Binary resolution order:

1. `--nvflash /path/to/nvflash` (explicit override)
2. repo copy: `nvflash_*_linux/<arch>/nvflash`, then `./nvflash`
3. `nvflash` on `PATH` (optional system install)

Preflight reports `nvflash OK` and prints the resolved path.

```bash
python3 check_cards.py --preflight   # shows: nvflash OK / nvflash: <repo>/nvflash_5.867_linux/x64/nvflash
```

Notes:

- nvflash requires root: the script invokes it via `sudo` — cache credentials with `sudo -v` first, or run the script as root.
- Only `--save` (read-only backup) is used; if it fails while the NVIDIA driver is bound, the script automatically falls back to the sysfs ROM method.
- Do **not** use the AUR nvflash package (known broken), and ignore Windows `.exe` builds.

Optional — install it system-wide anyway:

```bash
sudo install -m 755 nvflash_5.867_linux/x64/nvflash /usr/local/bin/nvflash
nvflash --version
sudo nvflash --list        # must show the 170HX
```

Refreshing the bundled copy: download `nvflash_5.867_linux.zip` from
https://www.techpowerup.com/download/nvidia-nvflash/ , verify SHA256
`73ab51e121129059f1377b3bf21a8001f6649d64af2471455499c803a9c80301`, extract and replace
the files (keep the executable bit: `chmod +x`).

## Usage (one card per run)

```bash
python3 check_cards.py               # check the connected card
python3 check_cards.py --preflight   # setup: tools, packages, harness clone
python3 check_cards.py --demo        # self-test with fake hardware (no GPU needed)
python3 check_cards.py --help        # all flags
```

Flow of a normal run:

1. Waits for a CMP 170HX on the bus — prompts you to insert/reseat the card, Enter triggers a PCI rescan (`q` quits)
2. Serial auto-read from nvflash UUID; falls back to a manual label prompt
3. VBIOS backup → `cards/<serial>/bios.rom` (nvflash, sysfs ROM fallback)
4. PCIe checks: link speed/width, AER counters, full config-space dump
5. Link retrain test + full bus rescan test (device must survive both)
6. Kernel log scrape (`dmesg`/`journalctl`) for this BDF: Xid / AER / NVRM lines
7. `nvidia-smi` snapshot + `harness/cmpunlocker/verify.sh` (read-only)
8. **Last step, optional**: `Run 5-min Vulkan VRAM stress test now? (y/N)` — refused by default, `--vulkan always|never` to override
9. Result printed + `report.json` written into the card's folder

Remove the card, plug the next one, repeat — no queue, no indices, folders just pile up in `cards/`.

## Checks & scoring (lenient)

| Result | Criteria |
|--------|----------|
| **FAIL** | no device detected, link down/untrained, device lost after rescan, link failed retrain, active uncorrectable AER errors |
| **WARN** | link width ≠ `x16` (capacitor mod missing/incomplete), Gen1 only (Gen2 unlock patch not active), correctable AER errors, kernel errors / Xid, VBIOS backup failed, cmpunlocker verify not `unlocked-ok`, errors during VRAM stress |
| **PASS** | everything clean |

Exit codes: `0` PASS/WARN · `1` FAIL · `2` no card found · `3` preflight blocking failure.

## Output

```
cards/
└── <serial>/
    ├── bios.rom              # VBIOS backup (+ sha256 in report)
    ├── report.json           # full result: link, AER before/after, retrain,
    │                          #   rescan, dmesg, verify, vulkan, PASS/WARN/FAIL
    ├── lspci.txt             # lspci -vvv (before)
    ├── lspci_after.txt       # lspci -vvv (after retrain/rescan)
    ├── config_space.hex      # lspci -nnvvxxxx full hex dump
    ├── dmesg.log             # kernel lines related to this card
    └── verify_unlocker.log   # cmpunlocker verify.sh output
```

## Flags

| Flag | Meaning |
|------|---------|
| `--preflight` | setup only: sudo precondition + package install + harness clone, then exit |
| `--demo` | run the whole pipeline against fake hardware (self-test) |
| `--nvflash PATH` | explicit nvflash binary |
| `--serial XYZ` | override serial (skips nvflash auto-read) |
| `--vulkan ask\|always\|never` | VRAM test policy (default: ask, last step) |
| `--skip-bios` | skip VBIOS backup |
| `--no-install` | never install packages (skips the sudo precondition) |
| `--no-clone` | don't clone the harness repos |

## Safety

This tool only **reads** card state: `verify.sh` is run without installing anything, no modules are patched, no firmware is flashed. The only writes are local files, PCI link retrain/rescan sysfs triggers, package installation during `--preflight` (explicitly gated behind the sudo precondition), and your own reports.