qbuissondebon
70a6ff9b1a
AGENTS.md now documents the real workflow: single remote origin (SSH), push only on explicit request, never force-push or add remotes unasked. README gains the Gitea repo link and points the driver requirement at cmpunlocker's driver/VERSION (610.xx / 615.xx).
147 lines
9.4 KiB
Markdown
Executable File
147 lines
9.4 KiB
Markdown
Executable File
# cmp170hx_check
|
||
|
||
Repo: https://git.qbuissondebon.eu.org/qbuissondebon/cmp170hx_check (SSH push: `git@git.ssh.qbuissondebon.eu.org:qbuissondebon/cmp170hx_check.git`)
|
||
|
||
Single-card checker for **NVIDIA CMP 170HX** cards (PCI IDs `10de:20c2` / `10de:2082`) — run it **one card at a time**: it waits for the card, verifies the PCIe capacitor mod, records PCIe bus health, backs up VBIOS, runs the cmpunlocker verification, offers the **opt-in** cmpunlocker unlock install (then a cold reboot and a re-run for the final PCIe check), and optionally stresses VRAM. Each card gets its own folder + JSON report named by serial, so the folders accumulate as you go through your batch (e.g. 23 cards).
|
||
|
||
Reference harness (auto-cloned into `harness/` by preflight):
|
||
|
||
- [170th-Street](https://github.com/amoghmunikote/170th-Street) — community docs, incl. `modifications/pcie-capacitor-mod.md` (the pass criteria: `LnkSta Width x16` after the mod, `x4` when missing)
|
||
- [cmpunlocker](https://github.com/amoghmunikote/cmpunlocker) — `verify.sh` is executed **read-only** per card; `install.sh` is only run from the separate, opt-in unlock step (see [Safety](#safety))
|
||
|
||
## Requirements
|
||
|
||
- Linux x86-64 (tested target: **Ubuntu 26.04**), Python 3.8+
|
||
- **sudo privileges** — precondition for `--preflight` package installation, for privileged checks (PCI rescan, link retrain, sysfs ROM, `dmesg`, `verify.sh`), and for the opt-in unlock install
|
||
- `pciutils` (`lspci`, `setpci`), `git` — installed automatically by preflight
|
||
- NVIDIA driver with `nvidia-smi` (nvidia-open, version list in cmpunlocker `driver/VERSION` — 610.xx / 615.xx) for unlock verification — preflight can install it for you (asks first)
|
||
- `nvflash` — **bundled in this repo** (`nvflash_5.867_linux/x64/nvflash`), used directly by the script; no system install needed (see [nvflash](#nvflash-bundled-in-this-repo))
|
||
- Optional: `vkmark` or `vulkan-tools` for the 5-minute VRAM stress test
|
||
|
||
## Preflight (setup step)
|
||
|
||
```bash
|
||
python3 check_cards.py --preflight
|
||
```
|
||
|
||
On Ubuntu 26.04 this:
|
||
|
||
1. **Checks the sudo precondition** — runs `sudo -v` if needed so credentials are cached; if sudo is unavailable it prints the fix and **does not install anything** (exit code 3 if blocking tools are still missing)
|
||
2. Installs required packages via `apt-get update && apt-get install`: `git`, `pciutils`, `vulkan-tools`, `vkmark`
|
||
3. Offers to install the **NVIDIA driver** (`ubuntu-drivers install`, falling back to the `nvidia-driver` metapackage) when `nvidia-smi` is missing — reboot afterwards
|
||
4. Clones the two harness repos into `harness/`
|
||
5. Prints a tool status table — the bundled `nvflash` is detected automatically and its path is shown
|
||
|
||
Skip installation with `--no-install` (no sudo needed at all).
|
||
|
||
## nvflash (bundled in this repo)
|
||
|
||
The Linux build of NVIDIA's `nvflash` 5.867 is committed to this repo at
|
||
`nvflash_5.867_linux/x64/nvflash` (plus `aarch64`/`ppc64`/`x86` variants) — the script
|
||
uses it **directly**, no system-wide installation required.
|
||
|
||
Binary resolution order:
|
||
|
||
1. `--nvflash /path/to/nvflash` (explicit override)
|
||
2. repo copy: `nvflash_*_linux/<arch>/nvflash`, then `./nvflash`
|
||
3. `nvflash` on `PATH` (optional system install)
|
||
|
||
Preflight reports `nvflash OK` and prints the resolved path.
|
||
|
||
```bash
|
||
python3 check_cards.py --preflight # shows: nvflash OK / nvflash: <repo>/nvflash_5.867_linux/x64/nvflash
|
||
```
|
||
|
||
Notes:
|
||
|
||
- nvflash requires root: the script invokes it via `sudo` — cache credentials with `sudo -v` first, or run the script as root.
|
||
- Only `--save` (read-only backup) is used; if it fails while the NVIDIA driver is bound, the script automatically falls back to the sysfs ROM method.
|
||
- Do **not** use the AUR nvflash package (known broken), and ignore Windows `.exe` builds.
|
||
|
||
Optional — install it system-wide anyway:
|
||
|
||
```bash
|
||
sudo install -m 755 nvflash_5.867_linux/x64/nvflash /usr/local/bin/nvflash
|
||
nvflash --version
|
||
sudo nvflash --list # must show the 170HX
|
||
```
|
||
|
||
Refreshing the bundled copy: download `nvflash_5.867_linux.zip` from
|
||
https://www.techpowerup.com/download/nvidia-nvflash/ , verify SHA256
|
||
`73ab51e121129059f1377b3bf21a8001f6649d64af2471455499c803a9c80301`, extract and replace
|
||
the files (keep the executable bit: `chmod +x`).
|
||
|
||
## Usage (one card per run)
|
||
|
||
```bash
|
||
python3 check_cards.py # check the connected card
|
||
python3 check_cards.py --preflight # setup: tools, packages, harness clone
|
||
python3 check_cards.py --demo # self-test with fake hardware (no GPU needed)
|
||
python3 check_cards.py --unlock # check + run the cmpunlocker install without asking
|
||
python3 check_cards.py --help # all flags
|
||
```
|
||
|
||
Flow of a normal run:
|
||
|
||
1. Waits for a CMP 170HX on the bus — prompts you to insert/reseat the card, Enter triggers a PCI rescan (`q` quits)
|
||
2. Serial auto-read from nvflash UUID; falls back to a manual label prompt
|
||
3. VBIOS backup → `cards/<serial>/bios.rom` (nvflash, sysfs ROM fallback)
|
||
4. PCIe checks: link speed/width, AER counters, full config-space dump
|
||
5. Link retrain test + full bus rescan test (device must survive both)
|
||
6. Kernel log scrape (`dmesg`/`journalctl`) for this BDF: Xid / AER / NVRM lines
|
||
7. `nvidia-smi` snapshot + `harness/cmpunlocker/verify.sh` (read-only)
|
||
8. **Opt-in unlock**: `cmpunlocker unlock (install)` — skipped with a note when memory is already unlocked, otherwise asks `Run cmpunlocker install.sh now? (y/N)` (default **n**; `--unlock` runs it without asking). On yes: `git pull --ff-only` on the harness clone, then `install.sh` runs from `harness/cmpunlocker/` with **live terminal output** (full log → `cards/<serial>/install_unlocker.log`; the kernel-module build takes 5–15 min)
|
||
9. **After a successful install**: the VRAM stress is skipped — **cold reboot the host** (`sudo shutdown -h now`, then power it back on) and **re-run the same command** for the final PCIe check, which must report **Gen2 x16**
|
||
10. **Last step, optional**: `Run 5-min Vulkan VRAM stress test now? (y/N)` — refused by default, `--vulkan always|never` to override
|
||
11. Result printed + `report.json` written into the card's folder
|
||
|
||
Remove the card, plug the next one, repeat — no queue, no indices, folders just pile up in `cards/`.
|
||
|
||
## Checks & scoring (lenient)
|
||
|
||
| Result | Criteria |
|
||
|--------|----------|
|
||
| **FAIL** | no device detected, link down/untrained, device lost after rescan, link failed retrain, active uncorrectable AER errors, **final post-unlock link other than Gen2 x16** (`[ERROR] final PCIe link after unlock is <speed> <width> - expected Gen2 x16`) |
|
||
| **WARN** | link width ≠ `x16` (capacitor mod missing/incomplete, pre-unlock runs only), Gen1 only (Gen2 unlock patch not active, pre-unlock runs only), correctable AER errors, kernel errors / Xid, VBIOS backup failed, cmpunlocker verify not `unlocked-ok`, cmpunlocker install failed, errors during VRAM stress |
|
||
| **PASS** | everything clean |
|
||
|
||
Exit codes: `0` PASS/WARN · `1` FAIL · `2` no card found · `3` preflight blocking failure.
|
||
|
||
## Output
|
||
|
||
```
|
||
cards/
|
||
└── <serial>/
|
||
├── bios.rom # VBIOS backup (+ sha256 in report)
|
||
├── report.json # full result: link, AER before/after, retrain,
|
||
│ # rescan, dmesg, verify, vulkan, PASS/WARN/FAIL
|
||
├── lspci.txt # lspci -vvv (before)
|
||
├── lspci_after.txt # lspci -vvv (after retrain/rescan)
|
||
├── config_space.hex # lspci -nnvvxxxx full hex dump
|
||
├── dmesg.log # kernel lines related to this card
|
||
├── verify_unlocker.log # cmpunlocker verify.sh output
|
||
└── install_unlocker.log # full install.sh output (only if the opt-in
|
||
# unlock step ran)
|
||
```
|
||
|
||
## Flags
|
||
|
||
| Flag | Meaning |
|
||
|------|---------|
|
||
| `--preflight` | setup only: sudo precondition + package install + harness clone, then exit |
|
||
| `--demo` | run the whole pipeline against fake hardware (self-test) |
|
||
| `--nvflash PATH` | explicit nvflash binary |
|
||
| `--serial XYZ` | override serial (skips nvflash auto-read) |
|
||
| `--vulkan ask\|always\|never` | VRAM test policy (default: ask, last step) |
|
||
| `--unlock` | run the cmpunlocker `install.sh` step without prompting (default: ask y/N) |
|
||
| `--skip-bios` | skip VBIOS backup |
|
||
| `--no-install` | never install packages (skips the sudo precondition) |
|
||
| `--no-clone` | don't clone the harness repos |
|
||
|
||
## Safety
|
||
|
||
This tool **reads** card state by default. The one step that changes the card is **opt-in**:
|
||
|
||
- **Unlock install (opt-in, explicit confirmation)** — runs only if you answer `y` to `Run cmpunlocker install.sh now? (y/N)` (default `n`) or pass `--unlock`. It executes `install.sh` **from the harness clone** (`harness/cmpunlocker/`, refreshed first with `git pull --ff-only`, no extra installer flags), needs root (sudo credentials are cached first), streams its output live, and saves the full log to `cards/<serial>/install_unlocker.log`. It is skipped entirely when memory is already unlocked, when the harness is not cloned, and in `--demo` mode. A successful install replaces kernel modules — the step then skips VRAM stress and asks you to cold reboot and re-run.
|
||
- **Everything else remains read-only**: `verify.sh` runs without installing anything, no modules are patched and no firmware is flashed (nvflash is only ever used with `--save`). The only other writes are local files (reports, logs), PCI link retrain/rescan sysfs triggers, and package installation during `--preflight` (explicitly gated behind the sudo precondition).
|