> ## Documentation Index
> Fetch the complete documentation index at: https://docs.warpscale.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# What is Warpscale?

> SRE platform for GPU infrastructure.

Warpscale is observability, automated diagnosis, and operational tooling for the GPUs running your training and inference workloads.

## Why Warpscale

Generic observability tools were not built for GPU clusters. Whether you are training or serving, they show you that something changed — utilization dropped, latency climbed — but not that one GPU out of 512 is leaking memory and will take everything down in eight minutes.

Warpscale names the GPU and rules out the alternatives:

> **rank 12** — memory leak, OOM in \~8 min. Not HBM failure: ECC clean.

That second sentence matters as much as the first. Every diagnosis carries the per-rank values behind it, where the ranks disagreed, and the patterns that were considered and rejected.

## How it works

### What it collects

* **Always-on monitoring.** A probe runs with minimal overhead on every GPU node and collects continuously, so the data is there when a job fails.
* **Every GPU, every node.** Signals are collected per GPU, so a diagnosis can name one GPU out of hundreds.
* **Deep telemetry.** Hardware (clocks, power, temperature, ECC, NVLink, PCIe), the CUDA runtime, NCCL collectives, the host (CPU, memory, disk and network I/O), and the framework — PyTorch for training, vLLM for serving.

### From telemetry to root cause

One path, from the first anomaly to the cause.

| Stage         |          | What happens                                                                                                                                      |
| ------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Telemetry** | collect  | The probe streams signals from the whole stack — GPU hardware, the CUDA runtime, NCCL, the host, and the framework — for every GPU on every node. |
| **Signals**   | distill  | Raw telemetry becomes named per-GPU features: memory pressure, backward-pass duration, sync waste, thermal headroom, link errors.                 |
| **Diagnosis** | diagnose | Anomalous features are matched against known failure patterns. One pattern is chosen, graded for severity, and attributed to specific ranks.      |
| **Issues**    | track    | A confirmed diagnosis becomes a tracked issue with a lifecycle and a running cost in GPU-hours. One fault is one issue, not a hundred alerts.     |

### How it decides

A GPU-specific knowledge graph and a domain model work together to find the root cause. The graph encodes how the stack fails: the failure patterns, the signals each one produces, and the heuristics that tell them apart.

The patterns are named and specific. Some examples include GPU out of memory, straggler ranks, NCCL timeouts, thermal throttling, power capping, SM occupancy collapse, HBM ECC failures, Xid faults, NVLink CRC storms, and GPUs falling off the bus.

### Where it fits

Your nodes ship telemetry outbound to Warpscale. You read diagnoses in the dashboard, or export them over **OpenTelemetry** to the backend you already run.

It connects to the infrastructure you have — Kubernetes and Slurm, PyTorch for training, vLLM for inference.

It does not replace experiment tracking. Weights & Biases and MLflow follow the training itself — loss, metrics, checkpoints. Warpscale monitors the GPUs those runs are using.

## Next steps

<Card title="Quickstart" href="/quickstart">
  From sign-up to your first diagnosed run in a few minutes.
</Card>