Skip to main content
The vLLM plugin reports per-step composition and request statistics from your vLLM server to the probe. It is how Warpscale observes inference.

​
Before you start

  • vLLM 0.23.x and Python 3.10 or newer.

​
Install the plugin

Install it into the same environment as vLLM.
The plugin declares no dependencies on purpose. vLLM pins an exact, CUDA-matched torch, so declaring vLLM here would let pip swap it and break your GPU environment. The plugin is pure Python and rides whatever vLLM is already installed — which is also why the 0.23.x requirement is on you to satisfy rather than enforced by pip.
See Set up vLLM to turn it on.

​
Upgrade

Check the supported vLLM version before upgrading either one. The plugin versions independently of the probe.

​
Uninstall

Remove the --distributed-executor-backend and --middleware flags from your vllm serve command, and drop the vLLM and OTLP settings from the probe.