vllm 大模型启动缓存相关环境变量 export

你设置的这些环境变量本质上是在把默认缓存目录从/root/.cache改到指定目录。逐个解释环境变量控制内容默认目录未设置时你现在设置后HF_HOMEHugging Face 模型/Tokenizer 缓存/root/.cache/huggingface/hyperai/input/input0/hubVLLM_CACHE_ROOTvLLM 编译缓存/root/.cache/vllm/local_nvme/vllm_cacheFLASHINFER_CACHE_DIRFlashInfer CUDA kernel 编译缓存/root/.cache/flashinfer/local_nvme/flashinfer_cacheTORCHINDUCTOR_CACHE_DIRPyTorch Inductor 编译缓存/tmp/torchinductor_user或用户 cache 目录版本有关/local_nvme/torch_cacheTRITON_CACHE_DIRTriton kernel 缓存~/.triton/cache/local_nvme/triton_cache1. HF_HOME你设置export HF_HOME/hyperai/input/input0/hub原来/root/.cache/huggingface/里面通常/root/.cache/huggingface/ ├── hub/ │ ├── models--xxx │ └── snapshots/ ├── assets/ └── datasets/设置后/hyperai/input/input0/hub/ ├── models--unsloth--Qwen3.6-27B-NVFP4/ │ └── snapshots/ │ └── ccdaab7... │ ├── config.json │ ├── model.safetensors │ └── tokenizer.json你现在这个models--unsloth--Qwen3.6-27B-NVFP4就是 HF cache 格式。2. VLLM_CACHE_ROOT你设置export VLLM_CACHE_ROOT/local_nvme/vllm_cache原来/root/.cache/vllm/例如/root/.cache/vllm/ └── torch_compile_cache/ └── cca9c83bef/ └── rank_0_0/ └── backbone/设置后/local_nvme/vllm_cache/ └── torch_compile_cache/ └── cca9c83bef/ └── rank_0_0/保存torch.compile graphvLLM 编译产物3. FLASHINFER_CACHE_DIR你设置export FLASHINFER_CACHE_DIR/local_nvme/flashinfer_cache原来/root/.cache/flashinfer/你之前看到/root/.cache/flashinfer/0.6.12/120f/设置后/local_nvme/flashinfer_cache/ └── 0.6.12/ └── 120f/ └── cached_ops/ └── fp4_gemm_cutlass_sm120/这里就是你的Qwen3.6-27B-NVFP4启动时编译的FP4 GEMM kernel4. TORCHINDUCTOR_CACHE_DIR你设置export TORCHINDUCTOR_CACHE_DIR/local_nvme/torch_cache原来可能/tmp/torchinductor_root/或者/root/.cache/torch/保存PyTorch 2.xtorch.compile | v TorchDynamo | v Inductor | v 生成CUDA代码例如/local_nvme/torch_cache/ ├── fxgraph/ ├── triton/ └── xxx.so5. TRITON_CACHE_DIR你设置export TRITON_CACHE_DIR/local_nvme/triton_cache原来/root/.triton/cache保存Triton kernel例如attention kernel layernorm kernel matmul kernel结构/local_nvme/triton_cache/ ├── xxx/ │ └── kernel.so你的最终目录结构大概会变成推荐/local_nvme/ | ├── vllm_cache/ │ └── torch_compile_cache/ | ├── flashinfer_cache/ │ └── 0.6.12/ | ├── torch_cache/ │ └── triton_cache/ /hyperai/input/input0/ | └── hub/ └── models--unsloth--Qwen3.6-27B-NVFP4/关系模型 | v HF_HOME ----------------- safetensors权重 启动vLLM | ---- FlashInfer | | | v | FLASHINFER_CACHE_DIR | ---- torch.compile | | | v | VLLM_CACHE_ROOT | ---- Triton | v TRITON_CACHE_DIR针对你现在的RTX PRO 6000 Qwen3.6-27B-NVFP4这个设置是合理的/hyperai/input/input0/hub可以共享模型/local_nvme/*cache每台 GPU 节点独立保存编译产物这样既避免重复下载模型也避免不同 GPU 节点之间污染 CUDA kernel 缓存。