PyTorch 2.15 and ExecuTorch 1.3.1: NCCL Backend Now Default

Key Changes #

PyTorch 2.15: Rewritten NCCL Backend Becomes the Default #

Rewritten NCCL Backend and Its Rollback Paths #

The PyTorch 2.15 Release Blog says 2.14 previewed the rewritten backend as the opt-in nccl2 backend. In 2.15 it is the default for every nccl process group. Either TORCH_DIST_USE_NCCL2=0 or the nccl-legacy backend name restores the legacy ProcessGroupNCCL, so a job needs only one of the two.

export TORCH_DIST_USE_NCCL2=0

Communicators are now created eagerly with the process group. Reconfiguration can opt into a shrink/grow fast path, which requires NCCL 2.29 or newer, so surviving ranks skip a full bootstrap. Reconfiguration for fault tolerance is also enabled on ROCm, with RCCL 2.30.7. Check the NCCL version in your job image before you request that path.

The backend-agnostic flight recorder writes a post-mortem to disk when a collective hangs. Watchdog health is published to the debug server, so a timeout on any rank triggers flight-recorder dumps from every rank. The release blog describes the recorder as capturing what each rank was doing at the moment of the hang. This suggests that a stall can be diagnosed from every rank's dump rather than only from the rank that timed out.

Python 3.10 and macOS 14 Removals #

Python 3.10 is no longer supported: 3.10 wheels are no longer built, 3.10 is no longer tested in CI, and building from source on 3.10 is not supported. PyTorch 2.15 supports Python 3.11 through 3.15, including the free-threaded 3.14t and 3.15t builds. Upcoming releases will remove the remaining 3.10 code paths, including in torch.compile (Dynamo), so 3.10 environments need a move before those removals arrive.

MPS now requires macOS 15 or newer, because macOS 14 has reached end of life and its MPS workarounds have been removed. PyTorch may still run on macOS 14, but it is not tested there, and issues on it will not be fixed. Apple Silicon hosts on macOS 14 should be moved to macOS 15 before the upgrade.

C++ Warnings, Extension Checks and Stable ABI #

C++ warnings from torch.ops and custom ops now propagate to Python. Extensions get a compile-time error if TORCH_TARGET_VERSION is newer than their libtorch headers, so check that target version against the headers each build uses.

torch::stable::sort joins the stable ABI, and parts of c10/util move into torch/headeronly with the old paths kept as forwarding includes. load_inline can generate pybind11 modules that are safe to import on free-threaded Python.

FSDP2 Gradient Dtypes and Precision Controls #

FSDP2 separates sharded and unsharded gradient dtypes, which changes the default behavior when reduce_dtype=None. Gradients now accumulate directly in .grad and follow each parameter's grad_dtype. The release blog does not spell out the new default, so compare the gradient dtype your configuration produces before and after the upgrade.

Precision-sensitive parameters, such as MoE router gates and norm weights, can stay in fp32 while the rest of the model trains in bf16, still with one all-gather per fully_shard through MixedPrecisionPolicy(param_dtype_override_fn=...). A full gradient-accumulation step can be captured in a single CUDA graph, and FSDP reduce-grads can overlap with subsequent compute.

Pipeline Parallelism Memory and Send Budgets #

Pipeline parallelism now allocates receive buffers just before each receive and releases forward outputs once their sends complete. Schedules accept a max_outstanding_sends budget, FSDP unshards are staggered across each rank's startup bubble, and each unique edge between pipeline-parallel ranks gets a dedicated P2P communicator. The release blog states that these changes make pipelining cudagraphable and deterministic, with an optimized memory profile.

CUDA 13.2 Default Wheel and ROCm 10.0 Packaging #

The default PyTorch wheel moves to CUDA 13.2 (13.2.2 toolkit). CUDA 13.4 GA wheels ship alongside it for Linux x86_64, Linux aarch64 and Windows, with experimental support for NVIDIA Rubin GPUs. Check which CUDA wheel your install command selects.

ROCm 10.0 wheels sit alongside the ROCm 7.14 wheels introduced in 2.14. All ROCm wheels are now built on AMD's TheRock packages and install ROCm as a pip dependency, the matching rocm package, instead of bundling the ROCm libraries.

pip show rocm

NVGEMM and Apple Silicon Kernel Changes #

NVGEMM folds the scalar dequantization factor of ModelOpt NVFP4 linear layers into the GEMM, which removes a separate pointwise launch per layer during decode. NVGEMM kernels can also use programmatic dependent launch, so a Triton consumer overlaps with the tail of the producer GEMM. The NVGEMM API is marked API Unstable.

torch.float8_e4m3fn is now supported on MPS, along with _scaled_mm using tensorwise or rowwise scaling and a gemv fast path for decode-sized shapes. MPS also gains a public stream API modeled on CUDA's, including torch.mps.Stream and Tensor.record_stream. Each performance figure below belongs to its own setup: the MoE decode gain is 1.32–1.34x on an M3 Ultra, the F.linear gain is 1.23x geometric-mean with up to 5x on small projections, and the 512 x 512 QR drops from about 7 seconds to 1.5 ms.

Also in This Release #

• FlyDSL kernels on gfx950 (optional, requires FlyDSL 0.3.x) and Tensor Data Mover support for gfx1250 (MI450, preview), with gfx1250 added to the nightly wheel architecture list.
• Intel XPU: SYCL free-function kernel support, IPC for torch.xpu.Event, and torch.xpu.MemPool with use_on_oom and non_split.
• Intel XPU profiling: XPU Graph replay visibility, cross-rank correlation for XCCL collectives, and per-kernel hardware counters.
• Linear algebra: several solvers were rewritten, and triangular solves are now blocked.
• Complete list of changes and contributors: PyTorch 2.15 Release Blog.

ExecuTorch 1.3.1: SME2 Builds Cut Streaming ASR Processing Time on Pixel 11 #

Chunk Size and Precision Set Streaming WER #

The ExecuTorch streaming ASR post measured host WER with the ExecuTorch runtime and XNNPACK CPU backend, over 14,019 utterances from four corpora. From 80 ms to 1,120 ms chunks, FP16 WER falls from 15.2% to 11.5%, and QC8W falls from 15.5% to 11.7%. FP16 differs from FP32 by less than 0.03 percentage points at every chunk size, while QC8W adds 0.21–0.31 percentage points.

accuracy must be measured at the chunk cadence used in deployment.

ExecuTorch streaming ASR post

The post also cites a cross-model study in which Qwen3-ASR-1.7B's WER rose by 77% at 2.4-second chunks. In that study's streaming comparison, NVIDIA Nemotron Speech Streaming 0.6B had the lowest average streaming WER over 560 ms chunks. These WER figures come from host runs, so they do not measure on-device latency.

SME2 on a Pixel 11 with One Inference Thread #

The post processed a 24.9-second recording on a Google Pixel 11 with 1,120 ms chunks and one inference thread pinned to a mid-core CPU. It compared two ExecuTorch 1.3.1 builds with SME2 toggled on and off, both with XNNPACK and KleidiAI and identical model artifacts. SME2 cuts FP16 processing from 9.79 to 4.61 seconds (2.12x faster), and QC8W from 5.20 to 3.24 seconds (1.61x faster). With SME2 on, the real-time factors are 0.19 for FP16 and 0.13 for QC8W.

The post notes that audio still has to arrive before it can be decoded, so this headroom does not remove the chunk-collection delay. It leaves capacity for other CPU work between chunks, which the post also links to managing thermal load during sustained transcription.

Where the Time Goes: GEMM and Data Movement #

ETDump times, summed across the encoder, predictor and joiner, show GEMM time falling from 8.2 to 3.1 seconds for FP16 and from 4.1 to 2.0 seconds for QC8W when SME2 is enabled. GEMM's share of processing time drops from 84% to 67% for FP16 and from 78% to 62% for QC8W. Data movement takes about the same time in both builds, so its share rises from 8% to 17% for FP16 and from 9% to 17% for QC8W. The post names this remaining data movement as a target for further optimization.

Which Native Kernels Ran on Pixel 11 #

Arm Streamline sampled native call paths and confirmed SME2-targeted KleidiAI GEMM kernels for both precisions. The FP16 build still runs some GEMM operations on NEON, and the QC8W build still runs some in FP32. The measured SME utilization of 38.4% for FP16 and 53.7% for QC8W reports how much of each capture the matrix hardware was allocated. The post says it is not a percentage of peak arithmetic throughput.

Choosing a Chunk Size for a Latency Budget #

The post pairs host WER with a Pixel 11 component benchmark that used synthetic input, fresh stream state and 75 measured samples per configuration. Its first-token estimate is conditional: audio collection time plus the median encoder–predictor–joiner sequence time, assuming the first output emits text. For the illustrative 13% WER ceiling, 320 ms chunks give 12.67% for FP16 and 12.97% for QC8W.

For the illustrative 800 ms latency budget, 560 ms chunks give FP16 12.01% WER with an estimated 691 ms delay, and QC8W 12.26% with 660 ms. Both 80 ms configurations miss the WER target, and both 1,120 ms configurations miss the latency budget. These measurements cover one device, Pixel 11.

Upgrade Checklist #

1. Before upgrading, confirm that each training and inference environment runs Python 3.11 through 3.15, because 3.10 wheels are no longer built.
2. Before upgrading, move Apple Silicon hosts that use MPS from macOS 14 to macOS 15 or newer.
3. Before upgrading, search FSDP2 configurations for reduce_dtype=None and record the gradient dtypes they produce.
4. Before upgrading, check that TORCH_TARGET_VERSION does not exceed the libtorch headers used by your extensions.
5. Before upgrading, run custom ops and note the C++ warnings that now propagate to Python.
6. During the upgrade, choose the CUDA 13.2 default wheel or the CUDA 13.4 wheel, and on ROCm run pip show rocm to confirm the matching package.
7. During the upgrade, for jobs that must keep the legacy backend, set TORCH_DIST_USE_NCCL2=0 or use the nccl-legacy backend name.
8. After the upgrade, confirm NCCL 2.29 or newer for any job that requests the shrink/grow path, or RCCL 2.30.7 on ROCm.
9. After the upgrade, run a multi-rank job and confirm that flight-recorder post-mortems are written when a collective hangs.
10. Before choosing an ExecuTorch chunk size, measure WER at the chunk size you deploy, since whole-recording accuracy does not predict streaming quality.