TorchTPU or Spyre: Choosing Between Two PyTorch Accelerator Backends

Two Accelerator Backends, Two Different Integration Stories #

Teams that want existing PyTorch programs to run on hardware other than the default device face a practical question: how much of the stack must change? The two sources in this comparison answer it from different positions. The TorchTPU announcement is written from the user side, describing how an existing model moves to a new device. The IBM Spyre post is written from the compiler side, describing how Inductor's decisions become a launchable program. This article compares the two on criteria both sources actually address, and it marks the gaps where a source is silent.

The comparison covers only what the two posts state, plus one foundation announcement that summarizes Spyre work. Where a row says a source does not cover a topic, that is a statement about the source, not a claim about the product.

Option 1: TorchTPU for Google TPUs #

According to its source, TorchTPU is an open source collaboration between Google and Meta that introduces a native PyTorch backend for Google TPUs. The post says the effort started at Google in mid 2025, in collaboration with the PyTorch team at Meta. It positions eager execution as a top priority and says existing PyTorch workloads run out of the box with trivial code changes.

The porting step is the most concrete part of the post. The authors give the example of changing .to("cuda") to .to("tpu"). They also note that this one-line change does not produce peak TPU performance, because a model's shapes, sharding and ops may be tuned for GPU architectures, while the model should still run. For distributed code, the post shows a side-by-side table in which the device and process-group backend change, as shown below.

device = torch.device("tpu")
dist.init_process_group(backend="tpu_dist")
model = model.to(device)

The post describes an autoload mechanism: when the TorchTPU wheel is installed on a TPU machine, importing PyTorch enables the path. On the compiler side, it states that torch.compile() is based on Torch Dynamo rather than Lazy Tensor, and that eager mode has a new DeferAndFuse mode. It reports support for about 450 Aten ops, including forward and backward passes for Core ATen ops needed by major deep learning architectures. Custom kernels can be authored through JAX/Pallas and the Helion DSL.

The post reports out-of-box validation on the 5,000 most downloaded Hugging Face models, across LLM, audio, vision and text-encoder families. It also says DTensor-based sharding, multi-host inference with vLLM and SGLang, and training with TorchTitan were validated, with Ray-based orchestration. On distributed primitives, it states that TorchTPU supports DDP, FSDP and DTensor by providing TPU collectives at the lowest level of the PyTorch stack, and that collective ops are lowered to XLA as StableHLO ops rather than through a separate library such as NCCL.

The post also lists limits. It says a TPU device can be accessed by only one user-space process, so RL colocation is not supported out of the box. It also says XLA and the PjRT runtime assume shapes are known at compile time, and that the team is adding bounded dynamism to address this.

• Choose TorchTPU if your target hardware is a Google TPU and you want PyTorch programs to run with the device-swap path the post describes.
• Choose TorchTPU if you need the eager and torch.compile paths together, with the op coverage and distributed APIs the post lists.
• Choose TorchTPU if RL colocation on a single device is not required, since the post says it is not supported out of the box.

Option 2: The IBM Spyre Backend #

The Spyre post describes the middle chapter of a three-part series. Its scope is the backend compiler. In its framing, torch-spyre's Inductor decides how each kernel's work is divided across Spyre's cores and how tensors are placed between on-chip scratchpad and off-chip memory. The backend compiler turns those decisions into code the hardware can run. The contract between them is KTIR (Kernel Tile IR), an MLIR representation.

KTIR is described as a tile-based, block-structured IR built on open-source MLIR. The post says it is designed to carry the mapping Inductor has already chosen, so the backend does not have to rediscover it. Arithmetic, reductions and control flow come from upstream dialects, and a dedicated ktdp dialect adds explicit memories and explicit accesses. A smaller spyreop dialect exposes Spyre intrinsics as ordinary operations. The post states that the work division is settled in the kernel through a grid attribute, and that the backend is expected to honor it.

The post describes the hardware abstraction: a set of cores, each with a compute engine and its own on-chip scratchpad, joined by an on-chip interconnect. It says Spyre has 32 cores on a ring with 2 MB of scratchpad each, backed by off-chip device memory. Data moves in 128-byte sticks that hold 64 fp16 values. The post also states that Spyre is a dataflow machine with decoupled access and execution, so loads, stores and compute run on separate engines that communicate through FIFOs.

The Dataflow Scheduler is the central component the post describes. It consumes KTIR and splits the kernel into programs. It starts from a simple load, compute, store pipeline, routes each transfer through the memory hierarchy given in an architecture description, and expands the pipeline into a nested, double-buffered, multi-instance schedule. The output is represented in DataflowIR. Device-specific stages then produce SpyreCode, a symbolic SpyreBinary plus a job plan. The post says the Spyre runtime prepares this artifact once and launches it many times, filling in tensor addresses and shapes at launch. The post also states that the scheduler and its dialects are open source under Apache-2.0.

For communication between compute tiles, the post lists inter_tile_produce and inter_tile_reduce, with an all-reduce of partial sums given as an example. The foundation announcement adds one figure: it states that IBM's Spyre software stack extends PyTorch's Inductor compiler and native runtime, reducing Spyre-specific compiler code by 7x while maintaining performance parity on tested workloads. The announcement does not state the baseline for that 7x figure, and it also reports that AI-assisted development enabled support for thousands of models with 44 adapters and approximately 10,000 lines of code.

• Choose the Spyre backend if your target is IBM Spyre and you work at the compiler or kernel level, where KTIR, DataflowIR and the scheduler are the relevant interfaces.
• Choose the Spyre backend if you need explicit control over tiling, scratchpad residency and inter-tile communication, which the post presents as expressed in the IR.
• Choose the Spyre backend if you want to read the scheduler and dialect sources, which the post says are Apache-2.0 licensed.

Side-by-Side Criteria #

The table below uses only criteria that at least one source addresses. Cells marked as not covered mean the source in that row does not discuss the topic.

Where the Sources Are Silent #

The two posts answer different questions, so several gaps matter for a decision. The Spyre KTIR post does not describe a user-level porting step, a model-validation list, or a dynamic-shape plan. The TorchTPU post does not describe the Spyre-style tile-level IR or a scratchpad model. Neither post provides performance numbers that can be compared across the two platforms, and the foundation announcement's 7x figure has no stated baseline. Any comparison of speed or cost between these backends would go beyond what these sources state.

The Spyre post also says it covers how decisions become a program, while the Inductor layout and tiling decisions are described in a separate chapter. Readers who want the full frontend story should consult that companion chapter, which this comparison does not cover.