NVIDIA BioIR Delivers 2.90x Boltz-2 Throughput on 8xH100

NVIDIA has published a technical breakdown of BioNeMo Inference Runtime (BioIR), a Python library that accelerates protein structure-prediction models on NVIDIA GPUs without leaving the standard PyTorch workflow. In a matched benchmark on 1,000 human dimer targets run on 8xH100 80GB GPUs, BioIR-accelerated Boltz-2 delivered 58.5K folded residues per GPU-hour versus 20.2K for a torch-compiled open-source Boltz-2, a 2.90x throughput gain. The library is available now as an open GitHub repository with precompiled CUBINs and requires no CUDA toolkit or nvcc to run.

What happened

Detail	Value
Benchmark hardware	8x H100 80GB GPUs
Benchmark targets	1,000 human dimer targets, sequence lengths below 2,800 residues
BioIR throughput	58.5K residues per GPU-hour
OSS Boltz-2 throughput	20.2K residues per GPU-hour
Throughput gain	2.90x
OSS baseline OOM failures	29 of 1,000 targets
Estimated energy, 1M targets (8-GPU TDP)	11 MWh (BioIR) vs 35 MWh (OSS)
AlphaFold DB expansion	31M candidate complexes, 4,777 proteomes, 1.81M high-confidence predictions released
Availability	Open GitHub repo, wheel with precompiled CUBINs, Python 3.12+ required

BioIR targets a specific bottleneck: when organizations need to fold thousands of proteins rather than one at a time, the limiting factor is no longer model accuracy but pipeline throughput. Parsing, featurization, GPU inference, and output writing all compete for time. BioIR is designed to optimize each of those stages without requiring researchers to change how their models are built.

The library already ran at production scale. NVIDIA used it to power the recent AlphaFold Database expansion, generating protein-complex structures across 4,777 proteomes and producing approximately 31 million candidate complexes, of which 1.81 million met the threshold for high-confidence release.

How does BioIR actually speed things up?

Acceleration happens at three distinct layers:

1. Kernel selection: For each supported operation (including Pairformer and Evoformer stacks, triangle operations, pairwise attention, diffusion transformers, and atom-level modules), BioIR picks the fastest compatible implementation from BioIR custom kernels, cuEquivariance, or a PyTorch fallback, based on GPU type, data type, and tensor shape.
2. Module optimization: An optimize() call enables CUDA Graph capture for compatible modules. CUDA Graph capture records a sequence of GPU operations and replays them with reduced launch overhead on each subsequent call.
3. Pipeline scaling via Ray: A Ray executor places one complete model replica on each visible GPU and distributes independent inputs across them. CPU stages (parsing, featurization, output writing) overlap with GPU folding time. One important constraint: Ray does not split a single forward pass across GPUs. It scales worklists of independent targets, not individual large targets. Context-parallel folding across GPUs is planned but not yet available.

Models remain ordinary torch.nn.Module objects throughout. There is no engine export step or separate build artifact between a checkpoint and a forward pass.

Benchmark numbers in detail

Both BioIR and the open-source Boltz-2 baseline used identical targets, staged MSAs (per-chain A3M multiple sequence alignments), and the same inference recipe: 3 recycles, 200 sampling steps, and 5 diffusion samples. The input size ceiling was combined sequence lengths below 2,800 residues.

At the model-forward level, NVIDIA reports geometric-mean speedups over a torch.compile baseline of 1.55x for OpenFold3, 1.78x for Boltz-2, and 2.56x for OpenFold2 monomer on H100. H200 numbers are similar at 1.54x, 1.75x, and 2.61x. These were measured across 17 inputs spanning 29 to 1,734 residues.

The end-to-end benchmark numbers (2.90x, 58.5K vs 20.2K residues per GPU-hour) cover the folding stage only and exclude MSA generation, preprocessing CPU allocations, storage, data transfer, and retries. NVIDIA explicitly warns against generalizing these figures to all BioIR-supported models or datasets.

Energy implications at scale

Extrapolating the benchmark linearly to 1 million comparable targets, NVIDIA estimates BioIR would need 11 MWh versus 35 MWh for the public implementation, using 8-GPU TDP (thermal design power) equivalents. Using full-node maximum-power equivalents, the estimate widens to 21 MWh versus 64 MWh. These are rated-power estimates for IT equipment only and exclude data center overhead such as power usage effectiveness (PUE). A saving in the range of 23 to 43 MWh per million targets is still a meaningful number for any team running proteome-scale campaigns.

What you need to run it

• Python 3.12 or higher
• A compatible NVIDIA GPU and driver
• A staged model checkpoint
• Per-chain A3M MSAs (BioIR does not run HHsearch or HMMsearch itself)

No nvcc, CUDA source, CMake, or the CUDA toolkit are required. There are two usage paths: an end-to-end processor that moves an input request through the full pipeline and writes PDB or mmCIF output, or direct PyTorch integration for developers who want to reuse selected optimized modules inside custom code.

Our take

This is a genuine infrastructure improvement, not a benchmark staged to flatter a headline. The fact that the OSS baseline failed on 29 targets due to out-of-memory errors while BioIR completed all 1,000 is arguably more useful than the throughput number. OOM failures at scale are a real operational pain.

The honest caveat from NVIDIA’s own blog is worth repeating: these numbers are folding-stage only, on one specific dataset and one hardware config. Teams working with longer sequences, mixed chain types, or non-dimer targets should run their own benchmarks before sizing infrastructure.

For most businesses, this is relevant if you work in drug discovery, biotech tooling, or any SaaS product built on structural biology APIs. If you are evaluating AI infrastructure costs at scale, the energy delta (11 MWh vs 35 MWh per million targets) is the kind of number worth putting in front of your cloud budget owner. Our work on AI integration for business workflows increasingly involves questions about GPU cost and throughput, and BioIR is a concrete answer to the “how do we make this affordable at scale” problem in the bio space.

If you are watching the broader GPU efficiency story, the context around Nvidia’s revenue trajectory helps explain why tooling like BioIR exists: NVIDIA has a strong incentive to show that its H100 and H200 hardware delivers compounding returns through software, not just raw silicon.

Run your own benchmark on a representative sample of your actual targets before committing to any infrastructure plan.

Source: Marktechpost