MLPerf Benchmarks

The NVIDIA AI platform showcases leading performance and versatility in MLPerf Training, Inference, and HPC for the most demanding, real-world AI workloads.

What Is MLPerf?

MLPerf™ benchmarks—developed by MLCommons, a consortium of AI leaders from academia, research labs, and industry— are designed to provide unbiased evaluations of training and inference performance for hardware, software, and services. They’re all conducted under prescribed conditions. To stay on the cutting edge of industry trends, MLPerf continues to evolve, holding new tests at regular intervals and adding new workloads that represent the state of the art in AI.

Chalmers University is one of the leading research institutions in Sweden, specializing in multiple areas from nanotechnology to climate studies. As we incorporate AI to advance our research endeavors, we find that the MLPerf benchmark provides a transparent apples-to-apples comparison across multiple AI platforms to showcase actual performance in diverse real-world use cases.

— Chalmers University of Technology, Sweden

TSMC is driving the cutting edge of global semiconductor manufacturing, like our latest 5nm node which leads the market in process technology. Innovations like machine learning based lithography and etch modeling dramatically improve our optical proximity correction (OPC) and etch simulation accuracy. To fully realize the potential of machine learning in model training and inference, we are working with the NVIDIA engineering team to port our Maxwell simulation and inverse lithography technology (ILT) engine to GPUs and see very significant speedups. The MLPerf benchmark is an important factor in our decision making.

— Dr. Danping Peng, Director, OPC Department, TSMC, San Jose, CA, USA

Computer vision and imaging are at the core of AI research, driving scientific discovery and readily representing core components of medical care. We've worked closely with NVIDIA to bring innovations like 3DUNet to the healthcare market. Industry-standard MLPerf benchmarks provide relevant performance data to the benefit of IT organizations and developers to get the right solution to accelerate their specific projects and applications.

— Prof. Dr. Klaus Maier-Hein, Head of Medical Image Computing, Deutsches Krebsforschungszentrum (DKFZ, German Cancer Research Center)

As the preeminent leader in research and manufacturing, Samsung uses AI to dramatically boost product performance and manufacturing productivity. Productizing these AI advances requires us to have the best computing platform available. The MLPerf benchmark streamlines our selection process by providing us with an open, direct evaluation method to assess uniformly across platforms.

— Samsung Electronics

MLPerf Submission Categories

MLPerf Training v3.0 is the eighth edition for training and tested eight different workloads across a diversity of use cases, including computer vision, large language models, and recommenders.

MLPerf Inference v3.1 is the eighth instantiation for inference and tested seven different use cases across seven different kinds of neural networks. Three of these use cases were for computer vision, one was for recommender systems, two were for language processing—including a new large language model test—and one was for medical imaging.

MLPerf HPC v2.0 is the third iteration for HPC and tested three different scientific computing use cases, including climate atmospheric river identification, cosmology parameter prediction, and quantum molecular modeling.

Image Classification

Image Classification

Assigns a label from a fixed set of categories to an input image, i.e., applies to computer vision problems. details.

Object Detection (Lightweight)

Object Detection (Lightweight)

Finds instances of real-world objects such as faces, bicycles, and buildings in images or videos and specifies a bounding box around each. details.

Object Detection (Heavyweight)

Object Detection (Heavyweight)

Detects distinct objects of interest appearing in an image and identifies a pixel mask for each. details.

Biomedical Image Segmentation

Biomedical Image Segmentation

Performs volumetric segmentation of dense 3D images for medical use cases. details.

Translation (Recurrent)

Translation (Recurrent)

Translates text from one language to another using a recurrent neural network (RNN). details.

Automatic Speech Recognition (ASR)

Automatic Speech Recognition (ASR)

Recognizes and transcribes audio in real time. details.

Natural Language Processing (NLP)

Natural Language Processing (NLP)

Understands text by using the relationship between different words in a block of text. Allows for question answering, sentence paraphrasing, and many other language-related use cases. details.



Delivers personalized results in user-facing services such as social media or ecommerce websites by understanding interactions between users and service items, like products or ads. details.

Large Language Model (LLM)

Large Language Models

Deep learning algorithms trained on large-scale datasets that can recognize, summarize, translate, predict, and generate content for a breadth of use cases. details.

Climate Atmospheric River Identification Category

Climate Atmospheric River Identification

Identify hurricanes and atmospheric rivers in climate simulation data. details.

Cosmology Parameter Prediction Category

Cosmology Parameter Prediction

Solve a 3D image regression problem on cosmological data. details.

Quantum Molecular Modeling Category

Quantum Molecular Modeling

Predict energies or molecular configurations. details.

NVIDIA’s MLPerf Benchmark Results

  • Training


  • Inference


  • HPC


The NVIDIA AI platform, powered by the H100 Tensor Core GPU, delivered record-setting performance across all MLPerf Training v3.0 tests, both per accelerator and at scale. This breakthrough performance came from full-stack craftsmanship spanning silicon, software, and system-level technologies. NVIDIA’s investment across the entire stack has pushed the boundaries of AI training performance with each MLPerf submission. The NVIDIA platform, unmatched in overall performance and versatility, is available everywhere—in the data center, at the edge, and in the cloud.

NVIDIA H100 Sets New Per-Accelerator Records on Every Workload

Benchmark Per-Accelerator Records
(NVIDIA H100 Tensor Core GPU)
Large Language Model (GPT-3 175B) 548 hours (23 days)
Natural Language Processing (BERT) 0.71 hours
Recommendation (DLRM-DCNv2) 0.56 hours
Speech Recognition (RNN-T) 2.2 hours
Image Classification (ResNet-50 v1.5) 1.8 hours
Object Detection, Heavyweight (Mask R-CNN) 2.6 hours
Object Detection, Lightweight (RetinaNet) 4.9 hours
Image Segmentation (3D U-Net) 1.6 hours

NVIDIA AI and H100 Tensor Core GPUs Set New Records at Scale

The NVIDIA AI platform, powered by NVIDIA H100 Tensor Core GPUs, set new records for AI training at scale in MLPerf Training v3.0 using NVIDIA’s Pre-Eos AI supercomputer as well as in a joint submission with CoreWeave on its HGX H100 infrastructure. This demonstrates the NVIDIA AI platform’s unmatched performance and versatility for deployments both on premises and in the cloud.

Max-Scale Records

Benchmark Max Scale Records
GPT-3 11 minutes
DLRM-DCNv2 1.6 minutes
BERT 0.13 minutes
ResNet-50 v1.5 0.18 minutes
Mask R-CNN 1.5 minutes
RetinaNet 1.5 minutes
3D U-Net 0.82 minutes
RNN-T 1.7 minutes

The NVIDIA H100 Tensor Core GPU powered the highest-throughput systems across every MLPerf Inference v3.1 data center workload and scenario. In its MLPerf debut, the NVIDIA GH200 Grace Hopper™ Superchip ran every workload and extended the exceptional performance of H100. The NVIDIA L4 Tensor Core GPU, optimized to be the most efficient NVIDIA accelerator for mainstream servers, also achieved great results across the board. For energy-efficient edge AI and robotics applications, NVIDIA Jetson AGX Orin™ and Jetson Orin NX continued to demonstrate outstanding system-on-module inference capabilities.

Offline Scenario for Data Center and Edge (Single GPU)

NVIDIA GH200 Grace Hopper Superchip (Inferences/Second) NVIDIA H100 (Inferences/Second) NVIDIA L4 (Inferences/Second) NVIDIA Jetson AGX Orin (Max Inferences/Query) NVIDIA Jetson Orin NX (Max Inferences/Query)
GPT-J (Large Language Model) 13.34 13.29 1.30 N/A N/A
DLRMv2 (Recommender) 49,002 42,856 3,673 N/A* N/A*
BERT (Natural Language Processing)** 8,646 7,878 631 554 195
ResNet-50 v1.5 (Image Classification) 93,198 88,526 12,882 6,424 2,641
RetinaNet (Object Detection) 1,849 1,761 226 149 67
RNN-T (Speech Recognition) 25,975 23,307 3,899 1,170 432
3D U-Net (Medical Imaging) 6.8 6.5 1.07 0.51 0.20

In MLPerf HPC v2.0, the NVIDIA Selene supercomputer, built on the NVIDIA DGX SuperPOD™ reference architecture, demonstrated leading performance across all three workloads spanning climate segmentation, cosmology parameter prediction, and quantum molecular modeling. The NVIDIA AI platform delivered the fastest time to train as measured by the strong scaling metric, as well as the highest throughput as measured by the weak scaling metric. Through full-stack innovation, the NVIDIA AI platform continues to raise the bar to meet the increasing compute demands fueled by the convergence of HPC and AI, delivering up to a 9X performance boost in just two years of MLPerf HPC.

Up to 9X More Performance in Two Years

NVIDIA's Full-Stack Innovation Delivers Continuous Improvements​

MLPerf HPC v2.0 Performance

The Technology Behind the Results

The complexity of AI demands a tight integration between all aspects of the platform. As demonstrated in MLPerf’s benchmarks, the NVIDIA AI platform delivers leadership performance with the world’s most advanced GPU, powerful and scalable interconnect technologies, and cutting-edge software—an end-to-end solution that can be deployed in the data center, in the cloud, or at the edge with amazing results.

Pre-trained models and Optimized Software from NVIDIA NGC

Optimized Software that Accelerates AI Workflows

An essential component of NVIDIA’s platform and MLPerf training and inference results, the NGC™ catalog is a hub for GPU-optimized AI, HPC, and data analytics software that simplifies and accelerates end-to-end workflows. With over 150 enterprise-grade containers—including workloads for generative AI, conversational AI, and recommender systems; hundreds of AI models; and industry-specific SDKs that can be deployed on premises, in the cloud, or at the edge—NGC enables data scientists, researchers, and developers to build best-in-class solutions, gather insights, and deliver business value faster than ever.

Leadership-Class AI Infrastructure

Achieving world-leading results across training and inference requires infrastructure that’s purpose-built for the world’s most complex AI challenges. The NVIDIA AI platform delivered leading performance powered by the NVIDIA GH200 Grace Hopper Superchip, the NVIDIA H100 Tensor Core GPU, the NVIDIA L4 Tensor Core GPU, and the scalability and flexibility of NVIDIA interconnect technologies—NVIDIA NVLink®, NVSwitch™, and Quantum-2 InfiniBand. These are at the heart of the NVIDIA data center platform, the engine behind our benchmark performance.

In addition, NVIDIA DGX™ systems offer the scalability, rapid deployment, and incredible compute power that enable every enterprise to build leadership-class AI infrastructure. 

Leadership-Class AI Infrastructure

Learn more about our data center training and inference product performance.