Picking the Best GPU for Computer Vision

Published in

Becoming Human: Artificial Intelligence Magazine

6 min readMay 20, 2021

What is the Best GPU to Use in Computer Vision Projects?

From facial recognition to monitoring crops, machine-learning models are being used to carry out a growing number of computer vision related tasks.

Training these models typically requires a large number of images or videos, which are translated into matrices of values reflecting properties such as pixel color and intensity.

During the process of training a machine learning model, these matrices of values will undergo repeated multiplications, until the output of the model is close to what it should be.

Computer vision has many applications, such as NVIDIA’s experimental DRIVE IX system for monitoring driver attentiveness. Image source: NVIDIA

Today, Graphics Processing Units or GPUs are typically used for training machine learning models, thanks to their ability to excel at carrying out operations on these matrices in parallel.

Although there are two major GPU manufacturers, if you’re looking to train a machine learning model then choosing an NVIDIA GPU is typically the best choice you can make. This is primarily due to the maturity of NVIDIA’s CUDA API for parallel computing, but also the addition of Tensor Cores to NVIDIA cards in recent years.

Tensor Cores are specialized silicon, tailored to handling common tasks in machine learning inference and training, such as matrix multiplications.

Mid-range NVIDIA cards from the past three generations onwards are all suitable for training computer vision models, although users looking to do more than dabble with machine learning should choose a card from the NVIDIA GeForce RTX 20 series onwards (including the recently released, and difficult to find RTX 30 series).=

The most recent NVIDIA GeForce RTX 30 series cards include third-generation Tensor Cores, with double the data throughput of those found in cards from the Nvidia GeForce RTX 20 series.

This is an important upgrade for machine learning, given a significant bottleneck when training machine learning models is transferring data from GPU memory to caches, registers, and shared memory used by the GPU’s streaming multiprocessors and Tensor Cores.

Currently, the RTX 30 Series includes:

RTX 3090
RTX 3080
RTX 3070
RTX 3060

Which Cards Offer the Best Bang for Your Buck?

In testing, the GeForce RTX 3080 struck the best balance between cost and performance, when used for workloads involving computer vision-targeted models, including Convolutional Neural Networks and networks based on Transformer architectures, which are beginning to find a use for computer vision tasks.

The NVIDIA GeForce RTX 3080. Image source: NVIDIA

The 3080 provided the best performance per dollar of a wide range of NVIDIA cards, ranging from mid-to-high end consumer cards in the GeForce 10 series through the GeForce 30 series, and also including NVIDIA’s data center class A100 and V100 tensor core GPUs.

As would be expected, given the RTX 3090’s near $1,000 price premium over the RTX 3080, the 3090 delivered the best performance of the consumer-grade cards, pulling just ahead of the 3080, while the A100 tensor core GPU with its 40GB VRAM and Ampere architecture delivered close to double the performance of the 3080. This isn’t too surprising given the A100’s $12,500 price tag.

The generational leap in performance from the NVIDIA GeForce 20 to 30 series was borne out by further testing. This demonstrated the GeForce RTX 3090 and RTX 3080 series were 50% and 35% faster than the 20 series Titan RTX and RTX 2080 Ti respectively when carrying out inference on TensorFlow 1.15 running FP32 calculations on ResNet50, a neural network used for a wide range of computer vision tasks.

Specs for the NVIDIA GeForce RTX 3080. Image source: NVIDIA

How Much Memory Does Your GPU Need?

A key consideration when choosing a GPU for training a computer vision model is available memory — with experts in the field saying computer vision typically requires at least 10GB of GPU memory — especially with rising VRAM requirements. When working with some state-of-the-art computer vision models and Generative Adversarial Networks (GANs), the requirements can be fairly high.

While the Ampere architecture of the GeForce 30 series has been upgraded to improve the speed at which it can train and run computer vision models, as outlined above, the available memory has been scaled back on several models of cards compared to GeForce 20 series, with the RTX 3080 only including 10GB VRAM.

While the memory is higher on some models of GeForce 20 series cards, the actual throughput of data is lower, with the 3080 using GDDR6X memory, which has double the bandwidth of the GDDR6 VRAM found on the 2080 Ti.

The NVIDIA GeForce RTX 3090. Image source: NVIDIA

For those who need more memory, the NVIDIA GeForce RTX 3090 offers 24GB of memory. However, there is speculation that NVIDIA may be planning to release a GeForce 30 series Titan card at a later stage, which might address some of the concerns over the cards’ double-precision floating point performance.

Other GPU Options for Getting Started with Computer Vision

If you’re looking for a more affordable option, NVIDIA has just released the GeForce RTX 3060 with a lower price, third-generation Tensor Cores and 12GB of GDDR6 memory.

Deep learning workstation with NVIDIA GPUs

If the latest cards are still unaffordable, which may be the case given recent GPU shortages, then a used RTX 2070 or RTX 2060 might be a good option.

If you’d rather buy a machine tailored specifically for computer vision, then check out SabrePC’s own custom-built workstations for carrying out deep learning, which are fitted with the latest NVIDIA GPUs, including the RTX 30 Series.

Don’t forget to give us your 👏 !