GPU-as-a-Service: What it is, Trends and Leading Providers

Download Listen

As enterprises worldwide seek to harness the potential of AI, the demand for powerful GPU chips continues to grow. GPU-as-a-Service (GPUaaS) solutions offer a cost-effective and scalable way for businesses to access this high-performance hardware without heavy upfront investment. In this article, we explore what GPUaaS is, how it works, and the key trends shaping this emerging market.

What is GPUaaS? A comprehensive overview of GPU-as-a-Service

The potential of AI has captured the excitement of consumers and enterprises globally. However, in order to train and run AI models you need access to graphics processing units (GPUs) which are more powerful than central processing units (CPUs) which have underpinned most computing applications in the past. GPUs are extremely adept at handling large, complex computations that AI and machine learning models require, including a greater ability to process parallel tasks, handle massive datasets and complete exceptionally complex operations.

This extra computing power does come at a cost: GPUs are more power hungry and more expensive than CPUs. In fact, running AI workloads bring significant challenges:

• Increased power and therefore cooling requirements
• Changes to space usage in data centres due to densification
• Need for new, bigger, stronger racks to support greater volume of equipment
• dMisalignment in equipment lifecycles

You can read more detail on the challenges facing data centres or customers wishing to host their own AI workloads in our recent article ‘How does AI impact data centres?’. A high-end GPU will also cost up to about 3x more than a high-end CPU, with significant extra maintenance and energy costs on top. While AI is still relatively nascent for many industries and enterprises, it is hard to predict the applications and use of AI in future and this complexity and cost is further increased when needing to scale applications.

As a result, many enterprises who will be wanting to run their own AI applications will likely be looking for cost-effective and scalable ways to do this, and the answer for many is likely to be GPUaaS. In the 2000s, Software-as-a-Service (SaaS) models gave wide access to powerful computing resources without a need for hardware investment or internal expertise, and GPUaaS will do the same. Being able to access a pay-as-you-go model which can easily scale up or down significantly reduces barriers to entry. We are therefore expecting GPUaaS offerings to play a key role in the proliferation of AI over the coming years.

Drive success in the expanding data centre landscape

Navigate the rapidly growing data centre market with our expert insights. Contact us to explore how we can support your growth.

Book a call with the Data Centres team

GPU-as-a-Service Providers: Leading companies offering GPUaaS

There are already many different GPUaaS providers in the market, and they can be broken down into several key categories for AI:

Hyperscale cloud providers

IBM (IBM Cloud, Research), Google (DeepMind, Google Cloud), AWS (Thinkbox), Azure

The hyperscale cloud providers have a wide range of general compute offerings which increasingly includes GPUaaS, alongside their storage and networking offerings. They have the resources and innovation to be very quick to market with new solutions and are already the earliest adopters of AI as some of the only companies globally with the scale and capabilities to train AI models. Some hyperscalers also have offerings tailored to specific customers or use cases such as IBM Research and Google DeepMind which are primarily for research purposes, and AWS Thinkbox which is for rendering.

Data centre operators

We are also seeing GPUaaS offerings from data centre operators. It is a logical step for large data centre operators to expand their offering vertically and offer GPUaaS, capturing more of the revenue opportunity when it comes to AI. Equinix offers direct access to NVIDIA DGX infrastructure to run AI in the private cloud, but for other large colo providers they may only offer access to GPUs through interconnection with hyperscaler data centres.

We expect to see an increase in vertically-integrated AI platforms from data centre operators, particularly those with access to renewable energy who are able to manage the high power requirements in a sustainable and cost-effective way. Verne provides GPUaaS from its energy-efficient data centres in Iceland and Finland, and Nscale provides services from its 60MW site in Norway.

Hardware providers

NVIDIA has a number of products as well as the GPU chips themselves, their DGX Cloud solution they describe as the world’s first ‘AI supercomputer in the cloud’ and they also have vGPU offerings. Graphcore also offer a number of services which allow access to their own IPUs (intelligence processing units) via the cloud. Access to GPUs (or IPUs) can be tricky to manage as lead times can be long and as the semiconductor industry finds itself at the heart of geopolitical rumblings. The manufacturers themselves are therefore at some advantage over competitors or at least a vital partner for other parties wishing to launch an offering.

Startups specialising in compute for AI

There are a number of start ups who are trying to capitalise on the AI opportunity by offering GPUaaS. Coreweave (who count NVIDIA as an early investor) have raised over $12 billion and have a hyperscale cloud offering designed for AI and high performance compute. Lambda Labs is a competitor with their GPU cloud giving quick access to AI capabilities and is similarly backed by NVIDIA.

There are other competitors such as vast.ai and run:ai (acquired by NVIDIA). We are expecting a rush of start ups trying to capture some of this opportunity and it remains to be seen which are able to achieve lasting-power in the market. It is vital for providers to work closely with the value chain, whether the colocation data centre or the chip manufacturers, so as to reduce vulnerability to energy price increases or long lead times for GPUs. We may see that those who survive are those who are most closely aligned with these other players, or indeed those who have a vertically-integrated offering and own and operate their own data centres.

GPU-as-a-Service market trends

The GPUaaS market is in its early phases. While AI hype is widespread, most enterprises are still figuring out what the use cases will be for them. It will only be once the AI market has further matured that we will really begin to see the demand for GPUaaS develop. It will be interesting to see which providers win out: will the hyperscalers dominate or is there room in the market for start-ups or new entrants to as-a-service models like colocation providers?

Matt Bamforth

Author

Matt Bamforth

Senior Consultant

Matt is a Senior Consultant at STL and has experience in consulting projects across a wide range of topics. These span areas such as 5G, private networks, telco cloud, and edge computing. Matt has previous experience in strategy consulting, as well as in the Fintech sector. He holds a BSc in Economics from University College London.

Are you looking for advisory services in data centres?

Data centre trends: Reflecting on 2024 & predicting 2025

Discover the latest data centre trends in AI, government intervention, and connectivity in 2024, plus our 2025 predictions for each.

How must data centre connectivity adapt to AI demands

Data centre connectivity is evolving to meet AI demands. Discover how networking tech is adapting to support growing data and infrastructure needs.

This year in data centres: The global push for scale and sustainability

A recap of some of the stories that we found interesting in the data centre world in 2024.