Edge computing infrastructure for AI workloads

AI has undoubtedly taken the spotlight this year, with its popularity steadily on the rise. However, rapid advancements in AI pose new requirements for edge computing infrastructure due to the increased demand for dense, compute-intensive infrastructure.

Following the introduction of ChatGPT, AI is now centre stage. The versatility of AI has been a crucial factor in its success, ranging from simplifying everyday manual tasks to serving industrial applications and beyond. Some AI applications come with specific requirements, including ultra-low latency, substantial compute power for model training, and large-scale data collection. Edge computing, with its ability to serve these needs, offers a compelling solution for meeting the infrastructure needs of some AI workloads.

What is the role of edge computing in AI?

Edge AI – a term for the use of edge infrastructure for AI development and deployment – possesses the ability to provide substantial speed and reliability improvements, low latency for mission-critical applications, and cost-effective solutions. These factors can be crucial for the delivery and training of AI models.

See how STL can help you capitalise on the edge computing opportunity

Develop a rich understanding of the edge computing opportunity with our research service. Book your demo today

Book a demo

The future of edge AI infrastructure: GPUs versus CPUs

The debate over GPUs versus CPUs is one crucial aspect of this landscape. CPUs, most commonly used in edge infrastructure deployments today and known for their suitability across a multitude of use cases and compute requirements. However, CPUs may struggle to handle heavier workloads, especially in the demanding field of AI, prompting discussions about a potential shift towards increased GPU usage.

GPUs are gaining traction because of their ability to handle intensive workloads, especially in applications like computer vision-driven video analytics. They also excel in the deep training of AI models, a crucial characteristic for the development of AI. GPUs, come at a higher cost compared to general-purpose CPUs, making them challenging to use at scale. Considered and strategic planning will be required from those looking to deploy GPUs at the edge to try and ensure that these expensive assets do not end up being under-utilised.

Figure 1: Advantages and disadvantages of CPU versus GPUs


With both CPUs and GPUs having different advantages, the choice between CPUs and GPUs will depend on the specific application requirements in the context of edge AI. For instance, for tasks that require lower power consumption, and versatility, CPUs may be a better choice due to their cost-effectiveness. However, if an application demands high-performance AI processing, especially for deep learning models, GPUs are likely the preferred option.

The question of whether GPUs will replace CPUs as the preferred infrastructure at the edge was raised in a recent survey by STL Partners. In the survey, 43% of the respondents thought GPUs would be used for AI/ML workloads at the edge, while 39% chose CPUs for these tasks, showing that the market remains split.

This perhaps should not come across as a surprise, partly because of the advancements made by large chipset players, notably Intel, who have produced AI-specific CPUs that can operate at the edge and handle high compute power. These developments begin to blur the fundamental differences between CPU and GPU in the first place.

To read more about STL’s analysis of the role of edge computing in enabling AI workloads, see Edge AI – How AI is sparking the adoption of edge computing.

Ela Eren


Ela Eren


Ela Eren is a Consultant at STL Partners, specialising in sustainability and telco cloud.

Edge computing market overview

Edge computing market overview

This 33-page document will provide you with a summary of our insights from our edge computing research and consulting work:

100 Edge computing companies to watch in 2024

As always, this list includes a range of companies, from start-ups to those established in the ecosystem. This year, we’ve asked companies to provide more information on their key proposition within the edge computing market, as well as who their customers are.

$8 billion invested in edge since 2020

This article summarises the key findings of the report, “Analysing Edge Computing M&A Trends,” which provides a comprehensive overview of the edge investments market over the last four years. Despite a slowdown in broader technology markets in 2023, investments in edge computing have shown remarkable resilience. This resilience underscores the sector’s strategic significance and its potential for continued growth.

Edge computing news 2023

Edge computing is a rapidly growing industry and evolving ecosystem. This article summarises the most important edge computing news and announcements from 2023. It captures information on M&A, partnerships, product announcements and other strategic plays.