Strategies for telco infrastructure in an AI world: Part 2

Download Listen

This second edition in our series on AI-driven opportunities for operators analyses the edge computing side of the equation and in particular the feasibility of operators hosting AI inference workloads.

Not the first rodeo

Edge computing is no new venture for operators. The first wave of telco interest in edge computing coincided with the initial roll out of 5G, where commercial and capital-investment synergies between the two – both offer low latency and require internal capabilities of operating distributed, cloud-native workloads – encouraged concurrent rollout. On top of this, edge computing looked like a stepping stone towards operator ambitions to diversify beyond connectivity. As such, many operators were early movers in the market, as illustrated by the graphic below.

Of note, while fixed players, such as Lumen, have also launched edge services, it was the rollout of 5G that truly brought this opportunity to the fore across the industry

Early movers in the telco edge market

*The graphic above is not exhaustive. See our network edge data centre forecast for a comprehensive global view.

Source: STL Partners

For the most part, these services struggled to bear fruit, with some early entrants, such as AT&T and Cox Communications deciding to exit the market or stall any further investment in response to the lack of short-term ROI. While, at STL, we have always argued that the network edge presents a long-term growth opportunity, it is clear that current network edge services have been difficult to monetise. We explore the experience of early movers and the reasons behind this sluggish adoption in our report here.

AI inference is pushing telcos back to the edge

Like an ex-lover they never quite got over, operators are once again turning to the edge as a strategic area of growth – a rekindled romance largely driven by the prospect of hosting AI inference workloads.

While initial model training will remain concentrated in hyperscale data centres (with some exceptions, i.e. where federated learning is employed), it is model inferencing where telcos spy an opportunity. Inferencing refers to the process whereby new data is applied to a trained AI model in order to infer conclusions from the data and enact decisions about what is occurring and how to respond to it.

Logically, it makes sense for inferencing to take place towards the ‘edge’ of the network, nearer to the processes or events it is interacting with, particularly in cases when this interaction needs to occur in near-real time, at ultra-low latencies and where it is too compute intensive to run on-device.

This is not to say that inferencing will solely be the preserve of edge locations, much of it will occur on-device, when connectivity or privacy are paramount, or in more centralised locations, when latency is not critical and cloud economics are desirable.

However, given telcos operate an infrastructure footprint that extends close to the end customer (at central offices or even the RAN) there is scope for operators to develop a platform to support inferencing, and court the workloads that fall between these on-device and hyperscale buckets. In addition, operators can leverage their position as national providers to provide these services with data sovereignty assurances. As illustrated below, several have launched or are trialling services to this very point.

The inferencing opportunity has encouraged new telco edge service launches

Source: STL Partners

• Many of these announcements would not qualify as what we define as “network edge”, which has often been the deployment mechanism of previous telco edge services. Network edge refers to edge data centres deployed within the network that offer local traffic breakout by hosting distributed User Plane Functions on-site. By contrast, many of the sites mentioned above would be considered regional edge sites – carrier-neutral facilities, typically larger in size, that do not offer local traffic breakout.

While some of the operators listed in the above graphic had previously offered edge computing services, i.e. SK Telecom, KDDI and Bell Canada, the inferencing hosting opportunity has provided the impetus for them to expand their edge computing capacity that has stagnated in recent years – in the case of KDDI and Bell Canada neither of them had expanded their edge site capacity since 2022. This drive towards creating a platform to support inferencing workloads falls within the broader AI strategies telcos have adopted to fully capitalise on the opportunities AI yields – for example, edge computing forms one component of SK Telecom’s “AI Pyramid” strategy. These strategies can also include the construction of AI factories; see our recent research on the topic here.

Where the network edge (still) fits

AI is accelerating the adoption of edge computing, yet this demand, at least for the moment, is concentrated around on-premise edge and regional edge. On-premise edge (as illustrated by our edge AI market forecast) is playing a critical role in providing a local and secure environment for AI inference, whereas the regional edge is supporting training and more compute-intensive inference workloads. The picture for network edge is less clear. The majority of AI use cases today do not have stringent network latency requirements so the commercial differentiation the network edge provides in terms of latency remains unclear, especially when considering the premium enterprises must pay to deploy applications in these environments. In addition, these latency advantages can be negligible for generative AI applications, and in particular reasoning models, where network latency forms only a small component of the total end to end latency.

It is when low latency, wide area use cases proliferate that the network edge will have a more defined proposition. This is the case for V2X use cases, where ultra-low latency and a wide infrastructure footprint are essential enablers. Indeed, Verizon has signalled its ambition to capture this opportunity with the recent launch of its Edge Transportation Exchange, leveraging its 19 network edge sites across the USA.

Where operators have a clearer avenue to differentiate their edge services is to leverage their position as domestic players to provide a sovereign infrastructure platform. In addition, given that most operators have access to a distributed footprint of thousands or even tens of thousands of central offices, there is scope to convert some of these sites (many of which are becoming idle following the decommissioning of legacy networks) into network edge data centres – this is by no means a straightforward process, but could provide a widespread platform of sites with a faster go-to-market than greenfield. On this note STL will soon be publishing an ROI analysis to answer this very question.

AI at the base station: a RANtastic idea?

There is also the question of whether the RAN can serve as a platform to support AI inferencing workloads, an idea being driven in large by the AI-RAN Alliance that now encompasses over 80 members, including seven operators. To learn more about the vision of this alliance, we have recently published an interview with Dr. Alex Jinsung Choi, Chair of the AI-RAN Alliance, accessible here. Ultimately, at STL we are sceptical of the operational and commercial mechanisms that would enable operators to generate a meaningful ROI in this endeavour. Indeed, Nvidia, who lays at the heart of the AI-RAN alliance, has in recent months signalled that inferencing may more be the preserve of more centralised locations, with the announcement of its hardware stack for cell sites, NVIDIA ARC-Compact, not (at least in its standard format) being equipped to host more intensive AI applications such as computer vision.

What next?

Ultimately, there is significant uncertainty as to how this AI era will unfold in terms of scale, time span, and location in the device to cloud continuum. What there is no doubt about, however, is that AI heralds a shift in how society functions, and opens up significant opportunity, and risk, for telcos as providers of critical connectivity infrastructure. This article examined the opportunity for operators to support inferencing workloads at network edge sites, but this forms just one element of what is a significant opportunity presented in this domain.

This article is the second in a three‑part series on the impact of AI on telco networks. The first instalment, focused on AI-driven connectivity opportunities is accessible here. The next instalment will explore how operators should approach sovereign AI and GPUaaS in this new AI-era.