Tag: latency

Telco edge computing: Turning vision into practice

The emerging opportunity for edge compute

There is ongoing interest in the telecoms industry about edge computing. The key rationale behind this is that telcos – through their distributed network assets – are in a unique position to push workloads closer to devices, reducing latency and/or data volumes over to the cloud, and thereby enabling new experiences and use cases, while enhancing existing ones.

After years of centralising workloads in the public cloud there is complementary demand emerging for more distributed compute. This is good news for telcos as it shows that the time is ripe for them to turn their ambition to edge computing. Telcos can exploit their own connectivity, unique network APIs and an existing distributed real-estate. Telcos are in a unique position to play a strong role in distributed and edge computing ecosystems.

Telcos’ excitement around edge is fuelled by new differentiation and revenue opportunities leveraging the dynamic application developer ecosystem which hitherto has been dominated by ever more sophisticated and technically advanced public clouds and proofs-of-concept (POCs). Furthermore, underlying trends in cloud computing are increasingly promising for distributed (edge) computing:

  • Hybrid and multi-cloud models and technologies will continue to facilitate more distributed compute scenarios beyond hyperscale-only and on-premise-only.
  • Lightweight compute models will enable the deployment of cloud-workloads on a smaller footprint (e.g. train AI models in the cloud and execute them at the edge, such as in a smartphone or a connected car). For example, containers and “serverless” compute models make it possible to run workloads more efficiently and elastically than virtual machines.
  • The adoption of more platform-agnostic deployment models (such as containers) will facilitate the shifting and moving of workloads within distributed and edge cloud environments.
  • Proliferation of edge gateways and IoT devices will drive processing and analytics outside the datacentre and closer to the customer (premises).
  • Regarding security, a more distributed computing model is well-suited to defending against certain types of attacks (e.g. DDOS). Furthermore, if/when breaches do occur, these can be quarantined to an edge “cloudlet”, limiting the potential damage and undermining the economics of an attack.

Our findings in this report are informed by a research programme STL Partners has conducted since January 2018, supported by and in cooperation with Aricent. For this research, STL Partners has conducted interviews with both telcos and technology companies, globally about their views and current efforts related to edge computing. Overall, the research forms part of STL Partners’ ongoing research work and consulting assignments around telco edge cloud.

Enter your details below to request an extract of the report

var MostRecentReportExtractAccess = “Most_Recent_Report_Extract_Access”;
var AllReportExtractAccess = “All_Report_Extract_Access”;
var formUrl = “https://go.stlpartners.com/l/859343/2022-02-16/dg485”;
var title = encodeURI(document.title);
var pageURL = encodeURI(document.location.href);
document.write(‘‘);

Key questions arising for telcos

Notwithstanding the strategic opportunity, telcos face some big questions in formulating edge initiatives. These include:

“What is the business case for telco edge – where is the money?”

“Will massive demand for low-latency compute drive demand from core/central to edge compute?”

“How can we compete with the big cloud players – won’t they expand and control the edge too?”

“How should we play in Enterprise edge – should we offer edge services on customer premises?”

“How can we architect and charge for different edge services – those requiring expensive, specialised hardware for accelerated computing to process machine learning/AI workloads?”

“What edge services should we offer and through what distribution channels?”

These are (real examples of) questions that telcos must address in defining and delivering edge services. This report provides a framework to tackle these (and other) questions in a structured way. We will revisit these questions (and the answers) throughout the report.

Enter your details below to request an extract of the report

var MostRecentReportExtractAccess = “Most_Recent_Report_Extract_Access”;
var AllReportExtractAccess = “All_Report_Extract_Access”;
var formUrl = “https://go.stlpartners.com/l/859343/2022-02-16/dg485”;
var title = encodeURI(document.title);
var pageURL = encodeURI(document.location.href);
document.write(‘‘);

Edge computing: Five viable telco business models

If you don’t subscribe to our research yet, you can download the free report as part of our sample report series.

This report has been produced independently by STL Partners, in co-operation with Hewlett Packard Enterprise and Intel.

Introduction

The idea behind Multi-Access Edge Computing (MEC) is to make compute and storage capabilities available to customers at the edge of communications networks. This will mean that workloads and applications are closer to customers, potentially enhancing experiences and enabling new services and offers. As we have discussed in our recent report, there is much excitement within telcos around this concept:

  • MEC promises to enable a plethora of vertical and horizontal use cases (e.g. leveraging lowlatency) implying significant commercial opportunities. This is critical as the whole industry is trying to uncover new sources of revenue, ideally where operators may be able to build a sustainable advantage.
  • MEC should also theoretically fit with telcos’ 5G and SDN/NFV deployments, which will run certain virtualised network functions in a distributed way, including at the edge of networks. In turn, MEC potentially benefits from the capabilities of a virtualised network to extract the full potential of distributed computing.

Figure 1: Defining MEC

Source: STL Partners

However, despite the excitement around the potentially transformative impact of MEC on telcos,viable commercial models that leverage MEC are still unclear and undefined. As an added complication, a diverse ecosystem around edge computing is emerging – of which telcos’ MEC is only one part.

From this, the following key questions emerge:

  • Which business models will allow telcos to realise the various potential MEC use cases in a commercially viable way?
  • What are the right MEC business models for which telco?
  • What is needed for success? What are the challenges?

Contents:

  • Preface
  • Introduction
  • The emerging edge computing ecosystem
  • Telcos’ MEC opportunity
  • Hyperscale cloud providers are an added complication for telcos
  • How should telcos position themselves?
  • 5 telco business models for MEC
  • Business model 1: Dedicated edge hosting
  • Business model 2: Edge IaaS/PaaS/NaaS
  • Business model 3: Systems integration
  • Business model 4: B2B2X solutions
  • Business model 5: End-to-end consumer retail applications
  • Mapping use cases to business models
  • Some business models will require a long-term view on the investment
  • Which business models are right for which operator and which operator division?
  • Conclusion

Figures:

  • Figure 1: Defining MEC
  • Figure 2: MEC potential benefits
  • Figure 3: Microsoft’s new mantra – “Intelligent Cloud, Intelligent Edge”
  • Figure 4: STL Partners has identified 5 telco business models for MEC
  • Figure 5: The dedicated edge hosting value
  • Figure 6: Quantified example – Dedicated edge hosting
  • Figure 7: The Edge IaaS/PaaS/NaaS value chain
  • Figure 8: Quantified example – Edge IaaS/PaaS/NaaS
  • Figure 9: The SI value chain
  • Figure 10: Quantified example – Systems integration
  • Figure 11: The B2B2X solutions value chain
  • Figure 12: Quantified example – B2B2x solutions
  • Figure 13: Graphical representation of the end-to-end consumer retail applications business model
  • Figure 14: Quantified example – End-to-end consumer retail applications
  • Figure 15: Mapping MEC business models to possible use cases
  • Figure 16: High IRR correlates with low terminal value
  • Figure 17: Telcos need patience for edge-enabled consumer applications to become profitable (breakeven only in year 5)
  • Figure 18: The characteristics and skills required of the MEC operator depend on the business models

MobiNEX: The Mobile Network Experience Index, H1 2016

Executive Summary

In response to customers’ growing usage of mobile data and applications, in April 2016 STL Partners developed MobiNEX: The Mobile Network Experience Index, which ranks mobile network operators by key measures relating to customer experience. To do this, we benchmark mobile operators’ network speed and reliability, allowing individual operators to see how they are performing in relation to the competition in an objective and quantitative manner.

Operators are assigned an individual MobiNEX score out of 100 based on their performance across four measures that STL Partners believes to be core drivers of customer app experience: download speed, average latency, error rate and latency consistency (the proportion of app requests that take longer than 500ms to fulfil).

Our partner Apteligent has provided us with the raw data for three out of the four measures, based on billions of requests made from tens of thousands of applications used by hundreds of millions of users in H1 2016. While our April report focused on the top three or four operators in just seven Western markets, this report covers 80 operators drawn from 25 markets spread across the globe in the first six months of this year.

The top ten operators were from Japan, France, the UK and Canada:

  • Softbank JP scores highest on the MobiNEX for H1 2016, with high scores across all measures and a total score of 85 out of 100.
  • Close behind are Bouygues FR (80) and Free FR (79), which came first and second respectively in the Q4 2015 rankings. Both achieve high scores for error rate, latency consistency and average latency, but are slightly let down by download speed.
  • The top six is completed by NTT DoCoMo JP (78), Orange FR (75) and au (KDDI) JP (71).
  • Slightly behind are Vodafone UK (65), EE UK (64), SFR FR (63), O2 UK (62) and Rogers CA (62). Except in the case of Rogers, who score similarly on all measures, these operators are let down by substantially worse download speeds.

The bottom ten operators all score a total of 16 or lower out of 100, suggesting a materially worse customer app experience.

  • Trailing the pack with scores of 1 or 2 across all four measures were Etisalat EG (4), Vodafone EG (4), Smart PH (5) and Globe PH (5).
  • Beeline RU (11) and Malaysian operators U Mobile MY (9) and Digi MY (9) also fare poorly, but benefit from slightly higher latency consistency scores. Slightly better overall, but still achieving minimum scores of 1 for download speed and average latency, are Maxis MY (14) and MTN ZA (12).

Overall, the extreme difference between the top and bottom of the table highlights a vast inequality in network quality customer experience across the planet. Customer app experience depends to a large degree on where one lives. However, our analysis shows that while economic prosperity does in general lead to a more advanced mobile experience as you might expect, it does not guarantee it. Norway, Sweden, Singapore and the US market are examples of high income countries with lower MobiNEX scores than might be expected against the global picture. STL Partners will do further analysis to uncover more on the drivers of differentiation between markets and players within them.

 

MobiNEX H1 2016 – included markets

MobiNEX H1 2016 – operator scores

 Source: Apteligent, OpenSignal, STL Partners analysis

 

  • About MobiNEX
  • Changes for H1 2016
  • MobiNEX H1 2016: results
  • The winners: top ten operators
  • The losers: bottom ten operators
  • The surprises: operators where you wouldn’t expect them
  • MobiNEX by market
  • MobiNEX H1 2016: segmentation
  • MobiNEX H1 2016: Raw data
  • Error rate
  • Latency consistency
  • Download speed
  • Average latency
  • Appendix 1: Methodology and source data
  • Latency, latency consistency and error rate: Apteligent
  • Download speed: OpenSignal
  • Converting raw data into MobiNEX scores
  • Setting the benchmarks
  • Why measure customer experience through app performance?
  • Appendix 2: Country profiles
  • Country profile: Australia
  • Country profile: Brazil
  • Country profile: Canada
  • Country profile: China
  • Country profile: Colombia
  • Country profile: Egypt
  • Country profile: France
  • Country profile: Germany
  • Country profile: Italy
  • Country profile: Japan
  • Country profile: Malaysia
  • Country profile: Mexico
  • Country profile: New Zealand
  • Country profile: Norway
  • Country profile: Philippines
  • Country profile: Russia
  • Country profile: Saudi Arabia
  • Country profile: Singapore
  • Country profile: South Africa
  • Country profile: Spain
  • Country profile: United Arab Emirates
  • Country profile: United Kingdom
  • Country profile: United States
  • Country profile: Vietnam

 

  • Figure 1: MobiNEX scoring breakdown, benchmarks and raw data used
  • Figure 2: MobiNEX H1 2016 – included markets
  • Figure 3: MobiNEX H1 2016 – operator scores breakdown (top half)
  • Figure 4: MobiNEX H1 2016 – operator scores breakdown (bottom half)
  • Figure 5: MobiNEX H1 2016 – average scores by country
  • Figure 6: MobiNEX segmentation dimensions
  • Figure 7: MobiNEX segmentation – network speed vs reliability
  • Figure 8: MobiNEX segmentation – network speed vs reliability – average by market
  • Figure 9: MobiNEX vs GDP per capita – H1 2016
  • Figure 10: MobiNEX vs smartphone penetration – H1 2016
  • Figure 11: Error rate per 10,000 requests, H1 2016 – average by country
  • Figure 12: Error rate per 10,000 requests, H1 2016 (top half)
  • Figure 13: Error rate per 10,000 requests, H1 2016 (bottom half)
  • Figure 14: Requests with total roundtrip latency > 500ms (%), H1 2016 – average by country
  • Figure 15: Requests with total roundtrip latency > 500ms (%), H1 2016 (top half)
  • Figure 16: Requests with total roundtrip latency > 500ms (%), H1 2016 (bottom half)
  • Figure 17: Average weighted download speed (Mbps), H1 2016 – average by country
  • Figure 18: Average weighted download speed (Mbps), H1 2016 (top half)
  • Figure 19: Average weighted download speed (Mbps), H1 2016 (bottom half)
  • Figure 20: Average total roundtrip latency (ms), H1 2016 – average by country
  • Figure 21: Average total roundtrip latency (ms), H1 2016 (top half)
  • Figure 22: Average total roundtrip latency (ms), H1 2016 (bottom half)
  • Figure 23: Benchmarks and raw data used

US Wireless Market: Early Warning Signs of Change

Introduction

The US national wireless market is currently the most influential of its kind on the planet. Not only is it big, it is also rich, with significantly higher ARPUs than other developed markets. Not only is it big and rich, it is advanced, with much higher 4G penetration than comparable markets. Further, it has frequently acted as a bellwether for the world wireless industry. The iPhone’s success in the US marked the smartphone’s transition from pioneer to early-adopter status worldwide; the much greater success of the iPhone 3GS and the Moto Droid marked the beginning of mass adoption, and the crisis of the mid-market vendors.

On the network side, Verizon Wireless’s early decision to abandon the CDMA2000 development path and choose LTE FDD signalled the end of the standards wars and the beginning of serious 4G deployment, threw Motorola even deeper into crisis, and positioned Alcatel-Lucent as the leading vendor in the first wave of LTE rollouts.

The inclusion of the 1800MHz band in the iPhone 5, meanwhile, transformed the world’s spectrum picture, redefining this legacy GSM/PCS allocation as a key asset for smartphone-focused operators. Today, the combination of US wireless operators and US semiconductor vendors is transforming industry technology strategies again, as Verizon Wireless, AT&T, and Qualcomm lead the charge for a mobile broadband-focused “early” 5G.

Clearly, the US market is as critical for global mobile as the European market was in the pre-iPhone era. As a result, Telco 2.0 finds it useful to monitor it closely. We covered the changing 5G ecosystem in MWC: 5G and Wireless Networks  and How 5G is Disrupting Cloud and Network Strategy Today. We covered AT&T’s key role in driving NFV and open-source telco software forwards in Fast Pivot to the NFV Future, and the fate of worldwide 4G deployments in 4G Rollout Analysis: Winning Strategies and 5G Implications. This picked out one US carrier in particular for closer attention. In the adjacent industries, we covered Microsoft in Pivoting to a Communications-Focused Business, Amazon.com in Amazon Web Services: Colossal, but Invincible?, the cable operators in Gigabit Cable Attacks This Year, and the top-brand tech sector generally in Amazon, Apple, Facebook, Google, Netflix: Whose digital content is king?.

In this note, we will review developments in the US national wireless sector, both on a long-term basis since the launch of 4G, and on a tactical basis over the last 12 months, including analysis of the results from our new, unique Mobile Network Experience Index product .

The US Wireless Market, 2011-2016

The last five years in the US cellular market have been characterised by two forces – disruption, and growth. The arrival of smartphones comprehensively disrupted what had been a rather stagnant sector. Later, T-Mobile USA initiated a price disruption which resulted in a wave of consolidation and a significant drop in industrywide ARPU. However, despite the “uncarrier”’s price cuts, the total industry profit pool has nonetheless grown dramatically in that timeframe, from $8.7bn/quarter to $14bn/quarter, as the revenue base has grown by some 20%, or 4% per annum.

Figure 1: The US wireless revenue base, 2011-2016

Source: STL Partners, company filings, themobileworld

Growth was as characteristic of the US market over the last 5 years as price disruption. T-Mobile’s strategy was to a large extent possible because there was a significant degree of “blue-ocean” competition, enlarging the subscriber base and deepening its use of smartphones and high-speed data service, as well as consolidation of minor operators. We show the net impact on operating profits in Figure 2.

Figure 2: Long term shifts in the US national wireless profit pool, 2011-2016

Source: STL Partners, themobileworld.com, company filings

Over the whole timeframe, the total annual pool of operating profit available in the market has grown by 59% or 11.8% per annum, or five times as fast as US GDP. This came in the context of a 20%, or 4% annualised, increase in total wireless revenues. At the same time, three operators have benefited disproportionately from this growth – AT&T, Verizon Wireless, and T-Mobile. In fact, AT&T’s gains have been quite modest compared to the triumphs at VZW and T-Mobile.

On the other hand, Sprint has seen its operating profits halve and halve again, while Leap, MetroPCS, and numerous minor operators have exited the market. Looking at these data in a time-series view, as we do in Figure 3, we see that the duopoly is still a real force, although Verizon, has done distinctly better than AT&T.

Verizon Wireless, which pursued a “premium carrier” strategy based on going first with 4G, using its 700MHz holdings to maximise coverage and densifying with 1800MHz, and holding the line on price as long as possible, has clearly maximised its operating-level profitability. Meanwhile, a vicious struggle for third place was waged between T-Mobile and Sprint. Both parties struggled at times with the cost of spectrum acquisitions and network investments, and the gap between them and the duopoly is unmistakable. However, T-Mobile has managed to keep in the black since 2013 and its profitability is gradually improving, breaking away from the also-rans over the last 12 months.

 

  • Executive Summary
  • Introduction
  • The US Wireless Market, 2011-2016
  • Where Do We Go From Here?
  • Duopolists, challengers, and exits
  • Valuations
  • Challenging the Premium Carrier
  • The Impact of IPv6 deployment
  • Conclusions
  • Disruptive Responses: 5G
  • Disruptive Responses: Content

 

  • Figure 1: The US wireless revenue base, 2011-2016
  • Figure 2: Long term shifts in the US national wireless profit pool, 2011-2016
  • Figure 3: Profits at US wireless carriers, 2011-2016 (time series)
  • Figure 4: The short-run profits pool
  • Figure 5: Five years of Verizon Vs T-Mobile
  • Figure 6: Long term share of connections growth, 2011-2016
  • Figure 7: Short term share of connections growth, 2015
  • Figure 8: Long term retail postpaid users, 2011-2016
  • Figure 9: Short term retail postpaid subscribers, 2015
  • Figure 10: Now, T-Mobile is gaining the right kind of subscribers
  • Figure 11: Retail postpaid connections over time
  • Figure 12: Prepaid subscribers over time
  • Figure 13: Long term change in prepaid users is mostly growth, and MetroPCS’s exit
  • Figure 14: Short term change in retail prepaid users, 2015
  • Figure 15: T-Mobile’s debts are far from zooming out of control
  • Figure 16: Duopolists, challengers, and exit candidates
  • Figure 17: Net income margins over time
  • Figure 18: Device sales surge, margins dive
  • Figure 19: Valuation – EV/EBITDA
  • Figure 20: T-Mobile leads on our MobiNEX score
  • Figure 21: A link between network metrics and customer satisfaction?
  • Figure 22: 3 out of 4 US MNOs are “challenged” in the world context
  • Figure 23: Download speed vs. percentage of LTE requests
  • Figure 24: T-Mobile is the lowest-latency US operator
  • Figure 25: T-Mobile is generating 25% fewer high latency events than AT&T
  • Figure 26: T-Mobile’s error rate catches up on the market leader
  • Figure 27: Quality across the board
  • Figure 28: IPv6 adoption, US wireless operators

MobiNEX: The Mobile Network Experience Index, Q4 2015

Executive Summary

In response to customers’ growing usage of mobile data and applications, STL Partners has developed MobiNEX: The Mobile Network Customer Experience Index, which benchmarks mobile operators’ network speed and reliability by measuring the consumer app experience, and allows individual players to see how they are performing in relation to competition in an objective and quantitative manner.

We assign operators an individual MobiNEX score based on their performance across four measures that are core drivers of customer app experience: download speed; average latency; error rate; latency consistency (the percentage of app requests that take longer than 500ms to fulfil). Apteligent has provided us with the raw data for three out of four of the measures based on billions of requests made from tens of thousands of applications used by hundreds of millions of users in Q4 2015. We plan to expand the index to cover other operators and to track performance over time with twice-yearly updates.

Encouragingly, MobiNEX scores are positively correlated with customer satisfaction in the UK and the US suggesting that a better mobile app experience contributes to customer satisfaction.

The top five performers across twenty-seven operators in seven countries in Europe and North America (Canada, France, Germany, Italy, Spain, UK, US) were all from France and the UK suggesting a high degree of competition in these markets as operators strive to improve relative to peers:

  • Bouygues Telecom in France scores highest on the MobiNEX for Q4 2015 with consistently high scores across all four measures and a total score of 76 out of 100.
  • It is closely followed by two other French operators. Free, the late entrant to the market, which started operations in 2012, scores 73. Orange, the former national incumbent, is slightly let down by the number of app errors experienced by users but achieves a healthy overall score of 70.
  • The top five is completed by two UK operators: EE (65) and O2 (61) with similar scores to the three French operators for everything except download speed which was substantially worse.

The bottom five operators have scores suggesting a materially worse customer app experience and we suggest that management focuses on improvements across all four measures to strengthen their customer relationships and competitive position. This applies particularly to:

  • E-Plus in Germany (now part of Telefónica’s O2 network but identified separately by Apteligent).
  • Wind in Italy, which is particularly let down by latency consistency and download speed.
  • Telefónica’s Movistar, the Spanish market share leader.
  • Sprint in the US with middle-ranking average latency and latency consistency but, like other US operators, poor scores on error rate and download speed.
  • 3 Italy, principally a result of its low latency consistency score.

Surprisingly, given the extensive deployment of 4G networks there, the US operators perform poorly and are providing an underwhelming customer app experience:

  • The best-performing US operator, T-Mobile, scores only 45 – a full 31 points below Bouygues Telecom and 4 points below the median operator.
  • All the US operators perform very poorly on error rate and, although 74% of app requests in the US were made on LTE in Q4 2015, no US player scores highly on download speed.

MobiNEX scores – Q4 2015

 Source: Apteligent, OpenSignal, STL Partners analysis

MobiNEX vs Customer Satisfaction

Source: ACSI, NCSI-UK, STL Partners

 

  • Introduction
  • Mobile app performance is dependent on more than network speed
  • App performance as a measure of customer experience
  • MobiNEX: The Mobile Network Experience Index
  • Methodology and key terms
  • MobiNEX Q4 2015 Results: Top 5, bottom 5, surprises
  • MobiNEX is correlated with customer satisfaction
  • Segmenting operators by network customer experience
  • Error rate
  • Quantitative analysis
  • Key findings
  • Latency consistency: Requests with latency over 500ms
  • Quantitative analysis
  • Key findings
  • Download speed
  • Quantitative analysis
  • Key findings
  • Average latency
  • Quantitative analysis
  • Key findings
  • Appendix: Source data and methodology
  • STL Partners and Telco 2.0: Change the Game
  • About Apteligent

 

  • MobiNEX scores – Q4 2015
  • MobiNEX vs Customer Satisfaction
  • Figure 1: MobiNEX – scoring methodology
  • Figure 2: MobiNEX scores – Q4 2015
  • Figure 3: Customer Satisfaction vs MobiNEX, 2015
  • Figure 4: MobiNEX operator segmentation – network speed vs network reliability
  • Figure 5: MobiNEX operator segmentation – with total scores
  • Figure 6: Major Western markets – error rate per 10,000 requests
  • Figure 7: Major Western markets – average error rate per 10,000 requests
  • Figure 8: Major Western operators – percentage of requests with total roundtrip latency greater than 500ms
  • Figure 9: Major Western markets – average percentage of requests with total roundtrip latency greater than 500ms
  • Figure 10: Major Western operators – average weighted download speed across 3G and 4G networks (Mbps)
  • Figure 11: Major European markets – average weighted download speed (Mbps)
  • Figure 12: Major Western markets – percentage of requests made on 3G and LTE
  • Figure 13: Download speed vs Percentage of LTE requests
  • Figure 14: Major Western operators – average total roundtrip latency (ms)
  • Figure 15: Major Western markets – average total roundtrip latency (ms)
  • Figure 16: MobiNEX benchmarks

Lag Kills! How App Latency Wrecks Customer Experience

Executive Summary

  • STL Partners’ analysis shows that while latency and app errors are only weakly correlated across the whole of Europe, once outlying operators (SFR, Wind and those in Germany) are removed, there is a strong positive correlation between the two: as latency increases so do app errors.
  • Intuitively, this makes sense: apps ‘time out’ waiting for responses causing errors and crashes.
  • Latency and app errors both negatively affect customer experience – customers are more likely to abandon apps as responsiveness and error rates increase:
    • 48% of users would uninstall or stop using an app if it regularly ran slowly.
    • 53% of users would uninstall or stop using an app if it regularly crashed, stopped responding or had errors.
  • Historically, customers have tended to hold the app developer responsible for errors (55% of users blame the app for problems and only 22% the mobile operator) but mobile operators have a significant impact on how quickly an app runs and how likely it is to experience an error and, as understanding of the operators’ role grows, users may well use this as a criterion when selecting their mobile service provider.
  • Performance among Europe’s operators for app latency and errors varies widely:
    • The worst-performing operator in Europe (3 Italy) experiences over three times the amount of requests with poor latency compared to the best-performer (Bouygues Telecom).
    • The worst-performing operator in Europe (O2 Germany) results in over twice the number of app errors than the best-performer (Bouygues Telecom again).
  • Improving customer experience is rapidly becoming a mantra of operators globally and for several players (in Europe at least) improving latency performance and reducing app errors caused by latency and other factors should be a key priority. For without improvement, poor performing operators will find themselves at a disadvantage and may struggle to retain existing customers and recruit new ones.

Introduction

Key objectives

Network latency is a key driver of user experience. In applications as diverse as e-commerce, VoIP, gaming, video or audio content delivery, search, online advertising, financial services, and the Internet of Things, increased latency has a direct and negative impact on customers. With higher latency, customers fail to complete tasks, leave applications, or experience application errors. This, in turn, results poorer core business KPIs for the application provider – lower ratings, fewer subscribers, or reduced advertising fees.

As we showed in a recent report titled Mobile app latency in Europe: French operators lead; Italian & Spanish lag, with the modern Internet dominated by flows of small packets on fast networks, latency accounts for the biggest share of total load times and tends to determine the actual data transfer rates users see. And, as web and mobile applications increasingly consist of large numbers of requests to independent ‘microservices’, jitter – the variation in latency – becomes a more significant threat to the consumer experience. Furthermore, we benchmarked major European mobile network operators (MNOs) on average latency and the rate of unacceptably high-latency events (over 500ms).

In this second report on latency, which again uses data provided by app analytics specialist Apteligent (formerly Crittercism), we look at the rate of app errors – evidently, something that could not impact user experience more directly – and its correlation with both latency, and the rate of unacceptable high-latency events. We explore how often apps fail across the same set of MNOs, test if latency is a driver of app errors, and then conclude whether or not our theory that it is a real driver of consumer experience is correct.

Source data and methodology

Our partner, Apteligent, collects a wide variety of analytics data from thousands of mobile apps used by hundreds of millions of people around the world in their every-day lives and work. To date, the primary purpose of the data has been to help app developers make better apps. We are now working with Crittercism to produce further insights from the data to serve the global community of mobile operators.

This data-set includes the average network latency experienced at the application layer, the percentage of network requests above 500ms round-trip time, the 5th and 95th percentiles, and the rate of application errors. All of these data points are useful in trying to understand the overall experience of customers using their mobile apps, and in particular the delays and problems they’ve experienced such as long screen wait times and applications failing to work.

We showed in the previous report how the longest round-trip delays or ‘app-lags’ (i.e. those over 500ms) are the most important KPI to look at when trying to understand customer experience. This is firstly because people really notice individual delays of this length. For people used to high speed broadband, it’s like going back to narrowband internet – it seems incredibly slow!

Importantly though, in modern apps, the distribution of delays is even more significant, as each app or web page typically makes multiple requests over the internet before it can load fully – and each of these requests will suffer some form of delay or latency.

A detailed explanation of this and of the collection methodology is available in the first report.

The Impact of latency on app errors

First glance: a positive correlation overall, but a weak one

The following chart shows the error rate per 10,000 app requests, plotted against the percentage of requests over 500ms round-trip time, by carrier. Each dot represents a week’s performance and we’ve looked at 12 weeks of data from 20 operators, from the week of 03/08/15 to the week beginning 19/10/2015. The hypothesis being that the more requests with unacceptable latency there are, the more app errors, because apps ‘time-out’ or key requests are not fulfilled in time causing an app error or, worse, a crash.

Figure 1: Latency and errors for the top 20 European MNOs over the last 12-weeks appear correlated, but there are some important outliers

Source: STL Partners, Apteligent

At first glance, there appears to be only a weak positive relationship between latency and error rates but there does seem to be a natural grouping found between the two hand-drawn dotted lines on the chart with the weeks above the upper boundary (potentially) being outliers, in which at least one other factor is driving application errors up.

The lower boundary seems to represent the underlying rate of app-errors that occur when there are no latency issues (between 20 and 50 errors per ten thousand plus an increasing error rate as higher latency kicks in. For example, when 10% of requests experience latency above 500ms, the minimum error rate is around 30 per 10,000 requests, rising to 50 at the 35% mark.

  • Executive Summary
  • Introduction
  • Key objectives
  • Source data and methodology
  • The Impact of Latency on App Errors
  • First glance: a positive correlation overall, but a weak one
  • Outliers are specific countries and operators
  • Strong positive correlation between latency and app errors once outliers are excluded
  • App Errors: The Impact on Customer Experience
  • Latency and errors – both bad for the customer
  • Appendix: Country Analysis
  • France: A Clear Relationship
  • The UK: Strong Latency-Error Correlation
  • Spain: A mixed picture, but latency is still predictive of app errors
  • Italy: Wind is a super-outlier
  • Germany: Nothing but Outliers?
  • STL Partners and Telco 2.0: Change the Game
  • About Apteligent (formerly Crittercism)

 

  • Figure 1: Latency and errors for the top 20 European MNOs over the last 12-weeks appear correlated, but there are some important outliers
  • Figure 2: 12-week average latency and app error performance by operator
  • Figure 3: After excluding the key outliers, high-latency events explain 75% of the app error rate across Europe’s top 20 operators
  • Figure 4: Expected number of errors when loading 20 web pages of Amazon
  • Figure 5: France shows both the best performers, and a very clear relationship between latency and app errors
  • Figure 6: The latency-error correlation is strongest in the UK
  • Figure 7: High variation in latency complicates the picture, but a third of app error variation is still driven by latency
  • Figure 8: Wind complicates the picture, but the trend is still there
  • Figure 9: Germany – is there any trend at all?
  • Figure 10: The source of the outliers – Germany in August

Mobile app latency in Europe: French operators lead; Italian & Spanish lag

Latency as a proxy for customer app experience

Latency is a measure of the time taken for a packet of data to travel from one designated point to another. The complication comes in defining the start and end point. For an operator seeking to measure its network latency, it might measure only the transmission time across its network.

However, to objectively measure customer app experience, it is better to measure the time it takes from the moment the user takes an action, such as pressing a button on a mobile device, to receiving a response – in effect, a packet arriving back and being processed by the application at the device.

This ‘total roundtrip latency’ time is what is measured by our partner, Crittercism, via embedded code within applications themselves on an aggregated and anonymised basis. Put simply, total roundtrip latency is the best measure of customer experience because it encompasses the total ‘wait time’ for a customer, not just a portion of the multi-stage journey

Latency is becoming increasingly important

Broadband speeds tend to attract most attention in the press and in operator advertising, and speed does of course impact downloads and streaming experiences. But total roundtrip latency has a bigger impact on many user digital experiences than speed. This is because of the way that applications are built.

In modern Web applications, the business logic is parcelled-out into independent ‘microservices’ and their responses re-assembled by the client to produce the overall digital user experience. Each HTTP request is often quite small, although an overall onscreen action can be composed of a number of requests of varying sizes so broadband speed is often less of a factor than latency – the time to send and receive each request. See Appendix 2: Why latency is important, for a more detailed explanation of why latency is such an important driver of customer app experience.

The value of using actual application latency data

As we have already explained, STL Partners prefers to use total roundtrip latency as an indicator of customer app experience as it measures the time that a customer waits for a response following an action. STL Partners believes that Crittercism data reflects actual usage in each market because it operates within apps – in hundreds of thousands of apps that people use in the Apple App Store and in Google Play. This is a quite different approach to other players which require users to download a specific app which then ‘pings’ a server and awaits a response. This latter approach has a couple of limitations:

1. Although there have been several million downloads of the OpenSignal and Actual Experience app, this doesn’t get anywhere near the number of people that have downloaded apps containing the Crittercism measurement code.

2. Because the Crittercism code is embedded within apps, it directly measures the latency experienced by users when using those apps1. A dedicated measurement app fails to do this. It could be argued that a dedicated app gives the ‘cleanest’ app reading – it isn’t affected by variations in app design, for example. This is true but STL Partners believes that by aggregating the data for apps such variation is removed and a representative picture of total roundtrip latency revealed. Crittercism data can also show more granular data. For example, although we haven’t shown it in this report, Crittercism data can show latency performance by application type – e.g. Entertainment, Shopping, and so forth – based on the categorisation of apps used by Google and Apple in their app stores.

A key premise of this analysis is that, because operators’ customer bases are similar within and across markets, the profile of app usage (and therefore latency) is similar from one operator to the next. The latency differences between operators are, therefore, down to the performance of the operator.

Why it isn’t enough to measure average latency

It is often said that averages hide disparities in data, and this is particularly true for latency and for customer experience. This is best illustrated with an example. In Figure 2 we show the distribution of latencies for two operators. Operator A has lots of very fast requests and a long tail of requests with high latencies.

Operator B has much fewer fast requests but a much shorter tail of poor-performing latencies. The chart clearly shows that operator B has a much higher percentage of requests with a satisfactory latency even though its average latency performance is lower than operator A (318ms vs 314ms). Essentially operator A is let down by its slowest requests – those that prevent an application from completing a task for a customer.

This is why in this report we focus on average latency AND, critically, on the percentage of requests that are deemed ‘unsatisfactory’ from a customer experience perspective.

Using latency as a measure of performance for customers

500ms as a key performance cut-off

‘Good’ roundtrip latency is somewhat subjective and there is evidence that experience declines in a linear fashion as latency increases – people incrementally drop off the site. However, we have picked 500ms (or half a second) as a measure of unsatisfactory performance as we believe that a delay of more than this is likely to impact mobile users negatively (expectations on the ‘fixed’ internet are higher). User interface research from as far back as 19682 suggests that anything below 100ms is perceived as “instant”, although more recent work3 on gamers suggests that even lower is usually better, and delay starts to become intrusive after 200-300ms. Google experiments from 20094 suggest that a lasting effect – users continued to see the site as “slow” for several weeks – kicked in above 400ms.

Percentage of app requests with total roundtrip latency above 500ms – markets

Five key markets in Europe: France, Germany, Italy, and the UK.

This first report looks at five key markets in Europe: France, Germany, Italy, and the UK. We explore performance overall for Europe by comparing the relative performance of each country and then dive into the performance of operators within each country.

We intend to publish other reports in this series, looking at performance in other regions – North America, the Middle East and Asia, for example. This first report is intended to provider a ‘taster’ to readers, and STL Partners would like feedback on additional insight that readers would welcome, such as latency performance by:

  • Operating system – Android vs Apple
  • Specific device – e.g. Samsung S6 vs iPhone 6
  • App category – e.g. shopping, games, etc.
  • Specific countries
  • Historical trends

Based on this feedback, STL Partners and Crittercism will explore whether it is valuable to provide specific total roundtrip latency measurement products.

Contents

  • Latency as a proxy for customer app experience
  • ‘Total roundtrip latency’ is the best measure for customer ‘app experience’
  • Latency is becoming increasingly important
  • STL Partners’ approach
  • Europe: UK, Germany, France, Italy, Spain
  • Quantitative Analysis
  • Key findings
  • UK: EE, O2, Vodafone, 3
  • Quantitative Analysis
  • Key findings
  • Germany: T-Mobile, Vodafone, e-Plus, O2
  • Quantitative Analysis
  • Key findings
  • France: Orange, SFR, Bouygues Télécom, Free
  • Quantitative Analysis
  • Key findings
  • Italy: TIM, Vodafone, Wind, 3
  • Quantitative Analysis
  • Key findings
  • Spain: Movistar, Vodafone, Orange, Yoigo
  • Quantitative Analysis
  • Key findings
  • About STL Partners and Telco 2.0
  • About Crittercism
  • Appendix 1: Defining latency
  • Appendix 2: Why latency is important

 

  • Figure 1: Total roundtrip latency – reflecting a user’s ‘wait time’
  • Figure 2: Why a worse average latency can result in higher customer satisfaction
  • Figure 3: Major European markets – average total roundtrip latency (ms)
  • Figure 4: Major European markets – percentage of requests above 500ms
  • Figure 5: The location of Google and Amazon’s European data centres favours operators in France, UK and Germany
  • Figure 6: European operators – average total roundtrip latency (ms)
  • Figure 7: European operators – percentage of requests with latency over 500ms
  • Figure 8: Customer app experience is likely to be particularly poor at 3 Italy, Movistar (Spain) and Telecom Italia
  • Figure 9: UK Operators – average latency (ms)
  • Figure 10: UK operators – percentage of requests with latency over 500ms
  • Figure 11: German Operators – average latency (ms)
  • Figure 12: German operators – percentage of requests with latency over 500ms
  • Figure 13: French Operators – average latency (ms)
  • Figure 14: French operators – percentage of requests with latency over 500ms
  • Figure 15: Italian Operators – average latency (ms)
  • Figure 16: Italian operators – percentage of requests with latency over 500ms
  • Figure 17: Spanish Operators – average latency (ms)
  • Figure 18: Spanish operators – percentage of requests with latency over 500ms
  • Figure 19: Breakdown of HTTP requests in facebook.com, by type and size

CDNs 2.0: should telcos compete with Akamai?

Content Delivery Networks (CDNs) such as Akamai’s are used to improve the quality and reduce costs of delivering digital content at volume. What role should telcos now play in CDNs? (September 2011, Executive Briefing Service, Future of the Networks Stream).
 Should telcos compete with Akamai?
  Read in Full (Members only)  Buy a single user license online  To Subscribe click here

Below is an extract from this 19 page Telco 2.0 Report that can be downloaded in full in PDF format by members of the Telco 2.0 Executive Briefing service and Future Networks Stream here. Non-members can subscribe here, buy a Single User license for this report online here for £795 (+VAT), or for multi-user licenses or other enquiries, please email contact@telco2.net / call +44 (0) 207 247 5003.

To share this article easily, please click:

//

Introduction

 

We’ve written about Akamai’s technology strategy for global CDN before as a fine example of the best practice in online video distribution and a case study in two-sided business models, to say nothing of being a company that knows how to work with the grain of the Internet. Recently, Akamai published a paper which gives an overview of its network and how it works. It’s a great paper, if something of a serious read. Having ourselves read, enjoyed and digested it, we’ve distilled the main elements in the following analysis, and used that as a basis to look at telcos’ opportunities in the CDN market.

Related Telco 2.0 Research

In the strategy report Mobile, Fixed and Wholesale Broadband Business Models – Best Practice Innovation, ‘Telco 2.0′ Opportunities, Forecasts and Future Scenarios we examined a number of different options for telcos to reduce costs and improve the quality of content delivery, including Content Delivery Networks (CDNs).

This followed on from Future Broadband Business Models – Beyond Bundling: winning the new $250Bn delivery game in which we looked at long term trends in network architectures, including the continuing move of intelligence and storage towards the edge of the network. Most recently, in Broadband 2.0: Delivering Video and Mobile CDNs we looked at whether there is now a compelling need for Mobile CDNs, and if so, should operators partner with existing players or build / buy their own?

We’ll also be looking in depth at the opportunities in mobile CDNs at the EMEA Executive Brainstorm in London on 9-10th November 2011.

Why have a CDN anyway?

The basic CDN concept is simple. Rather than sending one copy of a video stream, software update or JavaScript library over the Internet to each user who wants it, the content is stored inside their service provider’s network, typically at the POP level in a fixed ISP.

That way, there are savings on interconnect traffic (whether in terms of paid-for transit, capex, or stress on peering relationships), and by locating the servers strategically, savings are also possible on internal backhaul traffic. Users and content providers benefit from lower latency, and therefore faster download times, snappier user interface response, and also from higher reliability because the content servers are no longer a single point of failure.

What can be done with content can also be done with code. As well as simple file servers and media streaming servers, applications servers can be deployed in a CDN in order to bring the same benefits to Web applications. Because the content providers are customers of the CDN, it is possible to also apply content optimisation with their agreement at the time it is uploaded to the CDN. This makes it possible to save further traffic, and to avoid nasty accidents like this one.

Once the CDN servers are deployed, to make the network efficient, they need to be filled up with content and located so they are used effectively – so they need to be located in the right places. An important point of a CDN, and one that may play to telcos’ strengths, is that location is important.

Figure 1: With higher speeds, geography starts to dominate download times

CDN Akamai table distance throughput time Oct 2011 Telco 2.0

Source: Akamai

CDN Player Strategies

Market Overview

CDNs are a diverse group of businesses, with several major players, notably Akamai, the market leader, EdgeCast, and Limelight Networks, all of which are pure-play CDNs, and also a number of players that are part of either carriers or Web 2.0 majors. Level(3), which is widely expected to acquire the LimeLight CDN, is better known as a massive Internet backbone operator. BT Group and Telefonica both have CDN products. On the other hand, Google, Amazon, and Microsoft operate their own, very substantial CDNs in support of their own businesses. Amazon also provides a basic CDN service to third parties. Beyond these, there are a substantial number of small players.

Akamai is by far the biggest; Arbor Networks estimated that it might account for as much as 15% of Internet traffic once the actual CDN traffic was counted in, while the top five CDNs accounted for 10% of inter-domain traffic. The distinction is itself a testament to the effectiveness of CDN as a methodology.

The impact of CDN

As an example of the benefits of their CDN, above and beyond ‘a better viewing experience’, Akamai claim that they can demonstrate a 15% increase in completed transactions on an e-commerce site by using their application acceleration product. This doesn’t seem out of court, as Amazon.com has cited similar numbers in the past, in their case by reducing the volume of data needed to deliver a given web page rather than by accelerating its delivery.

As a consequence of these benefits, and the predicted growth in internet traffic, Akamai expect traffic on their platform to reach levels equivalent to the throughput of a US national broadcast TV station within 2-5 years. In the fixed world, Akamai claims offload rates of as much as 90%. The Jetstream CDN  blog points out that mobile operators might be able to offload as much as 65% of their traffic into the CDN. These numbers refer only to traffic sources that are customers of the CDN, but it ought to be obvious that offloading 90% of the YouTube or BBC iPlayer traffic is worth having.

In Broadband 2.0: Mobile CDNs and video distribution we looked at the early prospects for Mobile CDN, and indeed, Akamai’s own move into the mobile industry is only beginning. However, Telefonica recently announced that its internal, group-wide CDN has reached an initial capability, with service available in Europe and in Argentina. They intend to expand across their entire footprint. We are aware of at least one other mobile operator which is actively investing in CDN capabilities. The degree to which CDN capabilities can be integrated into mobile networks is dependent on the operator’s choice of network architecture, which we discuss later in this note.

It’s also worth noting that one of Akamai’s unique selling points is that it is very much a global operator. As usual, there’s a problem for operators, especially mobile operators, in that the big Internet platforms are global and operators are regional. Content owners can deal with one CDN for their services all around the world – they can’t deal with one telco. Also, big video sources like national TV broadcasters can usually deal with one ex-incumbent fixed operator and cover much of the market, but must deal with several mobile operators.

Application Delivery: the frontier of CDN

Akamai is already doing a lot of what we call “ADN” (Application-Delivery Networking) by analogy to CDN. In a CDN, content is served up near the network edge. In an ADN, applications are hosted in the same way in order to deliver them faster and more reliably. (Of course, the media server in a CDN node is itself a software application.) And the numbers we cited above regarding improved transaction completion rates are compelling.

However, we were a little under-whelmed by the details given of their Edge Computing product. It is restricted to J2EE and XSLT applications, and it seems quite limited in the power and flexibility it offers compared to the state of the art in cloud computing. Google App Engine and Amazon EC2 look far more interesting from a developer point of view. Obviously, they’re going for a different market. But we heartily agree with Dan Rayburn that the future of CDN is applications acceleration, and that this goes double for mobile with its relatively higher background levels of latency.

Interestingly, some of Akamai’s ADN customers aren’t actually distributing their code out to the ADN servers, but only making use of Akamai’s overlay network to route their traffic. Relatively small optimisations to the transport network can have significant benefits in business terms even before app servers are physically forward-deployed.

Other industry developments to watch

There are some shifts underway in the CDN landscape. Notably, as we mentioned earlier, there are rumours that Limelight Networks wants to exit the packet-pushing element of it in favour of the media services side – ingestion, transcoding, reporting and analytics. The most likely route is probably a sale or joint venture with Level(3). Their massive network footprint gives them both the opportunity to do global CDNing, and also very good reasons to do so internally. Being a late entrant, they have been very aggressive on price in building up a customer base (you may remember their role in the great Comcast peering war). They will be a formidable competitor and will probably want to move from macro-CDN to a more Akamai-like forward deployed model.

To read the note in full, including the following additional analysis…

  • Akamai’s technology strategy for a global CDN
  • Can Telcos compete with CDN Players?
  • Potential Telco Leverage Points
  • Global vs. local CDN strategies
  • The ‘fat head’ of content is local
  • The challenges of scale and experience
  • Strategic Options for Telcos
  • Cooperating with Akamai
  • Partnering with a Vendor Network
  • Part of the global IT operation?
  • National-TV-centred CDNs
  • A specialist, wholesale CDN role for challengers?
  • Federated CDN
  • Conclusion

…and the following charts…

  • Figure 1: With higher speeds, geography starts to dominate download times
  • Figure 2: Akamai’s network architecture
  • Figure 3: Architectural options for CDN in 3GPP networks
  • Figure 4: Mapping CDN strategic options

…Members of the Telco 2.0 Executive Briefing Subscription Service and Future Networks Stream can download the full 19 page report in PDF format here. Non-Members, please subscribe here, buy a Single User license for this report online here for £795 (+VAT), or for multi-user licenses or other enquiries, please email contact@telco2.net / call +44 (0) 207 247 5003.

Organisations, people and products referenced: 3UK, Akamai, Alcatel-Lucent, Amazon, Arbor Networks, BBC, BBC iPlayer, BitTorrent, BT, Cisco, Dan Rayburn, EC2, EdgeCast, Ericsson, Google, GSM, Internet HSPA, Jetstream, Level(3), Limelight Networks, MBNL, Microsoft, Motorola, MOVE, Nokia Siemens Networks, Orange, TalkTalk, Telefonica, T-Mobile, Velocix, YouTube.

Technologies and industry terms referenced: 3GPP, ADSL, App Engine, backhaul, Carrier-Ethernet, Content Delivery Networks (CDNs), DNS, DOCSIS 3, edge computing, FTTx, GGSN, Gi interface, HFC, HSPA+, interconnect, IT, JavaScript, latency, LTE, Mobile CDNs, online, peering, POPs (Points of Presence), RNC, SQL, UMTS, VPN, WLAN.