Login to access
Want to subscribe?
This article is part of: Executive Briefing Service
To find out more about how to join or access this report please contact us
For the first time, STL Partners quantifies the customer ‘app experience’ on twenty-seven mobile networks in seven countries. MobiNEX - The Mobile Network Experience Index - benchmarks mobile operators' network speed and reliability by measuring download speed; average latency; error rate and latency consistency. It’s based on billions of real customer data points provided by our partners Apteligent. Congratulations to the top three performers – Bouygues, Free and Orange (all in France).
Executive Summary
In response to customers’ growing usage of mobile data and applications, STL Partners has developed MobiNEX: The Mobile Network Customer Experience Index, which benchmarks mobile operators’ network speed and reliability by measuring the consumer app experience, and allows individual players to see how they are performing in relation to competition in an objective and quantitative manner.
We assign operators an individual MobiNEX score based on their performance across four measures that are core drivers of customer app experience: download speed; average latency; error rate; latency consistency (the percentage of app requests that take longer than 500ms to fulfil). Apteligent has provided us with the raw data for three out of four of the measures based on billions of requests made from tens of thousands of applications used by hundreds of millions of users in Q4 2015. We plan to expand the index to cover other operators and to track performance over time with twice-yearly updates.
Encouragingly, MobiNEX scores are positively correlated with customer satisfaction in the UK and the US suggesting that a better mobile app experience contributes to customer satisfaction.
The top five performers across twenty-seven operators in seven countries in Europe and North America (Canada, France, Germany, Italy, Spain, UK, US) were all from France and the UK suggesting a high degree of competition in these markets as operators strive to improve relative to peers:
- Bouygues Telecom in France scores highest on the MobiNEX for Q4 2015 with consistently high scores across all four measures and a total score of 76 out of 100.
- It is closely followed by two other French operators. Free, the late entrant to the market, which started operations in 2012, scores 73. Orange, the former national incumbent, is slightly let down by the number of app errors experienced by users but achieves a healthy overall score of 70.
- The top five is completed by two UK operators: EE (65) and O2 (61) with similar scores to the three French operators for everything except download speed which was substantially worse.
The bottom five operators have scores suggesting a materially worse customer app experience and we suggest that management focuses on improvements across all four measures to strengthen their customer relationships and competitive position. This applies particularly to:
- E-Plus in Germany (now part of Telefónica’s O2 network but identified separately by Apteligent).
- Wind in Italy, which is particularly let down by latency consistency and download speed.
- Telefónica’s Movistar, the Spanish market share leader.
- Sprint in the US with middle-ranking average latency and latency consistency but, like other US operators, poor scores on error rate and download speed.
- 3 Italy, principally a result of its low latency consistency score.
Surprisingly, given the extensive deployment of 4G networks there, the US operators perform poorly and are providing an underwhelming customer app experience:
- The best-performing US operator, T-Mobile, scores only 45 – a full 31 points below Bouygues Telecom and 4 points below the median operator.
- All the US operators perform very poorly on error rate and, although 74% of app requests in the US were made on LTE in Q4 2015, no US player scores highly on download speed.
MobiNEX scores – Q4 2015
Source: Apteligent, OpenSignal, STL Partners analysis
MobiNEX vs Customer Satisfaction
Source: ACSI, NCSI-UK, STL Partners
- Introduction
- Mobile app performance is dependent on more than network speed
- App performance as a measure of customer experience
- MobiNEX: The Mobile Network Experience Index
- Methodology and key terms
- MobiNEX Q4 2015 Results: Top 5, bottom 5, surprises
- MobiNEX is correlated with customer satisfaction
- Segmenting operators by network customer experience
- Error rate
- Quantitative analysis
- Key findings
- Latency consistency: Requests with latency over 500ms
- Quantitative analysis
- Key findings
- Download speed
- Quantitative analysis
- Key findings
- Average latency
- Quantitative analysis
- Key findings
- Appendix: Source data and methodology
- STL Partners and Telco 2.0: Change the Game
- About Apteligent
- MobiNEX scores – Q4 2015
- MobiNEX vs Customer Satisfaction
- Figure 1: MobiNEX – scoring methodology
- Figure 2: MobiNEX scores – Q4 2015
- Figure 3: Customer Satisfaction vs MobiNEX, 2015
- Figure 4: MobiNEX operator segmentation – network speed vs network reliability
- Figure 5: MobiNEX operator segmentation – with total scores
- Figure 6: Major Western markets – error rate per 10,000 requests
- Figure 7: Major Western markets – average error rate per 10,000 requests
- Figure 8: Major Western operators – percentage of requests with total roundtrip latency greater than 500ms
- Figure 9: Major Western markets – average percentage of requests with total roundtrip latency greater than 500ms
- Figure 10: Major Western operators – average weighted download speed across 3G and 4G networks (Mbps)
- Figure 11: Major European markets – average weighted download speed (Mbps)
- Figure 12: Major Western markets – percentage of requests made on 3G and LTE
- Figure 13: Download speed vs Percentage of LTE requests
- Figure 14: Major Western operators – average total roundtrip latency (ms)
- Figure 15: Major Western markets – average total roundtrip latency (ms)
- Figure 16: MobiNEX benchmarks