2018│12│20 – The 2018 Mobile Network Test in the United Kingdom
Download the report as PDF here
THE 2018 MOBILE NETWORK TEST IN THE UNITED KINGDOM
FOR THE FIFTH TIME IN A ROW, THE BENCHMARKING EXPERT UMLAUT AND THE ACKNOWLEDGED TELECOMMUNICATIONS MAGAZINE CONNECT HAVE CONDUCTED THEIR MOBILE NETWORK BENCHMARK IN THE UK. IN A MARKET CHARACTERISED BY ITS CONTINUOUS UPGRADING OF 4G NETWORKS, GROWING DATA RATES AND ENHANCEMENTS SUCH AS VOICE OVER LTE, WE WANT TO FIND OUT: WHICH OPERATOR OFFERS THE BEST MOBILE NETWORK?
Results in a nutshell
Umlaut‘s network benchmarks are widely accepted as being highly objective and defining the de-facto industry standard The carefully designed methodology of the 2018 benchmark in the UK combines drivetests and walktests for executing detailed voice and data measurements under controlled circumstances with a sophisticated crowdsourcing approach. This provides profound insights into the overall coverage of voice, data and 4G services, real-world User Download Speeds and Data Service Availability. For the first time in the UK, this year we have extended our walktest to also cover railway connections. Umlaut‘s holistic approach to network benchmarking includes both drivetest and walktest measurements as well as crowdsourcing. The drive and walktests allow for evaluating the cutting edge of the networks‘ capabilities. Crowdsourcing unveils the service quality, performance and coverage actually experienced by the users. We have thoroughly weighted these components in order to give a realistic and authoritative assessment of the rated networks‘ true potential and performance.
EE TAKES THE OVERALL LEAD, VODAFONE ALSO STRONG IN CITIES AND ON THE ROADS. THREE OFFERS BEST DATA PER-FORMANCE IN TOWNS, O2 RANKS SECOND IN CROWD SCORE.
In the voice category, EE is clearly leading, showing strong results in all tested scenarios and delivering excellent speech quality due to its use of the modern EVS (Enhanced Voice Services) codec. Vodafone offers excellent success ratios for voice calls in the city walktests together with EE. Three ranks third and O2 last, with both offering good voice results in the towns. In the data tests, EE shows the best results as well. This is applicable for the city drivetests, on the roads and – at a lower level – on railways. In the city walktests, EE and Vodafone are on a par, with the other two lagging behind. Three shows the best data performance in towns. In the crowd score, EE takes the lead in all categories, particulary in the crowdsourced quality. It did not suffer any service degradations from May to October 2018. O2 ranks second in the crowdsourcing and Vodafone third, only one point behind O2. Three follows at some distance on the last rank in the crowd discipline but proves to be also very reliable with only hour of service degradation in the observation period. All UK operators show considerable room for improvements on the railways.
With approximately 30 million customers, EE (formerly Everything Everywhere) is the biggest mobile network operator in the UK. Since January 2016, EE has been part of the British Telecom Group. EE started offering a 4G service in October 2012.Regarding their 4G coverage, EE has chosen to report geographic coverage instead of population coverage from now on. In this context, they refer to the Connected Nations Report by UK’s regulator Ofcom and thus quote to currently offer 84 per cent 4G geographic coverage. EE operates its 4G network at 800 MHz, 1800 MHz and 2600 MHz. Additionally, it operates 2G at 1800 MHz and 3G at 2100 MHz which also reach a population coverage in the 98 to 99 per cent range. EE operates a growing number LTE Advanced network cells that support speeds of up to 450 Mbps under the name “4GEE”. Also, EE offers Voice over LTE (VoLTE) in most of its 4G network. All plans come with 4G at no extra cost.
WITH THE VOLUME OF TRANSMITTED DATA PERMANENTLY GROWING, DATA CONNECTIVITY CONSTANTLY BECOMES MORE IMPORTANT. WHICH OPERATOR IN THE UK MANAGES BEST TO MEET THE INCREASING DEMAND?
EE TAKES THE OVERALL LEAD IN THE DATA DISCIPLINE AS WELL. IN THE CITY WALKTESTS, VODAFONE IS ON A PAR WITH EE. THREE SCORES BEST IN TOWNS.
Data connectivity is the most prestigious discipline in our benchmark and also in the operators’ marketing. All four UK networks claim to cover a large part of the population with LTE services – the claimed percentages range in the high nineties. Three states a combined 3G/4G data coverage of 99 per cent of the population and specifies that about 84 per cent of its 10.1 million customers are using 4G (also see page 3). All four operators are continuing to spend millions and billions on upgrading and expanding their networks to meet the growing demand. However, the four operators pursue different rollout strategies. EE and Vodafone chase each other with continually growing data rates that currently go up to 1 Gbps, based on the so-called carrier aggregation (the combination of multiple carrier frequencies). In contrast, O2 and Three stick around a solid 100 Mbps and mainly focus on enlarging their 4G footprints.
UMLAUT’S SCORING REWARDS HIGH PERFORMANCE AS WELL AS THE NETWORKS’ AVAILABILITY AND STABILITY
In order to assess the performance and reliability of data connections, each of our four drivetest cars and also the walktest team carried one Samsung Galaxy S8 per operator. Supporting the LTE category 16, these smartphones were able to benefit from carrier aggregation up to 1 Gbps. Umlaut‘s testing considers such fast throughputs as well as the networks‘ availability and stability.
IN THIS YEAR, THE RESULTS OF CROWDSOURCING ANALYSES ARE PART OF THE TOTAL SCORE FOR THE FIRST TIME. 268,000 USERS IN THE UK HAVE CONTRIBUTED TO THE DATA GATHERING THAT TOOK PLACE FROM AUGUST TO OCTOBER, 2018.
EE TAKES THE LEAD IN CROWDSOURCED QUALITY AND OVERALL IN ALL CROWDSOURCED CATEGORIES.
While the drivetests and walktests determine the peak performance of the examined networks, crowdsourcing can add important dimensions such as time, geography or variety in devices and tariff plans – if done in the right way. For the collection of crowd data, P3 has integrated a background diagnosis processes into more than 800 diverse Android apps. If one of these applications is installed on the enduser’s phone and the user authorizes the background analysis, data collection takes place 24/7, 365 days a year. Reports are generated for every quarter of an hour and sent daily to P3‘s cloud servers. Such reports generate just a small number of bytes per message and do not include any personal user data. A more detailed description of our crowdsourcing methodology can be found on page 15 of this report. Based on the total population count of 66 million people, one of 246 inhabitants of the United Kingdom has contributed to the gathering of 2.99 billion single samples of crowd data. The considered test area represents 99 per cent of the built-up area of the UK.
The umlaut connect Mobile Benchmark in the UK comprises of the results of extensive voice and data drivetests and walktests as well as a sophisticated crowdsourcing approach.
DRIVETESTS AND WALKTESTS
The drivetests and walktests in the UK took place from October 30 to December 3, 2018 All samples were collected during the day, between 8.00 a.m. and 10.00 p.m. The network tests covered inner-city, outer metropolitan and suburban areas. Measurements were also taken in smaller towns and on the connecting highways. The four measurement cars together covered about 3,750 kilometres in the cities, about 1,700 km in towns and about 6,300 km on the roads – resulting in a total of 11,750 kilometres. The combination of test areas has been selected to provide representative test results across the UK population. The areas selected for the 2018 test account for approximately 16.6 million people, or roughly 26.2 per cent of the total population of the UK. The drivetests covered 22 cities and 35 towns. Additionally, two teams conducted walktests in 10 cities and also on railway journeys between them. The routes are shown on page 1, all visited cities and towns are listed in the box on the right. The four drivetest cars as well as the battery-powered backpacks of the walktest teams were equipped with arrays of Samsung Galaxy S8 smartphones for the simultaneous measurement of voice and data services.
EE IS THE WINNER WHILST VODAFONE ACHIEVED A STRONG SECOND PLACE RANK. THREE SHOWS SOME SPECIFIC STRENGTHS BUT FELL BEHIND IN THE OVERALL SCORING, WHICH IS SIMILAR FOR VODAFONE. O2 PERFORMED SIMILARLY TO LAST YEAR.
EEs confirms its top rank from previous years and wins the umlaut connect Mobile Benchmark in the UK for the fifth time in a row (in 2016, EE and Vodafone won together). Compared to last year’s results, EE improves significantly in the voice discipline, but achieves a slightly decreased data score. In the now extended crowd analyses, EE is also the winner. Vodafone achieves a good second rank, showing especially good results in the big cities and on the roads. In comparison to the previous year, Vodafone managed to improve in the voice results, but lost some ground in the data scores. The same is also true for Three, which fell back to the grade “satisfactory”. A remarkable strength of this smallest operator in the UK is that it achieved the best data score in towns. Like all UK operators, Three was also able to increase its performance on the roads for both voice and data. O2 ranks last. Showing more or less the same performance than last year costs some points, because the requirements of our testing methodology increase year over year. In the crowdsourced assessment, O2 achieved the second-best results, one point ahead of Vodafone. However, our newly added railway measurements reveal big challenges for all UK operators.
The overall winner of the 2018 P3 connect Mobile Benchmark in the UK is EE. This operator clearly leads in all tested disciplines including the now extended crowd score and also leads the field in most of the single categories of our assessment. EE’s strong voice results are based on its use of the modern EVS codec.
Last year’s report can be downloaded here