2018│11│13 – The P3 Mobile Network Test in Spain
The benchmarking expert P3 and connect magazine have been testing the Spanish mobile networks since 2015. This year, we have conducted our authoritative benchmark for the fourth time, further enhancing its methodology.
The results reveal some surprises: Two of the examined operators hold their position, while the twoothers were able to improve considerably. This also leads to a change in the overall ranking compared to last year‘s results.
Results in a nutshell
Vodafone has won the P3 connect Mobile Benchmark Spain the fourth time in a row, with Orange and Movistar constantly competing for the second rank, and Yoigo steadily working on improvements. These overall trends are still true for 2018.
P3‘s network benchmarks are widely accepted as the de-facto industry standard being highly objective. The carefully designed methodology of our 2018 benchmark in Spain combines drivetests and walktests for executing detailed voice and data measurements under controlled circumstances with a sophisticated crowdsourcing approach. This provides profound insights into the overall coverage of voice, data and 4G services, real-world User Download Speeds and Data Service Availability.
P3‘s holistic approach to network benchmarking includes both drivetest and walktest measurements as well as crowdsourcing. The drive- and walktests allow for evaluating the cutting edge of the networks‘ capabilities. Crowdsourcing unveils the service quality and performance actually experienced by the users. We have thoroughly weighted these components in order to give a realistic and authoritative assessment of the rated networks‘ true potential and performance.
VODAFONE IS THE OVERALL WINNER, ORANGE OVERTAKES MOVISTAR, YOIGO IMPROVES CLEARLY
As in the three previous years, the overall winner is Vodafone. On the second rank, Orange manages to overtake Movistar thanks to distinct improvements particularly in its voice results. Movistar still shows a strong performance especially in the data category, but in the overall assessment only achieves the third rank this time. The smallest operator in Spain, Yoigo, has considerably improved compared to its results in the previous years. This is true in the data and also especially in the voice category. In a direct comparison to last year‘s results, Vodafone overall kept the same performance while Movistar dropped slightly.
In the additional crowdsourced assessmentwhich contributes 15 per cent to the total result this year, Orange takes the overall lead, scoring slightly ahead of Movistar. Vodafone achieves the best score for User Download Speed, Orange and Movistar are on par. However, all operators
show some room for improvement. Yoigo‘s improvements are also reflected in a decent crowd score. The Data Service Availability is excellent in all four Spanish networks.
Vodafone España has been present on the Spanish mobile communications market since the year 2000. Then, the British Vodafone Group acquired Airtel Móviles which had operated in Spain since 1994. In Spain, Vodafone reports 14.1 million mobile customers, adding up to a market share of about 26 per cent and making Vodafone the third largest operator in the country following at close distance behind its competitor Orange. In the fiscal year 2017/2018, Vodafone Spain achieved revenues of €4.5 billion which contributes about eleven per cent to the whole Vodafone Group‘s financial results. Vodafone‘s mobile network in Spain offers GSM service at 900 and 1800 MHz, UMTS/3G at 900 and 2100 MHz and LTE at 800, 1800, 2100 and 2600 MHz. The Vodafone 4G network in Spain supports LTE carrier aggregation (“4G+“) with maximum downlink speeds of 300 Mbps.
Vodafone España claims to offer the best LTE coverage in Spain.
A close look at Spain’s networks
P3, based in Aachen, Germany, is a world leader in mobile network testing. The company has over 3,700 employees worldwide and a turnover of more than 370 million Euros. P3 is partnering with the international telecommunications magazine connect, which has 25 years of editorial expertise and is one of the leading test authorities in Europe for telecommunications products and services. Together, P3 and connect have been conducting the most important network benchmark test in Germany for more than 15 years, extending it to other European countries since 2009.
A close look at the spanishalone, P3 compiled approximately 60,000 measurement hours in more than 80 countries across five continents, with its test cars covering almost one million kilometres. As the de-facto industry standard, P3‘s benchmarking methodology focuses on customer-perceived network quality.
The 2018 P3 connect Mobile Benchmark Spain consists of drivetests and walktests conducted throughout September 2018. Four drive test cars together covered about 12,400 kilometres. The test areas account for more than 12 million people, or roughly 25.8 per cent of the total population of Spain. In addition, the results of extensive crowdsourcing analyses considering July, August and September (Data Service Availability: May to September) 2018 are included in the score.
Many customers use voice services less intensely than data. However, when actually taking or placing a phone call, they expect reliable connections. How do the Spanish networks fulfil these expectations?
Among the four Spanish operators, Orange and Vodafone support Voice over LTE (VoLTE) in their networks. VoLTE transmits voice calls as data packets over a 4G connection. This way, the otherwise necessary “circuit-switched fallback“, which forces smartphones to switch back to 3G or 2G in order to take or place a phone call, can be avoided. Movistar and Yoigo still have to take this detour. Also, VoLTE codecs potentially support a wider audio bandwidth providing operators with the opportunity to deliver higher speech quality to their customers.
On their tour through Spain, P3‘s four test cars visited 19 cities with a population of more than 100,000 inhabitants, as well as 29 towns. Also, the cars covered about 8,400 km of connecting roads. Additionally,a walktest team conducted tests in seven cities. For the voice rating, each car carried a total of four Samsung Galaxy S8 smartphones (one per operator). The walktest team carried a total of four Galaxy S8 – one per operator. The phones in the cars always called a counterpart in one of the other cars. The phones carried by the walktest teams called a stationary counterpart. The connected testing equipment registered success ratios, setup times and speech quality. In order to simulate normal smartphone usage, data transfers took place in the background of the test calls.
VODAFONE SHOWS STRONGEST VOICE RESULTS IN THE DRIVETESTS IN CITIES AND ON THE CONNECTING ROADS
In the detailled analysis of the measurement values gathered in the voice tests, Vodafone shows the strongest performance in the drivetests conducted in the larger cities and also on the connecting roads. In these scenarios, Movistar distinctly ranks third and Yoigo fourth. However, compared to the results of the previous years, Yoigo has clearly improved. Orange achieves the shortest connection times in all scenarios. In terms of speech quality, Vodafone and Orange offer the best results in all cases. On the roads, Vodafone is slightly ahead of Orange.
VODAFONE SHOWS THE BEST VOICE RESULTS, ORANGE RANKS SECOND
In the drivetests conducted in smaller towns, Orange is slightly ahead of Vodafone, particularly thanks to its excellent call success ratios in these cases. Yoigo and Movistar reach the same rank in the towns, but in some of the considered KPIs such as success ratios or call setup times Yoigo is even slightly ahead of the Telefónica network. This makes up for the fact that Yoigo is behind the competition especially on the connecting roads to a certain extent.
ORANGE AND VODAFONE ON PAR IN THE WALKTEST, MOVISTAR AND YOIGO CLOSE TOGETHER
In the walktests that P3 conducted in seven larger cities (Barcelona, Madrid, Murcia, Palma de Mallorca, Sevilla, Valencia and Zaragoza), Orange and Vodafone are on a par. Movistar is ranking third, and Yoigo is following at close distance. So, for pedestrians or cyclists in the larger cities, Yoigo has definitely become a viable alternative to its bigger competitors.
VOICE RESULTS AT A GLANCE
The clear winner in the voice discipline is Vodafone, with Orange ranking second but still showing very good results. Movistar scores clearly behind the leading two contenders and ranks third in this discipline. Yoigo is behind its competitors particularly on the connecting roads but has improved considerably. In the towns, it even scores better than Movistar.
110,000 Spanish users have contributed around 500 million measurement samples in the period of July to September 2018. A thorough and sophisticated analysis of this extensive data base allows us to add a distinct picture of the actual user experience to our test results.
While the drivetests and walktests determine the peak performance of the examined networks, crowdsourcing can add important dimensions such as time, geography or variety in devices and tariff plans – if done in the right way. A detailed description of our crowdsourcing methodology can be found on page 13. Based on the total population count of 46.6 million people, one of
424 inhabitants of Spain have contributed to the gathering of our crowd data.The test area of our crowdsourcing represents 71 per cent of the built-up area of Spain
VERY GOOD VOICE AND DATA COVERAGE IN ALL FOUR NETWORKS, 4G COVERAGE SHOWS ROOM FOR IMROVEMENT
The determined coverage with voice and data services scores at high levels in the test area. The same applies for the so called Quality of Coverage which takes the actual availability of the respective services into account. As can be expected, both values are a little lower for the coverage with 4G services – especially here, the crowdsourcing results indicate some room for improvement.
VODAFONE DELIVERS BEST USER DOWNLOAD SPEEDS; SLIGHTLY AHEAD OF ORANGE AND MOVISTAR; YOIGO BEHIND
The results of the analysis of User Download Speeds on the whole confirms what could be expected based on the results of the data drivetests. In this category, Vodafone scores slightly ahead of Orange and Movistar with Yoigo following at a distinct distance. In comparison to the theoretical maximum data speeds, all operators show room for improvement. However, it should be taken into consideration that a part of the actual user base probably experiences data speed limitations caused by their mobile tariffs.
VODAFONE AND ORANGE SHOW NO DEGRADATIONS FROM MAY TO SEPTEMBER, OVERALL HIGH DATA SERVICE AVAILABILITY
Examining the Data Service Availability in the observation period (which for this KPI covered the
five months from May to September 2018), shows pleasant results.
For Vodafone and Orange we did not identify any degrations. The other two contenders suffered from only minor outages: For Movistar, the analysis found three degraded hours on two days in May and June. For Yoigo, we determined an outage of up to one hour on one observation
day in September.
CROWD RESULTS AT A GLANCE
In the crowd-based score, Orange takes the lead, slightly ahead of Movistar. Vodafone achieves the best score for User Download Speed, Orange and Movistar are on a par – but all operators show some room for improvement. Yoigo‘s improvements are also confirmed by a decent crowd score. The Data Service Availability is excellent in all four Spanish networks.