2018│11│13 – The P3 Mobile Network Test in Spain

The benchmarking expert P3 and connect magazine have been ­testing the Spanish mobile ­networks since 2015. This year, we have conducted our authoritative benchmark for the fourth time, further enhancing its methodology.

The results reveal some surprises: Two of the examined operators hold their position, while the twoothers were able to improve ­con­siderably. This also leads to a change in the overall ranking   compared to last year‘s results.

Results in a nutshell

Vodafone has won the P3 connect Mobile Benchmark Spain the fourth time in a row, with Orange and Movistar constantly competing for the ­second rank, and Yoigo steadily working on improvements. These overall trends are still true for 2018.

P3‘s network benchmarks are widely accepted as the de-facto industry standard being highly objective. The carefully designed ­methodology of our 2018 benchmark in Spain com­bines ­drivetests and walktests for executing detailed voice and data measurements under ­controlled circumstances with a sophisti­cated crowdsour­cing approach. This provides profound insights into the overall coverage of voice, data and 4G services, real-world User Down­load Speeds and Data Service Availability.

P3‘s holistic approach to network bench­marking includes both ­drivetest and walktest mea­surements as well as crowdsourcing. The drive- and walktests allow for evaluating the cutting edge of the networks‘ ­capabi­lities. Crowdsourcing unveils the ­service ­quality and ­performance ­actually experienced by the users. We have thoroughly weighted ­these components in order to give a realistic and ­authoritative assessment of the rated networks‘ true ­potential and performance.

VODAFONE IS THE OVERALL WINNER, ORANGE OVERTAKES MOVISTAR, YOIGO IMPROVES CLEARLY

As in the three previous years, the overall winner is Vodafone. On the second rank, ­Orange manages to overtake Movistar thanks to distinct improvements particularly in its voice results. Movistar still shows a strong ­performance especially in the data ­category, but in the overall assessment only achieves the third rank this time. The smallest operator in Spain, Yoigo, has con­siderably improved compared to its results in the previous years. This is true in the data and also ­especially in the voice ­category. In a direct comparison to last year‘s results, ­Vodafone overall kept the same performance while Movistar dropped slightly.

In the additional crowdsourced assessmentwhich contributes 15 per cent to the total result this year, Orange takes the overall lead, scoring slightly ahead of ­Movistar. ­Vodafone achieves the best score for User Download Speed, Orange and Movistar are on par. How­ever, all operators
show some room for improve­ment. Yoigo‘s improve­ments are also reflected in a decent crowd score. The Data Service Availability is excellent in all four Spanish networks.

Vodafone keeps its performance, Orange and ­Yoigo improved their scores ­considerably. Movistar lost a couple of points but still achieves good results.

Spain’s Operators

vodafone

orange_logo

logo_movistar

yoigo_azul

Vodafone España has been present on the Spanish mobile communications market since the ­year 2000. Then, the British ­Vodafone Group acquired ­Airtel ­Móviles which had operated in Spain ­since 1994. In Spain, ­Vodafone reports 14.1 ­million mo­bile ­customers, ­adding up to a market ­share of about 26 per cent and ­making ­Vodafone the third largest operator in the country following at close distance behind its competitor Orange. In the fiscal year 2017/2018, ­Vodafone Spain ­achieved revenues of €4.5 billion which ­contri­butes about eleven per cent to the whole ­Vodafone Group‘s finan­cial ­results. Vodafone‘s mobile network in Spain offers GSM ­service at 900 and 1800 MHz, UMTS/3G at 900 and 2100 MHz and LTE at 800, 1800, 2100 and 2600 MHz. The Vodafone 4G network in Spain supports LTE ­carrier aggregation (“4G+“) with maximum down­link speeds of 300 Mbps.

Vodafone ­España claims to offer the best LTE coverage in Spain.

Orange España is the brand ­name of France Telecom‘s mobile network in Spain. It has been operating under this name since 2006. Previously, the network was known as “Amena“ – this brand name lives on in Orange Spain‘s portfolio as a low-cost offer that is only available on the internet. Also, its network serves a number of mobile virtual network operators such as MasMovil, Carrefour Móvil and others. With 15.9 million custo­mers, ­Orange is the second largest Spanish mobile operator with a market share of about 30 per cent. In the fiscal year 2017, Orange Spain achieved a revenue of €5.3 ­billion which contributed 13 per cent to the whole Orange Group‘s results. Orange Spain has deployed 2G networks at 900 and 1800 MHz, 3G networks at 900 and 2100 MHz and 4G at 800, 1800 and 2600 MHz. The operator claims that its 4G network ­reaches more than 96 per cent of the Spanish ­population. ­Also, Orange was the first Spanish operator to offer VoLTE to its 4G customers.

Movistar is the brand name the Spanish telecom­muni­­cations company ­Telefónica causes for the ­mobile network in its home market. Tele­­fónica S.A. itself is one of the largest telco companies in the world. The ­operator is active in 17 countries with a total of 122,700 ­em­ployees and achieved worldwide ­revenues of over €52 ­billion in its fiscal year 2017.

While the company intro­duced the ­Movistar brand in Latin American countries in 2005, it has been active in Spain since the launch of GSM services back in 1995. Today, Movi­star is the largest mobile ­operator in Spain with about 17.6 million subscribers, which equals a market share of roughly 33 per cent. It ­offers GSM ­service at 900 and 1800 MHz, UMTS/ 3G at 900 and 2100 MHz and LTE at 800, 1800 and 2600 MHz. Since the end of 2014, ­Movistar has supported 4G+ carrier ­aggregation with maximum speeds rea­ching today up to 375 Mbps. The operator claims to provide 4G coverage to approximatley 95 per cent of the Spanish ­population.

Yoigo was the latest mobile operator to enter the Spanish market. Founded in 2000 under the name Xfera, the company started its actual operation in 2006, offering only a UMTS/3G network at 2100 MHz. At this time, the Swedish telecommunications company TeliaSonera acquired the majority of shares and rebranded the network as “Yoigo“. In June 2016, the former MVNO (mobile virtual network operator) Másmóvil bought the company. For its fiscal year 2017, Másmóvil reported revenues in Spain of €1.3 billion. Yoigo had a national roaming agreement with Movistar until the end of 2016. Since January 2017, Yoigo customers freely roam in the 2G and 3G networks of Orange at locations without Yoigo coverage. Yoigo claims a mobile customer base of 5.0 million mobile subscribers, which equals a market share of 11 per cent. Today, Yoigo operates 3G at 2100 MHz as well as 4G at 1800 MHz. The operator currently claims a LTE coverage of approximately 89 per cent of the Spanish population.

A close look at Spain’s networks

P3, based in Aachen, Germany, is a world leader in mobile network ­testing. The company has over 3,700 employees world­wide and a ­turnover of ­more than 370 mil­lion Euros. P3 is partnering with the ­international telecommunications magazine connect, which has 25 years of editorial expertise and is one of the leading test ­authorities in ­Europe for tele­communications ­products and services. ­Together, P3 and connect ­have been ­conducting the most impor­tant network benchmark test in Germany for more than 15 years, ­extending it to other ­European countries since 2009.

A close look at the spanishalone, P3 compiled approximately 60,000 mea­surement hours in more than 80 countries across five continents, with its test cars covering almost one million kilometres. As the de-facto industry standard, P3‘s benchmarking methodology focuses on customer-perceived network quality.

The 2018 P3 connect ­Mobile Benchmark Spain consists of drivetests and walktests conducted throughout September 2018. Four drive test cars together covered about 12,400 kilometres. The test areas account for more than 12 million people, or roughly 25.8 per cent of the total popula­tion of Spain. In addition, the results of ­extensive crowdsourcing ­analyses considering July, August and ­September (Data Service Availa­bility: May to September) 2018 are included in the score.

Hakan Ekmen, Managing Director P3 communications GmbH

“Our holistic approach clearly confirms that the Spanish operators have worked hard to improve their network performance and user experience. We are happy to see some considerable enhancements compared to the results of previous years and are quite sure that our demanding benchmarks have played a role in the serious efforts taken by the Spanish operators recently.”

Voice

Many customers use voice services less intensely than data. ­However, when actually taking or placing a phone call, they expect reliable connections. How do the Spanish networks fulfil these expectations?

Among the four Spanish operators, Orange and Vodafone support Voice over LTE (VoLTE) in their networks. VoLTE transmits voice calls as data packets over a 4G connection. This way, the otherwise necessary “­circuit-switched fallback“, which forces smartphones to switch back to 3G or 2G in order to take or place a phone call, can be avoided. ­Movistar and Yoigo still have to take this detour. Also, VoLTE codecs ­potentially ­support a wider audio bandwidth providing ­operators with the opportunity to deliver higher speech quality to their customers.

On their tour through Spain, P3‘s four test cars visited 19 cities with a population of more than 100,000 inhabitants, as well as 29 towns. Also, the cars covered about 8,400 km of connecting roads. Additionally,a walktest team conducted tests in seven cities. For the voice ­rating, each car carried a total of four Samsung ­Galaxy S8 smartphones (one per operator). The walktest team carried a total of four ­Galaxy S8 – one per operator. The ­phones in the cars always called a counterpart in one of the other cars. The phones carried by the walktest ­teams called a ­stationary counterpart. The connected ­testing equipment registered success ­ratios, ­setup ­times and speech quality. In order to simulate normal smartphone usage, data transfers took place in the back­ground of the test calls.

VODAFONE SHOWS STRONGEST VOICE RESULTS IN THE DRIVETESTS IN CITIES AND ON THE CONNECTING ROADS

In the detailled analysis of the measurement values gathered in the voice tests, Vodafone shows the strongest performance in the drivetests conducted in the ­larger cities and also on the connecting roads. In these scenarios, Movi­star distinctly ranks third and ­Yoigo fourth. However, ­compared to the results of the previous years, Yoigo has clearly improved. Orange achieves the shortest connection times in all ­scenarios. In terms of speech quality, ­Vodafone and Orange offer the best results in all cases. On the roads, ­Vodafone is slightly ahead of Orange.

VODAFONE SHOWS THE BEST VOICE RESULTS, ORANGE RANKS SECOND

In the drivetests conducted in smaller towns, ­Orange is ­slightly ahead of Vodafone, ­particularly thanks to its excellent call success ratios in these cases. Yoigo and Movistar reach the same rank in the towns, but in some of the ­considered KPIs such as success ratios or call setup ­times ­Yoigo is even slightly ahead of the Tele­fónica network. This makes up for the fact that Yoigo is behind the ­competition especially on the connecting roads to a certain extent.

ORANGE AND VODAFONE ON PAR IN THE WALKTEST, MOVISTAR AND YOIGO CLOSE TOGETHER

In the walktests that P3 con­ducted in seven larger ­cities (Barcelona, Madrid, Murcia, Palma de ­Mallorca, Sevilla, Valencia and ­Zaragoza), Orange and  Vodafone are on  a par. Movistar is ­ranking third, and Yoigo is following at ­close ­distance. So, for ­pedes­trians or ­cyclists in the larger ­cities, Yoigo has defi­nitely become a viable ­alternative to its bigger competitors.

VOICE RESULTS AT A GLANCE

The clear winner in the voice dis­cipline is Vodafone, with ­Orange ranking second but still ­showing very good results. Movistar scores clearly behind the leading two contenders and ranks third in this discipline. Yoigo is behind its competitors particularly on the connecting roads but has improved considerably. In the towns, it even scores better than Movistar.

Data

With the volume of transmitted data permanently growing, the data discipline constantly becomes more ­important. How do the Spanish operators keep up with the increasing demand resulting from this development?

According to their own claims (also see page 3), ­Movistar, Orange and Vodafone are competing for the best LTE coverage as well as for delivering the highest data rates to their customers. Yoigo is still mainly concentrating on expanding its 4G footprint and claims to have reached 89 per cent LTE coverage of the Spanish population now.

In order to assess the performance and reliability of data connections, each of our four drivetest cars and also the walktest team ­carried one Samsung ­Galaxy S8 per operator. Supporting the LTE category 9, these smartphone types were able to ­benefit from the so-called carrier aggregation – the combined use of three LTE ­carrier frequencies which can theoretically transmit up to 450 Mbps. P3‘s testing considers fast throughputs as well as the networks‘ ­availability and ­stability. In order to assess typical performance as well as peak speeds, we consider two values: the ­minimum data rate that is available in 90 per cent of the cases, and additionally the peak data rate that is surpassed in 10 per cent of the cases.

Web page and file downloads or file ­uploads ­reward fast speeds, while determination of success ­ratios and ­assessing YouTube playouts concentrate on ­reliability ­aspects. As YouTube streams videos at adaptive bitrates, the average value of the received video ­resolution is another important performance indicator.

CLOSE RACE BETWEEN VODAFONE AND MOVISTAR IN DATA DRIVETESTS CUNDUCTED IN LARGER CITIES

In the drive tests conducted in 19 cities with a population of more than 100,000 inhabitants, we see a close race between Vodafone and Movistar. Orange and Yoigo follow behind, with Yoigo ranking only one per cent behind its big competitor.

All operators achieve excellent success ratios in the cities. Only in the category of ­YouTube live ­videos, Movistar performs slightly weaker. Overall, ­Vodafone achie­ves the highest score for web browsing. In the cities, it is on par with Movistar regarding this KPI. This also applies to file ­downloads and uploads, where Vodafone and the Telefónica network are on a par as well.

VODAFONE AND MOVISTAR LEAD THE FIELD IN THE DATA WALKTESTS THAT TOOK PLACE IN LARGER CITIES

The results of the walktests conducted in Barcelona, Madrid, Murcia, Palma de Mallorca, ­Sevilla, Valencia and ­Zaragoza are similar to those of the drive­tests that took place in the larger cities: Vodafone and ­Movistar lead the field, with Orange and Yoigo follow as runner-ups at a not too big distance. The lead of ­Vodafone and ­Movistar is mainly ­based on higher data rates and also on strong performance when playing back YouTube videos. In terms of success ratios, all  four Spanish operators achieve very good results except for ­Movistar when it comes to playing back YouTube live videos.

VODAFONE LEADs IN DATA DRIVE-TESTS in SMALLER CITIES, THE OTHER THREE OPERATORS ALMOST ON PAR

In the data drivetests that our measurement cars performed in 29 smaller Spanish towns, the ranking is a little more distinct: In this discipline, Vodafone is clearly leading, the rest of the field follows at some distance – including ­Yoigo which scores only one ­per cent behind ­Orange and ­Movistar ranking equally. ­Vodafone‘s lead is ­largely based on its strong perfor­mance in the file ­download and ­upload tests, while Yoigo ­somehow ­­sur­prises with very good YouTube ­results in the towns.

VODAFONE AND MOVISTAR SHOW BEST RESULTS ON CONNECTING ROADS, ORANGE AND YOIGO CLOSE TOGETHER

On the 8,400 km of connecting roads covered by our test cars, Vodafone and Movistar performed particularly well. Here, ­Orange and Yoigo lag a little behind – with ­Yoigo only scoring two ­per cent behind Orange. Still, all four ­Spanish operators show ­pleasantly ­good results in this especially ­demanding category.

DATA RESULTS AT A GLANCE

Vodafone is the clear winner in the data discipline as well. ­Movistar manages to outperform Orange in this category and competes in a neck-and-neck race with Vodafone, reaching the second rank in the data tests. Orange and Yoigo each follow at some distance. Still, both perform pleasantly well in the important data discipline – including their performance and availability on the 8,400 kilo­metres of connecting roads that were ­covered by our measurement cars.

CROWD

110,000 Spanish users have contributed around 500 million measurement samples in the period of July to September 2018. A thorough and sophisticated analysis of this extensive data base allows us to add a distinct picture of the actual user experience to our test results.

While the drivetests and walktests determine the peak performance of the examined networks, ­­crowdsourcing can add important dimensions such as time, geo­graphy or variety in devices and tariff plans – if done in the right way. A detailed description of our crowdsourcing ­methodology can be found on page 13. ­Based on the total popu­lation count of 46.6 ­million people, one of
424 ­inhabitants of Spain have con­tri­buted to the gathering of our crowd data.The test area of our crowdsourcing ­represents 71 per cent of the ­built-up area of Spain

VERY GOOD VOICE AND DATA COVERAGE IN ALL FOUR NETWORKS, 4G COVERAGE SHOWS ROOM FOR IMROVEMENT

The determined coverage with voice and data ­services scores at high levels in the test area. The same applies for the so called Quality of ­Coverage which takes the actual availability of the ­respective services into account. As can be expected, both values are a little lower for the ­coverage with 4G ­services – ­especially here, the crowdsourcing ­results ­­indicate some room for improvement.

VODAFONE DELIVERS BEST USER DOWNLOAD SPEEDS; SLIGHTLY AHEAD OF ORANGE AND MOVISTAR; YOIGO BEHIND

The results of the analysis of User Download Speeds on the ­whole confirms what could be ­expected ­based on the results of the data drivetests. In this ­category, ­Vodafone scores slightly ahead of ­Orange and ­Movistar with Yoigo following at a distinct distance.  In comparison to the ­theoretical maximum data speeds, all ­ope­rators show room for ­improvement. ­However, it should be taken into consideration that a part of the actual user base probably experiences data speed limitations ­ caused by their ­mobile tariffs.

VODAFONE AND ORANGE SHOW NO DEGRADATIONS FROM MAY TO SEPTEMBER, OVERALL HIGH DATA SERVICE AVAILABILITY

Examining the Data Service Availability in the observation period (which for this KPI covered the
five months from May to September 2018), shows pleasant results.

For Vodafone and Orange we did not identify any degrations. The other two contenders suffered from only minor outages: For ­Movistar, the analysis found three ­degraded hours on two days in May and June. For Yoigo, we ­determined an outage of up to one hour on one observation
day in September.

CROWD RESULTS AT A GLANCE

In the crowd-based score, Orange takes the lead, slightly ahead of Movistar. Vodafone achieves the best score for User Download Speed, Orange and Movistar are on a par – but all operators show some room for improve­ment. Yoigo‘s improvements are also confirmed by a decent crowd score. The Data Service Availability is excellent in all four Spanish networks.