2017│11│28 – The Mobile Network Test in Germany, Austria and Switzerland
Download the report as PDF here
The great 2018 Mobile Network Test
In its 24th year, the connect mobile network test is still considered as the highest benchmark in the mobile communications industry. Together with our well-renowned benchmarking partner P3 communications, we have once again investigated which mobile operators in Germany, Austria and Switzerland are ahead – with the utmost effort and our customer-oriented testing methodology.
The procedure is reminiscent of exciting elections: After extensive preparations and intricate investigations, the candidates face their moment of truth. Going to the ballot box corresponds to the drivetests and walktests that the network benchmarking specialist P3 communications has conducted in Germany, Austria and Switzerland. Election campaigns and demoscopic surveys have their equivalents within the intensive preparations and the precise gathering of measurement values over the course of several weeks here.
At the end of the process, the P3 experts thoroughly check and evaluate all measurement values, while the network operators tensely anticipate the results – much like politicians wait for the outcome of the elections. In our case as well, the candidates vigorously speculate about their performances until the analyses are complete. Only at the very end, all facts are on hand.
In the next step, not all of the participants are exactly happy about the results. But everyone has to live with the consequences. While in politics, the negotiations about forming a government start, the mobile network operators are concerned about the meaning of their test results for their ongoing network upgrades and not least for their marketing activities.
But there are also some distinct differences between our network tests and the course of democracy: Opinion polls are completely irrelevant for our assessments, and the process leading to the final result is considerably more complicated than just ticking boxes on a list. After all, we permanently enhance our methodology and framework in order to ensure the validity of our test results.
So how do we consistently challenge the relevance of our criteria and evaluations? It begins with staying up-to-date, and representing state-of-the-art-technology. You can read the conclusions that we drew from these considerations from page 66 on. Also, you can find an outlook to our plans about even extending the scope of our quality assessments in the near future from page 64 on.
But now, let‘s make way for the analyses, trends and results of this year‘s mobile network test. One or two aspects may look familiar to frequent readers. But we can also guarantee a number of surprises on the following pages.
Even if many users are focussing on data communications, they still expect a high connection quality when talking on the phone. Where can it be found?
In order to make the mobile internet and a high voice quality coexist well on smart phones, all three German operators rely on VoLTE. The acronym stands for Voice over LTE, which means conducting phone calls via LTE-based data packets.
The drivetests and walktests conducted by P3 show that this is working effectively by now. The Samsung Galaxy S7 phones that were installed in the test cars as well as in the backpacks of the walktest teams permanently established voice calls to their defined counterparts. The calls that originated in the test cars were set up to the according smartphones in another car. The phones in the walktest backpacks called stationary receivers. In order to simulate common smart phone utilisation, the mobile phones transmitted parallel data streams in the background of the telephony tests.
This, however, does not mean that all test calls actually used VoLTE. If one of the participating mobile phones was not registered in a LTE network, the call was connected in the “circuit-switched“ mode of the older mobile communications standards.
Close voice race between Vodafone and Telekom
The test drives in larger cities conveyed the same picture that also could be observed in the subsequent disciplines: The race between Telekom and Vodafone was neck to neck, O2 followed at considerable distance.
Still, the Telefónica network achieved far better results in the big city walktests compared to the results of the drivetests. This indicates that O2 offers better coverage in inner-city locations where there is a lot of foot traffic.
Compared to last year‘s results, Vodafone managed to considerably improve the call set-up times in its network. In this regard, the Dusseldorf-based provider beats its competitors in all tested scenarios with the only exception of voice calls in railways. Deutsche Telekom holds up with slightly higher success rates – again, with the exception of rail journeys.
In smaller towns and on the connecting roads, both contenders offer high quality as well. Telefónica‘s network, that still suffers from the ongoing integration of O2‘s and E-Plus‘ formerly separate radio cells, falls back considerably outside of the big cities. In those locations, we see a distinct drop in the level of the measured values.
When it comes to conducting phone calls in trains, the performances of neither operator are a glorious chapter – especially when bearing in mind that the results in this discipline looked far better last year. Despite Vodafone being a small step ahead, there is much room for improvement in all considered networks.
The mobile internet and audio/video streaming are booming. How do the mobile networks cope with the high demand?
We are awarding 60 per cent of the achievable points in the data discipline. This pays tribute to the fact that internet communications and streaming applications nowadays have the biggest share in smartphone usage. The demanding measurements conducted by the drivetest and walktest teams are also reflecting these conditions: The smartphones regularly access the most popular web sites according to the renowned Alexa ranking. Furthermore, they invoke the static ETSI reference web page, also known as “Kepler page“. We evaluate the speed and reliability of data transfers with downloads of 3 MB and uploads of 1 MB test files. Furthermore, we gauge the amount of data travelling over the network in a seven-second period.
This is complemented with playing back Youtube videos. Here we examine the success ratios and how much time passes until the playback starts. Above that, we look at the percentage of videos that run through without interruptions as well as their average image resolution.
In order to account for the behaviour of various smartphones, we appointed two different phone models to the data tests. In addition to the Samsung Galaxy S7 phones that were also used for the voice tests, half of the measurements were conducted with Sony Xperia XZ.
The data tests in the big cities confirm the results of the data discipline – almost regardless of the number of the cities’ inhabitants. In comparison to the previous year‘s results, the leading duo Telekom and Vodafone succeed in taking their performances up a notch especially in urban areas. We are particularly impressed by the high success ratios of the leading duo. In big cities and smaller towns alike, this year‘s network test confirms the high availability and stability of LTE mobile network coverage.
However, Telekom takes a clear step ahead of Vodafone in the data discipline and builds the foundation of this year‘s overall win on this. The lead is most distinct regarding the upload and download data rates. Above that, the really fast transmission speeds in both networks are a good indication of the advanced stage of completion of the so-called “carrier aggregation“ – increasing data rates by means of combining several LTE frequency bands.
Strong Deutsche Telekom
Performance losses at O2
But our test results also expose the problems currently met by Telefónica and its customers. Since O2 acquired its former competitor E-Plus in October, 2014, the technicians have been working hard in order to consolidate the formerly separate cells of both networks. The measurement values at hand clearly indicate that this task is everything but trivial. The O2 network already ranked distinctly behind Telekom and Vodafone in our previous mobile network test (2016/2017). But this year, the gap did actually increase.
O2 manages to keep up its last year‘s level in the drivetests conducted in larger cities. However, the operator massively loses points in the inner city walktests. This suggests that fine tuning the merged networks is especially demanding at locations with high numbers of users simultaneously accessing the mobile network. Considerably lower success ratios and data rates than those delivered by Telekom and Vodafone result in a significant loss of points for Telefónica. Customers who join O2 due to its undoubtedly attractive tariffs must, at his point in time, be somewhat patient especially when they are using the mobile internet.
Vodafone makes up leeway over Telekom in smaller towns
In the drivetest conducted in smaller towns, O2 also ranks substantially lower than in the previous year. At least, the level of results rises a little in this scenario in comparison to the inner city walktests. As was to be expected, the leading duo also shows somewhat weaker results in the smaller towns than in the metropolises. Interestingly, in this category, Vodafone catches up closely to Telekom. Especially the success ratios of the Dusseldorf-based contender are a tiny step ahead of its Bonn-based competitor. Telekom counters this with slightly better data rates as well as an excellent Youtube performance.
All in all, Telekom and Vodafone customers in cities and towns can be very pleased with their operators. For O2, this currently applies only with some restrictions.
Communication and connectivity play an ever increasing role behind the wheel. So, what can be found out about mobile network coverage on German roads?
On their rides between the bigger cities and smaller towns considered in our test route, the four P3 test vehicles covered a distance of about 6,200 kilometres. This came on top of the approximately 3,400 kilometres that they drove within the cities and towns. On these roads, the cars also conducted permanent data measurements in order to investigate the quality and performance of online connectivity there.
Distinct ranking order on connecting roads
After observing the voice performance as well as the data tests in cities and towns, the ranking determined on the connecting roads comes hardly as a surprise: Telekom and Vodafone fight closely and at eye level. O2 follows at a distance, but achieves better results than in the voice discipline.
All in all, Telekom gains the win in this category – but Vodafone hold up very well too. When customers of O2 are travelling on the roads, they should expect lower success ratios when surfing the web, transmitting data or accessing audio/video streams as well as slower data rates.
Our test results clearly emphasize: For a high level of connectivity on German autobahns and country roads, there is currently hardly a viable alternative to the offerings of Telekom and Vodafone.
DATA ON RAILWAYS
Leaning back and using your mobile devices while riding a train – how well does this actually work?
“When it comes to connectivity in German railways, there remains a lot of work to be done.“ This was the conclusion of the corresponding section of our mobile network test one year ago. So, this year, our test teams started their tasks with eager expectations and successfully completed more than 30 hours of travelling in various ICE and IC trains. Especially in the ICE flagship trains, Deutsche Bahn has recently made extensive upgrades to their cellular and online coverage.
In the face of these efforts, the results are somewhat disappointing – particularly as all three German mobile operators were further ruffled in comparison to the mobile connectivity results obtained in the previous year. However, for the sake of fairness, we have to mention that the test connections did not make use of the WiFi hotspots that are meanwhile available in almost all ICE trains, but exclusively used the mobile networks. Still, they too should be receivable in good quality due to elaborate repeater technology in the railway waggons. Unfortunately, the result is still indisputable: When looking across the borders to Austria and especially to Switzerland, German railway customers can be only jealous. Both alpine countries are ahead in this discipline at an enormous distance.
Still need for improvement on German railways
It is not a very high level on which Vodafone beats the overall winner Telekom in this discipline. Success ratios and data rates of web-page downloads and Youtube playbacks as well as file transfers leave a lot to be desired in all three considered networks. The weakest candidate in this category, O2, failed at a quarter of all attempts to use an online connection in trains. When accessing Youtube, this number even rose to about half of the attempts. So this year‘s conclusion sadly is: When it comes to using the internet in German railways, there still remains a lot of work to be done.
Deutsche Telekom managed to grow once more on its results of the previous year. And although its Dusseldorf-based competitor also improved, the Bonn-based operator celebrates its seventh win in a row on the occasion of our network test 2017/2018. Telekom gains the overall win particularly in the data discipline. But also for Telekom, there still remains work to be done. This especially applies to voice and data connections in German railways.
In the voice discipline, Vodafone catches up with Telekom up to a distance of just one point. The final showdown and overall win of the Bonn-based competitor was decided in the data category – even if the Dusseldorfers also deliver a respectable performance in this discipline. So Vodafone fully deserves the overall grade “very good“. In the railways, Vodafone is even a small step ahead of Telekom – but there is still room for improvement in this category as well.
Compared to last year‘s results, O2 falls behind severely. The supposed reason for this are the challenges of the still ongoing network merger with E-Plus. So, in our network test 2017/ 2018, Telefónica again only achieves the overall grade “sufficient“. We send our best wishes to the Munich-based operator and its customers, hoping that this difficult phase may soon be over and that next year‘s test results will reflect true advancements