2018|12|13 – THE 2018 P3 CONNECT MOBILE BENCHMARK IN AUSTRALIA

THIS YEAR, THE BENCHMARKING EXPERT P3 AND THE IN­TERNATIONALLY ACKNOWLEDGED TELECOMMUNICATIONS MAGAZINE CONNECT HAVE CONDUCTED THEIR FIFTH MOBILE NETWORK TEST IN AUSTRALIA.  AFTER A CLOSE RACE BETWEEN OPTUS AND TELSTRA AND A GOOD THIRD RANK FOR VODAFONE IN THE PREVIOUS YEAR, HOW DID THE AUSTRALIAN OPERATORS  PERFORM THIS TIME?

RESULTS IN A NUTSHELL

AS IN THE PREVIOUS YEAR, WE AGAIN SAW A NECK-AND-NECK RACE BETWEEN THE TWO LARGEST AUSTRALIAN OPERATORS. THIS YEAR, TELSTRA MANAGES TO TAKE THE FIRST RANK BACK FROM AN ALSO VERY STRONG OPTUS. VODAFONE CONFIRMS ITS POSITION.

P3‘s network benchmarks are widely accepted as the de-facto industry standard being highly objective. The carefully designed methodology of our 2018 benchmark in Australia combines drivetests and walktests for executing detailed voice and data measurements under controlled circumstances with a sophisticated crowdsourcing approach. This provides profound insights into the overall coverage of voice, data and 4G services, real-world User Download Speeds and Data Service Availability.

P3‘s holistic approach to network benchmarking includes both drivetest and walktest measurements as well as crowdsourcing. The drive and walktests allow for evaluating the cutting edge of the networks‘ capabilities. Crowdsourcing unveils the service-quality, performance and coverage actually experienced by the users. We have thoroughly weighted these components in order to give a realistic and authoritative assessment of the rated networks‘ true potential and performance.

Telstra achieves a total of 884 points which equals the grade “very good”. Its narrow lead in the voice category is earned in the big city drivetests and its overall win by offering  the best speeds for the top 10 per cent of Evaluation Areas.
Optus ranks second with a total of 872 points, which is also a “very good” result. This operator leads in the data category, due to better results in smaller towns and on the roads. Also, Optus managed to improve its results in the voice and data categories compared to previous year’s score. In the crowdsourced assessment, Optus shows a particularly high Quality of 4G Coverage.
Vodafone can almost keep up in the cities and towns but clearly loses ground on the roads. The results of our crowdsourcing and additional examinations show no service degradations in the VHA network between July 2017 and October 2018.

 

Telstra are “Best in Test” with Optus achieving a “Very Good” second RANK. Vodafone confirms its position as “Good”. The Australian operators continue to show that they are focused on customer experience.

Telstra is the winner with strong crowd source and benchmark results. Optus achieved a very strong second rank, with Vodafone remaining in a good position.

AUSTRALIA’S OPERATORS

TELSTRA IS THE CLEAR LEADER IN AUSTRALIA IN TERMS OF MARKET SHARE, WHILE OPTUS UND VODAFONE COMPETE FOR THE SECOND RANK. ALL THREE OF THEM ­OFFER THE LATEST MOBILE TECHNOLOGIES LIKE LTE-ADVANCED AND VOLTE.

According to its own statement, Telstra currently has 17.7 million mobile subscribers in Australia which makes this operator the market leader.The history of Telstra dates far back: The company originated as part of the Postmaster-General‘s Department and was fully privatised by 2006. The last remaining government share was effectively sold in 2011.
Today, Telstra offers voice, mobile and internet access, as well as Pay TV and other entertainment services to Australian customers. The operator provides 3G (UMTS) services at 850 and 2100 MHz and a 4G (LTE including LTE-Advanced) network on the 700, 900, 1800 and 2600 MHz frequency bands. Like its competitors, Telstra has been continually increasing its LTE coverage and offerings, now providing LTE-Advanced under the name “4GX“.

With approximately 6.5 million subscribers on their 4G network, Singtel Optus is the second largest mobile operator in Australia. The company was originally founded as AusSat in 1981 and later changed its name to “Cable & Wireless Optus“. Since 2001, it is a wholly owned subsidiary of the Singapore-based SingTel. The company offers its mobile services mainly under the brand name Optus, but also maintains a number of subsidiary brands, such as Virgin Mobile Australia. Its offerings span fixed and mobile telephony and data networks, broadband access for both consumers and business customers, cable TV and satellite services. Optus provides a 3G network at 900 and 2100 MHz and 4G/LTE at the 700, 1800, 2100, 2300 and 2600 MHz bands. Its 4G offerings include LTE-Advanced under the brand name “4G Plus“.

Vodafone Australia quotes a mobile subscriber base of approx. 6 million mobile customers. This makes Vodafone Australia‘s third largest mobile operator.  The current company was formed in a merger of Vodafone Australia and Hutchison 3G Australia (and its former “3“ network) in 2009, as is typically referred to as “VHA“ (Vodafone-Hutchison Australia). VHA is a 50:50 joint venture between Vodafone Group Plc and Hutchison Telecommunications (Australia) Limited and provides mobile as well as fixed services to Australian customers. Today, VHA offers 3G at 900 and 2100 MHz as well as 4G at 850, 1800 and 2100 MHz. Upgrading and expanding its 4G service, Vodafone enabled carrier aggregation (LTE-Advanced) on all of their 4G bands.

CLOSER LOOK AT AUSTRALIA’S NETWORKS

WITH THE ADDITION OF CROWDSOURCING, P3 AND CONNECT HAVE EXTENDED THEIR SCOPE IN ORDER TO PRESENT AN EVEN MORE COMPLETE LOOK AT NETWORK QUALITY AND PERFORMANCE.

P3, headquartered in Aachen, Germany, is a world leader in mobile network testing. The company has over 3,500 employees worldwide and a turnover of more than 350 million Euros. P3 is partnering with the international telecommunications magazine connect, which has 25 years of editorial expertise and is one of the leading test authorities in Europe for telecommunications products and services. Together, P3 and connect have been conducting the most important network benchmark test in Germany for more than 15 years, extending it to other European countries since 2009. Above that, P3 has been conducting its authoritative Mobile Benchmarking in Australia since 2014 as well as examining many other mobile networks all over the world including those in the USA and Singapore.
The 2018 P3 connect Mobile Benchmark Australia consists of drivetests and walktests conducted in October and November, 2018. Four drive test cars together covered about 13,220 kilometres. The test areas account for more than 15.1 million people, or approximately 67 per cent of the total population of Australia. In addition, the results of extensive crowdsourcing analyses considering August, September and October (Data Service Availability: May to October) 2018 are included in the score.

Hakan Ekmen, CEO of P3 communications

In large-scale countries such as Australia, user experience is of especially high importance as the focus must be less on cove­ring extensive areas, but more on actually reaching the users. Therefore our unique crowdsourcing methodology is an important factor in the results. Our holistic approach clearly confirms that the Australian operators have worked hard to improve their networks’ performance and availability.

VOICE

MANY CUSTOMERS USE VOICE SERVICES LESS INTENSELY THAN DATA. HOWEVER, WHEN ACTUALLY TAKING OR PLACING A PHONE CALL, THEY EXPECT RELIABLE CONNECTIONS. HOW DO THE AUSTRALIAN NETWORKS FULLFIL THESE EXPECTATIONS?

All three Australian operators support Voice over LTE (VoLTE). VoLTE transmits voice calls as data packets over a 4G connection. This way, the otherwise necessary “circuit-switched fallback“, which forces smartphones to switch back to 3G in order to take or place a phone call, can be avoided. Also, VoLTE supports better audio codecs providing operators with the opportunity to deliver higher speech quality to their customers. For the voice rating, each drivetest car and each walktest team carried one Samsung Galaxy S8 smartphone per operator. The phones in the cars always called a counterpart in one of the other cars. The phones carried by the walktest teams called a stationary counterpart. The connected testing equipment registered success ratios, call setup times and speech quality. In order to simulate normal smartphone usage, data transfers took place in the background of the test calls.

 

TELSTRA LEADS IN THE DRIVETESTS CONDUCTED IN BIG CITIES
In the drivetests conducted in Adelaide, Brisbane, Canberra, Melbourne, Perth, Sydney, Toowoomba and Wollongong, Telstra takes the overall lead. This operator scores two percentage points ahead of its competitors mainly due to higher call success ratios. But ­Optus and Vodafone also show very robust results in this category.

OPTUS IS AHEAD IN THE WALKTESTS
Brisbane, Canberra, Melbourne and Sydney were additionally visited by P3’s walktest teams. In this category, Optus is ahead of the competition by two percentage points. This lead is earned by high call success ratios and also very fast call setup times. But again, the two other contenders also achieve convincing results.

OPTUS SHOWS ALSO BEST RESULTS IN SMALLER TOWNS
As in the big city walktests, Optus is also ahead in the smaller Australian towns that were part of our test routes. But in this case, the lead is only by a narrow margin – or one percentage point respectively. It is earned by the best call success ratios and the fastest call setup times. But as in the two categories before, Optus and Vodafone also show robust results.

OPTUS AHEAD OF TELSTRA ON THE ROADS BY A NARROW MARGIN, VODAFONE FOLLOWS AT A DISTINCT DISTANCE
Optus also achieves the best voice results on the connecting roads that were covered by our test vehicles. However, this time, the competition takes place at a significantly lower level. Optus gains 59 per cent of the possible points in this category, Telstra follows with 57 percent and Vodafone at a distinct distance with only 30 percent. Call success ratios dropping to about 88 per cent and lower reveal that permanent voice connectivity is not at all a given on Australian roads.

VOICE RESULTS AT A GLANCE
In the overall assessment, Telstra leads by one point in the voice category, which it earned in the big city drivetests. Optus is almost on a par and achieved slightly better results in the big city walktests, in  smaller towns and – at a lower performance level – on the connecting roads. Vodafone can almost keep up in the cities and towns but clearly loses ground on the roads.

DATA

WITH THE VOLUME OF TRANSMITTED DATA PERMANENTLY GROWING, DATA CONNECTIVITY CONSTANTLY BECOMES MORE IMPORTANT. WHICH AUSTRALIAN OPERATOR MANAGES BEST TO MEET THE INCREASING DEMAND?

OPTUS LEADS IN THE DATA CATEGORY, ESPECIALLY DUE TO STRONG RESULTS IN THE WALKTESTS, IN SMALLER TOWNS AND ON THE ROADS.

Data connectivity is the most prestigious discipline in our benchmark and also in the operators’ marketing. All three Australian networks claim to cover a large part of the population with LTE services – the claimed percentages range in the high nineties (see page 3). Telstra, Optus and Vodafone are continuing to spend billions of dollars on upgrading and expanding their networks to meet the increasing demand. According to the Australian Bureau of Statistics, the total amount of data dowloaded by mobile ­devices grew by 28 per cent between June 2017 and June 2018, reaching 123,147 terabytes at this latest point of the research period. Australian operators reacted by intro­ducing unlimited data plans in 2018 and also increasing the uncapped volumens included in their lower priced tariffs.

P3’S SCORING REWARDS HIGH PERFORMANCE AS WELL AS THE NETWORKS’ AVAILABILITY AND STABILITY
In order to assess the performance and reliability of data connections, each of our four drivetest cars and also the walktest team carried one Samsung Galaxy S8 per operator. Supporting the LTE category 16, these smartphones were able to benefit from the so-called carrier aggregation – the combined use of up to three LTE carrier frequencies which can theoretically transmit up to 1 Gbps. In the operators’ marketing, this functionality bears names such as “4GX”, “4G Plus” or “LTE-Advanced”.

P3‘s testing considers such fast throughputs as well as the networks‘ availability and stability. In order to assess typical performance as well as peak speeds, we consider two values: the minimum data rate that is available in 90 per cent of the cases, and additionally the peak data rate that is surpassed in 10 per cent of the cases. Web page and file downloads or uploads reward fast speeds, while determination of successratios and assessing YouTube playouts concentrate on reliability aspects. As YouTube streams videos at adaptive bitrates, the average value of the received video resolution is another important performance indicator.

TELSTRA AND OPTUS ALMOST ON A PAR IN BIG CITY DRIVETESTS
In the scoring of their performance determined during the drivetests conducted in big Australian cities, both Telstra and Optus achieve 95 per cent of the available points in this discipline. The success ratios are all in all on the same level, while Optus achieves slightly higher maximum download speeds and Telstra scores a little higher in the average download rates. With 92 per cent of the achievable points, Vodafone ranks third but can mostly keep up with its two slightly stronger competitors.

OPTUS ALSO SHOWS BEST DATA RESULTS IN BIG CITY WALKTESTS
As in the voice discipline before, Optus also leads the field in the data measurements taken during the walktests in the big cities. Telstra falls behind by 4 percentage points (Optus: 97 per cent, Telstra: 94 per cent), and Vodafone more distinctly by achieving 81 per cent of the possible points in this category. Part of an explanation for these results is that Optus particularly stands out with the best web browsing experience among all Australian operators. During the walk test phase of our benchmark in Australia a firmware issue was experienced on the smart-phones resulting in Telstra and Vodafone being unable to take advantage of carrier aggregation (CA). Therefore the walktest results are lower than anticipated, but reliability results were not affected. Due to the high performance levels of both these networks approximately two points were lost which did not impact the overall ranking.

This would have led to somewhat higher data rates and thus slightly better scores in a magnitude of about two points. However, this did not have any impact on the overall ranking.

OPTUS LEADS THE FIELD IN THE DATA TESTS CONDUCTED IN SMALLER TOWNS
This picture now almost becomes familiar: As in the voice discipline and as in the city walktests, Optus also achieves the best score in the data drivetests conducted in smaller Australian towns. This is valid for practically all data services. Telstra follows at a small distance, and Vodafone at a more pronounced one.

ACCEPTABLE RESULTS FOR OPTUS AND TELSTRA ON THE ROADS
Overall, Optus achieved 70 per cent of the available points for the data drivetests on the connecting roads. Telstra follows with 67 per cent. Both results are acceptable but not great. Vodafone shows the same weaknesses as in the voice category before and only achieves 30 per cent of the possible points. While Optus and Telstra offer success ratios of about 90 per cent and above, Vodafone drops to success ratios of 80 per cent and below.

DATA RESULTS AT A GLANCE
In the drivetests conducted in the big cities, Telstra and Optus are almost on a par, whereas Optus shows better performance for all data services in the smaller towns and on the roads. While Vodafone can mostly keep up with the other two contenders in the city drivetests, it falls behind in the walktests, in smaller towns and with a significant gap towards the competion on the connecting roads.

CROWD

IN THIS YEAR, THE RESULTS OF CROWD­ SOURCING ANALYSES ARE PART OF THE TOTAL SCORE FOR THE FIRST TIME. 63,000 AUSTRALIAN USERS HAVE CONTRIBUTED TO THE DATA GATHE­RING THAT TOOK PLACE FROM AUGUST TO OCTOBER, 2018.

TELSTRA IS THE WINNER IN THE CROWD ASSESSEMENT DUE TO BEST VOICE AND DATA COVERAGE AND FAST DATA RATES.

While the drivetests and walktests determine the peak performance of the examined networks, crowdsourcing can add important dimensions such as time, geography or variety in devices and tariff plans – if done in the right way. For the collection of crowd data, P3 has integrated a background diagnosis pro­cesses into more than 800 diverse Android apps. If one of these applications is installed on the end-user’s phone and the user ­authorizes the background analysis, data collection takes place 24/7, 365 days a year. Reports are generated for every quarter of an hour and sent daily to P3‘s cloud servers. Such reports generate just a small number of bytes per message and do not include any personal user data. A more detailed description of our crowdsourcing methodology can be found on page 15 of this report.

Based on the total population count of 24.6 million people, one of 390 inhabitants of the country has contributed to the gathering of 1.38 billion single samples of crowd data. The considered test area represents 81.8 per cent of the built-up area of Australia.

BEST COVERAGE RESULTS FOR VOICE AND DATA IN TELSTRA NETWORK. OPTUS OFFERS HIGHEST QUALITY OF 4G COVERAGE.
The Coverage and Quality of Coverage results determined in the crowdsourcing come hardly as a surprise: Telstra achieves the best results for the voice and data (3G + 4G) coverage, followed by Optus and – at a distinct distance – by Vodafone. However, Optus users have a slightly higher probability to being able to actually use 4G services (or a higher “Quality of Coverage”) than the customers of Telstra. Vodafone brings up the rear also in this examination.

TELSTRA SHOWS FASTEST DOWNLOAD SPEEDS IN TOP 10 PER CENT OF EVALUATION AREAS
In the assessment of download speeds available to the users, Telstra shows the best results for the top 10 per cent of Evaluation Areas. The values for the fastest users as well as the average user’s best throughput are practically on a par for all three contenders. Generally, it must be taken into consideration that a part of the actual user base probably experiences data speed limitations caused by their mobile tariffs.

VODAFONE DID NOT SHOW SERVICE DEGRADATIONS FROM JULY 2018 TO OCTOBER 2018
Examining the Data Service Availa­bility, shows pleasant results. Other than the rest of the crowd KPIs, this examination covers six months (May to ­October 2018). As a deeper investigation revealed, none of the operators suffered service degradations from July 2017 to April 2018. So the Vodafone network showed no impairment in an impressive period between July 2017 and October 2018. For Telstra and Optus we have identified only minor degradations (see chart below).

CROWD RESULTS AT A GLANCE
Telstra achieves the best results for the voice and data coverage, but Optus shows a higher Quality of 4G Coverage. Telstra shows the highest speeds for the top 10 per cent of Evaluation Areas. The fastest users as well as the average user’s best throughput are on a par. Vodafone can pride itself to have had no service degradations in more than 12 months.

THE LARGEST CITIES IN AUSTRALIA

FOR THE INHABITANTS OF THE DENSELY POPULATED URBAN AREAS IN ­AUSTRALIA, IT IS PARTICULARLY INTERESTING TO SEE HOW THE OPERATORS PERFORM IN THEIR HOMECITIES. THEREFORE, WE HAVE PERFORMED INDIVIDUAL ANALYSES FOR THE LARGEST AUSTRALIAN METROPOLITAN AREAS.

More than 14 million (or 60 per cent of all) Australians live in one of the five largest urban areas of the country – plus another 368,000 in its capital Canberra. Therefor we have taken a closer look at the individual results of Adelaide, Brisbane, Canberra, Melbourne, Perth and Sydney.

TELSTRA IS THE LOCAL CHAMPION IN SYDNEY, OPTUS AND VODAFONE ALSO STRONG.

In Sydney, Telstra takes the lead and Optus follows at a close distance of just seven points. As in the nationwide results, Vodafone ranks third. But as it is typical for the larger Australian cities, its gap to its two stronger competitors is much closer than in the overall benchmark results.

IN MELBOURNE AND CANBERRA, OPTUS TAKES THE LEAD AHEAD OF TELSTRA. VODAFONE FOLLOWS AT A WIDER GAP.

In Melbourne and Canberra, Optus takes a narrow lead ahead of Telstra, which derives mostly from strong data results. In the voice assessment, both operators are on a par. Vodafone falls behind a little more distinctly than in most of the other bigger cities. In the crowdsourcing analyses, all three contenders are close together – which can also be seen in many other cities.

IN BRISBANE, VODAFONE OVERTAKES OPTUS – BUT TELSTRA STILL LEADS.

In Brisbane, Vodafone is particularly strong, seizing the second rank behind the still leading Telstra and ahead of Optus. This lead is especially earned by a very robust performance in the data category.

TELSTRA WINS IN PERTH AND ADELAIDE.
In Perth, Optus and Vodafone are almost on a par, but still outdone by a very strong Telstra. In Adelaide, all three networks are almost equally strong, although Telstra still wins. As in Brisbane before, Vodafone overtakes Optus by a narrow margin.

A focused view at the country‘s largest ­cities (the six shown below plus Toowomba and Wollongong) shows that the Australian operators are especially strong in the densely populated  metropolitan areas.

COMPARISON

The comparison of Australia‘s six largest cities reveals different local champions as well as varying distances between the three contenders. In order to determine the crowd scores, we have analysed the Evaluation Areas of the respective greater areas.

HOW DOES SYDNEY COMPARE TO OTHER BUSINESS HOTSPOTS?

ANOTHER WAY TO EVALUATE THE RESULTS OF AUSTRALIAN METROPOLITAN AREAS IS TO COMPARE THEM WITH OTHER BUSINESS CAPITALS ALL OVER THE WORLD. WE HAVE USED OUR CROWD METRICS IN ORDER TO ALSO PROVIDE THIS VIEW.

A look at the average values of the best throughputs achieved by each individual user participating in our crowdsourcing (see chart below) shows that the Australian operators deliver considerably high performance also when compared to their global competitors.

SYDNEY AND SINGAPORE AHEAD OF OTHER WORLD BUSINESS CAPITALS

As can be clearly seen, only Singapore ranks on the same high level as Sydney which we have selected for this comparison. However, when evaluating the actual values, it must be taken into account that areas such as Singapore are densely covered with buildings, whereas cities like Berlin or London contain a higher share of open spaces such as parks or fringe areas, which are provided with less mobile coverage and also com­prise of less mobile users. In order to ensure statistical relevance, we had to exclude Jakarta‘s smallest operator Smartfren from the line-up. The analysis at hand concentrates on data throughputs. But an additional look at the Areas and Quality of Coverage with 4G services, 3G/2G data services as well as voice services in the considered cities also shows favourable results for Singapore and Sydney – however, some competitors such as London or Jakarta come close in one or the other of the analysed metrics. Deeper insights can be made available to interested parties on request.

TESTING METHODOLOGY

THE METHODOLOGY OF THE P3 CONNECT MOBILE BENCHMARK IS THE RESULT OF MORE THAN 15 YEARS OF TESTING MOBILE NETWORKS. TODAY, NETWORK TESTS ARE CONDUCTED IN MORE THAN 80 COUNTRIES. OUR METHODOLOGY WAS CAREFULLY DESIGNED TO EVALUATE AND OBJECTIVELY COMPARE THE PERFORMANCE AND SERVICE QUALITY OF MOBILE NETWORKS FROM THE USERS’ PERSPECTIVE.

The P3 connect Mobile Benchmark Australia comprises of the results of extensive voice and data drivetests and walktests as well as a sophisticated crowdsourcing approach.

DRIVETESTS AND WALKTESTS
The drivetests and walktests in Australia took place from October 29 to November 16, 2018 All samples were collected during the day, between 8.00 a.m. and 10.00 p.m. The network tests covered inner-city, outer metropolitan and suburban areas. Measurements were also taken in smaller towns and on the connecting highways. The four measurement cars together covered about 4,280 kilometres in the cities, about 940 km in towns and about 8,000 km on the roads – resulting in a total of 13,220 kilometres. The combination of test areas has been selected to provide representative test results across the Australian population. The areas selected for the 2018 test account for more than 15.1 million people, or roughly 67 per cent of the total population of Australia.
The tests were conducted according to P3‘s “Large Country“ model. This means that the measurements focused on metro­politan agglomerations and their closer environments. The routes and all visited cities and towns are shown on page 1 of this report. The four drivetest cars were equipped with Samsung Galaxy S8 smartphones. The simultaneous measurement of voice and data services was conducted with these mass market devices to obtain a realistic picture of the users‘ experience.

VOICE TESTING

One smartphone per operator in each car was used for the voice tests, setting up test calls from one car to another. The walktest team also carried one smartphone per operator for the voice tests.
In this case, the smartphones called a stationary counterpart. The audio quality of the calls was evaluated using the HD-voice capable and ITU standardised POLQA wideband algorithm.
All smartphones used for the voice tests were set to VoLTE preferred mode. In networks or areas where this modern 4G based voice technology was not available, they would perform a fallback to 3G. In order to account for typical smartphone use during the voice tests, background data traffic was generated through random injection of small amounts of HTTP traffic. The voice scores account for 34 per cent of the total results.

DATA TESTING

Data performance was measured by using three more Galaxy S8 in each car – one per operator. Their radio access technology was set to LTE preferred mode.

For the web tests, they accessed web pages according to the widely recognised Alexa ranking In addition, the static Kepler test web page as specified by  ETSI was used. In order to test the data service performance, files of 3 MB and 1 MB for download and upload were transferred from or to a test server located on the Internet. In addition, the peak data performance was tested in uplink and downlink directions by assessing the amount of data that was transferred within a seven seconds time period.
The evaluation of YouTube playback takes into account that YouTube dynamically adapts the video resolution to the available band width. So, in addition to success ratios, start times and playouts without interruptions, the measurements also determined average video resolution. All the tests were conducted with the best-performing mobile plan available from each operator. Data scores account for 51 per cent of the total results.

CROWDSOURCING

Additionally, P3 conducted crowd-based analyses of the Australian networks which contribute 15 per cent to the end result. They are based on data that gathered in August, September and October, 2018. For the collection of crowd data, P3 has integrated a background diagnosis processes into 800+ diverse Android apps. If one of these applications is installed on the enduser’s phone and the user authorizes the background analysis, data collection takes place 24/7, 365 days a year. Reports are generated for every quarter of an hour and sent daily to P3‘s cloud servers.

Such reports generate just a small number of bytes per message and do not include any personal user data. Interested parties can deliberately take part in the data gathering with the specific ”U get“ app (see box on the right).

NETWORK COVERAGE

For the assessment of network coverage, P3 lays a grid of 2 by 2 kilometres over the whole test area. The “evaluation areas“ generated this way are then sub-divided into 16 smaller tiles. To ensure statistical relevance, P3 requires a certain number of users and measurement values per operator for each tile and each evaluation area. If these thresholds are not met by one of the operators, this part of the map will not be considered in the assess­ment for the sake of fairness.  “Quality of Coverage“ reveals whether voice and data services actually work in an evaluation area. P3 does this because not in each area that allegedly provides network reception, mobile services can actually be used. We specify these values for the coverage of voice services (3G and 4G combined), data (3G and 4G combined) and 4G only.

DATA THROUGHPUTS

Additionally, P3 investigates the data rates that were actually available to each user. For this purpose, we determine the best obtained data rate for each user during the evaluation period and then calculate their average value. In addition, we determine the so-called P90 values for the top throughput of each evaluation area as well as of each user‘s best throughput. P90 values specify the threshold in a statistical distribution, below which 90 per cent of the gathered values are ranging and  depict how fast the network is under favorable conditions.

DATA SERVICE AVAILABILITY

This parameter indicates the number of outages or service degradations – events where data connectivity is impacted by a number of cases that significantly exceeds the expectation level. To judge this, the algorithm looks at a sliding window around the hour of interest. This ensures that we only consider actual degradations as opposed to a simple loss of network coverage due to prolonged indoor stays or similar reasons. In order to ensure statistical relevance, each operator must have sufficient statistics for trend and noise analyses per each evaluated hour. The exact number depends on the market size and number of operators.  A valid assessment month must comprise of at least 90 per cent of valid assessment hours. Deviating from the other crowd score elements, Data Service Availability is rated based on a six-month observation period – in this case from May to Oct 2018.