2018|12|13 – THE 2018 P3 CONNECT MOBILE BENCHMARK IN AUSTRALIA
THIS YEAR, THE BENCHMARKING EXPERT P3 AND THE INTERNATIONALLY ACKNOWLEDGED TELECOMMUNICATIONS MAGAZINE CONNECT HAVE CONDUCTED THEIR FIFTH MOBILE NETWORK TEST IN AUSTRALIA. AFTER A CLOSE RACE BETWEEN OPTUS AND TELSTRA AND A GOOD THIRD RANK FOR VODAFONE IN THE PREVIOUS YEAR, HOW DID THE AUSTRALIAN OPERATORS PERFORM THIS TIME?
RESULTS IN A NUTSHELL
AS IN THE PREVIOUS YEAR, WE AGAIN SAW A NECK-AND-NECK RACE BETWEEN THE TWO LARGEST AUSTRALIAN OPERATORS. THIS YEAR, TELSTRA MANAGES TO TAKE THE FIRST RANK BACK FROM AN ALSO VERY STRONG OPTUS. VODAFONE CONFIRMS ITS POSITION.
P3‘s network benchmarks are widely accepted as the de-facto industry standard being highly objective. The carefully designed methodology of our 2018 benchmark in Australia combines drivetests and walktests for executing detailed voice and data measurements under controlled circumstances with a sophisticated crowdsourcing approach. This provides profound insights into the overall coverage of voice, data and 4G services, real-world User Download Speeds and Data Service Availability.
P3‘s holistic approach to network benchmarking includes both drivetest and walktest measurements as well as crowdsourcing. The drive and walktests allow for evaluating the cutting edge of the networks‘ capabilities. Crowdsourcing unveils the service-quality, performance and coverage actually experienced by the users. We have thoroughly weighted these components in order to give a realistic and authoritative assessment of the rated networks‘ true potential and performance.
Telstra achieves a total of 884 points which equals the grade “very good”. Its narrow lead in the voice category is earned in the big city drivetests and its overall win by offering the best speeds for the top 10 per cent of Evaluation Areas.
Optus ranks second with a total of 872 points, which is also a “very good” result. This operator leads in the data category, due to better results in smaller towns and on the roads. Also, Optus managed to improve its results in the voice and data categories compared to previous year’s score. In the crowdsourced assessment, Optus shows a particularly high Quality of 4G Coverage.
Vodafone can almost keep up in the cities and towns but clearly loses ground on the roads. The results of our crowdsourcing and additional examinations show no service degradations in the VHA network between July 2017 and October 2018.
TELSTRA IS THE CLEAR LEADER IN AUSTRALIA IN TERMS OF MARKET SHARE, WHILE OPTUS UND VODAFONE COMPETE FOR THE SECOND RANK. ALL THREE OF THEM OFFER THE LATEST MOBILE TECHNOLOGIES LIKE LTE-ADVANCED AND VOLTE.
According to its own statement, Telstra currently has 17.7 million mobile subscribers in Australia which makes this operator the market leader.The history of Telstra dates far back: The company originated as part of the Postmaster-General‘s Department and was fully privatised by 2006. The last remaining government share was effectively sold in 2011.
Today, Telstra offers voice, mobile and internet access, as well as Pay TV and other entertainment services to Australian customers. The operator provides 3G (UMTS) services at 850 and 2100 MHz and a 4G (LTE including LTE-Advanced) network on the 700, 900, 1800 and 2600 MHz frequency bands. Like its competitors, Telstra has been continually increasing its LTE coverage and offerings, now providing LTE-Advanced under the name “4GX“.
CLOSER LOOK AT AUSTRALIA’S NETWORKS
WITH THE ADDITION OF CROWDSOURCING, P3 AND CONNECT HAVE EXTENDED THEIR SCOPE IN ORDER TO PRESENT AN EVEN MORE COMPLETE LOOK AT NETWORK QUALITY AND PERFORMANCE.
P3, headquartered in Aachen, Germany, is a world leader in mobile network testing. The company has over 3,500 employees worldwide and a turnover of more than 350 million Euros. P3 is partnering with the international telecommunications magazine connect, which has 25 years of editorial expertise and is one of the leading test authorities in Europe for telecommunications products and services. Together, P3 and connect have been conducting the most important network benchmark test in Germany for more than 15 years, extending it to other European countries since 2009. Above that, P3 has been conducting its authoritative Mobile Benchmarking in Australia since 2014 as well as examining many other mobile networks all over the world including those in the USA and Singapore.
The 2018 P3 connect Mobile Benchmark Australia consists of drivetests and walktests conducted in October and November, 2018. Four drive test cars together covered about 13,220 kilometres. The test areas account for more than 15.1 million people, or approximately 67 per cent of the total population of Australia. In addition, the results of extensive crowdsourcing analyses considering August, September and October (Data Service Availability: May to October) 2018 are included in the score.
MANY CUSTOMERS USE VOICE SERVICES LESS INTENSELY THAN DATA. HOWEVER, WHEN ACTUALLY TAKING OR PLACING A PHONE CALL, THEY EXPECT RELIABLE CONNECTIONS. HOW DO THE AUSTRALIAN NETWORKS FULLFIL THESE EXPECTATIONS?
All three Australian operators support Voice over LTE (VoLTE). VoLTE transmits voice calls as data packets over a 4G connection. This way, the otherwise necessary “circuit-switched fallback“, which forces smartphones to switch back to 3G in order to take or place a phone call, can be avoided. Also, VoLTE supports better audio codecs providing operators with the opportunity to deliver higher speech quality to their customers. For the voice rating, each drivetest car and each walktest team carried one Samsung Galaxy S8 smartphone per operator. The phones in the cars always called a counterpart in one of the other cars. The phones carried by the walktest teams called a stationary counterpart. The connected testing equipment registered success ratios, call setup times and speech quality. In order to simulate normal smartphone usage, data transfers took place in the background of the test calls.
WITH THE VOLUME OF TRANSMITTED DATA PERMANENTLY GROWING, DATA CONNECTIVITY CONSTANTLY BECOMES MORE IMPORTANT. WHICH AUSTRALIAN OPERATOR MANAGES BEST TO MEET THE INCREASING DEMAND?
OPTUS LEADS IN THE DATA CATEGORY, ESPECIALLY DUE TO STRONG RESULTS IN THE WALKTESTS, IN SMALLER TOWNS AND ON THE ROADS.
Data connectivity is the most prestigious discipline in our benchmark and also in the operators’ marketing. All three Australian networks claim to cover a large part of the population with LTE services – the claimed percentages range in the high nineties (see page 3). Telstra, Optus and Vodafone are continuing to spend billions of dollars on upgrading and expanding their networks to meet the increasing demand. According to the Australian Bureau of Statistics, the total amount of data dowloaded by mobile devices grew by 28 per cent between June 2017 and June 2018, reaching 123,147 terabytes at this latest point of the research period. Australian operators reacted by introducing unlimited data plans in 2018 and also increasing the uncapped volumens included in their lower priced tariffs.
P3’S SCORING REWARDS HIGH PERFORMANCE AS WELL AS THE NETWORKS’ AVAILABILITY AND STABILITY
In order to assess the performance and reliability of data connections, each of our four drivetest cars and also the walktest team carried one Samsung Galaxy S8 per operator. Supporting the LTE category 16, these smartphones were able to benefit from the so-called carrier aggregation – the combined use of up to three LTE carrier frequencies which can theoretically transmit up to 1 Gbps. In the operators’ marketing, this functionality bears names such as “4GX”, “4G Plus” or “LTE-Advanced”.
IN THIS YEAR, THE RESULTS OF CROWD SOURCING ANALYSES ARE PART OF THE TOTAL SCORE FOR THE FIRST TIME. 63,000 AUSTRALIAN USERS HAVE CONTRIBUTED TO THE DATA GATHERING THAT TOOK PLACE FROM AUGUST TO OCTOBER, 2018.
TELSTRA IS THE WINNER IN THE CROWD ASSESSEMENT DUE TO BEST VOICE AND DATA COVERAGE AND FAST DATA RATES.
While the drivetests and walktests determine the peak performance of the examined networks, crowdsourcing can add important dimensions such as time, geography or variety in devices and tariff plans – if done in the right way. For the collection of crowd data, P3 has integrated a background diagnosis processes into more than 800 diverse Android apps. If one of these applications is installed on the end-user’s phone and the user authorizes the background analysis, data collection takes place 24/7, 365 days a year. Reports are generated for every quarter of an hour and sent daily to P3‘s cloud servers. Such reports generate just a small number of bytes per message and do not include any personal user data. A more detailed description of our crowdsourcing methodology can be found on page 15 of this report.
THE LARGEST CITIES IN AUSTRALIA
IN MELBOURNE AND CANBERRA, OPTUS TAKES THE LEAD AHEAD OF TELSTRA. VODAFONE FOLLOWS AT A WIDER GAP.
In Melbourne and Canberra, Optus takes a narrow lead ahead of Telstra, which derives mostly from strong data results. In the voice assessment, both operators are on a par. Vodafone falls behind a little more distinctly than in most of the other bigger cities. In the crowdsourcing analyses, all three contenders are close together – which can also be seen in many other cities.
IN BRISBANE, VODAFONE OVERTAKES OPTUS – BUT TELSTRA STILL LEADS.
In Brisbane, Vodafone is particularly strong, seizing the second rank behind the still leading Telstra and ahead of Optus. This lead is especially earned by a very robust performance in the data category.
TELSTRA WINS IN PERTH AND ADELAIDE.
In Perth, Optus and Vodafone are almost on a par, but still outdone by a very strong Telstra. In Adelaide, all three networks are almost equally strong, although Telstra still wins. As in Brisbane before, Vodafone overtakes Optus by a narrow margin.
HOW DOES SYDNEY COMPARE TO OTHER BUSINESS HOTSPOTS?
THE METHODOLOGY OF THE P3 CONNECT MOBILE BENCHMARK IS THE RESULT OF MORE THAN 15 YEARS OF TESTING MOBILE NETWORKS. TODAY, NETWORK TESTS ARE CONDUCTED IN MORE THAN 80 COUNTRIES. OUR METHODOLOGY WAS CAREFULLY DESIGNED TO EVALUATE AND OBJECTIVELY COMPARE THE PERFORMANCE AND SERVICE QUALITY OF MOBILE NETWORKS FROM THE USERS’ PERSPECTIVE.
The P3 connect Mobile Benchmark Australia comprises of the results of extensive voice and data drivetests and walktests as well as a sophisticated crowdsourcing approach.
DRIVETESTS AND WALKTESTS
The drivetests and walktests in Australia took place from October 29 to November 16, 2018 All samples were collected during the day, between 8.00 a.m. and 10.00 p.m. The network tests covered inner-city, outer metropolitan and suburban areas. Measurements were also taken in smaller towns and on the connecting highways. The four measurement cars together covered about 4,280 kilometres in the cities, about 940 km in towns and about 8,000 km on the roads – resulting in a total of 13,220 kilometres. The combination of test areas has been selected to provide representative test results across the Australian population. The areas selected for the 2018 test account for more than 15.1 million people, or roughly 67 per cent of the total population of Australia.
The tests were conducted according to P3‘s “Large Country“ model. This means that the measurements focused on metropolitan agglomerations and their closer environments. The routes and all visited cities and towns are shown on page 1 of this report. The four drivetest cars were equipped with Samsung Galaxy S8 smartphones. The simultaneous measurement of voice and data services was conducted with these mass market devices to obtain a realistic picture of the users‘ experience.
One smartphone per operator in each car was used for the voice tests, setting up test calls from one car to another. The walktest team also carried one smartphone per operator for the voice tests.
In this case, the smartphones called a stationary counterpart. The audio quality of the calls was evaluated using the HD-voice capable and ITU standardised POLQA wideband algorithm.
All smartphones used for the voice tests were set to VoLTE preferred mode. In networks or areas where this modern 4G based voice technology was not available, they would perform a fallback to 3G. In order to account for typical smartphone use during the voice tests, background data traffic was generated through random injection of small amounts of HTTP traffic. The voice scores account for 34 per cent of the total results.
For the assessment of network coverage, P3 lays a grid of 2 by 2 kilometres over the whole test area. The “evaluation areas“ generated this way are then sub-divided into 16 smaller tiles. To ensure statistical relevance, P3 requires a certain number of users and measurement values per operator for each tile and each evaluation area. If these thresholds are not met by one of the operators, this part of the map will not be considered in the assessment for the sake of fairness. “Quality of Coverage“ reveals whether voice and data services actually work in an evaluation area. P3 does this because not in each area that allegedly provides network reception, mobile services can actually be used. We specify these values for the coverage of voice services (3G and 4G combined), data (3G and 4G combined) and 4G only.