Reliable network measurements enable QoE improvements

Mobile networks are rapidly evolving, and their data transport capacity has grown immensely over the last few years. At the same time, the data consumption of apps and services, and customer expectations around mobile data usage have increased drastically. Quality of Experience (QoE) is crucial for the success of mobile network operators because subscribers are increasingly sensitive to the perceived quality of service in mobile networks.

In many markets, operators will only grow their subscriber base if they can lure subscribers from competing operators by offering a better perceived QoE. Mobile network operators and service providers need to be able to quickly and easily perform accurate and advanced benchmarking tests that not only are in line with the latest standards and methodologies, but also reduce capital expenditures (CAPEX) and operating expenditures (OPEX).

There are various scoring methods claiming to measure the performance of mobile networks. Some scores are based on only technical metrics, others on quality of experience (QoE) measurements or surveys. Which score should a mobile network operator choose or rely on for accuracy? The one providing the highest score? The cheapest one? The one based on technical scores or rather the one reflecting QoE, indicating what the end user perceives? Should the score offer a transparent and stable method or be a black box?

Extracting insights from raw data translates into actionable decisions to ensure mobile network performance for applications. (Image: Rohde & Schwarz)

To really compare different networks, mobile network operators have to execute the same tests, at the same time, at the same geographical location and in different networks. This can be called “real benchmarking.” The performance of a service is not only determined by the transport capacity of the radio link but also the cell-site connections to the core network and the interconnectivity to the content delivery network and servers. Where and how content servers are placed and connected in the network and how the content management and caching is organized are essential.

To get a true view of a network’s performance, the actual service performance as perceived by the end user needs to be considered. Optimal network performance can only be achieved when all elements are in tune with each other. Therefore, when targeting focused investments for maximum benefit, the solution should include finding the bottleneck rather than investing aimlessly as in “the more, the better.”

Mobile network operators and service providers must look at the factors influencing accuracy and reliability of data acquisition and data analytics, and how the right measurement equipment and techniques can enable the efficient delivery of better services and higher quality of experience to customers.

Recommended
Measuring the QoE of mobile data applications for network optimization

Data collection
The data collection tools have to ensure that the data is accurate and reliable and that the measurement results can be repeated and validated for statistical purposes. This ensures that the test solution itself does not have a negative impact on the network measurement results.

Important factors for reliable and repeatable measurements include:

• Temperature control: The measurement probes’ temperature needs to be consistent, i.e., there is a correlation between increasing smartphone battery temperature and decreasing data performance.
• RF environment control: The measurement probes’ RF environment needs to be steady, i.e.,  measurement probes installed in-car are exposed to varying RF attenuation.
• Measurement probe location: Hosting the measurement probes in the test device containment modules (TCM) on the car roof ensures stable thermal environments and uniform RF conditions for all probes.

After data collection, the key question is how to extract insights from the raw data. The clearer the insights, the easier it is  to define actions.

Conclusion

In the fast-paced and highly competitive world of mobile network operators and service providers, performing regular benchmarking tests is an indispensable investment in the future. To make smart decisions about current performance as well as future expansions and deployments, reliable and reproducible network performance data is essential.