Telecommunication: Measuring Quality of Service
In the telecommunications industry, price is no longer the sole factor in purchasing decisions; quality is key. However, information about quality of service is hard to obtain, if it's even available. Some government agencies are trying to change.
Unlike prices, which users can easily compare across carriers, the telecommunications industry rarely publicizes quality of service (QoS). To address this gap, national regulatory agencies (NRAs) in many countries have introduced QoS programs.
In Poland, the country's regulatory agency for the telecom market, the Office of Electronic Communications (UKE), investigated those programs to find which best practices could be applied to a new QoS regulatory scheme.1 What is the scope of such regulations in other countries? Do they apply to all kinds of telecommunication services or just a few, and if so, which ones and why? What are the methods of measurement and who conducts it?
To help the UKE, A.T. Kearney sought the answers to these questions by offering a set of possibilities, based on a survey of 21 countries. This paper highlights the findings.
Evaluating Four Services
NRAs worldwide are requiring telecommunication operators to measure QoS for four separate services: fixed-line telephony, universal service, mobile telephony and Internet (see figure).
Fixed-line telephony.Fixed-line telephony is often a crucial element to verify in terms of quality of service as a part of national and international telecommunication systems. In almost every country we investigated, fixed-line telephony operators are required to measure and publish QoS data.
Universal service. Universal service is a package of basic services (usually in fixed-line telephony) that a licensed operator should be able to provide to every citizen at a reasonable price within a satisfactory level of quality. In all but three cases we studied, universal service providers are required by regulatory authorities to report on their quality of service and publish it, for example, on their website or in sales locations. Failing to meet minimal standards can lead to fines.
Mobile telephony and Internet connections. A few countries, including Italy, Latvia and India, have introduced minimum quality requirements in these still-evolving areas. The standards help provide higher quality of service throughout next-generation networks, mobile communications and digital technologies.
Regardless of service type, all operators in a country or area usually have to verify service quality. However, there are examples (France, Canada and, until recently, the United Kingdom) in which operators are limited to establishing required income levels, time in business, number of access lines or subscribers, and time of switched calls.
Measuring Quality of Service
We identified three dominant ways quality of service is measured and reported.
Key quality indicators (KQIs). These indicators (patterned after key performance indicators) gauge QoS and make the results comparable across time periods and carriers. Many NRAs, usually after a series of public consultations, have introduced sets of indicators for different services, depending on the scope of regulation, definitions, measurement guidelines and expected levels of quality. The sets vary and can relate to both customer service and technological issues. Unsuccessful call ratios, supply times for initial connection, response times for operator services and bill correctness are among the most popular indicators.
To make it less prone to measurement biases, the KQI method requires gathering all of the data under a specified, recurrent period, not just a one-time sample. Additionally, this method enables operators to distribute regulation costs proportionally across the market as usually these are operators who are obliged to gather and process data.
The method has been adopted in all countries we examined, and international bodies including the European Communication Standardization Institute and European Union have endorsed it.
Live testing. Another way of measuring quality of service is live testing. Companies use it irregularly and, because of its costs, it can only cover a representative sample of services and end-users. Only a few countries we studied have used these—France, Turkey, the United Kingdom, Latvia and India—to measure the quality of mobile services or Internet connections.
In measuring mobile telephony, the live tests may be performed with the help of a custom vehicle with dedicated equipment, antennas or other facilities capable of gathering QoS data. In car testing, the vehicle must adhere to a specified route, usually covering the biggest cities and most crowded travel routes. For broadband connection tests, access to customers' lines is essential. For example, British regulator Ofcom performed such live testing in the United Kingdom in 2008 and 2009 to verify the quality of its broadband network. Dedicated facilities with special software were placed on the access lines of around 1,600 consumers and data was collected for a three-month period.
The goal of this method is to present the quality of a given service at a particular moment in time.
Consumer survey. Another method of measuring quality of service is performing consumer surveys. This method can effectively pinpoint the weakest elements of service quality, giving operators great feedback, while allowing customers to compare opinions about various operators with other people. It is also a great addition to the indicator-based method of measurement. Contrasting those two sets of data can determine whether a weakness identified by consumers also falls among the low-levels of relevant indicator data. If not, proper verification of both activities can be performed. The Telecom Regulatory Agency of India is a good example here; it performs this type of joint measurement for 23 regions of the country to ensure the validity of QoS data. Customer surveys are used also in Germany.
Publishing Quality of Service Data
The frequency of QoS data publication and reporting to NRAs varies across countries, from quarterly to twice yearly to annual or only on the agency's request.
The most popular method of presenting QoS data is by publishing it on a corporate website. The reports, along with the regulatory schemes, should be up-to-date, presenting the most recent results in a way that customers can understand. The information should be precise, concise, and indicate the time period in which data was collected. A problem we identify in most countries is that operators typically publish only their own data, without comparing it to their competitors’ results. This makes comparing different carriers difficult for customers, who are forced to search for the information across multiple websites.
Periodic regulatory reports can also make QoS information more widely available. Usually, these reports present data for all operators over recurrent periods of time. They enable customers to compare quality differences more easily among several service providers. However, many end-users do not know that the QoS data is available, nor do many know about the NRAs’ websites that publish the reports.
New Technologies, New Strategies
Our findings suggest changes are taking place in QoS strategies. In some countries, for example, the previously introduced rules failed to reflect the real quality of services as new technologies such as VoIP telephony became available. NRAs decided that some QoS regulations —for example, technology-based indicators—no longer fulfilled their initial purpose of pro-viding consumers with clear and up-to-date and information about quality.
The U.K.'s telecommunications market regulator Ofcom offers an example about ensuring quality. Not long ago, Ofcom required fixed-line operators (those with more than £4 million in revenues) to measure QoS and publish results on a dedicated website. However, most consumers, Ofcom found out, were not aware of the website and hardly used it. On top of that, QoS data sets submitted by operators were often based on different indicators and measurement methods, which meant they were not truly comparable. In July 2009, after a series of public consultations, Ofcom decided to shut down the project and released operators from the QoS regulations.
The best direction for QoS is still being discussed in national and international forums, with NRAs still exploring the best methods for addressing consumer expectations. To date, no universal solution has been discovered. QoS information is still difficult for people to access and compare as both operators and NRAs often act without first consulting with those most interested in the matter—consumers. According to our findings, the following are essential to a successful quality of service strategy:
- Fair assessment of market imperfections and information issues
- Assessment of consumer's expectations, needs and demand for QoS information
- Easy access to the QoS data, presented in a comprehensible way
- Periodic verification of regulation
As our survey demonstrates, agencies can regulate QoS in many different ways. However, no NRAs have fully captured the nature of the telecom market's ever-changing landscape. Indeed, what seems to be a good regulatory tool now may soon prove inadequate, given the speed with which technology advances and customer habits and expectations change. Responding promptly and insightfully to new developments is becoming a vital skill for regulatory agencies.
At the same time, many telecommunication operators have been slow to realize the potential of using QoS as a competitive advantage. In mature markets, where price—and value-based strategies are less effective, quality of service may prove to be a key differentiator. Once carriers grasp this opportunity, QoS regulatory programs may become unnecessary.
1UKE stands for Urzqd Komunikacji Elektronicznej.