Sampling in quality monitoring: where does the 2% figure come from?

Sampling in quality monitoring: we traced the origin of the 1 to 2% and the 2 to 4 per agent figures. What COPC and ISO 18295 actually require.

BRIGGS Team10 min readLer em português

Introduction

Two numbers show up in almost every piece of content about quality monitoring in call centers: "1 to 2% of calls get audited" and "do 2 to 4 evaluations per agent per month." They circulate in vendor blogs, sales decks, supervisor training and procurement documents.

We went looking for the primary source of both. There isn't one.

What you find when you look for the source

Searching for "1 to 2%" and "2 to 4 evaluations per agent per month" always leads to the same kind of material: a vendor blog post with no attribution. The number appears as a market claim ("the industry average is..."), never as a citation of a study, a clause in a standard or a survey with stated methodology.

The closest numerical trail to an origin for "2 to 4" came from a PDF published by a call recording vendor, which showed a frequency distribution:

Evaluations per agent per month Share
2 to 4 34%
5 to 6 26%
7 to 10 19%
11 or more 14%

The problem with this table is what it leaves out. There is no named survey, no sample size, no collection date, no description of how respondents were selected. Without that, the table is a claim, not data. And note that even taking it at face value, "2 to 4" is the range for only 34% of cases. The majority (59%) do 5 or more. The number became an industry rule while being a minority in the very table that supposedly produced it.

Call Centre Helper, one of the most cited publications in the English-speaking industry, shows how shaky this foundation is. Over the years it has published recommendations that contradict each other: 4 to 8 evaluations per month, 5 to 6, 10 per month, 8% of total calls. It also published a reader poll in which 36% of respondents said they monitor more than 10 calls per agent per month, which directly contradicts the "2 to 4" legend. When four incompatible recommendations live in the same publication, what you are reading is editorial opinion shifting over time.

What COPC actually requires

COPC is the reference most often invoked when someone wants to lend authority to a sampling number. Two things are worth noting first: it is a proprietary standard owned by COPC Inc., not a public standard, and it does not define a percentage.

What the standard defines is the frequency and the properties of the sample. The literal requirement is:

"Each CSS must be monitored at least every month and must be monitored for each transaction type at least once each quarter"

In other words: each agent monitored at least once a month, and at least once a quarter for each transaction type they handle. It is a frequency floor with coverage by contact type. Nothing about a percentage of volume.

The second requirement is the one most people ignore: the sample selection method must be unbiased. The standard cares about how calls get into the sample, not how many. That flips the usual question. COPC doesn't ask "do you audit 2%?" It asks "were the calls you audit chosen in a way that doesn't distort the result?"

What COPC's own benchmark shows

COPC publishes the Global Benchmarking Series, a survey with stated methodology. In the 2022 edition (more than 900 executive respondents, data collected from September 1 to December 18, 2021, global scope), the relevant numbers are:

Indicator Result
Monitor at least once a month 88%
Select the sample randomly 63%
Have a formal QA program 92%
Use remote monitoring of recordings 95%
Agents per QA staff member 54 to 1

Look at what this set measures: frequency, selection method and team structure. Sampling percentage doesn't appear, because a question about percentage isn't one the standard considers relevant. The 63% random selection figure is the most revealing in the series: more than a third of responding operations admit they select their sample by some criterion that isn't random.

ISO 18295-1:2017: what the standard says and what it doesn't

ISO 18295-1:2017, "Customer contact centres, Part 1," exists and is the closest international reference to a contact center standard. It doesn't set a sample size either.

Clause 4.3 states that the contact center "shall take action to understand the experience customers have," and mentions that this can be done through quality evaluation, speech analytics or complaint analysis. It requires that a mechanism for understanding the experience exists, with freedom of method. There is no mention of sample size, percentage or minimum frequency.

The standard has an Annex A of metrics, but it is informative, not mandatory. That annex is paid content and we have not verified it, so we make no claims about what it contains. What can be said with confidence is that the normative requirement in clause 4.3 does not mention sampling.

PROBARE: the Brazilian reference

In Brazil, the self-regulation reference is PROBARE (the Brazilian self-regulation program for customer relationship centers), with its Management Maturity Standard, Revision 2, dated June 29, 2007, coordinated by ABEMD, ABRAREC and ABT (Brazilian industry associations for direct marketing, customer relations and telesservices).

PROBARE requires a monitoring process. It does not define a sample, a percentage or a number of audits. The two relevant items are:

  • 4.2.3.b: requires "Monitoring of the operation and management of results, with corrective actions taken"
  • 4.2.4.b: requires "Procedures that ensure the consistency of the monitoring process"

Item 4.2.4.b is the most relevant here, for the same reason as COPC: the concern is process consistency, not volume.

The standard also states explicitly that only the "framed" requirements in the document are mandatory, and that the "Examples of Management Practices" are not. Anyone citing PROBARE to justify a percentage is citing something that isn't there.

One more point: ABT and ABRAREC do not publish their own monitoring standard or benchmark. And there is no public standard (ISO, ABNT, Brazil's national standards body, or any other Brazilian standard) that sets a sampling percentage for quality monitoring.

The number matters less than the properties of the sample

Put together what the verifiable sources say and a pattern emerges. COPC asks for an unbiased sample. PROBARE asks for process consistency. ISO 18295 asks for a mechanism to understand the experience. None of the three asks for a percentage. What they require is a statistical property, and that property determines whether monitoring teaches you anything.

A random 2% sample lets you estimate the quality distribution of the entire operation, with a margin of error you can calculate. A 5% convenience sample does not. If the QA analyst picks calls from the top of the queue, the shortest ones (because they fit their schedule) or the ones the supervisor asked them to listen to, the result describes that slice and nothing more. Doubling the size of a biased sample doesn't fix the bias; it just gives you more confidence in a wrong number.

Three common forms of bias in practice:

  • Convenience bias: you audit what is easy to audit. Short calls get in more than long ones, and long calls are where the problems concentrate.
  • Time-of-day bias: you audit during the QA analyst's hours, almost always business hours. Evening peaks, overnight and weekends are left out of the picture.
  • Referral bias: you audit what someone flagged (a complaint, an escalation, a supervisor request). That is case investigation, useful in its own right, but not a sample of the operation.

Each one undermines inference, and none is fixed by raising the percentage.

The question that settles the argument

If you audit 2% of calls, what do you know about the other 98%?

With a random sample and a large enough N, the answer is: I can estimate the mean and spread of quality, with a margin of error I can state. With a convenience sample, the honest answer is: nothing. It's also worth acknowledging that traditional sampling was never designed to answer this. It grew out of a capacity constraint: there were more calls than people to listen to them, so a fraction got heard. The percentage was a consequence of QA team size and became "best practice" through repetition. A capacity constraint is not a methodological principle. When the constraint changes, the "1 to 2%" yardstick should be revisited, not quoted.

As long as sampling exists in your operation, two decisions matter more than its size:

  1. Make selection random and document the criteria. Draw by period, by agent and by contact type, with a record of how the draw was done. That is what satisfies the unbiased requirement.
  2. Keep sampling separate from investigation. Calls audited because they generated a complaint or escalation don't go into the average quality calculation. They form a second stream, with a different purpose.

Frequently asked questions

Is there any standard that defines a sampling percentage for quality monitoring? No. ISO 18295-1:2017, PROBARE and COPC do not set a percentage. There is no public Brazilian or international standard that establishes a number.

How many evaluations per agent does COPC require? It requires a minimum frequency, not a percentage: each agent monitored at least once a month and at least once a quarter for each transaction type. It also requires that sample selection be unbiased.

Where did "2 to 4 evaluations per agent per month" come from? The only trace we found was a distribution in a PDF from a call recording vendor, with no named survey, no sample size, no date and no methodology. In that table, the 2 to 4 range represented only 34% of cases.

Does PROBARE say anything about sample size? No. Items 4.2.3.b and 4.2.4.b require monitoring with corrective actions and procedures that ensure process consistency. Only the framed requirements are mandatory.

Is a 5% sample better than a 2% sample? It depends on how the calls were chosen. A random 2% sample supports inference about the whole; a 5% convenience sample describes only the slice that was heard. Increasing the size doesn't fix bias.

How do I know if my sample is biased? Compare the distribution of audited calls with the distribution of all calls along three axes: duration, time of day, and channel or contact type. If the profiles diverge, your selection is introducing bias.


Keep reading

To see what percentage of your operation goes unaudited today, the coverage calculator gives you the number in under a minute, free and with no sign-up.