The 12 Metrics Every Sample Buyer Should Be Requesting

ESOMAR 37’s final question lists twelve specific metrics a provider should be able to produce, in aggregate and by source. Here’s the list, and why each one is worth asking for before you sign.

---

ESOMAR 37 includes a question — number 37, fittingly the last one — that lists twelve specific metrics a sample provider should be able to produce for you, in aggregate and by source. Response rate trends. Rejection rates. Quarantine rates. Device mix. Tenure of paid completes. Feasibility ceilings under specific conditions.

Most buyers never ask for this list. Most providers are quietly relieved.

Here’s the list, and why each one is worth asking for — before you sign, not after a project goes sideways.

The metrics

1. Average qualifying or completion rate, trended by month. A single project’s completion rate tells you almost nothing. A twelve-month trend tells you whether a provider’s panel health is improving, degrading, or being propped up in ways that won’t hold.

2. Percent of paid completes rejected per month, trended. Rejection rate is a proxy for how seriously quality control is actually applied — not just described in a sales deck.

3. Percent of members/accounts removed or quarantined, trended. This is the panel’s immune system in numbers. A provider with a zero here either has a flawless panel or isn’t looking.

4. Percent of paid completes from 0–3 months tenure, trended. New-member concentration matters. A panel leaning heavily on brand-new members is either growing fast or churning fast — worth knowing which.

5. Percent of paid completes from smartphones, trended. Relevant to instrument design and data comparability, especially for trackers running over time.

6. Percent of paid completes from owned/branded relationships vs. intercept participants. This is the proprietary-versus-supplemented question in numbers rather than adjectives.

7. Average number of dispositions per member, trended. How often is the same person being asked to participate? Too high, and you’re looking at professional-respondent risk.

8. Average number of paid completes per member, trended. Same concern, different angle — concentration of survey activity within a small subset of the panel.

9. Active unique participants in the last 30 days. A basic but essential health check on panel size relative to claimed size.

10. Active unique 18–24 male participants in the last 30 days. A stand-in for a broader point: can the provider produce activity numbers for a genuinely hard-to-reach cell, not just the panel overall?

11. Maximum feasibility in a specific country with nat rep quotas, seven days in field, 100% incidence, ten-minute interview. A standardized stress test. If a provider can’t answer this cleanly, their other feasibility estimates deserve extra scrutiny.

12. Percent of quotas that reached full quota at time of delivery, trended by month. The most direct measure of whether promised sample actually shows up as promised.

Why this list is the real test

Any provider can tell you their panel is high quality. Far fewer can hand you twelve months of trended data proving it. The gap between those two things is usually the gap between marketing and methodology.

If you ask a provider for this list and get a confident, specific answer — by source, trended, no hedging — that’s a good sign about a lot of things beyond the twelve numbers themselves. If you get a paragraph about commitment to quality instead, you have your answer too.

Next in this series: why “proprietary” and “exclusive” aren’t the same thing, and why the distinction should change how you read a sample provider’s pitch.