Why Panels and Synthetic Data Are Better Together

The Unlocked Lab: What Research Looks Like When the Old Constraints Disappear

At the end of Part 1, I asked what research we’ve never attempted because we assumed we couldn’t. It’s worth sitting with that question for a moment before answering it.

Most of the design constraints we treat as permanent features of survey research aren’t methodological laws. They’re practical accommodations to real limitations: limited respondent pools, the cognitive cost of participation, the economics of recruiting rare audiences, the impossibility of tracking the same verified humans across years without losing them to attrition or panel fatigue. We built our entire research toolkit around those limitations. We internalized them so thoroughly that we stopped questioning them.

But what happens when the limitations change? What happens when high-quality verified human panels and rigorously trained synthetic data work together — not as competitors, but as partners? The answer, I think, is more interesting than most of the synthetic data conversation has acknowledged so far.

Panels and Synthetic Data Are Not Competing. They’re Compounding.

Let me be clear about something before going further, because this point gets muddled constantly in industry conversations: synthetic data without high-quality human panels is a house built on sand. The verified human respondent — properly recruited, identity-confirmed, behaviorally traceable — is not a relic of pre-AI research. It is the foundation that makes everything else defensible.

What synthetic data does is amplify what good panels make possible. It extends coverage to populations that panels can’t economically reach at scale. It eliminates the fatigue and attrition that degrade panel data over time. It allows researchers to run study designs that would be logistically impossible or prohibitively expensive with human-only samples. But the model is only as good as the human data it learned from. The panel is the ground truth. Synthetic is the multiplier.

That’s the partnership. Not replacement. Compounding.

And once you accept that framing, the research design possibilities become genuinely exciting.

Constraint #1: The Survey Had to End Somewhere

Every researcher who has ever fielded a study has had this conversation: “We need to cut questions. The survey is already 18 minutes and the client wants to add a brand equity battery, a NPS module, and six concept boards.”

Survey length has always been a proxy for respondent tolerance. We knew that beyond a certain point — somewhere between 12 and 20 minutes depending on the audience and incentive structure — data quality degrades. Speeders multiply. Straight-lining increases. Dropout rates climb. So we made hard choices about what to include and what to leave on the floor. Entire research questions got cut not because they were unimportant, but because we didn’t have the respondent bandwidth to ask them.

With synthetic data properly trained on verified human responses, that constraint dissolves. You can field a lean, high-quality human instrument — focused, engaging, methodologically tight — and then use the synthetic layer to answer the questions that didn’t make the cut. Want to know how the same respondent profile would react to three additional price points? How a different segment would score the same brand equity battery? What the attitudinal data would look like among a population you couldn’t economically recruit? The synthetic layer can model it, calibrated against the human anchor.

The survey doesn’t have to end where the respondent’s patience runs out. It ends where the research question does.

Constraint #2: Complex Exercises Were Rationed

Conjoint analysis is one of the most powerful tools in the research toolkit — and one of the most expensive to field correctly, not in dollars but in respondent cognitive load. A well-designed conjoint can tell you exactly how people trade off features against price, which attributes drive choice, and where the market is genuinely elastic. But a full conjoint with meaningful attribute depth requires sustained attention over a long exercise. Put it late in a survey and the data degrades. Put it early and you’ve used up most of your respondent’s engagement budget before you’ve asked anything else.

MaxDiff has the same profile. So do complex concept evaluation exercises, sequential monadic designs, and anything involving realistic product simulations or detailed scenario walkthroughs. We’ve always known these methods produce superior insights. We’ve also always known we couldn’t stack them.

A synthetic approach trained on validated conjoint and MaxDiff responses from high-quality verified panels changes that calculus. You run the full human exercise on a carefully managed subset, verify the quality of the data rigorously, and use the synthetic layer to extend the findings across a broader population or a wider attribute space. Researchers who have been rationing their best methodological tools because of respondent fatigue constraints suddenly have more room to work.

Constraint #3: Some Populations Were Effectively Unreachable

Here’s a research design problem I’ve watched frustrate clients for years: you need to understand the decision-making process of a very specific type of buyer. Not just “IT decision-makers” broadly — but IT decision-makers at manufacturing companies with between 500 and 2,000 employees who are currently evaluating ERP systems and have final budget authority. That population exists. It matters enormously to the client. And it is, for practical purposes, nearly impossible to recruit at meaningful scale through traditional panel methods without either accepting massive quality compromises or spending more on recruitment than the study is worth.

The same problem shows up across healthcare research (rare disease patient populations, narrow specialty physician segments), consumer research (very low-incidence behavioral segments), and financial services (ultra-high-net-worth individuals, niche product adopters). We’ve always known these populations were important. We’ve always struggled to study them rigorously.

Synthetic data trained on verified respondents from adjacent or overlapping populations — combined with whatever genuine hard-to-reach data can be collected — gives researchers a credible path to these segments for the first time. Not by fabricating responses, but by building models that understand how the behavioral and attitudinal patterns of verified respondents map onto populations that can’t be directly sampled at scale. It’s model-based inference made explicit, transparent, and honest — which, as I argued in Part 1, is exactly what we’ve been doing informally for decades anyway.

Constraint #4: Longitudinal Research Was Always Leaking

I’ll make this one personal, because I spent a large part of my career trying to fix it.

Tracking studies and longitudinal panels have always suffered from the same slow hemorrhage: attrition. People drop out. They age out. They become disengaged. They get flagged for quality issues. The panel at wave 6 of a tracker is never quite the same panel as wave 1, and the differences aren’t random — the people who stay are systematically different from the people who leave, which introduces bias that’s hard to measure and harder to correct.

I’ve sat across from clients trying to explain why their brand equity scores shifted six points between waves — and spent far too much of that conversation not discussing what the market did, but unpacking what the panel did. Was it attrition? Refreshment sample composition? Panel conditioning from repeated exposure to the same battery? Seasonal respondent behavior shifts? Weighting drift? The honest answer was often: probably some combination of all of the above, and we can’t fully decompose it. That’s not a satisfying answer when a client is trying to make a budget decision based on the trend.

We managed with refreshment samples, re-weighting schemes, and flagging waves where composition shifted meaningfully. Some of it worked reasonably well. None of it fully solved the problem. The tracker still drifted, and clients still had to make decisions based on trend lines that mixed genuine market movement with panel artifact. I helped a lot of clients get closer to the truth. But I rarely felt like we got all the way there.

A hybrid model that uses a verified human anchor — a carefully maintained, high-quality panel — combined with a synthetic extension changes that calculus meaningfully. The synthetic layer can maintain consistency in the respondent profile across waves, correcting for the systematic biases introduced by attrition and allowing researchers to separate genuine market movement from panel composition drift. The tracker becomes more trustworthy, not less, precisely because the modeling assumptions are made explicit rather than absorbed silently into the trend line.

Constraint #5: Qual and Quant Were Always Speaking Different Languages

For as long as I’ve been in this industry, qual and quant have operated as separate disciplines that inform each other imperfectly and expensively. You run a quant study, find something interesting — a segment behaving unexpectedly, a brand attribute scoring anomalously, a price point drawing a reaction you didn’t anticipate — and then you go back to the field with a qualitative study to understand why. By the time you have the qual findings in hand, weeks have passed, the quant data is aging, and the client is already acting on the numbers without the context.

The problem was never conceptual. Everyone agreed that quant tells you what is happening and qual tells you why. The problem was logistical and economic. You couldn’t run them simultaneously on the same respondent base without overwhelming participants. You couldn’t hold a focus group among the same synthetic personas you just surveyed. You couldn’t probe the language and reasoning behind a conjoint preference without recruiting an entirely separate group, briefing a moderator, and waiting weeks for transcripts.

That constraint is now dissolving. Once you have a well-validated synthetic respondent profile — built on rigorously collected human data — you can conduct qualitative exploration among those same synthetic personas at any point, on demand. What does this segment actually mean when they say they value ‘reliability’ over ‘innovation’? How do they talk through a purchase decision at this price point? What language do they use when they’re explaining a brand switch to a friend? You can probe those questions directly, in depth, without recruiting a single additional human participant.

Qual and quant, running together, against the same stable respondent base, without the logistical gap between them that has always forced researchers to choose one or accept a delay before getting the other. The insight doesn’t have to wait for the next phase anymore. That alone changes how research is designed, sequenced, and delivered.

Constraint #6: Concept Testing Was Always Rationed

How many times have you been in a research design meeting where the client has eight concepts and the budget to test four? Or fifteen ad executions and the capacity to evaluate six? The whole point of early-stage concept and creative testing is to cast a wide net — to find the signals in a large space of possibilities before investing in development. But the economics of fielding human surveys have always forced us to narrow that net before we cast it.

The result is that decisions about which concepts to test — decisions made before the research even starts — end up shaping the findings more than the research itself does. We never find out whether concept 9 would have outperformed concept 3, because concept 9 never made the cut.

Synthetic data trained on verified human concept evaluation responses can test the full space. Run a rigorous human study on a curated set of concepts. Validate the quality of that data hard. Then use the synthetic layer to model how the full concept space would have performed, prioritize the most promising candidates, and focus your next round of human research where it matters most. The funnel gets wider at the top, which is exactly where it should be.

Constraint #7: Behavioral Continuity Was Always a Snapshot

Survey research has always been episodic. We field a study, collect a moment-in-time snapshot of attitudes and reported behaviors, and then the research ends. We ask people what they did last month and hope their recall is good enough to be useful. We run the same tracker quarterly and hope that four snapshots a year give us enough resolution to see what’s actually happening in the market.

The future that good panels are already moving toward — and that synthetic data makes more scalable — is continuous behavioral intelligence. Not surveys alone, but panels that combine verified human survey responses with passively observed behavioral signals, triggered in-the-moment research at behaviorally meaningful inflection points, and synthetic models that fill the gaps between observations.

Imagine tracking a verified panel of financial services consumers not just through quarterly surveys, but through a combination of consented behavioral observation, contextually triggered surveys fired at the moment of a relevant financial decision, and synthetic modeling that connects those moments into a coherent continuous picture. The research isn’t a snapshot anymore. It’s a film. And the insights it generates — about how attitudes lead to behaviors, how behaviors shift attitudes, how both respond to market events in real time — are fundamentally richer than anything a survey alone can produce.

What This Means for How Researchers Should Be Thinking

None of this happens automatically, and I want to be careful not to oversell it. The research designs I’ve described above are not easy to execute. They require rigorous thinking about what the human anchor data needs to look like, how models are trained and validated, what the appropriate benchmarks are, and how transparent the methodology needs to be for the findings to be credible. The garbage-in-garbage-out principle I emphasized in Part 1 applies here with equal force.

But the shift in mindset I’m encouraging is real and I think it’s necessary. For a long time, research design has been a process of working backward from constraints. You start with what’s feasible — what you can field, what respondents will tolerate, what the budget allows — and you design the best study you can within those walls.

The walls are moving. Not disappearing — but moving. And researchers who are still designing studies as if the old constraints are fixed are going to find themselves producing research that’s less informative, less credible, and less valuable than what’s now possible.

The most exciting version of this industry’s future isn’t one where AI replaces the researcher. It’s one where researchers who understand both the power and the limits of these tools use them to ask questions we’ve never seriously been able to ask before. Where the bottleneck shifts from “what can we field?” to “what do we actually need to know?” Where verified human panels are recognized as the irreplaceable foundation they are — not because survey research is sacred, but because ground truth is.

The lab is unlocked. The question is what we build in it.

Disclaimer: The views expressed in this article are my own and do not represent the positions, strategies, or opinions of ROI Rocket or any affiliated organization.

References & Further Reading

Orme, B. K. (2010). Getting Started with Conjoint Analysis: Strategies for Product Design and Pricing Research (2nd ed.). Research Publishers LLC. — Practical and methodological foundation for conjoint analysis design, including the tradeoffs between attribute depth and respondent burden.

Louviere, J. J., Flynn, T. N., & Marley, A. A. J. (2015). Best-Worst Scaling: Theory, Methods and Applications. Cambridge University Press. — Theoretical and applied treatment of MaxDiff / best-worst scaling, including design efficiency and cognitive load considerations.

Lynn, P. (Ed.). (2009). Methodology of Longitudinal Surveys. Wiley. — Comprehensive treatment of attrition, panel conditioning, and refreshment sampling in longitudinal research designs.

Brick, J. M., & Williams, D. (2013). Explaining Rising Nonresponse Rates in Cross-Sectional Surveys. The ANNALS of the American Academy of Political and Social Science, 645(1), 36–59. — Documents the structural causes of declining response rates and the implications for survey inference over time.

Rubin, D. B. (1987). Multiple Imputation for Nonresponse in Surveys. Wiley. — Foundational text for model-based approaches to missing data — directly relevant to the theoretical basis for synthetic data generation.