It’s the first question we get, and it’s a fair one. If you’re used to market-sizing studies with thousands of respondents, twenty sounds tiny.
The honest answer: it depends what you’re trying to do. For the job we do, twenty of the right people is not just enough. It’s often better.
What small samples are bad at
Let’s be clear about the limits. A 20-person study can’t:
- Give you precise market-wide percentages with tight margins of error.
- Detect small differences between lots of subgroups.
- Stand in for a census of an entire industry.
If you need any of those things, for pricing decisions or investor sizing, for example, you need a bigger study. We’ll tell you so.
What small samples are great at
Peer intelligence is a different job. You’re not measuring a market. You’re capturing what a specific group of senior people think right now, and sharing it with others like them. For that, small samples have real advantages:
- Seniority. It’s far easier to get 20 genuine CISOs than 2,000. Large panels often fill up with people who only loosely match the brief.
- Depth. Small studies leave room for open questions and real quotes, which are often the most compelling part.
- Speed. You can go from question to outreach in weeks, while the topic is still hot.
- Relevance. “20 CISOs at banks like yours” is more interesting to a bank CISO than “500 IT decision-makers.”
A precise answer from the wrong people is worth less than an honest answer from the right ones.
How to stay honest
Small-sample research earns trust when it’s transparent. Our rules:
- Always state the sample size and who was asked.
- Prefer “7 in 10” or “14 of 20” to falsely precise decimals.
- Call findings directional, not definitive.
- Never extrapolate to “the whole industry.”
Done right, the small sample isn’t a weakness to hide. It’s part of the story: we asked twenty people exactly like you, and here’s what they said.