“The study included 10,000 families” sounds reassuring. A narrow confidence interval can also look authoritative. Both can tell you something about precision, but neither can rescue a badly framed question, a biased sample or an irrelevant outcome.
- Sample size is the number studied
- Ask “sample of whom?” before “how many?”
- A confidence interval shows a range of plausible estimates
- What the interval does not tell you
- Do not confuse statistical significance with importance
- A practical reading sequence
- Small samples are not automatically useless
- Sources and further reading
Sample size is the number studied
A sample is the group researchers observe in order to learn about a larger population. Larger samples often reduce random sampling variation, making an estimate more precise. But size is not representativeness. Ten thousand volunteers from one app may resemble each other more than the wider population.
The Office for National Statistics explains that different samples can produce different estimates and that sampling error is only one source of uncertainty. Non-sampling errors include people not responding, inaccurate answers and processing or analysis mistakes.
Ask “sample of whom?” before “how many?”
Check age, location, language, income, disability, family structure, recruitment method and exclusions. If a study about school routines includes only families who completed a lengthy online survey, families with less time or internet access may be underrepresented.
Also note attrition: how many started and how many remained at the end. A large starting sample can become much smaller, and the people who leave may differ from those who stay.
A confidence interval shows a range of plausible estimates
Researchers often report one best estimate plus an interval reflecting sampling uncertainty under their statistical assumptions. A narrower interval usually means greater precision; a wider interval means the data are compatible with a broader range of values.
For an invented example, suppose a survey estimates that 40% of families use a bedtime routine, with a confidence interval from 34% to 46%. The midpoint is not a promise that exactly 40% of every community follows that routine. The range does not account for every possible bias, poor question wording or missing group.
What the interval does not tell you
- Whether the study asked the right question.
- Whether participants represent your child or community.
- Whether an association is causal.
- Whether a measured difference matters in daily life.
- Whether researchers made appropriate analysis choices.
A very precise estimate can be precisely wrong if the underlying data are biased. A small study can still be valuable for rare conditions, early exploration or detailed qualitative work, though it should not be made to support claims beyond its design.
Do not confuse statistical significance with importance
Statistical significance addresses whether an observed difference may be difficult to explain by random sampling alone under a specified model. It does not tell you that the difference is large, helpful, harmful or relevant to your family. Look for the size of the effect and the range around it.
A practical reading sequence
- What population are researchers trying to understand?
- How was the sample recruited?
- How many people provided usable data?
- What is the estimate and its confidence interval?
- Are the possible values meaningfully different?
- What bias or missing data could remain?
Use the “I don’t know yet” script when the result is uncertain. For health decisions, take the study and your questions to a qualified professional; the appointment question cards can help you prepare.
Small samples are not automatically useless
Some questions involve rare conditions, intensive measurements or groups that are hard to recruit. In those settings, a small study may be the most feasible starting point. Judge whether the methods match the claim. Detailed interviews with 20 families can illuminate experiences, but they cannot estimate a national percentage precisely. The problem is not smallness alone; it is claiming more certainty or breadth than the design can support.
Always read the authors’ own limitations and compare the estimate with other well-designed studies before changing family practice.
