Is sample size calculation only about numbers?

Sample size calculation has been the most important universal question raised by almost all review committees during protocol review and is also the most bothering question for researchers, specially the clinicians. In this article, we have discussed about the need of calculating sample size and the prerequisites of a sample size calculation. We would try to discuss the following question:“Is sample size a number obtained as a result of putting few numbers in statistical software for calculation of sample size or more than that?”

The general perspective of a sample size calculation is largely misunderstood and is believed to be a customary essential part of a research protocol for approval by a research review committee, rather than an essential component of research for external validity. Most of the researchers, who are not adequately trained in epidemiology and biostatistics do not consider it as important for conducting a valid research and frequently approach an epidemiologist or a statistician for calculation of sample size with few articles and numbers in hand. The researchers generally feel that sample size calculation only requires some figures from similar previous studies, which need to be put in the statistical software to obtain another figure. However, sample size calculation is much beyond that. It is an essential component of any research study, because it provides the much-needed external validity or generalizability to the study. Sample size calculation has certain prerequisites or requirements.Some of these such as power, alpha error, proportions or means, margin of error, and number of groups are required as input in the statistical software;however other inputs which are hidden behind these required inputs are not explicit and therefore need to be understood.

Sample size calculation is based on the type of study design (Machin et al.,2009). Each and every study design requires a different sample size formula and different inputs for final calculation. It is therefore imperative that the researchers decide about their study design before approaching the statistician for sample size calculation. The formula for calculating sample size for cross sectional study is different from that of a case control study or a randomized controlled trial. Similarly, diagnostic test studies have different formulas for sample size calculation as do the cohort or follow up studies with observations done at multiple points.

Sample size calculation essentially depends upon the outcomes of the study,something that most of us miss to mention in the study protocol. The outcome may be in terms of quantitative variables, which are descriptively summarized as means, such as mean 6-minute walk distance and mean systolic blood pressure; or they may be categorical variables, summarized as proportions,such as proportion of people having infection after a surgical procedure versus another group with a different intervention. The outcomes and the objectives of a study are interlinked and the inability to frame a right objective reflects the lack of clarity with respect to the outcomes and study design. Thus, before starting for sample size calculation we also need to frame our objectives really well in terms of the SMART criteria, i.e., specific, measurable, achievable,relevant and time oriented. Amongst all these, the criterion, “measurable,”is most important with respect to sample size calculation. If we frame our objectives in a measurable manner, the research question becomes clear and operational and so does the sample size calculation.

For example, “Evaluate the effect of an intervention Avs. B on chronic obstructive pulmonary disease (COPD) patients” is a badly framed objective and does not give a clue about the outcome.

“Evaluate the effect of intervention Avs. B on forced expiratory volume in one second (FEV1) at 2 months among mild grade of COPD patients” is a good objective as it is specific, measurable (FEV1), achievable, relevant and time oriented (2 months). It also provides us an idea of the outcome, i.e.FEV1, which is a quantitative variable summarized as mean FEV1 and would be used for sample size calculation.

Sample sizes are calculated for two types of scenarios. In one scenario,we want to “estimate” an outcome, which is done in a cross-sectional study;in another scenario, we wish to “compare” the outcomes between two or more groups, such as in case control studies, cohort studies and randomized controlled trial. The number of these groups may be two or more than two depending on the research question. For estimating an outcome, which may be either quantitative (e.g., mean cholesterol level in a specified population)or categorical (e.g., proportion of people with hypertension), we require a confidence level (usually 95%), an estimate of mean/standard deviation or proportion from previous similar study, margin of error (determined by the researcher) and the study population size for population correction. We also require the sampling technique/strategy used for recruiting participants as anything that is not random requires an adjustment in sample size calculation known as design effect. So we need to know what we want to obtain from the study through a SMART objective and the sampling technique as well for calculating sample size.

Calculation of sample size for comparison of outcomes in two groups would require both alpha error (usually 0.05), power (usually 80%) and estimates in the two groups depending on the objectives in terms of categorical or quantitative variables. For example, for an objective of “To determine the effect of intervention A and B on mean FEV1 among patients of COPD after two months of follow up,” we need to have alpha error (usually 0.05), power(usually 80%) and the estimates of FEV1 in the two groups at 2 months from previous studies.

It is also important to note that while we input numbers from previous“similar” study for sample size calculation, we need to ensure that the study used has “similar” inclusion criteria as that of present study. Also, studies from“similar” geographical region are preferred over others. This is commonly overlooked both by the researcher and the review committees and therefore leads to inappropriate estimates (Levy and Lemeshaw, 1999).

The researchers are frequently found to report that no similar study has been conducted in the past and therefore they have no estimates. In such situations,an estimate of the outcome from clinical setting/ pilot testing/ other observational studies or from Phase 2 clinical trials in case of a randomized controlled trial should be obtained, since Phase 3 trials or randomized controlled trial are not permitted until the results from previous phases are available. Only in the rarest situation of no estimates should we take an estimate on our own.

Another factor that is important for sample size calculation is the nonresponse and dropout rate which need to be adjusted for, in final sample size calculation.

In summary, sample size calculation is not just about putting in few numbers for obtaining another magical number. It requires that the researcher is ready with other information such as study design, SMART objectives, specific measurable outcomes, sampling technique, estimates from studies with similar inclusion criteria and outcomes, expected dropout rate and non-response rate and the size of study population for population correction. It is therefore essential that each study protocol is discussed with an epidemiologist to ensure that all the elements of methodology are in place before the final sample size calculation is undertaken. Finally, a word of caution: if the calculated sample size is not achievable due to logistic reasons, it should be stated at the outset as well as in the limitations, instead of making adjustments in margin of error,alpha error and power, so that the evidence generated is meaningful.