Background : Traditional (Frequentist) Approach to Fixed Sample Size Trials

1. Sample size estimation is often a game to allow one to launch a study within a fixed budget no matter what is really required to detect a clinically relevant difference and no matter what precision is required for estimates of treatment effect
   • Studies are often sized to detect a more than clinically relevant difference
   • We often underestimate the variance of the response, which also results in sample sizes that are too small
     
     
   • The choices for a and b are arbitrary
   
   
2. The most common outcome of a clinical trial is `we don't know the treatment effect with any precision.' Thus all too often subjects who agree to participate in trials do not end up providing useful scientific information about the response variable.
3. Studies may be shortened sometimes; the answer may be known before the target sample size is reached.
4. Fixed sample size designs are inflexible, e.g., if the final P=0.06 we may not know what to conclude; often a definitive result may be obtained by extending a study somewhat
5. a-adjustment in the context of flexible sequential study designs is complex and there are many different methods for preserving type-I error
   
   
6. P-values are very frequently misinterpreted
   • Small P-values may not mean important treatment effects when n is large (i.e., statistical significance does not translate into clinical significance)
     
     
   • Large P-values mean nothing when n is small
     
     
   • P-values can provide evidence against a hypothesis but can never provide evidence in support of a hypothesis
7. There are good reasons to emphasize estimation instead of hypothesis testing; confidence intervals solve some of the problems of P-values

Bayesian Design before the Trial Begins

1. Choose patient response(s), randomization scheme, etc.
2. Choose a statistical model for the response(s): parametric (e.g., normal distribution) or rank-based
3. Describe state of prior knowledge about the effect of treatment on each response. Do this by choosing a prior probability distribution; this may be a flat distribution if there is no prior knowledge about the treatment effect. Some investigators choose a skeptical prior distribution that works against what they are trying to demonstrate; this will result in a study that is more convincing to skeptics.
   
   
4. Choose clinically relevant differences for secondary analyses of `clinical significance'
   
   
5. In a `non-inferiority' trial (e.g., equivalence trial, similarity trial) choose the lowest allowable treatment difference, e.g., we may want to compute the likelihood that a new drug is no more than 10% worse than the approved drug
6. If sufficient information is available, estimate minimum, average, and maximum sample sizes

As the Trial Proceeds

• Probability of efficacy, e.g., Prob[d < 0]; stop when > 0.95
• Probability of clinically important efficacy, e.g., Prob[ratio > 0.9]; stop when > 0.9
• Probability of similarity, e.g., Prob[-.2 < d < .2]; stop when < 0.8
• Can also stop when run out of money; latest current probabilities are still valid
• Futility analysis ¾ predict how acquiring more data would change the result
  
  
• Can optionally modify treatment allocation ratio as results unfold

Bayesian Design of Proton-Pump Inhibitor Laryngitis Study

1. Two-group parallel design, minimum sample size 15 patients/group
   
   
2. Response variable is the maximum of four symptom analog scale values (0-100)
   
   
3. Assume a normal distribution for the response
   
   
4. Let d denote the difference in population mean response (proton pump minus placebo)
   
   
5. Prior probability distribution for d: normal with mean zero and variance such that the probability that d > 50 or < -50 is only 0.05
6. As data accrue compute Prob[d < 0]; stop when > 0.95
   
   
7. Also compute Prob[-15 < d < 15]; stop when > .8