Insightful Short Course: Bootstrap Methods and Permutation Tests
Interest in computer based resampling methods has risen dramatically over the past 20 years. Two resampling methods, bootstrapping and permutation tests, has been applied successfully to areas of statistical modelling where "traditional" standard errors, confidence intervals and significance tests are unavailable or of doubtful accuracy.
Even in situations where traditional methods are usually applied, resampling methods are valuable as a validity check, and the answers may surprise many experienced statisticians. For example, the old rule of requiring sample sizes of at least 30 before applying Gaussian-based methods is inaccurate in the presence of skewness. Resampling methods offer graphical and numerical diagnostics for standard assumptions.
Resampling methods also offer practitioners greater flexibility in modeling. They are no longer constrained to use simple statistics such as sample means. They may use robust alternatives, and use resampling for inferences.
Similarly, resampling offers the flexibility to handle complex sampling situations, without the need for extensive analytical derivations. The basic rule is to resample in a way consistent with the original data collection. For example, when sampling from a finite population one should use a finite-population resampling method.
This course begins with a graphical approach to bootstrapping and permutation testing, illuminating basic statistical concepts of standard errors, confidence intervals, p-values and significance tests. We consider graphical and numerical diagnostic checks for the validity of traditional Gaussian-based inferences.
We then broaden our scope in three ways:
To a wider variety applications, including cases where bootstrapping fails, and how to recognize this
To consider additional sampling methods, including finite-sample and hierarchical sampling, and parametric bootstrapping
To discuss additional resampling methods, including the jackknife, influence methods, and cross-validation
The emphasis is on practical applications, with occasional notes about the underlying theory. Examples will be analysed using the statistical computing package S-PLUS, which has unparalleled resampling capabilities and the flexibility to deal with non-standard applications.
Statisticians faced with inferential problems where the use of standard results may be questionable or not available. Familiarity with S-PLUS is not necessary.
You will learn how to use resampling methods to for inferences, or to check the accuracy of standard methods, for a variety of statistical applications. Many attendees will gain a better understanding of statistical concepts such as standard errors, Gaussian approximations, and p-values.