Rotation testing is a framework for doing significance testing by computer simulations. An important application is the adjustment of univariate p-values in multiresponse experiments. When analysing several responses by individual significance tests, ordinary significance testing (F-test) is questionable since we will expect a lot of type I errors ("incorrect significance").

An alternative is to adjust the p-values according to the familywise error rate (FWE) criterion. The classical Bonferroni’s correction is, however, extremely conservative and becomes useless in cases with a large number of responses. By using rotation testing it is possible to adjust the p-values in an exact and non-conservative way (Langsrud, 2005).

FWE adjustment can be viewed as being too strict. Instead of considering the probability of at
least one type I error, an alternative is to estimate the false discovery rate (FDR) which is the
(expected) proportion of type I errors among all responses reported as significant.
The appendix (written by Ø. Langsrud) of Moen *et al.* (2005) describes a new way of
adjusting p-values according to FDR by using rotation testing (or permutation testing). Unlike other FDR
procedures, this method allows any kind of dependence among the responses.

ABSTRACT

This paper describes a generalised framework for doing Monte Carlo tests in multivariate linear regression. The rotation methodology assumes multivariate normality and is a true generalisation of the classical multivariate tests - any imaginable test statistic is allowed. The generalised test statistics are dependent on the unknown covariance matrix. Rotation testing handles this problem by conditioning on sufficient statistics.

Compared to permutation tests, we replace permutations by proper random rotations. Permutation tests avoid the multinormal assumption, but they are limited to relatively simple models. On the other hand, a rotation test can, in particular, be applied to any multivariate generalisation of the univariate F-test.

As an important application, a detailed description of how each single response p-value can be non-conservatively adjusted for multiplicity is given. This method is exact and non-conservative (unlike Bonferroni), and it is a generalisation of the ordinary F-test (except for the computation by simulations). Hence, this paper offers an exact Monte Carlo solution to a classical problem of multiple testing.

KEY WORDS: Conditional inference, Multiple testing, Random orthogonal matrix, Adjusted p-value, Multiple endpoints, Spherical distribution, Microarray data analysis.

ABSTRACT OF AN OLDER VERSION

Motivated by specialised significance tests,
Wedderburn (1975) ^{1)} described how to simulate
a multinormal sample conditioned on the sample mean and covariance matrix.
Unfortunately, Wedderburn died suddenly and his research report was never published.
Later, Cheng (1985) ^{2)} presented a simulation algorithm for the same problem.
However, during the years, the potential of such simulations has not been recognised.

The present paper extends Wedderburn's methodology to a generalised framework for doing Monte Carlo tests in multivariate linear regression. Compared to permutation tests, we replace permutations by proper random rotations. Thereby, more general problems can be solved. The rotation methodology is a true generalisation of the classical multivariate tests. The unknown parameters are handled by conditioning on their sufficient statistics. As a specific application, a detailed description of how each single response p-value can be non-conservatively adjusted for multiplicity is given.

1)
R. W. M. Wedderburn (1975),
Random Rotations and Multivariate Normal Simulation,
* Research Report, Rothamsted Experimental Station.*
Word file containing scanned images of the original document (2004 KB).

2)
R. C. H. Cheng (1985),
Generation of Multivariate Normal Samples with Given Sample Mean and Covariance Matrix,
* Journal of Statistical Computation and Simulation*,
**21**, 39-49.

**
Moen, B.,
Oust, A.,
Langsrud, Ø.,
Dorrell, N.,
Gemma, L.,
Marsden, G.L.,
Hinds, J.,
Kohler, A.,
Wren, B.W.
and Rudi, K. (2005),
An explorative multifactor approach for investigating global survival mechanisms
of Campylobacter jejuni under environmental conditions,
Applied and Environmental Microbiology,
71, 2086-2094.
**
[ pdf ]

ABSTRACT ( Copyright © American Society for Microbiology ) : Explorative approaches such as DNA microarray experiments are becoming increasingly important in microbial research. Despite these major technical advancements, approaches to study multifactor experiments are still lacking. We have addressed this problem using rotation testing and a novel MANOVA approach (50-50 MANOVA) to investigate interacting experimental factors in a complex experimental design. Furthermore, a new rotation testing based method was introduced to calculate false discovery rates for each response. This novel analytical concept was used to investigate global survival mechanisms in the environment of the major food borne pathogen

KEY WORDS:
*Campylobacter jejuni*,
survival in the environment,
microarray,
FT-IR spectroscopy,
50-50 MANOVA,
False Discovery Rate.