Home | 50-50 MANOVA | Rotation Tests | Software | Fisher's Exact Test | Publications

Rotation Tests

A generalized framework for doing Monte Carlo tests in multivariate linear regression



Introduction

Rotation testing is a framework for doing significance testing by computer simulations. An important application is the adjustment of univariate p-values in multiresponse experiments. When analysing several responses by individual significance tests, ordinary significance testing (F-test) is questionable since we will expect a lot of type I errors ("incorrect significance").

An alternative is to adjust the p-values according to the familywise error rate (FWE) criterion. The classical Bonferroni’s correction is, however, extremely conservative and becomes useless in cases with a large number of responses. By using rotation testing it is possible to adjust the p-values in an exact and non-conservative way (Langsrud, 2005).

FWE adjustment can be viewed as being too strict. Instead of considering the probability of at least one type I error, an alternative is to estimate the false discovery rate (FDR) which is the (expected) proportion of type I errors among all responses reported as significant. The appendix (written by Ø. Langsrud) of Moen et al. (2005) describes a new way of adjusting p-values according to FDR by using rotation testing (or permutation testing). Unlike other FDR procedures, this method allows any kind of dependence among the responses.


Papers

Langsrud, Ø. (2005), Rotation Tests, Statistics and Computing, 15, 53-60.

ABSTRACT

This paper describes a generalised framework for doing Monte Carlo tests in multivariate linear regression. The rotation methodology assumes multivariate normality and is a true generalisation of the classical multivariate tests - any imaginable test statistic is allowed. The generalised test statistics are dependent on the unknown covariance matrix. Rotation testing handles this problem by conditioning on sufficient statistics.

Compared to permutation tests, we replace permutations by proper random rotations. Permutation tests avoid the multinormal assumption, but they are limited to relatively simple models. On the other hand, a rotation test can, in particular, be applied to any multivariate generalisation of the univariate F-test.

As an important application, a detailed description of how each single response p-value can be non-conservatively adjusted for multiplicity is given. This method is exact and non-conservative (unlike Bonferroni), and it is a generalisation of the ordinary F-test (except for the computation by simulations). Hence, this paper offers an exact Monte Carlo solution to a classical problem of multiple testing.

KEY WORDS: Conditional inference, Multiple testing, Random orthogonal matrix, Adjusted p-value, Multiple endpoints, Spherical distribution, Microarray data analysis.

ABSTRACT OF AN OLDER VERSION

Motivated by specialised significance tests, Wedderburn (1975) 1) described how to simulate a multinormal sample conditioned on the sample mean and covariance matrix. Unfortunately, Wedderburn died suddenly and his research report was never published. Later, Cheng (1985) 2) presented a simulation algorithm for the same problem. However, during the years, the potential of such simulations has not been recognised.

The present paper extends Wedderburn's methodology to a generalised framework for doing Monte Carlo tests in multivariate linear regression. Compared to permutation tests, we replace permutations by proper random rotations. Thereby, more general problems can be solved. The rotation methodology is a true generalisation of the classical multivariate tests. The unknown parameters are handled by conditioning on their sufficient statistics. As a specific application, a detailed description of how each single response p-value can be non-conservatively adjusted for multiplicity is given.

1) R. W. M. Wedderburn (1975), Random Rotations and Multivariate Normal Simulation, Research Report, Rothamsted Experimental Station. Word file containing scanned images of the original document (2004 KB).
2) R. C. H. Cheng (1985), Generation of Multivariate Normal Samples with Given Sample Mean and Covariance Matrix, Journal of Statistical Computation and Simulation, 21, 39-49.


Moen, B., Oust, A., Langsrud, Ø., Dorrell, N., Gemma, L., Marsden, G.L., Hinds, J., Kohler, A., Wren, B.W. and Rudi, K. (2005), An explorative multifactor approach for investigating global survival mechanisms of Campylobacter jejuni under environmental conditions, Applied and Environmental Microbiology, 71, 2086-2094. [ pdf ]

ABSTRACT ( Copyright © American Society for Microbiology ) : Explorative approaches such as DNA microarray experiments are becoming increasingly important in microbial research. Despite these major technical advancements, approaches to study multifactor experiments are still lacking. We have addressed this problem using rotation testing and a novel MANOVA approach (50-50 MANOVA) to investigate interacting experimental factors in a complex experimental design. Furthermore, a new rotation testing based method was introduced to calculate false discovery rates for each response. This novel analytical concept was used to investigate global survival mechanisms in the environment of the major food borne pathogen C. jejuni. We simulated non-growth environmental conditions by investigating combinations of the factors temperature (5 and 25° C) and oxygen tension (anaerobic, microaerobic and aerobic). Data were generated using DNA microarrays for information about gene expression patterns coupled with FT-IR spectroscopy to study global macromolecular changes in the cell. Microarray analyses showed that most genes are either unchanged or down regulated compared to the reference (day 0) for the conditions tested, and that the 25° C anaerobic condition gave the most distinct expression pattern with the fewest genes expressed. The few up regulated genes are generally stress related and/or related to the cell envelope. We found, using FT-IR spectroscopy, that the amount of polysaccharides/oligosaccharides increases under the non-growth survival conditions. Potential mechanisms for survival could be to down regulate most functions to save energy, and produce polysaccharides/oligocaccharides for protection against harsh environments. Basic knowledge about the survival mechanisms is of fundamental importance in preventing transmission of this bacterium through the food chain.

KEY WORDS: Campylobacter jejuni, survival in the environment, microarray, FT-IR spectroscopy, 50-50 MANOVA, False Discovery Rate.