Permutation Hypothesis Test in R

Permutation tests, also known as exact tests or re-randomization tests, are non-parametric methods to test the null hypothesis by examining all possible rearrangements (permutations) of the observations. These tests are free from distributional assumptions and can be more robust than traditional parametric tests.

Basic Idea:

Combine the data from both groups into a single dataset.
Randomly divide the combined data into two groups, preserving the original group sizes.
Calculate the test statistic for these permuted groups.
Repeat steps 2 and 3 many times (e.g., 10,000 times) to build a null distribution of the test statistic.
Compare the observed test statistic to the null distribution to calculate the p-value.

Permutation Test in R:

Here's a basic example using a permutation test to compare the means of two independent groups:

# Generate some sample data
group1 <- c(23, 21, 22, 20, 25)
group2 <- c(30, 31, 29, 32, 28)

# Observed difference in means
obs_diff <- mean(group2) - mean(group1)

# Permutation test
n_permutations <- 10000
perm_diffs <- numeric(n_permutations)

for (i in 1:n_permutations) {
  # Combine and permute the data
  permuted_data <- sample(c(group1, group2))
  
  # Split the permuted data back into two groups
  perm_group1 <- permuted_data[1:length(group1)]
  perm_group2 <- permuted_data[(length(group1) + 1):length(permuted_data)]
  
  # Calculate the difference in means for the permuted groups
  perm_diffs[i] <- mean(perm_group2) - mean(perm_group1)
}

# Calculate the p-value
p_value <- mean(abs(perm_diffs) >= abs(obs_diff))

# Display results
cat("Observed difference:", obs_diff, "\n")
cat("P-value from permutation test:", p_value, "\n")

Libraries for Permutation Tests in R:

There are also R packages available that can perform permutation tests, which might be more efficient and convenient for more complex scenarios:

coin: This package provides a flexible framework for conditional inference procedures (i.e., permutation tests). Functions like independence_test can be used to perform various tests, including permutation-based t-tests.
perm: This package offers a variety of permutation tests.

To use one of these packages, you'll need to install it using install.packages() and then load it using the library() function.

Conclusion:

Permutation tests are a powerful non-parametric tool for hypothesis testing, especially when the assumptions of parametric tests are not met. They can be applied to a variety of scenarios, from simple two-sample tests to more complex experimental designs. In R, these tests can be implemented manually or with the assistance of specialized packages.

R code for permutation hypothesis testing:

Overview: Introduce the basic concept of permutation testing and provide a simple example.

Code:

# R code for permutation hypothesis testing
set.seed(123)
group1 <- c(5, 7, 8, 9, 6)
group2 <- c(10, 12, 14, 11, 13)

# Observed difference in means
observed_diff <- mean(group2) - mean(group1)

# Permutation test
permutation_test <- replicate(1000, {
  combined <- sample(c(group1, group2))
  perm_diff <- mean(combined[1:5]) - mean(combined[6:10])
  perm_diff
})

# Calculating p-value
p_value <- mean(abs(permutation_test) >= abs(observed_diff))

# Printing the results
print("Permutation Test Results:")
print(paste("Observed Difference:", observed_diff))
print(paste("Permutation-based p-value:", p_value))

Using permute package for hypothesis testing in R:

Overview: Introduce the permute package for conducting permutation tests in R.

Code:

# Using permute package for hypothesis testing
install.packages("permute")
library(permute)

# Example: Permutation test for difference in means
group1 <- c(5, 7, 8, 9, 6)
group2 <- c(10, 12, 14, 11, 13)

# Permutation test using permute package
result <- perm_test(group1 ~ group2, method = "exact")

# Printing the results
print("Permutation Test Results using permute:")
print(result)

R permutation test example for ANOVA:

Overview: Demonstrate a permutation test example for analysis of variance (ANOVA).

Code:

# R permutation test example for ANOVA
install.packages("permute")
library(permute)

# Example: Permutation test for ANOVA
data <- data.frame(
  value = c(10, 12, 14, 11, 13, 5, 7, 8, 9, 6),
  group = rep(c("A", "B"), each = 5)
)

# Permutation test for ANOVA
result <- aovp(value ~ group, data = data, perms = 1000)

# Printing the results
print("Permutation Test Results for ANOVA:")
print(result)

Permutation test for correlation in R:

Overview: Illustrate how to perform a permutation test for correlation.

Code:

# Permutation test for correlation in R
set.seed(123)
x <- c(1, 2, 3, 4, 5)
y <- c(5, 4, 3, 2, 1)

# Observed correlation
observed_corr <- cor(x, y)

# Permutation test for correlation
permutation_test <- replicate(1000, {
  perm_y <- sample(y)
  perm_corr <- cor(x, perm_y)
  perm_corr
})

# Calculating p-value
p_value <- mean(abs(permutation_test) >= abs(observed_corr))

# Printing the results
print("Permutation Test Results for Correlation:")
print(paste("Observed Correlation:", observed_corr))
print(paste("Permutation-based p-value:", p_value))

Permutation test for independent samples in R:

Overview: Demonstrate a permutation test for independent samples.

Code:

# Permutation test for independent samples in R
set.seed(123)
group1 <- c(10, 12, 14, 11, 13)
group2 <- c(5, 7, 8, 9, 6)

# Observed difference in means
observed_diff <- mean(group1) - mean(group2)

# Permutation test for independent samples
permutation_test <- replicate(1000, {
  combined <- sample(c(group1, group2))
  perm_diff <- mean(combined[1:5]) - mean(combined[6:10])
  perm_diff
})

# Calculating p-value
p_value <- mean(abs(permutation_test) >= abs(observed_diff))

# Printing the results
print("Permutation Test Results for Independent Samples:")
print(paste("Observed Difference:", observed_diff))
print(paste("Permutation-based p-value:", p_value))

R permTest function for permutation testing:

Overview: Introduce the permTest function from the lmPerm package for permutation testing.

Code:

# R permTest function for permutation testing
install.packages("lmPerm")
library(lmPerm)

# Example: permTest for difference in means
group1 <- c(5, 7, 8, 9, 6)
group2 <- c(10, 12, 14, 11, 13)

# Permutation test using permTest
result <- permTest(group1 ~ group2, data = data.frame(value = c(group1, group2)))

# Printing the results
print("Permutation Test Results using permTest:")
print(result)