| News and Announcements » |
This tutorial explains the use of several different mapping file category comparison techniques that are available in compare_categories.py. A majority of the currently available comparison techniques are based on the ANOVA family of statistical methods, which is a statistical method wherein observed variance in a particular variable is partitioned into components attributable to different sources of variation. Most of the methods available in compare_categories.py determine whether the grouping of samples by a given category is statistically significant. Many of these methods are nonparametric, which means they use permutations of the data to determine the p-value, or statistical significance.
Note: Many of these methods have been evaluated on empirical and simulated data sets to verify their usefulness on microbial community data. It has been found that these methods seem to suffer from low specificity (i.e. they always seem to detect significance of grouping even when there doesn’t appear to be any). Even though the methods do not assume normality of the data due to their nonparametric nature, many of the methods assume that each group of samples has equal variance. Thus, it may be possible that the low specificity of the methods is due to violations of this equal variance assumption. Therefore, the methods may be detecting significant differences in the variability of sample groups instead of differences in center (i.e. mean). These methods need to be tested more thoroughly to verify these claims, so please be cautious with your use of these methods until then.
One common request when working with sample data is the ability to analyze the data sets with available statistical methods in the field. compare_categories.py provides this functionality. This tutorial will demonstrate how to work with this script in order to anaylze your data sets that have been produced through QIIME’s workflow processing.
You can obtain the files used in this tutorial here. The files are taken from a study (Fierer et al., 2010) where samples were collected from the fingertips of three individuals and the keys of their keyboards. Researchers found that an individual’s fingertip samples clustered very closely with his or her corresponding keyboard samples. These results have potential application in the field of forensic science. Throughout this tutorial, we will explore various ways of determining whether the clustering of fingertip and keyboard samples is statistically significant by analyzing the groups of samples.
Depending on which statistical method you run, the output file(s) generated by this script will vary. They will all be placed in the directory specified by the required -o option. Most of the output files will be tab-separated text files containing information about the test that was performed and its results. These can easily be viewed in a spreadsheet program such as Excel.
Adonis is a nonparametric statistical method that takes a QIIME distance matrix file such as a UniFrac distance matrix, a mapping file, and a category in the mapping file to determine sample grouping from. It computes a p-value to determine the statistical significance of the groups of samples.
Adonis creates a set by first identifying the relevant centroids of the data and then calculates the squared deviations from these points. After that significance tests are performed using F-tests based on sequential sums of squares from permutations of the raw data.
Suppose we wanted to determine whether the samples grouped by host individual (i.e. person) are statistically significant. Run the following command:
compare_categories.py --method adonis -i unweighted_unifrac_dm.txt -m map.txt -c HOST_SUBJECT_ID -o adonis_out -n 999
When broken down the command syntax means:
compare_categories.py - This is the script being run
--method adonis - This is the method to perform, in this case it's 'adonis'.
-i unweighted_unifrac_dm.txt - This is the unweighted UniFrac distance matrix being passed in for this statistical test.
-m map.txt - This is the metadata mapping file associated with this data set, being passed in.
-c HOST_SUBJECT_ID - This specifies the category in the metadata mapping file to analyze. In this case it's 'HOST_SUBJECT_ID'.
-o adonis_out - This specifies the output directory to place the results of this method in. In this case it's 'adonis_out'.
-n 999 - This specifies the number of permutations that will be used to calculate the p-value.
This command will create a new output directory named adonis_out, which will contain a single text file called adonis_results.txt. Open up adonis_results.txt to see the results of the test:
Call:
adonis(formula = as.dist(qiime.data$distmat) ~ qiime.data$map[[opts$category]], permutations = opts$num_permutations)
Terms added sequentially (first to last)
Df SumsOfSqs MeanSqs F.Model R2 Pr(>F)
qiime.data$map[[opts$category]] 10 7.1804 0.71804 5.1788 0.33243 0.001
Residuals 104 14.4196 0.13865 0.66757
Total 114 21.6000 1.00000
qiime.data$map[[opts$category]] ***
Residuals
Total
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
The p-value of 0.001 indicates that at an alpha of 0.05, the grouping of samples by individual is statistically significant. These results confirm the results obtained by the original researchers of this study.
ANOSIM is a method that tests whether two or more groups of samples are significantly different (similar to Adonis, above). You can specify a category in the metadata mapping file to separate samples into groups and then test whether there are significant differences between those groups. For example, you might test whether Control samples are significantly different from Fast samples. Since ANOSIM is non-parametric, significance is determined through permutations.
Note: ANOSIM only works with a categorical variable that is used to do the grouping. Mantel is recommended for continuous variables.
The R statistic that is calculated by ANOSIM is determined by the following formula:
R = (rb-rw)/(N(N-1)/4)
where rb is the mean rank of all distances between groups and rw is the mean rank of all distances within groups. An R value near +1 means that there is dissimilarity between the groups, while an R value near 0 indicates no significant dissimilarity between the groups (http://folk.uio.no/ohammer/past/multivar.html).
We will attempt to answer the same question posed in the Adonis section (above). Run the following command:
compare_categories.py --method anosim -i unweighted_unifrac_dm.txt -m map.txt -c HOST_SUBJECT_ID -o anosim_out -n 999
When broken down the command syntax means:
compare_categories.py - This is the script being run
--method anosim - This is the method to perform, in this case it's 'anosim'.
-i unweighted_unifrac_dm.txt - This is the unweighted UniFrac distance matrix being passed in for this statistical test.
-m map.txt - This is the metadata mapping file associated with this data set, being passed in.
-c HOST_SUBJECT_ID - This specifies the category in the metadata mapping file to analyze. In this case it's 'HOST_SUBJECT_ID'.
-o anosim_out - This specifies the output directory to place the results of this method in. In this case it's 'anosim_out'.
-n 999 - This specifies the number of permutations that will be used to calculate the p-value.
This command will create a new output directory named anosim_out, which will contain a single text file called anosim_results.txt. Open up anosim_results.txt to see the results of the test:
Method Name R-value P-value
ANOSIM 0.794026410205 0.001
The p-value of 0.001 indicates that at an alpha of 0.05, the grouping of samples by individual is statistically significant. These results confirm the results obtained by the original researchers of this study, as well as our results when we applied Adonis to this dataset (above).
Fierer, N. et al. Forensic identification using skin bacterial communities. Proc. Natl. Acad. Sci. USA. 107: 6477-6481 (2010).
