| News and Announcements » |
QIIME consists of native code and additionally wraps many external applications. This gives the user flexibility to easily build their own analysis pipelines, making use of popular microbial community analysis tools. QIIME handles the processing of input and output of these applications, so the user can spend time analyzing their data rather than parsing, writing, and converting file formats.
As a consequence of this ‘pipeline’ architecture, depending on the features of QIIME that you plan to use, you may or may not need all of QIIME’s dependencies. Getting all of these applications working correctly can be difficult, which is why we distribute the QIIME virtual box. If you are a beginner user, or just testing QIIME to see if it will meet your needs, you may want to begin with the virtual box which will greatly ease installation.
Alternatively, if you are interested in installing QIIME on a Linux system, we recommend using the qiime-deploy tool, particularly if you are using Ubuntu. This tool allows for the easy (and native) installation of QIIME and its various dependencies.
The following programs and datasets were used to generate the QIIME tutorial. QIIME has been tested with the same versions listed below. Other versions are not guaranteed to work. Download links are provided for convenience, but these are subject to change. We’ll do our best to keep these up-to-date.
You should follow instructions provided by the package developers to install the dependencies.
To install most of following dependencies you need to have a build environment on your machine. On OS X, this involves installing the developer tools. On Debian-based Linux (e.g., Ubuntu), this involves installing the build-essential package:
sudo apt-get install build-essential
The following are required by QIIME:
PyNAST alignment, tree-building, taxonomy assignment, OTU picking, and other data generation steps (required in default pipeline):
Alignment, tree-building, taxonomy assignment, OTU picking, and other data generation steps (required for alternative pipelines):
Processing sff files:
Denoising 454 data:
Visualization and plotting steps:
Supervised learning (supervised_learning.py) and compare_categories.py:
Assigning taxonomy using BLAST or picking OTUs against Greengenes filtered at 97% identity:
If you plan to build the QIIME documentation locally:
If you plan to use remote mapping files (stored as Google Spreadsheets) with QIIME (see the tutorial here):
If you plan to use SourceTracker with QIIME:
We have attempted to provide accurate licensing information for the above dependencies for the convenience of our users. This information is by no means definitive and may contain errors. Any questions about licenses or the legality of specific uses of these software packages should be directed to the authors of the software. Do not rely solely on the license information presented above!
For simplicity throughout this document, we assume that you have downloaded QIIME in $HOME/. You should consider all occurrences of $HOME/ in the remainder of this document as references to the directory which contains the QIIME directory which you’ll have after downloading and unpacking QIIME.
First, change to the directory where you would like to download QIIME:
cd $HOME
Currently the most stable version of QIIME is our 1.7.0 release, which you can download from here.
To get the latest development version of QIIME, you should check it out of our git repository, which is hosted on GitHub. While this code is subject to changes in interface and hasn’t been as extensively tested as the release version, it will provide access to the latest and greatest QIIME features. The official web documentation is likely to be out-of-date with respect to the development software. You should instead refer to the documentation in Qiime/doc. Check out the latest version of QIIME using git with the command:
git clone git://github.com/qiime/qiime.git Qiime
If you are using the latest development version of QIIME, you should periodically update your checkout by running the following command (from within your checkout):
git pull
After downloading the QIIME release tar file you’ll need to unpack the code. For simplicity in this document, we will assume that you have downloaded QIIME to the directory $HOME/.
Unpack the release .tar.gz file with the commands:
cd $HOME
tar -xvzf qiime-1.7.0.tar.gz
ln -s $HOME/qiime-1.7.0 $HOME/Qiime
If you have downloaded the development version from GitHub, QIIME is already unpacked.
QIIME consists of library code (in Qiime/qiime), test code (in Qiime/tests), documentation (in Qiime/doc), and scripts (in Qiime/scripts). Installing QIIME consists of running the tests (optional, but highly recommend), installing the library code in a place where python knows where to find it, and installing the scripts in a place where the shell looks for executable files.
Using Qiime/setup.py (and thereby python’s distutils package) is the recommended way of installing the Qiime library code and scripts. You can optionally specify where the library code and scripts should be installed – depending on your setup, you may want to do this. By default, the QIIME library code will be placed under python’s site-packages, and the QIIME scripts will be place in /usr/local/bin/. You may need to run setup.py using sudo if you do not have permission to place files in the default locations.
First, ensure that you are in the top-level QIIME directory:
cd $HOME/Qiime
By default the QIIME scripts will be installed in /usr/local/bin. As there are a lot of QIIME scripts, we highly recommend customizing the script directory to keep your system organized. This can be customized with the --install_scripts option. You also can specify an alternate directory for the library files with --install-purelib. An example command is:
python setup.py install --install-scripts=$HOME/bin/ --install-purelib=$HOME/lib/
For a complete discussion of customizations related to the setup.py script, see this page.
If you used default values for --install-scripts and --install-purelib (by not specifying them), your installation should be complete. If you specified an alternate value for --install-scripts, you’ll need to ensure that the shell knows where to look for the scripts. If you are using the bash shell and the locations specified in the examples above, you can do this with the following command:
echo "export PATH=$HOME/bin/:$PATH" >> $HOME/.bashrc
If you specified an alternate value for --install-purelib, you’ll need to be sure that python knows where to look for Qiime. If you are using the bash shell and the locations specified in the examples above, you can do this with the following command:
echo "export PYTHONPATH=$HOME/lib/:$PYTHONPATH" >> $HOME/.bashrc
The source your .bashrc:
source $HOME/.bashrc
Finally, you’ll need to create and edit a custom qiime_config file to tell QIIME where to look for the QIIME scripts. Create a custom qiime_config file by copying the default qiime_config packaged with QIIME:
cp $HOME/Qiime/qiime/support_files/qiime_config $HOME/.qiime_config
Open the new file, $HOME/.qiime_config, in a text editor such as TextEdit (on Mac), gedit (on Linux), vim, or emacs (but not Microsoft Word, which is a word processor, not a text editor!). Find the line beginning qiime_scripts_dir and add a tab, followed by the QIIME scripts directory. If you’ve used the default value (i.e., you didn’t specify --install-scripts) the value you add will be /usr/local/bin/. Otherwise, specify the value that you provided for --install-scripts. In the example above, this would look like:
qiime_scripts_dir $HOME/bin/
Note that the delimiter between the key and the value here is a tab, not a space! For additional information on the qiime_config file, see this document.
Next you should run the test suite. Execute the following commands:
cd $HOME/Qiime/tests/
python all_tests.py
You will see test output on the terminal indicating test successes and failures. Some failures are OK. The all_tests.py command will complete with a summary of test failures. Some tests may fail due to missing external applications – these will be noted separately from other test failures. If these are related to features of QIIME that you are not using, this is acceptable. Otherwise, you’ll need to ensure that you have the external applications installed correctly (and the correct versions), and re-run the tests.
If QIIME is installed correctly, you should be able to run the QIIME scripts. Try the following:
cd
align_seqs.py -h
This should give you help text describing the interface to the align_seqs.py script. (Note that if you do not have a $HOME/.bashrc you may get an error at the source step. If you did not specify alternate values for --install-purelib or --install-scripts this shouldn’t be a problem.)
External applications used by QIIME need to be visible to the shell by existing in executable search path (i.e., listed in the $PATH environment variable). For example, if you plan to use cd-hit, and have the cd-hit executables installed in $HOME/bin you can add this directory to your system path with the commands:
echo "export PATH=$HOME/bin/:$PATH" >> $HOME/.bashrc
source $HOME/.bashrc
Qiime, PyCogent, and NumPy must be visible to python for all features of QIIME. matplotlib must be visible to python if you plan to use graphics features of QIIME; PyNAST must be visible to python if you plan to use PyNAST for multiple sequence alignment; and Denoiser must be visible to python if you plan to denoise 454 data. With the exception of Denoiser, all of these packages come with setup.py scripts. If you have used these, you should not need to modify your PYTHONPATH to make the library code visible. If you haven’t used the respective setup.py scripts, or if you specified an alternate value for --install-purelib, you may need to add the locations of these libraries to your PYTHONPATH environment variable.
For example, if you’ve installed PyNAST in $HOME/PyNAST you can add this to your PYTHONPATH with the commands:
echo "export PYTHONPATH=$HOME/PyNAST/:$PYTHONPATH" >> $HOME/.bashrc
source $HOME/.bashrc
If you plan to use the RDP classifier for taxonomy assignment you must define an RDP_JAR_PATH environment variable. If you downloaded and unzipped the RDP classifier folder in $HOME/app/, you can do this with the following commands:
echo "export RDP_JAR_PATH=$HOME/app/rdp_classifier_2.2/rdp_classifier-2.2.jar" >> $HOME/.bashrc
source $HOME/.bashrc
Note that you will need the contents inside rdp_classifier_2.2 for the program to function properly.
The uclust binary must be called uclust, which differs from the names of the posted binaries, but is the name of the binary if you build from source. If you’ve installed the binary uclust1.2.21q_i86linux64 as $HOME/bin/uclust1.2.21q_i86linux64, we recommend creating a symbolic link to this file:
ln -s $HOME/bin/uclust1.2.21q_i86linux64 $HOME/bin/uclust
The usearch binary must be called usearch, which differs from the names of the posted binaries, but is the name of the binary if you build from source. If you’ve installed the binary usearch5.2.236_i86linux32 as $HOME/bin/usearch5.2.236_i86linux32, we recommend creating a symbolic link to this file:
ln -s $HOME/bin/usearch5.2.236_i86linux32 $HOME/bin/usearch
ChimeraSlayer can only be run from the directory where it was unpacked and built as it depends on several of its dependencies being in specific places relative to the executable (ChimeraSlayer/ChimeraSlayer.pl). Carefully follow the ChimeraSlayer install instructions. Then add the directory containing ChimeraSlayer.pl to your $PATH environment variable. If your ChimeraSlayer folder is in $HOME/app/ you can set the $PATH environment variable as follows:
echo "export PATH=$HOME/app/ChimeraSlayer:$PATH" >> $HOME/.bashrc
source $HOME/.bashrc
If you’re having trouble getting ChimeraSlayer to work via QIIME, you should first check to see if you can run it directly from a directory other than its install directory. For example, try running ChimeraSlayer.pl from your home directory.
Once you have configured Qiime, you can test your ChimeraSlayer install by running:
print_qiime_config.py -t
This includes a check for obvious problems with your ChimeraSlayer install, and should help you determine if you have it installed correctly.
To install R visit http://www.r-project.org/ and follow the install instructions. Once R is installed, run R and excecute the following commands:
install.packages('randomForest')
install.packages('optparse')
install.packages('vegan')
install.packages('ape')
install.packages('MASS')
install.packages('gtools')
install.packages('klaR')
install.packages('RColorBrewer')
q()
AmpliconNoise requires that several environment variables are set. After you’ve installed AmpliconNoise, you can set these with the following commands (assuming your AmpliconNoise install directory is $HOME/AmpliconNoiseV1.27/):
echo "export PATH=$HOME/AmpliconNoiseV1.27/Scripts:$HOME/AmpliconNoiseV1.27/bin:$PATH" >> $HOME/.bashrc
echo "export PYRO_LOOKUP_FILE=$HOME/AmpliconNoiseV1.27/Data/LookUp_E123.dat" >> $HOME/.bashrc
echo "export SEQ_LOOKUP_FILE=$HOME/AmpliconNoiseV1.27/Data/Tran.dat" >> $HOME/.bashrc
If you do not install QIIME using setup.py and you plan to use the QIIME Denoiser, you’ll need to compile the FlowgramAlignment program. To do this you’ll need to have ghc installed. Then from the Qiime/qiime/support_files/denoiser/FlowgramAlignment/ directory, run the following command:
make ; make install
If you are using the development version of QIIME, you may want to build the documentation locally for access to the latest version. You can change to the Qiime/doc directory and run:
make html
We try to update the documentation as we update the code, but development version users may notice some discrepancies. After building the documentation, you can view it in a web browser by opening the file Qiime/doc/_build/html/index.html. You may want to bookmark that page for easy access.
