Diversity analysis

Introduction¶

Microbes have a profound impact on human life, influencing the environment, health, disease, and ecosystems. Understanding the diversity and composition of these ecological units is fundamental to answering broader ecological questions Cassol et al., 2025.

Depending on the scale of the ecological units, diversity is measured at three levels Andermann et al., 2022.

• Alpha diversity: Describes the richness and evenness within a single community.

• Beta diversity: Analyzes the differentiation between distinct communities.

• Gamma diversity: Studies diversity at a larger landscape or geographic scale.

In the QIIME 2 tutorial, the focus is on alpha and beta diversity. The subsequent steps involve constructing a phylogenetic tree and generating core metrics matrices to facilitate these analyses.

qiime phylogeny align-to-tree-mafft-fasttree \
  --i-sequences rep-seqs.qza \
  --o-alignment aligned-rep-seqs.qza \
  --o-masked-alignment masked-aligned-rep-seqs.qza \
  --o-tree unrooted-tree.qza \
  --o-rooted-tree rooted-tree.qza

qiime diversity core-metrics-phylogenetic \
  --i-phylogeny rooted-tree.qza \
  --i-table table.qza \
  --p-sampling-depth 1103 \
  --m-metadata-file sample-metadata.tsv \
  --output-dir diversity-core-metrics-phylogenetic

Workflow¶

The output from the DADA2 algorithm, “ASV sequences”, is utilized to compute a rooted phylogenetic tree at the Phylogenetics step. Meanwhile the Sequence table (BIOM count table of ASVs $\times$ samples, also generated by DADA2) is stardardized via Rarefaction. This rarefied table is then used to compute core matrices (non-phylogenetic) and integrated with the rooted tree to calculate phylogenetic matrices.

Phylogenetics¶

The ASV sequences generated by the DADA2 algorithm are aligned using MAFFT v7.526 Katoh & Standley, 2013. By default, the progressive FFT-NS-2 method is employed. In this workflow, a distance matrix is calculated based on shared 6-tuples between pairs of sequences, and a phylogenetic guide tree is constructed using the UPGMA method prior to performing the Multiple Sequence Alignment (MSA).

Following the MSA step, positions are masked to retain if satisfying the gap frequency is $\le 1.0$ and the frequency of the most prevalent character (not include gaps) is $\ge 0.4$ (default thresholds).

Using the MSA, FastTree v2.2.0 Price et al., 2010 generates an initial unrooted tree via a heuristic Neighbor-Joining (NJ) method. Nearest-Neighbor Interchanges (NNIs) are then employed to explore alternative topologies and branch lengths. These candidate trees serve as the basis for Maximum Likelihood (ML) computations, which statistically estimate the most likely topology and branch lengths to produce the final phylogenetic tree.

Finally, the workflow utilizes the Midpoint Rooting (MPR) method Farris, 1972, implemented in the scikit bio package, to restructure the tree and determine the root. The algorithmic implementation can be referenced in MPR.py.

Rarefaction¶

Before computing core matrices, the abundance table is randomly subsampled (rarefied) to ensure the total number of sequences across all ASVs in each sample is exactly 1103 (--p-sampling-depth). Samples are discarded if their total sequence count is below this threshold.

Note: The red dash line is the threshold 1103.

As the rarefaction threshold increases, the number of retained samples decreases, while the total sequence count increases due to the greater contribution from highly abundant samples.

Setting the threshold too low results in insufficient data for a robust analysis. Conversely, setting it too high leads to the elimination of too many samples, reducing statistical power. The goal is to select a threshold that retains as many samples as possible while capturing a “large enough” number of sequences to represent the community. This critical step ensures evenness and comparability across all samples in the study.

Computation of core matrices¶

The refied feature table is used to calculate core matrices:

• Alpha matrices: Measure diversity within individual samples (observed feature, shannon entropy, pielou evenness)

• Beta matrices: Compute dissimilarity matrices, containing distance between pairs of samples (jaccard, bray curtis)

Note:

• $p_i$: The relative abundance (probability) of $\text{ASV}_i$ in a sample, where $p_i = \frac{\text{\#ASV}_i}{\text{\#total sequences}}$. (#total_sequences is 1103 in our case)

• $n$: The total number of observed ASVs in the sample.

At the end, the beta diversity (distance) matrices, jaccard and bray curtis, are utilized to calculate principal coordinates through Principal Coordinates Analysis (PCoA) Gower, 1966. This dimensionality reduction technique projects the multi-dimensional distance data into a lower-dimensional space, typically visualized as 2D or 3D plots to reveal ecological patterns.

Computation of phylogenetic matrices¶

While alpha diversity matrices quantify the richness and abundance of a community, they fail to account for the evolutionary relationships among ASVs.

For instance, consider a scenario where Sample A contains more unique ASVs than Sample B. If all ASVs in Sample A share extremely high sequence similarity, it is difficult to argue that Sample A is truly more diverse than Sample B in a biological sense.

The problem are addressed by phylogenetic matrices, includes:

• Alpha matrices: faithpd

• Beta matrices: unweighted_unifrac and weighted_unifrac

Similar to the non-phylogenetic core matrices, the distance matrices generated via unweighted and weighted UniFrac are used as inputs for PCoA.

Summary¶

Identifying the root of the phylogenetic tree is a mechanical necessity for diversity analysis. Faith’s PD requires a directed hierarchy to accumulate branch lengths from successors up to the common ancestor (see the algorithmic code). Furthermore, a rooted tree provides the fixed reference points needed to calculate meaningful shared and unique evolutionary branch lengths between samples in UniFrac matrices.

During this deep dive, I also noticed that MAFFT suspiciously “picks out” specific conserved nucleotide positions in the ASV sequences after MSA (see my MAFFT issue).

Ultimately, while non-phylogenetic (core) matrices treat every ASV as an independent entity, phylogenetic matrices leverage the shared evolutionary history encoded within the sequences to provide a deeper biological context.