Scope

The Protein Clusters dataset consists of organized groups (clusters) of proteins encoded by complete and draft genomes from the NCBI Reference Sequence (RefSeq) collection of microorganisms: prokaryotes, viruses, fungi, protozoans; it also includes protein clusters from RefSeq genomes of plants, chloroplasts, and mitochondria. Clusters for each group are created and curated separately and given a different accession prefix. The primary goal of Protein Clusters is to provide the support to functional annotation of RefSeq genomes. Functional annotation of novel proteins is based on the assumption that proteins with high levels of sequence similarity are likely to share the same function. This oversimplified model of a linear evolution where similar proteins evolve from a single ancestor is further complicated by the events of gene duplication. Clusters of related (homologous) proteins include both orthologs and paralogs. Orthologs are genes in different organisms (species) that evolved from a common ancestral gene by speciation; paralogs are genes related by duplication within a genome. The definition was first introduced by Fitch in 1970 (7). The analysis of protein families from various organisms has shown that this definition does not embrace all the complexity of relationships of genes from different organisms. For more details see Koonin et al. 2005 (16). The NCBI Protein Clusters dataset contains automatically generated clusters that do not distinguish orthologs and paralogs. During manual evaluation some clusters containing paralogs can be split by curators, especially if the paralogs are known to have different functions.

History

The first complete bacterial genome of Haemophilus influenzae Rd KW20 sequenced and released in 1995 opened a new era in genome analysis (8). In the following year four more genomes were completed producing an unprecedented variety of protein sequences from all three major kingdoms (Archaea, Eubacteria, and Eukaryota). Comparative analysis of homologous genes has been used in evolutionary studies and functional classification since the first sequence became available, but for the first time the whole proteome of several organisms became available for comparison. New genome-scale methods were needed to provide an understanding of the true orthologous relationships of protein sequences. The protein database of Clusters of Orthologous Groups (COGs), a pioneering work of NCBI scientists, was the first attempt at creating a phylogenetic classification of the complete complement of proteins encoded by complete genomes (23). The COG approach is based on the simple notion that any group of at least three proteins from distant genomes that are more similar to each other than they are to any other proteins from the same genome are most likely to form an orthologous set. The COG project has proved to be an excellent approach for understanding microbial evolution and the diversity of protein functions encoded by their genomes. However, the major difficulty of any genomic data resource in the modern era of rapid genomic sequencing is keeping the genomic data and the annotations up-to-date.

The RefSeq project, which contains non-redundant sets of curated transcript, gene, and protein information in eukaryotes, and gene and protein information in prokaryotes, has been a very successful way to maintain and update annotated data. Given the increasing number of prokaryotic genomes being deposited, it became apparent that annotating protein families as a group was a convenient and efficient way to functionally annotate these data. The Protein Clusters database was constructed with two goals in mind: first, to routinely update RefSeq genomes with curated gene and protein information from such clusters; and second, to provide a central aggregation source for information collected from a wide variety of sources that would be useful for scientists studying protein-level or genomic-level molecular functions. In addition, curators routinely parse the scientific literature for reports of experimentally verified functions as the basis for existing or potential connections to genes/proteins, and such connections are added as annotations on each cluster. The first release of NCBI Protein Clusters in 2007 contained ~1 million proteins encoded by complete chromosomes and plasmids from three major groups: prokaryotes, bacteriophages, and the organelles (15). Since then the scope has been expanded to other taxonomic groups and proteins from draft genomes.

As of April 2013 the dataset represents more than 20 million proteins.

Data Model

Clustering is a well-known method in statistics and computer science. For a given set of entities, clusters are defined as subsets that are homogeneous and well separated. The cluster analysis should start from a definition of homogeneity and separation. Most clustering methods rely upon similarities (or distance) between entities. Protein clusters are aimed to be groups of homologous proteins. The similarity between two protein sequences is measured by maximum alignment between the sequences calculated by BLAST. There are multiple ways of defining various types of clusters that are based on criteria used to express separation or homogeneity of a cluster and separation from other clusters. NCBI Protein Clusters uses two methods for clustering, both resulting in building cliques, one based on partitioning and the other based on hierarchical aggregation.

Once clustered, each protein cluster is assigned a cluster ID and accession (letter prefix followed by digits) that is stable from version to version as long as the majority of its proteins don’t change. A protein cluster also includes certain attributes aggregated from proteins: “Gene names” (locus), “COG functional categories,” “EC numbers,” and “Conserved Domains.” An attribute “Conserved in” defines the common taxonomical name of genomes included in the cluster. The Protein Clusters database also includes a set of “Related Clusters”. Besides these attributes, each protein record in the database has “Organism name,” “Protein name,” “Protein accession,” “Locus tag,” “Length,” and UniProtKB / SwissProt accession. These attributes are easily searchable within a cluster and also through the whole database.

Example of a bacterial cluster PCLA_5029913 glycoside hydrolase

Clustering Methods

Hierarchical Aggregation in Cliques

A new approach implemented for prokaryotic genomes is based on hierarchical clustering. While a hierarchical structure is conventionally represented by a dendrogram and clusters are selected as a sub-tree corresponding to a certain threshold (14, 17, 18), the hierarchical structure goes beyond simple clustering (1, 3). First, all the proteins are organized in global clusters, then links between clusters are calculated reflecting the similarity between the clusters based on several criteria.

Protein Clustering Procedure

The similarity of proteins is determined from the aggregated BLAST hits obtained by blastp with e-value ${10}^{-3}$. Two proteins are considered connected if there is an aggregated BLAST hit between them satisfying criteria on hit length and score. More specifically, we require the aggregated hit lengths on each protein, $l_{ij}^{\left(1\right)}$ and $l_{ij}^{\left(2\right)}$, satisfy the inequalities $l_{ij}^{\left(1\right)} \ge \epsilon {\bullet l}_i$ and $l_{ij}^{\left(1\right)} \ge \epsilon {\bullet l}_j$, where $l_i$ and $l_j$ are lengths of proteins, and $0.5< \epsilon <1$, and the aggregated BLAST score $S_{ij}$satisfy the inequality $S_{ij}\ge \gamma \bullet \mathrm{m}\mathrm{a}\mathrm{x}(S_{ii}, S_{jj})$, where $S_{ii}$ and $S_{jj}$are self-scores.

The modified BLAST distance is defined as

$d_{ij}^{\mathrm{\alpha }}=1-\frac{S_{ij}}{\max \left({\chi }_{ij}^{\left(1\right)}{\bullet S}_{ii}, { {\chi }_{ij}^{\left(2\right)}\bullet S}_{jj}, { S}_{ij} \right)}$,

where the score modifications are ${\chi }_{ij}^{\left(1\right)}=\mathrm{m}\mathrm{a}\mathrm{x}(\frac{l_{ij}^{\left(1\right)}}{l_i}$, 1-$\alpha )$ and ${\chi }_{ij}^{\left(2\right)}=\mathrm{m}\mathrm{a}\mathrm{x}(\frac{l_{ij}^{\left(2\right)}}{l_i}$, 1-$\alpha )$, and $0<\alpha \ll 1.$ Using $\alpha >0$ allows some flexibility at the end of the sequences (the distance is reduced to $d_{ij}^0=1- \frac{S_{ij}}{\mathrm{m}\mathrm{a}\mathrm{x}(S_{ii}, S_{jj})}$ when $\alpha =0$). Clusters are aggregated in a hierarchical manner using the complete linkage distance, with an additional requirement that the minimum distance between clusters $d^{\mathrm{m}\mathrm{i}\mathrm{n}}(\mathrm{\Lambda },\mathrm{ }$Ω) should not exceed threshold $\delta ,$ where $0<\delta <1-\gamma ,$for clusters $\mathrm{\Lambda }$ and Ω to be merged. Note that we calculate and use both $d^{\mathrm{m}\mathrm{i}\mathrm{n}}(\mathrm{\Lambda },\mathrm{ }$Ω) and $d^{\mathrm{m}\mathrm{a}\mathrm{x}}(\mathrm{\Lambda },\mathrm{ }$Ω) in our clustering procedure (see Figure 1). Because of the sparse nature of connections and applied thresholds, we build a family of trees that we consider clusters. Currently, we use the values $\epsilon =0.7$, $\gamma =0.2,$ $\alpha =0.1$, and $\delta =0.5$.

Figure 1.

Minimal and maximum distance between clusters

Establishing Links between Related Clusters

Each protein within a cluster should be similar to all other proteins in the same cluster, satisfying coverage and similarity criteria. Still, a pair of proteins in different clusters could be similar. Such clusters are designated as related clusters (1, 3, 12, 24). Links between related clusters are stored in link indexes, which are used to show neighborhoods of clusters in Entrez search.

Organization of computations. First, we eliminate redundancy and near-redundancy in the protein dataset (2, 12). Representative proteins from groups of redundant and nearly-redundant proteins are selected by the program USEARCH (5).

In order to perform clustering in parallel, the dataset is partitioned in disjoint sets (Figure 2) using a parallel implementation based on a disjoint-set forest with union-by-rank heuristics (4, 22), and then clustering is performed concurrently in partitions. When looking for disjoint sets, we only consider connections with $d_{ij}^{\mathrm{\alpha }}\le \delta .$

Figure 2.

Disjoint sets.

After the clustering is performed, link indexes are also calculated in parallel from the aggregated BLAST hit and protein assignment to clusters.

Dataflow

Input data are proteins from complete and draft (WGS) genomes that pass certain quality filters.

Proteins marked as incomplete in metadata (“incomplete,” “no start,” “no end,” “fragment,” etc.) are removed and only proteins that are presumed complete are analyzed. Bacterial genome clustering has a different dataflow compared to other genomes as indicated in Figure 3.

Figure 3.

Dataflow for prokaryotic and eukaryotic genomes

Manual Curation

One of the most important aspects of the curation process of Protein Clusters is the assignment of function that is obtained from the literature. Curated functional annotation can be propagated to all proteins within the cluster. That process improves the functional annotation of RefSeq genomes and unifies and standardizes the naming rules across various organisms and different annotation pipelines. In addition to providing functional annotation that is required for each cluster, other data are also added, such as the gene name, a detailed description about the protein, the E.C. number, and relevant publications.

Cluster Display

A protein cluster is represented by a list of protein identifiers (accessions) and the genomes that code for the proteins. Each cluster has a stable unique identifier and a functional cluster name. The cluster name is automatically calculated and followed by manual review. Each cluster provides statistics to indicate the number of proteins within that cluster and other important features including the Protein Table.

Access

Protein Clusters are presented in NCBI’s Entrez system (http://www.ncbi.nlm.nih.gov/sites/entrez?db=proteinclusters)

The first public release of the Protein Clusters in NCBI’s Entrez interface was in April 2007, and initially consisted of only prokaryotic clusters (15). The Entrez system provides a mechanism for the search, retrieval, and linkage between Protein Clusters and other NCBI databases, as well as external resources. Clusters can be searched by general text terms, and also by specific protein or gene names.

Limits and Advanced search allow clusters to be browsed by function and filtered by size and organism group. A table browser allows users to sort by the content of each column by clicking on the column header.

Protein clusters are available for download from the FTP directory (ftp://ftp.ncbi.nih.gov/genomes/Bacteria/CLUSTERS/) by date and by major taxonomic groups.

Related Tools

References

1.

Ahn YY, Bagrow J, Lehmann S. Link communities reveal multiscale complexity in networks. Nature. 2010 Aug 5;466(5):761–765. [PubMed: 20562860]

2.

Cameron M, Bernstein Y, Williams HE. Clustered Sequence Representation for Fast Homology Search. J Comput Biol. 2007 June;14(5):594–614. [PubMed: 17683263]

3.

Clauset A, Moore C, Newman ME. Hierarchical structure and the prediction of missing links in networks. Nature. 2008 May 1;453:98–100. [PubMed: 18451861]

4.

Cormen TH, Leiserson CE, Rivest RL, Stein C. Introduction to algorithms. 3rd Edition, The MIT Press; 2009.

5.

Edgar RC. Search and clustering orders of magnitude faster than BLAST. Bioinformatics. 2010 Oct 1;26(19):2460–2461. [PubMed: 20709691]

6.

Fairman-Williams ME, Jankowsky E. Unwinding initiation by the viral RNA helicase NPH-II. J Mol Biol. 2012 Feb 3;415(5):819–832. [PMC free article: PMC3267895] [PubMed: 22155080]

7.

Fitch WM. Distinguishing homologous from analogous proteins. Syst Zool. 1970 Jun;19:99–106. [PubMed: 5449325]

8.

Fleischmann RD, Adams MD, White O, Clayton RA, Kirkness EF, Kerlavage AR, Bult CJ, Tomb JF, Dougherty BA, Merrick JM, et al. Whole-genome random sequencing and assembly of Haemophilus influenzae Rd. Science. 1995 Jul 28;269(5223):496–512. [PubMed: 7542800]

9.

Gargantini PR, Serradell MC, Torri A, Lujan HD. Putative SF2 helicases of the early-branching eukaryote Giardia lamblia are involved in antigenic variation and parasite differentiation into cysts. BMC Microbiol. 2012 Nov 28;12:284. [PMC free article: PMC3566956] [PubMed: 23190735]

10.

Goh LL, Sim TS. Homology modeling and mutagenesis analyses of Plasmodium falciparum falcipain 2A: implications for rational drug design. Biochem Biophys Res Commun. 2004 Oct 15;323(2):565–572. [PubMed: 15369788]

11.

Hawkins LK, Wold WS. A. 12,500 MW protein is coded by region E3 of adenovirus. Virology. 1992 Jun;188(2):486–494. [PubMed: 1585632]

12.

Holm L, Sander C. Removing near-neighbor redundancy from large protein sequence collections. Bioinformatics. 1998 Jun;14(5):423–429. [PubMed: 9682055]

13.

Hughes AL, Irausquin S, Friedman R. The evolutionary biology of poxviruses. Infect Genet Evol. 2010 Jan;10(1):50–59. [PMC free article: PMC2818276] [PubMed: 19833230]

14.

Kaplan N, Sasson O, Inbar U, Friedlich M, Fromer M, Fleischer H, Portugaly E, Linial N, Linial M. ProtoNet 4.0: a hierarchical classification of one million protein sequences. Nucleic Acids Res. 2005 Jan 1;33(Database issue):D216–D218. [PMC free article: PMC539961] [PubMed: 15608180]

15.

Klimke W, Agarwala R, Badretdin A, Chetvernin S, Ciufo S, Fedorov B, Kiryutin B, O'Neill K, Resch W, Resenchuk S, Schafer S, Tolstoy I, Tatusova T. The National Center for Biotechnology Information's Protein Clusters Database. Nucleic Acids Res. 2009 Jan;37(Database issue):D216–23. [PMC free article: PMC2686591] [PubMed: 18940865]

16.

Koonin EV. Orthologs, paralogs, and evolutionary genomics. Annu Rev Genet. 2005;39:309–38. [PubMed: 16285863]

17.

Krause A, Stoye J, Vingron M. Large scale hierarchical clustering of protein sequences. BMC Bioinformatics. 2005 Jan 22;6:6–15. [PMC free article: PMC547898] [PubMed: 15663796]

18.

Loewenstein Y, Portugaly E, Fromer M, Linial M. Efficient algorithms for accurate hierarchical clustering of huge datasets: tackling the entire protein space. Bioinformatics. 2008 Jul 1;24(13):i41–49. [PMC free article: PMC2718652] [PubMed: 18586742]

19.

Ranji A, Boris-Lawrie K. RNA helicases: emerging roles in viral replication and the host innate response. RNA Biol. 2010 Nov-Dec;7(6):775–87. [PMC free article: PMC3073335] [PubMed: 21173576]

20.

Shenai BR, Sijwali PS, Singh A, Rosenthal PJ. Characterization of native and recombinant falcipain-2, a principal trophozoite cysteine protease and essential hemoglobinase of Plasmodium falciparum. J Biol Chem. 2000 Sep 15;275(37):29000–29010. [PubMed: 10887194]

21.

Takano M, Inagaki N, Xie X, Yuzurihara N, Hihara F, Ishizuka T, Yano M, Nishimura M, Miyao A, Hirochika H, Shinomura T. Distinct and cooperative functions of phytochromes A, B, and C in the control of deetiolation and flowering in rice. Plant Cell. 2005 Dec;17(12):3311–3325. [PMC free article: PMC1315371] [PubMed: 16278346]

22.

Tarjan RE, Data structures and network algorithms, CBMS 44, Society for Industrial and Applied Mathematics, Philadelphia, PA; 1983.

23.

Tatusov RL, Koonin EV, Lipman DJ. A genomic perspective on protein families. Science. 1997 Oct 24;278(5338):631–637. [PubMed: 9381173]

24.

Zaslavsky L, Tatusova T. Mining the NCBI influenza sequence database: adaptive grouping of BLAST results using precalculated neighbor indexing. PLoS Curr. 2009;1:RRN1124. [PMC free article: PMC2771650] [PubMed: 20029662]

25.

Plücken H1. Müller B, Grohmann D, Westhoff P, Eichacker LA. The HCF136 protein is essential for assembly of the photosystem II reaction center in Arabidopsis thaliana. FEBS Lett. 2002 Dec 4;532(1-2):85–90. [PubMed: 12459468]

26.

Meurer J, Plücken H, Kowallik KV, Westhoff P. A nuclear-encoded protein of prokaryotic origin is essential for the stability of photosystem II in Arabidopsis thaliana. EMBO J. 1998 Sep 15;17(18):5286-5297. [PMC free article: PMC1170856] [PubMed: 9736608]

27.

Peltier JB, Emanuelsson O, Kalume DE, Ytterberg J, Friso G, Rudella A, Liberles DA, Söderberg L, Roepstorff P, von Heijne G, van Wijk KJ. Central functions of the lumenal and peripheral thylakoid proteome of Arabidopsis determined by experimentation and genome-wide prediction. Plant Cell. 2002 Jan;14(1):211–236. [PMC free article: PMC150561] [PubMed: 11826309]