Wednesday, October 8, 2014

How Monarch Integrates and Curates Biological Data

As with most biomedical databases, the first step is to identify relevant data from the research community. The Monarch Initiative is focused primarily on phenotype-related resources. We bring in data associated with those phenotypes so that our users can begin to make connections among other biological entities of interest, such as:
  • genes
  • genotypes
  • gene variants (including SNPs, SNVs, QTLs, CNVs, and other rearrangements big and small)
  • models (including cell lines, animal strains, species, breeds, as well as targeted mutants)
  • pathways
  • orthologs
  • phenotypes
  • publications

We import data from a variety of data sources in formats including databases, spreadsheets, delimited text files, XML, JSON, and Web APIs, on a monthly schedule, which is placed into a Postgres database (hosted by the NIF). Our curation team semantically maps each resource into our data model, primarily using ontologies. This involves both typing relevant columns, mappings between columns (such as between identifier and labels, but also more complex associations, such as between a genotype-phenotype association and the publication it was mentioned in), and value-level mapping. Because our focus is on genotype-phenotype data, we focus our efforts on ensuring that each resources’ variants, genes, genotypes, strains, and phenotypes are well-typed using ontologies and standardized identifiers. Internally, we map all genes to NCBI gene identifiers, diseases to the Disease Ontology, and phenotypes into our unified phenotype ontology, Uberpheno.

The Monarch Initiative data workflow.

With many resources integrated into a single database, we can join across the various data sources to produce integrated views. We have started with the big players including ClinVar and OMIM, but are equally interested in boutique databases (which you will see more of in the coming months). You can learn more about the sources of data that populate our system from our sources page.

Once curated, we generate views and semantically index them into a Solr instance, and the data is served to our Monarch application via REST services through NIF. That way when a user is interested in exploring abnormalities of the ear, a single query can retrieve all relevant data from the system. Our web application wraps NIF’s REST services.

Since all of our data is curated using ontologies, we are currently exploring the use of a graph database (based on Neo4j) to serve up all our data and ontologies. This will have the side benefit of providing the community our semantically mapped data in RDF.