- gene variants (including SNPs, SNVs, QTLs, CNVs, and other rearrangements big and small)
- models (including cell lines, animal strains, species, breeds, as well as targeted mutants)
We import data from a variety of data sources in formats including databases, spreadsheets, delimited text files, XML, JSON, and Web APIs, on a monthly schedule, which is placed into a Postgres database (hosted by the NIF). Our curation team semantically maps each resource into our data model, primarily using ontologies. This involves both typing relevant columns, mappings between columns (such as between identifier and labels, but also more complex associations, such as between a genotype-phenotype association and the publication it was mentioned in), and value-level mapping. Because our focus is on genotype-phenotype data, we focus our efforts on ensuring that each resources’ variants, genes, genotypes, strains, and phenotypes are well-typed using ontologies and standardized identifiers. Internally, we map all genes to NCBI gene identifiers, diseases to the Disease Ontology, and phenotypes into our unified phenotype ontology, Uberpheno.
With many resources integrated into a single database, we can join across the various data sources to produce integrated views. We have started with the big players including ClinVar and OMIM, but are equally interested in boutique databases (which you will see more of in the coming months). You can learn more about the sources of data that populate our system from our sources page.
Once curated, we generate views and semantically index them into a Solr instance, and the data is served to our Monarch application via REST services through NIF. That way when a user is interested in exploring abnormalities of the ear, a single query can retrieve all relevant data from the system. Our web application wraps NIF’s REST services.
Since all of our data is curated using ontologies, we are currently exploring the use of a graph database (based on Neo4j) to serve up all our data and ontologies. This will have the side benefit of providing the community our semantically mapped data in RDF.