Manoj Ganesan, Software Architect at 23andMe — ML products at 23andMe Data and Machine Learning are core components of 23andMe’s Health + Ancestry and 23andMe+ services. Since 23andMe launched in 2007, over 80% of our approximately 12 million customers have consented to participate in research and contribute their genetic and self-reported survey data to help the advancement of science. We use…

Anuved Verma and Anja Bog, Software Engineers at 23andMe — Are you planning to run “some kind of workflow” and don’t want to worry about infrastructure, monitoring, or resumability? Do you want to orchestrate different kinds of tasks, like Python scripts, command-line tools, or applications written in other programming languages? Yes? …

Patrick Yee and Allie Sanzi, Software Engineers at 23andMe — Summary This article illustrates how to architect an AWS Lambda function, written in Python, to stream input data from an S3 object, pipe the data stream through an external program, and then pipe the output stream to an object in S3. Introduction AWS Lambda function is a handy serverless computing service for…

Prad Pagadala, Software Engineer at 23andMe — AWS Elastic Map Reduce (EMR), a popular big data platform, powers a lot of computation at 23andMe. One particular use case is to find associations between genes and traits using Hail, which is built on top of Apache Spark. In the course of figuring out how to optimize processing for…

Tulasi Paradarami, Sr. Engineering Manager at 23andMe — Introduction In bioinformatics, Variant Call Format (VCF) is a popular text file format for storing genetic variation data, and is the standard output for popular imputation methodologies like minimac. It’s unambiguous, it’s flexible, and it supports arbitrary metadata. A drawback of the VCF format, however, is that it’s a text file…

Joe Banks, Tech Lead at 23andMe — I have a disability. Two, actually. I was diagnosed with Usher Syndrome Type 2A, a rare genetic disease characterized by hearing loss and progressive vision loss. When I was 3, my parents discovered that I was born with moderate-severe hearing loss in both of my ears. …