Sometimes things happen in the lab that you wouldn’t expect. Back in 2009, when Matei Zaharia was a grad student at UC Berkeley’s AMPLab, he started a project called Spark to serve as a pilot workload for Mesos, an open-source project to manage clusters. Since then, Mesos has faded, while Spark has become the widely adopted successor to the Hadoop distributed processing framework—faster, smarter, and, unlike its predecessor, a robust platform for streaming analytics and machine learning.

Today Zaharia is CTO of Databricks, a cloud-based provider of Spark and machine learning as a service, though he still keeps one foot in academia as an assistant professor of computer science at Stanford. One testament to his ingenuity: According to Databricks CEO Ali Ghodsi, Zaharia once informed him that he had an interest in biology and was taking a class. Not long after a project emerged that he created in collaboration with AMPLap colleagues: the Scalable Nucleotide Alignment Program (SNAP), a sequence aligner that is three to 20 times faster than competing sequencing solutions.

To read this article in full, please click here