Virginia Smith

When Virginia Smith began her PhD in artificial intelligence, she had a question: How do you train a neural network on data that is stored across multiple machines? 

Her attempts to answer it have made her a leader in the field of federated learning, which seeks to handle data spread across hundreds, or even millions, of remote sources. 

Google researchers first introduced federated learning in 2017 to use with the company’s mobile devices. The method they devised involved training millions of neural networks locally before sending them to a company server to be merged together in a master model. It allowed the master model to train on data from every device without making it necessary to centralize that data. This not only reduced latency in the mobile experience but could also improve each user’s data privacy.

But combining millions of AI models also risks creating a central model that performs well on average but poorly for outliers—for example, voice recognition software that fails when the speaker has an unfamiliar accent.

So Smith proposed a new technique for more “personalized” federated learning. Rather than merge a million localized models into one, it merges the most similar localized models into a few—the more heterogeneous the data, the greater the number of final models. Each model still learns from many devices but is also tailored to specific subsets of the user population.

Smith also works to overcome other challenges in federated learning, such as accounting for different power and memory constraints on different devices. To encourage more research, she co-created an open-source tool that lets researchers test their federated techniques on more realistic data sets and in more realistic environments.