PyTorch Distributed: Experiences on Accelerating Data Parallel Training
Shen Li, Yanli Zhao, Rohan Verma, Omkar Salpekar, Pieter Noordhuis, Teng Li, Adam Paszke, Jeff Smith, Brian Vaughan, Pritam Damania, Soumith Chintala
People
Teng Li
Research Scientist Manager
I am a research scientist and engineering manager at Facebook AI Research (FAIR). I currently support the research engineering team of reinforcement learning and robotics, based in the Menlo Park headquarters. At FAIR, I was part of the PyTorch team and led the area of PyTorch distributed training, as well as the engineering efforts of FAIR’s deep learning platform. Before joining FAIR, I worked on Facebook’s core data infrastructure.
I received my PhD in computer engineering from the George Washington University in Washington, DC. My PhD research is primarily within the area of GPGPU, parallel and distributed systems, and high-performance computing.
Interests
Parallel computing, GPGPU, deep learning, high-performance computing, and databases.