I am a Ph.D. student advised by Prof. Russ Tedrake at MIT Computer Science and Artificial Intelligence Laboratory (MIT CSAIL). I also work closely with Prof. Kaiming He. Before coming to MIT, I received my B.S. and M.S. degrees at University of Washington where I was very fortunate to work with Prof. Dieter Fox and collaborate with NVIDIA.
My research interest lies in Machine Learning and Robotics. In particular, I am interested in developing algorithms and systems that can generalize in the complex and unstructured real-world environments. To achieve this, I work on fleet learning that can scale with the heterogeneous data.
Email: liruiw [at] mit (dot) edu [Google Scholar] / [Github] / [Twitter] / [LinkedIn]
Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformers Lirui Wang, Xinlei Chen, Jialiang Zhao, Kaiming He Neural Information Processing Systems (Spotlight), Neurips 2024 arxiv, Code, Website, Models, HF Code, Video, Bibtex |
GenSim2: Scaling Robot Data Generation with Multi-modal and Reasoning LLMs Pu Hua*, Minghuan Liu*, Annabella Macaluso*, Yunfeng Lin, Weinan Zhang, Huazhe Xu, Lirui Wang Conference on Robot Learning (CoRL), 2024 arxiv, Code, Video, Demo, Website, Bibtex |
PoCo: Policy Composition From and For Heterogeneous Robot Learning Lirui Wang, Jialiang Zhao, Yilun Du, Edward Adelson, Russ Tedrake Robotics: Science and Systems (R:SS), 2024 arxiv, Bibtex, Website, Video, News, |
GenSim: Generating Robotic Simulation Tasks via Large Language Models Lirui Wang, Yiyang Ling*, Zhecheng Yuan*, Mohit Shridhar, Chen Bao, Yuzhe Qin, Bailin Wang, Huazhe Xu, Xiaolong Wang Workshop on Language Grounding and Robot Learning (Best Paper), CoRL 2023 International Conference on Learning Representations (Spotlight), ICLR 2024 arXiv, Bibtex , Website, Demo, Code, Dataset, Video, News |
Robot Fleet Learning via Policy Merging Lirui Wang, Kaiqing Zhang, Allan Zhou, Max Simchowitz, Russ Tedrake International Conference on Learning Representations (ICLR), 2024 arxiv, Bibtex, Code, Video |
Does Decentralized Learning with non-IID Unlabeled Data Benefit from Self Supervision?
Lirui Wang, Kaiqing Zhang, Yunzhu Li, Yonglong Tian, and Russ Tedrake International Conference on Learning Representations (ICLR), 2023 arXiv, Bibtex, Code, Video |
Transferable Tactile Transformers for Representation Learning Across Diverse Sensors and Tasks Jialiang Zhao, Yuxiang Ma, Lirui Wang, Edward H. Adelson Conference on Robot Learning (CoRL), 2024 arxiv, Bibtex, Code, Website |
3D Neural Embedding Likelihood: Probabilistic Inverse Graphics for Robust 6D Pose Estimation Guangyao Zhou*, Nishad Gothoskar*, Lirui Wang, Joshua B. Tenenbaum, Dan Gutfreund, Miguel Lázaro-Gredilla, Dileep George, Vikash K. Mansinghka International Conference on Computer Vision (ICCV), 2023 arXiv, Bibtex, Website, Code |
NeRF in the Palm of Your Hand: Corrective Augmentation for Robotics via Novel-View Synthesis
Allan Zhou*, Moo Jin Kim*, Lirui Wang, Pete Florence, Chelsea Finn The IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR), 2023 arxiv, Website, Bibtex |
PredictionNet: Real-Time Joint Probabilistic Traffic Prediction for Planning, Control, and Simulation
Alexey Kamenev, Lirui Wang, Ollin Boer Bohan, Ishwar Kulkarni, Bilal Kartal, Artem Molchanov, Stan Birchfield, David Nistér, Nikolai Smolyanskiy International Conference on Robotics and Automation (ICRA), 2022 arXiv, Bibtex, Video, Blog, Poster |
Hierarchical Policies for Cluttered-Scene Grasping with Latent Plans
Lirui Wang, Xiangyun Meng, Yu Xiang, Dieter Fox IEEE Robotics and Automation Letters (RAL), 2022 arXiv, Bibtex, Video, Poster, Website, Code |
Goal-Auxiliary Actor-Critic for 6D Robotic Grasping with Point Clouds
Lirui Wang, Yu Xiang, Wei Yang, Arsalan Mousavian, Dieter Fox The Conference on Robot Learning (CoRL), 2021 arXiv, Bibtex, Video, Poster, Website, Code |
Manipulation Trajectory Optimization with Online Grasp Synthesis and Selection Lirui Wang, Yu Xiang, Dieter Fox Robotics: Science and Systems (R:SS), 2020 arXiv, Bibtex, Video, Website, Code |