Andrew (YuXuan) Liu

I am a Member of the Technical Staff at OpenAI. Previously I was a Tech Lead Manager at Covariant, leading the ML team for autonomous robots. While working at Covariant, I started and completed my PhD at UC Berkeley with Pieter Abbeel on Perception for Real-World Robotic Applications. I completed my Masters and Undergraduate studies at UC Berkeley, also with Pieter Abbeel. During my undergrad, I interned at Quora on language classification and Dropbox on iOS.

 /  CV (2018)  /  Github  /  Linkedin  /  Twitter

profile photo
Research
Self-Supervised Instance Segmentation by Grasping
YuXuan Liu, Xi Chen, Pieter Abbeel
International Conference on Intelligent Robots and Systems (IROS), 2023
arXiv

We use before and after grasp images to augment instance segmentation supervision in a self-supervised manner, resulting in 10x data efficiency.

Distributional Instance Segmentation: Modeling Uncertainty and High Confidence Predictions with Latent-MaskRCNN
YuXuan Liu, Nikhil Mishra, Pieter Abbeel, Xi Chen
International Conference on Robotics and Automation (ICRA), 2023
patent / project page / dataset / arXiv / code

We augment instance segmentation models with a latent code in a VAE manner to model uncertainty and propose a confidence mask method that can significantly reduce robotic picking errors.

Autoregressive Uncertainty Modeling for 3D Bounding Box Prediction
YuXuan Liu, Nikhil Mishra, Maximilian Sieb, Yide Shentu, Pieter Abbeel, Xi Chen
European Conference on Computer Vision (ECCV), 2022
patent / project page / dataset / arXiv / code / blog

We augment instance segmentation models with a latent code in a VAE manner to model uncertainty and propose a confidence mask method that can significantly reduce robotic picking errors.

Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation
YuXuan Liu*, Abhishek Gupta*, Pieter Abbeel, Sergey Levine
International Conference on Robotics and Automation (ICRA), 2018
project page / arXiv / code

We propose an imitation learning method from video demonstractions by translating task representations between viewpoints.

Self-Consistent Trajectory Autoencoder: Hierarchical Reinforcement Learning with Trajectory Embeddings
John D. Co-Reyes*, YuXuan Liu*, Abhishek Gupta*, Ben Eysenbach, Pieter Abbeel, Sergey Levine
International Conference on Machine Learning (ICML), 2018
project page / video / arXiv / code

We encode trajectories into a latent space which can be decoded for hierarchical planning and executed as a policy.

Meta-Reinforcement Learning of Structured Exploration Strategies
Abhishek Gupta, Russell Mendonca, YuXuan Liu, Pieter Abbeel, Sergey Levine
Neural Information Processing Systems (NeurIPS), 2018
arXiv / code

We meta-learn coherent exploration strategies and use this learned representation space for more meaningful exploration.

Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning
Abhishek Gupta*, Coline Devin*, YuXuan Liu, Pieter Abbeel, Sergey Levine
International Conference on Learning Representations (ICLR), 2017
video / arXiv

We learn an invariant representation space to transfer policies between different robots.

iOS Projects
Robo Call Screener - Phone AI

Call screening with live transcription for iOS.

Available on App Store, 2021

Download on the App Store

iSwipe Keyboard

First gesture-based keyboard for jailbroken iOS (pre-iOS 8.0) using a dynamic programming algorithm to match words with gesture angle differences.

code / released 2011

DreamBoard

Powerful iOS theming platform with interactive widgets, custom script parser, and over 11 million downloads in Cydia.

code / released 2011

Hackathon Projects
ARCode
HackRice 2015. 1st Place Winner. Best use of Microsoft Product
project page / video

We designed an audio transmission protocol to send messages, links, and pictures using the Fast-Fourier Transform.

Mindchat
TreeHacks 2015. Best Microsoft Hack. Best use of Vertical Response
project page / video

We use EEG states to encode brain activity and transmit messages using Huffman encoding.

Intellifit
Hack The North, 2014. Best Myo Hack
We built an iOS fitness tracker that collects real-time motion data and classifies activities using dynamic time warping.

H-Auth
HackTech, 2016. Best Synaptics Hack - Touch.
We learn handwriting styles with a neural network and built an authentication platform based on handwriting.

Instatag
HackRice, 2016. 3rd Place Winner.
We automatically generate Instagram tags using image classification and real-time Instagram data.

Meme Generator
Cal Hacks 3.0, 2016. Best ML Hack.
We trained an image captioning model to generate meme captions conditioned on input images.

Scribe
HackTX, 2015.
We applied speech recognition on lecture videos to match lectures with corresponding textbook material using tf-idf.

Kerve
HackSC, 2014.
We implemented a pose classification algorithm with the Kinect for learning Yoga.

SmartRoomba
HackTech, 2017. Best Use of iRobot
We hacked a Roomba to follow you with real-time face detection.

Watchpoint
SoHacks, 2014. Pebble Prize
We built a smartwatch app that alerts you when your phone is out of a certain range.

Misc

Teaching

CS188 Introduction to Artificial Intelligence: Spring 2017
EE16A Design of Information Devices and Systems: Spring 2016, Fall 2016, Fall 2017
Outstanding Graduate Student Instructor Award: May 2017

Reviewer

NeurIPS, ICML, ICLR, ICRA, IROS


Website template from Jon Barron.

*indicates equal contribution