Andrew YuXuan Liu

Andrew (YuXuan) Liu

I am a Tech Lead Manager at Covariant, leading the ML team for autonomous robots. While working at Covariant, I started and completed my PhD at UC Berkeley with Pieter Abbeel on Perception for Real-World Robotic Applications.

Previously, I did my Masters and Undergraduate studies at UC Berkeley, also with Pieter Abbeel. During my undergrad, I interned at Quora on language classification and Dropbox on iOS. Before Berkeley, I wrote the first iOS swipe keyboard and a dynamic theming platform for the jailbroken iOS platform.

/ CV / Github / Linkedin / Twitter

Research

	Self-Supervised Instance Segmentation by Grasping YuXuan Liu, Xi Chen, Pieter Abbeel International Conference on Intelligent Robots and Systems (IROS), 2023 arXiv We use before and after grasp images to augment instance segmentation supervision in a self-supervised manner, resulting in 10x data efficiency.
	Distributional Instance Segmentation: Modeling Uncertainty and High Confidence Predictions with Latent-MaskRCNN YuXuan Liu, Nikhil Mishra, Pieter Abbeel, Xi Chen International Conference on Robotics and Automation (ICRA), 2023 patent / project page / dataset / arXiv / code We augment instance segmentation models with a latent code in a VAE manner to model uncertainty and propose a confidence mask method that can significantly reduce robotic picking errors.
	Autoregressive Uncertainty Modeling for 3D Bounding Box Prediction YuXuan Liu, Nikhil Mishra, Maximilian Sieb, Yide Shentu, Pieter Abbeel, Xi Chen European Conference on Computer Vision (ECCV), 2022 patent / project page / dataset / arXiv / code / blog We augment instance segmentation models with a latent code in a VAE manner to model uncertainty and propose a confidence mask method that can significantly reduce robotic picking errors.
	Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation YuXuan Liu, Abhishek Gupta, Pieter Abbeel, Sergey Levine International Conference on Robotics and Automation (ICRA), 2018 project page / arXiv / code We propose an imitation learning method from video demonstractions by translating task representations between viewpoints.
	Self-Consistent Trajectory Autoencoder: Hierarchical Reinforcement Learning with Trajectory Embeddings John D. Co-Reyes, YuXuan Liu, Abhishek Gupta, Ben Eysenbach, Pieter Abbeel, Sergey Levine International Conference on Machine Learning (ICML)*, 2018 project page / video / arXiv / code We encode trajectories into a latent space which can be decoded for hierarchical planning and executed as a policy.
	Meta-Reinforcement Learning of Structured Exploration Strategies Abhishek Gupta, Russell Mendonca, YuXuan Liu, Pieter Abbeel, Sergey Levine Neural Information Processing Systems (NeurIPS), 2018 arXiv / code We meta-learn coherent exploration strategies and use this learned representation space for more meaningful exploration.
	Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning Abhishek Gupta, Coline Devin, YuXuan Liu, Pieter Abbeel, Sergey Levine International Conference on Learning Representations (ICLR), 2017 video / arXiv We learn an invariant representation space to transfer policies between different robots.

iOS Projects

Robo Call Screener - Phone AI

Call screening with live transcription for iOS.

Available on App Store, 2021

iSwipe Keyboard

First gesture-based keyboard for jailbroken iOS (pre-iOS 8.0) using a dynamic programming algorithm to match words with gesture angle differences.

code / released 2011

DreamBoard

Powerful iOS theming platform with interactive widgets, custom script parser, and over 11 million downloads in Cydia.

code / released 2011

Technical Reports

	Information Maximizing Preference Learning IEOR290 Machine Learning for Operations with Humans, 2018 pdf We model the reward as a Bayesian neural network and maximize information gain to efficiently learn from human preferences.
	Joint Embedding Spaces for Language and Behavior INFO259 Natural Language Processing, 2017 pdf We learn an embeddings space where trajectories are points and language feedback are vectors that modulate between different behaviors in latent trajectory space.
	Information Maximizing Exploration and Control EE229A Information Theory, 2017 pdf We maximize state entropy for exploration and mutual information with a latent variable for controllability.
	Temporal Convolutional Policy Networks YuXuan Liu, Tony Duan, Wesley Hsieh CS294 Deep Neural Networks, 2016 pdf We show that temporal convolutional policies can be effective for partially observed tasks including classical control and pong.

Hackathon Projects

	ARCode HackRice 2015. 1st Place Winner. Best use of Microsoft Product project page / video We designed an audio transmission protocol to send messages, links, and pictures using the Fast-Fourier Transform.
	Mindchat TreeHacks 2015. Best Microsoft Hack. Best use of Vertical Response project page / video We use EEG states to encode brain activity and transmit messages using Huffman encoding.

Intellifit
Hack The North, 2014. Best Myo Hack
We built an iOS fitness tracker that collects real-time motion data and classifies activities using dynamic time warping.

H-Auth
HackTech, 2016. Best Synaptics Hack - Touch.
We learn handwriting styles with a neural network and built an authentication platform based on handwriting.

Instatag
HackRice, 2016. 3rd Place Winner.
We automatically generate Instagram tags using image classification and real-time Instagram data.

Meme Generator
Cal Hacks 3.0, 2016. Best ML Hack.
We trained an image captioning model to generate meme captions conditioned on input images.

Scribe
HackTX, 2015.
We applied speech recognition on lecture videos to match lectures with corresponding textbook material using tf-idf.

Kerve
HackSC, 2014.
We implemented a pose classification algorithm with the Kinect for learning Yoga.

SmartRoomba
HackTech, 2017. Best Use of iRobot
We hacked a Roomba to follow you with real-time face detection.

Watchpoint
SoHacks, 2014. Pebble Prize
We built a smartwatch app that alerts you when your phone is out of a certain range.

Misc

Teaching

CS188 Introduction to Artificial Intelligence: Spring 2017
EE16A Design of Information Devices and Systems: Spring 2016, Fall 2016, Fall 2017
Outstanding Graduate Student Instructor Award: May 2017

Reviewer

NeurIPS, ICML, ICLR, ICRA, IROS

Website template from Jon Barron.

*indicates equal contribution