Research

Self-Supervised Instance Segmentation by Grasping
YuXuan Liu, Xi Chen, and Pieter Abbeel
[pdf] [arxiv]

Distributional Instance Segmentation: Modeling Uncertainty and High Confidence Predictions with Latent-MaskRCNN
YuXuan Liu, Nikhil Mishra, Pieter Abbeel, and Xi Chen
In the International Conference on Robotics and Automation (ICRA), 2023
[pdf] [arxiv] [dataset] [code]

Autoregressive Uncertainty Modeling for 3D Bounding Box Prediction
YuXuan Liu, Nikhil Mishra, Maximilian Sieb, Yide Shentu, Pieter Abbeel, and Xi Chen
In the European Conference on Computer Vision (ECCV), 2022
[pdf] [arxiv] [dataset] [code] [blog]

Self-Consistent Trajectory Autoencoder: Hierarchical Reinforcement Learning with Trajectory Embeddings
John D. Co-Reyes*, YuXuan Liu*, Abhishek Gupta*, Benjamin Eysenbach, Pieter Abbeel, Sergey Levine
In the International Conference on Machine Learning (ICML), 2018
[pdf] [arxiv] [video] [code]

Meta-Reinforcement Learning of Structured Exploration Strategies
Abhishek Gupta, Russell Mendonca, YuXuan Liu, Pieter Abbeel, Sergey Levine
In the Advances in Neural Information Processing Systems (NeurIPS), 2018
[pdf] [arxiv] [code]

Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation
YuXuan Liu*, Abhishek Gupta*, Pieter Abbeel, Sergey Levine
In the International Conference on Robotics and Automation (ICRA), 2018
[pdf] [arxiv] [video] [code]

Learning Invariant Feature Spaces to Transfer Skills with Reinforcement Learning
Abhishek Gupta*, Coline Devin*, YuXuan Liu, Pieter Abbeel, Sergey Levine
In the International Conference on Learning Representations (ICLR), 2017
[pdf] [arxiv] [video]

Projects

Robo Call Screener - Phone AI

Filter robocalls from humans with live transcription and responses at your fingertips

• Have our robot answer your robocalls so you don't have to
• Screen humans from robots without having to pick up the phone
• Live transcription and text responses give you the flexibility to respond when you can't talk

Temporal Convolutional Policy Networks [pdf]
We show that temporal convolutional policies can be effective for partially observed tasks including classical control and pong.

ARCode [project page] [video]
We designed an audio transmission protocol to send messages, links, and pictures using the Fast-Fourier Transform.

iSwipe [github]
First gesture-based keyboard for iOS using a dynamic programming algorithm to match words with vector angle differences

[show more projects]

MindChat [project page] [video]
We learn EEG states to encode brain activity and transmit messages using Huffman encoding.

Dreamboard [github]
Powerful iOS theming platform with interactive widgets, custom script parser, and over 11 million downloads in Cydia.

Intellifit [project page]
We built an iOS fitness tracker that collects real-time motion data and classifies activities using dynamic time warping.

H-Auth [project page]
We learn handwriting styles with a neural network and built an authentication platform based on handwriting.

Instatag [project page] [video]
We automatically generate Instagram tags using image classification and real-time Instagram data.

Meme Generator [project page]
We trained an image captioning model to generate meme captions conditioned on input images.

Scribe [project page] [video]
We applied speech recognition on lecture videos to match lectures with corresponding textbook material using tf-idf.

HouseRank [github]
We trained a ranking system on housing preferences to filter listings on Craigslist.

Kerve [project page]
We implemented a pose classification algorithm with the Kinect for learning Yoga.

MineCraft Mini [github]
Minecraft clone in Java OpenGL with physics simulation, water flood fill, and perlin noise random map generation.

HoloScan [project page]
We built a real-time facial detection and recognition system that can identify people you are looking at.

SmartRoomba [project page] [video]
We hacked a Roomba to follow you with real-time face detection.

Watchpoint [project page]
We built a smartwatch app that alerts you when your phone is out of a certain range.

[hackathon projects] [twitter]