Local Neural Descriptor Fields: Locally Conditioned Object Representations for Manipulation

Ethan Chun, Yilun Du, Anthony Simeonov, Tomas Lozano-Perez, Leslie Kaelbling

Neural Descriptor Fields are a way to implicitly represent three dimensional geometry in a neural network. Similarly shaped objects produce similar descriptor fields, thus one can use Neural Descriptor Fields to encode object poses. Unfortunately, conventional Neural Descriptor Fields encode information at the object level -- a handle on a mug is treated differently than if that handle were glued to a bottle. In this paper, we present an architecture which restrict Neural Descriptors to local geometry, allowing for more varied robotic manipulation.

Modulation of Prosthetic Ankle Plantarflexion Through Direct Myoelectric Control of a Subject-Optimized Neuromuscular Model

Tony Shu, Christopher Shallal, Ethan Chun, Aashini Shah, Angel Bu, Daniel Levine, Seong Ho Yeon, Matthew Carney, Hyungeun Song, Tsung-Han Hsieh, and Hugh M. Herr

The focus of this paper is on a novel EMG control paradigm using a Neuromuscular model of trans-tibial amputee subject. However, in order to test this paradigm, we also needed a fully function robotic ankle -- hence my role. While Tony and Chris focused on the controls aspect, I worked on sensor integration, mechanical system upgrades, new firmware to accommodate the EMG board, and various other mechatronics related tasks. It's a lot of fun building robots!

Novel Shape Generation with SO(3)-Equivariant Auto-Encoders

This is my final project for Prof. Tess Smidt's course on Equivariant Neural Networks. The main idea is that we wanted to find a more efficient way to generate 3D shapes. To do so, we leverage the inherent SO(3) symmetry in 3D data and propose an SO(3) equivariant auto-encoding neural network. We represent 3D shapes as the coefficients of the spherical harmonic basis functions and design a rotation equivariant interpolation method to traverse the latent space. We demonstrate robust reconstruction of a dataset of 100 randomly generated boxes and show coherent traversals of the latent space when trained on two and 100 randomly selected objects.

Understanding Probabilistic Inverse Graphics through Optical Digit Recognition with 2D Gaussians

This is my final project for Prof. Josh Tenenbaum's course on Computational Cognitive Science. I wanted to make a simple demonstration of probabilistic inverse graphics. To do so, I targeted optical digit recognition, one of the simplest inverse graphics problems known. I modeled each digit as a collection of 2D Gaussians and ran inference on noisy point set images. I demonstrated a success rate of 0.95 over 50 trials per digit. Furthermore, I presented two of my past attempts at this problem and provided intuitive reasoning on their strengths and pitfalls.

Tiny View Conditioned Latent Diffusion Models

View conditioned latent diffusion models show great potential to improve scene understanding in robotics applications. However, training times are slow and while the former focuses on image quality, robotic applications prioritize efficiency. To fix this, we present a scaled down version of the popular Latent Diffusion Model, trained on the SRNCars dataset. Inspired by the work, Zero-1-to-3, we provide image and pose conditioning conditioning to allow for novel view synthesis from single images. Our model reduces the number of trainable parameters from 860 million in state of the art models to less than 1.7 million parameters and trains in less than four hours on a consumer GPU. This was my final project for Vincent Sitzmann's Inverse Graphics seminar at MIT.

Stereo Occupancy Networks

Current 3D neural implicit reconstruction algorithms often use single images or point clouds as input data. However, the noise artifacts or lack of information from these data can degrade the robustness of their implicit representations. What if we leverage recent advances in stereo perception and use stereo images as input data? Click here to find out more! This was my final project for MIT's Computer Vision course (6.8301).

catBot: Solving the Falling Cat Problem

Have you ever seen a cat fall? They always land on their feet! How is this possible? Is it magic? This is the problem I explore in a set of papers on the falling cat problem. In the first paper, I explore the use of trajectory optimization to solve mimic the cat's motion. In the second, I test two Reinforcement Learning Algorithms, Soft Actor Critic and Proximal Policy Optimization, and compare their performance on solving the problem. Read on to learn how cats always land on their feet. Spoilers: it's not magic!

ChessBot: A Single View Perception and Manipulation System for Robotic Chess

Now presenting, a robot playing chess! Using a single RGBD sensor, this robot system can perceive and act on the state of a 3d simulated chess board. Winning a 2022 Outstanding Project Award, this is my final project for MIT's course on robotic manipulation.

Mechatronics Upgrades to the TF8 Powered Prosthetic Knee

Over the past year, I've been working with Tony Shu to upgrade and refine the TF8 actuator system for use as a powered prosthetic knee. On my part, this involved various custom designed hardware, included a new endstop system and electronics mounts.

A Single-board Mechatronics Platform for Control of a Powered Prosthesis

During my early time at Biomechatronics, I worked extensively with Dan Levine and Tony Shu to develop a new Prosthesis Controller. This system, dubbed Talaria Verdin, enables much of the complex movement seen in the Biomechatronics publication above and is currently in use on three powered prosthesis systems.

Bird Lamp

I made this bird lamp for a friend's birthday. It's designed after the small birds hopping around MIT's campus. The shell was resin printed and the lights are controlled by a small arduino-like microcontroller.

Maker Portfolio

This is a collection of images and descriptions from the maker portfolio that I used to apply to MIT. I haven't edited it too much but its a good overview of some of my previous projects