Human-Machine Collaborative Systems

CIRL currently supports four main thrusts in the area of human-machine collaborative systems: systems for collaboration, object recognition and scene understanding, fine-grained action recognition, and deep learning of robot trajectories from expert demonstrations. Many of these projects were funded by the NSF National Robotics Initiative grant Multilateral Manipulation by Human-Robot Collaborative Systems.

CoSTAR: The Collaborative System for Task Automation and Recognition

Traditional robot manipulators are prohibitively costly for many small lot manufacturing entities (SMEs) companies to due to their rapidly changing assembly lines and aging equipment with highly variable performance. We aim to simplify robot integration, improve efficiency of collaborative industrial robot systems, and enable their application in complex, dynamic environments.

Our CoSTAR system provides a powerful visual programming interface to a broad range of robot capabilities. We demonstrated the power of this system at the KUKA Innovation Award competition at Hannover Messe 2016, where we won a €20,000 prize over 25 other entrants and 5 other finalist teams.

Related Projects

Object Recognition and Scene Understanding

Visual perception is crucial for many robotics applications such as autonomous navigation, home service robots, and industrial manufacturing. However, most methods are not robust to difficult scenarios involving densely cluttered scenes, heavy object occlusions and large intra-class object variation. We look at problems including object classification, semantic segmentation and 6-DoF object pose estimation.

Screenshot from 2016-05-24 13:25:40

Related Projects

Fine-grained Action Analysis

The ability to automatically recognize a sequence of actions is important for applications ranging from collaborative manufacturing to surgical skill assessment. In this work we jointly temporally segment and classify a sequence of actions in situated settings using sensor data (e.g. robot kinematics, accelerometer data) and video.


Related Projects

Planning with Knowledge Learned from Expert Demonstrations

In order for robots to be able to solve difficult problems, they must be able to learn from human experts. Humans expect robots to solve tasks in ways that make intuitive sense to them. We propose to give robots this intuition through algorithms for combined task and motion planning using a combination of knowledge learned from watching humans perform tasks and through exploration.

In this work, we propose methods for grounding robot actions based on human demonstrations. We then describe the algorithms necessary to create task plans that sequence these actions to accomplish high-level goals.

Related Projects


Robot manipulation from pixels

In this project, we investigate how to complete robot manipulation tasks with only visual inputs. We propose to extract object-centric representations from raw images and then model the interactive relationships among them. Robot pose estimation is also studied.


Comments are closed.