During my internship at UTS (University of Technology of Sydney), we studied the situation awareness for Human-Robot Collaboration. We designed a specific task: building a tower in Duplo (big lego). That task involves developing multiple algorithms.
First, we used a custom color recognition algorithm in Python, for the Duplo detection that uses the PR2 camera combined with a PointCloud algorithm produced by the Kinect.
Then, we implement several gestures to grab elements and superpose them in order to build a tower.
We evaluate several conditions such as gaze, action intention , with custom Duplo disposition in order to understand in which situation humans need feedback from robots to design effective Human-Robot Interaction.