Paper Notes: Helping Operators Keep Up with Collaborative Robots

Posted on by Curt 4 min read

Working next to a cobot means sharing a task with a machine that runs on its own for a while and then needs you. The robot’s half takes care of itself. The hard part is guessing when your half is coming, and for how long.

During my time at the People and Robots Laboratory at UW-Madison, I worked on this exact problem, in two halves. The first half was about the shape of the teaming itself. We compared three levels of task interdependence between the human and the robot. In pooled teaming each works independently and the results combine at the end. In sequential teaming one’s output feeds the other’s task, assembly-line style. In reciprocal teaming the work cycles back and forth between the two. Workers in the reciprocal arrangement reported less stress and were more likely to describe the robot as a collaborator instead of a tool.

A participant's hands and the UR3e arm building block structures together on a taped-off table, with a sheet of target patterns at the edge.
Participant and UR3e sharing the build during the interdependence study. From Zhao et al., RO-MAN 2020. © 2020 IEEE.

That result sets up the second half. Once the human and robot are actually interleaved, when does the robot need you, and for how long? Robot Attention Demand (RAD) already existed as a metric. It splits the robot’s task into stretches it handles alone (neglect time) and stretches that need a person (interaction time), and the ratio tells you how demanding the robot is to supervise. The catch is that RAD was a post-task analysis number, something you computed after the run was over. We expanded it into pRAD (predictive RAD). Since the task segments are known ahead of time, the whole attention schedule can be predicted and shown to the operator up front, telling them when the cobot will need them and for how long.

Top-down diagram of the study workspace showing assembly and storage zones with the robot, an interaction button, the operator's sorting zone, and the pRAD display.
The study workspace. The pRAD display faces the operator from across the sorting zone. From Henrichs et al., RO-MAN 2021. © 2021 IEEE.

We built two interface prototypes, a timeline showing the entire task sequence and a countdown timer with traffic-light color coding (green, yellow, red). The ROS-based implementation ran on a UR3e cobot with a Robotiq gripper performing a toy house assembly task while participants juggled a secondary sorting task during the robot’s autonomous phases. How the robot found its blocks in the first place is its own post. In a 34-participant study against a no-interface baseline, the timeline made the biggest difference. People felt less loaded down and said they could manage their attention across both tasks instead of hovering near the robot. Both interfaces beat having nothing on usability.

The timeline and countdown timer interfaces shown in their three states. Green while the robot runs alone, yellow as a warning, red when interaction is due.
Both displays over one cycle: neglect, warn, interact. From Henrichs et al., RO-MAN 2021. © 2021 IEEE.

The common thread is that neither half makes the robot any smarter. The gains come from restructuring the teamwork and giving the operator a clearer picture of when they’re actually needed.

Both halves were published at IEEE RO-MAN (2020 and 2021).

Papers:

The header image is from Henrichs et al., RO-MAN 2021. © 2021 IEEE.