Robotics for Elderly
Human-Robot Interaction
Keywords
Robotics, Human-Robot Interaction, Robotics Operating System (ROS), Human Centred Design, Python
Goal
An internship assignment at Heemskerk Innovative Technology (HIT) was to develop human-robot interaction modules for elderly living in a healthcare facility. The robot used for the assignment was TIAGo from Pal Robotics. The human-robot interaction consisted of three steps:
- Approaching the Patient.
- Addressing the Patient.
- Understanding Patient’s Needs.
An elderly patient needs to be approached followed by gazing to address the patient and finally utilisation of an a speech recognition module to understand their needs.
Approaching the Patient
An elderly patient needed to be approached by the robot in an orderly fashion (not rushed, not too slow) while keeping a fair distance from the patient, since patients can be intimidated by a robot. The approaching module used input from a computer vision module running YOLO (You Only Look Once) algorithm which provided X,Y,Z coordinates in 3D space of the detected patients, this information was used to approach the patient and while finding an open space around the patient using on board LIDAR. Basic algorithm was for the robot to approach the patient in a circle and choose an empty area around the patient.
Addressing the Patient
For a robot to address a human, it needs to use body language which the human can understand. From multiple researches, we know that gazing is a good possible body language a robot can use to address the humans.
We used gazing as a language to address the approached patient in oder to let the patient know, that the robot is listening or attentive to the patient. We also added a feature to address the closest patient in the surrounding first as seen in the video. The gaze feature automatically prompts patients to engage with the robot verbally.
Understanding Patient's Needs
After approaching and addressing the patient, we had to understand their needs for which we used a wrapper for google speech to text service in order to convert voice data of the patient into text which could be passed on to a natural language processor to understand what the patient requires and hence the robot can carry out the tasks the patients have asked for or provide with an appropriate reply.
Connecting the Dots
All three modules work together based on the messages states defined in the High Level State Image, collectively called as Human-Robot Interaction Module. At the final stage the module was integrated into a finite state machine which switched between multiple states based on the output of each individual states as also seen in the second image. For instance, while patrolling if a robot detects a human, the approaching module begins followed by gazing module, speech to text module and then back patrolling module.
