Robot Cooking

Title	Robot Cooking
Summary	Capability to physically perform first aid for an autonomous (mobile) robot
Keywords	Robots, First Aid, Visual Recognition, Healthcare
TimeFrame	2017/1/1-2017/8/30
References	-first aid Travers, A. H., Rea, T. D., Bobrow, B. J., et al. (2010). Part 4: CPR overview 2010 American Heart Association guidelines for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation, 122(18 suppl 3), S676-S684. -pose recognition Jamie Shotton, Ross Girshick, Andrew Fitzgibbon, Toby Sharp, Mat Cook, Mark Finocchio, Richard Moore, Pushmeet Kohli, Antonio Criminisi, Alex Kipman, Andrew Blake, "Efficient Human Pose Estimation from Single Depth Images", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 12, pp. 2821-2840, Dec. 2013, doi:10.1109/TPAMI.2012.241
Prerequisites	Strong multidisciplinary interest, strong work ethic, software (also ability to work with libraries), possibly some hardware/electronics
Author	Chandrashekhar Shankarrao Nasurade, Vamsi Krishna Nathani
Supervisor	Martin Cooney
Level	Master
Status	Ongoing

The student will be part of a research drive at HH connected to the Intelligent Home to develop the capability for home robots to help save people's lives in emergencies. So far, although smart phones have acquired huge popularity, robots have failed to gain wide acceptance in people's homes; the cause is that a very good reason for having a versatile (expensive) embodiment in a home has not yet been established. We think first aid can be such a reason. First, many of us know of someone who has fallen down and not received immediate medical attention; the repercussions can be very serious. Second, first aid is a complex problem which requires a complex robotic embodiment; a simple vacuum cleaner-like machine is not enough to help. Within this context, the student will use an excellent robot (Baxter from Rodney Brooks, on a Clearpath mobile base) to develop some capability to conduct first aid. Some notes:

• In this exploratory project the robot will touch a mannequin in place of a real human, due to ethics and safety concerns.
• We will seek to obtain advice from various experts, e.g., in nursing and computer vision.
• The robot will also be used from time to time by others (researchers and students).
• Various possibilities exist for contributions to intelligent systems:
  • an approach for carrying out basic first aid steps: e.g., chest compressions, adjusting pose to assist airway, and artificial ventilation. This could involve designing a special gripper and approach for using it. OR
  • an approach for stemming bleeding by detecting bleeding (rate, location of wounds and arteries) and treating (cooling a wound with a peltier, applying pressure to the wound with pads or to arteries, and elevating limbs). OR
  • an approach for assessing a person's mental state/confusion such as AVPU or the Glascow coma scale, which requires the robot to execute multimodal behavior like touching a person and recognize reactions such as the manner of contraction of an arm.

Goal: the robot will, while recognizing, conduct some basic steps for first aid on a mannequin (involving physical touch)

Relation to some previous work:

Several student projects have focused on preparation for robotic first aid. In 2014, Meyr built a mobile robot system which could be commanded to go to the location of an emergency and ask humans if they were okay; Lazaro built a computer vision algorithm to distinguish fallen humans from objects. In 2015, Zhang and Zhao built a recognition system using Kinect data to recognize health signs in an unconscious person related to first aid. In 2016, Hotze created a mobile robot system to find fallen humans and locate on a map body parts important for first aid, to allow for cpr to be conducted. The current project will draw insight from this previous work by students at HH and be the first to result in a system capable of physically conducting some form of first aid (physically touching a human mock-up).

Possible Steps:

•Preparation: (January) Becoming (more) familiar with OpenCV, ROS, (Arduino,) Baxter(/Ridgeback), getting robot/robot arms to move

•(February) Basic literature review; last preparation, such as attaching secondary systems which might be needed such as touch sensors to the robot

•Main part: (March-April) Sensing, Planning, Acting For example: (sense) the robot will use its sensors to locate important first aid points in 3d (plan) the robot will use inverse kinematics to position its end effectors to do first aid (act) the robot will move to do first aid, controlling its motions via feedback from sensors

•After-work: (May) Evaluating; writing/presenting

Possible evaluation metrics: e.g., objective measures (time, positioning, force); qualitative assessment by an expert; and/or comparison to a novice human first aider

Expected results: a thesis/report, code, video (for this project, the student is expected to be willing to also write a six page shortened version of the thesis, to be submitted to a conference)