Difference between revisions of "Robot Cooking"

From ISLAB/CAISR
(Created page with "{{StudentProjectTemplate |Summary=Capability to physically perform first aid for an autonomous (mobile) robot |Keywords=Robots, First Aid, Visual Recognition, Healthcare |Tim...")
 
 
(7 intermediate revisions by one user not shown)
Line 1: Line 1:
 
{{StudentProjectTemplate
 
{{StudentProjectTemplate
−
|Summary=Capability to physically perform first aid for an autonomous (mobile) robot  
+
|Summary=Common sense for a robot to cook healthy food
−
|Keywords=Robots, First Aid, Visual Recognition, Healthcare
+
|Keywords=Robots, Healthcare, Visual Recognition
−
|TimeFrame=2017/1/1-2017/8/30  
+
|TimeFrame=2017/1/1-2017/8/30
−
|References=-first aid
+
|References=-robot cooking
  
−
Travers, A. H., Rea, T. D., Bobrow, B. J., et al. (2010). Part 4: CPR overview
+
Christian Østergaard Laursen, Søren Pedersen, Timothy Merritt, Ole Caprani. Robot-Supported Food Experiences: Exploring Aesthetic Plating with Design Prototypes. In J.T.K.V. Koh et al. (Eds.): Cultural Robotics 2015, LNAI 9549, pp. 107–130, 2016. DOI: 10.1007/978-3-319-42945-8 10 Springer International Publishing Switzerland 2016.012.241
−
2010 American Heart Association guidelines for cardiopulmonary resuscitation and
+
−
emergency cardiovascular care. Circulation, 122(18 suppl 3), S676-S684.  
+
  
−
-pose recognition
+
-common sense acquisition
  
−
Jamie Shotton, Ross Girshick, Andrew Fitzgibbon, Toby Sharp, Mat Cook, Mark Finocchio, Richard Moore, Pushmeet Kohli, Antonio Criminisi, Alex Kipman, Andrew Blake, "Efficient Human Pose Estimation from Single Depth Images", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 12, pp. 2821-2840, Dec. 2013, doi:10.1109/TPAMI.2012.241
+
Rakesh Gupta, Mykel J. Kochenderfer. Common Sense Data Acquisition for Indoor Mobile Robots. ROBOTICS
−
 
+
|Prerequisites=Strong multidisciplinary interest, strong work ethic, software (also ability to work with libraries), possibly some small work with hardware/electronics
−
|Prerequisites=Strong multidisciplinary interest, strong work ethic, software (also ability to work with libraries), possibly some hardware/electronics
+
|Supervisor=Martin Cooney,
−
|Supervisor=Martin Cooney,  
+
 
|Examiner=Antanas, Slawomir
 
|Examiner=Antanas, Slawomir
 +
|Author=Chandrashekhar Shankarrao Nasurade, Vamsi Krishna Nathani
 
|Level=Master
 
|Level=Master
−
|Status=Open
+
|Status=Ongoing
 
}}
 
}}
 +
This project will be about designing a capability for "common sense" in a robot, within the context of helping an elderly person at home with cooking.
 +
Our motivation is that robots could be helpful to people and contribute to their well-being, health, and quality of life, but first some major challenges must be overcome. One challenge is that everyday tasks which humans perform can be complex and confusing. This is especially a problem with elderly persons with declined physical and cognitive abilities (e.g. dementia), when the person can no longer function by themselves but requires support, which is sometimes not available from other humans. To support such a person, we believe a robot should have some degree of what we call here "common sense"; related to robustness, this is an ability to function correctly in the presence of some errors which a healthy adult person can typically detect.
 +
In the case of cooking, to cook in a healthy and good way, this means that a robot should be able to detect and compensate for errors in three main facets of cooking: the recipe, the tools used, and the ingredients.
 +
For example, a human can determine that if a recipe for one person calls for 10kg of salt, this is probably a mistake, and conclude from experience with similar recipes that it should be 10g of salt.
 +
If a recipe calls for a spatula but instead a knife has been provided, a human can determine that this is wrong and seek from experience some tool which is more appropriate.
 +
If instead of salt, a bag of sugar has been provided, a human can determine that this is wrong and seek more appropriate ingredients.
 +
To address the challenge, in this project the robot will learn a model of common sense regarding these three main facets of cooking by unsupervised learning (forming clusters, detecting anomalies, and inferring how anomalies can be rectified).
 +
The robot used will be Baxter on a Ridgeback mobile base, which will be shared with other students and researchers.
 +
The evaluation will measure the degree to which the robot can complete a simple cooking task in the presence of various errors which we introduce into the cooking process.
  
−
The student will be part of a research drive at HH connected to the Intelligent Home to develop the capability for home robots to help save people's lives in emergencies.
+
Timeline:
−
So far, although smart phones have acquired huge popularity, robots have failed to gain wide acceptance in people's homes; the cause is that a very good reason for having a versatile (expensive) embodiment in a home has not yet been established.
+
−
We think first aid can be such a reason.
+
−
First, many of us know of someone who has fallen down and not received immediate medical attention; the repercussions can be very serious.
+
−
Second, first aid is a complex problem which requires a complex robotic embodiment; a simple vacuum cleaner-like machine is not enough to help.
+
−
Within this context, the student will use an excellent robot (Baxter from Rodney Brooks, on a Clearpath mobile base) to develop some capability to conduct first aid.
+
−
Some notes:  
+
−
*In this exploratory project the robot will touch a mannequin in place of a real human, due to ethics and safety concerns.
+
−
*We will seek to obtain advice from various experts, e.g., in nursing and computer vision.
+
−
*The robot will also be used from time to time by others (researchers and students).
+
−
*Various possibilities exist for contributions to intelligent systems:
+
−
**an approach for carrying out basic first aid steps: e.g., chest compressions, adjusting pose to assist airway, and artificial ventilation. This could involve designing a special gripper and approach for using it. OR
+
−
**an approach for stemming bleeding by detecting bleeding (rate, location of wounds and arteries) and treating (cooling a wound with a peltier, applying pressure to the wound with pads or to arteries, and elevating limbs). OR
+
−
**an approach for assessing a person's mental state/confusion such as AVPU or the Glascow coma scale, which requires the robot to execute multimodal behavior like touching a person and recognize reactions such as the manner of contraction of an arm.
+
−
 
+
−
Goal: the robot will, while recognizing, conduct some basic steps for first aid on a mannequin (involving physical touch)
+
−
 
+
−
Relation to some previous work:
+
−
 
+
−
Several student projects have focused on preparation for robotic first aid.
+
−
In 2014, Meyr built a mobile robot system which could be commanded to go to the location of an emergency and ask humans if they were okay; Lazaro built a computer vision algorithm to distinguish fallen humans from objects.
+
−
In 2015, Zhang and Zhao built a recognition system using Kinect data to recognize health signs in an unconscious person related to first aid.
+
−
In 2016, Hotze created a mobile robot system to find fallen humans and locate on a map body parts important for first aid, to allow for cpr to be conducted.
+
−
The current project will draw insight from this previous work by students at HH and be the first to result in a system capable of physically conducting some form of first aid (physically touching a human mock-up).
+
−
 
+
−
Possible Steps:
+
−
 
+
−
•Preparation: (January) Becoming (more) familiar with OpenCV, ROS, (Arduino,) Baxter(/Ridgeback), getting robot/robot arms to move
+
−
 
+
−
•(February) Basic literature review; last preparation, such as attaching secondary systems which might be needed such as touch sensors to the robot
+
  
−
•Main part: (March-April) Sensing, Planning, Acting
+
January-February: Preparation: literature review; setting up basic capability for a robot to do a simple cooking task (formulate instructions from a recipe, detect tools and ingredients, and carry out instructions)
−
For example:
+
−
(sense) the robot will use its sensors to locate important first aid points in 3d
+
−
(plan) the robot will use inverse kinematics to position its end effectors to do first aid
+
−
(act) the robot will move to do first aid, controlling its motions via feedback from sensors
+
  
−
•After-work: (May) Evaluating; writing/presenting
+
March-April: Main point:learning a common sense model to avoid errors
  
−
Possible evaluation metrics: e.g., objective measures (time, positioning, force); qualitative assessment by an expert; and/or comparison to a novice human first aider
+
May: Evaluation, writing/presenting
  
−
Expected results: a thesis/report, code, video  
+
Expected results: a thesis/report, code, video (we also hope to offer some food cooked by the robot)
−
(for this project, the student is expected to be willing to also write a six page shortened version of the thesis, to be submitted to a conference)
+

Latest revision as of 13:09, 28 November 2016

Title Robot Cooking
Summary Common sense for a robot to cook healthy food
Keywords Robots, Healthcare, Visual Recognition
TimeFrame 2017/1/1-2017/8/30
References -robot cooking

Christian Østergaard Laursen, Søren Pedersen, Timothy Merritt, Ole Caprani. Robot-Supported Food Experiences: Exploring Aesthetic Plating with Design Prototypes. In J.T.K.V. Koh et al. (Eds.): Cultural Robotics 2015, LNAI 9549, pp. 107–130, 2016. DOI: 10.1007/978-3-319-42945-8 10 Springer International Publishing Switzerland 2016.012.241

-common sense acquisition

Rakesh Gupta, Mykel J. Kochenderfer. Common Sense Data Acquisition for Indoor Mobile Robots. ROBOTICS

Prerequisites Strong multidisciplinary interest, strong work ethic, software (also ability to work with libraries), possibly some small work with hardware/electronics
Author Chandrashekhar Shankarrao Nasurade, Vamsi Krishna Nathani
Supervisor Martin Cooney
Level Master
Status Ongoing


This project will be about designing a capability for "common sense" in a robot, within the context of helping an elderly person at home with cooking. Our motivation is that robots could be helpful to people and contribute to their well-being, health, and quality of life, but first some major challenges must be overcome. One challenge is that everyday tasks which humans perform can be complex and confusing. This is especially a problem with elderly persons with declined physical and cognitive abilities (e.g. dementia), when the person can no longer function by themselves but requires support, which is sometimes not available from other humans. To support such a person, we believe a robot should have some degree of what we call here "common sense"; related to robustness, this is an ability to function correctly in the presence of some errors which a healthy adult person can typically detect. In the case of cooking, to cook in a healthy and good way, this means that a robot should be able to detect and compensate for errors in three main facets of cooking: the recipe, the tools used, and the ingredients. For example, a human can determine that if a recipe for one person calls for 10kg of salt, this is probably a mistake, and conclude from experience with similar recipes that it should be 10g of salt. If a recipe calls for a spatula but instead a knife has been provided, a human can determine that this is wrong and seek from experience some tool which is more appropriate. If instead of salt, a bag of sugar has been provided, a human can determine that this is wrong and seek more appropriate ingredients. To address the challenge, in this project the robot will learn a model of common sense regarding these three main facets of cooking by unsupervised learning (forming clusters, detecting anomalies, and inferring how anomalies can be rectified). The robot used will be Baxter on a Ridgeback mobile base, which will be shared with other students and researchers. The evaluation will measure the degree to which the robot can complete a simple cooking task in the presence of various errors which we introduce into the cooking process.

Timeline:

January-February: Preparation: literature review; setting up basic capability for a robot to do a simple cooking task (formulate instructions from a recipe, detect tools and ingredients, and carry out instructions)

March-April: Main point:learning a common sense model to avoid errors

May: Evaluation, writing/presenting

Expected results: a thesis/report, code, video (we also hope to offer some food cooked by the robot)

Retrieved from "https://wiki.hh.se/caisr/index.php?title=Robot_Cooking&oldid=3385"