Thermal Detection of Subtle Human Cues for a Robot Magic Performance
| Title | Thermal Detection of Subtle Human Cues for a Robot Magic Performance |
|---|---|
| Summary | Thermal Detection of Subtle Human Cues for a Robot Magic Performance (NOT AVAILABLE HT22/VT23) |
| Keywords | |
| TimeFrame | |
| References | Martin Cooney, & Alexey Vinel. “Magic in Human-Robot Interaction (HRI).” In the 34th annual workshop of the Swedish Artificial Intelligence Society (SAIS 2022), 2022.
Cho, Y., Bianchi-Berthouze, N., Marquardt, N., & Julier, S. J. (2018, April). Deep thermal imaging: Proximate material type recognition in the wild through deep learning of spatial surface temperature patterns. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (pp. 1-13). Xu, Z., Wang, Q., Li, D., Hu, M., Yao, N., & Zhai, G. (2020). Estimating departure time using thermal camera and heat traces tracking technique. Sensors, 20(3), 782. Cooney, M., & Bigun, J. (2017). PastVision+: Thermovisual inference of recent Medicine intake by Detecting heated Objects and cooled lips. Frontiers in Robotics and AI, 4, 61. |
| Prerequisites | |
| Author | (NOT AVAILABLE HT22/VT23) |
| Supervisor | Martin Cooney |
| Level | Master |
| Status | Draft |
(THIS PROJECT WILL NOT BE AVAILABLE FALL 2022/SPRING 2023 DUE TO PARENTAL LEAVE.)
Generations have been entertained and inspired by mentalist magicians and fantastical accounts of detectives like Sherlock Holmes, who draw extensive conclusions from minute observations.
As AI rapidly becomes more advanced, robots will one day become able to leverage superhuman abilities in sensing and calculation to infer in such a way from subtle cues, to better help people, quickly recognize what people want and providing it in a good way.
But in 2022, most robots are still remarkably "dense":
As a simple example, if one goes up to a Pepper robot and taps it on the head, the odds are that it will not react.
This kind of lack of observation and interactivity acts as a barrier to acceptance of robots, which can be seen in the failures of various robot start up companies.
A speculative prototyping approach will be followed:
- speculative step: a list of potentially useful subtle cues will be compiled (e.g., reflections, shadows, etc.) and proposals made on how they could be implemented in robots.
For this, some review of potential recognition (e.g., DL) techniques will be required.
- prototyping step: the use of current methods to recognize and leverage one subtle cue will be explored.
This will probably involve Deep Learning (DL) to carry out inference on data from a thermal camera in some challenging situation. This could possibly involve detection of light touches, at distance, multiple cameras, etc. Furthermore, the recognition capability will be incorporated into the interaction design for a robot, within the application area of robot magic, to carry out a simplified magic performance.
Expected outcomes: a thesis/report, code, video. Ideally the results should be sufficient to form the basis for a paper, and we will hopefully participate in an International Robot Magic Competition.
