Detection and intention prediction of pedestrians in zebra crossings

Title Detection and intention prediction of pedestrians in zebra crossings
Summary Detection and intention prediction of pedestrians in zebra crossings
Keywords Deep Learning, pedestrian detection, facial analysis
TimeFrame October 2017 to June 2018
References Computer Vision Datasets:

Computer Vision Resources: Caffee Model Zoo: R. Klette, “Concise Computer Vision”, Springer, 2014

Prerequisites Good knowledge of applied mathematics, signal and image processing. Programming skills (preferably Matlab).
Author Dimitrios Varytimidis
Supervisor Fernando Alonso-Fernandez, Cristofer Englund, Boris Duran
Level Master
Status Finished

Keywords: Deep Learning, pedestrian detection, facial analysis


A key issue for semi- and fully-autonomous cars is safety with respect to pedestrians. To this end, fully automated solutions for pedestrian detection and behaviour modelling are necessary, especially in zebra crossings and other areas where pedestrians may have priority, or traffic lights regulate vehicles and people flow.

In this project we aim to model pedestrian behaviour and intention in zebra crossings from videos, in order to estimate if the pedestrian will stop or if s/he will continue crossing. The aim is to incorporate such information into semi- or fully-autonomous vehicles, which will raise opportune alarms and ultimately, will stop the car if deemed opportune.

Project description

The project is divided into three parts: (1) data gathering, (2) pedestrian detection, and (3) pedestrian behaviour modelling.

The use case will be a car approaching a zebra crossing, with pedestrians approaching at the same time. A part of the project will be devoted to gather relevant data (videos) of pedestrians approaching zebra crossings available from other projects. Students will be required also to investigate state of the art models and features for pedestrian detection and behaviour analysis, which can include (among others): face expression, face pose, eye gaze, hand movements, or body motion. We specially seek to apply deep learning techniques in order to solve these tasks. Ultimately, the target will be to obtain reliable predictions using proposed models indicating whether the pedestrian will cross/will not cross the zebra crossing.

Research questions

  • Which features extracted from pedestrian videos are relevant to predict the behaviour?
  • What characterize a pedestrian that will stop vs. a pedestrian that will not stop?
  • How to combine such features to increase reliability?


Halmstad University RISE Viktoria

Contact: Fernando Alonso-Fernandez (, Cristofer Englund (, Boris Duran (

Resources & References