Zero-Shot Learning for Semantic Segmentation
| Title | Zero-Shot Learning for Semantic Segmentation |
|---|---|
| Summary | Zero-Shot Learning for Semantic Segmentation |
| Keywords | Deep Learning, Computer Vision, Semantic Segmentation |
| TimeFrame | |
| References | https://arxiv.org/pdf/1707.00600.pdf
https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/41473.pdf |
| Prerequisites | Excellent Programming Skills; Excellent Grasp of Deep Learning and Pytorch |
| Author | Tiago Cortinhal |
| Supervisor | Tiago Cortinhal, Eren Erdal Aksoy |
| Level | Master |
| Status | Open |
Currently we are working of a GAN approach to be able to generate segmentation maps from a sensor modality to another (from/to LiDAR and RGB).
The currently available datasets are not perfectly aligned (SemanticKitti and Cityscape), to try to overcome this problem techniques like Zero-Shot Learning could be applied.
Zero-Shot Learning can be seen as a type of Domain Adaptation, where a given set of classes are used for training and we have another set of unseen classes to which we wish to have segmentation as well.
To be able to perform this, a kind of embedding of the classes need to be done, as to be able to adapt the pre-trained model to unseen classes by combining the embeddings of the learnt classes.
The project consists of using the pre-trained GAN generator model and apply Zero-Shot Learning.
Research Questions: Can Zero-Shot Learning besides of learning unseen classes also improve the overall IoU (Intersection over Union)? Should Zero-Shot or N-Shot be applied in this scenario (N-Shot: N examples are showed during training) Compared with Domain transfer/finetuning techniques which could provide faster/better results?
