Rethinking genre classification with fine grained semantic clustering

Ed Fish, Jon Weinbren , Andrew Gilbert,
Unviersity of Surrey
IEEE International Conference on Image Processing (ICIP), 2021
Paper

We expand these ‘coarse’ genre labels by identifying ‘fine-grained’ contextual relationships within the multi-modal content of videos. By leveraging pretrained ‘expert’ networks, we learn the influence of different combinations of modes for multi-label genre classification.

Video Presentation

Abstract

Movie genre classification is an active research area in machine learning; however, the content of movies can vary widely within a single genre label. We expand these ‘coarse’ genre labels by identifying ‘fine-grained’ contextual relationships within the multi-modal content of videos. By leveraging pretrained ‘expert’ networks, we learn the influence of different combinations of modes for multi-label genre classification. Then, we continue to fine-tune this ‘coarse’ genre classification network self-supervised to sub-divide the genres based on the multi-modal content of the videos. Our approach is demonstrated on a new multi-modal 37,866,450 frame, 8,800 movie trailer dataset, MMX-Trailer-20, which includes pre-computed audio, location, motion, and image embeddings.

An overview of the approach. Image and audio features from movie trailers are extracted and their influence is learnt via a collaborative weighting to classify broad genres such as Action, Adventure and Sci-Fi. A self-supervised network then compares these embeddings to generate contextually appropriate sub-genres.

Poster

BibTeX

@inproceedings{Fish:ICIP:2021,
        AUTHOR = Fish, Ed and  Weinbren, Jan and  Gilbert Andrew",
        TITLE = "Rethinking genre classification with fine grained semantic clustering",
        BOOKTITLE = "IEEE International Conference on Image Processing (ICIP),	2021",
        YEAR = "2023",
        ​}
Rethinking genre classification with fine grained semantic clustering E Fish, J Weinbren, A Gilbert 2021 IEEE International Conference on Image Processing (ICIP), 1274-1278 11* 2021 [Rethinking_genre](https://andrewjohngilbert.github.io/Rethinking_genre/)