VU Researcher Develops Models to Assess the Emotional Impact of Art and Advertising

Human emotions are subjective and shaped by a wide range of factors, yet researchers continue to explore ways to translate this complex phenomenon into a form computers can understand. One of them is Dr Modestas Motiejauskas from the Faculty of Mathematics and Informatics of Vilnius University (VU), who successfully defended his doctoral dissertation and developed models capable of recognising visual emotions. According to him, such models could help us better understand the emotions evoked by works of art and enable marketing specialists to evaluate how brands affect their audiences.

Where psychology meets computer science

A physical disability did not prevent Dr Motiejauskas from completing his Bachelor’s, Master’s, and doctoral studies. ‘My studies were pretty typical, just like those of any other student. Lectures, seminars, coursework, and exams – the familiar path followed by all students.’

He explains that his dissertation topic – visual emotion recognition in general-purpose images – is a relatively new and challenging field of research. 

‘We can effectively and reliably recognise, classify, or segment physical objects because they have clearly defined distinguishing features. The interpretation and perception of emotions, however, depend on human subjectivity, as well as psychological and cultural factors,’ said the researcher.

According to him, this means that we cannot determine with certainty which emotion a particular image conveys: ‘For example, emotions such as contentment, amusement, and awe are visually similar and reflect closely related emotional states. The same applies to negative emotions such as fear and sadness. Although they differ in nature, these emotions can visually look quite similar.’

Dr Modestas Motiejauskas. Photo by Vilnius University.

Despite the complexity of the topic, Dr Motiejauskas combined two disciplines – psychology and computer science – to develop models capable of recognising emotions in general-purpose images.

‘The model receives an image as input and predicts the emotion that best matches the labelled examples in its training data. It is trained using annotated images, and new images are then assigned to one of the predefined emotion categories. This operating principle is a typical example of a classification task,’ explained the VU researcher.

A model for content recommendation

According to Dr Motiejauskas, such models could be applied in many areas – from content recommendation to the analysis of artworks and marketing materials.

‘The model could be used to recommend images, works of art, or other visual content based on their emotional characteristics. It could also help computer systems better understand the emotional impact that displayed images have on users and adapt the content it presents accordingly,’ he said.

‘Another important area is art. The model could help shed light on the emotions conveyed by or evoked through works of art, as well as evaluate the emotional impact of advertising and brand imagery on audiences,’ the researcher added.

Doctoral thesis defence. Photo by Vilnius University.

Reflecting on his experience at VU, Dr Motiejauskas says he particularly values the supportive environment, the continuous guidance he received from his supervisor, Prof. Gintautas Dzemyda, and the support provided by the teaching staff.

‘I am grateful to Vilnius University for giving me the opportunity to pursue and complete my Bachelor’s, Master’s, and doctoral studies. I am also grateful to the Doctoral Studies Committee for the Field of Informatics for allowing me to participate in my studies and complete assessments remotely,’ he concluded.