Lorenzo Baraldi

Lorenzo Baraldi is a researcher in vision-and-language AI at the University of Modena and Reggio Emilia, where he works in the AImageLab group. After a 2017 internship at Facebook AI Research in Paris, where he worked on a video-matching algorithm adopted in production, he earned a Ph.D. cum laude in ICT from Modena in 2018. With Marcella Cornia and Rita Cucchiara, he coauthored the captioning models Show, Control and Tell, accepted at CVPR 2019, and the Meshed-Memory Transformer, which ranked first on the COCO captioning leaderboard in 2019. He was elected an ELLIS Scholar in 2021, and in 2024 he and his group introduced the LLaVA-MORE multimodal models. Since 2024, he is an associate professor at Modena, where he coordinates the Modena ELLIS Unit and leads work package 2, "Foundation Models", of the EU project ELLIOT.

Now at: University of Modena and Reggio Emilia and European Laboratory for Learning and Intelligent Systems
Badges: Bridge

Timeline

2017

An internship at Facebook AI Research

From July to October 2017, he interns at Facebook AI Research in Paris under Hervé Jégou. There he helps develop a video-matching algorithm that is adopted in production on the social network to detect abusive content.
2018

A doctorate at Modena

He receives a Ph.D. cum laude in ICT from the University of Modena and Reggio Emilia in 2018. His paper on Temporal Match Kernels is accepted at CVPR 2018.
2019

Show, Control and Tell

Written with Marcella Cornia and Rita Cucchiara, Show, Control and Tell is a captioning framework that generates descriptions grounded on image regions chosen by a control signal. The paper is accepted at CVPR 2019.
2019–2020

The Meshed-Memory Transformer

He coauthors the Meshed-Memory Transformer for image captioning with Marcella Cornia, Matteo Stefanini and Rita Cucchiara; the paper appears at CVPR 2020. In December 2019, the architecture ranks first on the COCO captioning leaderboard with a CIDEr-D of 1.321.
2021

NVIDIA spotlight and ELLIS Scholar

NVIDIA's Meet the Researcher series profiles him in February 2021 as an assistant professor at the University of Modena and Reggio Emilia and an Associate Editor of Pattern Recognition Letters. In 2021, he is elected a Scholar of ELLIS, the European Laboratory for Learning and Intelligent Systems.
2022

Best paper at CBMI 2022

A paper from his lab, "Retrieval-Augmented Transformer for Image Captioning", is selected as best paper at the International Conference on Content-based Multimedia Indexing (CBMI 2022). In 2022, he also gives a research seminar at the Bibliotheca Hertziana on multimodal and embodied AI for digital humanities.
2024–2025

LLaVA-MORE multimodal models

In August 2024, he and his group introduce LLaVA-MORE, a family of models that enhances LLaVA by integrating LLaMA 3.1 as the language model. In 2025, he coauthors the LLaVA-MORE paper with Federico Cocchi, Nicholas Moratelli, Davide Caffagni, Sara Sarto, Marcella Cornia and Rita Cucchiara, a study of language models and visual backbones for visual instruction tuning.
Since 2024

Associate professor and ELLIOT lead

Since 2024, he is an associate professor at the University of Modena and Reggio Emilia, where he works in AImageLab on deep learning, vision-and-language integration and large-scale models, and he coordinates the Modena ELLIS Unit. He obtains the Italian National Habilitation as Full Professor in Information Processing Systems in 2025. In July 2026, he takes on leading WP2, "Foundation Models", of the Horizon Europe project ELLIOT, and in October 2026 his project PREMISE is funded under PRIN 2026 with him as principal investigator.