Post-Doctoral Research Visit – Postdoc in Foundation Models for Human-Robot Interaction

Type de contrat : CDD Contexte et atouts du poste The HUCEBOT team is dedicated to advancing algorithms for human-centered robots: robots that are not working autonomously in isolation, but that instead react, interact, collaborate, and assist humans. To do so, these robots need to intertwine a multi-contact whole-body controller, a digital simulation of the interacting humans, and machine learning models to predict and respond to human movements and intentions. In a crescendo of complexity, the team  tackles scenarios that involve collaboration with cobots, assistance with exoskeletons, and collaboration with humanoid robots. The application domains span from industrial robotics to space teleoperation. The main robots of the team are the mobile robot Shelfy, TiagoPro bimanual mobile manipulator, the Unitree G1 humanoid, and the Talos humanoid robot. The team also works with Franka cobots and exoskeletons. The team currently consists of about 20+ members, including permanent researchers, PhD students and post-doctoral students. Serena Ivaldi, head of HUCEBOT, is holding the chair in Robotics and AI of the Cluster IA ENACT project (https://cluster-ia-enact.ai/) that is funding this PhD thesis. In the chair, she wants to push the research in Natural Language to assist humans in different scenarios of collaboration with robots, where safety is paramount. The ambition is to create a foundation that bridges natural language commands into interpretable commands for the robot, leading to robot actions that are contextualized and intrinsically safe. Mission confiée Recent advancements in foundation models in VLMs and VLAs has drastically improved the contextual understanding and motion control for simple tasks. They allow the robot to execute plans and execute a task from visual input alone. However, there are known problems of trajectory hallucinations, safety and scaling to the long horizon problems. These problems make it difficult to make robust and reliable autonomous robot architectures. Tuning these generalized architectures and combining them with classical motion planning and control significantly improves the success rate.  When the robots have to work along with humans, strong safety guarantees are required along with clear understanding of how to interact with humans. We want to exploit the contextual understanding and action planning skills of VLMs and VLAs for motion planning in HRI with focus on safety, interaction and communication modelling and environmental adaptability. Socially aware robot navigation across various kinds of robots and environments will be the application area of this research position. This postdoctoral researcher will be funded by the ENACT project, but will contribute more generally to the research of the team in robot learning, HRI. Principales activités Main activities: implement, test and develop novel algorithms for real robots that use language models and foundation models. Write papers and present them at conferences. Write, test, validate and document its associated software. Experiments with real robots are mandatory. This position involves contributing in the supervision of the team engineers and PhD students, together with the supervisor. The postdoc will also be involved in the activities organized by the Cluster-AI project ENACT, which may involve dissemination actions, meetings and presentations to relevant stakeholders (Europe, France, industries, etc). Compétences Good skills in Python (Pytorch). Ideally, prior experience with LLM, VLM and Foundation Models. Good knowledge of robotics. Languages: English (English is the official language of the team and many members do no speak French). Proactivity and curiosity, daily communication, ability to work in a team are fundamental. Inria, l'institut national de recherche dans les sciences et technologies du numérique, est en appui de l’État pour les stratégies nationales de recherche et d’innovation du numérique en tant qu'Agence de programmes. Inria mène plus de 300 projets de recherche et d’innovation avec ses 3500 scientifiques, ingénieurs et personnels d’appui, en partenariat avec les universités et l’écosystème numérique (entreprises, entrepreneurs, acteurs publics). Ensemble, nous explorons des domaines clés comme l'intelligence artificielle, la cybersécurité, l’informatique quantique, le Cloud, la transformation numérique de la santé, les jumeaux numériques ou encore les technologies numériques pour la défense. Nous construisons des solutions concrètes telles que des logiciels, des startups technologiques, des partenariats avec les entreprises du tissu national et des formations de pointe. Notre objectif : l’impact scientifique, technologique et industriel au service de la souveraineté numérique de la France.