Research Engineer: Detailed riggable humans from multi-view video

Type de contrat : CDD Contexte et atouts du poste Within the framework of a partnership  • The project's findings will contribute to a follow up 4-year BPI transfer project (Banque Publique d'investissement) with a consortium of French production companies. The project is liable to begin during this contract period, with discussions with involved partners Mission confiée Context Many works nowadays provide solutions for the avatarization process, i.e. obtaining a 3D, animatable model from one or several images. This is a very hard problem as shape fidelity, animatability, fast computation time and low number of required input cameras are all desirable, but hardly realizable simultaneously. For example, obtaining plausible models from a single camera video is nowadays feasable, but often at the cost of shape quality due to the use of low-parametric models such as SMPL. Using many videos for redundancy can allow to acquire more detail, but at the expense of computation speed. And all this detail needs to be animatable, which gets more complex with the scale of detail (i.e. millimeter shape with only a human kinematic rig), again putting a burden on the model and its computation time. Mission In recent years the Morpheo team has come up with very precise multi-view reconstruction approaches [1]. In this project, we wish to examine the problem of animating this type of detailed model and rig it, by exploring the stream of recent methods. Principales activités During his activity, the candidate is expected to tackle the following tasks • establish a more complete bibliography of relevant methods based on the initial suggested references • propose and discuss likely and realizable methodological and architecture innovations / reparametrizations that allow to rig and estimate a detailed animated model from images, grounded in this existing work • propose and discuss dataset enhancements that would enrich the training toward better performance for these tasks • demonstrate a proof of concept implementation of an envisioned solution discussed with team research advisers • identify existing datasets that are relevant for comparative evaluation of performance of his proposals. In-house datasets such as 4DHumanOutfit[2] Inria, l'institut national de recherche dans les sciences et technologies du numérique, est en appui de l’État pour les stratégies nationales de recherche et d’innovation du numérique en tant qu'Agence de programmes. Inria mène plus de 300 projets de recherche et d’innovation avec ses 3500 scientifiques, ingénieurs et personnels d’appui, en partenariat avec les universités et l’écosystème numérique (entreprises, entrepreneurs, acteurs publics). Ensemble, nous explorons des domaines clés comme l'intelligence artificielle, la cybersécurité, l’informatique quantique, le Cloud, la transformation numérique de la santé, les jumeaux numériques ou encore les technologies numériques pour la défense. Nous construisons des solutions concrètes telles que des logiciels, des startups technologiques, des partenariats avec les entreprises du tissu national et des formations de pointe. Notre objectif : l’impact scientifique, technologique et industriel au service de la souveraineté numérique de la France.