PhD Position F/M Physically-based skin models for 3D reconstruction and temporal tracking from multi-video observations

Le descriptif de l’offre ci-dessous est en Anglais

Type de contrat : CDD

Niveau de diplôme exigé : Bac + 5 ou équivalent

Fonction : Doctorant

A propos du centre ou de la direction fonctionnelle

The Centre Inria de l’Université de Grenoble groups together almost 600 people in 26 research teams and 8 research support departments.

Staff is present on three campuses in Grenoble, in close collaboration with other research and higher education institutions (Université Grenoble Alpes, CNRS, CEA, INRAE, …), but also with key economic players in the area.

The Centre Inria de l’Université Grenoble Alpes is active in the fields of high-performance computing, verification and embedded systems, modeling of the environment at multiple levels, and data science and artificial intelligence. The center is a top-level scientific institute with an extensive network of international collaborations in Europe and the rest of the world.

Contexte et atouts du poste

We propose a PhD subject combining interests in computer vision, computer graphics, surface and appearance modeling, exploring direct applicability of the findings to the field of medical research.
In particular, this PhD investigates scalable, physically-based encoding of skin radiance and SDF surfaces to enable high-fidelity 3D reconstruction and temporal tracking from multi-view video.
The topic targets explainable skin radiance models, targeting contributions both in fundamentals of computer vision, geometric and photometric modeling, where interpetable skin parameters and the high quality geometry resulting from improved modeling can unlock groundbreaking, non-invasive techniques for patient diagnosis and monitoring. In particular, we will be looking at applying such results in the context of a longstanding collaboration with CHU Grenoble, for the benefit of scoliosis patients.

Adolescent Idiopathic Scoliosis (AIS) is a three-dimensional deformity of the spine affecting between 1 and 4% of the population [1]. It can affect the mobility of the patient restricting the spine torsion and bendings. At the start of puberty, half of mild to moderate scoliosis progress to severe scoliosis. These patients are treated by the means of bracing or, in the very severe cases, with surgery. Scoliosis grading is done with a single angle measurement (Cobb [2]). While highly adopted, the Cobb measurement reduces a highly complex 4D phenomenon – the patient anatomic and kinematic change over time – to a few numbers, making it difficult for clinicians to understand how the 3D deformity evolves, how effective the treatments are and their consequences on the patient’s mobility and overall wellbeing.

Accurately reconstructing the 3D shape and motion of patients from non-invasive measurements, such as multi-view videos, presents thus an opportunity to quantify how the 3D deformity evolves.

Mission confiée

This PhD will explore new encodings of the SDF-grid based on a combination of factored tensors [A,B] and efficient factored representations of the radiance function, rooted in analytical Gaussian lobe decompositions of the radiance function as for [C,D]. The former have been used to model opacity fields in the context of volumetric NeRF, but seldom applied to SDF-based rendering approaches. We believe the latter can be significantly simplified to the point of eliminating most neural components to focus on physically accurate, interpretable radiance parameterizations, in a way that can be combined with spatially efficient encodings. Recent work [D] shows a promising lead to obtain high precision reconstructions with more explainable and complete surface radiance models. But their approach does not easily scale to real-world complex human data, with a very high compute overhead. Most interestingly, simplified but expressive models of skin subscattering have been proposed on the basis of dipole (2-lobe) angular Gaussian parametrizations [30], allowing to explicitly model different Fitzpatrick skin indices [F]. We thus postulate that a natural and unified Gaussian-lobe parametrization of light interaction exists and would simultaneously lead to a sparse, lightweight, relightable and differentiable representation of the scene that could complement current surface estimation algorithms, and ultimately drastically improve their performance with natural scenes containing humans. Our intuition is also that such a proposition would easily lend itself to scalable implementations able to reach millimetric detail with enhanced reconstruction performance due to better radiance model expressivity. The scalability can be achieved by using recent factored models projecting coordinates of a 3D query point on lower dimensional spaces such as planes [A,G,H], which have been recently generalized to a more versatile framework [B] with multiscale capabilities and yet very simple implementations. This multiscale capability is particularly interesting to
encode hierarchical feature sets based on sparse Gaussian lobe sets that could be combined over various spatial levels in the hierarchy for improved expressivity. Various novel research contributions will be proposed and explored on the basis of such an encoding to optimize pipeline decoding, ray-batching, ray-marching, appearance and color decoding benefiting from this new targeted combination of models.

High Performance Computing (HPC) methods will be studied to overcome the computational complexity burden.

[A] Chen, Anpei, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. "Tensorf: Tensorial radiance fields." In European conference on computer vision, pp. 333-350. Cham: Springer Nature Switzerland, 2022.

[B] Chen, Anpei, Zexiang Xu, Xinyue Wei, Siyu Tang, Hao Su, and Andreas Geiger. "Dictionary
fields: Learning a neural basis decomposition." ACM Transactions on Graphics (TOG) 42, no. 4 (2023): 1-12.

[C] Wang, Jiaping, Peiran Ren, Minmin Gong, John Snyder, and Baining Guo. "All-frequency rendering of dynamic, spatially-varying reflectance. " In ACM SIGGRAPH Asia 2009 papers, pp.1-10. 2009.

[D] Fan, Yue, Ningjing Fan, Ivan Skorokhodov, Oleg Voynov, Savva Ignatyev, Evgeny Burnaev, Peter Wonka, and Yiqun Wang. "Factored-neus: Reconstructing surfaces, illumination, and materials of possibly glossy objects. " In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 21317-21327. 2025.

[E] Donner, Craig, and Henrik Wann Jensen. "Light diffusion in multi-layered translucent materials." ACM Transactions on Graphics (ToG) 24, no. 3 (2005): 1032-1039.

[F] Fitzpatrick, Thomas B. "The validity and practicality of sun-reactive skin types I through VI." Archives of dermatology 124, no. 6 (1988): 869-871.

[G] Fridovich-Keil, Sara, Giacomo Meanti, Frederik Rahbæk Warburg, Benjamin Recht, and Angjoo Kanazawa.
"K-planes: Explicit radiance fields in space, time, and appearance. " In Proceedings of the IEEE/CVF Conference on CVPR, pp. 12479-12488. 2023.

[H] Cao, Ang, and Justin Johnson. "Hexplane: A fast representation for dynamic scenes." In
Proceedings of the IEEE/CVF Conference on CVPR, pp. 130-141. 2023.

Principales activités

• Developing physics-based 3D reconstruction algorithms from images and videos
• Designing mathematical models for geometry, lighting, materials, and camera behavior
• Implementing and optimizing computer vision pipelines using tools such as Python, PyTorch, CUDA, and OpenCV
• Running experiments, training models, and evaluating reconstruction accuracy on real or synthetic datasets
• Reading scientific literature, maintaining bibliography databases, and conducting state-of-the-art reviews
• Writing scientific papers, preparing figures/results, and submitting work to conferences and journals
• Presenting research in meetings, conferences, and collaborations with other researchers or industry partners

Compétences

Mandatory:

Master degree in applied mathematics or computer science or physics;

Strong theoretical knowledge in mathematics/Physics/Engineering;

Methodological knowledge in paper writting and programming

Stron coding skills (Python, PyTorch and/or Tensor Flow)

English (B2 or C1);

The ideal candidate should have preliminary experience in: image processing - machine learning - 3D reconstruction - physics-based differentiable models - temporal series.

• A specific section in the application letter must briefly describe the personal experience in these areas.

Avantages

See benefits offered by UGA

Rémunération

According to UGA reglementation

Employer : Grenoble Alpes University

Informations générales

• Thème/Domaine : Vision, perception et interprétation multimedia

  Systèmes d'information (BAP E)

• Ville : Montbonnot

• Centre Inria : Centre Inria de l'Université Grenoble Alpes

• Date de prise de fonction souhaitée : 2026-10-01

• Durée de contrat : 3 ans

• Date limite pour postuler : 2026-06-20

Attention: Les candidatures doivent être déposées en ligne sur le site Inria. Le traitement des candidatures adressées par d'autres canaux n'est pas garanti.

Consignes pour postuler

Applications must be submitted online via the Inria website.

Applications must include a CV, motivation letter, recommendation letter(s), if available evaluation reports.

Sécurité défense :
Ce poste est susceptible d’être affecté dans une zone à régime restrictif (ZRR), telle que définie dans le décret n°2011-1425 relatif à la protection du potentiel scientifique et technique de la nation (PPST). L’autorisation d’accès à une zone est délivrée par le chef d’établissement, après avis ministériel favorable, tel que défini dans l’arrêté du 03 juillet 2012, relatif à la PPST. Un avis ministériel défavorable pour un poste affecté dans une ZRR aurait pour conséquence l’annulation du recrutement.

Politique de recrutement :

Dans le cadre de sa politique diversité, tous les postes Inria sont accessibles aux personnes en situation de handicap.

Contacts

• Équipe Inria : KINETIX

• Directeur de thèse :

  Pujades Sergi / sergi.pujades-rocamora@inria.fr

L'essentiel pour réussir

Observe, Think, Act

A propos d'Inria

Inria est l’institut national de recherche dédié aux sciences et technologies du numérique. Il emploie 2600 personnes. Ses 215 équipes-projets agiles, en général communes avec des partenaires académiques, impliquent plus de 3900 scientifiques pour relever les défis du numérique, souvent à l’interface d’autres disciplines. L’institut fait appel à de nombreux talents dans plus d’une quarantaine de métiers différents. 900 personnels d’appui à la recherche et à l’innovation contribuent à faire émerger et grandir des projets scientifiques ou entrepreneuriaux qui impactent le monde. Inria travaille avec de nombreuses entreprises et a accompagné la création de plus de 200 start-up. L'institut s'efforce ainsi de répondre aux enjeux de la transformation numérique de la science, de la société et de l'économie.