Paul Maria Scheikl

I am an applied scientist at Amazon Robotics in Berlin, working on the Vulcan project. I was a postdoc at Johns Hopkins University working under the guidance of Axel Krieger in the Laboratory for Computational Sensing and Robotics (LCSR). I received my PhD (Dr.-Ing.) from the Department of Artificial Intelligence in Biomedical Engineering at the Friedrich-Alexander University Erlangen-Nürnberg in Germany, where I was advised by Franziska Mathis-Ullrich.

Email  /  Scholar  /  Github  /  Doctoral Thesis

Research

My research interests lie at the intersection of robotics, computer vision, and machine learning. I am particularly interested in learning visumotor policies for complex object manipulation tasks. During my doctoral studies, I worked on imitation learning, reinforcement learnig, semantic segmentation, deformable object simulation, and sim-to-real transfer.

Publicly available multimodal dataset for imitation learning of long-horizon surgical workflows. Provides over 18,000 demonstrations from 34 ex-vivo porcine cholecystectomies (≈20 hours), segmented into 17 distinct tasks during the clipping and cutting phase. Combines endoscopic video from multiple viewpoints with da Vinci Research Kit kinematics, and includes both optimal executions and recovery maneuvers for robust error recovery.

Real-time reconstruction of deformable tissue from monocular endoscopic video. Constrains 3D Gaussian primitives along the view-aligned axis to form surface-aligned elliptical splats, sharpening tool-occlusion handling and preserving fine anatomical details over prior 3DGS- and NeRF-based methods. A lightweight Fully Fused Deformation MLP predicts surfel motion fields up to 5x faster than a standard MLP.

Learns a hierarchical policy for autonomous surgery using language-conditioned imitation learning. The high-level policy receives image data and produces a natural language instruction. The low-level policy receives the language instruction and image data and generates robotic motions. The framework is evaluated on the da Vinci Research Kit on ex-vivo porcine organs for laparoscopic cholecystectomy.

Presents the first robotic system to demonstrate autonomous clip positioning on a physical phantom in laparoscopic surgery. The system segments a colorless point cloud from a single camera, extracts target positions for the clips using spline interpolation, and allows human operators to adjust the targets. The segmentation model is trained on only 60 hand-labeled real point clouds, reflecting data scarcity in the surgical domain. The system achieves a localization precision of 0.75 mm at a 95% success rate and executes autonomous clip positioning with a 100% success rate.

Vision-guided, autonomous approach for palliative resection of tracheal tumors. Models the tracheal surface with a fifth-degree polynomial to plan tool trajectories. Aegmentation pipeline identifies the trachea and tumor boundaries.

Reconstructs a complete deformable object, including deformed internal structures, from a single point cloud observation. We use the reconstructed object to plan a robotic path to puncture internal regions of interest. Deformable object reconstruction eliminates the need for deformable object registration!

A 3D mapping pipeline that uses only RGB images to create segmented point clouds of the target anatomy, removing the need for bulky depth cameras in space-limited surgical settings. Compares structure-from-motion algorithms for mapping central airway obstructions and shows that monocular reconstruction can match -- and in some metrics exceed -- RGB-D performance in a downstream tumor resection task.

Differentiable rendering approach to estimate surgical tool kinematics from monocular video, eliminating the need for ground-truth kinematic recordings. Tool trajectories and joint angles are inferred by minimizing the discrepancy between rendered and observed images. Imitation learning policies trained on estimated kinematics achieve success rates comparable to those trained on measured kinematics, opening a path to large-scale learning from online surgical videos.

Couples a learned tissue retraction policy with a model-based resection planner for automated tumor removal in central airway obstruction surgery. The retraction network exposes the surgical target so the planner can compute resection trajectories on the visible tissue.

Image segmentation that produces visual observations to support feedback control of an articulated endoscope.

Demonstrates simulation and policy learning of lens capsule tearing in cataract surgery using reinforcement learning.

Combines the versatility of diffusion-based imitation learning with the high-quality motion generation capabilities of Probabilistic Dynamic Movement Primitives. Achieves gentle manipulation of deformable objects, while maintaining data efficiency critical for surgical applications where demonstration data is scarce. Evaluated in simulation and on real robotic hardware.

3D reconstruction and segmentation of deformable objects from a single view point cloud. Also introduces a simple sampling algorithm to generate better training data for occupancy learning.

Defines a surgical process model that captures intraoperative activities -- actor, instrument, action, target -- in laparoscopic cholecystectomy. Activities, instrument presence, and surgical phases are annotated across 20 videos of human and ex-vivo porcine procedures (~25 hours). A Distilled-Swin transformer is trained on the dataset for automatic action triplet recognition, providing a foundation for context-aware collaborative surgical robots.

Reinforcement learning framework for robot-assisted laparoscopic surgery. Builds on the open-source, fast, interactive FEM simulation backend SOFA. Deformable object manipulation, topological changes (cutting), grasping, image observation modalities (RGB, depth, segmentation, point clouds).

Integrate sensory information to ground Graph Network Simulators on real world observations. Predict the mesh state of deformable objects by utilizing point cloud data.

Training a visumotor policy for deformable object manipulation in simulation with reinforcement learning. Transferring the policy to the real world with the daVinci Research Kit using unpaired image-to-image translation.

Learns decentralized policies without a human in the loop with multi-agent reinforcement learning. Evaluates the learned policies in cooperation with a human surgeon for deformable object manipulation.

Evaluates several architectures and training strategies for semantic segmentation of organs and tissues in laparoscopic surgery.