News  /  Education  /  Experience  /  Publications  /  Projects  /  Contact  /  ▶ More ...

Rafay Mohiuddin

Greetings, I'm Rafay 👋. I hold a Master’s degree from the TUM School of Engineering and Design, where I focused on scientific computing, robotics and 3D computer vision. My master's thesis, conducted at the Huawei Intelligent Cloud Technologies Lab, centered on advancing Open World 3D Scene Understanding. Previously, I served as a Robotics Research Intern, concentrating on lifelong localization and mapping. During my studies, I also worked as a Student Research Assistant at the Chair of Computational Modeling and Simulation, contributing to various robotics projects.

My primary interests lie in robotics, computer vision, machine learning, and physics simulation, with ~ three years of experience in R&D, I have worked on a wide range of projects across these fields.

Email  /  GitHub  /  Orcid  /  LinkedIn  /  CV

profile photo
project image
Huawei Munich
Masters Thesis Research - Intelligent Clound Technologies Lab
Oct 2023 - Apr 2024

Conducted R&D in the area of zero shot approches for “Open World 3D Scene Understanding” utilizing foundation and large language models; co-supervised by Huawei-ICTL and TUM-CMS.
  • Developed OpenSU3D an incremental and scalable approch for zero-shot 3D scene understanding utilizing foundation and large language models.
  • Quantitatively and qualitatively demonstrated effectiveness of open vocabulary segmentation and context-based instance retrieval, exceeding state-of-the-art approaches.
  • Explored downstream applications such as 3D instance segmentation, automatic labeling, and spatial reasoning (chat with scene using LLM).
  • Currently, working on research paper to publish findings.


Research Intern - Intelligent Clound Technologies Lab
Apr 2023 - Oct 2024

Contributed as a Robotics Research Intern at Huawei's Intelligent Cloud Technologies Lab in the area of 3D robotics perception, primarily focusing on LiDAR-based Life Long Localization and Mapping.
  • Research cutting-edge techniques to improve LiDAR-based lifelong localization and mapping systems. Implemented hierarchical bundle adjustment, pose graph optimization, etc. to enhance localization and mapping performance.
  • Refactor and modularize the SLAM pipeline to facilitate scalability, maintainability, and ease of integration with future enhancements.
  • Studied and implemented recent methods, and explored potential research directions in the areas of 3D Robotics Perception, Neural Radiance Field, and Scene Understanding.

thesis_3
project image
Technical University of Munich
Research Assistant - Chair of Computational Modeling and Simulation
Apr 2022 - Apr 2023

Contributed to the Intriped Project as a student research assistant and the Additive Manufacturing Project as part of software practicum at the Chair of Computational Modeling and Simulation.
  • Proposed methodology for robust, real-time control of material flow in additive manufacturing for construction using an RGB camera. Successfully deployed this solution on a scaled-down setup based on UR10 robot.
  • Developed a lightweight Blender-based LiDAR simulation tool tailored for large 3D building models. The tool automatically plans paths based on the 3D model and exports LiDAR scans according to configured specifications. Additionally, enhanced the existing 2D BIM-to-SLAM simulation module for 3D environ- ments by incorporating support for Gazebo.
  • Collaborated in the development of mapping system based on GO1 quadruped robot. Streamlined process for sensor calibration, synchronization, communication, and data recording. Dockerized and configured open-sourced ROS-based SLAM libraries for seamless use in indoor and outdoor environments.

go1
project image
National University of Science and Technology
Research Engineer - Integrated Navigation Terrestrial Electromagnetic Lab
Jul 2020 - Oct 2021

Contributed to a diverse set of projects focused on the development of aerial robots at the Integrated Navigation Terrestrial Electromagnetic Lab at National University of Science and University.
  • Developed setup for Software in the Loop (SITL) simulation for tuning the flight controller. Integrated the X-Plane flight simulator with the flight controller firmware. Based on aeronautical design, developed models of vertical take-off and fixed-wing aerial vehicles for the simulator.
  • Carried out end-to-end development of vertical take-off and fixed-wing aerial vehicles, from preliminary design (aeronautics calculations), detailed design (detailed CAD assemblies) to the export of manufacturing plans. Optimized design, aerodynamics, and structural components by conducting CFD, FEA, FSI simulations.
  • Provided guidance and mentorship to undergraduate interns to support project development.

vtol
project image
NED University of Engineering and Technology
Research Assistant - Department of Mechanical Engineering
Jul 2019 - Jan 2020

Assisted research activities in Mechanical Department of NED University under the supervision of Dr.-Ing. Usman Allauddin.
  • Conducted comparative studies to numerically assess different RANS turbulence models for prediction of the secondary peak in Nusselt number resulting from a hydraulic jump in jet impingement cooling.
  • Extensively reviewed literature and conducted numerical simulations related to impinging jets focusing on the influence of different types of fins, surface topology, pulsating/synthetic jets, and nanofluids on heat transfer characteristics. Collaborate on two peer-reviewed research papers based on findings.
  • Mentored undergraduate student groups, on their research project.

cfd
project image Technical University of Munich
Masters of Science Computational Mechanics
Oct 2021 - Oct 2024

Open World 3D Scene Understanding using Foundation Models. OpenSU3D
Course Work:

  • Computation in Engineering I & II (C++), Numerical Optimization, Elements of Machine Learning, Cyber Physical System, Advance Deep Learning for Physics, Professional Software Development, Visual Data Analytics …

project image
NED University of Engineering and Technology
Bachelors of Science Mechanical Engineering
Dec 2015 - Sep 2019

Numerical Investigation of Heat Transfer Enhancement Methods in Impinging Jets. Link
Course Work:

  • Advanced Calculus & Linear Algebra, Advanced Mathematical Techniques, Applied Probability & Statistics, Operations Management, Machine Control Systems, Clean Energy Technologies, Computer Programming & Applications …

project image

OpenSU3D: Open Word 3D Scene Understanding using Foundation Models


Rafay Mohiuddin,Sai Manoj Prakhya, Fiona Collins, Ziyuan Liu, Andre Borrmann
(in review) - , 2024
paper / web /

project image

Optical Process Control for extrusion-based Additive Manufacturing methods in construction


Rafay Mohiuddin, Miranda Cruz Policroniades, Martin Slepicka, André Borrmann
ISARC - 40th International Symposium on Automation and Robotics in Construction, 2023
paper / web /

project image

Nanoscale heat transfer investigation of an array of impinging jet systems with different working fluids under crossflow with and without pin fins


Usman Allauddin, Rafay Mohiuddin, Hafiz Mohammad Usman Khan, Naseem Uddin, Waqar A. Khan
Wiley - Journal of Heat Transfer, 2020
paper / web /

project image

Nanoscale heat transfer investigation of an array of impinging jet systems with different working fluids under crossflow with and without micro pin fins


Usman Allauddin, Tariq Jamil, Muhammad Shakaib, H.M. Usman Khan, Rafay Mohiuddin ,M. Saad Saeed, Haseeb Ahsan & Naseem Uddin
Begell House - Journal of Enhnaced Heat Transfer, 2020
paper / web /




These include coursework, assignments, side projects ... (GitHub)

project image

AI for Science FourCastNet


EuroCC2 AI for Science Bootcamp
2024-06-26

Training NeuralOperator, FourCastNet on nvidia gpu cluster.
project image

Diffusion Model (DDPM)


SideProject
2024-06-15

Implementation of DDPM paper in pytorch, training on MNIST and CIFAR10
project image

Advanced Deep Learning for Physics


TUM ADL4P: Advanced Deep Learning for Physics
2023-08-01

Optimization, Differential Simulation, Time Series Prediction, Inverse Design etc.
project image

Optimal Path - Differentiable Physics


TUM ADL4P: Advanced Deep Learning for Physics
2023-08-01

Differential physics training to find optimal path.
project image

Kuramoto-Sivashinsky Learning


TUM ADL4P: Advanced Deep Learning for Physics
2023-08-01

Learnining using differentiable solver for Kuramoto-Sivashinsky equation.
project image

Attachment Point Optimization


TUM RDC: Numerical Optimizationnn for Robotic Design and Control
2023-04-10

Optimal attachment point location to achive optimal wrench feasible workspace (minimim positive tension reqquired) for cable driven parallel robot.
project image

2R-planar robot: Bi-level Optimisation


TUM RDC: Numerical Optimizationnn for Robotic Design and Control
2023-04-10

Optimal control input and physical parameters to achive maximum workspace with minimum energy required for 2R-planner robot.
project image

Real Time Optical Process Monitering & Controll for Additive Manufacturing


TUM SL: Software Lab
2023-01-10

Real-time optical process monitoring and control using RGBD camera (ZED mini) for additive manufacturing system using a 6DOF robotic arm (UR10) for application in construction. (Accepted for ISARC)
project image

Copy Move Forgery Detection and Localization


TUM EoML: Elements of Machine Learning
2023-01-01

Copy move forgery detection and localization in images using SIFT and DBSCAN, tested on MICC-F220 dataset.
project image

Visualization of Scientific Data


TUM SV: Scientific Visualization
2022-12-25

Visualization of Scientific data using ParaView and ImageVis3D.
project image

F1Tenth Autonomous Car


TUM CPS: Cyber Physical Systems Lab
2022-09-14

Implementation of reactive autonomous navigation strategies on F1Tenth car.
project image

F1Tenth Autonomous Car - Simulation


TUM CPS: Cyber Physical Systems Lab
2022-09-07

Implementation and comparison of reactive autonomous navigation strategies in simulated environment.
project image

Computational Fluid Dynamics


TUM CFD: Computationl Fluid Dynamics
2022-07-30

Numerical methods in computational fluid dynamics. 1D-2D advection diffusion equation, 2D Navier Stokes Equation, explicit/implicit euler & central differnce scheme, crank nichlson analysis, runge kutta, gauss seidel, rhi-chow interpolation.
project image

FEM - Discritization using B-Spline basis


TUM CIE2: Computation in Engineering II
2022-07-01

FE code (C++) that discretizes a two-dimensional Poisson’s equation using B-Spline basis functions.
project image

Spline Kernal (BSpline)


TUM CIE2: Computation in Engineering II
2022-06-01

C++ implimentation for spline formation using approximation.
project image

Faster R-CNN Implementation


SideProject
2022-02-24

Implementation of Faster RCNN in pytorch + other CNN based projects object detection, facial keypoint, semantic segmentation …
project image

Implicit Geomtry


TUM CIE1: Computation in Engineering I
2022-02-01

Inplimentation of implicit geometry formation using quad tree in C++.

Design and source code from Jon Barron's website