Ongoing Projects



Educational video is the key driver to e-learning, as it offers rich context, flexibility, and personalized learning speed. Interaction possibilities, however, are often limited to a sequential viewing. To overcome this limitation, AMIGO's goal is an interaction with videos just like with PDFs: You can search for text in e-lectures, copy text from them, navigate between pages/slides, get links to interesting stuff, etc. Key to that is an automatic indexer for e-lectures that - given a video and presentation slides - automatically localizes each slide in the video, estimating a pixel- and frame-accurate position of each text box.

[AMIGO live system (access restricted)] [AMIGO paper]



Virtual reality applications often requires to bake complex light situations for specific objects in a virtual scene into textures, so called lightmaps. These lightmaps can be used in any 3D realtime applications like games or visualization apps. With precalculated lightmaps it is possible to improve the performance and the visual quality in realtime applications. In this project we develop a software tool that supports at least baking of Global Illumination, Ambient Occlusion and Direct Lighting / Shadow Baking.

Dieses Projekt (HA-Projekt-Nr. 229/10-05) wurde im Rahmen von Hessen ModellProjekte aus Mitteln der LOEWE – Landes-Offensive zur Entwicklung Wissenschaftlich-ökonomischer Exzellenz, Förderlinie 3: KMU-Verbundprojektvorhaben gefördert.


DemoToGo BMBF_GWK_logo

Content Analytics is targeted at extracting knowledge from heterogenous data sources, mainly natural language text. Here, recent models based on neural networks (deep learning) open up possibilities for improving search, tagging, categorization, and exploration. DeepCA's mission is to interlink neural models with conventional semantic knowledge modelling using ontologies. A cognitive service for text and knowledge graph analysis will be developed that can easily adapted for the exploration of new domains and leads to innovative solutions for semantic text annotation and search in heterogenous case bases. Through this, DeepCA will form the basis for an end-user centric, efficient knowledge engineering.
DeepCA is conducted in collaboration with the company partners Empolis and SER, and with the German Research Center for Artificial Intelligence (DFKI) and the University of Trier as academic partners. The project is funded by the Federal Ministry of Education and Research.

Demo To Go

DemoToGo BMBF_GWK_logo

Demonstrationen sind wesentliche Elemente für den Technologieaustausch. Dazu sollen zunächst herkömmliche Demozentren eingerichtet werden, in denen ohne großen Vorbereitungsaufwand Gästen der Hochschule Forschungsresultate aus dem Anwendungsbereich Smart Home / Smart Energy / Smart Mobility sowie Technologie- und Dienstleistungsdemos aus dem Forschungsschwerpunkt "Smarte Systeme für Mensch und Technik" demonstriert werden können. In einem zweiten Schritt sollen Demos mit Methoden der Virtuellen Realität realisiert werden, um eine Ortsunabhängigkeit und in Ergänzung zu reinen Videos eine Interaktivität zu erreichen. Damit können diese Demos auf hybriden Plattformen vorgeführt werden, z.B. kann eine Demo, die besonderes Equipment am Hochschulstandort Rüsselsheim benötigt, auch am Standort Wiesbaden gezeigt werden. Andere Orte sind z.B. Messen oder Vorort-Demonstrationen bei Externen. In einem dritten Schritt sollen die Demonstrationen so erweitert werden, dass sie nicht nur von eingewiesenem Hochschulpersonal sondern auch Unternehmen und Konsumenten selber durchgeführt werden können. Methoden der Serious Games sollen genutzt werden, um diesen ein motivierendes und unterhaltsames Demo-Erlebnis zu vermitteln. Die Demonstrationen sollen auch in der Lehre Verwendung finden.


Kephalos BMBF_logo

Orthodontic motivated radiographs substantial contributes to radiation exposure of children and adolescents. Especially since the introduction of dental digital volume tomography (DVT), the use of 3D radiographs is increasingly propagated. Because DVT devices typically generate 3D information from several hundred 2D radiographs, the effective dose of a DVT recording is at least an order of magnitude greater than that of a single digital lateral cephalogram. However, for many diagnostics a complete 3D model often is not needed, but only the outer (facial) surface of the facial skeleton. The aim of Kephalos is to develope a new technique for calculating the exact facial surface of the facial bones based on a) a single lateral cephalogram, b) an optical face scan and c) a statistical model of the correlation between the facial surface and the facial bones.
Kephalos is funded by the Federal Ministry of Education and Research.



The research project OPTIGAMA (Optical Short Game Motion Analyzer) is embedded in the research project SHOGAMA (Short Game Motion Analyzer) which is a joint project together with the Science & Motion Sports GmbH in Flörsheim and the institute of computer science of the Johannes Gutenberg University Mainz. The aim of SHOGAMA is to develop a highly accurate realtime system for three-dimensional tracking of the golf cub during short game. The complete system will be composed of an inertial measurement unit (IMU) attached to the golfs cub handle as well as a camera observing the head of the golf cub. The aim of OPTIGAMA is to determine the golf cub position and orientation (pose) while it is inside the cameras field of view based on the camera footage. The determined pose can be additionally refined and verified based on the data obtained by the IMU. On the other hand pose information can be achieved only based on IMU data if the golf cub is outside the cameras field of view. This combination allows to monitor the complete golf stroke mechanics.



Im Rahmen des FuE-Projekts wird ein neuartiges System entstehen, mit dem die Bewegung eines Golfschlägers beim sogenannten Kurzspiel (Putten und Chippen) räumlich und hochpräzise erfasst werden kann. Dabei wird ein optisches, monokulares, passives Trackingverfahren in Kombination mit einer intertialen Messeinheit (englisch intertial measurement unit, IMU) entwickelt werden. Die beiden Sensorquellen gleichen hierbei jeweils ihre gegenseitigen Schwächen aus, um so den Messraum und die Messgenauigkeit des Gesamtsystems zu vergrößern bzw. zu erhöhen. Die Kamera wird so positioniert, dass sie die Schlägerbewegungen nahe des Balls, inklusive des Abschlags, mit maximaler Präzision erfassen kann. Sobald der Schläger den Sichtbereich der Kamera verlässt, wird die Bewegung mit der IMU anhand eines physikalischen Schlagmodells rekonstruiert. Der zeitliche Drift der den IMU Messdaten innewohnt, wird durch die absoluten Messungen der Kamera korregiert, sobald der Schläger wieder in deren Sichtbereich eintritt. Die Algorithmen werden auf einem eingebetteten System implementiert werden, welches dem Golfer innerhalb weniger Sekunden Feedback in Form einer 3D Visualisierung des Schlages liefert.
Die Hochschule RheinMain bearbeitet hierbei im Teilprojekt OPTIGAMA (Optical Short Game Motion Analyzer) im wesentlichen Fragestellungen im Bereich des optischen Trackings von Golfschlägern.



The research project TRACKSHOT (Tracking of Shotgun and Target for Olympic Trap and Skeet) is a joint research project with the German Shooting Federation (DSB).The aim of the project is to detect and track the position and orientation (pose) of the shotgun together with the position of the target simultaneously in a common coordinate system. The resulting performance-diagnostic hardware component and a suitable software application should enable both the athlete and the trainer to have an accurate and timely evaluation and interpretation of the data. Beside the development of a robust and accurate marker-based tracking system, one of the main focuses of the project is to set the basis for a completely marker-less system, that enables tracking without the need of hardware to be attached to the sports equipment in order to have a completely non-reactive system. Particularly suitable for this initial tests for the development of such a marker less optical tracking system is the shotgun due to its size and thus good visibility. Thus, a feasibility study for marker-less tracking is conducted within the project. If this proves to be feasible, it would be dealt with in follow-up applications for other sports equipment such as air gun or bow.



Many tasks in the field of urban planning, such as noise protection measures, the incidence of light, or radio coverage require 3D data. Complete 3D city models thus gradually replace conventional maps. But the creation and maintenance of 3D city models is very complex and costly. The goal of this research project is to develop a system that allow editing of 3D city models by various users using al kind of different devices. Users should be placed in a position to visualize and collaboratively edit the models on all kind of devices from the traditional desktop PC to Tablet PCs and smartphones.

Dieses Projekt (HA-Projekt-Nr. 315/12-05) wurde im Rahmen von Hessen ModellProjekte aus Mitteln der LOEWE – Landes-Offensive zur Entwicklung Wissenschaftlich-ökonomischer Exzellenz, Förderlinie 3: KMU-Verbundprojektvorhaben gefördert.



WASABI brings together computer vision experts from HSRM with colleagues from the National University University of Sciences and Technology (NUST) in Islamabad / Pakistan. Pakistan ranks third in water scarcity world-wide, and suffers from a lack of proper water resource management. While drying up on the one hand, the country experiences catastrophic floods on the other. Therefore, to better manage water resources so that to enforce an even distribution of water throughout the year, a solution is required to densely supervise the volume of water bodies (including rivers, lakes, and reservoirs) and predict the ratio at which the water resources are varying. To this end, the project WASABI (WAter resource estimation by SAtellite Based Image analysis) aims at laying the foundations for a system that quantitatively monitors Pakistan’s water bodies and helps national authorities with a more proactive water supply management. To do so, the WASABI system will employ remote sensing satellite imagery of Pakistan’s water bodies. This data will be processed with state-of-the-art machine learning / deep learning techniques to estimate the current quantity of water and - based on historic time series of population and seasonal capacity fluctuation - predict the amount of water available in the near future. WASABI‘s core research challenge lies in the fact that labeled data for Pakistani waterbodies is scarce, which is why unsupervised learning and transfer learning from densely labeled regions such as Europe will be explored. The project also features an academic exchange program, with interns from NUST visiting Wiesbaden on an annual basis.