Research Projects

This project aims to develop portable measurement tools with in-situ visualization for the construction industry. A future measurement tool will provide a direct augmented reality view of measured properties over the real environment together with instructions as to where and how a certain task can be completed. For example, a metal detection tool should be able to provide direct visual feedback on the location of hidden metallic structures over a live video view of the inspected wall area. Furthermore it can guide a construction engineer to the optimal position for drilling a hole, avoiding any damage to existing structures.

Thus the tools should combine information from several sources to provide interactive and contextaware

guidance: Measurements from built-in sensors; location-aware through online tracking and registration; spatial, semantic information retrieved from a building information system (BIM). At the same time, future tools need to be simple to be used by non-expert users; therefore the system needs to be intuitive and guide users in the correct operation to fulfil their tasks. To accomplish this goal, The project addresses the following challenges:

• Tracking for mobile devices in changing and unknown environments for correct visual overlays. We will investigate the combination of visual online reconstruction methods with range finders and coarse models for absolute registration.
• X-Ray visualization of hidden and abstract information in unknown environments. Here we will investigate automatic approaches that take the environment’s appearance and the virtual information into account to select the best visualization method.
• User guidance based on measured and plan information requires automatic analysis of the spatial arrangements and automatic visualization.

The growth of information is nowadays enormous and at a level which had never been reached before. We currently produce almost more data in one year than was produced in the entire history of mankind so far. In particular the trend to a full digitization of audiovisual content is contributing to this explosion of available material. The exponential growth of online video, most notably YouTube among the many prominent video portals is just one example for that. Even if international studies are not arriving at exactly the same results, the figures are impressive: digital production in 2006 was approximately 160 Exabyte, and is predicted to rise to 990 Exabyte in 2010.

Any video processing /editing software has to keep pace with these extraordinary data rates which requires special efforts from the hardware and the software. Fortunately we see also an extraordinary increase in processing power, especially when looking at recent developments of graphics cards (GPUs). These cards offer massive parallelism (ideally suited for video processing) at a rather modest price. All these facts make this hardware an ideal candidate for video processing. But in order to make full use of the hardware the algorithms have to be highly parallel. Typical tasks encountered in video processing (which will also be tackled by the proposed project are):

Superresolution: With the advent of HDTVs in many homes there is an increasing need to produce also HDTV content. In order to make use of existing (low-resolution) material one can use so called superresolution algorithms. These methods generate from a sequence of low resolution frames a high resolution image by exploiting the high interframe redundancy.

Denoising: There are many sources of noise in a video, either the material is historic or during production/compression etc. noise is added to the video. A basic task is to remove the noise but still preserve all fine scale details.

Interactive video editing: For post production purposes one wants to mark objects in a video (of course the object should only be marked in a single frame and then segmented automatically in all subsequent frames) and either remove them (which requires inpainting methods to fill the holes with meaningful content), place them somewhere else in the video or replace them with different objects. Since these tasks are done interactively this requires interactive framerates.

Fortunately all of these tasks can be addressed by so called variational methods. The basic idea is to formulate the task as a minimization problem of a suitable energy functional. Besides other desirable properties these methods can be implemented in a highly parallel fashion which makes them ideal candidates for implementation on modern GPUs.

The project VM-GPU fits exactly to the FIT-IT Visual Computing call. It is a combination of computer vision and graphics methods to offer solutions to a problem of great relevance for industry. In particular,

1. VM-GPU will address modern variational methods for computer vision. These methods are mathematically well understood and provide novel means for such diverse tasks as denoising, segmentation, 3D matching registration etc. One short coming of these methods is that due to their iterative nature they are usually slow to implement, therefore these methods have not been used in an industrial settings. 2. Modern graphics processing units (GPUs) offer a tremendous processing speed (the new Nvidia series is supposed to offer 500GFlops) and the increase in processing speed is much faster than for standard CPUs. Recent features of GPUs (e.g. floating point, highly parallel architecture etc.) make them attractive for general purpose calculations and in particular for computer vision tasks. 3. Machine Vision and industrial image processing is a fast growing market with a lot of challenging tasks to be solved. In order to keep pace with the production process the methods need to be fast.

The goal of VM-GPU is to make variational methods available for industrial problems by using modern graphics hardware. If successful this project will have a large impact on the machine vision industry, it will allow for the first time to use variational methods in an industrial setting, in addition having graphics cards available as computing platforms will offer completely new ways of addressing industrial vision problems (e.g., it is very easy to scale up by just using a second graphics card).

After medical treatment of visually handicapped people it is desirable to evaluate the benefit of the treatment for the patient. Especially the capability of the patient to orient himself in a three-dimensional environment, to navigate and recognize obstacles is of interest.

For a clinical evaluation under controlled circumstances a labyrinth has been built through which the patient ha to navigate. Obstacles may be randomly placed in the labyrinth. A multi-camera system keeps track of the patients movements and extracts parameters such as position, speed, head rotation etc.

Development of a vision system for accurate calibration of the kinematic chain of an articulated robot arm.

The absolute positioning error of articulated robot arms is typically by an order of ten higher than their repeatability error. Inaccurate blueprint kinematic models typically account for 90% of this discrepancy. In this work a calibration procedure is developed which calibrates the kinematic model of a robot arm using fixed stereo rig and a calibration target mounted on the robot hand. In a single calibration framework the following parameters are automatically determined:

• DH-parameters of the robot arm
• Relative pose of calibration target and tooltip
• Relative pose of stereo rig and robot coordinate frame
• Interior camera parameters
• Lens distortion parameters of both cameras
• Relative pose of the stereo cameras
• Inaccuracies/deformation of the calibration target

The procedure is fully automatic and does not require expensive, precalibrated equipment.

Development of Computer Vision Algorithms for Robust Object Detection and Tracking on an Embedded DSP Platform. FREQUENTIS was working on an embedded platform called "TRICam". The platform is capable of acting as a high quality Motion-JPEG or MPEG4 Encoder or Decoder; furthermore, it can perform a set of selected image processing algorithms for object detection and tracking in real-time. The focus was mainly put on traffic monitoring, vehicle detection and tracking. The goal was to also deal with various algorithms for video surveillance in public places and public transport.

We propose to develop a robotic system for the exact pose estimation of rigid objects in six degrees of freedom (DoF). Our approach is based on the following scenario: Given be a set of unordered objects in arbitrary position which come from a small variety (<10) of different types as may happen on a conveyor belt during an assembly process. We assume the objects to be mostly un-occluded and their CAD-model given. Our goal is to pick with a robot a specific object in such a way that it has a defined position. The robot is equipped with multiple black and white cameras (e.g., 2-3). In order to achieve maximum generality we do not intend to use range information nor calculate a 3D-representation from stereo images. This task requires a solution for the following sub-problems: Recognition and initial pose estimation, which will be approached using robust appearance-based recognition methods; Pose refinement will be handled by model-based methods; Movement planning to enhance the accuracy of the pose estimation.

Partnerlist:

M&R Holding GmbH