Recent advances in key technologies have made Unmanned Aerial Vehicles (UAVs) widely used for civilian, homeland security, and defense applications including mission planning, hostage rescue, reconnaissance, surveys of natural disaster areas, and even traffic and accident monitoring. In the military, the use of UAV technology can reduce the risk to military personnel and has taken on increased importance in light of the U.S.’s new war on terrorism. However, the volume of video data obtained by these UAVs is overwhelming compared to the amount of human intervention needed to visually inspect videos from UAVs. It is thus imperative to develop robust algorithms and software tools that can facilitate the otherwise labor-intensive process. Since in many cases 3D models are more descriptive than frames of the original video, one important tasks is 3D urban scene modeling from UAV video. This is a very challenging task because UAV videos tend to have lower resolution, poor contrast, and significant noise. Despite the advancements in 3D urban scene modeling, existing techniques are not robust enough to handle these types of data. To date, there is no working system that can directly reconstruct 3D urban scenes from UAV video.
We are developing a prototype system that can automatically reconstruct 3D urban scenes from UAV videos. To meet the aforementioned challenges we are developing robust, efficient algorithms for all major components of the system including image registration, image segmentation, as well as 3D reconstruction. Besides UAV video, the proposed system can also be extended to reconstruct 3D scene from other image modalities as well, e.g. high- resolution oblique satellite and/or airborne images.