Cited 0 time in webofscience Cited 0 time in scopus

A Large-scale Virtual Dataset and Egocentric Localization for Disaster Responses

A Large-scale Virtual Dataset and Egocentric Localization for Disaster Responses
Jeon, Hae-GonIm, SunghoonLee, Byeong-UkRameau, Franc¸oisChoi, Dong-GeolOh, JeanKweon, In SoHebert, Martial
DGIST Authors
Jeon, Hae-Gon; Im, Sunghoon; Lee, Byeong-Uk; Rameau, Franc¸ois; Choi, Dong-Geol; Oh, Jean; Kweon, In So; Hebert, Martial
Issue Date
IEEE Transactions on Pattern Analysis and Machine Intelligence, 1-1
Author Keywords
Large-scale datasetLocation awarenessSemanticsTask analysisThree-dimensional displaysvisual odometryVisualizationBuildingscamera relocalizationCamerasdisaster scenariosegocentric localization
CamerasComputer visionConvolutional neural networksDisastersLarge datasetOptical flowsSemanticsTexturesDisaster responseDisaster scenarioDisaster situationsEgocentric localizationGround truth dataHigh resolution stereoState-of-the-art methodsVisual observationsStereo image processing
With the increasing social demands of disaster response, methods of visual observation for rescue and safety have become increasingly important. However, because of the shortage of datasets for disaster scenarios, there has been little progress in computer vision and robotics in this field. With this in mind, we present the first large-scale synthetic dataset of egocentric viewpoints for disaster scenarios. We simulate pre- and post-disaster cases with drastic changes in appearance, such as buildings on fire and earthquakes. The dataset consists of more than 300K high-resolution stereo image pairs, all annotated with ground-truth data for the semantic label, depth in metric scale, optical flow with sub-pixel precision, and surface normal as well as their corresponding camera poses. To create realistic disaster scenes, we manually augment the effects with 3D models using physically-based graphics tools. We train various state-of-the-art methods to perform computer vision tasks using our dataset, evaluate how well these methods recognize the disaster situations, and produce reliable results of virtual scenes as well as real-world images. We also present a convolutional neural network-based egocentric localization method that is robust to drastic appearance changes, such as the texture changes in a fire, and layout changes from a collapse. To address these key challenges, we propose a new model that learns a shape-based representation by training on stylized images, and incorporate the dominant planes of query images as approximate scene coordinates. We evaluate the proposed method using various scenes including a simulated disaster dataset to demonstrate the effectiveness of our method when confronted with significant changes in scene layout. Experimental results show that our method provides reliable camera pose predictions despite vastly changed conditions. IEEE
IEEE Computer Society
Related Researcher
  • Author Im, Sunghoon Computer Vision Lab.
  • Research Interests Computer Vision; Deep Learning; Robot Vision
There are no files associated with this item.
Department of Electrical Engineering and Computer ScienceComputer Vision Lab.1. Journal Articles

qrcode mendeley

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.