Title

Multimodal Data Management in Disks and Main Memory for Deep Learning

Alternative Title

디스크 및 메인 메모리에서 딥러닝을 위한 멀티모달 데이터 관리

DGIST Authors

Lee, Dooseok ; Lee, Sanghyeon ; Kim, Min-Soo

Advisor

김민수

Co-Advisor(s)

Dooseok Lee

Issued Date

2019

Awarded Date

2019-02

Citation

Sanghyeon Lee. (2019). Multimodal Data Management in Disks and Main Memory for Deep Learning. doi: 10.22699/thesis.200000171479

Type

Thesis

Table Of Contents

Ⅰ. INTRODUCTION·························································································· 1
ⅠⅠ. BACKGROUND························································································ 4
2.1 LMDB ···································································································· 4
2.2 Apache Kafka························································································· 5
ⅠⅠⅠ. MULTIMODAL DATA MANAGEMENT SYSTEM···································· 6
3.1 Batch and stream data ··········································································· 6
3.2 Batch and stream processing ································································· 7
ⅠV. MULTIMODAL DATA MANAGEMENT METHOD······································· 9
4.1 MDF ········································································································ 9
4.2 RID Table ······························································································ 11
4.3 Image MDF ·························································································· 13
4.4 Relational MDF ····················································································· 14
V. STREAM PROCESSING FOR DEEP LEARNING··············································· 16
5.1 Stream platform ····················································································· 17
5.2 Stream processing method ····································································· 18
5.3 Stream processing module ····································································· 20
VⅠ. EXPERIMENTS·························································································· 22
6.1 Experimental setup················································································· 22
6.2 Data conversion······················································································ 22
6.3 Stream processing for deep learning······················································· 24
6.3.1 Surveillance and drama video data··················································· 25
6.3.2 Surveillance data with noise images················································· 26
6.3.3 Unrelated video consisting of MasterCOCO dataset······························· 27
VⅠⅠ. CONCLUSION AND FUTURE WORK························································ 28
VⅠⅠⅠ. REFERENCE························································································· 29

URI

http://dgist.dcollection.net/common/orgView/200000171479
http://hdl.handle.net/20.500.11750/10739

DOI

10.22699/thesis.200000171479

Degree

MASTER

Department

Information and Communication Engineering

Publisher

DGIST

Show Full Item Record