Cited 0 time in webofscience Cited 0 time in scopus

Improving the Performance of Natural Language Deep Learning Models by Using Dimension Attribute Values

Title
Improving the Performance of Natural Language Deep Learning Models by Using Dimension Attribute Values
Translated Title
Dimension 속성값을 이용한 자연언어 딥러닝 모델의 성능 향상
Authors
Myeonghwa Lee
DGIST Authors
Myeonghwa Lee; Jemin Lee; Min-Soo Kim
Advisor(s)
이제민
Co-Advisor(s)
Min-Soo Kim
Issue Date
2021
Available Date
2022-07-07
Degree Date
2021/02
Type
Thesis
Keywords
BERT, OLAP, NLU, NLP, SRL, BERT, OLAP, 자연어 이해, 자연어 처리, 의미역 결정
Table Of Contents
1. Introduction 1 1.1 Motivation 2 1.2 Contributions 3 2. Background 5 2.1 Language Modeling 5 2.2 Context-independent and Context-sensitive Text Representation 6 2.3 Pre-train and Fine-tune Paradigm in the Field of Natural Language Processing 8 2.4 Semantic Role Labeling 9 2.5 Citation Intent Classification 10 3. Related Work 11 3.1 BERT (Bidirectional Transformers for Language Understanding) 11 3.2 BERT Variant Models 13 3.2.1 BERT Models Pre-trained with Domain-specific Corpus 13 3.2.2 BERT Models Pre-trained with New Tasks 14 3.2.3 BERT Models Pre-trained with Additional Features 15 4. Our Methods for OLAP-BERT 17 4.1 Method 1: Additional Features Affect Text Tokens Differently 17 4.2 Method 2: Additional Features Affect Text Tokens Equally 20 5. Our Datasets for OLAP-BERT 22 5.1 DBLP-RC: A Record-based Corpus 22 5.2 Record-based Labeled Datasets 24 5.2.1 DBLP-RDfSRL: A Record-based Dataset for Semantic Role Labeling 24 5.2.2 DBLP-RDfCIC: A Record-based Dataset for Citation Intent Classification 25 6. Experiments 26 6.1 Datasets 26 6.1.1 Record-based Corpus for Pre-training 26 6.1.2 Record-based Datasets for Fine-training 28 6.2 Experimental Setup 28 6.2.1 Pre-training for Natural Language Understanding Models 29 6.2.2 Fine-tuning for Task-specific Models 31 6.3 Experimental Results 32 6.3.1 Results of the Pre-training 32 6.3.2 Results of the Fine-tuning 34 7. Discussion 36 8. Conclusions 38 9. References 40
URI
http://dgist.dcollection.net/common/orgView/200000363154
http://hdl.handle.net/20.500.11750/16675
DOI
10.22677/thesis.200000363154
Degree
Master
Department
Information and Communication Engineering
University
DGIST
Files:
There are no files associated with this item.
Collection:
Department of Electrical Engineering and Computer ScienceThesesMaster


qrcode mendeley

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

BROWSE