Title

A Fast and Elastic Distributed Matrix Computation Engine using GPUs

Alternative Title

그래픽 처리 장치를 사용하는 빠르고 탄력적인 분산 행렬 계산 엔진

DGIST Authors

Donghyoung Han ; Daehoon Kim ; Kyongseok Park

Advisor

김대훈

Co-Advisor(s)

Kyongseok Park

Issued Date

2021

Awarded Date

2021/02

Citation

Donghyoung Han. (2021). A Fast and Elastic Distributed Matrix Computation Engine using GPUs. doi: 10.22677/thesis.200000364466

Type

Thesis

Subject

Matrix computation, large-scale matrix multiplication, GPU processing, and operation fusion, 분산 행렬 계산, 대규모 행렬 곱셈, GPU 가속화 및 연산 융합

Table Of Contents

Chapter 1. Introduction 1
1.1 Motivation and objectives 1
1.2 Main contributions 8
1.3 Structure of thesis 9
Chapter 2. Background 11
2.1 Matrix Computation Engine 11
2.1.1 Matrix Partitioning Schemes 11
2.1.2 Distributed Matrix Multiplication Methods 12
2.1.3 Operator Fusion 17
2.1.4 Distributed Fused Operator 23
Chapter 3. DistME: Elastic Distributed Matrix Multiplication 28
3.1 Cuboid Matrix Multiplication 28
3.1.1 (P; Q; R)-Cuboid partitioning 28
3.1.2 Optimization of CuboidMM 31
3.2 Acceleration of CuboidMM using GPUs 33
3.2.1 Subcuboid partitioning 33
3.2.2 Optimization of subcuboids for GPU 35
3.2.3 GPU streaming of subcuboids 36
3.2.4 Algorithm 38
3.3 Experimental Evaluation 38
3.3.1 Experimental setup 40
3.3.2 Performance of CuboidMM 41
3.3.3 Performance of DistME 44
3.3.4 Performance of GNMF 47
3.3.5 Comparison with Systems in HPC 48
Chapter 4. FuseME: Operation Fusion Method 51
4.1 Cuboid-based fused Operator 51
4.1.1 Cuboid-based Fusion 51
4.1.2 Cost Optimization 59
4.2 Cuboid-based Fusion Generation 66
4.2.1 Fusion Plan 66
4.2.2 Union and Cut of Fusion plan 71
4.3 Experimental Evaluation 73
4.3.1 Experimental setup 74
4.3.2 Performance of Cuboid-based fusion 75
4.3.3 Optimization of Cuboid-based fusion 78
4.3.4 Performance Depending on Fusion plan . 80
Chapter 5. Related Work 82
5.1 Related Work 82
Chapter 6. Conclusion 84
6.1 Conclusions 84
Bibliography 86

URI

http://dgist.dcollection.net/common/orgView/200000364466
http://hdl.handle.net/20.500.11750/16698

DOI

10.22677/thesis.200000364466

Degree

Doctor

Department

Information and Communication Engineering

Publisher

DGIST

Show Full Item Record