DGIST Scholar: A Fast and Elastic Distributed Matrix Computation Engine using GPUs

Citation: Donghyoung Han. (2021). A Fast and Elastic Distributed Matrix Computation Engine using GPUs. doi: 10.22677/thesis.200000364466

Subject: Matrix computation, large-scale matrix multiplication, GPU processing, and operation fusion, 분산 행렬 계산, 대규모 행렬 곱셈, GPU 가속화 및 연산 융합

Table Of Contents: Chapter 1. Introduction 1
1.1 Motivation and objectives 1
1.2 Main contributions 8
1.3 Structure of thesis 9
Chapter 2. Background 11
2.1 Matrix Computation Engine 11
2.1.1 Matrix Partitioning Schemes 11
2.1.2 Distributed Matrix Multiplication Methods 12
2.1.3 Operator Fusion 17
2.1.4 Distributed Fused Operator 23
Chapter 3. DistME: Elastic Distributed Matrix Multiplication 28
3.1 Cuboid Matrix Multiplication 28
3.1.1 (P; Q; R)-Cuboid partitioning 28
3.1.2 Optimization of CuboidMM 31
3.2 Acceleration of CuboidMM using GPUs 33
3.2.1 Subcuboid partitioning 33
3.2.2 Optimization of subcuboids for GPU 35
3.2.3 GPU streaming of subcuboids 36
3.2.4 Algorithm 38
3.3 Experimental Evaluation 38
3.3.1 Experimental setup 40
3.3.2 Performance of CuboidMM 41
3.3.3 Performance of DistME 44
3.3.4 Performance of GNMF 47
3.3.5 Comparison with Systems in HPC 48
Chapter 4. FuseME: Operation Fusion Method 51
4.1 Cuboid-based fused Operator 51
4.1.1 Cuboid-based Fusion 51
4.1.2 Cost Optimization 59
4.2 Cuboid-based Fusion Generation 66
4.2.1 Fusion Plan 66
4.2.2 Union and Cut of Fusion plan 71
4.3 Experimental Evaluation 73
4.3.1 Experimental setup 74
4.3.2 Performance of Cuboid-based fusion 75
4.3.3 Optimization of Cuboid-based fusion 78
4.3.4 Performance Depending on Fusion plan . 80
Chapter 5. Related Work 82
5.1 Related Work 82
Chapter 6. Conclusion 84
6.1 Conclusions 84
Bibliography 86

URI: http://dgist.dcollection.net/common/orgView/200000364466
http://hdl.handle.net/20.500.11750/16698

Show Full Item Record