With the rapid advancement of hardware accelerators, the bottleneck in deep learning systems is moving from computation to I/O. This bottleneck can be observed especially during transfer learning that uses a fixed feature extractor because the feature extraction is an I/O intensive task. Due to the limited bandwidth of Direct Media Interface (DMI), GPUs failed to fully perform despite the advent of high-performance SSDs. To address the problem that the performance of transfer learning is limited by the DMI bandwidth, we propose a novel transfer learning system that adapts in-storage processing. The feature extraction is executed in SSDs using a high-performance mobile GPU. Because the feature extraction can be executed in parallel, the feature extraction in aggregated SSDs is fast and scalable. Moreover, the feature extraction can be optimized by adapting optimization techniques such as 16-bit floating-point quantization, layer fusion, kernel auto-tuning, removing transformation overhead, and data prefetching. Our proposed system could catch up with the GPU system using 6 aggregated SSDs in a power-efficient way.
Table Of Contents
I. Introduction II. Background 2.1 Transfer Learning 2.2 Direct Media Interface (DMI) III. Motivation IV. Design of SSD-Off 4.1 System Overview 4.2 Feature Extraction on SSD 4.3 Serialized Files in SSD 4.4 Operation Process V. Optimization 5.1 Feature Extraction Optimization 5.2 Transformation Overhead Removal 5.3 Data Prefetching VI. Evaluation 6.1 Experiment Setup 6.2 Experiment Results 6.2.1 Effect of Transformation Overhead Removal 6.2.2 Effect of Feature Extraction Optimization and Data Prefetching 6.2.3 Performance of Single SSD 6.2.4 Performance of Aggregated SSDs 6.2.5 Performance of SSD Cluster VII. Related Works 7.1 Inference-Enabled Solid-State Drives (IESSD) 7.2 In-storage Processing Using Edge TPU VIII. Conclusions 8.1 Conclusions 8.2 Future work References