Title

Deep Reinforcement Learning for Data Freshness in Blockchain-enabled Wireless Sensor Networks

DGIST Authors

Dongsun Kim ; Sungjin Lee ; Jemin Lee

Advisor

이성진

Co-Advisor(s)

Jemin Lee

Issued Date

2024

Awarded Date

2024-08-01

Citation

Dongsun Kim. (2024). Deep Reinforcement Learning for Data Freshness in Blockchain-enabled Wireless Sensor Networks. doi: 10.22677/THESIS.200000802743

Type

Thesis

Description

Permissioned blockchain, multi-agent reinforcement learning, transaction generation control, latency-sensitive services, wireless sensor networks, age of information, missing data, imputation

Table Of Contents

I. INTRODUCTION 1
1.1 Backgrounds 1
1.2 Outline and Contributions 2
1.2.1 Chapter 2 2
1.2.2 Chapter 3 2
1.2.3 Chapter 4 3
1.2.4 Chapter 5 3
II. Intelligent Transaction Generation Control for Permissioned Blockchain-based Services 4
2.1 Introduction 4
2.2 Related Works 5
2.2.1 Performance Evaluation of Permissioned Blockchain 5
2.2.2 Performance Optimization in Blockchain 6
2.3 Blockchain-based Services Model 7
2.3.1 HLF Transaction Flow 7
2.3.2 Main Parameters Affecting HLF Performance 8
2.4 Intelligent Transaction Generation Control 9
2.4.1 Effective Throughput of Blockchain-based Services 9
2.4.2 Effective Throughput Maximization Problem 10
2.4.3 Dec-POMDP for intelligent transaction generation control (i-TGC) 12
2.4.4 i-TGC Algorithm 13
2.5 Experiment Results 16
2.5.1 Experiment Environments and Baseline 16
2.5.2 i-TGC vs p-TGC 18
2.5.3 Effect of Blockchain Parameters 22
2.5.4 Effect of Number of Clients 23
2.6 Conclusion 24
III. Reinforcement Learning-based Sensing Decision for Data Freshness in Blockchain-enabled Wireless Networks 25
3.1 Introduction 25
3.2 System Model and Problem Formulation 26
3.2.1 System Model 26
3.2.2 Transaction Flow at blockchain (BC) 27
3.2.3 Problem Formulation 28
3.3 Reinforcement Learning-based Sensing Decision 29
3.3.1 decentralized partially observable Markov decision process (dec-POMDP) Formulation 30
3.3.2 reinforcement learning-based sensing decision (RL-SD) Algorithm 31
3.4 Experiment Results 32
3.5 Conclusion 35
IV. IA-MARL: Imputation Assisted Multi-Agent Reinforcement Learning for Missing Training Data 36
4.1 Introduction 36
4.2 Related Works 37
4.3 Background 39
4.3.1 DDPG and MADDPG 39
4.3.2 IQL and QMIX 40
4.3.3 Imputation 41
4.4 Methods 41
4.4.1 Missing Training Data Problem and Imputation Method 41
4.4.2 IA-MARL: Imputation Assisted Multi-Agent Reinforcement Learning 43
4.5 Experimental Results 46
4.5.1 Environment and Hyperparameters 46
4.5.2 Performance of IA-MARL 49
4.5.3 Effect of Mask-based Update and Imputation Accuracy 51
4.5.4 Effect of Missing Training Data on MADDPG and QMIX 52
4.5.5 Computation Time for Training and Evaluation 53
4.6 Conclusion 54
V. CONCLUSIONS 56
References 58

URI

http://hdl.handle.net/20.500.11750/57588
http://dgist.dcollection.net/common/orgView/200000802743

DOI

10.22677/THESIS.200000802743

Degree

Doctor

Department

Department of Electrical Engineering and Computer Science

Publisher

DGIST

Show Full Item Record