Cited time in webofscience Cited time in scopus

A Transformer Model with Spatiotemporal Input Embedding for fNIRS data-Driven Neural Decoding

Title
A Transformer Model with Spatiotemporal Input Embedding for fNIRS data-Driven Neural Decoding
Author(s)
Lee, HyunminKim, TaehunAn, Jinung
Issued Date
2024-02-26
Citation
12th International Winter Conference on Brain-Computer Interface, BCI 2024, pp.1 - 4
Type
Conference Paper
ISBN
9798350309430
ISSN
2572-7672
Abstract
Model structures proposed as spatial embedding of fNIRS data have a problem of embedding channels in a row into one channel without considering the brain area. This paper proposes a neural decoding model, ORC-T, using input embedding that reflects fNIRS data's spatiotemporal characteristics. ORC-T embeds the fNIRS channels into new channels with a depth-wise convolution layer based on the fNIRS optod layout. The embedded data is transferred to the classifier via a 2D convolution layer and a transformer encoder. An open dataset (mental arithmetic, BNCI Horizon 2020) and a consciousness dataset were trained to validate model performance. The ORC-T achieved a k-fold CV accuracy of 81.78% and LOSO CV accuracy of 90.19% in open data, showing lower or equivalent performance than the existing model, fNIRS-T. In the consciousness state dataset, the ORC-T achieved a k-fold CV accuracy of 97.09% and LOSO CV accuracy of 59.43%, showing the equivalent performance as the fNIRS-T. The performance of ORC-T and fNIRS-T hardly decreased even though the number of channels input to the model was reduced. Considering the possibility that spatial embedding introduced in ORC-T did not effectively reflect the spatial information of the fNIRS channels, a model using a convolution layer with a kernel in the channel array's horizontal, vertical, and diagonal directions is in the planning process. © 2024 IEEE.
URI
http://hdl.handle.net/20.500.11750/57516
DOI
10.1109/BCI60775.2024.10480484
Publisher
Institute of Electrical and Electronics Engineers Inc.
Related Researcher
Files in This Item:

There are no files associated with this item.

Appears in Collections:
Division of Intelligent Robot 2. Conference Papers
Division of Intelligent Robot Brain Robot Augmented InteractioN(BRAIN) Laboratory 2. Conference Papers

qrcode

  • twitter
  • facebook
  • mendeley

Items in Repository are protected by copyright, with all rights reserved, unless otherwise indicated.

BROWSE