Ключевые слова
нейрофизиологические данные
методы машинного обучения
нейронные сети глубокого обучения
сверточные нейронные сети
рекуррентные нейронные сети
Аннотация
Введение. В последнее время все большую популярность для анализа нейрофизиологических данных набирают методы машинного обучения, являющиеся составной частью методов искусственного интеллекта. Для изучения нейрокогнитивных механизмов в настоящее время активно применяют функциональную спектроскопию в ближнем инфракрасном диапазоне (фБИК-спектроскопию). Данная технология регистрации гемодинамических данных обладает рядом преимуществ, таких как точная локализация сигнала, неинвазивность, возможность проводить исследования в естественных условиях, что объясняет растущую популярность технологии среди исследователей. Теоретическое обоснование. Анализ результатов фБИК-спектроскопии зависит от последовательности и выбранных методов предварительной очистки и обработки исходных данных, а также от применяемых моделей для классификации полученных зависимостей. В настоящем обзоре рассмотрены различные методы предварительной обработки и детально проанализированы подходы к классификации данных фБИК-спектроскопии. При предварительной обработке сигнала важным моментом является удаление из исходных данных физиологических артефактов, для чего используются следующие алгоритмы: фильтрация, отбеливание сигнала, метод главных компонент (PCA) и метод независимых компонент (ICA), метод регистрации коротковолновых каналов (short-channel). Для удаления артефактов движения применяются такие методы, как вейвлет-фильтрация (wavelet), сплайн-интерполяция (spline interpolation), фильтрация Калмана. Обсуждение результатов. Обзор направлен на детальное рассмотрение методов машинного обучения, таких как рекуррентные нейронные сети (RNN) и сверточные нейронные сети (CNN), которые применялись в различных исследованиях для анализа данных фБИК-спектроскопии. В обзоре показано, что применение нейронных сетей глубокого обучения позволяет при анализе сигнала фБИК-спектроскопии сократить длительность предварительной обработки сигнала и при этом получить точность, превосходящую точность классических подходов в обработке нейрокогнитивных данных.
Библиографические ссылки
Селивёрстов, Ю. А., Селивёрстова, Е. В., Коновалов, Р. Н., Котенкова, М. В., & Иллариошкин, С. Н. (2014). Функциональная магнитно-резонансная томография покоя: возможности и будущее мето-да. Бюллетень Национального общества по изучению болезни Паркинсона и расстройств движений, (1), 16–19.
Asgher, U., Khalil, K., Khan, M. J., Ahmad, R., Butt, S. I., Ayaz, Y., ... & Nazir, S. (2020). Enhanced accuracy for mul-ticlass mental workload detection using long short-term memory for brain–computer interface. Frontiers in neuroscience, 14, 584. https://doi.org/10.3389/fnins.2020.00584
Barker, J. W., Aarabi, A., & Huppert, T. J. (2013). Autoregressive model-based algorithm for correcting motion and serially correlated errors in fNIRS. Biomedical optics express, 4(8), 1366–1379.
Benerradi, J., A. Maior, H., Marinescu, A., Clos, J., & L. Wilson, M. (2019, November). Exploring machine learning approaches for classifying mental workload using fNIRS data from HCI tasks. In Proceedings of the Halfway to the Future Symposium 2019 (pp. 1–11). https://doi.org/10.1145/3363384.3363392
Blanco, B., Molnar, M., & Caballero-Gaudes, C. (2018). Effect of prewhitening in resting-state functional near-infrared spectroscopy data. Neurophotonics, 5(4), 040401–040401. https://doi.org/10.1117/1.NPh.5.4.040401
Brigadoi, S., & Cooper, R. J. (2015). How short is short? Optimum source–detector distance for short-separation channels in functional near-infrared spectroscopy. Neurophotonics, 2(2), 025005–025005. https://doi.org/10.1117/1.NPh.2.2.025005
Chen, W. L., Wagner, J., Heugel, N., Sugar, J., Lee, Y. W., Conant, L., ... & Whelan, H. T. (2020). Functional near-infrared spectroscopy and its clinical application in the field of neuroscience: advances and future direc-tions. Frontiers in neuroscience, 14, 724. https://doi.org/10.3389/fnins.2020.00724
Chhabra, H., Shajil, N., & Venkatasubramanian, G. (2020). Investigation of deep convolutional neural network for classification of motor imagery fNIRS signals for BCI applications. Biomedical Signal Processing and Con-trol, 62, 102133. https://doi.org/10.1016/j.bspc.2020.102133
Chiarelli, A. M., Croce, P., Merla, A., & Zappasodi, F. (2018). Deep learning for hybrid EEG-fNIRS brain–computer in-terface: application to motor imagery classification. Journal of neural engineering, 15(3), 036028. https://doi.org/10.1088/1741-2552/aaaf82
Cohen, D. (1968). Magnetoencephalography: evidence of magnetic fields produced by alpha-rhythm cur-rents. Science, 161(3843), 784–786. https://doi.org/10.1126/science.161.3843.784
Cooney, C., Folli, R., & Coyle, D. (2021). A bimodal deep learning architecture for EEG-fNIRS decoding of overt and imagined speech. IEEE Transactions on Biomedical Engineering, 69(6), 1983–1994. https://doi.org/10.1109/TBME.2021.3132861
Cordes, D., Haughton, V. M., Arfanakis, K., Carew, J. D., Turski, P. A., Moritz, C. H., ... & Meyerand, M. E. (2001). Fre-quencies contributing to functional connectivity in the cerebral cortex in “resting-state” data. American Jour-nal of Neuroradiology, 22(7), 1326–1333.
Cui, Z., Chen, W., & Chen, Y. (2016). Multi-scale convolutional neural networks for time series classification. arXiv preprint arXiv:1603.06995. https://doi.org/10.48550/arXiv.1603.06995
Dargazany, A. R., Abtahi, M., & Mankodiya, K. (2019). An end-to-end (deep) neural network applied to raw EEG, fNIRs and body motion data for data fusion and BCI classification task without any pre-/post-processing. arXiv preprint arXiv:1907.09523. https://doi.org/10.48550/arXiv.1907.09523
Dolmans, T. C., Poel, M., van’t Klooster, J. W. J., & Veldkamp, B. P. (2021). Perceived mental workload classification using intermediate fusion multimodal deep learning. Frontiers in human neuroscience, 14, 609096. https://doi.org/10.3389/fnhum.2020.609096
Eastmond, C., Subedi, A., De, S., & Intes, X. (2022). Deep learning in fNIRS: a review. Neurophotonics, 9(4), 041411. https://doi.org/https://doi.org/10.1117/1.NPh.9.4.041411
Erdoĝan, S. B., Özsarfati, E., Dilek, B., Kadak, K. S., Hanoĝlu, L., & Akın, A. (2019). Classification of motor imagery and execution signals with population-level feature sets: implications for probe design in fNIRS based BCI. Journal of neural engineering, 16(2), 026029. https://doi.org/10.1088/1741-2552/aafdca
Funane, T., Sato, H., Yahata, N., Takizawa, R., Nishimura, Y., Kinoshita, A., ... & Kiguchi, M. (2015). Concurrent fNIRS-fMRI measurement to validate a method for separating deep and shallow fNIRS signals by using multidis-tance optodes. Neurophotonics, 2(1), 015003–015003. https://doi.org/10.1117/1.NPh.2.1.015003
Gagnon, L., Yücel, M. A., Boas, D. A., & Cooper, R. J. (2014). Further improvement in reducing superficial contami-nation in NIRS using double short separation measurements. Neuroimage, 85, 127–135. https://doi.org/10.1016/j.neuroimage.2013.01.073
Gao, Y., Chao, H., Cavuoto, L., Yan, P., Kruger, U., Norfleet, J. E., ... & Intes, X. (2022). Deep learning-based motion artifact removal in functional near-infrared spectroscopy. Neurophotonics, 9(4), 041406–041406. https://doi.org/10.1117/1.NPh.9.4.041406
Gao, Y., Yan, P., Kruger, U., Cavuoto, L., Schwaitzberg, S., De, S., & Intes, X. (2020). Functional brain imaging relia-bly predicts bimanual motor skill performance in a standardized surgical task. IEEE Transactions on Biomedi-cal Engineering, 68(7), 2058–2066. https://doi.org/10.1109/TBME.2020.3014299
Ghonchi, H., Fateh, M., Abolghasemi, V., Ferdowsi, S., & Rezvani, M. (2020а). Deep recurrent–convolutional neural network for classification of simultaneous EEG–fNIRS signals. IET Signal Processing, 14(3), 142–153. https://doi.org/10.1049/iet-spr.2019.0297
Ghonchi, H., Fateh, M., Abolghasemi, V., Ferdowsi, S., & Rezvani, M. (2020b). Spatio-temporal deep learning for EEG-fNIRS brain computer interface. In 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (pp. 124–127). IEEE. https://doi.org/10.1109/EMBC44109.2020.9176183
Glorot, X., Bordes, A., & Bengio, Y. (2011, June). Deep sparse rectifier neural networks. In Proceedings of the four-teenth international conference on artificial intelligence and statistics (pp. 315–323). JMLR Workshop and Conference Proceedings.
Graves, A. (2012). Long Short-Term Memory. In: Supervised Sequence Labelling with Recurrent Neural Networks. Studies in Computational Intelligence, 385, 37–45. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24797-2_4
Hakimi, N., Jodeiri, A., Mirbagheri, M., & Setarehdan, S. K. (2020). Proposing a convolutional neural network for stress assessment by means of derived heart rate from functional near infrared spectroscopy. Computers in Biology and Medicine, 121, 103810. https://doi.org/10.1016/j.compbiomed.2020.103810
Hamid, H., Naseer, N., Nazeer, H., Khan, M. J., Khan, R. A., & Shahbaz Khan, U. (2022). Analyzing classification per-formance of fNIRS-BCI for gait rehabilitation using deep neural networks. Sensors, 22(5), 1932. https://doi.org/10.3390/s22051932
Ho, T. K. K., Gwak, J., Park, C. M., & Song, J. I. (2019). Discrimination of mental workload levels from multi-channel fNIRS using deep leaning-based approaches. Ieee Access, 7, 24392–24403. https://doi.org/10.1109/ACCESS.2019.2900127
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
Huppert, T. J. (2016). Commentary on the statistical properties of noise and its implication on general linear mod-els in functional near-infrared spectroscopy. Neurophotonics, 3(1), 010401–010401. https://doi.org/10.1117/1.NPh.3.1.010401
Hyvärinen, A., & Oja, E. (2000). Independent component analysis: algorithms and applications. Neural net-works, 13(4–5), 411–430. https://doi.org/10.1016/s0893-6080(00)00026-5
Izzetoglu, M., Chitrapu, P., Bunce, S., & Onaral, B. (2010). Motion artifact cancellation in NIR spectroscopy using discrete Kalman filtering. Biomedical engineering online, 9, 1–10. https://doi.org/10.1186/1475-925X-9-16
Jahani, S., Setarehdan, S. K., Boas, D. A., & Yücel, M. A. (2018). Motion artifact detection and correction in function-al near-infrared spectroscopy: a new hybrid method based on spline interpolation method and Savitzky–Golay filtering. Neurophotonics, 5(1), 015003–015003. https://doi.org/10.1117/1.NPh.5.1.015003
Kim, M., Lee, S., Dan, I., & Tak, S. (2022). A deep convolutional neural network for estimating hemodynamic re-sponse function with reduction of motion artifacts in fNIRS. Journal of Neural Engineering, 19(1), 016017. https://doi.org/10.1088/1741-2552/ac4bfc
Kohno, S., Miyai, I., Seiyama, A., Oda, I., Ishikawa, A., Tsuneishi, S., ... & Shimizu, K. (2007). Removal of the skin blood flow artifact in functional near-infrared spectroscopic imaging data through independent component analysis. Journal of Biomedical Optics, 12(6), 062111–062111. https://doi.org/10.1117/1.2814249
Kwon, J., & Im, C. H. (2021). Subject-independent functional near-infrared spectroscopy-based brain–computer interfaces based on convolutional neural networks. Frontiers in Human Neuroscience, 15, 646915. https://doi.org/10.3389/fnhum.2021.646915
Light, G. A., Williams, L. E., Minow, F., Sprock, J., Rissling, A., Sharp, R., ... & Braff, D. L. (2010). Electroencephalog-raphy (EEG) and event‐related potentials (ERPs) with human participants. Current protocols in neurosci-ence, 52(1), 6–25. https://doi.org/10.1002/0471142301.ns0625s52
Liu, Y., Ayaz, H., & Shewokis, P. A. (2017). Multisubject “learning” for mental workload classification using concur-rent EEG, fNIRS, and physiological measures. Frontiers in human neuroscience, 11, 389. https://doi.org/10.3389/fnhum.2017.00389
Lu, J., Yan, H., Chang, C., & Wang, N. (2020). Comparison of machine learning and deep learning approaches for decoding brain computer interface: an fNIRS study. In Intelligent Information Processing X: 11th IFIP TC 12 In-ternational Conference, IIP 2020, Hangzhou, China, July 3–6, 2020, Proceedings 11 (pp. 192-201). Springer International Publishing. https://doi.org/10.1007/978-3-030-46931-3_18
Luke, R., Larson, E. D., Shader, M. J., Innes-Brown, H., Van Yper, L., Lee, A. K., ... & McAlpine, D. (2021). Analysis methods for measuring passive auditory fNIRS responses generated by a block-design paradigm. Neuropho-tonics, 8(2), 025008. https://doi.org/10.1117/1.NPh.8.2.025008
Ma, T., Lyu, H., Liu, J., Xia, Y., Qian, C., Evans, J., ... & He, S. (2020). Distinguishing bipolar depression from major depressive disorder using fnirs and deep neural network. Progress In Electromagnetics Research, 169, 73–86. https://doi.org/10.2528/PIER20102202.
Ma, T., Wang, S., Xia, Y., Zhu, X., Evans, J., Sun, Y., & He, S. (2021). CNN-based classification of fNIRS signals in motor imagery BCI system. Journal of Neural Engineering, 18(5), 056019. https://doi.org/10.1088/1741-2552/abf187
Molavi, B., and Dumont, G. A. (2012). Wavelet-based motion artifact removal for functional near-infrared spec-troscopy. Physiological Measurement, 33, 259–270. https://doi.org/10.1088/0967-3334/33/2/259
Naseer, N., Qureshi, N. K., Noori, F. M., & Hong, K. S. (2016). Analysis of different classification techniques for two-class functional near-infrared spectroscopy-based brain-computer interface. Computational Intelligence and Neuroscience, 2016. https://doi.org/10.1155/2016/5480760
Nguyen, H. D., Yoo, S. H., Bhutta, M. R., & Hong, K. S. (2018). Adaptive filtering of physiological noises in fNIRS da-ta. Biomedical Engineering Online, 17, 1–23. https://doi.org/10.1186/s12938-018-0613-2
Orrù, G., Monaro, M., Conversano, C., Gemignani, A., & Sartori, G. (2020). Machine learning in psychometrics and psychological research. Frontiers in Psychology, 10, 2970. https://doi.org/10.3389/fpsyg.2019.02970
Ortega, P., & Faisal, A. (2021, May). HemCNN: deep learning enables decoding of fNIRS cortical signals in hand grip motor tasks. In 2021 10th International IEEE/EMBS Conference on Neural Engineering (NER) (pp. 718-721). IEEE. https://doi.org/10.1109/NER49283.2021.9441323
Ortega, P., & Faisal, A. A. (2021). Deep learning multimodal fNIRS and EEG signals for bimanual grip force decod-ing. Journal of Neural Engineering, 18(4), 0460e6. https://doi.org/10.1088/1741-2552/ac1ab3
Osharina, V., Ponchel, E., Aarabi, A., Grebe, R., & Wallois, F. (2010). Local haemodynamic changes preceding inter-ictal spikes: a simultaneous electrocorticography (ECoG) and near-infrared spectroscopy (NIRS) analysis in rats. Neuroimage, 50(2), 600–607. https://doi.org/10.1016/j.neuroimage.2010.01.009
Pinti, P., Aichelburg, C., Gilbert, S., Hamilton, A., Hirsch, J., Burgess, P., & Tachtsidis, I. (2018). A review on the use of wearable functional near‐infrared spectroscopy in naturalistic environments. Japanese Psychological Re-search, 60(4), 347–373. https://doi.org/10.1111/jpr.12206
Quaresima, V., & Ferrari, M. (2019, August). A mini-review on functional near-infrared spectroscopy (fNIRS): where do we stand, and where should we go?. Photonics, 6(3). https://doi.org/10.3390/photonics6030087
Robertson, F. C., Douglas, T. S., & Meintjes, E. M. (2010). Motion artifact removal for functional near infrared spec-troscopy: a comparison of methods. IEEE Transactions on Biomedical Engineering, 57(6), 1377–1387. https://doi.org/10.1109/TBME.2009.2038667
Rojas, R. F., Romero, J., Lopez-Aparicio, J., & Ou, K. L. (2020). Pain assessment based on fNIRS using bidirectional LSTMs. arXiv preprint arXiv:2012.13231. URL: http://arxiv.org/abs/2012.13231
Saadati, M., Nelson, J., & Ayaz, H. (2019, October). Mental workload classification from spatial representation of fnirs recordings using convolutional neural networks. In 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP) (pp. 1-6). IEEE. https://doi.org/10.1109/MLSP.2019.8918861
Scholkmann, F., Kleiser, S., Metz, A. J., Zimmermann, R., Pavia, J. M., Wolf, U., & Wolf, M. (2014). A review on con-tinuous wave functional near-infrared spectroscopy and imaging instrumentation and methodology. Neu-roimage, 85, 6-27. https://doi.org/10.1016/j.neuroimage.2013.05.004
Scholkmann, F., Spichtig, S., Muehlemann, T., & Wolf, M. (2010). How to detect and reduce movement artifacts in near-infrared imaging using moving standard deviation and spline interpolation. Physiological measure-ment, 31(5), 649. https://doi.org/10.1088/0967-3334/31/5/004
Sirpal, P., Kassab, A., Pouliot, P., Nguyen, D. K., & Lesage, F. (2019). fNIRS improves seizure detection in multimodal EEG-fNIRS recordings. Journal of Biomedical Optics, 24(5), 051408–051408. https://doi.org/10.1117/1.jbo.24.5.051408
Sitnikova, M. A., & Malykh, S. B. (2021). Functional near-infrared spectroscopy applications in developmental cog-nitive neuroscience. I.P. Pavlov Journal of Higher Nervous Activity, 71(4), 485–499.
Sun, Z., Huang, Z., Duan, F., & Liu, Y. (2020). A novel multimodal approach for hybrid brain–computer inter-face. IEEE Access, 8, 89909–89918. https://doi.org/10.1109/ACCESS.2020.2994226
Tanveer, M. A., Khan, M. J., Qureshi, M. J., Naseer, N., & Hong, K. S. (2019). Enhanced drowsiness detection using deep learning: an fNIRS study. IEEE access, 7, 137920–137929. https://doi.org/10.1109/ACCESS.2019.2942838
Trakoolwilaiwan, T., Behboodi, B., Lee, J., Kim, K., & Choi, J. W. (2018). Convolutional neural network for high-accuracy functional near-infrared spectroscopy in a brain–computer interface: three-class classification of rest, right-, and left-hand motor execution. Neurophotonics, 5(1), 011008–011008. https://doi.org/10.1117/1.nph.5.1.011008
Van Houdt, G., Mosquera, C., & Nápoles, G. (2020). A review on the long short-term memory model. Artificial In-telligence Review, 53, 5929–5955. https://doi.org/10.1007/s10462-020-09838-1
Wang, R., Hao, Y., Yu, Q., Chen, M., Humar, I., & Fortino, G. (2021). Depression analysis and recognition based on functional near-infrared spectroscopy. IEEE Journal of Biomedical and Health Informatics, 25(12), 4289–4299. https://doi.org/10.1109/JBHI.2021.3076762
Wang, Z., & Oates, T. (2015). Spatially encoding temporal correlations to classify temporal data using convolu-tional neural networks. arXiv preprint arXiv:1509.07481. https://doi.org/10.48550/arXiv.1509.07481
Wickramaratne, S. D., & Mahmud, M. S. (2020, November). A Ternary Bi-Directional LSTM Classification for Brain Activation Pattern Recognition Using fNIRS. In 2020 5th International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS) (pp. 202-207). IEEE. https://doi.org/10.48550/arXiv.2101.05892
Wickramaratne, S. D., & Mahmud, M. S. (2021). Conditional-GAN based data augmentation for deep learning task classifier improvement using fNIRS data. Frontiers in big Data, 4, 659146. https://doi.org/10.3389/fdata.2021.659146
Woo, S. W., Kang, M. K., & Hong, K. S. (2020). Classification of finger tapping tasks using convolutional neural net-work based on augmented data with deep convolutional generative adversarial network. In 2020 8th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob). IEEE. https://doi.org/10.1109/BioRob49111.2020.9224386
Xu, L., Choy, C. S., & Li, Y. W. (2016, September). Deep sparse rectifier neural networks for speech denoising. In 2016 IEEE International Workshop on Acoustic Signal Enhancement (IWAENC) (pp. 1-5). IEEE. https://doi.org/10.1109/IWAENC.2016.7602891
Xu, L., Geng, X., He, X., Li, J., & Yu, J. (2019). Prediction in autism by deep learning short-time spontaneous hemo-dynamic fluctuations. Frontiers in Neuroscience, 13, 1120. https://doi.org/10.3389/fnins.2019.01120
Yang, D., Huang, R., Yoo, S. H., Shin, M. J., Yoon, J. A., Shin, Y. I., & Hong, K. S. (2020). Detection of mild cognitive impairment using convolutional neural network: temporal-feature maps of functional near-infrared spec-troscopy. Frontiers in Aging Neuroscience, 12, 141. https://doi.org/10.3389/fnagi.2020.00141
Yücel, M. A., Lühmann, A. V., Scholkmann, F., Gervain, J., Dan, I., Ayaz, H., ... & Wolf, M. (2021). Best practices for fNIRS publications. Neurophotonics, 8(1), 012101–012101. https://doi.org/10.1117/1.NPh.8.1.012101
Zhao, Q., Li, C., Xu, J., & Jin, H. (2019, July). FNIRS based brain-computer interface to determine whether motion task to achieve the ultimate goal. In 2019 IEEE 4th International Conference on Advanced Robotics and Mechatronics (ICARM) (pp. 136–140). IEEE. https://doi.org/10.1109/ICARM.2019.8833883
Zhang, X., Noah, J. A., & Hirsch, J. (2016). Separation of the global and local components in functional near-infrared spectroscopy signals using principal component spatial filtering. Neurophotonics, 3(1), 015004–015004. https://doi.org/10.1117/1.NPh.3.1.015004
Zheng, Y., Liu, Q., Chen, E., Ge, Y., & Zhao, J. L. (2014). Time series classification using multi-channels deep con-volutional neural networks. In Web-Age Information Management: 15th International Conference, WAIM 2014, Macau, China, June 16-18, 2014. Proceedings 15. Springer International Publishing. https://doi.org/10.1007/978-3-319-08010-9_33
Это произведение доступно по лицензии Creative Commons «Attribution» («Атрибуция») 4.0 Всемирная.
Copyright (c) 2024 Рустам Г. Асадуллаев, Мария А. Ситникова, Александр А. Слетов, Андрей В. Ситников, Сергей Б. Малых