研究生: |
林志韋 Lin, Chih-Wei |
---|---|
論文名稱: |
用於光學同調斷層掃描之基於深度學習和聯邦學習框架之視網膜積液分割技術 Retinal Fluid Segmentation Technology Based on Deep Learning and Federated Learning Framework for Optical coherence tomography |
指導教授: |
呂成凱
Lu, Cheng-Kai |
口試委員: |
呂成凱
Lu, Cheng-Kai 連中岳 Lien, Chung-Yueh 林承鴻 Lin, Cheng-Hung |
口試日期: | 2024/07/15 |
學位類別: |
碩士 Master |
系所名稱: |
電機工程學系 Department of Electrical Engineering |
論文出版年: | 2024 |
畢業學年度: | 112 |
語文別: | 中文 |
論文頁數: | 93 |
中文關鍵詞: | 視網膜積液分割 、深度學習 、卷積神經網路 、聯邦學習 |
英文關鍵詞: | Retinal Fluid Segmentation, Deep learning, Convolutional Neural Network, Federated Learning |
研究方法: | 實驗設計法 、 準實驗設計法 |
DOI URL: | http://doi.org/10.6345/NTNU202401376 |
論文種類: | 學術論文 |
相關次數: | 點閱:99 下載:1 |
分享至: |
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
在眼科領域,光學相干斷層掃描(OCT)是檢測眼病的關鍵技術。偏鄉資源有限僅能使用輕量化設備,但其計算能力不足,難以支撐較為大型模型的訓練,以及數據缺乏和隱私問題阻礙醫院數據共享。首先針對輕量化設備,基於LEDNet設計了高效的LEDNet(α)模型,通過調整通道、添加Shuffle Attention模塊和Group Normalization。使用成本低廉的樹莓派5進行訓練,適合偏鄉需求,為解決隱私問題,引入聯邦學習,通過上傳本地模型參數聚合全局模型,避免資料直接上傳。本研究提出Krum(α)算法,在客戶端損失函數中添加近端項並考慮模型自適應性,改善淘汰機制,改進基於歐氏距離淘汰惡意模型的Krum算法。最後實驗結果顯示,在AROI、DUKE、UMN和RETOUCH數據集上,AROI積液類別提高了3.4%,DUKE提高了5.9%,UMN提高了2.4%,RETOUCH提高了1.4%。
In the field of ophthalmology, Optical Coherence Tomography (OCT) is a key technology for detecting eye diseases. In resource-limited rural areas, only lightweight devices can be used, but it’s computational power is insufficient to support the training of large models. Additionally, data scarcity and privacy issues hinder data sharing between hospitals. First, for lightweight devices, an efficient LEDNet(α) model was designed based on LEDNet by adjusting channels, adding Shuffle Attention modules, and using Group Normalization. Training was conducted using the low-cost Raspberry Pi 5, suitable for rural needs. To solve privacy issues, federated learning was introduced, aggregating a global model by uploading local model parameters, avoiding direct data upload. This study proposes the Krum(α) algorithm, improving the Krum algorithm based on Euclidean distance by adding a proximal term in the client's loss function and considering model adaptability, improving the elimination mechanism. Finally, experimental results showed that on the AROI, DUKE, UMN, and RETOUCH datasets, the AROI exudate category improved by 3.4%, DUKE by 5.9%, UMN by 2.4%, and RETOUCH by 1.4%.
[1] 聯邦數據保護法“New Federal Act on Data Protection (nFADP),” Avaliable:https://www.kmu.admin.ch/kmu/en/home/facts-and-trends/digitization/data-protection/new-federal-act-on-data-protection-nfadp.html.
[2] 歐盟第9條規定“General Data Protection Regulation,” Available: https://gdpr-text.com/read/article-9/
[3] Q. Li. Y. Diao, Q. Chen, and B. He, “Federated Learning on Non-IID Data Silos: An Experimental Study,” IEEE 38th International Conference on Data Engineering (ICDE)., Kuala Lumpur, Malaysia, 2022, pp. 965-978.
[4] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. Arcas, “Communication-efficient learning of deep networks from decentralized data,” Artificial Intelligence and Statistics (AISTATS)., April 2017, pp. 1273-1282.
[5] M. Wojtkowski, M. Srinivasan, K. Ko, J. Fujimoto, and R. G. M. St. John, “Real-time in vivo imaging by high-speed spectral optical coherence tomography,” Optics Letters, vol. 28, no. 19, pp. 1745-1747, Oct. 2003.
[6] Z. Wang, Y. Zhong, M. Yao, Y. Ma, W. Zhang, C. Li, and B. Yan, “Automated segmentation of macular edema for the diagnosis of ocular disease using deep learning method,” Scientific Reports., vol. 11, no. 1, Art. no. 13392, 2021.
[7] D. K. Hwang, W. K. Yu, T. C. Lin, S. J. Chou, A. Yarmishyn, Z. K. Kao, and Y. C. Jheng, “Smartphone-based diabetic macula edema screening with an offline artificial intelligence,” J. Chin. Med. Assoc, vol. 83, no. 12, pp. 1102-1106, Dec. 2020.
[8] B. Kayalibay, G. Jensen, and P. van der Smagt, “CNN-based segmentation of medical imaging data,” arXiv preprint arXiv:1701.03056, Jan. 2017.
[9] O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” Medical Image Computing and Computer-Assisted Intervention (MICCAI)., Munich, Germany, Oct. 5-9, 2015, pp. 234-241.
[10] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” IEEE Conf. Computer Vision and Pattern Recognition (CVPR)., Boston, MA, June 7-12, 2015, pp. 3431-3440.
[11] H. D. Ghael, L. Solanki, and G. Sahu, “A review paper on raspberry pi and its applications,” Int. J. Adv. Eng. Manag. (IJAEM)., vol. 2, no. 12, pp. 4, 2020.
[12] D. Pan, L. Jia, A. Zeng, and X. Song, “Applications of generative adversarial networks in medical image processing,” Sheng Wu Yi Xue Gong Cheng Xue Za Zhi (J. Biomedical Eng)., vol. 35, no. 6, pp. 970-976, 2018.
[13] J. D. Malone, I. Hussain, and A. K. Bowden, “SmartOCT: smartphone-integrated optical coherence tomography,” Biomedical Optics Express, vol. 14, no. 7, pp. 3138-3151, Jul. 2023.
[14] Z. Wang, Y. Zhong, M. Yao, Y. Ma, W. Zhang, C. Li, and B. Yan, “Automated segmentation of macular edema for the diagnosis of ocular disease using deep learning method,” Scientific Reports, vol. 11, no. 1, pp. 13392, 2021.
[15] M. Melinščak, M. Radmilovič, Z. Vatavuk and S. Lončarić, “AROI: Annotated Retinal OCT Images Database,"2021 44th International Convention on Information, Communication and Electronic Technology (MIPRO)., Opatija, Croatia, 2021, pp. 371-376.
[16] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, Sep. 2014.
[17] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)., Las Vegas, NV, USA, 2016, pp. 770-778.
[18] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR)., Boston, MA, June 7-12, 2015, pp. 3431-3440.
[19] T. Schlegl, S. M. Waldstein, H. Bogunovic, F. Endstraber, A. Sadeghipour, A. M. Philip, and U. Schmidt-Erfurth, “Fully automated detection and quantification of macular fluid in OCT using deep learning,” Ophthalmology, vol. 125, no. 4, pp. 549-558, Apr. 2018.
[20] Z. Chen, D. Li, H. Shen, H. Mo, Z. Zeng, and H. Wei, “Automated segmentation of fluid regions in optical coherence tomography B-scan images of age-related macular degeneration,” Optics Laser Technol, vol. 122, pp. 105830, Dec. 2020.
[21] Z. Wang, Y. Zhong, M. Yao, Y. Ma, W. Zhang, C. Li, and B. Yan, “Automated segmentation of macular edema for the diagnosis of ocular disease using deep learning method,” Scientific Reports, vol. 11, no. 1, pp. 13392, 2021.
[22] L. B. Sappa, I. P. Okuwobi, M. Li, Y. Zhang, S. Xie, S. Yuan, and Q. Chen, “RetFluidNet: retinal fluid segmentation for SD-OCT images using convolutional neural network,” J. Digit. Imaging, vol. 34, no. 3, pp. 691-704, Jun. 2021.
[23] S. J. Pawan, R. Sankar, A. Jain, M. Jain, D. V. Darshan, B. N. Anoop, and J. Rajan, “Capsule Network–based architectures for the segmentation of sub-retinal serous fluid in optical coherence tomography images of central serous chorioretinopathy,” Med. Biol. Eng. Comput, vol. 59, no. 6, pp. 1245-1259, Jun. 2021.
[24] M. Melinščak, M. Radmilović, Z. Vatavuk, and S. Lončarić, “Annotated retinal optical coherence tomography images (AROI) database for joint retinal layer and fluid segmentation,” Automatika: Časopis za Automatiku, Mjerenje, Elektroniku, Računarstvo i Komunikacije, vol. 62, no. 3-4, pp. 375-385, Sep. 2021.
[25] M. Melinščak, “Attention-based U-net: Joint Segmentation of Layers and Fluids from Retinal OCT Images,” 2023 46th MIPRO ICT and Electronics Convention (MIPRO)., Opatija, Croatia, May 22-26, 2023, pp. 391-396.
[26] A. G. Roy, S. Conjeti, S. P. K. Karri, D. Sheet, A. Katouzian, C. Wachinger, and N. Navab, “ReLayNet: retinal layer and fluid segmentation of macular optical coherence tomography using fully convolutional networks,” Biomed. Opt. Express., vol. 8, no. 8, pp. 3627-3642, Aug. 2017.
[27] A. Farshad, Y. Yeganeh, P. Gehlbach, and N. Navab, “Y-net: A spatiospectral dual-encoder network for medical image segmentation,”International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI)., Switzerland, Sep. 2022, pp. 582-592.
[28] C. S. Lee, A. J. Tyring, N. P. Deruyter, Y. Wu, A. Rokem, and A. Y. Lee, “Deep-learning based, automated segmentation of macular edema in optical coherence tomography,” Biomed., Opt. Express, vol. 8, no. 7, pp. 3440-3448, Jul. 2017.
[29] X. Liu, J. Cao, T. Fu, Z. Pan, W. Hu, K. Zhang, and J. Liu, “Semi-supervised automatic segmentation of layer and fluid region in retinal optical coherence tomography images using adversarial learning,” IEEE Access, vol. 7, pp. 3046-3061, 2018.
[30] R. Tennakoon, A. K. Gostar, R. Hoseinnezhad, and A. Bab-Hadiashar, “Retinal fluid segmentation in OCT images using adversarial loss based convolutional neural networks,” 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018)., Washington, DC, USA, Apr. 2018, pp. 1436–1440.
[31] H. Wei and P. Peng, “The segmentation of retinal layer and fluid in SD-OCT images using mutex dice loss based fully convolutional networks,” IEEE Access, vol. 8, pp. 60929-60939, 2020.
[32] Z. Chen, D. Li, H. Shen, H. Mo, Z. Zeng, and H. Wei, “Automated segmentation of fluid regions in optical coherence tomography B-scan images of age-related macular degeneration,” Optics Laser Technol, vol. 122, Art. no. 105830, Dec. 2020.
[33] S. Loizillon, C. Meurée, C. Breuil, T. Faucon, and A. Lambert, “Deep-learning based retinal fluid segmentation in optical coherence tomography images using a cascade of ENets,” BioRxiv, 2021.
[34] E. Parra-Mora and L. A. Da Silva Cruz, “LOCTseg: A lightweight fully convolutional network for end-to-end optical coherence tomography segmentation,” Comput. Biol. Med, vol. 150, Art. no. 106174, Dec. 2022.
[35] J. Zhuang, J. Yang, L. Gu, and N. Dvornek, “Shelfnet for fast semantic segmentation,” in Proc. IEEE/CVF Int. Conf. Comput. Vision Workshops (ICCVW)., Seoul, South Korea, Oct. 2019, pp. 0-0.
[36] B. Baheti, S. Innani, S. Gajre, and S. Talbar, “Eff-unet: A novel architecture for semantic segmentation in unstructured environment,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, Jun. 2020, pp. 358-359.
[37] C. Yu, C. Gao, J. Wang, G. Yu, C. Shen, and N. Sang, “BiSeNet v2: Bilateral network with guided aggregation for real-time semantic segmentation,” Int. J. Comput. Vision, vol. 129, pp. 3051-3068, 2021.
[38] A. Cazañas-Gordón and L. A. da Silva Cruz, “Multiscale attention gated network (MAGNet) for retinal layer and macular cystoid edema segmentation,” IEEE Access, vol. 10, pp. 85905-85917, 2022.
[39] X. He, W. Song, Y. Wang, F. Poiesi, J. Yi, M. Desai, and Y. Wan, “Light-weight retinal layer segmentation with global reasoning,” IEEE Trans. Instrum, vol. 73, no. 1, pp. 1-12, Jan. 2024.
[40] Y C. Yang and F. Gao, “EDA-Net: Dense aggregation of deep and shallow information achieves quantitative photoacoustic blood oxygenation imaging deep in human breast,” Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, Oct. 13–17, 2019, vol. 22, pp. 246-254.
[41] A. Lou and M. Loew, “CFPNet: Channel-wise feature pyramid for real-time semantic segmentation,” in 2021 IEEE International Conference on Image Processing (ICIP)., Bracelona, Spain, Sep. 2021, pp. 1894-1898.
[42] S. Woo, J. Park, J. Y. Lee, and I. S. Kweon, “CBAM: Convolutional block attention module,” in Proc. European Conf. Comput. Vision (ECCV), Munich, Germany, Sep. 2018, pp. 3-19.
[43] Q. Wang, B. Wu, P. Zhu, P. Li, W. Zuo, and Q. Hu, “ECA-Net: Efficient channel attention for deep convolutional neural networks,” in Proc. IEEE/CVF Conf. Comput. Vision Pattern Recognit. (CVPR), Seattle, WA, Jun. 2020, pp. 11534-11542.
[44] Y. Wang, “LEDNet: A lightweight encoder-decoder network for real-time semantic segmentation,” in Proc. IEEE Int. Conf. Image Process. (ICIP), Taipei, Taiwan, Sep. 2019, pp. 1860-1864.
[45] M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang, “Deep learning with differential privacy,” in Proc. 2016 ACM SIGSAC Conf. Comput. Commun, Security, Vienna, Austria, Oct. 2016, pp. 308-318.
[46] D. C. Nguyen, Q. V. Pham, P. N. Pathirana, M. Ding, A. Seneviratne, Z. Lin, ... and W. J. Hwang, “Federated learning for smart healthcare: A survey,” ACM Comput. Surv. (CSUR)., vol. 55, no. 3, pp. 1-37, 2022.
[47] T. Sun, D. Li, and B. Wang, “Decentralized federated averaging,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 4, pp. 4289-4301, Apr. 2022.
[48] D. Yin, Y. Chen, R. Kannan, and P. Bartlett, “Byzantine-robust distributed learning: Towards optimal statistical rates,” in International Conference on Machine Learning (ICML)., Stockholm, Sweden, Jul. 2018, pp. 5650-5659.
[49] D. Yin, Y. Chen, R. Kannan, and P. Bartlett, “Byzantine-robust distributed learning: Towards optimal statistical rates,” in International Conference on Machine Learning (ICML)., Stockholm, Sweden, Jul. 2018, pp. 5650-5659.
[50] P. Blanchard, E. M. E. Mhamdi, R. Guerraoui, and J. Stainer, “Byzantine-tolerant machine learning,” arXiv preprint arXiv:1703.02757, Mar. 2017.
[51] Z. Tang and T. H. Chang, “FedLion: Faster adaptive federated optimization with fewer communication,” in Proc. ICASSP 2024-2024 IEEE Int. Conf. Acoustics, Speech Signal Process. (ICASSP)., Melbourne, Australia, Apr. 2024, pp. 13316-13320.
[52] C. Thapa, P. C. M. Arachchige, S. Camtepe, and L. Sun, “SplitFed: When federated learning meets split learning,” in Proc. AAAI Conf. Artif. Intell., vol. 36, no. 8, pp. 8485-8493, 2022.
[53] Ceballos, V. Sharma, E. Mugica, A. Singh, A. Roman, P. Vepakomma, and R. Raskar, “SplitNN-driven vertical partitioning,” arXiv preprint arXiv:2008.04137, Aug. 2020.
[54] Zhang, M., L. Qu, P. Singh, J. Kalpathy-Cramer, and D. L. Rubin, “Splitavg: A heterogeneity-aware federated deep learning method for medical imaging,” IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 9, pp. 4635-4644, Sept. 2022.
[55] M. Zhang, L. Qu, P. Singh, J. Kalpathy-Cramer, and D. L. Rubin, “SplitAVG: A heterogeneity-aware federated deep learning method for medical imaging,” IEEE J. Biomed. Health Informatics, vol. 26, no. 9, pp. 4635-4644, Sep. 2022.
[56] M. G. Arivazhagan, V. Aggarwal, A. K. Singh, and S. Choudhary, “Federated learning with personalization layers,” arXiv preprint arXiv:1912.00818, Dec. 2019.
[57] T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith, “Federated optimization in heterogeneous networks,” in Proceedings of Machine Learning and System.s, vol. 2, pp. 429-450, 2020.
[58] S. P. Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich, and A. T. Suresh, “Scaffold: Stochastic controlled averaging for federated learning,” in Proceedings of the International Conference on Machine Learning (ICML)., Nov. 2020, pp. 5132-5143.
[59] J. Miao, Z. Yang, L. Fan, and Y. Yang, “Fedseg: Class-heterogeneous federated learning for semantic segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 8042-8052.
[60] A. Brecko, E. Kajati, J. Koziorek, and I. Zolotova, “Federated learning for edge computing: A survey,” Applied Sciences, vol. 12, no. 18, pp. 9124, 2022.
[61] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data," in Artificial Intelligence and Statistics, PMLR, Apr. 2017, pp. 1273-1282.
[62] A. Ziller, A. Trask, A. Lopardo, B. Szymkow, B. Wagner, E. Bluemke,, “Pysyft: A library for easy federated learning,” in Federated Learning Systems: Towards Next-Generation AI, pp. 111-139, 2021.
[63] C. He, S. Li, J. So, X. Zeng, M. Zhang, H. Wang, “FedML: A research library and benchmark for federated machine learning,” arXiv:2007.13518, 2020.
[64] D. J. Beutel, T. Topal, A. Mathur, X. Qiu, J. Fernandez-Marques, Y. Gao, "Flower: A friendly federated learning research framework," arXiv:2007.14390, 2020.
[65] X. Liu, T. Shi, C. Xie, Q. Li, K. Hu, H. Kim, “Unifed: A benchmark for federated learning frameworks,” arXiv:2207.10308., 2022.
[66] S. J. Chiu, M. J. Allingham, P. S. Mettu, S. W. Cousins, J. A. Izatt, and S. Farsiu, “Kernel regression based segmentation of optical coherence tomography images with diabetic macular edema,” Biomedical Optics Express, vol. 6, no. 4, pp. 1172-1194, 2015.
[67] S. J. Chiu, M. J. Allingham, P. S. Mettu, S. W. Cousins, J. A. Izatt, and S. Farsiu, “Kernel regression based segmentation of optical coherence tomography images with diabetic macular edema,” Biomedical Optics Express, vol. 6, no. 4, pp. 1172-1194, 2015.
[68] M. D. Abràmoff, M. K. Garvin and M. Sonka, “Retinal Imaging and Image Analysis,” in IEEE Reviews in Biomedical Engineering, vol. 3, pp. 169-208, 2010.
[69] A. Hübers ., “Retinal involvement in amyotrophic lateral sclerosis: a study with optical coherence tomography and diffusion tensor imaging,” J. Neural Transm, vol. 123, pp. 281-287, 2016.
[70] Q. L. Zhang and Y. B. Yang, “Sa-net: Shuffle attention for deep convolutional neural networks,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)., Jun. 2021, pp. 2235-2239.
[71] Y. Wu and K. He, “Group normalization,” in Proceedings of the European conference on computer vision (ECCV)., 2018, pp. 3-19.
[72] C. Li, Z. Qiu, X. Cao, Z. Chen, H. Gao, and Z. Hua, “Hybrid dilated convolution with multi-scale residual fusion network for hyperspectral image classification,” Micromachines, vol. 12, no. 5, pp. 545, 2021.