All Issue

2024 Vol.40, Issue 5 Preview Page
Data Construction through Oversampling Techniques and Outlier Removal Methods

Oversampling 기법 및 이상치 제거 방법을 통한 데이터 구축 연구

Byeong-Su Jang1
,
Gyu-Hyun Go2
,
YoungSeok Kim3
,
Sewon Kim4
,
Hyun-Jun Choi5
,
Hyung-Koo Yoon6*
장 병수1
,
고 규현2
,
김 영석3
,
김 세원4
,
최 현준5
,
윤 형구6*
1Member, Ph.D. Student, Dept. of Disaster Safety Engineering, Daejeon Univ.
2Member, Associate Prof., Dept. of civil Engineering, Kumoh National Institure of Tech.
3Member, Senior Research Fellow, Northern Infrastructure Specialized Team, Korea Institute of Civil Engineering and Building Technology
4Researcher, Department of Geotechnical Engineering Research, Korea Institute of Civil Engineering and Building Technology
5Member, Senior Researcher, Northern Infrastructure Specialized Team, Korea Institute of Civil Engineering and Building Technology
6Member, Prof., Dept. of Disaster Safety Engineering, Daejeon Univ.
1정회원, 대전대학교 재난안전공학과 박사과정
2정회원, 금오공과대학교 토목공학과 부교수
3정회원, 한국건설기술연구원 북방인프라특화팀 선임연구위원
4비회원, 한국건설기술연구원 지반연구본부 연구원
5정회원, 한국건설기술연구원 북방인프라특화팀 수석연구원
6정회원, 대전대학교 재난안전공학과 정교수
*Corresponding Author
31 October 2024. pp. 93-101
PDF XML Citation
Abstract

Numerical analysis methods are widely used to assess the safety of hydrogen storage facilities; however, obtaining data under various conditions poses significant challenges. This study aims to expand the dataset using oversampling algorithms and utilize these enhanced datasets as diverse input parameters for numerical analysis. The oversampling techniques applied include SMOTE, Borderline-SMOTE, ADASYN, and CTGAN, with data amplified by factors of 2, 5, and 100 relative to the original dataset. This approach increases data volume based on the characteristics of the existing data, which may consequently introduce outliers. To address this, statistical methods such as the 3-sigma rule and the confidence level method are employed to identify and remove outliers beyond the normal distribution range. The reliability of the conditions generated through data amplification and outlier analysis is evaluated by comparing them with trends observed in the original dataset. Additionally, the SHAP algorithm is utilized to analyze changes in the importance values of each parameter. The SHAP values derived from the original dataset and those processed through AI techniques and outlier analysis exhibit similar trends, validating the proposed methodologies. The methods proposed in this paper are applicable not only to hydrogen storage facilities but also to the systematic construction of data for assessing the stability of various geotechnical structures.

Keywords
Hydrogen storage facilities
Outlier analysis
Oversampling algorithm
SHAP
Stability

수소 저장 시설의 안정성을 평가하기 위해서 주로 수치해석 방법이 활용되나, 여러 조건의 데이터 확보에는 어려움이 따른다. 해당 연구의 목적은 oversampling 알고리즘을 활용하여 데이터 그룹의 양을 확대하고 수치해석 시 다양한 입력 인자로 이용되도록 하는 것이다. Oversampling 알고리즘은 AI 분야에서 데이터 불균형 문제를 해소하고자 제안된 SMOTE, Borderline-SMOTE, ADASYN 그리고 CTGAN 기법을 적용하였으며, 기존 데이터 대비 2배, 5배 그리고 100배로 증폭하였다. 해당 방법은 기존 데이터 특성을 기반으로 양을 증폭하는 방식으로 최종 데이터 그룹은 이상치가 포함될 가능성이 있다. 이를 해소하고자 통계기법인 3 sigma rule과 confidence level 방법으로 데이터의 정규분포 특성의 일정한 범위 외에 있는 값들은 이상치로 판단하여 제거하였다. 데이터 증폭과 이상치 분석을 통해 구축된 다양한 조건의 값의 신뢰성은 기존 데이터의 경향과 비교하여 판단하고자 하였으며, SHAP 알고리즘을 통해 각 물성치들의 중요도 값의 변화를 살펴보았다. 기존 데이터와 AI 기법 및 이상치 분석을 수행한 데이터의 SHAP 값은 모두 유사하게 나타나 해당 논문에서 제안한 방법이 타당함을 입증하였다. 해당 논문에서 제안한 방법은 수소 저장 시설뿐 아니라 다양한 지반 구조물의 안정성 평가 시 합리적인 데이터 구축에 활용 가능할 것으로 판단된다.

References
1

Chawla, N. V., Bowyer, K. W., Hall, L. O., and Kegelmeyer, W. P. (2002), "SMOTE: Synthetic Minority Over-sampling Technique", Journal of Artificial Intelligence Research, Vol.16, pp.321-357.

10.1613/jair.953
2

Choi, H. J., Kim, S. W., and Kim, Y. S. (2022), "A Basic Study on Effect Analysis of Adjacent Structures due to Explosion of Underground Hydrogen Infrastructure", Journal of Korean Geosynthetics Society, pp.21-27.

3

Cordón, I., García, S., Fernández, A., and Herrera, F. (2018), "Imbalance: Oversampling algorithms for Imbalanced Classification in R", Knowledge-Based Systems, Vol.161, pp.329-341.

10.1016/j.knosys.2018.07.035
4

Go, G. H., Jeon, J. S., Kim, Y. S., Kim, H. W., and Choi, H. J. (2022), "Prediction of Hydrodynamic Behavior of Unsaturated Ground Due to Hydrogen Gas Leakage in a Low-depth Underground Hydrogen Storage Facility", Journal of the Korean Geotechnical Society, Vol.38, No.11, pp.107-118.

5

Han, H., Wang, W. Y., and Mao, B. H. (2005, August), "Borderline-SMOTE: A New Over-sampling Method in Imbalanced Data Sets Learning", In International Conference on Intelligent Computing (pp.878-887), Berlin, Heidelberg: Springer Berlin Heidelberg.

10.1007/11538059_91
6

He, H., Bai, Y., Garcia, E. A., and Li, S. (2008, June), "ADASYN: Adaptive Synthetic Sampling Approach for Imbalanced Learning", In 2008 IEEE International Joint Conference on Neural Networks (IEEE world congress on computational intelligence) (pp.1322-1328), Ieee.

10.1109/IJCNN.2008.4633969
7

Kim, S. Y., Lee, D., Yu, J. D., and Yoon, H. K. (2024), "A Study on the Characteristics of Applying Oversampling Algorithms to Fosberg Fire-Weather Index (FFWI) data", Smart Structures and Systems, Vol.34, No.1, p.9.

8

Lee, W. S., Kim, Y., Shinn, Y., Wang, J., Moon, B., Park, H., ... and Kwon, O. (2021), "Role of Blue Hydrogen for Developing National Hydrogen Supply Infrastructure", Journal of the Korean Society of Mineral and Energy Resources Engineers, Vol.58, No.5, pp.503-520.

10.32390/ksmer.2021.58.5.503
9

Ning, Z. X., Su, M. X., Xue, Y. G., Qiu, D. H., Li, Z. Q., and Fu, K. (2021), "Reevaluation of the Design and Excavation of Underground Oil Storage Cavern Groups Using Numerical and Monitoring Approaches", Geomech Eng, Vol.27, No.3, pp.291-307.

10

Panfilov, M. (2016), Underground and Pipeline Hydrogen Storage, In Compendium of Hydrogen Energy (pp.91-115), Woodhead Publishing.

10.1016/B978-1-78242-362-1.00004-3
11

Rekha, G. and Reddy, V. K. (2018), "A Novel Approach for Handling Outliers in Imbalance Data", International Journal of Engineering & Technology, Vol.7, No.3.1, pp.1-5.

10.14419/ijet.v7i3.1.16783
12

Shin, J. W. (2023), "Damage Evaluation of Adjacent Structures for Detonation of Hydrogen Storage Facilities", Korean Society of Disaster & Security, Vol.16, No.1, pp.61-70.

13

Taylor, J. B., Alderson, J. E. A., Kalyanam, K. M., Lyle, A. B., and Phillips, L. A. (1986), "Technical and Economic Assessment of Methods for the Storage of Large Quantities of Hydrogen", International Journal of Hydrogen Energy, Vol.11, No.1, pp.5-22.

10.1016/0360-3199(86)90104-7
14

Zivar, D., Kumar, S., and Foroozesh, J. (2021), "Underground Hydrogen Storage: A Comprehensive Review", International Journal of Hydrogen Energy, Vol.46, No.45, pp.23436-23462.

10.1016/j.ijhydene.2020.08.138
Information
  • Publisher :The Korean Geotechnical Society
  • Publisher(Ko) :한국지반공학회
  • Journal Title :Journal of the Korean Geotechnical Society
  • Journal Title(Ko) :한국지반공학회 논문집
  • Volume : 40
  • No :5
  • Pages :93-101
  • Received Date : 2024-09-30
  • Revised Date : 2024-10-15
  • Accepted Date : 2024-10-18
  • DOI :https://doi.org/10.7843/kgs.2024.40.5.93
Journal Informaiton Journal of the Korean Geotechnical Society Journal of the Korean Geotechnical Society
Archives
: Vol.23~28 (2007~2012)
  • NRF
  • KOFST
  • KISTI Current Status
  • KISTI Cited-by
  • crosscheck
  • orcid
  • open access
  • ccl
Journal Informaiton Journal Informaiton - close