All Issue

2025 Vol.41, Issue 4 Preview Page
Optimization of DEM Pluviation Simulation for Sandy Soil Specimen Preparation Using High-performance Computing

고성능 컴퓨팅을 활용한 사질토 시료 조성의 DEM 낙사 시뮬레이션 최적화

Jin-Sun Lee1*
이 진선1*
1Member, Prof., Dept. of Civil and Environmental Engrg., Wonkwang Univ.
1정회원, 원광대학교 건설환경공학과 교수
*Corresponding Author
31 August 2025. pp. 111-128
PDF XML Citation
Abstract

This study aims to optimize the computational efficiency and parallel scalability of the discrete element method (DEM) by utilizing high-performance computing techniques. A DEM simulation was conducted to model the pluviation of sand particles, and several parallel processing methods, including Message Passing Interface (MPI), Open Multiprocessing (OpenMP, OMP), and Advanced Vector Extensions (AVX-512), were applied and benchmarked on Intel Xeon and AMD Threadripper CPU architectures. In particular, the application of Dynamic Load Balancing effectively mitigated computational load imbalance during MPI-based parallelization, significantly reducing total simulation time. The findings of this study provide practical guidelines for hardware and software configurations that enhance parallel performance in DEM analysis. These insights are expected to promote more efficient use of computational resources and enable large-scale laboratory simulations of granular soil behavior.

Keywords
DEM
Discrete element method
High performance computing
HPC
LAMMPS
Particle behavior

본 연구는 고성능 컴퓨팅(High Performance Computing, HPC)을 활용하여 개별요소법(Discrete Element Method, DEM)의 연산 효율과 병렬 확장성을 최적화하고자 하였다. 사질토 입자의 낙사 과정을 DEM해석으로 시뮬레이션하고, 다양한 병렬 연산 기법인 MPI(Message Passing Interface), OpenMP(Open Multi-Processing, OMP), AVX-512(Advanced Vector Extensions)를 Intel Xeon 및 AMD Threadripper CPU 환경에 적용하여 연산성능을 비교·분석하였다. 특히 동적 부하 분산(Dynamic Load Balancing, DLB) 기능을 통해 MPI 병렬화 과정에서 발생하는 연산 부하의 불균형을 개선함으로써 해석시간을 효과적으로 단축할 수 있음을 확인하였다. 본 연구는 DEM 해석에서 병렬연산 성능의 최적화를 위한 하드웨어 및 소프트웨어 설정 기준 및 벤치마킹 절차를 제시하며, 향후 사질토 입자 시뮬레이션의 실내시험 크기 확장과 연산 자원 활용의 효율화를 가능하게 할 것으로 기대한다.

References
1

Advanced Micro Devices (2022), AMD Ryzen™ Threadripper™ PRO processors: The third wave of workstation computing (White paper), https://www.amd.com/content/dam/amd/en/documents/processor-tech-docs/white-papers/threadripperpro-white-paper-third-wave-workstation-computing.pdf.

2

Altair Engineering, Inc. (n.d.), EDEM Applications, https://altair.co.kr/edem-applications.

3

Amdahl, G. M. (1967), “Validity of the single processor approach to achieving large scale computing capabilities”, In Proceedings of the AFIPS Spring Joint Computer Conference, 483-485, https://doi.org/10.1145/1465482.1465560.

10.1145/1465482.1465560
4

Angelidakis, V., Boschi, K., Brzeziński, K., Caulk, R.A., Chareyre, B., del Valle, C.A., Duriez, J., Gladky, A., van der Haven, D.L.H., Kozicki, J., Pekmezi, G., Scholtes, L., and Thoeni, K. (2024), “YADE - An Extensible Framework for the Interactive Simulation of Multiscale, Multiphase, and Multiphysics Particulate Systems”, Computer Physics Communications, Vol.304, 109293, https://doi.org/10.1016/j.cpc.2024.109293.

10.1016/j.cpc.2024.109293
5

ANSYS, Inc. (n.d.), ANSYS Rocky, https://www.ansys.com/products/fluids/ansys-rocky.

6

Berger, R., Kloss, C., Kohlmeyer, A., and Pirker, S. (2015), “Hybrid parallelization of the LIGGGHTS open-source DEM code”, Powder Technology, Vol.278, pp.234-247.

10.1016/j.powtec.2015.03.019
7

Canonical Ltd. (n.d.), Ubuntu Server, https://ubuntu.com/server.

8

Cundall, PA and Strack, OD (1979), “A discrete numerical model for granular assemblies”, Géotechnique, Vol.29, pp.47-65, https://doi.org/10.1680/geot.1979.29.1.47.

10.1680/geot.1979.29.1.47
9

DCS Computing GmbH (n.d.), Aspherix® - Advanced DEM Simulation Software. https://www.aspherix-dem.com/.

10

DCS Computing GmbH (n.d.), LIGGGHTS® - Open Source Discrete Element Method Particle Simulation Code, https://www.cfdem.com/liggghts-open-source-discrete-element-method-particle-simulation-code.

11

Dosta, M. and Skorych, V. (2020), “MUSEN: An open-source framework for GPU-accelerated DEM simulations”, SoftwareX, Vol.12, 100618, https://doi.org/10.1016/j.softx.2020.100618.

10.1016/j.softx.2020.100618
12

Dosta, M., et al. (2024), “Comparing open-source DEM frameworks for simulations of common bulk processes”, Computer Physics Communications, Vol.296, pp.109066, https://doi.org/10.1016/j.cpc.2023.109066.

10.1016/j.cpc.2023.109066
13

Dutta, A.K. and Penumadu, D. (2007), “Hardness and Modulus of Individual Sand Particles Using Nanoindentation”, Geo-Denver 2007: Advances in Measurement and Modeling of Soil Behavior, https://doi.org/10.1061/40917(236)34.

10.1061/40917(236)34
14

GROMACS (n.d.), GROMACS molecular dynamics software, https://www.gromacs.org/.

15

Hertz, H. (1881), “Über die Berührung fester elastischer Körper”, Journal für die reine und angewandte Mathematik, Vol.92, pp.156-171, https://doi.org/10.1515/crll.1882.92.156.

10.1515/crll.1882.92.156
16

HPC Wiki (n.d.), Scaling, https://hpc-wiki.info/hpc/Scaling.

17

Intel Corporation (n.d.), oneAPI Overview, https://www.intel.com/content/www/us/en/developer/tools/oneapi/overview.html.

18

Itasca Consulting Group (n.d.), PFC https://www.itascacg.com

19

Khronos Group (2025), OpenCL 3.0 unified specification, https://registry.khronos.org/OpenCL/.

20

Kitware, Inc. (n.d.), VTK: The Visualization Toolkit, https://vtk.org/.

21

Kitware, Inc. (n.d.), CMake, https://cmake.org/.

22

Kitware, Inc. (n.d.), ParaView: Open-Source Scientific Visualization, https://www.paraview.org/.

23

Kloss, C., Goniva, C., Hager, A., Amberger, S., and Pirker, S. (2012), “Models, algorithms and validation for opensource DEM and CFD-DEM”, Progress in Computational Fluid Dynamics, Vol.12(2-3), pp.140-152, https://doi.org/10.1504/PCFD.2012.047457.

10.1504/PCFD.2012.047457
24

Kokkos Project (n.d.), Kokkos: Performance portability programming model, GitHub, https://github.com/kokkos/kokkos.

25

Kuhn, MR, Renken, HE, Mixsell, AD, and Kramer, SL (2014), “Investigation of Cyclic Liquefaction with Discrete Element Simulations”, J. Geotech. Geoenviron. Eng., Vol.140, https://doi.org/10.1061/(ASCE)GT.1943-5606.0001181.

10.1061/(ASCE)GT.1943-5606.0001181
26

Mindlin, R.D. and Deresiewicz, H. (1953), “Elastic Spheres in Contact under Varying Oblique Force”, Trans. ASME, J. Appl. Mech., Vol.20, pp.327-344, https://doi.org/10.1115/1.4010702.

10.1115/1.4010702
27

MPICH Project (n.d.), MPICH: Official website, https://www.mpich.org/.

28

NVIDIA (n.d.), CUDA Toolkit Documentation, https://developer.nvidia.com/cuda-zone.

29

OpenMP Architecture Review Board (n.d.), https://www.openmp.org.

30

Open MPI Project (n.d.), Open MPI: Official website, https://www.open-mpi.org/.

31

O’Sullivan C (2017), Particulate Discrete Element Modelling: A Geomechanics Perspective, CRC Press. ISBN 9781138074897.

32

Owens, JD, Luebke, D, Govindaraju, N, Harris, M, Krüger, J, Lefohn, AE, and Purcell, T (2007), “A survey of general‑purpose computation on graphics hardware”, Computer Graphics Forum, Vol.26(1), pp.80-113, https://doi.org/10.1111/j.1467-8659.2007.01012.x.

10.1111/j.1467-8659.2007.01012.x
33

ROCm Project (n.d.), HIP (Heterogeneous-computing Interface for Portability), https://github.com/ROCm/hip.

34

Syamlal M., Rogers W., and O’Brien T. (1993), “MFIX Documentation: Theory Guide”, DOE/METC-94/1004, https://doi.org/10.2172/10145548.

10.2172/10145548
35

Thompson, AP et al. (2022), “LAMMPS - A flexible simulation tool for particle-based materials modeling at the atomic, meso, and continuum scales”, Computer Physics Communications, Vol.271, pp.108171, https://doi.org/10.1016/j.cpc.2021.108171.

10.1016/j.cpc.2021.108171
36

Tsuji, Y., Tanaka, T., and Ishida, T. (1992), “Lagrangian numerical simulation of plug flow of cohesionless particles in a horizontal pipe”, Powder Technology, Vol.71, pp.239-250, https://doi.org/10.1016/0032-5910(92)88030-L.

10.1016/0032-5910(92)88030-L
37

Wang, G., Huang, D., and Wei, J. (2018), “Discrete Element Simulation of Soil Liquefaction: Fabric Evolution, Large Deformation, and Multi-Directional Loading”, Proceedings of Geotechnical Earthquake Engineering and Soil Dynamics V., ASCE, https://doi.org/10.1061/9780784481479.013.

10.1061/9780784481479.013
38

Wang, L., Park, J.-Y., and Fu, Y. (2007), “Representation of real particles for DEM simulation using X-ray tomography”, Constr. Build. Mater, Vol.21, pp.338-346, http://doi.org/10.1016/j.conbuildmat.2005.08.013.

10.1016/j.conbuildmat.2005.08.013
39

Wang, Y. and Mora, P. (2009), “The ESyS_Particle: A New 3-D Discrete Element Model with Single Particle Rotation”, In: Advances in Geocomputing, Lecture Notes in Earth Sciences, Vol.119, pp.183-194, https://doi.org/10.1007/978-3-540-85879-9_6.

10.1007/978-3-540-85879-9_6
40

Wikipedia contributors (n.d.), AVX-512, Wikipedia, https://en.wikipedia.org/wiki/AVX-512.

41

Wikipedia contributors (n.d.), LLVM, Wikipedia, https://ko.wikipedia.org/wiki/LLVM.

42

Wikipedia contributors (n.d.), NUMA: Non-uniform memory access, Wikipedia, https://en.wikipedia.org/wiki/Non-uniform_memory_access.

43

Wikipedia contributors (n.d.), Verlet integration, Wikipedia, https://en.wikipedia.org/wiki/Verlet_integration.

44

Xu, M.Q., Guo, N., and Yang, Z.X. (2021), “Particle shape effects on the shear behaviors of granular assemblies: irregularity and elongation”, Granular Matter, Vol.23, 25, https://doi.org/10.1007/s10035-021-01096-4.

10.1007/s10035-021-01096-4
45

Zhang, H.P. and Makse, H.A. (2005), “Jamming Transition in Emulsions and Granular Materials”, Physical Review E, Vol.72(1), pp.011301, https://doi.org/10.1103/PhysRevE.72.011301.

10.1103/PhysRevE.72.011301
46

Zhao, J., Zhao, S., and Luding, S. (2023), “The role of particle shape in computational modelling of granular matter”, Nat Rev Phys 5, 505-525, https://doi.org/10.1038/s42254-023-00617-9.

10.1038/s42254-023-00617-9
47

Zheng, J. and Hryciw, R.D. (2016), “Roundness and Sphericity of Soil Particles in Assemblies by Computational Geometry”, Journal of Computing in Civil Engineering, Vol.30, No.6, https://doi.org/10.1061/(ASCE)CP.1943-5487.0000578.

10.1061/(ASCE)CP.1943-5487.0000578
48

Zheng, J. and Hryciw, R.D. (2017), “An image based clump library for DEM simulations”, Granular Matter, Vol.19, 26, https://doi.org/10.1007/s10035-017-0713-x.

10.1007/s10035-017-0713-x
49

Zhou, Y., Wang, H., Zhou, B., and, Li J. (2018), “DEM-aided direct shear testing of granular sands incorporating realistic particle shape”, Granular Matter, Vol.20, 55, https://doi.org/10.1007/s10035-018-0828-8.

10.1007/s10035-018-0828-8
Information
  • Publisher :The Korean Geotechnical Society
  • Publisher(Ko) :한국지반공학회
  • Journal Title :Journal of the Korean Geotechnical Society
  • Journal Title(Ko) :한국지반공학회 논문집
  • Volume : 41
  • No :4
  • Pages :111-128
  • Received Date : 2025-07-07
  • Revised Date : 2025-07-24
  • Accepted Date : 2025-07-31
  • DOI :https://doi.org/10.7843/kgs.2025.41.4.111
Journal Informaiton Journal of the Korean Geotechnical Society Journal of the Korean Geotechnical Society
Archives
: Vol.23~28 (2007~2012)
  • NRF
  • KOFST
  • KISTI Current Status
  • KISTI Cited-by
  • crosscheck
  • orcid
  • open access
  • ccl
Journal Informaiton Journal Informaiton - close