1. 17-Nov-18
Parallel 2D and 3D Acoustic Modeling Application for hybrid computing platform of PARAM Yuva II
Abhishek Srivastava, Ashutosh Londhe*, Richa Rastogi
PARCOMPTECH 2017
Presented By:
Ashutosh Londhe
Project Engineer
Seismic Data Processing (SDP) team,
High Performance Computing – Scientific and Engineering Application (HPC-S&EA) group,
Centre for Development of Advanced Computing (CDAC), Pune

2. Content Background Objective Methodology Computational complexity
17-Nov-18
Content
Background
Objective
Methodology
Computational complexity
Serial algorithm
Parallel implementation
Results
Conclusion
17-Nov-18

3. Background Synthetic seismogram Wavefield BP 2004 velocity model
17-Nov-18
Background
Synthetic seismogram
BP 2004 velocity model
Source wavelet
Wavefield
17-Nov-18

4. 17-Nov-18
Objective
Development of parallel 2D/3D seismic acoustic modeling application using staggered grid Finite Difference Method (FDM).
Objective of acoustic modelling is to generate synthetic seismogram for given earth model by propagating acoustic wave through earth subsurface model.
17-Nov-18

5. Methodology – Finite Difference operator
17-Nov-18
Methodology – Finite Difference operator
𝜕 2 𝑃 𝜕 𝑡 2 = 𝑐 2 𝑃 𝜕 𝑥 2 𝜕 𝑥 𝜕 𝑧 2 𝜕 𝑧 𝐹(𝑥,𝑧) (1)
𝜕 2 𝑃 𝜕 𝑡 2 = 𝑐 2 𝑃 𝜕 𝑥 2 𝜕 𝑥 𝜕 𝑦 2 𝜕 𝑦 𝜕 𝑧 2 𝜕 𝑧 𝐹(𝑥,𝑦,𝑧) (2)
Where, P is acoustic pressure, c is acoustic wave speed, F is the source and (x, y, z) is dimensional positions
approximation on bounded model space for corresponding unbounded model space
17-Nov-18

6. Methodology – Staggered grid and Leap frog scheme
17-Nov-18
Methodology – Staggered grid and Leap frog scheme
Staggered grid: Leap frog scheme:
Importance and uses
What is staggered grid- acoustic wave equation has two variables pressure and velocity. In case of staggered pressure variable are located at grid center and velocity variables are located on grid face.
Use: enable evaluation of partial derivatives over smaller grid interval which increases resolution of results and decrease truncation error.
truncation error in fdm depends upon number of terms used in Taylor’s series.
Leap frog schemes is used for discretisation of continuous function. In this scheme position and velocity is updated at interleaved time steps. Which u can see in fig that position is updated at dt timestep and velocity at dt+1/2 time step
17-Nov-18

7. Computational complexity
17-Nov-18
Computational complexity
Description
Values
Total grid points
FD Time Step
2500
Grid point evaluated
9.58 * 109
Floating-point Operations to update one grid point 18
Memory required by one grid point (in Bytes)
4 Bytes
Total floating-point operations for one simulation
1.73 * 1011
Total memory required (in Bytes)
38.32 * X*
X*: Runtime memory
Floating point operations/grid point = [ (7*k)/2] + 4, where k is order of FDM
model size: 540m X 3500m X 4180m with grid spacing of 20m
Time sampling rate: 0.002s and seismogram recorded for 5s
17-Nov-18

8. 2D/3D Acoustic modeling Algorithm
17-Nov-18
2D/3D Acoustic modeling Algorithm
Algorithm to generate synthetic seismogram using acoustic modelling is depicted in fig. Velocity and density model are two major inputs. Other inputs to the application include, grid spacing, time sampling rate, peak frequency for source wavelet and source and receiver geometry for which seismogram has to be generated and other relevant inputs.
The velocity model is first checked for dispersion and stability criteria. Dispersion check is performed for checking whether the selected grid spacing will result in dispersion in result or not.  finite difference approximation is stable if the errors (truncation, round-off etc) decay as the computation proceeds from one marching step to the next.
Then model is extended for absorbing wave at boundaries so there will be no reflection.
Now for each shot we have to propagate the wave using FDM and record the reflection arises from interfaces in earth subsurface model on the receivers of the shot.
Source wavelet is used to generate wave which get propagated in earth subsurface as acoustic wave. Here we have used a Ricker wavelet as a source wavelet.
The above steps are repeated for each shot.
17-Nov-18

9. Parallel implementation
17-Nov-18
Parallel implementation
The fig. shows the algorithm implementation. The data decomposition in our case shot workload distribution is achieved using MPI. Further parallelisation is achieved thro’ OpenMP implementation in FD module as shown by red box.
17-Nov-18

10. 17-Nov-18
Optimizations
Apart from MPI and OpenMP parallelization the application performance is enhanced using the different optimization methodologies shown in fig.
17-Nov-18

11. List of relevant parameter used for 2D & 3D acoustic modeling
17-Nov-18
List of relevant parameter used for 2D & 3D acoustic modeling
Description
2D Synthetic Data
3D Extracted Data
Model Y dimension (y) -
540m
Model X dimension (x)
60000m
13500m
Model Z dimension (z)
40000m
4180m
Grid size in Y dimension (dy)
20m
Grid size in X dimension (dx)
40m
Grid size in Z dimension (dz)
Grid point in Y direction (ny) 27
Grid point in X direction (nx)
1501
676
Grid point in Z direction (nz)
1001
210
Sampling interval for FDTD (dt)
0.003s
0.002s
Resampling interval (Seisdt)
0.008s
Record Length (RecLen)
15.0s
5.0s
Grid points for PML (NSP) 60
17-Nov-18

12. CPU compute time for 100 shots of 2D/3D wave propagation
17-Nov-18
2D/3D parallel acoustic modeling compute time
CPU compute time for 100 shots of 2D/3D wave propagation
17-Nov-18

13. CPU Node scaling & efficiency for 100 shots of 2D wave propagation
17-Nov-18
2D parallel acoustic modeling node scaling and efficiency
CPU Node scaling & efficiency for 100 shots of 2D wave propagation
17-Nov-18

14. CPU Node scaling & efficiency for 100 shots of 3D wave propagation
17-Nov-18
3D parallel acoustic modeling node scaling and efficiency
CPU Node scaling & efficiency for 100 shots of 3D wave propagation
17-Nov-18

15. 2D wave propagation snapshots at different time
17-Nov-18
2D acoustic wavefield
2D Velocity model
2D wave propagation snapshots at different time
17-Nov-18

16. Generated 2D synthetic seismogram.
17-Nov-18
2D acoustic seismogram
Generated 2D synthetic seismogram.
17-Nov-18

17. 3D wave propagation snapshot at 1 s, 2 s, 3 s and 5 s
3D acoustic wavefield
3D Velocity model
17-Nov-18
3D wave propagation snapshot at 1 s, 2 s, 3 s and 5 s

18. Generated 3D synthetic seismogram.
17-Nov-18
3D acoustic seismogram
Generated 3D synthetic seismogram.
17-Nov-18

19. Conclusion
The developed modeling technique is efficient, scalable and robust for modeling complex subsurface.
Load imbalance occurs in case of uneven distribution of shot points.
Further work in this is to implement domain decomposition for 3D modeling.
It will be major module for seismic imaging techniques such as Full Waveform Inversion (FWI) and Reverse Time Migration (RTM) which give detailed structure of earths subsurface.
17-Nov-18

20. 17-Nov-18
Thank You
17-Nov-18