.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "gallery/tutorials/cli.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_gallery_tutorials_cli.py: 07. Command-line interface ========================== The implemented command-line interface (CLI) can be powerful to use `emg3d` from any programming environment. It enables, for instance, to use `emg3d` as a forward modelling kernel in your inversion code running on a server. The capabilities of the CLI are more limited than of the Python interface, but they are sufficient to compute the responses of a given `survey` for a provided `model`, and to compute the gradient of the misfit function. The biggest difficulty to work with the CLI is, at the moment, the file formats, or better the lack of their documentation. Surveys, models, and grids can be stored with `emg3d` to either h5, npz, or json files. The status of the documentation will improve towards the v1.0.0 release of `emg3d`. The "easiest" way at the moment is to generate such a file in Python, and reproduce its structure in your language or program of choice. Please feel free to open an issue on GitHub if there are questions in this regard. The idea for the future is to go through https://github.com/softwareunderground/subsurface to interface to various file formats. .. GENERATED FROM PYTHON SOURCE LINES 23-31 .. code-block:: default import os import pooch import subprocess # Adjust this path to a folder of your choice. data_path = os.path.join('..', 'download', '') .. GENERATED FROM PYTHON SOURCE LINES 33-38 Note that everything shown in this example is meant to be executed in a terminal, nothing is executed in Python. However, as this example gallery is generated in Python we have to use a work-around to show the functionality, the function ``bash(command)``. You would execute the provided ``command`` in your terminal. .. GENERATED FROM PYTHON SOURCE LINES 38-52 .. code-block:: default def bash(command): "Prints the `command`, executes it in bash, and prints the output.""" # Print command print(f"$ {command}") # Move to data_path command = f"cd {data_path}; " + command # Carry out command msg = subprocess.run(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) # Print output print(msg.stdout.decode()) .. GENERATED FROM PYTHON SOURCE LINES 53-55 We have to fetch the example files. You can also download these manually or provide your own survey, model, and parameter files. .. GENERATED FROM PYTHON SOURCE LINES 55-71 .. code-block:: default model = "GemPy-II.h5" pooch.retrieve( 'https://raw.github.com/emsig/data/2021-05-21/emg3d/models/'+model, 'ea8c23be80522d3ca8f36742c93758370df89188816f50cb4e1b2a6a3012d659', fname=model, path=data_path, ) survey = 'GemPy-II-survey-A.h5' pooch.retrieve( 'https://raw.github.com/emsig/data/2021-05-21/emg3d/surveys/'+survey, '5f2ed0b959a4f80f5378a071e6f729c6b7446898be7689ddc9bbd100a8f5bce7', fname=survey, path=data_path, ) .. rst-class:: sphx-glr-script-out .. code-block:: none '/home/dtr/3-GitHub/emsig/emg3d-gallery/examples/download/GemPy-II-survey-A.h5' .. GENERATED FROM PYTHON SOURCE LINES 72-76 Getting started - help ---------------------- The best way to get started is, as with any bash command, to have a look at its in-built help. .. GENERATED FROM PYTHON SOURCE LINES 76-80 .. code-block:: default bash("emg3d --help") .. rst-class:: sphx-glr-script-out .. code-block:: none $ emg3d --help usage: emg3d [-h] [-n NPROC] [-f | -m | -g] [--path PATH] [--survey SURVEY] [--model MODEL] [--output OUTPUT] [--save SAVE] [--load LOAD] [--cache CACHE] [--clean] [-l] [-d] [--verbosity {-1,0,1,2} | -v | -q] [--report] [--version] [config] Multigrid solver for 3D electromagnetic diffusion. positional arguments: config name of config file; default is 'emg3d.cfg'; consult https://emg3d.emsig.xyz/en/stable/manual/cli.html for its format optional arguments: -h, --help show this help message and exit -n NPROC, --nproc NPROC number of processors -f, --forward compute synthetic data (default) -m, --misfit compute synthetic data and their misfit -g, --gradient compute synthetic data, misfit, and its gradient --path PATH path (abs or rel); file names are relative to path --survey SURVEY input survey file name; default is 'survey.h5' --model MODEL input model file name; default is 'model.h5' --output OUTPUT output files base name; default is 'emg3d_out' --save SAVE file-name to save simulation; not used by default --load LOAD file-name to load simulation; not used by default; if set, the provided survey and almost all parameters in the config file are ignored; provided model is only used if --clean is set --cache CACHE shortcut: file-name for --load --save (cache overrules load and save) --clean replace model and all computed data of loaded simulation -l, --layered use a layered model with empymod for each src-rec pair -d, --dry-run only display what would have been done --verbosity {-1,0,1,2} set verbosity; default is 0 -v, --verbose increase verbosity; can be used multiple times -q, --quiet decrease verbosity --report only display emg3d report --version only display emg3d version .. GENERATED FROM PYTHON SOURCE LINES 81-92 Configuration file ------------------ The CLI is driven by a configuration file, that is by default named ``emg3d.cfg`` and must be in the same directory as you execute the command. If it has a different name or a different location you have to provide the full or relative path and filename as the first argument to ``emg3d``. The configuration parameters are described in the documentation, consult `Manual -> CLI `_. Let's write a simple example file. .. GENERATED FROM PYTHON SOURCE LINES 92-114 .. code-block:: default with open(f"{data_path}emg3d.cfg", "w") as f: f.write("""[files] survey = GemPy-II-survey-A.h5 model = GemPy-II.h5 output = emg3d_out.json [simulation] name = CLI Test gridding = single [solver_opts] # nothing specified [gridding_opts] center_on_edge = False [data] sources = TxED-1 receivers = RxEP-08, RxEP-38 frequencies = f-1""") .. GENERATED FROM PYTHON SOURCE LINES 115-125 The file defines the location of the survey and model files, the name and format of the output file, and selects a source, two receivers, and a frequency to compute. Compute the forward responses ----------------------------- We just compute the forward response here, but the usage for the misfit or the gradient is very similar. We use the most verbose version here, to see what it does internally. .. GENERATED FROM PYTHON SOURCE LINES 125-129 .. code-block:: default bash("emg3d --forward -vv") .. rst-class:: sphx-glr-script-out .. code-block:: none $ emg3d --forward -vv :: emg3d CLI forward START :: Wed Aug 31 21:34:45 2022 :: v1.8.0 -------------------------------------------------------------------------------- Date: Wed Aug 31 21:34:45 2022 CEST OS : Linux CPU(s) : 4 Machine : x86_64 Architecture : 64bit RAM : 15.5 GiB Environment : Python File system : ext4 Python 3.9.12 | packaged by conda-forge | (main, Mar 24 2022, 23:22:55) [GCC 10.3.0] numpy : 1.22.4 scipy : 1.9.0 numba : 0.55.2 emg3d : 1.8.0 empymod : 2.2.0 xarray : 2022.6.0 discretize : 0.8.2 h5py : 3.7.0 matplotlib : 3.4.3 tqdm : 4.64.0 IPython : 8.4.0 Intel(R) oneAPI Math Kernel Library Version 2022.0-Product Build 20211112 for Intel(R) 64 architecture applications -------------------------------------------------------------------------------- :: CONFIGURATION :: /home/dtr/3-GitHub/emsig/emg3d-gallery/examples/download/emg3d.cfg { "data": { "frequencies": [ "f-1" ], "receivers": [ "RxEP-08", "RxEP-38" ], "sources": [ "TxED-1" ] }, "files": { "load": false, "log": "/home/dtr/3-GitHub/emsig/emg3d-gallery/examples/download/emg3d_out.log", "model": "/home/dtr/3-GitHub/emsig/emg3d-gallery/examples/download/GemPy-II.h5", "output": "/home/dtr/3-GitHub/emsig/emg3d-gallery/examples/download/emg3d_out.json", "save": false, "survey": "/home/dtr/3-GitHub/emsig/emg3d-gallery/examples/download/GemPy-II-survey-A.h5" }, "noise_kwargs": {}, "simulation_options": { "gridding": "single", "gridding_opts": { "center_on_edge": false }, "name": "CLI Test" } } :: LOAD SURVEY AND MODEL :: Data loaded from «/home/dtr/3-GitHub/emsig/emg3d-gallery/examples/download/GemPy-II-survey-A.h5» [emg3d v0.17.1.dev22+ge23c468 (format 1.0) on 2021-04-14T22:24:03.229285]. Data loaded from «/home/dtr/3-GitHub/emsig/emg3d-gallery/examples/download/GemPy-II.h5» [emg3d v1.0.0rc3.dev5+g0cd9e09 (format 1.0) on 2021-05-21T18:40:16.721968]. :: SIMULATION :: :: Simulation «CLI Test» :: - Survey «GemPy-II Survey A»: 1 sources; 2 receivers; 1 frequencies - Model: resistivity; isotropic; 100 x 100 x 102 (1,020,000) - Gridding: Single grid for all sources and frequencies; 128 x 64 x 64 (524,288) :: MESHES :: = Source: all; Frequency: all = == GRIDDING IN X == Skin depth [m] : 390 / 7117625 / 7117625 [corr. to `properties`] Survey dom. DS [m] : 4000 - 16000 Comp. dom. DC [m] : -96000 - 116000 Final extent [m] : -101181 - 121187 Cell widths [m] : 130 / 130 / 28717 [min(DS) / max(DS) / max(DC)] Number of cells : 128 (94 / 34 / 0) [Total (DS/DC/remain)] Max stretching : 1.000 (1.000) / 1.374 [DS (seasurface) / DC] == GRIDDING IN Y == Skin depth [m] : 390 / 7117625 / 7117625 [corr. to `properties`] Survey dom. DS [m] : 5500 - 9500 Comp. dom. DC [m] : -94500 - 109500 Final extent [m] : -97055 - 153667 Cell widths [m] : 130 / 130 / 41613 [min(DS) / max(DS) / max(DC)] Number of cells : 64 (31 / 33 / 0) [Total (DS/DC/remain)] Max stretching : 1.000 (1.000) / 1.404 [DS (seasurface) / DC] == GRIDDING IN Z == Skin depth [m] : 390 / 2251 / 7117625 [corr. to `properties`] Survey dom. DS [m] : -6725 - -1722 Comp. dom. DC [m] : -20867 - 98278 Final extent [m] : -23065 - 102852 Cell widths [m] : 130 / 130 / 32306 [min(DS) / max(DS) / max(DC)] Number of cells : 64 (39 / 25 / 0) [Total (DS/DC/remain)] Max stretching : 1.000 (1.000) / 1.444 [DS (seasurface) / DC] TensorMesh: 524,288 cells MESH EXTENT CELL WIDTH FACTOR dir nC min max min max max --- --- --------------------------- ------------------ ------ x 128 -101,180.95 121,187.06 129.95 28,716.91 1.37 y 64 -97,054.53 153,667.41 129.95 41,612.88 1.40 z 64 -23,064.76 102,852.13 129.95 32,306.29 1.44 :: FORWARD COMPUTATION :: Compute efields 0/1 [00:00] Compute efields ██████████ 1/1 [00:42] Compute efields ██████████ 1/1 [00:42] - SOLVER INFO - = Source TxED-1; Frequency 0.5 Hz = 7.5e-07; 1(5); 0:00:42; CONVERGED :: SAVE RESULTS :: Data saved to «/home/dtr/3-GitHub/emsig/emg3d-gallery/examples/download/emg3d_out.json» [emg3d v1.8.0 (format 1.0) on 2022-08-31T21:35:28.151599]. :: emg3d CLI forward END :: Wed Aug 31 21:35:28 2022 :: runtime = 0:00:43 .. GENERATED FROM PYTHON SOURCE LINES 130-136 Generated files --------------- The run created the files ``emg3d_out.json`` (name and file-type can be changed in the config file), which contains the responses, and the file ``emg3d_out.log`` with some information. .. GENERATED FROM PYTHON SOURCE LINES 136-140 .. code-block:: default bash("ls emg3d* -1") .. rst-class:: sphx-glr-script-out .. code-block:: none $ ls emg3d* -1 emg3d.cfg emg3d_out.json emg3d_out.log .. GENERATED FROM PYTHON SOURCE LINES 141-146 Log --- Let's have a look at the log, which is mostly the same as was printed above with the verbosity flag: .. GENERATED FROM PYTHON SOURCE LINES 146-150 .. code-block:: default bash("cat emg3d_out.log") .. rst-class:: sphx-glr-script-out .. code-block:: none $ cat emg3d_out.log :: emg3d CLI forward START :: Wed Aug 31 21:34:45 2022 :: v1.8.0 -------------------------------------------------------------------------------- Date: Wed Aug 31 21:34:45 2022 CEST OS : Linux CPU(s) : 4 Machine : x86_64 Architecture : 64bit RAM : 15.5 GiB Environment : Python File system : ext4 Python 3.9.12 | packaged by conda-forge | (main, Mar 24 2022, 23:22:55) [GCC 10.3.0] numpy : 1.22.4 scipy : 1.9.0 numba : 0.55.2 emg3d : 1.8.0 empymod : 2.2.0 xarray : 2022.6.0 discretize : 0.8.2 h5py : 3.7.0 matplotlib : 3.4.3 tqdm : 4.64.0 IPython : 8.4.0 Intel(R) oneAPI Math Kernel Library Version 2022.0-Product Build 20211112 for Intel(R) 64 architecture applications -------------------------------------------------------------------------------- :: CONFIGURATION :: /home/dtr/3-GitHub/emsig/emg3d-gallery/examples/download/emg3d.cfg { "data": { "frequencies": [ "f-1" ], "receivers": [ "RxEP-08", "RxEP-38" ], "sources": [ "TxED-1" ] }, "files": { "load": false, "log": "/home/dtr/3-GitHub/emsig/emg3d-gallery/examples/download/emg3d_out.log", "model": "/home/dtr/3-GitHub/emsig/emg3d-gallery/examples/download/GemPy-II.h5", "output": "/home/dtr/3-GitHub/emsig/emg3d-gallery/examples/download/emg3d_out.json", "save": false, "survey": "/home/dtr/3-GitHub/emsig/emg3d-gallery/examples/download/GemPy-II-survey-A.h5" }, "noise_kwargs": {}, "simulation_options": { "gridding": "single", "gridding_opts": { "center_on_edge": false }, "name": "CLI Test" } } :: LOAD SURVEY AND MODEL :: Data loaded from «/home/dtr/3-GitHub/emsig/emg3d-gallery/examples/download/GemPy-II-survey-A.h5» [emg3d v0.17.1.dev22+ge23c468 (format 1.0) on 2021-04-14T22:24:03.229285]. Data loaded from «/home/dtr/3-GitHub/emsig/emg3d-gallery/examples/download/GemPy-II.h5» [emg3d v1.0.0rc3.dev5+g0cd9e09 (format 1.0) on 2021-05-21T18:40:16.721968]. :: SIMULATION :: :: Simulation «CLI Test» :: - Survey «GemPy-II Survey A»: 1 sources; 2 receivers; 1 frequencies - Model: resistivity; isotropic; 100 x 100 x 102 (1,020,000) - Gridding: Single grid for all sources and frequencies; 128 x 64 x 64 (524,288) :: MESHES :: = Source: all; Frequency: all = == GRIDDING IN X == Skin depth [m] : 390 / 7117625 / 7117625 [corr. to `properties`] Survey dom. DS [m] : 4000 - 16000 Comp. dom. DC [m] : -96000 - 116000 Final extent [m] : -101181 - 121187 Cell widths [m] : 130 / 130 / 28717 [min(DS) / max(DS) / max(DC)] Number of cells : 128 (94 / 34 / 0) [Total (DS/DC/remain)] Max stretching : 1.000 (1.000) / 1.374 [DS (seasurface) / DC] == GRIDDING IN Y == Skin depth [m] : 390 / 7117625 / 7117625 [corr. to `properties`] Survey dom. DS [m] : 5500 - 9500 Comp. dom. DC [m] : -94500 - 109500 Final extent [m] : -97055 - 153667 Cell widths [m] : 130 / 130 / 41613 [min(DS) / max(DS) / max(DC)] Number of cells : 64 (31 / 33 / 0) [Total (DS/DC/remain)] Max stretching : 1.000 (1.000) / 1.404 [DS (seasurface) / DC] == GRIDDING IN Z == Skin depth [m] : 390 / 2251 / 7117625 [corr. to `properties`] Survey dom. DS [m] : -6725 - -1722 Comp. dom. DC [m] : -20867 - 98278 Final extent [m] : -23065 - 102852 Cell widths [m] : 130 / 130 / 32306 [min(DS) / max(DS) / max(DC)] Number of cells : 64 (39 / 25 / 0) [Total (DS/DC/remain)] Max stretching : 1.000 (1.000) / 1.444 [DS (seasurface) / DC] TensorMesh: 524,288 cells MESH EXTENT CELL WIDTH FACTOR dir nC min max min max max --- --- --------------------------- ------------------ ------ x 128 -101,180.95 121,187.06 129.95 28,716.91 1.37 y 64 -97,054.53 153,667.41 129.95 41,612.88 1.40 z 64 -23,064.76 102,852.13 129.95 32,306.29 1.44 :: FORWARD COMPUTATION :: - SOLVER INFO - = Source TxED-1; Frequency 0.5 Hz = 7.5e-07; 1(5); 0:00:42; CONVERGED :: SAVE RESULTS :: Data saved to «/home/dtr/3-GitHub/emsig/emg3d-gallery/examples/download/emg3d_out.json» [emg3d v1.8.0 (format 1.0) on 2022-08-31T21:35:28.151599]. :: emg3d CLI forward END :: Wed Aug 31 21:35:28 2022 :: runtime = 0:00:43 .. GENERATED FROM PYTHON SOURCE LINES 151-157 Result / output --------------- We stored the output as a json-file, just so we can easily cat it. However, you might want to use h5 files, as they are compressed. Note that json cannot store complex numbers, so the responses are stored as ``[[Re], [Im]]``. .. GENERATED FROM PYTHON SOURCE LINES 157-161 .. code-block:: default bash("cat emg3d_out.json") .. rst-class:: sphx-glr-script-out .. code-block:: none $ cat emg3d_out.json { "configuration": { "files": { "save": false, "load": false, "survey": "/home/dtr/3-GitHub/emsig/emg3d-gallery/examples/download/GemPy-II-survey-A.h5", "model": "/home/dtr/3-GitHub/emsig/emg3d-gallery/examples/download/GemPy-II.h5", "output": "/home/dtr/3-GitHub/emsig/emg3d-gallery/examples/download/emg3d_out.json", "log": "/home/dtr/3-GitHub/emsig/emg3d-gallery/examples/download/emg3d_out.log" }, "simulation_options": { "gridding": "single", "name": "CLI Test", "gridding_opts": { "center_on_edge": false } }, "data": { "sources": [ "TxED-1" ], "receivers": [ "RxEP-08", "RxEP-38" ], "frequencies": [ "f-1" ] }, "noise_kwargs": {} }, "data__complex__array-float64": [ [ [ [ -3.643887691173835e-08 ], [ NaN ] ] ], [ [ [ -3.3514219146614606e-09 ], [ NaN ] ] ] ], "_date": "2022-08-31T21:35:28.151599", "_version": "emg3d v1.8.0", "_format": "1.0" } .. GENERATED FROM PYTHON SOURCE LINES 162-167 Report ------ The report-command prints an extensive list of your python environment of components which are important for `emg3d`. If you report an issue it helps a lot if you provide this info. .. GENERATED FROM PYTHON SOURCE LINES 167-169 .. code-block:: default bash("emg3d --report") .. rst-class:: sphx-glr-script-out .. code-block:: none $ emg3d --report -------------------------------------------------------------------------------- Date: Wed Aug 31 21:35:31 2022 CEST OS : Linux CPU(s) : 4 Machine : x86_64 Architecture : 64bit RAM : 15.5 GiB Environment : Python File system : ext4 Python 3.9.12 | packaged by conda-forge | (main, Mar 24 2022, 23:22:55) [GCC 10.3.0] numpy : 1.22.4 scipy : 1.9.0 numba : 0.55.2 emg3d : 1.8.0 empymod : 2.2.0 xarray : 2022.6.0 discretize : 0.8.2 h5py : 3.7.0 matplotlib : 3.4.3 tqdm : 4.64.0 IPython : 8.4.0 Intel(R) oneAPI Math Kernel Library Version 2022.0-Product Build 20211112 for Intel(R) 64 architecture applications -------------------------------------------------------------------------------- .. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 50.098 seconds) **Estimated memory usage:** 9 MB .. _sphx_glr_download_gallery_tutorials_cli.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: cli.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: cli.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_