GMS:MODFLOW with HDF5: Difference between revisions

From XMS Wiki
Jump to navigationJump to search
(move to external link)
 
(6 intermediate revisions by 2 users not shown)
Line 5: Line 5:
The HDF5 file format is a cross platform binary format for storing scientific data. One advantage of using HDF5 is that the data is stored in binary format so the disk reads and writes are much faster than with ASCII files. View the data stored in an HDF5 file by using [http://hdf.ncsa.uiuc.edu/hdf-java-html/hdfview/index.html HDFView] (or some other available viewer).
The HDF5 file format is a cross platform binary format for storing scientific data. One advantage of using HDF5 is that the data is stored in binary format so the disk reads and writes are much faster than with ASCII files. View the data stored in an HDF5 file by using [http://hdf.ncsa.uiuc.edu/hdf-java-html/hdfview/index.html HDFView] (or some other available viewer).


Another advantage of using the HDF5 libary with MODFLOW package data and array data is that HDF5 will compress the data. For example, if there is a transient simulation using the river package the typical line in a river package file would look like this:
Another advantage of using the HDF5 library with MODFLOW package data and array data is that HDF5 will compress the data. For example, if there is a transient simulation using the river package the typical line in a river package file would look like this:


'''[K] [I] [J] [STAGE] [CONDUCTANCE] [ELEVATION]'''
'''[K] [I] [J] [STAGE] [CONDUCTANCE] [ELEVATION]'''
Line 19: Line 19:
In one example a river file that was 437 MB was saved to HDF5 format with a compression level of 1; the resulting file was 14 MB. In another example an HDF5 file that contained drain and general head data was 8 MB while the drain file was 253 MB and the general head 141 MB. Not only do the files compress extremely well but the read/write times for the HDF5 file are much faster than the ASCII files.
In one example a river file that was 437 MB was saved to HDF5 format with a compression level of 1; the resulting file was 14 MB. In another example an HDF5 file that contained drain and general head data was 8 MB while the drain file was 253 MB and the general head 141 MB. Not only do the files compress extremely well but the read/write times for the HDF5 file are much faster than the ASCII files.


==Changes to MODFLOW 2000 & 2005<ref name="GMS8"/> code==
==Changes to MODFLOW 2000 & 2005 Code==
The following subroutines in ult6.f were modified: U2DREL, U2DINT, and ULSTRD. The stream package reading routines were also modified in the str6.f file.
The following subroutines in ult6.f were modified:<ref name="GMS8"/> U2DREL, U2DINT, and ULSTRD. The stream package reading routines were also modified in the str6.f file.


===<big>ult.f modifications</big>===
===ult.f Modifications===
U2DREL and U2DINT were modified so that if the key word HDF5 was encountered when attempting to read an array then an external procedure is called to read in the data. We have added support for the following:
U2DREL and U2DINT were modified so that if the key word HDF5 was encountered when attempting to read an array then an external procedure is called to read in the data. We have added support for the following:


Line 122: Line 122:
|}
|}


===<big>str6.f modifications</big>===
===str6.f Modifications===
Since the STR package does not use the ULSTRD utility the str6.f file had to be modified to read the same data that ULSTRD was modified to read: '''GMS_HDF5_01 "FNAME" "pathInFile" SP'''.
Since the STR package does not use the ULSTRD utility the str6.f file had to be modified to read the same data that ULSTRD was modified to read: '''GMS_HDF5_01 "FNAME" "pathInFile" SP'''.


Line 130: Line 130:
In previous versions of GMS,  [[GMS:Pilot Points|pilot points]] were supported by using the multiplier arrays in MODFLOW. Beginning with version 6.5 the pilot point interpolation takes place with in our external routine that is called by MODFLOW. For example, if MODFLOW calls our routine to read an HK array and that array has parameter ''key'' values that are associated with a parameter that is defined using pilot points then our routine will perform the pilot point interpolation and substitute the appropriate value. This includes any log interpolation that the user has specified.
In previous versions of GMS,  [[GMS:Pilot Points|pilot points]] were supported by using the multiplier arrays in MODFLOW. Beginning with version 6.5 the pilot point interpolation takes place with in our external routine that is called by MODFLOW. For example, if MODFLOW calls our routine to read an HK array and that array has parameter ''key'' values that are associated with a parameter that is defined using pilot points then our routine will perform the pilot point interpolation and substitute the appropriate value. This includes any log interpolation that the user has specified.


==Reading MODFLOW files which include HDF5 data==
==Reading MODFLOW Files which Include HDF5 Data==
When GMS writes out the MODFLOW files for the list based packages, the first item written to the file is #GMS_HDF5_01. When GMS reads these MODFLOW files with this special comment, it does not read any more of the package file and the data from the *.h5 file will be used.
When GMS writes out the MODFLOW files for the list based packages, the first item written to the file is #GMS_HDF5_01. When GMS reads these MODFLOW files with this special comment, it does not read any more of the package file and the data from the *.h5 file will be used.


When GMS saves the RCH or EVT package with parameter that are defined with cluster (and instances). Then a different special comment is written to the top of the file: #GMS_PARAM_CLUSTERS. If GMS is reading a project file that includes a MODFLOW simulation then when this special comment is encountered GMS will not read any more of the package file. However, if a user is reading a MODFLOW model (not a project file) then the rest of this package file will be read and new datasets will be created on the 3D grid that are used to define the clusters used by the parameters.  
When GMS saves the RCH or EVT package with parameter that are defined with cluster (and instances). Then a different special comment is written to the top of the file: #GMS_PARAM_CLUSTERS. If GMS is reading a project file that includes a MODFLOW simulation then when this special comment is encountered GMS will not read any more of the package file. However, if a user is reading a MODFLOW model (not a project file) then the rest of this package file will be read and new datasets will be created on the 3D grid that are used to define the clusters used by the parameters.


==Exporting Native MODFLOW Files==
==Exporting Native MODFLOW Files==
Line 157: Line 157:


==External links==
==External links==
*More information about [http://hdf.ncsa.uiuc.edu/HDF5/ HDF5]
*More information about [https://portal.hdfgroup.org/display/knowledge/What+is+HDF5 HDF5]




Latest revision as of 21:28, 7 August 2020

MODFLOW
Pre-processing
MODFLOW Commands
Building a MODFLOW Model
Map to MODFLOW
Calibration
Packages Supported in GMS
Saving a MODFLOW Simulation
Importing MODFLOW Files
Unsupported MODFLOW Features
Run MODFLOW
Post-processing
MODFLOW Display Options
MODFLOW Post-Processing Viewing Options
Reading a MODFLOW Simulation
Tutorials
Packages
Flow: BCF6, HUF, LPF, UPW
Solvers:

DE4, GMG, NWT, PCG,

PCGN, LMG, SIP, SOR,

SMS
Other:

BAS6, BFH, CHD1, CLN,

DRN1, DRT1, EVT1, ETS1,

GAGE, GHB1, GNC, HFB1,

HUF, LAK3, MNW1, MNW2,

OUT1, RCH1, RIV1, SFR2,

STR1, SUB1, SWI2, WEL1,

UZF1
v • e

The versions of MODFLOW 2000, MODFLOW 2005[1], SEAWAT[1], MODFLOW-NWT[2], and MODFLOW-LGR[3] that ship with the GMS software have been modified to be able to read input data stored in HDF5 files. GMS saves some MODFLOW data to an HDF5 file that is saved with the other MODFLOW files. Some MODFLOW files will have references to the HDF5 file.

Advantages to HDF5

The HDF5 file format is a cross platform binary format for storing scientific data. One advantage of using HDF5 is that the data is stored in binary format so the disk reads and writes are much faster than with ASCII files. View the data stored in an HDF5 file by using HDFView (or some other available viewer).

Another advantage of using the HDF5 library with MODFLOW package data and array data is that HDF5 will compress the data. For example, if there is a transient simulation using the river package the typical line in a river package file would look like this:

[K] [I] [J] [STAGE] [CONDUCTANCE] [ELEVATION]

1 3 4 10.2 5.8 9.5

Then during the next stress period typically the stage would change. The line in the file would look like this:

1 3 4 10.8 5.8 9.5

So we repeated all of the data except that 10.2 changed to 10.8. With compression those repeated values do not take up much space on disk so that the resulting file is smaller.

In one example a river file that was 437 MB was saved to HDF5 format with a compression level of 1; the resulting file was 14 MB. In another example an HDF5 file that contained drain and general head data was 8 MB while the drain file was 253 MB and the general head 141 MB. Not only do the files compress extremely well but the read/write times for the HDF5 file are much faster than the ASCII files.

Changes to MODFLOW 2000 & 2005 Code

The following subroutines in ult6.f were modified:[1] U2DREL, U2DINT, and ULSTRD. The stream package reading routines were also modified in the str6.f file.

ult.f Modifications

U2DREL and U2DINT were modified so that if the key word HDF5 was encountered when attempting to read an array then an external procedure is called to read in the data. We have added support for the following:

Explanation of variables
HDF5 a card to indicate that this is read from an HDF5 file
CNSTNT the multiplier for the array
IPRN print flag for the output control for MODFLOW
"FNAME" the HDF5 file where the array is stored
"pathInFile" path to the dataset in the HDF5 file
nDim number of dimensions that the dataset has (this is 1, 2, 3)
start1 the index (NOTE: these are 0 based not 1) for the starting point to read the dataset in the first dimension
nToRead1 the number of values to read in the first dimension
start2 the index for the starting point to read the dataset in the second dimension
nToRead2 the number of values to read in the second dimension
start3 the index for the starting point to read the dataset in the third dimension
nToRead3 the number of values to read in the third dimension
example: "HDF5 1.0 0 "input.h5" "Recharge/07. Property" 3 0 1 0 132 0 1"
This type of formatting will work for any HDF5 dataset that has 3 dimensions or less and the dataset can be a 4 byte float or an 8 byte double. The other available format that can be used with the HDF5 key word for the array reading utilities is the following:
HDF5 CONSTANT CNSTNT
example: HDF5 CONSTANT 3.0
This type of formating will assign a constant value to the array. This type of formatting may seem unnecessary. However, the constant identifier can be useful when using parameters to define arrays. Parameters will be explained in another section.

ULSTRD was also modified to read data from an HDF5 file. This format is specific to the types of HDF5 files that GMS creates.

GMS_HDF5_01 "FNAME" "pathInFile" SP
Explanation of variables
GMS_HDF5_01 a card to indicate that this is read from an HDF5 file
"FNAME" the HDF5 file where the array is stored
"pathInFile" path to the group in the HDF5 file where all of the data for the boundary condition is stored
SP stress period number
Here is an example of a regular modflow drain file and a drain file with the HDF5 modifications.
Comparison of MODFLOW Drain files
Traditional Drain file Drain file with HDF5

3 40 AUX IFACE AUX CONDFACT AUX CELLGRP
3 0
1 3 2 19.0 10.0 6 1.0 -1
1 4 3 19.0 10.0 4 1.0 -1
1 5 4 19.0 10.0 5 1.0 -1
3 0
1 3 2 18.9 11.0 6 1.0 -1
1 4 3 18.9 11.0 4 1.0 -1
1 5 4 18.9 11.0 5 1.0 -1
3 0
1 3 2 19.0 10.0 6 1.0 -1
1 4 3 19.0 10.0 4 1.0 -1
1 5 4 19.0 10.0 5 1.0 -1

 #GMS_HDF5_01

3 40 AUX IFACE AUX CONDFACT AUX CELLGRP
3 0 0
GMS_HDF5_01 "sg_t_pest_65.h5" "Drain" 1
3 0 0
GMS_HDF5_01 "sg_t_pest_65.h5" "Drain" 2
3 0 0
GMS_HDF5_01 "sg_t_pest_65.h5" "Drain" 3

str6.f Modifications

Since the STR package does not use the ULSTRD utility the str6.f file had to be modified to read the same data that ULSTRD was modified to read: GMS_HDF5_01 "FNAME" "pathInFile" SP.

Parameters

MODFLOW 2000 introduced the concept of parameters as native input to MODFLOW. Starting with version 6.5, GMS no longer writes out native MODFLOW parameters (except when clusters are defined) and the use of the MODFLOW PES process is not available from within GMS. Even when native MODFLOW files are exported from GMS, native MODFLOW parameters are not used (except when clusters are defined) and instead the parameter starting values are written. However, GMS still supports PEST and users may create parameters in the GMS interface as they have been able to do previously. Parameters are created in GMS by assigning a "key" value (usually a negative number) to a MODFLOW input. When MODFLOW is running and calls our external routine to read in the HDF5 a check is made in the data that is read from the HDF5 file. If one of the parameter "key" values is found in the data then the parameter value is substituted into the array or list.

By adopting this approach it was much easier to support transient parameters and very large sets of pilot points.

Also, adopting this approach allows GMS users to use parameters with the BCF package. This feature is not available with the MODFLOW PES process.

Pilot Points

In previous versions of GMS, pilot points were supported by using the multiplier arrays in MODFLOW. Beginning with version 6.5 the pilot point interpolation takes place with in our external routine that is called by MODFLOW. For example, if MODFLOW calls our routine to read an HK array and that array has parameter key values that are associated with a parameter that is defined using pilot points then our routine will perform the pilot point interpolation and substitute the appropriate value. This includes any log interpolation that the user has specified.

Reading MODFLOW Files which Include HDF5 Data

When GMS writes out the MODFLOW files for the list based packages, the first item written to the file is #GMS_HDF5_01. When GMS reads these MODFLOW files with this special comment, it does not read any more of the package file and the data from the *.h5 file will be used.

When GMS saves the RCH or EVT package with parameter that are defined with cluster (and instances). Then a different special comment is written to the top of the file: #GMS_PARAM_CLUSTERS. If GMS is reading a project file that includes a MODFLOW simulation then when this special comment is encountered GMS will not read any more of the package file. However, if a user is reading a MODFLOW model (not a project file) then the rest of this package file will be read and new datasets will be created on the 3D grid that are used to define the clusters used by the parameters.

Exporting Native MODFLOW Files

To export native MODFLOW ASCII files from GMS 6.5 up to GMS 9.2, use the following steps:

  1. Open a project in GMS (6.5 or later).
  2. In the 3D Grid Module, select MODFLOW | Export Native MF2K text... (GMS 8.3 and below) or MODFLOW | Advanced | Export Native MF2K text... (GMS 9.0 and above).
  3. In the window that appears, double-click the Name file of the MODFLOW model that the user wants to convert to ASCII format.
  4. A DOS prompt will appear. Press the Enter key when the executable is finished.
  5. A folder called "Out_Mf2k" is created in the same folder as the name file the user selected. "Out_Mf2k" contains all the MODFLOW input files in ASCII format.

If the DOS prompt does not appear after selecting the name file, then go to Edit | Preferences... and select Models. Verify that the Gms2Mf2k executable (the application that converts MODFLOW files from HDF5 to ASCII format) has the correct path next to it. If it doesn't, select Browse... and locate the executable. It will probably be located at C:\Program Files\GMS 8.1\models\mf2k\Gms2Mf2k.exe.

If not using MODFLOW-USG then to export native MODFLOW ASCII files from GMS 10.0 (or later) use the following steps:

  1. Open the project in GMS
  2. Select the MODFLOW | Global Options menu command.
  3. Under the MODFLOW version section of the dialog turn on the Save native text copy option.
  4. Save the project.

When a GMS project is saved with a MODFLOW model there will be a project file (ex: myProject.gpr) and a MODFLOW folder (ex: myProject_MODFLOW). With the Save native text copy option turned on another folder is created (ex: myProject_MODFLOW_text) that contains the MODFLOW files in native text format.

Notes

  1. ^ a b c Supported starting at GMS version 8.0
  2. ^ Supported starting at GMS version 8.2
  3. ^ Supported starting at GMS version 9.2

External links

  • More information about HDF5


v • e
GMS – Groundwater Modeling System
Modules:2D Grid  •  2D Mesh  •  2D Scatter Point  •  3D Grid  •  3D Mesh  •  3D Scatter Point  •  Boreholes  •  GIS  •  Map  •  Solid  •  TINs  •  UGridsGMS Logo2.png
Models:FEFLOW  •  FEMWATER  •  HydroGeoSphere  •   MODAEM  •  MODFLOW  •  MODPATH  •  mod-PATH3DU  •  MT3DMS  •  MT3D-USGS  •  PEST  •  PHT3D  •  RT3D  •  SEAM3D  •  SEAWAT  •  SEEP2D  •  T-PROGS  •  ZONEBUDGET
Aquaveo
Retrieved from "https://www.xmswiki.com/index.php?title=GMS:MODFLOW_with_HDF5&oldid=147786"