HERMES_GR_PREPROC job fail with auto-MONARCH
@ctena, I'm trying to run an experiment, but autosubmit gets stuck at the HERMES_GR_PREPROC step.
It seems the issue is related to the temporal_coords.nc file. You can see the full error in: a980_20221215_000_HERMES_GR_PREPROC.20250507150318.err
File "/gpfs/scratch/bsc32/bsc143869/a980/HERMESv3/sources/HERMESv3_GR/hermesv3_gr/hermes.py", line 147, in run
self.grid = select_grid(self.comm, self.options)
File "/gpfs/scratch/bsc32/bsc143869/a980/HERMESv3/sources/HERMESv3_GR/hermesv3_gr/modules/grids/grid.py", line 49, in select_grid
date = HermesGr(Config()).main()
File "/gpfs/scratch/bsc32/bsc143869/a980/HERMESv3/sources/HERMESv3_GR/hermesv3_gr/hermes.py", line 69, in __init__
grid = RotatedGrid(arguments.domain_type, arguments.vertical_description, arguments.output_timestep_num,
File "/gpfs/scratch/bsc32/bsc143869/a980/HERMESv3/sources/HERMESv3_GR/hermesv3_gr/modules/grids/grid_rotated.py", line 80, in __init__
self.grid = select_grid(self.comm, self.options)
File "/gpfs/scratch/bsc32/bsc143869/a980/HERMESv3/sources/HERMESv3_GR/hermesv3_gr/modules/grids/grid.py", line 49, in select_grid
total_area = self.get_cell_area()
File "/gpfs/scratch/bsc32/bsc143869/a980/HERMESv3/sources/HERMESv3_GR/hermesv3_gr/modules/grids/grid.py", line 235, in get_cell_area
grid = RotatedGrid(arguments.domain_type, arguments.vertical_description, arguments.output_timestep_num,
File "/gpfs/scratch/bsc32/bsc143869/a980/HERMESv3/sources/HERMESv3_GR/hermesv3_gr/modules/grids/grid_rotated.py", line 80, in __init__
total_area = self.get_cell_area()
File "/gpfs/scratch/bsc32/bsc143869/a980/HERMESv3/sources/HERMESv3_GR/hermesv3_gr/modules/grids/grid.py", line 235, in get_cell_area
dataset = Dataset(self.coords_netcdf_file, mode='r')
File "src/netCDF4/_netCDF4.pyx", line 2353, in netCDF4._netCDF4.Dataset.__init__
dataset = Dataset(self.coords_netcdf_file, mode='r')
File "src/netCDF4/_netCDF4.pyx", line 2353, in netCDF4._netCDF4.Dataset.__init__
File "src/netCDF4/_netCDF4.pyx", line 1963, in netCDF4._netCDF4._ensure_nc_success
File "src/netCDF4/_netCDF4.pyx", line 1963, in netCDF4._netCDF4._ensure_nc_success
OSError: [Errno -101] NetCDF: HDF error: b'/gpfs/scratch/bsc32/bsc143869/a980/HERMESv3/auxiliar_files/d01/temporal_coords.nc'
OSError: [Errno -101] NetCDF: HDF error: b'/gpfs/scratch/bsc32/bsc143869/a980/HERMESv3/auxiliar_files/d01/temporal_coords.nc'
The file in question is: /gpfs/scratch/bsc32/bsc143869/a980/HERMESv3/auxiliar_files/d01/temporal_coords.nc It doesn't appear to be correct, the coastlines shift when viewing the cell_area variable, which seems suspicious.
Canceling the job and rerunning it without deleting anything seems to make the trick, and the rest of the experiment runs.
FYI: @mguevara