Initialize big.matrix

Hi @aho!

This brach partially solves the problem.

I have installed this branch and use it with your minimal example and the following Compute call:

 res_n4 <- Compute(wf,
                   chunks = list(sdate = 1),
                   threads_load = 1,
                   threads_compute = 4,
                   cluster = list(queue_host = 'nord4',
                   queue_type = 'slurm',
                   temp_dir = temp_dir,
                   lib_dir = 
"/.statelite/tmpfs/gpfs/home/bsc32/bsc32339/R/x86_64-pc-linux-gnu-library/3.6",
                   cores_per_job = 2,
                   job_wallclock = '01:00:00',
                   max_jobs = 4,
                   bidirectional = FALSE,
                   polling_period = 10
                 ),
                 ecflow_suite_dir = ecflow_suite_dir,
                 wait = T)

The result is correct in this case:

summary(res_n4$output1)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
  244.6   251.4   256.9   259.9   272.3   274.0       8 

However, when setting threads_load = 2 it fails:

* Loading... This may take several minutes...
* Progress: 0%Error in parallel::clusterApplyLB(cluster, work_pieces, .LoadDataFile,  :
  object 'transform_crop_domain' not found
In addition: Warning messages:
1: ! Warning: Parameter 'pattern_dims' not specified. Taking the first dimension,
!   'dat' as 'pattern_dims'.
2: ! Warning: Could not find any pattern dim with explicit data set descriptions (in
!   the form of list of lists). Taking the first pattern dim, 'dat', as
!   dimension with pattern specifications.
3: In filebacked.big.matrix(nrow = nrow, ncol = ncol, type = type,  :
  No descriptor file given, it will be named _2842325790_1_1_1_1_1_1_.desc

The data cannot be loaded.
See the original error message:
object 'found_files' not found

Well, I'll keep testing it.

Cheers,

Núria