Last updated: 2019-05-27

Checks: 6 0

Knit directory: W_shredder/

This reproducible R Markdown analysis was created with workflowr (version 1.3.0). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.

Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.

Great job! The global environment was empty. Objects defined in the global environment can affect the analysis in your R Markdown file in unknown ways. For reproduciblity it’s best to always run the code in an empty environment.

The command set.seed(20180716) was run prior to running the code in the R Markdown file. Setting a seed ensures that any results that rely on randomness, e.g. subsampling or permutations, are reproducible.

Great job! Recording the operating system, R version, and package versions is critical for reproducibility.

Nice! There were no cached chunks for this analysis, so you can be confident that you successfully produced the results during this run.

Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility. The version displayed above was the version of the Git repository at the time these results were generated.

Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated:

Ignored files:
    Ignored:    .DS_Store
    Ignored:    .Rhistory
    Ignored:    .Rproj.user/
    Ignored:    Proc_B_manuscript/.DS_Store
    Ignored:    data/sim_results/
    Ignored:    figures/.DS_Store
    Ignored:    submission_pack/.DS_Store

Unstaged changes:
    Modified:   figures/S1_fig.pdf
    Modified:   figures/S3_fig.pdf
    Modified:   figures/S4_fig.pdf
    Modified:   figures/S5_fig.pdf
    Modified:   figures/S5_fig.rds
    Modified:   figures/figure4.pdf

Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.

These are the previous versions of the R Markdown and HTML files. If you’ve configured a remote Git repository (see ?wflow_git_remote), click on the hyperlinks in the table below to view them.

File Version Author Date Message
Rmd b70693c lukeholman 2019-05-20 tweak
Rmd 063e21f lukeholman 2019-05-20 tweak
Rmd 41aa9f7 lukeholman 2019-05-20 tweak
html 4d78c50 lukeholman 2019-05-06 Build site.
Rmd 8d839bb lukeholman 2019-05-06 wflow_publish(files = "*")
html 216445e lukeholman 2019-05-06 Build site.
Rmd 635d151 lukeholman 2019-05-06 wflow_publish(files = "*")
html 635d151 lukeholman 2019-05-06 wflow_publish(files = "*")
Rmd 1fca665 lukeholman 2019-04-26 increase SLURM time, and writing
Rmd 0ccc722 Luke Holman 2018-12-28 change to 48h
Rmd a587610 Luke Holman 2018-12-21 bigger chunks
Rmd e882e57 Luke Holman 2018-12-04 single core for faster queueing
Rmd 8a6ce50 Luke Holman 2018-12-03 change resources
Rmd 3b94528 Luke Holman 2018-12-03 Tweaks
Rmd ab69fc7 Luke Holman 2018-11-23 fix else
Rmd e60d0d1 Luke Holman 2018-11-23 bug hunt
Rmd b531be6 Luke Holman 2018-11-23 Tweak handling of parameter set up
Rmd e01c7fe Luke Holman 2018-11-22 bug fix
Rmd 6c2a74e Luke Holman 2018-11-22 Added new slurm script to check paras
Rmd 68469d2 Luke Holman 2018-11-22 fix wd typo
Rmd 79a4634 Luke Holman 2018-11-21 Change counting of old parameters
Rmd 7bf9997 Luke Holman 2018-11-20 added print to find bug
Rmd ed47ecc Luke Holman 2018-11-20 go to single core again
Rmd 9d2045c Luke Holman 2018-11-20 Fix parameter file
Rmd ee185ed Luke Holman 2018-11-19 Extra params
Rmd 64ce9e9 Luke Holman 2018-11-18 Less memory?
Rmd 0e180bc Luke Holman 2018-11-18 Fix memory spec
Rmd a9c7dfb Luke Holman 2018-11-16 More memory pls
Rmd 4c91977 Luke Holman 2018-11-16 Get more computation time
Rmd 4e630f4 Luke Holman 2018-11-15 Make combine_results_files.R
Rmd 2416673 Luke Holman 2018-11-15 missed bracket
Rmd 2c0e001 Luke Holman 2018-11-15 Bug fix
Rmd 4be43ee Luke Holman 2018-11-14 fixing bugs
Rmd b02f7d0 Luke Holman 2018-11-14 more nodes pls
Rmd 1cafc43 Luke Holman 2018-11-14 Added fitness to density calculation
Rmd fb9316c Luke Holman 2018-11-13 Request 32GB memory
Rmd 7560af7 Luke Holman 2018-11-13 change node number
Rmd b0fdeed Luke Holman 2018-11-13 multicore again
Rmd 8983cff Luke Holman 2018-11-13 Fixed chunks
Rmd 710d342 Luke Holman 2018-11-13 cpu = 1
Rmd 5f4f83b Luke Holman 2018-11-13 fixed job chunks
Rmd 9440576 Luke Holman 2018-11-13 change name
Rmd de9e0ff Luke Holman 2018-11-13 Added slurm capacity
Rmd 99e93c7 Luke Holman 2018-11-03 Many bug fixes with density dependence
html 99e93c7 Luke Holman 2018-11-03 Many bug fixes with density dependence

# This bit is for the unimelb cluster, Spartan
working_directory <- "/data/projects/punim0243/W_shredder"
source_rmd <- function(file){
  options(knitr.duplicate.label = "allow")
  tempR <- tempfile(tmpdir = ".", fileext = ".R")
  knitr::purl(file, output = tempR, quiet = TRUE)
  source(tempR, local = globalenv())
custom_functions <- ls()

Define the parameter space still not yet run

This is defined in an R script that sets up the parameter space, and runs everything that has not already completed.


Now launch lots of SLURM jobs to run the remaining parameter spaces

chunk_size <- 4000
cpus <- 1
sopt <- list(time = '168:00:00',  # max run time per node in hours
             mem  = '32768')     # 32GB memory across all 8 cores

chunks <- split(1:nrow(parameters),
number_of_chunks <- length(chunks)

sjob <- slurm_apply(
  f = function(i) {
                          over_write = FALSE, 
                          cores = cpus,
                          wd = working_directory))
  params = data.frame(i = 1:length(chunks)),
  add_objects = c("do_all_parameters", 
                  "parameters", "cpus",
                  "chunks", "number_of_chunks",
  jobname = 'W_shredder',
  nodes = number_of_chunks, 
  cpus_per_node = cpus, 
  slurm_options = sopt)

R version 3.5.1 (2018-07-02)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS High Sierra 10.13.6

Matrix products: default
BLAS: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

[1] en_AU.UTF-8/en_AU.UTF-8/en_AU.UTF-8/C/en_AU.UTF-8/en_AU.UTF-8

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods  
[8] base     

other attached packages:
[1] tibble_2.0.99.9000 readr_1.1.1        rslurm_0.4.0      
[4] Rcpp_1.0.1         reshape2_1.4.3     stringr_1.4.0     
[7] tidyr_0.8.2        purrr_0.3.2        dplyr_0.8.0.1     

loaded via a namespace (and not attached):
 [1] knitr_1.22        whisker_0.3-2     magrittr_1.5     
 [4] workflowr_1.3.0   hms_0.4.2         tidyselect_0.2.5 
 [7] R6_2.4.0          rlang_0.3.4       plyr_1.8.4       
[10] tools_3.5.1       xfun_0.6          git2r_0.23.0     
[13] htmltools_0.3.6   yaml_2.2.0        rprojroot_1.3-2  
[16] digest_0.6.18     assertthat_0.2.1  crayon_1.3.4     
[19] fs_1.3.1          glue_1.3.1.9000   evaluate_0.13    
[22] rmarkdown_1.10    stringi_1.4.3     compiler_3.5.1   
[25] pillar_1.3.1.9000 backports_1.1.2   pkgconfig_2.0.2