# Adjustable Variables¶

We have now adjusted the disease file for the lurgy, which has resulted in a model run that has involved a larger section of the population in the disease outbreak. However, the values we chose for the parameters in the disease file were made up. They had no basis in reality, which means that it is difficult to use this model to make good predictions.

metawards is designed to support parameter searches that seek to adjust the disease parameters so that predictions from model runs can be matched to observed reality. The idea is that variables are adjusted until the model runs can reliably capture the observed behaviour of an outbreak, at which point the model may then act as a good prediction of the future.

## Choosing what to adjust¶

In the last section we introduced a new stage to the lurgy in which infected individuals were very mobile (too_ill_to_move was 0.0), but also very infectious (beta was 0.5).

This was introduced as stage 3 of the lurgy. We need therefore to tune the beta[2] and too_ill_to_move[2] parameters (remembering that we count from zero). What we want to do is vary these two parameters and see how they affect the model outbreak.

To do this, in your current directory create a file called lurgyparams.csv and copy in the text below;

beta[2]  too_ill_to_move[2]
0.3         0.00
0.4         0.00
0.5         0.00
0.3         0.25
0.4         0.25
0.5         0.25
0.3         0.50
0.4         0.50
0.5         0.50


This file has two columns of numbers; beta[2] which shows how to adjust the beta[2] value, and too_ill_to_move[2] which shows how to adjust the too_ill_to_move[2] value. Each row of this file represents a single model run using this pair of values. For example, the line 0.3       0.00 will set beta[2] to 0.3 and too_ill_to_move[2] to 0.0.

Note

The format of this file is very flexible. Columns can be space-separated or comma-separated, as long as this is consistent in the entire file. You can choose to adjust any of the parameters for any stage of a disease. Simply title the column with the parameter name, e.g. beta, progress, too_ill_to_move or contrib_foi, and add the stage number in square brackets, e.g. beta[0]. Remember that we count from zero, so stage one has index zero.

You can pass metawards this input file using the --input (or -i) command-line option. This input file has nine pairs of values of beta[2] and too_ill_to_move[2], which means that nine model runs need to be performed. To run them all, use this command;

metawards -d lurgy2 -a ExtraSeedsLondon.dat --input lurgyparams.csv


This will run all nine jobs. If you have nine or more processor cores on your computer then all of them will be run in parallel (with individual model runs then using any other cores you have). If, like me, you are running this on your laptop, then this may take 10+ minutes to complete.

Note

If you want to distribute this work over a set of disconnected computers, you can tell metawards to only adjust parameters from a subset of the lines of the input file. To do this, use the --line (or -l) command line argument to specify the line or lines to process. Lines are counted from 0 being the top line (containing the header), and multiple lines or ranges can be specified, e.g. -l 1 2 3 will use lines one to three, while -l 4-6 would use lines four to six (inclusive).

Warning

We are only going to use one repeat of each pair of values, which means that our results will suffer from a lot of random error. Ideally you should ask for multiple repeats using the --repeats command-line argument. A good value would be at least eight repeats. In this case, eight repeats would require 72 model runs. If you parallelise each run over 16 cores then this needs 1152 cores. Fortunately, for these jobs, metawards runs well in parallel across a slurm or PBS cluster.

Once the jobs are complete (which took 15 minutes on my laptop), you should have output that looks similar to this;

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ MULTIPROCESSING ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Computing model run ✔
┌────────────────────────────────────────────────────────────────────────────────────────┐
│                                                                                        │
│  Completed job 1 of 9                                                                  │
│  (beta[2]=0.3, too_ill_to_move[2]=0)[repeat 1]                                         │
│  2020-12-15: DAY: 209 S: 7978312    E: 0    I: 0    R: 48103765    IW: 1   UV: 1.0     │
│  TOTAL POPULATION 56082077                                                             │
│                                                                                        │
└────────────────────────────────────────────────────────────────────────────────────────┘
Computing model run ✔
┌────────────────────────────────────────────────────────────────────────────────────────┐
│                                                                                        │
│  Completed job 2 of 9                                                                  │
│  (beta[2]=0.4, too_ill_to_move[2]=0)[repeat 1]                                         │
│  2020-11-28: DAY: 192 S: 7046459    E: 0    I: 0    R: 49035618    IW: 0   UV: 1.0     │
│  TOTAL POPULATION 56082077                                                             │
│                                                                                        │
└────────────────────────────────────────────────────────────────────────────────────────┘
Computing model run ✔
┌────────────────────────────────────────────────────────────────────────────────────────┐
│                                                                                        │
│  Completed job 3 of 9                                                                  │
│  (beta[2]=0.5, too_ill_to_move[2]=0)[repeat 1]                                         │
│  2020-11-17: DAY: 181 S: 6221586    E: 0    I: 0    R: 49860491    IW: 1   UV: 1.0     │
│  TOTAL POPULATION 56082077                                                             │
│                                                                                        │
└────────────────────────────────────────────────────────────────────────────────────────┘
Computing model run ✔
┌────────────────────────────────────────────────────────────────────────────────────────┐
│                                                                                        │
│  Completed job 4 of 9                                                                  │
│  (beta[2]=0.3, too_ill_to_move[2]=0.25)[repeat 1]                                      │
│  2020-12-13: DAY: 207 S: 8010218    E: 0    I: 0    R: 48071859    IW: 0   UV: 1.0     │
│  TOTAL POPULATION 56082077                                                             │
│                                                                                        │
└────────────────────────────────────────────────────────────────────────────────────────┘
Computing model run ✔
┌────────────────────────────────────────────────────────────────────────────────────────┐
│                                                                                        │
│  Completed job 5 of 9                                                                  │
│  (beta[2]=0.4, too_ill_to_move[2]=0.25)[repeat 1]                                      │
│  2020-12-02: DAY: 196 S: 7071208    E: 0    I: 0    R: 49010869    IW: 1   UV: 1.0     │
│  TOTAL POPULATION 56082077                                                             │
│                                                                                        │
└────────────────────────────────────────────────────────────────────────────────────────┘
Computing model run ✔
┌────────────────────────────────────────────────────────────────────────────────────────┐
│                                                                                        │
│  Completed job 6 of 9                                                                  │
│  (beta[2]=0.5, too_ill_to_move[2]=0.25)[repeat 1]                                      │
│  2020-12-01: DAY: 195 S: 6260263    E: 0    I: 0    R: 49821814    IW: 0   UV: 1.0     │
│  TOTAL POPULATION 56082077                                                             │
│                                                                                        │
└────────────────────────────────────────────────────────────────────────────────────────┘
Computing model run ✔
┌────────────────────────────────────────────────────────────────────────────────────────┐
│                                                                                        │
│  Completed job 7 of 9                                                                  │
│  (beta[2]=0.3, too_ill_to_move[2]=0.5)[repeat 1]                                       │
│  2021-01-01: DAY: 226 S: 8031161    E: 0    I: 0    R: 48050916    IW: 0   UV: 1.0     │
│  TOTAL POPULATION 56082077                                                             │
│                                                                                        │
└────────────────────────────────────────────────────────────────────────────────────────┘
Computing model run ✔
┌────────────────────────────────────────────────────────────────────────────────────────┐
│                                                                                        │
│  Completed job 8 of 9                                                                  │
│  (beta[2]=0.4, too_ill_to_move[2]=0.5)[repeat 1]                                       │
│  2020-11-27: DAY: 191 S: 7103861    E: 0    I: 0    R: 48978216    IW: 0   UV: 1.0     │
│  TOTAL POPULATION 56082077                                                             │
│                                                                                        │
└────────────────────────────────────────────────────────────────────────────────────────┘
Computing model run ✔
┌────────────────────────────────────────────────────────────────────────────────────────┐
│                                                                                        │
│  Completed job 9 of 9                                                                  │
│  (beta[2]=0.5, too_ill_to_move[2]=0.5)[repeat 1]                                       │
│  2020-11-16: DAY: 180 S: 6293958    E: 0    I: 0    R: 49788119    IW: 0   UV: 1.0     │
│  TOTAL POPULATION 56082077                                                             │
│                                                                                        │
└────────────────────────────────────────────────────────────────────────────────────────┘
┌────────────────────────────────────────────────────────────────────────────────────────┐
│                                                                                        │
│  Writing a summary of all results into the csv file                                    │
│  /Users/chris/GitHub/tutorial/test/output/results.csv.bz2. You can use this to         │
│  quickly look at statistics across all runs using e.g. R or pandas                     │
│                                                                                        │
└────────────────────────────────────────────────────────────────────────────────────────┘


## Output directories¶

The output files for each repeat are placed into subdirectories of the main output directory. By default, these subdirectories are named according to the fingerprint of the adjustable variables used for each run, e.g. listing the contents of the output directory using;

$ls output 0i3v0i0x001 0i3v0i5x001 0i4v0i25x001 0i5v0i0x001 0i5v0i5x001 console.log.bz2 0i3v0i25x001 0i4v0i0x001 0i4v0i5x001 0i5v0i25x001 config.yaml results.csv.bz2  The fingerprint is a unique key used for each run, e.g. 0i3v0i0x001 refers to the run using values 0.3 0.0, and the first repeat. The i represents a decimal point, v is used to separate values, and x001 means the first repeat. Similarly, 0i4v0i25x001 refers to the first repeat of the values 0.4 0.25. Sometimes you may want to specify the names of the output directories yourself. You can do this by adding a output column to your scan file, e.g. beta[2] too_ill_to_move[2] output 0.3 0.00 beta_0i3_ill_0i00 0.4 0.00 beta_0i4_ill_0i00 0.5 0.00 beta_0i5_ill_0i00 0.3 0.25 beta_0i3_ill_0i25 0.4 0.25 beta_0i4_ill_0i25 0.5 0.25 beta_0i5_ill_0i25 0.3 0.50 beta_0i3_ill_0i50 0.4 0.50 beta_0i4_ill_0i50 0.5 0.50 beta_0i5_ill_0i50  Running metawards with this file would place output in the following directories; $ ls output
beta_0i3_ill_0i00 beta_0i4_ill_0i00 beta_0i5_ill_0i00 config.yaml
beta_0i3_ill_0i25 beta_0i4_ill_0i25 beta_0i5_ill_0i25 console.log.bz2
beta_0i3_ill_0i50 beta_0i4_ill_0i50 beta_0i5_ill_0i50 results.csv.bz2


If you run multiple repeats of these jobs, e.g. using the --repeats keyword via;

metawards -d lurgy2 -a ExtraSeedsLondon.dat --input lurgyparams.csv --repeats 2


then the repeat number will be automatically added to the directory names, e.g.

\$ ls output
beta_0i3_ill_0i00     beta_0i4_ill_0i00     beta_0i5_ill_0i00     config.yaml
beta_0i3_ill_0i00x002 beta_0i4_ill_0i00x002 beta_0i5_ill_0i00x002 console.log.bz2
beta_0i3_ill_0i25     beta_0i4_ill_0i25     beta_0i5_ill_0i25     results.csv.bz2
beta_0i3_ill_0i25x002 beta_0i4_ill_0i25x002 beta_0i5_ill_0i25x002
beta_0i3_ill_0i50     beta_0i4_ill_0i50     beta_0i5_ill_0i50
beta_0i3_ill_0i50x002 beta_0i4_ill_0i50x002 beta_0i5_ill_0i50x002