Model Data Get and Set Quick Start

From OpenM++
Jump to: navigation, search

Contents

Overview

If want to create your own quick and simple script to run the model then:

  • use parameter value specified as command line argument: modelOne.exe -Parameter.StartingSeed 1234
  • use parameter value specified inside of ini-file [Parameter] section
  • use parameter.csv files and run model as modelOne.exe -OpenM.ParamDir C:\MyCsv\
  • use dbcopy tool to export from model database into .json and .csv files: dbcopy.exe -m modelOne
  • use dbcopy tool to import .json and .csv files into model database

Please check Model Run Cycle: How model finds input parameters for more details.

There multiple ways to get model output results or set values of input parameters and c++ and Go libraries avaliable. Also openMpp R package is probably cover all necessary operations. For example:

Sub-values: sub-samples, members, replicas

Following terms: "simulation member", "replica", "sub-sample" are often used in micro-simulation conversations interchangeably, depending on context. To avoid terminology discussion openM++ uses "sub-value" as equivalent of all above and some older pages of that wiki may contain "sub-sample" in that case.

Model output tables: sub-values, accumulators and expressions

There are two kind of model output tables:

  • accumulators table: output sub-values (similar to Modgen sub-samples)
  • expressions table: model output value calculated as accumulators aggregated across sub-values (e.g. mean or CV or SE)

All output accumulator tables always contain same number of sub-values, for example model run:

model.exe -OpenM.Subvalues 16

will create 16 sub-values for each accumulator in each output accumulator table.

Model parameters: sub-values (optional)

OpenM++ parameters can also contain sub-values. Parameters sub-values are not required, it is a user choice to run the model and supply sub-values for some parameters.

For example, if user wants to describe statistical uncertanty of parameter SalaryByYearByProvince then csv file with 16 sub-values can be supplied to run the model:

model.exe -OpenM.Subvalues 16 Subvalue.SalaryByYearByProvince csv -OpenM.ParamDir C:\MyCsv\

Parameter: Value as command line argument

It is possible to specify value of any scalar parameter as command line argument, i.e.:

model.exe -Parameter.RandomSeed 1234

There is an example of such technique at Run model from R: simple loop over model parameter page, where we using NewCaseBased model to study effect of Mortality Hazard input parameter on Duration of Life output:

for (mortalityValue from 0.014 to 0.109 by step 0.005)
{
  # run the model
  NewCaseBased.exe -Parameter.MortalityHazard mortalityValue
}

If parameter is enum-based (e.g. classification) then you can specify code or enum id:

modelOne.exe -Parameter.baseSalary Full
modelOne.exe -Parameter.baseSalary 22 -OpenM.IdParameterValue true

Parameter: Sub-values [0, N-1] as command line argument

If we want to run the model with multiple sub-values (a.k.a. sub-samples) and want "Some" parameter sub-values to be created as [0, N-1] then:

model.exe -OpenM.Subvalues 16 -Subvalue.Some iota

as result sub-values parameter Some would be: [0, ..., 15]

Parameter: Value inside of ini.file

Also any scalar parameter can be defined in model ini-file, i.e.:

model.exe -ini my.ini
; inside of my.ini file:
;
[Parameter]
Z_Parameter = XYZ      ; string parameter
SomeInt     = 1234     ; integer parameter
OrLogical   = true     ; boolean parameter
Anumber     = 9.876e5  ; float parameter

Parameters: Csv files

It is also possible to supply some (or even all) model parameters as csv-file(s). For example:

model.exe -OpenM.ParamDir C:\my_csv

If directory C:\my_csv\ exist and contains parameterName.csv files then model will use it parameter values. For example, C:\my_csv\Sex.csv values of "Sex" parameter:

sub_id,dim0,param_value
0,     F,   true
0,     M,   false

Parameters: Csv files with multiple sub-values

If user want to supply up to 32 sub-values of "Sex" parameter:

sub_id,dim0,param_value
0,     F,   true
0,     M,   false
1,     F,   true
1,     M,   true
.................
31,    F,   false
31,    M,   true

Important: Presence of multiple sub-values in csv file (or in database) does not mean model will use all parameter sub-values. Only explicitly specified parameter(s) receiving sub-values.

For example, if user run the model 3 times:

model.exe -OpenM.Subvalues 16
model.exe -OpenM.Subvalues 16 -OpenM.ParamDir C:\my_csv
model.exe -OpenM.Subvalues 16 -OpenM.ParamDir C:\my_csv -Subvalue.Sex csv
  1. "Sex" parameter expected to be in database and no sub-values used
  2. "Sex" parameter value is sub-value 0 from C:\my_csv\Sex.csv
  3. "Sex" parameter using sub-values [0, 15] from C:\my_csv\Sex.csv

Use dbcopy: Export entire model into text files

dbcopy -m modelOne
dbcopy -m modelOne -dbcopy.Zip

It will create modelOne directory and modelOne.Zip file with:

  • all model metadata (e.g. parameters, description, notes,...) in .json files
  • csv files with sets of model input parameters
  • csv files with model run results and input parameters

Use dbcopy: Export entire model into csv files

dbcopy -m modelOne -dbcopy.To csv
dbcopy -m modelOne -dbcopy.To csv -dbcopy.Zip

It will create modelOne directory and modelOne.Zip file with:

  • all model metadata (e.g. parameters, description, notes,...) in .csv files
  • csv files with sets of model input parameters
  • csv files with model run results and input parameters

Use dbcopy: Export set of input parameters into text files

dbcopy -m modelOne -s modelOne_other -dbcopy.ParamDir pDir

It will create pDir directory with:

  • input parameters set metadata (name, description, notes,...) in .json file
  • csv files with sets of model input parameters

Use dbcopy: Export model run results into text files

dbcopy -m modelOne -dbcopy.RunId 101
dbcopy -m modelOne -dbcopy.RunName modelOne_2016_11_22_11_38_49_0945_101

It will create a directory with:

  • model run metadata (name, description, notes,...) in .json file
  • csv files with input parameters used to run the model
  • csv files with model output tables values

Use dbcopy: Import parameters from csv files into database

dbcopy -m myModel -s MyInput -dbcopy.ParamDir P -dbcopy.ToDatabase "Database=myModel.sqlite;OpenMode=ReadWrite"

If any parameterName.csv file(s) exist in directory P then it will be loaded into MyInput set of input parameters.

It is recommended to run dbcopy -m modelOne -s modelOne_other -dbcopy.ParamDir P to get familiar how csv files look like.

Use dbcopy: Import parameters, description and notes from text files into database

dbcopy -m myModel -s MyInput -dbcopy.ToDatabase "Database=myModel.sqlite;OpenMode=ReadWrite"

It will insert or update MyInput set of input parameters in database with:

  • if json metadata file exist then input set description, notes and parameter value note updated
  • if any parameterName.csv files exist then it will be loaded into database

It is recommended to run dbcopy -m modelOne -s modelOne_other -dbcopy.ParamDir P to get familiar how json and csv files look like.

Example of json metadata file for "ageSexData" input set of parameters with description, notes and ageSex parameter value notes:

{
  "ModelName" : "modelOne",
  "Name" : "ageSexData",
  "Txt" : [{
      "LangCode" : "EN",
      "Descr" : "Model One set of parameters"
    }
  ],
  "Param" : [{
      "Name" : "ageSex",
      "SubCount" : 1,
      "Txt" : [{
          "LangCode" : "EN",
          "Note" : "Age by Sex values"
        }
    ]
  ]
}

It is also must exist csv file with parameter values: ageSex.csv

Example of json metadata file for "emptyData" input set of parameters with description and notes in English and French:

{
  "ModelName" : "modelOne",
  "Name" : "emptyData",
  "Txt" : [{
      "LangCode" : "EN",
      "Descr" : "Model One set of parameters",
      "Note" : "Notes for model One set of parameters"
    }, {
      "LangCode" : "FR",
      "Descr" : "Je suis désolé je ne parle pas français"
    }
  ]
}


<metadesc>OpenM++: open source microsimulation platform</metadesc>