6. Importing data

6.1. data files

As of now, you can only import CSV files, one file for each entity. Their first row should contain the name of the fields. You need at least two integer columns: “id” and “period” (though they do not necessarily need to be named like that in the csv file).

6.2. description file

To import CSV files, you need to create a description file. Those description files have the following general format:

output: <path_of_hdf5_file>.csv

# compression is optional. compression type can be 'zlib', 'bzip2' or 'lzo'
# level is a digit from 1 to 9 and is optional (defaults to 5).
# Examples of valid compression strings are: zlib, lzo-1, bzip2-9.
# You should experiment to see which compression scheme (if any) offers the
# best trade-off for your dataset.
compression: <type>-<level>

globals:
    periodic:
        path: <path_of_globals_file>.csv
        # if the csv file is transposed (each field is on a row instead of a
        # column and the field names are in the first column, instead of the
        # first row), you can use "transpose: true". You do not need to
        # specify anything if the file is not transposed.
        transposed: true

entities:
    <entity1_name>:
        path: <path_to_entity1_data>.csv

        # if you want to manually select the fields to be used, and/or
        # specify their types, you can do so in the following section.
        # If you want to use all the fields present in the csv file, you
        # can simply omit this section. The field types will be
        # automatically detected.
        fields:
            # period and id are implicit
            - <field1_name>: <field1_type>
            - <field2_name>: <field2_type>
            - ...

        # if you want to keep your csv files intact but use different
        # names in your simulation than in the csv files, you can specify
        # name changes here.
        oldnames:
            <fieldX_newname>: <fieldX_oldname>
            <fieldY_newname>: <fieldY_oldname>

        # if you want to invert the value of some boolean fields
        # (True -> False and False -> True), add them to the "invert" list
        # below.
        invert: [list, of, boolean, fields, to, invert]

    <entity2_name>:
        ...

Most elements of this description file are optional. The only required elements are “output” and “entities”. If an element is not specified, it uses the following default value:

• if path is omitted, it defaults to a file named after the entity in the same directory than the description file (ie local_path\name_of_the_entity.csv).
• if the fields section is omitted, all columns of the csv file will be imported and their type will be detected automatically.
• if compression is omitted, the output will not be compressed.

Note that if an “entity section” is entirely empty, you need to use the special code: “{}”.

example

output: normal.h5

globals:
    periodic:
        path: input\globals_transposed.csv
        transposed: true

entities:
    household:
        path: input\household.csv

    person:
        path: input\person.csv
        fields:
            - age:           int
            - gender:        bool
            - workstate:     int
            - civilstate:    int
            - partner_id:    int

        oldnames:
            gender: male

simpler example

output: simple.h5

globals:
    periodic:
        path: input\globals.csv

entities:
    household:
        path: input\household.csv

    person:
        path: input\person.csv

simplest example

output: simplest.h5

entities:
    household: {}
    person: {}

This will try to load all the fields of the household and person entities in “household.csv” and “person.csv” in the same directory than the description file.

6.3. importing the data

Once you have your data as CSV files and created a description file, you can import your data.

• If you are using the bundled editor, simply open the description file and press F5.

• If you are using the command line, use:

  liam2 import <path_to_description_file>