9.2.3 Setting Up Experiments

In this section we show how SIESTA is typically used. When starting up, the initialization file is read in which the hosts to be used are defined via define-all-hosts. The information about the hosts will be used later by the task manager which schedules the execution of the various simulation tools. Hosts may later be enabled and disabled interactively by the user. Furthermore, for every host a function can be provided which decides if a host is usable right now; this is useful when certain hosts must not be used at certain times of day.

Commands are entered in any number of interactive listeners. After loading a file containing the definition of an experiment, the run command starts an optimization run. The value returned by run can be stored in a file, although the automatically generated log file contains all the results and information about the progress of the optimization. The result can also be used as a starting value for the next run.

Experiments are defined using define-experiment. The definition of an experiment consists of an optional description, the list of the free variables, their interval and their default values, the list of user variables which enable sophisticated customized setups, the goal function, the default value of the goal function to be used when no attempt was successful, the constraint function, and the configuration of one or more optimizers.

States correspond to points in the search space or to individuals of the population in the language of evolutionary computation. A state consists of a list of all (free) variables and their respective values, the experiment it belongs to and - after evaluation - the value of the goal function. States can be manipulated with the make-state, copy-state, with-state-vars, setq-in-copied-state, and show-state operators.

Setting up the evaluation function is usually the hardest part of defining an experiment. test-run can be used to evaluate the goal function on the default values interactively and to see if it works satisfactorily.

The following simple example shows the important steps when setting up new experiments.

Example.

? (defun sum-of-all-free-vars (state)
    (reduce #'+ (free-vars state) :key #'value))

> SUM-OF-ALL-FREE-VARS

? (define-experiment :e1 (genetic-experiment)
    :description "A simple test experiment."
    :vars '((var1 (interval 0 10) :default 5)
            (var2 (interval 0 10) :default 5)
            (var3 (interval 0 10) :default 5 :free nil))
    :evaluation-function 'sum-of-all-free-vars
    :last-resort 0
    :optimizer-configuration
    (make-genopt-configuration :e1
                               :minimize-or-maximize :maximize
                               :algorithm "GASteadyStateGA"
                               :crossover "TwoPointCrossover"
                               :population-size 50
                               :number-of-generations 12))

> (#<GENETIC-EXPERIMENT :E1>)

? (test-run :e1)

> 10

? (sum-of-all-free-vars (setq-in-copied-state (make-state :e1) var1 1 var2 2))

> 3

? (run :e1)

;; Several lines deleted.
========== Generation 12 ==========
Number of requests evaluated: 426
Best: #<REQUEST (id 637) (score 20.0) (vars ((VAR1 10.0) (VAR2 10.0)))>
> #<REQUEST (id 637) (score 20.0) (vars ((VAR1 10.0) (VAR2 10.0)))>