Run OCD

OCD exposes a single executable with various subcommands:

• ocd run: the main ocd command
• ocd config: handle configuration files
• ocd run_shot: run a HETDEX shot
• ocd allow_hetdex: turn on/off the shot execution
• ocd db_replay: replay a sqlite3 database
• ocd docker_mysql: manage a docker container with a MySQL database

ocd run

This command runs the main OCD program. When starting, it connects to one or more URLS, as provided from the command line, and begins listening for events to parse. As events come in, relevant information are stored and analysed. To stop OCD hit ctrl+c.

Run ocd run -h for the full list of arguments [1]:

usage: ocd run [-h] [--config [CONFIG]] [-u EVENT-URLS [EVENT-URLS ...]]
            [-t EXTRA-TOPICS [EXTRA-TOPICS ...]] [-a {yes,no}]
            [--n-ocd-main-loop N_OCD_MAIN_LOOP]
            [--n-ocd-run-shot N_OCD_RUN_SHOT]

Subcommand to run OCD

optional arguments:
-h, --help            show this help message and exit
--config [CONFIG]     Override the builtin configurations with the given
                      configuration file. If the option is used without a
                      value, "ocd.cfg" is used. Non existing files are
                      skipped and a warning is printed to the screen.
                      (default: None)

Override options in the [ulrs] section:
-u EVENT-URLS [EVENT-URLS ...], --event-urls EVENT-URLS [EVENT-URLS ...]
                      Address to which to bind the ZMQ socket. (default:
                      None)

Override options in the [topics] section.:
-t EXTRA-TOPICS [EXTRA-TOPICS ...], --extra-topics EXTRA-TOPICS [EXTRA-TOPICS ...]
                      Topics to subscribe to. OCD automatically subscribe
                      only to the topics necessary for it to work. If for
                      any reason other topics are needed, add them here.
                      (default: None)
-a {yes,no}, --auto-subscribe {yes,no}
                      Do not subscribe to the topics used by OCD but only to
                      the ones given by -t/--topic option. (default: None)

Override options in the [run] section:
--n-ocd-main-loop N_OCD_MAIN_LOOP
                      Which of the urls for the ``ocd_main_loop`` option of
                      the ``[urls]`` section use to setup a ZMQ server
                      (default: None)
--n-ocd-run-shot N_OCD_RUN_SHOT
                      Which of the urls for the ``ocd_run_shot`` option of
                      the ``[urls]`` section use to setup a ZMQ server
                      (default: None)

The configuration overriding mechanism works only on existing options: make
sure that the corresponding sections and options exist. The section names are
given in the description of the command and the option names are, unless
otherwise stated, the same as the command line options long names.

Note

The OCD Orchestrator listen for events on a number of urls. The event_urls option in the [urls] section of the Master configuration file, contain the addresses that TCS uses to stream events, while ocd_db_replay contain the list of addresses used by ocd db_replay to stream events from a sqlite3 database.

Since the latter tool has been designed to replay TCS events stored into a database, allowing OCD to listen to both channels might give rise to confusion and make OCD misbehave. Therefore ocd run aborts if both entries are given. The other OCD subcommands prints a warning instead.

For refernce and implementation see issue #2240.

ocd config

copy

Most of the options are passed via a configuration file. OCD parses the configuration file in the following way:

1. loads the master configuration, shipped with the package;
2. if the --config option is used, loads the given file. Entries in this file can override the default settings. If the file does not exist print a warning.

Because of this, the file passed with the --config option doesn’t need to be complete, but can contains the sections and options that the user wants to override.

The master configuration file can be obtained using the ocd config copy command. Run ocd config copy -h for the full list of arguments [1]:

usage: ocd config copy [-h] [-V] [-t TO_DIR] [-f] [-b]

Copy the configuration files in the desired directory

optional arguments:
  -h, --help            show this help message and exit
  -V, --version         show program's version number and exit
  -t TO_DIR, --to-dir TO_DIR
                        Copy the configuration files to the given directory.
                        By default use the current directory (default: .)
  -f, --force           Force overwriting existing configuration files. If
                        this option is used, all local modifications will be
                        lost (default: False)
  -b, --backup          Backup existing files before copying the new ones. If
                        this option is used, the `-f/--force` is ignored
                        (default: False)

send

There are situations in which the user would like to update some configuration options while the OCD main loop is active, i.e. without restarting ocd run. The following command sends a new value for one option of one section:

usage: ocd config send [-h] [-V] [--config [CONFIG]] [-n MAX_ATTEMPTS]
                    [-I INTERVAL] [-o OCD_CONFIG [OCD_CONFIG ...]]
                    [-i OCD_MAIN_LOOP [OCD_MAIN_LOOP ...]]
                    [-N N_OCD_CONFIG]
                    section option value

Send a configuration option to the OCD main loop

positional arguments:
section               Configuration section
option                Configuration option to override
value                 New configuration value

optional arguments:
-h, --help            show this help message and exit
-V, --version         show program's version number and exit
--config [CONFIG]     Override the builtin configurations with the given
                      configuration file. If the option is used without a
                      value, "ocd.cfg" is used. Non existing files are
                      skipped and a warning is printed to the screen.
                      (default: None)
-n MAX_ATTEMPTS, --max-attempts MAX_ATTEMPTS
                      Maximum number of times "ocd config send" attempts to
                      enable/disable HETDEX shot execution (default: 5)
-I INTERVAL, --interval INTERVAL
                      Wait "interval" seconds before a new attempt.
                      (default: 1)

Override options in the [urls] section:
-o OCD_CONFIG [OCD_CONFIG ...], --ocd-config OCD_CONFIG [OCD_CONFIG ...]
                      Urls/paths used to send the signal to update the
                      configuration entries. (default: None)
-i OCD_MAIN_LOOP [OCD_MAIN_LOOP ...], --ocd-main-loop OCD_MAIN_LOOP [OCD_MAIN_LOOP ...]
                      Url/path from where the confirmation of the update
                      arrives. (default: None)

Override options in the [config] section:
-N N_OCD_CONFIG, --n-ocd-config-hetdex N_OCD_CONFIG
                      Which of the urls for the ``ocd_config`` option of the
                      ``[urls]`` section use to setup a ZMQ server (default:
                      None)

The configuration overriding mechanism works only on existing options: make
sure that the corresponding sections and options exist. The section names are
given in the description of the command and the option names are, unless
otherwise stated, the same as the command line options long names.

Some of the options are used during the initialization of the OCD components, thus modifications via this command won’t have any effect. The configuration file contains information whether using ocd config send has effect on the OCD behaviour.

ocd run_shot

This command exposes the OCD functionality to run a HETDEX shot. It uses the same configuration file of ocd run and provides command line options to overrides some of the relevant configuration options. The implementation is base on the syscmd commands, as created by hetdex-shuffle, to setup the telescope and the hetdex-dither.py script to execute the observations. This executable can by used as a hetdex-dither.py substitute. Run ocd run_shot -h for the full list of arguments [1]:

usage: ocd run_shot [-h] [--config [CONFIG]] [-e] [-c] [-t TIMEOUT]
                    [-m METADATA] [-d {yes,no}] [-w {yes,no}]
                    [-N N_OCD_RUN_SHOT] [--shuffle-shot SHUFFLE_SHOT]
                    [--shuffle-conf-template SHUFFLE_CONF_TEMPLATE]
                    [--acam-dest-file ACAM_DEST_FILE]
                    [--ocd-run-shot OCD_RUN_SHOT [OCD_RUN_SHOT ...]]
                    shotid obs_number ra dec azimuth {0,1} exp01 exp02 exp03

Subcommand to run a single hetdex shot

positional arguments:
shotid                ID of the shot to run
obs_number            Counter for the current observation. The number is
                      used to set the directory name where the output fits
                      files are saved.
ra                    Right ascension of the shot
dec                   Declination of the shot
azimuth               Azimuth at which the observationis performed
{0,1}                 Whether it is EAST or WEST track
exp01                 Exposure time of the first exposure in seconds
exp02                 Exposure time of the second exposure in seconds
exp03                 Exposure time of the third exposure in seconds

optional arguments:
-h, --help            show this help message and exit
--config [CONFIG]     Override the builtin configurations with the given
                      configuration file. If the option is used without a
                      value, "ocd.cfg" is used. Non existing files are
                      skipped and a warning is printed to the screen.
                      (default: None)
-e, --emit-events     Emit events via ZeroMQ. If used, emits a number of
                      events using the internal OCD ZeroMQ servers. Use with
                      caution, in particular if OCD is running. (default:
                      False)
-c, --wait-for-connection
                      Try to establish a connection with the OCD main loop
                      before starting to run the shot. This option is used
                      only if ``-e/--emit-events`` is activated. If the
                      connection is not established within 10 seconds, abort
                      the execution. (default: False)
-t TIMEOUT, --timeout TIMEOUT
                      If ``-c/--wait-for-connection`` is given, wait at most
                      'timeout' seconds before aborting (default: 10)
-m METADATA, --metadata METADATA
                       Dictionary of metadata to send to TCS. It must be a
                       JSON string encoding a dictionary. ``json.loads`` is
                       used to deserialize the string to a python dictionary
                       (default: None)

Override options in the [run_shot] section:
-d {yes,no}, --dither-with-probes {yes,no}
                      Use the guide probes for dithering (default: None)
-w {yes,no}, --wait-last-write {yes,no}
                      Wait for the last write to happen before exiting
                      (default: None)
-N N_OCD_RUN_SHOT, --n-ocd-run-shot N_OCD_RUN_SHOT
                      Which of the urls for the ``ocd_run_shot`` option of
                      the ``[urls]`` section use to setup a ZMQ server
                      (default: None)

Override options in the [paths] section.:
--shuffle-shot SHUFFLE_SHOT
                      Directory containing the shuffle output directoried
                      (default: None)
--shuffle-conf-template SHUFFLE_CONF_TEMPLATE
                      Template for the name of the configuration files
                      generated by shuffle (default: None)
--acam-dest-file ACAM_DEST_FILE
                      File where to copy the ACAM image (default: None)

Override options in the [urls] section:
--ocd-run-shot OCD_RUN_SHOT [OCD_RUN_SHOT ...]
                      Urls/paths used to send the signal when running a
                      shot. Used only if ``-e/--emit-events`` is enabled.
                      (default: None)

The configuration overriding mechanism works only on existing options: make
sure that the corresponding sections and options exist. The section names are
given in the description of the command and the option names are, unless
otherwise stated, the same as the command line options long names.

ocd allow_hetdex

When ocd run starts, OCD is not allowed to execute shots. With ocd allow_hetdex it is possible to interact with the main OCD program to enable or disable the HETDEX shot execution. Run ocd allow_hetdex -h for the full list of arguments [1]:

usage: ocd allow_hetdex [-h] [--config [CONFIG]] [-n MAX_ATTEMPTS]
                        [-I INTERVAL]
                        [-o OCD_ALLOW_HETDEX [OCD_ALLOW_HETDEX ...]]
                        [-i OCD_MAIN_LOOP [OCD_MAIN_LOOP ...]]
                        [-N N_OCD_ALLOW_HETDEX]
                        {start,stop}

Subcommand to enable or disable the execution of HETDEX shots, even if the
conditions are good.

positional arguments:
{start,stop}          "start": allow OCD to execute HETDEX shots. "stop":
                      don't start the execution of new HETDEX shot; if a
                      shot is running, it will finish.

optional arguments:
-h, --help            show this help message and exit
--config [CONFIG]     Override the builtin configurations with the given
                      configuration file. If the option is used without a
                      value, "ocd.cfg" is used. Non existing files are
                      skipped and a warning is printed to the screen.
                      (default: None)
-n MAX_ATTEMPTS, --max-attempts MAX_ATTEMPTS
                      Maximum number of times "ocd allow_hetdex" attempts to
                      enable/disable HETDEX shot execution (default: 5)
-I INTERVAL, --interval INTERVAL
                      Wait "interval" seconds before a new attempt.
                      (default: 1)

Override options in the [urls] section:
-o OCD_ALLOW_HETDEX [OCD_ALLOW_HETDEX ...], --ocd-allow-hetdex OCD_ALLOW_HETDEX [OCD_ALLOW_HETDEX ...]
                      Urls/paths used to send the signal to enable/disable
                      the HETDEX execution. (default: None)
-i OCD_MAIN_LOOP [OCD_MAIN_LOOP ...], --ocd-main-loop OCD_MAIN_LOOP [OCD_MAIN_LOOP ...]
                      Url/path from where the confirmation of the
                      enabling/disabling arrive. (default: None)

Override options in the [allow_hetdex] section:
-N N_OCD_ALLOW_HETDEX, --n-ocd-allow-hetdex N_OCD_ALLOW_HETDEX
                      Which of the urls for the ``ocd_allow_hetdex`` option
                      of the ``[urls]`` section use to setup a ZMQ server
                      (default: None)

The configuration overriding mechanism works only on existing options: make
sure that the corresponding sections and options exist. The section names are
given in the description of the command and the option names are, unless
otherwise stated, the same as the command line options long names.

ocd db_replay

This command allows to replay a TCS sqlite3 database, e.g. as created during night operation. This is very useful for testing and debugging OCD. The command can be executed only with a running ocd run instance [2]. Run ocd db_replay -h for the full list of arguments [1]:

usage: ocd db_replay [-h] [--config [CONFIG]] [--serve-forever] [-t TIMEOUT]
                     [-s SPEEDUP] [-T TOPICS [TOPICS ...]]
                     [-S {none,__data_time,__wire_time}] [--no-convert-number]
                     [--no-convert-bool]
                     [--ocd-db-replay OCD_DB_REPLAY [OCD_DB_REPLAY ...]]
                     db_name

Subcommand to replay a sqlite3 database, typically from night operations.

positional arguments:
db_name               Name of the database to replay

optional arguments:
-h, --help            show this help message and exit
--config [CONFIG]     Override the builtin configurations with the given
                      configuration file. If the option is used without a
                      value, "ocd.cfg" is used. Non existing files are
                      skipped and a warning is printed to the screen.
                      (default: None)
--serve-forever       After the database finishes, restart serving it.
                      (default: False)
-t TIMEOUT, --timeout TIMEOUT
                      When trying to connect with the OCD main loop, wait at
                      most 'timeout' seconds before aborting (default: 10)
-s SPEEDUP, --speedup SPEEDUP
                      Speedup to apply when replaying the database. For
                      example, speedup=1 plays back at the original speed;
                      speedup=2 plays it twice as fast (default: 1.0)
-T TOPICS [TOPICS ...], --topics TOPICS [TOPICS ...]
                      List of topics to publish. If no topic is specified,
                      all the topics will be published. (default: None)
-S {none,__data_time,__wire_time}, --sort-by {none,__data_time,__wire_time}
                      Method to time-sort the events before serving them. If
                      'none', no sorting is done and the order of the
                      database is respected. (default: __wire_time)

Override options in the [urls] section:
--ocd-db-replay OCD_DB_REPLAY [OCD_DB_REPLAY ...]
                      Urls/paths used to send the signal when replaying a
                      database. (default: None)

The configuration overriding mechanism works only on existing options: make
sure that the corresponding sections and options exist. The section names are
given in the description of the command and the option names are, unless
otherwise stated, the same as the command line options long names.

ocd docker_mysql

Note

This subcommand is available only if the specific dependences, namely Docker SDK for Python, is installed. The easiest way to have it is to install OCD with the [docker] specifier.

Shortly before submitting a new shot, OCD queries a MySQL database to retrieve the observation number to use and creates as new entry with the new number. Setting up a MySQL server and database is not trivial and the commands to do this depends on the version of MySQL in use. To allow easy deployment of a temporary MySQL server for testing OCD outside of HET, OCD provides the docker_mysql subcommand:

usage: ocd docker_mysql [-h] {up,down,info} ...

OCD subcommand that manages a docker container used to run a mysql server
useful for test OCD.

optional arguments:
  -h, --help      show this help message and exit

Docker-mysql
                                            subcommands:
Type 'ocd docker_mysql cmd -h' for detailed information about the
subcommands

{up,down,info}
    up            Start the mysql docker container
    down          Stop and remove the mysql docker container
    info          Get information about the docker image

It consists of three actions:

• ocd docker_mysql up: starts a docker container running a MySQL server, wait for it to come alive, adds the table vl_obsnum and insert one entry.
• ocd docker_mysql info: prints basic information about the docker container
• ocd docker_mysql down: stop and remove the container and the associated volumes

Each action has different options: run ocd docker_mysql <action> -h for the full list of arguments. The name of the docker container is set in the container_name option of the [docker_mysql] configuration section and can be easily overwritten by the -c/--container-name command line option.

See MySQL database for more information about the database

Footnotes

[1]	(1, 2, 3, 4, 5) we try to keep the usage in the documentation in sinc with the code. However always rely on the output of -h/--help for up-to-date usage.

[2]	if this requirement is too stringent, let the developers know.