Input files¶

OCD requires a number of input files to run. The files are described here.

Master configuration file¶

The master configuration file is shipped with OCD and is loaded every time OCD starts. When loading the configuration file, the environment variables, as given by os.environ, are set as defaults, making them available to every section. The file can be retrieved as explained in ocd config. The current configuration file is:

[general]
# Master OCD configuration file
# The environment variables, as returned by ``os.environ`` is injected at run
# time as DEFAULT and are available to the whole configuration

# The command ``ocd config send`` allows to modify configuration options at
# runtime. However not all modifications have an effect on OCD behaviour, since
# some are used to initialize OCD. Legend:
# * {conf modifiable}: modifications through the above command have an effect on OCD
# * {conf static}: modifications through the above command have no effect
# * {conf do not modify}: options that should *NOT* be modified
# * {conf can-be-mod}: the options marked in this way are currently "{conf
#   static}" but can be switched to "{conf modifiable}" if required

# {optional} {conf modifiable}
# If true log the incoming events as DEBUG. This should be used only for
# debugging purposes as it adds a lot of very long log messages. Default: false
log_events = false

[run]
# configuration necessary to initialize the ZeroMQ servers in the ``ocd run``
# command

# {optional} {conf do not modify}
# the following configurations tell ``ocd run`` to use the first urls of the
# ``ocd_main_loop`` and ``ocd_run_shot`` options in ``[urls]`` to initialize
# the ZeroMQ servers necessary
n_ocd_main_loop = 0
n_ocd_run_shot = 0

[run_shot]
# configuration entries used when running a shot
# Unless otherwise stated, all the configurations in this section are
# {conf modifiable}

# {mandatory} path to the shuffle directory/ies. Any path in the configuration
# file, e.g. acam_image, must be relative to this directory. The file name is
# constructed using the format syntax
# <https://docs.python.org/3/library/string.html#formatstrings>`. The following
# fields are allowed: ``{shotid}``, ``{ra}``, ``{dec}``,  ``{track_name}``
# (EAST/WEST), ``{track_number}`` (0/1), ``{track_initial}`` (E/W), all the
# names passed to the ``ocd run_shot`` command or the ``ocd.run_shot.run``
# function as metadata. For the shots run by OCD main loop, this means all the
# Q* fields present in the input shot file. When running ``ocd run_shot`` by
# hand, it means anything given as metadata from the command line
shuffle_shot = /path/to/shuffle/output
# {mandatory} template for the name of the configuration file containing the
# information from shuffle for the shot. Same substitutions as shuffle_shot
shuffle_conf_template = %(shuffle_shot)s/{shotid}_{track_number}/{shotid}_{track_number}_000_{track_initial}.cfg

# {mandatory} absolute tolerance to use when comparing ra, dec, azimuth values
# and track between what OCD plans and what shuffle produces
abs_tol_ra = 1
abs_tol_dec = 1
abs_tol_azimuth = 1
# track is wither 0 or 1. Set it to a value larger than 1 to skip checking the
# track correctness
abs_tol_track = 0.1 

# {mandatory} destination file for the ACAM image
acam_dest_file =  /data1/nossy/www/html/noss/htopx2/acam_dss.jpg

# {mandatory} If true/yes offset the fiducial position of the star in the guide probes to
# execute the dithersing. If false/no use the dither mechanism
dither_with_probes = false

# {mandatory} If true/yes after the last exposure, wait for the readout to
# happen
wait_last_readout = true

# {mandatory} Comma separated list of names of the filters for the guide probes
# to load from the shuffle configuration file
guider_shuffle_filters = g, r, i
# {mandatory} Comma separated list; names of the same filters of
# ``shuffle_filters`` to use when calling
# ``pas.{probe}_SetObjectAndMagnitudes``. The number of elements must match
guider_pas_filters = g`, r`, i`

# {mandatory} timeout in seconds to pass to ``tcs.wait_for_setup``. If
# negative, no timeout used; if positive, wait at most
# ``wait_for_setup_timeout`` seconds for the setup to be confirmed: if this
# doesn't happen an exception is raised
wait_for_setup_timeout = -1
# {mandatory} decide what to do if ``tcs.wait_for_setup`` times out before the
# setup is confirmed: if ``false`` abort the shot execution, if ``true``
# continue running the shot
continue_on_timeout = false

# {mandatory} after the telescope setup and before an exposure starts trigger
# the exposure time countdown display
# executable
message_board = mboard_message -exp {exptime} -inst virus

# {mandatory} after the telescope setup and after the exposures are done play a
# sound
# executable
play_exe = play -q
# sound played after the setup after ``go_next`` is done
setup_sound = alert-3.ogg
# sounds played after each exposure is done
finish_exp_1_sound = %(setup_sound)s
finish_exp_2_sound = %(setup_sound)s
finish_exp_3_sound = %(setup_sound)s
# sound played if the shot execution crashes
failure_sound = %(setup_sound)s

# {optional} {conf do not modify}
# configuration necessary to initialize the ZeroMQ servers in the
# ``ocd run_shot`` command.
# If you think that you are going to use ``ocd run_shot`` together with ``ocd
# run``, then you might need two ocd_run_shot urls and the following entry
# should be set to ``1``
n_ocd_run_shot = 1

[allow_hetdex]
# configuration necessary to initialize the ZeroMQ servers in the ``ocd
# allow_hetdex`` command

# {optional} {conf do not modify}
# the following configurations tell ``ocd allow_hetdex`` to use the first urls
# of the ``ocd_allow_hetdex`` option in ``[urls]`` to initialize the ZeroMQ
# servers necessary
n_ocd_allow_hetdex = 0

[db_replay]
# configuration necessary to initialize the ZeroMQ servers in the ``ocd
# db_replay`` command

# {optional} {conf do not modify}
# the following configurations tell ``ocd db_replay`` to use the first urls
# of the ``ocd_db_replay`` option in ``[urls]`` to initialize the ZeroMQ
# servers necessary
n_ocd_db_replay = 0

[config]
# configuration necessary to initialize the ZeroMQ servers in the ``ocd
# config send`` command

# {optional} {conf do not modify}
# the following configurations tell ``ocd config send`` to use the first urls
# of the ``ocd_config`` option in ``[urls]`` to initialize the ZeroMQ
# servers necessary
n_ocd_config = 0

[docker_mysql]
# configuration entries specific to the ``docker_mysql`` command

# {mandatory}
# name of the docker container to use
container_name = ocd_mysql

[urls]
# section with the urls necessary to run OCD
# Unless otherwise stated, all the configurations in this section are
# {conf do not modify}

# {mandatory} Comma-separated list of urls to which the main OCD event listener
# connect. They should be the urls, or any protocol accepted by ZeroMQ, used by
# TCS and the other systems to send events 
event_urls = tcp://127.0.0.1:5556

# {mandatory} Comma-separated list of urls/file names for a protocol `accepted
# by zeroMQ <http://zeromq.org/docs:features#toc6>`
# Server used to send events from the OCD main loop
ocd_main_loop = tcp://127.0.0.1:6600
# Server used to send events from the run_shot module
ocd_run_shot = tcp://127.0.0.1:6601, tcp://127.0.0.1:6621
# Server used to send events from the ``ocd allow_hetdex`` command
ocd_allow_hetdex = tcp://127.0.0.1:6602
# Server used to send events from the ``ocd db_replay`` command
# If both ``event_urls`` and ``ocd_db_replay`` are present, ``ocd run`` aborts.
# This is done to avoid mixing the TCS event stream and the events coming from
# a replayed sqlite3 database, which can lead to confusion and inconsistencies.
# ocd_db_replay = tcp://127.0.0.1:6603
ocd_db_replay =
# Server used to send events from the ``ocd config send`` command
ocd_config = tcp://127.0.0.1:6604

# TCS logging and subsystems configurations
# {mandatory} TCS logging system (TCSLog.TCSLog)
tcs_log = tcp://192.168.66.99:31000
# {mandatory} if TCSLog is not available, a mock class is created and
# initialised. This class is a proxy for a standard python logger with name
# 'ocd.tcs_proxy.MockTCSLog'. By default the NullHandler is attached. However
# it is possible to customize the loggers, using a configuration file according
# to
# https://docs.python.org/3/library/logging.config.html#configuration-file-format
# this options should be either empty or a path to a valid configuration file.
# If the logging configuration file contains all the relevant configurations,
# there is no need to pass it to every ``*_mock_path``
tcs_log_mock_path = 

# {mandatory} name of the subsystems that must be initialized. The
# corresponding configuration entries are below.
# See
# http://www.mpe.mpg.de/~montefra/documentation/tcs_lib/latest/tcs_proxy.html
# for more information
subsystem_names = tcs, virus, pfip, pas

# {mandatory} TCS subsystem (tcssubsystem.TCSSubSystem)
tcs= tcp://192.168.66.31:30300
# {mandatory} if TCSSubSystem is not available, a mock class is created and
# initialised. This class is a proxy for a standard python logger with name
# 'ocd.tcs_proxy.tcs'. See the ``tcs_log_mock_path`` for more info.
tcs_mock_path = 

# {mandatory} virus subsystem (tcssubsystem.TCSSubSystem)
virus = tcp://192.168.66.40:31300
# {mandatory} if TCSSubSystem is not available, a mock class is created and
# initialised. This class is a proxy for a standard python logger with name
# 'ocd.tcs_proxy.virus'. See the ``tcs_log_mock_path`` for more info.
virus_mock_path = 

# {mandatory} pfip subsystem (tcssubsystem.TCSSubSystem)
pfip = tcp://192.168.66.31:30900
# {mandatory} if TCSSubSystem is not available, a mock class is created and
# initialised. This class is a proxy for a standard python logger with name
# 'ocd.tcs_proxy.pfip'. See the ``tcs_log_mock_path`` for more info.
pfip_mock_path = 

# {mandatory} pas subsystem (tcssubsystem.TCSSubSystem)
pas = tcp://192.168.66.31:30700
# {mandatory} if TCSSubSystem is not available, a mock class is created and
# initialised. This class is a proxy for a standard python logger with name
# 'ocd.tcs_proxy.pas'. See the ``tcs_log_mock_path`` for more info.
pas_mock_path = 

[topics]
# {mandatory} {conf static}
# Comma-separated list of OCD subscribes to all the topics necessary for it.
# This option can be used to provide additional topics. If empty or not found,
# no extra topic added
extra_topics =

# {mandatory} {conf static}
# Turn off the topic auto-subscription in OCD, but subscribe only to the topics
# given with ``extra_topics``. If not found or empty, default to ``true``
auto_subscribe = yes

[containers]
# Data container configurations

# Unless otherwise stated, all the configurations in this section are
# {conf can-be-mod}

# {mandatory}
# Name of the key in the pas.Guider{1,2}.metrology_data from which the sky
# magnitude is extracted. As of 21.12.2017 these are the options
photometry_skymag = photometry.kron_skymag
# photometry_skymag = photometry.fixed_skymag

# {mandatory} Name of the key in the pas.Guider{1,2}.metrology_data from which
# the star magnitude is extracted. As of 21.12.2017 these are the options
photometry_trans = transparency
# photometry_trans = photometry.kron_mag
# photometry_trans = photometry.fixed_mag
# photometry_trans = fit.moffat_mag
# photometry_trans = fit.gauss_mag
# {optional}{temporary} value of the illumination correction to use to compute
# transparency. If not provided uses 0.86.
# Note: this option will be removed once the illumination correction is
# included into the events
# illumination_correction = 0.86

# {mandatory} Name of the key in the pas.Guider{1,2}.metrology_data
# containing the image quality calculated from the guider data.
image_quality = image_quality
# photometry_iq = image_quality_gauss

# {mandatory} {conf static}
# Maximum length of the elements stored in the containers.  To leave the size
# unbound, comment the entry above and uncomment the one below. Notice the lack
# of ``=`` sign: this is intentional and necessary in order to associate
# ``None`` to maxlen.
maxlen = 50
# maxlen

# {mandatory} When adding a new value keep at most values that are
# ``delta_timestamp`` seconds older than the new added. To avoid using a delta
# time uncomment the second entry (as for maxlen)
# delta_timestamp = 1000
delta_timestamp

# {mandatory} [minimum, maximum] values of the fwhm range. Units: arcsecond.
# It must be comma separated list of two floats.
ref_fwhm = 0, 1.9

# {mandatory} [minimum, maximum] values of the sky magnitude range. Units:
# magnitudes. It must be comma separated list of two floats.
ref_skymag = 16, 30

# {mandatory} [minimum, maximum] values of the transparency range. Units: none.
# It must be comma separated list of two floats
# ps: Now is set to 2 to accommodate values above 1 (in case of problems or
# numeral errors)
ref_transparency = 0.75, 2

# {mandatory} [minimum, maximum] values of the allowed transparency range.
# Values outside this range, are considered invalid, and not used for transparency
# calculations. Units: none.
# It must be comma separated list of two floats
# ps: Now is set to 2 to accommodate values above 1 (in case of problems or
# numeral errors)
valid_transparency = 0.1, 5

# {mandatory} [minimum, maximum] values of the allowed image quality range.
# Values outside this range, are considered invalid, and not used for fwhm
# calculations. Units: arcsec.
# It must be comma separated list of two floats
valid_image_quality = 0.05, 1000.

# {mandatory} Whether to consider good conditions if both guide probes are good
# or if one good probe is enough
both_gp_good = true

[database]
# configurations for handling the OCD internal sqlite3 and the
# external MySQL databases

# {mandatory} {conf static}
# name of the database. It can be one of the following 3 options
# * a file path: use this method for persistence
# * :memory:: for an in memory database
# * :tmpfile:: for a temporary file in a /tmp/ocd-\d* directory
database_name = :tmpfile:

# {optional} {conf static}
# if found and true, drops existing tables in the database and recreate them,
# effectively cleaning up the database. By default doesn't drop tables.
# Effective only if the database_name is a path to an existing file
drop_existing_tables = false

# {mandatory} {conf modifiable}
# configuration for the mysql database containing the vl_obsnum table 
mysql_host=127.0.0.1
mysql_port=3306
mysql_database=test_db
mysql_user=test_user
mysql_password=test
# {optional}{conf modifiable}
# if the following entry is false, do not insert in the mysql database the new
# observation number. This options is should be set to false for testing and
# when running OCD in listening mode. Default: true
mysql_update_obsnum = false
# {mandatory} {conf modifiable}
# If the observation number is equal or larger that the following number, it
# will not be added back to the MySQL database and the shot will not be run
max_obsnum = 1000

[shots]

# {mandatory} {conf static}
# path to the file containing the shot files to load into OCD see the
# documentation for the description of the necessary columns
shot_file = /path/to/shot.list

# {optional} {conf static}
# if found and true, clear the table before inserting the shots from the
# shot_file
clear_shot_table = true

# {optional} {conf static}
# If the shots database table is not empty, as ``shotid``s are unique, we need
# to avoid conflicts between ``shotid`` already in the table and ones coming
# from the shot file.  If update_existing_ids is true, the entries from the
# shot files can update the ``shotid``s in the database, otherwise matching
# ``shotid``s are skipped
update_existing_ids = false

# {mandatory} {conf modifiable}
# Directory where to put the files with the list of shots stored in the
# database. If :tmpdir: a temporary directory is created and used throughout
# the execution of OCD. If the directory does not exist, OCD attempts to create
# it recursively
out_shot_dir = :tmpdir:

# {mandatory} {conf modifiable}
# template for the output shot file name. It can contain at most one unnamed
# format placeholder. If it contain more than one placeholder, the value
# 'ocd_shot_{}.list' is used
out_shot_file_template = ocd_shot_{}.list

# {optional} {conf modifiable}
# number of output shot file to keep when a new one is created. If set to -1 or
# not found, no output shot file is removed. This option has an effect only if
# out_shot_file_template contains one format place holder
keep_shot_files = 5

# {optional} {conf modifiable}
# If found and true, writes to the output shot file also the shots with no
# remaining observations
all_shots = false

[autoschedule]
# Options related to the automatic scheduling and execution of shots

# Unless otherwise stated, all the configurations in this section are
# {conf modifiable}

# {mandatory} deal with shots that happen in the future. All times here are in
# seconds
# Imagine that a shot is planned to start in ``dtime`` seconds:
# * if ``dtime >= skip_shot_delta_sec`` skip the shot
# * if ``wait_shot_delta_sec <= dtime < skip_shot_delta_sec``: submit the shot
#   but sleep for ``dtime - wait_shot_delta_sec`` seconds before starting the
#   telescope setup
# * if ``dtime < wait_shot_delta_sec``, start the shot immediately
skip_shot_delta_sec = 360
wait_shot_delta_sec = 60

# {optional} Although a shot is available and can be run, do not execute it.
# This option should be set to ``true`` when one want to run OCD in read-only
# mode but want that OCD plans and prepare shots. If this options is set to
# ``true``, the option ``mysql_update_obsnum`` of the ``[database]`` section is
# set to ``false``. Default: ``false``
skip_shot_submission = false

# Options to run autoschedule_main
# If you are using OCD, you probably already have ``CUREBIN`` set in the
# environment and the variables below will be likely be expanded for you.
# {mandatory} name of the executable. If you don't have it set, set explicitly the path to the
# executable
exe_name = %(CUREBIN)s/autoschedule_main

# {mandatory} Init file for the scheduler. See the ``plan`` manual for more
# info. 
init_file = %(CUREBIN)s/../plan/autoschedule_init.dat

# {mandatory} LAE detection fraction wrt new Moon as a function of lunation,
# used only if the Moon flag is set to 1. 
lae_dec = lae_dec.dat

# {mandatory} RA-DEC of the moon at the relevant dates 
moon_loc = %(CUREBIN)s/../plan/moon_loc.dat

# {mandatory} Comma-separated list of 3 names for the files with illumination
# tables at optimal AZ and +- DELTA_AZ_TABLES. If they don't exist, new tables
# are computed (very slow) 
illum_tables = %(CUREBIN)s/../plan/Illum_0.dat,%(CUREBIN)s/../plan/Illum_2.dat,%(CUREBIN)s/../plan/Illum-2.dat

[dates]

# {optional} {conf static}
# if this value is provided, it must contain a UTC date/time that
# astropy.time.Time can parse (http://docs.astropy.org/en/stable/time/#id3).
# If the option not is used, the times used to run e.g. ``autoschedule_main``
# refers to current UTC times
# If the option is used, a mock object is initialized with the ``mock_time``,
# and calls to ocd.utils.get_utc and ocd.utils.get_jd return a new time ``n``
# seconds after ``mock_time``, where ``n`` is the time between initializing the
# mock object and the get_* function call.
# If the option is used the user is asked to proceed to avoid troubles during
# operation
# mock_time = 

Shot file, input¶

When starting OCD loads onto a database a file containing the list of possible shots. The file must have the columns described here:

ocd.shots_db.load_shot_file(shot_file)[source]

Load the shot file.

The file must contain at least the columns given by COLUMN_NAMES:

shotid (string): the ID of the shot; must be unique;
ra (float): ra of the shot in decimal hours;
dec (float): dec of the shot in decimal degrees;
track (integer): must be one of 0, 1, 2; 0: force EAST track; 1: force WEST track; 2: let the software decide the optimal track;
priority (int): priority for the shot; 1 is the highest priority and decreases as the number increases; must be equal or larger than 1;
n_obs (int): number of times the shot must be observed; if not present defaults to 1;
forced_az (float): if -1, the scheduling algorithm decides the best azimuth to use, if positive forces the azimuth to the given value.
Q* (from METADATA_NAMES): extra column that connect each HETDEX shot to the global HET queue

The shot file can be in any format supported by astropy ascii.

If file must an header with all the COLUMN_NAMES columns.

Parameters:	shot_file : string name of the file to load
Returns:	table : `astropy.table.Table` instance relevant part of the shot file
Raises:	ShotColumnError if a mandatory column is missing

Although the mechanism that OCD uses to load the shot file is very flexible, we recommend follow these recommendations, so that the file can be also used without modification as input for $CUREBIN/autoschedule_main:

use spaces/tabs as separator;
keep the header in comment lines;
order the columns as in

ocd.shots_db.COLUMN_NAMES = ['shotid', 'ra', 'dec', 'track', 'priority', 'n_obs', 'forced_az', 'QIDX', 'QPROG', 'QOBJECT', 'QIFU', 'QRA', 'QDEC', 'QEQUINOX', 'QPMRA', 'QPMDEC', 'QEPOCH']

Name of the mandatory columns of the input shot file

A typical shot file looks like:

# shotid  ra        dec       track  priority n_obs forced_az  QIDX  QPROG QOBJECT QIFU QRA QDEC QEQUINOX QPMRA QPMDEC QEPOCH
# ra: decimal hours
# dec: decimal degrees
#
# test input file for autoschedule_main
# track can be: 0: force EAST; 1: force WEST; 2: let autoschedule decide
#
# $CUREBIN/autoschedule_main -v autoschedule_n_obs.dat 2457923.70 1000 1.5 0.6 1
DEX00001  12.401733 56.146400 2     1      1     -1  89     HET18-2-100     HS0015_E        000     10:55:31.00     +51:07:40.44    2000    0.0     0.0     2000
DEX00005  14.663467 53.661500 2     2      3     322 90     HET18-2-100     HS0016_E        000     10:56:12.10     +51:07:40.44    2000    0.0     0.0     2000
DEX00006  14.924933 49.587000 2     3      1     -1  91     HET18-2-100     HS0018_E        000     10:57:33.40     +51:07:40.44    2000    0.0     0.0     2000
DEX00012  13.000000 56.472000 2     1      1     -1  92     HET18-2-100     HS0019_E        000     10:58:14.50     +51:07:40.44    2000    0.0     0.0     2000
DEX00013  13.566600 56.040500 2     1      2     -1  93     HET18-2-100     HS0021_E        000     10:59:35.80     +51:07:40.44    2000    0.0     0.0     2000
DEX00014  14.180333 55.226000 2     1      2     -1  94     HET18-2-100     HS0024_E        000     11:01:38.20     +51:07:40.44    2000    0.0     0.0     2000

DEX00001 has a higher priority than DEX00005 and DEX00006. DEX00005 must be observed observed 3 times at azimuth 322.

Autoschedule¶

Every so often OCD runs $CUREBIN/autoschedule_main to determine the next shot to run or to obtain a list of probable shots to execute in the future. The name of the executable, exe_name, as well as the names of the support files (see below), comes from the section [autoschedule] of the configuration file described in Master configuration file.

To run, the code needs:

shot list: the shot file, as described in Shot file, output, is created from internal database every time autoschedule_main is run and passed to the code;
julian date: date when running the code;
azimuth: current telescope azimuth; it’s used to minimize azimuth moves;
seeing, transparency, sky_background: current conditions from the TCS stream
init file: initialization file, whose name come from the init_file configuration option; the structure of the file is:
```
start_day start_month start_year
end_day end_month end_year
moon_flag
calendar_flag
field_flag
threashold_acceptability
max_exposure_extension
```
where:
- moon_flag: 0 for moon down only
- calendar_flag: 0: reads from the file “cal.times”, 1: computes for given date range; should always be set to 1
- field_flag: ignored
lae detection fraction: LAE detection fraction wrt new Moon as a function of lunation, used only if the moon_flag is set to 1; the name comes from the lae_dec configuration option; the file looks like:
```
# Lunar age (days, new=0)       LAE detection rate
0             1.00
1             0.95
```

position of the moon: RA/DEC of the moon at a given date; the name comes from the moon_loc configuration option; the file looks like:

# Date (eve/morn)           MJDmid   %     RA     Dec
Tue Jan 01/Wed Jan 02  6294.8   78  10 28.9 +03 58
Wed Jan 02/Thu Jan 03  6295.8   69  11 17.3 -00 37
Thu Jan 03/Fri Jan 04  6296.8   59  12 06.6 -05 16
Fri Jan 04/Sat Jan 05  6297.8   48  12 57.7 -09 46
Sat Jan 05/Sun Jan 06  6298.8   37  13 51.3 -13 52
Sun Jan 06/Mon Jan 07  6299.8   27  14 47.9 -17 17
Mon Jan 07/Tue Jan 08  6300.8   17  15 47.5 -19 45
Tue Jan 08/Wed Jan 09  6301.8    9  16 49.6 -20 58
Wed Jan 09/Thu Jan 10  6302.8    4  17 52.7 -20 46

illumination tables: name of the files with illumination tables at optimal azimuth and +- DELTA_AZ_TABLES; if they don’t exist, new tables are computed; the names come from the illum_tables configuration option.

Warning

The computation of new illumination tables is very slow and autoschedule is supposted to be very fast in order to have real time planning informations. We strongly suggest to make sure that the illumination tables are present before autoschedule_main runs to avoid it to lock up OCD for a long time

A copy of the files, described in the points 5.-8. is shipped together with cure and can be found in the cure/plan directory.

Logging configuration file¶

The TCS proxy module, ocd.tcs_proxy, can setup a number of loggers in the case the TCS modules are not found. The loggers can be setup as described in official logging documentation. OCD ships a (hopefully) working configuration file and it can be retrieved together with the Master configuration file as described in ocd config. The current logging configuration file is:

[loggers]
keys = root, tcs_log, tcs, virus, pfip, pas

[handlers]
keys = null, stdout_tcs_log, stdout_tcs, stdout_virus, stdout_pfip, stdout_pas

[formatters]
keys = tcs_log, tcs, virus, pfip, pas

# root logger (mandatory)
[logger_root]
level = NOTSET
handlers = null

[handler_null]
class = NullHandler
level = NOTSET
args = ()

# tcs_log logger
[logger_tcs_log]
level = DEBUG
handlers = stdout_tcs_log
propagate = 1
qualname = ocd.tcs_proxy.tcs_log

[handler_stdout_tcs_log]
class = StreamHandler
level = DEBUG
formatter = tcs_log
args = ()

[formatter_tcs_log]
format = [Mock TCSLog - %(asctime)s - %(levelname)s] %(message)s
datefmt =

# tcs subsystem logger
[logger_tcs]
level = DEBUG
handlers = stdout_tcs
propagate = 1
qualname = ocd.tcs_proxy.tcs

[handler_stdout_tcs]
class = StreamHandler
level = DEBUG
formatter = tcs
args = ()

[formatter_tcs]
format = [tcs subsystem - %(asctime)s - %(levelname)s] %(message)s
datefmt =

# virus subsystem logger
[logger_virus]
level = DEBUG
handlers = stdout_virus
propagate = 1
qualname = ocd.tcs_proxy.virus

[handler_stdout_virus]
class = StreamHandler
level = DEBUG
formatter = virus
args = ()

[formatter_virus]
format = [virus subsystem - %(asctime)s - %(levelname)s] %(message)s
datefmt =

# pfip subsystem logger
[logger_pfip]
level = DEBUG
handlers = stdout_pfip
propagate = 1
qualname = ocd.tcs_proxy.pfip

[handler_stdout_pfip]
class = StreamHandler
level = DEBUG
formatter = pfip
args = ()

[formatter_pfip]
format = [pfip subsystem - %(asctime)s - %(levelname)s] %(message)s
datefmt =
[logger_pas]
level = DEBUG
handlers = stdout_pas
propagate = 1
qualname = ocd.tcs_proxy.pas

[handler_stdout_pas]
class = StreamHandler
level = DEBUG
formatter = pas
args = ()

[formatter_pas]
format = [pas subsystem - %(asctime)s - %(levelname)s] %(message)s
datefmt =

Shuffle configuration files¶

These files are produced by shuffle and contain the information needed by OCD to setup the telescope for the next shot. The name of the file is stored in the Master configuration file. The file is loaded and used in ocd.run_shot. As of revision 143 the file has the following structure:

[image]
acam_output = ACAMimages/20032651+2952000-R_066_E.jpg

[trajectory]
az = 80.6822
ra = 20.060988
dec = 29.869333
equinox = 2000.0
dir = EAST

[guider1]
id = GAIA:20294212161502510080
ra = 20.054997
dec = 29.715554
equinox = 2000.0
trajectory = next
u = 0.000
g = 15.904
r = 15.904
i = 0.000
z = 0.000

[guider2]
id = GAIA:20298089314361771520
ra = 20.067912
dec = 29.993369
equinox = 2000.0
trajectory = next
u = 0.000
g = 17.931
r = 17.931
i = 0.000
z = 0.000

[wfs1]
id = GAIA:20294285347746805760
ra = 20.071655
dec = 29.795891
equinox = 2000.0
trajectory = next
u = 0.000
g = 14.904
r = 14.904
i = 0.000
z = 0.000

[wfs2]
id = GAIA:20294256141969131520
ra = 20.049655
dec = 29.796961
equinox = 2000.0
trajectory = next
u = 0.000
g = 15.785
r = 15.785
i = 0.000
z = 0.000

[go_next]
move_structure = true
move_dome = true
move_probes = true

Output files¶

Shot file, output¶

Before running Autoschedule, an shot file is created from the content of the database. The output shot file has structure described in Shot file, input, with the difference that only the following columns are saved:

ocd.shots_db.SHOT_NAMES = ['shotid', 'ra', 'dec', 'track', 'priority', 'n_obs', 'forced_az']: Name of the columns in the input and output shot file that define a shot and its priority, number of observations, …

The file is create in the directory given by the out_shot_dir option of the [shots] section. Its value can be either a directory in the file system or :tmpdir:. In the former case, if the directory is not found, it created recursively (using os.makedirs()). In the latter case, a temporary directory is created using tempfile.mkdtemp() and the prefix tempfile.mkdtemp; the directory is created only the first time and then reused.

The file name is also taken from the configuration file, option out_shot_file_template and can be any of the following:

a normal string, e.g. out_shot.list: the shot list is saved always on the same file;
a string with one anonymous replacement field ({}) or one or more replacement fields with index 0 ({0}), e.g. ocd_shot_{}.list or ocd_shot_{0}.list: any time the shot file must be written a new file is created formatting the replacement field with an integer counter, that is increased in order to avoid accidental overwrites; in the above example the file will be called ocd_shot_0.list the first time, ocd_shot_1.list the second and so on; the rotation functionality is provided by FileNameRotator;
if the file name contains multiple anonymous fields, one or more named fields or fields with index larger than 1, the shot file templates defaults to ocd_shot_{}.list and the file is rotated as described above.

In the case of the file rotation, it is possible to decide how many files are retained when a new one is made with the keep_shot_files option of the [shots] section. If is set to 5, it means that after the nth file all the files with a counter less than n-5 are removed. If the option is not found or set to -1 all files will be kept.

By default the output shot file will contain all the shots with observations still to be done, i.e. with n_obs larger than zero. If all shots are required, set the configuration option all_shots to true.