Input files =========== OCD requires a number of input files to run. The files are described here. .. _master_conf: Master configuration file ------------------------- The master configuration file is shipped with OCD and is loaded every time OCD starts. When loading the configuration file, the environment variables, as given by :data:`os.environ`, are set as defaults, making them available to every section. The file can be retrieved as explained in :ref:`ocd_conf`. The current configuration file is: .. literalinclude:: /../../ocd/static/ocd.cfg :language: cfg .. _shot_file_in: Shot file, input ---------------- When starting OCD loads onto a database a file containing the list of possible shots. The file must have the columns described here: .. autofunction:: ocd.shots_db.load_shot_file :noindex: Although the mechanism that OCD uses to load the shot file is very flexible, we recommend follow these recommendations, so that the file can be also used without modification as input for ``$CUREBIN/autoschedule_main``: * use spaces/tabs as separator; * keep the header in comment lines; * order the columns as in .. autodata:: ocd.shots_db.COLUMN_NAMES :noindex: A typical shot file looks like:: # shotid ra dec track priority n_obs forced_az QIDX QPROG QOBJECT QIFU QRA QDEC QEQUINOX QPMRA QPMDEC QEPOCH # ra: decimal hours # dec: decimal degrees # # test input file for autoschedule_main # track can be: 0: force EAST; 1: force WEST; 2: let autoschedule decide # # $CUREBIN/autoschedule_main -v autoschedule_n_obs.dat 2457923.70 1000 1.5 0.6 1 DEX00001 12.401733 56.146400 2 1 1 -1 89 HET18-2-100 HS0015_E 000 10:55:31.00 +51:07:40.44 2000 0.0 0.0 2000 DEX00005 14.663467 53.661500 2 2 3 322 90 HET18-2-100 HS0016_E 000 10:56:12.10 +51:07:40.44 2000 0.0 0.0 2000 DEX00006 14.924933 49.587000 2 3 1 -1 91 HET18-2-100 HS0018_E 000 10:57:33.40 +51:07:40.44 2000 0.0 0.0 2000 DEX00012 13.000000 56.472000 2 1 1 -1 92 HET18-2-100 HS0019_E 000 10:58:14.50 +51:07:40.44 2000 0.0 0.0 2000 DEX00013 13.566600 56.040500 2 1 2 -1 93 HET18-2-100 HS0021_E 000 10:59:35.80 +51:07:40.44 2000 0.0 0.0 2000 DEX00014 14.180333 55.226000 2 1 2 -1 94 HET18-2-100 HS0024_E 000 11:01:38.20 +51:07:40.44 2000 0.0 0.0 2000 ``DEX00001`` has a higher priority than ``DEX00005`` and ``DEX00006``. ``DEX00005`` must be observed observed 3 times at azimuth ``322``. .. _autoschedule: Autoschedule ------------ Every so often OCD runs ``$CUREBIN/autoschedule_main`` to determine the next shot to run or to obtain a list of probable shots to execute in the future. The name of the executable, ``exe_name``, as well as the names of the support files (see below), comes from the section ``[autoschedule]`` of the configuration file described in :ref:`master_conf`. To run, the code needs: #. shot list: the shot file, as described in :ref:`shot_file_out`, is created from internal database every time ``autoschedule_main`` is run and passed to the code; #. julian date: date when running the code; #. azimuth: current telescope azimuth; it's used to minimize azimuth moves; #. seeing, transparency, sky_background: current conditions from the TCS stream #. init file: initialization file, whose name come from the ``init_file`` configuration option; the structure of the file is:: start_day start_month start_year end_day end_month end_year moon_flag calendar_flag field_flag threashold_acceptability max_exposure_extension where: * moon_flag: 0 for moon down only * calendar_flag: 0: reads from the file “cal.times”, 1: computes for given date range; should always be set to 1 * field_flag: ignored #. lae detection fraction: LAE detection fraction wrt new Moon as a function of lunation, used only if the moon_flag is set to 1; the name comes from the ``lae_dec`` configuration option; the file looks like:: # Lunar age (days, new=0) LAE detection rate 0 1.00 1 0.95 #. position of the moon: RA/DEC of the moon at a given date; the name comes from the ``moon_loc`` configuration option; the file looks like:: # Date (eve/morn) MJDmid % RA Dec 2013 Tue Jan 01/Wed Jan 02 6294.8 78 10 28.9 +03 58 2013 Wed Jan 02/Thu Jan 03 6295.8 69 11 17.3 -00 37 2013 Thu Jan 03/Fri Jan 04 6296.8 59 12 06.6 -05 16 2013 Fri Jan 04/Sat Jan 05 6297.8 48 12 57.7 -09 46 2013 Sat Jan 05/Sun Jan 06 6298.8 37 13 51.3 -13 52 2013 Sun Jan 06/Mon Jan 07 6299.8 27 14 47.9 -17 17 2013 Mon Jan 07/Tue Jan 08 6300.8 17 15 47.5 -19 45 2013 Tue Jan 08/Wed Jan 09 6301.8 9 16 49.6 -20 58 2013 Wed Jan 09/Thu Jan 10 6302.8 4 17 52.7 -20 46 #. illumination tables: name of the files with illumination tables at optimal azimuth and +- DELTA_AZ_TABLES; if they don't exist, new tables are computed; the names come from the ``illum_tables`` configuration option. .. warning:: The computation of new illumination tables is very slow and autoschedule is supposted to be very fast in order to have real time planning informations. We strongly suggest to make sure that the illumination tables are present before ``autoschedule_main`` runs to avoid it to lock up OCD for a long time A copy of the files, described in the points 5.-8. is shipped together with ``cure`` and can be found in the ``cure/plan`` directory. Logging configuration file -------------------------- The TCS proxy module, :mod:`ocd.tcs_proxy`, can setup a number of loggers in the case the TCS modules are not found. The loggers can be setup as described in `official logging documentation `_. OCD ships a (hopefully) working configuration file and it can be retrieved together with the :ref:`master_conf` as described in :ref:`ocd_conf`. The current logging configuration file is: .. literalinclude:: /../../ocd/static/loggers.cfg :language: cfg Shuffle configuration files --------------------------- These files are produced by shuffle and contain the information needed by OCD to setup the telescope for the next shot. The name of the file is stored in the :ref:`master_conf`. The file is loaded and used in :mod:`ocd.run_shot`. As of revision 143 the file has the following structure: .. code-block:: cfg [image] acam_output = ACAMimages/20032651+2952000-R_066_E.jpg [trajectory] az = 80.6822 ra = 20.060988 dec = 29.869333 equinox = 2000.0 dir = EAST [guider1] id = GAIA:20294212161502510080 ra = 20.054997 dec = 29.715554 equinox = 2000.0 trajectory = next u = 0.000 g = 15.904 r = 15.904 i = 0.000 z = 0.000 [guider2] id = GAIA:20298089314361771520 ra = 20.067912 dec = 29.993369 equinox = 2000.0 trajectory = next u = 0.000 g = 17.931 r = 17.931 i = 0.000 z = 0.000 [wfs1] id = GAIA:20294285347746805760 ra = 20.071655 dec = 29.795891 equinox = 2000.0 trajectory = next u = 0.000 g = 14.904 r = 14.904 i = 0.000 z = 0.000 [wfs2] id = GAIA:20294256141969131520 ra = 20.049655 dec = 29.796961 equinox = 2000.0 trajectory = next u = 0.000 g = 15.785 r = 15.785 i = 0.000 z = 0.000 [go_next] move_structure = true move_dome = true move_probes = true Output files ============ .. _shot_file_out: Shot file, output ----------------- Before running :ref:`autoschedule`, an shot file is created from the content of the database. The output shot file has structure described in :ref:`shot_file_in`, with the difference that only the following columns are saved: .. autodata:: ocd.shots_db.SHOT_NAMES :noindex: The file is create in the directory given by the ``out_shot_dir`` option of the ``[shots]`` section. Its value can be either a directory in the file system or ``:tmpdir:``. In the former case, if the directory is not found, it created recursively (using :func:`os.makedirs`). In the latter case, a temporary directory is created using :func:`tempfile.mkdtemp` and the prefix ``tempfile.mkdtemp``; the directory is created only the first time and then reused. The file name is also taken from the configuration file, option ``out_shot_file_template`` and can be any of the following: * a normal string, e.g. ``out_shot.list``: the shot list is saved always on the same file; * a string with one anonymous replacement field (``{}``) or one or more replacement fields with index 0 (``{0}``), e.g. ``ocd_shot_{}.list`` or ``ocd_shot_{0}.list``: any time the shot file must be written a new file is created formatting the replacement field with an integer counter, that is increased in order to avoid accidental overwrites; in the above example the file will be called ``ocd_shot_0.list`` the first time, ``ocd_shot_1.list`` the second and so on; the rotation functionality is provided by :class:`~pyhetdex.tools.files.file_tools.FileNameRotator`; * if the file name contains multiple anonymous fields, one or more named fields or fields with index larger than 1, the shot file templates defaults to ``ocd_shot_{}.list`` and the file is rotated as described above. In the case of the file rotation, it is possible to decide how many files are retained when a new one is made with the ``keep_shot_files`` option of the ``[shots]`` section. If is set to 5, it means that after the *nth* file all the files with a counter less than ``n-5`` are removed. If the option is not found or set to ``-1`` all files will be kept. By default the output shot file will contain all the shots with observations still to be done, i.e. with ``n_obs`` larger than zero. If all shots are required, set the configuration option ``all_shots`` to ``true``.