Contribute to ocd
¶
How To¶
The suggested workflow for implementing bug fixes and/or new features is the following:
Identify or, if necessary, add to our redmine issue tracker one or more issues to tackle. Multiple issues can be addressed together if they belong together. Assign the issues to yourself.
Create a new branch from the trunk with a name either referring to the topic or the issue to solve. E.g. if you need to add a new executable, tracked by issue #1111
do_something
:svn cp ^/trunk ^/branches/do_something_1111\ -m 'create branch to solve issue #1111'
Switch to the branch:
svn switch ^/branches/do_something_1111
Implement the required changes and don’t forget to track your progress on Redmine. If the feature/bug fix requires a large amount of time, we suggest, when possible, to avoid one big commit at the end in favour of smaller commits. In this way, in case of breakages, is easier to traverse the branch history and find the offending code. For each commit you should add an entry in the
Changelog
file.If you work on multiple issues on the same branch, close one issue before proceeding to the next. When closing one issue is good habit to add in the description on the redmine the revision that resolves it.
Every function or class added or modified should be adequately documented as described in Coding style.
Documentation is essential both for users and for your fellow developers to understand the scope and signature of functions and classes. If a new module is added, it should be also added to the documentation in the appropriate place. See the existing documentation for examples.
Each executable should be documented and its description should contain enough information and examples to allow users to easily run it.
Every functionality should be thoroughly tested for python 2.7, 3.4, 3.5 and 3.6 in order to ensure that the code behaves as expected and that future modifications will not break existing functionalities. When fixing bugs, add tests to ensure that the bug will not repeat. For more information see Testing.
Once the issue(s) are solved and the branch is ready, merge any pending change from the trunk:
svn merge ^/trunk
While doing the merge, you might be asked to manually resolve one or more conflicts. Once all the conflicts have been solved, commit the changes with a meaningful commit message, e.g.:
merge ^/trunk into ^/branches/do_something_1111
. Then rerun the test suite to make sure your changes do not break functionalities implemented while you were working on your branch.Then contact the maintainer of
ocd
and ask to merge your branch back to the trunk.
Information about branching and merging can be found in the svn book. For any questions or if you need support do not hesitate to contact the maintainer or the other developers.
Coding style¶
All the code should be compliant with the official python style guidelines described in PEP 8. To help you keep the code in spec, we suggest to install plugins that check the code for you, like Synstastic for vim or flycheck for Emacs.
The code should also be thoroughly documented using the numpy style. See the existing documentation for examples.
Testing¶
ocd
uses the testing framework provided by the pytest package. The tests should cover every
aspect of a function or method. If exceptions are explicitly raised, this
should also tested to ensure that the implementation behaves as expected.
The preferred way to run the tests is using tox, an automatised test help
package. If you have installed tox, with e.g. pip install tox
, you can run
it by typing:
tox
It will take care of creating virtual environments for every supported version
of python (2.7, 3.4, 3.5 and 3.6), if it exists on the system, install
ocd
, its dependences and the packages necessary to run the tests and
runs py.test
.
You can run the tests for a specific python version using:
py.test
or:
python setup.py test
The latter command fetches all the needed dependences, among others pytest
itself, will be fetched and installed in a .eggs
directory. Then it will run
py.test
. This command might fail when running in a virtual environment. Use
the option --addopts
to pass additional options to py.test
.
You can run specific tests providing the file name(s) and, optionally the name of a test. E.g.:
py.test tests/test_utils.py # runs only the tests in the test_utils.py file
py.test tests/test_utils.py::test_common_parser_arguments # runs only one test
A code coverage report is also created thanks to the pytest-cov plugin and can be visualized opening
into a browser cover/index.html
. If you want a recap of the coverage
directly in the terminal you can provide one of the following options when
running py.test
:
--cov-report term
--cov-report term-missing
Besides running the tests, the tox
command also builds, by default, the
documentation and collates the coverage tests from the various python
interpreters and can copy then to some directory. To do the latter create, if
necessary, the configuration file ~/.config/little_deploy.cfg
and add to it
a section called ocd
with either one or both of the following options:
[ocd]
# if given the deploys the documentation to the given dir
doc = /path/to/dir
# if given the deploys the coverage report to the given dir
cover = /path/to/other/dir
# it's also possible to insert the project name and the type of the document
# to deploy using the {project} and {type_} placeholders. E.g
# cover = /path/to/dir/{project}_{type_}
# will be expanded to /path/to/dir/ocd_cover
For more information about the configuration file check little_deploy.
The some of the tests require a running docker
daemon. If docker
is not
found or not running, some of the tests will not run. Tests requiring
docker
can also been skipped using the following option:
--skip-docker Skip tests using docker
This option can also be useful to cut setup time there running a subset of the
tests that do not require docker
.
In order to test the ZeroMQ-based communication functionalities, the tests suite constructs a number of addresses or paths used to initialise ZeroMQ sockets. By default temporary files are created in the pytest’s tmpdir <https://docs.pytest.org/en/latest/tmpdir.html> and used to create IPC paths. Since on UNIX systems paths used with the IPC protocol are limited to 107 characters, if the path is longer that this amount the tests fall back to TCP/IP addresses. The use of TCS/IP addresses can also be forced using the following option:
--force-tcp Force the use of TCS address to initialise ZeroMQ
sockets. The default is to use IPC paths
For other command line arguments type:
py.test -h
For a list of available fixtures type:
py.test --fixtures tests/
tox
and pyenv
¶
Many systems have a limited number of python versions installed. pyenv provides ways to have multiple python
versions that can be used by tox
via the tox-pyenv plugin.
Here we outline the steps necessary to make tox
use pyenv
:
Install
pyenv
following these instructions. We suggest to usebrew
under Mac OS X or the automatic installer. When is done, follow the instructions to enablespyenv
.Install the python versions that you need. E.g. if you have python 2.7 and 3.6 on you system, you can install only missing versions, e.g.:
pyenv install 3.4.6 pyenv install 3.5.3
Of course you can also install 2.7 and 3.6 using pyenv.
Install
tox-pyenv
in the same place wheretox
is installed, i.e. either on the system, a virtual environment or apyenv
instance:pip install tox-pyenv
This way
tox
is can usepyenv which
to locate a required python versionThe last step consists in letting
pyenv
know which python versions to use. If you have already set pyenv global to all the version required for testing you should be done. Otherwise go to theocd
directory and run pyenv local:pyenv local system 3.4.6 3.5.3
This command creates a file called
.python-version
that contains the following three lines:system 3.4.6 3.5.3
It will make
pyenv which
look for python versions in the system directories as well as within thepyenv
directory.If you did installed also other versions of python (e.g. 3.6.0 and 2.7.13) under
pyenv
and want to use them instead of the system ones, you can use:pyenv local 3.6.0 3.4.6 3.5.3 2.7.13
Run
tox
: now you will be able to use all the python version that tox requires.
Documentation¶
To build the documentation you need the additional dependences described in
Optional dependences. They can be installed by hand or during ocd
installation
by executing one of the following commands on a local copy:
pip install /path/to/ocd[doc]
pip install /path/to/ocd[livedoc]
The first install sphinx
, the alabaster
theme and the numpydoc
extension; the second also installs sphinx-autobuild
.
To build the documentation in html format go to the doc
directory and run:
make html
The output is saved in doc/build/html
. For the full list of available
targets type make help
.
If you are updating the documentation and want avoid the
edit-compile-browser refresh
cycle, and you have installed
sphinx-autobuild
, type:
make livehtml
then visit http://127.0.0.1:8000. The html documentation is automatically rebuilt after every change of the source and the browser reloaded.
A note on the state machines¶
The module ocd.states
implements the state machines needed by OCD. The
state machines are implemented using the transitions. Among other functionalities,
it is possible to created diagrams to visualize the states and the transitions
of the machines. To improve the documentation, ocd.states – OCD state machines shows the machine
diagrams below the documentation of every state class.
The diagrams are generated using the script scripts/states_graphs.py
in the
ocd repository. The script monkey patch transitions.Machine
with a
class derived from transitions.extensions.GraphMachine
and then loop
through the OCD states machines to create the plot. To run the script
pygraphviz is required. You can install it
issuing pip install pygraphviz
. If the installation fails, probably it is
because you are missing the graphviz
development packages. On Linux systems
they are called graphviz-devel
or graphviz-dev
. Try to install it with
your package manager and then reinstall pygraphviz
. See python3
scripts/states_graphs.py -h
for the script command line options.
When you modify one or more existing state machines, changing the number of
states or the transitions, the graphs should be recreated and then copied to
doc/source/_static
. This can be achieved with the command:
python3 scripts/states_graphs.py -o doc/source/_static/
When you add one state machine make sure to:
- add it it to ocd.states – OCD state machines as a new section; use any other entry as example
- add the class to the
state_classes
inscripts/states_graphs.py
- add the base name of the output file to the
basenames
inscripts/states_graphs.py
When you remove one state machine make sure to remove all the references to it.