Commit 6a5deeea authored by Luc Hermitte's avatar Luc Hermitte
Browse files

Merge branch '60-add-guideline-of-tasks-required-to-publish-a-new-version' into 'develop'

Resolve "Add guideline of tasks required to publish a new version"

Closes #60

See merge request s1-tiling/s1tiling!29
parents 4a0833f6 396653d6
......@@ -64,7 +64,7 @@ Dask: tasks
-----------
Given :ref:`pipeline descriptions <dev_pipeline>`, a requested S2 tile and its
intersecing S1 images, S1 Tiling builds a set of dependant :std:doc:`Dask tasks
intersecting S1 images, S1 Tiling builds a set of dependant :std:doc:`Dask tasks
<graphs>`. Each task corresponds to an actual pipeline which will transform a
given image into another named image product.
......@@ -90,12 +90,12 @@ Step Factories
Steps
+++++
Step types are usually instanciated automatically.
Step types are usually instantiated automatically.
- :class:`FirstStep <s1tiling.libs.otbpipeline.FirstStep>` is instanciated
- :class:`FirstStep <s1tiling.libs.otbpipeline.FirstStep>` is instantiated
automatically by the program from existing files (downloaded, or produced by
a pipeline earlier in the sequence of pipelines)
- :class:`MergeStep <s1tiling.libs.otbpipeline.MergeStep>` is also instanciated
- :class:`MergeStep <s1tiling.libs.otbpipeline.MergeStep>` is also instantiated
automatically as an alternative to :class:`FirstStep
<s1tiling.libs.otbpipeline.FirstStep>` in the case of steps that expect
several input files. This is for instance the case of :class:`Concatenate
......@@ -105,7 +105,7 @@ Step types are usually instanciated automatically.
- :class:`Step <s1tiling.libs.otbpipeline.Step>` is the main class for steps
that execute an OTB application.
- :class:`AbstractStep <s1tiling.libs.otbpipeline.AbstractStep>` is the root
class of steps hierarchy. It still get instanciated automatically for steps
class of steps hierarchy. It still get instantiated automatically for steps
not related to an OTB application.
``AbstractStep``
......@@ -141,7 +141,7 @@ Existing processings
++++++++++++++++++++
The :ref:`domain processings <processings>` are defined through
:class:`StepFactory` subclasses, which in turn will instanciate domain unaware
:class:`StepFactory` subclasses, which in turn will instantiate domain unaware
subclasses of :class:`AbstractStep` for the actual processing.
``AnalyseBorders``
......@@ -192,80 +192,3 @@ subclasses of :class:`AbstractStep` for the actual processing.
.. autoclass:: s1tiling.libs.otbwrappers.SmoothBorderMask
:members:
:show-inheritance:
======================================================================
How To's...
======================================================================
How to add a new processing?
----------------------------
This is done by deriving :class:`StepFactory
<s1tiling.libs.otbpipeline.StepFactory>`. You'll find many examples in the
default :ref:`step factories <Existing Processings>`.
The important points are to decide:
- Where should the step happen in the sequence of pipelines? |br|
In all cases, don't forget to add it in a pipeline registered in the sequence
of pipelines.
- Shall its result be considered as a public product, or an intermediary step?
|br|
A public product is expected to be always produced. It shall then conclude a
:ref:`pipeline <Pipelines>`. Also the pipeline shall be registered with
``product_required=True`` in that case.
- What would be the name of the result files? |br|
Override :func:`build_step_output_filename()
<s1tiling.libs.otbpipeline.StepFactory.build_step_output_filename>` with the
answer.
.. note::
Even if there is no OTB application behind the step, this method needs to
forward the filename of the input as done in
:func:`AnalyseBorders.build_step_output_filename()
<s1tiling.libs.otbwrappers.AnalyseBorders.build_step_output_filename>`.
- Which configuration options would be needed? |br|
Copy them from the constructor that will be passed the
:class:`s1tiling.libs.configuration.Configure` object.
- What meta information should be filled-in? |br|
This should be done in :func:`complete_meta()
<s1tiling.libs.otbpipeline.StepFactory.complete_meta>`. |br|
Meta information can be used:
- immediately by other methods like :func:`parameters()
<s1tiling.libs.otbpipeline.StepFactory.parameters>`,
- or by later steps in the pipeline.
- If there is an OTB application behind the step -- which should be the case
for most processing steps.
In case the step relates to an OTB application:
- What would be the name of the temporary files while they are being produced? |br|
Return the information from :func:`build_step_output_tmp_filename()
<s1tiling.libs.otbpipeline.StepFactory.build_step_output_tmp_filename>`,
- Where the product should be produced? |br|
Return the information from :func:`output_directory()
<s1tiling.libs.otbpipeline.StepFactory.output_directory>` -- this is
typicallly used from :func:`build_step_output_filename()
<s1tiling.libs.otbpipeline.StepFactory.build_step_output_filename>`.
- What parameters shall be sent to the OTB application? |br|
Return the information from :func:`parameters()
<s1tiling.libs.otbpipeline.StepFactory.parameters>`.
- What are the parameters expected by the OTB application from the images that
could be passed in-memory? |br|
The default are ``"in"`` and ``"out"`` but could be overridden in the
constructor of the new step factory through the parameters ``param_in`` and
``param_out``. See for instance
:func:`s1tiling.libs.otbwrappers.OrthoRectify.__init__` implementation.
- What is the OTB application? |br|
Its name is expected to be passed to the constructor of the parent class,
from the constructor of the new class.
Technically all other methods from :class:`StepFactory
<s1tiling.libs.otbpipeline.StepFactory>` could be overriden. For instance,
:func:`create_step() <s1tiling.libs.otbpipeline.StepFactory.create_step>` could
be overridden to change the type of :ref:`Steps` instanciated.
.. # define a hard line break for HTML
.. |br| raw:: html
<br />
.. _howto_dev:
.. index:: Developer documentation (how to)
======================================================================
How To's...
======================================================================
How to add a new processing?
----------------------------
This is done by deriving :class:`StepFactory
<s1tiling.libs.otbpipeline.StepFactory>`. You'll find many examples in the
default :ref:`step factories <Existing Processings>`.
The important points are to decide:
- Where should the step happen in the sequence of pipelines? |br|
In all cases, don't forget to add it in a pipeline registered in the sequence
of pipelines.
- Shall its result be considered as a public product, or an intermediary step?
|br|
A public product is expected to be always produced. It shall then conclude a
:ref:`pipeline <Pipelines>`. Also the pipeline shall be registered with
``product_required=True`` in that case.
- What would be the name of the result files? |br|
Override :func:`build_step_output_filename()
<s1tiling.libs.otbpipeline.StepFactory.build_step_output_filename>` with the
answer.
.. note::
Even if there is no OTB application behind the step, this method needs to
forward the filename of the input as done in
:func:`AnalyseBorders.build_step_output_filename()
<s1tiling.libs.otbwrappers.AnalyseBorders.build_step_output_filename>`.
- Which configuration options would be needed? |br|
Copy them from the constructor that will be passed the
:class:`s1tiling.libs.configuration.Configure` object.
- What meta information should be filled-in? |br|
This should be done in :func:`complete_meta()
<s1tiling.libs.otbpipeline.StepFactory.complete_meta>`. |br|
Meta information can be used:
- immediately by other methods like :func:`parameters()
<s1tiling.libs.otbpipeline.StepFactory.parameters>`,
- or by later steps in the pipeline.
- If there is an OTB application behind the step -- which should be the case
for most processing steps.
In case the step relates to an OTB application:
- What would be the name of the temporary files while they are being produced? |br|
Return the information from :func:`build_step_output_tmp_filename()
<s1tiling.libs.otbpipeline.StepFactory.build_step_output_tmp_filename>`,
- Where the product should be produced? |br|
Return the information from :func:`output_directory()
<s1tiling.libs.otbpipeline.StepFactory.output_directory>` -- this is
typically used from :func:`build_step_output_filename()
<s1tiling.libs.otbpipeline.StepFactory.build_step_output_filename>`.
- What parameters shall be sent to the OTB application? |br|
Return the information from :func:`parameters()
<s1tiling.libs.otbpipeline.StepFactory.parameters>`.
- What are the parameters expected by the OTB application from the images that
could be passed in-memory? |br|
The default are ``"in"`` and ``"out"`` but could be overridden in the
constructor of the new step factory through the parameters ``param_in`` and
``param_out``. See for instance
:func:`s1tiling.libs.otbwrappers.OrthoRectify.__init__` implementation.
- What is the OTB application? |br|
Its name is expected to be passed to the constructor of the parent class,
from the constructor of the new class.
Technically all other methods from :class:`StepFactory
<s1tiling.libs.otbpipeline.StepFactory>` could be overridden. For instance,
:func:`create_step() <s1tiling.libs.otbpipeline.StepFactory.create_step>` could
be overridden to change the type of :ref:`Steps` instantiated.
Release a new version
---------------------
Here is a short list of the actions to do for each new release.
1. Update the :ref:`release notes <release_notes>`
2. Make sure :file:`__meta__.py` version matches the name of the version to be
released.
Don't forget the `rcX` suffix if need be.
Version format is expected to follow the following convention:
``M.m(.p)(rcX)`` See
https://packaging.python.org/guides/distributing-packages-using-setuptools/#standards-compliance-for-interoperability
Let's extract version number into a variable to simplify following steps
.. code:: bash
version="$(awk '/version/ {print $3}' s1tiling/__meta__.py | xargs )"
echo "version: ${version}"
3. Handle all the issues associated for the related milestone.
4. Push ``develop`` branch and wait for the doc to be completely updated.
.. code::
git checkout develop && git push
4. Merge ``develop`` branch into ``master``
.. code::
git checkout master && git merge develop
5. Push ``master`` branch and wait for the doc to be completely updated.
.. important::
Given current limitations in the generation of the documentation
(multiple tags/branches) cannot be generated simultaneously, **we have
to wait for the documentation for `develop` branch to be updated**
before pushing ``master`` branch.
.. code::
# WAIT...
git checkout master && git push
6 Create a git tag matching the version number
.. code::
git tag -a "${version}"
# And fill in version information
7. Wait the doc to be updated, then
.. important::
Given current limitations in the generation of the documentation
(multiple tags/branches) cannot be generated simultaneously, **we have
to wait for the documentation for `master` branch to be updated**
before pushing the new tag.
.. code::
git push --tags
8. Prepare the packets for pipy
.. code::
python3 setup.py sdist bdist_wheel
9. Push to pipy
.. code::
python3 -m twine upload --repository s1tiling dist/S1Tiling-${version}*
10. Update :file:`__meta__.py` version to the next expected version.
Do not use the `rcX` suffix for the moment.
......@@ -46,6 +46,7 @@ On demand Ortho-rectification of Sentinel-1 data on Sentinel-2 grid.
contribute
developers
how-to-developers
testing
Indices and tables
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment