Commit 99723cb9 authored by Jordi Inglada's avatar Jordi Inglada

Merge branch 'develop' of https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb into develop

parents 53c2697b 6fcdf04b
......@@ -9,3 +9,5 @@ Describe as precisely as possible how to reproduce the bug. Try to isolate a min
### Configuration information
OS, OTB version or tag, information related to build (binaries, superbuild, system libs ...)
/label ~bug
Short summary of the requested feature
/label ~feature
......@@ -10,11 +10,11 @@ Explain the rationale for the changes (possible link to a Request For Comments o
#### Classes and files
Give an overview of the implementation: main changes made to classes, files and module. Do not paste complete diff, as it is available in the merge request already.
Give an overview of the implementation: main changes made to classes, files and modules. Do not paste complete diff, as it is available in the merge request already.
#### Applications
Describe any changes made to existing applications, or new application that have been added.
Describe any changes made to existing applications, or new applications that have been added.
#### Tests
......@@ -22,7 +22,7 @@ Describe the testing strategy for new features.
### Documentation
List or link documentation modification that were made (doxygen, example, software guide, application documentation, cookbook).
List or link documentation modifications that were made (doxygen, example, software guide, application documentation, cookbook).
### Additional notes
......
# How to contribute to Orfeo ToolBox ?
Thank you for taking the time to contribute to OTB! This document will guide you
through the workflow and best practices you need to know to send your
contribution.
There are many ways to contribute to OTB:
* [Reporting a bug](#reporting-bugs)
* [Making a feature request](#feature-requests-and-discussions)
* [Improving documentation](#documentation-improvements)
* [Contributing code (C++, Python, CMake, etc.)](#code-contribution)
* [Publishing a remote module](#remote-modules)
Our main workflow uses GitLab for source control, issues and task tracking. We
use a self-hosted gitlab instance:
[`https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb`](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb)
Remember to check out also our [developers mailing list](https://groups.google.com/forum/?hl=fr#!forum/otb-developers/join),
where we discuss some features, improvements and high level project planning.
You are welcome to ask questions there as a beginner or future OTB contributor!
## Reporting bugs
If you have found a bug, you can first [search the existing issues](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/issues?label_name%5B%5D=bug)
to see if it has already been reported.
If it's a new bug, please [open a new issue on GitLab](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/issues/new).
The 'Bug' issue template will help you provide all important information and
help fixing the bug quicker. Remember to add as much information as possible!
## Feature requests and discussions
Feature requests are welcome! Generally you are welcome to simply [open an issue](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/issues)
and discuss your idea there. For more complex requests there is an issue
template for in depth description called 'Request for Comments'.
## Documentation improvements
The two main OTB documentations are the [Software Guide](https://www.orfeo-toolbox.org/SoftwareGuide/index.html)
and the [CookBook](https://www.orfeo-toolbox.org/CookBook/). Their sources are
hosted in the main OTB repository in the `Documentation/` directory. Then, to
contribute documentation use the same workflow as for code contributions (see
below).
See also the [Compiling documentation](https://wiki.orfeo-toolbox.org/index.php/Compiling_documentation)
wiki page for help on building the Sphinx and Latex source.
## Code contribution
The OTB workflow is based on GitLab [Merge Requests](https://docs.gitlab.com/ee/gitlab-basics/add-merge-request.html).
Clone the repository, create a feature branch, commit your changes, push the
feature branch to a fork (or the main repository if you are a core developer),
then send a merge request.
Note that we also accept PRs on our [GitHub mirror](https://github.com/orfeotoolbox/OTB)
which we will manually merge.
### Commit message
On your feature branch, write a good [commit message](https://xkcd.com/1296/):
short and descriptive. If fixing an issue or bug, put the issue number in the
commit message so that GitLab can [crosslink it](https://docs.gitlab.com/ce/user/project/issues/crosslinking_issues.html).
You can prefix your commit message with an indicating flag (DOC, BUG, PKG,
TEST, SuperBuild, etc.).
Standard prefixes for OTB commit messages:
BUG: Fix for runtime crash or incorrect result
COMP: Compiler error or warning fix
DOC: Documentation change
ENH: New functionality
PERF: Performance improvement
STYLE: No logic impact (indentation, comments)
WIP: Work In Progress not ready for merge
For example, here are some good commit messages:
BUG: #1701 Warn users if parameter string is unset
DOC: Fix typo in Monteverdi French translation
COMP: Allow GeoTIFF and TIFF to be disabled when no 3rd party drags them
### Merge request
Your contribution is ready to be added to the main OTB repository? Send a Merge
Request against the `develop` branch on GitLab using the merge request
template. The merge request will then be discussed by the community and the core
OTB team.
* Merge requests can not be merged until all discussions have been resolved (this is enforced by GitLab)
* Merge requests **must receive at least 2 positives votes from PSC members** before being merged
* The merger is responsible for checking that the branch is up-to-date with develop
* Merge requests can be merged by anyone (not just PSC or RM) with push access to develop
* Merge requests can be merged once the dashboard is proven green for this branch
Branches can be registered for dashboard testing by adding one line in `Config/feature_branches.txt` in [otb-devutils repository](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb-devutils.git).
For branches in the main repository, the syntax is the following:
```
branch_name [otb-data_branch_name]
```
The second branch name is optional. It can be set if you need to modify [otb-data](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb-data.git) according to your changes.
For branches in forks, the syntax is the following:
```
user/branch_name [user/otb-data_branch_name]
```
Again, the second branch name is optional.
For users without push access to [otb-devutils repository](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb-devutils.git), the modification can be asked through a merge requests to this repository.
Once the feature branch is registered for testing, it should appear in the *FeatureBranches* section of the [OTB dashboard](https://dash.orfeo-toolbox.org/index.php?project=OTB) next day (remember tests are run on a nighlty basis).
Do not forget to remove the feature branch for testing once it has been merged.
## Remote modules
[Remote Modules](https://wiki.orfeo-toolbox.org/index.php/Remote_Modules) are
the prefered way if you wish to make your apps and filters available to the
community while keeping control and maintenance of their sources. Remote
modules are just like regular modules, except they are not distributed inside
OTB source code. Under some conditions (dependencies, official acceptance
process, etc.), we are also able to distribute your remote module in the
official standalone binaries. See [the wiki](https://wiki.orfeo-toolbox.org/index.php/Remote_Modules)
for more information.
## Gitlab guidelines
In order to organize the issues in our Gitlab instance, we use both labels and
milestones.
The [milestones](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/milestones) should be used to track in which release a feature is merged.
Gitlab can then provide a summary of all features and bugs added to a given release
version.
Regarding labels, we use the following set:
* ~story: significant feature to be implemented with a detailed work plan, it can
correspond to a Request for Comments that has turned into a development action
* ~bug: Bug, crash or unexpected behavior, reported by a user or a developer
* ~feature: Feature request expressed by an OTB user/developer
* ~"To Do": action is planned
* ~Doing: work in progress
* ~api ~app ~documentation ~monteverdi ~packaging ~qgis: optional context information
.. _extended-filenames:
Extended filenames
================================
Extended filenames is an interesting feature of OTB. With it, you can control
several aspects of the beahvior of the OTB in the OTB-Applications or in our
own C++ applications. Historically this feature has been desingn to solve
an issue with how to handle geo-referencing information.
Indeed, there are multiple ways to define geo-referencing information. For
instance, one can use a geographic transform, a cartographic projection,
or a sensor model with RPC coefficients. A single image may contain
several of these elements, such as in the “ortho-ready” products: this
is a type of product still in sensor geometry (the sensor model is
supplied with the image) but it also contains an approximative
geographic transform that can be used to have a quick estimate of the
image localisation. For instance, your product may contain a “.TIF” file
for the image, along with a “.RPB” file that contains the sensor model
coefficients and an “.IMD” file that contains a cartographic projection.
This case leads to the following question: which geo-referencing
element should be used when opening this image in OTB. In
fact, it depends on the users need. For an orthorectification
application, the sensor model must be used. In order to specify which
information should be skipped, a syntax of extended filenames has been
developed for both reading and writing.
Since the development of this feature we have extend this mechanism for
other aspaects: like band or overview selection in reader part or support
create option of gdal in writer part.The reader and writer extended filename
support is based on the same syntax, only the options are different.
To benefit from the extended file name mechanism, the following syntax
is to be used:
::
Path/Image.ext?&key1=<value1>&key2=<value2>
**Note that you’ll probably need to “quote” the filename, especially if calling
applications from the bash command line.**
Reader options
^^^^^^^^^^^^^^
::
&geom=<path/filename.geom>
- Contains the file name of a valid geom file
- Use the content of the specified geom file instead of
image-embedded geometric information
- empty by default, use the image-embedded information if available
-----------------------------------------------
::
&sdataidx=<(int)idx>
- Select the sub-dataset to read
- 0 by default
-----------------------------------------------
::
&resol=<(int)resolution factor>
- Select the JPEG2000 sub-resolution image to read
- 0 by default
-----------------------------------------------
::
&bands=r1,r2,...,rn
- Select a subset of bands from the input image
- The syntax is inspired by Python indexing syntax with
bands=r1,r2,r3,...,rn where each ri is a band range that can be :
- a single index (1-based) :
- :code:`2` means 2nd band
- :code:`-1` means last band
- or a range of bands :
- :code:`3:` means 3rd band until the last one
- :code:`:-2` means the first bands until the second to last
- :code:`2:4` means bands 2,3 and 4
- empty by default (all bands are read from the input image)
-----------------------------------------------
::
&skipcarto=<(bool)true>
- Skip the cartographic information
- Clears the projectionref, set the origin to :math:`[0,0]` and the
spacing to :math:`[1/max(1,r),1/max(1,r)]` where :math:`r` is the resolution
factor.
- Keeps the keyword list
- false by default
-----------------------------------------------
::
&skipgeom=<(bool)true>
- Skip geometric information
- Clears the keyword list
- Keeps the projectionref and the origin/spacing information
- false by default.
-----------------------------------------------
::
&skiprpctag=<(bool)true>
- Skip the reading of internal RPC tags (see
[sec:TypesofSensorModels] for details)
- false by default.
Writer options
^^^^^^^^^^^^^^
::
&writegeom=<(bool)false>
- To activate writing of external geom file
- true by default
-----------------------------------------------
::
&writerpctags=<(bool)true>
- To activate writing of RPC tags in TIFF files
- false by default
-----------------------------------------------
::
&gdal:co:<GDALKEY>=<VALUE>
- To specify a gdal creation option
- For gdal creation option information, see dedicated gdal documentation for each driver. For example, you can find `here <http://www.gdal.org/frmt_gtiff.html>`_ the information about the GeoTiff create options
- None by default
-----------------------------------------------
::
&streaming:type=<VALUE>
- Activates configuration of streaming through extended filenames
- Override any previous configuration of streaming
- Allows to configure the kind of streaming to perform
- Available values are:
- auto: tiled or stripped streaming mode chosen automatically
depending on TileHint read from input files
- tiled: tiled streaming mode
- stripped: stripped streaming mode
- none: explicitly deactivate streaming
- Not set by default
-----------------------------------------------
::
&streaming:sizemode=<VALUE>
- Allows to choose how the size of the streaming pieces is computed
- Available values are:
- auto: size is estimated from the available memory setting by
evaluating pipeline memory print
- height: size is set by setting height of strips or tiles
- nbsplits: size is computed from a given number of splits
- Default is auto
-----------------------------------------------
::
&streaming:sizevalue=<VALUE>
- Parameter for size of streaming pieces computation
- Value is :
- if sizemode=auto: available memory in Mb
- if sizemode=height: height of the strip or tile in pixels
- if sizemode=nbsplits: number of requested splits for streaming
- If not provided, the default value is set to 0 and result in
different behaviour depending on sizemode (if set to height or
nbsplits, streaming is deactivated, if set to auto, value is
fetched from configuration or cmake configuration file)
-----------------------------------------------
::
&box=<startx>:<starty>:<sizex>:<sizey>
- User defined parameters of output image region
- The region must be set with 4 unsigned integers (the separator
used is the colon ’:’). Values are:
- startx: first index on X (starting with 0)
- starty: first index on Y (starting with 0)
- sizex: size along X
- sizey: size along Y
- The definition of the region follows the same convention as
itk::Region definition in C++. A region is defined by two classes:
the itk::Index and itk::Size classes. The origin of the region
within the image with which it is associated is defined by Index
-----------------------------------------------
::
&bands=r1,r2,...,rn
- Select a subset of bands from the output image
- The syntax is inspired by Python indexing syntax with
bands=r1,r2,r3,...,rn where each ri is a band range that can be :
- a single index (1-based) :
- :code:`2` means 2nd band
- :code:`-1` means last band
- or a range of bands :
- :code:`3:` means 3rd band until the last one
- :code:`:-2` means the first bands until the second to last
- :code:`2:4` means bands 2,3 and 4
- Empty by default (all bands are write from the output image)
The available syntax for boolean options are:
- ON, On, on, true, True, 1 are available for setting a ’true’ boolean
value
- OFF, Off, off, false, False, 0 are available for setting a ’false’
boolean value
Examples
^^^^^^^^^^^^^^
You can find below some examples:
- Write a file with blockSize equal to 256 and with DEFLATE compression
::
$ otbcli_Convert -in OTB-Data/Examples/QB_1_ortho.tif -out "/tmp/example1.tif?&gdal:co:TILED=YES&gdal:co:COMPRESS=DEFLATE"
- Process only first band from a file
::
$ otbcli_Convert -in "OTB-Data/Examples/QB_1_ortho.tif?&bands=1" -out /tmp/example2.tif
......@@ -189,7 +189,7 @@ The first time, you can get the source code using:
::
git clone https://git@git.orfeo-toolbox.org/git/otb.git
git clone https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb.git
Then you can build OTB as usual using this directory as the source
(refer to build instructions). Later if you want to update your source,
......@@ -428,7 +428,7 @@ You can get the base doing:
::
git clone https://git@git.orfeo-toolbox.org/git/otb-data.git
git clone https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb-data.git
This is about 1 GB of data, so it will take a while, but you have to do
it only once, as after, a simple
......
......@@ -54,7 +54,7 @@ Download, Configure and build OTB
::
mkdir -p /opt/OTB/build && cd /opt/OTB
git clone --depth=1 --branch=develop https://git@git.orfeo-toolbox.org/git/otb.git source
git clone --depth=1 --branch=develop https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb.git source
cd build && cmake ../source
make -j2
......
......@@ -321,7 +321,7 @@ Conclusion
~~~~~~~~~~
The images used in this documentation can be found in the OTB-Data
repository (https://git.orfeo-toolbox.org/otb-data.git):
repository (https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb-data.git):
- in OTB-Data/Input:
......
......@@ -360,296 +360,3 @@ among 560 cpus and took only 56 seconds.
Note that this MPI parallel invocation of applications is only
available for command-line calls to OTB applications, and only for
images output parameters.
.. _extended-filenames:
Extended filenames
------------------
There are multiple ways to define geo-referencing information. For
instance, one can use a geographic transform, a cartographic projection,
or a sensor model with RPC coefficients. A single image may contain
several of these elements, such as in the “ortho-ready” products: this
is a type of product still in sensor geometry (the sensor model is
supplied with the image) but it also contains an approximative
geographic transform that can be used to have a quick estimate of the
image localisation. For instance, your product may contain a “.TIF” file
for the image, along with a “.RPB” file that contains the sensor model
coefficients and an “.IMD” file that contains a cartographic projection.
This case leads to the following question: which geo-referencing
element should be used when opening this image in OTB. In
fact, it depends on the users need. For an orthorectification
application, the sensor model must be used. In order to specify which
information should be skipped, a syntax of extended filenames has been
developed for both reading and writing.
The reader and writer extended file name support is based on the same
syntax, only the options are different. To benefit from the extended
file name mechanism, the following syntax is to be used:
::
Path/Image.ext?&key1=<value1>&key2=<value2>
Note that you’ll probably need to “quote” the filename, especially if calling
applications from the bash command line.
Reader options
^^^^^^^^^^^^^^
::
&geom=<path/filename.geom>
- Contains the file name of a valid geom file
- Use the content of the specified geom file instead of
image-embedded geometric information
- empty by default, use the image-embedded information if available
-----------------------------------------------
::
&sdataidx=<(int)idx>
- Select the sub-dataset to read
- 0 by default
-----------------------------------------------
::
&resol=<(int)resolution factor>
- Select the JPEG2000 sub-resolution image to read
- 0 by default
-----------------------------------------------
::
&bands=r1,r2,...,rn
- Select a subset of bands from the input image
- The syntax is inspired by Python indexing syntax with
bands=r1,r2,r3,...,rn where each ri is a band range that can be :
- a single index (1-based) :
- :code:`2` means 2nd band
- :code:`-1` means last band
- or a range of bands :
- :code:`3:` means 3rd band until the last one
- :code:`:-2` means the first bands until the second to last
- :code:`2:4` means bands 2,3 and 4
- empty by default (all bands are read from the input image)
-----------------------------------------------
::
&skipcarto=<(bool)true>
- Skip the cartographic information
- Clears the projectionref, set the origin to :math:`[0,0]` and the
spacing to :math:`[1/max(1,r),1/max(1,r)]` where :math:`r` is the resolution
factor.
- Keeps the keyword list
- false by default
-----------------------------------------------
::
&skipgeom=<(bool)true>
- Skip geometric information
- Clears the keyword list
- Keeps the projectionref and the origin/spacing information
- false by default.
-----------------------------------------------
::
&skiprpctag=<(bool)true>
- Skip the reading of internal RPC tags (see
[sec:TypesofSensorModels] for details)
- false by default.
Writer options
^^^^^^^^^^^^^^
::
&writegeom=<(bool)false>
- To activate writing of external geom file
- true by default
-----------------------------------------------
::
&writerpctags=<(bool)true>
- To activate writing of RPC tags in TIFF files
- false by default
-----------------------------------------------
::
&gdal:co:<GDALKEY>=<VALUE>
- To specify a gdal creation option
- For gdal creation option information, see dedicated gdal documentation
- None by default
-----------------------------------------------
::
&streaming:type=<VALUE>
- Activates configuration of streaming through extended filenames
- Override any previous configuration of streaming
- Allows to configure the kind of streaming to perform