Commit d892dc2d authored by Guillaume Pasero's avatar Guillaume Pasero
Browse files

MRG: synchronize with 'develop'

Conflicts:
	Modules/Wrappers/SWIG/src/otbApplication.i
parents 36815b0c 3d8ddf16
### Description
Describe what happens and why you think it is a bug
### Steps to reproduce
Describe as precisely as possible how to reproduce the bug. Try to isolate a minimal number of steps. Also describe reproducibility (always, random ...).
### Configuration information
OS, OTB version or tag, information related to build (binaries, superbuild, system libs ...)
/label ~bug
### Target documentation ressources
Can be a combination of Software guide, cookbook, doxygen, blog, applications doc, websites
### Change requested
Describe precisely the changes that are required.
Short summary of the requested feature
/label ~feature
### What changes will be made and why they would make a better Orfeo ToolBox?
#### High level description
#### Risks and benefits
#### Alternatives for implementations
### Who will be developing the proposed changes?
### Summary
Gives a short summary of the changes.
### Rationale
Explain the rationale for the changes (possible link to a Request For Comments or to an issue).
### Implementation Details
#### Classes and files
Give an overview of the implementation: main changes made to classes, files and modules. Do not paste complete diff, as it is available in the merge request already.
#### Applications
Describe any changes made to existing applications, or new applications that have been added.
#### Tests
Describe the testing strategy for new features.
### Documentation
List or link documentation modifications that were made (doxygen, example, software guide, application documentation, cookbook).
### Additional notes
List remaining open issues if any, and additional notes.
# How to contribute to Orfeo ToolBox ?
Thank you for taking the time to contribute to OTB! This document will guide you
through the workflow and best practices you need to know to send your
contribution.
There are many ways to contribute to OTB:
* [Reporting a bug](#reporting-bugs)
* [Making a feature request](#feature-requests-and-discussions)
* [Improving documentation](#documentation-improvements)
* [Contributing code (C++, Python, CMake, etc.)](#code-contribution)
* [Publishing a remote module](#remote-modules)
Our main workflow uses GitLab for source control, issues and task tracking. We
use a self-hosted gitlab instance:
[`https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb`](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb)
Remember to check out also our [developers mailing list](https://groups.google.com/forum/?hl=fr#!forum/otb-developers/join),
where we discuss some features, improvements and high level project planning.
You are welcome to ask questions there as a beginner or future OTB contributor!
## Reporting bugs
If you have found a bug, you can first [search the existing issues](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/issues?label_name%5B%5D=bug)
to see if it has already been reported.
If it's a new bug, please [open a new issue on GitLab](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/issues/new).
The 'Bug' issue template will help you provide all important information and
help fixing the bug quicker. Remember to add as much information as possible!
## Feature requests and discussions
Feature requests are welcome! Generally you are welcome to simply [open an issue](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/issues)
and discuss your idea there. For more complex requests there is an issue
template for in depth description called 'Request for Comments'.
## Documentation improvements
The two main OTB documentations are the [Software Guide](https://www.orfeo-toolbox.org/SoftwareGuide/index.html)
and the [CookBook](https://www.orfeo-toolbox.org/CookBook/). Their sources are
hosted in the main OTB repository in the `Documentation/` directory. Then, to
contribute documentation use the same workflow as for code contributions (see
below).
See also the [Compiling documentation](https://wiki.orfeo-toolbox.org/index.php/Compiling_documentation)
wiki page for help on building the Sphinx and Latex source.
## Code contribution
The OTB workflow is based on GitLab [Merge Requests](https://docs.gitlab.com/ee/gitlab-basics/add-merge-request.html).
Clone the repository, create a feature branch, commit your changes, push the
feature branch to a fork (or the main repository if you are a core developer),
then send a merge request.
Note that we also accept PRs on our [GitHub mirror](https://github.com/orfeotoolbox/OTB)
which we will manually merge.
### Commit message
On your feature branch, write a good [commit message](https://xkcd.com/1296/):
short and descriptive. If fixing an issue or bug, put the issue number in the
commit message so that GitLab can [crosslink it](https://docs.gitlab.com/ce/user/project/issues/crosslinking_issues.html).
You can prefix your commit message with an indicating flag (DOC, BUG, PKG,
TEST, SuperBuild, etc.).
Standard prefixes for OTB commit messages:
BUG: Fix for runtime crash or incorrect result
COMP: Compiler error or warning fix
DOC: Documentation change
ENH: New functionality
PERF: Performance improvement
STYLE: No logic impact (indentation, comments)
WIP: Work In Progress not ready for merge
For example, here are some good commit messages:
BUG: #1701 Warn users if parameter string is unset
DOC: Fix typo in Monteverdi French translation
COMP: Allow GeoTIFF and TIFF to be disabled when no 3rd party drags them
### Merge request
Your contribution is ready to be added to the main OTB repository? Send a Merge
Request against the `develop` branch on GitLab using the merge request
template. The merge request will then be discussed by the community and the core
OTB team.
* Merge requests can not be merged until all discussions have been resolved (this is enforced by GitLab)
* Merge requests **must receive at least 2 positives votes from PSC members** before being merged
* The merger is responsible for checking that the branch is up-to-date with develop
* Merge requests can be merged by anyone (not just PSC or RM) with push access to develop
* Merge requests can be merged once the dashboard is proven green for this branch
Branches can be registered for dashboard testing by adding one line in `Config/feature_branches.txt` in [otb-devutils repository](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb-devutils.git).
For branches in the main repository, the syntax is the following:
```
branch_name [otb-data_branch_name]
```
The second branch name is optional. It can be set if you need to modify [otb-data](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb-data.git) according to your changes.
For branches in forks, the syntax is the following:
```
user/branch_name [user/otb-data_branch_name]
```
Again, the second branch name is optional.
For users without push access to [otb-devutils repository](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb-devutils.git), the modification can be asked through a merge requests to this repository.
Once the feature branch is registered for testing, it should appear in the *FeatureBranches* section of the [OTB dashboard](https://dash.orfeo-toolbox.org/index.php?project=OTB) next day (remember tests are run on a nighlty basis).
Do not forget to remove the feature branch for testing once it has been merged.
## Remote modules
[Remote Modules](https://wiki.orfeo-toolbox.org/index.php/Remote_Modules) are
the prefered way if you wish to make your apps and filters available to the
community while keeping control and maintenance of their sources. Remote
modules are just like regular modules, except they are not distributed inside
OTB source code. Under some conditions (dependencies, official acceptance
process, etc.), we are also able to distribute your remote module in the
official standalone binaries. See [the wiki](https://wiki.orfeo-toolbox.org/index.php/Remote_Modules)
for more information.
## Gitlab guidelines
In order to organize the issues in our Gitlab instance, we use both labels and
milestones.
The [milestones](https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/milestones) should be used to track in which release a feature is merged.
Gitlab can then provide a summary of all features and bugs added to a given release
version.
Regarding labels, we use the following set:
* ~story: significant feature to be implemented with a detailed work plan, it can
correspond to a Request for Comments that has turned into a development action
* ~bug: Bug, crash or unexpected behavior, reported by a user or a developer
* ~feature: Feature request expressed by an OTB user/developer
* ~"To Do": action is planned
* ~Doing: work in progress
* ~api ~app ~documentation ~monteverdi ~packaging ~qgis: optional context information
.. _extended-filenames:
Extended filenames
================================
Extended filenames is an interesting feature of OTB. With it, you can control
several aspects of the beahvior of the OTB in the OTB-Applications or in our
own C++ applications. Historically this feature has been desingn to solve
an issue with how to handle geo-referencing information.
Indeed, there are multiple ways to define geo-referencing information. For
instance, one can use a geographic transform, a cartographic projection,
or a sensor model with RPC coefficients. A single image may contain
several of these elements, such as in the “ortho-ready” products: this
is a type of product still in sensor geometry (the sensor model is
supplied with the image) but it also contains an approximative
geographic transform that can be used to have a quick estimate of the
image localisation. For instance, your product may contain a “.TIF” file
for the image, along with a “.RPB” file that contains the sensor model
coefficients and an “.IMD” file that contains a cartographic projection.
This case leads to the following question: which geo-referencing
element should be used when opening this image in OTB. In
fact, it depends on the users need. For an orthorectification
application, the sensor model must be used. In order to specify which
information should be skipped, a syntax of extended filenames has been
developed for both reading and writing.
Since the development of this feature we have extend this mechanism for
other aspaects: like band or overview selection in reader part or support
create option of gdal in writer part.The reader and writer extended filename
support is based on the same syntax, only the options are different.
To benefit from the extended file name mechanism, the following syntax
is to be used:
::
Path/Image.ext?&key1=<value1>&key2=<value2>
**Note that you’ll probably need to “quote” the filename, especially if calling
applications from the bash command line.**
Reader options
^^^^^^^^^^^^^^
::
&geom=<path/filename.geom>
- Contains the file name of a valid geom file
- Use the content of the specified geom file instead of
image-embedded geometric information
- empty by default, use the image-embedded information if available
-----------------------------------------------
::
&sdataidx=<(int)idx>
- Select the sub-dataset to read
- 0 by default
-----------------------------------------------
::
&resol=<(int)resolution factor>
- Select the JPEG2000 sub-resolution image to read
- 0 by default
-----------------------------------------------
::
&bands=r1,r2,...,rn
- Select a subset of bands from the input image
- The syntax is inspired by Python indexing syntax with
bands=r1,r2,r3,...,rn where each ri is a band range that can be :
- a single index (1-based) :
- :code:`2` means 2nd band
- :code:`-1` means last band
- or a range of bands :
- :code:`3:` means 3rd band until the last one
- :code:`:-2` means the first bands until the second to last
- :code:`2:4` means bands 2,3 and 4
- empty by default (all bands are read from the input image)
-----------------------------------------------
::
&skipcarto=<(bool)true>
- Skip the cartographic information
- Clears the projectionref, set the origin to :math:`[0,0]` and the
spacing to :math:`[1/max(1,r),1/max(1,r)]` where :math:`r` is the resolution
factor.
- Keeps the keyword list
- false by default
-----------------------------------------------
::
&skipgeom=<(bool)true>
- Skip geometric information
- Clears the keyword list
- Keeps the projectionref and the origin/spacing information
- false by default.
-----------------------------------------------
::
&skiprpctag=<(bool)true>
- Skip the reading of internal RPC tags (see
[sec:TypesofSensorModels] for details)
- false by default.
Writer options
^^^^^^^^^^^^^^
::
&writegeom=<(bool)false>
- To activate writing of external geom file
- true by default
-----------------------------------------------
::
&writerpctags=<(bool)true>
- To activate writing of RPC tags in TIFF files
- false by default
-----------------------------------------------
::
&gdal:co:<GDALKEY>=<VALUE>
- To specify a gdal creation option
- For gdal creation option information, see dedicated gdal documentation for each driver. For example, you can find `here <http://www.gdal.org/frmt_gtiff.html>`_ the information about the GeoTiff create options
- None by default
-----------------------------------------------
::
&streaming:type=<VALUE>
- Activates configuration of streaming through extended filenames
- Override any previous configuration of streaming
- Allows to configure the kind of streaming to perform
- Available values are:
- auto: tiled or stripped streaming mode chosen automatically
depending on TileHint read from input files
- tiled: tiled streaming mode
- stripped: stripped streaming mode
- none: explicitly deactivate streaming
- Not set by default
-----------------------------------------------
::
&streaming:sizemode=<VALUE>
- Allows to choose how the size of the streaming pieces is computed
- Available values are:
- auto: size is estimated from the available memory setting by
evaluating pipeline memory print
- height: size is set by setting height of strips or tiles
- nbsplits: size is computed from a given number of splits
- Default is auto
-----------------------------------------------
::
&streaming:sizevalue=<VALUE>
- Parameter for size of streaming pieces computation
- Value is :
- if sizemode=auto: available memory in Mb
- if sizemode=height: height of the strip or tile in pixels
- if sizemode=nbsplits: number of requested splits for streaming
- If not provided, the default value is set to 0 and result in
different behaviour depending on sizemode (if set to height or
nbsplits, streaming is deactivated, if set to auto, value is
fetched from configuration or cmake configuration file)
-----------------------------------------------
::
&box=<startx>:<starty>:<sizex>:<sizey>
- User defined parameters of output image region
- The region must be set with 4 unsigned integers (the separator
used is the colon ’:’). Values are:
- startx: first index on X (starting with 0)
- starty: first index on Y (starting with 0)
- sizex: size along X
- sizey: size along Y
- The definition of the region follows the same convention as
itk::Region definition in C++. A region is defined by two classes:
the itk::Index and itk::Size classes. The origin of the region
within the image with which it is associated is defined by Index
-----------------------------------------------
::
&bands=r1,r2,...,rn
- Select a subset of bands from the output image
- The syntax is inspired by Python indexing syntax with
bands=r1,r2,r3,...,rn where each ri is a band range that can be :
- a single index (1-based) :
- :code:`2` means 2nd band
- :code:`-1` means last band
- or a range of bands :
- :code:`3:` means 3rd band until the last one
- :code:`:-2` means the first bands until the second to last
- :code:`2:4` means bands 2,3 and 4
- Empty by default (all bands are write from the output image)
The available syntax for boolean options are:
- ON, On, on, true, True, 1 are available for setting a ’true’ boolean
value
- OFF, Off, off, false, False, 0 are available for setting a ’false’
boolean value
Examples
^^^^^^^^^^^^^^
You can find below some examples:
- Write a file with blockSize equal to 256 and with DEFLATE compression
::
$ otbcli_Convert -in OTB-Data/Examples/QB_1_ortho.tif -out "/tmp/example1.tif?&gdal:co:TILED=YES&gdal:co:COMPRESS=DEFLATE"
- Process only first band from a file
::
$ otbcli_Convert -in "OTB-Data/Examples/QB_1_ortho.tif?&bands=1" -out /tmp/example2.tif
......@@ -189,7 +189,7 @@ The first time, you can get the source code using:
::
git clone https://git@git.orfeo-toolbox.org/git/otb.git
git clone https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb.git
Then you can build OTB as usual using this directory as the source
(refer to build instructions). Later if you want to update your source,
......@@ -243,22 +243,21 @@ OFF the CMake variable OTB\_USE\_EXTERNAL\_ITK.
OTB compilation and Windows platform
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To build OTB on Windows, we highly recommend using OSGeo4W which
provides all the necessary dependencies.
To build OTB on Windows, you should prepare an environment with the following
tools:
Currently it is not possible to build OTB in Debug when using the
dependencies provided by OSGeo4W. If you want to build OTB in Debug for
Windows, you will need to build and install manually each dependency
needed by OTB. You should use the same compiler for all the
dependencies, as much as possible.
* Visual Studio 2015 or later
* CMake 3.1 or later
* OTB XDK : download a Windows binary package of OTB and use the supplied
uninstall script to remove OTB binaries and headers. Now, this package only
contains the dependencies needed to build OTB.
Therefore, we highly recommend you to use OSGeo4W shell environment to
build OTB. You can use the 32 or 64 bit installer, since OSGeo4W
provides all the necessary dependencies in the two cases. Please follow
carefully the procedure provided in the Software Guide.
Then, you can download OTB sources (preferably, a version compatible with your
XDK), and compile them as a standard CMake project. More details are available
in the SoftwareGuide.
Typically, when using the dependencies provided by OSGeo4W, compile OTB
in Release or RelWithDebInfo mode.
There is an other solution, using OSGeo4W distribution. However, the
dependencies may be outdated.
Using OTB
---------
......@@ -429,7 +428,7 @@ You can get the base doing:
::
git clone https://git@git.orfeo-toolbox.org/git/otb-data.git
git clone https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb-data.git
This is about 1 GB of data, so it will take a while, but you have to do
it only once, as after, a simple
......
......@@ -39,8 +39,8 @@ archive is extracted, the directory structure consists of:
- ``share``: A folder containing common resources and copyright
mentions.
- ``tool``: A folder containing usefull scripts to test the installation or
to uninstall OTB libraries and headers while keeping all the depedencies.
- ``tool``: A folder containing useful scripts to test the installation or
to uninstall OTB libraries and headers while keeping all the dependencies.
The applications can be launched from the Mapla launcher. If you want to
use the otbcli and otbgui launchers, you can initialize your environment
......
......@@ -54,7 +54,7 @@ Download, Configure and build OTB
::
mkdir -p /opt/OTB/build && cd /opt/OTB
git clone --depth=1 --branch=develop https://git@git.orfeo-toolbox.org/git/otb.git source
git clone --depth=1 --branch=develop https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb.git source
cd build && cmake ../source
make -j2
......
......@@ -28,8 +28,8 @@ Contents of OTB-|release|-Darwin64 is briefly listed below:
- ``share``: A folder containing common resources and copyright
mentions.
- ``tool``: A folder containing usefull scripts to test the installation or
to uninstall OTB libraries and headers while keeping all the depedencies.
- ``tool``: A folder containing useful scripts to test the installation or
to uninstall OTB libraries and headers while keeping all the dependencies.
The applications can be launched from the Mapla launcher. If you want to
use the otbcli and otbgui launchers, you can initialize your environment
......@@ -90,6 +90,7 @@ A: You can get this error at startup running Monteverdi.app or Mapla.app. The
solution is to run in a terminal the following command:
::
xcode-select --install
And then try to restart Monteverdi or Mapla.
......@@ -21,8 +21,8 @@ and their launchers (both command line and graphical launchers are provided):
- ``include``: A folder containing all the necessary headers to compile OTB