Release notes v2.3
==================

This release introduces several new features and improvements of the
engine calculators. Moreover, our packaging strategy has been
revolutionized and now we provide packages for most operating
systems - including previously
abandoned systems such as Ubuntu 12.04 - by using the exact same
versions of the libraries on every supported platform (Linux, Windows, Mac).

More than 20 pull requests were closed in oq-hazardlib and more than
70 pull requests were closed in oq-engine. For the complete list of
changes, please see the changelogs:
https://github.com/gem/oq-hazardlib/blob/engine-2.3/debian/changelog
and https://github.com/gem/oq-engine/blob/engine-2.3/debian/changelog.

A summary follows.

New features in the engine
--------------------------

From this release it is possible to take into account the *topography
of the region of interest*. It is enough to write the elevation of the
hazard sites in the sites file or in the site model file, as a
negative depth, and the engine will compute the ground shaking
correctly. Before there was no way to specify the elevation of the hazard sites
and the ground shaking was computed only at the sea level.

From this release it is also possible to use *magnitude-dependent
integration distances*. This is expected to have a substantial impact on the
calculation time, since small magnitude ruptures will affects a lot
less sites than before. In order to use the feature the user has to
specify in the `job.ini` file a list of pairs (magnitude, distance) for each
tectonic region type. In the past the user would write

`maximum_distance = {'Active Shallow Crust': 200}`

and the engine would use an integration distance of 200 km for all
ruptures of the given tectonic region type, disregarding the magnitude.
Now, the user can write

`maximum_distance = {'Active Shallow Crust': [(8, 200), (7, 100), (5, 20)]}`

and the engine will use an integration distance of 200 km for ruptures
of magnitude 8, of 100 km for ruptures of magnitude 7, and of 20 km
for ruptures of magnitude 5. Intermediate magnitudes will use a
linearly interpolated distance. It is up to the user to provide a
sound magnitude-distance function. For the future, we plan to release
tools to help the users in the task of inferring magnitude-distance functions.
If in doubt, you can always use the old syntax, equivalent to a
constant maximum distance for all magnitudes, and the engine will keep
working as in the past.

A lot of work went into the UCERF calculators, but they are still officially
marked as experimental and are left undocumented on purpose. There is a
new time-dependent classical calculator, while the old calculators were
substantially improved.

There is an experimental command `run_tiles` to split a `sites.csv`
file in tiles and run multiple calculations at once. This is meant for
power users only.

There is a new command `oq reset` that will remove all calculations of
the current user from the database and the filesystem. It is meant for
cleaning up a testing machine were there are no important calculations.
Use it with care.

New features in hazardlib
--------------------------

New GMPEs - site adjusted Pezeshk et al. 2011, plus a few modified GMPEs
for Armenia - were added.

The new `GriddedSurface` object introduced in engine 2.2 can now be
serialized in the NRML 0.5 format properly.

The calculator `calc_hazard_curves` now supports mutually exclusive
sources and ruptures too.

Performance improvements
------------------------

The calculation of ground motion fields is a lot faster than before in
situations with a large amount of hazard sites. The reason is that now
the distances are computed in the workers, whereas before they were
computed in the controller node. This has a significant impact on
users with a cluster.

Changes
------------

There are two new limits in the event based calculators:

1. it is impossible to run calculations with more than 65536 tasks
2. it is impossible to have tasks generating more than 65536 events each

These limits have been added to prevent users from running impossibly
large calculations: an error message will be displayed early rather
than causing machines to crash.

The event loss table exporter is now faster than in
release 2.2. It does not export the tectonic region type string
anymore, however you can infer it from the `event_tag` field which
contains the source group ID. The association between the source group
ID and the tectonic region type is contained in the new export
`sourcegroups.csv`.  The `source_id` has been stripped from the `event
tag` since it was taking a lot of memory in the case of extra-large
calculations (UCERF).

The calculation of quantiles hazard curves used different algorithms
in the case of logic tree full enumeration and logic tree sampling.
For the sake of consistency, we now use always the same algorithm, the
one of full enumeration. This change may produce small differences in
the quantile curves if you were doing computations with sampling of
the logic tree.

The separator for the exported file `ruptures.csv` has changed from comma
to tab, since this file now contains MULTIPOLYGON geometries containing commas.

We do not compute the mean anymore if there is a single realization, even
if the user has set `mean_hazard_curves=true`.

We changed the DbServer port from port 1999 to port 1908. The
reason is that some anti-viruses were blocking port 1999.

Fixes to the Web UI
-------------------

The "Remove" button now actually removes the calculation. Before
it was only hiding it from the Web UI. Be careful with it, since it is
final. If you remove a calculation, the only way to restore it is
to repeat the computation.

We fixed the names of the files downloaded from the Web UI.  The names
of files downloaded from the Web UI previously contained slash ('/')
characters that were then replaced with dashes or underscores by the
browser. We no longer include slash characters in file names.

The named of the zipped outputs contained a spurious ".". This has been
fixed.

Other
------

We removed a warning about missing the `rtree` library that was too noisy.

A bug in the logic tree storage was fixed. Now it possible to run a
computation on a machine, copy the datastore on a different machine
and to export the results from there.

The was a bug in scenario_risk calculations in the case of exposures
featuring multiple assets of the same taxonomy on the same point
without insured losses. This is now fixed.

There was a bug when combining hazard curves happening in rare situations,
when a source group was generating probabilities of exceedence all zeros.
This has been fixed.

Configuration files with empty `conditional_loss_poes` and non-empty
`loss_ratios` were giving an ugly error message at export
time. Now the non-existing loss maps are simply not exported.

The parallelization library is now more robust against unexpected
exceptions.

Calculations in which a single task is generated are now executed in core,
i.e. no subprocess is spawned.

As usual the internal representation of several data structures has
changed in the datastore. In particular, we removed several usages of
variable-length strings, since they were buggy in the case of large
arrays. As always, users should not rely on the stability of the
internal datastore formats.

UCERF calculations (and more generally any calculations requiring a
shared directory) now work even on clusters without a shared
directory: they will only use the controller node, but at least they
will not fail.

The header of the exported losses for scenario calculations with
occupants was the ugly string `occupants:float32`: now it is just
`occupants`.

The command `oq engine --make-html-report` now works with Python 3 too.

The commands `oq info` and `oq info --report` have been fixed and improved; in
particular, now the report contains the correct number of effective ruptures
(i.e. ruptures within the integration distance) even for classical
calculations.

The `openquake.cfg` file is included in the installation, so users that
installed the engine via `pip install openquake.engine` will not get
any complain about the missing configuration file.

The repository with the latex manual has been merged into the oq-engine
repository, to help reducing disalignement between the code and the
documentation.

We fixed an issue with rabbitmq on Ubuntu 16.04 by updating our configuration
file `openquake.cfg` to not use the `guest` account.