openquake.server.db package

openquake.server.db.actions module

exception openquake.server.db.actions.InvalidCalculationID

Bases: exceptions.Exception

openquake.server.db.actions.calc_info(calc_id)

Parameters:calc_id – calculation ID Returns:dictionary of info about the given calculation

openquake.server.db.actions.check_hazard_risk_consistency(haz_job, risk_mode)

Make sure that the provided hazard job is the right one for the current risk calculator.

Parameters:

• job – an OqJob instance referring to the previous hazard calculation
• risk_mode – the calculation_mode string of the current risk calculation

openquake.server.db.actions.check_outdated()

Check if the db is outdated, called before starting anything

openquake.server.db.actions.create_job(calc_mode, description, user_name, datadir, hc_id=None)

Create job for the given user, return it.

Parameters:

• calc_mode (str) – Calculation mode, such as classical, event_based, etc
• user_name – User who owns/started this job.
• datadir – Data directory of the user who owns/started this job.
• description – Description of the calculation
• hc_id – If not None, then the created job is a risk job

Returns:

openquake.server.db.models.OqJob instance.

openquake.server.db.actions.create_outputs(job_id, dskeys)

Build a correspondence between the outputs in the datastore and the ones in the database.

Parameters:

• job_id – ID of the current job
• dskeys – a list of datastore keys

openquake.server.db.actions.del_calc(job_id, user)

Delete a calculation and all associated outputs.

Parameters:job_id – ID of a OqJob.

openquake.server.db.actions.delete_uncompleted_calculations(user)

Delete the uncompleted calculations of the given user

openquake.server.db.actions.fetch(templ, *args)

Run queries directly on the database. Return header + rows

openquake.server.db.actions.finish(job_id, status)

Set the job columns is_running, status, and stop_time

openquake.server.db.actions.get_calc_id(datadir, job_id=None)

Return the latest calc_id by looking both at the datastore and the database.

openquake.server.db.actions.get_calcs(request_get_dict, user_name, user_acl_on=False, id=None)

Returns:

list of tuples (job_id, user_name, job_status, job_type,

job_is_running, job_description)

openquake.server.db.actions.get_dbpath()

Returns the path to the database file

openquake.server.db.actions.get_job_id(job_id, username)

If job_id is negative, return the last calculation of the current user, otherwise returns the job_id unchanged.

openquake.server.db.actions.get_log(job_id)

Extract the logs as a big string

openquake.server.db.actions.get_log_size(calc_id)

Get a slice of the calculation log as a JSON list of rows

openquake.server.db.actions.get_log_slice(calc_id, start, stop)

Get a slice of the calculation log as a JSON list of rows

openquake.server.db.actions.get_output(output_id)

Parameters:output_id – ID of an Output object Returns:(ds_key, calc_id, dirname)

openquake.server.db.actions.get_outputs(job_id)

Parameters:job_id – ID of a calculation. Returns:A sequence of openquake.server.db.models.Output objects

openquake.server.db.actions.get_result(result_id)

Returns:(job_id, job_status, datadir, datastore_key)

openquake.server.db.actions.get_results(job_id)

Returns:(datadir, datastore_keys)

openquake.server.db.actions.get_traceback(calc_id)

Return the traceback of the given calculation as a list of lines. The list is empty if the calculation was successful.

openquake.server.db.actions.list_calculations(job_type, user_name)

Yield a summary of past calculations.

Parameters:job_type – ‘hazard’ or ‘risk’

openquake.server.db.actions.list_outputs(job_id, full=True)

List the outputs for a given OqJob.

Parameters:

• job_id – ID of a calculation.
• full (bool) – If True produce a full listing, otherwise a short version

openquake.server.db.actions.log(job_id, timestamp, level, process, message)

Write a log record in the database

openquake.server.db.actions.log_to_json(log)

Convert a log record into a list of strings

openquake.server.db.actions.print_outputs_summary(outputs, full=True)

List of openquake.server.db.models.Output objects.

openquake.server.db.actions.reset_is_running()

Reset the flag job.is_running to False. This is called when the Web UI is re-started: the idea is that it is restarted only when all computations are completed.

openquake.server.db.actions.save_performance(job_id, records)

Save in the database the performance information about the given job

openquake.server.db.actions.set_relevant(calc_id, flag)

Set the relevant field of the given calculation record

openquake.server.db.actions.upgrade_db()

openquake.server.db.actions.version_db()

openquake.server.db.actions.what_if_I_upgrade(extract_scripts)

openquake.server.db.models module

Model representations of the OpenQuake DB tables.

openquake.server.db.models.DEFAULT_SRID = 4326

System Reference ID used for geometry objects

openquake.server.db.models.LOSS_TYPES = ['structural', 'nonstructural', 'fatalities', 'contents']

Kind of supported type of loss outputs

class openquake.server.db.models.Log(*args, **kwargs)

Bases: django.db.models.base.Model

Log table for calculations

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

Log.LOG_LEVEL_CHOICES = ((u'debug', u'Debug'), (u'info', u'Info'), (u'warn', u'Warn'), (u'error', u'Error'), (u'critical', u'Critical'))

exception Log.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Log.get_level_display(*moreargs, **morekwargs)

Log.get_next_by_timestamp(*moreargs, **morekwargs)

Log.get_previous_by_timestamp(*moreargs, **morekwargs)

Log.job

Log.objects = <django.db.models.manager.Manager object>

openquake.server.db.models.MAX_SINT_32 = 2147483647

Maximum value for a seed number

openquake.server.db.models.MIN_SINT_32 = -2147483648

Minimum value for a seed number

exception openquake.server.db.models.NotFound

Bases: exceptions.Exception

Raised when a Django object does not exist

class openquake.server.db.models.OqJob(*args, **kwargs)

Bases: django.db.models.base.Model

An OpenQuake engine run started by the user

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception OqJob.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

OqJob.STATUS_CHOICES = ((u'created', u'Created'), (u'executing', u'Executing'), (u'post_executing', u'Post-Executing'), (u'post_processing', u'Post-Processing'), (u'export', u'Exporting results'), (u'clean_up', u'Cleaning up'), (u'complete', u'Complete'))

OqJob.get_next_by_start_time(*moreargs, **morekwargs)

OqJob.get_next_by_stop_time(*moreargs, **morekwargs)

OqJob.get_oqparam()

Return an OqParam object as read from the database

OqJob.get_previous_by_start_time(*moreargs, **morekwargs)

OqJob.get_previous_by_stop_time(*moreargs, **morekwargs)

OqJob.get_status_display(*moreargs, **morekwargs)

OqJob.hazard_calculation

OqJob.job_type

‘hazard’ or ‘risk’

OqJob.log_set

OqJob.objects = <django.db.models.manager.Manager object>

OqJob.oqjob_set

OqJob.output_set

OqJob.performance_set

class openquake.server.db.models.Output(*args, **kwargs)

Bases: django.db.models.base.Model

A single artifact which is a result of an OpenQuake job. The data may reside in a file or in the database.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception Output.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Output.get_next_by_last_update(*moreargs, **morekwargs)

Output.get_previous_by_last_update(*moreargs, **morekwargs)

Output.objects = <openquake.server.db.models.OutputManager object>

Output.oq_job

class openquake.server.db.models.OutputManager

Bases: django.db.models.manager.Manager

Manager class to filter and create Output objects

create_output(job, display_name, ds_key)

Create an output for the given job, display_name and ds_key

class openquake.server.db.models.Performance(*args, **kwargs)

Bases: django.db.models.base.Model

Contains performance information about a job.

exception DoesNotExist

Bases: django.core.exceptions.ObjectDoesNotExist

exception Performance.MultipleObjectsReturned

Bases: django.core.exceptions.MultipleObjectsReturned

Performance.job

Performance.objects = <django.db.models.manager.Manager object>

openquake.server.db.models.RISK_ATOL = 0.01

absolute tolerance to consider two risk outputs (almost) equal

openquake.server.db.models.RISK_RTOL = 0.05

relative tolerance to consider two risk outputs (almost) equal

openquake.server.db.models.STAT_CHOICES = ((u'mean', u'Mean'), (u'quantile', u'Quantile'))

Kind of supported curve statistics

openquake.server.db.models.get(djm, **kw)

Small wrapper around djm.objects.get

openquake.server.db.models.oqparam(job_id)

Parameters:job_id – ID of openquake.server.db.models.OqJob Returns:instance of openquake.commonlib.oqvalidation.OqParam

openquake.server.db.upgrade_manager module

exception openquake.server.db.upgrade_manager.DuplicatedVersion

Bases: exceptions.RuntimeError

class openquake.server.db.upgrade_manager.UpgradeManager(upgrade_dir, version_table='revision_info', version_pattern='\d\d\d\d', flag_pattern='(-slow|-danger)?')

Bases: object

The package containing the upgrade scripts should contain an instance of the UpgradeManager called upgrader in the __init__.py file. It should also specify the initializations parameters

Parameters:

• upgrade_dir – the directory were the upgrade script reside
• version_table – the name of the versioning table (default revision_info)
• version_pattern – a regulation expression for the script version number (dddd)

ENGINE_URL = 'https://github.com/gem/oq-engine/tree/master/'

UPGRADES = 'openquake/server/db/schema/upgrades/'

check_versions(conn)

Parameters:conn – a DB API 2 connection Returns:a message with the versions that will be applied or None

extract_upgrade_scripts()

Extract the OpenQuake upgrade scripts from the links in the GitHub page

get_db_versions(conn)

Get all the versions stored in the database as a set.

Parameters:conn – a DB API 2 connection

init(conn)

Create the version table and run the base script on an empty database.

Parameters:conn – a DB API 2 connection

install_versioning(conn)

Create the version table into an already populated database and insert the base script.

Parameters:conn – a DB API 2 connection

classmethod instance(conn, pkg_name='openquake.server.db.schema.upgrades')

Return an UpgradeManager instance.

Parameters:

• conn – a DB API 2 connection
• pkg_name (str) – the name of the package with the upgrade scripts

parse_script_name(script_name)

Parse a script name and return a dictionary with fields fname, name, version and ext (or None if the name does not match).

Parameters:name – name of the script

read_scripts(minversion=None, maxversion=None, skip_versions=())

Extract the upgrade scripts from a directory as a list of dictionaries, ordered by version.

Parameters:

• minversion – the minimum version to consider
• maxversion – the maximum version to consider
• skipversions – the versions to skip

upgrade(conn, skip_versions=())

Upgrade the database from the current version to the maximum version in the upgrade scripts.

Parameters:

• conn – a DBAPI 2 connection
• skip_versions – the versions to skip

exception openquake.server.db.upgrade_manager.VersionTooSmall

Bases: exceptions.RuntimeError

exception openquake.server.db.upgrade_manager.VersioningNotInstalled

Bases: exceptions.RuntimeError

class openquake.server.db.upgrade_manager.WrappedConnection(conn, debug=False)

Bases: object

This is an utility class that wraps a DB API-2 connection providing a couple of convenient features.

1. it is possible to set a debug flag to print on stdout the executed queries;
2. there is a .run method to run a query with a dedicated cursor; it returns the cursor, which can be iterated over

Parameters:conn – a DB API2-compatible connection

run(templ, *args)

A simple utility to run SQL queries.

Parameters:

• templ – a query or query template
• args – the arguments (or the empty tuple)

Returns:

the DB API 2 cursor used to run the query

openquake.server.db.upgrade_manager.apply_sql_script(conn, fname)

Apply the given SQL script to the database

Parameters:

• conn – a DB API 2 connection
• fname – full path to the creation script

openquake.server.db.upgrade_manager.check_script(upgrade, conn, dry_run=True, debug=True)

An utility to debug upgrade scripts written in Python

Parameters:

• upgrade – upgrade procedure
• conn – a DB API 2 connection
• dry_run – if True, do not change the database
• debug – if True, print the queries which are executed

openquake.server.db.upgrade_manager.upgrade_db(conn, pkg_name='openquake.server.db.schema.upgrades', skip_versions=())

Upgrade a database by running several scripts in a single transaction.

Parameters:

• conn – a DB API 2 connection
• pkg_name (str) – the name of the package with the upgrade scripts
• skip_versions (list) – the versions to skip

Returns:

the version numbers of the new scripts applied the database

openquake.server.db.upgrade_manager.version_db(conn, pkg_name='openquake.server.db.schema.upgrades')

Parameters:

• conn – a DB API 2 connection
• pkg_name (str) – the name of the package with the upgrade scripts

Returns:

the current version of the database

openquake.server.db.upgrade_manager.what_if_I_upgrade(conn, pkg_name='openquake.server.db.schema.upgrades', extract_scripts='extract_upgrade_scripts')

Parameters:

• conn – a DB API 2 connection
• pkg_name (str) – the name of the package with the upgrade scripts
• extract_scripts – name of the method to extract the scripts

Module contents

This is needed for imports to work.