openquake.server.db package

openquake.server.db.actions module

class openquake.server.db.actions.List

Bases: list

openquake.server.db.actions.calc_info(db, calc_id)

Parameters:

• db – a openquake.server.dbapi.Db instance
• calc_id – calculation ID

Returns:

dictionary of info about the given calculation

openquake.server.db.actions.check_outdated(db)

Check if the db is outdated, called before starting anything

Parameters:db – a openquake.server.dbapi.Db instance

openquake.server.db.actions.create_job(db, calc_mode, description, user_name, datadir, hc_id=None)

Create job for the given user, return it.

Parameters:

• db – a openquake.server.dbapi.Db instance
• calc_mode (str) – Calculation mode, such as classical, event_based, etc
• user_name – User who owns/started this job.
• datadir – Data directory of the user who owns/started this job.
• description – Description of the calculation
• hc_id – If not None, then the created job is a risk job

Returns:

openquake.server.db.models.OqJob instance.

openquake.server.db.actions.create_outputs(db, job_id, keysize, ds_size)

Build a correspondence between the outputs in the datastore and the ones in the database. Also, update the datastore size in the job table.

Parameters:

• db – a openquake.server.dbapi.Db instance
• job_id – ID of the current job
• keysize – a list of pairs (key, size_mb)
• ds_size – total datastore size in MB

openquake.server.db.actions.db_version(db)

Parameters:db – a openquake.server.dbapi.Db instance

openquake.server.db.actions.del_calc(db, job_id, user)

Delete a calculation and all associated outputs, if possible.

Parameters:

• db – a openquake.server.dbapi.Db instance
• job_id – job ID, can be an integer or a string
• user – username

Returns:

None if everything went fine or an error message

openquake.server.db.actions.delete_uncompleted_calculations(db, user)

Delete the uncompleted calculations of the given user.

Parameters:

• db – a openquake.server.dbapi.Db instance
• user – user name

openquake.server.db.actions.fetch(db, templ, *args)

Run generic queries directly on the database. See the documentation of the dbapi module.

Parameters:

• db – a openquake.server.dbapi.Db instance
• templ – a SQL query template
• args – arguments to pass to the template

openquake.server.db.actions.find(db, description)

Parameters:

• db – a openquake.server.dbapi.Db instance
• description – job description, used in a case-insensitive LIKE clause

openquake.server.db.actions.finish(db, job_id, status)

Set the job columns is_running, status, and stop_time.

Parameters:

• db – a openquake.server.dbapi.Db instance
• job_id – ID of the current job
• status – a string such as ‘successful’ or ‘failed’

openquake.server.db.actions.get_calc_id(db, datadir, job_id=None)

Return the latest calc_id by looking both at the datastore and the database.

Parameters:

• db – a openquake.server.dbapi.Db instance
• datadir – the directory containing the datastores
• job_id – a job ID; if None, returns the latest job ID

openquake.server.db.actions.get_calcs(db, request_get_dict, allowed_users, user_acl_on=False, id=None)

Parameters:

• db – a openquake.server.dbapi.Db instance
• request_get_dict – a dictionary
• allowed_users – a list of users
• user_acl_on – if True, returns only the calculations owned by the user or the group
• id – if given, extract only the specified calculation

Returns:

list of tuples (job_id, user_name, job_status, calculation_mode,

job_is_running, job_description)

openquake.server.db.actions.get_dbpath(db)

Parameters:db – a openquake.server.dbapi.Db instance Returns:the path to the database file.

openquake.server.db.actions.get_executing_jobs(db)

Parameters:db – a openquake.server.dbapi.Db instance Returns:(id, pid, user_name, start_time) tuples

openquake.server.db.actions.get_job(db, job_id, username=None)

If job_id is negative, return the last calculation of the current user, otherwise returns the job_id unchanged.

Parameters:

• db – a openquake.server.dbapi.Db instance
• job_id – a job ID (can be negative and can be nonexisting)
• username – an user name (if None, ignore it)

Returns:

a valid job or None if the original job ID was invalid

openquake.server.db.actions.get_log(db, job_id)

Extract the logs as a big string

Parameters:

• db – a openquake.server.dbapi.Db instance
• job_id – a job ID

openquake.server.db.actions.get_log_size(db, job_id)

Get a slice of the calculation log as a JSON list of rows.

Parameters:

• db – a openquake.server.dbapi.Db instance
• job_id – a job ID

openquake.server.db.actions.get_log_slice(db, job_id, start, stop)

Get a slice of the calculation log as a JSON list of rows

Parameters:

• db – a openquake.server.dbapi.Db instance
• job_id – a job ID
• start – start of the slice
• stop – end of the slice (the last element is excluded)

openquake.server.db.actions.get_longest_jobs(db)

Parameters:db – a openquake.server.dbapi.Db instance Returns:(id, user_name, days) tuples

openquake.server.db.actions.get_output(db, output_id)

Parameters:

• db – a openquake.server.dbapi.Db instance
• output_id – ID of an Output object

Returns:

(ds_key, calc_id, dirname)

openquake.server.db.actions.get_outputs(db, job_id)

Parameters:

• db – a openquake.server.dbapi.Db instance
• job_id – ID of a calculation.

Returns:

A sequence of openquake.server.db.models.Output objects

openquake.server.db.actions.get_path(db)

Parameters:db – a openquake.server.dbapi.Db instance Returns:the full path to the dbserver codebase

openquake.server.db.actions.get_result(db, result_id)

Parameters:

• db – a openquake.server.dbapi.Db instance
• result_id – a result ID

Returns:

(job_id, job_status, datadir, datastore_key)

openquake.server.db.actions.get_results(db, job_id)

Parameters:

• db – a openquake.server.dbapi.Db instance
• job_id – a job ID

Returns:

(datadir, datastore_keys)

openquake.server.db.actions.get_traceback(db, job_id)

Return the traceback of the given calculation as a list of lines. The list is empty if the calculation was successful.

Parameters:

• db – a openquake.server.dbapi.Db instance
• job_id – a job ID

openquake.server.db.actions.import_job(db, calc_id, calc_mode, description, user_name, status, hc_id, datadir)

Insert a calculation inside the database, if calc_id is not taken

openquake.server.db.actions.list_calculations(db, job_type, user_name)

Yield a summary of past calculations.

Parameters:

• db – a openquake.server.dbapi.Db instance
• job_type – ‘hazard’ or ‘risk’
• user_name – an user name

openquake.server.db.actions.list_outputs(db, job_id, full=True)

List the outputs for a given OqJob.

Parameters:

• db – a openquake.server.dbapi.Db instance
• job_id – ID of a calculation.
• full (bool) – If True produce a full listing, otherwise a short version

openquake.server.db.actions.log(db, job_id, timestamp, level, process, message)

Write a log record in the database.

Parameters:

• db – a openquake.server.dbapi.Db instance
• job_id – a job ID
• timestamp – timestamp to store in the log record
• level – logging level to store in the log record
• process – process ID to store in the log record
• message – message to store in the log record

openquake.server.db.actions.reset_is_running(db)

Reset the flag job.is_running to False. This is called when the Web UI is re-started: the idea is that it is restarted only when all computations are completed.

Parameters:db – a openquake.server.dbapi.Db instance

openquake.server.db.actions.save_performance(db, job_id, records)

Save in the database the performance information about the given job.

Parameters:

• db – a openquake.server.dbapi.Db instance
• job_id – a job ID
• records – a list of performance records

openquake.server.db.actions.set_status(db, job_id, status)

Set the status ‘created’, ‘executing’, ‘complete’, ‘failed’, ‘aborted’ consistently with is_running.

Parameters:

• db – a openquake.server.dbapi.Db instance
• job_id – ID of the current job
• status – status string

openquake.server.db.actions.update_job(db, job_id, dic)

Update the given calculation record.

Parameters:

• db – a openquake.server.dbapi.Db instance
• job_id – a job ID
• dic – a dictionary of valid field/values for the job table

openquake.server.db.actions.update_parent_child(db, parent_child)

Set hazard_calculation_id (parent) on a job_id (child)

openquake.server.db.actions.upgrade_db(db)

Parameters:db – a openquake.server.dbapi.Db instance

openquake.server.db.actions.what_if_I_upgrade(db, extract_scripts)

Parameters:

• db – a openquake.server.dbapi.Db instance
• extract_scripts – scripts to extract

openquake.server.db.upgrade_manager module

exception openquake.server.db.upgrade_manager.DuplicatedVersion

Bases: RuntimeError

class openquake.server.db.upgrade_manager.UpgradeManager(upgrade_dir, version_table='revision_info', version_pattern='\d\d\d\d', flag_pattern='(-slow|-danger)?')

Bases: object

The package containing the upgrade scripts should contain an instance of the UpgradeManager called upgrader in the __init__.py file. It should also specify the initializations parameters

Parameters:

• upgrade_dir – the directory were the upgrade script reside
• version_table – the name of the versioning table (default revision_info)
• version_pattern – a regulation expression for the script version number (dddd)

ENGINE_URL = 'https://github.com/gem/oq-engine/tree/master/'

UPGRADES = 'openquake/server/db/schema/upgrades/'

check_versions(conn)

Parameters:conn – a DB API 2 connection Returns:a message with the versions that will be applied or None

extract_upgrade_scripts()

Extract the OpenQuake upgrade scripts from the links in the GitHub page

get_db_versions(conn)

Get all the versions stored in the database as a set.

Parameters:conn – a DB API 2 connection

init(conn)

Create the version table and run the base script on an empty database.

Parameters:conn – a DB API 2 connection

install_versioning(conn)

Create the version table into an already populated database and insert the base script.

Parameters:conn – a DB API 2 connection

classmethod instance(conn, pkg_name='openquake.server.db.schema.upgrades')

Return an UpgradeManager instance.

Parameters:

• conn – a DB API 2 connection
• pkg_name (str) – the name of the package with the upgrade scripts

parse_script_name(script_name)

Parse a script name and return a dictionary with fields fname, name, version and ext (or None if the name does not match).

Parameters:name – name of the script

read_scripts(minversion=None, maxversion=None, skip_versions=())

Extract the upgrade scripts from a directory as a list of dictionaries, ordered by version.

Parameters:

• minversion – the minimum version to consider
• maxversion – the maximum version to consider
• skipversions – the versions to skip

upgrade(conn, skip_versions=())

Upgrade the database from the current version to the maximum version in the upgrade scripts.

Parameters:

• conn – a DBAPI 2 connection
• skip_versions – the versions to skip

exception openquake.server.db.upgrade_manager.VersionTooSmall

Bases: RuntimeError

exception openquake.server.db.upgrade_manager.VersioningNotInstalled

Bases: RuntimeError

class openquake.server.db.upgrade_manager.WrappedConnection(conn, debug=False)

Bases: object

This is an utility class that wraps a DB API-2 connection providing a couple of convenient features.

1. it is possible to set a debug flag to print on stdout the executed queries;
2. there is a .run method to run a query with a dedicated cursor; it returns the cursor, which can be iterated over

Parameters:conn – a DB API2-compatible connection

run(templ, *args)

A simple utility to run SQL queries.

Parameters:

• templ – a query or query template
• args – the arguments (or the empty tuple)

Returns:

the DB API 2 cursor used to run the query

openquake.server.db.upgrade_manager.apply_sql_script(conn, fname)

Apply the given SQL script to the database

Parameters:

• conn – a DB API 2 connection
• fname – full path to the creation script

openquake.server.db.upgrade_manager.check_script(upgrade, conn, dry_run=True, debug=True)

An utility to debug upgrade scripts written in Python

Parameters:

• upgrade – upgrade procedure
• conn – a DB API 2 connection
• dry_run – if True, do not change the database
• debug – if True, print the queries which are executed

openquake.server.db.upgrade_manager.db_version(conn, pkg_name='openquake.server.db.schema.upgrades')

Parameters:

• conn – a DB API 2 connection
• pkg_name (str) – the name of the package with the upgrade scripts

Returns:

the current version of the database

openquake.server.db.upgrade_manager.upgrade_db(conn, pkg_name='openquake.server.db.schema.upgrades', skip_versions=())

Upgrade a database by running several scripts in a single transaction.

Parameters:

• conn – a DB API 2 connection
• pkg_name (str) – the name of the package with the upgrade scripts
• skip_versions (list) – the versions to skip

Returns:

the version numbers of the new scripts applied the database

openquake.server.db.upgrade_manager.what_if_I_upgrade(conn, pkg_name='openquake.server.db.schema.upgrades', extract_scripts='extract_upgrade_scripts')

Parameters:

• conn – a DB API 2 connection
• pkg_name (str) – the name of the package with the upgrade scripts
• extract_scripts – name of the method to extract the scripts

Module contents