openquake.server.db package

openquake.server.db.actions module

class openquake.server.db.actions.List[source]

Bases: list

openquake.server.db.actions.add_checksum(db, job_id, value)[source]
Parameters:
openquake.server.db.actions.calc_info(db, calc_id)[source]
Parameters:
Returns:

dictionary of info about the given calculation
openquake.server.db.actions.check_outdated(db)[source]

Check if the db is outdated, called before starting anything

Parameters:db – a openquake.server.dbapi.Db instance
openquake.server.db.actions.create_job(db, datadir)[source]

Create job for the given user, return it.

Parameters:
Returns:

the job ID
openquake.server.db.actions.create_outputs(db, job_id, keysize, ds_size)[source]

Build a correspondence between the outputs in the datastore and the ones in the database. Also, update the datastore size in the job table.

Parameters:
  • db – a openquake.server.dbapi.Db instance
  • job_id – ID of the current job
  • keysize – a list of pairs (key, size_mb)
  • ds_size – total datastore size in MB
openquake.server.db.actions.db_version(db)[source]
Parameters:db – a openquake.server.dbapi.Db instance
openquake.server.db.actions.del_calc(db, job_id, user, force=False)[source]

Delete a calculation and all associated outputs, if possible.

Parameters:
  • db – a openquake.server.dbapi.Db instance
  • job_id – job ID, can be an integer or a string
  • user – username
  • force – delete even if there are dependent calculations
Returns:

None if everything went fine or an error message
openquake.server.db.actions.delete_uncompleted_calculations(db, user)[source]

Delete the uncompleted calculations of the given user.

Parameters:
openquake.server.db.actions.fetch(db, templ, *args)[source]

Run generic queries directly on the database. See the documentation of the dbapi module.

Parameters:
openquake.server.db.actions.find(db, description)[source]
Parameters:
openquake.server.db.actions.finish(db, job_id, status)[source]

Set the job columns is_running, status, and stop_time.

Parameters:
  • db – a openquake.server.dbapi.Db instance
  • job_id – ID of the current job
  • status – a string such as ‘successful’ or ‘failed’
openquake.server.db.actions.get_calc_id(db, datadir, job_id=None)[source]

Return the latest calc_id by looking both at the datastore and the database.

Parameters:
  • db – a openquake.server.dbapi.Db instance
  • datadir – the directory containing the datastores
  • job_id – a job ID; if None, returns the latest job ID
openquake.server.db.actions.get_calc_ids(db, user)[source]
Returns:calculation IDs of the given user
openquake.server.db.actions.get_calcs(db, request_get_dict, allowed_users, user_acl_on=False, id=None)[source]
Parameters:
  • db – a openquake.server.dbapi.Db instance
  • request_get_dict – a dictionary
  • allowed_users – a list of users
  • user_acl_on – if True, returns only the calculations owned by the user or the group
  • id – if given, extract only the specified calculation
Returns:

list of tuples (job_id, user_name, job_status, calculation_mode,

job_is_running, job_description)
openquake.server.db.actions.get_checksum_from_job(db, job_id)[source]
Parameters:
Returns:

the value of the checksum or 0
openquake.server.db.actions.get_dbpath(db)[source]
Parameters:db – a openquake.server.dbapi.Db instance
Returns:the path to the database file.
openquake.server.db.actions.get_executing_jobs(db)[source]
Parameters:db – a openquake.server.dbapi.Db instance
Returns:(id, pid, user_name, start_time) tuples
openquake.server.db.actions.get_job(db, job_id, username=None)[source]

If job_id is negative, return the last calculation of the current user, otherwise returns the job_id unchanged.

Parameters:
  • db – a openquake.server.dbapi.Db instance
  • job_id – a job ID (can be negative and can be nonexisting)
  • username – an user name (if None, ignore it)
Returns:

a valid job or None if the original job ID was invalid
openquake.server.db.actions.get_job_from_checksum(db, checksum)[source]
Parameters:
Returns:

the job associated to the checksum or None
openquake.server.db.actions.get_log(db, job_id)[source]

Extract the logs as a big string

Parameters:
openquake.server.db.actions.get_log_size(db, job_id)[source]

Get a slice of the calculation log as a JSON list of rows.

Parameters:
openquake.server.db.actions.get_log_slice(db, job_id, start, stop)[source]

Get a slice of the calculation log as a JSON list of rows

Parameters:
  • db – a openquake.server.dbapi.Db instance
  • job_id – a job ID
  • start – start of the slice
  • stop – end of the slice (the last element is excluded)
openquake.server.db.actions.get_longest_jobs(db)[source]
Parameters:db – a openquake.server.dbapi.Db instance
Returns:(id, user_name, days) tuples
openquake.server.db.actions.get_output(db, output_id)[source]
Parameters:
Returns:

(ds_key, calc_id, dirname)
openquake.server.db.actions.get_outputs(db, job_id)[source]
Parameters:
Returns:

A sequence of openquake.server.db.models.Output objects
openquake.server.db.actions.get_path(db)[source]
Parameters:db – a openquake.server.dbapi.Db instance
Returns:the full path to the dbserver codebase
openquake.server.db.actions.get_result(db, result_id)[source]
Parameters:
Returns:

(job_id, job_status, datadir, datastore_key)
openquake.server.db.actions.get_results(db, job_id)[source]
Parameters:
Returns:

(datadir, datastore_keys)
openquake.server.db.actions.get_traceback(db, job_id)[source]

Return the traceback of the given calculation as a list of lines. The list is empty if the calculation was successful.

Parameters:
openquake.server.db.actions.import_job(db, calc_id, calc_mode, description, user_name, status, hc_id, datadir)[source]

Insert a calculation inside the database, if calc_id is not taken

openquake.server.db.actions.list_calculations(db, job_type, user_name)[source]

Yield a summary of past calculations.

Parameters:
openquake.server.db.actions.list_outputs(db, job_id, full=True)[source]

List the outputs for a given OqJob.

Parameters:
  • db – a openquake.server.dbapi.Db instance
  • job_id – ID of a calculation.
  • full (bool) – If True produce a full listing, otherwise a short version
openquake.server.db.actions.log(db, job_id, timestamp, level, process, message)[source]

Write a log record in the database.

Parameters:
  • db – a openquake.server.dbapi.Db instance
  • job_id – a job ID
  • timestamp – timestamp to store in the log record
  • level – logging level to store in the log record
  • process – process ID to store in the log record
  • message – message to store in the log record
openquake.server.db.actions.reset_is_running(db)[source]

Reset the flag job.is_running to False. This is called when the DbServer is restarted: the idea is that it is restarted only when all computations are completed.

Parameters:db – a openquake.server.dbapi.Db instance
openquake.server.db.actions.set_status(db, job_id, status)[source]

Set the status ‘created’, ‘executing’, ‘complete’, ‘failed’, ‘aborted’ consistently with is_running.

Parameters:
openquake.server.db.actions.update_job(db, job_id, dic)[source]

Update the given calculation record.

Parameters:
  • db – a openquake.server.dbapi.Db instance
  • job_id – a job ID
  • dic – a dictionary of valid field/values for the job table
openquake.server.db.actions.update_job_checksum(db, job_id, checksum)[source]
Parameters:
openquake.server.db.actions.update_parent_child(db, parent_child)[source]

Set hazard_calculation_id (parent) on a job_id (child)

openquake.server.db.actions.upgrade_db(db)[source]
Parameters:db – a openquake.server.dbapi.Db instance
openquake.server.db.actions.what_if_I_upgrade(db, extract_scripts)[source]
Parameters:

openquake.server.db.upgrade_manager module

exception openquake.server.db.upgrade_manager.DuplicatedVersion[source]

Bases: RuntimeError

class openquake.server.db.upgrade_manager.UpgradeManager(upgrade_dir, version_table='revision_info', version_pattern='\d\d\d\d', flag_pattern='(-slow|-danger)?')[source]

Bases: object

The package containing the upgrade scripts should contain an instance of the UpgradeManager called upgrader in the __init__.py file. It should also specify the initializations parameters

Parameters:
  • upgrade_dir – the directory were the upgrade script reside
  • version_table – the name of the versioning table (default revision_info)
  • version_pattern – a regulation expression for the script version number (dddd)
ENGINE_URL = 'https://github.com/gem/oq-engine/tree/master/'
UPGRADES = 'openquake/server/db/schema/upgrades/'
check_versions(conn)[source]
Parameters:conn – a DB API 2 connection
Returns:a message with the versions that will be applied or None
extract_upgrade_scripts()[source]

Extract the OpenQuake upgrade scripts from the links in the GitHub page

get_db_versions(conn)[source]

Get all the versions stored in the database as a set.

Parameters:conn – a DB API 2 connection
init(conn)[source]

Create the version table and run the base script on an empty database.

Parameters:conn – a DB API 2 connection
install_versioning(conn)[source]

Create the version table into an already populated database and insert the base script.

Parameters:conn – a DB API 2 connection
classmethod instance(conn, pkg_name='openquake.server.db.schema.upgrades')[source]

Return an UpgradeManager instance.

Parameters:
  • conn – a DB API 2 connection
  • pkg_name (str) – the name of the package with the upgrade scripts
parse_script_name(script_name)[source]

Parse a script name and return a dictionary with fields fname, name, version and ext (or None if the name does not match).

Parameters:name – name of the script
read_scripts(minversion=None, maxversion=None, skip_versions=())[source]

Extract the upgrade scripts from a directory as a list of dictionaries, ordered by version.

Parameters:
  • minversion – the minimum version to consider
  • maxversion – the maximum version to consider
  • skipversions – the versions to skip
upgrade(conn, skip_versions=())[source]

Upgrade the database from the current version to the maximum version in the upgrade scripts.

Parameters:
  • conn – a DBAPI 2 connection
  • skip_versions – the versions to skip
exception openquake.server.db.upgrade_manager.VersionTooSmall[source]

Bases: RuntimeError

exception openquake.server.db.upgrade_manager.VersioningNotInstalled[source]

Bases: RuntimeError

class openquake.server.db.upgrade_manager.WrappedConnection(conn, debug=False)[source]

Bases: object

This is an utility class that wraps a DB API-2 connection providing a couple of convenient features.

  1. it is possible to set a debug flag to print on stdout the executed queries;
  2. there is a .run method to run a query with a dedicated cursor; it returns the cursor, which can be iterated over
Parameters:conn – a DB API2-compatible connection
run(templ, *args)[source]

A simple utility to run SQL queries.

Parameters:
  • templ – a query or query template
  • args – the arguments (or the empty tuple)
Returns:

the DB API 2 cursor used to run the query
openquake.server.db.upgrade_manager.apply_sql_script(conn, fname)[source]

Apply the given SQL script to the database

Parameters:
  • conn – a DB API 2 connection
  • fname – full path to the creation script
openquake.server.db.upgrade_manager.check_script(upgrade, conn, dry_run=True, debug=True)[source]

An utility to debug upgrade scripts written in Python

Parameters:
  • upgrade – upgrade procedure
  • conn – a DB API 2 connection
  • dry_run – if True, do not change the database
  • debug – if True, print the queries which are executed
openquake.server.db.upgrade_manager.db_version(conn, pkg_name='openquake.server.db.schema.upgrades')[source]
Parameters:
  • conn – a DB API 2 connection
  • pkg_name (str) – the name of the package with the upgrade scripts
Returns:

the current version of the database
openquake.server.db.upgrade_manager.upgrade_db(conn, pkg_name='openquake.server.db.schema.upgrades', skip_versions=())[source]

Upgrade a database by running several scripts in a single transaction.

Parameters:
  • conn – a DB API 2 connection
  • pkg_name (str) – the name of the package with the upgrade scripts
  • skip_versions (list) – the versions to skip
Returns:

the version numbers of the new scripts applied the database
openquake.server.db.upgrade_manager.what_if_I_upgrade(conn, pkg_name='openquake.server.db.schema.upgrades', extract_scripts='extract_upgrade_scripts')[source]
Parameters:
  • conn – a DB API 2 connection
  • pkg_name (str) – the name of the package with the upgrade scripts
  • extract_scripts – name of the method to extract the scripts

Module contents