References

All functionalities references.

Database

Database manipulation classes.

Database manager

Abstract database operations using SQLAlchemy.

Store all errors in exception object.
exception.logger: logger object.
exception.message: error message.
class database_manager.DatabaseManager(configuration, *, url=None, prefix='', **kwargs)

Manage database interaction. Create a new Engine instance using a configuration dictionary. He must contains a key (assuming the default prefix) url, which provides the database URL.

dialect+driver://username:password@host:port/database
Otherwise the url keyword arg must be provided with the following parameters.
  • drivername – the name of the database backend. This name will correspond to a module in sqlalchemy/databases or a third party plug-in.
  • username – The user name.
  • password – database password.
  • host – The name of the host.
  • port – The port number.
  • database – The database name.
  • query – A dictionary of options to be passed to the dialect and/or
    the DBAPI upon connect.

Warning

If url argument is provided it will override configuration url key but kwargs will override it.

Note

Engine is created with sqlalchemy.engine_from_config, url is created with sqlalchemy.engine.url.URL, you can find more informations on http://docs.sqlalchemy.org/en/latest/core/engines.html.

Parameters:
  • configuration (dict) – engine configuration.
  • url (dict) – arguments to construct the database URL, default to None.
  • prefix (str) – prefix to match and then strip from keys in ‘configuration’, default to ‘’.
  • kwargs (dict) – each keyword argument overrides the corresponding item taken from the ‘configuration’ dictionary. Keyword arguments should not be prefixed.
add(entity)

Add one entity into current database, flush and expunge it.

Parameters:entity (obj) – Entity class with data to feed database.
add_all(entities)

Add list of entity into current database, flush and expunge it.

Parameters:entities (list) – List of entity class with data to feed.
bulk_insert_mappings(obj, data)

Execute a bulk insert of data.

Parameters:
  • obj (obj) – Class representing table schema.
  • data (list) – list of dict representing data to add.
bulk_insert_mappings_core(obj, data, chunk_size=1000)

Execute a bulk insert of data split in chunks using SQLAlchemy CORE.

Parameters:
  • obj (obj) – Class representing table schema.
  • data (list) – list of dict representing data to add.
  • chunk_size (int) – size of chunk to insert at one time, default to 1000.
bulk_save_objects(data, return_defaults=False)

Execute a bulk save of data.

Parameters:data (list) – list of table object with data to add.
create_schema(name)

Create a schema.

Parameters:name (str) – name of schema.
create_table_obj(name, *, schema=None, autoload=True)

Create an sqlalchemy.Table object.

Parameters:
  • name (str) – name of table.
  • schema (str) – table schema name. Default to None.
  • autoload (bool) – autoload table structure. Default to True.
delete(entity, merge=False)

Delete one entity in current database.

Parameters:entity (obj) – Entity instance to delete.
drop_schema(name)

Drop a schema.

Parameters:name (str) – name of schema.
entity_exists(filter)

Check if an entity exists on given filter.

Parameters:entity (obj) – filter to apply to where clause.
Returns:True if exists otherwise False.
map_tables(tables, *, schema=None)

Map list of tables into python object.

Parameters:
  • tables (list) – list of tables name to map.
  • schema (str) – table schema name. Default to None.
Returns:

list of python object containing tables schemas.
read(table)

Read all entry for one table, expunge and return it.

Parameters:table (obj) – Entity class with table schema.
Returns:list of database row as entity class.
schema_exists(name)

Check if schema exists.

Parameters:name (str) – schema name.
Returns:True if exists otherwise False.
session(*, raise_err=True, session_config=None)

Context manager for session, yield a Session instance if session_config is not provided autocommit is turn to True and expire_on_commit to False. In case of error log exception and rollback session, and finally close the session.

Note

Session object is created with scoped_session, this ensure to always have the same session object for a database. More informations on http://docs.sqlalchemy.org/en/latest/orm/contextual.html.

Parameters:
  • raise_err (bool) – raise exception in case of error if True. Default to True.
  • session_config (dict) – session configuration.
update(entity, keys)

Update one entity, select by id.

Parameters:
  • entity (obj) – entity object to update.
  • keys (list) – list of keys to update.

Database singleton

Singleton who store a database manager.

class database_singleton.DatabaseSingleton(configuration, *, url=None, prefix='', **kwargs)

Singleton for database manager. Use this class like a DatabaseManager.

Attr instance:_DatabaseSingleton instance.

Entity

Database entities creation.

Abstract entity

class abc_entity.ABCEntity

Abstract entity to represent table schema. Contains base method to handle table.

classmethod create(db)

Create table in database.

Parameters:db (obj) – DatabaseManager.
Returns:cls object.
classmethod drop(db)

Drop table in database.

Parameters:db (obj) – DatabaseManager.
Returns:cls object.
classmethod exists(db)

Check if tables exists in database.

Parameters:db (obj) – DatabaseManager.
Returns:True if exists otherwise False.

Base entity

Basic Entity as declarative to inherite from.

Flask

Flask interaction classes.

Flask engine

Engine for web server with Flask and swagger.

class flask_engine.FlaskEngine(views, swagger_config)

Handle flask app with swagger. Load all view from views dictionary with key ‘cls’. You can define REST method in they ‘method’ key.

Parameters:
  • views (dict) – Web views data.
  • swagger_config (dict) – swagger configuration.
run()

Run the flask app with debug.

Logging

Logging setup and utils.

Logging decorator

Suite of decorator for logging.

logging_decorator.lambda_logger(level, *, start=None, end=None, err=None, log_err=False)

Decorator for logging before and after function execution with callable as message, usefull if you want to use decorated function params in logging message.. Decorated function must be method in class with logger attribute. Log with exception level if decorated func raise an exception. class (cls) or instance (self) is always pass as cls parameter.

Parameters:
  • level (str) – level of logger.
  • start (func) – string to log before function exec, default: None.
  • end (func) – string to log after function exec, default: None.
  • err (func) – string to log if exception is raised, default: ‘func_name error’.
  • log_err (bool) – log error if True, default: False.
logging_decorator.simple_logger(level, *, start=None, end=None, err=None, log_err=False)

Decorator for logging before and after function execution. Decorated function must be method in class with logger attribute. Log with exception level if decorated func raise an exception.

Parameters:
  • level (str) – level of logger.
  • start (str) – string to log before function exec, default: None.
  • end (str) – string to log after function exec, default: None.
  • err (str) – string to log if exception is raised, default: ‘func_name error’.
  • log_err (bool) – log error if True, default: False.

Logging filters

Custom filters for logging.

class logging_filters.LvlFilter(low, high)

Custom filter based on level, log only if level is equal or between two value.

Parameters:
  • low (int) – low level value.
  • high (int) – high level value.
filter(record)

Determine if the specified record is to be logged.

Parameters:record (obj) – record obj to log.
Returns:True if record must be logged otherwise False.

Logging handlers

Custom handlers for logging.

class logging_handlers.SlackHandler(token, channel, proxies=None)

Handler for slack. Use color dict attr for the message attachments.

color = {
    'ERROR': '#F00', (RED)
    'OTHER': '#3AA3E3', (BLUE)
    'SLACK': '#00ff2b', (GREEN)
    'WARNING': '#f07900', (YELLOW)
}
Parameters:
  • token (str) – auth token for slack.
  • channel (str) – channel name to post message.
emit(record)

Send message on slack channel.

Logging level

Custom level for logging.

class logging_level.LoggingLevel

Custom logging level.

static mail(self, message, *args, **kwargs)

Custom level for mail logging.

static slack(self, message, *args, **kwargs)

Custom level for slack logging.

Parser

Parsers creation.

Arg parser

class arg_parser.ArgParser(parser_options, commands)

Parser for CLI creation.

Parameters:
  • parser_options (dict) – arguments for argparse.ArgumentParser.
  • commands (dict) – dict of all commands as key: command name, value: callable.
parse()

Parse command argument and execute the command, print helper if no command.

Yaml parser

Yaml parsing and setup.

class yaml_parser.YamlConstructor

Collection of custom Yaml constructor.

static simple_join(loader, node)

Yaml join list of string on empty string. usage: !join [param, …]

Parameters:
  • loader (obj) – PyYaml Loader.
  • node (obj) – Pyyaml node.
Returns:

joined string.
static string_join(loader, node)

Yaml join list of string by given string as first param. usage: !string_join [param, …]

Parameters:
  • loader (obj) – PyYaml Loader.
  • node (obj) – Pyyaml node.
Returns:

joined string.
static yaml_getter(loader, node, *, data)

Constructor to get data from dictionary.

Parameters:
  • loader (obj) – PyYaml Loader.
  • node (obj) – Pyyaml node.
  • directory (dict) – project directory path.
Returns:

value at given key.
class yaml_parser.YamlParser(debug, *, prod_block='prod', dev_block='dev', custom_constructors=None)

Parse yaml file using custom constructor with two level of settings prod and dev, dev inherite from prod and override it.

Parameters:
  • debug (bool) – bool to setup prod or dev settings.
  • prod_block (str) – yaml block name for production settings, default: prod.
  • dev_block (str) – yaml block name for development settings, default: dev.
  • directory (dict) – project directory path, default: {}.
add_custom_constructors(custom_constructors)

Add custom constructors to yaml.

Parameters:custom_constructors (dict) – custom constructors to add as key: tag, value: func.
add_getter_constructor(name, data)

Set a yaml constructor to get data from other yaml.

Parameters:
  • name (str) – name of the getter.
  • data (dict) – data dictionary to get from.
parse(yaml_to_dict, yaml_dir_path)

Extract .yml conf files into info dict.

Parameters:
  • yaml_to_dict (dict) – dict to map yaml file with variable.
  • yaml_dir_path (str) – path directory for all yaml files.

Utils

Utils classes and functions.

property

Custom Property decorator.

class property.classproperty(fget=None, doc=None)

Decorator to set a property as classmethod. Work only as getter.

usage: @classproperty.

class property.staticproperty(fget=None, doc=None)

Decorator to set a property as staticmethod. Work only as getter.

usage: @staticproperty.