conductor.lib.client_db module

class conductor.lib.client_db.FilesDB(db_filepath, thread_safe=True)

Bases: conductor.lib.client_db.TableDB

classmethod add_file(file_info, db_filepath=None, thread_safe=True)

Add the given file to the files table

Parameters:file_info (dict[str]) –

A dictionaries with the following keys:

  • ”filepath”,
  • ”modtime”,
  • ”filesize”
classmethod add_files(files_info, db_filepath=None, thread_safe=True)

Add the given list of files to the files db table.

Parameters:file_info (dict[str]) –

A dictionaries with the following keys:

  • ”filepath”,
  • ”modtime”,
  • ”filesize”
columns = ({'sqlite_type': 'TEXT', 'name': 'filepath', 'python_type': <type 'str'>, 'primary_key': True}, {'sqlite_type': 'TIMESTAMP', 'name': 'modtime', 'python_type': <type 'datetime.datetime'>}, {'sqlite_type': 'INTEGER', 'name': 'size', 'python_type': (<type 'int'>, <type 'long'>)}, {'sqlite_type': 'TEXT', 'name': 'md5', 'python_type': <type 'str'>})
classmethod get_cached_file(file_info, db_filepath=None, thread_safe=True)

For the given file (file_info), return it’s db entry if is not considered “stale”, otherwise return None

classmethod get_comparison_column_names()

Return a list of column names whose data (for a given file row), should be used to compare against the data of a file from disk (e.g. to check whether a file on disk has stale cache or not).

This should return names such as “filepath”, “modtime, and “size”, but not “md5” (bc the file on disk doesn’t have the md5 info available…which is the whole point of quering the db for it :) )

classmethod query_file(filepath, db_filepath=None, thread_safe=True)

Query the db for all files which match the given filepaths.

Note that this achieved through chunked queries so not to breach sqlite’s maximum of 999 arguments

classmethod query_files(filepaths, return_dict=False, db_filepath=None, thread_safe=True)

Query the db for all files which match the given filepaths.

Note that this achieved through chunked queries so not to breach sqlite’s maximum of 999 arguments

one: bool. If True, treat th

table_name = 'files'
class conductor.lib.client_db.TableDB(db_filepath, thread_safe=True)

Bases: object

Represents a single sql table to query/operate upon. This has admittedly limited functionality (as there can only be as single table to interact with).

close_connection()
column_parameters = ['name', 'sqlite_type']
columns = []
classmethod connnect_to_db(db_filepath, timeout=300, db_perms=438)

Create a connection to the database with the specified database filepath and return the connection object

Parameters:timeout (float) –

The amount of seconds that a connection will wait to establish itself before it times out and raises an “OperationalError: database is locked” exception.

This is important when threading because sqlite can’t handle that many concurrent connections and will quickly throw that exception unless the timeout is high enough.

Honestly this is kind of a hack and may not work in all circumstances. We should really just query the db in a single thread (IMO -lws)

create_table()

create the table (if it does not already exist in the db), with the self.columns

classmethod dict_to_row_tuple(input_dict)

Convert the the given dictionary of data into a tuple of data that is suitable to use for a db row insert.

classmethod get_column_names()

Return the name of all columns in the table

classmethod get_table_sql()

create a table with the columns defined in self.columns

insert_row(row_dict, replace=True)

Add the given row data (dictionary) to the the db

Parameters:
  • row_data (dict) – Where the keys are the columns names.
  • replace (bool) – When True, will replace the the existing row in the db (if there is one) that matches the row’s Primary Key.
insert_rows(row_dicts, replace=True)

Add the given list of of row data (dictionaries) to the the db

Parameters:
  • row_data (dict) – Where the keys are the columns names.
  • replace (bool) – When True, will replace the the existing row in the db (if there is one) that matches the row’s Primary Key.
classmethod row_to_dict(row)

Return the give sqlite row object as a python dict

sql_execute(sql, params=None, many=False)

Execute the given sql command

Parameters:
  • new_connection (bool) – If True, will instantiate a new sql connection object. This is necessary when running this method across multiple threads.
  • many (bool) – If True, will execute the given sql command in batch, using the given params as a list of variables for each call.
sql_fetch(sql, params=None)

Fetch data from the db via the given sql string and paramaters

classmethod sqlite_dict_factory(cursor, row)
table_name = None
conductor.lib.client_db.chunker(list_, chunk_size)

For the given list, return a tuple which breaks creates smaller lists of the given size (length), containing the contents of the the original list

conductor.lib.client_db.get_default_db_filepath()

Return a default filepath to use for storing the sqlite. Depending on the platform, this will be located in some sort of temporary directory, such as:

  • /usr/temp for Linux
  • c:\users\<username>ppdata\local        emp for Windows