python.task_manager.background_task_manager module

Background task manager.

class BackgroundTaskManager(parent, start_processing=False, max_threads=8)

Bases: sphinx.ext.autodoc.importer._MockObject

Main task manager class. Manages a queue of tasks running them asynchronously through a pool of worker threads.

The BackgroundTaskManager class itself is reentrant but not thread-safe so its methods should only be called from the thread it is created in. Typically this would be the main thread of the application.

Signal task_completed(uid, group, result):  Emitted when a task has been completed. The uid parameter holds the unique id associated with the task, the group is the group that the task is associated with and the result is the data returned by the task. Signal task_failed(uid, group, message, traceback_str):  Emitted when a task fails for some reason. The uid parameter holds the unique id associated with the task, the group is the group that the task is associated with, the message is a short error message and the traceback_str holds a full traceback. Signal task_group_finished(group):  Emitted when all tasks in a group have finished. The group is the group that has completed.

add_pass_through_task(priority=None, group=None, upstream_task_ids=None, task_kwargs=None)

Add a pass-through task to the queue. A pass-through task doesn’t perform any work but can be useful when synchronising other tasks (e.g. pulling the results from multiple upstream tasks into a single task)

Parameters:

• priority – The priority this task should be run with. Tasks with higher priority are run first.
• group – The group this task belongs to. Task groups can be used to simplify task management (e.g. stop a whole group, be notified when a group is complete). A group is expressed as a string, for example ‘thumbnails’, ‘IO’ or ‘shotgun’.
• upstream_task_ids – A list of any upstream tasks that should be completed before this task is run. The results from any upstream tasks are appended to the kwargs for this task.
• task_kwargs – A dictionary of named parameters that will be appended to the result of the pass-through task.

Returns:

A unique id representing the task.

add_task(cbl, priority=None, group=None, upstream_task_ids=None, task_args=None, task_kwargs=None)

Add a new task to the queue. A task is a callable method/class together with any arguments that should be passed to the callable when it is called.

Parameters:

• cbl – The callable function/class to call when executing the task
• priority – The priority this task should be run with. Tasks with higher priority are run first.
• group – The group this task belongs to. Task groups can be used to simplify task management (e.g. stop a whole group, be notified when a group is complete)
• upstream_task_ids – A list of any upstream tasks that should be completed before this task is run. The results from any upstream tasks are appended to the kwargs for this task.
• task_args – A list of unnamed parameters to be passed to the callable when running the task
• task_kwargs – A dictionary of named parameters to be passed to the callable when running the task

Returns:

A unique id representing the task.

next_group_id()

Return the next available group id

Returns:A unique group id to be used for tasks that belong to the same group.

pause_processing()

Pause processing of tasks - any currently running tasks will complete as normal.

shut_down()

Shut down the task manager. This clears the task queue and gracefully stops all running threads. Completion/failure of any currently running tasks will be ignored.

start_processing()

Start processing of tasks

stop_all_tasks()

Stop all currently queued or running tasks. If any tasks are already running then they will complete but their completion/failure signals will be ignored.

stop_task(task_id, stop_upstream=True, stop_downstream=True)

Stop the specified task from running. If the task is already running then it will complete but the completion/failure signal will be ignored.

Parameters:

• task_id – The id of the task to stop
• stop_upstream – If true then all upstream tasks will also be stopped
• stop_downstream – If true then all downstream tasks will also be stopped

stop_task_group(group, stop_upstream=True, stop_downstream=True)

Stop all tasks in the specified group from running. If any tasks are already running then they will complete but their completion/failure signals will be ignored.

Parameters:

• group – The task group to stop
• stop_upstream – If true then all upstream tasks will also be stopped
• stop_downstream – If true then all downstream tasks will also be stopped

task_completed

Used by autodoc_mock_imports.

task_failed

Used by autodoc_mock_imports.

task_group_finished

Used by autodoc_mock_imports.