dootle.jobs.expand

Actions for turning data into delayed specs, for then queuing

For Example, with state(blah=[‘a’,’b’,’c’]): {do=’job.expand’, from_=’blah’, template=’simple::task’, inject={literal=[‘val’]}, update_=’specs’} Or: {do=’job.match’, from_=’blah’, mapping={…}, inject={literal=[‘val’]}, update_=’specs’} then: {do=”job.queue”, from_=”specs”}

Classes

JobExpandAction

Expand data into a number of delayed specs.

MatchExpansionAction

Create subtasks, but with context specific mapping

JobQueueAction

An action that returns a list of tasks to queue, instead of a dict,

Module Contents

class dootle.jobs.expand.JobExpandAction

Bases: doot.workflow.actions.DootBaseAction

Expand data into a number of delayed specs. Does not queue the tasks, thats for job.queue

from : source data to inject inject : how to map the data as literals template : taskname of the base spec to use

registered as: job.expand TODO change job subtask naming scheme

_prep_data(
data: jgdv.Maybe[int | str | pathlib.Path | list],
) → list

ensure the data for subtasks is correct

Parameters:

data (jgdv.Maybe[int | str | pathlib.Path | list])

Return type:

list

_delay_spec(
*,
target: doot.workflow._interface.TaskName_p,
template: doot.workflow._interface.TaskName_p,
literal: Any,
inject: jgdv.Maybe[doot.workflow._interface.InjectSpec_i],
state: dict,
overrides: dict,
) → doot.workflow.factory.DelayedSpec
Parameters:

  • target (doot.workflow._interface.TaskName_p)

  • template (doot.workflow._interface.TaskName_p)

  • literal (Any)

  • inject (jgdv.Maybe[doot.workflow._interface.InjectSpec_i])

  • state (dict)

  • overrides (dict)

Return type:

doot.workflow.factory.DelayedSpec

class dootle.jobs.expand.MatchExpansionAction

Bases: JobExpandAction

Create subtasks, but with context specific mapping

Take a mapping of {pattern -> task} and a list of data, and build a list of delayed specs

use prepfn to get a value from a taskspec to match on.

> prepfn :: λ(spec) -> Maybe[str|TaskName]

defaults to getting JobMatchAction._default_prep_fn, which tries spec.fpath for a suffix

registered as: job.match

_default_prep_fn(
x: pathlib.Path | str,
) → jgdv.Maybe[str]

The default matcher. get the ‘fpath’ from a spec

Parameters:

x (pathlib.Path | str)

Return type:

jgdv.Maybe[str]

class dootle.jobs.expand.JobQueueAction

An action that returns a list of tasks to queue, instead of a dict,

which signals the doot backend to queue the items of the list

  1. Queue Named Tasks: {do=’job.queue’, args=[‘group::task’] }

  2. Queue Expanded TaskSpecs: {do=’job.queue’, from_=’state_key’ }

tasks can be specified by name in args and from prior expansion state vars with from_ (accepts a list)

after can be used to specify additional depends_on entries. (the job head is specified using $head$)

registered as: job.queue