dagstd package

Subpackages

• dagstd.constants package
  • Submodules
  • dagstd.constants.constant module
  • dagstd.constants.numbers module
  • Module contents
• dagstd.operations package
  • Submodules
  • dagstd.operations.maths module
  • dagstd.operations.strings module
  • Module contents
• dagstd.sphinx package
  • Submodules
  • dagstd.sphinx.parser module
  • Module contents

Submodules

dagstd.env module

env.py contains ops for working with environment variables.

(op)dagstd.env.or_default(key: str, default: str) → str

Returns the value of the environment variable with the given key, or the given default if the variable is not set.

(op)dagstd.env.or_empty(key: str) → str

Returns the value of the environment variable with the given key, or an empty string if the variable is not set.

(op)dagstd.env.or_none(key: str) → Union[str, NoneType]

Returns the value of the environment variable with the given key, or None if the variable is not set.

(op)dagstd.env.or_raise(key: str) → str

Returns the value of the environment variable with the given key, or raises an error if the variable is not set.

dagstd.utils module

utils.py contains useful helper ops that are not specific to any part of the Dagstd library.

(op)dagstd.utils.no_op(value: Any) → Any

Does nothing. Allows unlinked dependencies for graphs.

Examples

In this example, even though graph_2 has no data dependencies on graph_1, it will still wait for graph_1 to complete before starting due to the use of the no_op operation.

This is useful when you have graphs that need to activate in sequence, but the earlier graphs don’t pass data. For example, you might run a data sync with Airbyte, and then run a dbt project to transform that data in your warehouse.

from dagster import graph, job, In

from dagstd.constants import Constant, One, Two
from dagstd.operations import add, fmt
from dagstd.utils import no_op

@graph
def graph_1():
     name = Constant('Isaac')
     age = Constant(20)
     return fmt(Constant('{} is {} years old'), name, age)

@graph(ins={'arg_1': In(int), 'arg_2': In(int), 'deps': In(Any)})
def graph_2(arg_1, arg_2, deps):
     no_op(deps)
     return add([arg_1, arg_2])


@job
def my_job():
    string = graph_1()
    num = graph_2(One(), Two(), deps=string)

Module contents