pythonbuilder.pythonbuilder

module documentation

Define build targets with Target and run the build with updateTargets(). The functions starting with "state" can be used for the state parameter of Target(). Some have a string shortcut. See there.

Class	`Target`	Represents a target with an item to create, its dependencies, how to compute its state (hashed content, fingerprint), how to (re-)create it and an optional nice name.
Exception	`BuildException`	Raised on failures detected by pythonbuilder.
Function	`ansiGreen`	Wraps text into ansi sequences to render it green.
Function	`ansiRed`	Wraps text into ansi sequences to render it red.
Function	`findTarget`	Looks up a target by its name. For convenience, if the parameter is already a `Target`, it is returned as is.
Function	`flatten`	Flattens a multiply nested Iterable into the given result. Any element not `Iterable` and not `str` is added to the result as `str(element)`
Function	`getLogger`	The logger we use.
Function	`getStatefile`	Gets the `Path` of the file where target states are persisted. See `setStatefile`.
Function	`getTargets`	Returns a dictionary of the currently defined targets mapped by their `.name` field.
Function	`graphToDot`	Convenience function to create a dependency graph of all targets.
Function	`levelVerbose`	Additional logging level, `"VERBOSE"` logging somewhat more than `"INFO"`.
Function	`logcmd`	A standard way to log external commands.
Function	`pbexec`	Shallow wrapper around subprocess.run()
Function	`rawUpdateTargets`	Updates all targets in turn without resetting the traversal state in between, but see `updateTargets` too.
Function	`registerDependencyGraphTarget`	Creates a target to create the dependency graph.
Function	`resetTraversalState`	Resets the traversal state in all targets.
Function	`setLogger`	Sets the logger we use.
Function	`setStatefile`	Sets the `Path` of the file in which to persist target states.
Function	`stateDirContent`	Feeds all recursively contained files of a directory into `stateFileIteratorContent`.
Function	`stateFileContent`	Feeds the path a single file name into `stateFileIteratorContent` and returns the result.
Function	`stateFileIteratorContent`	Creates a combined state over files.
Function	`stateFixed`	Always returns the same state string, independent of the input.
Function	`stateNew`	Returns a state string which is never the same, independent of the input
Function	`stateValue`	Computes a state string from `item`, so it can be used for in-memory targets like, for example, a version string generated from the current date.
Function	`toGraphviz`	Creates the dependency graph of all targets in dot format.
Function	`updateTargets`	Wraps `rawUpdateTargets` to log times and trim any exception to just its message.
Type Variable	`T`	The data type wrapped by a `Target`.
Class	`_StateStore`	Undocumented
Function	`_escdq`	Undocumented
Function	`_eschtml`	Undocumented
Function	`_formatNanoDelta`	Undocumented
Function	`_targetToGraphviz`	Undocumented
Function	`_updateTarget`	Undocumented
Constant	`_LOG`	Undocumented
Constant	`_STATEFILE`	Undocumented
Constant	`_TARGETS`	Undocumented
Constant	`_VERBOSE`	Undocumented

def ansiGreen(text: str) -> str: ¶

Wraps text into ansi sequences to render it green.

Parameters
text:`str`	to wrap

Returns
`str`	wrapped text

def ansiRed(text: str) -> str: ¶

Wraps text into ansi sequences to render it red.

Parameters
text:`str`	to wrap

Returns
`str`	wrapped text

def findTarget(target: str | Target) -> Target | None: ¶

Looks up a target by its name. For convenience, if the parameter is already a Target, it is returned as is.

Parameters
target:`str \| Target`	to look up

Returns
`Target \| None`	either the target or `None`

def flatten(result: list[str], nested: Any): ¶

Flattens a multiply nested Iterable into the given result. Any element not Iterable and not str is added to the result as str(element)

Parameters
result:`list[str]`	to which elements are appended
nested:`Any`	either an `Iterable` to flatten or a string or other object to add to the result

def getLogger() -> logging.Logger: ¶

The logger we use.

For consistent logging, the build script can import and use it too, rather than creating its own. This logger is used by pythonbuilder throughout. If the environment variable PBLOGLEVEL is set, it is used as the default, otherwise it is "INFO". The levels are used as follows:

INFO: Show only output from logcmd which, in particular, is used by pbexec, but is also recommended to be used for target generating commands which don't call external programs.
VERBOSE: also shows targets considered and whether they are determined up-to-date or not (see levelVerbose)
DEBUG: also shows internal steps and data

def getStatefile() -> Path: ¶

Gets the Path of the file where target states are persisted. See setStatefile.

def getTargets() -> dict[str, Target]: ¶

Returns a dictionary of the currently defined targets mapped by their .name field.

This is not needed in normal operation because adding targets is achieved by the Target constructor automatically. To reset the traversal state, use resetTraversalState.

def graphToDot(targetPath: Path, _unused: Path): ¶

Convenience function to create a dependency graph of all targets.

The function calls toGraphviz() and pipes the output through the dot program, writing the image file to the targetPath.

Parameters
targetPath:`Path`	to write the graph to. Its suffix is used as the image type requested from `dot`, like `svg` or `pdf`.
_unused:`Path`	not used, facilitates used as a `cmd` parameter of `Target`

def levelVerbose() -> int: ¶

Additional logging level, "VERBOSE" logging somewhat more than "INFO".

def logcmd(cmd: list[str]): ¶

A standard way to log external commands.

These are often run with subprocess.run() which accepts a string list as the command line. This is the standard way to log the command list also used by pbexec.

Parameters
cmd:`list[str]`	to log

def pbexec(*args: Any): ¶

Shallow wrapper around subprocess.run()

logs the command line at INFO level
runs subprocess.run() with the flattened input list
calls check_returncode() on the result

If you need more, use subprocess.run() directly, possibly with flatten.

Parameters
*args:`Any`	a possibly multiply-nested list which will be flattened with `flatten` before passed to `subprocess.run()`.

def rawUpdateTargets(*targets: str | Target): ¶

Updates all targets in turn without resetting the traversal state in between, but see updateTargets too.

If target T1 depends on target T0, in the call updateTargets(T1, T0), then T0 is redundant, because it will be updated already when doing T1.

Parameters
*targets:`str \| Target`	the targets to update where any `str` argument is looked up with `findTarget`

def registerDependencyGraphTarget(graphFile: Path, name: str, buildscript: Target[Path] | Path = Path('pb')) -> Target[Path]: ¶

Creates a target to create the dependency graph.

If the given buildscript is a Path a Target is created for it.

The extension of the graphFile will be used as the image type requested from the dot program. If the buildscript is already a Target it is used as the dependency.

Parameters
graphFile:`Path`	the dependency graph after converting the output of `toGraphviz`.
name:`str`	of the `Target` for the graph file
buildscript:`Target[Path] \| Path`	to depend on such that the graph is re-created if the script changes

Returns
`Target[Path]`	the `Target` for the dependency graph

def resetTraversalState(): ¶

Resets the traversal state in all targets.

If updateTargets is run again afterwards, it considers all targets in turn again.

NOTE: this does not reset the persited target state, so without changing any data, a subsequent run of updateTargets should do nothing.

def setLogger(logger: logging.Logger): ¶

Sets the logger we use.

def setStatefile(path: Path): ¶

Sets the Path of the file in which to persist target states.

Defaults to Path("build", "pbstates").

Set this before calling updateTargets. To force all targets to be rebuild, delete this file.

def stateDirContent(targetPath: Path) -> str: ¶

Feeds all recursively contained files of a directory into stateFileIteratorContent.

Parameters
targetPath:`Path`	of the directory to compute the state from

Returns
`str`	a hash

def stateFileContent(targetPath: Path) -> str: ¶

Feeds the path a single file name into stateFileIteratorContent and returns the result.

Parameters
targetPath:`Path`	of the file to compute the state from

Returns
`str`	a hash

def stateFileIteratorContent(iter: Iterable[Path]) -> str: ¶

Creates a combined state over files.

If a path from the iterable exists in the file system, its content is fed into the checksum, otherwise a stateNew is fed in. If the path string denotes a directory, it is ignored. This is a convenience to be able to use this with Path.rglob() without extra directory name filter.

To make a difference between sets of non existing files, all file names are also hashed.

NOTE 1: This state function is best used together with a target wrapping a pbutil.Paths item.

NOTE 2: When using this as the state parameter of a Target, the iterable wrapped in the target should typically be lazily computed, that is, only when this function is actually called. Using a constant list like

files: list[Path] = generateVoodooFileList()
voodoo = Target(files, name="voodoo", state=stateFileIteratorContent)

the list is created way before the Target is actually considered. This may miss files which are only created by processing the dependencies of the target. An additional minor quirk is that the list is even computed if the voodoo target is never considered in the build.

Example: stateFileIteratorContent(Path("src").rglob("*.py")) will run all python files recursively found in src through a hash function and return the result.

Parameters
iter:`Iterable[Path]`	an iterable (or iterator or generator) over path names of files which may or may not exist.

Returns
`str`	a hash of all file names and file contents. If the iterator is empty, or contains only empty directories, `stateNew()` is returned.

def stateFixed(_t: Any = None) -> str: ¶

Always returns the same state string, independent of the input.

Parameters
_t:`Any`	unused

Returns
`str`	always the same implementation dependent string

def stateNew(_t: Any = None) -> str: ¶

Returns a state string which is never the same, independent of the input

This is subject to time.time_ns() changing more frequently than this function is called.

Parameters
_t:`Any`	unused

Returns
`str`	a different implementation dependent string for each call

def stateValue(item: Any) -> str: ¶

Computes a state string from item, so it can be used for in-memory targets like, for example, a version string generated from the current date.

Parameters
item:`Any`	to compute the state string from

Returns
`str`	an implementation dependent string is always the same for same `t.item` values and typically changes for changed `t.item` values

def toGraphviz() -> str: ¶

Creates the dependency graph of all targets in dot format.

Can be used to pipe through the 'dot' program to create an SVG or PDF. The suggested use is to define a target and let it depend on the build script.

Returns
`str`	graph in dot format

def updateTargets(*targets: str | Target) -> bool: ¶

Wraps rawUpdateTargets to log times and trim any exception to just its message.

Returns
`bool`	False if the build failed with an exception, otherwise True. Use this for a `sys.exit(1)` in the fail case.

T = ¶

The data type wrapped by a Target.

Value

TypeVar('T',
        bound=Hashable)

def _escdq(s: str) -> str: ¶

Undocumented

def _eschtml(s: str, _eschtmlMap: dict[str, str] = {'&': '&', '<': '<', '>': '>'}) -> str: ¶

Undocumented

def _formatNanoDelta(start: int, now: int, _UNITS: list[tuple[str, int, int]] = [('ns', 10000, 1000), ('µs', 10000, 1000), ('ms', 10000, 1000), ('s', 120, 60)]) -> str: ¶

Undocumented

def _targetToGraphviz(target: Target) -> str: ¶

Undocumented

def _updateTarget(states: _StateStore, target: Target[T], onbehalf: list[Target]): ¶

Undocumented

_LOG = ¶

Undocumented

Value

logging.getLogger()

_STATEFILE = ¶

Undocumented

Value

Path('build', 'pbstates')

_TARGETS: dict[str, Target] = ¶

Undocumented

Value

dict()

_VERBOSE = ¶

Undocumented

Value

logging.INFO - 5