2021-02-05

(sph io path-pipe-chain)

call procedures with input/output arguments in a chained manner to allow data flow between them.

part of sph-lib

library description

procedures can be configured to take paths or ports as arguments and the links are made compatible automatically.

for example paths are temporarily created fifo pipes and ports are guile ports

module name

(sph io path-pipe-chain)

exported bindings

procedure: path-pipe-chain first-input last-output config ... ->
string/port/any string/port/any #(symbol symbol procedure:{string/port string/port -> any}) ... -> (any ...)
first-input last-output (path/port/nothing path/port/nothing procedure:{in out -> result}) ... -> (result ...)
like pipe-chain but additionally supports specifying the type of port between procedures.
procedures can take file paths or ports for input or output. input output combinations between procedures are automatically made compatible.
caveats:
* in port-path and path-port links, input or output are not ports but procedures that return
the path that has to be opened for read or write in a separate thread or process.
it blocks until the next procedure has opened the other end. this is typical behaviour of
named pipes and there does not seem to be any applicable way around it
* intermediate ports must be closed after read/write finished
* intermediate named pipe paths should be deleted after read/write finished. otherwise they accumulate
in the systems temporary directory (but do not take up space in the filesystem)
* ports are not automatically closed when an exception occurs
* to make it possible to read while a previous procedure is writing a user might want to create threads or processes
* named pipes are used when paths are requested. they normally block for each end until the other end is connected,
and they do not send end of file before the writer is closed
most of these caveats are to keep this procedure generic enough to be useful in many situations
usage example:
(path-pipe-chain \"/tmp/test\" (current-output-port) (vector (q nothing) (q port) (l (in out) (file->port in out))) (vector (q path) (q port) (l (in out) (do-stuff path port))))
algorithm:
* create a vector of all input/output type names to get previous, current and next type per proc
* create previous-current and current-next output/input link objects
* handle first and last input/output separately because they are foreign objects not to be created by path-pipe-chain
* prepare and call procedures in order to complete the chain at the last input.
the output is passed to the current and the input to the next procedure.
one of the biggest issues when developing this was to avoid blocking deadlocks with named pipes
variable: sph-io-path-pipe-chain-description