Processing module

Processing module

This module contians all processing classes derived from the base class. The base class has mechanisms to handle the data passing between each processing block.

Tracking UPDATES:

Date

Description

Jul-17-2019

t_general_block is added and the output queues are added in add_Qs.

class processing.base_processing

Bases: object

The abstract class for all signal processing ecosystem.

class processing.p_general_block(BlockName, NInPorts, NOutPorts)

Bases: processing.process_blocks

The block used to process the input data stream from other processes and deliver processed data to other processes. TODO: check if it is possible to collect and distribute the incoming and outgoing streams from and to multiple processes.

the problem mightb be passing multiple Queues to the process from different processes.

Attributes
authkey
daemon

Return whether process is a daemon

exitcode

Return exit code of process or None if it has yet to stop

ident

Return identifier (PID) of process or None if it has yet to start

name
pid

Return identifier (PID) of process or None if it has yet to start

sentinel

Return a file descriptor (Unix) or handle (Windows) suitable for waiting for process termination.

Methods

close(self)

Close the Process object.

is_alive(self)

Return whether process is alive

join(self[, timeout])

Wait until child process terminates

kill(self)

Terminate process; sends SIGKILL signal or uses TerminateProcess()

run(self)

Method to be run in sub-process; can be overridden in sub-class

start(self)

Start child process

terminate(self)

Terminate process; sends SIGTERM signal or uses TerminateProcess()

delay_calc

worker

run(self)

Method to be run in sub-process; can be overridden in sub-class

worker(self, \*ins)
class processing.p_sink_block(BlockName, NPorts)

Bases: processing.process_blocks

Consuming data from other processes.

Attributes
authkey
daemon

Return whether process is a daemon

exitcode

Return exit code of process or None if it has yet to stop

ident

Return identifier (PID) of process or None if it has yet to start

name
pid

Return identifier (PID) of process or None if it has yet to start

sentinel

Return a file descriptor (Unix) or handle (Windows) suitable for waiting for process termination.

Methods

close(self)

Close the Process object.

is_alive(self)

Return whether process is alive

join(self[, timeout])

Wait until child process terminates

kill(self)

Terminate process; sends SIGKILL signal or uses TerminateProcess()

run(self)

Method to be run in sub-process; can be overridden in sub-class

start(self)

Start child process

terminate(self)

Terminate process; sends SIGTERM signal or uses TerminateProcess()

delay_calc

worker

run(self)

Method to be run in sub-process; can be overridden in sub-class

worker(self, \*ins)
class processing.p_source_block(BlockName, NPorts)

Bases: processing.process_blocks

The process block source which is passing generated data down to other processes.

Attributes
authkey
daemon

Return whether process is a daemon

exitcode

Return exit code of process or None if it has yet to stop

ident

Return identifier (PID) of process or None if it has yet to start

name
pid

Return identifier (PID) of process or None if it has yet to start

sentinel

Return a file descriptor (Unix) or handle (Windows) suitable for waiting for process termination.

Methods

close(self)

Close the Process object.

is_alive(self)

Return whether process is alive

join(self[, timeout])

Wait until child process terminates

kill(self)

Terminate process; sends SIGKILL signal or uses TerminateProcess()

run(self)

Method to be run in sub-process; can be overridden in sub-class

start(self)

Start child process

terminate(self)

Terminate process; sends SIGTERM signal or uses TerminateProcess()

delay_calc

worker

run(self)

Method to be run in sub-process; can be overridden in sub-class

worker(self)
class processing.process_blocks(name)

Bases: processing.base_processing, multiprocessing.context.Process

This is to derive all multiprocessed blocks to achieve full parallelism. TODO: setting daemon returns error:

assert self._popen is None, ‘process has already started’ AttributeError: ‘DCAReader’ object has no attribute ‘_popen’

Attributes
authkey
daemon

Return whether process is a daemon

exitcode

Return exit code of process or None if it has yet to stop

ident

Return identifier (PID) of process or None if it has yet to start

name
pid

Return identifier (PID) of process or None if it has yet to start

sentinel

Return a file descriptor (Unix) or handle (Windows) suitable for waiting for process termination.

Methods

close(self)

Close the Process object.

is_alive(self)

Return whether process is alive

join(self[, timeout])

Wait until child process terminates

kill(self)

Terminate process; sends SIGKILL signal or uses TerminateProcess()

run(self)

Method to be run in sub-process; can be overridden in sub-class

start(self)

Start child process

terminate(self)

Terminate process; sends SIGTERM signal or uses TerminateProcess()

delay_calc

worker

delay_calc(self)
run(self)

Method to be run in sub-process; can be overridden in sub-class

worker(self)
class processing.t_general_block(BlockName, NInPorts, NOutPorts)

Bases: processing.thread_blocks

All threaded general blocks should be derived from this class. Initializing parameters are:

Parameters
  • BlockName (str) – name of the processing block

  • NInPorts (int) – number of output queues

  • NOutPorts (int) – number of output queues

Attributes
daemon

A boolean value indicating whether this thread is a daemon thread.

ident

Thread identifier of this thread or None if it has not been started.

name

A string used for identification purposes only.

Methods

isAlive(self)

Return whether the thread is alive.

is_alive(self)

Return whether the thread is alive.

join(self[, timeout])

Wait until the thread terminates.

run(self)

Method representing the thread’s activity.

start(self)

Start the thread’s activity.

delay_calc

getName

isDaemon

setDaemon

setName

worker

run(self)

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

worker(self, \*ins)
class processing.t_sink_block(BlockName, NInPorts)

Bases: processing.thread_blocks

All threaded sink blocks should be derived from this class. Initializing parameters are:

Parameters
  • BlockName (str) – name of the processing block

  • NInPorts (int) – number of output queues

Attributes
daemon

A boolean value indicating whether this thread is a daemon thread.

ident

Thread identifier of this thread or None if it has not been started.

name

A string used for identification purposes only.

Methods

isAlive(self)

Return whether the thread is alive.

is_alive(self)

Return whether the thread is alive.

join(self[, timeout])

Wait until the thread terminates.

run(self)

Method representing the thread’s activity.

start(self)

Start the thread’s activity.

delay_calc

getName

isDaemon

setDaemon

setName

worker

run(self)

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

worker(self, \*ins)
class processing.t_source_block(BlockName, NPorts)

Bases: processing.thread_blocks

All threaded source blocks should be derived from this class. Initializing parameters are:

Parameters
  • BlockName (str) – Name of the processing block

  • NQs (int) – Number of output ports

Attributes
daemon

A boolean value indicating whether this thread is a daemon thread.

ident

Thread identifier of this thread or None if it has not been started.

name

A string used for identification purposes only.

Methods

isAlive(self)

Return whether the thread is alive.

is_alive(self)

Return whether the thread is alive.

join(self[, timeout])

Wait until the thread terminates.

run(self)

Method representing the thread’s activity.

start(self)

Start the thread’s activity.

delay_calc

getName

isDaemon

setDaemon

setName

worker

run(self)

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

worker(self)
class processing.thread_blocks(name)

Bases: processing.base_processing, threading.Thread

This is to derive multithreaded blocks.

Attributes
daemon

A boolean value indicating whether this thread is a daemon thread.

ident

Thread identifier of this thread or None if it has not been started.

name

A string used for identification purposes only.

Methods

isAlive(self)

Return whether the thread is alive.

is_alive(self)

Return whether the thread is alive.

join(self[, timeout])

Wait until the thread terminates.

run(self)

Method representing the thread’s activity.

start(self)

Start the thread’s activity.

delay_calc

getName

isDaemon

setDaemon

setName

worker

delay_calc(self)
run(self)

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

worker(self)
class processing.threaded_process_blocks(ProcName)

Bases: processing.process_blocks, processing.thread_blocks

Multithreaded processes containing a flowgraph taking data from at least one tp_source_block or t_source_block`and deliver outputs to at least one `tp_sink_block or t_sink_block. # TODO: complete the class definition

Attributes
authkey
daemon

Return whether process is a daemon

exitcode

Return exit code of process or None if it has yet to stop

ident

Return identifier (PID) of process or None if it has yet to start

name
pid

Return identifier (PID) of process or None if it has yet to start

sentinel

Return a file descriptor (Unix) or handle (Windows) suitable for waiting for process termination.

Methods

close(self)

Close the Process object.

isAlive(self)

Return whether the thread is alive.

is_alive(self)

Return whether process is alive

join(self[, timeout])

Wait until child process terminates

kill(self)

Terminate process; sends SIGKILL signal or uses TerminateProcess()

run(self)

Method to be run in sub-process; can be overridden in sub-class

start(self)

Start child process

terminate(self)

Terminate process; sends SIGTERM signal or uses TerminateProcess()

delay_calc

getName

get_threads

isDaemon

setDaemon

setName

worker

get_threads(self, ListThreads)
run(self)

Method to be run in sub-process; can be overridden in sub-class

worker(self)
class processing.top_block

Bases: processing.top_flow

All signal processing blocks in the flowgraph should be connected by this class. It manages for the correct flow connection of the graph from the source set to the sink set. It manages the IOs between the blocks.

Methods

add_Qs

connect

cpu_trace

flow_start

flow_stop

get_A

get_cpu_usage

handle_A

mem_available

mem_used

num_cores

print_arcs

print_nodes

set_A

throughput_calc

update_A

add_Qs(self, blck1: processing.base_processing, port1, blck2: processing.base_processing, port2)
connect(self, block1, port1: int, block2, port2: int)
cpu_trace(self)
print_arcs(self)
print_nodes(self)
throughput_calc(self)
update_A(self, A_)
class processing.top_flow

Bases: processing.base_processing

The signal processing flowgraph parent class which handles the signal flow in the graph.

Methods

connect

flow_start

flow_stop

get_A

get_cpu_usage

handle_A

mem_available

mem_used

num_cores

set_A

connect(self)
flow_start(self)
flow_stop(self)
get_A(self)
get_cpu_usage(self)
handle_A(self)
mem_available(self)
mem_used(self)
num_cores(self)
set_A(self, A_)
class processing.tp_sink_block(BlockName, NPortsIn, NPortsOut)

Bases: processing.thread_blocks

This is used to take data from threads in a process created it and relay them to other processes. # TODO: complete the class definition

Attributes
daemon

A boolean value indicating whether this thread is a daemon thread.

ident

Thread identifier of this thread or None if it has not been started.

name

A string used for identification purposes only.

Methods

isAlive(self)

Return whether the thread is alive.

is_alive(self)

Return whether the thread is alive.

join(self[, timeout])

Wait until the thread terminates.

run(self)

Method representing the thread’s activity.

start(self)

Start the thread’s activity.

delay_calc

getName

isDaemon

setDaemon

setName

worker

run(self)

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

worker(self, \*ins)
class processing.tp_source_block(ProcName, NPortsIn, NPortsOut)

Bases: processing.thread_blocks

A source for threaded process which takes a list of input multiprocess.Queue() to collect data from other processes and pass them down to the threads in the process by output queue.Queue(). In fact, this is a bridge between processes to this thread in order to get data from other processes and relay them to this thread which created this block.

Attributes
daemon

A boolean value indicating whether this thread is a daemon thread.

ident

Thread identifier of this thread or None if it has not been started.

name

A string used for identification purposes only.

Methods

isAlive(self)

Return whether the thread is alive.

is_alive(self)

Return whether the thread is alive.

join(self[, timeout])

Wait until the thread terminates.

run(self)

Method representing the thread’s activity.

start(self)

Start the thread’s activity.

delay_calc

getName

isDaemon

setDaemon

setName

worker

run(self)

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

worker(self, \*ins)