Processing module¶

This module contians all processing classes derived from the base class. The base class has mechanisms to handle the data passing between each processing block.

Tracking UPDATES:¶

Date	Description
Jul-17-2019	`t_general_block` is added and the output queues are added in `add_Qs`.

class processing.base_processing¶

Bases: object

The abstract class for all signal processing ecosystem.

class processing.p_general_block(BlockName, NInPorts, NOutPorts)¶

Bases: processing.process_blocks

The block used to process the input data stream from other processes and deliver processed data to other processes. TODO: check if it is possible to collect and distribute the incoming and outgoing streams from and to multiple processes.

the problem mightb be passing multiple Queues to the process from different processes.

Attributes

authkey
daemon: Return whether process is a daemon
exitcode: Return exit code of process or None if it has yet to stop
ident: Return identifier (PID) of process or None if it has yet to start
name
pid: Return identifier (PID) of process or None if it has yet to start
sentinel: Return a file descriptor (Unix) or handle (Windows) suitable for waiting for process termination.

Methods

`close`(self)	Close the Process object.
`is_alive`(self)	Return whether process is alive
`join`(self[, timeout])	Wait until child process terminates
`kill`(self)	Terminate process; sends SIGKILL signal or uses TerminateProcess()
`run`(self)	Method to be run in sub-process; can be overridden in sub-class
`start`(self)	Start child process
`terminate`(self)	Terminate process; sends SIGTERM signal or uses TerminateProcess()

delay_calc
worker

run(self)¶: Method to be run in sub-process; can be overridden in sub-class

worker(self, \*ins)¶

class processing.p_sink_block(BlockName, NPorts)¶

Bases: processing.process_blocks

Consuming data from other processes.

Attributes

authkey
daemon: Return whether process is a daemon
exitcode: Return exit code of process or None if it has yet to stop
ident: Return identifier (PID) of process or None if it has yet to start
name
pid: Return identifier (PID) of process or None if it has yet to start
sentinel: Return a file descriptor (Unix) or handle (Windows) suitable for waiting for process termination.

Methods

`close`(self)	Close the Process object.
`is_alive`(self)	Return whether process is alive
`join`(self[, timeout])	Wait until child process terminates
`kill`(self)	Terminate process; sends SIGKILL signal or uses TerminateProcess()
`run`(self)	Method to be run in sub-process; can be overridden in sub-class
`start`(self)	Start child process
`terminate`(self)	Terminate process; sends SIGTERM signal or uses TerminateProcess()

delay_calc
worker

run(self)¶: Method to be run in sub-process; can be overridden in sub-class

worker(self, \*ins)¶

class processing.p_source_block(BlockName, NPorts)¶

Bases: processing.process_blocks

The process block source which is passing generated data down to other processes.

Attributes

authkey
daemon: Return whether process is a daemon
exitcode: Return exit code of process or None if it has yet to stop
ident: Return identifier (PID) of process or None if it has yet to start
name
pid: Return identifier (PID) of process or None if it has yet to start
sentinel: Return a file descriptor (Unix) or handle (Windows) suitable for waiting for process termination.

Methods

`close`(self)	Close the Process object.
`is_alive`(self)	Return whether process is alive
`join`(self[, timeout])	Wait until child process terminates
`kill`(self)	Terminate process; sends SIGKILL signal or uses TerminateProcess()
`run`(self)	Method to be run in sub-process; can be overridden in sub-class
`start`(self)	Start child process
`terminate`(self)	Terminate process; sends SIGTERM signal or uses TerminateProcess()

delay_calc
worker

run(self)¶: Method to be run in sub-process; can be overridden in sub-class

worker(self)¶

class processing.process_blocks(name)¶

Bases: processing.base_processing, multiprocessing.context.Process

This is to derive all multiprocessed blocks to achieve full parallelism. TODO: setting daemon returns error:

assert self._popen is None, ‘process has already started’ AttributeError: ‘DCAReader’ object has no attribute ‘_popen’

Attributes

authkey
daemon: Return whether process is a daemon
exitcode: Return exit code of process or None if it has yet to stop
ident: Return identifier (PID) of process or None if it has yet to start
name
pid: Return identifier (PID) of process or None if it has yet to start
sentinel: Return a file descriptor (Unix) or handle (Windows) suitable for waiting for process termination.

Methods

`close`(self)	Close the Process object.
`is_alive`(self)	Return whether process is alive
`join`(self[, timeout])	Wait until child process terminates
`kill`(self)	Terminate process; sends SIGKILL signal or uses TerminateProcess()
`run`(self)	Method to be run in sub-process; can be overridden in sub-class
`start`(self)	Start child process
`terminate`(self)	Terminate process; sends SIGTERM signal or uses TerminateProcess()

delay_calc
worker

delay_calc(self)¶

run(self)¶: Method to be run in sub-process; can be overridden in sub-class

worker(self)¶

class processing.t_general_block(BlockName, NInPorts, NOutPorts)¶

Bases: processing.thread_blocks

All threaded general blocks should be derived from this class. Initializing parameters are:

Parameters

BlockName (str) – name of the processing block
NInPorts (int) – number of output queues
NOutPorts (int) – number of output queues

Attributes

daemon: A boolean value indicating whether this thread is a daemon thread.
ident: Thread identifier of this thread or None if it has not been started.
name: A string used for identification purposes only.

Methods

`isAlive`(self)	Return whether the thread is alive.
`is_alive`(self)	Return whether the thread is alive.
`join`(self[, timeout])	Wait until the thread terminates.
`run`(self)	Method representing the thread’s activity.
`start`(self)	Start the thread’s activity.

delay_calc
getName
isDaemon
setDaemon
setName
worker

run(self)¶

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

worker(self, \*ins)¶

class processing.t_sink_block(BlockName, NInPorts)¶

Bases: processing.thread_blocks

All threaded sink blocks should be derived from this class. Initializing parameters are:

Parameters

BlockName (str) – name of the processing block
NInPorts (int) – number of output queues

Attributes

daemon: A boolean value indicating whether this thread is a daemon thread.
ident: Thread identifier of this thread or None if it has not been started.
name: A string used for identification purposes only.

Methods

`isAlive`(self)	Return whether the thread is alive.
`is_alive`(self)	Return whether the thread is alive.
`join`(self[, timeout])	Wait until the thread terminates.
`run`(self)	Method representing the thread’s activity.
`start`(self)	Start the thread’s activity.

delay_calc
getName
isDaemon
setDaemon
setName
worker

run(self)¶

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

worker(self, \*ins)¶

class processing.t_source_block(BlockName, NPorts)¶

Bases: processing.thread_blocks

All threaded source blocks should be derived from this class. Initializing parameters are:

Parameters

BlockName (str) – Name of the processing block
NQs (int) – Number of output ports

Attributes

daemon: A boolean value indicating whether this thread is a daemon thread.
ident: Thread identifier of this thread or None if it has not been started.
name: A string used for identification purposes only.

Methods

`isAlive`(self)	Return whether the thread is alive.
`is_alive`(self)	Return whether the thread is alive.
`join`(self[, timeout])	Wait until the thread terminates.
`run`(self)	Method representing the thread’s activity.
`start`(self)	Start the thread’s activity.

delay_calc
getName
isDaemon
setDaemon
setName
worker

run(self)¶

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

worker(self)¶

class processing.thread_blocks(name)¶

Bases: processing.base_processing, threading.Thread

This is to derive multithreaded blocks.

Attributes

daemon: A boolean value indicating whether this thread is a daemon thread.
ident: Thread identifier of this thread or None if it has not been started.
name: A string used for identification purposes only.

Methods

`isAlive`(self)	Return whether the thread is alive.
`is_alive`(self)	Return whether the thread is alive.
`join`(self[, timeout])	Wait until the thread terminates.
`run`(self)	Method representing the thread’s activity.
`start`(self)	Start the thread’s activity.

delay_calc
getName
isDaemon
setDaemon
setName
worker

delay_calc(self)¶

run(self)¶

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

worker(self)¶

class processing.threaded_process_blocks(ProcName)¶

Bases: processing.process_blocks, processing.thread_blocks

Multithreaded processes containing a flowgraph taking data from at least one tp_source_block or t_source_block`and deliver outputs to at least one `tp_sink_block or t_sink_block. # TODO: complete the class definition

Attributes

authkey
daemon: Return whether process is a daemon
exitcode: Return exit code of process or None if it has yet to stop
ident: Return identifier (PID) of process or None if it has yet to start
name
pid: Return identifier (PID) of process or None if it has yet to start
sentinel: Return a file descriptor (Unix) or handle (Windows) suitable for waiting for process termination.

Methods

`close`(self)	Close the Process object.
`isAlive`(self)	Return whether the thread is alive.
`is_alive`(self)	Return whether process is alive
`join`(self[, timeout])	Wait until child process terminates
`kill`(self)	Terminate process; sends SIGKILL signal or uses TerminateProcess()
`run`(self)	Method to be run in sub-process; can be overridden in sub-class
`start`(self)	Start child process
`terminate`(self)	Terminate process; sends SIGTERM signal or uses TerminateProcess()

delay_calc
getName
get_threads
isDaemon
setDaemon
setName
worker

get_threads(self, ListThreads)¶

run(self)¶: Method to be run in sub-process; can be overridden in sub-class

worker(self)¶

class processing.top_block¶

Bases: processing.top_flow

All signal processing blocks in the flowgraph should be connected by this class. It manages for the correct flow connection of the graph from the source set to the sink set. It manages the IOs between the blocks.

Methods

add_Qs
connect
cpu_trace
flow_start
flow_stop
get_A
get_cpu_usage
handle_A
mem_available
mem_used
num_cores
print_arcs
print_nodes
set_A
throughput_calc
update_A

add_Qs(self, blck1: processing.base_processing, port1, blck2: processing.base_processing, port2)¶

connect(self, block1, port1: int, block2, port2: int)¶

cpu_trace(self)¶

print_arcs(self)¶

print_nodes(self)¶

throughput_calc(self)¶

update_A(self, A_)¶

class processing.top_flow¶

Bases: processing.base_processing

The signal processing flowgraph parent class which handles the signal flow in the graph.

Methods

connect
flow_start
flow_stop
get_A
get_cpu_usage
handle_A
mem_available
mem_used
num_cores
set_A

connect(self)¶

flow_start(self)¶

flow_stop(self)¶

get_A(self)¶

get_cpu_usage(self)¶

handle_A(self)¶

mem_available(self)¶

mem_used(self)¶

num_cores(self)¶

set_A(self, A_)¶

class processing.tp_sink_block(BlockName, NPortsIn, NPortsOut)¶

Bases: processing.thread_blocks

This is used to take data from threads in a process created it and relay them to other processes. # TODO: complete the class definition

Attributes

daemon: A boolean value indicating whether this thread is a daemon thread.
ident: Thread identifier of this thread or None if it has not been started.
name: A string used for identification purposes only.

Methods

`isAlive`(self)	Return whether the thread is alive.
`is_alive`(self)	Return whether the thread is alive.
`join`(self[, timeout])	Wait until the thread terminates.
`run`(self)	Method representing the thread’s activity.
`start`(self)	Start the thread’s activity.

delay_calc
getName
isDaemon
setDaemon
setName
worker

run(self)¶

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

worker(self, \*ins)¶

class processing.tp_source_block(ProcName, NPortsIn, NPortsOut)¶

Bases: processing.thread_blocks

A source for threaded process which takes a list of input multiprocess.Queue() to collect data from other processes and pass them down to the threads in the process by output queue.Queue(). In fact, this is a bridge between processes to this thread in order to get data from other processes and relay them to this thread which created this block.

Attributes

daemon: A boolean value indicating whether this thread is a daemon thread.
ident: Thread identifier of this thread or None if it has not been started.
name: A string used for identification purposes only.

Methods

`isAlive`(self)	Return whether the thread is alive.
`is_alive`(self)	Return whether the thread is alive.
`join`(self[, timeout])	Wait until the thread terminates.
`run`(self)	Method representing the thread’s activity.
`start`(self)	Start the thread’s activity.

delay_calc
getName
isDaemon
setDaemon
setName
worker

run(self)¶

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

worker(self, \*ins)¶