Welcome to COMPSs!
COMP Superscalar (COMPSs) is a programming model which aims to ease the development of applications for distributed infrastructures, such as Clusters, Grids and Clouds. COMP Superscalar also features a runtime system that exploits the inherent parallelism of applications at execution time.
For the sake of programming productivity, the COMPSs model has four key characteristics:
- Sequential programming: COMPSs programmers do not need to deal with the typical duties of parallelization and distribution, such as thread creation and synchronization, data distribution, messaging or fault tolerance. Instead, the model is based on sequential programming, which makes it appealing to users that either lack parallel programming expertise or are looking for better programmability.
- Infrastructure unaware: COMPSs offers a model that abstracts the application from the underlying distributed infrastructure. Hence, COMPSs programs do not include any detail that could tie them to a particular platform, like deployment or resource management. This makes applications portable between infrastructures with diverse characteristics.
- Standard programming languages: COMPSs is based on the popular programming language Java, but also offers language bindings for Python and C/C++ applications. This facilitates the learning of the model, since programmers can reuse most of their previous knowledge.
- No APIs: In the case of COMPSs applications in Java, the model does not require to use any special API call, pragma or construct in the application; everything is pure standard Java syntax and libraries. With regard the Python and C/C++ bindings, a small set of API calls should be used on the COMPSs applications.
This manual is divided in 9 sections:
What is COMPSs?
COMP Superscalar (COMPSs) is a programming model which aims to ease the development of applications for distributed infrastructures, such as Clusters, Grids and Clouds. COMP Superscalar also features a runtime system that exploits the inherent parallelism of applications at execution time.
For the sake of programming productivity, the COMPSs model has four key characteristics:
- Sequential programming: COMPSs programmers do not need to deal with the typical duties of parallelization and distribution, such as thread creation and synchronization, data distribution, messaging or fault tolerance. Instead, the model is based on sequential programming, which makes it appealing to users that either lack parallel programming expertise or are looking for better programmability.
- Infrastructure unaware: COMPSs offers a model that abstracts the application from the underlying distributed infrastructure. Hence, COMPSs programs do not include any detail that could tie them to a particular platform, like deployment or resource management. This makes applications portable between infrastructures with diverse characteristics.
- Standard programming languages: COMPSs is based on the popular programming language Java, but also offers language bindings for Python and C/C++ applications. This facilitates the learning of the model, since programmers can reuse most of their previous knowledge.
- No APIs: In the case of COMPSs applications in Java, the model does not require to use any special API call, pragma or construct in the application; everything is pure standard Java syntax and libraries. With regard the Python and C/C++ bindings, a small set of API calls should be used on the COMPSs applications.
Quickstart
Install COMPSs
- Choose the installation method:
Pip - Local to the user
JAVA_HOME environment variable points to the Java JDK folder, that the GRADLE_HOME environment variable points to the GRADLE folder, and the gradle binary is in the PATH environment variable.$HOME/.local/ folder (or alternatively within the active virtual environment).$ pip install pycompss -vImportant
Please, update the environment after installing COMPSs:
$ source ~/.bashrc # or alternatively reboot the machineIf installed within a virtual environment, deactivate and activate it to ensure that the environment is propperly updated.
Warning
If using Ubuntu 18.04 or higher, you will need to comment some lines of your
.bashrcand do a complete logout. Please, check the Post installation Section for detailed instructions.
Pip - Systemwide
JAVA_HOME environment variable points to the Java JDK folder, that the GRADLE_HOME environment variable points to the GRADLE folder, and the gradle binary is in the PATH environment variable./usr/lib64/pythonX.Y/site-packages/pycompss/ folder.$ sudo -E pip install pycompss -vImportant
Please, update the environment after installing COMPSs:
$ source /etc/profile.d/compss.sh # or alternatively reboot the machineWarning
If using Ubuntu 18.04 or higher, you will need to comment some lines of your
.bashrcand do a complete logout. Please, check the Post installation Section for detailed instructions.
Build from sources - Local to the user
JAVA_HOME environment variable points to the Java JDK folder, that the GRADLE_HOME environment variable points to the GRADLE folder, and the gradle binary is in the PATH environment variable.$HOME/COMPSs/ folder.$ git clone https://github.com/bsc-wdc/compss.git $ cd compss $ ./submodules_get.sh $ ./submodules_patch.sh $ cd builders/ $ export INSTALL_DIR=$HOME/COMPSs/ $ ./buildlocal ${INSTALL_DIR}
$ ./buildlocal -h
Build from sources - Systemwide
JAVA_HOME environment variable points to the Java JDK folder, that the GRADLE_HOME environment variable points to the GRADLE folder, and the gradle binary is in the PATH environment variable./opt/COMPSs/ folder.$ git clone https://github.com/bsc-wdc/compss.git $ cd compss $ ./submodules_get.sh $ ./submodules_patch.sh $ cd builders/ $ export INSTALL_DIR=/opt/COMPSs/ $ sudo -E ./buildlocal ${INSTALL_DIR}
$ ./buildlocal -h
Supercomputer
Docker - PyCOMPSs Player
pip as follows:$ python3 -m pip install pycompss-player
Tip
Please, check the PyCOMPSs player Installation Section for the further information with regard to the requirements installation and troubleshooting.
Write your first app
Choose your flavour:
Java
Application Overview
A COMPSs application is composed of three parts:
- Main application code: the code that is executed sequentially and contains the calls to the user-selected methods that will be executed by the COMPSs runtime as asynchronous parallel tasks.
- Remote methods code: the implementation of the tasks.
- Task definition interface: It is a Java annotated interface which declares the methods to be run as remote tasks along with metadata information needed by the runtime to properly schedule the tasks.
The main application file name has to be the same of the main class and starts with capital letter, in this case it is Simple.java. The Java annotated interface filename is application name + Itf.java, in this case it is SimpleItf.java. And the code that implements the remote tasks is defined in the application name + Impl.java file, in this case it is SimpleImpl.java.
All code examples are in the /home/compss/tutorial_apps/java/ folder
of the development environment.
Main application code
In COMPSs, the user’s application code is kept unchanged, no API calls need to be included in the main application code in order to run the selected tasks on the nodes.
The COMPSs runtime is in charge of replacing the invocations to the user-selected methods with the creation of remote tasks also taking care of the access to files where required. Let’s consider the Simple application example that takes an integer as input parameter and increases it by one unit.
The main application code of Simple application is shown in the following code block. It is executed sequentially until the call to the increment() method. COMPSs, as mentioned above, replaces the call to this method with the generation of a remote task that will be executed on an available node.
package simple;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import simple.SimpleImpl;
public class Simple {
public static void main(String[] args) {
String counterName = "counter";
int initialValue = args[0];
//--------------------------------------------------------------//
// Creation of the file which will contain the counter variable //
//--------------------------------------------------------------//
try {
FileOutputStream fos = new FileOutputStream(counterName);
fos.write(initialValue);
System.out.println("Initial counter value is " + initialValue);
fos.close();
}catch(IOException ioe) {
ioe.printStackTrace();
}
//----------------------------------------------//
// Execution of the program //
//----------------------------------------------//
SimpleImpl.increment(counterName);
//----------------------------------------------//
// Reading from an object stored in a File //
//----------------------------------------------//
try {
FileInputStream fis = new FileInputStream(counterName);
System.out.println("Final counter value is " + fis.read());
fis.close();
}catch(IOException ioe) {
ioe.printStackTrace();
}
}
}
Remote methods code
The following code contains the implementation of the remote method of the Simple application that will be executed remotely by COMPSs.
package simple;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.FileNotFoundException;
public class SimpleImpl {
public static void increment(String counterFile) {
try{
FileInputStream fis = new FileInputStream(counterFile);
int count = fis.read();
fis.close();
FileOutputStream fos = new FileOutputStream(counterFile);
fos.write(++count);
fos.close();
}catch(FileNotFoundException fnfe){
fnfe.printStackTrace();
}catch(IOException ioe){
ioe.printStackTrace();
}
}
}
Task definition interface
This Java interface is used to declare the methods to be executed remotely along with Java annotations that specify the necessary metadata about the tasks. The metadata can be of three different types:
- For each parameter of a method, the data type (currently File type, primitive types and the String type are supported) and its directions (IN, OUT, INOUT, COMMUTATIVE or CONCURRENT).
- The Java class that contains the code of the method.
- The constraints that a given resource must fulfill to execute the method, such as the number of processors or main memory size.
The task description interface of the Simple app example is shown in the following figure. It includes the description of the Increment() method metadata. The method interface contains a single input parameter, a string containing a path to the file counterFile. In this example there are constraints on the minimum number of processors and minimum memory size needed to run the method.
Interface of the Simple application (SimpleItf.java)package simple; import es.bsc.compss.types.annotations.Constraints; import es.bsc.compss.types.annotations.task.Method; import es.bsc.compss.types.annotations.Parameter; import es.bsc.compss.types.annotations.parameter.Direction; import es.bsc.compss.types.annotations.parameter.Type; public interface SimpleItf { @Constraints(computingUnits = "1", memorySize = "0.3") @Method(declaringClass = "simple.SimpleImpl") void increment( @Parameter(type = Type.FILE, direction = Direction.INOUT) String file ); }
Application compilation
A COMPSs Java application needs to be packaged in a jar file containing the class files of the main code, of the methods implementations and of the Itf annotation. This jar package can be generated using the commands available in the Java SDK or creating your application as a Apache Maven project.
To integrate COMPSs in the maven compile process you just need to add the compss-api artifact as dependency in the application project.
<dependencies>
<dependency>
<groupId>es.bsc.compss</groupId>
<artifactId>compss-api</artifactId>
<version>${compss.version}</version>
</dependency>
</dependencies>
To build the jar in the maven case use the following command
$ mvn package
Next we provide a set of commands to compile the Java Simple application (detailed at Java Sample applications).
$ cd tutorial_apps/java/simple/src/main/java/simple/
$~/tutorial_apps/java/simple/src/main/java/simple$ javac *.java
$~/tutorial_apps/java/simple/src/main/java/simple$ cd ..
$~/tutorial_apps/java/simple/src/main/java$ jar cf simple.jar simple/
$~/tutorial_apps/java/simple/src/main/java$ mv ./simple.jar ../../../jar/
In order to properly compile the code, the CLASSPATH variable has to contain the path of the compss-engine.jar package. The default COMPSs installation automatically add this package to the CLASSPATH; please check that your environment variable CLASSPATH contains the compss-engine.jar location by running the following command:
$ echo $CLASSPATH | grep compss-engine
If the result of the previous command is empty it means that you are missing the compss-engine.jar package in your classpath. We recommend to automatically load the variable by editing the .bashrc file:
$ echo "# COMPSs variables for Java compilation" >> ~/.bashrc
$ echo "export CLASSPATH=$CLASSPATH:/opt/COMPSs/Runtime/compss-engine.jar" >> ~/.bashrc
Application execution
A Java COMPSs application is executed through the runcompss script. An example of an invocation of the script is:
$ runcompss --classpath=/home/compss/tutorial_apps/java/simple/jar/simple.jar simple.Simple 1
A comprehensive description of the runcompss command is available in the Executing COMPSs applications section.
In addition to Java, COMPSs supports the execution of applications written in other languages by means of bindings. A binding manages the interaction of the no-Java application with the COMPSs Java runtime, providing the necessary language translation.
Python
Let’s write your first Python application parallelized with PyCOMPSs.
Consider the following code:
increment.pyimport time
from pycompss.api.api import compss_wait_on
from pycompss.api.task import task
@task(returns=1)
def increment(value):
time.sleep(value * 2) # mimic some computational time
return value + 1
def main():
values = [1, 2, 3, 4]
start = time.time()
for pos in range(len(values)):
values[pos] = increment(values[pos])
values = compss_wait_on(values)
assert values == [2, 3, 4, 5]
print(values)
print("Elapsed time: " + str(time.time() - start_time))
if __name__=='__main__':
main()
This code increments the elements of an array (values) by calling
iteratively to the increment function.
The increment function sleeps the number of seconds indicated by the
value parameter to represent some computational time.
On a normal python execution, each element of the array will be
incremented after the other (sequentially), accumulating the
computational time.
PyCOMPSs is able to parallelize this loop thanks to its @task
decorator, and synchronize the results with the compss_wait_on
API call.
Note
If you are using the PyCOMPSs player (pycompss-player), it is time to deploy the COMPSs environment within your current folder:
$ pycompss init
Please, be aware that the first time needs to download the docker image from the repository, and it may take a while.
Copy and paste the increment code it into increment.py.
Execution
Now let’s execute increment.py. To this end, we will use the
runcompss script provided by COMPSs:
$ runcompss -g increment.py
[Output in next step]
Or alternatively, the pycompss run command if using the PyCOMPSs player
(which wraps the runcompss command and launches it within the COMPSs’ docker
container):
$ pycompss run -g increment.py
[Output in next step]
Note
The -g flag enables the task dependency graph generation (used later).
The runcompss command has a lot of supported options that can be checked with the -h flag.
They can also be used within the pycompss run command.
Tip
It is possible to run also with the python command using the pycompss module,
which accepts the same flags as runcompss:
$ python -m pycompss -g increment.py # Parallel execution
[Output in next step]
Having PyCOMPSs installed also enables to run the same code sequentially without the need of removing the PyCOMPSs syntax.
$ python increment.py # Sequential execution
[2, 3, 4, 5]
Elapsed time: 20.0161030293
Output
$ runcompss -g increment.py
[ INFO] Inferred PYTHON language
[ INFO] Using default location for project file: /opt/COMPSs/Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs/Runtime/configuration/xml/resources/default_resources.xml
[ INFO] Using default execution type: compss
----------------- Executing increment.py --------------------------
WARNING: COMPSs Properties file is null. Setting default values
[(433) API] - Starting COMPSs Runtime v2.7 (build 20200519-1005.r6093e5ac94d67250e097a6fad9d3ec00d676fe6c)
[2, 3, 4, 5]
Elapsed time: 11.5068922043
[(4389) API] - Execution Finished
------------------------------------------------------------
Nice! it run successfully in my 8 core laptop, we have the expected output,
and PyCOMPSs has been able to run the increment.py application in almost half
of the time required by the sequential execution. What happened under the hood?
COMPSs started a master and one worker (by default configured to execute up to four tasks at the same time) and executed the application (offloading the tasks execution to the worker).
Let’s check the task dependency graph to see the parallelism that COMPSs has extracted and taken advantage of.
Task dependency graph
COMPSs stores the generated task dependecy graph within the
$HOME/.COMPSs/<APP_NAME>_<00-99>/monitor directory in dot format.
The generated graph is complete_graph.dot file, which can be
displayed with any dot viewer.
Tip
COMPSs provides the compss_gengraph script which converts the
given dot file into pdf.
$ cd $HOME/.COMPSs/increment.py_01/monitor
$ compss_gengraph complete_graph.dot
$ evince complete_graph.pdf # or use any other pdf viewer you like
It is also available within the PyCOMPSs player:
$ cd $HOME/.COMPSs/increment.py_01/monitor
$ pycompss gengraph complete_graph.dot
$ evince complete_graph.pdf # or use any other pdf viewer you like
And you should see:
The dependency graph of the increment application
COMPSs has detected that the increment of each element is independent,
and consequently, that all of them can be done in parallel. In this
particular application, there are four increment tasks, and since
the worker is able to run four tasks at the same time, all of them can
be executed in parallel saving precious time.
Check the performance
Let’s run it again with the tracing flag enabled:
$ runcompss -t increment.py
[ INFO] Inferred PYTHON language
[ INFO] Using default location for project file: /opt/COMPSs//Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs//Runtime/configuration/xml/resources/default_resources.xml
[ INFO] Using default execution type: compss
----------------- Executing increment.py --------------------------
Welcome to Extrae 3.5.3
[... Extrae prolog ...]
WARNING: COMPSs Properties file is null. Setting default values
[(434) API] - Starting COMPSs Runtime v2.7 (build 20200519-1005.r6093e5ac94d67250e097a6fad9d3ec00d676fe6c)
[2, 3, 4, 5]
Elapsed time: 13.1016821861
[... Extrae eplilog ...]
mpi2prv: Congratulations! ./trace/increment.py_compss_trace_1587562240.prv has been generated.
[(24117) API] - Execution Finished
------------------------------------------------------------
The execution has finished successfully and the trace has been generated
in the $HOME/.COMPSs/<APP_NAME>_<00-99>/trace directory in prv format,
which can be displayed and analysed with PARAVER.
$ cd $HOME/.COMPSs/increment.py_02/trace
$ wxparaver increment.py_compss_trace_*.prv
Note
In the case of using the PyCOMPSs player, the trace will be generated
in the .COMPSs/<APP_NAME>_<00-99>/trace directory:
$ cd .COMPSs/increment.py_02/trace
$ wxparaver increment.py_compss_trace_*.prv
Once Paraver has started, lets visualize the tasks:
- Click in
Fileand then inLoad Configuration - Look for
/PATH/TO/COMPSs/Dependencies/paraver/cfgs/compss_tasks.cfgand clickOpen.
Note
In the case of using the PyCOMPSs player, the configuration files can be obtained by downloading them from the COMPSs repositoy.
And you should see:
Trace of the increment application
The X axis represents the time, and the Y axis the deployed processes
(the first three (1.1.1-1.1.3) belong to the master and the fourth belongs
to the master process in the worker (1.2.1) whose events are
shown with the compss_runtime.cfg configuration file).
The increment tasks are depicted in blue.
We can quickly see that the four increment tasks have been executed in parallel
(one per core), and that their lengths are different (depending on the
computing time of the task represented by the time.sleep(value * 2) line).
Paraver is a very powerful tool for performance analysis. For more information, check the Tracing Section.
Note
If you are using the PyCOMPSs player, it is time to stop the COMPSs environment:
$ pycompss stop
C/C++
Application Overview
As in Java, the application code is divided in 3 parts: the Task definition interface, the main code and task implementations. These files must have the following notation,: <app_ame>.idl, for the interface file, <app_name>.cc for the main code and <app_name>-functions.cc for task implementations. Next paragraphs provide an example of how to define this files for matrix multiplication parallelised by blocks.
Task Definition Interface
As in Java the user has to provide a task selection by means of an interface. In this case the interface file has the same name as the main application file plus the suffix “idl”, i.e. Matmul.idl, where the main file is called Matmul.cc.
interface Matmul
{
// C functions
void initMatrix(inout Matrix matrix,
in int mSize,
in int nSize,
in double val);
void multiplyBlocks(inout Block block1,
inout Block block2,
inout Block block3);
};
The syntax of the interface file is shown in the previous code. Tasks can be declared as classic C function prototypes, this allow to keep the compatibility with standard C applications. In the example, initMatrix and multiplyBlocks are functions declared using its prototype, like in a C header file, but this code is C++ as they have objects as parameters (objects of type Matrix, or Block).
The grammar for the interface file is:
["static"] return-type task-name ( parameter {, parameter }* );
return-type = "void" | type
ask-name = <qualified name of the function or method>
parameter = direction type parameter-name
direction = "in" | "out" | "inout"
type = "char" | "int" | "short" | "long" | "float" | "double" | "boolean" |
"char[<size>]" | "int[<size>]" | "short[<size>]" | "long[<size>]" |
"float[<size>]" | "double[<size>]" | "string" | "File" | class-name
class-name = <qualified name of the class>
Main Program
The following code shows an example of matrix multiplication written in C++.
#include "Matmul.h"
#include "Matrix.h"
#include "Block.h"
int N; //MSIZE
int M; //BSIZE
double val;
int main(int argc, char **argv)
{
Matrix A;
Matrix B;
Matrix C;
N = atoi(argv[1]);
M = atoi(argv[2]);
val = atof(argv[3]);
compss_on();
A = Matrix::init(N,M,val);
initMatrix(&B,N,M,val);
initMatrix(&C,N,M,0.0);
cout << "Waiting for initialization...\n";
compss_wait_on(B);
compss_wait_on(C);
cout << "Initialization ends...\n";
C.multiply(A, B);
compss_off();
return 0;
}
The developer has to take into account the following rules:
- A header file with the same name as the main file must be included, in this case Matmul.h. This header file is automatically generated by the binding and it contains other includes and type-definitions that are required.
- A call to the compss_on binding function is required to turn on the COMPSs runtime.
- As in C language, out or inout parameters should be passed by reference by means of the “&” operator before the parameter name.
- Synchronization on a parameter can be done calling the compss_wait_on binding function. The argument of this function must be the variable or object we want to synchronize.
- There is an implicit synchronization in the init method of Matrix. It is not possible to know the address of “A” before exiting the method call and due to this it is necessary to synchronize before for the copy of the returned value into “A” for it to be correct.
- A call to the compss_off binding function is required to turn off the COMPSs runtime.
Functions file
The implementation of the tasks in a C or C++ program has to be provided in a functions file. Its name must be the same as the main file followed by the suffix “-functions”. In our case Matmul-functions.cc.
#include "Matmul.h"
#include "Matrix.h"
#include "Block.h"
void initMatrix(Matrix *matrix,int mSize,int nSize,double val){
*matrix = Matrix::init(mSize, nSize, val);
}
void multiplyBlocks(Block *block1,Block *block2,Block *block3){
block1->multiply(*block2, *block3);
}
In the previous code, class methods have been encapsulated inside a function. This is useful when the class method returns an object or a value and we want to avoid the explicit synchronization when returning from the method.
Additional source files
Other source files needed by the user application must be placed under the directory “src”. In this directory the programmer must provide a Makefile that compiles such source files in the proper way. When the binding compiles the whole application it will enter into the src directory and execute the Makefile.
It generates two libraries, one for the master application and another for the worker application. The directive COMPSS_MASTER or COMPSS_WORKER must be used in order to compile the source files for each type of library. Both libraries will be copied into the lib directory where the binding will look for them when generating the master and worker applications.
Application Compilation
The user command “compss_build_app” compiles both master and worker for a single architecture (e.g. x86-64, armhf, etc). Thus, whether you want to run your application in Intel based machine or ARM based machine, this command is the tool you need.
When the target is the native architecture, the command to execute is very simple;
$~/matmul_objects> compss_build_app Matmul
[ INFO ] Java libraries are searched in the directory: /usr/lib/jvm/java-1.8.0-openjdk-amd64//jre/lib/amd64/server
[ INFO ] Boost libraries are searched in the directory: /usr/lib/
...
[Info] The target host is: x86_64-linux-gnu
Building application for master...
g++ -g -O3 -I. -I/Bindings/c/share/c_build/worker/files/ -c Block.cc Matrix.cc
ar rvs libmaster.a Block.o Matrix.o
ranlib libmaster.a
Building application for workers...
g++ -DCOMPSS_WORKER -g -O3 -I. -I/Bindings/c/share/c_build/worker/files/ -c Block.cc -o Block.o
g++ -DCOMPSS_WORKER -g -O3 -I. -I/Bindings/c/share/c_build/worker/files/ -c Matrix.cc -o Matrix.o
ar rvs libworker.a Block.o Matrix.o
ranlib libworker.a
...
Command successful.
Application Execution
The following environment variables must be defined before executing a COMPSs C/C++ application:
- JAVA_HOME
- Java JDK installation directory (e.g. /usr/lib/jvm/java-8-openjdk/)
After compiling the application, two directories, master and worker, are generated. The master directory contains a binary called as the main file, which is the master application, in our example is called Matmul. The worker directory contains another binary called as the main file followed by the suffix “-worker”, which is the worker application, in our example is called Matmul-worker.
The runcompss script has to be used to run the application:
$ runcompss /home/compss/tutorial_apps/c/matmul_objects/master/Matmul 3 4 2.0
The complete list of options of the runcompss command is available in Section Executing COMPSs applications.
Task Dependency Graph
COMPSs can generate a task dependency graph from an executed code. It is indicating by a
$ runcompss -g /home/compss/tutorial_apps/c/matmul_objects/master/Matmul 3 4 2.0
The generated task dependency graph is stored within the
$HOME/.COMPSs/<APP_NAME>_<00-99>/monitor directory in dot format.
The generated graph is complete_graph.dot file, which can be
displayed with any dot viewer. COMPSs also provides the compss_gengraph script
which converts the given dot file into pdf.
$ cd $HOME/.COMPSs/Matmul_02/monitor $ compss_gengraph complete_graph.dot $ evince complete_graph.pdf # or use any other pdf viewer you like
The following figure depicts the task dependency graph for the Matmul application in its object version with 3x3 blocks matrices, each one containing a 4x4 matrix of doubles. Each block in the result matrix accumulates three block multiplications, i.e. three multiplications of 4x4 matrices of doubles.
Matmul Execution Graph.
The light blue circle corresponds to the initialization of matrix “A” by means of a method-task and it has an implicit synchronization inside. The dark blue circles correspond to the other two initializations by means of function-tasks; in this case the synchronizations are explicit and must be provided by the developer after the task call. Both implicit and explicit synchronizations are represented as red circles.
Each green circle is a partial matrix multiplication of a set of 3. One block from matrix “A” and the correspondent one from matrix “B”. The result is written in the right block in “C” that accumulates the partial block multiplications. Each multiplication set has an explicit synchronization. All green tasks are method-tasks and they are executed in parallel.
Useful information
Choose your flavour:
Java
- Syntax detailed information -> Java
- Constraint definition -> Constraints
- Execution details -> Executing COMPSs applications
- Graph, tracing and monitoring facilities -> COMPSs Tools
- Other execution environments (Supercomputers, Docker, etc.) -> Supercomputers
- Performance analysis -> Tracing
- Troubleshooting -> Troubleshooting
- Sample applications -> Java Sample applications
- Using COMPSs with persistent storage frameworks (e.g. dataClay, Hecuba) -> Persistent Storage
Python
- Syntax detailed information -> Python Binding
- Constraint definition -> Constraints
- Execution details -> Executing COMPSs applications
- Graph, tracing and monitoring facilities -> COMPSs Tools
- Other execution environments (Supercomputers, Docker, etc.) -> Supercomputers
- Performance analysis -> Tracing
- Troubleshooting -> Troubleshooting
- Sample applications -> Python Sample applications
- Using COMPSs with persistent storage frameworks (e.g. dataClay, Hecuba) -> Persistent Storage
C/C++
- Syntax detailed information -> C/C++ Binding
- Constraint definition -> Constraints
- Execution details -> Executing COMPSs applications
- Graph, tracing and monitoring facilities -> COMPSs Tools
- Other execution environments (Supercomputers, Docker, etc.) -> Supercomputers
- Performance analysis -> Tracing
- Troubleshooting -> Troubleshooting
- Sample applications -> C/C++ Sample applications
Installation and Administration
This section is intended to walk you through the COMPSs installation.
Dependencies
Next we provide a list of dependencies for installing COMPSs package. The exact names may vary depending on the Linux distribution but this list provides a general overview of the COMPSs dependencies. For specific information about your distribution please check the Depends section at your package manager (apt, yum, zypper, etc.).
| Module | Dependencies |
|---|---|
| COMPSs Runtime | openjdk-8-jre, graphviz, xdg-utils, openssh-server |
| COMPSs Python Binding | libtool, automake, build-essential, python (>= 2.7 | >=3.5), python-dev | python3-dev, python-setuptools|python3-setuptools, libpython2.7 |
| COMPSs C/C++ Binding | libtool, automake, build-essential, libboost-all-dev, libxml2-dev |
| COMPSs Autoparallel | libgmp3-dev, flex, bison, libbison-dev, texinfo, libffi-dev, astor, sympy, enum34, islpy |
| COMPSs Tracing | libxml2 (>= 2.5), libxml2-dev (>= 2.5), gfortran, papi |
As an example for some distributions:
Ubuntu 20.04
Ubuntu 20.04 dependencies installation commands:
$ sudo apt-get install -y openjdk-8-jdk graphviz xdg-utils libtool automake build-essential python python-dev libpython2.7 python3 python3-dev libboost-serialization-dev libboost-iostreams-dev libxml2 libxml2-dev csh gfortran libgmp3-dev flex bison texinfo python3-pip libpapi-dev
$ sudo wget https://services.gradle.org/distributions/gradle-5.4.1-bin.zip -O /opt/gradle-5.4.1-bin.zip
$ sudo unzip /opt/gradle-5.4.1-bin.zip -d /opt
Attention
Before installing it is important to have a proper JAVA_HOME environment
variable definition. This variable must contain a valid path to a Java JDK
(as a remark, it must point to a JDK, not JRE).
So, please, export this variable and include it into your .bashrc:
$ echo 'export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/' >> ~/.bashrc
$ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/
Ubuntu 18.04
Ubuntu 18.04 dependencies installation commands:
$ sudo apt-get install -y openjdk-8-jdk graphviz xdg-utils libtool automake build-essential python python-dev libpython2.7 python3 python3-dev libboost-serialization-dev libboost-iostreams-dev libxml2 libxml2-dev csh gfortran libgmp3-dev flex bison texinfo python3-pip libpapi-dev
$ sudo wget https://services.gradle.org/distributions/gradle-5.4.1-bin.zip -O /opt/gradle-5.4.1-bin.zip
$ sudo unzip /opt/gradle-5.4.1-bin.zip -d /opt
Attention
Before installing it is important to have a proper JAVA_HOME environment
variable definition. This variable must contain a valid path to a Java JDK
(as a remark, it must point to a JDK, not JRE).
So, please, export this variable and include it into your .bashrc:
$ echo 'export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/' >> ~/.bashrc
$ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/
Ubuntu 16.04
Ubuntu 16.04 dependencies installation commands:
$ sudo apt-get install -y openjdk-8-jdk graphviz xdg-utils libtool automake build-essential python2.7 libpython2.7 libboost-serialization-dev libboost-iostreams-dev libxml2 libxml2-dev csh gfortran python-pip libpapi-dev
$ sudo wget https://services.gradle.org/distributions/gradle-5.4.1-bin.zip -O /opt/gradle-5.4.1-bin.zip
$ sudo unzip /opt/gradle-5.4.1-bin.zip -d /opt
Attention
Before installing it is important to have a proper JAVA_HOME environment
variable definition. This variable must contain a valid path to a Java JDK
(as a remark, it must point to a JDK, not JRE).
So, please, export this variable and include it into your .bashrc:
$ echo 'export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/' >> ~/.bashrc
$ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/
OpenSuse Tumbleweed
OpenSuse Tumbleweed dependencies installation commands:
$ sudo zypper install --type pattern -y devel_basis
$ sudo zypper install -y java-1_8_0-openjdk-headless java-1_8_0-openjdk java-1_8_0-openjdk-devel graphviz xdg-utils python python-devel python3 python3-devel python3-decorator libtool automake libboost_headers1_71_0-devel libboost_serialization1_71_0 libboost_iostreams1_71_0 libxml2-2 libxml2-devel tcsh gcc-fortran papi libpapi gcc-c++ papi-devel gmp-devel
$ sudo wget https://services.gradle.org/distributions/gradle-5.4.1-bin.zip -O /opt/gradle-5.4.1-bin.zip
$ sudo unzip /opt/gradle-5.4.1-bin.zip -d /opt
Attention
Before installing it is important to have a proper JAVA_HOME environment
variable definition. This variable must contain a valid path to a Java JDK
(as a remark, it must point to a JDK, not JRE).
So, please, export this variable and include it into your .bashrc:
$ echo 'export JAVA_HOME=/usr/lib64/jvm/java-1.8.0-openjdk/' >> ~/.bashrc
$ export JAVA_HOME=/usr/lib64/jvm/java-1.8.0-openjdk/
OpenSuse Leap 15.1
OpenSuse Leap 15.1 dependencies installation commands:
$ sudo zypper install --type pattern -y devel_basis
$ sudo zypper install -y java-1_8_0-openjdk-headless java-1_8_0-openjdk java-1_8_0-openjdk-devel graphviz xdg-utils python python-devel python-decorator python3 python3-devel python3-decorator libtool automake libboost_headers1_66_0-devel libboost_serialization1_66_0 libboost_iostreams1_66_0 libxml2-2 libxml2-devel tcsh gcc-fortran papi libpapi gcc-c++ papi-devel gmp-devel
$ sudo wget https://services.gradle.org/distributions/gradle-5.4.1-bin.zip -O /opt/gradle-5.4.1-bin.zip
$ sudo unzip /opt/gradle-5.4.1-bin.zip -d /opt
Attention
Before installing it is important to have a proper JAVA_HOME environment
variable definition. This variable must contain a valid path to a Java JDK
(as a remark, it must point to a JDK, not JRE).
So, please, export this variable and include it into your .bashrc:
$ echo 'export JAVA_HOME=/usr/lib64/jvm/java-1.8.0-openjdk/' >> ~/.bashrc
$ export JAVA_HOME=/usr/lib64/jvm/java-1.8.0-openjdk/
OpenSuse 42.2
OpenSuse 42.2 dependencies installation commands:
$ sudo zypper install --type pattern -y devel_basis
$ sudo zypper install -y java-1_8_0-openjdk-headless java-1_8_0-openjdk java-1_8_0-openjdk-devel graphviz xdg-utils python python-devel libpython2_7-1_0 python-decorator libtool automake boost-devel libboost_serialization1_54_0 libboost_iostreams1_54_0 libxml2-2 libxml2-devel tcsh gcc-fortran python-pip papi libpapi gcc-c++ papi-devel gmp-devel
$ sudo wget https://services.gradle.org/distributions/gradle-5.4.1-bin.zip -O /opt/gradle-5.4.1-bin.zip
$ sudo unzip /opt/gradle-5.4.1-bin.zip -d /opt
Warning
OpenSuse provides Python 3.4 from its repositories, which is not supported
by the COMPSs python binding.
Please, update Python 3 (python and python-devel) to a higher
version if you expect to install COMPSs from sources.
Alternatively, you can use a virtual environment.
Attention
Before installing it is important to have a proper JAVA_HOME environment
variable definition. This variable must contain a valid path to a Java JDK
(as a remark, it must point to a JDK, not JRE).
So, please, export this variable and include it into your .bashrc:
$ echo 'export JAVA_HOME=/usr/lib64/jvm/java-1.8.0-openjdk/' >> ~/.bashrc
$ export JAVA_HOME=/usr/lib64/jvm/java-1.8.0-openjdk/
Fedora 32
Fedora 32 dependencies installation commands:
$ sudo dnf install -y java-1.8.0-openjdk java-1.8.0-openjdk-devel graphviz xdg-utils libtool automake python27 python3 python3-devel boost-devel boost-serialization boost-iostreams libxml2 libxml2-devel gcc gcc-c++ gcc-gfortran tcsh @development-tools bison flex texinfo papi papi-devel gmp-devel
$ # If the libxml softlink is not created during the installation of libxml2, the COMPSs installation may fail.
$ # In this case, the softlink has to be created manually with the following command:
$ sudo ln -s /usr/include/libxml2/libxml/ /usr/include/libxml
$ sudo wget https://services.gradle.org/distributions/gradle-5.4.1-bin.zip -O /opt/gradle-5.4.1-bin.zip
$ sudo unzip /opt/gradle-5.4.1-bin.zip -d /opt
Attention
Before installing it is important to have a proper JAVA_HOME environment
variable definition. This variable must contain a valid path to a Java JDK
(as a remark, it must point to a JDK, not JRE).
So, please, export this variable and include it into your .bashrc:
$ echo 'export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk/' >> ~/.bashrc
$ export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk/
Fedora 25
Fedora 25 dependencies installation commands:
$ sudo dnf install -y java-1.8.0-openjdk java-1.8.0-openjdk-devel graphviz xdg-utils libtool automake python python-libs python-pip python-devel python2-decorator boost-devel boost-serialization boost-iostreams libxml2 libxml2-devel gcc gcc-c++ gcc-gfortran tcsh @development-tools redhat-rpm-config papi
$ # If the libxml softlink is not created during the installation of libxml2, the COMPSs installation may fail.
$ # In this case, the softlink has to be created manually with the following command:
$ sudo ln -s /usr/include/libxml2/libxml/ /usr/include/libxml
$ sudo wget https://services.gradle.org/distributions/gradle-5.4.1-bin.zip -O /opt/gradle-5.4.1-bin.zip
$ sudo unzip /opt/gradle-5.4.1-bin.zip -d /opt
Attention
Before installing it is important to have a proper JAVA_HOME environment
variable definition. This variable must contain a valid path to a Java JDK
(as a remark, it must point to a JDK, not JRE).
So, please, export this variable and include it into your .bashrc:
$ echo 'export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk/' >> ~/.bashrc
$ export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk/
Debian 8
Debian 8 dependencies installation commands:
$ su -
$ echo "deb http://ppa.launchpad.net/webupd8team/java/ubuntu xenial main" | tee /etc/apt/sources.list.d/webupd8team-java.list
$ echo "deb-src http://ppa.launchpad.net/webupd8team/java/ubuntu xenial main" | tee -a /etc/apt/sources.list.d/webupd8team-java.list
$ apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys EEA14886
$ apt-get update
$ apt-get install oracle-java8-installer
$ apt-get install graphviz xdg-utils libtool automake build-essential python python-decorator python-pip python-dev libboost-serialization1.55.0 libboost-iostreams1.55.0 libxml2 libxml2-dev libboost-dev csh gfortran papi-tools
$ wget https://services.gradle.org/distributions/gradle-5.4.1-bin.zip -O /opt/gradle-5.4.1-bin.zip
$ unzip /opt/gradle-5.4.1-bin.zip -d /opt
Attention
Before installing it is important to have a proper JAVA_HOME environment
variable definition. This variable must contain a valid path to a Java JDK
(as a remark, it must point to a JDK, not JRE). A possible value is the following:
$ echo $JAVA_HOME
/usr/lib64/jvm/java-openjdk/
So, please, check its location, export this variable and include it into your .bashrc
if it is not already available with the previous command.
$ echo 'export JAVA_HOME=/usr/lib64/jvm/java-openjdk/' >> ~/.bashrc
$ export JAVA_HOME=/usr/lib64/jvm/java-openjdk/
CentOS 7
CentOS 7 dependencies installation commands:
$ sudo rpm -iUvh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
$ sudo yum -y update
$ sudo yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel graphviz xdg-utils libtool automake python python-libs python-pip python-devel python2-decorator boost-devel boost-serialization boost-iostreams libxml2 libxml2-devel gcc gcc-c++ gcc-gfortran tcsh @development-tools redhat-rpm-config papi
$ sudo pip install decorator
Attention
Before installing it is important to have a proper JAVA_HOME environment
variable definition. This variable must contain a valid path to a Java JDK
(as a remark, it must point to a JDK, not JRE). A possible value is the following:
$ echo $JAVA_HOME
/usr/lib64/jvm/java-openjdk/
So, please, check its location, export this variable and include it into your .bashrc
if it is not already available with the previous command.
$ echo 'export JAVA_HOME=/usr/lib64/jvm/java-openjdk/' >> ~/.bashrc
$ export JAVA_HOME=/usr/lib64/jvm/java-openjdk/
Attention
Before installing it is also necessary to export the GRADLE_HOME environment
variable and include its binaries path into the PATH environment variable:
$ echo 'export GRADLE_HOME=/opt/gradle-5.4.1' >> ~/.bashrc
$ export GRADLE_HOME=/opt/gradle-5.4.1
$ echo 'export PATH=/opt/gradle-5.4.1/bin:$PATH' >> ~/.bashrc
$ export PATH=/opt/gradle-5.4.1/bin:$PATH
Build Dependencies
To build COMPSs from sources you will also need wget, git and maven.
To install with Pip, pip for the target Python version is required.
Optional Dependencies
For the Python binding it is also recommended to have dill and
guppy/guppy3 installed.
The dill package increases the variety of serializable objects by Python
(for example: lambda functions), and the guppy/guppy3 package is needed to use the
@local decorator. Both packages can be found in pyPI and can be installed via pip.
Building from sources
This section describes the steps to install COMPSs from the sources.
The first step is downloading the source code from the Git repository.
$ git clone https://github.com/bsc-wdc/compss.git
$ cd compss
Then, you need to download the embedded dependencies from the git submodules.
$ compss> ./submodules_get.sh
$ compss> ./submodules_patch.sh
Finally you just need to run the installation script. You have two options:
For all users
For installing COMPSs for all users run the following command:
$ compss> cd builders/
$ builders> export INSTALL_DIR=/opt/COMPSs/
$ builders> sudo -E ./buildlocal ${INSTALL_DIR}
Attention
Root access is required.
For the current user
For installing COMPSs for the current user run the following commands:
$ compss> cd builders/
$ builders> INSTALL_DIR=$HOME/opt/COMPSs/
$ builders> ./buildlocal ${INSTALL_DIR}
Tip
The buildlocal script allows to disable the installation of
components. The options can be foun in the command help:
$ compss> cd builders/
$ builders> ./buildlocal -h
Usage: ./buildlocal [options] targetDir
* Options:
--help, -h Print this help message
--opts Show available options
--version, -v Print COMPSs version
--monitor, -m Enable Monitor installation
--no-monitor, -M Disable Monitor installation
Default: true
--bindings, -b Enable bindings installation
--no-bindings, -B Disable bindings installation
Default: true
--pycompss, -p Enable PyCOMPSs installation
--no-pycompss, -P Disable PyCOMPSs installation
Default: true
--tracing, -t Enable tracing system installation
--no-tracing, -T Disable tracing system installation
Default: true
--autoparallel, -a Enable autoparallel module installation
--no-autoparallel, -A Disable autoparallel module installation
Default: true
--kafka, -k Enable Kafka module installation
--no-kafka, -K Disable Kafka module installation
Default: true
--nothing, -N Disable all previous options
Default: unused
--user-exec=<str> Enables a specific user execution for maven compilation
When used the maven install is not cleaned.
Default: false
--skip-tests Disables MVN unit tests
Default:
* Parameters:
targetDir COMPSs installation directory
Default: /opt/COMPSs
Post installation
Once your COMPSs package has been installed remember to log out and back in again to end the installation process.
Caution
Using Ubuntu version 18.04 or higher requires to comment the following
lines in your .bashrc in order to have the appropriate environment
after logging out and back again (which in these distributions it must be
from the complete system (e.g. gnome) not only from the terminal,
or restart the whole machine).
# If not running interactively, don't do anything
# case $- in #
# *i*) ;; # Comment these lines before logging out
# *) return;; # from the whole gnome (or restart the machine).
# esac #
In addition, COMPSs requires ssh passwordless access. If you need to set up your machine for the first time please take a look at Additional Configuration Section for a detailed description of the additional configuration.
Pip
Pre-requisites
In order to be able to install COMPSs and PyCOMPSs with Pip, the
dependencies (excluding the COMPSs packages) mentioned
in the Dependencies Section must be satisfied (do not forget
to have proper JAVA_HOME and GRADLE_HOME environment variables pointing to the
java JDK folder and Gradle home respectively, as well as the gradle binary in the
PATH environment variable) and Python pip.
Installation
Depending on the machine, the installation command may vary. Some of the possible scenarios and their proper installation command are:
Install systemwide
Install systemwide:
$ sudo -E pip install pycompss -v
Attention
Root access is required.
It is recommended to restart the user session once the installation process has finished. Alternatively, the following command sets all the COMPSs environment in the current session.
$ source /etc/profile.d/compss.sh
Install in user local folder
Install in user home folder (.local):
$ pip install pycompss -v
It is recommended to restart the user session once the installation process has finished. Alternatively, the following command sets all the COMPSs environment.
$ source ~/.bashrc
Within a virtual environment
Within a Python virtual environment:
(virtualenv) $ pip install pycompss -v
In this particular case, the installation includes the necessary variables in the activate script. So, restart the virtual environment in order to set all the COMPSs environment.
Post installation
If you need to set up your machine for the first time please take a look at Additional Configuration Section for a detailed description of the additional configuration.
Supercomputers
The COMPSs Framework can be installed in any Supercomputer by installing its packages as in a normal distribution. The packages are ready to be reallocated so the administrators can choose the right location for the COMPSs installation.
However, if the administrators are not willing to install COMPSs through the packaging system, we also provide a COMPSs zipped file containing a pre-build script to easily install COMPSs. Next subsections provide further information about this process.
Prerequisites
In order to successfully run the installation script some dependencies must be present on the target machine. Administrators must provide the correct installation and environment of the following software:
- Autotools
- BOOST
- Java 8 JRE
The following environment variables must be defined:
- JAVA_HOME
- BOOST_CPPFLAGS
The tracing system can be enhanced with:
- PAPI, which provides support for harware counters
- MPI, which speeds up the tracing merge (and enables it for huge traces)
Installation
To perform the COMPSs Framework installation please execute the following commands:
$ # Check out the last COMPSs release
$ wget http://compss.bsc.es/repo/sc/stable/COMPSs_<version>.tar.gz
$ # Unpackage COMPSs
$ tar -xvzf COMPSs_<version>.tar.gz
$ # Install COMPSs at your preferred target location
$ cd COMPSs
$ ./install <targetDir> [<supercomputer.cfg>]
$ # Clean downloaded files
$ rm -r COMPSs
$ rm COMPSs_<version>.tar.gz
The installation script will create a COMPSs folder inside the given
<targetDir> so the final COMPSs installation will be placed under
the <targetDir>/COMPSs folder.
Attention
If the <targetDir>/COMPSs folder already exists it will be automatically erased.
After completing the previous steps, administrators must ensure that the nodes have passwordless ssh access. If it is not the case, please contact the COMPSs team at support-compss@bsc.es.
The COMPSs package also provides a compssenv file that loads the
required environment to allow users work more easily with COMPSs. Thus,
after the installation process we recomend to source the
<targetDir>/COMPSs/compssenv into the users .bashrc.
Once done, remember to log out and back in again to end the installation process.
Configuration
To maintain the portability between different environments, COMPSs has a
pre-built structure of scripts to execute applications in Supercomputers.
For this purpose, users must use the enqueue_compss script provided in the
COMPSs installation and specify the supercomputer configuration with
--sc_cfg flag.
When installing COMPSs for a supercomputer, system administrators must define
a configuration file for the specific Supercomputer parameters.
This document gives and overview about how to modify the configuration files
in order to customize the enqueue_compss for a specific queue system and
supercomputer.
As overview, the easier way to proceed when creating a new configuration is to
modify one of the configurations provided by COMPSs. System sdministrators can
find configurations for LSF, SLURM, PBS and SGE as well as
several examples for Supercomputer configurations in
<installation_dir>/Runtime/scripts/queues.
For instance, the configuration for the MareNostrum IV Supercomputer and the
Slurm queue system, can be used as base file for new supercomputer and queue
system cfgs. Sysadmins can modify these files by changing the flags,
parameters, paths and default values that corresponds to your supercomputer.
Once, the files have been modified, they must be copied to the queues folder
to make them available to the users. The following paragraph describe more
in detail the scripts and configuration files
If you need help, contact support-compss@bsc.es.
COMPSs Queue structure overview
All the scripts and cfg files shown in Figure 4 are located
in the <installation_dir>/Runtime/scripts/ folder.
enqueue_compss and launch_compss (launch.sh in the figure) are in
the user subfolder and submit.sh and the cfgs are located in queues.
There are two types of cfg files: the queue system cfg files, which are
located in queues/queue_systems; and the supercomputers cfg files, which
are located in queues/supercomputers.
Structure of COMPSs queue scripts. In Blue user scripts, in Green queue scripts and in Orange system dependant scripts
Configuration Files
The cfg files contain a set of bash variables which are used by the other scripts. On the one hand, the queue system cfgs contain the variables to indicate the commands used by the system to submit and spawn processes, the commands or variables to get the allocated nodes and the directives to indicate the number of nodes, processes, etc. Below you can see an example of the most important variable definition for Slurm
# Submission command (submit.sh)
SUBMISSION_CMD="sbatch"
SUBMISSION_PIPE="< "
...
# Variables to define the directives as #${QUEUE_CMD} ${ARG_*}${QUEUE_SEPARATOR}value (submit.sh)
QUEUE_CMD="SBATCH"
QUEUE_SEPARATOR=""
QARG_JOB_NAME="--job-name="
QARG_JOB_OUT="-o"
QARG_JOB_ERROR="-e"
QARG_WD="--workdir="
QARG_WALLCLOCK="-t"
QARG_NUM_NODES="-N"
QARG_NUM_PROCESSES="-n"
...
#vars to customize the commands know job id and allocated nodes (submit.sh)
ENV_VAR_JOB_ID="SLURM_JOB_ID"
ENV_VAR_NODE_LIST="SLURM_JOB_NODELIST"
HOSTLIST_CMD="scontrol show hostname"
HOSTLIST_TREATMENT="| awk {' print \$1 '} | sed -e 's/\.[^\ ]*//g'"
...
#vars to customize worker process spawn inside the job (launch_compss)
LAUNCH_CMD="srun"
LAUNCH_PARAMS="-n1 -N1 --nodelist="
LAUNCH_SEPARATOR=""
CMD_SEPARATOR=""
To adapt this script to your queue system, you just need to change the variable value to the command, argument or value required in your system. If you find that some of this variables are not available in your system, leave it empty.
On the other hand, the supercomputers cfg files contains a set of variables to indicate the queue system used by a supercomputer, paths where the shared disk is mounted, the default values that COMPSs will set in the project and resources files when they are not set by the user and flags to indicate if a functionality is available or not in a supercomputer. The following lines show examples of this variables for the MareNostrum IV supercomputer.
QUEUE_SYSTEM="slurm"
# Default values enqueue_compss
DEFAULT_EXEC_TIME=10
DEFAULT_NUM_NODES=2
DEFAULT_QUEUE=default
DEFAULT_CPUS_PER_NODE=48
DEFAULT_NODE_MEMORY_SIZE=92
DEFAULT_MASTER_WORKING_DIR=.
MINIMUM_NUM_NODES=1
MINIMUM_CPUS_PER_NODE=1
...
# Enabling/disabling queue system features
DISABLE_QARG_MEMORY=true
DISABLE_QARG_CONSTRAINTS=false
DISABLE_QARG_QOS=false
DISABLE_QARG_OVERCOMMIT=true
DISABLE_QARG_CPUS_PER_TASK=false
HETEROGENEOUS_MULTIJOB=false
...
#Paths
SCRATCH_DIR="/scratch/tmp"
GPFS_PREFIX="/gpfs/"
...
#Other values
REMOTE_EXECUTOR="none" #disable the ssh spawn at runtime
NETWORK_INFINIBAND_SUFFIX="-ib0" #hostname suffix to add in order to use infiniband
NETWORK_DATA_SUFFIX="-data" #hostname suffix to add in order to use infiniband
MASTER_NAME_CMD=hostname #command to know the mastername
To adapt this script to your supercomputer, you just need to change the variables to commands paths or values which are set in your system. If you find that some of this values are not available in your system, leave them empty or as they are in the MareNostrum IV.
How are cfg files used in scripts?
The submit.sh is in charge of getting some of the arguments from
enqueue_compss, generating the a temporal job submission script for the
queue_system (function create_normal_tmp_submit) and performing the
submission in the scheduler (function submit).
The functions used in submit.sh are implemented in common.sh.
If you look at the code of this script, you will see that most of the code is
customized by a set of bash vars which are mainly defined in the cfg files.
For instance the submit command is customized in the following way:
eval ${SUBMISSION_CMD} ${SUBMISSION_PIPE}${TMP_SUBMIT_SCRIPT}
Where ${SUBMISSION_CMD} and ${SUBMISSION_PIPE} are defined in the
queue_system.cfg. So, for the case of Slurm, at execution time it is
translated to something like sbatch < /tmp/tmp_submit_script
The same approach is used for the queue system directives defined in the submission script or in the command to get the assigned host list.
The following lines show the examples in these cases.
#${QUEUE_CMD} ${QARG_JOB_NAME}${QUEUE_SEPARATOR}${job_name}
In the case of Slurm in MN, it generates something like #SBATCH --job-name=COMPSs
host_list=\$(${HOSTLIST_CMD} \$${ENV_VAR_NODE_LIST}${env_var_suffix} ${HOSTLIST_TREATMENT})
The same approach is used in the launch_compss script where it is using
the defined vars to customize the project.xml and resources.xml file
generation and spawning the master and worker processes in the assigned resources.
At first, you should not need to modify any script. The goal of the cfg files is that sysadmins just require to modify the supercomputers cfg, and in the case that the used queue system is not in the queue_systems, folder it should create a new one for the new one.
If you think that some of the features of your system are not supported in the current implementation, please contact us at support-compss@bsc.es. We will discuss how it should be incorporated in the scripts.
Post installation
To check that COMPSs Framework has been successfully installed you may run:
$ # Check the COMPSs version
$ runcompss -v
COMPSs version <version>
For queue system executions, COMPSs provides several prebuild queue scripts than can be accessible throgh the enqueue_compss command. Users can check the available options by running:
$ enqueue_compss -h
Usage: /apps/COMPSs/2.7/Runtime/scripts/user/enqueue_compss [queue_system_options] [COMPSs_options] application_name application_arguments
* Options:
General:
--help, -h Print this help message
--heterogeneous Indicates submission is going to be heterogeneous
Default: Disabled
Queue system configuration:
--sc_cfg=<name> SuperComputer configuration file to use. Must exist inside queues/cfgs/
Default: default
Submission configuration:
General submision arguments:
--exec_time=<minutes> Expected execution time of the application (in minutes)
Default: 10
--job_name=<name> Job name
Default: COMPSs
--queue=<name> Queue name to submit the job. Depends on the queue system.
For example (MN3): bsc_cs | bsc_debug | debug | interactive
Default: default
--reservation=<name> Reservation to use when submitting the job.
Default: disabled
--constraints=<constraints> Constraints to pass to queue system.
Default: disabled
--qos=<qos> Quality of Service to pass to the queue system.
Default: default
--cpus_per_task Number of cpus per task the queue system must allocate per task.
Note that this will be equal to the cpus_per_node in a worker node and
equal to the worker_in_master_cpus in a master node respectively.
Default: false
--job_dependency=<jobID> Postpone job execution until the job dependency has ended.
Default: None
--storage_home=<string> Root installation dir of the storage implementation
Default: null
--storage_props=<string> Absolute path of the storage properties file
Mandatory if storage_home is defined
Normal submission arguments:
--num_nodes=<int> Number of nodes to use
Default: 2
--num_switches=<int> Maximum number of different switches. Select 0 for no restrictions.
Maximum nodes per switch: 18
Only available for at least 4 nodes.
Default: 0
--agents=<string> Hierarchy of agents for the deployment. Accepted values: plain|tree
Default: tree
--agents Deploys the runtime as agents instead of the classic Master-Worker deployment.
Default: disabled
Heterogeneous submission arguments:
--type_cfg=<file_location> Location of the file with the descriptions of node type requests
File should follow the following format:
type_X(){
cpus_per_node=24
node_memory=96
...
}
type_Y(){
...
}
--master=<master_node_type> Node type for the master
(Node type descriptions are provided in the --type_cfg flag)
--workers=type_X:nodes,type_Y:nodes Node type and number of nodes per type for the workers
(Node type descriptions are provided in the --type_cfg flag)
Launch configuration:
--cpus_per_node=<int> Available CPU computing units on each node
Default: 48
--gpus_per_node=<int> Available GPU computing units on each node
Default: 0
--fpgas_per_node=<int> Available FPGA computing units on each node
Default: 0
--io_executors=<int> Number of IO executors on each node
Default: 0
--fpga_reprogram="<string> Specify the full command that needs to be executed to reprogram the FPGA with
the desired bitstream. The location must be an absolute path.
Default:
--max_tasks_per_node=<int> Maximum number of simultaneous tasks running on a node
Default: -1
--node_memory=<MB> Maximum node memory: disabled | <int> (MB)
Default: disabled
--node_storage_bandwidth=<MB> Maximum node storage bandwidth: <int> (MB)
Default: 450
--network=<name> Communication network for transfers: default | ethernet | infiniband | data.
Default: infiniband
--prolog="<string>" Task to execute before launching COMPSs (Notice the quotes)
If the task has arguments split them by "," rather than spaces.
This argument can appear multiple times for more than one prolog action
Default: Empty
--epilog="<string>" Task to execute after executing the COMPSs application (Notice the quotes)
If the task has arguments split them by "," rather than spaces.
This argument can appear multiple times for more than one epilog action
Default: Empty
--master_working_dir=<path> Working directory of the application
Default: .
--worker_working_dir=<name | path> Worker directory. Use: scratch | gpfs | <path>
Default: scratch
--worker_in_master_cpus=<int> Maximum number of CPU computing units that the master node can run as worker. Cannot exceed cpus_per_node.
Default: 24
--worker_in_master_memory=<int> MB Maximum memory in master node assigned to the worker. Cannot exceed the node_memory.
Mandatory if worker_in_master_cpus is specified.
Default: 50000
--worker_port_range=<min>,<max> Port range used by the NIO adaptor at the worker side
Default: 43001,43005
--jvm_worker_in_master_opts="<string>" Extra options for the JVM of the COMPSs Worker in the Master Node.
Each option separed by "," and without blank spaces (Notice the quotes)
Default:
--container_image=<path> Runs the application by means of a container engine image
Default: Empty
--container_compss_path=<path> Path where compss is installed in the container image
Default: /opt/COMPSs
--container_opts="<string>" Options to pass to the container engine
Default: empty
--elasticity=<max_extra_nodes> Activate elasticity specifiying the maximum extra nodes (ONLY AVAILABLE FORM SLURM CLUSTERS WITH NIO ADAPTOR)
Default: 0
--automatic_scaling=<bool> Enable or disable the runtime automatic scaling (for elasticity)
Default: true
--jupyter_notebook=<path>, Swap the COMPSs master initialization with jupyter notebook from the specified path.
--jupyter_notebook Default: false
Runcompss configuration:
Tools enablers:
--graph=<bool>, --graph, -g Generation of the complete graph (true/false)
When no value is provided it is set to true
Default: false
--tracing=<level>, --tracing, -t Set generation of traces and/or tracing level ( [ true | basic ] | advanced | scorep | arm-map | arm-ddt | false)
True and basic levels will produce the same traces.
When no value is provided it is set to 1
Default: 0
--monitoring=<int>, --monitoring, -m Period between monitoring samples (milliseconds)
When no value is provided it is set to 2000
Default: 0
--external_debugger=<int>,
--external_debugger Enables external debugger connection on the specified port (or 9999 if empty)
Default: false
--jmx_port=<int> Enable JVM profiling on specified port
Runtime configuration options:
--task_execution=<compss|storage> Task execution under COMPSs or Storage.
Default: compss
--storage_impl=<string> Path to an storage implementation. Shortcut to setting pypath and classpath. See Runtime/storage in your installation folder.
--storage_conf=<path> Path to the storage configuration file
Default: null
--project=<path> Path to the project XML file
Default: /apps/COMPSs/2.7//Runtime/configuration/xml/projects/default_project.xml
--resources=<path> Path to the resources XML file
Default: /apps/COMPSs/2.7//Runtime/configuration/xml/resources/default_resources.xml
--lang=<name> Language of the application (java/c/python)
Default: Inferred is possible. Otherwise: java
--summary Displays a task execution summary at the end of the application execution
Default: false
--log_level=<level>, --debug, -d Set the debug level: off | info | debug
Warning: Off level compiles with -O2 option disabling asserts and __debug__
Default: off
Advanced options:
--extrae_config_file=<path> Sets a custom extrae config file. Must be in a shared disk between all COMPSs workers.
Default: null
--trace_label=<string> Add a label in the generated trace file. Only used in the case of tracing is activated.
Default: None
--comm=<ClassName> Class that implements the adaptor for communications
Supported adaptors:
├── es.bsc.compss.nio.master.NIOAdaptor
└── es.bsc.compss.gat.master.GATAdaptor
Default: es.bsc.compss.nio.master.NIOAdaptor
--conn=<className> Class that implements the runtime connector for the cloud
Supported connectors:
├── es.bsc.compss.connectors.DefaultSSHConnector
└── es.bsc.compss.connectors.DefaultNoSSHConnector
Default: es.bsc.compss.connectors.DefaultSSHConnector
--streaming=<type> Enable the streaming mode for the given type.
Supported types: FILES, OBJECTS, PSCOS, ALL, NONE
Default: NONE
--streaming_master_name=<str> Use an specific streaming master node name.
Default: null
--streaming_master_port=<int> Use an specific port for the streaming master.
Default: null
--scheduler=<className> Class that implements the Scheduler for COMPSs
Supported schedulers:
├── es.bsc.compss.scheduler.data.DataScheduler
├── es.bsc.compss.scheduler.fifo.FIFOScheduler
├── es.bsc.compss.scheduler.fifodata.FIFODataScheduler
├── es.bsc.compss.scheduler.lifo.LIFOScheduler
├── es.bsc.compss.components.impl.TaskScheduler
└── es.bsc.compss.scheduler.loadbalancing.LoadBalancingScheduler
Default: es.bsc.compss.scheduler.loadbalancing.LoadBalancingScheduler
--scheduler_config_file=<path> Path to the file which contains the scheduler configuration.
Default: Empty
--library_path=<path> Non-standard directories to search for libraries (e.g. Java JVM library, Python library, C binding library)
Default: Working Directory
--classpath=<path> Path for the application classes / modules
Default: Working Directory
--appdir=<path> Path for the application class folder.
Default: /home/group/user
--pythonpath=<path> Additional folders or paths to add to the PYTHONPATH
Default: /home/group/user
--base_log_dir=<path> Base directory to store COMPSs log files (a .COMPSs/ folder will be created inside this location)
Default: User home
--specific_log_dir=<path> Use a specific directory to store COMPSs log files (no sandbox is created)
Warning: Overwrites --base_log_dir option
Default: Disabled
--uuid=<int> Preset an application UUID
Default: Automatic random generation
--master_name=<string> Hostname of the node to run the COMPSs master
Default:
--master_port=<int> Port to run the COMPSs master communications.
Only for NIO adaptor
Default: [43000,44000]
--jvm_master_opts="<string>" Extra options for the COMPSs Master JVM. Each option separed by "," and without blank spaces (Notice the quotes)
Default:
--jvm_workers_opts="<string>" Extra options for the COMPSs Workers JVMs. Each option separed by "," and without blank spaces (Notice the quotes)
Default: -Xms1024m,-Xmx1024m,-Xmn400m
--cpu_affinity="<string>" Sets the CPU affinity for the workers
Supported options: disabled, automatic, user defined map of the form "0-8/9,10,11/12-14,15,16"
Default: automatic
--gpu_affinity="<string>" Sets the GPU affinity for the workers
Supported options: disabled, automatic, user defined map of the form "0-8/9,10,11/12-14,15,16"
Default: automatic
--fpga_affinity="<string>" Sets the FPGA affinity for the workers
Supported options: disabled, automatic, user defined map of the form "0-8/9,10,11/12-14,15,16"
Default: automatic
--fpga_reprogram="<string>" Specify the full command that needs to be executed to reprogram the FPGA with the desired bitstream. The location must be an absolute path.
Default:
--io_executors=<int> IO Executors per worker
Default: 0
--task_count=<int> Only for C/Python Bindings. Maximum number of different functions/methods, invoked from the application, that have been selected as tasks
Default: 50
--input_profile=<path> Path to the file which stores the input application profile
Default: Empty
--output_profile=<path> Path to the file to store the application profile at the end of the execution
Default: Empty
--PyObject_serialize=<bool> Only for Python Binding. Enable the object serialization to string when possible (true/false).
Default: false
--persistent_worker_c=<bool> Only for C Binding. Enable the persistent worker in c (true/false).
Default: false
--enable_external_adaptation=<bool> Enable external adaptation. This option will disable the Resource Optimizer.
Default: false
--gen_coredump Enable master coredump generation
Default: false
--python_interpreter=<string> Python interpreter to use (python/python2/python3).
Default: python Version: 3
--python_propagate_virtual_environment=<true> Propagate the master virtual environment to the workers (true/false).
Default: true
--python_mpi_worker=<false> Use MPI to run the python worker instead of multiprocessing. (true/false).
Default: false
* Application name:
For Java applications: Fully qualified name of the application
For C applications: Path to the master binary
For Python applications: Path to the .py file containing the main program
* Application arguments:
Command line arguments to pass to the application. Can be empty.
If none of the pre-build queue configurations adapts to your infrastructure (lsf, pbs, slurm, etc.) please contact the COMPSs team at support-compss@bsc.es to find out a solution.
If you are willing to test the COMPSs Framework installation you can run any of the applications available at our application repository http://compss.bsc.es/projects/bar. We suggest to run the java simple application following the steps listed inside its README file.
For further information about either the installation or the usage please check the README file inside the COMPSs package.
Additional Configuration
Configure SSH passwordless
By default, COMPSs uses SSH libraries for communication between nodes. Consequently, after COMPSs is installed on a set of machines, the SSH keys must be configured on those machines so that COMPSs can establish passwordless connections between them. This requires to install the OpenSSH package (if not present already) and follow these steps on each machine:
Generate an SSH key pair
$ ssh-keygen -t rsaDistribute the public key to all the other machines and configure it as authorized
$ # For every other available machine (MACHINE): $ scp ~/.ssh/id_rsa.pub MACHINE:./myRSA.pub $ ssh MACHINE "cat ./myRSA.pub >> ~/.ssh/authorized_keys; rm ./myRSA.pub"
Check that passwordless SSH connections are working fine
$ # For every other available machine (MACHINE): $ ssh MACHINE
For example, considering the cluster shown in Figure 5, users will have to execute the following commands to grant free ssh access between any pair of machines:
me@localhost:~$ ssh-keygen -t id_rsa
# Granting access localhost -> m1.bsc.es
me@localhost:~$ scp ~/.ssh/id_rsa.pub user_m1@m1.bsc.es:./me_localhost.pub
me@localhost:~$ ssh user_m1@m1.bsc.es "cat ./me_localhost.pub >> ~/.ssh/authorized_keys; rm ./me_localhost.pub"
# Granting access localhost -> m2.bsc.es
me@localhost:~$ scp ~/.ssh/id_rsa.pub user_m2@m2.bsc.es:./me_localhost.pub
me@localhost:~$ ssh user_m2@m2.bsc.es "cat ./me_localhost.pub >> ~/.ssh/authorized_keys; rm ./me_localhost.pub"
me@localhost:~$ ssh user_m1@m1.bsc.es
user_m1@m1.bsc.es:~> ssh-keygen -t id_rsa
user_m1@m1.bsc.es:~> exit
# Granting access m1.bsc.es -> localhost
me@localhost:~$ scp user_m1@m1.bsc.es:~/.ssh/id_rsa.pub ~/userm1_m1.pub
me@localhost:~$ cat ~/userm1_m1.pub >> ~/.ssh/authorized_keys
# Granting access m1.bsc.es -> m2.bsc.es
me@localhost:~$ scp ~/userm1_m1.pub user_m2@m2.bsc.es:~/userm1_m1.pub
me@localhost:~$ ssh user_m2@m2.bsc.es "cat ./userm1_m1.pub >> ~/.ssh/authorized_keys; rm ./userm1_m1.pub"
me@localhost:~$ rm ~/userm1_m1.pub
me@localhost:~$ ssh user_m2@m2.bsc.es
user_m2@m2.bsc.es:~> ssh-keygen -t id_rsa
user_m2@m2.bsc.es:~> exit
# Granting access m2.bsc.es -> localhost
me@localhost:~$ scp user_m2@m1.bsc.es:~/.ssh/id_rsa.pub ~/userm2_m2.pub
me@localhost:~$ cat ~/userm2_m2.pub >> ~/.ssh/authorized_keys
# Granting access m2.bsc.es -> m1.bsc.es
me@localhost:~$ scp ~/userm2_m2.pub user_m1@m1.bsc.es:~/userm2_m2.pub
me@localhost:~$ ssh user_m1@m1.bsc.es "cat ./userm2_m2.pub >> ~/.ssh/authorized_keys; rm ./userm2_m2.pub"
me@localhost:~$ rm ~/userm2_m2.pub
Cluster example
Configure the COMPSs Cloud Connectors
This section provides information about the additional configuration needed for some Cloud Connectors.
OCCI (Open Cloud Computing Interface) connector
In order to execute a COMPSs application using cloud resources, the rOCCI (Ruby OCCI) connector [1] has to be configured properly. The connector uses the rOCCI CLI client (upper versions from 4.2.5) which has to be installed in the node where the COMPSs main application runs. The client can be installed following the instructions detailed at http://appdb.egi.eu/store/software/rocci.cli
| [1] | https://appdb.egi.eu/store/software/rocci.cli |
Configuration Files
The COMPSs runtime has two configuration files: resources.xml and
project.xml . These files contain information about the execution
environment and are completely independent from the application.
For each execution users can load the default configuration files or
specify their custom configurations by using, respectively, the
--resources=<absolute_path_to_resources.xml> and the
--project=<absolute_path_to_project.xml> in the runcompss
command. The default files are located in the
/opt/COMPSs/Runtime/configuration/xml/ path.
Next sections describe in detail the resources.xml and the
project.xml files, explaining the available options.
Resources file
The resources file provides information about all the available
resources that can be used for an execution. This file should normally
be managed by the system administrators. Its full definition schema
can be found at /opt/COMPSs/Runtime/configuration/xml/resources/resource_schema.xsd.
For the sake of clarity, users can also check the SVG schema located at
/opt/COMPSs/Runtime/configuration/xml/resources/resource_schema.svg.
This file contains one entry per available resource defining its name and its capabilities. Administrators can define several resource capabilities (see example in the next listing) but we would like to underline the importance of ComputingUnits. This capability represents the number of available cores in the described resource and it is used to schedule the correct number of tasks. Thus, it becomes essential to define it accordingly to the number of cores in the physical resource.
compss@bsc:~$ cat /opt/COMPSs/Runtime/configuration/xml/resources/default_resources.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ResourcesList>
<ComputeNode Name="localhost">
<Processor Name="P1">
<ComputingUnits>4</ComputingUnits>
<Architecture>amd64</Architecture>
<Speed>3.0</Speed>
</Processor>
<Processor Name="P2">
<ComputingUnits>2</ComputingUnits>
</Processor>
<Adaptors>
<Adaptor Name="es.bsc.compss.nio.master.NIOAdaptor">
<SubmissionSystem>
<Interactive/>
</SubmissionSystem>
<Ports>
<MinPort>43001</MinPort>
<MaxPort>43002</MaxPort>
</Ports>
</Adaptor>
</Adaptors>
<Memory>
<Size>16</Size>
</Memory>
<Storage>
<Size>200.0</Size>
</Storage>
<OperatingSystem>
<Type>Linux</Type>
<Distribution>OpenSUSE</Distribution>
</OperatingSystem>
<Software>
<Application>Java</Application>
<Application>Python</Application>
</Software>
</ComputeNode>
</ResourcesList>
Project file
The project file provides information about the resources used in a
specific execution. Consequently, the resources that appear in this file
are a subset of the resources described in the resources.xml file.
This file, that contains one entry per worker, is usually edited by the
users and changes from execution to execution. Its full definition
schema can be found at
/opt/COMPSs/Runtime/configuration/xml/projects/project_schema.xsd.
For the sake of clarity, users can also check the SVG schema located at
/opt/COMPSs/Runtime/configuration/xml/projects/project_schema.xsd.
We emphasize the importance of correctly defining the following entries:
- installDir
- Indicates the path of the COMPSs installation inside the resource (not necessarily the same than in the local machine).
- User
- Indicates the username used to connect via ssh to the resource. This user must have passwordless access to the resource (see Configure SSH passwordless Section). If left empty COMPSs will automatically try to access the resource with the same username as the one that lauches the COMPSs main application.
- LimitOfTasks
- The maximum number of tasks that can be simultaneously scheduled to a resource. Considering that a task can use more than one core of a node, this value must be lower or equal to the number of available cores in the resource.
compss@bsc:~$ cat /opt/COMPSs/Runtime/configuration/xml/projects/default_project.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Project>
<!-- Description for Master Node -->
<MasterNode></MasterNode>
<!--Description for a physical node-->
<ComputeNode Name="localhost">
<InstallDir>/opt/COMPSs/</InstallDir>
<WorkingDir>/tmp/Worker/</WorkingDir>
<Application>
<AppDir>/home/user/apps/</AppDir>
<LibraryPath>/usr/lib/</LibraryPath>
<Classpath>/home/user/apps/jar/example.jar</Classpath>
<Pythonpath>/home/user/apps/</Pythonpath>
</Application>
<LimitOfTasks>4</LimitOfTasks>
<Adaptors>
<Adaptor Name="es.bsc.compss.nio.master.NIOAdaptor">
<SubmissionSystem>
<Interactive/>
</SubmissionSystem>
<Ports>
<MinPort>43001</MinPort>
<MaxPort>43002</MaxPort>
</Ports>
<User>user</User>
</Adaptor>
</Adaptors>
</ComputeNode>
</Project>
Configuration examples
In the next subsections we provide specific information about the
services, shared disks, cluster and cloud configurations and several
project.xml and resources.xml examples.
Parallel execution on one single process configuration
The most basic execution that COMPSs supports is using no remote workers
and running all the tasks internally within the same process that hosts
the application execution. To enable the parallel execution of the
application, the user needs to set up the runtime and provide a
description of the resources available on the node. For that purpose,
the user describes within the <MasterNode> tag of the
project.xml file the resources in the same way it describes other
nodes’ resources on the using the resources.xml file. Since there is
no inter-process communication, adaptors description is not allowed. In
the following example, the master will manage the execution of tasks on
the MainProcessor CPU of the local node - a quad-core amd64 processor at
3.0GHz - and use up to 16 GB of RAM memory and 200 GB of storage.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Project>
<MasterNode>
<Processor Name="MainProcessor">
<ComputingUnits>4</ComputingUnits>
<Architecture>amd64</Architecture>
<Speed>3.0</Speed>
</Processor>
<Memory>
<Size>16</Size>
</Memory>
<Storage>
<Size>200.0</Size>
</Storage>
</MasterNode>
</Project>
If no other nodes are available, the list of resources on the
resources.xml file is empty as shown in the following file sample.
Otherwise, the user can define other nodes besides the master node as
described in the following section, and the runtime system will
orchestrate the task execution on both the local process and on the
configured remote nodes.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ResourcesList>
</ResourcesList>
Cluster and grid configuration (static resources)
In order to use external resources to execute the applications, the following steps have to be followed:
- Install the COMPSs Worker package (or the full COMPSs Framework package) on all the new resources.
- Set SSH passwordless access to the rest of the remote resources.
- Create the WorkingDir directory in the resource (remember this path
because it is needed for the
project.xmlconfiguration). - Manually deploy the application on each node.
The resources.xml and the project.xml files must be configured
accordingly. Here we provide examples about configuration files for Grid
and Cluster environments.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ResourcesList>
<ComputeNode Name="hostname1.domain.es">
<Processor Name="MainProcessor">
<ComputingUnits>4</ComputingUnits>
</Processor>
<Adaptors>
<Adaptor Name="es.bsc.compss.nio.master.NIOAdaptor">
<SubmissionSystem>
<Interactive/>
</SubmissionSystem>
<Ports>
<MinPort>43001</MinPort>
<MaxPort>43002</MaxPort>
</Ports>
</Adaptor>
<Adaptor Name="es.bsc.compss.gat.master.GATAdaptor">
<SubmissionSystem>
<Batch>
<Queue>sequential</Queue>
</Batch>
<Interactive/>
</SubmissionSystem>
<BrokerAdaptor>sshtrilead</BrokerAdaptor>
</Adaptor>
</Adaptors>
</ComputeNode>
<ComputeNode Name="hostname2.domain.es">
...
</ComputeNode>
</ResourcesList>
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Project>
<MasterNode/>
<ComputeNode Name="hostname1.domain.es">
<InstallDir>/opt/COMPSs/</InstallDir>
<WorkingDir>/tmp/COMPSsWorker1/</WorkingDir>
<User>user</User>
<LimitOfTasks>2</LimitOfTasks>
</ComputeNode>
<ComputeNode Name="hostname2.domain.es">
...
</ComputeNode>
</Project>
Cloud configuration (dynamic resources)
In order to use cloud resources to execute the applications, the following steps have to be followed:
- Prepare cloud images with the COMPSs Worker package or the full COMPSs Framework package installed.
- The application will be deployed automatically during execution but the users need to set up the configuration files to specify the application files that must be deployed.
The COMPSs runtime communicates with a cloud manager by means of connectors. Each connector implements the interaction of the runtime with a given provider’s API, supporting four basic operations: ask for the price of a certain VM in the provider, get the time needed to create a VM, create a new VM and terminate a VM. This design allows connectors to abstract the runtime from the particular API of each provider and facilitates the addition of new connectors for other providers.
The resources.xml file must contain one or more
<CloudProvider> tags that include the information about a
particular provider, associated to a given connector. The tag must
have an attribute Name to uniquely identify the provider. Next
example summarizes the information to be specified by the user inside
this tag.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ResourcesList>
<CloudProvider Name="PROVIDER_NAME">
<Endpoint>
<Server>https://PROVIDER_URL</Server>
<ConnectorJar>CONNECTOR_JAR</ConnectorJar>
<ConnectorClass>CONNECTOR_CLASS</ConnectorClass>
</Endpoint>
<Images>
<Image Name="Image1">
<Adaptors>
<Adaptor Name="es.bsc.compss.nio.master.NIOAdaptor">
<SubmissionSystem>
<Interactive/>
</SubmissionSystem>
<Ports>
<MinPort>43001</MinPort>
<MaxPort>43010</MaxPort>
</Ports>
</Adaptor>
</Adaptors>
<OperatingSystem>
<Type>Linux</Type>
</OperatingSystem>
<Software>
<Application>Java</Application>
</Software>
<Price>
<TimeUnit>100</TimeUnit>
<PricePerUnit>36.0</PricePerUnit>
</Price>
</Image>
<Image Name="Image2">
<Adaptors>
<Adaptor Name="es.bsc.compss.nio.master.NIOAdaptor">
<SubmissionSystem>
<Interactive/>
</SubmissionSystem>
<Ports>
<MinPort>43001</MinPort>
<MaxPort>43010</MaxPort>
</Ports>
</Adaptor>
</Adaptors>
</Image>
</Images>
<InstanceTypes>
<InstanceType Name="Instance1">
<Processor Name="P1">
<ComputingUnits>4</ComputingUnits>
<Architecture>amd64</Architecture>
<Speed>3.0</Speed>
</Processor>
<Processor Name="P2">
<ComputingUnits>4</ComputingUnits>
</Processor>
<Memory>
<Size>1000.0</Size>
</Memory>
<Storage>
<Size>2000.0</Size>
</Storage>
</InstanceType>
<InstanceType Name="Instance2">
<Processor Name="P1">
<ComputingUnits>4</ComputingUnits>
</Processor>
</InstanceType>
</InstanceTypes>
</CloudProvider>
</ResourcesList>
The project.xml complements the information about a provider listed
in the resources.xml file. This file can contain a <Cloud>
tag where to specify a list of providers, each with a
<CloudProvider> tag, whose name attribute must match one of
the providers in the resources.xml file. Thus, the project.xml
file must contain a subset of the providers specified in the
resources.xml file. Next example summarizes the information to be
specified by the user inside this <Cloud> tag.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Project>
<Cloud>
<InitialVMs>1</InitialVMs>
<MinimumVMs>1</MinimumVMs>
<MaximumVMs>4</MaximumVMs>
<CloudProvider Name="PROVIDER_NAME">
<LimitOfVMs>4</LimitOfVMs>
<Properties>
<Property Context="C1">
<Name>P1</Name>
<Value>V1</Value>
</Property>
<Property>
<Name>P2</Name>
<Value>V2</Value>
</Property>
</Properties>
<Images>
<Image Name="Image1">
<InstallDir>/opt/COMPSs/</InstallDir>
<WorkingDir>/tmp/Worker/</WorkingDir>
<User>user</User>
<Application>
<Pythonpath>/home/user/apps/</Pythonpath>
</Application>
<LimitOfTasks>2</LimitOfTasks>
<Package>
<Source>/home/user/apps/</Source>
<Target>/tmp/Worker/</Target>
<IncludedSoftware>
<Application>Java</Application>
<Application>Python</Application>
</IncludedSoftware>
</Package>
<Package>
<Source>/home/user/apps/</Source>
<Target>/tmp/Worker/</Target>
</Package>
<Adaptors>
<Adaptor Name="es.bsc.compss.nio.master.NIOAdaptor">
<SubmissionSystem>
<Interactive/>
</SubmissionSystem>
<Ports>
<MinPort>43001</MinPort>
<MaxPort>43010</MaxPort>
</Ports>
</Adaptor>
</Adaptors>
</Image>
<Image Name="Image2">
<InstallDir>/opt/COMPSs/</InstallDir>
<WorkingDir>/tmp/Worker/</WorkingDir>
</Image>
</Images>
<InstanceTypes>
<InstanceType Name="Instance1"/>
<InstanceType Name="Instance2"/>
</InstanceTypes>
</CloudProvider>
<CloudProvider Name="PROVIDER_NAME2">
...
</CloudProvider>
</Cloud>
</Project>
For any connector the Runtime is capable to handle the next list of properties:
| Name | Description |
|---|---|
| provider-user | Username to login in the provider |
| provider-user-credential | Credential to login in the provider |
| time-slot | Time slot |
| estimated-creation-time | Estimated VM creation time |
| max-vm-creation-time | Maximum VM creation time |
Additionally, for any connector based on SSH, the Runtime automatically handles the next list of properties:
| Name | Description |
|---|---|
| vm-user | User to login in the VM |
| vm-password | Password to login in the VM |
| vm-keypair-name | Name of the Keypair to login in the VM |
| vm-keypair-location | Location (in the master) of the Keypair to login in the VM |
Finally, the next sections provide a more accurate description of each of the currently available connector and its specific properties.
Cloud connectors: rOCCI
The connector uses the rOCCI binary client [1] (version newer or equal than 4.2.5) which has to be installed in the node where the COMPSs main application is executed.
This connector needs additional files providing details about the
resource templates available on each provider. This file is located
under
<COMPSs_INSTALL_DIR>/configuration/xml/templates path.
Additionally, the user must define the virtual images flavors and
instance types offered by each provider; thus, when the runtime
decides the creation of a VM, the connector selects the appropriate
image and resource template according to the requirements (in terms of
CPU, memory, disk, etc) by invoking the rOCCI client through Mixins
(heritable classes that override and extend the base templates).
Table 4 contains the rOCCI specific properties
that must be defined under the Provider tag in the project.xml
file and Table 5 contains the specific properties
that must be defined under the Instance tag.
| Name | Description |
|---|---|
| auth | Authentication method, x509 only supported |
| user-cred | Path of the VOMS proxy |
| ca-path | Path to CA certificates directory |
| ca-file | Specific CA filename |
| owner | Optional. Used by the PMES Job-Manager |
| jobname | Optional. Used by the PMES Job-Manager |
| timeout | Maximum command time |
| username | Username to connect to the back-end cloud provider |
| password | Password to connect to the back-end cloud provider |
| voms | Enable VOMS authentication |
| media-type | Media type |
| resource | Resource type |
| attributes | Extra resource attributes for the back-end cloud provider |
| context | Extra context for the back-end cloud provider |
| action | Extra actions for the back-end cloud provider |
| mixin | Mixin definition |
| link | Link |
| trigger-action | Adds a trigger |
| log-to | Redirect command logs |
| skip-ca-check | Skips CA checks |
| filter | Filters command output |
| dump-model | Dumps the internal model |
| debug | Enables the debug mode on the connector commands |
| verbose | Enables the verbose mode on the connector commands |
| Instance | Multiple entries of resource templates. |
|---|---|
| Type | Name of the resource template. It has to be the same name than in the previous files |
| CPU | Number of cores |
| Memory | Size in GB of the available RAM |
| Disk | Size in GB of the storage |
| Price | Cost per hour of the instance |
Cloud connectors: JClouds
The JClouds connector is based on the JClouds API version 1.9.1. Table Table 6 shows the extra available options under the Properties tag that are used by this connector.
| Instance | Description |
|---|---|
| provider | Back-end provider to use with JClouds (i.e. aws-ec2) |
Cloud connectors: Docker
This connector uses a Java API client from
https://github.com/docker-java/docker-java, version 3.0.3. It has not
additional options. Make sure that the image/s you want to load are
pulled before running COMPSs with docker pull IMAGE. Otherwise, the
connectorn will throw an exception.
Cloud connectors: Mesos
The connector uses the v0 Java API for Mesos which has to be installed in the node where the COMPSs main application is executed. This connector creates a Mesos framework and it uses Docker images to deploy workers, each one with an own IP address.
By default it does not use authentication and the timeout timers are set to 3 minutes (180.000 milliseconds). The list of optional properties available from connector is shown in Table 7.
| Instance | Description |
|---|---|
| mesos-framework-name | Framework name to show in Mesos. |
| mesos-woker-name | Worker names to show in Mesos. |
| mesos-framework-hostname | Framework hostname to show in Mesos. |
| mesos-checkpoint | Checkpoint for the framework. |
| mesos-authenticate | Uses authentication? (true/false) |
| mesos-principal | Principal for authentication. |
| mesos-secret | Secret for authentication. |
| mesos-framework-register-timeout | Timeout to wait for Framework to register. |
| mesos-framework-register-timeout-units | Time units to wait for register. |
| mesos-worker-wait-timeout | Timeout to wait for worker to be created. |
| mesos-worker-wait-timeout-units | Time units for waiting creation. |
| mesos-worker-kill-timeout | Number of units to wait for killing a worker. |
| mesos-worker-kill-timeout-units | Time units to wait for killing. |
| mesos-docker-command | Command to use at start for each worker. |
| mesos-containerizer | Containers to use: (MESOS/DOCKER) |
| mesos-docker-network-type | Network type to use: (BRIDGE/HOST/USER) |
| mesos-docker-network-name | Network name to use for workers. |
| mesos-docker-mount-volume | Mount volume on workers? (true/false) |
| mesos-docker-volume-host-path | Host path for mounting volume. |
| mesos-docker-volume-container-path | Container path to mount volume. |
TimeUnit avialable values: DAYS, HOURS, MICROSECONDS,
MILLISECONDS, MINUTES, NANOSECONDS, SECONDS.
Services configuration
To allow COMPSs applications to use WebServices as tasks, the
resources.xml can include a special type of resource called
Service. For each WebService it is necessary to specify its wsdl, its
name, its namespace and its port.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ResourcesList>
<ComputeNode Name="localhost">
...
</ComputeNode>
<Service wsdl="http://bscgrid05.bsc.es:20390/hmmerobj/hmmerobj?wsdl">
<Name>HmmerObjects</Name>
<Namespace>http://hmmerobj.worker</Namespace>
<Port>HmmerObjectsPort</Port>
</Service>
</ResourcesList>
When configuring the project.xml file it is necessary to include the
service as a worker by adding an special entry indicating only the name
and the limit of tasks as shown in the following example:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Project>
<MasterNode/>
<ComputeNode Name="localhost">
...
</ComputeNode>
<Service wsdl="http://bscgrid05.bsc.es:20390/hmmerobj/hmmerobj?wsdl">
<LimitOfTasks>2</LimitOfTasks>
</Service>
</Project>
| [1] | https://appdb.egi.eu/store/software/rocci.cli |
Application development
This section is intended to walk you through the development of COMPSs applications.
Java
This section illustrates the steps to develop a Java COMPSs application, to compile and to execute it. The Simple application will be used as reference code. The user is required to select a set of methods, invoked in the sequential application, that will be run as remote tasks on the available resources.
Programming Model
This section shows how the COMPSs programming model is used to develop a Java task-based parallel application for distributed computing. First, We introduce the structure of a COMPSs Java application and with a simple example. Then, we will provide a complete guide about how to define the application tasks. Finally, we will show special API calls and other optimization hints.
Application Overview
A COMPSs application is composed of three parts:
- Main application code: the code that is executed sequentially and contains the calls to the user-selected methods that will be executed by the COMPSs runtime as asynchronous parallel tasks.
- Remote methods code: the implementation of the tasks.
- Task definition interface: It is a Java annotated interface which declares the methods to be run as remote tasks along with metadata information needed by the runtime to properly schedule the tasks.
The main application file name has to be the same of the main class and starts with capital letter, in this case it is Simple.java. The Java annotated interface filename is application name + Itf.java, in this case it is SimpleItf.java. And the code that implements the remote tasks is defined in the application name + Impl.java file, in this case it is SimpleImpl.java.
All code examples are in the /home/compss/tutorial_apps/java/ folder
of the development environment.
Main application code
In COMPSs, the user’s application code is kept unchanged, no API calls need to be included in the main application code in order to run the selected tasks on the nodes.
The COMPSs runtime is in charge of replacing the invocations to the user-selected methods with the creation of remote tasks also taking care of the access to files where required. Let’s consider the Simple application example that takes an integer as input parameter and increases it by one unit.
The main application code of Simple application is shown in the following code block. It is executed sequentially until the call to the increment() method. COMPSs, as mentioned above, replaces the call to this method with the generation of a remote task that will be executed on an available node.
package simple;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import simple.SimpleImpl;
public class Simple {
public static void main(String[] args) {
String counterName = "counter";
int initialValue = args[0];
//--------------------------------------------------------------//
// Creation of the file which will contain the counter variable //
//--------------------------------------------------------------//
try {
FileOutputStream fos = new FileOutputStream(counterName);
fos.write(initialValue);
System.out.println("Initial counter value is " + initialValue);
fos.close();
}catch(IOException ioe) {
ioe.printStackTrace();
}
//----------------------------------------------//
// Execution of the program //
//----------------------------------------------//
SimpleImpl.increment(counterName);
//----------------------------------------------//
// Reading from an object stored in a File //
//----------------------------------------------//
try {
FileInputStream fis = new FileInputStream(counterName);
System.out.println("Final counter value is " + fis.read());
fis.close();
}catch(IOException ioe) {
ioe.printStackTrace();
}
}
}
Remote methods code
The following code contains the implementation of the remote method of the Simple application that will be executed remotely by COMPSs.
package simple;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.FileNotFoundException;
public class SimpleImpl {
public static void increment(String counterFile) {
try{
FileInputStream fis = new FileInputStream(counterFile);
int count = fis.read();
fis.close();
FileOutputStream fos = new FileOutputStream(counterFile);
fos.write(++count);
fos.close();
}catch(FileNotFoundException fnfe){
fnfe.printStackTrace();
}catch(IOException ioe){
ioe.printStackTrace();
}
}
}
Task definition interface
This Java interface is used to declare the methods to be executed remotely along with Java annotations that specify the necessary metadata about the tasks. The metadata can be of three different types:
- For each parameter of a method, the data type (currently File type, primitive types and the String type are supported) and its directions (IN, OUT, INOUT, COMMUTATIVE or CONCURRENT).
- The Java class that contains the code of the method.
- The constraints that a given resource must fulfill to execute the method, such as the number of processors or main memory size.
The task description interface of the Simple app example is shown in the following figure. It includes the description of the Increment() method metadata. The method interface contains a single input parameter, a string containing a path to the file counterFile. In this example there are constraints on the minimum number of processors and minimum memory size needed to run the method.
Interface of the Simple application (SimpleItf.java)package simple; import es.bsc.compss.types.annotations.Constraints; import es.bsc.compss.types.annotations.task.Method; import es.bsc.compss.types.annotations.Parameter; import es.bsc.compss.types.annotations.parameter.Direction; import es.bsc.compss.types.annotations.parameter.Type; public interface SimpleItf { @Constraints(computingUnits = "1", memorySize = "0.3") @Method(declaringClass = "simple.SimpleImpl") void increment( @Parameter(type = Type.FILE, direction = Direction.INOUT) String file ); }
The following sections show a detailed guide of how to implement complex applications.
Task definition reference guide
The task definition interface is a Java annotated interface where developers define tasks as annotated methods in the interfaces. Annotations can be of three different types:
- Task-definition annotations are method annotations to indicate which type of task is a method declared in the interface.
- The Parameter annotation provides metadata about the task parameters, such as data type, direction and other property for runtime optimization.
- The Constraints annotation describes the minimum capabilities that a given resource must fulfill to execute the task, such as the number of processors or main memory size.
- Scheduler hint annotation provides information about how to deal with tasks of this type at scheduling and execution
A complete and detailed explanation of the usage of the metadata includes:
Task-definition Annotations
For each declared methods, developers has to define a task type. The following list enumerates the possible task types:
@Method: Defines the Java method as a task
- declaringClass (Mandatory) String specifying the class that implements the Java method.
- targetDirection This field specifies the direction of the target object of an object method. It can be defined as: INOUT” (default value) if the method modifies the target object, “CONCURRENT” if this object modification can be done concurrently, or “IN” if the method does not modify the target object. ().
- priority “true” if the task takes priority and “false” otherwise. This parameter is used by the COMPSs scheduler (it is a String not a Java boolean).
- onFailure Expected behaviour if the task fails. OnFailure.RETRY (default value) makes the task be executed again, OnFailure.CANCEL_SUCCESSORS ignores the failure and cancels the succesor tasks, OnFailure.FAIL stops the whole application in a save mode once a task fails or OnFailure.IGNORE ignores the failure and continues with normal runtime execution.
@Binary: Defines the Java method as a binary invokation
- binary (Mandatory) String defining the full path of the binary that must be executed.
- workingDir Full path of the binary working directory inside the COMPSs Worker.
- priority “true” if the task takes priority and “false” otherwise. This parameter is used by the COMPSs scheduler (it is a String not a Java boolean).
@MPI: Defines the Java method as a MPI invokation
- mpiRunner (Mandatory) String defining the mpi runner command.
- binary (Mandatory) String defining the full path of the binary that must be executed.
- processes String defining the number of MPI processes spawn in the task execution. This can be combined with the constraints annotation to create define a MPI+OpenMP task. (Default is 1)
- scaleByCU It indicates that the defined processes will be scaled by the defined computingUnits in the constraints. So, the total MPI processes will be processes multiplied by computingUnits. This functionality is used to groups MPI processes per node. Number of groups will be set in processes and the number of processes per node will be indicated by computingUnits
- workingDir Full path of the binary working directory inside the COMPSs Worker.
- priority “true” if the task takes priority and “false” otherwise. This parameter is used by the COMPSs scheduler (it is a String not a Java boolean).
@OmpSs: Defines the Java method as a OmpSs invokation
- binary (Mandatory) String defining the full path of the binary that must be executed.
- workingDir Full path of the binary working directory inside the COMPSs Worker.
- priority “true” if the task takes priority and “false” otherwise. This parameter is used by the COMPSs scheduler (it is a String not a Java boolean).
@Service: Mandatory. It specifies the service properties.
- namespace Mandatory. Service namespace
- name Mandatory. Service name.
- port Mandatory. Service port.
- operation Operation type.
- priority “true” if the service takes priority, “false” otherwise. This parameter is used by the COMPSs scheduler (it is a String not a Java boolean).
Parameter-level annotations
For each parameter of task (method declared in the interface), the user must include a @Parameter annotation. The properties
Direction: Describes how a task uses the parameter (Default is IN).
- Direction.IN: Task only reads the data.
- Direction.INOUT: Task reads and modifies
- Direction.OUT: Task completely modify the data, or previous content or not modified data is not important.
- Direction.COMMUTATIVE: An INOUT usage of the data which can be re-ordered with other executions of the defined task.
- Direction.CONCURRENT: The task allow concurrent modifications of this data. It requires a storage backend that manages concurrent modifications.
Type: Describes the data type of the task parameter. By default, the runtime infers the type according to the Java datatype. However, it is mandatory to define it for files, directories and Streams.
COMPSs supports the following types for task parameters:
- Basic types: To indicate a parameter is a Java primitive type use the follwing types: Type.BOOLEAN, Type.CHAR, Type.BYTE, Type.SHORT, Type.INT, Type.LONG, Type.FLOAT, Type.DOUBLE. They can only have IN direction, since primitive types in Java are always passed by value.
- String: To indicate a parameter is a Java String use Type.STRING. It can only have IN direction, since Java Strings are immutable.
- File: The real Java type associated with a file parameter is a String that contains the path to the file. However, if the user specifies a parameter as Type.FILE, COMPSs will treat it as such. It can have any direction (IN, OUT, INOUT, CONMMUTATIVE or CONCURRENT).
- Directory: The real Java type associated with a directory parameter is a String that contains the path to the directory. However, if the user specifies a parameter as Type.DIRECTORY, COMPSs will treat it as such. It can have any direction (IN, OUT, INOUT, CONMMUTATIVE or CONCURRENT).
- Object: An object parameter is defined with Type.Object. It can have any direction (IN, INOUT, COMMUTATIVE or CONCURRENT).
- Streams: A Task parameters can be defined as stream with Type.STREAM. It can have direction IN, if the task pull data from the stream, or OUT if the task pushes data to the stream.
Return type: Any object or a generic class object. In this case the direction is always OUT. Basic types are also supported as return types. However, we do not recommend to use them because they cause an implicit synchronization
StdIOStream: For non-native tasks (binaries, MPI, and OmpSs) COMPSs supports the automatic redirection of the Linux streams by specifying StdIOStream.STDIN, StdIOStream.STDOUT or StdIOStream.STDERR. Notice that any parameter annotated with the stream annotation must be of type Type.FILE, and with direction Direction.IN for StdIOStream.STDIN or Direction.OUT/ Direction.INOUT for StdIOStream.STDOUT and StdIOStream.STDERR.
Prefix: For non-native tasks (binaries, MPI, and OmpSs) COMPSs allows to prepend a constant String to the parameter value to use the Linux joint-prefixes as parameters of the binary execution.
Weight: Provides a hint of the size of this parameter compared to a default one. For instance, if a parameters is 3 times larger than the others, set the weigh property of this paramenter to 3.0. (Default is 1.0).
keepRename: Runtime rename files to avoid some data dependencies. It is transparent to the final user because we rename back the filename when invoking the task at worker. This management creates an overhead, if developers know that the task is not name nor extension sensitive (i.e can work with rename), they can set this property to true to reduce the overhead.
Constraints annotations
- @Constraints: The user can specify the capabilities that a resource must have in order to run a method. For example, in a cloud execution the COMPSs runtime creates a VM that fulfils the specified requirements in order to perform the execution. A full description of the supported constraints can be found in Table 14.
Scheduler annotations
@SchedulerHints: It specifies hints for the scheduler about how to treat the task.
- isReplicated “true” if the method must be executed in all the worker nodes when invoked from the main application (it is a String not a Java boolean).
- isDistributed “true” if the method must be scheduled in a forced round robin among the available resources (it is a String not a Java boolean).
Alternative method implementations
Since version 1.2, the COMPSs programming model allows developers to define sets of alternative implementations of the same method in the Java annotated interface. Code 10 depicts an example where the developer sorts an integer array using two different methods: merge sort and quick sort that are respectively hosted in the packagepath.Mergesort and packagepath.Quicksort classes.
@Method(declaringClass = "packagepath.Mergesort")
@Method(declaringClass = "packagepath.Quicksort")
void sort(
@Parameter(type = Type.OBJECT, direction = Direction.INOUT)
int[] array
);
As depicted in the example, the name and parameters of all the implementations must coincide; the only difference is the class where the method is implemented. This is reflected in the attribute declaringClass of the @Method annotation. Instead of stating that the method is implemented in a single class, the programmer can define several instances of the @Method annotation with different declaring classes.
As independent remote methods, the sets of equivalent methods might have common restrictions to be fulfilled by the resource hosting the execution. Or even, each implementation can have specific constraints. Through the @Constraints annotation, developers can specify the common constraints for a whole set of methods. In the following example (Code 11) only one core is required to run the method of both sorting algorithms.
@Constraints(computingUnits = "1")
@Method(declaringClass = "packagepath.Mergesort")
@Method(declaringClass = "packagepath.Quicksort")
void sort(
@Parameter(type = Type.OBJECT, direction = Direction.INOUT)
int[] array
);
However, these sorting algorithms have different memory consumption, thus each algorithm might require a specific amount of memory and that should be stated in the implementation constraints. For this purpose, the developer can add a @Constraints annotation inside each @Method annotation containing the specific constraints for that implementation. Since the Mergesort has a higher memory consumption than the quicksort, the Code 12 sets a requirement of 1 core and 2GB of memory for the mergesort implementation and 1 core and 500MB of memory for the quicksort.
@Constraints(computingUnits = "1")
@Method(declaringClass = "packagepath.Mergesort", constraints = @Constraints(memorySize = "2.0"))
@Method(declaringClass = "packagepath.Quicksort", constraints = @Constraints(memorySize = "0.5"))
void sort(
@Parameter(type = Type.OBJECT, direction = Direction.INOUT)
int[] array
);
Java API calls
COMPSs also provides a explicit synchronization call, namely barrier, which can be used through the COMPSs Java API. The use of barrier forces to wait for all tasks that have been submitted before the barrier is called. When all tasks submitted before the barrier have finished, the execution continues (Code 13).
import es.bsc.compss.api.COMPSs;
public class Main {
public static void main(String[] args) {
// Setup counterName1 and counterName2 files
// Execute task increment 1
SimpleImpl.increment(counterName1);
// API Call to wait for all tasks
COMPSs.barrier();
// Execute task increment 2
SimpleImpl.increment(counterName2);
}
}
When an object if used in a task, COMPSs runtime store the references of these object in the runtime data structures and generate replicas and versions in remote workers. COMPSs is automatically removing these replicas for obsolete versions. However, the reference of the last version of these objects could be stored in the runtime data-structures preventing the garbage collector to remove it when there are no references in the main code. To avoid this situation, developers can indicate the runtime that an object is not going to use any more by calling the deregisterObject API call. Code 14 shows a usage example of this API call.
import es.bsc.compss.api.COMPSs;
public class Main {
public static void main(String[] args) {
final int ITERATIONS = 10;
for (int i = 0; i < ITERATIONS; ++i) {
Dummy d = new Dummy(d);
TaskImpl.task(d);
/*Allows garbage collector to delete the
object from memory when the task is finished */
COMPSs.deregisterObject((Object) d);
}
}
}
To synchronize files, the getFile API call synchronizes a file, returning the last version of file with its original name. Code 15 contains an example of its usage.
import es.bsc.compss.api.COMPSs;
public class Main {
public static void main(String[] args) {
for (int i=0; i<1; i++) {
TaskImpl.task(FILE_NAME, i);
}
/*Waits until all tasks have finished and
synchronizes the file with its last version*/
COMPSs.getFile(FILE_NAME);
}
}
Application Compilation
A COMPSs Java application needs to be packaged in a jar file containing the class files of the main code, of the methods implementations and of the Itf annotation. This jar package can be generated using the commands available in the Java SDK or creating your application as a Apache Maven project.
To integrate COMPSs in the maven compile process you just need to add the compss-api artifact as dependency in the application project.
<dependencies>
<dependency>
<groupId>es.bsc.compss</groupId>
<artifactId>compss-api</artifactId>
<version>${compss.version}</version>
</dependency>
</dependencies>
To build the jar in the maven case use the following command
$ mvn package
Next we provide a set of commands to compile the Java Simple application (detailed at Java Sample applications).
$ cd tutorial_apps/java/simple/src/main/java/simple/
$~/tutorial_apps/java/simple/src/main/java/simple$ javac *.java
$~/tutorial_apps/java/simple/src/main/java/simple$ cd ..
$~/tutorial_apps/java/simple/src/main/java$ jar cf simple.jar simple/
$~/tutorial_apps/java/simple/src/main/java$ mv ./simple.jar ../../../jar/
In order to properly compile the code, the CLASSPATH variable has to contain the path of the compss-engine.jar package. The default COMPSs installation automatically add this package to the CLASSPATH; please check that your environment variable CLASSPATH contains the compss-engine.jar location by running the following command:
$ echo $CLASSPATH | grep compss-engine
If the result of the previous command is empty it means that you are missing the compss-engine.jar package in your classpath. We recommend to automatically load the variable by editing the .bashrc file:
$ echo "# COMPSs variables for Java compilation" >> ~/.bashrc
$ echo "export CLASSPATH=$CLASSPATH:/opt/COMPSs/Runtime/compss-engine.jar" >> ~/.bashrc
If you are using an IDE (such as Eclipse or NetBeans) we recommend you
to add the compss-engine.jar file as an external file to the project.
The compss-engine.jar file is available at your current COMPSs
installation under the following path: /opt/COMPSs/Runtime/compss-engine.jar
Please notice that if you have performed a custom installation, the location of the package can be different.
Application Execution
A Java COMPSs application is executed through the runcompss script. An example of an invocation of the script is:
$ runcompss --classpath=/home/compss/tutorial_apps/java/simple/jar/simple.jar simple.Simple 1
A comprehensive description of the runcompss command is available in the Executing COMPSs applications section.
In addition to Java, COMPSs supports the execution of applications written in other languages by means of bindings. A binding manages the interaction of the no-Java application with the COMPSs Java runtime, providing the necessary language translation.
Python Binding
COMPSs features a binding for Python 2 and 3 applications. The next subsections explain how to program a Python application for COMPSs and a brief overview on how to execute it.
Programming Model
Task Selection
As in the case of Java, a COMPSs Python application is a Python sequential program that contains calls to tasks. In particular, the user can select as a task:
- Functions
- Instance methods: methods invoked on objects.
- Class methods: static methods belonging to a class.
The task definition in Python is done by means of Python decorators
instead of an annotated interface. In particular, the user needs to add
a @task decorator that describes the task before the
definition of the function/method.
As an example (Code 16), let us assume that the application calls a function func, which receives a file path (string parameter) and an integer parameter. The code of func updates the file.
def func(file_path, value):
# update the file 'file_path'
def main():
my_file = '/tmp/sample_file.txt'
func(my_file, 1)
if __name__ == '__main__':
main()
In order to select func as a task, the corresponding @task decorator needs to be placed right before the definition of the function, providing some metadata about the parameters of that function. The @task decorator has to be imported from the pycompss library (Code 17).
from pycompss.api.task import task
@task()
def func():
...
Function parameters
The @task decorator does not interfere with the function parameters, Consequently, the user can define the function parameters as normal python functions (Code 18).
@task()
def func(param1, param2):
...
The use of *args and **kwargs as function parameters is supported (Code 19).
@task(returns=int)
def argkwarg_func(*args, **kwargs):
...
And even with other parameters, such as usual parameters and default defined arguments. Code 20 shows an example of a task with two three parameters (whose one of them (’s’) has a default value), *args and **kwargs.
@task(returns=int)
def multiarguments_func(v, w, s = 2, *args, **kwargs):
...
Tasks within classes
Functions within classes can also be declared as tasks as normal functions.
The main difference is the existence of the self parameter which enables
to modify the callee object.
For tasks corresponding to instance methods, by default the task is assumed to modify the callee object (the object on which the method is invoked). The programmer can tell otherwise by setting the target_direction argument of the @task decorator to IN (Code 21).
class MyClass(object):
...
@task(target_direction=IN)
def instance_method(self):
... # self is NOT modified here
Class methods and static methods can also be declared as tasks. The only
requirement is to place the @classmethod or @staticmethod over
the @task decorator (Code 22).
Note that there is no need to use the target_direction flag within the
@task decorator.
@classmethod and @staticmethod tasks exampleclass MyClass(object):
...
@classmethod
@task()
def class_method(cls, a, b, c):
...
@staticmethod
@task(returns=int)
def static_method(a, b, c):
...
Tip
Tasks inheritance and overriding supported!!!
Caution
The objects used as task parameters MUST BE serializable:
- Implement the
__getstate__and__setstate__functions in their classes for those objects that are not automatically serializable.- The classes must not be declared in the same file that contains the main method (
if __name__=='__main__') (known pickle issue).
Important
For instances of user-defined classes, the classes of these objects should have an empty constructor, otherwise the programmer will not be able to invoke task instance methods on those objects (Code 23).
# In file utils.py
from pycompss.api.task import task
class MyClass(object):
def __init__(self): # empty constructor
...
@task()
def yet_another_task(self):
# do something with the self attributes
...
...
# In file main.py
from pycompss.api.task import task
from utils import MyClass
@task(returns=MyClass)
def ret_func():
...
myc = MyClass()
...
return myc
def main():
o = ret_func()
# invoking a task instance method on a future object can only
# be done when an empty constructor is defined in the object's
# class
o.yet_another_task()
if __name__=='__main__':
main()
Task Parameters
The metadata corresponding to a parameter is specified as an argument of
the @task decorator, whose name is the formal parameter’s name and whose
value defines the type and direction of the parameter. The parameter types and
directions can be:
- Types
- Primitive types (integer, long, float, boolean)
- Strings
- Objects (instances of user-defined classes, dictionaries, lists, tuples, complex numbers)
- Files
- Direction
- Read-only (IN - default)
- Read-write (INOUT)
- Write-only (OUT)
- Concurrent (CONCURRENT)
- Conmutative (CONMUTATIVE)
COMPSs is able to automatically infer the parameter type for primitive types, strings and objects, while the user needs to specify it for files. On the other hand, the direction is only mandatory for INOUT and OUT parameters. Thus, when defining the parameter metadata in the @task decorator, the user has the following options:
| PARAMETER | DESCRIPTION |
|---|---|
| IN | The parameter is read-only. The type will be inferred. |
| INOUT | The parameter is read-write. The type will be inferred. |
| OUT | The parameter is write-only. The type will be inferred. |
| CONCURRENT | The parameter is read-write with concurrent access. The type will be inferred. |
| CONMUTATIVE | The parameter is read-write with conmutative access. The type will be inferred. |
| FILE/FILE_IN | The parameter is a file. The direction is assumed to be IN. |
| FILE_INOUT | The parameter is a read-write file. |
| FILE_OUT | The parameter is a write-only file. |
| DIRECTORY_IN | The parameter is a directory and the direction is IN. The directory will be compressed before any transfer amongst nodes. |
| DIRECTORY_INOUT | The parameter is a read-write directory. The directory will be compressed before any transfer amongst nodes. |
| DIRECTORY_OUT | The parameter is a write-only directory. The directory will be compressed before any transfer amongst nodes. |
| FILE_CONCURRENT | The parameter is a concurrent read-write file. |
| FILE_CONMUTATIVE | The parameter is a conmutative read-write file. |
| COLLECTION_IN | The parameter is read-only collection. |
| COLLECTION_INOUT | The parameter is read-write collection. |
| COLLECTION_OUT | The parameter is write-only collection. |
| COLLECTION_FILE/COLLECTION_FILE_IN | The parameter is read-only collection of files. |
| COLLECTION_FILE_INOUT | The parameter is read-write collection of files. |
| COLLECTION_FILE_OUT | The parameter is write-only collection of files. |
Consequently, please note that in the following cases there is no need to include an argument in the @task decorator for a given task parameter:
- Parameters of primitive types (integer, long, float, boolean) and strings: the type of these parameters can be automatically inferred by COMPSs, and their direction is always IN.
- Read-only object parameters: the type of the parameter is automatically inferred, and the direction defaults to IN.
The parameter metadata is available from the pycompss library (Code 24)
from pycompss.api.parameter import *
Continuing with the example, in Code 25 the decorator specifies that func has a parameter called f, of type FILE and INOUT direction. Note how the second parameter, i, does not need to be specified, since its type (integer) and direction (IN) are automatically inferred by COMPSs.
from pycompss.api.task import task # Import @task decorator
from pycompss.api.parameter import * # Import parameter metadata for the @task decorator
@task(f=FILE_INOUT)
def func(f, i):
fd = open(f, 'r+')
...
The user can also define that the access to a parameter is concurrent with CONCURRENT or to a file FILE_CONCURRENT (Code 26). Tasks that share a “CONCURRENT” parameter will be executed in parallel, if any other dependency prevents this. The CONCURRENT direction allows users to have access from multiple tasks to the same object/file during their executions. However, note that COMPSs does not manage the interaction with the objects or files used/modified concurrently. Taking care of the access/modification of the concurrent objects is responsibility of the developer.
from pycompss.api.task import task # Import @task decorator
from pycompss.api.parameter import * # Import parameter metadata for the @task decorator
@task(f=FILE_CONCURRENT)
def func(f, i):
...
Or even, the user can also define that the access to a parameter is conmutative with CONMUTATIVE or to a file FILE_CONMUTATIVE (Code 27). The execution order of tasks that share a “CONMUTATIVE” parameter can be changed by the runtime following the conmutative property.
from pycompss.api.task import task # Import @task decorator
from pycompss.api.parameter import * # Import parameter metadata for the @task decorator
@task(f=FILE_CONMUTATIVE)
def func(f, i):
...
Moreover, it is possible to specify that a parameter is a collection of elements (e.g. list) and its direction (COLLECTION_IN or COLLECTION_INOUT) (Code 28). In this case, the list may contain sub-objects that will be handled automatically by the runtime. It is important to annotate data structures as collections if in other tasks there are accesses to individual elements of these collections as parameters. Without this annotation, the runtime will not be able to identify data dependences between the collections and the individual elements.
from pycompss.api.task import task # Import @task decorator
from pycompss.api.parameter import * # Import parameter metadata for the @task decorator
@task(my_collection=COLLECTION)
def func(my_collection):
for element in my_collection:
...
The sub-objects of the collection can be collections of elements (and recursively). In this case, the runtime also keeps track of all elements contained in all sub-collections. In order to improve the performance, the depth of the sub-objects can be limited through the use of the depth parameter (Code 29)
@task(my_collection={Type:COLLECTION_IN, Depth:2})
def func(my_collection):
for inner_collection in my_collection:
for element in inner_collection:
# The contents of element will not be tracked
...
Other Task Parameters
Task time out
The user is also able to define the time out of a task within the @task decorator
with the time_out=<TIME_IN_SECONDS> hint.
The runtime will cancel the task if the time to execute the task exceeds the time defined by the user.
For example, Code 30 shows how to specify that the unknown_duration_task
maximum duration before canceling (if exceeded) is one hour.
@task(time_out=3600)
def unknown_duration_task(self):
...
Scheduler hints
The programmer can provide hints to the scheduler through specific arguments within the @task decorator.
For instance, the programmer can mark a task as a high-priority task
with the priority argument of the @task decorator (Code 31).
In this way, when the task is free of dependencies, it will be scheduled before
any of the available low-priority (regular) tasks. This functionality is
useful for tasks that are in the critical path of the application’s task
dependency graph.
@task(priority=True)
def func():
...
Moreover, the user can also mark a task as distributed with the is_distributed argument or as replicated with the is_replicated argument (Code 32). When a task is marked with is_distributed=True, the method must be scheduled in a forced round robin among the available resources. On the other hand, when a task is marked with is_replicated=True, the method must be executed in all the worker nodes when invoked from the main application. The default value for these parameters is False.
@task(is_distributed=True)
def func():
...
@task(is_replicated=True)
def func2():
...
On failure task behaviour
In case a task fails, the whole application behaviour can be defined using the on_failure argument (Code 33). It has four possible values: ‘RETRY’, ’CANCEL_SUCCESSORS’, ’FAIL’ and ’IGNORE’. ’RETRY’ is the default behaviour, making the task to be executed again (on the same worker or in another worker if the failure remains). ’CANCEL_SUCCESSORS’ ignores the failed task and cancels the execution of the successor tasks, ’FAIL’ stops the whole execution once a task fails and ’IGNORE’ ignores the failure and continues with the normal execution.
@task(on_failure='CANCEL_SUCCESSORS')
def func():
...
Task Parameters Summary
Table 8 summarizes all arguments that can be found in the @task decorator.
| Argument | Value | |
|---|---|---|
| Formal parameter name | (default: empty) | The parameter is an object or a simple tipe that will be inferred. |
| IN | Read-only parameter, all types. | |
| INOUT | Read-write parameter, all types except file (primitives, strings, objects). | |
| OUT | Write-only parameter, all types except file (primitives, strings, objects). | |
| CONCURRENT | Concurrent read-write parameter, all types except file (primitives, strings, objects). | |
| CONMUTATIVE | Conmutative read-write parameter, all types except file (primitives, strings, objects). | |
| FILE(_IN) | Read-only file parameter. | |
| FILE_INOUT | Read-write file parameter. | |
| FILE_OUT | Write-only file parameter. | |
| FILE_CONCURRENT | Concurrent read-write file parameter. | |
| FILE_CONMUTATIVE | Conmutative read-write file parameter. | |
| DIRECTORY(_IN) | The parameter is a read-only directory. | |
| DIRECTORY_INOUT | The parameter is a read-write directory. | |
| DIRECTORY_OUT | the parameter is a write-only directory. | |
| COLLECTION(_IN) | Read-only collection parameter (list). | |
| COLLECTION_INOUT | Read-write collection parameter (list). | |
| COLLECTION_OUT | Read-only collection parameter (list). | |
| COLLECTION_FILE(_IN) | Read-only collection of files parameter (list of files). | |
| COLLECTION_FILE_INOUT | Read-write collection of files parameter (list of files). | |
| COLLECTION_FILE_OUT | Read-only collection of files parameter (list opf files). | |
| Dictionary: {Type:(empty=object)/FILE/COLLECTION, Direction:(empty=IN)/IN/INOUT/OUT/CONCURRENT} | ||
| returns | int (for integer and boolean), long, float, str, dict, list, tuple, user-defined classes | |
| target_direction | INOUT (default), IN or CONCURRENT | |
| priority | True or False (default) | |
| is_distributed | True or False (default) | |
| is_replicated | True or False (default) | |
| on_failure | ’RETRY’ (default), ’CANCEL_SUCCESSORS’, ’FAIL’ or ’IGNORE’ | |
| time_out | int (time in seconds) | |
Task Return
If the function or method returns a value, the programmer can use the returns argument within the @task decorator. In this argument, the programmer can specify the type of that value (Code 34).
@task(returns=int)
def ret_func():
return 1
Moreover, if the function or method returns more than one value, the programmer can specify how many and their type in the returns argument. Code 35 shows how to specify that two values (an integer and a list) are returned.
@task(returns=(int, list))
def ret_func():
return 1, [2, 3]
Alternatively, the user can specify the number of return statements as an integer value (Code 36). This way of specifying the amount of return eases the returns definition since the user does not need to specify explicitly the type of the return arguments. However, it must be considered that the type of the object returned when the task is invoked will be a future object. This consideration may lead to an error if the user expects to invoke a task defined within an object returned by a previous task. In this scenario, the solution is to specify explicitly the return type.
@task(returns=1)
def ret_func():
return "my_string"
@task(returns=2)
def ret_func():
return 1, [2, 3]
Important
If the programmer selects as a task a function or method that returns a value, that value is not generated until the task executes (Code 37).
@task(return=MyClass)
def ret_func():
return MyClass(...)
...
if __name__=='__main__':
o = ret_func() # o is a future object
The object returned can be involved in a subsequent task call, and the COMPSs runtime will automatically find the corresponding data dependency. In the following example, the object o is passed as a parameter and callee of two subsequent (asynchronous) tasks, respectively (Code 38).
if __name__=='__main__':
# o is a future object
o = ret_func()
...
another_task(o)
...
o.yet_another_task()
Tip
PyCOMPSs is able to infer if the task returns something and its amount in most cases. Consequently, the user can specify the task without returns argument. But this is discouraged since it requires code analysis, including an overhead that can be avoided by using the returns argument.
Tip
PyCOMPSs is compatible with Python 3 type hinting. So, if type hinting is present in the code, PyCOMPSs is able to detect the return type and use it (there is no need to use the returns):
@task()
def ret_func() -> str:
return "my_string"
@task()
def ret_func() -> (int, list):
return 1, [2, 3]
Other task types
In addition to this API functions, the programmer can use a set of decorators for other purposes.
For instance, there is a set of decorators that can be placed over the @task decorator in order to define the task methods as a binary invocation (with the Binary decorator), as a OmpSs invocation (with the OmpSs decorator), as a MPI invocation (with the MPI decorator), as a COMPSs application (with the COMPSs decorator), or as a task that requires multiple nodes (with the Multinode decorator). These decorators must be placed over the @task decorator, and under the @constraint decorator if defined.
Consequently, the task body will be empty and the function parameters will be used as invocation parameters with some extra information that can be provided within the @task decorator.
The following subparagraphs describe their usage.
Binary decorator
The @binary decorator shall be used to define that a task is going to invoke a binary executable.
In this context, the @task decorator parameters will be used as the binary invocation parameters (following their order in the function definition). Since the invocation parameters can be of different nature, information on their type can be provided through the @task decorator.
Code 40 shows the most simple binary task definition without/with constraints (without parameters); please note that @constraint decorator has to be provided on top of the others.
from pycompss.api.task import task
from pycompss.api.binary import binary
@binary(binary="mybinary.bin")
@task()
def binary_func():
pass
@constraint(computingUnits="2")
@binary(binary="otherbinary.bin")
@task()
def binary_func2():
pass
The invocation of these tasks would be equivalent to:
$ ./mybinary.bin
$ ./otherbinary.bin # in resources that respect the constraint.
The @binary decorator supports the working_dir parameter to define
the working directory for the execution of the defined binary.
Code 41 shows a more complex binary invocation, with files as parameters:
from pycompss.api.task import task
from pycompss.api.binary import binary
from pycompss.api.parameter import *
@binary(binary="grep", working_dir=".")
@task(infile={Type:FILE_IN_STDIN}, result={Type:FILE_OUT_STDOUT})
def grepper():
pass
# This task definition is equivalent to the folloowing, which is more verbose:
@binary(binary="grep", working_dir=".")
@task(infile={Type:FILE_IN, StdIOStream:STDIN}, result={Type:FILE_OUT, StdIOStream:STDOUT})
def grepper(keyword, infile, result):
pass
if __name__=='__main__':
infile = "infile.txt"
outfile = "outfile.txt"
grepper("Hi", infile, outfile)
The invocation of the grepper task would be equivalent to:
$ # grep keyword < infile > result
$ grep Hi < infile.txt > outfile.txt
Please note that the keyword parameter is a string, and it is respected as is in the invocation call.
Thus, PyCOMPSs can also deal with prefixes for the given parameters. Code 42 performs a system call (ls) with specific prefixes:
from pycompss.api.task import task
from pycompss.api.binary import binary
from pycompss.api.parameter import *
@binary(binary="ls")
@task(hide={Type:FILE_IN, Prefix:"--hide="}, sort={Prefix:"--sort="})
def myLs(flag, hide, sort):
pass
if __name__=='__main__':
flag = '-l'
hideFile = "fileToHide.txt"
sort = "time"
myLs(flag, hideFile, sort)
The invocation of the myLs task would be equivalent to:
$ # ls -l --hide=hide --sort=sort
$ ls -l --hide=fileToHide.txt --sort=time
This particular case is intended to show all the power of the @binary decorator in conjuntion with the @task decorator. Please note that although the hide parameter is used as a prefix for the binary invocation, the fileToHide.txt would also be transfered to the worker (if necessary) since its type is defined as FILE_IN. This feature enables to build more complex binary invocations.
In addition, the @binary decorator also supports the fail_by_exit_value
parameter to define the failure of the task by the exit value of the binary
(Code 43).
It accepts a boolean (True to consider the task failed if the exit value is
not 0, or False to ignore the failure by the exit value (default)), or
a string to determine the environment variable that defines the fail by
exit value (as boolean).
The default behaviour (fail_by_exit_value=False) allows users to receive
the exit value of the binary as the task return value, and take the
necessary decissions based on this value.
fail_by_exit_value@binary(binary="mybinary.bin", fail_by_exit_value=True)
@task()
def binary_func():
pass
OmpSs decorator
The @ompss decorator shall be used to define that a task is going to invoke a OmpSs executable (Code 44).
from pycompss.api.ompss import ompss
@ompss(binary="ompssApp.bin")
@task()
def ompss_func():
pass
The OmpSs executable invocation can also be enriched with parameters, files and prefixes as with the @binary decorator through the function parameters and @task decorator information. Please, check Binary decorator for more details.
MPI decorator
The @mpi decorator shall be used to define that a task is going to invoke a MPI executable (Code 45).
from pycompss.api.mpi import mpi
@mpi(binary="mpiApp.bin", runner="mpirun", computing_nodes=2)
@task()
def mpi_func():
pass
The MPI executable invocation can also be enriched with parameters, files and prefixes as with the @binary decorator through the function parameters and @task decorator information. Please, check Binary decorator for more details.
COMPSs decorator
The @compss decorator shall be used to define that a task is going to be a COMPSs application (Code 46). It enables to have nested PyCOMPSs/COMPSs applications.
from pycompss.api.compss import compss
@compss(runcompss="${RUNCOMPSS}", flags="-d",
app_name="/path/to/simple_compss_nested.py", computing_nodes="2")
@task()
def compss_func():
pass
The COMPSs application invocation can also be enriched with the flags accepted by the runcompss executable. Please, check execution manual for more details about the supported flags.
Multinode decorator
The @multinode decorator shall be used to define that a task is going to use multiple nodes (e.g. using internal parallelism) (Code 47).
from pycompss.api.multinode import multinode
@multinode(computing_nodes="2")
@task()
def multinode_func():
pass
The only supported parameter is computing_nodes, used to define the number of nodes required by the task (the default value is 1). The mechanism to get the number of nodes, threads and their names to the task is through the COMPSS_NUM_NODES, COMPSS_NUM_THREADS and COMPSS_HOSTNAMES environment variables respectively, which are exported within the task scope by the COMPSs runtime before the task execution.
Other task types summary
Next tables summarizes the parameters of these decorators.
- @binary
Parameter Description binary (Mandatory) String defining the full path of the binary that must be executed. working_dir Full path of the binary working directory inside the COMPSs Worker.
- @ompss
Parameter Description binary (Mandatory) String defining the full path of the binary that must be executed. working_dir Full path of the binary working directory inside the COMPSs Worker.
- @mpi
Parameter Description binary (Mandatory) String defining the full path of the binary that must be executed. working_dir Full path of the binary working directory inside the COMPSs Worker. runner (Mandatory) String defining the MPI runner command. computing_nodes Integer defining the number of computing nodes reserved for the MPI execution (only a single node is reserved by default).
- @compss
Parameter Description runcompss (Mandatory) String defining the full path of the runcompss binary that must be executed. flags String defining the flags needed for the runcompss execution. app_name (Mandatory) String defining the application that must be executed. computing_nodes Integer defining the number of computing nodes reserved for the COMPSs execution (only a single node is reserved by default).
- @multinode
Parameter Description computing_nodes Integer defining the number of computing nodes reserved for the task execution (only a single node is reserved by default).
In addition to the parameters that can be used within the
@task decorator, Table 9
summarizes the StdIOStream parameter that can be used within the
@task decorator for the function parameters when using the
@binary, @ompss and @mpi decorators. In
particular, the StdIOStream parameter is used to indicate that a parameter
is going to be considered as a FILE but as a stream (e.g. 
,
and 
in bash) for the @binary,
@ompss and @mpi calls.
| Parameter | Description |
|---|---|
| (default: empty) | Not a stream. |
| STDIN | Standard input. |
| STDOUT | Standard output. |
| STDERR | Standard error. |
Moreover, there are some shorcuts that can be used for files type definition as parameters within the @task decorator (Table 10). It is not necessary to indicate the Direction nor the StdIOStream since it may be already be indicated with the shorcut.
| Alias | Description |
|---|---|
| COLLECTION(_IN) | Type: COLLECTION, Direction: IN |
| COLLECTION_INOUT | Type: COLLECTION, Direction: INOUT |
| COLLECTION_OUT | Type: COLLECTION, Direction: OUT |
| COLLECTION_FILE(_IN) | Type: COLLECTION (File), Direction: IN |
| COLLECTION_FILE_INOUT | Type: COLLECTION (File), Direction: INOUT |
| COLLECTION_FILE_OUT | Type: COLLECTION (File), Direction: OUT |
| FILE(_IN)_STDIN | Type: File, Direction: IN, StdIOStream: STDIN |
| FILE(_IN)_STDOUT | Type: File, Direction: IN, StdIOStream: STDOUT |
| FILE(_IN)_STDERR | Type: File, Direction: IN, StdIOStream: STDERR |
| FILE_OUT_STDIN | Type: File, Direction: OUT, StdIOStream: STDIN |
| FILE_OUT_STDOUT | Type: File, Direction: OUT, StdIOStream: STDOUT |
| FILE_OUT_STDERR | Type: File, Direction: OUT, StdIOStream: STDERR |
| FILE_INOUT_STDIN | Type: File, Direction: INOUT, StdIOStream: STDIN |
| FILE_INOUT_STDOUT | Type: File, Direction: INOUT, StdIOStream: STDOUT |
| FILE_INOUT_STDERR | Type: File, Direction: INOUT, StdIOStream: STDERR |
| FILE_CONCURRENT | Type: File, Direction: CONCURRENT |
| FILE_CONCURRENT_STDIN | Type: File, Direction: CONCURRENT, StdIOStream: STDIN |
| FILE_CONCURRENT_STDOUT | Type: File, Direction: CONCURRENT, StdIOStream: STDOUT |
| FILE_CONCURRENT_STDERR | Type: File, Direction: CONCURRENT, StdIOStream: STDERR |
| FILE_CONMUTATIVE | Type: File, Direction: CONMUTATIVE |
| FILE_CONMUTATIVE_STDIN | Type: File, Direction: CONMUTATIVE, StdIOStream: STDIN |
| FILE_CONMUTATIVE_STDOUT | Type: File, Direction: CONMUTATIVE, StdIOStream: STDOUT |
| FILE_CONMUTATIVE_STDERR | Type: File, Direction: CONMUTATIVE, StdIOStream: STDERR |
These parameter keys, as well as the shortcuts, can be imported from the PyCOMPSs library:
from pycompss.api.parameter import *
Task Constraints
It is possible to define constraints for each task. To this end, the decorator @constraint followed by the desired constraints needs to be placed ON TOP of the @task decorator (Code 48).
Important
Please note the the order of @constraint and @task decorators is important.
from pycompss.api.task import task
from pycompss.api.constraint import constraint
from pycompss.api.parameter import INOUT
@constraint(computing_units="4")
@task(c=INOUT)
def func(a, b, c):
c += a * b
...
This decorator enables the user to set the particular constraints for each task, such as the amount of Cores required explicitly. Alternatively, it is also possible to indicate that the value of a constraint is specified in a environment variable (Code 49). A full description of the supported constraints can be found in Table 14.
For example:
from pycompss.api.task import task
from pycompss.api.constraint import constraint
from pycompss.api.parameter import INOUT
@constraint(computing_units="4",
app_software="numpy,scipy,gnuplot",
memory_size="$MIN_MEM_REQ")
@task(c=INOUT)
def func(a, b, c):
c += a * b
...
Or another example requesting a CPU core and a GPU (Code 50).
from pycompss.api.task import task
from pycompss.api.constraint import constraint
@constraint(processors=[{'processorType':'CPU', 'computingUnits':'1'},
{'processorType':'GPU', 'computingUnits':'1'}])
@task(returns=1)
def func(a, b, c):
...
return result
When the task requests a GPU, COMPSs provides the information about the assigned GPU through the COMPSS_BINDED_GPUS, CUDA_VISIBLE_DEVICES and GPU_DEVICE_ORDINAL environment variables. This information can be gathered from the task code in order to use the GPU.
Please, take into account that in order to respect the constraints, the peculiarities of the infrastructure must be defined in the resources.xml file.
Multiple Task Implementations
As in Java COMPSs applications, it is possible to define multiple implementations for each task. In particular, a programmer can define a task for a particular purpose, and multiple implementations for that task with the same objective, but with different constraints (e.g. specific libraries, hardware, etc). To this end, the @implement decorator followed with the specific implementations constraints (with the @constraint decorator, see Section [subsubsec:constraints]) needs to be placed ON TOP of the @task decorator. Although the user only calls the task that is not decorated with the @implement decorator, when the application is executed in a heterogeneous distributed environment, the runtime will take into account the constraints on each implementation and will try to invoke the implementation that fulfills the constraints within each resource, keeping this management invisible to the user (Code 51).
from pycompss.api.implement import implement
@implement(source_class="sourcemodule", method="main_func")
@constraint(app_software="numpy")
@task(returns=list)
def myfunctionWithNumpy(list1, list2):
# Operate with the lists using numpy
return resultList
@task(returns=list)
def main_func(list1, list2):
# Operate with the lists using built-int functions
return resultList
Please, note that if the implementation is used to define a binary, OmpSs, MPI, COMPSs or multinode task invocation (see Other task types), the @implement decorator must be always on top of the decorators stack, followed by the @constraint decorator, then the @binary/@ompss/@mpi/@compss/@multinode decorator, and finally, the @task decorator in the lowest level.
Main Program
The main program of the application is a sequential code that contains calls to the selected tasks. In addition, when synchronizing for task data from the main program, there exist seven API functions that can to be invoked:
- compss_file_exists(file_name)
- Check if a file exists. If it does not exist, it check if file has been accessed before by calling the runtime.
- compss_open(file_name, mode=’r’)
- Similar to the Python open() call. It synchronizes for the last version of file file_name and returns the file descriptor for that synchronized file. It can have an optional parameter mode, which defaults to ’r’, containing the mode in which the file will be opened (the open modes are analogous to those of Python open()).
- compss_delete_file(file_name)
- Notifies the runtime to delete a file.
- compss_wait_on_file(file_name)
- Synchronizes for the last version of the file file_name. Returns True if success (False otherwise).
- compss_wait_on_directory(directory_name)
- Synchronizes for the last version of the directory directory_name. Returns True if success (False otherwise).
- compss_delete_object(object)
- Notifies the runtime to delete all the associated files to a given object.
- compss_barrier(no_more_tasks=False)
- Performs a explicit synchronization, but does not return any object. The use of compss_barrier() forces to wait for all tasks that have been submitted before the compss_barrier() is called. When all tasks submitted before the compss_barrier() have finished, the execution continues. The no_more_tasks is used to specify if no more tasks are going to be submitted after the compss_barrier().
- compss_barrier_group(group_name)
- Performs a explicit synchronization over the tasks that belong to the group group_name, but does not return any object. The use of compss_barrier_group() forces to wait for all tasks that belong to the given group submitted before the compss_barrier_group() is called. When all group tasks submitted before the compss_barrier_group() have finished, the execution continues. See Group Tasks for more information about task groups.
- compss_wait_on(obj, to_write=True)
- Synchronizes for the last version of object obj and returns the synchronized object. It can have an optional boolean parameter to_write, which defaults to True, that indicates whether the main program will modify the returned object. It is possible to wait on a list of objects. In this particular case, it will synchronize all future objects contained in the list.
- TaskGroup(group_name, implicit_barrier=True)
- Python context to define a group of tasks. All tasks submitted within the context will belong to group_name context and are sensitive to wait for them while the rest are being executed. Tasks groups are depicted within a box into the generated task dependency graph. See Group Tasks for more information about task groups.
To illustrate the use of the aforementioned API functions, the following example (Code 52) first invokes a task func that writes a file, which is later synchronized by calling compss_open(). Later in the program, an object of class MyClass is created and a task method method that modifies the object is invoked on it; the object is then synchronized with compss_wait_on(), so that it can be used in the main program from that point on.
Then, a loop calls again ten times to func task. Afterwards, the barrier performs a synchronization, and the execution of the main user code will not continue until the ten func tasks have finished.
from pycompss.api.api import compss_file_exists
from pycompss.api.api import compss_open
from pycompss.api.api import compss_delete_file
from pycompss.api.api import compss_delete_object
from pycompss.api.api import compss_wait_on
from pycompss.api.api import compss_wait_on_file
from pycompss.api.api import compss_wait_on_directory
from pycompss.api.api import compss_barrier
if __name__=='__main__':
my_file = 'file.txt'
func(my_file)
if compss_file_exists(my_file):
print("Exists")
else:
print("Not exists")
...
fd = compss_open(my_file)
...
my_file2 = 'file2.txt'
func(my_file2)
compss_delete_file(my_file2)
...
my_file3 = 'file3.txt'
func(my_file3)
compss_wait_on_file(my_file3)
...
my_directory = '/tmp/data'
func_dir(my_directory)
compss_wait_on_directory(my_directory)
...
my_obj1 = MyClass()
my_obj1.method()
compss_delete_object(my_obj1)
...
my_obj2 = MyClass()
my_obj2.method()
my_obj2 = compss_wait_on(my_obj2)
...
for i in range(10):
func(str(i) + my_file)
compss_barrier()
...
The corresponding task selection for the example above would be (Code 53):
@task(f=FILE_OUT)
def func(f):
...
class MyClass(object):
...
@task()
def method(self):
... # self is modified here
Tip
It is possible to synchronize a list of objects. This is particularly useful when the programmer expect to synchronize more than one elements (using the compss_wait_on function) (Code 54. This feature also works with dictionaries, where the value of each entry is synchronized. In addition, if the structure synchronized is a combination of lists and dictionaries, the compss_wait_on will look for all objects to be synchronized in the whole structure.
if __name__=='__main__':
# l is a list of objects where some/all of them may be future objects
l = []
for i in range(10):
l.append(ret_func())
...
l = compss_wait_on(l)
Important
In order to make the COMPSs Python binding function correctly, the programmer should not use relative imports in the code. Relative imports can lead to ambiguous code and they are discouraged in Python, as explained in: http://docs.python.org/2/faq/programming.html#what-are-the-best-practices-for-using-import-in-a-module
Group Tasks
COMPSs also enables to specify task groups. To this end, COMPSs provides the TaskGroup context (Code 55) which can be tuned with the group name, and a second parameter (boolean) to perform an implicit barrier for the whole group. Users can also define task groups within task groups.
from pycompss.api.task import task
from pycompss.api.api import TaskGroup
from pycompss.api.api import compss_barrier_group
@task()
def func1():
...
@task()
def func2():
...
def test_taskgroup():
# Creation of group
with TaskGroup('Group1', False):
for i in range(NUM_TASKS):
func1()
func2()
...
compss_barrier_group('Group1')
...
...
if __name__=='__main__':
test_taskgroup()
API
Table 11 summarizes the API functions to be used in the main program of a COMPSs Python application.
| API Function | Description |
|---|---|
| compss_file_exists(file_name) | Check if a file exists. |
| compss_open(file_name, mode=’r’) | Synchronizes for the last version of a file and returns its file descriptor. |
| compss_delete_file(file_name) | Notifies the runtime to remove a file. |
| compss_wait_on_file(file_name) | Synchronizes for the last version of a file. |
| compss_wait_on_directory(directory_name) | Synchronizes for the last version of a directory. |
| compss_delete_object(object) | Notifies the runtime to delete the associated file to this object. |
| compss_barrier(no_more_tasks=False) | Wait for all tasks submitted before the barrier. |
| compss_barrier_group(group_name) | Wait for all tasks that belong to group_name group submitted before the barrier. |
| compss_wait_on(obj, to_write=True) | Synchronizes for the last version of an object (or a list of objects) and returns it. |
| TaskGroup(group_name, implicit_barrier=True) | Context to define a group of tasks. implicit_barrier forces waiting on context exit. |
Local Decorator
Besides the synchronization API functions, the programmer has also a decorator for automatic function parameters synchronization at his disposal. The @local decorator can be placed over functions that are not decorated as tasks, but that may receive results from tasks (Code 56). In this case, the @local decorator synchronizes the necessary parameters in order to continue with the function execution without the need of using explicitly the compss_wait_on call for each parameter.
from pycompss.api.task import task
from pycompss.api.api import compss_wait_on
from pycompss.api.parameter import INOUT
from pycompss.api.local import local
@task(returns=list)
@task(v=INOUT)
def append_three_ones(v):
v += [1, 1, 1]
@local
def scale_vector(v, k):
return [k*x for x in v]
if __name__=='__main__':
v = [1,2,3]
append_three_ones(v)
# v is automatically synchronized when calling the scale_vector function.
w = scale_vector(v, 2)
Exceptions
COMPSs is able to deal with exceptions raised during the execution of the applications. In this case, if a user/python defined exception happens, the user can choose the task behaviour using the on_failure argument within the @task decorator (with four possible values: ‘RETRY’, ’CANCEL_SUCCESSORS’, ’FAIL’ and ’IGNORE’. ’RETRY’ is the default behaviour).
However, COMPSs provides an exception (COMPSsException) that the user can
raise when necessary and can be catched in the main code for user defined
behaviour management (Code 57). This mechanism avoids
any synchronization, and enables applications to react under particular
circunstances.
from pycompss.api.task import task
from pycompss.api.exceptions import COMPSsException
@task()
def func():
...
raise COMPSsException("Something happened!")
...
if __name__=='__main__':
try:
func()
except COMPSsException:
... # React to the exception (maybe calling other tasks or with other parameters)
In addition, the COMPSsException can be combined with task groups, so that the tasks which belong to the group will also be cancelled as soon as the COMPSsException is raised (Code 58)
from pycompss.api.task import task
from pycompss.api.exceptions import COMPSsException
from pycompss.api.api import TaskGroup
@task()
def func(v):
...
if v == 8:
raise COMPSsException("8 found!")
...
if __name__=='__main__':
try:
with TaskGroup('exceptionGroup1'):
for i in range(10):
func(i)
except COMPSsException:
... # React to the exception (maybe calling other tasks or with other parameters)
Application Execution
The next subsections describe how to execute applications with the COMPSs Python binding.
Environment
The following environment variables must be defined before executing a COMPSs Python application:
- JAVA_HOME
- Java JDK installation directory (e.g.
/usr/lib/jvm/java-8-openjdk/)
Command
In order to run a Python application with COMPSs, the runcompss script
can be used, like for Java and C/C++ applications. An example of an
invocation of the script is:
compss@bsc:~$ runcompss \
--lang=python \
--pythonpath=$TEST_DIR \
$TEST_DIR/application.py arg1 arg2
Or alternatively, use the pycompss module:
compss@bsc:~$ python -m pycompss \
--pythonpath=$TEST_DIR \
$TEST_DIR/application.py arg1 arg2
Tip
The runcompss command is able to detect the application language.
Consequently, the --lang=python is not mandatory.
Tip
The --pythonpath flag enables the user to add directories to the
PYTHONPATH environment variable and export them into the workers, so
that the tasks can resolve successfully its imports.
For full description about the options available for the runcompss command please check the Executing COMPSs applications Section.
Integration with Jupyter notebook
PyCOMPSs can also be used within Jupyter notebooks. This feature allows users to develop and run their PyCOMPSs applications in a Jupyter notebook, where it is possible to modify the code during the execution and experience an interactive behaviour.
Environment Variables
The following libraries must be present in the appropiate environment variables in order to enable PyCOMPSs within Jupyter notebook:
- PYTHONPATH
- The path where PyCOMPSs is installed (e.g.
/opt/COMPSs/Bindings/python/). Please, note that the path contains the folder2and/or3. This is due to the fact that PyCOMPSs is able to choose the appropiate one depending on the kernel used with jupyter. - LD_LIBRARY_PATH
- The path where the
libbindings-commons.solibrary is located (e.g.<COMPSS_INSTALLATION_PATH>/Bindings/bindings-common/lib/) and the path where thelibjvm.solibrary is located (e.g./usr/lib/jvm/java-8-openjdk/jre/lib/amd64/server/).
API calls
In this case, the user is responsible of starting and stopping the
COMPSs runtime during the jupyter notebook execution.
To this end, PyCOMPSs provides a module with two main API calls:
one for starting the COMPSs runtime, and another for stopping it.
This module can be imported from the pycompss library:
import pycompss.interactive as ipycompss
And contains two main functions: start and stop. These functions can then be invoked as follows for the COMPSs runtime deployment with default parameters:
# Previous user code/cells
ipycompss.start()
# User code/cells that can benefit from PyCOMPSs
ipycompss.stop()
# Subsequent code/cells
Between the start and stop function calls, the user can write its own python code including PyCOMPSs imports, decorators and synchronization calls described in the Programming Model Section. The code can be splitted into multiple cells.
The start and stop functions accept parameters in order to customize
the COMPSs runtime (such as the flags that can be selected with the
runcompss command). Table 12 summarizes
the accepted parameters of the start function. Table 13
summarizes the accepted parameters of
the stop function.
| Parameter Name | Parameter Type | Description |
|---|---|---|
| log_level | String | Log level Options: "off", "info" and "debug". (Default: "off") |
| debug | Boolean | COMPSs runtime debug (Default: False) (overrides log level) |
| o_c | Boolean | Object conversion to string when possible (Default: False) |
| graph | Boolean | Task dependency graph generation (Default: False) |
| trace | Boolean | Paraver trace generation (Default: False) |
| monitor | Integer | Monitor refresh rate (Default: None - Monitoring disabled) |
| project_xml | String | Path to the project XML file (Default: "$COMPSS/Runtime/configuration/xml/projects/default project.xml") |
| resources_xml | String | Path to the resources XML file (Default: "$COMPSs/Runtime/configuration/xml/resources/default resources.xml") |
| summary | Boolean | Show summary at the end of the execution (Default: False) |
| storage_impl | String | Path to an storage implementation (Default: None) |
| storage_conf | String | Storage configuration file path (Default: None) |
| task_count | Integer | Number of task definitions (Default: 50) |
| app_name | String | Application name (Default: "Interactive") |
| uuid | String | Application uuid (Default: None - Will be random) |
| base_log_dir | String | Base directory to store COMPSs log files (a .COMPSs/ folder will be created inside this location)|pyjbr| (Default: User homeBase log path) |
| specific_log_dir | String | Use a specific directory to store COMPSs log files (the folder MUST exist and no sandbox is created) (Default: Disabled) |
| extrae_cfg | String | Sets a custom extrae config file. Must be in a shared disk between all COMPSs workers (Default: None) |
| comm | String | Class that implements the adaptor for communications. Supported adaptors: - "es.bsc.compss.nio.master.NIOAdaptor" - "es.bsc.compss.gat.master.GATAdaptor" (Default: "es.bsc.compss.nio.master.NIOAdaptor") |
| conn | String | Class that implements the runtime connector for the cloud. Supported connectors: - "es.bsc.compss.connectors.DefaultSSHConnector" - "es.bsc.compss.connectors.DefaultNoSSHConnector" (Default: "es.bsc.compss.connectors.DefaultSSHConnector") |
| master_name | String | Hostname of the node to run the COMPSs master (Default: "") |
| master_port | String | Port to run the COMPSs master communications (Only for NIO adaptor) (Default: "[43000,44000]") |
| scheduler | String | Class that implements the Scheduler for COMPSs. Supported schedulers: - "es.bsc.compss.scheduler.fullGraphScheduler.FullGraphScheduler" - "es.bsc.compss.scheduler.fifoScheduler.FIFOScheduler" - "es.bsc.compss.scheduler.resourceEmptyScheduler. ResourceEmptyScheduler" (Default: "es.bsc.compss.scheduler.loadBalancingScheduler.LoadBalancingScheduler") |
| jvm_workers | String | Extra options for the COMPSs Workers JVMs. Each option separed by “,” and without blank spaces (Default: "-Xms1024m,-Xmx1024m,-Xmn400m") |
| cpu_affinity | String | Sets the CPU affinity for the workers. Supported options: "disabled", "automatic", user defined map of the form "0-8/9,10,11/12-14,15,16" (Default: "automatic") |
| gpu_affinity | String | Sets the GPU affinity for the workers. Supported options: "disabled", "automatic", user defined map of the form "0-8/9,10,11/12-14,15,16" (Default: "automatic") |
| profile_input | String | Path to the file which stores the input application profile (Default: "") |
| profile_output | String | Path to the file to store the application profile at the end of the execution (Default: "") |
| scheduler_config | String | Path to the file which contains the scheduler configuration (Default: "") |
| external_adaptation | Boolean | Enable external adaptation (this option will disable the Resource Optimizer) (Default: False) |
| propatage_virtual_environment | Boolean | Propagate the master virtual environment to the workers (Default: False) |
| verbose | Boolean | Verbose mode (Default: False) |
| Parameter Name | Parameter Type | Description |
|---|---|---|
| sync | Boolean | Synchronize the objects left on the user scope. (Default: False) |
The following code snippet shows how to start a COMPSs runtime with tracing and graph generation enabled (with trace and graph parameters), as well as enabling the monitor with a refresh rate of 2 seconds (with the monitor parameter). It also synchronizes all remaining objects in the scope with the sync parameter when invoking the stop function.
# Previous user code
ipycompss.start(graph=True, trace=True, monitor=2000)
# User code that can benefit from PyCOMPSs
ipycompss.stop(sync=True)
# Subsequent code
Notebook execution
The application can be executed as a common Jupyter notebook by steps or the whole application.
Attention
Once the COMPSs runtime has been stopped it is NECESSARY to restart the python kernel in Jupyter before starting another COMPSs runtime.
To this end, click on “Kernel” and “Restart” (or “Restart & Clear Output” or “Restart & Run All”, depending on the need).
Notebook example
Sample notebooks can be found in the PyCOMPSs Notebooks Section.
Integration with Numba
PyCOMPSs can also be used with Numba. Numba (http://numba.pydata.org/) is an Open Source JIT compiler for Python which provides a set of decorators and functionalities to translate Python functios to optimized machine code.
Basic usage
PyCOMPSs’ tasks can be decorated with Numba’s
@jit/@njit decorator (with the appropiate
parameters) just below the @task decorator in order to apply
Numba to that task.
from pycompss.api.task import task # Import @task decorator
from numba import jit
@task(returns=1)
@jit()
def numba_func(a, b):
...
The task will be optimized by Numba within the worker node, enabling COMPSs to use the most efficient implementation of the task (and exploiting the compilation cache – any task that has already been compiled does not need to be recompiled in subsequent invocations).
Advanced usage
PyCOMPSs can be also used in conjuntion with the Numba’s
@vectorize, @guvectorize, @stencil and @cfunc.
But since these decorators do not preserve the original argument specification
of the original function, their usage is done through the numba parameter
withih the @task decorator.
The numba parameter accepts:
- Boolean:
- True: Applies jit to the function.
- Dictionary{k, v}:
- Applies jit with the dictionary parameters to the function (allows to specify specific jit parameters (e.g.*nopython=True*)).
- String:
- “jit”: Applies jit to the function.
- “njit”: Applies jit with nopython=True to the function.
- “generated_jit”: Applies generated_jit to the function.
- “vectorize”: Applies vectorize to the function. Needs some extra flags in the @task decorator:
- numba_signature: String with the vectorize signature.
- “guvectorize”: Applies guvectorize to the function. Needs some extra flags in the @task decorator:
- numba_signature: String with the guvectorize signature.
- numba_declaration: String with the guvectorize declaration.
- “stencil”: Applies stencil to the function.
- “cfunc”: Applies cfunc to the function. Needs some extra flags in the @task decorator:
- numba_signature: String with the cfunc signature.
Moreover, the @task decorator also allows to define specific flags for the jit, njit, generated_jit, vectorize, guvectorize and cfunc functionalities with the numba_flags hint. This hint is used to declare a dictionary with the flags expected to use with these numba functionalities. The default flag included by PyCOMPSs is the cache=True in order to exploit the function caching of Numba across tasks.
For example, to apply Numba jit to a task:
from pycompss.api.task import task
@task(numba='jit') # Aternatively: @task(numba=True)
def jit_func(a, b):
...
And if the developer wants to use specific flags with jit (e.g. parallel=True), the numba_flags must be defined with a dictionary where the key is the numba flag name, and the value, the numba flag value to use):
from pycompss.api.task import task
@task(numba='jit', numba_flags={'parallel':True})
def jit_func(a, b):
...
Other Numba’s functionalities require the specification of the function signature and declaration. In the next example a task that will use the vectorize with three parameters and a specific flag to target the cpu is shown:
from pycompss.api.task import task
@task(returns=1,
numba='vectorize',
numba_signature=['float32(float32, float32, float32)'],
numba_flags={'target':'cpu'})
def vectorize_task(a, b, c):
return a * b * c
Details about numba and the specification of the signature, declaration and flags can be found in the Numba’s webpage (http://numba.pydata.org/).
C/C++ Binding
COMPSs provides a binding for C and C++ applications. The new C++ version in the current release comes with support for objects as task parameters and the use of class methods as tasks.
Programming Model
As in Java, the application code is divided in 3 parts: the Task definition interface, the main code and task implementations. These files must have the following notation,: <app_ame>.idl, for the interface file, <app_name>.cc for the main code and <app_name>-functions.cc for task implementations. Next paragraphs provide an example of how to define this files for matrix multiplication parallelised by blocks.
Task Definition Interface
As in Java the user has to provide a task selection by means of an interface. In this case the interface file has the same name as the main application file plus the suffix “idl”, i.e. Matmul.idl, where the main file is called Matmul.cc.
interface Matmul
{
// C functions
void initMatrix(inout Matrix matrix,
in int mSize,
in int nSize,
in double val);
void multiplyBlocks(inout Block block1,
inout Block block2,
inout Block block3);
};
The syntax of the interface file is shown in the previous code. Tasks can be declared as classic C function prototypes, this allow to keep the compatibility with standard C applications. In the example, initMatrix and multiplyBlocks are functions declared using its prototype, like in a C header file, but this code is C++ as they have objects as parameters (objects of type Matrix, or Block).
The grammar for the interface file is:
["static"] return-type task-name ( parameter {, parameter }* );
return-type = "void" | type
ask-name = <qualified name of the function or method>
parameter = direction type parameter-name
direction = "in" | "out" | "inout"
type = "char" | "int" | "short" | "long" | "float" | "double" | "boolean" |
"char[<size>]" | "int[<size>]" | "short[<size>]" | "long[<size>]" |
"float[<size>]" | "double[<size>]" | "string" | "File" | class-name
class-name = <qualified name of the class>
Main Program
The following code shows an example of matrix multiplication written in C++.
#include "Matmul.h"
#include "Matrix.h"
#include "Block.h"
int N; //MSIZE
int M; //BSIZE
double val;
int main(int argc, char **argv)
{
Matrix A;
Matrix B;
Matrix C;
N = atoi(argv[1]);
M = atoi(argv[2]);
val = atof(argv[3]);
compss_on();
A = Matrix::init(N,M,val);
initMatrix(&B,N,M,val);
initMatrix(&C,N,M,0.0);
cout << "Waiting for initialization...\n";
compss_wait_on(B);
compss_wait_on(C);
cout << "Initialization ends...\n";
C.multiply(A, B);
compss_off();
return 0;
}
The developer has to take into account the following rules:
- A header file with the same name as the main file must be included, in this case Matmul.h. This header file is automatically generated by the binding and it contains other includes and type-definitions that are required.
- A call to the compss_on binding function is required to turn on the COMPSs runtime.
- As in C language, out or inout parameters should be passed by reference by means of the “&” operator before the parameter name.
- Synchronization on a parameter can be done calling the compss_wait_on binding function. The argument of this function must be the variable or object we want to synchronize.
- There is an implicit synchronization in the init method of Matrix. It is not possible to know the address of “A” before exiting the method call and due to this it is necessary to synchronize before for the copy of the returned value into “A” for it to be correct.
- A call to the compss_off binding function is required to turn off the COMPSs runtime.
Functions file
The implementation of the tasks in a C or C++ program has to be provided in a functions file. Its name must be the same as the main file followed by the suffix “-functions”. In our case Matmul-functions.cc.
#include "Matmul.h"
#include "Matrix.h"
#include "Block.h"
void initMatrix(Matrix *matrix,int mSize,int nSize,double val){
*matrix = Matrix::init(mSize, nSize, val);
}
void multiplyBlocks(Block *block1,Block *block2,Block *block3){
block1->multiply(*block2, *block3);
}
In the previous code, class methods have been encapsulated inside a function. This is useful when the class method returns an object or a value and we want to avoid the explicit synchronization when returning from the method.
Additional source files
Other source files needed by the user application must be placed under the directory “src”. In this directory the programmer must provide a Makefile that compiles such source files in the proper way. When the binding compiles the whole application it will enter into the src directory and execute the Makefile.
It generates two libraries, one for the master application and another for the worker application. The directive COMPSS_MASTER or COMPSS_WORKER must be used in order to compile the source files for each type of library. Both libraries will be copied into the lib directory where the binding will look for them when generating the master and worker applications.
The following sections provide a more detailed view of the C++ Binding. It will include the available API calls, how to deal with objects and having tasks as method objects as well as how to define constraints and task versions.
Binding API
Besides the aforementioned compss_on, compss_off and compss_wait_on functions, the C/C++ main program can make use of a variety of other API calls to better manage the synchronization of data generated by tasks. These calls are as follows:
- void compss_ifstream(char * filename, ifstream* & * ifs)
- Given an uninitialized input stream ifs and a file filename, this function will synchronize the content of the file and initialize ifs to read from it.
- void compss_ofstream(char * filename, ofstream* & * ofs)
- Behaves the same way as compss_ifstream, but in this case the opened stream is an output stream, meaning it will be used to write to the file.
- FILE* compss_fopen(char * file_name, char * mode)
- Similar to the C/C++ fopen call. Synchronizes with the last version of file file_name and returns the FILE* pointer to further reference it. As the mode parameter it takes the same that can be used in fopen (r, w, a, r+, w+ and a+).
- void compss_wait_on(T** & * obj) or T compss_wait_on(T* & * obj)
- Synchronizes for the last version of object obj, meaning that the execution will stop until the value of obj up to that point of the code is received (and thus all tasks that can modify it have ended).
- void compss_delete_file(char * file_name)
- Makes an asynchronous delete of file filename. When all previous tasks have finished updating the file, it is deleted.
- void compss_delete_object(T** & * obj)
- Makes an asynchronous delete of an object. When all previous tasks have finished updating the object, it is deleted.
- void compss_barrier()
- Similarly to the Python binding, performs an explicit synchronization without a return. When a compss_barrier is encountered, the execution will not continue until all the tasks submitted before the compss_barrier have finished.
Functions file
The implementation of the tasks in a C or C++ program has to be provided in a functions file. Its name must be the same as the main file followed by the suffix “-functions”. In our case Matmul-functions.cc.
#include "Matmul.h"
#include "Matrix.h"
#include "Block.h"
void initMatrix(Matrix *matrix,int mSize,int nSize,double val){
*matrix = Matrix::init(mSize, nSize, val);
}
void multiplyBlocks(Block *block1,Block *block2,Block *block3){
block1->multiply(*block2, *block3);
}
In the previous code, class methods have been encapsulated inside a function. This is useful when the class method returns an object or a value and we want to avoid the explicit synchronization when returning from the method.
Additional source files
Other source files needed by the user application must be placed under the directory “src”. In this directory the programmer must provide a Makefile that compiles such source files in the proper way. When the binding compiles the whole application it will enter into the src directory and execute the Makefile.
It generates two libraries, one for the master application and another for the worker application. The directive COMPSS_MASTER or COMPSS_WORKER must be used in order to compile the source files for each type of library. Both libraries will be copied into the lib directory where the binding will look for them when generating the master and worker applications.
Class Serialization
In case of using an object as method parameter, as callee or as return of a call to a function, the object has to be serialized. The serialization method has to be provided inline in the header file of the object’s class by means of the “boost” library. The next listing contains an example of serialization for two objects of the Block class.
#ifndef BLOCK_H
#define BLOCK_H
#include <vector>
#include <boost/archive/text_iarchive.hpp>
#include <boost/archive/text_oarchive.hpp>
#include <boost/serialization/serialization.hpp>
#include <boost/serialization/access.hpp>
#include <boost/serialization/vector.hpp>
using namespace std;
using namespace boost;
using namespace serialization;
class Block {
public:
Block(){};
Block(int bSize);
static Block *init(int bSize, double initVal);
void multiply(Block block1, Block block2);
void print();
private:
int M;
std::vector< std::vector< double > > data;
friend class::serialization::access;
template<class Archive>
void serialize(Archive & ar, const unsigned int version) {
ar & M;
ar & data;
}
};
#endif
For more information about serialization using “boost” visit the related documentation at www.boost.org <www.boost.org>.
Method - Task
A task can be a C++ class method. A method can return a value, modify the this object, or modify a parameter.
If the method has a return value there will be an implicit synchronization before exit the method, but for the this object and parameters the synchronization can be done later after the method has finished.
This is because the this object and the parameters can be accessed inside and outside the method, but for the variable where the returned value is copied to, it can’t be known inside the method.
#include "Block.h"
Block::Block(int bSize) {
M = bSize;
data.resize(M);
for (int i=0; i<M; i++) {
data[i].resize(M);
}
}
Block *Block::init(int bSize, double initVal) {
Block *block = new Block(bSize);
for (int i=0; i<bSize; i++) {
for (int j=0; j<bSize; j++) {
block->data[i][j] = initVal;
}
}
return block;
}
#ifdef COMPSS_WORKER
void Block::multiply(Block block1, Block block2) {
for (int i=0; i<M; i++) {
for (int j=0; j<M; j++) {
for (int k=0; k<M; k++) {
data[i][j] += block1.data[i][k] * block2.data[k][j];
}
}
}
this->print();
}
#endif
void Block::print() {
for (int i=0; i<M; i++) {
for (int j=0; j<M; j++) {
cout << data[i][j] << " ";
}
cout << "\r\n";
}
}
Task Constraints
The C/C++ binding also supports the definition of task constraints. The task definition specified in the IDL file must be decorated/annotated with the @Constraints. Below, you can find and example of how to define a task with a constraint of using 4 cores. The list of constraints which can be defined for a task can be found in Section [sec:Constraints]
interface Matmul
{
@Constraints(ComputingUnits = 4)
void multiplyBlocks(inout Block block1,
in Block block2,
in Block block3);
};
Task Versions
Another COMPSs functionality supported in the C/C++ binding is the definition of different versions for a tasks. The following code shows an IDL file where a function has two implementations, with their corresponding constraints. It show an example where the multiplyBlocks_GPU is defined as a implementation of multiplyBlocks using the annotation/decoration @Implements. It also shows how to set a processor constraint which requires a GPU processor and a CPU core for managing the offloading of the computation to the GPU.
interface Matmul
{
@Constraints(ComputingUnits=4);
void multiplyBlocks(inout Block block1,
in Block block2,
in Block block3);
// GPU implementation
@Constraints(processors={
@Processor(ProcessorType=CPU, ComputingUnits=1)});
@Processor(ProcessorType=GPU, ComputingUnits=1)});
@Implements(multiplyBlocks);
void multiplyBlocks_GPU(inout Block block1,
in Block block2,
in Block block3);
};
Use of programming models inside tasks
To improve COMPSs performance in some cases, C/C++ binding offers the possibility to use programming models inside tasks. This feature allows the user to exploit the potential parallelism in their application’s tasks.
OmpSs
COMPSs C/C++ binding supports the use of the programming model OmpSs. To use OmpSs inside COMPSs tasks we have to annotate the implemented tasks. The implementation of tasks was described in section [sec:functionsfile]. The following code shows a COMPSs C/C++ task without the use of OmpSs.
void compss_task(int* a, int N) {
int i;
for (i = 0; i < N; ++i) {
a[i] = i;
}
}
This code will assign to every array element its position in it. A possible use of OmpSs is the following.
void compss_task(int* a, int N) {
int i;
for (i = 0; i < N; ++i) {
#pragma omp task
{
a[i] = i;
}
}
}
This will result in the parallelization of the array initialization, of course this can be applied to more complex implementations and the directives offered by OmpSs are much more. You can find the documentation and specification in https://pm.bsc.es/ompss.
There’s also the possibility to use a newer version of the OmpSs programming model which introduces significant improvements, OmpSs-2. The changes at user level are minimal, the following image shows the array initialization using OmpSs-2.
void compss_task(int* a, int N) {
int i;
for (i = 0; i < N; ++i) {
#pragma oss task
{
a[i] = i;
}
}
}
Documentation and specification of OmpSs-2 can be found in https://pm.bsc.es/ompss-2.
Application Compilation
To compile user’s applications with the C/C++ binding two commands are used: The “compss_build_app’ command allows to compile applications for a single architecture, and the “compss_build_app_multi_arch” command for multiple architectures. Both commands must be executed in the directory of the main application code.
Single architecture
The user command “compss_build_app” compiles both master and worker for a single architecture (e.g. x86-64, armhf, etc). Thus, whether you want to run your application in Intel based machine or ARM based machine, this command is the tool you need.
When the target is the native architecture, the command to execute is very simple;
$~/matmul_objects> compss_build_app Matmul
[ INFO ] Java libraries are searched in the directory: /usr/lib/jvm/java-1.8.0-openjdk-amd64//jre/lib/amd64/server
[ INFO ] Boost libraries are searched in the directory: /usr/lib/
...
[Info] The target host is: x86_64-linux-gnu
Building application for master...
g++ -g -O3 -I. -I/Bindings/c/share/c_build/worker/files/ -c Block.cc Matrix.cc
ar rvs libmaster.a Block.o Matrix.o
ranlib libmaster.a
Building application for workers...
g++ -DCOMPSS_WORKER -g -O3 -I. -I/Bindings/c/share/c_build/worker/files/ -c Block.cc -o Block.o
g++ -DCOMPSS_WORKER -g -O3 -I. -I/Bindings/c/share/c_build/worker/files/ -c Matrix.cc -o Matrix.o
ar rvs libworker.a Block.o Matrix.o
ranlib libworker.a
...
Command successful.
In order to build an application for a different architecture e.g. armhf, an environment must be provided, indicating the compiler used to cross-compile, and also the location of some COMPSs dependencies such as java or boost which must be compliant with the target architecture. This environment is passed by flags and arguments;
Please note that to use cross compilation features and multiple architecture builds, you need to do the proper installation of COMPSs, find more information in the builders README.
$~/matmul_objects> compss_build_app --cross-compile --cross-compile-prefix=arm-linux-gnueabihf- --java_home=/usr/lib/jvm/java-1.8.0-openjdk-armhf Matmul
[ INFO ] Java libraries are searched in the directory: /usr/lib/jvm/java-1.8.0-openjdk-armhf/jre/lib/arm/server
[ INFO ] Boost libraries are searched in the directory: /usr/lib/
[ INFO ] You enabled cross-compile and the prefix to be used is: arm-linux-gnueabihf-
...
[ INFO ] The target host is: arm-linux-gnueabihf
Building application for master...
g++ -g -O3 -I. -I/Bindings/c/share/c_build/worker/files/ -c Block.cc Matrix.cc
ar rvs libmaster.a Block.o Matrix.o
ranlib libmaster.a
Building application for workers...
g++ -DCOMPSS_WORKER -g -O3 -I. -I/Bindings/c/share/c_build/worker/files/ -c Block.cc -o Block.o
g++ -DCOMPSS_WORKER -g -O3 -I. -I/Bindings/c/share/c_build/worker/files/ -c Matrix.cc -o Matrix.o
ar rvs libworker.a Block.o Matrix.o
ranlib libworker.a
...
Command successful.
[The previous outputs have been cut for simplicity]
The –cross-compile flag is used to indicate the users desire to cross-compile the application. It enables the use of –cross-compile-prefix flag to define the prefix for the cross-compiler. Setting $CROSS_COMPILE environment variable will also work (in case you use the environment variable, the prefix passed by arguments is overrided with the variable value). This prefix is added to $CC and $CXX to be used by the user Makefile and lastly by the GNU toolchain . Regarding java and boost, –java_home and –boostlib flags are used respectively. In this case, users can also use teh $JAVA_HOME and $BOOST_LIB variables to indicate the java and boost for the target architecture. Note that these last arguments are purely for linkage, where $LD_LIBRARY_PATH is used by Unix/Linux systems to find libraries, so feel free to use it if you want to avoid passing some environment arguments.
Multiple architectures
The user command “compss_build_app_multi_arch” allows a to compile an application for several architectures. Users are able to compile both master and worker for one or more architectures. Environments for the target architectures are defined in a file specified by *c*fg flag. Imagine you wish to build your application to run the master in your Intel-based machine and the worker also in your native machine and in an ARM-based machine, without this command you would have to execute several times the command for a single architecture using its cross compile features. With the multiple architecture command is done in the following way.
$~/matmul_objects> compss_build_app_multi_arch --master=x86_64-linux-gnu --worker=arm-linux-gnueabihf,x86_64-linux-gnu Matmul
[ INFO ] Using default configuration file: /opt/COMPSs/Bindings/c/cfgs/compssrc.
[ INFO ] Java libraries are searched in the directory: /usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/amd64/server
[ INFO ] Boost libraries are searched in the directory: /usr/lib/
...
Building application for master...
g++ -g -O3 -I. -I/Bindings/c/share/c_build/worker/files/ -c Block.cc Matrix.cc
ar rvs libmaster.a Block.o Matrix.o
ranlib libmaster.a
Building application for workers...
g++ -DCOMPSS_WORKER -g -O3 -I. -I/Bindings/c/share/c_build/worker/files/ -c Block.cc -o Block.o
g++ -DCOMPSS_WORKER -g -O3 -I. -I/Bindings/c/share/c_build/worker/files/ -c Matrix.cc -o Matrix.o
ar rvs libworker.a Block.o Matrix.o
ranlib libworker.a
...
Command successful. # The master for x86_64-linux-gnu compiled successfuly
...
[ INFO ] Java libraries are searched in the directory: /usr/lib/jvm/java-1.8.0-openjdk-armhf/jre/lib/arm/server
[ INFO ] Boost libraries are searched in the directory: /opt/install-arm/libboost
...
Building application for master...
arm-linux-gnueabihf-g++ -g -O3 -I. -I/Bindings/c/share/c_build/worker/files/ -c Block.cc Matrix.cc
ar rvs libmaster.a Block.o Matrix.o
ranlib libmaster.a
Building application for workers...
arm-linux-gnueabihf-g++ -DCOMPSS_WORKER -g -O3 -I. -I/Bindings/c/share/c_build/worker/files/ -c Block.cc -o Block.o
arm-linux-gnueabihf-g++ -DCOMPSS_WORKER -g -O3 -I. -I/Bindings/c/share/c_build/worker/files/ -c Matrix.cc -o Matrix.o
ar rvs libworker.a Block.o Matrix.o
ranlib libworker.a
...
Command successful. # The worker for arm-linux-gnueabihf compiled successfuly
...
[ INFO ] Java libraries are searched in the directory: /usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/lib/amd64/server
[ INFO ] Boost libraries are searched in the directory: /usr/lib/
...
Building application for master...
g++ -g -O3 -I. -I/Bindings/c/share/c_build/worker/files/ -c Block.cc Matrix.cc
ar rvs libmaster.a Block.o Matrix.o
ranlib libmaster.a
Building application for workers...
g++ -DCOMPSS_WORKER -g -O3 -I. -I/Bindings/c/share/c_build/worker/files/ -c Block.cc -o Block.o
g++ -DCOMPSS_WORKER -g -O3 -I. -I/Bindings/c/share/c_build/worker/files/ -c Matrix.cc -o Matrix.o
ar rvs libworker.a Block.o Matrix.o
ranlib libworker.a
...
Command successful. # The worker for x86_64-linux-gnu compiled successfuly
[The previous output has been cut for simplicity]
Building for single architectures would lead to a directory structure quite different than the one obtained using the script for multiple architectures. In the single architecture case, only one master and one worker directories are expected. In the multiple architectures case, one master and one worker is expected per architecture.
.
|-- arm-linux-gnueabihf
| `-- worker
| `-- gsbuild
| `-- autom4te.cache
|-- src
|-- x86_64-linux-gnu
| |-- master
| | `-- gsbuild
| | `-- autom4te.cache
| `-- worker
| `-- gsbuild
| `-- autom4te.cache
`-- xml
(Note than only directories are shown).
Using OmpSs
As described in section [sec:ompss] applications can use OmpSs and
OmpSs-2 programming models. The compilation process differs a little bit
compared with a normal COMPSs C/C++ application. Applications using
OmpSs must be compiled using the --ompss option in the
compss_build_app command.
$~/matmul_objects> compss_build_app --ompss Matmul
Executing the previous command will start the compilation of the
application. Sometimes due to configuration issues OmpSs can not be
found, the option --with_ompss=/path/to/ompss specifies the OmpSs
path that the user wants to use in the compilation.
Applications using OmpSs-2 are similarly compiled. The options to
compile with OmpSs-2 are --ompss-2 and --with_ompss-2=/path/to/ompss-2
$~/matmul_objects> compss_build_app --with_ompss-2=/home/mdomingu/ompss-2 --ompss-2 Matmul
Remember that additional source files can be used in COMPSs C/C++ applications, if the user expects OmpSs or OmpSs-2 to be used in those files she, must be sure that the files are properly compiled with OmpSs or OmpSs-2.
Application Execution
The following environment variables must be defined before executing a COMPSs C/C++ application:
- JAVA_HOME
- Java JDK installation directory (e.g. /usr/lib/jvm/java-8-openjdk/)
After compiling the application, two directories, master and worker, are generated. The master directory contains a binary called as the main file, which is the master application, in our example is called Matmul. The worker directory contains another binary called as the main file followed by the suffix “-worker”, which is the worker application, in our example is called Matmul-worker.
The runcompss script has to be used to run the application:
$ runcompss /home/compss/tutorial_apps/c/matmul_objects/master/Matmul 3 4 2.0
The complete list of options of the runcompss command is available in Section Executing COMPSs applications.
Task Dependency Graph
COMPSs can generate a task dependency graph from an executed code. It is indicating by a
$ runcompss -g /home/compss/tutorial_apps/c/matmul_objects/master/Matmul 3 4 2.0
The generated task dependency graph is stored within the
$HOME/.COMPSs/<APP_NAME>_<00-99>/monitor directory in dot format.
The generated graph is complete_graph.dot file, which can be
displayed with any dot viewer. COMPSs also provides the compss_gengraph script
which converts the given dot file into pdf.
$ cd $HOME/.COMPSs/Matmul_02/monitor $ compss_gengraph complete_graph.dot $ evince complete_graph.pdf # or use any other pdf viewer you like
The following figure depicts the task dependency graph for the Matmul application in its object version with 3x3 blocks matrices, each one containing a 4x4 matrix of doubles. Each block in the result matrix accumulates three block multiplications, i.e. three multiplications of 4x4 matrices of doubles.
Matmul Execution Graph.
The light blue circle corresponds to the initialization of matrix “A” by means of a method-task and it has an implicit synchronization inside. The dark blue circles correspond to the other two initializations by means of function-tasks; in this case the synchronizations are explicit and must be provided by the developer after the task call. Both implicit and explicit synchronizations are represented as red circles.
Each green circle is a partial matrix multiplication of a set of 3. One block from matrix “A” and the correspondent one from matrix “B”. The result is written in the right block in “C” that accumulates the partial block multiplications. Each multiplication set has an explicit synchronization. All green tasks are method-tasks and they are executed in parallel.
Constraints
This section provides a detailed information about all the supported constraints by the COMPSs runtime for Java, Python and C/C++ languages. The constraints are defined as key-value pairs, where the key is the name of the constraint. Table 14 details the available constraints names for Java, Python and C/C++, its value type, its default value and a brief description.
| Java | Python | C / C++ | Value type | Default value | Description |
|---|---|---|---|---|---|
| computingUnits | computing_units | ComputingUnits | string |
“1” | Required number of computing units |
| processorName | processor_name | ProcessorName | string |
“[unassigned]” | Required processor name |
| processorSpeed | processor_speed | ProcessorSpeed | string |
“[unassigned]” | Required processor speed |
| processorArchitecture | processor_architecture | ProcessorArchitecture | string |
“[unassigned]” | Required processor architecture |
| processorType | processor_type | ProcessorType | string |
“[unassigned]” | Required processor type |
| processorPropertyName | processor_property_name | ProcessorPropertyName | string |
“[unassigned]” | Required processor property |
| processorPropertyValue | processor_property_value | ProcessorPropertyValue | string |
“[unassigned]” | Required processor property value |
| processorInternalMemorySize | processor_internal_memory_size | ProcessorInternalMemorySize | string |
“[unassigned]” | Required internal device memory |
| processors | processors | List@Processor |
“{}” | Required processors (check Table 15 for Processor details) | |
| memorySize | memory_size | MemorySize | string |
“[unassigned]” | Required memory size in GBs |
| memoryType | memory_type | MemoryType | string |
“[unassigned]” | Required memory type (SRAM, DRAM, etc.) |
| storageSize | storage_size | StorageSize | string |
“[unassigned]” | Required storage size in GBs |
| storageType | storage_type | StorageType | string |
“[unassigned]” | Required storage type (HDD, SSD, etc.) |
| operatingSystemType | operating_system_type | OperatingSystemType | string |
“[unassigned]” | Required operating system type (Windows, MacOS, Linux, etc.) |
| operatingSystemDistribution | operating_system_distribution | OperatingSystemDistribution | string |
“[unassigned]” | Required operating system distribution (XP, Sierra, openSUSE, etc.) |
| operatingSystemVersion | operating_system_version | OperatingSystemVersion | string |
“[unassigned]” | Required operating system version |
| wallClockLimit | wall_clock_limit | WallClockLimit | string |
“[unassigned]” | Maximum wall clock time |
| hostQueues | host_queues | HostQueues | string |
“[unassigned]” | Required queues |
| appSoftware | app_software | AppSoftware | string |
“[unassigned]” | Required applications that must be available within the remote node for the task |
All constraints are defined with a simple value except the HostQueue and AppSoftware constraints, which allow multiple values.
The processors constraint allows the users to define multiple processors for a task execution. This constraint is specified as a list of @Processor annotations that must be defined as shown in Table 15
| Annotation | Value type | Default value | Description |
|---|---|---|---|
| processorType | string |
“CPU” | Required processor type (e.g. CPU or GPU) |
| computingUnits | string |
“1” | Required number of computing units |
| name | string |
“[unassigned]” | Required processor name |
| speed | string |
“[unassigned]” | Required processor speed |
| architecture | string |
“[unassigned]” | Required processor architecture |
| propertyName | string |
“[unassigned]” | Required processor property |
| propertyValue | string |
“[unassigned]” | Required processor property value |
| internalMemorySize | string |
“[unassigned]” | Required internal device memory |
Execution Environments
This section is intended to show how to execute the COMPSs applications.
Local
This section is intended to walk you through the COMPSs usage in local machines.
Executing COMPSs applications
Prerequisites
Prerequisites vary depending on the application’s code language: for Java applications the users need to have a jar archive containing all the application classes, for Python applications there are no requirements and for C/C++ applications the code must have been previously compiled by using the buildapp command.
For further information about how to develop COMPSs applications please refer to Application development.
Runcompss command
COMPSs applications are executed using the runcompss command:
compss@bsc:~$ runcompss [options] application_name [application_arguments]
The application name must be the fully qualified name of the application in Java, the path to the .py file containing the main program in Python and the path to the master binary in C/C++.
The application arguments are the ones passed as command line to main application. This parameter can be empty.
The runcompss command allows the users to customize a COMPSs
execution by specifying different options. For clarity purposes,
parameters are grouped in Runtime configuration, Tools enablers and
Advanced options.
compss@bsc:~$ runcompss -h
Usage: /opt/COMPSs/Runtime/scripts/user/runcompss [options] application_name application_arguments
* Options:
General:
--help, -h Print this help message
--opts Show available options
--version, -v Print COMPSs version
Tools enablers:
--graph=<bool>, --graph, -g Generation of the complete graph (true/false)
When no value is provided it is set to true
Default: false
--tracing=<level>, --tracing, -t Set generation of traces and/or tracing level ( [ true | basic ] | advanced | scorep | arm-map | arm-ddt | false)
True and basic levels will produce the same traces.
When no value is provided it is set to 1
Default: 0
--monitoring=<int>, --monitoring, -m Period between monitoring samples (milliseconds)
When no value is provided it is set to 2000
Default: 0
--external_debugger=<int>,
--external_debugger Enables external debugger connection on the specified port (or 9999 if empty)
Default: false
--jmx_port=<int> Enable JVM profiling on specified port
Runtime configuration options:
--task_execution=<compss|storage> Task execution under COMPSs or Storage.
Default: compss
--storage_impl=<string> Path to an storage implementation. Shortcut to setting pypath and classpath. See Runtime/storage in your installation folder.
--storage_conf=<path> Path to the storage configuration file
Default: null
--project=<path> Path to the project XML file
Default: /opt/COMPSs//Runtime/configuration/xml/projects/default_project.xml
--resources=<path> Path to the resources XML file
Default: /opt/COMPSs//Runtime/configuration/xml/resources/default_resources.xml
--lang=<name> Language of the application (java/c/python)
Default: Inferred is possible. Otherwise: java
--summary Displays a task execution summary at the end of the application execution
Default: false
--log_level=<level>, --debug, -d Set the debug level: off | info | debug
Warning: Off level compiles with -O2 option disabling asserts and __debug__
Default: off
Advanced options:
--extrae_config_file=<path> Sets a custom extrae config file. Must be in a shared disk between all COMPSs workers.
Default: null
--trace_label=<string> Add a label in the generated trace file. Only used in the case of tracing is activated.
Default: None
--comm=<ClassName> Class that implements the adaptor for communications
Supported adaptors:
├── es.bsc.compss.nio.master.NIOAdaptor
└── es.bsc.compss.gat.master.GATAdaptor
Default: es.bsc.compss.nio.master.NIOAdaptor
--conn=<className> Class that implements the runtime connector for the cloud
Supported connectors:
├── es.bsc.compss.connectors.DefaultSSHConnector
└── es.bsc.compss.connectors.DefaultNoSSHConnector
Default: es.bsc.compss.connectors.DefaultSSHConnector
--streaming=<type> Enable the streaming mode for the given type.
Supported types: FILES, OBJECTS, PSCOS, ALL, NONE
Default: NONE
--streaming_master_name=<str> Use an specific streaming master node name.
Default: null
--streaming_master_port=<int> Use an specific port for the streaming master.
Default: null
--scheduler=<className> Class that implements the Scheduler for COMPSs
Supported schedulers:
├── es.bsc.compss.scheduler.data.DataScheduler
├── es.bsc.compss.scheduler.fifo.FIFOScheduler
├── es.bsc.compss.scheduler.fifodata.FIFODataScheduler
├── es.bsc.compss.scheduler.lifo.LIFOScheduler
├── es.bsc.compss.components.impl.TaskScheduler
└── es.bsc.compss.scheduler.loadbalancing.LoadBalancingScheduler
Default: es.bsc.compss.scheduler.loadbalancing.LoadBalancingScheduler
--scheduler_config_file=<path> Path to the file which contains the scheduler configuration.
Default: Empty
--library_path=<path> Non-standard directories to search for libraries (e.g. Java JVM library, Python library, C binding library)
Default: Working Directory
--classpath=<path> Path for the application classes / modules
Default: Working Directory
--appdir=<path> Path for the application class folder.
Default: /home/user/gitlab/documentation/COMPSs_Manuals/source
--pythonpath=<path> Additional folders or paths to add to the PYTHONPATH
Default: /home/user/gitlab/documentation/COMPSs_Manuals/source
--base_log_dir=<path> Base directory to store COMPSs log files (a .COMPSs/ folder will be created inside this location)
Default: User home
--specific_log_dir=<path> Use a specific directory to store COMPSs log files (no sandbox is created)
Warning: Overwrites --base_log_dir option
Default: Disabled
--uuid=<int> Preset an application UUID
Default: Automatic random generation
--master_name=<string> Hostname of the node to run the COMPSs master
Default:
--master_port=<int> Port to run the COMPSs master communications.
Only for NIO adaptor
Default: [43000,44000]
--jvm_master_opts="<string>" Extra options for the COMPSs Master JVM. Each option separed by "," and without blank spaces (Notice the quotes)
Default:
--jvm_workers_opts="<string>" Extra options for the COMPSs Workers JVMs. Each option separed by "," and without blank spaces (Notice the quotes)
Default: -Xms1024m,-Xmx1024m,-Xmn400m
--cpu_affinity="<string>" Sets the CPU affinity for the workers
Supported options: disabled, automatic, user defined map of the form "0-8/9,10,11/12-14,15,16"
Default: automatic
--gpu_affinity="<string>" Sets the GPU affinity for the workers
Supported options: disabled, automatic, user defined map of the form "0-8/9,10,11/12-14,15,16"
Default: automatic
--fpga_affinity="<string>" Sets the FPGA affinity for the workers
Supported options: disabled, automatic, user defined map of the form "0-8/9,10,11/12-14,15,16"
Default: automatic
--fpga_reprogram="<string>" Specify the full command that needs to be executed to reprogram the FPGA with the desired bitstream. The location must be an absolute path.
Default:
--io_executors=<int> IO Executors per worker
Default: 0
--task_count=<int> Only for C/Python Bindings. Maximum number of different functions/methods, invoked from the application, that have been selected as tasks
Default: 50
--input_profile=<path> Path to the file which stores the input application profile
Default: Empty
--output_profile=<path> Path to the file to store the application profile at the end of the execution
Default: Empty
--PyObject_serialize=<bool> Only for Python Binding. Enable the object serialization to string when possible (true/false).
Default: false
--persistent_worker_c=<bool> Only for C Binding. Enable the persistent worker in c (true/false).
Default: false
--enable_external_adaptation=<bool> Enable external adaptation. This option will disable the Resource Optimizer.
Default: false
--gen_coredump Enable master coredump generation
Default: false
--python_interpreter=<string> Python interpreter to use (python/python2/python3).
Default: python Version: 2
--python_propagate_virtual_environment=<true> Propagate the master virtual environment to the workers (true/false).
Default: true
--python_mpi_worker=<false> Use MPI to run the python worker instead of multiprocessing. (true/false).
Default: false
* Application name:
For Java applications: Fully qualified name of the application
For C applications: Path to the master binary
For Python applications: Path to the .py file containing the main program
* Application arguments:
Command line arguments to pass to the application. Can be empty.
Running a COMPSs application
Before running COMPSs applications the application files must be in
the CLASSPATH. Thus, when launching a COMPSs application, users can
manually pre-set the CLASSPATH environment variable or can add the
--classpath option to the runcompss command.
The next three sections provide specific information for launching COMPSs applications developed in different code languages (Java, Python and C/C++). For clarity purposes, we will use the Simple application (developed in Java, Python and C++) available in the COMPSs Virtual Machine or at https://compss.bsc.es/projects/bar webpage. This application takes an integer as input parameter and increases it by one unit using a task. For further details about the codes please refer to Sample Applications.
Running Java applications
A Java COMPSs application can be launched through the following command:
compss@bsc:~$ cd tutorial_apps/java/simple/jar/
compss@bsc:~/tutorial_apps/java/simple/jar$ runcompss simple.Simple <initial_number>
compss@bsc:~/tutorial_apps/java/simple/jar$ runcompss simple.Simple 1
[ INFO] Using default execution type: compss
[ INFO] Using default location for project file: /opt/COMPSs/Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs/Runtime/configuration/xml/resources/default_resources.xml
[ INFO] Using default language: java
----------------- Executing simple.Simple --------------------------
WARNING: COMPSs Properties file is null. Setting default values
[(1066) API] - Starting COMPSs Runtime v<version>
Initial counter value is 1
Final counter value is 2
[(4740) API] - Execution Finished
------------------------------------------------------------
In this first execution we use the default value of the --classpath
option to automatically add the jar file to the classpath (by executing
runcompss in the directory which contains the jar file). However, we can
explicitly do this by exporting the CLASSPATH variable or by
providing the --classpath value. Next, we provide two more ways to
perform the same execution:
compss@bsc:~$ export CLASSPATH=$CLASSPATH:/home/compss/tutorial_apps/java/simple/jar/simple.jar
compss@bsc:~$ runcompss simple.Simple <initial_number>
compss@bsc:~$ runcompss --classpath=/home/compss/tutorial_apps/java/simple/jar/simple.jar \
simple.Simple <initial_number>
Running Python applications
To launch a COMPSs Python application users have to provide the
--lang=python option to the runcompss command. If the extension of
the main file is a regular Python extension (.py or .pyc) the
runcompss command can also infer the application language without
specifying the lang flag.
compss@bsc:~$ cd tutorial_apps/python/simple/
compss@bsc:~/tutorial_apps/python/simple$ runcompss --lang=python ./simple.py <initial_number>
compss@bsc:~/tutorial_apps/python/simple$ runcompss simple.py 1
[ INFO] Using default execution type: compss
[ INFO] Using default location for project file: /opt/COMPSs/Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs/Runtime/configuration/xml/resources/default_resources.xml
[ INFO] Inferred PYTHON language
----------------- Executing simple.py --------------------------
WARNING: COMPSs Properties file is null. Setting default values
[(616) API] - Starting COMPSs Runtime v<version>
Initial counter value is 1
Final counter value is 2
[(4297) API] - Execution Finished
------------------------------------------------------------
Attention
Executing without debug (e.g. default log level or --log_level=off)
uses -O2 compiled sources, disabling asserts and __debug__.
Alternatively, it is possible to execute the a COMPSs Python application
using pycompss as module:
compss@bsc:~$ python -m pycompss <runcompss_flags> <application> <application_parameters>
Consequently, the previous example could also be run as follows:
compss@bsc:~$ cd tutorial_apps/python/simple/
compss@bsc:~/tutorial_apps/python/simple$ python -m pycompss simple.py <initial_number>
If the -m pycompss is not set, the application will be run ignoring
all PyCOMPSs imports, decorators and API calls, that is, sequentially.
In order to run a COMPSs Python application with a different interpreter, the runcompss command provides a specific flag:
compss@bsc:~$ cd tutorial_apps/python/simple/
compss@bsc:~/tutorial_apps/python/simple$ runcompss --python_interpreter=python3 ./simple.py <initial_number>
However, when using the pycompss module, it is inferred from the python used in the call:
compss@bsc:~$ cd tutorial_apps/python/simple/
compss@bsc:~/tutorial_apps/python/simple$ python3 -m pycompss simple.py <initial_number>
Finally, both runcompss and pycompss module provide a particular
flag for virtual environment propagation
(--python_propagate_virtual_environment=<bool>). This, flag is
intended to activate the current virtual environment in the worker nodes
when set to true.
Running C/C++ applications
To launch a COMPSs C/C++ application users have to compile the
C/C++ application by means of the buildapp command. For
further information please refer to C/C++ Binding. Once
complied, the --lang=c option must be provided to the runcompss
command. If the main file is a C/C++ binary the runcompss command
can also infer the application language without specifying the lang
flag.
compss@bsc:~$ cd tutorial_apps/c/simple/
compss@bsc:~/tutorial_apps/c/simple$ runcompss --lang=c simple <initial_number>
compss@bsc:~/tutorial_apps/c/simple$ runcompss ~/tutorial_apps/c/simple/master/simple 1
[ INFO] Using default execution type: compss
[ INFO] Using default location for project file: /opt/COMPSs/Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs/Runtime/configuration/xml/resources/default_resources.xml
[ INFO] Inferred C/C++ language
----------------- Executing simple --------------------------
JVM_OPTIONS_FILE: /tmp/tmp.ItT1tQfKgP
COMPSS_HOME: /opt/COMPSs
Args: 1
WARNING: COMPSs Properties file is null. Setting default values
[(650) API] - Starting COMPSs Runtime v<version>
Initial counter value is 1
[ BINDING] - @compss_wait_on - Entry.filename: counter
[ BINDING] - @compss_wait_on - Runtime filename: d1v2_1497432831496.IT
Final counter value is 2
[(4222) API] - Execution Finished
------------------------------------------------------------
Additional configurations
The COMPSs runtime has two configuration files: resources.xml and
project.xml . These files contain information about the execution
environment and are completely independent from the application.
For each execution users can load the default configuration files or
specify their custom configurations by using, respectively, the
--resources=<absolute_path_to_resources.xml> and the
--project=<absolute_path_to_project.xml> in the runcompss
command. The default files are located in the
/opt/COMPSs/Runtime/configuration/xml/ path. Users can manually edit
these files or can use the Eclipse IDE tool developed for COMPSs. For
further information about the Eclipse IDE please refer to
COMPSs IDE Section.
For further details please check the Configuration Files.
Results and logs
Results
When executing a COMPSs application we consider different type of results:
- Application Output: Output generated by the application.
- Application Files: Files used or generated by the application.
- Tasks Output: Output generated by the tasks invoked from the application.
Regarding the application output, COMPSs will preserve the application
output but will add some pre and post output to indicate the COMPSs
Runtime state. Figure 7 shows the standard output
generated by the execution of the Simple Java application. The green box
highlights the application stdout while the rest of the output is
produced by COMPSs.
Output generated by the execution of the Simple Java application with COMPSs
Regarding the application files, COMPSs does not modify any of them and thus, the results obtained by executing the application with COMPSs are the same than the ones generated by the sequential execution of the application.
Regarding the tasks output, COMPSs introduces some modifications due to the fact that tasks can be executed in remote machines. After the execution, COMPSs stores the stdout and the stderr of each job (a task execution) inside the ``/home/$USER/.COMPSs/$APPNAME/$EXEC_NUMBER/jobs/`` directory of the main application node.
Figure 8 and Figure 9 show an example of the
results obtained from the execution of the Hello Java application.
While Figure 8 provides the output of the sequential
execution of the application (without COMPSs), Figure 9
provides the output of the equivalent COMPSs
execution. Please note that the sequential execution produces the
Hello World! (from a task) message in the stdout while the
COMPSs execution stores the message inside the job1_NEW.out file.
Sequential execution of the Hello java application
COMPSs execution of the Hello java application
Logs
COMPSs includes three log levels for running applications but users can
modify them or add more levels by editing the logger files under the
/opt/COMPSs/Runtime/configuration /log/ folder. Any of these log
levels can be selected by adding the --log_level=<debug | info | off>
flag to the runcompss command. The default value is off.
The logs generated by the NUM_EXEC execution of the application APP
by the user USER are stored under
/home/$USER/.COMPSs/$APP/$EXEC_NUMBER/ folder (from this point on:
base log folder). The EXEC_NUMBER execution number is
automatically used by COMPSs to prevent mixing the logs of data of
different executions.
When running COMPSs with log level off only the errors are reported.
This means that the base log folder will contain two empty files
(runtime.log and resources.log) and one empty folder (jobs).
If somehow the application has failed, the runtime.log and/or the
resources.log will not be empty and a new file per failed job will
appear inside the jobs folder to store the stdout and the
stderr. Figure 10 shows the logs generated by
the execution of the Simple java application (without errors) in off
mode.
Structure of the logs folder for the Simple java application in off mode
When running COMPSs with log level info the base log folder will
contain two files (runtime.log and resources.log) and one folder
(jobs). The runtime.log file contains the execution information
retrieved from the master resource, including the file transfers and the
job submission details. The resources.log file contains information
about the available resources such as the number of processors of each
resource (slots), the information about running or pending tasks in the
resource queue and the created and destroyed resources. The jobs folder
will be empty unless there has been a failed job. In this case it will
store, for each failed job, one file for the stdout and another for
the stderr. As an example, Figure 11 shows the
logs generated by the same execution than the previous case but with
info mode.
Structure of the logs folder for the Simple java application in info mode
The runtime.log and resources.log are quite large files, thus
they should be only checked by advanced users. For an easier
interpretation of these files the COMPSs Framework includes a monitor
tool. For further information about the COMPSs Monitor please check
COMPSs Monitor.
Figure 12 and Figure 13 provide the content of these two files generated by the execution of the Simple java application.
runtime.log generated by the execution of the Simple java application
resources.log generated by the execution of the Simple java application
Running COMPSs with log level debug generates the same files as the
info log level but with more detailed information. Additionally, the
jobs folder contains two files per submitted job; one for the
stdout and another for the stderr. In the other hand, the COMPSs
Runtime state is printed out on the stdout.
Figure 14 shows the logs generated by the same execution
than the previous cases but with debug mode.
The runtime.log and the resources.log files generated in this mode can be extremely large. Consequently, the users should take care of their quota and manually erase these files if needed.
Structure of the logs folder for the Simple java application in debug mode
When running Python applications a pycompss.log file is written
inside the base log folder containing debug information about the
specific calls to PyCOMPSs.
Furthermore, when running runcompss with additional flags (such as
monitoring or tracing) additional folders will appear inside the base
log folder. The meaning of the files inside these folders is explained
in COMPSs Tools.
COMPSs Tools
Application graph
At the end of the application execution a dependency graph can be
generated representing the order of execution of each type of task and
their dependencies. To allow the final graph generation the -g flag
has to be passed to the runcompss command; the graph file is written
in the base_log_folder/monitor/complete_graph.dot at the end of the
execution.
Figure 15 shows a dependency graph example of a SparseLU java application. The graph can be visualized by running the following command:
compss@bsc:~$ compss_gengraph ~/.COMPSs/sparseLU.arrays.SparseLU_01/monitor/complete_graph.dot
The dependency graph of the SparseLU application
COMPSs Monitor
The COMPSs Framework includes a Web graphical interface that can be used to monitor the execution of COMPSs applications. COMPSs Monitor is installed as a service and can be easily managed by running any of the following commands:
compss@bsc:~$ /etc/init.d/compss-monitor usage
Usage: compss-monitor {start | stop | reload | restart | try-restart | force-reload | status}
Service configuration
The COMPSs Monitor service can be configured by editing the
/opt/COMPSs/Tools/monitor/apache-tomcat/conf/compss-monitor.conf file which contains
one line per property:
- COMPSS_MONITOR
- Default directory to retrieve monitored applications
(defaults to the
.COMPSsfolder inside therootuser). - COMPSs_MONITOR_PORT
- Port where to run the compss-monitor web service (defaults to 8080).
- COMPSs_MONITOR_TIMEOUT
- Web page timeout between browser and server (defaults to 20s).
Usage
In order to use the COMPSs Monitor users need to start the service as shown in Figure 16.
COMPSs Monitor start command
And use a web browser to open the specific URL:
compss@bsc:~$ firefox http://localhost:8080/compss-monitor &
The COMPSs Monitor allows to monitor applications from different users and thus, users need to first login to access their applications. As shown in Figure 17, the users can select any of their executed or running COMPSs applications and display it.
COMPSs monitoring interface
To enable all the COMPSs Monitor features, applications must run the
runcompss command with the -m flag. This flag allows the COMPSs
Runtime to store special information inside inside the
log_base_folder under the monitor folder (see
Figure 17 and Figure 18). Only
advanced users should modify or delete any of these files. If the
application that a user is trying to monitor has not been executed with
this flag, some of the COMPSs Monitor features will be disabled.
compss@bsc:~/tutorial_apps/java/simple/jar$ runcompss -dm simple.Simple 1
[ INFO] Using default execution type: compss
[ INFO] Using default location for project file: /opt/COMPSs/Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs/Runtime/configuration/xml/resources/default_resources.xml
[ INFO] Using default language: java
----------------- Executing simple.Simple --------------------------
WARNING: COMPSs Properties file is null. Setting default values
[(799) API] - Deploying COMPSs Runtime v<version>
[(801) API] - Starting COMPSs Runtime v<version>
[(801) API] - Initializing components
[(1290) API] - Ready to process tasks
[(1293) API] - Opening /home/compss/tutorial_apps/java/simple/jar/counter in mode OUT
[(1338) API] - File target Location: /home/compss/tutorial_apps/java/simple/jar/counter
Initial counter value is 1
[(1340) API] - Creating task from method increment in simple.SimpleImpl
[(1340) API] - There is 1 parameter
[(1341) API] - Parameter 1 has type FILE_T
Final counter value is 2
[(4307) API] - No more tasks for app 1
[(4311) API] - Getting Result Files 1
[(4340) API] - Stop IT reached
[(4344) API] - Stopping Graph generation...
[(4344) API] - Stopping Monitor...
[(6347) API] - Stopping AP...
[(6348) API] - Stopping TD...
[(6509) API] - Stopping Comm...
[(6510) API] - Runtime stopped
[(6510) API] - Execution Finished
------------------------------------------------------------
Logs generated by the Simple java application with the monitoring flag enabled
Graphical Interface features
In this section we provide a summary of the COMPSs Monitor supported features available through the graphical interface:
- Resources information Provides information about the resources used by the application
- Tasks information Provides information about the tasks definition used by the application
- Current tasks graph Shows the tasks dependency graph currently stored into the COMPSs Runtime
- Complete tasks graph Shows the complete tasks dependecy graph of the application
- Load chart Shows different dynamic charts representing the evolution over time of the resources load and the tasks load
- Runtime log Shows the runtime log
- Execution Information Shows specific job information allowing users to easily select failed or uncompleted jobs
- Statistics Shows application statistics such as the accumulated cloud cost.
Important
To enable all the COMPSs Monitor features applications must run with the -m flag.
The webpage also allows users to configure some performance parameters of the monitoring service by accessing the Configuration button at the top-right corner of the web page.
For specific COMPSs Monitor feature configuration please check our FAQ section at the top-right corner of the web page.
Application tracing
COMPSs Runtime can generate a post-execution trace of the execution of the application. This trace is useful for performance analysis and diagnosis.
A trace file may contain different events to determine the COMPSs master state, the task execution state or the file-transfers. The current release does not support file-transfers informations.
During the execution of the application, an XML file is created in the worker nodes to keep track of these events. At the end of the execution, all the XML files are merged to get a final trace file.
In this manual we only provide information about how to obtain a trace and about the available Paraver (the tool used to analyze the traces) configurations. For further information about the application instrumentation or the trace visualization and configurations please check the Tracing Section.
Trace Command
In order to obtain a post-execution trace file one of the following
options -t, --tracing, --tracing=true, --tracing=basic must
be added to the runcompss command. All this options activate the
basic tracing mode; the advanced mode is activated with the option
--tracing=advanced. For further information about advanced mode check
the COMPSs applications tracing
Section. Next, we provide an example of the command execution with the basic
tracing option enabled for a java K-Means application.
compss@bsc:~$ runcompss -t kmeans.Kmeans
*** RUNNING JAVA APPLICATION KMEANS
[ INFO] Relative Classpath resolved: /path/to/jar/kmeans.jar
----------------- Executing kmeans.Kmeans --------------------------
Welcome to Extrae VERSION
Extrae: Parsing the configuration file (/opt/COMPSs/Runtime/configuration/xml/tracing/extrae_basic.xml) begins
Extrae: Warning! <trace> tag has no <home> property defined.
Extrae: Generating intermediate files for Paraver traces.
Extrae: <cpu> tag at <counters> level will be ignored. This library does not support CPU HW.
Extrae: Tracing buffer can hold 100000 events
Extrae: Circular buffer disabled.
Extrae: Dynamic memory instrumentation is disabled.
Extrae: Basic I/O memory instrumentation is disabled.
Extrae: System calls instrumentation is disabled.
Extrae: Parsing the configuration file (/opt/COMPSs/Runtime/configuration/xml/tracing/extrae_basic.xml) has ended
Extrae: Intermediate traces will be stored in /user/folder
Extrae: Tracing mode is set to: Detail.
Extrae: Successfully initiated with 1 tasks and 1 threads
WARNING: COMPSs Properties file is null. Setting default values
[(751) API] - Deploying COMPSs Runtime v<version>
[(753) API] - Starting COMPSs Runtime v<version>
[(753) API] - Initializing components
[(1142) API] - Ready to process tasks
...
...
...
merger: Output trace format is: Paraver
merger: Extrae 3.3.0 (revision 3966 based on extrae/trunk)
mpi2prv: Assigned nodes < Marginis >
mpi2prv: Assigned size per processor < <1 Mbyte >
mpi2prv: File set-0/TRACE@Marginis.0000001904000000000000.mpit is object 1.1.1 on node Marginis assigned to processor 0
mpi2prv: File set-0/TRACE@Marginis.0000001904000000000001.mpit is object 1.1.2 on node Marginis assigned to processor 0
mpi2prv: File set-0/TRACE@Marginis.0000001904000000000002.mpit is object 1.1.3 on node Marginis assigned to processor 0
mpi2prv: File set-0/TRACE@Marginis.0000001980000001000000.mpit is object 1.2.1 on node Marginis assigned to processor 0
mpi2prv: File set-0/TRACE@Marginis.0000001980000001000001.mpit is object 1.2.2 on node Marginis assigned to processor 0
mpi2prv: File set-0/TRACE@Marginis.0000001980000001000002.mpit is object 1.2.3 on node Marginis assigned to processor 0
mpi2prv: File set-0/TRACE@Marginis.0000001980000001000003.mpit is object 1.2.4 on node Marginis assigned to processor 0
mpi2prv: File set-0/TRACE@Marginis.0000001980000001000004.mpit is object 1.2.5 on node Marginis assigned to processor 0
mpi2prv: Time synchronization has been turned off
mpi2prv: A total of 9 symbols were imported from TRACE.sym file
mpi2prv: 0 function symbols imported
mpi2prv: 9 HWC counter descriptions imported
mpi2prv: Checking for target directory existance... exists, ok!
mpi2prv: Selected output trace format is Paraver
mpi2prv: Stored trace format is Paraver
mpi2prv: Searching synchronization points... done
mpi2prv: Time Synchronization disabled.
mpi2prv: Circular buffer enabled at tracing time? NO
mpi2prv: Parsing intermediate files
mpi2prv: Progress 1 of 2 ... 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% done
mpi2prv: Processor 0 succeeded to translate its assigned files
mpi2prv: Elapsed time translating files: 0 hours 0 minutes 0 seconds
mpi2prv: Elapsed time sorting addresses: 0 hours 0 minutes 0 seconds
mpi2prv: Generating tracefile (intermediate buffers of 838848 events)
This process can take a while. Please, be patient.
mpi2prv: Progress 2 of 2 ... 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% done
mpi2prv: Warning! Clock accuracy seems to be in microseconds instead of nanoseconds.
mpi2prv: Elapsed time merge step: 0 hours 0 minutes 0 seconds
mpi2prv: Resulting tracefile occupies 991743 bytes
mpi2prv: Removing temporal files... done
mpi2prv: Elapsed time removing temporal files: 0 hours 0 minutes 0 seconds
mpi2prv: Congratulations! ./trace/kmeans.Kmeans_compss_trace_1460456106.prv has been generated.
[ API] - Execution Finished
------------------------------------------------------------
At the end of the execution the trace will be stored inside the
trace folder under the application log directory.
compss@bsc:~$ cd .COMPSs/kmeans.Kmeans_01/trace/
compss@bsc:~$ ls -1
kmeans.Kmeans_compss_trace_1460456106.pcf
kmeans.Kmeans_compss_trace_1460456106.prv
kmeans.Kmeans_compss_trace_1460456106.row
Trace visualization
The traces generated by an application execution are ready to be visualized with Paraver. Paraver is a powerful tool developed by BSC that allows users to show many views of the trace data by means of different configuration files. Users can manually load, edit or create configuration files to obtain different trace data views.
If Paraver is installed, issue the following command to visualize a given tracefile:
compss@bsc:~$ wxparaver path/to/trace/trace_name.prv
For further information about Paraver please visit the following site: http://www.bsc.es/computer-sciences/performance-tools/paraver
COMPSs IDE
COMPSs IDE is an Integrated Development Environment to develop, compile, deploy and execute COMPSs applications. It is available through the Eclipse Market as a plugin and provides an even easier way to work with COMPSs.
For further information please check the COMPSs IDE User Guide available at: http://compss.bsc.es .
Supercomputers
This section is intended to walk you through the COMPSs usage in Supercomputers.
Common usage
Available COMPSs modules
COMPSs is configured as a Linux Module. As shown in next Figure, the
users can type the module available COMPSs command to list the
supported COMPSs modules in the supercomputer. The users can also
execute the module load COMPSs/<version> command to load an specific
COMPSs module.
$ module available COMPSs
---------- /apps/modules/modulefiles/tools ----------
COMPSs/1.3
COMPSs/1.4
COMPSs/2.0
COMPSs/2.1
COMPSs/2.2
COMPSs/2.3
COMPSs/2.4
COMPSs/2.5
COMPSs/2.6
COMPSs/2.7
COMPSs/release(default)
COMPSs/trunk
$ module load COMPSs/release
load java/1.8.0u66 (PATH, MANPATH, JAVA_HOME, JAVA_ROOT, JAVA_BINDIR,
SDK_HOME, JDK_HOME, JRE_HOME)
load MKL/11.0.1 (LD_LIBRARY_PATH)
load PYTHON/2.7.3 (PATH, MANPATH, LD_LIBRARY_PATH, C_INCLUDE_PATH)
load COMPSs/release (PATH, MANPATH, COMPSS_HOME)
The following command can be run to check if the correct COMPSs version has been loaded:
$ enqueue_compss --version
COMPSs version <version>
Configuration
The COMPSs module contains all the COMPSs dependencies, including
Java, Python and MKL. Modifying any of these dependencies can cause
execution failures and thus, we do not recomend to change them.
Before running any COMPSs job please check your environment and, if
needed, comment out any line inside the .bashrc file that loads
custom COMPSs, Java, Python and/or MKL modules.
The COMPSs module needs to be loaded in all the nodes that will run
COMPSs jobs. Consequently, the module load must be included in
your .bashrc file. To do so, please run the following command with
the corresponding COMPSs version:
$ cat "module load COMPSs/release" >> ~/.bashrc
Log out and back in again to check that the file has been correctly edited. The next listing shows an example of the output generated by well loaded COMPSs installation.
$ exit
$ ssh USER@SC
load java/1.8.0u66 (PATH, MANPATH, JAVA_HOME, JAVA_ROOT, JAVA_BINDIR,
SDK_HOME, JDK_HOME, JRE_HOME)
load MKL/11.0.1 (LD_LIBRARY_PATH)
load PYTHON/2.7.3 (PATH, MANPATH, LD_LIBRARY_PATH, C_INCLUDE_PATH)
load COMPSs/release (PATH, MANPATH, COMPSS_HOME)
USER@SC$ enqueue_compss --version
COMPSs version <version>
Important
Please remember that COMPSs runs in several nodes and your current enviroment is not exported to them. Thus, all the needed environment variables must be loaded through the .bashrc file.
Important
Please remember that PyCOMPSs uses Python 2.7 by default. In order to use Python 3, the Python 2.7 module must be unloaded after loading COMPSs module, and then load the Python 3 module.
COMPSs Job submission
COMPSs jobs can be easily submited by running the enqueue_compss command. This command allows to configure any runcompss option and some particular queue options such as the queue system, the number of nodes, the wallclock time, the master working directory, the workers working directory and number of tasks per node.
Next, we provide detailed information about the enqueue_compss
command:
$ enqueue_compss -h
Usage: /apps/COMPSs/2.7/Runtime/scripts/user/enqueue_compss [queue_system_options] [COMPSs_options] application_name application_arguments
* Options:
General:
--help, -h Print this help message
--heterogeneous Indicates submission is going to be heterogeneous
Default: Disabled
Queue system configuration:
--sc_cfg=<name> SuperComputer configuration file to use. Must exist inside queues/cfgs/
Default: default
Submission configuration:
General submision arguments:
--exec_time=<minutes> Expected execution time of the application (in minutes)
Default: 10
--job_name=<name> Job name
Default: COMPSs
--queue=<name> Queue name to submit the job. Depends on the queue system.
For example (MN3): bsc_cs | bsc_debug | debug | interactive
Default: default
--reservation=<name> Reservation to use when submitting the job.
Default: disabled
--constraints=<constraints> Constraints to pass to queue system.
Default: disabled
--qos=<qos> Quality of Service to pass to the queue system.
Default: default
--cpus_per_task Number of cpus per task the queue system must allocate per task.
Note that this will be equal to the cpus_per_node in a worker node and
equal to the worker_in_master_cpus in a master node respectively.
Default: false
--job_dependency=<jobID> Postpone job execution until the job dependency has ended.
Default: None
--storage_home=<string> Root installation dir of the storage implementation
Default: null
--storage_props=<string> Absolute path of the storage properties file
Mandatory if storage_home is defined
Normal submission arguments:
--num_nodes=<int> Number of nodes to use
Default: 2
--num_switches=<int> Maximum number of different switches. Select 0 for no restrictions.
Maximum nodes per switch: 18
Only available for at least 4 nodes.
Default: 0
--agents=<string> Hierarchy of agents for the deployment. Accepted values: plain|tree
Default: tree
--agents Deploys the runtime as agents instead of the classic Master-Worker deployment.
Default: disabled
Heterogeneous submission arguments:
--type_cfg=<file_location> Location of the file with the descriptions of node type requests
File should follow the following format:
type_X(){
cpus_per_node=24
node_memory=96
...
}
type_Y(){
...
}
--master=<master_node_type> Node type for the master
(Node type descriptions are provided in the --type_cfg flag)
--workers=type_X:nodes,type_Y:nodes Node type and number of nodes per type for the workers
(Node type descriptions are provided in the --type_cfg flag)
Launch configuration:
--cpus_per_node=<int> Available CPU computing units on each node
Default: 48
--gpus_per_node=<int> Available GPU computing units on each node
Default: 0
--fpgas_per_node=<int> Available FPGA computing units on each node
Default: 0
--io_executors=<int> Number of IO executors on each node
Default: 0
--fpga_reprogram="<string> Specify the full command that needs to be executed to reprogram the FPGA with
the desired bitstream. The location must be an absolute path.
Default:
--max_tasks_per_node=<int> Maximum number of simultaneous tasks running on a node
Default: -1
--node_memory=<MB> Maximum node memory: disabled | <int> (MB)
Default: disabled
--node_storage_bandwidth=<MB> Maximum node storage bandwidth: <int> (MB)
Default: 450
--network=<name> Communication network for transfers: default | ethernet | infiniband | data.
Default: infiniband
--prolog="<string>" Task to execute before launching COMPSs (Notice the quotes)
If the task has arguments split them by "," rather than spaces.
This argument can appear multiple times for more than one prolog action
Default: Empty
--epilog="<string>" Task to execute after executing the COMPSs application (Notice the quotes)
If the task has arguments split them by "," rather than spaces.
This argument can appear multiple times for more than one epilog action
Default: Empty
--master_working_dir=<path> Working directory of the application
Default: .
--worker_working_dir=<name | path> Worker directory. Use: scratch | gpfs | <path>
Default: scratch
--worker_in_master_cpus=<int> Maximum number of CPU computing units that the master node can run as worker. Cannot exceed cpus_per_node.
Default: 24
--worker_in_master_memory=<int> MB Maximum memory in master node assigned to the worker. Cannot exceed the node_memory.
Mandatory if worker_in_master_cpus is specified.
Default: 50000
--worker_port_range=<min>,<max> Port range used by the NIO adaptor at the worker side
Default: 43001,43005
--jvm_worker_in_master_opts="<string>" Extra options for the JVM of the COMPSs Worker in the Master Node.
Each option separed by "," and without blank spaces (Notice the quotes)
Default:
--container_image=<path> Runs the application by means of a container engine image
Default: Empty
--container_compss_path=<path> Path where compss is installed in the container image
Default: /opt/COMPSs
--container_opts="<string>" Options to pass to the container engine
Default: empty
--elasticity=<max_extra_nodes> Activate elasticity specifiying the maximum extra nodes (ONLY AVAILABLE FORM SLURM CLUSTERS WITH NIO ADAPTOR)
Default: 0
--automatic_scaling=<bool> Enable or disable the runtime automatic scaling (for elasticity)
Default: true
--jupyter_notebook=<path>, Swap the COMPSs master initialization with jupyter notebook from the specified path.
--jupyter_notebook Default: false
Runcompss configuration:
Tools enablers:
--graph=<bool>, --graph, -g Generation of the complete graph (true/false)
When no value is provided it is set to true
Default: false
--tracing=<level>, --tracing, -t Set generation of traces and/or tracing level ( [ true | basic ] | advanced | scorep | arm-map | arm-ddt | false)
True and basic levels will produce the same traces.
When no value is provided it is set to 1
Default: 0
--monitoring=<int>, --monitoring, -m Period between monitoring samples (milliseconds)
When no value is provided it is set to 2000
Default: 0
--external_debugger=<int>,
--external_debugger Enables external debugger connection on the specified port (or 9999 if empty)
Default: false
--jmx_port=<int> Enable JVM profiling on specified port
Runtime configuration options:
--task_execution=<compss|storage> Task execution under COMPSs or Storage.
Default: compss
--storage_impl=<string> Path to an storage implementation. Shortcut to setting pypath and classpath. See Runtime/storage in your installation folder.
--storage_conf=<path> Path to the storage configuration file
Default: null
--project=<path> Path to the project XML file
Default: /apps/COMPSs/2.7//Runtime/configuration/xml/projects/default_project.xml
--resources=<path> Path to the resources XML file
Default: /apps/COMPSs/2.7//Runtime/configuration/xml/resources/default_resources.xml
--lang=<name> Language of the application (java/c/python)
Default: Inferred is possible. Otherwise: java
--summary Displays a task execution summary at the end of the application execution
Default: false
--log_level=<level>, --debug, -d Set the debug level: off | info | debug
Warning: Off level compiles with -O2 option disabling asserts and __debug__
Default: off
Advanced options:
--extrae_config_file=<path> Sets a custom extrae config file. Must be in a shared disk between all COMPSs workers.
Default: null
--trace_label=<string> Add a label in the generated trace file. Only used in the case of tracing is activated.
Default: None
--comm=<ClassName> Class that implements the adaptor for communications
Supported adaptors:
├── es.bsc.compss.nio.master.NIOAdaptor
└── es.bsc.compss.gat.master.GATAdaptor
Default: es.bsc.compss.nio.master.NIOAdaptor
--conn=<className> Class that implements the runtime connector for the cloud
Supported connectors:
├── es.bsc.compss.connectors.DefaultSSHConnector
└── es.bsc.compss.connectors.DefaultNoSSHConnector
Default: es.bsc.compss.connectors.DefaultSSHConnector
--streaming=<type> Enable the streaming mode for the given type.
Supported types: FILES, OBJECTS, PSCOS, ALL, NONE
Default: NONE
--streaming_master_name=<str> Use an specific streaming master node name.
Default: null
--streaming_master_port=<int> Use an specific port for the streaming master.
Default: null
--scheduler=<className> Class that implements the Scheduler for COMPSs
Supported schedulers:
├── es.bsc.compss.scheduler.data.DataScheduler
├── es.bsc.compss.scheduler.fifo.FIFOScheduler
├── es.bsc.compss.scheduler.fifodata.FIFODataScheduler
├── es.bsc.compss.scheduler.lifo.LIFOScheduler
├── es.bsc.compss.components.impl.TaskScheduler
└── es.bsc.compss.scheduler.loadbalancing.LoadBalancingScheduler
Default: es.bsc.compss.scheduler.loadbalancing.LoadBalancingScheduler
--scheduler_config_file=<path> Path to the file which contains the scheduler configuration.
Default: Empty
--library_path=<path> Non-standard directories to search for libraries (e.g. Java JVM library, Python library, C binding library)
Default: Working Directory
--classpath=<path> Path for the application classes / modules
Default: Working Directory
--appdir=<path> Path for the application class folder.
Default: /home/group/user
--pythonpath=<path> Additional folders or paths to add to the PYTHONPATH
Default: /home/group/user
--base_log_dir=<path> Base directory to store COMPSs log files (a .COMPSs/ folder will be created inside this location)
Default: User home
--specific_log_dir=<path> Use a specific directory to store COMPSs log files (no sandbox is created)
Warning: Overwrites --base_log_dir option
Default: Disabled
--uuid=<int> Preset an application UUID
Default: Automatic random generation
--master_name=<string> Hostname of the node to run the COMPSs master
Default:
--master_port=<int> Port to run the COMPSs master communications.
Only for NIO adaptor
Default: [43000,44000]
--jvm_master_opts="<string>" Extra options for the COMPSs Master JVM. Each option separed by "," and without blank spaces (Notice the quotes)
Default:
--jvm_workers_opts="<string>" Extra options for the COMPSs Workers JVMs. Each option separed by "," and without blank spaces (Notice the quotes)
Default: -Xms1024m,-Xmx1024m,-Xmn400m
--cpu_affinity="<string>" Sets the CPU affinity for the workers
Supported options: disabled, automatic, user defined map of the form "0-8/9,10,11/12-14,15,16"
Default: automatic
--gpu_affinity="<string>" Sets the GPU affinity for the workers
Supported options: disabled, automatic, user defined map of the form "0-8/9,10,11/12-14,15,16"
Default: automatic
--fpga_affinity="<string>" Sets the FPGA affinity for the workers
Supported options: disabled, automatic, user defined map of the form "0-8/9,10,11/12-14,15,16"
Default: automatic
--fpga_reprogram="<string>" Specify the full command that needs to be executed to reprogram the FPGA with the desired bitstream. The location must be an absolute path.
Default:
--io_executors=<int> IO Executors per worker
Default: 0
--task_count=<int> Only for C/Python Bindings. Maximum number of different functions/methods, invoked from the application, that have been selected as tasks
Default: 50
--input_profile=<path> Path to the file which stores the input application profile
Default: Empty
--output_profile=<path> Path to the file to store the application profile at the end of the execution
Default: Empty
--PyObject_serialize=<bool> Only for Python Binding. Enable the object serialization to string when possible (true/false).
Default: false
--persistent_worker_c=<bool> Only for C Binding. Enable the persistent worker in c (true/false).
Default: false
--enable_external_adaptation=<bool> Enable external adaptation. This option will disable the Resource Optimizer.
Default: false
--gen_coredump Enable master coredump generation
Default: false
--python_interpreter=<string> Python interpreter to use (python/python2/python3).
Default: python Version: 3
--python_propagate_virtual_environment=<true> Propagate the master virtual environment to the workers (true/false).
Default: true
--python_mpi_worker=<false> Use MPI to run the python worker instead of multiprocessing. (true/false).
Default: false
* Application name:
For Java applications: Fully qualified name of the application
For C applications: Path to the master binary
For Python applications: Path to the .py file containing the main program
* Application arguments:
Command line arguments to pass to the application. Can be empty.
MareNostrum 4
Basic queue commands
The MareNostrum supercomputer uses the SLURM (Simple Linux Utility for Resource Management) workload manager. The basic commands to manage jobs are listed below:
- sbatch Submit a batch job to the SLURM system
- scancel Kill a running job
- squeue -u <username> See the status of jobs in the SLURM queue
For more extended information please check the SLURM: Quick start user guide at https://slurm.schedmd.com/quickstart.html .
Tracking COMPSs jobs
When submitting a COMPSs job a temporal file will be created storing the job information. For example:
$ enqueue_compss \
--exec_time=15 \
--num_nodes=3 \
--cpus_per_node=16 \
--master_working_dir=. \
--worker_working_dir=gpfs \
--lang=python \
--log_level=debug \
<APP> <APP_PARAMETERS>
SC Configuration: default.cfg
Queue: default
Reservation: disabled
Num Nodes: 3
Num Switches: 0
GPUs per node: 0
Job dependency: None
Exec-Time: 00:15
Storage Home: null
Storage Properties: null
Other:
--sc_cfg=default.cfg
--cpus_per_node=48
--master_working_dir=.
--worker_working_dir=gpfs
--lang=python
--classpath=.
--library_path=.
--comm=es.bsc.compss.nio.master.NIOAdaptor
--tracing=false
--graph=false
--pythonpath=.
<APP> <APP_PARAMETERS>
Temp submit script is: /scratch/tmp/tmp.pBG5yfFxEo
$ cat /scratch/tmp/tmp.pBG5yfFxEo
#!/bin/bash
#
#SBATCH --job-name=COMPSs
#SBATCH --workdir=.
#SBATCH -o compss-%J.out
#SBATCH -e compss-%J.err
#SBATCH -N 3
#SBATCH -n 144
#SBATCH --exclusive
#SBATCH -t00:15:00
...
In order to trac the jobs state users can run the following command:
$ squeue
JOBID PARTITION NAME USER TIME_LEFT TIME_LIMIT START_TIME ST NODES CPUS NODELIST
474130 main COMPSs XX 0:15:00 0:15:00 N/A PD 3 144 -
The specific COMPSs logs are stored under the ~/.COMPSs/ folder;
saved as a local runcompss execution. For further details please check the
Executing COMPSs applications Section.
MinoTauro
Basic queue commands
The MinoTauro supercomputer uses the SLURM (Simple Linux Utility for Resource Management) workload manager. The basic commands to manage jobs are listed below:
- sbatch Submit a batch job to the SLURM system
- scancel Kill a running job
- squeue -u <username> See the status of jobs in the SLURM queue
For more extended information please check the SLURM: Quick start user guide at https://slurm.schedmd.com/quickstart.html .
Tracking COMPSs jobs
When submitting a COMPSs job a temporal file will be created storing the job information. For example:
$ enqueue_compss \
--exec_time=15 \
--num_nodes=3 \
--cpus_per_node=16 \
--master_working_dir=. \
--worker_working_dir=gpfs \
--lang=python \
--log_level=debug \
<APP> <APP_PARAMETERS>
SC Configuration: default.cfg
Queue: default
Reservation: disabled
Num Nodes: 3
Num Switches: 0
GPUs per node: 0
Job dependency: None
Exec-Time: 00:15
Storage Home: null
Storage Properties: null
Other:
--sc_cfg=default.cfg
--cpus_per_node=16
--master_working_dir=.
--worker_working_dir=gpfs
--lang=python
--classpath=.
--library_path=.
--comm=es.bsc.compss.nio.master.NIOAdaptor
--tracing=false
--graph=false
--pythonpath=.
<APP> <APP_PARAMETERS>
Temp submit script is: /scratch/tmp/tmp.pBG5yfFxEo
$ cat /scratch/tmp/tmp.pBG5yfFxEo
#!/bin/bash
#
#SBATCH --job-name=COMPSs
#SBATCH --workdir=.
#SBATCH -o compss-%J.out
#SBATCH -e compss-%J.err
#SBATCH -N 3
#SBATCH -n 48
#SBATCH --exclusive
#SBATCH -t00:15:00
...
In order to trac the jobs state users can run the following command:
$ squeue
JOBID PARTITION NAME USER ST TIME NODES NODELIST (REASON)
XXXX projects COMPSs XX R 00:02 3 nvb[6-8]
The specific COMPSs logs are stored under the ~/.COMPSs/ folder;
saved as a local runcompss execution. For further details please check the
Executing COMPSs applications Section.
Nord 3
Basic queue commands
The Nord3 supercomputer uses the LSF (Load Sharing Facility) workload manager. The basic commands to manage jobs are listed below:
- bsub Submit a batch job to the LSF system
- bkill Kill a running job
- bjobs See the status of jobs in the LSF queue
- bqueues Information about LSF batch queues
For more extended information please check the IBM Platform LSF Command Reference at https://www.ibm.com/support/knowledgecenter/en/SSETD4_9.1.2/lsf_kc_cmd_ref.html .
Tracking COMPSs jobs
When submitting a COMPSs job a temporal file will be created storing the job information. For example:
$ enqueue_compss \
--exec_time=15 \
--num_nodes=3 \
--cpus_per_node=16 \
--master_working_dir=. \
--worker_working_dir=gpfs \
--lang=python \
--log_level=debug \
<APP> <APP_PARAMETERS>
SC Configuration: default.cfg
Queue: default
Reservation: disabled
Num Nodes: 3
Num Switches: 0
GPUs per node: 0
Job dependency: None
Exec-Time: 00:15
Storage Home: null
Storage Properties: null
Other:
--sc_cfg=default.cfg
--cpus_per_node=16
--master_working_dir=.
--worker_working_dir=gpfs
--lang=python
--classpath=.
--library_path=.
--comm=es.bsc.compss.nio.master.NIOAdaptor
--tracing=false
--graph=false
--pythonpath=.
<APP> <APP_PARAMETERS>
Temp submit script is: /scratch/tmp/tmp.pBG5yfFxEo
$ cat /scratch/tmp/tmp.pBG5yfFxEo
#!/bin/bash
#
#BSUB -J COMPSs
#BSUB -cwd .
#BSUB -oo compss-%J.out
#BSUB -eo compss-%J.err
#BSUB -n 3
#BSUB -R "span[ptile=1]"
#BSUB -W 00:15
...
In order to trac the jobs state users can run the following command:
$ bjobs
JOBID USER STAT QUEUE FROM_HOST EXEC_HOST JOB_NAME SUBMIT_TIME
XXXX bscXX PEND XX login1 XX COMPSs Month Day Hour
The specific COMPSs logs are stored under the ~/.COMPSs/ folder;
saved as a local runcompss execution. For further details please check the
Executing COMPSs applications Section.
Enabling COMPSs Monitor
Configuration
As supercomputer nodes are connection restricted, the better way to enable the COMPSs Monitor is from the users local machine. To do so please install the following packages:
- COMPSs Runtime
- COMPSs Monitor
- sshfs
For further details about the COMPSs packages installation and configuration please refer to Installation and Administration Section. If you are not willing to install COMPSs in your local machine please consider to download our Virtual Machine available at our webpage.
Once the packages have been installed and configured, users need to
mount the sshfs directory as follows. The SC_USER stands for your
supercomputer’s user, the SC_ENDPOINT to the supercomputer’s public
endpoint and the TARGET_LOCAL_FOLDER to the local folder where you
wish to deploy the supercomputer files):
compss@bsc:~$ scp $HOME/.ssh/id_rsa.pub ${SC_USER}@mn1.bsc.es:~/id_rsa_local.pub
compss@bsc:~$ ssh SC_USER@SC_ENDPOINT \
"cat ~/id_rsa_local.pub >> ~/.ssh/authorized_keys; \
rm ~/id_rsa_local.pub"
compss@bsc:~$ mkdir -p TARGET_LOCAL_FOLDER/.COMPSs
compss@bsc:~$ sshfs -o IdentityFile=$HOME/.ssh/id_rsa -o allow_other \
SC_USER@SC_ENDPOINT:~/.COMPSs \
TARGET_LOCAL_FOLDER/.COMPSs
Whenever you wish to unmount the sshfs directory please run:
compss@bsc:~$ sudo umount TARGET_LOCAL_FOLDER/.COMPSs
Execution
Access the COMPSs Monitor through its webpage
(http://localhost:8080/compss-monitor by default) and log in with the
TARGET_LOCAL_FOLDER to enable the COMPSs Monitor for MareNostrum.
Please remember that to enable all the COMPSs Monitor features applications must be ran with the -m flag. For further details please check the Executing COMPSs applications Section.
Figure 19 illustrates how to login and Figure 20 shows the COMPSs Monitor main page for an application run inside a Supercomputer.
COMPSs Monitor login for Supercomputers
COMPSs Monitor main page for a test application at Supercomputers
Docker
What is Docker?
Docker is an open-source project that automates the deployment of applications inside software containers, by providing an additional layer of abstraction and automation of operating-system-level virtualization on Linux. In addition to the Docker container engine, there are other Docker tools that allow users to create complex applications (Docker-Compose) or to manage a cluster of Docker containers (Docker Swarm).
COMPSs supports running a distributed application in a Docker Swarm cluster.
Requirements
In order to use COMPSs with Docker, some requirements must be fulfilled:
- Have Docker and Docker-Compose installed in your local machine.
- Have an available Docker Swarm cluster and its Swarm manager ip and port to access it remotely.
- A Dockerhub account. Dockerhub is an online repository for Docker images. We don’t currently support another sharing method besides uploading to Dockerhub, so you will need to create a personal account. This has the advantage that it takes very little time either upload or download the needed images, since it will reuse the existing layers of previous images (for example the COMPSs base image).
Execution in Docker
The runcompss-docker execution workflow uses Docker-Compose, which is in charge of spawning the different application containers into the Docker Swarm manager. Then the Docker Swarm manager schedules the containers to the nodes and the application starts running. The COMPSs master and workers will run in the nodes Docker Swarm decides. To see where the masters and workers are located in runtime, you can use:
$ docker -H '<swarm_manager_ip:swarm_port>' ps -a
The execution of an application using Docker containers with COMPSs consists of 2 steps:
Execution step 1: Creation of the application image
The very first step to execute a COMPSs application in Docker is creating your application Docker image.
This must be done only once for every new application, and then you can run it as many times as needed. If the application is updated for whatever reason, this step must be done again to create and share the updated image.
In order to do this, you must use the compss_docker_gen_image tool, which is available in the standard COMPSs application. This tool is the responsible of taking your application, create the needed image, and upload it to Dockerhub to share it.
The image is created injecting your application into a COMPSs base image. This base image is available in Dockerhub. In case you need it, you can pull it using the following command:
$ docker pull compss/compss
The compss_docker_gen_image script receives 2 parameters:
| --c, --context-dir | |
Specifies the context directory path of the application. This path MUST BE ABSOLUTE, not relative. The context directory is a local directory that must contain the needed binaries and input files of the app (if any). In its simplest case, it will contain the executable file (a .jar for example). Keep the context-directory as lightest as possible. For example: –context-dir=’/home/compss-user/my-app-dir’ (where ’my-app-dir’ contains ’app.jar’, ’data1.dat’ and ’data2.csv’). For more details, this context directory will be recursively copied into a COMPSs base image. Specifically, it will create all the path down to the context directory inside the image. | |
| --image-name | Specifies a name for the created image. It MUST have this format:
’DOCKERHUB-USERNAME/image-name’.
The DOCKERHUB_USERNAME must be the username of your personal
Dockerhub account.
The image_name can be whatever you want, and will be used as the
identifier for the image in Dockerhub. This name will be the one
you will use to execute the application in Docker.
For example, if my Dockerhub username is john123 and I want my
image to be named “my-image-app”:
As stated before, this is needed to share your container application image with the nodes that need it. Image tags are also supported (for example “john123/my-image-app:1.23). |
Important
After creating the image, be sure to write down the absolute
context-directory and the absolute classpath (the absolute path to the
executable jar). You will need it to run the application using
runcompss-docker. In addition, if you plan on distributing the
application, you can use the Dockerhub image’s information tab to
write them, so the application users can retrieve them.
Execution step 2: Run the application
To execute COMPSs in a Docker Swarm cluster, you must use the
runcompss-docker command, instead of runcompss.
The command runcompss-docker has some additional arguments
that will be needed by COMPSs to run your application in a distributed
Docker Swarm cluster environment. The rest of typical arguments
(classpath for example) will be delegated to runcompss command.
These additional arguments must go before the typical runcompss arguments. The runcompss-docker additional arguments are:
| --w, --worker-containers | |
Specifies the number of worker containers the app will execute
on. One more container will be created to host the master. If you
have enough nodes in the Swarm cluster, each container will be
executed by one node. This is the default schedule strategy used by
Swarm.
For example: --worker-containers=3 | |
| --s, --swarm-manager | |
Specifies the Swarm manager ip and port (format: ip:port).
For example: --swarm-manager=’129.114.108.8:4000’ | |
| --i, --image-name | |
Specify the image name of the application image in Dockerhub.
Remember you must generate this with compss_docker_gen_image
Remember as well that the format must be:
’DOCKERHUB_USERNAME/APP_IMAGE_NAME:TAG’ (the :TAG is optional).
For example: --image-name=’john123/my-compss-application:1.9’ | |
| --c, --context-dir | |
Specifies the context directory of the app. It must be specified
by the application image provider.
For example: --context-dir=’/home/compss-user/my-app-context-dir’ | |
As optional arguments:
| --c-cpu-units | Specifies the number of cpu units used by each container (default value is 4).
For example: *--c-cpu-units:=16 |
| --c-memory | Specifies the physical memory used by each container in GB (default value is 8 GB).
For example, in this case, each container would use as maximum 32 GB
of physical memory: --c-memory=32 |
Here is the format you must use with runcompss-docker command:
$ runcompss-docker --worker-containers=N \
--swarm-manager='<ip>:<port>' \
--image-name='DOCKERHUB_USERNAME/image_name' \
--context-dir='CTX_DIR' \
[rest of classic runcompss args]
Or alternatively, in its shortest form:
$ runcompss-docker --w=N --s='<ip>:<port>' --i='DOCKERHUB_USERNAME/image_name' --c='CTX_DIR' \
[rest of classic runcompss args]
Execution with TLS
If your cluster uses TLS or has been created using Docker-Machine, you will have to export two environment variables before using runcompss-docker:
On one hand, DOCKER_TLS_VERIFY environment variable will tell Docker that you are using TLS:
export DOCKER_TLS_VERIFY="1"
On the other hand, DOCKER_CERT_PATH variable will tell Docker where to find your TLS certificates. As an example:
export DOCKER_CERT_PATH="/home/compss-user/.docker/machine/machines/my-manager-node"
In case you have created your cluster using docker-machine, in order to know what your DOCKER_CERT_PATH is, you can use this command:
$ docker-machine env my-swarm-manager-node-name | grep DOCKER_CERT_PATH
In which swarm-manager-node-name must be changed by the name
docker-machine has assigned to your swarm manager node.
With these environment variables set, you are ready to use
runcompss-docker in a cluster using TLS.
Execution results
The execution results will be retrieved from the master container of your application.
If your context-directory name is ’matmul’, then your results will be saved in the ’matmul-results’ directory, which will be located in the same directory you executed runcompss-docker on.
Inside the ’matmul-results’ directory you will have:
- A folder named ’matmul’ with all the result files that were in the same directory as the executable when the application execution ended. More precisely, this will contain the context-directory state right after finishing your application execution. Additionally, and for more advanced debug purposes, you will have some intermediate files created by runcompss-docker (Dockerfile, project.xml, resources.xml), in case you want to check for more complex errors or details.
- A folder named ’debug’, which (in case you used the runcompss debug option (-d)), will contain the ’.COMPSs’ directory, which contains another directory in which there are the typical debug files runtime.log, jobs, etc. Remember .COMPSs is a hidden directory, take this into account if you do ls inside the debug directory (add the -a option).
To make it simpler, we provide a tree visualization of an example of what your directories should look like after the execution. In this case we executed the Matmul example application that we provide you:
Result and log folders of a Matmul execution with COMPSs and Docker
Execution examples
Next we will use the Matmul application as an example of a Java application running with COMPSs and Docker.
Imagine we have our Matmul application in /home/john/matmul and
inside the matmul directory we only have the file matmul.jar.
We have created a Dockerhub account with username ’john123’.
The first step will be creating the image:
$ compss_docker_gen_image --context-dir='/home/john/matmul' \
--image-name='john123/matmul-example'
Now, we write down the context-dir (/home/john/matmul) and the
classpath (/home/john/matmul/matmul.jar). We do this because they will be
needed for future executions.
Since the image is created and uploaded, we won’t need to do this step
anymore.
Now we are going to execute our Matmul application in a Docker cluster.
Take as assumptions:
- We will use 5 worker docker containers.
- The swarm-manager ip will be 129.114.108.8, with the Swarm manager listening to the port 4000.
- We will use debug (-d).
- Finally, as we would do with the typical runcompss, we specify the main class name and its parameters (16 and 4 in this case).
In addition, we know from the former step that the image name is
john123/matmul-example, the context directory is
/home/john/matmul, and the classpath is
/home/john/matmul/matmul.jar. And this is how you would run
runcompss-docker:
$ runcompss-docker --worker-containers=5 \
--swarm-manager='129.114.108.8:4000' \
--context-dir='/home/john/matmul' \
--image-name='john123/matmul-example' \
--classpath=/home/john/matmul/matmul.jar \
-d \
matmul.objects.Matmul 16 4
Here we show another example using the short arguments form, with the KMeans example application, that is also provided as an example COMPSs application to you:
First step, create the image once:
$ compss_docker_gen_image --context-dir='/home/laura/apps/kmeans' \
--image-name='laura-67/my-kmeans'
And now execute with 30 worker containers, and Swarm located in ’110.3.14.159:26535’.
$ runcompss-docker --w=30 \
--s='110.3.14.159:26535' \
--c='/home/laura/apps/kmeans' \
--image-name='laura-67/my-kmeans' \
--classpath=/home/laura/apps/kmeans/kmeans.jar \
kmeans.KMeans
Chameleon
What is Chameleon?
The Chameleon project is a configurable experimental environment for large-scale cloud research based on a OpenStack KVM Cloud. With funding from the National Science Foundation (NSF), it provides a large-scale platform to the open research community allowing them explore transformative concepts in deeply programmable cloud services, design, and core technologies. The Chameleon testbed, is deployed at the University of Chicago and the Texas Advanced Computing Center and consists of 650 multi-core cloud nodes, 5PB of total disk space, and leverage 100 Gbps connection between the sites.
The project is led by the Computation Institute at the University of Chicago and partners from the Texas Advanced Computing Center at the University of Texas at Austin, the International Center for Advanced Internet Research at Northwestern University, the Ohio State University, and University of Texas at San Antoni, comprising a highly qualified and experienced team. The team includes members from the NSF supported FutureGrid project and from the GENI community, both forerunners of the NSFCloud solicitation under which this project is funded. Chameleon will also sets of partnerships with commercial and academic clouds, such as Rackspace, CERN and Open Science Data Cloud (OSDC).
For more information please check https://www.chameleoncloud.org/ .
Execution in Chameleon
Currently, COMPSs can only handle the Chameleon infrastructure as a cluster (deployed inside a lease). Next, we provide the steps needed to execute COMPSs applications at Chameleon:
- Make a lease reservation with 1 minimum node (for the COMPSs master instance) and a maximum number of nodes equal to the number of COMPSs workers needed plus one
- Instantiate the master image (based on the published image COMPSs__CC-CentOS7)
- Attach a public IP and login to the master instance (the instance is correctly contextualized for COMPSs executions if you see a COMPSs login banner)
- Set the instance as COMPSs master by running
/etc/init.d/chameleon_init start - Copy your CH file (API credentials) to the Master and source it
- Run the
chameleon_cluster_setupscript and fill the information when prompted (you will be asked for the name of the master instance, the reservation id and number of workers). This scripts may take several minutes since it sets up the all cluster. - Execute your COMPSs applications normally using the
runcompssscript
As an example you can check this video https://www.youtube.com/watch?v=BrQ6anPHjAU performing a full setup and execution of a COMPSs application at Chameleon.
Dynamic infrastructures
Opposing to well-established deployments with an almost-static set of computing resources and hardly-varying interconnection conditions such as a single-computer, a cluster or a supercomputer; dynamic infrastructures, like Fog environments, require a different kind of deployment able to adapt to rapidly-changing conditions. Such infrastructures are likely to comprise several mobile devices whose connectivity to the infrastructure is temporary. When the device is within the network range, it joins an already existing COMPSs deployment and interacts with the other resources to offload tasks onto them or viceversa. Eventually, the connectivity of that mobile device could be disrupted to never reestablish. If the leaving device was used as a worker node, the COMPSs master needs to react to the departure and reassign the tasks running on that node. If the device was the master node, it should be able to carry on with the computation being isolated from the rest of the infrastructure or with another set of available resources.
What are COMPSs Agents?
COMPSs Agents is a deployment approach especially designed to fit in this kind of environments. Each device is an autonomous individual with processing capabilities hosting the execution of a COMPSs runtime as a background service. Applications - running on that device or on another - can contact this service to request the execution of a function in a serverless, stateless manner (resembling the Function-as-a-Service model). If the requested function follows the COMPSs programming model, the runtime will parallelise its execution as if it were the main function of a regular COMPSs application.
Agents can associate with other agents by offering their embedded computing resources to execute functions to achieve a greater purpose; in exchange, they receive a platform where they can offload their computation in the same manner, and, thus, achieve lower response times. As opossed to the master-worker approach followed by the classic COMPSs deployment, where a single node produces the all the workload, in COMPSs Agents deployments, any of the nodes within the platform becomes a potential source of computation to distribute. Therefore, this master-centric approach where workload producer to orchestrate holistically the execution is no longer valid. Besides, concentrating all the knowledge of several applications and handling the changes of infrastructure represents an important computational burden for the resource assuming the master role, especially if it is a resource-scarce device like a mobile. For this two reasons, COMPSs agents proposes a hierachic approach to organize the nodes. Each node will only be aware of some devices with which it has direct connection and only decides whether the task runs on its embedded computing devices or if the responsability of executing the task is delegated onto one of the other agents. In the latter case, the receiver node will face the same problem and decide whether it should host the execution or forward it to a different node.
The following image illustrates an example of a COMPSs agents hierarchy that could be deployed in any kind of facilities; for instance, a university campus. In this case, students only interact directly with their mobile phones and laptops to run their applications; however, the computing workload produced by them is distributed across the whole system. To do so, the mobile devices need to connect to one of the edge devices devices scattered across the facilities acting as a Wi-Fi Hotspot (in the example, raspberry Pi) which runs a COMPSs agent. To submit the operation execution to the platform, mobile devices can either contact a COMPSs agent running in the device or the application can directly contact the remote agent running on the rPI. All rPi agents are connected to an on-premise server within the campus that also runs a COMPSs Agent. Upon an operation request by a user device, the rPi can host the computation on its own devices or forward the request to one of its neighbouring agents: the on-premise server or another user’s device running a COMPSs agent. In the case that the rPi decides to move up the request through the hierarchy, the on-premise server faces a similar problem: hosting the computation on its local devices, delegating the execution onto one of the rPi – which in turn could forward the execution back to another user’s device –, or submit the request to a cloud. Internally, the Cloud can also be organized with COMPSs Agents hierarchy; thus, one of its nodes can act as the gateway to receive external requests and share the workload across the whole system.
Deploying a COMPSs Agent
COMPSs Agents are deployed using the compss_agent_start command:
compss@bsc:~$ compss_agent_start [OPTION]
There is one mandatory parameter --hostname that indicates the name that other agents and itself use to refer to the agent. Bear in mind that agents are not able to dynamically modify its classpath; therefore, the --classpath parameter becomes important to indicate the application available on the agent. Any public method available on the classpath is an execution request candidate.
The following command raises an agent with name 192.168.1.100 and any of the public methods of the classes encapsulated in the jarfile /app/path.jar can be executed.
compss@bsc:~$ compss_agent_start --hostname=192.168.1.100 --classpath=/app/path.jar
The compss_agent_start command allows users to set up the COMPSs runtime by specifyng different options in the same way as done for the runcompss command. To indicate the available resources, the device administrator can use the --project and --resources option exactly in the same way as for the runcompss command. For further details on how to dynamically modify the available resources, please, refer to section Modifying the available resources.
Currently, COMPSs agents allow interaction through two interfaces: the Comm interface and the REST interface. The Comm interface leverages on a propietary protocol to submit operations and request updates on the current resource configuration of the agent. Although users and applications can use this interface, its design purpose is to enable high-performance interactions among agents rather than supporting user interaction. The REST interface takes the completely opposed approach; Users should interact with COMPSs agents through it rather than submitting tasks with the Comm interface. The COMPSs agent allows to enact both interfaces at a time; thus, users can manually submit operations using the REST interface, while other agents can use the Comm interface. However, the device owner can decide at deploy time which of the interfaces will be available on the agent and through which port the API will be exposed using the rest_port and comm_port options of the compss_agent_start command. Other agents can be configured to interact with the agent through any of the interfaces. For further details on how to configure the interaction with another agent, please, refer to section Modifying the available resources.
compss@bsc:~$ compss_agent_start -h
Usage: /opt/COMPSs/Runtime/scripts/user/compss_agent_start [OPTION]...
COMPSs options:
--appdir=<path> Path for the application class folder.
Default: /home/flordan/git/compss/framework/builders
--classpath=<path> Path for the application classes / modules
Default: Working Directory
--comm=<className> Class that implements the adaptor for communications with other nodes
Supported adaptors:
├── es.bsc.compss.nio.master.NIOAdaptor
├── es.bsc.compss.gat.master.GATAdaptor
├── es.bsc.compss.agent.rest.Adaptor
└── es.bsc.compss.agent.comm.CommAgentAdaptor
Default: es.bsc.compss.agent.comm.CommAgentAdaptor
--comm_port=<int> Port on which the agent sets up a Comm interface. (<=0: Disabled)
-d, --debug Enable debug. (Default: disabled)
--hostname Name with which itself and other agents will identify the agent.
--library_path=<path> Non-standard directories to search for libraries (e.g. Java JVM library, Python library, C binding library)
Default: Working Directory
--log_dir=<path> Log directory. (Default: /tmp/)
--log_level=<level> Set the debug level: off | info | api | debug | trace
Default: off
--master_port=<int> Port to run the COMPSs master communications.
(Only when es.bsc.compss.nio.master.NIOAdaptor is used. The value is overriden by the comm_port value.)
Default: [43000,44000]
--pythonpath=<path> Additional folders or paths to add to the PYTHONPATH
Default: /home/flordan/git/compss/framework/builders
--project=<path> Path of the project file
(Default: /opt/COMPSs/Runtime/configuration/xml/projects/examples/local/project.xml)
--resources=<path> Path of the resources file
(Default: /opt/COMPSs/Runtime/configuration/xml/resources/examples/local/resources.xml)
--rest_port=<int> Port on which the agent sets up a REST interface. (<=0: Disabled)
--scheduler=<className> Class that implements the Scheduler for COMPSs
Supported schedulers:
├── es.bsc.compss.scheduler.data.DataScheduler
├── es.bsc.compss.scheduler.fifo.FIFOScheduler
├── es.bsc.compss.scheduler.fifodata.FIFODataScheduler
├── es.bsc.compss.scheduler.lifo.LIFOScheduler
├── es.bsc.compss.components.impl.TaskScheduler
└── es.bsc.compss.scheduler.loadbalancing.LoadBalancingScheduler
Default: es.bsc.compss.scheduler.loadbalancing.LoadBalancingScheduler
--scheduler_config_file=<path> Path to the file which contains the scheduler configuration.
Default: Empty
--summary Displays a task execution summary at the end of the application execution
Default: false
Other options:
--help prints this message
Executing an operation
The compss_agent_call_operation commands interacts with the REST interface of the COMPSs agent to submit an operation.
compss@bsc:~$ compss_agent_call_operation [options] application_name application_arguments
The command has two mandatory flags --master_node and --master_port to indicate the endpoint of the COMPSs Agent. By default, the command submits an execution of the main method of the Java class with the name passed in as the application_name and gathering all the application arguments in a single String[] instance. To execute Python methods, the user can use the --lang=PYTHON option and the Agent will execute the python script with the name passed in as application_name. Operation invocations can be customized by using other options of the command. The --method_name option allow to execute a specific method; in the case of specifying a method, each of the parameters will be passed in as a different parameter to the function and it is necessary to indicate the --array flag to encapsulate all the parameters as an array.
compss@bsc.es:~$ compss_agent_call_operation -h
Usage: compss_agent_call_operation [options] application_name application_arguments
* Options:
General:
--help, -h Print this help message
--opts Show available options
--version, -v Print COMPSs version
--master_node=<string> Node where to run the COMPSs Master
Mandatory
--master_port=<string> Node where to run the COMPSs Master
Mandatory
Launch configuration:
--cei=<string> Canonical name of the interface declaring the methods
Default: No interface declared
--lang=<string> Language implementing the operation
Default: JAVA
--method_name=<string> Name of the method to invoke
Default: main and enables array parameter
--parameters_array, --array Parameters are encapsulated as an array
Default: disabled
For example, to submit the execution of the demoFunction method from the es.bsc.compss.tests.DemoClass class passing in a single parameter with value 1 on the agent 127.0.0.1 with a REST interface listening on port 46101, the user should execute the following example command:
compss@bsc.es:~$ compss_agent_call_operation --master_node="127.0.0.1" --master_port="46101" --method_name="demoFunction" es.bsc.compss.test.DemoClass 1
For the agent to detect inner tasks within the operation execution, the COMPSs Programming model requires an interface selecting the methods to be replaced by asyncrhonous task creations. An invoker should use the --cei option to specify the name of the interface selecting the tasks.
Modifying the available resources
Finally, the COMPSs framework offers tree commands to control dynamically the pool of resources available for the runtime un one agent. These commands are compss_agent_add_resources, compss_agent_reduce_resources and compss_agent_lost_resources.
The compss_agent_add_resources commands interacts with the REST interface of the COMPSs agent to attach new resources to the Agent.
compss@bsc.es:~$ compss_agent_add_resources [options] resource_name [<adaptor_property_name=adaptor_property_value>]
By default, the command modifies the resource pool of the agent deployed on the node running the command listenning on port 46101; however, this can be modified by using the options --agent_node and --agent_port to indicate the endpoint of the COMPSs Agent. The other options passed in to the command modify the characteristics of the resources to attach; by default, it adds one single CPU core. However, it also allows to modify the amount of GPU cores, FPGAs, memory type and size and OS details.
compss@bsc.es:~$ compss_agent_add_resources -h
Usage: compss_agent_add_resources [options] resource_name [<adaptor_property_name=adaptor_property_value>]
* Options:
General:
--help, -h Print this help message
--opts Show available options
--version, -v Print COMPSs version
--agent_node=<string> Name of the node where to add the resource
Default:
--agent_port=<string> Port of the node where to add the resource
Default:
Resource description:
--comm=<string> Canonical class name of the adaptor to interact with the resource
Default: es.bsc.compss.agent.comm.CommAgentAdaptor
--cpu=<integer> Number of cpu cores available on the resource
Default: 1
--gpu=<integer> Number of gpus devices available on the resource
Default: 0
--fpga=<integer> Number of fpga devices available on the resource
Default: 0
--mem_type=<string> Type of memory used by the resource
Default: [unassigned]
--mem_size=<string> Size of the memory available on the resource
Default: -1
--os_type=<string> Type of operating system managing the resource
Default: [unassigned]
--os_distr=<string> Distribution of the operating system managing the resource
Default: [unassigned]
--os_version=<string> Version of the operating system managing the resource
Default: [unassigned]
If resource_name matches the name of the Agent, the capabilities of the device are increased according to the description; otherwise, the runtime adds a remote worker to the resource pool with the specified characteristics. Notice that, if there is another resource within the pool with the same name, the agent will increase the resources of such node instead of adding it as a new one. The --comm option is used for selecting which adaptor is used for interacting with the remote node; the default adaptor (CommAgent) interacts with the remote node through the Comm interface of the COMPSs agent.
The following command adds a new Agent onto the pool of resources of the Agent deployed at IP 192.168.1.70 with a REST Interface on port 46101. The new agent, which has 4 CPU cores, is deployed on IP 192.168.1.72 and has a Comm interface endpoint on port 46102.
compss@bsc.es:~$ compss_agent_add_resources --agent_node=192.168.1.70 --agent_port=46101 --cpu=4 192.168.1.72 Port=46102
Conversely, the compss_agent_reduce_resources command allows to reduce the number of resources configured in an agent. Executing the command causes the target agent to reduce the specified amount of resources from one of its configured neighbors. At the moment the reception of the resource removal request, the agent might be using actively using those remote resources by executing some tasks. If that is the case, the agent will register the resource reduction request, stop submitting more workload to the corresponding node, and, when the idle resources of the node match the request, the agent removes them from the pool. If upon the completion of the compss_agent_reduce_resources command no resources are associated to the reduced node, the node is completely removed from the resource pool of the agent. The options and default values are the same than for the compss_agent_add_resources command. Notice that --comm option is not available because only one resource can associated to that name regardless the selected adaptor.
compss@bsc.es:~$ compss_agent_reduce_resources -h
Usage: compss_agent_reduce_resources [options] resource_name
* Options:
General:
--help, -h Print this help message
--opts Show available options
--version, -v Print COMPSs version
--agent_node=<string> Name of the node where to add the resource
Default:
--agent_port=<string> Port of the node where to add the resource
Default:
Resource description:
--cpu=<integer> Number of cpu cores available on the resource
Default: 1
--gpu=<integer> Number of gpus devices available on the resource
Default: 0
--fpga=<integer> Number of fpga devices available on the resource
Default: 0
--mem_type=<string> Type of memory used by the resource
Default: [unassigned]
--mem_size=<string> Size of the memory available on the resource
Default: -1
--os_type=<string> Type of operating system managing the resource
Default: [unassigned]
--os_distr=<string> Distribution of the operating system managing the resource
Default: [unassigned]
--os_version=<string> Version of the operating system managing the resource
Default: [unassigned]
Finally, the last command to control the pool of resources configured, compss_agent_lost_resources, immediately removes from an agent’s pool all the resources corresponding to the remote node associated to that name.
compss@bsc.es:~$ compss_agent_lost_resources [options] resource_name
In this case, the only available options are those used for identifying the endpoint of the agent:--agent_node and --agent_port. As with the previous commands, by default, the request is submitted to the agent deployed on the IP address 127.0.0.1 and listenning on port 46101.
Tracing
COMPSs is instrumented with EXTRAE, which enables to produce PARAVER traces for performance profiling. This section is intended to walk you through the tracing of your COMPSs applications in order to analyse the performance with great detail.
COMPSs applications tracing
COMPSs Runtime has a built-in instrumentation system to generate post-execution tracefiles of the applications’ execution. The tracefiles contain different events representing the COMPSs master state, the tasks’ execution state, and the data transfers (transfers’ information is only available when using NIO adaptor), and are useful for both visual and numerical performance analysis and diagnosis. The instrumentation process essentially intercepts and logs different events, so it adds overhead to the execution time of the application.
The tracing system uses Extrae [1] to generate tracefiles of the execution that, in turn, can be visualized with Paraver [2]. Both tools are developed and maintained by the Performance Tools team of the BSC and are available on its web page http://www.bsc.es/computer-sciences/performance-tools.
For each worker node and the master, Extrae keeps track of the events in an intermediate format file (with .mpit extension). At the end of the execution, all intermediate files are gathered and merged with Extrae’s mpi2prv command in order to create the final tracefile, a Paraver format file (.prv). See the Visualization Section for further information about the Paraver tool.
When instrumentation is activated, Extrae outputs several messages corresponding to the tracing initialization, intermediate files’ creation, and the merging process.
At present time, COMPSs tracing features two execution modes:
- Basic
- Aimed at COMPSs applications developers
- Advanced
- For COMPSs developers and users with access to its source code or custom installations
Next sections describe the information provided by each mode and how to use them.
Basic Mode
This mode is aimed at COMPSs’ apps users and developers. It instruments computing threads and some management resources providing information about tasks’ executions, data transfers, and hardware counters if PAPI is available (see PAPI: Hardware Counters for more info).
Basic Mode Usage
In order to activate basic tracing one needs to provide one of the following arguments to the execution command:
-t--tracing--tracing=basic--tracing=true
Examples given:
$ runcompss --tracing application_name application_args
Figure 22 was generated as follows:
$ runcompss \
--lang=java \
--tracing \
--classpath=/path/to/jar/kmeans.jar \
kmeans.KMeans
When tracing is activated, Extrae generates additional output to help the user ensure that instrumentation is turned on and working without issues. On basic mode this is the output users should see when tracing is working correctly:
*** RUNNING JAVA APPLICATION KMEANS
Resolved: /path/to/jar/kmeans.jar:
----------------- Executing kmeans.Kmeans --------------------------
Welcome to Extrae VERSION
Extrae: Parsing the configuration file (/opt/COMPSs/Runtime/configuration/xml/tracing/extrae_basic.xml) begins
Extrae: Tracing package is located on /opt/COMPSs/Dependencies/extrae/
Extrae: Generating intermediate files for Paraver traces.
Extrae: PAPI domain set to USER for HWC set 1
Extrae: HWC set 1 contains following counters < PAPI_TOT_INS (0x80000032) PAPI_TOT_CYC (0x8000003b) PAPI_LD_INS (0x80000035) PAPI_SR_INS (0x80000036) > - changing every 500000000 nanoseconds
Extrae: PAPI domain set to USER for HWC set 2
Extrae: HWC set 2 contains following counters < PAPI_TOT_INS (0x80000032) PAPI_TOT_CYC (0x8000003b) PAPI_LD_INS (0x80000035) PAPI_SR_INS (0x80000036) PAPI_L2_DCM (0x80000002) > - changing every 500000000 nanoseconds
WARNING: COMPSs Properties file is null. Setting default values
[(751) API] - Deploying COMPSs Runtime v<version>
[(753) API] - Starting COMPSs Runtime v<version>
[(753) API] - Initializing components
[(1142) API] - Ready to process tasks
...
...
...
merger: Output trace format is: Paraver
merger: Extrae VERSION
mpi2prv: Assigned nodes < Marginis >
mpi2prv: Assigned size per processor < <1 Mbyte >
mpi2prv: File set-0/TRACE@Marginis.0000001904000000000000.mpit is object 1.1.1 on node Marginis assigned to processor 0
mpi2prv: File set-0/TRACE@Marginis.0000001904000000000001.mpit is object 1.1.2 on node Marginis assigned to processor 0
mpi2prv: File set-0/TRACE@Marginis.0000001904000000000002.mpit is object 1.1.3 on node Marginis assigned to processor 0
mpi2prv: File set-0/TRACE@Marginis.0000001980000001000000.mpit is object 1.2.1 on node Marginis assigned to processor 0
mpi2prv: File set-0/TRACE@Marginis.0000001980000001000001.mpit is object 1.2.2 on node Marginis assigned to processor 0
mpi2prv: File set-0/TRACE@Marginis.0000001980000001000002.mpit is object 1.2.3 on node Marginis assigned to processor 0
mpi2prv: File set-0/TRACE@Marginis.0000001980000001000003.mpit is object 1.2.4 on node Marginis assigned to processor 0
mpi2prv: File set-0/TRACE@Marginis.0000001980000001000004.mpit is object 1.2.5 on node Marginis assigned to processor 0
mpi2prv: Time synchronization has been turned off
mpi2prv: A total of 9 symbols were imported from TRACE.sym file
mpi2prv: 0 function symbols imported
mpi2prv: 9 HWC counter descriptions imported
mpi2prv: Checking for target directory existance... exists, ok!
mpi2prv: Selected output trace format is Paraver
mpi2prv: Stored trace format is Paraver
mpi2prv: Searching synchronization points... done
mpi2prv: Time Synchronization disabled.
mpi2prv: Circular buffer enabled at tracing time? NO
mpi2prv: Parsing intermediate files
mpi2prv: Progress 1 of 2 ... 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% done
mpi2prv: Processor 0 succeeded to translate its assigned files
mpi2prv: Elapsed time translating files: 0 hours 0 minutes 0 seconds
mpi2prv: Elapsed time sorting addresses: 0 hours 0 minutes 0 seconds
mpi2prv: Generating tracefile (intermediate buffers of 838848 events)
This process can take a while. Please, be patient.
mpi2prv: Progress 2 of 2 ... 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% done
mpi2prv: Warning! Clock accuracy seems to be in microseconds instead of nanoseconds.
mpi2prv: Elapsed time merge step: 0 hours 0 minutes 0 seconds
mpi2prv: Resulting tracefile occupies 991743 bytes
mpi2prv: Removing temporal files... done
mpi2prv: Elapsed time removing temporal files: 0 hours 0 minutes 0 seconds
mpi2prv: Congratulations! ./trace/kmeans.Kmeans_compss_trace_1460456106.prv has been generated.
[ API] - Execution Finished
Extrae: Tracing buffer can hold 100000 events
Extrae: Circular buffer disabled.
Extrae: Warning! <dynamic-memory> tag will be ignored. This library does support instrumenting dynamic memory calls.
Extrae: Warning! <input-output> tag will be ignored. This library does support instrumenting I/O calls.
Extrae: Dynamic memory instrumentation is disabled.
Extrae: Basic I/O memory instrumentation is disabled.
Extrae: Parsing the configuration file (/opt/COMPSs/Runtime/scripts/user/../../configuration/xml/tracing/extrae_basic.xml) has ended
Extrae: Intermediate traces will be stored in /home/kurtz/compss/tests_local/app10
Extrae: Tracing mode is set to: Detail.
Extrae: Successfully initiated with 1 tasks and 1 threads
It contains diverse information about the tracing, for example, Extrae
version used (VERSION will be replaced by the actual number during
executions), the XML configuration file used (extrae_basic.xml), the
amount of threads instrumented (objects through 1.1.1 to 1.2.5),
available hardware counters (PAPI_TOT_INS (0x80000032) …
PAPI_L3_TCM (0x80000008) ) or the name of the generated tracefile
(./trace/kmeans. Kmeans_compss_trace_1460456106.prv). When using
NIO communications adaptor with debug activated, the log of each worker
also contains the Extrae initialization information.
N.B. when using Python, COMPSs needs to perform an extra merging step in order to add the Python-produced events to the main tracefile. If Python events are not shown, check runtime.log file and search for the following expected output of this merging process to find possible errors:
[(9788)(2016-11-15 11:22:27,687) Tracing] @generateTrace - Tracing: Generating trace
[(9851)(2016-11-15 11:22:27,750) Tracing] @<init> - Trace's merger initialization successful
[(9851)(2016-11-15 11:22:27,750) Tracing] @merge - Parsing master sync events
[(9905)(2016-11-15 11:22:27,804) Tracing] @merge - Proceeding to merge task traces into master
[(9944)(2016-11-15 11:22:27,843) Tracing] @merge - Merging finished,
[(9944)(2016-11-15 11:22:27,843) Tracing] @merge - Temporal task folder removed.
Instrumented Threads in Basic Mode
Basic traces instrument the following threads:
- Master node (3 threads)
- COMPSs runtime
- Task Dispatcher
- Access Processor
- Worker node (1 + Computing Units)
- Main thread
- Number of threads available for computing
Information Available in Basic Traces
The basic mode tracefiles contain three kinds of information:
- Events
- Marking diverse situations such as the runtime start, tasks’ execution or synchronization points.
- Communications
- Showing the transfers and requests of the parameters needed by COMPSs tasks.
- Hardware counters
- Of the execution obtained with Performance API (see PAPI: Hardware Counters)
Basic Trace Example
Figure 22 is a tracefile generated by the execution of a k-means clustering algorithm. Each timeline contains information of a different resource, and each event’s name is on the legend. Depending on the number of computing threads specified for each worker, the number of timelines varies. However the following threads are always shown:
- Master - Thread 1.1.1
- This timeline shows the actions performed by the main thread of the COMPSs application
- Task Dispatcher - Thread 1.1.2
- Shows information about the state and scheduling of the tasks to be executed.
- Access Processor - Thread 1.1.3
- All the events related to the tasks’ parameters management, such as dependencies or transfers are shown in this thread.
- Worker X Master - Thread 1.X.1
- This thread is the master of each worker and handles the computing resources and transfers. Is is repeated for each available resource. All data events of the worker, such as requests, transfers and receives are marked on this timeline (when using the appropriate configurations).
- Worker X Computing Unit Y - Thread 1.X.Y
- Shows the actual tasks execution information and is repeated as many times as computing threads has the worker X
Basic mode tracefile for a k-means algorithm visualized with compss_runtime.cfg
Advanced Mode
This mode is for more advanced COMPSs’ users and developers who want to customize further the information provided by the tracing or need rawer information like pthreads calls or Java garbage collection. With it, every single thread created during the execution is traced.
N.B.: The extra information provided by the advanced mode is only available on the workers when using NIO adaptor.
Advanced Mode Usage
In order to activate the advanced tracing add the following option to the execution:
--tracing=advanced
Examples given:
$ runcompss --tracing=advanced application_name application_args
Figure 23 was generated as follows:
$ runcompss \
--lang=java \
--tracing=advanced \
--classpath=/path/to/jar/kmeans.jar \
kmeans.KMeans
When advanced tracing is activated, the configuration file reported on the output is extrae_advanced.xml.
*** RUNNING JAVA APPLICATION KMEANS
...
...
...
Welcome to Extrae VERSION
Extrae: Parsing the configuration file (/opt/COMPSs/Runtime/scripts/user/../../configuration/xml/tracing/extrae_advanced.xml) begins
This is the default file used for advanced tracing. However, advanced users can modify it in order to customize the information provided by Extrae. The configuration file is read first by the master on the runcompss script. When using NIO adaptor for communication, the configuration file is also read when each worker is started (on persistent_worker.sh or persistent_worker_starter.sh depending on the execution environment).
If the default file is modified, the changes always affect the master, and also the workers when using NIO. Modifying the scripts which turn on the master and the workers is possible to achieve different instrumentations for master/workers. However, not all Extrae available XML configurations work with COMPSs, some of them can make the runtime or workers crash so modify them at your discretion and risk. More information about instrumentation XML configurations on Extrae User Guide at: https://www.bsc.es/computer-sciences/performance-tools/trace-generation/extrae/extrae-user-guide.
Instrumented Threads in Advanced Mode
Advanced mode instruments all the pthreads created during the application execution. It contains all the threads shown on basic traces plus extra ones used to call command-line commands, I/O streams managers and all actions which create a new process. Due to the temporal nature of many of this threads, they may contain little information or appear just at specific parts of the execution pipeline.
Information Available in Advanced Traces
The advanced mode tracefiles contain the same information as the basic ones:
- Events
- Marking diverse situations such as the runtime start, tasks’ execution or synchronization points.
- Communications
- Showing the transfers and requests of the parameters needed by COMPSs tasks.
- Hardware counters
- Of the execution obtained with Performance API (see PAPI: Hardware Counters)
Advanced Trace Example
Figure Figure 23 shows the total completed instructions for a sample program executed with the advanced tracing mode. Note that the thread - resource correspondence described on the basic trace example is no longer static and thus cannot be inferred. Nonetheless, they can be found thanks to the named events shown in other configurations such as compss_runtime.cfg.
Advanced mode tracefile for a testing program showing the total completed instructions
For further information about Extrae, please visit the following site:
Custom Installation and Configuration
Custom Extrae
COMPSs uses the environment variable EXTRAE_HOME to get the
reference to its installation directory (by default:
/opt/COMPSs/Dependencies/extrae ). However, if the variable is
already defined once the runtime is started, COMPSs will not override
it. User can take advantage of this fact in order to use custom extrae
installations. Just set the EXTRAE_HOME environment variable to
the directory where your custom package is, and make sure that it is
also set for the worker’s environment.
Be aware that using different Extrae packages can break the runtime
and executions so you may change it at your own risk.
Custom Configuration file
COMPSs offers the possibility to specify an extrae custom configuration file in order to harness all the tracing capabilities further tailoring which information about the execution is displayed. To do so just pass the file as an execution parameter as follows:
--extrae_config_file=/path/to/config/file.xml
The configuration file must be in a shared disk between all COMPSs workers because a file’s copy is not distributed among them, just the path to that file.
| [1] | For more information: https://www.bsc.es/computer-sciences/extrae |
| [2] | For more information: https://www.bsc.es/computer-sciences/performance-tools/paraver |
Visualization
Paraver is the BSC tool for trace visualization. Trace events are encoded in Paraver format (.prv) by the Extrae tool. Paraver is a powerful tool and allows users to show many views of the trace data using different configuration files. Users can manually load, edit or create configuration files to obtain different tracing views.
The following subsections explain how to load a trace file into Paraver, open the task events view using an already predefined configuration file, and how to adjust the view to display the data properly.
For further information about Paraver, please visit the following site:
http://www.bsc.es/computer-sciences/performance-tools/paraver
Trace Loading
The final trace file in Paraver format (.prv) is at the base log folder of the application execution inside the trace folder. The fastest way to open it is calling the Paraver binary directly using the tracefile name as the argument.
$ wxparaver /path/to/trace/trace.prv
Configurations
To see the different events, counters and communications that the
runtime generates, diverse configurations are available with the COMPSs
installation. To open one of them, go to the “Load Configuration” option
in the main window and select “File”. The configuration files are under
the following path for the default installation
/opt/COMPSs/Dependencies/ paraver/cfgs/. A detailed list of all
the available configurations can be found in
Paraver: configurations.
The following guide uses the compss_tasks.cfg as an example to illustrate the basic usage of Paraver. After accepting the load of the configuration file, another window appears showing the view. Figure 24 and Figure 25 show an example of this process.
Paraver menu
Trace file
View Adjustment
In a Paraver view, a red exclamation sign may appear in the bottom-left corner (see Figure 25 in the previous section). This means that some event values are not being shown (because they are out of the current view scope), so little adjustments must be made to view the trace correctly:
- Fit window: modifies the view scope to fit and display all the events
in the current window.
- Right click on the trace window
- Choose the option Fit Semantic Scale / Fit Both
Paraver view adjustment: Fit window
- View Event Flags: marks with a green flag all the emitted the events.
- Right click on the trace window
- Chose the option View / Event Flags
Paraver view adjustment: View Event Flags
- Show Info Panel: display the information panel. In the tab “Colors”
we can see the legend of the colors shown in the view.
- Right click on the trace window
- Check the Info Panel option
- Select the Colors tab in the panel
Paraver view adjustment: Show info panel
- Zoom: explore the tracefile more in-depth by zooming into the most
relevant sections.
- Select a region in the trace window to see that region in detail
- Repeat the previous step as many times as needed
- The undo-zoom option is in the right click panel
Paraver view adjustment: Zoom configuration
Paraver view adjustment: Zoom configuration
Interpretation
This section explains how to interpret a trace view once it has been adjusted as described in the previous section.
- The trace view has on its horizontal axis the execution time and on the vertical axis one line for the master at the top, and below it, one line for each of the workers.
- In a line, the light blue color is associated with an idle state, i.e. there is no event at that time.
- Whenever an event starts or ends a flag is shown.
- In the middle of an event, the line shows a different color. Colors are assigned depending on the event type.
- The info panel contains the legend of the assigned colors to each event type.
Trace interpretation
Analysis
This section gives some tips to analyze a COMPSs trace from two different points of view: graphically and numerically.
Graphical Analysis
The main concept is that computational events, the task events in this case, must be well distributed among all workers to have a good parallelism, and the duration of task events should be also balanced, this means, the duration of computational bursts.
Basic trace view of a Hmmpfam execution.
In the previous trace view, all the tasks of type “hmmpfam” in dark blue appear to be well distributed among the four workers, each worker executes four “hmmpfam” tasks.
However, some workers finish earlier than the others, worker 1.2.3 finish the first and worker 1.2.1 the last. So there is an imbalance in the duration of “hmmpfam” tasks. The programmer should analyze then whether all the tasks process the same amount of input data and do the same thing in order to find out the reason for such imbalance.
Another thing to highlight is that tasks of type “scoreRatingSameDB” are not equally distributed among all the workers. Some workers execute more tasks of this type than the others. To understand better what happens here, one needs to take a look to the execution graph and also zoom in the last part of the trace.
Data dependencies graph of a Hmmpfam execution.
Zoomed in view of a Hmmpfam execution.
There is only one task of type “scoreRatingSameSeq”. This task appears in red in the trace (and in light-green in the graph). With the help of the graph we see that the “scoreRatingSameSeq” task has dependences on tasks of type “scoreRatingSameDB”, in white (or yellow).
When the last task of type “hmmpfam” (in dark blue) ends, the previous dependencies are solved, and if we look at the graph, this means going across a path of three dependencies of type “scoreRatingSameDB” (in yellow). Moreover, because these are sequential dependencies (one depends on the previous) no more than a worker can be used at the same time to execute the tasks. This is the reason of why the last three task of type “scoreRatingSameDB” (in white) are executed in worker 1.2.1 sequentially.
Numerical Analysis
Here we show another trace from a different parallel execution of the Hmmer program.
Original sample trace interval corresponding to the obtained Histogram.
Paraver offers the possibility of having different histograms of the trace events. Click the “New Histogram” button in the main window and accept the default options in the “New Histogram” window that will appear.
Paraver Menu - New Histogram
After that, the following table is shown. In this case for each worker, the time spent executing each type of task is shown. Task names appear in the same color than in the trace view. The color of a cell in a row corresponding to a worker ranges from light-green for lower values to dark-blue for higher ones. This conforms a color based histogram.
Hmmpfam histogram corresponding to previous trace
The previous table also gives, at the end of each column, some extra statistical information for each type of tasks (as the total, average, maximum or minimum values, etc.).
In the window properties of the main window, it is possible to change the semantic of the statistics to see other factors rather than the time, for example, the number of bursts.
Paraver histogram options menu
In the same way as before, the following table shows for each worker the number of bursts for each type of task, this is, the number or tasks executed of each type. Notice the gradient scale from light-green to dark-blue changes with the new values.
Hmmpfam histogram with the number of bursts
PAPI: Hardware Counters
The applications instrumentation supports hardware counters through the performance API (PAPI). In order to use it, PAPI needs to be present on the machine before installing COMPSs.
During COMPSs installation it is possible to check if PAPI has been detected in the Extrae config report:
Package configuration for Extrae VERSION based on extrae/trunk rev. XXXX:
-----------------------
Installation prefix: /opt/COMPSs/Dependencies/extrae
Cross compilation: no
...
...
...
Performance counters: yes
Performance API: PAPI
PAPI home: /usr
Sampling support: yes
Caution
PAPI detection is only performed in the machine where COMPSs is installed. User is responsible of providing a valid PAPI installation to the worker machines to be used (if they are different from the master), otherwise workers will crash because of the missing libpapi.so.
PAPI installation and requirements depend on the OS. On Ubuntu 14.04 it is available under textitpapi-tools package; on OpenSuse textitpapi and textitpapi-dev. For more information check https://icl.cs.utk.edu/projects/papi/wiki/Installing_PAPI.
Extrae only supports 8 active hardware counters at the same time. Both basic and advanced mode have the same default counters list:
- PAPI_TOT_INS
- Instructions completed
- PAPI_TOT_CYC
- Total cycles
- PAPI_LD_INS
- Load instructions
- PAPI_SR_INS
- Store instructions
- PAPI_BR_UCN
- Unconditional branch instructions
- PAPI_BR_CN
- Conditional branch instructions
- PAPI_VEC_SP
- Single precision vector/SIMD instructions
- RESOURCE_STALLS
- Cycles Allocation is stalled due to Resource Related reason
The XML config file contains a secondary set of counters. In order to activate it just change the starting-set-distribution from 2 to 1 under the cpu tag. The second set provides the following information:
- PAPI_TOT_INS
- Instructions completed
- PAPI_TOT_CYC
- Total cycles
- PAPI_L1_DCM
- Level 1 data cache misses
- PAPI_L2_DCM
- Level 2 data cache misses
- PAPI_L3_TCM
- Level 3 cache misses
- PAPI_FP_INS
- Floating point instructions
To further customize the tracked counters, modify the XML to suit your needs. To find the available PAPI counters on a given computer issue the command papi_avail -a. For more information about Extrae’s XML configuration refer to https://www.bsc.es/computer-sciences/performance-tools/trace-generation/extrae/extrae-user-guide.
Paraver: configurations
Table 16, Table 17
and Table 18 provide information about the different
pre-build configurations that are distributed with COMPSs and that can
be found under the /opt/COMPSs/Dependencies/ paraver/cfgs/
folder. The cfgs folder contains all the basic views, the python
folder contains the configurations for Python events, and finally the
comm folder contains the configurations related to communications.
| Configuration File Name | Description |
|---|---|
| 2dp_runtime_state.cfg | 2D plot of runtime state |
| 2dp_tasks.cfg | 2D plot of tasks duration |
| 3dh_duration_runtime.cfg | 3D Histogram of runtime execution |
| 3dh_duration_tasks.cfg | 3D Histogram of tasks duration |
| Interval_between_runtime.cfg | Interval between runtime events |
| compss_runtime.cfg | Shows COMPSs Runtime events (master and workers) |
| compss_runtime_master.cfg | Shows COMPSs Runtime master events |
| compss_storage.cfg | Shows COMPSs persistent storage events |
| compss_tasks.cfg | Shows tasks execution |
| compss_tasks_and_runtime.cfg | Shows COMPSs Runtime events (master and workers) and tasks execution |
| compss_tasks_numbers.cfg | Shows tasks execution by task id |
| compss_waiting_tasks.cfg | Shows waiting tasks |
| nb_executing_tasks.cfg | Number of executing tasks |
| nb_requested_cpus.cfg | Number of requested CPUs |
| nb_requested_disk_bw.cfg | Number of requested disk bandwidth |
| nb_requested_gpus.cfg | Number of requested GPUs |
| nb_executing_mem.cfg | Number of executing memory |
| task_duration.cfg | Shows tasks duration |
| thread_cpu.cfg | Shows the initial executing CPU |
| time_betw_tasks.cfg | Shows the time between tasks |
| user_events.cfg | Shows the user events (type 9000000) |
| Configuration File Name | Description |
|---|---|
| 3dh_events_inside_task.cfg | 3D Histogram of python events |
| events_inside_tasks.cfg | Events showing python information such as user function execution time, modules imports, or serializations |
| Time_between_events_inside_tasks.cfg | Shows the time between events inside tasks |
| nb_user_code_executing.cfg | Number of user code executing |
| Configuration File Name | Description |
|---|---|
| sr_bandwith.cfg | Send/Receive bandwith view for each node |
| send_bandwith.cfg | Send bandwith view for each node |
| receive_bandwith.cfg | Receive bandwith view for each node |
| process_bandwith.cfg | Send/Receive bandwith table for each node |
| compss_tasks_scheduling_transfers.cfg | Task’s transfers requests for scheduling (gradient of tasks ID) |
| compss_tasksID_transfers.cfg | Task’s transfers request for each task (task with its IDs are also shown) |
| compss_data_transfers.cfg | Shows data transfers for each task’s parameter |
| communication_matrix.cfg | Table view of communications between each node |
User Events in Python
Users can emit custom events inside their python tasks. Thanks to the fact that python is not a compiled language, users can emit events inside their own tasks using the available EXTRAE instrumentation object because it is already loaded and available in the PYTHONPATH when running with tracing enabled.
To emit an event first import pyextrae:
import pyextrae.sequential as pyextraeto emit events from the main code.import pyextrae.multiprocessing as pyextraeto emit events within tasks code.
And then just use the call pyextrae.event(type, id) (or
pyextrae.eventandcounters (type, id) if you also want to emit PAPI
hardware counters).
Tip
It must be used a type number higher than 8000050 in order to avoid type
conflicts.
We suggest to use 9000000 since we provide the user_events.cfg
configuration file to visualize the user events of this type in PARAVER.
Events in main code
The following code snippet shows how to emit an event from the main code (or
any other code which is not within a task). In this case it is necessary to
import pyextrae.sequential.
from pycompss.api.api import compss_wait_on
from pycompss.api.task import task
import pyextrae.sequential as pyextrae
@task(returns=1)
def increment(value):
return value + 1
def main():
value = 1
pyextrae.eventandcounters(9000000, 2)
result = increment(value)
result = compss_wait_on(result)
pyextrae.eventandcounters(9000000, 0)
print("result: " + str(result))
if __name__ == "__main__":
main()
Events in task code
The following code snippet shows how to emit an event from the task code.
In this case it is necessary to import pyextrae.multiprocessing.
from pycompss.api.task import task
@task()
def compute():
import pyextrae.multiprocessing as pyextrae
pyextrae.eventandcounters(9000000, 2)
...
# Code to wrap within event 2
...
pyextrae.eventandcounters(9000000, 0)
Caution
Please, note that the import pyextrae.multiprocessing as pyextrae is
performed within the task. If the user needs to add more events to tasks
within the same module (excluding the applicatin main module) and wants to
put this import in the top of the module making pyextrae available for
all of them, it is necessary to enable the tracing hook on the tasks that
emit events:
from pycompss.api.task import task
import pyextrae.multiprocessing as pyextrae
@task(tracing_hook=True)
def compute():
pyextrae.eventandcounters(9000000, 2)
...
# Code to wrap within event 2
...
pyextrae.eventandcounters(9000000, 0)
The tracing_hook is disabled by default in order to reduce the overhead
introduced by tracing avoiding to intercept all function calls within the
task code.
Result trace
The events will appear automatically on the generated trace.
In order to visualize them, just load the user_events.cfg configuration file
in PARAVER.
If a different type value is choosen, take the same user_events.cfg and go
to Window Properties -> Filter -> Events -> Event Type and change
the value labeled Types for your custom events type.
Tip
If you want to name the events, you will need to manually add them to the
.pcf file with the corresponding name for each value.
Practical example
Consider the following application where we define an event in the main code
(1) and another within the task (2).
The increment task is invoked 8 times (with a mimic computation time of
the value received as parameter.)
from pycompss.api.api import compss_wait_on
from pycompss.api.task import task
import time
@task(returns=1)
def increment(value):
import pyextrae.multiprocessing as pyextrae
pyextrae.eventandcounters(9000000, 2)
time.sleep(value) # mimic some computation
pyextrae.eventandcounters(9000000, 0)
return value + 1
def main():
import pyextrae.sequential as pyextrae
elements = [1, 2, 3, 4, 5, 6, 7, 8]
results = []
pyextrae.eventandcounters(9000000, 1)
for element in elements:
results.append(increment(element))
results = compss_wait_on(results)
pyextrae.eventandcounters(9000000, 0)
print("results: " + str(results))
if __name__ == "__main__":
main()
After launching with tracing enabled (-t flag), the trace has been
generated into the logs folder:
$HOME/.COMPSs/events.py_01/traceif usingruncompss.$HOME/.COMPSs/<JOB_ID>/traceif usingenqueue_compss.
Now it is time to modify the .pcf file including the folling text at
the end of the file with your favourite text editor:
EVENT_TYPE
0 9000000 User events
VALUES
0 End
1 Main code event
2 Task event
Caution
Keep value 0 with the End message.
Add all values defined in the application with a descriptive short name to ease the event identification in PARAVER.
Open PARAVER, load the tracefile (.prv) and open the user_events.cfg
configuration file. The result (see Figure 40) shows that
there are 8 “Task event” (in white), and 1 “Main code event” (in blue) as
we expected.
Their length can be seen with the event flags (green flags), and measured
by double clicking on the event of interest.
User events trace file
Paraver uses by default the .pcf with the same name as the tracefile so
if you add them to one, you can reuse it just by changing its name to
the tracefile.
Persistent Storage
COMPSs is able to interact with Persistent Storage frameworks. To this end, it is necessary to take some considerations in the application code and on its execution. This section is intended to walk you through the COMPSs’ storage interface and its integration with some Persistent Storage frameworks.
First steps
COMPSs relies on a Storage API to enable the interation with persistent storage frameworks (Figure 41), which is composed by two main modules: Storage Object Interface (SOI) and Storage Runtime Interface (SRI)
COMPSs with persistent storage architecture
Any COMPSs application aimed at using a persistent storage framework has to include calls to:
- The SOI in order to define the data model (see Defining the data model), and relies on COMPSs, which interacts with the persistent storage framework through the SRI.
- The SRI in order to interact directly with the storage backend (e.g. retrieve data, etc.) (see Interacting with the persistent storage).
In addition, it must be taken into account that the execution of an application
using a persistent storage framework requires some specific flags in
runcompss and enqueue_compss
(see Running with persistent storage).
Currently, there exists storage interfaces for dataClay, Hecuba and Redis. They are thoroughly described from the developer and user point of view in Sections:
The interface is open to any other storage framework by implementing the required functionalities described in Implement your own Storage interface for COMPSs.
Defining the data model
The data model consists of a set of related classes programmed in one of the supported languages aimed are representing the objects used in the application (e.g. in a wordcount application, the data model would be text).
In order to define that the application objects are going to be stored in the underlying persistent storage backend, the data model must be enriched with the Storage Object Interface (SOI).
The SOI provides a set of functionalities that all objects stored in the persistent storage backend will need. Consequently, the user must inherit the SOI on its data model classes, and give some insights of the class attributes.
The following subsections detail how to enrich the data model in Java and Python applications.
Java
To define that a class objects are going to be stored in the persistent storage
backend, the class must extend the StorageObject class (as well as
implement the Serializable interface). This class is provided by the
persistent storage backend.
import storage.StorageObject;
import java.io.Serializable;
class MyClass extends StorageObject implements Serializable {
private double[] vector;
/**
* Write here your class-specific
* constructors, attributes and methods.
*/
}
The StorageObject object enriches the class with some methods that allow the
user to interact with the persistent storage backend. These methods can be
found in Table 19.
| Name | Returns | Comments |
|---|---|---|
| makePersistent(String id) | Nothing | Inserts the object in the database with the id.
If id is null, a random UUID will be computed instead.
|
| deletePersistent() | Nothing | Removes the object from the storage.
It does nothing if it was not already there.
|
| getID() | String | Returns the current object identifier if the object is not persistent (null instead).
|
These functions can be used from the application in order to persist an object
(pushing the object into the persistent storage) with make_persistent,
remove it from the persistent storage with delete_persistent or
getting the object identifier with getID for the later interaction with
the storage backend.
import MyPackage.MyClass;
class Test{
// ...
public static void main(String args[]){
// ...
MyClass my_obj = new MyClass();
my_obj.matrix = new double[10];
my_obj.makePersistent(); // make persistent without parameter
String obj_id = my_obj.getID(); // get the idenfier provided by the storage framework
// ...
my_obj.deletePersistent();
// ...
MyClass my_obj2 = new MyClass();
my_obj2.matrix = new double[20];
my_obj2.makePersistent("obj2"); // make persistent providing identifier
// ...
my_obj2.delete_persistent();
// ...
}
}
Python
To define that a class objects are going to be stored in the persistent storage
backend, the class must inherit the StorageObject class. This class
is provided by the persistent storage backend.
from storage.api import StorageObject
class MyClass(StorageObject):
...
In addition, the user has to give details about the class attributes using
the class documentation.
For example, if the user wants to define a class containing a numpy ndarray as
attribute, the user has to specify this attribute starting with @ClassField
followed by the attribute name and type:
from storage.api import StorageObject
class MyClass(StorageObject):
"""
@ClassField matrix numpy.ndarray
"""
pass
Important
Methods inside the class are not supported by all storage backends. dataClay is currently the only backend that provides support for them (see Enabling COMPSs applications with dataClay).
Then, the user can use the instantiated object normally:
from MyFile import MyClass
import numpy as np
my_obj = MyClass()
my_obj.matrix = np.random.rand(10, 2)
...
The following code snippet gives some examples of several types of attributes:
from storage.api import StorageObject
class MyClass(StorageObject):
"""
# Elemmental types
@ClassField field1 int
@ClassField field2 str
@ClassField field3 np.ndarray
# Structured types
@ClassField field4 list <int>
@ClassField field5 set <list<float>>
# Another class instance as attribute
@ClassField field6 AnotherClassName
# Complex dictionaries:
@ClassField field7 dict <<int,str>, dict<<int>, list<str>>>
@ClassField field8 dict <<int>, AnotherClassName>
# Dictionary with structured value:
@ClassField field9 dict <<k1: int, k2: int>, tuple<v1: int, v2: float, v3: text>>
# Plain definition of the same dictionary:
@ClassField field10 dict <<int,int>, str>
"""
pass
Finally, the StorageObject class includes some functions in the class that
will be available from the instantiated objects
(Table 20).
| Name | Returns | Comments |
|---|---|---|
| make_persistent(String id) | Nothing | Inserts the object in the database with the id.
If id is null, a random UUID will be computed instead.
|
| delete_persistent() | Nothing | Removes the object from the storage.
It does nothing if it was not already there.
|
| getID() | String | Returns the current object identifier if the object is not persistent (
None instead). |
These functions can be used from the application in order to persist an object
(pushing the object into the persistent storage) with make_persistent,
remove it from the persistent storage with delete_persistent or
getting the object identifier with getID for the later interaction with
the storage backend.
import numpy as np
my_obj = MyClass()
my_obj.matrix = np.random.rand(10, 2)
my_obj.make_persistent() # make persistent without parameter
obj_id = my_obj.getID() # get the idenfier provided by the storage framework
...
my_obj.delete_persistent()
...
my_obj2 = MyClass()
my_obj2.matrix = np.random.rand(10, 3)
my_obj2.make_persistent('obj2') # make persistent providing identifier
...
my_obj2.delete_persistent()
...
C/C++
Unsupported
Persistent storage is not supported with C/C++ COMPSs applications.
Interacting with the persistent storage
The Storage Runtime Interface (SRI) provides some functions to interact with the storage backend. All of them are aimed at enabling the COMPSs runtime to deal with persistent data across the infrastructure.
However, the function to retrieve an object from the storage backend from its
identifier can be useful for the user.
Consequently, users can import the SRI and use the getByID function
when needed necessary. This function requires a String parameter with
the object identifier, and returns the object associated with that identifier
(null or None otherwise).
The following subsections detail how to call the getByID function in Java
and Python applications.
Java
Import the getByID function from the storage api and use it:
import storage.StorageItf;
import MyPackage.MyClass;
class Test{
// ...
public static void main(String args[]){
// ...
obj = StorageItf.getByID("my_obj");
// ...
}
}
Python
Import the getByID function from the storage api and use it:
from storage.api import getByID
..
obj = getByID('my_obj')
...
C/C++
Unsupported
Persistent storage is not supported with C/C++ COMPSs applications.
Running with persistent storage
Local
In order to run a COMPSs application locally, the runcompss command is used.
The runcompss command includes some flags to execute the application
considering a running persistent storage framework. These flags are:
--classpath, --pythonpath and --storage_conf.
Consequently, the runcompss requirements to run an application with a
running persistent storage backend are:
| --classpath | Add the --classpath=${path_to_storage_api.jar} flag to the
runcompss command. |
| --pythonpath | If you are running a python application, also add the
--pythonpath=${path_to_the_storage_api}/python
flag to the runcompss command. |
| --storage_conf | Add the flag --storage_conf=${path_to_your_storage_conf_dot_cfg_file}
to the runcompss command. The storage configuration file (usually
storage_conf.cfg) contains the configuration parameters needed by the
storage framework for the execution (it depends on the storage framework). |
As usual, the project.xml and resources.xml files must be correctly set.
Supercomputer
In order to run a COMPSs application in a Supercomputer or cluster, the
enqueue_compss command is used.
The enqueue_compss command includes some flags to execute the application
considering a running persistent storage framework. These flags are:
--classpath, --pythonpath, --storage-home and --storage-props.
Consequently, the enqueue_compss requirements to run an application with a
running persistent storage backend are:
| --classpath | --classpath=${path_to_storage_interface.jar} As with the runcompss
command, the JAR with the storage API must be specified. It is usally
available in a environment variable (check the persistent storage framework). |
| --pythonpath | If you are running a Python application, also add the
--pythonpath=${path_to_the_storage_api}/python flag.
It is usally available in a environment variable (check the persistent
storage framework). |
| --storage-home | --storage-home=${path_to_the_storage_api} This must point to
the root of the storage folder. This folder must contain a scripts
folder where the scripts to start and stop the persistent framework are.
It is usally available in a environment variable (check the persistent
storage framework). |
| --storage-props | |
--storage-props=${path_to_the_storage_props_file} This must point
to the storage properties configuration file (usually storage_props.cfg)
It contains the configuration parameters needed by the storage framework
for the execution (it depends on the storage framework). | |
COMPSs + dataClay
Warning
Under construction
COMPSs + dataClay Dependencies
dataClay
Other dependencies
Enabling COMPSs applications with dataClay
Java
Python
C/C++
Unsupported
C/C++ COMPSs applications are not supported with dataClay.
Executing a COMPSs application with dataClay
Launching using an existing dataClay deployment
Launching on queue system based environments
COMPSs + Hecuba
Warning
Under construction
COMPSs + Hecuba Dependencies
Hecuba
Other dependencies
Enabling COMPSs applications with Hecuba
Java
Unsupported
Java COMPSs applications are not supported with Hecuba.
Python
C/C++
Unsupported
C/C++ COMPSs applications are not supported with Hecuba.
Executing a COMPSs application with Hecuba
Launching using an existing Hecuba deployment
Launching on queue system based environments
COMPSs + Redis
COMPSs provides a built-in interface to use Redis as persistent storage from COMPSs’ applications.
Note
We assume that COMPSs is already installed. See Installation and Administration
The next subsections focus on how to install the Redis utilities and the storage API for COMPSs.
Hint
It is advisable to read the Redis Cluster tutorial for beginners [1] in order to understand all the terminology that is used.
COMPSs + Redis Dependencies
The required dependencies are:
Redis Server
redis-server is the core Redis program. It allows to create
standalone Redis instances that may form part of a cluster in the
future. redis-server can be obtained by following these steps:
- Go to
https://redis.io/downloadand download the last stable version. This should download aredis-${version}.tar.gzfile to your computer, where${version}is the current latest version. - Unpack the compressed file to some directory, open a terminal on it
and then type
sudo make installif you want to install Redis for all users. If you want to have it installed only for yourself you can simply typemake redis-server. This will leave theredis-serverexecutable file inside the directorysrc, allowing you to move it to a more convenient place. By convenient place we mean a folder that is in yourPATHenvironment variable. It is advisable to not delete the uncompressed folder yet. - If you want to be sure that Redis will work well on your machine then
you can type
make test. This will run a very exhaustive test suite on Redis features.
Important
Do not delete the uncompressed folder yet.
Redis Cluster script
Redis needs an additional script to form a cluster from various Redis
instances. This script is called redis-trib.rb and can be found in
the same tar.gz file that contains the sources to compile
redis-server in src/redis-trib.rb. Two things must be done to
make this script work:
Move it to a convenient folder. By convenient folder we mean a folder that is in your
PATHenvironment variable.Make sure that you have Ruby and
geminstalled. Typegem install redis.In order to use COMPSs + Redis with Python you must also install the
redisandredis-py-clusterPyPI packages.Hint
It is also advisable to have the PyPI package
hiredis, which is a library that makes the interactions with the storage to go faster.
COMPSs-Redis Bundle
COMPSs-Redis Bundle is a software package that contains the
following:
- A java JAR file named
compss-redisPSCO.jar. This JAR contains the implementation of a Storage Object that interacts with a given Redis backend. We will discuss the details later. - A folder named
scripts. This folder contains a bunch of scripts that allows a COMPSs-Redis app to create a custom, in-place cluster for the application. - A folder named
pythonthat contains the Python equivalent tocompss-redisPSCO.jar
This package can be obtained from the COMPSs source as follows:
- Go to
trunk/utils/storage/redisPSCO - Type
./make_bundle. This will leave a folder namedCOMPSs-Redis-bundlewith all the bundle contents.
Enabling COMPSs applications with Redis
Java
This section describes how to develop Java applications with the
Redis storage. The application project should have the
dependency induced by compss-redisPSCO.jar satisfied.
That is, it should be included in the application’s pom.xml if you are
using Maven, or it should be listed in the
dependencies section of the used development tool.
The application is almost identical to a regular COMPSs application except for the presence of Storage Objects. A Storage Object is an object that it is capable to interact with the storage backend. If a custom object extends the Redis Storage Object and implements the Serializable interface then it will be ready to be stored and retrieved from a Redis database. An example signature could be the following:
import storage.StorageObject;
import java.io.Serializable;
/**
* A PSCO that contains a KD point
*/
class RedisPoint
extends StorageObject implements Serializable {
// Coordinates of our point
private double[] coordinates;
/**
* Write here your class-specific
* constructors, attributes and methods.
*/
double getManhattanDistance(RedisPoint other) {
...
}
}
The StorageObject object has some inherited methods that allow the
user to write custom objects that interact with the Redis backend. These
methods can be found in Table 21.
| Name | Returns | Comments |
|---|---|---|
| makePersistent(String id) | Nothing | Inserts the object in the database with the id.
If id is null, a random UUID will be computed instead.
|
| deletePersistent() | Nothing | Removes the object from the storage.
It does nothing if it was not already there.
|
| getID() | String | Returns the current object identifier if the object is not persistent (null instead).
|
Caution
Redis Storage Objects that are used as INOUTs must be manually updated.
This is due to the fact that COMPSs does not know the exact effects of
the interaction between the object and the storage, so the runtime cannot
know if it is necessary to call makePersistent after having used an
INOUT or not (other storage approaches do live modifications to its storage
objects). The followingexample illustrates this situation:
/**
* A is passed as INOUT
*/
void accumulativePointSum(RedisPoint a, RedisPoint b) {
// This method computes the coordinate-wise sum between a and b
// and leaves the result in a
for(int i=0; i<a.getCoordinates().length; ++i) {
a.setComponent(i, a.getComponent(i) + b.getComponent(i));
}
// Delete the object from the storage and
// re-insert the object with the same old identifier
String objectIdentifier = a.getID();
// Redis contains the old version of the object
a.deletePersistent();
// Now we will insert the updated one
a.makePersistent(objectIdentifier);
}
If the last three statements were not present, the changes would never
be reflected on the RedisPoint a object.
Python
Redis is also available for Python. As happens with Java, we
first need to define a custom Storage Object. Let’s suppose that we want
to write an application that multiplies two matrices 
, and
by blocks. We can define a Block object that lets us store
and write matrix blocks in our Redis backend:
from storage.storage_object import StorageObject
import storage.api
class Block(StorageObject):
def __init__(self, block):
super(Block, self).__init__()
self.block = block
def get_block(self):
return self.block
def set_block(self, new_block):
self.block = new_block
Let’s suppose that we are multiplying our matrices in the usual blocked way:
for i in range(MSIZE):
for j in range(MSIZE):
for k in range(MSIZE):
multiply(A[i][k], B[k][j], C[i][j])
Where and are Block objects and 
is a
regular Python object (e.g: a Numpy matrix), then we can define
multiply as a task as follows:
@task(c = INOUT)
def multiply(a_object, b_object, c, MKLProc):
c += a_object.block * b_object.block
Let’s also suppose that we are interested to store the final result in our storage. A possible solution is the following:
for i in range(MSIZE):
for j in range(MSIZE):
persist_result(C[i][j])
Where persist_result can be defined as a task as follows:
@task()
def persist_result(obj):
to_persist = Block(obj)
to_persist.make_persistent()
This way is preferred for two main reasons:
- we avoid to bring the resulting matrix to the master node,
- and we can exploit the data locality by executing the task in the node
where last version of
objis located.
C/C++
Unsupported
C/C++ COMPSs applications are not supported with Redis.
Executing a COMPSs application with Redis
Launching using an existing Redis Cluster
If there is already a running Redis Cluster on the node/s where the COMPSs application will run then only the following steps must be followed:
- Create a
storage_conf.cfgfile that lists, one per line, the nodes where the storage is present. Only hostnames or IPs are needed, ports are not necessary here. - Add the flag
--classpath=${path_to_COMPSs-redisPSCO.jar}to theruncompsscommand that launches the application. - Add the flag
--storage_conf=${path_to_your_storage_conf_dot_cfg_file}to theruncompsscommand that launches the application. - If you are running a python app, also add the
--pythonpath=${app_path}:${path_to_the_bundle_folder}/pythonflag to theruncompsscommand that launches the application.
As usual, the project.xml and resources.xml files must be
correctly set. It must be noted that there can be Redis nodes that are
not COMPSs nodes (although this is a highly unrecommended practice).
As a requirement, there must be at least one Redis instance on each
COMPSs node listening to the official Redis port 6379 [2]. This is
required because nodes without running Redis instances would cause a
great amount of transfers (they will always need data that must be
transferred from another node). Also, any locality policy will likely
cause this node to have a very low workload, rendering it almost
useless.
Launching on queue system based environments
COMPSs-Redis-Bundle also includes a collection of scripts that allow
the user to create an in-place Redis cluster with his/her COMPSs
application. These scripts will create a cluster using only the COMPSs
nodes provided by the queue system (e.g. SLURM, PBS, etc.).
Some parameters can be tuned by the user via a
storage_props.cfg file. This file must have the following form:
REDIS_HOME=some_path
REDIS_NODE_TIMEOUT=some_nonnegative_integer_value
REDIS_REPLICAS=some_nonnegative_integer_value
There are some observations regarding to this configuration file:
- REDIS_HOME
- Must be equal to a path to some location that is not shared between nodes. This is the location where the Redis sandboxes for the instances will be created.
- REDIS_NODE_TIMEOUT
- Must be a nonnegative integer number that represents the amount of milliseconds that must pass before Redis declares the cluster broken in the case that some instance is not available.
- REDIS_REPLICAS
- Must be equal to a nonnegative integer. This value will represent the amount of replicas that a given shard will have. If possible, Redis will ensure that all replicas of a given shard will be on different nodes.
In order to run a COMPSs + Redis application on a queue system the user
must add the following flags to the enqueue_compss command:
--storage-home=${path_to_the_bundle_folder}This must point to the root of the COMPSs-Redis bundle.--storage-props=${path_to_the_storage_props_file}This must point to thestorage_props.cfgmentioned above.--classpath=${path_to_COMPSs-redisPSCO.jar}As in the previous section, the JAR with the storage API must be specified.- If you are running a Python application, also add the
--pythonpath=${app_path}:${path_to_the_bundle_folder}flag
Caution
As a requirement, the supercomputer MUST NOT kill daemonized processes running on the provided computing nodes during the execution.
| [1] | https://redis.io/topics/cluster-tutorial |
| [2] | https://en.wikipedia.org/wiki/List_of_TCP_and_UDP_port_numbers |
Implement your own Storage interface for COMPSs
In order to implement an interface for a Storage framework, it is necessary to implement the Java SRI (mandatory), and depending on the desired language, implement the Python SRI and the specific SOI inheriting from the generic SOI provided by COMPSs.
Generic Storage Object Interface
Table 22 shows the functions that must exist in the storage object interface, that enables the object that inherits it to interact with the storage framework.
| Name | Returns | Comments |
|---|---|---|
| Constructor | Nothing | Instantiates the object.
|
| get_by_alias(String id) | Object | Retrieve the object with alias “name”.
|
| makePersistent(String id) | Nothing | Inserts the object in the storage framework with the id.
If id is null, a random UUID will be computed instead.
|
| deletePersistent() | Nothing | Removes the object from the storage.
It does nothing if it was not already there.
|
| getID() | String | Returns the current object identifier if the object is not persistent (null instead).
|
For example, the makePersistent function is intended to store the object content into the persistent storage, deletePersistent to remove it, and getID to provide the object identifier.
Important
An object will be considered persisted if the getID function retrieves
something different from None.
This interface must be implemented in the target language desired (e.g. Java or Python).
Generic Storage Runtime Interfaces
Table 23 shows the functions that must exist in the storage runtime interface, that enables the COMPSs runtime to interact with the storage framework.
| Name | Returns | Comments | Signature |
|---|---|---|---|
init(String storage_conf)
|
Nothing | Do any initialization action before
starting to execute the application.
Receives the storage configuration
file path defined in the
runcompssor
enqueue_composs command. |
public static void init(String storageConf) throws StorageException {} |
finish()
|
Nothing | Do any finalization action after
executing the application.
|
public static void finish() throws StorageException |
getLocations(String id)
|
List<String> | Retrieve the locations where a particular
object is from its identifier.
|
public static List<String> getLocations(String id) throws StorageException |
getByID(String id)
|
Object | Retrieve an object from its identifier.
|
public static Object getByID(String id) throws StorageException |
newReplica(String id,
String hostName)
|
String | Create a new replica of an object in the
storage framework.
|
public static void newReplica(String id, String hostName) throws StorageException |
newVersion(String id,
String hostname)
|
String | Create a new version of an object in the
storage framework.
|
public static String newVersion(String id, String hostName) throws StorageException |
consolidateVersion(String id)
|
Nothing | Consolidate a version of an object in the
storage framework.
|
public static void consolidateVersion(String idFinal) throws StorageException |
executeTask(String id, …)
|
String | Execute the task into the datastore.
|
public static String executeTask(String id, String descriptor, Object[] values, String hostName, CallbackHandler callback) throws StorageException |
getResult(CallbackEvent event())
|
Object | Retrieve the result of the execution into
the storage framework.
|
public static Object getResult(CallbackEvent event) throws StorageException |
This functions enable the COMPSs runtime to keep the data consistency through the distributed execution.
In addition, Table 24 shows the functions that must exist in the storage runtime interface, that enables the COMPSs Python binding to interact with the storage framework. It is only necessary if the target language is Python.
| Name | Returns | Comments | Signature |
|---|---|---|---|
| init(String storage_conf) | Nothing | Do any initialization action before starting to execute the application.
Receives the storage configuration file path defined in the
runcompss orenqueue_composs command. |
def initWorker(config_file_path=None, **kwargs)
# Does not return
|
| finish() | Nothing | Do any finalization action after executing the application.
|
def finishWorker(**kwargs)
# Does not return
|
| getByID(String id) | Object | Retrieve an object from its identifier.
|
def getByID(id)
# Returns the object with Id ‘id’
|
| TaskContext | Context | Define a task context (task enter/exit actions).
|
class TaskContext(object):
def __init__(self, logger, values, config_file_path=None, **kwargs):
self.logger = logger
self.values = values
self.config_file_path = config_file_path
def __enter__(self):
# Do something for task prolog
def __exit__(self, type, value, traceback):
# Do something for task epilog
|
Storage Interface usage
Using runcompss
The first consideration is to deploy the storage framework, and then follow the next steps:
- Create a
storage_conf.cfgfile with the configuation required by theinitSRIs functions. - Add the flag
--classpath=${path_to_SRI.jar}to theruncompsscommand. - Add the flag
--storage_conf="path to storage_conf.cfg fileto theruncompsscommand. - If you are running a Python app, also add the
--pythonpath=${app_path}:${path_to_the_bundle_folder}/pythonflag to theruncompsscommand.
As usual, the project.xml and resources.xml files must be
correctly set. It must be noted that there can be nodes that are
not COMPSs nodes (although this is a highly unrecommended practice since
they will always need data that must be transferred from another node).
Also, any locality policy will likely cause this node to have a very low workload.
Using enqueue_compss
In order to run a COMPSs + your storage on a queue system the user
must add the following flags to the enqueue_compss command:
--storage-home=${path_to_the_user_storage_folder}This must point to the root of the user storage folder, where the scripts for starting (storage_init.sh) and stopping (storage_stop.sh) the storage framework must exist.storage_init.shis called before the application execution and itis intended to deploy the storage framework within the nodes provided by the queuing system. The parameters that receives are (in order):
- JOBID
The job identifier provided by the queuing system.
- MASTER_NODE
The name of the master node considered by COMPSs.
- STORAGE_MASTER_NODE
The name of the node to be considere the master for the Storage framework.
- WORKER_NODES
The set of nodes provided by the queuing system that will be considered as worker nodes by COMPSs.
- NETWORK
Network interface (e.g. ib0)
- STORAGE_PROPS
Storage properties file path (defined as
enqueue_compssflag).- VARIABLES_TO_BE_SOURCED
If environment variables for the Storage framework need to be defined COMPSs provides an empty file to be filled by the
storage_init.shscript, that will be sourced afterwards. This file is cleaned inmediately after sourcing it.
storage_stop.shis called after the application execution and itis intended to stop the storage framework within the nodes provided by the queuing system. The parameters that receives are (in order):
- JOBID
The job identifier provided by the queuing system.
- MASTER_NODE
The name of the master node considered by COMPSs.
- STORAGE_MASTER_NODE
The name of the node to be considere the master for the Storage framework.
- WORKER_NODES
The set of nodes provided by the queuing system that will be considered as worker nodes by COMPSs.
- NETWORK
Network interface (e.g. ib0)
- STORAGE_PROPS
Storage properties file path (defined as
enqueue_compssflag).
--storage-props=${path_to_the_storage_props_file}This must point to thestorage_props.cfgspecific for the storage framework that will be used by the start and stop scripts provided in the--storage-homepath.--classpath=${path_to_SRI.jar}As in the previous section, the JAR with the Java SRI must be specified.If you are running a Python application, also add the
--pythonpath=${app_path}:${path_to_the_user_storage_folder}flag, where the SOI for Python must exist.
Sample Applications
This section is intended to walk you through some COMPSs applications.
Java Sample applications
The first two examples in this section are simple applications developed in COMPSs to easily illustrate how to code, compile and run COMPSs applications. These applications are executed locally and show different ways to take advantage of all the COMPSs features.
The rest of the examples are more elaborated and consider the execution in a cloud platform where the VMs mount a common storage on /sharedDisk directory. This is useful in the case of applications that require working with big files, allowing to transfer data only once, at the beginning of the execution, and to enable the application to access the data directly during the rest of the execution.
The Virtual Machine available at our webpage (http://compss.bsc.es/)
provides a development environment with all the applications listed in
the following sections. The codes of all the applications can be found
under the /home/compss/tutorial_apps/java/ folder.
Hello World
The Hello Wolrd is a Java application that creates a task and prints a Hello World! message. Its purpose is to clarify that the COMPSs tasks output is redirected to the job files and it is not available at the standard output.
Next we provide the important parts of the application’s code.
// hello.Hello
public static void main(String[] args) throws Exception {
// Check and get parameters
if (args.length != 0) {
usage();
throw new Exception("[ERROR] Incorrect number of parameters");
}
// Hello World from main application
System.out.println("Hello World! (from main application)");
// Hello World from a task
HelloImpl.sayHello();
}
As shown in the main code, this application has no input arguments.
// hello.HelloImpl
public static void sayHello() {
System.out.println("Hello World! (from a task)");
}
Remember that, to run with COMPSs, java applications must provide an interface. For simplicity, in this example, the content of the interface only declares the task which has no parameters:
// hello.HelloItf
@Method(declaringClass = "hello.HelloImpl")
void sayHello(
);
Notice that there is a first Hello World message printed from the main code and, a second one, printed inside a task. When executing sequentially this application users will be able to see both messages at the standard output. However, when executing this application with COMPSs, users will only see the message from the main code at the standard output. The message printed from the task will be stored inside the job log files.
Let’s try it. First we proceed to compile the code by running the following instructions:
compss@bsc:~$ cd ~/tutorial_apps/java/hello/src/main/java/hello/
compss@bsc:~/tutorial_apps/java/hello/src/main/java/hello$ javac *.java
compss@bsc:~/tutorial_apps/java/hello/src/main/java/hello$ cd ..
compss@bsc:~/tutorial_apps/java/hello/src/main/java$ jar cf hello.jar hello
compss@bsc:~/tutorial_apps/java/hello/src/main/java$ mv hello.jar ~/tutorial_apps/java/hello/jar/
Alternatively, this example application is prepared to be compiled with maven:
compss@bsc:~$ cd ~/tutorial_apps/java/hello/
compss@bsc:~/tutorial_apps/java/hello$ mvn clean package
Once done, we can sequentially execute the application by directly invoking the jar file.
compss@bsc:~$ cd ~/tutorial_apps/java/hello/jar/
compss@bsc:~/tutorial_apps/java/hello/jar$ java -cp hello.jar hello.Hello
Hello World! (from main application)
Hello World! (from a task)
And we can also execute the application with COMPSs:
compss@bsc:~$ cd ~/tutorial_apps/java/hello/jar/
compss@bsc:~/tutorial_apps/java/hello/jar$ runcompss -d hello.Hello
[ INFO] Using default execution type: compss
[ INFO] Using default location for project file: /opt/COMPSs/Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs/Runtime/configuration/xml/resources/default_resources.xml
----------------- Executing hello.Hello --------------------------
WARNING: COMPSs Properties file is null. Setting default values
[(928) API] - Deploying COMPSs Runtime v<version>
[(931) API] - Starting COMPSs Runtime v<version>
[(931) API] - Initializing components
[(1472) API] - Ready to process tasks
Hello World! (from main application)
[(1474) API] - Creating task from method sayHello in hello.HelloImpl
[(1474) API] - There is 0 parameter
[(1477) API] - No more tasks for app 1
[(4029) API] - Getting Result Files 1
[(4030) API] - Stop IT reached
[(4030) API] - Stopping AP...
[(4031) API] - Stopping TD...
[(4161) API] - Stopping Comm...
[(4163) API] - Runtime stopped
[(4166) API] - Execution Finished
------------------------------------------------------------
Notice that the COMPSs execution is using the -d option to allow the job logging. Thus, we can check out the application jobs folder to look for the task output.
compss@bsc:~$ cd ~/.COMPSs/hello.Hello_01/jobs/
compss@bsc:~/.COMPSs/hello.Hello_01/jobs$ ls -1
job1_NEW.err
job1_NEW.out
compss@bsc:~/.COMPSs/hello.Hello_01/jobs$ cat job1_NEW.out
[JAVA EXECUTOR] executeTask - Begin task execution
WORKER - Parameters of execution:
* Method type: METHOD
* Method definition: [DECLARING CLASS=hello.HelloImpl, METHOD NAME=sayHello]
* Parameter types:
* Parameter values:
Hello World! (from a task)
[JAVA EXECUTOR] executeTask - End task execution
Simple
The Simple application is a Java application that increases a counter by means of a task. The counter is stored inside a file that is transferred to the worker when the task is executed. Thus, the tasks inferface is defined as follows:
// simple.SimpleItf
@Method(declaringClass = "simple.SimpleImpl")
void increment(
@Parameter(type = Type.FILE, direction = Direction.INOUT) String file
);
Next we also provide the invocation of the task from the main code and the increment’s method code.
// simple.Simple
public static void main(String[] args) throws Exception {
// Check and get parameters
if (args.length != 1) {
usage();
throw new Exception("[ERROR] Incorrect number of parameters");
}
int initialValue = Integer.parseInt(args[0]);
// Write value
FileOutputStream fos = new FileOutputStream(fileName);
fos.write(initialValue);
fos.close();
System.out.println("Initial counter value is " + initialValue);
//Execute increment
SimpleImpl.increment(fileName);
// Write new value
FileInputStream fis = new FileInputStream(fileName);
int finalValue = fis.read();
fis.close();
System.out.println("Final counter value is " + finalValue);
}
// simple.SimpleImpl
public static void increment(String counterFile) throws FileNotFoundException, IOException {
// Read value
FileInputStream fis = new FileInputStream(counterFile);
int count = fis.read();
fis.close();
// Write new value
FileOutputStream fos = new FileOutputStream(counterFile);
fos.write(++count);
fos.close();
}
Finally, to compile and execute this application users must run the following commands:
compss@bsc:~$ cd ~/tutorial_apps/java/simple/src/main/java/simple/
compss@bsc:~/tutorial_apps/java/simple/src/main/java/simple$ javac *.java
compss@bsc:~/tutorial_apps/java/simple/src/main/java/simple$ cd ..
compss@bsc:~/tutorial_apps/java/simple/src/main/java$ jar cf simple.jar simple
compss@bsc:~/tutorial_apps/java/simple/src/main/java$ mv simple.jar ~/tutorial_apps/java/simple/jar/
compss@bsc:~$ cd ~/tutorial_apps/java/simple/jar
compss@bsc:~/tutorial_apps/java/simple/jar$ runcompss simple.Simple 1
compss@bsc:~/tutorial_apps/java/simple/jar$ runcompss simple.Simple 1
[ INFO] Using default execution type: compss
[ INFO] Using default location for project file: /opt/COMPSs/Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs/Runtime/configuration/xml/resources/default_resources.xml
----------------- Executing simple.Simple --------------------------
WARNING: COMPSs Properties file is null. Setting default values
[(772) API] - Starting COMPSs Runtime v<version>
Initial counter value is 1
Final counter value is 2
[(3813) API] - Execution Finished
------------------------------------------------------------
Increment
The Increment application is a Java application that increases N times three different counters. Each increase step is developed by a separated task. The purpose of this application is to show parallelism between the three counters.
Next we provide the main code of this application. The code inside the increment task is the same than the previous example.
// increment.Increment
public static void main(String[] args) throws Exception {
// Check and get parameters
if (args.length != 4) {
usage();
throw new Exception("[ERROR] Incorrect number of parameters");
}
int N = Integer.parseInt(args[0]);
int counter1 = Integer.parseInt(args[1]);
int counter2 = Integer.parseInt(args[2]);
int counter3 = Integer.parseInt(args[3]);
// Initialize counter files
System.out.println("Initial counter values:");
initializeCounters(counter1, counter2, counter3);
// Print initial counters state
printCounterValues();
// Execute increment tasks
for (int i = 0; i < N; ++i) {
IncrementImpl.increment(fileName1);
IncrementImpl.increment(fileName2);
IncrementImpl.increment(fileName3);
}
// Print final counters state (sync)
System.out.println("Final counter values:");
printCounterValues();
}
As shown in the main code, this application has 4 parameters that stand for:
- N: Number of times to increase a counter
- InitialValue1: Initial value for counter 1
- InitialValue2: Initial value for counter 2
- InitialValue3: Initial value for counter 3
Next we will compile and run the Increment application with the -g option to be able to generate the final graph at the end of the execution.
compss@bsc:~$ cd ~/tutorial_apps/java/increment/src/main/java/increment/
compss@bsc:~/tutorial_apps/java/increment/src/main/java/increment$ javac *.java
compss@bsc:~/tutorial_apps/java/increment/src/main/java/increment$ cd ..
compss@bsc:~/tutorial_apps/java/increment/src/main/java$ jar cf increment.jar increment
compss@bsc:~/tutorial_apps/java/increment/src/main/java$ mv increment.jar ~/tutorial_apps/java/increment/jar/
compss@bsc:~$ cd ~/tutorial_apps/java/increment/jar
compss@bsc:~/tutorial_apps/java/increment/jar$ runcompss -g increment.Increment 10 1 2 3
[ INFO] Using default execution type: compss
[ INFO] Using default location for project file: /opt/COMPSs/Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs/Runtime/configuration/xml/resources/default_resources.xml
----------------- Executing increment.Increment --------------------------
WARNING: COMPSs Properties file is null. Setting default values
[(1028) API] - Starting COMPSs Runtime v<version>
Initial counter values:
- Counter1 value is 1
- Counter2 value is 2
- Counter3 value is 3
Final counter values:
- Counter1 value is 11
- Counter2 value is 12
- Counter3 value is 13
[(4403) API] - Execution Finished
------------------------------------------------------------
By running the compss_gengraph command users can obtain the task graph of the above execution. Next we provide the set of commands to obtain the graph show in Figure 42.
compss@bsc:~$ cd ~/.COMPSs/increment.Increment_01/monitor/
compss@bsc:~/.COMPSs/increment.Increment_01/monitor$ compss_gengraph complete_graph.dot
compss@bsc:~/.COMPSs/increment.Increment_01/monitor$ evince complete_graph.pdf
Java increment tasks graph
Matrix multiplication
The Matrix Multiplication (Matmul) is a pure Java application that multiplies two matrices in a direct way. The application creates 2 matrices of N x N size initialized with values, and multiply the matrices by blocks.
This application provides three different implementations that only differ on the way of storing the matrix:
- matmul.objects.Matmul
- Matrix stored by means of objects
- matmul.files.Matmul
- Matrix stored in files
- matmul.arrays.Matmul
- Matrix represented by an array
Matrix multiplication
In all the implementations the multiplication is implemented in the multiplyAccumulative method that is thus selected as the task to be executed remotely. As example, we we provide next the task implementation and the tasks interface for the objects implementation.
// matmul.objects.Block
public void multiplyAccumulative(Block a, Block b) {
for (int i = 0; i < M; i++) {
for (int j = 0; j < M; j++) {
for (int k = 0; k < M; k++) {
data[i][j] += a.data[i][k]*b.data[k][j];
}
}
}
}
// matmul.objects.MatmulItf
@Method(declaringClass = "matmul.objects.Block")
void multiplyAccumulative(
@Parameter Block a,
@Parameter Block b
);
In order to run the application the matrix dimension (number of blocks) and the dimension of each block have to be supplied. Consequently, any of the implementations must be executed by running the following command.
compss@bsc:~$ runcompss matmul.<IMPLEMENTATION_TYPE>.Matmul <matrix_dim> <block_dim>
Finally, we provide an example of execution for each implementation.
compss@bsc:~$ cd ~/tutorial_apps/java/matmul/jar/
compss@bsc:~/tutorial_apps/java/matmul/jar$ runcompss matmul.objects.Matmul 8 4
[ INFO] Using default execution type: compss
[ INFO] Using default location for project file: /opt/COMPSs/Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs/Runtime/configuration/xml/resources/default_resources.xml
----------------- Executing matmul.objects.Matmul --------------------------
WARNING: COMPSs Properties file is null. Setting default values
[(887) API] - Starting COMPSs Runtime v<version>
[LOG] MSIZE parameter value = 8
[LOG] BSIZE parameter value = 4
[LOG] Allocating A/B/C matrix space
[LOG] Computing Result
[LOG] Main program finished.
[(7415) API] - Execution Finished
------------------------------------------------------------
compss@bsc:~$ cd ~/tutorial_apps/java/matmul/jar/
compss@bsc:~/tutorial_apps/java/matmul/jar$ runcompss matmul.files.Matmul 8 4
[ INFO] Using default execution type: compss
[ INFO] Using default location for project file: /opt/COMPSs/Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs/Runtime/configuration/xml/resources/default_resources.xml
----------------- Executing matmul.files.Matmul --------------------------
WARNING: COMPSs Properties file is null. Setting default values
[(907) API] - Starting COMPSs Runtime v<version>
[LOG] MSIZE parameter value = 8
[LOG] BSIZE parameter value = 4
[LOG] Computing result
[LOG] Main program finished.
[(9925) API] - Execution Finished
------------------------------------------------------------
compss@bsc:~$ cd ~/tutorial_apps/java/matmul/jar/
compss@bsc:~/tutorial_apps/java/matmul/jar$ runcompss matmul.arrays.Matmul 8 4
[ INFO] Using default execution type: compss
[ INFO] Using default location for project file: /opt/COMPSs/Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs/Runtime/configuration/xml/resources/default_resources.xml
----------------- Executing matmul.arrays.Matmul --------------------------
WARNING: COMPSs Properties file is null. Setting default values
[(1062) API] - Starting COMPSs Runtime v<version>
[LOG] MSIZE parameter value = 8
[LOG] BSIZE parameter value = 4
[LOG] Allocating C matrix space
[LOG] Computing Result
[LOG] Main program finished.
[(7811) API] - Execution Finished
------------------------------------------------------------
Sparse LU decomposition
SparseLU multiplies two matrices using the factorization method of LU decomposition, which factorizes a matrix as a product of a lower triangular matrix and an upper one.
Sparse LU decomposition
The matrix is divided into N x N blocks on where 4 types of operations will be applied modifying the blocks: lu0, fwd, bdiv and bmod. These four operations are implemented in four methods that are selecetd as the tasks that will be executed remotely. In order to run the application the matrix dimension has to be provided.
As the previous application, the sparseLU is provided in three different implementations that only differ on the way of storing the matrix:
- sparseLU.objects.SparseLU Matrix stored by means of objects
- sparseLU.files.SparseLU Matrix stored in files
- sparseLU.arrays.SparseLU Matrix represented by an array
Thus, the commands needed to execute the application is with each implementation are:
compss@bsc:~$ cd tutorial_apps/java/sparseLU/jar/
compss@bsc:~/tutorial_apps/java/sparseLU/jar$ runcompss sparseLU.objects.SparseLU 16 8
[ INFO] Using default execution type: compss
[ INFO] Using default location for project file: /opt/COMPSs/Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs/Runtime/configuration/xml/resources/default_resources.xml
----------------- Executing sparseLU.objects.SparseLU --------------------------
WARNING: COMPSs Properties file is null. Setting default values
[(1221) API] - Starting COMPSs Runtime v<version>
[LOG] Running with the following parameters:
[LOG] - Matrix Size: 16
[LOG] - Block Size: 8
[LOG] Initializing Matrix
[LOG] Computing SparseLU algorithm on A
[LOG] Main program finished.
[(13642) API] - Execution Finished
------------------------------------------------------------
compss@bsc:~$ cd tutorial_apps/java/sparseLU/jar/
compss@bsc:~/tutorial_apps/java/sparseLU/jar$ runcompss sparseLU.files.SparseLU 4 8
[ INFO] Using default execution type: compss
[ INFO] Using default location for project file: /opt/COMPSs/Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs/Runtime/configuration/xml/resources/default_resources.xml
----------------- Executing sparseLU.files.SparseLU --------------------------
WARNING: COMPSs Properties file is null. Setting default values
[(1082) API] - Starting COMPSs Runtime v<version>
[LOG] Running with the following parameters:
[LOG] - Matrix Size: 16
[LOG] - Block Size: 8
[LOG] Initializing Matrix
[LOG] Computing SparseLU algorithm on A
[LOG] Main program finished.
[(13605) API] - Execution Finished
------------------------------------------------------------
compss@bsc:~$ cd tutorial_apps/java/sparseLU/jar/
compss@bsc:~/tutorial_apps/java/sparseLU/jar$ runcompss sparseLU.arrays.SparseLU 8 8
[ INFO] Using default execution type: compss
[ INFO] Using default location for project file: /opt/COMPSs/Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs/Runtime/configuration/xml/resources/default_resources.xml
----------------- Executing sparseLU.arrays.SparseLU --------------------------
WARNING: COMPSs Properties file is null. Setting default values
[(1082) API] - Starting COMPSs Runtime v<version>
[LOG] Running with the following parameters:
[LOG] - Matrix Size: 16
[LOG] - Block Size: 8
[LOG] Initializing Matrix
[LOG] Computing SparseLU algorithm on A
[LOG] Main program finished.
[(13605) API] - Execution Finished
------------------------------------------------------------
BLAST Workflow
BLAST is a widely-used bioinformatics tool for comparing primary biological sequence information, such as the amino-acid sequences of different proteins or the nucleotides of DNA sequences with sequence databases, identifying sequences that resemble the query sequence above a certain threshold. The work performed by the COMPSs Blast workflow is computationally intensive and embarrassingly parallel.
The COMPSs Blast workflow
The workflow describes the three blocks of the workflow implemented in the Split, Align and Assembly methods. The second one is the only method that is chosen to be executed remotely, so it is the unique method defined in the interface file. The Split method chops the query sequences file in N fragments, Align compares each sequence fragment against the database by means of the Blast binary, and Assembly combines all intermediate files into a single result file.
This application uses a database that will be on the shared disk space avoiding transferring the entire database (which can be large) between the virtual machines.
compss@bsc:~$ cp ~/workspace/blast/package/Blast.tar.gz /home/compss/
compss@bsc:~$ tar xzf Blast.tar.gz
The command line to execute the workflow:
compss@bsc:~$ runcompss blast.Blast <debug> \
<bin_location> \
<database_file> \
<sequences_file> \
<frag_number> \
<tmpdir> \
<output_file>
Where:
- debug: The debug flag of the application (true or false).
- bin_location: Path of the Blast binary.
- database_file: Path of database file; the shared disk /sharedDisk/ is suggested to avoid big data transfers.
- sequences_file: Path of sequences file.
- frag_number: Number of fragments of the original sequence file, this number determines the number of parallel Align tasks.
- tmpdir: Temporary directory (/home/compss/tmp/).
- output_file: Path of the result file.
Example:
compss@bsc:~$ runcompss blast.Blast true \
/home/compss/tutorial_apps/java/blast/binary/blastall \
/sharedDisk/Blast/databases/swissprot/swissprot \
/sharedDisk/Blast/sequences/sargasso_test.fasta \
4 \
/tmp/ \
/home/compss/out.txt
Python Sample applications
The first two examples in this section are simple applications developed in COMPSs to easily illustrate how to code, compile and run COMPSs applications. These applications are executed locally and show different ways to take advantage of all the COMPSs features.
The rest of the examples are more elaborated and consider the execution in a cloud platform where the VMs mount a common storage on /sharedDisk directory. This is useful in the case of applications that require working with big files, allowing to transfer data only once, at the beginning of the execution, and to enable the application to access the data directly during the rest of the execution.
The Virtual Machine available at our webpage (http://compss.bsc.es/)
provides a development environment with all the applications listed in
the following sections. The codes of all the applications can be found
under the /home/compss/tutorial_apps/python/ folder.
Simple
The Simple application is a Python application that increases a counter by means of a task. The counter is stored inside a file that is transfered to the worker when the task is executed. Next, we provide the main code and the task declaration:
from pycompss.api.task import task
from pycompss.api.parameter import FILE_INOUT
@task(filePath = FILE_INOUT)
def increment(filePath):
# Read value
fis = open(filePath, 'r')
value = fis.read()
fis.close()
# Write value
fos = open(filePath, 'w')
fos.write(str(int(value) + 1))
fos.close()
def main_program():
from pycompss.api.api import compss_open
# Check and get parameters
if len(sys.argv) != 2:
exit(-1)
initialValue = sys.argv[1]
fileName="counter"
# Write value
fos = open(fileName, 'w')
fos.write(initialValue)
fos.close()
print "Initial counter value is " + initialValue
# Execute increment
increment(fileName)
# Write new value
fis = compss_open(fileName, 'r+')
finalValue = fis.read()
fis.close()
print "Final counter value is " + finalValue
if __name__=='__main__':
main_program()
The simple application can be executed by invoking the runcompss command
with the application file name and the initial counter value.
The following lines provide an example of its execution.
compss@bsc:~$ cd ~/tutorial_apps/python/simple/
compss@bsc:~/tutorial_apps/python/simple$ runcompss ~/tutorial_apps/python/simple/simple.py 1
[ INFO] Using default execution type: compss
[ INFO] Using default location for project file: /opt/COMPSs/Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs/Runtime/configuration/xml/resources/default_resources.xml
----------------- Executing simple.py --------------------------
WARNING: COMPSs Properties file is null. Setting default values
[(639) API] - Starting COMPSs Runtime v<version>
Initial counter value is 1
Final counter value is 2
[(6230) API] - Execution Finished
------------------------------------------------------------
Increment
The Increment application is a Python application that increases N times three different counters. Each increase step is developed by a separated task. The purpose of this application is to show parallelism between the three counters.
Next we provide the main code of this application. The code inside the increment task is the same than the previous example.
from pycompss.api.task import task
from pycompss.api.parameter import FILE_INOUT
@task(filePath = FILE_INOUT)
def increment(filePath):
# Read value
fis = open(filePath, 'r')
value = fis.read()
fis.close()
# Write value
fos = open(filePath, 'w')
fos.write(str(int(value) + 1))
fos.close()
def main_program():
# Check and get parameters
if len(sys.argv) != 5:
exit(-1)
N = int(sys.argv[1])
counter1 = int(sys.argv[2])
counter2 = int(sys.argv[3])
counter3 = int(sys.argv[4])
# Initialize counter files
initializeCounters(counter1, counter2, counter3)
print "Initial counter values:"
printCounterValues()
# Execute increment
for i in range(N):
increment(FILENAME1)
increment(FILENAME2)
increment(FILENAME3)
# Write final counters state (sync)
print "Final counter values:"
printCounterValues()
if __name__=='__main__':
main_program()
As shown in the main code, this application has 4 parameters that stand for:
- N
- Number of times to increase a counter
- counter1
- Initial value for counter 1
- counter2
- Initial value for counter 2
- counter3
- Initial value for counter 3
Next we run the Increment application with the -g option to be able to
generate the final graph at the end of the execution.
compss@bsc:~/tutorial_apps/python/increment$ runcompss --lang=python -g ~/tutorial_apps/python/increment/increment.py 10 1 2 3
[ INFO] Using default execution type: compss
[ INFO] Using default location for project file: /opt/COMPSs/Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs/Runtime/configuration/xml/resources/default_resources.xml
----------------- Executing increment.py --------------------------
WARNING: COMPSs Properties file is null. Setting default values
[(670) API] - Starting COMPSs Runtime v<version>
Initial counter values:
- Counter1 value is 1
- Counter2 value is 2
- Counter3 value is 3
Final counter values:
- Counter1 value is 11
- Counter2 value is 12
- Counter3 value is 13
[(7390) API] - Execution Finished
------------------------------------------------------------
By running the compss_gengraph command users can obtain the task
graph of the above execution. Next we provide the set of commands to
obtain the graph show in Figure 46.
compss@bsc:~$ cd ~/.COMPSs/increment.py_01/monitor/
compss@bsc:~/.COMPSs/increment.py_01/monitor$ compss_gengraph complete_graph.dot
compss@bsc:~/.COMPSs/increment.py_01/monitor$ evince complete_graph.pdf
Python increment tasks graph
Kmeans
KMeans is machine-learning algorithm (NP-hard), popularly employed for cluster analysis in data mining, and interesting for benchmarking and performance evaluation.
The objective of the Kmeans algorithm to group a set of multidimensional points into a predefined number of clusters, in which each point belongs to the closest cluster (with the nearest mean distance), in an iterative process.
import numpy as np
import time
from sklearn.metrics import pairwise_distances
from sklearn.metrics.pairwise import paired_distances
from pycompss.api.task import task
from pycompss.api.api import compss_wait_on
from pycompss.api.api import compss_barrier
@task(returns=np.ndarray)
def partial_sum(fragment, centres):
partials = np.zeros((centres.shape[0], 2), dtype=object)
close_centres = pairwise_distances(fragment, centres).argmin(axis=1)
for center_idx, _ in enumerate(centres):
indices = np.argwhere(close_centres == center_idx).flatten()
partials[center_idx][0] = np.sum(fragment[indices], axis=0)
partials[center_idx][1] = indices.shape[0]
return partials
@task(returns=dict)
def merge(*data):
accum = data[0].copy()
for d in data[1:]:
accum += d
return accum
def converged(old_centres, centres, epsilon, iteration, max_iter):
if old_centres is None:
return False
dist = np.sum(paired_distances(centres, old_centres))
return dist < epsilon ** 2 or iteration >= max_iter
def recompute_centres(partials, old_centres, arity):
centres = old_centres.copy()
while len(partials) > 1:
partials_subset = partials[:arity]
partials = partials[arity:]
partials.append(merge(*partials_subset))
partials = compss_wait_on(partials)
for idx, sum_ in enumerate(partials[0]):
if sum_[1] != 0:
centres[idx] = sum_[0] / sum_[1]
return centres
def kmeans_frag(fragments, dimensions, num_centres=10, iterations=20,
seed=0., epsilon=1e-9, arity=50):
"""
A fragment-based K-Means algorithm.
Given a set of fragments, the desired number of clusters and the
maximum number of iterations, compute the optimal centres and the
index of the centre for each point.
:param fragments: Number of fragments
:param dimensions: Number of dimensions
:param num_centres: Number of centres
:param iterations: Maximum number of iterations
:param seed: Random seed
:param epsilon: Epsilon (convergence distance)
:param arity: Reduction arity
:return: Final centres
"""
# Set the random seed
np.random.seed(seed)
# Centres is usually a very small matrix, so it is affordable to have it in
# the master.
centres = np.asarray(
[np.random.random(dimensions) for _ in range(num_centres)]
)
# Note: this implementation treats the centres as files, never as PSCOs.
old_centres = None
iteration = 0
while not converged(old_centres, centres, epsilon, iteration, iterations):
print("Doing iteration #%d/%d" % (iteration + 1, iterations))
old_centres = centres.copy()
partials = []
for frag in fragments:
partial = partial_sum(frag, old_centres)
partials.append(partial)
centres = recompute_centres(partials, old_centres, arity)
iteration += 1
return centres
def parse_arguments():
"""
Parse command line arguments. Make the program generate
a help message in case of wrong usage.
:return: Parsed arguments
"""
import argparse
parser = argparse.ArgumentParser(description='KMeans Clustering.')
parser.add_argument('-s', '--seed', type=int, default=0,
help='Pseudo-random seed. Default = 0')
parser.add_argument('-n', '--numpoints', type=int, default=100,
help='Number of points. Default = 100')
parser.add_argument('-d', '--dimensions', type=int, default=2,
help='Number of dimensions. Default = 2')
parser.add_argument('-c', '--num_centres', type=int, default=5,
help='Number of centres. Default = 2')
parser.add_argument('-f', '--fragments', type=int, default=10,
help='Number of fragments.' +
' Default = 10. Condition: fragments < points')
parser.add_argument('-m', '--mode', type=str, default='uniform',
choices=['uniform', 'normal'],
help='Distribution of points. Default = uniform')
parser.add_argument('-i', '--iterations', type=int, default=20,
help='Maximum number of iterations')
parser.add_argument('-e', '--epsilon', type=float, default=1e-9,
help='Epsilon. Kmeans will stop when:' +
' |old - new| < epsilon.')
parser.add_argument('-a', '--arity', type=int, default=50,
help='Arity of the reduction carried out during \
the computation of the new centroids')
return parser.parse_args()
@task(returns=1)
def generate_fragment(points, dim, mode, seed):
"""
Generate a random fragment of the specified number of points using the
specified mode and the specified seed. Note that the generation is
distributed (the master will never see the actual points).
:param points: Number of points
:param dim: Number of dimensions
:param mode: Dataset generation mode
:param seed: Random seed
:return: Dataset fragment
"""
# Random generation distributions
rand = {
'normal': lambda k: np.random.normal(0, 1, k),
'uniform': lambda k: np.random.random(k),
}
r = rand[mode]
np.random.seed(seed)
mat = np.asarray(
[r(dim) for __ in range(points)]
)
# Normalize all points between 0 and 1
mat -= np.min(mat)
mx = np.max(mat)
if mx > 0.0:
mat /= mx
return mat
def main(seed, numpoints, dimensions, num_centres, fragments, mode, iterations,
epsilon, arity):
"""
This will be executed if called as main script. Look at the kmeans_frag
for the KMeans function.
This code is used for experimental purposes.
I.e it generates random data from some parameters that determine the size,
dimensionality and etc and returns the elapsed time.
:param seed: Random seed
:param numpoints: Number of points
:param dimensions: Number of dimensions
:param num_centres: Number of centres
:param fragments: Number of fragments
:param mode: Dataset generation mode
:param iterations: Number of iterations
:param epsilon: Epsilon (convergence distance)
:param arity: Reduction arity
:return: None
"""
start_time = time.time()
# Generate the data
fragment_list = []
# Prevent infinite loops
points_per_fragment = max(1, numpoints // fragments)
for l in range(0, numpoints, points_per_fragment):
# Note that the seed is different for each fragment.
# This is done to avoid having repeated data.
r = min(numpoints, l + points_per_fragment)
fragment_list.append(
generate_fragment(r - l, dimensions, mode, seed + l)
)
compss_barrier()
print("Generation/Load done")
initialization_time = time.time()
print("Starting kmeans")
# Run kmeans
centres = kmeans_frag(fragments=fragment_list,
dimensions=dimensions,
num_centres=num_centres,
iterations=iterations,
seed=seed,
epsilon=epsilon,
arity=arity)
compss_barrier()
print("Ending kmeans")
kmeans_time = time.time()
print("-----------------------------------------")
print("-------------- RESULTS ------------------")
print("-----------------------------------------")
print("Initialization time: %f" % (initialization_time - start_time))
print("Kmeans time: %f" % (kmeans_time - initialization_time))
print("Total time: %f" % (kmeans_time - start_time))
print("-----------------------------------------")
centres = compss_wait_on(centres)
print("CENTRES:")
print(centres)
print("-----------------------------------------")
if __name__ == "__main__":
options = parse_arguments()
main(**vars(options))
The kmeans application can be executed by invoking the runcompss command
with the desired parameters (in this case we use -g to generate the
task depedency graph) and application.
The following lines provide an example of its execution considering 10M points,
of 3 dimensions, divided into 8 fragments, looking for 8 clusters and a maximum
number of iterations set to 10.
compss@bsc:~$ runcompss -g kmeans.py -n 10240000 -f 8 -d 3 -c 8 -i 10
[ INFO] Inferred PYTHON language
[ INFO] Using default location for project file: /opt/COMPSs//Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs//Runtime/configuration/xml/resources/default_resources.xml
[ INFO] Using default execution type: compss
----------------- Executing kmeans.py --------------------------
WARNING: COMPSs Properties file is null. Setting default values
[(436) API] - Starting COMPSs Runtime v2.7 (build 20200519-1005.r6093e5ac94d67250e097a6fad9d3ec00d676fe6c)
Generation/Load done
Starting kmeans
Doing iteration #1/5
Doing iteration #2/5
Doing iteration #3/5
Doing iteration #4/5
Doing iteration #5/5
Ending kmeans
-----------------------------------------
-------------- RESULTS ------------------
-----------------------------------------
Initialization time: 8.625658
Kmeans time: 6.110023
Total time: 14.735682
-----------------------------------------
CENTRES:
[[0.72244748 0.73760837 0.47839032]
[0.555741 0.20736841 0.21758715]
[0.25766653 0.73309038 0.77668994]
[0.20623714 0.67588471 0.25750168]
[0.73305652 0.7013741 0.15204797]
[0.22431367 0.22614948 0.66875431]
[0.76540302 0.75721277 0.83083206]
[0.75688812 0.24817146 0.72752128]]
-----------------------------------------
[(16137) API] - Execution Finished
------------------------------------------------------------
Figure 47 depicts the generated task dependency graph. The dataset generation can be identified in the 8 blue tasks, while the five iterations appear next. Between the iteration there is a synchronization which corresponds to the convergence/max iterations check.
Python kmeans tasks graph
Kmeans with Persistent Storage
KMeans is machine-learning algorithm (NP-hard), popularly employed for cluster analysis in data mining, and interesting for benchmarking and performance evaluation.
The objective of the Kmeans algorithm to group a set of multidimensional points into a predefined number of clusters, in which each point belongs to the closest cluster (with the nearest mean distance), in an iterative process.
In this application we make use of the persistent storage API.
In particular, the dataset fragments are considered StorageObject,
delegating its content into the persistent framework.
Since the data model (object declared as storage object) includes functions,
it can run efficiently with dataClay.
First, lets see the data model (storage_model/fragment.py)
from storage.api import StorageObject
try:
from pycompss.api.task import task
from pycompss.api.parameter import IN
except ImportError:
# Required since the pycompss module is not ready during the registry
from dataclay.contrib.dummy_pycompss import task, IN
from dataclay import dclayMethod
import numpy as np
from sklearn.metrics import pairwise_distances
class Fragment(StorageObject):
"""
@ClassField points numpy.ndarray
@dclayImport numpy as np
@dclayImportFrom sklearn.metrics import pairwise_distances
"""
@dclayMethod()
def __init__(self):
super(Fragment, self).__init__()
self.points = None
@dclayMethod(num_points='int', dim='int', mode='str', seed='int')
def generate_points(self, num_points, dim, mode, seed):
"""
Generate a random fragment of the specified number of points using the
specified mode and the specified seed. Note that the generation is
distributed (the master will never see the actual points).
:param num_points: Number of points
:param dim: Number of dimensions
:param mode: Dataset generation mode
:param seed: Random seed
:return: Dataset fragment
"""
# Random generation distributions
rand = {
'normal': lambda k: np.random.normal(0, 1, k),
'uniform': lambda k: np.random.random(k),
}
r = rand[mode]
np.random.seed(seed)
mat = np.asarray(
[r(dim) for __ in range(num_points)]
)
# Normalize all points between 0 and 1
mat -= np.min(mat)
mx = np.max(mat)
if mx > 0.0:
mat /= mx
self.points = mat
@task(returns=np.ndarray, target_direction=IN)
@dclayMethod(centres='numpy.ndarray', return_='anything')
def partial_sum(self, centres):
partials = np.zeros((centres.shape[0], 2), dtype=object)
arr = self.points
close_centres = pairwise_distances(arr, centres).argmin(axis=1)
for center_idx, _ in enumerate(centres):
indices = np.argwhere(close_centres == center_idx).flatten()
partials[center_idx][0] = np.sum(arr[indices], axis=0)
partials[center_idx][1] = indices.shape[0]
return partials
Now we can focus in the main kmeans application (kmeans.py):
import time
import numpy as np
from pycompss.api.task import task
from pycompss.api.api import compss_wait_on
from pycompss.api.api import compss_barrier
from storage_model.fragment import Fragment
from sklearn.metrics.pairwise import paired_distances
@task(returns=dict)
def merge(*data):
accum = data[0].copy()
for d in data[1:]:
accum += d
return accum
def converged(old_centres, centres, epsilon, iteration, max_iter):
if old_centres is None:
return False
dist = np.sum(paired_distances(centres, old_centres))
return dist < epsilon ** 2 or iteration >= max_iter
def recompute_centres(partials, old_centres, arity):
centres = old_centres.copy()
while len(partials) > 1:
partials_subset = partials[:arity]
partials = partials[arity:]
partials.append(merge(*partials_subset))
partials = compss_wait_on(partials)
for idx, sum_ in enumerate(partials[0]):
if sum_[1] != 0:
centres[idx] = sum_[0] / sum_[1]
return centres
def kmeans_frag(fragments, dimensions, num_centres=10, iterations=20,
seed=0., epsilon=1e-9, arity=50):
"""
A fragment-based K-Means algorithm.
Given a set of fragments (which can be either PSCOs or future objects that
point to PSCOs), the desired number of clusters and the maximum number of
iterations, compute the optimal centres and the index of the centre
for each point.
PSCO.mat must be a NxD float np.ndarray, where D = dimensions
:param fragments: Number of fragments
:param dimensions: Number of dimensions
:param num_centres: Number of centres
:param iterations: Maximum number of iterations
:param seed: Random seed
:param epsilon: Epsilon (convergence distance)
:param arity: Arity
:return: Final centres and labels
"""
# Set the random seed
np.random.seed(seed)
# Centres is usually a very small matrix, so it is affordable to have it in
# the master.
centres = np.asarray(
[np.random.random(dimensions) for _ in range(num_centres)]
)
# Note: this implementation treats the centres as files, never as PSCOs.
old_centres = None
iteration = 0
while not converged(old_centres, centres, epsilon, iteration, iterations):
print("Doing iteration #%d/%d" % (iteration + 1, iterations))
old_centres = centres.copy()
partials = []
for frag in fragments:
partial = frag.partial_sum(old_centres)
partials.append(partial)
centres = recompute_centres(partials, old_centres, arity)
iteration += 1
return centres
def parse_arguments():
"""
Parse command line arguments. Make the program generate
a help message in case of wrong usage.
:return: Parsed arguments
"""
import argparse
parser = argparse.ArgumentParser(description='KMeans Clustering.')
parser.add_argument('-s', '--seed', type=int, default=0,
help='Pseudo-random seed. Default = 0')
parser.add_argument('-n', '--numpoints', type=int, default=100,
help='Number of points. Default = 100')
parser.add_argument('-d', '--dimensions', type=int, default=2,
help='Number of dimensions. Default = 2')
parser.add_argument('-c', '--num_centres', type=int, default=5,
help='Number of centres. Default = 2')
parser.add_argument('-f', '--fragments', type=int, default=10,
help='Number of fragments.' +
' Default = 10. Condition: fragments < points')
parser.add_argument('-m', '--mode', type=str, default='uniform',
choices=['uniform', 'normal'],
help='Distribution of points. Default = uniform')
parser.add_argument('-i', '--iterations', type=int, default=20,
help='Maximum number of iterations')
parser.add_argument('-e', '--epsilon', type=float, default=1e-9,
help='Epsilon. Kmeans will stop when:' +
' |old - new| < epsilon.')
parser.add_argument('-a', '--arity', type=int, default=50,
help='Arity of the reduction carried out during \
the computation of the new centroids')
return parser.parse_args()
from storage_model.fragment import Fragment # this will have to be removed
@task(returns=Fragment)
def generate_fragment(points, dim, mode, seed):
"""
Generate a random fragment of the specified number of points using the
specified mode and the specified seed. Note that the generation is
distributed (the master will never see the actual points).
:param points: Number of points
:param dim: Number of dimensions
:param mode: Dataset generation mode
:param seed: Random seed
:return: Dataset fragment
"""
fragment = Fragment()
# Make persistent before since it is populated in the task
fragment.make_persistent()
fragment.generate_points(points, dim, mode, seed)
def main(seed, numpoints, dimensions, num_centres, fragments, mode, iterations,
epsilon, arity):
"""
This will be executed if called as main script. Look at the kmeans_frag
for the KMeans function.
This code is used for experimental purposes.
I.e it generates random data from some parameters that determine the size,
dimensionality and etc and returns the elapsed time.
:param seed: Random seed
:param numpoints: Number of points
:param dimensions: Number of dimensions
:param num_centres: Number of centres
:param fragments: Number of fragments
:param mode: Dataset generation mode
:param iterations: Number of iterations
:param epsilon: Epsilon (convergence distance)
:param arity: Arity
:return: None
"""
start_time = time.time()
# Generate the data
fragment_list = []
# Prevent infinite loops in case of not-so-smart users
points_per_fragment = max(1, numpoints // fragments)
for l in range(0, numpoints, points_per_fragment):
# Note that the seed is different for each fragment.
# This is done to avoid having repeated data.
r = min(numpoints, l + points_per_fragment)
fragment_list.append(
generate_fragment(r - l, dimensions, mode, seed + l)
)
compss_barrier()
print("Generation/Load done")
initialization_time = time.time()
print("Starting kmeans")
# Run kmeans
centres = kmeans_frag(fragments=fragment_list,
dimensions=dimensions,
num_centres=num_centres,
iterations=iterations,
seed=seed,
epsilon=epsilon,
arity=arity)
compss_barrier()
print("Ending kmeans")
kmeans_time = time.time()
print("-----------------------------------------")
print("-------------- RESULTS ------------------")
print("-----------------------------------------")
print("Initialization time: %f" % (initialization_time - start_time))
print("Kmeans time: %f" % (kmeans_time - initialization_time))
print("Total time: %f" % (kmeans_time - start_time))
print("-----------------------------------------")
centres = compss_wait_on(centres)
print("CENTRES:")
print(centres)
print("-----------------------------------------")
if __name__ == "__main__":
options = parse_arguments()
main(**vars(options))
Tip
This code can work with Hecuba and Redis if the functions declared in
the data model are declared outside the data model, and the kmeans
application uses the points attribute explicitly.
Since this code is going to be executed with dataClay, it is be necessary to
declare the client.properties, session.properties and
storage_props.cfg files into the dataClay_confs with the following
contents as example (more configuration options can be found in the
dataClay manual):
- client.properties
HOST=127.0.0.1 TCPPORT=11034
- session.properties
Account=bsc_user Password=bsc_user StubsClasspath=./stubs DataSets=hpc_dataset DataSetForStore=hpc_dataset DataClayClientConfig=./client.properties
- storage_props.cfg
BACKENDS_PER_NODE=48
An example of the submission script that can be used in MareNostrum IV to launch this kmeans with PyCOMPSs and dataClay is:
dsafasdf
#!/bin/bash -e
module load gcc/8.1.0
export COMPSS_PYTHON_VERSION=3-ML
module load COMPSs/2.7
module load mkl/2018.1
module load impi/2018.1
module load opencv/4.1.2
module load DATACLAY/2.4.dev
# Retrieve script arguments
job_dependency=${1:-None}
num_nodes=${2:-2}
execution_time=${3:-5}
tracing=${4:-false}
exec_file=${5:-$(pwd)/kmeans.py}
# Freeze storage_props into a temporal
# (allow submission of multiple executions with varying parameters)
STORAGE_PROPS=`mktemp -p ~`
cp $(pwd)/dataClay_confs/storage_props.cfg "${STORAGE_PROPS}"
if [[ ! ${tracing} == "false" ]]
then
extra_tracing_flags="\
--jvm_workers_opts=\"-javaagent:/apps/DATACLAY/dependencies/aspectjweaver.jar\" \
--jvm_master_opts=\"-javaagent:/apps/DATACLAY/dependencies/aspectjweaver.jar\" \
"
echo "Adding DATACLAYSRV_START_CMD to storage properties file"
echo "\${STORAGE_PROPS}=${STORAGE_PROPS}"
echo "" >> ${STORAGE_PROPS}
echo "DATACLAYSRV_START_CMD=\"--tracing\"" >> ${STORAGE_PROPS}
fi
# Define script variables
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
WORK_DIR=${SCRIPT_DIR}/
APP_CLASSPATH=${SCRIPT_DIR}/
APP_PYTHONPATH=${SCRIPT_DIR}/
# Define application variables
graph=$tracing
log_level="off"
qos_flag="--qos=debug"
workers_flag=""
constraints="highmem"
CPUS_PER_NODE=48
WORKER_IN_MASTER=0
shift 5
# Those are evaluated at submit time, not at start time...
COMPSS_VERSION=`module load whatis COMPSs 2>&1 >/dev/null | awk '{print $1 ; exit}'`
DATACLAY_VERSION=`module load whatis DATACLAY 2>&1 >/dev/null | awk '{print $1 ; exit}'`
# Enqueue job
enqueue_compss \
--job_name=kmeansOO_PyCOMPSs_dataClay \
--job_dependency="${job_dependency}" \
--exec_time="${execution_time}" \
--num_nodes="${num_nodes}" \
\
--cpus_per_node="${CPUS_PER_NODE}" \
--worker_in_master_cpus="${WORKER_IN_MASTER}" \
--scheduler=es.bsc.compss.scheduler.loadbalancing.LoadBalancingScheduler \
\
"${workers_flag}" \
\
--worker_working_dir=/gpfs/scratch/user/ \
\
--constraints=${constraints} \
--tracing="${tracing}" \
--graph="${graph}" \
--summary \
--log_level="${log_level}" \
"${qos_flag}" \
\
--classpath=${DATACLAY_JAR} \
--pythonpath=${APP_PYTHONPATH}:${PYCLAY_PATH}:${PYTHONPATH} \
--storage_props=${STORAGE_PROPS} \
--storage_home=$COMPSS_STORAGE_HOME \
--prolog="$DATACLAY_HOME/bin/dataclayprepare,$(pwd)/storage_model/,$(pwd)/,storage_model,python" \
\
${extra_tracing_flags} \
\
--lang=python \
\
"$exec_file" $@ --use_storage
Matmul
The matmul performs the matrix multiplication of two matrices.
import time
import numpy as np
from pycompss.api.task import task
from pycompss.api.parameter import INOUT
from pycompss.api.api import compss_barrier
from pycompss.api.api import compss_wait_on
@task(returns=1)
def generate_block(size, num_blocks, seed=0, set_to_zero=False):
"""
Generate a square block of given size.
:param size: <Integer> Block size
:param num_blocks: <Integer> Number of blocks
:param seed: <Integer> Random seed
:param set_to_zero: <Boolean> Set block to zeros
:return: Block
"""
np.random.seed(seed)
if not set_to_zero:
b = np.random.random((size, size))
# Normalize matrix to ensure more numerical precision
b /= np.sum(b) * float(num_blocks)
else:
b = np.zeros((size, size))
return b
@task(C=INOUT)
def fused_multiply_add(A, B, C):
"""
Multiplies two Blocks and accumulates the result in an INOUT Block (FMA).
:param A: Block A
:param B: Block B
:param C: Result Block
:return: None
"""
C += np.dot(A, B)
def dot(A, B, C):
"""
A COMPSs blocked matmul algorithm.
:param A: Block A
:param B: Block B
:param C: Result Block
:return: None
"""
n, m = len(A), len(B[0])
# as many rows as A, as many columns as B
for i in range(n):
for j in range(m):
for k in range(n):
fused_multiply_add(A[i][k], B[k][j], C[i][j])
def main(num_blocks, elems_per_block, seed):
"""
Matmul main.
:param num_blocks: <Integer> Number of blocks
:param elems_per_block: <Integer> Number of elements per block
:param seed: <Integer> Random seed
:return: None
"""
start_time = time.time()
# Generate the dataset in a distributed manner
# i.e: avoid having the master a whole matrix
A, B, C = [], [], []
matrix_name = ["A", "B"]
for i in range(num_blocks):
for l in [A, B, C]:
l.append([])
# Keep track of blockId to initialize with different random seeds
bid = 0
for j in range(num_blocks):
for ix, l in enumerate([A, B]):
l[-1].append(generate_block(elems_per_block,
num_blocks,
seed=seed + bid))
bid += 1
C[-1].append(generate_block(elems_per_block,
num_blocks,
set_to_zero=True))
compss_barrier()
initialization_time = time.time()
# Do matrix multiplication
dot(A, B, C)
compss_barrier()
multiplication_time = time.time()
print("-----------------------------------------")
print("-------------- RESULTS ------------------")
print("-----------------------------------------")
print("Initialization time: %f" % (initialization_time -
start_time))
print("Multiplication time: %f" % (multiplication_time -
initialization_time))
print("Total time: %f" % (multiplication_time - start_time))
print("-----------------------------------------")
def parse_args():
"""
Arguments parser.
Code for experimental purposes.
:return: Parsed arguments.
"""
import argparse
description = 'COMPSs blocked matmul implementation'
parser = argparse.ArgumentParser(description=description)
parser.add_argument('-b', '--num_blocks', type=int, default=1,
help='Number of blocks (N in NxN)'
)
parser.add_argument('-e', '--elems_per_block', type=int, default=2,
help='Elements per block (N in NxN)'
)
parser.add_argument('--seed', type=int, default=0,
help='Pseudo-Random seed'
)
return parser.parse_args()
if __name__ == "__main__":
opts = parse_args()
main(**vars(opts))
The matrix multiplication application can be executed by invoking the
runcompss command with the desired parameters (in this case we use -g
to generate the task depedency graph) and application.
The following lines provide an example of its execution considering 4 x 4 Blocks
of 1024 x 1024 elements each block, which conforms matrices of 4096 x 4096 elements.
compss@bsc:~$ runcompss -g matmul.py -b 4 -e 1024
[ INFO] Inferred PYTHON language
[ INFO] Using default location for project file: /opt/COMPSs//Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs//Runtime/configuration/xml/resources/default_resources.xml
[ INFO] Using default execution type: compss
----------------- Executing matmul.py --------------------------
WARNING: COMPSs Properties file is null. Setting default values
[(439) API] - Starting COMPSs Runtime v2.7 (build 20200519-1005.r6093e5ac94d67250e097a6fad9d3ec00d676fe6c)
-----------------------------------------
-------------- RESULTS ------------------
-----------------------------------------
Initialization time: 4.112615
Multiplication time: 2.366103
Total time: 6.478717
-----------------------------------------
[(5609) API] - Execution Finished
------------------------------------------------------------
Figure 48 depicts the generated task dependency graph. The dataset generation can be identified in the blue tasks, while the white tasks represent the multiplication of a block with another.
Python matrix multiplication tasks graph
Lysozyme in water
This example will guide a new user through the usage of the @binary,
@mpi and @constraint decorators for setting up a simulation system
containing a set of proteins (lysozymes) in boxes of water with ions.
Each step contains an explanation of input and output,
using typical settings for general use.
Extracted from: http://www.mdtutorials.com/gmx/lysozyme/index.html Originally done by: Justin A. Lemkul, Ph.D. From: Virginia Tech Department of Biochemistry
Note
This example reaches up to stage 4 (energy minimization).
Important
This application requires Gromacs gmx and gmx_mpi.
from os import listdir
from os.path import isfile, join
import sys
from pycompss.api.task import task
from pycompss.api.constraint import constraint
from pycompss.api.binary import binary
from pycompss.api.mpi import mpi
from pycompss.api.parameter import *
# ############ #
# Step 1 tasks #
# ############ #
@binary(binary='${GMX_BIN}/gmx')
@task(protein=FILE_IN,
structure=FILE_OUT,
topology=FILE_OUT)
def generate_topology(mode='pdb2gmx',
protein_flag='-f', protein=None,
structure_flag='-o', structure=None,
topology_flag='-p', topology=None,
flags='-ignh',
forcefield_flag='-ff', forcefield='oplsaa',
water_flag='-water', water='spce'):
# Command: gmx pdb2gmx -f protein.pdb -o structure.gro -p topology.top -ignh -ff amber03 -water tip3p
pass
# ############ #
# Step 2 tasks #
# ############ #
@binary(binary='${GMX_BIN}/gmx')
@task(structure=FILE_IN,
structure_newbox=FILE_OUT)
def define_box(mode='editconf',
structure_flag='-f', structure=None,
structure_newbox_flag='-o', structure_newbox=None,
center_flag='-c',
distance_flag='-d', distance='1.0',
boxtype_flag='-bt', boxtype='cubic'):
# Command: gmx editconf -f structure.gro -o structure_newbox.gro -c -d 1.0 -bt cubic
pass
# ############ #
# Step 3 tasks #
# ############ #
@binary(binary='${GMX_BIN}/gmx')
@task(structure_newbox=FILE_IN,
protein_solv=FILE_OUT,
topology=FILE_IN)
def add_solvate(mode='solvate',
structure_newbox_flag='-cp', structure_newbox=None,
configuration_solvent_flag='-cs', configuration_solvent='spc216.gro',
protein_solv_flag='-o', protein_solv=None,
topology_flag='-p', topology=None):
# Command: gmx solvate -cp structure_newbox.gro -cs spc216.gro -o protein_solv.gro -p topology.top
pass
# ############ #
# Step 4 tasks #
# ############ #
@binary(binary='${GMX_BIN}/gmx')
@task(conf=FILE_IN,
protein_solv=FILE_IN,
topology=FILE_IN,
output=FILE_OUT)
def assemble_tpr(mode='grompp',
conf_flag='-f', conf=None,
protein_solv_flag='-c', protein_solv=None,
topology_flag='-p', topology=None,
output_flag='-o', output=None):
# Command: gmx grompp -f ions.mdp -c protein_solv.gro -p topology.top -o ions.tpr
pass
@binary(binary='${GMX_BIN}/gmx')
@task(ions=FILE_IN,
output=FILE_OUT,
topology=FILE_IN,
group={Type:FILE_IN, StdIOStream:STDIN})
def replace_solvent_with_ions(mode='genion',
ions_flag='-s', ions=None,
output_flag='-o', output=None,
topology_flag='-p', topology=None,
pname_flag='-pname', pname='NA',
nname_flag='-nname', nname='CL',
neutral_flag='-neutral',
group=None):
# Command: gmx genion -s ions.tpr -o 1AKI_solv_ions.gro -p topol.top -pname NA -nname CL -neutral < ../config/genion.group
pass
# ############ #
# Step 5 tasks #
# ############ #
computing_units = "24"
computing_nodes = "1"
@constraint(computing_units=computing_units)
@mpi(runner="mpirun", binary="gmx_mpi", computing_nodes=computing_nodes)
@task(em=FILE_IN,
em_energy=FILE_OUT)
def energy_minimization(mode='mdrun',
verbose_flag='-v',
ompthreads_flag='-ntomp', ompthreads='0',
em_flag='-s', em=None,
em_energy_flag='-e', em_energy=None):
# Command: gmx mdrun -v -s em.tpr
pass
# ############ #
# Step 6 tasks #
# ############ #
@binary(binary='${GMX_BIN}/gmx')
@task(em=FILE_IN,
output=FILE_OUT,
selection={Type:FILE_IN, StdIOStream:STDIN})
def energy_analisis(mode='energy',
em_flag='-f', em=None,
output_flag='-o', output=None,
selection=None):
# Command: gmx energy -f em.edr -o output.xvg
pass
# ############# #
# MAIN FUNCTION #
# ############# #
def main(dataset_path, output_path, config_path):
print("Starting demo")
protein_names = []
protein_pdbs = []
# Look for proteins in the dataset folder
for f in listdir(dataset_path):
if isfile(join(dataset_path, f)):
protein_names.append(f.split('.')[0])
protein_pdbs.append(join(dataset_path, f))
proteins = zip(protein_names, protein_pdbs)
# Iterate over the proteins and process them
result_image_paths = []
for name, pdb in proteins:
# 1st step - Generate topology
structure = join(output_path, name + '.gro')
topology = join(output_path, name + '.top')
generate_topology(protein=pdb,
structure=structure,
topology=topology)
# 2nd step - Define box
structure_newbox = join(output_path, name + '_newbox.gro')
define_box(structure=structure,
structure_newbox=structure_newbox)
# 3rd step - Add solvate
protein_solv = join(output_path, name + '_solv.gro')
add_solvate(structure_newbox=structure_newbox,
protein_solv=protein_solv,
topology=topology)
# 4th step - Add ions
# Assemble with ions.mdp
ions_conf = join(config_path, 'ions.mdp')
ions = join(output_path, name + '_ions.tpr')
assemble_tpr(conf=ions_conf,
protein_solv=protein_solv,
topology=topology,
output=ions)
protein_solv_ions = join(output_path, name + '_solv_ions.gro')
group = join(config_path, 'genion.group') # 13 = SOL
replace_solvent_with_ions(ions=ions,
output=protein_solv_ions,
topology=topology,
group=group)
# 5th step - Minimize energy
# Reasemble with minim.mdp
minim_conf = join(config_path, 'minim.mdp')
em = join(output_path, name + '_em.tpr')
assemble_tpr(conf=minim_conf,
protein_solv=protein_solv_ions,
topology=topology,
output=em)
em_energy = join(output_path, name + '_em_energy.edr')
energy_minimization(em=em,
em_energy=em_energy)
# 6th step - Energy analysis (generate xvg image)
energy_result = join(output_path, name + '_potential.xvg')
energy_selection = join(config_path, 'energy.selection') # 10 = potential
energy_analisis(em=em_energy,
output=energy_result,
selection=energy_selection)
if __name__=='__main__':
config_path = sys.argv[1]
dataset_path = sys.argv[2]
output_path = sys.argv[3]
main(dataset_path, output_path, config_path)
This application can be executed by invoking the runcompss command defining
the config_path, dataset_path and output_path where the application
inputs and outputs are. For the sake of completeness, we show how to execute
this application in a Supercomputer. In this case, the execution will be
enqueued in the supercomputer queuing system (e.g. SLURM) through the use
of the enqueue_compss command, where all parameters used in runcompss
must appear, as well as some parameters required for the queuing system (e.g. walltime).
The following code shows a bash script to submit the execution in MareNostrum IV supercomputer:
#!/bin/bash -e
# Define script variables
scriptDir=$(pwd)/$(dirname $0)
execFile=${scriptDir}/src/lysozyme_in_water.py
appClasspath=${scriptDir}/src/
appPythonpath=${scriptDir}/src/
# Retrieve arguments
numNodes=$1
executionTime=$2
tracing=$3
# Leave application args on $@
shift 3
# Load necessary modules
module purge
module load intel/2017.4 impi/2017.4 mkl/2017.4 bsc/1.0
module load COMPSs/2.7
module load gromacs/2016.4 # exposes gmx_mpi binary
export GMX_BIN=/home/user/lysozyme5.1.2/bin # exposes gmx binary
# Enqueue the application
enqueue_compss \
--num_nodes=$numNodes \
--exec_time=$executionTime \
--master_working_dir=. \
--worker_working_dir=scratch \
--tracing=$tracing \
--graph=true \
-d \
--classpath=$appClasspath \
--pythonpath=$appPythonpath \
--lang=python \
$execFile $@
######################################################
# APPLICATION EXECUTION EXAMPLE
# Call:
# ./launch_md.sh <NUMBER_OF_NODES> <EXECUTION_TIME> <TRACING> <CONFIG_PATH> <DATASET_PATH> <OUTPUT_PATH>
#
# Example:
# ./launch_md.sh 2 10 false $(pwd)/config/ $(pwd)/dataset/ $(pwd)/output/
#
#####################################################
Having the 1aki.pdb, 1u3m.pdb and 1xyw.pdb`` proteins in the dataset
folder, the execution of this script produces the submission of the job with
the following output:
$ ./launch_md.sh 2 10 false $(pwd)/config/ $(pwd)/dataset/ $(pwd)/output/
remove mkl/2017.4 (LD_LIBRARY_PATH)
remove impi/2017.4 (PATH, MANPATH, LD_LIBRARY_PATH)
Set INTEL compilers as MPI wrappers backend
load impi/2017.4 (PATH, MANPATH, LD_LIBRARY_PATH)
load mkl/2017.4 (LD_LIBRARY_PATH)
load java/8u131 (PATH, MANPATH, JAVA_HOME, JAVA_ROOT, JAVA_BINDIR, SDK_HOME, JDK_HOME, JRE_HOME)
load papi/5.5.1 (PATH, LD_LIBRARY_PATH, C_INCLUDE_PATH)
Loading default Python 2.7.13.
* For alternative Python versions, please set the COMPSS_PYTHON_VERSION environment variable with 2, 3, 2-jupyter or 3-jupyter before loading the COMPSs module.
load PYTHON/2.7.13 (PATH, MANPATH, LD_LIBRARY_PATH, LIBRARY_PATH, PKG_CONFIG_PATH, C_INCLUDE_PATH, CPLUS_INCLUDE_PATH, PYTHONHOME)
load lzo/2.10 (LD_LIBRARY_PATH,PKG_CONFIG_PATH,CFLAGS,CXXFLAGS,LDFLAGS)
load boost/1.64.0_py2 (LD_LIBRARY_PATH, LIBRARY_PATH, C_INCLUDE_PATH, CPLUS_INCLUDE_PATH, BOOST_ROOT)
load COMPSs/2.7 (PATH, CLASSPATH, MANPATH, GAT_LOCATION, COMPSS_HOME, JAVA_TOOL_OPTIONS, LDFLAGS, CPPFLAGS)
load gromacs/2016.4 (PATH, LD_LIBRARY_PATH)
SC Configuration: default.cfg
JobName: COMPSs
Queue: default
Reservation: disabled
Num Nodes: 2
Num Switches: 0
GPUs per node: 0
Job dependency: None
Exec-Time: 00:10:00
QoS: debug
Constraints: disabled
Storage Home: null
Storage Properties:
Other:
--sc_cfg=default.cfg
--qos=debug
--master_working_dir=.
--worker_working_dir=/gpfs/home/user/lysozyme
--tracing=false
--graph=true
--classpath=/home/user/lysozyme/./src/
--pythonpath=/home/user/lysozyme/./src/
--lang=python /home/user/lysozyme/./src/lysozyme_in_water.py /home/user/lysozyme/config/ /home/user/lysozyme/dataset/ /home/user/lysozyme/output/
Temp submit script is: /scratch/tmp/tmp.sMHLsaTUJj
Requesting 96 processes
Submitted batch job 10178129
Once executed, it produces the compss-10178129.out file, containing all the
standard output messages flushed during the execution:
$ cat compss-10178129.out
------ Launching COMPSs application ------
[ INFO] Using default execution type: compss
[ INFO] Relative Classpath resolved: /home/user/lysozyme/./src/:
----------------- Executing lysozyme_in_water.py --------------------------
[(590) API] - Starting COMPSs Runtime v2.7 (build 20200519-1005.r6093e5ac94d67250e097a6fad9d3ec00d676fe6c)
Starting demo
# Here it takes some time to process the dataset
[(290788) API] - Execution Finished
------------------------------------------------------------
[LAUNCH_COMPSS] Waiting for application completion
Since the execution has been performed with the task dependency graph generation enabled, the result is depicted in Figure 49. It can be identified that PyCOMPSs has been able to analyse the three given proteins in parallel.
Python Lysozyme in Water tasks graph
The output of the application is a set of files within the output folder.
It can be seen that the files decorated with FILE_OUT are stored in this
folder. In particular, potential (.xvg) files represent the final results
of the application, which can be visualized with GRACE.
user@login:~/lysozyme/output> ls -l
total 79411
-rw-r--r-- 1 user group 8976 may 19 17:06 1aki_em_energy.edr
-rw-r--r-- 1 user group 1280044 may 19 17:03 1aki_em.tpr
-rw-r--r-- 1 user group 88246 may 19 17:03 1aki.gro
-rw-r--r-- 1 user group 1279304 may 19 17:03 1aki_ions.tpr
-rw-r--r-- 1 user group 88246 may 19 17:03 1aki_newbox.gro
-rw-r--r-- 1 user group 2141 may 19 17:06 1aki_potential.xvg <-------
-rw-r--r-- 1 user group 1525186 may 19 17:03 1aki_solv.gro
-rw-r--r-- 1 user group 1524475 may 19 17:03 1aki_solv_ions.gro
-rw-r--r-- 1 user group 577616 may 19 17:03 1aki.top
-rw-r--r-- 1 user group 577570 ene 24 16:11 #1aki.top.1#
-rw-r--r-- 1 user group 577601 may 19 16:59 #1aki.top.10#
-rw-r--r-- 1 user group 577570 may 19 17:03 #1aki.top.11#
-rw-r--r-- 1 user group 577601 may 19 17:03 #1aki.top.12#
-rw-r--r-- 1 user group 577601 ene 24 16:11 #1aki.top.2#
-rw-r--r-- 1 user group 577570 ene 24 16:20 #1aki.top.3#
-rw-r--r-- 1 user group 577601 ene 24 16:20 #1aki.top.4#
-rw-r--r-- 1 user group 577570 ene 24 16:25 #1aki.top.5#
-rw-r--r-- 1 user group 577601 ene 24 16:25 #1aki.top.6#
-rw-r--r-- 1 user group 577570 ene 24 16:31 #1aki.top.7#
-rw-r--r-- 1 user group 577601 ene 24 16:31 #1aki.top.8#
-rw-r--r-- 1 user group 577570 may 19 16:59 #1aki.top.9#
-rw-r--r-- 1 user group 8976 may 19 17:08 1u3m_em_energy.edr
-rw-r--r-- 1 user group 1416272 may 19 17:03 1u3m_em.tpr
-rw-r--r-- 1 user group 82046 may 19 17:03 1u3m.gro
-rw-r--r-- 1 user group 1415196 may 19 17:03 1u3m_ions.tpr
-rw-r--r-- 1 user group 82046 may 19 17:03 1u3m_newbox.gro
-rw-r--r-- 1 user group 2151 may 19 17:08 1u3m_potential.xvg <-------
-rw-r--r-- 1 user group 1837046 may 19 17:03 1u3m_solv.gro
-rw-r--r-- 1 user group 1836965 may 19 17:03 1u3m_solv_ions.gro
-rw-r--r-- 1 user group 537950 may 19 17:03 1u3m.top
-rw-r--r-- 1 user group 537904 ene 24 16:11 #1u3m.top.1#
-rw-r--r-- 1 user group 537935 may 19 16:59 #1u3m.top.10#
-rw-r--r-- 1 user group 537904 may 19 17:03 #1u3m.top.11#
-rw-r--r-- 1 user group 537935 may 19 17:03 #1u3m.top.12#
-rw-r--r-- 1 user group 537935 ene 24 16:11 #1u3m.top.2#
-rw-r--r-- 1 user group 537904 ene 24 16:20 #1u3m.top.3#
-rw-r--r-- 1 user group 537935 ene 24 16:20 #1u3m.top.4#
-rw-r--r-- 1 user group 537904 ene 24 16:25 #1u3m.top.5#
-rw-r--r-- 1 user group 537935 ene 24 16:25 #1u3m.top.6#
-rw-r--r-- 1 user group 537904 ene 24 16:31 #1u3m.top.7#
-rw-r--r-- 1 user group 537935 ene 24 16:31 #1u3m.top.8#
-rw-r--r-- 1 user group 537904 may 19 16:59 #1u3m.top.9#
-rw-r--r-- 1 user group 8780 may 19 17:08 1xyw_em_energy.edr
-rw-r--r-- 1 user group 1408872 may 19 17:03 1xyw_em.tpr
-rw-r--r-- 1 user group 80112 may 19 17:03 1xyw.gro
-rw-r--r-- 1 user group 1407844 may 19 17:03 1xyw_ions.tpr
-rw-r--r-- 1 user group 80112 may 19 17:03 1xyw_newbox.gro
-rw-r--r-- 1 user group 2141 may 19 17:08 1xyw_potential.xvg <-------
-rw-r--r-- 1 user group 1845237 may 19 17:03 1xyw_solv.gro
-rw-r--r-- 1 user group 1845066 may 19 17:03 1xyw_solv_ions.gro
-rw-r--r-- 1 user group 524026 may 19 17:03 1xyw.top
-rw-r--r-- 1 user group 523980 ene 24 16:11 #1xyw.top.1#
-rw-r--r-- 1 user group 524011 may 19 16:59 #1xyw.top.10#
-rw-r--r-- 1 user group 523980 may 19 17:03 #1xyw.top.11#
-rw-r--r-- 1 user group 524011 may 19 17:03 #1xyw.top.12#
-rw-r--r-- 1 user group 524011 ene 24 16:11 #1xyw.top.2#
-rw-r--r-- 1 user group 523980 ene 24 16:20 #1xyw.top.3#
-rw-r--r-- 1 user group 524011 ene 24 16:20 #1xyw.top.4#
-rw-r--r-- 1 user group 523980 ene 24 16:25 #1xyw.top.5#
-rw-r--r-- 1 user group 524011 ene 24 16:25 #1xyw.top.6#
-rw-r--r-- 1 user group 523980 ene 24 16:31 #1xyw.top.7#
-rw-r--r-- 1 user group 524011 ene 24 16:31 #1xyw.top.8#
-rw-r--r-- 1 user group 523980 may 19 16:59 #1xyw.top.9#
Figure 50 depicts the potential results obtained for the 1xyw protein.
1xyw Potential result (plotted with GRACE)
C/C++ Sample applications
The first two examples in this section are simple applications developed in COMPSs to easily illustrate how to code, compile and run COMPSs applications. These applications are executed locally and show different ways to take advantage of all the COMPSs features.
The rest of the examples are more elaborated and consider the execution in a cloud platform where the VMs mount a common storage on /sharedDisk directory. This is useful in the case of applications that require working with big files, allowing to transfer data only once, at the beginning of the execution, and to enable the application to access the data directly during the rest of the execution.
The Virtual Machine available at our webpage (http://compss.bsc.es/)
provides a development environment with all the applications listed in
the following sections. The codes of all the applications can be found
under the /home/compss/tutorial_apps/c/ folder.
Simple
The Simple application is a C application that increases a counter by means of a task. The counter is stored inside a file that is transfered to the worker when the task is executed. Thus, the tasks inferface is defined as follows:
// simple.idl
interface simple {
void increment(inout File filename);
};
Next we also provide the invocation of the task from the main code and the increment’s method code.
// simple.cc
int main(int argc, char *argv[]) {
// Check and get parameters
if (argc != 2) {
usage();
return -1;
}
string initialValue = argv[1];
file fileName = strdup(FILE_NAME);
// Init compss
compss_on();
// Write file
ofstream fos (fileName);
if (fos.is_open()) {
fos << initialValue << endl;
fos.close();
} else {
cerr << "[ERROR] Unable to open file" << endl;
return -1;
}
cout << "Initial counter value is " << initialValue << endl;
// Execute increment
increment(&fileName);
// Read new value
string finalValue;
ifstream fis;
compss_ifstream(fileName, fis);
if (fis.is_open()) {
if (getline(fis, finalValue)) {
cout << "Final counter value is " << finalValue << endl;
fis.close();
} else {
cerr << "[ERROR] Unable to read final value" << endl;
fis.close();
return -1;
}
} else {
cerr << "[ERROR] Unable to open file" << endl;
return -1;
}
// Close COMPSs and end
compss_off();
return 0;
}
//simple-functions.cc
void increment(file *fileName) {
cout << "INIT TASK" << endl;
cout << "Param: " << *fileName << endl;
// Read value
char initialValue;
ifstream fis (*fileName);
if (fis.is_open()) {
if (fis >> initialValue) {
fis.close();
} else {
cerr << "[ERROR] Unable to read final value" << endl;
fis.close();
}
fis.close();
} else {
cerr << "[ERROR] Unable to open file" << endl;
}
// Increment
cout << "INIT VALUE: " << initialValue << endl;
int finalValue = ((int)(initialValue) - (int)('0')) + 1;
cout << "FINAL VALUE: " << finalValue << endl;
// Write new value
ofstream fos (*fileName);
if (fos.is_open()) {
fos << finalValue << endl;
fos.close();
} else {
cerr << "[ERROR] Unable to open file" << endl;
}
cout << "END TASK" << endl;
}
Finally, to compile and execute this application users must run the following commands:
compss@bsc:~$ cd ~/tutorial_apps/c/simple/
compss@bsc:~/tutorial_apps/c/simple$ compss_build_app simple
compss@bsc:~/tutorial_apps/c/simple$ runcompss --lang=c --project=./xml/project.xml --resources=./xml/resources.xml ~/tutorial_apps/c/simple/master/simple 1
[ INFO] Using default execution type: compss
----------------- Executing simple --------------------------
JVM_OPTIONS_FILE: /tmp/tmp.n2eZjgmDGo
COMPSS_HOME: /opt/COMPSs
Args: 1
WARNING: COMPSs Properties file is null. Setting default values
[(617) API] - Starting COMPSs Runtime v<version>
Initial counter value is 1
[ BINDING] - @GS_register - Ref: 0x7fffa35d0f48
[ BINDING] - @GS_register - ENTRY ADDED
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: counter
[ BINDING] - @GS_register - setting filename: counter
[ BINDING] - @GS_register - Filename: counter
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @compss_wait_on - Entry.type: 9
[ BINDING] - @compss_wait_on - Entry.classname: File
[ BINDING] - @compss_wait_on - Entry.filename: counter
[ BINDING] - @compss_wait_on - Runtime filename: /home/compss/.COMPSs/simple_01/tmpFiles/d1v2_1479141705574.IT
[ BINDING] - @compss_wait_on - File renaming: /home/compss/.COMPSs/simple_01/tmpFiles/d1v2_1479141705574.IT to counter
Final counter value is 2
[(3755) API] - Execution Finished
------------------------------------------------------------
Increment
The Increment application is a C application that increases N times three different counters. Each increase step is developed by a separated task. The purpose of this application is to show parallelism between the three counters.
Next we provide the main code of this application. The code inside the increment task is the same than the previous example.
// increment.cc
int main(int argc, char *argv[]) {
// Check and get parameters
if (argc != 5) {
usage();
return -1;
}
int N = atoi( argv[1] );
string counter1 = argv[2];
string counter2 = argv[3];
string counter3 = argv[4];
// Init COMPSs
compss_on();
// Initialize counter files
file fileName1 = strdup(FILE_NAME1);
file fileName2 = strdup(FILE_NAME2);
file fileName3 = strdup(FILE_NAME3);
initializeCounters(counter1, counter2, counter3, fileName1, fileName2, fileName3);
// Print initial counters state
cout << "Initial counter values: " << endl;
printCounterValues(fileName1, fileName2, fileName3);
// Execute increment tasks
for (int i = 0; i < N; ++i) {
increment(&fileName1);
increment(&fileName2);
increment(&fileName3);
}
// Print final state
cout << "Final counter values: " << endl;
printCounterValues(fileName1, fileName2, fileName3);
// Stop COMPSs
compss_off();
return 0;
}
As shown in the main code, this application has 4 parameters that stand for:
- N: Number of times to increase a counter
- counter1: Initial value for counter 1
- counter2: Initial value for counter 2
- counter3: Initial value for counter 3
Next we will compile and run the Increment application with the -g option to be able to generate the final graph at the end of the execution.
compss@bsc:~$ cd ~/tutorial_apps/c/increment/
compss@bsc:~/tutorial_apps/c/increment$ compss_build_app increment
compss@bsc:~/tutorial_apps/c/increment$ runcompss --lang=c -g --project=./xml/project.xml --resources=./xml/resources.xml ~/tutorial_apps/c/increment/master/increment 10 1 2 3
[ INFO] Using default execution type: compss
----------------- Executing increment --------------------------
JVM_OPTIONS_FILE: /tmp/tmp.mgCheFd3kL
COMPSS_HOME: /opt/COMPSs
Args: 10 1 2 3
WARNING: COMPSs Properties file is null. Setting default values
[(655) API] - Starting COMPSs Runtime v<version>
Initial counter values:
- Counter1 value is 1
- Counter2 value is 2
- Counter3 value is 3
[ BINDING] - @GS_register - Ref: 0x7ffea17719f0
[ BINDING] - @GS_register - ENTRY ADDED
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file1.txt
[ BINDING] - @GS_register - setting filename: file1.txt
[ BINDING] - @GS_register - Filename: file1.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea17719f8
[ BINDING] - @GS_register - ENTRY ADDED
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file2.txt
[ BINDING] - @GS_register - setting filename: file2.txt
[ BINDING] - @GS_register - Filename: file2.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea1771a00
[ BINDING] - @GS_register - ENTRY ADDED
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file3.txt
[ BINDING] - @GS_register - setting filename: file3.txt
[ BINDING] - @GS_register - Filename: file3.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea17719f0
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file1.txt
[ BINDING] - @GS_register - setting filename: file1.txt
[ BINDING] - @GS_register - Filename: file1.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea17719f8
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file2.txt
[ BINDING] - @GS_register - setting filename: file2.txt
[ BINDING] - @GS_register - Filename: file2.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea1771a00
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file3.txt
[ BINDING] - @GS_register - setting filename: file3.txt
[ BINDING] - @GS_register - Filename: file3.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea17719f0
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file1.txt
[ BINDING] - @GS_register - setting filename: file1.txt
[ BINDING] - @GS_register - Filename: file1.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea17719f8
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file2.txt
[ BINDING] - @GS_register - setting filename: file2.txt
[ BINDING] - @GS_register - Filename: file2.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea1771a00
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file3.txt
[ BINDING] - @GS_register - setting filename: file3.txt
[ BINDING] - @GS_register - Filename: file3.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea17719f0
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file1.txt
[ BINDING] - @GS_register - setting filename: file1.txt
[ BINDING] - @GS_register - Filename: file1.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea17719f8
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file2.txt
[ BINDING] - @GS_register - setting filename: file2.txt
[ BINDING] - @GS_register - Filename: file2.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea1771a00
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file3.txt
[ BINDING] - @GS_register - setting filename: file3.txt
[ BINDING] - @GS_register - Filename: file3.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea17719f0
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file1.txt
[ BINDING] - @GS_register - setting filename: file1.txt
[ BINDING] - @GS_register - Filename: file1.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea17719f8
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file2.txt
[ BINDING] - @GS_register - setting filename: file2.txt
[ BINDING] - @GS_register - Filename: file2.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea1771a00
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file3.txt
[ BINDING] - @GS_register - setting filename: file3.txt
[ BINDING] - @GS_register - Filename: file3.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea17719f0
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file1.txt
[ BINDING] - @GS_register - setting filename: file1.txt
[ BINDING] - @GS_register - Filename: file1.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea17719f8
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file2.txt
[ BINDING] - @GS_register - setting filename: file2.txt
[ BINDING] - @GS_register - Filename: file2.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea1771a00
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file3.txt
[ BINDING] - @GS_register - setting filename: file3.txt
[ BINDING] - @GS_register - Filename: file3.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea17719f0
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file1.txt
[ BINDING] - @GS_register - setting filename: file1.txt
[ BINDING] - @GS_register - Filename: file1.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea17719f8
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file2.txt
[ BINDING] - @GS_register - setting filename: file2.txt
[ BINDING] - @GS_register - Filename: file2.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea1771a00
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file3.txt
[ BINDING] - @GS_register - setting filename: file3.txt
[ BINDING] - @GS_register - Filename: file3.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea17719f0
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file1.txt
[ BINDING] - @GS_register - setting filename: file1.txt
[ BINDING] - @GS_register - Filename: file1.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea17719f8
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file2.txt
[ BINDING] - @GS_register - setting filename: file2.txt
[ BINDING] - @GS_register - Filename: file2.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea1771a00
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file3.txt
[ BINDING] - @GS_register - setting filename: file3.txt
[ BINDING] - @GS_register - Filename: file3.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea17719f0
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file1.txt
[ BINDING] - @GS_register - setting filename: file1.txt
[ BINDING] - @GS_register - Filename: file1.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea17719f8
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file2.txt
[ BINDING] - @GS_register - setting filename: file2.txt
[ BINDING] - @GS_register - Filename: file2.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea1771a00
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file3.txt
[ BINDING] - @GS_register - setting filename: file3.txt
[ BINDING] - @GS_register - Filename: file3.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea17719f0
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file1.txt
[ BINDING] - @GS_register - setting filename: file1.txt
[ BINDING] - @GS_register - Filename: file1.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea17719f8
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file2.txt
[ BINDING] - @GS_register - setting filename: file2.txt
[ BINDING] - @GS_register - Filename: file2.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @GS_register - Ref: 0x7ffea1771a00
[ BINDING] - @GS_register - ENTRY FOUND
[ BINDING] - @GS_register - Entry.type: 9
[ BINDING] - @GS_register - Entry.classname: File
[ BINDING] - @GS_register - Entry.filename: file3.txt
[ BINDING] - @GS_register - setting filename: file3.txt
[ BINDING] - @GS_register - Filename: file3.txt
[ BINDING] - @GS_register - Result is 0
[ BINDING] - @compss_wait_on - Entry.type: 9
[ BINDING] - @compss_wait_on - Entry.classname: File
[ BINDING] - @compss_wait_on - Entry.filename: file1.txt
[ BINDING] - @compss_wait_on - Runtime filename: /home/compss/.COMPSs/increment_01/tmpFiles/d1v11_1479142004112.IT
[ BINDING] - @compss_wait_on - File renaming: /home/compss/.COMPSs/increment_01/tmpFiles/d1v11_1479142004112.IT to file1.txt
[ BINDING] - @compss_wait_on - Entry.type: 9
[ BINDING] - @compss_wait_on - Entry.classname: File
[ BINDING] - @compss_wait_on - Entry.filename: file2.txt
[ BINDING] - @compss_wait_on - Runtime filename: /home/compss/.COMPSs/increment_01/tmpFiles/d2v11_1479142004112.IT
[ BINDING] - @compss_wait_on - File renaming: /home/compss/.COMPSs/increment_01/tmpFiles/d2v11_1479142004112.IT to file2.txt
[ BINDING] - @compss_wait_on - Entry.type: 9
[ BINDING] - @compss_wait_on - Entry.classname: File
[ BINDING] - @compss_wait_on - Entry.filename: file3.txt
[ BINDING] - @compss_wait_on - Runtime filename: /home/compss/.COMPSs/increment_01/tmpFiles/d3v11_1479142004112.IT
[ BINDING] - @compss_wait_on - File renaming: /home/compss/.COMPSs/increment_01/tmpFiles/d3v11_1479142004112.IT to file3.txt
Final counter values:
- Counter1 value is 2
- Counter2 value is 3
- Counter3 value is 4
[(4288) API] - Execution Finished
------------------------------------------------------------
By running the compss_gengraph command users can obtain the task graph of the above execution. Next we provide the set of commands to obtain the graph show in Figure 51.
compss@bsc:~$ cd ~/.COMPSs/increment_01/monitor/
compss@bsc:~/.COMPSs/increment_01/monitor$ compss_gengraph complete_graph.dot
compss@bsc:~/.COMPSs/increment_01/monitor$ evince complete_graph.pdf
C increment tasks graph
PyCOMPSs Player
The PyCOMPSs player (pycompss-player) provides a tool to use PyCOMPSs within
local machines interactively through docker containers. This tool has been
implemented on top of the PyCOMPSs programming model,
and it is being developed by the Workflows and Distributed Computing
group of the Barcelona Supercomputing
Center, and can be easily downloaded and installed
from the Pypi repository.
Requirements and Installation
Installation
Install Docker (continue with step 2 if already installed):
1.1. Suggested Docker installation instructions:
- Docker for Mac. Or, if you prefer to use Homebrew.
- Docker for Ubuntu.
- Docker for Arch Linux.
Be aware that for some distributions the Docker package has been renamed from
dockertodocker-ce. Make sure you install the new package.1.2. Add user to docker group to run the containers as a non-root user:
1.3. Check that docker is correctly installed:
$ docker --version $ docker ps # this should be empty as no docker processes are yet running.Install docker for python (continue with step 3 if already installed):
$ python3 -m pip install docker
Install pycompss-player:
Since the PyCOMPSs playerpackage is available in Pypi, it can be easly installed with
pipas follows:$ python3 -m pip install pycompss-playerCheck the pycompss-player installation:
In order to check that it is correctly installed, check that the pycompss-player executables (
pycompss,compssanddislib, which can be used indiferently) are available from your command line.$ pycompss [PyCOMPSs player options will be shown]
Tip
Some Linux distributions do not include the
$HOME/.local/binfolder in thePATHenvironment variable, preventing to access to thepycompss-playercommands (and any other Python packages installed in the user HOME).If you experience that the
pycompss|compss|dislibcommand is not available after the installation, you may need to include the following line into your.bashrcand execute it in your current session:$ export PATH=${HOME}/.local/bin:${PATH}
Usage
pycompss-player provides the pycompss command line tool (compss
and dislib are also alternatives to pycompss).
This command line tool enables to deal with docker in order to deploy a COMPSs infrastructure in containers.
The supported flags are:
$ pycompss
PyCOMPSs|COMPSS Player:
Usage: pycompss COMMAND | compss COMMAND | dislib COMMAND
Available commands:
init -w [WORK_DIR] -i [IMAGE]: initializes COMPSs in the current working dir or in WORK_DIR if -w is set.
The COMPSs docker image to be used can be specified with -i (it can also be
specified with the COMPSS_DOCKER_IMAGE environment variable).
kill: stops and kills all instances of the COMPSs.
update: updates the COMPSs docker image (use only when installing master branch).
exec CMD: executes the CMD command inside the COMPSs master container.
run [OPTIONS] FILE [PARAMS]: runs FILE with COMPSs, where OPTIONS are COMPSs options and PARAMS are application parameters.
monitor [start|stop]: starts or stops the COMPSs monitoring.
jupyter [PATH|FILE]: starts jupyter-notebook in the given PATH or FILE.
gengraph [FILE.dot]: converts the .dot graph into .pdf
components list: lists COMPSs actives components.
components add RESOURCE: adds the RESOURCE to the pool of workers of the COMPSs.
Example given: pycompss components add worker 2 # to add 2 local workers.
Example given: pycompss components add worker <IP>:<CORES> # to add a remote worker
Note: compss and dislib can be used instead of pycompss in both examples.
components remove RESOURCE: removes the RESOURCE to the pool of workers of the COMPSs.
Example given: pycompss components remove worker 2 # to remove 2 local workers.
Example given: pycompss components remove worker <IP>:<CORES> # to remove a remote worker
Note: compss and dislib can be used instead of pycompss in both examples.
Start COMPSs infrastructure in your development directory
Initialize the COMPSs infrastructure where your source code will be (you can re-init anytime). This will allow docker to access your local code and run it inside the container.
$ pycompss init # operates on the current directory as working directoyr.
Note
The first time needs to download the docker image from the repository, and it may take a while.
Alternatively, you can specify the working directory, the COMPSs docker image to use, or both at the same time:
$ # You can also provide a path
$ pycompss init -w /home/user/replace/path/
$
$ # Or the COMPSs docker image to use
$ pycompss init -i compss/compss-tutorial:2.7
$
$ # Or both
$ pycompss init -w /home/user/replace/path/ -i compss/compss-tutorial:2.7
Running applications
In order to show how to run an application, clone the PyCOMPSs’ tutorial apps repository:
$ git clone https://github.com/bsc-wdc/tutorial_apps.git
Init the COMPSs environment in the root of the repository. The source
files path are resolved from the init directory which sometimes can be
confusing. As a rule of thumb, initialize the library in a current
directory and check the paths are correct running the file with
python3 path_to/file.py (in this case
python3 python/simple/src/simple.py).
$ cd tutorial_apps
$ pycompss init
Now we can run the simple.py application:
$ pycompss run python/simple/src/simple.py 1
The log files of the execution can be found at $HOME/.COMPSs.
You can also init the COMPSs environment inside the examples folder. This will mount the examples directory inside the container so you can execute it without adding the path:
$ cd python/simple/src
$ pycompss init
$ pycompss run simple.py 1
Running the COMPSs monitor
The COMPSs monitor can be started using the pycompss monitor start
command. This will start the COMPSs monitoring facility which enables to
check the application status while running. Once started, it will show
the url to open the monitor in your web browser
(i.e. http://127.0.0.1:8080/compss-monitor)
Important
Include the --monitor=<REFRESH_RATE_MS> flag in the execution before
the binary to be executed.
$ cd python/simple/src
$ pycompss init
$ pycompss monitor start
$ pycompss run --monitor=1000 -g simple.py 1
$ # During the execution, go to the URL in your web browser
$ pycompss monitor stop
If running a notebook, just add the monitoring parameter into the COMPSs runtime start call.
Once finished, it is possible to stop the monitoring facility by using
the pycompss monitor stop command.
Running Jupyter notebooks
Notebooks can be run using the pycompss jupyter command. Run the
following snippet from the root of the project:
$ cd tutorial_apps/python
$ pycompss init
$ pycompss jupyter ./notebooks
An alternative and more flexible way of starting jupyter is using the
pycompss run command in the following way:
$ pycompss run jupyter-notebook ./notebooks --ip=0.0.0.0 --NotebookApp.token='' --allow-root
And access interactively to your notebook by opening following the http://127.0.0.1:8888/ URL in your web browser.
Caution
If the notebook process is not properly closed, you might get the following warning when trying to start jupyter notebooks again:
The port 8888 is already in use, trying another port.
To fix it, just restart the container with pycompss init.
Generating the task graph
COMPSs is able to produce the task graph showing the dependencies that
have been respected. In order to producee it, include the --graph flag in
the execution command:
$ cd python/simple/src
$ pycompss init
$ pycompss run --graph simple.py 1
Once the application finishes, the graph will be stored into the
~\.COMPSs\app_name_XX\monitor\complete_graph.dot file. This dot file
can be converted to pdf for easier visualilzation through the use of the
gengraph parameter:
$ pycompss gengraph .COMPSs/simple.py_01/monitor/complete_graph.dot
The resulting pdf file will be stored into the
~\.COMPSs\app_name_XX\monitor\complete_graph.pdf file, that is, the
same folder where the dot file is.
Tracing applications or notebooks
COMPSs is able to produce tracing profiles of the application execution
through the use of EXTRAE. In order to enable it, include the --tracing
flag in the execution command:
$ cd python/simple/src
$ pycompss init
$ pycompss run --tracing simple.py 1
If running a notebook, just add the tracing parameter into the COMPSs runtime start call.
Once the application finishes, the trace will be stored into the
~\.COMPSs\app_name_XX\trace folder. It can then be analysed with
Paraver.
Adding more nodes
Note
Adding more nodes is still in beta phase. Please report issues, suggestions, or feature requests on Github.
To add more computing nodes, you can either let docker create more workers for you or manually create and config a custom node.
For docker just issue the desired number of workers to be added. For example, to add 2 docker workers:
$ pycompss components add worker 2
You can check that both new computing nodes are up with:
$ pycompss components list
If you want to add a custom node it needs to be reachable through ssh
without user. Moreover, pycompss will try to copy the working_dir
there, so it needs write permissions for the scp.
For example, to add the local machine as a worker node:
$ pycompss components add worker '127.0.0.1:6'
- ‘127.0.0.1’: is the IP used for ssh (can also be a hostname like ‘localhost’ as long as it can be resolved).
- ‘6’: desired number of available computing units for the new node.
Important
Please be aware** that pycompss components will not list your
custom nodes because they are not docker processes and thus it can’t be
verified if they are up and running.
Removing existing nodes
Note
Removing nodes is still in beta phase. Please report issues, suggestions, or feature requests on Github.
For docker just issue the desired number of workers to be removed. For example, to remove 2 docker workers:
$ pycompss components remove worker 2
You can check that the workers have been removed with:
$ pycompss components list
If you want to remove a custom node, you just need to specify its IP and number of computing units used when defined.
$ pycompss components remove worker '127.0.0.1:6'
Stop pycompss
The infrastructure deployed can be easily stopped and the docker instances closed with the following command:
$ pycompss kill
PyCOMPSs Notebooks
This section contains all PyCOMPSs related tutorial notebooks (sources available in https://github.com/bsc-wdc/notebooks).
It is divided into three main folders:
- Syntax: Contains the main tutorial notebooks. They cover the syntax and main functionalities of PyCOMPSs.
- Hands-On: Contains example applications and hands-on exercises.
- Demos: Contains demonstration notebooks.
Syntax
Here you will find the syntax notebooks used in the tutorials.
Basics of programming with PyCOMPSs
In this example we will see basics of programming with PyCOMPSs: - Runtime start - Task definition - Task invocation - Runtime stop
Let’s get started with a simple example
First step
- Import the PyCOMPSs library
[1]:
import pycompss.interactive as ipycompss
Second step
- Initialize COMPSs runtime. Parameters indicates if the execution will generate task graph, tracefile, monitor interval and debug information.
[2]:
import os
if 'BINDER_SERVICE_HOST' in os.environ:
ipycompss.start(graph=True,
project_xml='../xml/project.xml',
resources_xml='../xml/resources.xml')
else:
ipycompss.start(graph=True, monitor=1000) # debug=True, trace=True
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
* .-~~-.--. _____ ________ *
* : ) |____ \ |____ / *
* .~ ~ -.\ /.- ~~ . ___) | / / *
* > `. .' < / ___/ / / *
* ( .- -. ) | |___ _ / / *
* `- -.-~ `- -' ~-.- -' |_____| |_| /__/ *
* ( : ) _ _ .-: *
* ~--. : .--~ .-~ .-~ } *
* ~-.-^-.-~ \_ .~ .-~ .~ *
* \ \ ' \ '_ _ -~ *
* \`.\`. // *
* . - ~ ~-.__\`.\`-.// *
* .-~ . - ~ }~ ~ ~-.~-. *
* .' .-~ .-~ :/~-.~-./: *
* /_~_ _ . - ~ ~-.~-._ *
* ~-.< *
******************************************************
* - Starting COMPSs runtime... *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun! *
******************************************************
Third step
- Import task module before annotating functions or methods
[3]:
from pycompss.api.task import task
Fourth step
- Declare functions and decorate with @task those that should be tasks
[4]:
@task(returns=int)
def square(val1):
return val1 * val1
Found task: square
[5]:
@task(returns=int)
def add(val2, val3):
return val2 + val3
Found task: add
[6]:
@task(returns=int)
def multiply(val1, val2):
return val1 * val2
Found task: multiply
Fifth step
- Invoke tasks
[7]:
a = square(2)
[8]:
b = add(a, 4)
[9]:
c = multiply(b, 5)
Sixth step (last)
- Stop COMPSs runtime. All data can be synchronized in the main program .
[10]:
ipycompss.stop(sync=True)
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Synchronizing all future objects left on the user scope.
Found a future object: b
Found a future object: a
Found a future object: c
****************************************************
[11]:
print("Results after stopping PyCOMPSs: ")
print("a: %d" % a)
print("b: %d" % b)
print("c: %d" % c)
Results after stopping PyCOMPSs:
a: 4
b: 8
c: 40
PyCOMPSs: Synchronization
In this example we will see how to synchronize with PyCOMPSs.
Import the PyCOMPSs library
[1]:
import pycompss.interactive as ipycompss
Start the runtime
Initialize COMPSs runtime Parameters indicates if the execution will generate task graph, tracefile, monitor interval and debug information.
[2]:
import os
if 'BINDER_SERVICE_HOST' in os.environ:
ipycompss.start(graph=True, debug=False,
project_xml='../xml/project.xml',
resources_xml='../xml/resources.xml')
else:
ipycompss.start(graph=True, monitor=1000, trace=False)
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
* .-~~-.--. _____ ________ *
* : ) |____ \ |____ / *
* .~ ~ -.\ /.- ~~ . ___) | / / *
* > `. .' < / ___/ / / *
* ( .- -. ) | |___ _ / / *
* `- -.-~ `- -' ~-.- -' |_____| |_| /__/ *
* ( : ) _ _ .-: *
* ~--. : .--~ .-~ .-~ } *
* ~-.-^-.-~ \_ .~ .-~ .~ *
* \ \ ' \ '_ _ -~ *
* \`.\`. // *
* . - ~ ~-.__\`.\`-.// *
* .-~ . - ~ }~ ~ ~-.~-. *
* .' .-~ .-~ :/~-.~-./: *
* /_~_ _ . - ~ ~-.~-._ *
* ~-.< *
******************************************************
* - Starting COMPSs runtime... *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun! *
******************************************************
Importing task and parameter modules
Import task module before annotating functions or methods
[3]:
from pycompss.api.task import task
from pycompss.api.parameter import *
from pycompss.api.api import compss_wait_on
Declaring tasks
Declare functions and decorate with @task those that should be tasks
[4]:
@task(returns=int)
def square(val1):
return val1 * val1
Found task: square
[5]:
@task(returns=int)
def add(val2, val3):
return val2 + val3
Found task: add
[6]:
@task(returns=int)
def multiply(val1, val2):
return val1 * val2
Found task: multiply
Invoking tasks
[7]:
a = square(2)
[8]:
b = add(a, 4)
[9]:
c = multiply (b, 5)
Accessing data outside tasks requires synchronization
[10]:
c = compss_wait_on(c)
[11]:
c = c + 1
[12]:
print("a: %s" % a)
print("b: %s" % b)
print("c: %d" % c)
a: <pycompss.runtime.binding.Future object at 0x7f8024052250>
b: <pycompss.runtime.binding.Future object at 0x7f808cf77250>
c: 41
[13]:
a = compss_wait_on(a)
[14]:
print("a: %d" % a)
a: 4
Stop the runtime
[15]:
ipycompss.stop(sync=True)
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Synchronizing all future objects left on the user scope.
Found a future object: b
****************************************************
[16]:
print("Results after stopping PyCOMPSs: ")
print("a: %d" % a)
print("b: %d" % b)
print("c: %d" % c)
Results after stopping PyCOMPSs:
a: 4
b: 8
c: 41
PyCOMPSs: Using objects, lists, and synchronization
In this example we will see how classes and objects can be used from PyCOMPSs, and that class methods can become tasks.
Import the PyCOMPSs library
[1]:
import pycompss.interactive as ipycompss
Start the runtime
Initialize COMPSs runtime Parameters indicates if the execution will generate task graph, tracefile, monitor interval and debug information.
[2]:
import os
if 'BINDER_SERVICE_HOST' in os.environ:
ipycompss.start(graph=True, debug=True,
project_xml='../xml/project.xml',
resources_xml='../xml/resources.xml')
else:
ipycompss.start(graph=True, monitor=1000, debug=True)
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
* .-~~-.--. _____ ________ *
* : ) |____ \ |____ / *
* .~ ~ -.\ /.- ~~ . ___) | / / *
* > `. .' < / ___/ / / *
* ( .- -. ) | |___ _ / / *
* `- -.-~ `- -' ~-.- -' |_____| |_| /__/ *
* ( : ) _ _ .-: *
* ~--. : .--~ .-~ .-~ } *
* ~-.-^-.-~ \_ .~ .-~ .~ *
* \ \ ' \ '_ _ -~ *
* \`.\`. // *
* . - ~ ~-.__\`.\`-.// *
* .-~ . - ~ }~ ~ ~-.~-. *
* .' .-~ .-~ :/~-.~-./: *
* /_~_ _ . - ~ ~-.~-._ *
* ~-.< *
******************************************************
* - Starting COMPSs runtime... *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun! *
******************************************************
Importing task and arguments directionality modules
Import task module before annotating functions or methods
[3]:
from pycompss.api.api import compss_barrier
from pycompss.api.api import compss_wait_on
Declaring a class
[4]:
%%writefile my_shaper.py
from pycompss.api.task import task
from pycompss.api.parameter import IN
class Shape(object):
def __init__(self,x,y):
self.x = x
self.y = y
@task(returns=int)
def area(self):
return self.x * self.y
@task(returns=int)
def perimeter(self):
return 2 * self.x + 2 * self.y
def describe(self,text):
self.description = text
@task()
def scaleSize(self,scale):
self.x = self.x * scale
self.y = self.y * scale
@task(target_direction=IN)
def infoShape(self):
print('Shape x=', self.x, 'y= ', self.y)
Writing my_shaper.py
Invoking tasks
[5]:
from my_shaper import Shape
[6]:
my_shapes = []
my_shapes.append(Shape(100,45))
my_shapes.append(Shape(50,50))
[7]:
all_areas = []
[8]:
for this_shape in my_shapes:
all_areas.append(this_shape.area())
[9]:
# Need it if we want to synchonize nested objects
all_areas = compss_wait_on(all_areas)
print(all_areas)
[4500, 2500]
[10]:
rectangle = Shape(200,25)
rectangle.scaleSize(5)
area_rectangle = rectangle.area()
rectangle = compss_wait_on(rectangle)
print('X = %d' % rectangle.x)
area_rectangle = compss_wait_on(area_rectangle)
print('Area = %d' % area_rectangle)
X = 1000
Area = 125000
[11]:
all_perimeters=[]
my_shapes.append(rectangle)
for this_shape in my_shapes:
this_shape.infoShape()
all_perimeters.append(this_shape.perimeter())
[12]:
all_perimeters = compss_wait_on(all_perimeters)
print(all_perimeters)
[290, 200, 2250]
Stop the runtime
[13]:
ipycompss.stop(sync=True)
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Synchronizing all future objects left on the user scope.
Found an object to synchronize: this_shape
****************************************************
PyCOMPSs: Using objects, lists, and synchronization
In this example we will see how classes and objects can be used from PyCOMPSs, and that class methods can become tasks.
Import the PyCOMPSs library
[1]:
import pycompss.interactive as ipycompss
Start the runtime
Initialize COMPSs runtime Parameters indicates if the execution will generate task graph, tracefile, monitor interval and debug information.
[2]:
import os
if 'BINDER_SERVICE_HOST' in os.environ:
ipycompss.start(graph=True, debug=True,
project_xml='../xml/project.xml',
resources_xml='../xml/resources.xml')
else:
ipycompss.start(graph=True, monitor=1000, debug=True, trace=False)
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
* .-~~-.--. _____ ________ *
* : ) |____ \ |____ / *
* .~ ~ -.\ /.- ~~ . ___) | / / *
* > `. .' < / ___/ / / *
* ( .- -. ) | |___ _ / / *
* `- -.-~ `- -' ~-.- -' |_____| |_| /__/ *
* ( : ) _ _ .-: *
* ~--. : .--~ .-~ .-~ } *
* ~-.-^-.-~ \_ .~ .-~ .~ *
* \ \ ' \ '_ _ -~ *
* \`.\`. // *
* . - ~ ~-.__\`.\`-.// *
* .-~ . - ~ }~ ~ ~-.~-. *
* .' .-~ .-~ :/~-.~-./: *
* /_~_ _ . - ~ ~-.~-._ *
* ~-.< *
******************************************************
* - Starting COMPSs runtime... *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun! *
******************************************************
Importing task and arguments directionality modules
Import task module before annotating functions or methods
[3]:
from pycompss.api.api import compss_barrier
from pycompss.api.api import compss_wait_on
from pycompss.api.task import task
Declaring a class
[4]:
%%writefile my_shaper.py
from pycompss.api.task import task
from pycompss.api.parameter import IN
class Shape(object):
def __init__(self,x,y):
self.x = x
self.y = y
description = "This shape has not been described yet"
@task(returns=int)
def area(self):
return self.x * self.y
@task(returns=int)
def perimeter(self):
return 2 * self.x + 2 * self.y
def describe(self,text):
self.description = text
@task()
def scaleSize(self,scale):
self.x = self.x * scale
self.y = self.y * scale
@task(target_direction=IN)
def infoShape(self):
print('Shape x=', self.x, 'y= ', self.y)
Overwriting my_shaper.py
[5]:
@task(returns=int)
def addAll(*mylist):
sum = 0
for ll in mylist:
sum = sum + ll
return sum
Task definition detected.
Found task: addAll
Invoking tasks
[6]:
from my_shaper import Shape
[7]:
my_shapes = []
my_shapes.append(Shape(100,45))
my_shapes.append(Shape(50,50))
my_shapes.append(Shape(10,100))
my_shapes.append(Shape(20,30))
[8]:
all_areas = []
[9]:
for this_shape in my_shapes:
all_areas.append(this_shape.area())
[10]:
# Need it if we want to synchonize nested objects
all_areas = compss_wait_on(all_areas)
print(all_areas)
[4500, 2500, 1000, 600]
[11]:
rectangle = Shape(200,25)
rectangle.scaleSize(5)
area_rectangle = rectangle.area()
rectangle = compss_wait_on(rectangle)
print('X = %d' % rectangle.x)
area_rectangle = compss_wait_on(area_rectangle)
print('Area = %d' % area_rectangle)
X = 1000
Area = 125000
[12]:
all_perimeters=[]
my_shapes.append(rectangle)
for this_shape in my_shapes:
this_shape.infoShape()
all_perimeters.append(this_shape.perimeter())
[13]:
# all_perimeters = compss_wait_on(all_perimeters)
# print all_perimeters
[14]:
mysum = addAll(*all_perimeters)
mysum = compss_wait_on(mysum)
print(mysum)
3060
Stop the runtime
[15]:
ipycompss.stop(sync=True)
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Synchronizing all future objects left on the user scope.
Found an object to synchronize: this_shape
****************************************************
PyCOMPSs: Using objects, lists, and synchronization. Using collections.
In this example we will see how classes and objects can be used from PyCOMPSs, and that class methods can become tasks. The example also illustrates the use of collections
Import the PyCOMPSs library
[1]:
import pycompss.interactive as ipycompss
Start the runtime
Initialize COMPSs runtime Parameters indicates if the execution will generate task graph, tracefile, monitor interval and debug information.
[2]:
import os
if 'BINDER_SERVICE_HOST' in os.environ:
ipycompss.start(graph=True, debug=True,
project_xml='../xml/project.xml',
resources_xml='../xml/resources.xml')
else:
ipycompss.start(graph=True, monitor=1000, debug=True, trace=False)
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
* .-~~-.--. _____ ________ *
* : ) |____ \ |____ / *
* .~ ~ -.\ /.- ~~ . ___) | / / *
* > `. .' < / ___/ / / *
* ( .- -. ) | |___ _ / / *
* `- -.-~ `- -' ~-.- -' |_____| |_| /__/ *
* ( : ) _ _ .-: *
* ~--. : .--~ .-~ .-~ } *
* ~-.-^-.-~ \_ .~ .-~ .~ *
* \ \ ' \ '_ _ -~ *
* \`.\`. // *
* . - ~ ~-.__\`.\`-.// *
* .-~ . - ~ }~ ~ ~-.~-. *
* .' .-~ .-~ :/~-.~-./: *
* /_~_ _ . - ~ ~-.~-._ *
* ~-.< *
******************************************************
* - Starting COMPSs runtime... *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun! *
******************************************************
Importing task and arguments directionality modules
Import task module before annotating functions or methods
[3]:
from pycompss.api.api import compss_barrier
from pycompss.api.api import compss_wait_on
from pycompss.api.task import task
from pycompss.api.parameter import *
Declaring a class
[4]:
%%writefile my_shaper.py
from pycompss.api.task import task
from pycompss.api.parameter import IN
class Shape(object):
def __init__(self,x,y):
self.x = x
self.y = y
description = "This shape has not been described yet"
@task(returns=int, target_direction=IN)
def area(self):
import time
time.sleep(4)
return self.x * self.y
@task()
def scaleSize(self,scale):
import time
time.sleep(4)
self.x = self.x * scale
self.y = self.y * scale
@task(returns=int, target_direction=IN)
def perimeter(self):
import time
time.sleep(4)
return 2 * self.x + 2 * self.y
def describe(self,text):
self.description = text
@task(target_direction=IN)
def infoShape(self):
import time
time.sleep(1)
print('Shape x=', self.x, 'y= ', self.y)
Overwriting my_shaper.py
[5]:
#Operations with collections: previous to release 2.5
@task(returns=1)
def addAll(*mylist):
import time
time.sleep(1)
sum = 0
for ll in mylist:
sum = sum + ll
return sum
Task definition detected.
Found task: addAll
[6]:
@task(returns=int, mylist = COLLECTION_IN)
def addAll_C(mylist):
import time
time.sleep(4)
sum = 0
for ll in mylist:
sum = sum + ll
return sum
Task definition detected.
Found task: addAll_C
[7]:
@task(returns=2, mylist = COLLECTION_IN, my_otherlist = COLLECTION_IN)
def addAll_C2(mylist, my_otherlist):
import time
time.sleep(4)
sum = 0
sum2 = 0
for ll in mylist:
sum = sum + ll
for jj in my_otherlist:
sum2 = sum2 + jj
return sum, sum2
Task definition detected.
Found task: addAll_C2
[8]:
@task (mylist = COLLECTION_INOUT)
def scale_all (mylist, scale):
import time
time.sleep(4)
for ll in mylist:
ll.x = ll.x * scale
ll.y = ll.y * scale
Task definition detected.
Found task: scale_all
Invoking tasks
[9]:
from my_shaper import Shape
[10]:
my_shapes = []
my_shapes.append(Shape(100,45))
my_shapes.append(Shape(50,50))
my_shapes.append(Shape(10,100))
my_shapes.append(Shape(20,30))
[11]:
all_areas = []
[12]:
for this_shape in my_shapes:
all_areas.append(this_shape.area())
Synchronizing results from tasks
[13]:
all_areas = compss_wait_on(all_areas)
print(all_areas)
[4500, 2500, 1000, 600]
[14]:
rectangle = Shape(200,25)
rectangle.scaleSize(5)
area_rectangle = rectangle.area()
rectangle = compss_wait_on(rectangle)
print('X =', rectangle.x)
area_rectangle = compss_wait_on(area_rectangle)
print('Area =', area_rectangle)
('X =', 1000)
('Area =', 125000)
Accessing data in collections
[15]:
all_perimeters=[]
my_shapes.append(rectangle)
for this_shape in my_shapes:
all_perimeters.append(this_shape.perimeter())
[16]:
mysum = addAll_C(all_perimeters)
mysum = compss_wait_on(mysum)
print(mysum)
3060
[17]:
# Previous version without collections
# mysum = addAll(*all_perimeters)
# mysum = compss_wait_on(mysum)
# print(mysum)
Accessing two collections
[18]:
all_perimeters=[]
all_areas=[]
for this_shape in my_shapes:
all_perimeters.append(this_shape.perimeter())
all_areas.append(this_shape.area())
[19]:
[my_per, my_area] = addAll_C2(all_perimeters, all_areas)
[my_per, my_area] = compss_wait_on([my_per, my_area])
print([my_per, my_area])
[3060, 133600]
Scattering data from a collection
[20]:
scale_all(my_shapes,2)
scaled_areas=[]
for this_shape in my_shapes:
scaled_areas.append(this_shape.area())
scaled_areas = compss_wait_on(scaled_areas)
print(scaled_areas)
[18000, 10000, 4000, 2400, 500000]
Stop the runtime
[21]:
ipycompss.stop(sync=True)
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Synchronizing all future objects left on the user scope.
Found an object to synchronize: this_shape
****************************************************
PyCOMPSs: Using objects, lists, and synchronization. Managing fault-tolerance.
In this example we will see how classes and objects can be used from PyCOMPSs, and that class methods can become tasks. The example also illustrates the current fault-tolerance management provided by the runtime.
Import the PyCOMPSs library
[1]:
import pycompss.interactive as ipycompss
Start the runtime
Initialize COMPSs runtime Parameters indicates if the execution will generate task graph, tracefile, monitor interval and debug information.
[2]:
import os
if 'BINDER_SERVICE_HOST' in os.environ:
ipycompss.start(graph=True, debug=False,
project_xml='../xml/project.xml',
resources_xml='../xml/resources.xml')
else:
ipycompss.start(graph=True, monitor=1000, trace=False, debug=False)
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
* .-~~-.--. _____ ________ *
* : ) |____ \ |____ / *
* .~ ~ -.\ /.- ~~ . ___) | / / *
* > `. .' < / ___/ / / *
* ( .- -. ) | |___ _ / / *
* `- -.-~ `- -' ~-.- -' |_____| |_| /__/ *
* ( : ) _ _ .-: *
* ~--. : .--~ .-~ .-~ } *
* ~-.-^-.-~ \_ .~ .-~ .~ *
* \ \ ' \ '_ _ -~ *
* \`.\`. // *
* . - ~ ~-.__\`.\`-.// *
* .-~ . - ~ }~ ~ ~-.~-. *
* .' .-~ .-~ :/~-.~-./: *
* /_~_ _ . - ~ ~-.~-._ *
* ~-.< *
******************************************************
* - Starting COMPSs runtime... *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun! *
******************************************************
Importing task and arguments directionality modules
Import task module before annotating functions or methods
[3]:
from pycompss.api.api import compss_barrier
from pycompss.api.api import compss_wait_on
from pycompss.api.task import task
from pycompss.api.parameter import *
Declaring a class
[4]:
%%writefile my_shaper.py
from pycompss.api.task import task
from pycompss.api.parameter import IN
import sys
class Shape(object):
def __init__(self,x,y):
self.x = x
self.y = y
description = "This shape has not been described yet"
@task(returns=int, target_direction=IN)
def area(self):
return self.x * self.y
@task()
def scaleSize(self,scale):
self.x = self.x * scale
self.y = self.y * scale
# on_failure= 'IGNORE', on_failure= 'RETRY', on_failure= 'FAIL', 'CANCEL_SUCCESSORS'
@task(on_failure= 'CANCEL_SUCCESSORS')
def downScale(self,scale):
if (scale <= 0):
sys.exit(1)
else:
self.x = self.x/scale
self.y = self.y/scale
@task(returns=int, target_direction=IN)
def perimeter(self):
return 2 * self.x + 2 * self.y
def describe(self,text):
self.description = text
@task(target_direction=IN)
def infoShape(self):
print('Shape x=', self.x, 'y= ', self.y)
Overwriting my_shaper.py
Invoking tasks
[5]:
from my_shaper import Shape
[6]:
my_shapes = []
my_shapes.append(Shape(100,45))
my_shapes.append(Shape(50,50))
my_shapes.append(Shape(10,100))
my_shapes.append(Shape(20,30))
my_shapes.append(Shape(200,25))
[7]:
all_perimeters = []
[8]:
i=4
for this_shape in my_shapes:
this_shape.scaleSize(2)
this_shape.area()
i = i - 1
this_shape.downScale(i)
all_perimeters.append(this_shape.perimeter())
Synchronizing results from tasks
[9]:
# all_perimeters = compss_wait_on(all_perimeters)
# print all_perimeters
Stop the runtime
[10]:
ipycompss.stop(sync=False)
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Warning: some of the variables used with PyCOMPSs may
have not been brought to the master.
****************************************************
PyCOMPSs: Using files
In this example we will how files can be used with PyCOMPSs.
Import the PyCOMPSs library
[1]:
import pycompss.interactive as ipycompss
Start the runtime
Initialize COMPSs runtime Parameters indicates if the execution will generate task graph, tracefile, monitor interval and debug information.
[2]:
import os
if 'BINDER_SERVICE_HOST' in os.environ:
ipycompss.start(graph=True, debug=False,
project_xml='../xml/project.xml',
resources_xml='../xml/resources.xml')
else:
ipycompss.start(graph=True, monitor=1000, trace=False, debug=False)
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
* .-~~-.--. _____ ________ *
* : ) |____ \ |____ / *
* .~ ~ -.\ /.- ~~ . ___) | / / *
* > `. .' < / ___/ / / *
* ( .- -. ) | |___ _ / / *
* `- -.-~ `- -' ~-.- -' |_____| |_| /__/ *
* ( : ) _ _ .-: *
* ~--. : .--~ .-~ .-~ } *
* ~-.-^-.-~ \_ .~ .-~ .~ *
* \ \ ' \ '_ _ -~ *
* \`.\`. // *
* . - ~ ~-.__\`.\`-.// *
* .-~ . - ~ }~ ~ ~-.~-. *
* .' .-~ .-~ :/~-.~-./: *
* /_~_ _ . - ~ ~-.~-._ *
* ~-.< *
******************************************************
* - Starting COMPSs runtime... *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun! *
******************************************************
Importing task and parameter modules
Import task module before annotating functions or methods
[3]:
from pycompss.api.task import task
from pycompss.api.parameter import FILE_IN, FILE_OUT, FILE_INOUT
from pycompss.api.api import compss_wait_on, compss_open
Declaring tasks
Declare functions and decorate with @task those that should be tasks
[4]:
@task(fout=FILE_OUT)
def write(fout, content):
with open(fout, 'w') as fout_d:
fout_d.write(content)
Found task: write
[5]:
@task(finout=FILE_INOUT)
def append(finout):
finout_d = open(finout, 'a')
finout_d.write("\n===> INOUT FILE ADDED CONTENT")
finout_d.close()
Found task: append
[6]:
@task(fin=FILE_IN, returns=str)
def readFile(fin):
fin_d = open(fin, 'r')
content = fin_d.read()
fin_d.close()
return content
Found task: readFile
Invoking tasks
[7]:
f = "myFile.txt"
content = "OUT FILE CONTENT"
write(f, content)
[8]:
append(f)
[9]:
readed = readFile(f)
[10]:
append(f)
Accessing data outside tasks requires synchronization
[11]:
readed = compss_wait_on(readed)
print(readed)
OUT FILE CONTENT
===> INOUT FILE ADDED CONTENT
[12]:
with compss_open(f) as fd:
f_content = fd.read()
print(f_content)
OUT FILE CONTENT
===> INOUT FILE ADDED CONTENT
===> INOUT FILE ADDED CONTENT
Stop the runtime
[13]:
ipycompss.stop(sync=True)
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Synchronizing all future objects left on the user scope.
****************************************************
PyCOMPSs: Using constraints
In this example we will how to define task constraints with PyCOMPSs.
Import the PyCOMPSs library
[1]:
import pycompss.interactive as ipycompss
Starting runtime
Initialize COMPSs runtime Parameters indicates if the execution will generate task graph, tracefile, monitor interval and debug information.
[2]:
import os
if 'BINDER_SERVICE_HOST' in os.environ:
ipycompss.start(graph=True, debug=False,
project_xml='../xml/project.xml',
resources_xml='../xml/resources.xml')
else:
ipycompss.start(graph=True, monitor=1000, trace=True, debug=False)
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
* .-~~-.--. _____ ________ *
* : ) |____ \ |____ / *
* .~ ~ -.\ /.- ~~ . ___) | / / *
* > `. .' < / ___/ / / *
* ( .- -. ) | |___ _ / / *
* `- -.-~ `- -' ~-.- -' |_____| |_| /__/ *
* ( : ) _ _ .-: *
* ~--. : .--~ .-~ .-~ } *
* ~-.-^-.-~ \_ .~ .-~ .~ *
* \ \ ' \ '_ _ -~ *
* \`.\`. // *
* . - ~ ~-.__\`.\`-.// *
* .-~ . - ~ }~ ~ ~-.~-. *
* .' .-~ .-~ :/~-.~-./: *
* /_~_ _ . - ~ ~-.~-._ *
* ~-.< *
******************************************************
* - Starting COMPSs runtime... *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun! *
******************************************************
Importing task and arguments directionality modules
Import task module before annotating functions or methods
[3]:
from pycompss.api.task import task
from pycompss.api.parameter import *
from pycompss.api.api import compss_barrier
from pycompss.api.constraint import constraint
from pycompss.api.implement import implement
Declaring tasks
Declare functions and decorate with @task those that should be tasks
[4]:
@constraint(computing_units="2")
@task(returns=int)
def square(val1):
return val1 * val1
Found task: square
[5]:
@constraint(computing_units="1")
@task(returns=int)
def add(val2, val3):
return val2 + val3
Found task: add
[6]:
@constraint(computing_units="4")
@task(returns=int)
def multiply(val1, val2):
return val1 * val2
Found task: multiply
Invoking tasks
[7]:
for i in range(20):
r1 = square(i)
r2 = add(r1,i)
r3 = multiply(r2,r1)
compss_barrier()
Stop the runtime
[8]:
ipycompss.stop(sync=True)
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Synchronizing all future objects left on the user scope.
Found a future object: r2
Found a future object: r1
Found a future object: r3
****************************************************
[9]:
print(r1)
print(r2)
print(r3)
361
380
137180
PyCOMPSs: Polymorphism
In this example we will how to use polimorphism with PyCOMPSs.
Import the PyCOMPSs library
[1]:
import pycompss.interactive as ipycompss
Start the runtime
Initialize COMPSs runtime Parameters indicates if the execution will generate task graph, tracefile, monitor interval and debug information.
[2]:
import os
if 'BINDER_SERVICE_HOST' in os.environ:
ipycompss.start(graph=True, debug=False,
project_xml='../xml/project.xml',
resources_xml='../xml/resources.xml')
else:
ipycompss.start(graph=True, monitor=1000, trace=False, debug=False)
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
* .-~~-.--. _____ ________ *
* : ) |____ \ |____ / *
* .~ ~ -.\ /.- ~~ . ___) | / / *
* > `. .' < / ___/ / / *
* ( .- -. ) | |___ _ / / *
* `- -.-~ `- -' ~-.- -' |_____| |_| /__/ *
* ( : ) _ _ .-: *
* ~--. : .--~ .-~ .-~ } *
* ~-.-^-.-~ \_ .~ .-~ .~ *
* \ \ ' \ '_ _ -~ *
* \`.\`. // *
* . - ~ ~-.__\`.\`-.// *
* .-~ . - ~ }~ ~ ~-.~-. *
* .' .-~ .-~ :/~-.~-./: *
* /_~_ _ . - ~ ~-.~-._ *
* ~-.< *
******************************************************
* - Starting COMPSs runtime... *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun! *
******************************************************
Create a file to define the tasks
Importing task, implement and constraint modules
[3]:
%%writefile module.py
from pycompss.api.task import task
from pycompss.api.implement import implement
from pycompss.api.constraint import constraint
Writing module.py
Declaring tasks into the file
Declare functions and decorate with @task those that should be tasks
[4]:
%%writefile -a module.py
@constraint(computing_units='1')
@task(returns=list)
def addtwovectors(list1, list2):
for i in range(len(list1)):
list1[i] += list2[i]
return list1
Appending to module.py
[5]:
%%writefile -a module.py
@implement(source_class="module", method="addtwovectors")
@constraint(computing_units='4')
@task(returns=list)
def addtwovectorsWithNumpy(list1, list2):
import numpy as np
x = np.array(list1)
y = np.array(list2)
z = x + y
return z.tolist()
Appending to module.py
Invoking tasks
[6]:
from pycompss.api.api import compss_wait_on
from module import addtwovectors # Just import and use addtwovectors
from random import random
vectors = 100
vector_length = 5000
vectors_a = [[random() for i in range(vector_length)] for i in range(vectors)]
vectors_b = [[random() for i in range(vector_length)] for i in range(vectors)]
results = []
for i in range(vectors):
results.append(addtwovectors(vectors_a[i], vectors_b[i]))
Accessing data outside tasks requires synchronization
[7]:
results = compss_wait_on(results)
print(len(results))
print(results[0])
100
[1.768494493198583, 1.4586562536056813, 0.8441866404028181, 1.1063441132937109, 1.1084718629145844, 0.36955190772124735, 1.3811422763120502, 1.1286614323406154, 1.0576102402034677, 0.9782033218632993, 1.408191857769348, 0.6878254104159688, 0.323112039940443, 1.5801901552786781, 1.2231248827059296, 0.5857981618762707, 1.0258930359824174, 1.0840330717552462, 0.6093654435241589, 1.5234961280184112, 0.7634916148073189, 1.2911846095807626, 1.8687434856043383, 1.6012726320749655, 0.8747566516215126, 1.2196619362318097, 1.878545507107222, 1.913525006295211, 0.48610333625670155, 1.0927450762471345, 1.2362929217842575, 0.4093005628468286, 0.4743460568465909, 0.9806087966180191, 0.8642825028671067, 1.4041312783626183, 0.8541342586718479, 1.7525260317695208, 0.8982230917162601, 0.5265680075012115, 1.1358962368942915, 1.1139560539897726, 0.7911933525841417, 1.174682557524768, 0.6091128408012625, 1.280194521748916, 0.6565847703634651, 1.4729487319426782, 0.9965572873249281, 0.6419527716084441, 0.19654489469950565, 1.0700826765023335, 1.1119201353555914, 1.0588381624814782, 1.4844048130921885, 0.5844203973407232, 1.4597098684888143, 0.8335863590139848, 1.3184848712467079, 0.4487638195525635, 0.97893438004842, 1.10756741318523, 0.48119172567265445, 0.5653797923114816, 1.2927301706463423, 1.4062799909285872, 0.44369855653967627, 0.9866881621823611, 1.5249073821201597, 0.918086321105754, 1.1800939728721471, 0.5738682324020625, 0.9671415166985388, 0.8555830866024451, 0.7137929010586325, 0.6805326316444796, 1.677594473648915, 1.6834949385394187, 1.2194489886735747, 1.3101260460164308, 1.4400731131549787, 1.0303974584615334, 1.1444417510810594, 1.4314410896090504, 0.9560224725888148, 1.503725992200886, 0.2614883568961929, 0.5545905131871809, 0.45534555015876776, 1.04491335283876, 0.488249581091047, 1.404369554485935, 1.24661015269761, 0.7264359706270782, 0.8908760963818223, 1.1986969799026066, 1.3950171776599145, 0.9160319070919636, 1.5322010638408385, 1.1196529913114428, 0.8315852973652229, 0.9503388943216038, 1.8964992587393281, 0.7850167454422428, 1.3259242695031788, 0.40220037075610315, 1.3924546998563425, 0.6886235131357672, 0.5899244355602115, 1.2446777331626109, 0.5244248023685747, 1.0314343366020662, 0.6909862142001905, 0.7924380445198032, 1.0236215014543548, 0.8959701916924075, 0.8171457342668484, 0.875637284001392, 0.7479194317073946, 0.7047263223765342, 0.5460108646241774, 0.755436872859533, 0.7265736608215391, 0.5499028157023066, 1.1195939746911108, 1.1893117410581504, 0.6313013705721447, 0.9605994129959913, 1.3212292403252084, 0.866889925273197, 1.2315186809546628, 0.9930412886968937, 1.668119289959577, 0.5799391205717043, 0.7842340860310806, 1.795799208934422, 0.24186913849413016, 0.4761457240575484, 1.565821106978258, 0.9295625639637609, 0.6146090430721882, 0.49337717963674554, 0.9073658646675467, 1.344174665421093, 0.3691047179676791, 0.7337944219309906, 0.6214934293608398, 0.8782525184164499, 1.0355242609449402, 1.3217032114226908, 1.29985090281781, 0.7291267185123564, 0.5636633241360172, 1.0920459501119324, 1.2415799254919249, 0.7848951395770315, 1.0330664886446217, 1.0947179429099334, 1.1926204699023213, 0.4128910279172324, 0.5191174564117699, 1.1913230062525377, 0.8936184036519684, 1.4881650909519462, 1.425523721685146, 0.8699251369462485, 1.4542967442863441, 1.135566822918407, 1.248198073674873, 0.25793843810959793, 0.7607965606599789, 1.4384544300218591, 1.6351813542882319, 0.8874977531636492, 1.186612289537385, 1.2897607229101595, 1.2018612149786556, 0.16797665928720273, 1.0411393610389235, 1.6390243428960134, 1.0648095901159982, 1.4090456265001037, 1.2038335732118566, 0.4331858839322419, 1.6620659148580472, 1.2548051400460987, 1.398492233129768, 0.8488682197737033, 1.2477528741016028, 1.9093024376029981, 1.483248827670741, 1.0506909698281683, 1.1109332012577924, 1.1411238784565079, 0.8765915715539236, 1.2488255410733162, 1.6118274795667946, 0.8451633878217952, 1.026907742387124, 1.1834276724987156, 1.5154526179028964, 1.4683746927199677, 1.0312823451929913, 1.0876673680419597, 1.0058122506777833, 0.36100742553485976, 1.039819015664698, 1.0333573775393465, 0.969768248540223, 0.7685417140385872, 1.4930685365914447, 0.7067083462205153, 1.0675506269359178, 1.9348964428609439, 0.712304591341247, 1.0750394233635931, 1.834113193511942, 1.7821029869158322, 1.1757917294393785, 1.9418144836319073, 1.5407664976241904, 0.8195608789445906, 1.131059371091499, 1.202621102619241, 0.908865774869891, 1.1430024555594442, 0.9696385034486777, 0.8713543394232806, 0.34994998984840864, 1.039978218764225, 0.9572277610314784, 0.6100815379139216, 1.042142078249348, 0.9809074382963212, 0.835806983505511, 1.1006578149481498, 0.5536715431770683, 1.454711880339551, 0.5668701408442717, 1.2893069065507126, 0.9088954159240594, 0.940774710882133, 0.40190883206567096, 1.2610974246360924, 1.4342705036696597, 1.478812301081842, 1.292199368424147, 1.3295762714780053, 1.3858446030185956, 0.9692474112985321, 1.4014919321377857, 1.4566697058180949, 1.0474005952462653, 0.8845488233613712, 0.24189096918721786, 0.46222770435497096, 1.5631266941440778, 0.9659679998093386, 0.6971531952034133, 0.9258147099378691, 0.7601624870286469, 0.39530046722894463, 0.6190113614003636, 0.6395501284550484, 1.2592567640608183, 0.7227962618651551, 1.45004583618653, 0.6088460127065299, 0.972420323121303, 1.272378355827812, 0.9424736215222161, 1.4100238468941468, 1.4494312526912294, 0.5799454679116338, 1.4359807801948816, 0.6974077201270353, 1.0287928747847555, 0.9845455301775531, 1.1158213698147779, 1.2694789006030651, 1.0708765163945357, 0.8854840836908394, 1.4527699784405788, 1.1215706273348012, 1.2495209928438362, 0.34367247430281145, 1.354194725082281, 0.8970197886823629, 1.4175066989192002, 0.34129478167813676, 1.611479953058864, 0.9146322131349749, 1.420050163491061, 0.6989547772855847, 0.16133378948322985, 1.747701708024148, 1.3368371162466566, 1.1365206439795414, 1.4784110019648176, 0.572182723666451, 1.181681750878285, 0.5489474234722523, 0.9040062439236236, 1.2511193298207468, 0.8690258443051432, 1.610135285987098, 1.551774949291286, 1.000735433275163, 1.3269374147852249, 1.6078062348443614, 0.9866550145018862, 1.08627918078205, 0.8276315410754872, 1.0438365141498018, 1.1200870493620707, 0.5712311957238856, 1.0838355433763298, 0.9934941497948127, 0.3968125514347408, 0.9472719189141305, 0.5059794634770473, 1.4922335390132846, 0.7542762055011092, 1.0795449697150172, 1.176050356689129, 0.6650858173909182, 1.435390261804801, 1.2787046701771678, 0.44297762103300686, 1.0092730652736677, 0.9445837755995842, 0.7635215302814147, 1.2136142428346819, 1.242358058729108, 0.9439804686723962, 1.0123389077541896, 1.4107321369144603, 1.3645599095547951, 1.7494553836624855, 1.4472340109125115, 0.44875774315088324, 1.3136831557483823, 0.7036173875604419, 0.383320573990366, 1.360840846451214, 0.9719995008525365, 1.1938820608079779, 0.3175613203104144, 0.8265170517141646, 0.6890749859804391, 0.4303258516944005, 0.7284183028378056, 1.0967476561452782, 1.8422910281436518, 1.3941965513547003, 0.9819931109845836, 0.9218375859036672, 1.556932544989066, 1.2528041725239583, 0.8192279764844977, 1.4165423547798486, 1.0585773034826427, 0.8437320625688423, 1.2875648465276022, 1.4017581924703157, 1.0248964132230887, 0.6458625002104273, 1.8305098398398776, 1.0098496096097338, 0.5353791244271978, 0.8752106033158263, 0.7299823977756268, 0.996078353204495, 1.156970165163533, 0.6527142895323373, 1.7855857398741346, 0.2959949648575364, 0.686157332476223, 1.1298798901505807, 0.7252292318588848, 1.199844343093066, 0.7265254897405955, 1.3879489519253205, 1.3129527561014829, 0.18535984000340822, 0.7957600984278232, 1.549504403878913, 1.5653166966925567, 1.0727501055296327, 1.2157562772672916, 0.8132286288010384, 0.31297616278122176, 1.6723550400176799, 0.6380788928490759, 1.2840139210571542, 1.3711784131344191, 0.7865316569429049, 1.0821911113356482, 1.0178210921263278, 0.6296233590285634, 1.2877199586355659, 0.9939187849484572, 0.8970430810609669, 0.7936728689498298, 0.7859922700512979, 0.8998285912552847, 1.3203049742073616, 0.822036705232024, 1.4022196619861194, 0.28360253099399824, 0.7823982715359704, 0.7698763487499012, 1.0446078779006234, 0.9565606094496266, 1.646443074465688, 0.5671249275864304, 1.120596241944057, 0.4681651273690933, 1.2108477683230818, 1.1339394553840962, 1.0441262864898384, 0.797713559443618, 0.660142359656358, 1.472243939368796, 0.4060613038610543, 0.7939588002961834, 1.7060413495090252, 0.8209428277676825, 1.627872191923926, 0.16649126390600721, 0.8675235190702518, 1.141010671344945, 0.9340206542540544, 0.6496268984621149, 0.5903008750826126, 1.3821140965803806, 0.884445543386215, 1.4466634056481804, 0.1578885460539302, 1.3366192751401718, 0.8735636358037295, 1.4966970448814587, 0.5351676633259758, 0.3212837897060473, 1.1723207976157026, 0.5860076358984346, 0.9308134896152942, 0.9951220810667493, 0.44076275632126005, 0.6954103653990655, 1.4831918321861002, 1.1592326362253114, 0.7038640393769118, 0.8195478524086252, 0.28485133900725224, 0.8374999909817752, 0.1768815278532888, 0.8738903166737051, 0.5919980347306723, 0.8295879877015947, 0.9383758957229613, 1.2782518830403504, 0.9897342109792259, 0.7728286360257718, 0.9286510849044157, 1.0382095798663724, 1.2990508942194907, 0.9892425003413093, 1.1036629324898046, 1.2405231681852948, 1.2676577381032188, 1.6982694650333197, 1.2513196246283922, 0.6501643619669323, 0.8865915450279394, 0.7104595104005436, 0.2286338220002868, 0.5470262318861608, 0.716412560913958, 1.215426165749165, 0.7176699469366594, 1.3525195031026525, 1.5296464407001702, 1.3199954739391595, 1.2428963618942737, 1.2238861114895347, 1.0259216667059678, 0.41830531277650507, 0.9644581314998436, 0.9944740293834983, 1.3523562265875186, 1.0466571469195942, 1.5629906986994304, 1.3918412145772197, 0.776883846565769, 1.456343701814947, 0.8563866330597505, 1.6796475005212455, 0.829222352763344, 0.5731002173769754, 1.1332259079461522, 1.435019296826314, 0.9232288902777912, 0.838843936896777, 0.8900624064810624, 1.3179985443693134, 1.0602726839252794, 0.46828453761046995, 0.5261138717251594, 1.2725593115045721, 1.1225355332307858, 0.42223955897098564, 0.8940600392592631, 1.0168629744029234, 1.1645937440382923, 0.4721470230113417, 0.6302835886257002, 1.007874474913054, 1.008499747141938, 0.5705654796156403, 0.3685052007189048, 1.327277991387205, 1.862492096524665, 1.6032963425741418, 1.4061526017881771, 0.7532191173895123, 1.1080918442070704, 0.45408536297034896, 0.6059666543930446, 1.0779700972005082, 0.5817994717424685, 1.8807534706139606, 0.8010877310376636, 1.0174998220074079, 1.6920104007515429, 0.921019394260788, 1.5016819852281462, 1.6119561914020135, 1.2524909603360983, 1.4057099477826926, 0.7947549901478483, 1.2040823491423427, 0.30951741834940405, 0.8210830519198113, 0.9836649594375706, 0.904349484476273, 0.18758720410942853, 1.7260032800444773, 0.41309935611258153, 1.3746027881191056, 1.2318710259581014, 0.7980364522563083, 0.8216364792694228, 1.5169180135844953, 1.366222128809646, 0.8325146182494279, 0.7420911641496685, 1.426513183534817, 1.924157282562841, 0.7975461649302497, 0.3449693949340905, 1.0052661729883554, 1.031418903117765, 1.0487663101633211, 0.6525707382750344, 0.9336683926493612, 1.3390023110733118, 1.3669061813197563, 0.9114024773803097, 1.7843121142525975, 0.5686278280452893, 1.304444526083994, 1.2842813144419816, 1.0758675039756724, 0.46870986228658174, 0.9717367626076451, 1.24167358952479, 0.16302767830205245, 1.5024598338708204, 0.39111723357952655, 0.8628173896008107, 0.8934461810886372, 0.9863400235339546, 0.9869293308556617, 0.8386599438949605, 0.6296878177236168, 0.6870496884738766, 1.045781901831355, 0.9764567945722977, 1.470379926126817, 0.9638639546595481, 0.8880335283906554, 1.0096910464511528, 1.3484545072575584, 1.4661559041329952, 1.105073322256939, 0.7900484197634258, 1.0129986432872549, 1.3111888894528534, 0.6782148371430667, 1.0103424657155884, 0.31423849010868266, 0.75536652356199, 1.1454676075307166, 1.2607146191385254, 0.23931057085839735, 1.1507726677600458, 1.0870405124079252, 1.1239496588417155, 0.7963054919309651, 1.6663306487298153, 1.031485475612301, 0.5510833937863979, 1.294630992307504, 0.689763600142018, 0.8752397447394447, 0.7832435471610051, 1.6453073028598344, 1.255817321341058, 0.7538095016321364, 0.749421672977623, 1.3006048783556026, 0.8226512204511924, 1.2436506831244905, 1.1005413231478638, 1.2143930090714827, 1.147800682738124, 1.1361805318100486, 1.2289930811460765, 0.25228189607862306, 1.5205276279943725, 0.35550954882332386, 1.0417612891846988, 0.6626442875464231, 0.09191234067277265, 0.9188578615218588, 0.10507564337199127, 1.1383391455650176, 0.8826369130875734, 1.6578226850722348, 0.6256000797221637, 1.543484135150321, 0.8061764943512865, 1.3199033266566267, 0.6661182102457497, 0.7665139255689863, 0.4344425164144622, 0.5647169706347072, 0.7998423501377879, 0.5041835766161807, 0.782421656780592, 0.78450402885067, 1.5919082246851624, 0.5417460174957371, 0.7532401814807993, 0.8465038156987259, 1.1758136761630158, 0.9663699477218938, 0.5047966374201045, 0.47162348824258915, 1.2721212771584551, 1.4727943703065374, 1.1912422159559668, 1.2337833947540093, 1.0926760432784801, 1.102220288726997, 0.48440276737553534, 0.5143877748229285, 1.3116821244068975, 1.6491180892448007, 1.4161688797029446, 0.4006588617641057, 1.0814755001536285, 0.8277309471372819, 1.2199927023397428, 1.1459475138218564, 1.2389417452490292, 1.7513694863549918, 1.5001409901178855, 0.8754763710651617, 0.9122135232409017, 1.109620249334679, 1.3060692325549423, 0.3140559227838309, 1.1256907868001218, 0.9940014318858219, 0.8435891771248071, 0.5382688611019979, 0.7120795102128182, 0.8378467399735293, 0.515606268523143, 0.7767996263905469, 0.9688314102270158, 1.2619378046744323, 0.9390307366245285, 0.7826360777033352, 1.249236173184756, 0.3964395225842319, 1.379056216573193, 1.1484280255544483, 1.7534597444770856, 1.121353352429069, 1.2513690987330697, 1.181871429030354, 1.502606542651236, 0.4619415890915257, 0.9474367883179393, 1.2973910576545307, 1.1239762162100178, 0.9853445079352103, 0.7161379352860066, 0.8744891602272802, 0.40205026998916593, 1.4757938743239438, 1.219769585508846, 1.1090289317233433, 0.8171109801443068, 0.977750934725106, 1.2946001243378533, 1.2626660934352194, 0.9155156456186587, 1.1300867793076967, 1.155514364374685, 1.177093603735611, 0.737184464734749, 0.6624258885774911, 0.9058609441541425, 1.2572636358532439, 1.0425645143464242, 1.6595887061561116, 0.7654123939038345, 1.1185713649261757, 1.8809914197694035, 1.0496520648691694, 1.289204097109439, 1.290471948910123, 1.033747784496232, 0.24807769433890925, 0.5306737617092917, 1.0694652260380555, 0.9823974828036454, 1.4039391307626512, 1.0737547310907567, 0.7347453512742181, 1.8281694084381186, 1.3926154686395784, 1.486313464495483, 0.8698429513446206, 0.7092728562503665, 0.7042130466751653, 0.9657847221096125, 0.9171892029878967, 1.0505362108189273, 1.1208670610814722, 1.4251636004200332, 0.6271043237867686, 1.1596572935902867, 1.0086792454509197, 0.4840660562782456, 0.7945542552890362, 0.9341422468617658, 1.8605434929512494, 1.1085556617087131, 0.4512468204990939, 0.6923699971169759, 1.328989360563431, 1.6004683653311031, 0.9017544033224089, 0.8531113938808719, 0.906234236723886, 1.3683540244384642, 0.9043129138667212, 1.4694205648958494, 0.7259632464602851, 1.0022372119438838, 1.4308888784074005, 1.0985112845356948, 0.37050414201278226, 0.4803046535772517, 1.6244872295455826, 0.30959387172174957, 1.1042785204900798, 1.027124647583738, 1.6492461399601641, 1.72591923146006, 1.3785231220596807, 0.4751908437627701, 0.7296612900144354, 0.9608234631035105, 0.7719993513616887, 1.377126082451167, 1.244308968753207, 0.955701846401525, 0.8913131515147242, 1.5489167306980067, 0.6122887539247265, 0.6250478786040892, 0.6732080473391652, 0.9072806955433451, 1.6122953414547643, 0.5826259045867865, 0.868329757132542, 1.4551212271192342, 0.7909390966221237, 0.5375900680200089, 1.2181059701431751, 1.3012162764367317, 0.9308664605547485, 0.5737034730982483, 1.2528671448061452, 1.3104300651519676, 0.4106496893520345, 0.49875459296053426, 0.6789489437374194, 0.9834942856731064, 0.47277636376177834, 0.9014574785280268, 0.2077337735357515, 0.9747183941688597, 0.8326594059869902, 0.8249917507176062, 0.45135184481097534, 1.042880363858059, 0.6280084243185405, 1.3181795772050338, 0.49147834104483856, 1.1344240629176836, 1.2413452901001838, 0.21694109536218986, 1.1498005834583214, 1.1651288768563848, 0.6760169567053346, 1.2745787835826752, 0.3229031287199714, 0.9913169758691758, 0.6029575087380389, 1.6154543144384585, 1.259068732286455, 0.9480184934567168, 0.24210547771472712, 0.9410459595404336, 0.7822359067882023, 0.40897909973364244, 0.5909672159381808, 1.0489906331812815, 1.0658842536936373, 0.9449392288066814, 1.531847495525152, 0.9723242322464029, 1.469296135459006, 1.3335742782476272, 0.6275800620173049, 1.2779856603262605, 0.3475489180005725, 1.43379960242125, 0.3977423943891557, 1.5193046588681813, 0.9708516570630248, 1.1884060808802963, 0.5133087628403022, 0.8184774988082608, 1.4932356890935856, 1.2781309170621669, 1.1376183510914197, 0.6258871762465653, 1.5042741445637913, 0.8575508180763759, 0.9837272493856498, 0.860627423737973, 0.9608579477820758, 1.6612266018503852, 0.1782995754893184, 1.3034686825391604, 0.6325988266040427, 1.1031131600301283, 1.1510815180766725, 1.3725763690045896, 0.07691021695286282, 1.6511444845210992, 0.5397699237024784, 0.47018197081026025, 1.1405569393941373, 0.7632911779477061, 0.6311283411281733, 0.6406593249514473, 1.5996068472667717, 0.9736772484637146, 0.8871506279145207, 0.7099010294898114, 0.9325845402354381, 1.2362090329057298, 1.1933313891229047, 0.05992720837539689, 0.2794535710558479, 1.3838062713930568, 0.6230164787683429, 0.6782910560465697, 0.9057201567181941, 1.569113554406605, 0.17259491472446253, 0.4549855398621152, 0.5970582495096387, 1.577319693311758, 0.7091804369064295, 0.3057275459877761, 0.6558749179539991, 1.03048822370617, 1.1259671928255697, 0.5740313698508842, 0.42666455194942543, 0.35523950725372055, 0.9740724797718542, 1.6327775134243514, 1.555130774415519, 0.8537097143708994, 0.8447367876045919, 1.1804746190096673, 1.2713000474856535, 1.556322421177879, 0.33688076113077814, 1.684064888391545, 0.5135982462934731, 1.417052648316755, 0.3363560749689293, 1.0300968682861311, 0.9348613529229653, 1.8137823637124464, 1.1941383430771761, 0.3249934780430922, 1.1602233737380399, 0.5107185576935268, 0.6115019765669417, 1.175986395601488, 1.622888548056904, 1.3981227543327157, 0.5832756827767779, 1.0338478885122155, 0.9975444226166675, 0.12614972318980489, 0.5697629111302531, 0.7152660630929162, 1.7555983355065043, 1.1034616954333205, 1.0115044147258878, 1.3511993991624658, 1.0981153604939062, 1.4877987038341733, 0.8330118754106653, 1.0238476400042047, 1.0746983669870485, 0.19253315685686512, 0.28560311378706793, 0.6490481311514377, 1.1616365476175718, 0.7299354875961351, 1.3308025801211354, 1.4423633227196198, 1.0630815595787135, 1.6953214365688483, 1.002923588407266, 1.0987312053547944, 1.5043398640591108, 1.0101392785500565, 0.9432275570053495, 1.3456748585152905, 1.2138144612627966, 0.6427727448280647, 0.2606802615796491, 0.7892030727538875, 1.768874618481092, 1.2597618834594138, 0.9004681019018959, 0.8995256142455288, 1.7700386031412552, 0.2927295349368598, 0.830390816939656, 0.38682058247914985, 1.2032626918588416, 0.7510336184439953, 1.5481546517339022, 1.6302550269281384, 0.8228931971389335, 0.4633957626896038, 1.6526545213476571, 0.9187144556154772, 1.1058992997801464, 1.1506890127321256, 1.137251936198722, 0.9566255984104047, 0.8071911558833526, 1.0117164988676746, 0.9320099443589764, 0.9202840291026586, 1.6905814683361988, 0.17616291798097128, 0.7662169893632058, 1.7849874676883255, 0.652733758700786, 0.9564947114798017, 1.3705461436544217, 1.1716154673200636, 0.7161697007517867, 1.1221713075519593, 1.7153451728209483, 1.0065861152767925, 1.5654572454850988, 0.9841366736922016, 0.5308096986067521, 1.782493462118829, 0.6562191890481827, 0.8397916444427901, 1.1418403489202165, 0.6273868141937411, 0.7377696279323082, 0.5615795601480378, 1.823805914483728, 1.0735799547558127, 0.8543723857851248, 0.6957663234890977, 1.19474193070418, 1.455830903220761, 1.2688307259511553, 1.0739385417201872, 1.050501869266904, 1.0789886875969605, 0.5241604882358809, 1.7582767677679045, 0.6640360519691778, 0.6179684237900752, 1.0037009360298208, 1.81984974453882, 1.4880917460012517, 0.44133693318441325, 0.7073857629110224, 1.4916276122564172, 1.4776693122437141, 1.1888932497541655, 1.8036566189502983, 1.3367697855346257, 1.0189466965479035, 0.837721358231347, 1.2148510449805316, 0.6887827767076115, 1.0146658930972863, 0.8256459161565963, 0.777891101836066, 0.9240181994425823, 1.1041011952814586, 0.885078962252689, 1.0748373304938812, 0.9845164021849389, 0.44675821286818185, 0.546162569468852, 0.8782130539023775, 1.0593752006459853, 1.538184751752738, 0.9005288638033602, 1.2437887786275188, 1.2966878135791968, 1.534290151220374, 1.0252660862994738, 0.84834796780886, 1.15320285149228, 1.3906966948438506, 1.2385885288331941, 0.7996791254196338, 1.352287328567721, 1.279418342071557, 1.0458743392450793, 0.9265465045244152, 0.40965572336103206, 1.4326549535665611, 1.648849892730992, 0.9381967744070876, 1.288723327807444, 1.1730520995579772, 0.9265701607437347, 1.4494891331604058, 1.0106254790264733, 0.6108744892110486, 0.275390018513442, 1.7370478482610066, 0.15466057529531219, 0.5920732804753053, 0.971669086300956, 1.5049070501821433, 0.6624807564682954, 1.3982887793681387, 1.2392080173156712, 0.15466997273463967, 0.6554456201365081, 0.8985923774428568, 1.0505946067928686, 1.3597654263205434, 1.7747676391749878, 0.8420919060996049, 0.623765693643185, 0.9451039915803163, 1.129087571027175, 1.5605467620440303, 0.8966152987120655, 0.8536096760635951, 1.155933447759831, 0.8845891968233774, 1.390342693845258, 1.2840314558210553, 0.81260219170999, 0.714889225779514, 0.9709774653661153, 1.4161040412443868, 1.6152461864890348, 0.5654988551326118, 1.1019993886751944, 0.8987821519667298, 1.1288880446574532, 1.5257792704866804, 0.9456418019386215, 0.5233355700159757, 0.602463823287987, 0.9181089900444337, 1.210672870336007, 1.8326368369952624, 1.4940455847914502, 0.9287732228123642, 0.44810097732009235, 0.9178205610330513, 1.2427760115993278, 0.6278756323442631, 1.7446012751724707, 1.3535318251150263, 1.77898816404032, 1.074565473185369, 0.49310316666318854, 1.4238805169766575, 0.5545105917002441, 0.7313340030886639, 0.3797152980878772, 0.5216649031029206, 1.032908619599966, 1.2087128567985417, 0.8007887283471601, 0.4368915264643535, 1.3489880996955899, 0.2572404893553123, 1.8460347679397766, 0.8512499912625072, 1.5074368060887742, 1.4771033390979476, 1.367600522527527, 1.1158265970187113, 1.3439287369204487, 1.4561174433437207, 1.171418011882569, 0.4503020519134754, 0.39654060008539227, 1.1129794575609684, 0.8574783587797372, 1.2489970086668112, 0.761974274061903, 1.3243189766628323, 0.48706290265941665, 0.5079385842703972, 0.682516543162964, 0.4834348958501896, 1.0464473252142805, 0.6009771555685143, 1.04349107912747, 0.8591233311611232, 1.1451991389407707, 1.079168465630702, 0.6830066624743809, 1.1436793616178687, 0.8353786162693965, 1.3920439749401505, 1.1930127467110414, 1.0661396787052797, 1.36817072725611, 0.6651048270946313, 0.7740396389656052, 0.6876512250679214, 0.5208327005939345, 0.9038549072567181, 1.1906992266969287, 0.7670896343470667, 0.9125227862282026, 0.771736965351928, 1.2977144839897252, 0.6084939243051503, 1.0349835156269211, 0.7670548083286232, 1.1672143367849093, 1.1120218274194733, 1.3789266032851435, 1.5882582926452447, 1.3639803351290518, 1.3121405424145671, 1.1007784483830607, 1.3173246120622437, 0.9500584076710783, 1.0808380628409013, 0.4654329452559838, 1.5257018145057892, 0.3303824307505131, 1.0016333705646048, 0.8045852409858643, 1.7765114682308956, 1.4118813605371257, 1.1807669474854532, 1.0878225563780033, 1.3053347745424237, 1.0225093555570224, 1.4199252109290077, 0.8247933794525798, 0.6403706784738071, 1.5680329025630408, 0.9829617490713813, 0.7673180394042527, 0.44980363393515355, 1.206301901808607, 0.7240454748788787, 1.0094704604056712, 1.411151007795347, 0.9866421546649263, 0.5982512363526634, 0.8518010142834663, 1.21021576647904, 0.8650990608709068, 1.6491331335467172, 1.6186221717559537, 0.8790738016066971, 1.7483056669791843, 1.237717468108296, 0.980852249613393, 1.5082324761170853, 0.8870018128609946, 0.8632022775685193, 1.3409978873231103, 0.9800516643252064, 0.5408091372387036, 1.271182465789681, 1.4493646518048577, 0.94078222707717, 0.6738752224957223, 1.5649774138903991, 0.7757525504421258, 1.2322181863220996, 0.6668339892131985, 0.5454000326204471, 1.5457900209319524, 1.2106145333135607, 1.204002639190955, 0.9138839353427493, 0.652398517239185, 0.9485726964956849, 0.40597077781124946, 1.5571285840128035, 0.8007224588393981, 1.1264674525283511, 1.3484025063272047, 1.1232108284866769, 1.17109074110883, 1.2341440516317197, 0.8716191256000814, 1.3188472122038393, 0.5540720223067984, 1.4401986233403885, 0.7770817378339285, 0.6246188851907767, 1.4315498937370252, 1.3009776378699633, 1.8320629240273965, 1.1061910598835754, 0.4956437558396538, 0.9245408733085486, 1.2148542302181387, 0.9476228903085832, 1.2825853686669957, 0.8998558045204272, 0.8406139980581734, 1.3921333002904086, 1.2075737332069605, 1.708504623631538, 1.6614885964935553, 0.9997619559811809, 1.5277588508572126, 0.820085450879886, 1.4812750043926366, 1.2583103421505692, 1.2442033983771972, 1.2175227991415445, 1.4494344079639936, 1.5241779786065233, 0.9442585014911857, 1.336741786934494, 1.827182007055757, 0.6306910330806458, 1.0843358487090269, 0.6211402212462833, 1.2987958403458935, 0.7899378871664458, 1.1609810474185953, 1.5886826473807356, 1.2228822818198952, 0.8578036580471533, 0.7242217175666438, 1.2429710412957675, 0.6051960942958474, 0.7374036517614365, 0.24567167462526018, 0.7889784323271352, 1.1908759556488548, 0.638631396881371, 0.8416589171122073, 0.989084685959704, 0.6729686256216995, 1.5052116126764297, 0.7496947812673969, 1.7348176017946746, 0.5732974301119157, 0.9230756882230767, 0.9904558136328625, 1.125328385817566, 1.6960022355225506, 1.2446221264928279, 0.9599783512691819, 0.3168063941117728, 0.8035168492429496, 1.454570137994866, 1.3272613074092952, 1.1384631628557575, 1.4311990110112265, 0.9839846862217912, 1.4815147010539442, 0.819984718357881, 1.2680900969447477, 0.498850623547644, 0.1630867451112532, 0.7066037613125999, 1.593822073691595, 1.2404494478975576, 1.2672275300619233, 1.360041471629032, 0.9126203746086805, 0.34433983520065115, 0.3408045092465085, 1.4482075717627936, 0.5517059951977228, 0.859207608380577, 1.0641781366739598, 1.0548639122563108, 1.8702865631582122, 0.8713169050822345, 1.3322355602788458, 1.512195439747838, 1.1928237339907972, 1.5141003558062742, 0.8066819483883736, 1.0215804257808272, 1.0562018160292936, 1.0587985651549878, 1.0493716446667603, 0.658402591074876, 0.931374919553049, 1.3224642072078372, 1.4088271933710677, 0.9450289467704784, 1.025612759226167, 1.2642653141602815, 1.0238953275753362, 0.4609081156239584, 1.1921424675380525, 1.255711168358077, 1.342911907488119, 1.9037571736512446, 0.7484402956758349, 0.790626036768087, 0.4652751813993453, 1.1925199414828433, 1.2511888041703152, 0.8193481749085861, 1.0832798984324292, 1.3675154157625236, 0.7717928147436188, 1.417415649753868, 0.9339085453152919, 0.9250314639270736, 0.05539858178858614, 1.5832146473429074, 0.5057758328423768, 1.6861247412337277, 1.1075598173944585, 1.3141907908064352, 0.8619694279814708, 0.6401562392138357, 1.068380694861687, 1.6751874183487945, 1.306219956813976, 0.7902447963170774, 1.3280678088798277, 0.17035329256604248, 1.5818053511133892, 1.0788754477153906, 1.5391395241391277, 1.1787208080404143, 0.6235680715225875, 0.4448162826398361, 0.6157548599784413, 1.1726560230487224, 1.2601960242614174, 0.8502609054574942, 0.28548803830801417, 0.8072925139426813, 1.056247474105063, 1.0927670307131583, 0.6545325086483309, 0.5998935970783291, 1.0562650629695152, 0.36027103619519985, 0.8940327566458441, 1.3601756382207308, 0.6393585823786047, 0.5605796175071143, 0.6078613898467332, 1.540134747677437, 0.3686326271428998, 0.7170339515602717, 0.7243268989199703, 1.525216982949909, 0.7485644113828928, 1.4479728019178792, 0.8151311301124362, 0.47647387844078426, 0.9967602559783701, 1.1722308792141263, 0.7461185334174864, 1.4965951647140732, 0.5930757731146687, 1.295800052390765, 1.0527549743224434, 0.29617957882747004, 1.0384515547872621, 1.5131933744966437, 1.3096473078513213, 1.2048695097985305, 0.6485632873437003, 1.4901360857144783, 1.040490394312032, 1.6561640842348888, 0.694793994931053, 0.6126156806230534, 0.48219927488139036, 1.5715574828490824, 1.2002370189794125, 0.8570787927606056, 1.1273387563535586, 1.62646508325171, 0.8988231902464291, 1.5911531481829084, 0.6448863352335544, 1.01607273341337, 0.5400637501944414, 1.0893731587025546, 1.355552448322018, 0.9305135884971779, 0.4448202424119613, 1.1611381875777047, 0.7597869682425075, 1.681131845629263, 1.6403165845736227, 1.257458892413238, 0.97192714699633, 1.2325335794033536, 0.595706786261289, 0.5764330137875172, 0.5124643485906483, 0.988751838401434, 1.0949650160538509, 0.7158047956951687, 0.7386832899737782, 0.8219096952910292, 1.016155633145228, 1.4300842349100558, 1.8026976137322885, 0.7248469011138541, 0.9430655861449817, 0.6256937560158254, 0.9438261934363666, 0.5425533718801618, 1.2343418946048583, 1.5827093167668767, 0.5083246593399596, 1.4666899107994893, 1.0041790014187681, 0.9725410766379935, 0.895889568529991, 1.0257890916779702, 0.507756117926806, 0.9193956480980011, 0.4266225368808827, 1.2846704599881946, 1.2676015156787637, 0.813747076336849, 0.7486934236701679, 1.1725772270358041, 1.159954386092705, 0.3488178113655578, 0.9567162475386325, 1.139336477329718, 0.9871502389845402, 0.9746787969064872, 1.4098118264833746, 0.7883215059849901, 1.1294616361361083, 0.6821184428301422, 1.218019782580141, 0.80601072457145, 1.0862204147085195, 1.3730412644966892, 1.1300286285060928, 0.9570648501232921, 0.9636963017148757, 1.1463581874894717, 0.4156938965389263, 0.47871669922606586, 1.2908385360030774, 1.589122990689496, 1.0929895386915773, 1.0587007755446542, 0.19185555673549537, 0.8450745483757613, 0.16597258840285078, 1.1729206273890553, 0.11118494242865129, 0.4703496975749466, 1.4384425198782722, 1.0088289563987112, 0.37329777523212393, 0.9317908044154961, 0.6494493212021523, 1.3109230769144642, 0.8001659740376115, 1.4721255981446266, 0.3189918766748434, 0.7019189129976915, 1.5545748317636408, 1.441188992321622, 0.46097368842723474, 1.5158373992009213, 0.829985926636855, 1.7755132841141816, 1.6166507604750349, 0.9307739398180361, 0.6977773273621873, 0.4122726047482491, 1.0377943873302544, 1.239897630311453, 0.4192826332940407, 0.8795091502260703, 1.4049664571998, 0.8115282484065962, 1.2021733703691753, 0.9225168817691906, 1.1779182021748844, 1.3766769670963925, 0.4345046341527089, 0.9950297174998775, 1.5382124243537518, 1.1409813383398406, 0.7589822907125431, 0.11381704921231517, 1.3297465136667652, 0.9812159458776616, 1.5324008838221048, 1.006700553830224, 0.14389975318348125, 0.3508765159999969, 0.713015613771331, 1.3076295897799461, 1.14904238818737, 0.6871877008400913, 1.032678549014011, 1.4742084701453835, 0.6032198846979561, 0.5446230894544232, 0.6771587297486457, 1.3508667790568807, 0.9457971985772892, 0.9107605150906627, 0.5869820230024551, 0.8536625582416859, 0.6169950565460275, 0.9745485346011149, 0.6143261563148361, 1.073915944853134, 1.2064700589385882, 1.129540960669197, 0.3988717725205384, 0.9272880046204031, 1.6057998810729956, 1.4060910975815617, 1.1557144040897067, 1.0744821674666554, 0.7699691272360154, 1.689948919319936, 0.8213451289903891, 1.7202332301297198, 0.5018951249713779, 0.2224436750392631, 1.1176840886119226, 1.6700132378305181, 1.0510293882354687, 1.2486741510625758, 1.0835415627078566, 1.082217226110139, 0.9582281508325321, 1.4514750122974756, 1.2857293082433796, 0.5813797688902201, 0.1964344776657796, 0.6479021478793254, 1.012058723162517, 1.6977943173084582, 1.033219441973102, 1.547341788222739, 0.46305072381279755, 1.5114300055319225, 0.9734608029620458, 0.9711106247507477, 1.336155061625829, 1.111514440790622, 1.2841327517583425, 1.6305790089921002, 0.8576627820221208, 0.2971170555227469, 0.9849838315108295, 0.8838047589450844, 1.0137773929019938, 1.716310680165941, 0.6786029052898469, 1.3593201413474372, 1.0814128087638883, 0.13739449268612292, 0.8079425249621073, 1.1913947932991285, 0.3979880203960976, 1.765884381249014, 0.3195049571326769, 1.3646291230929872, 1.5776583863014886, 0.7250739872432089, 1.2944736663165761, 1.176487183595489, 1.252019736792234, 1.04940401337111, 1.273485242043162, 0.7120850430926252, 0.3005598252900644, 0.5815437131984857, 1.3462079702072014, 1.6308679738968523, 0.4985519678381688, 0.8599234011057542, 1.131105779709715, 1.5789635588933888, 1.116646777848585, 1.2124020085883618, 0.914091760403147, 1.1580856607495673, 1.902677167892521, 0.9979489357714303, 0.15130684426201324, 1.1034135302998829, 0.900779492688712, 0.43423528898438757, 1.5274083457370127, 0.9952753868381339, 1.2441584596188129, 1.5870301739341757, 0.4437845266832845, 1.3941140819935158, 1.396953973279263, 1.270366948114534, 0.8871059386240867, 0.990925830846749, 1.020511785263237, 0.9996037186370654, 0.325656839576283, 0.9026131171127024, 0.33416462554327186, 0.8996092664058417, 1.7876833369266967, 0.8042888880330078, 1.5756579875432206, 1.3667629665894312, 0.7111997779482828, 1.6238343097111803, 1.2802488312417653, 0.8503042975355688, 0.7003509075066274, 1.13139656783807, 0.4484333790008378, 0.8240686011645454, 0.6662375404547144, 1.3342577531080129, 1.1747798170376436, 0.837942915523813, 0.35034941612511894, 1.072118254382739, 1.0125914876922208, 1.2234810819142887, 1.1838206365242998, 1.4173021148964708, 0.7132820129884414, 1.2593845439424856, 1.5845439661048601, 0.7379184590092914, 0.9928310957074591, 1.1333037797585201, 0.8774314976233429, 0.2274049498742755, 1.8112302678892676, 0.5949545353767558, 1.3559063458818572, 0.7837904560807627, 0.9624506771303782, 0.38456502838192697, 0.9284371577623491, 1.2123042722069448, 1.5204162677276436, 1.1742405846857609, 0.9429291187611349, 1.3767711152659068, 0.3944757886670913, 0.6229857318281539, 1.5209182399962753, 1.5653689485326625, 0.8137338029585162, 1.333528195651151, 1.6345567241640746, 1.2551000594834554, 0.6651208129315099, 0.4339710370180708, 0.9491672042377768, 0.8528565861581773, 1.03000254751643, 1.0779370663045476, 0.5334225616581221, 1.6594788389806694, 0.9176767281907582, 1.1819768521402025, 1.810988780684325, 1.3652288914278548, 1.6116701773708213, 0.7173048796918721, 1.4291067169066092, 1.655102305501595, 1.085482038404793, 0.61695273731499, 0.36831870246326903, 1.5865161394916705, 1.2329693364290644, 1.0165557187192482, 0.9394266081378413, 0.17528116203264277, 0.9288978105145935, 1.5679438030754724, 1.7958533790915105, 0.48920055186644085, 0.1926730624602524, 1.3478590190214046, 1.0364604942022801, 1.2889416534339078, 1.5036151605954737, 1.4593355475220835, 1.6464050444024296, 0.8238848907881491, 1.4243472369631598, 1.139715718815022, 1.1038945130925333, 1.4385750784814868, 1.532474310645556, 1.5696735156668615, 0.7384935663276654, 1.0482868378869026, 1.2133401194664024, 1.425205014695927, 1.0583897885213895, 0.6060219407216693, 0.491586237913876, 1.5309282237698423, 1.4737613891686485, 1.3034717807373195, 1.2740534619353432, 1.5884388105878913, 1.043591894769191, 0.8628918053579389, 1.1272647909507385, 0.8364137720444335, 1.1902729923256992, 0.7926282435236263, 0.4934462264710039, 1.5128757168615006, 0.8634067159989075, 0.5767431652333668, 0.8040401659030051, 1.2252225784086777, 0.6465582852284858, 1.2822688891034608, 1.6570852620914367, 0.7192910597319147, 0.8899999925668809, 0.5402165021479997, 1.0339024434384196, 0.3953513326500414, 1.6145057241923038, 1.3729312803069957, 1.2827601448935346, 0.6195000381920159, 1.540613277780308, 0.859780247272692, 1.3213337087171722, 0.7388345024218389, 1.2470268626501655, 0.7654272675851364, 0.9667196597922704, 0.2166272970804054, 1.626256403793934, 1.5612291045319373, 1.0096886364113669, 1.891568289048826, 0.42314243325953615, 0.8460477786362504, 1.1117119864106884, 1.4934389033354027, 1.3383186086553618, 1.006111787928737, 0.8803946969086411, 1.6257537529562835, 1.2698412931457579, 1.2900891996710504, 0.8621674580270969, 0.9493593718539316, 1.2050955036792028, 1.4962212498605436, 1.3002745077417344, 0.983549846538114, 0.4214472443157924, 0.8773971844821641, 0.6967009205139177, 1.16609727623631, 1.1368050706928174, 1.1645132567407308, 0.8476242675122831, 0.8852570637173315, 0.6283483333073602, 0.530813374789288, 1.2746586002973572, 0.9310291214534618, 0.608011287068717, 0.7515518582531425, 0.160861976184276, 1.2774651719917083, 1.348582002183214, 1.3215208573331476, 0.5339267613316038, 1.338298312265387, 0.9930531174503007, 0.15148620346741526, 0.5494478004682999, 1.3372570732487084, 1.2131859371185212, 0.9826670478685685, 1.5637093684464434, 1.1171883534291678, 0.7656583768474378, 1.1349085867383657, 1.0216120682057945, 1.3473731039697066, 1.1521894921956768, 0.5412072983409297, 0.9862507273125465, 1.4630723112351904, 1.4878392545144963, 1.2668738297343096, 0.9779964531662453, 0.8645323912626036, 0.6331728764998122, 0.8890037500270308, 0.9093016650768033, 1.7538530498188172, 1.3336898437252285, 0.5413691180663192, 0.921123237773793, 0.7504017090098919, 0.18422946529950546, 1.7527837244414928, 0.4106255622020534, 0.45468366839881524, 1.3949431015082936, 0.7944671072084759, 0.58215705470166, 0.8966929817836666, 0.7055118588003688, 1.0751503398101852, 1.2151900918395564, 0.5196947954540048, 1.4607971814675436, 1.1040699331736228, 1.3874260369677, 1.34984225480552, 0.5964877200483122, 0.7994257423054716, 1.100368502735881, 0.9513352868948249, 1.7312456579729463, 0.2551520632593224, 1.1576357617307802, 1.0855663091586578, 0.6140262939422996, 0.5303797526569843, 1.5342730951694628, 0.5622203250961831, 0.5745368547772053, 0.7298979917858054, 1.1013329570749983, 0.11838881136793555, 1.4195087550120813, 0.7043909161009428, 0.6513614978155969, 1.815171261626212, 1.2949462109261456, 0.28948380363950166, 0.5372788306721186, 0.8926901087617662, 1.3360272314137394, 1.0042552101863917, 0.6441146968546676, 1.44649817285533, 1.1139535607136641, 0.31331189459775444, 1.0648547721745598, 0.2622095905921512, 1.2273619205770407, 0.8321064082392293, 0.19648814854828145, 0.9895994601560884, 0.9125259562255764, 1.2943498108470974, 0.8992308400110125, 0.7017391984255246, 1.1381764538227053, 0.8293718747681255, 0.6861871746538057, 0.7980583858816698, 1.0062242088729223, 1.214411278004583, 1.2531733575544484, 0.9509295621974356, 1.3327713714573344, 0.8347550984010099, 0.7494671898123357, 0.3891587945599946, 0.305479607829358, 1.7269157471240073, 0.5048017988503584, 1.1642231039761097, 0.38374286446948014, 1.3065938317560448, 0.6797392116193974, 0.7397023329650503, 0.36125244164829584, 0.7869259996418041, 1.750988613714412, 0.8624266138510792, 0.888096241224441, 0.9268677360138174, 0.9201081627532647, 0.8578450161995437, 1.2989103263555728, 1.326152789499543, 1.009072286809257, 0.4890225936034006, 0.8161148585846542, 0.6542403380691126, 1.1161407960472327, 1.1936766143837265, 1.2057545552944546, 1.436758620828722, 0.9063113103220797, 1.0125654313361232, 1.1096152692146348, 1.0030931495038005, 1.3799282067836705, 1.5012849463988975, 0.12247085313583794, 1.1575197837499935, 0.807397157590008, 0.9723245893610267, 0.7214869233499855, 1.5049045681273774, 1.1482725550385653, 0.5425842083831681, 0.5405898861049027, 1.5271798161531078, 1.4652850698264914, 0.48261350274140935, 0.9087723544965811, 0.19942417646641153, 0.4378441811439615, 0.32848956656200656, 1.545055862510007, 1.0505108159305538, 1.3555906037773453, 1.1916740331292646, 0.7575357460353795, 0.7572649155716441, 1.4956786994017666, 1.0344367086518969, 1.2468286266632145, 0.847647476426327, 1.4044119912130721, 1.7351745056320957, 1.38953306988109, 1.1985160002815864, 0.5484617319980601, 1.2809286145994812, 1.1728297327266237, 0.9811318698813214, 1.3496704194823441, 1.2365979568866412, 0.21865837624082973, 0.5090531227411444, 0.7749435591912667, 1.5130203169545686, 1.4787459127819376, 1.144116641698528, 0.7714739597832553, 0.4766114997640425, 0.35433035471533425, 0.6862013388110059, 1.3259241534857078, 0.1578833421442204, 1.0304948119511967, 0.94915477996509, 0.7842957030754243, 1.8290796309786308, 1.0813292313513498, 1.4848905000004562, 0.5253235778192767, 0.8880409104015158, 0.9350029078947749, 1.1142689157575616, 1.6145079798927142, 0.8975151020071074, 0.27580522536453367, 0.7838183590184109, 0.6955435344995518, 0.6964937313002711, 1.1379585478561007, 1.0568023279981587, 1.049463465573524, 0.4236173048618702, 1.7733414074389717, 0.9069676212770955, 1.7391763643109697, 0.23181604764205554, 0.9515752591499198, 0.9687416932681615, 1.1872133683760344, 1.1001876990047335, 1.2217076856335969, 0.9252871270318295, 1.6694340908009344, 0.9947958016344008, 1.0714983575895665, 0.9749142376154201, 0.34974815202012466, 1.1855701757873318, 0.6702305486922732, 0.6085975804830337, 0.6541522627067867, 1.085283437690514, 0.6869894402483914, 1.2232645228989263, 1.1046040332025795, 1.0761875046514087, 1.3896513172225982, 1.0434200403345049, 0.5864659854357496, 0.3147861400053915, 0.9847788580166034, 0.7362141346149326, 1.07785163883447, 1.3018035360080251, 0.8167098055793058, 1.0686828282265377, 1.6854139618056851, 0.620570123934897, 1.3852195994521594, 1.539286754682151, 1.1876342762568823, 1.0493277012432212, 1.1698435063561905, 1.3624008163883903, 0.2906703813809055, 1.1132303881594185, 0.5884050576052853, 1.6091135668382914, 0.507859291170455, 1.2946051258949627, 0.30748982021117055, 0.8487125064985054, 0.6670833968740209, 1.0748750657953943, 0.5635880783191435, 1.2096394193777842, 1.4954888816434209, 1.2034307726935545, 0.9327269671836772, 0.12366676416626421, 1.2248413168464047, 0.4684474107376825, 0.9031873230030669, 0.9924249153815408, 1.2856699268347644, 1.7564049248188447, 1.3713709897955484, 1.1834797863464357, 0.4508495427414084, 0.5824580625367113, 1.6877466223428246, 1.78463638834274, 0.5708760672287503, 0.8480833325130198, 1.5079641984534233, 0.1386313686751922, 0.7889726169800925, 0.8423992493061464, 1.0447174160135617, 0.45360153401771475, 1.0977667677122835, 1.3689183260032187, 0.6399057333667881, 1.046468259928575, 1.7822175236242055, 0.542788685492665, 1.2033637363394591, 0.47384716739292254, 1.411233958995798, 0.8996474528004365, 1.2372839820892907, 0.7481162656531035, 1.3014531760871146, 1.4126763915250893, 0.7633698422266225, 0.5456457736393479, 0.7357417854902927, 1.5213567779783848, 1.306214085845451, 1.5977014661230742, 0.40577558611841813, 0.999994431008148, 1.1099453267794643, 1.2990233343844195, 0.5255278939169888, 1.4933614784181892, 1.0392404453974196, 1.4730259613905272, 0.8377354636235145, 1.4199160123887244, 1.342213259810633, 1.305001984965262, 1.4103777364647332, 0.6653711947101759, 0.9713990432020739, 1.2767725075856542, 0.8059462602738663, 0.44933904830636373, 0.762269171066449, 0.67071807533961, 0.6005367854120859, 1.2011070177236374, 1.4614025836514486, 1.371827496239823, 1.296983770039214, 1.1544461073437602, 1.797012357470564, 1.6696204857627155, 1.70513909336317, 0.7710056445876544, 0.81736256698842, 0.7726558549739044, 0.6691916080589566, 0.9901773020102598, 0.7361156638854899, 1.353313901040679, 1.6948258812461612, 1.9249535691455841, 1.3887078262899757, 0.7008242064223494, 0.886074518757202, 1.334134373054955, 0.4508555624427446, 1.0826644171427082, 0.08453140498890765, 1.0188000233184973, 0.4110970057070785, 0.3393981374483105, 0.9422587281726106, 1.2068502975711977, 0.9064499722280275, 1.5287324710211143, 0.6904579694541232, 0.9913583934431662, 0.1401243127563, 0.7704796379443912, 0.7556669177238937, 1.4188642979652344, 0.6653488828339488, 0.48020729211341884, 1.147206296041008, 0.5348578697511984, 0.9931172586737497, 1.4055837321465585, 0.4089748087566245, 1.3573977840080351, 0.9833388194228363, 1.2138575394431568, 0.47475307141188394, 0.45308319819100307, 1.3181908033652827, 0.7300415563173944, 0.4673891189132586, 0.4042528759322176, 1.2310934681012449, 1.1236695278767441, 1.4284273462435593, 0.7393184906937481, 0.5180506685156326, 0.7005647233323248, 1.0821139556866561, 0.8619499757022162, 1.5141957258898995, 0.6370942796151763, 1.1916142328644679, 1.0064474103777807, 1.4469705104082657, 1.0766672413712621, 1.412039728422177, 1.1728821261245708, 1.2974188057557066, 1.4326797425430597, 0.6439055001760301, 0.9883817142126817, 0.40487084816084495, 0.5981002481935442, 1.339038141430239, 0.8408575217539335, 0.22181581697258035, 1.0077560313908718, 1.4877597775778173, 1.0894486086198893, 0.7600046094069539, 1.0550427123604664, 0.6015708193709611, 0.7617566722623222, 1.1247288342540118, 0.927239258424355, 1.140349849371502, 1.4254193846017005, 1.3044042667136933, 1.6287948784381348, 0.5180377487510466, 1.0502183976693633, 0.5704612265600492, 0.731085819904403, 1.6855978394931455, 0.5789501591211713, 1.0567901494786374, 0.8267627290651786, 0.8882006377191394, 0.29080762237959346, 1.4421202505428448, 1.1678641012792013, 1.0391163767718283, 0.8119960785250877, 1.383269235468941, 1.2768990467224022, 1.2268707299873896, 0.7559635413065591, 1.8760489355962888, 0.3844662339741861, 0.8518524498149298, 0.865883662156312, 1.6923038740440597, 1.1930488050082086, 1.5308602427544646, 0.9426936776646212, 0.9511454563219022, 0.8943474724390662, 1.5832554319974288, 0.5639975907394597, 1.2638219155211683, 1.785423539392931, 0.26524674409823423, 0.7400192915195173, 0.8780069527050727, 1.1414671792557367, 1.006425167134559, 1.3173868031657263, 1.5408955337554189, 1.04647365015712, 1.2007603902646267, 1.3748182754935308, 0.8582968309652875, 1.6366866902898436, 0.9045441882399224, 0.795882105201158, 1.389776605731298, 0.730212715509968, 1.9358825406125761, 0.8385440407757894, 1.1300144016931648, 1.3415776016079821, 0.7383270605590527, 0.9130021026515183, 1.1137721300137942, 0.8538604477922853, 1.2198358423013311, 1.1776866989589243, 1.2406046340119312, 1.0078178587384516, 0.9159815922987543, 0.9262392420852783, 0.5256193408688896, 1.9043341583422513, 1.1242451763748509, 1.3876902247047962, 1.572660511300322, 1.1095072474341925, 0.24627990840338576, 0.43238394260070134, 1.908108381262862, 0.8910237759524156, 1.1225028769185994, 0.9300459666317908, 0.3964204380690768, 1.099140097065526, 0.9402256810460059, 1.32851697994725, 1.5698918366113406, 0.4709488132230688, 0.7680303021572649, 1.802634623813213, 1.054932008781817, 1.1737207544594088, 0.5624266495434335, 0.7263389766704661, 0.8287862262854138, 1.2547591082847553, 1.1632677122399107, 0.9185656654730119, 0.9601762481132421, 1.113291958891136, 1.2830346533465553, 1.2041870772239363, 1.4573987330937461, 1.3702240263363494, 1.20676758999929, 1.0100061638316182, 1.5746141671294729, 0.8890004584162733, 1.6332070804312644, 1.281502494787727, 0.8541147742141524, 1.1428651394684466, 1.410846293753851, 0.6758074840334541, 1.4941545385122494, 1.0937552742317695, 0.3375308317720014, 1.4446695716319078, 1.199267069580804, 0.5749690749225671, 1.4892500936465694, 1.5958355414264598, 1.0289713028439391, 0.46983400522826024, 1.244842145198897, 1.1653039082639185, 0.9582619848142164, 1.1432289837562264, 1.661369449968707, 1.1077602499331274, 0.3109618579636566, 0.8111756151864762, 0.7741552937220726, 1.6792223985613983, 1.6186035908257308, 0.8259279416062002, 1.3240747443012095, 0.9712747385355188, 0.33016281406546666, 1.2352197041180077, 0.46906013427509263, 0.847960230613164, 1.2168611539745209, 0.8353441604325049, 0.9182337988126318, 0.7278636001142251, 0.2619711373553869, 1.0841164719110241, 1.6585338661507563, 1.444143300055064, 0.1796238774903789, 0.7555184663626436, 1.152739850382798, 0.9189652629333809, 1.8794173959914928, 1.1185225748363787, 0.868746382591976, 1.282174783529471, 0.4640898141430737, 1.074428753615293, 1.0207683465182256, 0.5717105487188094, 0.8985278964276477, 0.021692647488153027, 0.984333647288328, 1.0848660768844027, 1.694999113757692, 0.5051256261904202, 1.4066858112923897, 0.7879593850724722, 0.7812180076145416, 1.6050752744396457, 0.6714508896544821, 0.2321070421470106, 1.774646217219586, 1.3733344414434845, 1.3939873322086695, 1.2013599837604625, 0.5453699294458418, 1.3046882332829286, 0.9222137838918169, 0.991669473347905, 1.0180759467938394, 1.3534924470781022, 1.1921711818051306, 1.3028283023800402, 0.6606020262821667, 0.8698892896964617, 0.3412819946678589, 0.9799987903379875, 1.1252008038277288, 0.8458078869918353, 1.0796318710824697, 1.4947668600062785, 1.6036527191240828, 0.5855355119017565, 0.6906164598559433, 1.364115077046258, 1.1600371973459889, 0.8432620625506254, 1.4211448200078935, 0.40787335184278306, 0.772596620860406, 0.9507289187001235, 1.215858284036692, 1.88683701215527, 1.675959213033769, 1.8090087413041804, 0.7607849315015457, 0.6162475888217883, 0.78267588822101, 0.8424664492536399, 0.9369152995030494, 0.9847367079357124, 0.46161114789632207, 1.3911278490616323, 1.3018745356675883, 1.0512018162604249, 1.1447102964652633, 1.0213527212956595, 0.9951960381046552, 0.9948618033590096, 1.175024370136581, 1.2370803880879286, 1.2242510455240323, 0.6138078018279997, 1.7927342185512423, 1.178142218355436, 0.6430460780461457, 0.4393831823611726, 0.5371135872658209, 1.092904613747851, 0.958327199515212, 1.3964845484308457, 0.6145144323165582, 1.1975649265451183, 1.1278872013432037, 1.2264566321875283, 1.3255922826660664, 0.6407784842810877, 1.1528095463925867, 0.6991858573060331, 0.7543660647810708, 0.6822611380522572, 1.0043187358338812, 1.6042374587925154, 0.5544582670962612, 1.456280737505125, 0.7600423857521436, 1.4442833696448971, 1.13435111451227, 1.0388701871775337, 0.7259014423495784, 0.5723671663867103, 1.5984964315047332, 0.8677092716723604, 0.9273285145709272, 0.36833930358690326, 0.5450086497807992, 0.7437662618415595, 0.7516307111447812, 0.7193942057375604, 1.461080177717093, 1.6380645023063445, 1.4213523798475574, 1.4376776992464766, 1.013745944519382, 0.8841026585639946, 1.328828725192174, 1.2910950690373166, 0.8176651883140061, 1.035178312256066, 1.351297262340762, 1.2804495817487367, 0.6308203678999764, 1.1881194241984971, 0.902244712035882, 0.9109700814676376, 0.7762423163157406, 0.9942469250828871, 1.1729327912207554, 0.7940431377188062, 0.6205525983796322, 0.6208226209593227, 1.3370805804588848, 0.560298573557654, 1.066212139829513, 0.8427545959171665, 1.3293006521082364, 1.0137309276011495, 0.8401075472559214, 1.7920837237335783, 0.724435893044891, 0.4238694757722681, 0.8252174699859706, 0.4064272783936064, 0.7831264169829598, 1.0063522206695383, 0.6935395098967516, 0.614060692057746, 1.047233703187704, 1.3802853148441803, 1.0269614856471145, 0.6900390494204064, 0.516580457520228, 1.1828143031194027, 1.0313694073677047, 0.917592778017681, 0.8991647836539581, 1.041693912274372, 1.173652439882368, 1.2519734141075478, 1.0921878397580598, 0.6340459097827544, 1.4944995800572785, 1.2507113345399208, 1.6281222428634174, 1.5364130844940336, 1.3058293522502327, 0.9807400339821346, 1.952723778534869, 0.5035036299100002, 1.46698218553392, 0.7938907579400141, 0.775208033601852, 0.6667244477305834, 1.1082199631380643, 1.1423661969783696, 1.8418250917177108, 1.086027871124514, 0.5240171714552477, 1.0945748327679676, 1.0033348234486, 0.9700386899211176, 1.3219419289379997, 1.1268779881846411, 1.4347011288141116, 1.240907527623538, 1.4917871960294655, 0.788049700164515, 1.8221321392420924, 1.8552514446225823, 0.8040236690594225, 0.7360380612134978, 0.9013775093524647, 1.056978115584931, 1.207432176233838, 1.1322091658575364, 0.573489242178977, 1.3442936385193478, 0.7134616121076968, 0.5466251125278732, 0.7395441824846608, 0.8611605495547296, 1.4255859289943065, 1.702638812183487, 1.0319085679415974, 1.1611409201282017, 1.1380137573280908, 1.2166791652572746, 1.406282054944485, 0.8795222086882512, 1.5548499372228821, 0.33051598270899174, 0.40341886645487957, 0.39613317828381367, 0.6207173971413472, 1.2138187693697438, 1.624572439476283, 1.3698920832908512, 1.204110241451691, 1.7051410059192929, 1.0327896172374331, 1.4578740153992054, 0.7748901586865982, 1.153678439087563, 1.1212270901919994, 1.2674510832450003, 0.4035618014451253, 0.7277781439690265, 0.32665944897115884, 1.1071060683419733, 1.1011332913898642, 1.2209426166618953, 0.11790630192553309, 1.6078235956551197, 1.0236711018479678, 0.5662612394063714, 0.8187842495322382, 1.638041883458595, 1.2099648345453347, 1.3534182890936983, 1.2132968907040667, 0.9438325995519609, 1.2067914677915175, 0.6282945449304194, 1.1238102545574309, 0.7207686890069978, 1.016128791947719, 1.3036864152384728, 1.5967537609081808, 1.388153887219718, 1.2699145366692721, 1.8033215010177042, 0.7433232495764, 1.0920994915225537, 0.6301774991103906, 1.3426868528573022, 1.395907314952101, 0.4024916817521351, 0.789277234932041, 1.151424130110002, 1.0413075502589897, 0.655929464578735, 1.0255673243607681, 0.8131858773024837, 0.9755621204235381, 1.521308842304025, 0.259605324843443, 0.8761655931651441, 0.716170259031545, 1.2923785721965402, 1.245989530556954, 1.0180215279986138, 0.5203462634901947, 0.5653708779310405, 0.8508720527953277, 1.475288323387535, 0.8533931813476793, 1.069606319905278, 1.2564303678963693, 1.0505380754409444, 0.6565805931634282, 1.307122098033808, 0.96498500779283, 0.9509339682468749, 1.498583443886329, 0.8342215130328893, 1.1993348478237773, 0.7331629245017766, 0.8281659380438643, 1.817609705856074, 0.39337358763321073, 0.8459025645058513, 0.3836841230584861, 0.5674758121921671, 0.7208321745869957, 0.8076419589651981, 1.1546307083979026, 0.9063774558282407, 0.9766274375375235, 1.6613431224530637, 0.7042852566998425, 1.091020610761474, 0.5586892876728404, 0.6408729086610664, 1.4365409691756876, 1.034138139953514, 0.35555933614061985, 0.5171550432957445, 0.13373736671356717, 1.140726652646507, 1.0257187940340837, 1.4330910342707426, 1.2861849382707042, 1.212673774542627, 1.2479825137244864, 1.064037825607762, 1.3107244096356365, 0.4665190244916442, 1.0974592111049488, 1.3693070013336226, 1.4003926418535912, 1.4090508558927162, 0.31052917962395155, 0.7265429708641382, 1.4974382427694435, 1.7098396115939771, 0.49441767227817623, 0.7257868205469282, 0.980732981782899, 0.49849732928158563, 0.8308888587291734, 0.3290769152572067, 1.0673126729828986, 1.4494930818197536, 1.1603145602235525, 0.2763372730372058, 1.5504081840114545, 1.050540970821284, 0.5022144498960147, 0.3425986801336347, 0.8677631173217337, 1.195216055174094, 1.5079459467329217, 0.5220641578292501, 1.363340128848109, 1.5692573716204374, 1.5646312233946262, 0.7154554714604795, 1.1917572687757962, 0.8088869810145678, 0.7157449370942978, 0.9743842774977717, 1.0672901334021363, 1.869484229363652, 0.9028674475265612, 0.3498842189931921, 1.0680196361634882, 1.3008967598063186, 0.6126341199992762, 1.6703585394926814, 0.9006772266080746, 1.0195470056127411, 1.1656989545227137, 1.3261966121908975, 1.071199567369514, 1.3295378128161217, 1.8004482767786674, 1.2537173143497502, 0.4565050262817403, 0.685276790223785, 0.924709874480779, 0.5042029792728246, 0.3309448750320526, 0.4292117739007062, 0.839661942388849, 1.0700401281559029, 1.132722772705395, 1.103409995865884, 1.4596528002877203, 1.696948119237417, 1.1021075118795505, 0.579699022636756, 0.4872624721451314, 0.9680775674534442, 0.7905756317772552, 1.808734614596777, 0.6633381069418454, 1.7585978040556658, 1.3939022165467856, 0.9757011770909636, 1.215379237688171, 0.517735311356861, 1.9082151161996506, 1.2113040143657061, 1.1374597798603414, 1.0371149985345158, 1.5159390097743066, 1.42397132967334, 1.5919003738513862, 0.4257492993264249, 0.32673761865497497, 1.0213504322126619, 1.085596610300351, 1.141964470883678, 0.6498251679963476, 1.025430058456745, 0.9527598770941778, 1.0768558563072155, 0.4182486283146114, 0.9879785135531242, 0.8710960837614642, 0.7398005004126726, 1.505256395531601, 1.2196803051986835, 0.7260324789040222, 0.3862308073415963, 1.1957437337727033, 0.4301661687660461, 1.2525749826222377, 0.5929654087246875, 0.6146216541146715, 1.049148696870183, 0.9372059501125496, 0.7037111346545852, 0.7834781659392194, 1.1300664940201641, 0.9844797190570016, 0.9946306993798085, 0.9718140799802862, 0.856609379292242, 0.08412955546830159, 1.4948485274315066, 0.9289505966364382, 1.2773984358453645, 1.5309815348830533, 0.4278595485072386, 1.2723518226626855, 1.2175768062165597, 0.8646114452197791, 0.758061132090039, 0.8245063545349569, 1.720254333426372, 1.186342222714332, 1.2959935804162717, 1.6691100359472495, 1.278288020765587, 1.3248459354363118, 0.8582999023285778, 0.8139267800637043, 0.5815790964433027, 0.7559613864773943, 0.8915372759233038, 0.7647002397280044, 0.7081435425409294, 0.8814664093754748, 0.8922145294855975, 0.8735196545667672, 1.3107102373687307, 0.54121223887506, 1.4882327448822654, 0.4799147365268528, 0.23770991067381164, 1.7150812429185056, 1.1180584925717825, 0.8091587777928488, 1.2549740399220561, 1.3119170138689538, 1.0095772976299515, 0.5577422521019757, 1.4583792909021647, 0.4320294686266153, 1.5425704538730616, 1.2649917222460028, 1.1797112343096763, 1.2024041026881176, 1.6095581845504938, 1.4800414166095215, 1.0408565120492637, 0.9070755219032733, 1.1931434546408648, 1.2317360608422545, 0.9077380616279279, 0.8126132916152002, 0.7759072833065771, 1.0204350268155826, 0.6785547359908735, 1.273405323698725, 1.0386684609670522, 0.3860766622831012, 0.9841013841896178, 0.8449217113282546, 0.8751645074957699, 0.7667846478290861, 0.5856387112814884, 1.1060815259553172, 1.0741959965432089, 0.15438964100109986, 0.922614059736342, 0.5732518479861202, 0.34829279896563947, 0.5267185732375883, 0.3416749208532902, 1.2205644681074244, 1.5646286161092433, 0.3096355281472105, 0.5496580852229634, 1.536688454536888, 0.22375691632051142, 1.452131281733834, 0.9291765716273361, 1.2000082565250847, 1.6311805089948948, 0.3605577136587582, 0.3640544453139325, 0.8864274260490898, 1.2663567383090635, 1.6162130404515644, 0.9953369543074925, 1.3404031073588403, 0.9246241130926309, 1.1422538295621063, 1.195153304659769, 0.6975345667083531, 1.1006394803451416, 1.1837228754103075, 1.4310005083034696, 1.3069883498820798, 1.448454057901103, 1.4079043567545129, 1.1085194955629976, 0.8977950575226414, 1.7891865752600236, 0.8132036158441539, 1.3397152592325001, 0.5459781061589736, 0.6530630537807761, 0.808386579763142, 1.0085568478149483, 0.44911857480975126, 1.729760663513713, 1.3489079644248987, 1.4073986476417812, 1.0378910266082917, 0.5968043351177, 1.7843721645619186, 1.6115136538320403, 1.7390828900859723, 1.341932163511684, 1.7809959488149718, 0.9549665472123442, 1.4436103637379931, 0.8738029380711054, 0.8748321902602344, 0.6123719231358502, 0.8886882765717353, 1.6956792403035577, 0.894745055444271, 0.7372892634025937, 0.40955721458105276, 1.5515142576188063, 1.6931468449859581, 1.218291703726369, 0.6853793413280815, 0.901445567198838, 1.2510876997639397, 1.1662261366937638, 1.0608046228720223, 0.9640982525993533, 1.6161279792433545, 1.3696629186950553, 1.3997700517334728, 0.5342621853707615, 0.6493893484876057, 0.929584800351711, 0.8235744332337901, 0.6449157992088643, 1.353054363445891, 0.7468247771717869, 0.8300171706296534, 1.0810195427094487, 0.8019251523524994, 1.0475664965078133, 1.0496689560554893, 0.6603390125961153, 1.0770019775269284, 0.8840308386090143, 0.9475594766474733, 0.8801655097490585, 1.5098937753775679, 1.2672748172881554, 0.5387912542422143, 1.1091073349274458, 1.0076494061708647, 1.301378357112115, 1.0315387993371794, 1.1672749210293585, 0.45344596724738484, 0.4267799308372009, 1.244387906399706, 0.9640947614843838, 1.6712291407417177, 1.264063903637215, 1.1605238994432097, 1.3308876694845369, 1.0369414503139458, 1.5024917999605978, 1.079497612020003, 1.3915628044933168, 1.1487916347163667, 0.90961300695513, 0.8339419218913506, 0.5689891939193437, 1.2309565350471932, 0.11273963082755145, 0.7963303849967392, 0.6787620310907793, 1.2626318898319737, 0.16519010166076686, 1.7489620280251872, 0.8516110432634336, 0.9998683521966772, 0.989258088001836, 0.6475627885913666, 1.3899000636662238, 1.729760772120187, 1.0579121786358732, 0.8023693387206391, 1.2948058124568935, 1.3676223727136083, 1.0265577126200165, 0.6053280126149252, 0.5092155441484609, 0.6367861994492381, 1.2941327119649215, 0.9084340251945336, 1.1721759705374537, 1.0627221261758715, 1.302387875405692, 0.9796524698922898, 0.428365538490619, 0.819055875009787, 0.974995227892834, 0.7081195661769343, 0.4811898677517753, 0.8201001647107825, 0.8753616047766515, 1.2115204568060332, 1.1374807975006345, 1.2684480867474988, 1.1585562563375975, 0.8917596391281074, 1.8101261388674996, 0.9579642312984575, 0.8979613147741402, 0.9184462490713019, 1.3229451341225598, 0.7107181306688649, 1.5629357001516704, 1.3744220117541601, 1.1935093811193354, 0.8999988084618663, 0.17971634668219227, 0.5909118887782403, 1.0873347997791785, 0.5331485060499327, 0.6883288713166872, 0.47613607502321564, 1.0812115458826408, 1.5302114650263676, 1.0579858449625783, 1.0752513512792534, 0.9285029753157383, 0.4396366292548637, 0.3523595791264066, 1.140923144107318, 1.1725588416532147, 0.3526797660884642, 0.7543874049268328, 0.6091743172692113, 0.4919739065057025, 1.3476849515644322, 1.9096895565914989, 0.7897365317920945, 0.8913831255346288, 0.8989730276189443, 0.8032170148706531, 1.2831682327022098, 0.26666474433810616, 0.20705693239080591, 0.9282342316918097, 0.9375649253810155, 1.4936154484871298, 0.6882509585142262, 0.5755729203532317, 1.366777542575764, 1.5492295099849307, 0.99322840979034, 0.45307557909227636, 1.4556795024250506, 1.632772073810747, 1.3828868686979128, 1.3253464838521065, 1.0230930243805967, 1.2524210840053094, 1.0786447170538656, 1.5827449058762566, 0.7799888455659052, 0.5945471644031382, 1.0953579739327384, 1.1538883469238705, 0.3935462063486739, 1.3199580211325201, 1.3339170235881754, 0.9689464601860037, 1.1239713122182682, 0.7682910135845121, 1.518846108865886, 0.837890290307409, 1.0243951415532089, 0.5363353304307485, 1.2447870571207658, 1.812805290422681, 1.4391083865738932, 1.1178942353716037, 0.6375287152500704, 1.0851412231896482, 0.9530642726070803, 0.5993080714899572, 0.12692952552820902, 0.2610222628141672, 1.8422232221517296, 0.7634687930561651, 1.1160548650033748, 0.9606363057984538, 1.2300434398129467, 1.5594515825034247, 1.5611064458081865, 1.7595323540430585, 0.6604281509082898, 1.015892020638841, 1.1568389047027563, 1.5376394111012894, 1.6054351745972286, 1.6612740901855154, 0.43382210375864017, 0.4029957193191003, 0.9615262088498056, 0.5194453422650744, 1.0704777195426654, 0.8863829601303846, 0.90875305156974, 1.6546523666678563, 1.6140536079367793, 1.153524493371934, 1.5871168129435764, 1.0385661602325367, 0.875270136866142, 1.0374614233145287, 0.7359276136072208, 0.7772618647311192, 1.4272162746860133, 0.296356657039338, 1.385986434815963, 0.5146815717276945, 1.0317594434031583, 1.362375763513684, 1.4400458176593838, 0.9441039182869276, 0.3166237944849899, 1.2554360452621354, 0.10735625634305734, 0.9692562352236996, 1.4216365928031736, 1.6103279240535553, 1.5191331667945622, 0.12720364675648255, 0.6413068711066336, 1.5781624517510258, 1.0754895240313467, 1.0221799642434102, 0.12521618120762879, 0.7603361691816409, 1.717508474282111, 0.8669211958492424, 1.7824936687726924, 0.2591802423667371, 1.4605025124922475, 0.7991038384449548, 1.5338270504364306, 1.1717045394746006, 0.5295469755315722, 0.4105678333760705, 1.8430230300188588, 1.7578455893921552, 1.3134564306100671, 1.0583682210581586, 1.2604887736835368, 0.5358675113702652, 1.3928320646081425, 1.0072127309421195, 1.1844391206132907, 1.8692324811348013, 1.3059054598197677, 0.8383278455548877, 1.1413180983473006, 1.3442682902496472, 1.3208042299424985, 0.8540009731220881, 1.2659942614431685, 1.0610289912567894, 1.240128075161564, 0.6806522669036251, 1.3685907001114637, 0.6716013711424944, 0.8448828724630741, 1.1370627131314412, 0.22049366402605675, 0.8975397137651953, 1.424402396335489, 1.0086671155020297, 1.1793648949727515, 1.5341178100198396, 1.0913274506165922, 0.8787901053728144, 1.0831216073958265, 1.753576320636165, 0.3870911841814283, 0.727864772459194, 0.8895398214493218, 0.5002086819961066, 0.672770582468433, 1.047459778495917, 0.623027161127642, 1.3565853151869987, 0.8317951309315373, 0.1793413269821421, 1.3378905056190464, 0.8840666551365799, 0.41215216830517565, 1.2502568950126882, 0.3612890028781295, 1.4577042436496985, 0.843696715446421, 0.977838864310978, 1.0496844694349126, 1.2786139682468867, 1.0121702490116804, 0.912964630384788, 1.5948696094035082, 0.5336064125287553, 0.6626451147811101, 0.5874961873094593, 0.9388706826252461, 0.8156487042903957, 1.1840196704984742, 0.6855068094810749, 0.5070567208709696, 0.4648239651712843, 1.212081563009939, 0.5038631956588249, 1.2384693894541048, 1.4098548638220034, 0.9819075641223429, 0.781648831390926, 0.9300566778372966, 0.39605881196995374, 0.6582927551945855, 0.3588071826113688, 0.12580065562283138, 0.3962209957736119, 0.7560163789883705, 1.3787089963882913, 1.928602658841235, 1.038650702810405, 1.1958162989286665, 1.7520494810433354, 0.2820739723452028, 1.3133820684660769, 0.6786519502579568, 0.47426095751005803, 0.5129950784578371, 0.9440195042046959, 1.5670048621463621, 1.7044054997873694, 1.2496029691192283, 0.5387959230062489, 1.5882383927974377, 1.3310280162194954, 0.8825595217801356, 1.0258696013945108, 1.504065685635026, 1.1314978504107447, 1.2990910531394226, 0.3632719155279994, 1.3255889119644673, 1.1365300719491613, 1.5650964365817213, 1.4198282243368612, 1.2289905245091302, 0.8289133229125261, 1.0936746926386836, 1.038057393452202, 0.9906556753500789, 1.4011939390026287, 1.0406818142261844, 1.4352565703853695, 1.394659156884198, 0.4088149133720149, 0.8016739320682753, 0.3398161449802479, 1.7353989378627932, 1.812748044383004, 0.8433577692304761, 0.9281706420515003, 0.7409070019635384, 1.7611203723673143, 1.0936140323106094, 0.8883709935768089, 0.8501792058627655, 0.8567450610200766, 0.6351377653370877, 1.0195510005762798, 0.6453243272831184, 1.4459196989193441, 0.5899339588111283, 0.5017839178999497, 0.9981249629405783, 1.706550135568269, 1.548780258563756, 1.6530300408439205, 0.9620010600172467, 0.7348126902102559, 0.6900180558304898, 1.7668659283296142, 0.6070561086351192, 0.9161409026626646, 1.2404452440798455, 0.5120462477576478, 0.664908795808306, 0.7092197367100606, 1.3305524008590033, 1.625050796201322, 0.5207808306221661, 1.174379864522092, 1.0817019584579999, 0.7956415728213303, 1.219717446785237, 0.7327528035010072, 1.5565120069415497, 0.8190838643814552, 1.6846762286135244, 0.7651770640175256, 1.1545945428631312, 0.8096587976221941, 0.7397913858825858, 0.9356601477756995, 1.04348008600082, 1.2086262700607664, 1.7195166380181912, 0.981527497195322, 0.20781295131573474, 1.1560259871266045, 1.6113689594417246, 0.23542460566871837, 1.5877156612596184, 1.0332432412559782, 1.1031049139956148, 0.7885157129484157, 0.6899611909135158, 0.26116532264879744, 0.5573754507670448, 1.1928808323723992, 1.5353899893748308, 0.6685800954088497, 1.2592708363355238, 1.1168180405056674, 0.7135109055474553, 0.8010068242714943, 1.7183487888767734, 0.6429907800643523, 0.9801350168304244, 0.6478453896112312, 1.4140580571520465, 1.1087782171449776, 0.6282838214519031, 1.6004326396007473, 1.3432040193275965, 0.7944613589490968, 0.9193660857693591, 0.8910703813828572, 0.27148802309301956, 0.48102979412956104, 1.2836274593441788, 0.4867660697813887, 1.3774855995637232, 1.2343455515445798, 1.2700990700897181, 0.9255841482815598, 1.4029246850787134, 0.4338988729633637, 0.8173299463253108, 1.4886166600605024, 0.07337138102761609, 1.032058973519116, 0.8829131120798228, 0.9075840744288364, 1.6967666837712445, 0.9741241905830039, 0.8906323606979041, 0.7068162819140981, 0.785732908496795, 0.8226365246129453, 1.3985026045197932, 0.9799544844559976, 0.5194570591228124, 1.4242941632827244, 0.841010894829009, 1.5038286033946955, 0.30550985240204476, 0.789767531832407, 0.34967735307320136, 0.8945799137859546, 1.2043894088770695, 1.200434749482847, 1.1169918885275032, 1.352758697262663, 0.6221129283566128, 1.208587237311626, 1.7171704357461581, 1.7257756971535327, 0.48222265160273925, 0.9158965697151078, 1.3261428269237392, 0.7727654568820839, 0.6725682043529169, 1.1806779493193766, 0.9587815478821056, 0.9104575441728024, 0.19935877918844858, 1.6118639054402875, 1.2174246363796164, 1.1130823185448113, 1.3546884639651244, 1.1961184247779921, 1.0261669255751165, 1.2483534895214343, 0.6712396168937937, 0.4906104084717037, 1.0854597866215343, 1.4197869904331615, 0.8389298546433732, 1.3581638295166623, 1.0877373045659082, 0.12840316891015946, 1.2490514203002392, 0.2732319431316924, 0.30907295415153213, 0.7180491769629886, 1.3446601927540338, 0.5741262448513066, 1.4337624766668697, 0.5417290468261979, 0.6308904961027115, 0.8430553946396132, 1.2664135132427856, 0.9281245127274119, 1.7499263021185238, 0.1405101733303803, 0.8392053551284764, 1.2380728778749084, 1.0308210071773467, 0.7652076913415734, 1.2947247684146708, 1.311625473783881, 0.35592540691526886, 0.8525378223176885, 1.6003080996576395, 0.3547863911662583, 0.9159403590583106, 0.21152184537646634, 1.682127432561108, 1.0847376221031504, 1.9119190503173464, 1.1403867888626456, 1.115059945381026, 1.0165183486989395, 1.0188730282028637, 0.4844030740711337, 1.3756055658401407, 0.7482344642205285, 1.483117140957889, 1.4424928371040115, 1.1383975302095393, 1.3913459399039994, 1.5736165743423747, 1.1893606596315165, 1.3216831515760108, 0.2473248706826442, 1.3634505097860021, 0.7548116894846485, 1.4887130636658596, 1.8409297447611865, 0.7888461355506057, 0.7865863740424, 1.3266374700622228, 0.13117098702360708, 1.3316472857519759, 1.5351047028217608, 1.0101166539189566, 0.4893324148873863, 1.260334651583582, 1.3287249975690514, 0.9392510535651424, 0.9515606543302801, 0.9732678286015032, 0.6082071456364688, 0.5674659459226921, 0.9259228087478617, 0.632158877492185, 1.6226269605845647, 0.3833848761666979, 1.0871537045703041, 1.1046458985607313, 1.0970211456423384, 1.0738218403975686, 0.8167131913863087, 0.9239306524171605, 1.6788454651593532, 0.32603498206285697, 0.6465558172360354, 1.2807258051392347, 0.5699025667358881, 0.8804463451411529, 0.7738741661419176, 0.5323017193652718, 1.6660406861784471, 1.5980365324391492, 0.38992101651608924, 1.2325022462539321, 1.8888786773590511, 0.1894905165625097, 0.9243423162306937, 0.1965160068063211, 0.5992207697351538, 0.5263501351541259, 1.5172601566074388, 1.026362953680905, 0.40215015030429535, 0.7196632433665472, 1.4497061640341449, 1.502037436603668, 0.37772215404597154, 1.7211278265085503, 0.6920662162976489, 0.3696761039175037, 1.0344766894956123, 0.826434155306302, 1.102919089938975, 1.1545095380980408, 1.1225667751578026, 0.40517024413186875, 1.7655870126715798, 1.6053560887937506, 0.579482001188681, 0.40951232911859603, 0.8229930161154491, 0.5659606131652402, 0.7665378594629471, 1.2037859703271454, 0.990539972900149, 0.7265735624685463, 1.0142942566917503, 1.0454994491865226, 0.5841537151172964, 0.7411972298861658, 0.46057086534653935, 1.281876039396836, 0.4384973418529845, 1.6791933186764483, 1.062429456133135, 0.9254048350468603, 1.6176416656962342, 1.0365282712853452, 1.0533378827467068, 0.9248838709228099, 0.20485084573968526, 1.0372141096843617, 0.5266071928672125, 0.1319022165071574, 1.0557071255345316, 0.3700425971341097, 1.4177870819962375, 0.9146998698817186, 0.8557198898350131, 0.8123136448378722, 0.8651771961823066, 1.074721296298013, 0.813565897732932, 1.0895638762726287, 0.25443793794056035, 1.3033918924363537, 0.5039213688621063, 0.5521716897136397, 1.7020534994833845, 0.4265986179597653, 0.4244019326944435, 1.387678359741968, 0.9663216301060065, 1.6137902201282057, 1.5525724652253547, 0.8357285206382684, 1.2130485854199402, 1.0749045191495024, 0.17090658645453638, 1.44837456472208, 1.0427793731380977, 1.0538863491169135, 0.9004296240831673, 0.7030101334888175, 1.4854305517948898, 0.6027313805023762, 0.6144890192782347, 0.5256946480193451, 1.076666070135846, 0.8959528964022204, 1.2156101079294213, 0.868374750809994, 1.852458878948056, 1.4108885444648074, 1.2980122425142118, 0.540748845499966, 1.1402348191536307, 0.9766231380457581, 0.6978876443514868, 0.8116017043666099, 1.1353175819150128, 0.6957049642012538, 0.8015217604466782, 0.9938789372728019, 0.710078659230147, 1.1011074267843677, 0.7690396611647787, 1.8615606747454652, 0.9316105581611076, 0.516256749124595, 1.3379671372579618, 1.179105881767234, 0.36287265690481163, 0.14868817073390794, 1.1505031305581908, 1.855015011604141, 1.6953240919947596, 1.3993717701562565, 1.5546675294762584, 0.9344334737412612, 0.8560306960847694, 1.0680211919994131, 0.6294368774425241, 0.9308532226588409, 0.942050793063304, 1.6096903703413212, 0.44811924833236716, 0.9569699053730252, 0.62598238088944, 0.6815120773006954, 1.3669461905389175, 0.8501220500940961, 0.9634947358534407, 0.8210510694431984, 1.3104043175719804, 0.5349596084649548, 0.8837142791429976, 0.9701899227004357, 0.6242757483487351, 1.6205612545098869, 0.9802282042881997, 1.4713534804189128, 1.2708196510159837, 0.8009743715096255, 0.7771089427956697, 0.7927116708883144, 0.7540186553166669, 1.1977389080533514, 0.6906741341395451, 0.8671160262601936, 0.7663468411633229, 1.710191897485047, 0.5854468665796535, 1.3041306924180227, 0.1378348912474805, 0.5098092535294854, 0.24641659344971634, 1.1840559085533944, 0.9538768650212702, 0.3104742313001774, 1.2107936597108446, 1.589753811430409, 1.2082494117622866, 1.1482031551283405, 0.798637467003657, 0.5190031488156197, 1.1250495392940598, 1.1738722262007841, 0.9596870574554409, 1.3263089020834502, 1.4107551033713555, 0.8784261642811291, 0.955063682741532, 0.3952445306634548, 0.26880829896448444, 0.8293042947504428, 0.39754118795291127, 0.9976484090410476, 1.4850227940014529, 0.9854495699156395, 0.9879363846170991, 0.6133101205382486, 0.7773159099792807, 1.8789198066709172, 1.549173885750128, 1.114879413896229, 1.0488758567645402, 0.9852301635175413, 1.2138037912666515, 0.9973778530969502, 1.261769217696035, 1.8042719302072694, 0.935054258143708, 0.7683876612111721, 1.6003021406172682, 1.103039231312672, 0.7806972994716975, 1.088186155168206, 0.41378657780133854, 0.9001162257234926, 0.4983054348403504, 1.8845811170319582, 1.4691778921569156, 0.7282217457944985, 1.5248515338133843, 0.20141063626739886, 0.4223723856837437, 1.0112145557288488, 1.6458697808339156, 1.104833551578838, 1.285875237521308, 0.45607800809437804, 0.9732396068034822, 0.9954917363777285, 0.994882068512542, 0.4242054674661001, 1.6053227017946117, 0.9094271325515695, 0.978645770499411, 0.2358757156054384, 1.4779362015283257, 1.8297230632852424, 1.4076702311060991, 0.8619714903170704, 0.7936124121297993, 1.3893938575902163, 1.4179580246386585, 0.27439447653571436, 1.3704602898488347, 1.1332519712977818, 0.7391964259537506, 1.291653177642344, 1.032106021229664, 0.8211246785946233, 0.4711749472629042, 0.645505432060212, 1.2565696140386815, 0.625961684888004, 0.5813822274748087, 0.7709571612959203, 0.8136812405831763, 1.0253656313862167, 0.7247102400219905, 0.25936148157278227, 0.2798302374333069, 1.3532252168271053, 1.6580034099962124, 1.2236089137870771, 1.0252684934249312, 0.9073211387387733, 0.7045512913111375, 0.6932022149229176, 1.0498208048630482, 1.346527103862947, 0.6138686886271104, 1.1164676874722674, 0.2024908376924388, 1.1370879612217697, 1.1763941776636206, 1.611595086599057, 1.4016100833032417, 1.3872291868160418, 1.6877886659159769, 0.21973627656205363, 0.5278885067467792, 1.5038319141688634, 1.0367496400274046, 1.3065258242754842, 1.1292633585303449, 0.9948546145077458, 0.8288110435474876, 0.8675163543965343, 1.4791677887242003, 1.3869827324185373, 1.111403694777063, 1.0223069396269722, 1.5726032274239028, 1.405565050342454, 0.9870004820327183, 0.7942303007336183, 0.444017846404425, 0.7926708917621925, 0.07726641805491319, 1.843687339856083, 0.8519567257345644, 1.1745643963045256, 0.723122414200272, 0.5722613350942165, 1.5583228960278843, 1.480856114906413, 0.9011883594959761, 1.1738707816977998, 1.091198749706709, 0.6336201516157651, 1.008547538702028, 0.2048454327246214, 0.88598308799412, 0.7439320911257358, 1.6935564470829951, 0.7581224744977295, 1.1304688139156056, 1.5603551962823898, 0.5034323811636703, 0.534845328464618, 1.3076367471536874, 0.19394254360520502, 0.6990106444138423, 0.911277053336431, 1.1080274306171658, 1.4256428490722053, 1.250516827157444, 1.4166020754507622, 1.4538526976586745, 1.0033171649765187, 1.8674418386905822, 0.9051238131976812, 1.6342005381486016, 0.8465643623553358, 0.781753502146001, 1.1120593790642666, 0.3018459905966838, 1.2887424280685058, 0.5283377664064246, 0.5136973737533613, 0.873164561408737, 0.18924877093493198, 0.9271090304902919, 0.33140053913831746, 0.9704964299554603, 0.41126271388364743, 1.404039345025839, 1.3791955010190569, 0.8259513237591641, 0.5630236270055051, 0.5324786774917606, 0.20632822027061504, 0.9922418818020603, 1.2836662374354058, 1.7560512699451993, 0.8070837894247569, 1.4512839059231668, 0.4841071046498552, 0.11117213255321712, 1.3189236564831857, 0.862350213972602, 1.406381156840602, 1.2122871586843287, 1.62068021719673, 1.9495721786241225, 0.4692234237735178, 0.8299135201917013, 1.6474798276058782, 0.8827326143891872, 0.671963671299201, 1.1767997937267816, 1.007585625235544, 0.6245059330741927, 0.42608254383006905, 0.7993352653891197, 1.2016289078364402, 1.018973043085989, 1.7777286498722367, 0.7103758925880921, 0.5301592963688551, 0.16261459688872304, 0.9741765729645804, 0.9490461254023606, 0.7326180051058634, 0.7803734903526646, 1.31024868179169, 1.121203715680263, 1.438793118368844, 1.2408170814669972, 0.43167212842595426, 1.3301851419392257, 1.2172676350371683, 1.0367037599821485, 0.5941548156981592, 1.2309050188404684, 1.489986657773667, 1.0567408401507268, 1.013364275444187, 1.4027337601358765, 0.9741229391812932, 0.9502018135629253, 0.9182256613710708, 1.3620425925395006, 1.3273023117325762, 1.3349784350860867, 1.4526654669171593, 0.8864434373734834, 0.5857722648589656, 1.3691764481230568, 1.5418322865057528, 0.7137944529340815, 1.4892116748864073, 1.3322541615855812, 1.321571509662887, 1.7652925027585054, 1.0948520614480568, 0.5865836486722912, 0.7573187997161943, 0.5440688088063231, 0.6862074936784297, 1.0982352171138494, 1.1786370671417958, 1.326567176611949, 0.30811159062212123, 1.0589763227466198, 0.6489952844035712, 0.23530270966897038, 0.42632389104435764, 1.0390112453738483, 1.2049866913542289, 1.5326700683808276, 0.9883814297145673, 0.9413644384145543, 0.4946950516698627, 1.0506661063860752, 1.0767783393932207, 1.123212784845964, 0.6667985075305753, 0.9677114911403741, 0.8981489659946978, 1.2775535660001731, 0.7028569288386203, 1.3543834805192645, 1.427586971699266, 1.3475383359610063, 1.5408552465940817, 1.4324743262999358, 1.4640022832829196, 1.1226155316205613, 1.0076124464104206, 0.6015918855830891, 1.0633932958986505, 1.0415388915922292, 0.9926782940111427, 0.2561035336990881, 1.1151107338593627, 0.9773888231160682, 0.5218773312419482, 0.7207048165268808, 0.3972058807825892, 0.9922897663335106, 0.3057750056669799, 0.854842319727852, 0.9874480010807214, 1.3430402067222187, 1.15217128239717, 1.5767805363566343, 0.49691205256379045, 1.0756068350294568, 0.8721969972694215, 0.8369815432775146, 0.6046628972152219, 0.9553329960193696, 1.6907391586851332, 0.4809063867461475, 0.7972775518862603, 0.6931205868224647, 0.5368995622188237, 0.5238955372308072, 0.9330765374808656, 0.4145480428278949, 1.6014193759704645, 1.176933303448601, 1.7570251724205002, 1.8193836067567886, 1.1110482837124038, 1.2174705592265278, 1.6313304119547127, 0.9177419638125062, 1.464939220676944, 0.2215005307524175, 0.3043226915270836, 0.9241371283203831, 0.6360152022227302, 1.202208410031705, 1.175932314073013, 0.7837471971226777, 1.134458347396282, 1.4617835707956797, 0.687084989384668, 0.5227621084134053, 0.8356524118533908, 0.9581407730455715, 0.8067497006849093, 1.0205052422706022, 1.4180839557253755, 0.04362633865140131, 1.0088217781635538, 1.6093695756568978, 0.5461746903796131, 1.8096112750956812, 0.6917573757988662, 1.0167804734054227, 1.3497580599382957, 1.0022866374000645, 0.9668477809155707, 0.4508058266377539, 1.7920847683551282, 1.0052939418787703, 0.6728013013623556, 0.6193965570163489, 0.5090343936872499, 0.5888706734224044, 0.7417989186260979, 0.8933174314434261, 0.9341798296903764, 0.832893650151219, 1.2066365643243953, 0.9648393458606529, 0.311029356743565, 1.6114406226645737, 0.4132900861493801, 0.8490247976175248, 1.636399534317474, 0.8449074671062289, 1.8635040426959044, 1.5317976623985907, 0.8596997960731658, 0.9964400375856216, 0.6236111660252355, 1.2212849693623538, 0.6240131396459176, 1.5750445779192335, 1.5662290553390161, 0.42041349549403095, 1.6405841849369835, 0.6569795994649418, 0.8288961265943304, 0.815671451798209, 1.2175428933458612, 0.7913385035725936, 1.3220704509848302, 0.8421354296906924, 1.457176426443603, 1.001802353498455, 0.2273327980255384, 1.2007576958398234, 0.751864775700439, 0.5972485092940752, 0.7029423232627964, 1.1785108480392348, 0.9551267374621021, 1.3112057966405484, 1.3448404432954408, 0.6063902199637045, 0.9380560692545876, 1.0570280511407872, 0.5166804119706666, 1.5226694282116042, 1.0758282191375341, 1.3318162296002702, 1.685071627609505, 1.441744488908588, 1.399424114536445, 1.0518216203412916, 0.7999617779189155, 1.261092794972699, 0.7544509243452737, 0.4976644366329095, 1.3215670867513434, 0.9704007567452597, 0.4008183769543153, 0.7730613711883296, 1.6325308892627912, 0.9473979648861686, 1.3011757753858095, 0.8828144284572451, 0.6393421684615771, 1.5178571776406016, 1.2975832015603448, 0.3069396347910395, 0.9808709666942892, 0.931250024100476, 1.175833853925584, 0.3981147777176238, 1.0624625947617305, 1.6406282084108064, 1.1299790138046215, 0.9522189915029098, 0.5785534461730782, 0.5883375260758852, 1.638049110560206, 1.598375762542084, 0.762682357005135, 1.1399890370858037, 0.9882592036958254, 0.15930689178480417, 1.854666137872473, 1.188263581032888, 0.8181973657659095, 0.20261319908391862, 1.2360523445296376, 0.7987266133330034, 1.5534126658788732, 1.071843150300727, 1.1111383854157124, 1.6302504749365458, 0.5503328049642194, 1.1878366786118213, 1.082151067325375, 0.4338004093879023, 0.443957116237068, 1.2706409550602247, 0.6363377473108847, 1.7837010055541271, 0.6659835984093443, 0.8562906492809185, 0.9027690700832829, 0.9938234061119533, 1.240350828293804, 0.833719354588132, 0.8068507723700834, 0.8682152472574645, 0.8703016216897106, 0.5898903978886145, 0.6644398312736386, 0.2715589558052215, 0.5042040819696729, 0.9888231626774884, 0.670645029037861, 0.7234419464989966, 1.5750966715158774, 1.1868449861796353, 1.4106226748714918, 1.5391066673815157, 1.0844251284715365, 1.081614228102619, 0.8219684083679922, 1.5018519149383562, 0.9107336093634988, 1.6280714698522853, 0.4843843899992861, 0.47462966393355466, 0.7763094533118132, 1.2975438078749164, 0.5995558893410954, 0.8453237047914877, 1.326513046959234, 0.7059135341793985, 0.6554461805735655, 0.9743877357213285, 1.1761190093523441, 1.2574641020981687, 0.7580582416733616, 1.040821386370704, 0.536766682640881, 1.0143255816048589, 1.3893236039253904, 0.9962657498306956, 1.6015382211014488, 0.14319307704471396, 1.3561525965846912, 0.3490394449483485, 0.7507927131405787, 0.8657212148444344, 0.4174348688492816, 0.8880807144315781, 1.1612362868848032, 1.1255578605897176, 0.8954227289143787, 1.4278052128203353, 1.294551049188, 1.71664463910552, 1.482266622844671, 1.0665553331192363, 1.580370920796902, 1.0847629324038879, 0.8136079140308521, 1.1679257898924198, 1.2505781866379664, 0.5172329229426306, 1.2715887967036166, 1.1129294264566956, 1.1055361340140737, 0.5895272939544961, 0.5621575887715535, 0.9015195713019988, 1.521218495092461, 0.15621364883261435, 0.3292291108937069, 0.8028122083037629, 1.1136569592827636, 0.5767668008756213, 1.026828172986149, 1.2186719556521246, 0.7490057433781987, 1.477018864311058, 1.4842854941985433, 0.4239646407112976, 0.7229990396785148, 1.5152538641310915, 1.3684662327270902, 0.5139838846289324, 0.8523146395762871, 0.7616375502466394, 0.7218661745323576, 0.8579905775451826, 1.8184282935067144, 0.968585362497049, 1.1103007335811004, 1.8108241913772478, 0.9177318656173244, 0.726086456446213, 1.9211672902045955, 1.255526288860953, 0.5841448383131738, 1.9271894036343995, 1.3746734099515034, 0.4318151746974064, 0.793908777167802, 1.243187873838892, 0.7433437852403643, 0.935964988578669, 0.9831088590580317, 1.3022062257335523, 0.7503298618376854, 0.3568697319812778, 1.4264982889353683, 0.7463139646918489, 0.025038765567554577, 0.6580886934915428, 0.9266155040376641, 1.4772649408434968, 1.0088249531726636, 1.1539376521012925, 0.49761335658374484, 0.5134630887257432, 0.9275531875658151, 1.4882575453955853, 1.7130504594270435, 0.9472944276670263, 0.8885730419559281, 1.4896881948640233, 1.7229258741005058, 1.5650694117675776, 1.010735327410559, 0.9476924325703608, 1.2076138436424104, 0.8219700356127811, 0.22718838159146504, 0.8174748420460222, 1.124801634973362, 1.6717115933361955, 1.0157957783137093, 1.2765208199729017, 1.1649104327394892, 1.0284948693954448, 1.156479935846051, 1.1405820406204341, 0.8919238720474694, 0.9808152465564033, 1.1775992435717368, 1.0816787863108903, 0.2983909530318323, 1.446543831187297, 1.1790713280502314, 1.2563042077563744, 1.5248828548794973, 0.8603500708297273, 1.1142357786249604, 0.9149467316966114, 0.6162465469388293, 0.1151642560502435, 0.7467921891821909, 1.137511406404021, 0.3019600289677298, 1.4596881925477052, 1.094534029491236, 0.8794873601652403, 0.3444518499195395, 0.8619078204116215, 0.611644330352241, 0.8030247776891448, 1.4049725337367862, 0.8477046643811895, 1.2190501211154423, 1.0528877398527794, 1.748674938007417, 1.0117852365987015, 0.09925274488599267, 1.1162878272977192, 1.4058666116389225, 0.3059168511493837, 1.085498110659602, 1.1005055874116785, 1.3418557027108147, 1.4932728455316442, 1.7038191626407708, 0.8975391783008894, 1.6035510957355588, 0.5640846979333675, 0.5479181323110057, 1.9172234764511416, 1.1399692588419517, 1.7045242387195907, 0.8073111491943982, 0.7690638851379002, 1.6299285532300378, 1.678459317060872, 0.723922041724839, 1.0450500151254452, 1.670577797264325, 1.11289397287445, 1.0921324083419042, 1.380708995454336, 0.21381718983840914, 1.2822561167104383, 1.2265099584938417, 1.9618237990179517, 1.1235285578961682, 0.7172654917374147, 1.33121206072689, 1.0903165382969675, 0.7757124343197463, 0.8887458286114053, 0.5354030606654204, 1.4748310202631387, 1.2776653376249483, 0.10409067756235213, 0.4745999715611289, 1.164612911297882, 1.4111158799126269, 0.22003090655488233, 1.3028995216490595, 1.577512976967485, 0.7482929737951091, 0.360917195439749, 0.43228376367626775, 1.167034299940095, 1.0391722414146145, 1.4535954115353835, 1.3123227993264601, 0.5560843128916989, 0.8950925756059468, 1.3357467993413037, 0.593936691645008, 1.1677553458263654, 0.7216862164539637, 1.5261719387263994, 0.7672906772941379, 1.6625604496763235, 1.0873946902749576, 0.9253010058763458, 1.2913404568701967, 1.6749452743342625, 0.33234960071178565, 0.27002875439610474, 1.5453981971967456, 0.9849964747689347, 1.4520999860692207, 1.023317455902176, 1.0113244500302554, 1.7947644386179338, 1.0632532878889724, 0.19027761741417304, 0.9436949116882363, 1.1819011804751645, 0.9646357085647763, 1.0024072276106994, 0.3563028594816212, 1.3204104304840043, 0.6050749748612345, 1.7837663122994694, 0.9741124653165157, 1.5537844997869181, 1.0528418993152746, 1.8256427177317631, 1.4318809808828075, 0.4368326330223209, 1.0557615431834226, 1.5891850309420559, 0.535466026088287, 0.9485168852535344, 0.9528868268639138, 1.2487181299621142, 0.5708872208525728, 1.0324353094943117, 1.2768952582931448, 1.1806425293092917, 0.5470916699600299, 1.2752815707531453, 1.5611521666745682, 1.311990313745912, 0.7187260885272431, 1.272016451543994, 1.2077808430581864, 0.5945013957983932, 0.8195865330697911, 0.47765446065427974, 0.8594464405020183, 1.1916258345050585, 0.9950824600832873, 1.0108036901464372, 0.6839949874370256, 1.0921728081338538, 0.5747662386135555, 1.0319204860576179, 1.08085494331001, 0.684928716494977, 0.6787721827734834, 1.7958086055737583, 0.33589935402977944, 0.4387427121246704, 0.6206481065458996, 1.191551874919698, 0.5907856281623467, 1.338058278815263, 0.9803095803486738, 1.0377869391653802, 1.1533228087010907, 1.0178790759720846, 1.4147577996338971, 1.7099073130832978, 0.13043729171160023, 1.4745242905349265, 1.1861174756907995, 0.8996989432624732, 0.6373964714948436, 0.5960973173272943, 1.1241243001404015, 0.3946740393509123, 0.2663573700316152, 0.6490850947416068, 0.8290435022451491, 0.9217554624918215, 0.7157399836683705, 0.9957166494589649, 0.845464267391801, 1.0198214293982617, 0.8288653553826102, 1.6594206282663713, 0.7107830995156514, 0.9332053527089129, 0.6227821345711477, 1.8231718963937102, 1.0197486797007378, 0.37094265240432933, 0.7318882880505314, 1.3584032663482528, 1.1726416115974374, 0.9765456087534844, 0.515508727781062, 1.42106372014594, 1.064613059652313, 0.743376360278359, 0.977192638144535, 0.7043373380288307, 1.6219827669594915, 1.0056254366003388, 0.6552309867928133, 0.8501722835690856, 1.0829662038949392, 0.31854488945080217, 0.9121788549872126, 1.2932947099204863, 0.7028809562894474, 0.23654276746441272, 0.5370044698902839, 1.8361779620773944, 0.6677358593570663, 0.7520140410142696, 1.004245091032117, 1.5622023492363046, 0.8326482170852579, 1.2533156816471953, 1.3206111742719728, 0.41249846997321105, 0.8898447073715099, 0.40585041568933133, 0.9525771150440467, 0.8721398089836663, 0.3460021068825452, 1.1533561252680915, 1.6584535870961596, 1.0634715643243706, 1.3680199896350014, 1.6650719619103338, 0.5627141485256443, 0.6208932688781136, 1.3071587710053199, 0.1416700137635163, 0.9547222985673014, 1.6989862696455291, 0.2804973705951106, 1.3020645353358884, 0.9801107794734651, 0.9240729549541574, 0.4503631059669889, 0.8127226993186792, 1.3113042029015403, 1.5058748985208155, 0.7593680887908676, 1.5433755838757581, 1.3837720243648923, 0.5778395807417879, 0.30472730521511326, 0.6070970707239934, 1.0782265433572935, 0.617560817138614, 1.2140212302831541, 1.3757022961208065, 0.7133528797637423, 1.298918093599398, 0.9009395011968638, 1.8421089459275528, 0.9383047309729783, 0.9993180918762333, 1.3921061133334178, 0.8360665829753785, 0.7578343486840561, 1.29357217902789, 1.1292634287484076, 1.5032679652674021, 0.2860437927261704, 0.5825480548392632, 0.27075063853379866, 0.9961090549387308, 0.7658555165470118, 0.9963127230119017, 1.153836657566321, 0.5582241134547475, 1.172233878982302, 0.44778354378091634, 0.9362190081064267, 1.017625095280925, 1.6469656146551508, 0.9530816873008614, 1.3362141927453721, 1.0780115177747769, 1.2390291934635476, 1.6866322162390213, 1.286074599891161, 0.23083372407462954, 0.3361946461282025, 0.768451072394043, 1.5455925327729938, 0.8559676110541327, 0.7679180257962205, 0.8243554909638229, 1.1843599232637914, 0.9076773967146894, 0.7664341598118981, 1.152091165376254, 1.1916036817614581, 1.2481496622084058, 1.4035173296901617, 0.09777459192980975, 0.8597542308347942, 0.3440797089134502, 0.3425533366163732, 1.8935186632125454, 1.2394900111425886, 0.8750582068401028, 1.0263615391199226, 1.1366009947976135, 0.5922044339956318, 0.628306186583761, 1.8191183264482582, 1.2047588739752042, 0.27097180976413915, 1.291172259053235, 0.9488830405946108, 0.06138803103209001, 1.1855235594474935, 1.4518716831827558, 0.8749960220808065, 0.3718776071322465, 0.15296357166678032, 0.7093771343903228, 1.2703846629061624, 0.11896700455859555, 0.6322807805699668, 1.268503673090482, 0.937316332264034, 1.0599969691447924, 1.3013601248918212, 0.8669517683052644, 1.2271433600109387, 0.40150369081253723, 0.6541869073085391, 1.5812361107576542, 0.8078352702530114, 1.6776192384679764, 0.9928589189459276, 1.1774709189706631, 0.49662497545604367, 0.3791556725405554, 1.0260704664071738, 1.6226982758369766, 1.605044899457809, 1.5060806487574894, 1.3473653601751947, 1.3719849849936465, 0.9046009624400265, 1.113282417079121, 1.2290963446140841, 0.9166787691187973, 0.8585400612280851, 1.1330081258521025, 1.2744828867004463, 0.604149274319127, 0.04213591955865126, 0.6560163548340946, 0.5310449870271383, 1.8139355886451356, 0.32604256845775437, 1.0939074078826092, 0.9213140107354965, 1.0613206633589045, 1.5271260395404664, 0.7843675589127213, 0.41043090324628984, 1.1069088899551454, 0.916397627958068, 0.7570030576971741, 0.28451474334897275, 0.9778211575417696, 1.539378093955803, 1.2867414344151455, 0.7720044042830703, 0.8036210978991808, 0.8768753093283082, 0.9307650182820922, 0.6017997706477368, 0.4930388475719314, 1.3760406543504864, 1.4685231076831355, 0.9548072684623707, 0.7303103228662753, 1.5193758464979787, 1.3291851909519399, 1.6988931442342747, 0.9827158352423898, 0.6204010327497481, 1.3395139451447133, 0.20754194437578366, 0.45118101075269323, 1.7424745671885937, 1.5154668463592942, 1.6677029326727464, 0.8760780150064849, 0.6964432837215943, 0.6466503653170323, 0.7268940775552369, 1.2337361218161975, 1.0595816228398791, 1.2432734373100658, 1.1691544106256833, 1.4341847294447665, 0.8185049487634138, 1.206536521188565, 1.305774933547971, 0.6890698402965197, 1.8057285495755946, 1.371095567676142, 0.3217437709766321, 1.5203510569507914, 1.2618509852942972, 0.9546290505590352, 1.5498639534959517, 1.0609203188943028, 0.8406809775568864, 0.5634692900648305, 0.9641999004540313, 0.950955022439398, 1.0126887093805692, 1.2362507500172115, 1.0674608239360859, 0.4671064197175574, 1.7574789740402548, 1.5936177252946613, 1.5262966938540186, 0.7522890205018901, 1.6822986132507962, 0.9521553542321253, 1.2409798217380805, 0.7518217139192954, 1.2604053773595552, 1.3575591792988133, 1.1034946907138004, 1.6553828618734852, 1.4430844992517726, 1.3896852140039015, 0.33669094796275123, 0.7256029339290792, 0.5978115451197892, 1.6345094705563898, 1.098846118098252, 1.6042090580848707, 0.6486197698478454, 1.0537458648264724, 1.4871725653234544, 0.8780915487299088, 1.3503220802963558, 0.8497657312752449, 1.5985441645246807, 0.9554230119290573, 1.5751098223462603, 1.5413513207297456, 1.864230954673483, 1.0968292220769167, 0.4231336880675193, 0.715121896573262, 0.7560785198415086, 0.3916157622425872, 0.4813022699042798, 0.6631392354487036, 1.3360994219216644, 0.8233955412107452, 0.3266898024195838, 0.9707305481948241, 1.4126005038594625, 0.7219727270203533, 1.239799348805816, 1.0342943979108856, 1.1016829494684477, 0.6724215557109912, 1.9688696855189356, 1.7052857980556801, 0.7882736782287548, 0.9516813625191675, 1.1714252663770046, 0.1983296129649954, 0.6199874516275886, 1.0929852436214407, 0.7272705202917563, 1.1933129193503733, 0.4044021021065517, 1.1821015446749876, 0.6424281862962791, 1.3792825608488997, 0.5988184712201394, 1.2753172606094498, 1.3821985014704166, 1.3245241943318224, 0.5893028292697725, 1.8490787064277219, 1.5373636116635692, 1.0634824079575602, 1.2138493633000356, 1.2485812134152106, 0.8396754460654092, 0.13569710388808187, 0.48219814807466155, 0.9319834001310333, 1.042579405672706, 1.3998493909479313, 0.816898516849061, 0.7496232938080494, 0.4390782914483453, 1.3051747709594597, 0.6834593156108381, 1.1373520138694118, 0.930277674734766, 1.1499995833325918, 0.9747977885641635, 1.3113932585470685, 0.6594193075421791, 0.8851158791037514, 1.5709312444472756, 1.180015009724586, 0.8388865531532258, 1.444605682257963, 0.772644160176738, 1.160519667236934, 1.6518683432103676, 1.011881223620073, 0.4602317089035005, 1.605275113879265, 0.9785836409903146, 0.7183409412355918, 0.7240023964434642, 0.8069942591069451, 0.848609747474426, 1.2161814858622286, 0.2380543700866644, 1.0463745650369154, 0.6518314868944769, 1.3353195658027028, 0.536018067436691, 0.873407830238836, 1.3088355960020603, 1.180805618555965, 0.8095932490341667, 0.9577072364795068, 1.2596951866876767, 1.0039178198453018, 0.833393852172298, 0.3872561560593303, 1.2810826864664242, 0.7466111024247024, 0.8619301978047699, 1.1365794341434672, 0.5193642685620291, 0.6413227227942541, 1.2932174657492677, 0.9461469349755092, 1.2679846375941866, 1.1274449149068475, 1.5473222998429605, 0.707572013096146, 0.32976869950221566, 0.8288997854161629, 0.37160470605020823, 1.155619442159459, 1.002254871870536, 0.919671070833335, 1.0159884863168158, 0.5916457243095763, 0.4652991237998907, 0.7952005865943527, 0.5746731201033065, 0.15993029136736292, 0.07896529677092046, 0.238851749384264, 0.5823859123760231, 0.8419734988510202, 0.7795769143034944, 0.7183353978667145, 0.14330364557696917, 1.154353277254349, 1.0665700897836996, 0.5517631845868344, 1.7778637598773233, 0.6977304208539423, 0.8581887805481812, 0.3276610634238375, 0.8617617817134594, 1.3386715010305013, 0.8278936009378179, 0.6023491948008943, 1.4351646194347967, 0.7418385452365969, 0.4704351734226545, 1.8592895650159178, 1.1407410896772758, 1.1529549533308643, 1.3371700346072304, 0.7195762783696457, 0.4678625636345307, 1.3509735127142855, 0.6128830338532043, 1.1228282297636505, 1.5821556476200218, 1.3393130239512083, 1.3886291030465696, 1.2595780418367437, 1.5061241861012258, 0.21093398680613962, 0.813681524392799, 1.6782240986290824, 0.9013561889101332, 1.0695096328339182, 1.8311252291145244, 0.4381051879579324, 1.3752476908316753, 0.07780428338960543, 1.8391146181071027, 1.162507144809653, 1.5702661892667151, 1.289698994700693, 0.7586488888480281, 0.9488739238067784, 1.1729371662132535, 1.0900663027603616, 0.7037642721930079, 0.7242571230947347, 0.5138759921455044, 1.276281964485901, 0.6188419117255499, 0.6910725569776877, 0.6111237687430621, 0.957420027904532, 0.3344673731343757, 1.2749551497937142, 0.5161346375630277, 0.669666387380999, 1.158690680456067, 0.7327149183141739, 1.546308415603925, 1.0003547793672842, 0.2630540036503003, 1.8857124342766043, 1.0487647686388253, 1.3845187348134258, 0.7322135975772348, 0.8777657421577839, 0.9670250477273902, 0.7136845471451373, 1.4621694575992836, 1.0191311878117284, 0.8142358581709577, 1.086484538046805, 0.8802426575671713, 1.2330330602717865, 0.844258469081715, 0.9818685873099879, 1.081217814390083, 1.1824159107856553, 0.30370243948296805, 1.0188810715417016, 1.6397643015664922, 1.2935711711980191, 0.9003028884836122, 0.4296591188562917, 0.8489467108039611, 0.7379600515414215, 0.7533774719791428, 1.0038107751326968, 1.500117921983855, 1.079805143171819, 0.3457351765546629, 0.43296586757032907, 1.5982202501662885, 1.1337880190872935, 1.6115747616459262, 1.260711827860856, 1.9440527350644747, 1.0137967532908145, 1.2687475728787565, 1.3448279448485851, 1.1053181859839736, 1.7261979145774529, 1.1926065849538947, 0.9343308981853065, 0.19430397550284362, 0.9090218978373004, 0.6319984510253326, 1.1590967290573992, 1.3685891659625948, 0.4835779208014592, 1.073822448266561, 1.0944514763793491, 0.689693387066232, 1.2544319079128579, 1.019044227421395, 0.573541173455945, 0.8039617832812939, 1.2011875352442394, 0.5146774113823851, 0.9895336299929607, 1.1903408594023799, 0.9858224627882686, 1.3172328495570431, 1.2701081815811304]
Stop the runtime
[8]:
ipycompss.stop(sync=True)
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Synchronizing all future objects left on the user scope.
****************************************************
PyCOMPSs: Other decorators - Binary
In this example we will how to invoke binaries as tasks with PyCOMPSs.
Import the PyCOMPSs library
[1]:
import pycompss.interactive as ipycompss
Start the runtime
Initialize COMPSs runtime Parameters indicates if the execution will generate task graph, tracefile, monitor interval and debug information.
[2]:
import os
if 'BINDER_SERVICE_HOST' in os.environ:
ipycompss.start(debug=False,
project_xml='../xml/project.xml',
resources_xml='../xml/resources.xml')
else:
ipycompss.start(graph=True, monitor=1000, trace=True, debug=True)
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
* .-~~-.--. _____ ________ *
* : ) |____ \ |____ / *
* .~ ~ -.\ /.- ~~ . ___) | / / *
* > `. .' < / ___/ / / *
* ( .- -. ) | |___ _ / / *
* `- -.-~ `- -' ~-.- -' |_____| |_| /__/ *
* ( : ) _ _ .-: *
* ~--. : .--~ .-~ .-~ } *
* ~-.-^-.-~ \_ .~ .-~ .~ *
* \ \ ' \ '_ _ -~ *
* \`.\`. // *
* . - ~ ~-.__\`.\`-.// *
* .-~ . - ~ }~ ~ ~-.~-. *
* .' .-~ .-~ :/~-.~-./: *
* /_~_ _ . - ~ ~-.~-._ *
* ~-.< *
******************************************************
* - Starting COMPSs runtime... *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun! *
******************************************************
Importing task and binary modules
Import task module before annotating functions or methods
[3]:
from pycompss.api.task import task
from pycompss.api.binary import binary
from pycompss.api.parameter import *
Declaring tasks
Declare functions and decorate with @task those that should be tasks and with @binary the ones that execute a binary file
[4]:
@binary(binary="sed")
@task(file=FILE_INOUT)
def sed(flag, expression, file):
# Equivalent to: $ sed flag expression file
pass
Task definition detected.
Found task: sed
[5]:
@binary(binary="grep")
@task(infile={Type:FILE_IN, StdIOStream:STDIN}, result={Type:FILE_OUT, StdIOStream:STDOUT})
def grep(keyword, infile, result):
# Equivalent to: $ grep keyword < infile > result
pass
Task definition detected.
Found task: grep
Invoking tasks
[6]:
from pycompss.api.api import compss_open
finout = "inoutfile.txt"
with open(finout, 'w') as finout_d:
finout_d.write("Hi, this a simple test!")
finout_d.write("\nHow are you?")
sed('-i', 's/Hi/Hello/g', finout)
fout = "outfile.txt"
grep("Hello", finout, fout)
Accessing data outside tasks requires synchronization
[7]:
# Check the result of 'sed'
with compss_open(finout, "r") as finout_r:
sedresult = finout_r.read()
print(sedresult)
Hello, this a simple test!
How are you?
[8]:
# Check the result of 'grep'
with compss_open(fout, "r") as fout_r:
grepresult = fout_r.read()
print(grepresult)
Hello, this a simple test!
Stop the runtime
[9]:
ipycompss.stop(sync=True)
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Synchronizing all future objects left on the user scope.
****************************************************
PyCOMPSs: Integration with Numba
In this example we will how to use Numba with PyCOMPSs.
Import the PyCOMPSs library
[1]:
import pycompss.interactive as ipycompss
Starting runtime
Initialize COMPSs runtime Parameters indicates if the execution will generate task graph, tracefile, monitor interval and debug information.
[2]:
import os
if 'BINDER_SERVICE_HOST' in os.environ:
ipycompss.start(graph=True, debug=False,
project_xml='../xml/project.xml',
resources_xml='../xml/resources.xml')
else:
ipycompss.start(graph=True, monitor=1000, trace=True, debug=False)
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
* .-~~-.--. _____ ________ *
* : ) |____ \ |____ / *
* .~ ~ -.\ /.- ~~ . ___) | / / *
* > `. .' < / ___/ / / *
* ( .- -. ) | |___ _ / / *
* `- -.-~ `- -' ~-.- -' |_____| |_| /__/ *
* ( : ) _ _ .-: *
* ~--. : .--~ .-~ .-~ } *
* ~-.-^-.-~ \_ .~ .-~ .~ *
* \ \ ' \ '_ _ -~ *
* \`.\`. // *
* . - ~ ~-.__\`.\`-.// *
* .-~ . - ~ }~ ~ ~-.~-. *
* .' .-~ .-~ :/~-.~-./: *
* /_~_ _ . - ~ ~-.~-._ *
* ~-.< *
******************************************************
* - Starting COMPSs runtime... *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun! *
******************************************************
Importing task and arguments directionality modules
Import task module before annotating functions or methods
[3]:
from pycompss.api.task import task
from pycompss.api.parameter import *
from pycompss.api.api import compss_barrier
from pycompss.api.api import compss_wait_on
Importing other modules
Import the time and numpy modules
[4]:
import time
import numpy as np
Declaring tasks
Declare functions and decorate with @task those that should be tasks – Note that they are exactly the same but the “numba” parameter in the @task decorator
[5]:
@task(returns=1) # Default: numba=False
def ident_loops(x):
r = np.empty_like(x)
n = len(x)
for i in range(n):
r[i] = np.cos(x[i]) ** 2 + np.sin(x[i]) ** 2
return r
Found task: ident_loops
[6]:
@task(returns=1, numba=True)
def ident_loops_jit(x):
r = np.empty_like(x)
n = len(x)
for i in range(n):
r[i] = np.cos(x[i]) ** 2 + np.sin(x[i]) ** 2
return r
Found task: ident_loops_jit
Invoking tasks
[7]:
size = 100000
ntasks = 8
# Run some tasks without numba jit
start = time.time()
for i in range(ntasks):
out = ident_loops(np.arange(size))
compss_barrier()
end = time.time()
# Run some tasks with numba jit
start_jit = time.time()
for i in range(ntasks):
out_jit = ident_loops_jit(np.arange(size))
compss_barrier()
end_jit = time.time()
# Get the last result of each run to compare that the results are ok
out = compss_wait_on(out)
out_jit = compss_wait_on(out_jit)
print("TIMING RESULTS:")
print("* ident_loops : %s seconds" % str(end - start))
print("* ident_loops_jit : %s seconds" % str(end_jit - start_jit))
if len(out) == len(out_jit) and list(out) == list(out_jit):
print("* SUCCESS: Results match.")
else:
print("* FAILURE: Results are different!!!")
TIMING RESULTS:
* ident_loops : 4.26820993423 seconds
* ident_loops_jit : 0.774371147156 seconds
* SUCCESS: Results match.
Stop the runtime
[8]:
ipycompss.stop(sync=False)
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Warning: some of the variables used with PyCOMPSs may
have not been brought to the master.
****************************************************
Dislib tutorial
This tutorial will show the basics of using dislib.
Setup
First, we need to start an interactive PyCOMPSs session:
[1]:
import pycompss.interactive as ipycompss
import os
if 'BINDER_SERVICE_HOST' in os.environ:
ipycompss.start(graph=True,
project_xml='../xml/project.xml',
resources_xml='../xml/resources.xml')
else:
ipycompss.start(graph=True, monitor=1000)
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
* .-~~-.--. _____ ________ *
* : ) |____ \ |____ / *
* .~ ~ -.\ /.- ~~ . ___) | / / *
* > `. .' < / ___/ / / *
* ( .- -. ) | |___ _ / / *
* `- -.-~ `- -' ~-.- -' |_____| |_| /__/ *
* ( : ) _ _ .-: *
* ~--. : .--~ .-~ .-~ } *
* ~-.-^-.-~ \_ .~ .-~ .~ *
* \ \ ' \ '_ _ -~ *
* \`.\`. // *
* . - ~ ~-.__\`.\`-.// *
* .-~ . - ~ }~ ~ ~-.~-. *
* .' .-~ .-~ :/~-.~-./: *
* /_~_ _ . - ~ ~-.~-._ *
* ~-.< *
******************************************************
* - Starting COMPSs runtime... *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun! *
******************************************************
Next, we import dislib and we are all set to start working!
[2]:
import dislib as ds
Distributed arrays
The main data structure in dislib is the distributed array (or ds-array). These arrays are a distributed representation of a 2-dimensional array that can be operated as a regular Python object. Usually, rows in the array represent samples, while columns represent features.
To create a random array we can run the following NumPy-like command:
[3]:
x = ds.random_array(shape=(500, 500), block_size=(100, 100))
print(x.shape)
x
(500, 500)
[3]:
ds-array(blocks=(...), top_left_shape=(100, 100), reg_shape=(100, 100), shape=(500, 500), sparse=False)
Now x is a 500x500 ds-array of random numbers stored in blocks of 100x100 elements. Note that x is not stored in memory. Instead, random_array generates the contents of the array in tasks that are usually executed remotely. This allows the creation of really big arrays.
The content of x is a list of Futures that represent the actual data (wherever it is stored).
To see this, we can access the _blocks field of x:
[4]:
x._blocks[0][0]
[4]:
<pycompss.runtime.binding.Future at 0x7fb45006c4a8>
block_size is useful to control the granularity of dislib algorithms.
To retrieve the actual contents of x, we use collect, which synchronizes the data and returns the equivalent NumPy array:
[5]:
x.collect()
[5]:
array([[0.64524889, 0.69250058, 0.97374899, ..., 0.36928149, 0.75461806,
0.98784805],
[0.61753454, 0.092557 , 0.4967433 , ..., 0.87482708, 0.44454572,
0.0022951 ],
[0.78473344, 0.15288068, 0.70022708, ..., 0.82488172, 0.16980005,
0.30608108],
...,
[0.16257112, 0.94326181, 0.26206143, ..., 0.49725598, 0.80564738,
0.69616444],
[0.25089352, 0.10652958, 0.79657793, ..., 0.86936011, 0.67382938,
0.78140887],
[0.17716041, 0.24354163, 0.52866266, ..., 0.12053584, 0.9071268 ,
0.55058659]])
Another way of creating ds-arrays is using array-like structures like NumPy arrays or lists:
[6]:
x1 = ds.array([[1, 2, 3], [4, 5, 6]], block_size=(1, 3))
x1
[6]:
ds-array(blocks=(...), top_left_shape=(1, 3), reg_shape=(1, 3), shape=(2, 3), sparse=False)
Distributed arrays can also store sparse data in CSR format:
[7]:
from scipy.sparse import csr_matrix
sp = csr_matrix([[0, 0, 1], [1, 0, 1]])
x_sp = ds.array(sp, block_size=(1, 3))
x_sp
[7]:
ds-array(blocks=(...), top_left_shape=(1, 3), reg_shape=(1, 3), shape=(2, 3), sparse=True)
In this case, collect returns a CSR matrix as well:
[8]:
x_sp.collect()
[8]:
<2x3 sparse matrix of type '<class 'numpy.int64'>'
with 3 stored elements in Compressed Sparse Row format>
Loading data
A typical way of creating ds-arrays is to load data from disk. Dislib currently supports reading data in CSV and SVMLight formats like this:
[9]:
x, y = ds.load_svmlight_file("./files/libsvm/1", block_size=(20, 100), n_features=780, store_sparse=True)
print(x)
csv = ds.load_txt_file("./files/csv/1", block_size=(500, 122))
print(csv)
ds-array(blocks=(...), top_left_shape=(20, 100), reg_shape=(20, 100), shape=(61, 780), sparse=True)
ds-array(blocks=(...), top_left_shape=(500, 122), reg_shape=(500, 122), shape=(4235, 122), sparse=False)
Slicing
Similar to NumPy, ds-arrays support the following types of slicing:
(Note that slicing a ds-array creates a new ds-array)
[10]:
x = ds.random_array((50, 50), (10, 10))
Get a single row:
[11]:
x[4]
[11]:
ds-array(blocks=(...), top_left_shape=(10, 10), reg_shape=(10, 10), shape=(1, 50), sparse=False)
Get a single element:
[12]:
x[2, 3]
[12]:
ds-array(blocks=(...), top_left_shape=(1, 1), reg_shape=(1, 1), shape=(1, 1), sparse=False)
Get a set of rows or a set of columns:
[13]:
# Consecutive rows
print(x[10:20])
# Consecutive columns
print(x[:, 10:20])
# Non consecutive rows
print(x[[3, 7, 22]])
# Non consecutive columns
print(x[:, [5, 9, 48]])
ds-array(blocks=(...), top_left_shape=(10, 10), reg_shape=(10, 10), shape=(10, 50), sparse=False)
ds-array(blocks=(...), top_left_shape=(10, 10), reg_shape=(10, 10), shape=(50, 10), sparse=False)
ds-array(blocks=(...), top_left_shape=(10, 10), reg_shape=(10, 10), shape=(3, 50), sparse=False)
ds-array(blocks=(...), top_left_shape=(10, 10), reg_shape=(10, 10), shape=(50, 3), sparse=False)
Get any set of elements:
[14]:
x[0:5, 40:45]
[14]:
ds-array(blocks=(...), top_left_shape=(10, 10), reg_shape=(10, 10), shape=(5, 5), sparse=False)
Other functions
Apart from this, ds-arrays also provide other useful operations like transpose and mean:
[15]:
x.mean(axis=0).collect()
[15]:
array([0.551327 , 0.49044018, 0.47707326, 0.505077 , 0.48609203,
0.5046837 , 0.48576743, 0.50085461, 0.59721606, 0.52625186,
0.54670041, 0.51213254, 0.4790549 , 0.56172745, 0.54560812,
0.52904012, 0.54524971, 0.52370167, 0.44268389, 0.58325103,
0.4754059 , 0.4990062 , 0.55135695, 0.53576298, 0.44728801,
0.57918722, 0.46289081, 0.48786927, 0.46635653, 0.53229369,
0.50648353, 0.53486796, 0.5159688 , 0.50703502, 0.46818427,
0.59886016, 0.48181209, 0.51001161, 0.46914881, 0.57437929,
0.52491673, 0.49711231, 0.50817903, 0.50102322, 0.42250693,
0.51456321, 0.53393824, 0.47924624, 0.49860827, 0.49424366])
[16]:
x.transpose().collect()
[16]:
array([[0.75899854, 0.83108977, 0.28083382, ..., 0.65348721, 0.10747938,
0.13453309],
[0.70515755, 0.22656129, 0.60863163, ..., 0.10640133, 0.3311688 ,
0.50884584],
[0.71224037, 0.95907871, 0.6010006 , ..., 0.41099068, 0.5671029 ,
0.06170055],
...,
[0.39789773, 0.69988175, 0.93784369, ..., 0.24439267, 0.45685381,
0.93017544],
[0.22410234, 0.13491992, 0.75906239, ..., 0.96917569, 0.96204333,
0.14629864],
[0.81496796, 0.96925576, 0.58510411, ..., 0.65520011, 0.05744591,
0.78974985]])
Machine learning with dislib
Dislib provides an estimator-based API very similar to scikit-learn. To run an algorithm, we first create an estimator. For example, a K-means estimator:
[17]:
from dislib.cluster import KMeans
km = KMeans(n_clusters=3)
Now, we create a ds-array with some blob data, and fit the estimator:
[18]:
from sklearn.datasets import make_blobs
# create ds-array
x, y = make_blobs(n_samples=1500)
x_ds = ds.array(x, block_size=(500, 2))
km.fit(x_ds)
[18]:
KMeans(arity=50, init='random', max_iter=10, n_clusters=3,
random_state=<mtrand.RandomState object at 0x7fb51401f9d8>, tol=0.0001,
verbose=False)
Finally, we can make predictions on new (or the same) data:
[19]:
y_pred = km.predict(x_ds)
y_pred
[19]:
ds-array(blocks=(...), top_left_shape=(500, 1), reg_shape=(500, 1), shape=(1500, 1), sparse=False)
y_pred is a ds-array of predicted labels for x_ds
Let’s plot the results
[20]:
%matplotlib inline
import matplotlib.pyplot as plt
centers = km.centers
# set the color of each sample to the predicted label
plt.scatter(x[:, 0], x[:, 1], c=y_pred.collect())
# plot the computed centers in red
plt.scatter(centers[:, 0], centers[:, 1], c='red')
[20]:
<matplotlib.collections.PathCollection at 0x7fb42b4fc518>
Note that we need to call y_pred.collect() to retrieve the actual labels and plot them. The rest is the same as if we were using scikit-learn.
Now let’s try a more complex example that uses some preprocessing tools.
First, we load a classification data set from scikit-learn into ds-arrays.
Note that this step is only necessary for demonstration purposes. Ideally, your data should be already loaded in ds-arrays.
[21]:
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
x, y = load_breast_cancer(return_X_y=True)
x_train, x_test, y_train, y_test = train_test_split(x, y)
x_train = ds.array(x_train, block_size=(100, 10))
y_train = ds.array(y_train.reshape(-1, 1), block_size=(100, 1))
x_test = ds.array(x_test, block_size=(100, 10))
y_test = ds.array(y_test.reshape(-1, 1), block_size=(100, 1))
Next, we can see how support vector machines perform in classifying the data. We first fit the model (ignore any warnings in this step):
[22]:
from dislib.classification import CascadeSVM
csvm = CascadeSVM()
csvm.fit(x_train, y_train)
/usr/lib/python3.6/site-packages/dislib-0.4.0-py3.6.egg/dislib/classification/csvm/base.py:374: RuntimeWarning: overflow encountered in exp
k = np.exp(k)
/usr/lib/python3.6/site-packages/dislib-0.4.0-py3.6.egg/dislib/classification/csvm/base.py:342: RuntimeWarning: invalid value encountered in double_scalars
delta = np.abs((w - self._last_w) / self._last_w)
[22]:
CascadeSVM(c=1, cascade_arity=2, check_convergence=True, gamma='auto',
kernel='rbf', max_iter=5, random_state=None, tol=0.001,
verbose=False)
and now we can make predictions on new data using csvm.predict(), or we can get the model accuracy on the test set with:
[23]:
score = csvm.score(x_test, y_test)
score represents the classifier accuracy, however, it is returned as a Future. We need to synchronize to get the actual value:
[24]:
from pycompss.api.api import compss_wait_on
print(compss_wait_on(score))
0.5944055944055944
The accuracy should be around 0.6, which is not very good. We can scale the data before classification to improve accuracy. This can be achieved using dislib’s StandardScaler.
The StandardScaler provides the same API as other estimators. In this case, however, instead of making predictions on new data, we transform it:
[25]:
from dislib.preprocessing import StandardScaler
sc = StandardScaler()
# fit the scaler with train data and transform it
scaled_train = sc.fit_transform(x_train)
# transform test data
scaled_test = sc.transform(x_test)
Now scaled_train and scaled_test are the scaled samples. Let’s see how SVM perfroms now.
[26]:
csvm.fit(scaled_train, y_train)
score = csvm.score(scaled_test, y_test)
print(compss_wait_on(score))
0.972027972027972
The new accuracy should be around 0.9, which is a great improvement!
Close the session
To finish the session, we need to stop PyCOMPSs:
[27]:
ipycompss.stop()
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Warning: some of the variables used with PyCOMPSs may
have not been brought to the master.
****************************************************
Machine Learning with dislib
This tutorial will show the different algorithms available in dislib.
Setup
First, we need to start an interactive PyCOMPSs session:
[1]:
import pycompss.interactive as ipycompss
import os
if 'BINDER_SERVICE_HOST' in os.environ:
ipycompss.start(graph=True,
project_xml='../xml/project.xml',
resources_xml='../xml/resources.xml')
else:
ipycompss.start(graph=True, monitor=1000)
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
* .-~~-.--. _____ ________ *
* : ) |____ \ |____ / *
* .~ ~ -.\ /.- ~~ . ___) | / / *
* > `. .' < / ___/ / / *
* ( .- -. ) | |___ _ / / *
* `- -.-~ `- -' ~-.- -' |_____| |_| /__/ *
* ( : ) _ _ .-: *
* ~--. : .--~ .-~ .-~ } *
* ~-.-^-.-~ \_ .~ .-~ .~ *
* \ \ ' \ '_ _ -~ *
* \`.\`. // *
* . - ~ ~-.__\`.\`-.// *
* .-~ . - ~ }~ ~ ~-.~-. *
* .' .-~ .-~ :/~-.~-./: *
* /_~_ _ . - ~ ~-.~-._ *
* ~-.< *
******************************************************
* - Starting COMPSs runtime... *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun! *
******************************************************
Next, we import dislib and we are all set to start working!
[2]:
import dislib as ds
Load the MNIST dataset
[3]:
x, y = ds.load_svmlight_file('/tmp/mnist/mnist', # Download the dataset
block_size=(10000, 784), n_features=784, store_sparse=False)
[4]:
x.shape
[4]:
(60000, 784)
[5]:
y.shape
[5]:
(60000, 1)
[6]:
y_array = y.collect()
y_array
[6]:
array([5., 0., 4., ..., 5., 6., 8.])
[7]:
img = x[0].collect().reshape(28,28)
[8]:
%matplotlib inline
import matplotlib.pyplot as plt
plt.imshow(img)
[8]:
<matplotlib.image.AxesImage at 0x7f6fab182fd0>
[9]:
int(y[0].collect())
[9]:
5
dislib algorithms
Preprocessing
[10]:
from dislib.preprocessing import StandardScaler
from dislib.decomposition import PCA
Clustering
[11]:
from dislib.cluster import KMeans
from dislib.cluster import DBSCAN
from dislib.cluster import GaussianMixture
Classification
[12]:
from dislib.classification import CascadeSVM
from dislib.classification import RandomForestClassifier
Recommendation
[13]:
from dislib.recommendation import ALS
Model selection
[14]:
from dislib.model_selection import GridSearchCV
Others
[15]:
from dislib.regression import LinearRegression
from dislib.neighbors import NearestNeighbors
Examples
KMeans
[16]:
kmeans = KMeans(n_clusters=10)
pred_clusters = kmeans.fit_predict(x).collect()
Get the number of images of each class in the cluster 0:
[17]:
from collections import Counter
Counter(y_array[pred_clusters==0])
[17]:
Counter({2.0: 4006,
6.0: 244,
3.0: 148,
4.0: 20,
8.0: 36,
1.0: 14,
7.0: 22,
9.0: 16,
0.0: 9,
5.0: 7})
GaussianMixture
Fit the GaussianMixture with the painted pixels of a single image:
[18]:
import numpy as np
img_filtered_pixels = np.stack([np.array([i, j]) for i in range(28) for j in range(28) if img[i,j] > 10])
img_pixels = ds.array(img_filtered_pixels, block_size=(50,2))
gm = GaussianMixture(n_components=7, random_state=0)
gm.fit(img_pixels)
Get the parameters that define the Gaussian components:
[19]:
from pycompss.api.api import compss_wait_on
means = compss_wait_on(gm.means_)
covariances = compss_wait_on(gm.covariances_)
weights = compss_wait_on(gm.weights_)
Use the Gaussian mixture model to sample random pixels replicating the original distribution:
[20]:
samples = np.concatenate([np.random.multivariate_normal(means[i], covariances[i], int(weights[i]*1000))
for i in range(7)])
plt.scatter(samples[:,1], samples[:,0])
plt.gca().set_aspect('equal', adjustable='box')
plt.gca().invert_yaxis()
plt.draw()
PCA
[21]:
pca = PCA()
pca.fit(x)
[21]:
PCA(arity=50, n_components=None)
Calculate the explained variance of the 10 first eigenvectors:
[22]:
sum(pca.explained_variance_[0:10])/sum(pca.explained_variance_)
[22]:
0.48814980354934007
Show the weights of the first eigenvector:
[23]:
plt.imshow(np.abs(pca.components_[0]).reshape(28,28))
[23]:
<matplotlib.image.AxesImage at 0x7f6f9dd987f0>
RandomForestClassifier
[24]:
rf = RandomForestClassifier(n_estimators=5, max_depth=3)
rf.fit(x, y)
[24]:
RandomForestClassifier(distr_depth='auto', hard_vote=False, max_depth=3,
n_estimators=5, random_state=None,
sklearn_max=100000000.0, try_features='sqrt')
Use the test dataset to get an accuracy score:
[25]:
x_test, y_test = ds.load_svmlight_file('/tmp/mnist/mnist.test', block_size=(10000, 784), n_features=784, store_sparse=False)
score = rf.score(x_test, y_test)
print(compss_wait_on(score))
0.6359
Close the session
To finish the session, we need to stop PyCOMPSs:
[26]:
ipycompss.stop()
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Warning: some of the variables used with PyCOMPSs may
have not been brought to the master.
****************************************************
Hands-on
Here you will find the hands on notebooks used in the tutorials.
Sort by Key
Algorithm that sorts the elements of a set of files and merges the partial results respecting the order.
First of all - Create a dataset
This step can be avoided if the dataset already exists.
If not, this code snipped creates a set of files with dictionary on each one generated randomly. Uses pickle.
[1]:
def datasetGenerator(directory, numFiles, numPairs):
import random
import pickle
import os
if os.path.exists(directory):
print("Dataset directory already exists... Removing")
import shutil
shutil.rmtree(directory)
os.makedirs(directory)
for f in range(numFiles):
fragment = {}
while len(fragment) < numPairs:
fragment[random.random()] = random.randint(0, 1000)
filename = 'file_' + str(f) + '.data'
with open(directory + '/' + filename, 'wb') as fd:
pickle.dump(fragment, fd)
print('File ' + filename + ' has been created.')
[2]:
numFiles = 2
numPairs = 10
directoryName = 'mydataset'
datasetGenerator(directoryName, numFiles, numPairs)
Dataset directory already exists... Removing
File file_0.data has been created.
File file_1.data has been created.
[3]:
# Show the files that have been created
%ls -l $directoryName
total 8
-rw-r--r-- 1 user users 268 may 28 12:58 file_0.data
-rw-r--r-- 1 user users 266 may 28 12:58 file_1.data
Algorithm definition
[4]:
import pycompss.interactive as ipycompss
[5]:
import os
if 'BINDER_SERVICE_HOST' in os.environ:
ipycompss.start(graph=True,
project_xml='../xml/project.xml',
resources_xml='../xml/resources.xml')
else:
ipycompss.start(graph=True, monitor=1000)
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
* .-~~-.--. _____ ________ *
* : ) |____ \ |____ / *
* .~ ~ -.\ /.- ~~ . ___) | / / *
* > `. .' < / ___/ / / *
* ( .- -. ) | |___ _ / / *
* `- -.-~ `- -' ~-.- -' |_____| |_| /__/ *
* ( : ) _ _ .-: *
* ~--. : .--~ .-~ .-~ } *
* ~-.-^-.-~ \_ .~ .-~ .~ *
* \ \ ' \ '_ _ -~ *
* \`.\`. // *
* . - ~ ~-.__\`.\`-.// *
* .-~ . - ~ }~ ~ ~-.~-. *
* .' .-~ .-~ :/~-.~-./: *
* /_~_ _ . - ~ ~-.~-._ *
* ~-.< *
******************************************************
* - Starting COMPSs runtime... *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun! *
******************************************************
[6]:
from pycompss.api.task import task
from pycompss.api.parameter import FILE_IN
[7]:
@task(returns=list, dataFile=FILE_IN)
def sortPartition(dataFile):
'''
Reads the dataFile and sorts its content which is assumed to be a dictionary {K: V}
:param path: file that contains the data
:return: a list of (K, V) pairs sorted.
'''
import pickle
import operator
with open(dataFile, 'rb') as f:
data = pickle.load(f)
# res = sorted(data, key=lambda (k, v): k, reverse=not ascending)
partition_result = sorted(data.items(), key=operator.itemgetter(0), reverse=False)
return partition_result
Found task: sortPartition
[8]:
@task(returns=list, priority=True)
def reducetask(a, b):
'''
Merges two partial results (lists of (K, V) pairs) respecting the order
:param a: Partial result a
:param b: Partial result b
:return: The merging result sorted
'''
partial_result = []
i = 0
j = 0
while i < len(a) and j < len(b):
if a[i] < b[j]:
partial_result.append(a[i])
i += 1
else:
partial_result.append(b[j])
j += 1
if i < len(a):
partial_result + a[i:]
elif j < len(b):
partial_result + b[j:]
return partial_result
Found task: reducetask
[9]:
def merge_reduce(function, data):
import sys
if sys.version_info[0] >= 3:
import queue as Queue
else:
import Queue
q = Queue.Queue()
for i in data:
q.put(i)
while not q.empty():
x = q.get()
if not q.empty():
y = q.get()
q.put(function(x, y))
else:
return x
MAIN
Parameters (that can be configured in the following cell): * datasetPath: The path where the dataset is (default: the same as created previously).
[10]:
import os
import time
from pycompss.api.api import compss_wait_on
datasetPath = directoryName # Where the dataset is
files = []
for f in os.listdir(datasetPath):
files.append(datasetPath + '/' + f)
startTime = time.time()
partialSorted = []
for f in files:
partialSorted.append(sortPartition(f))
result = merge_reduce(reducetask, partialSorted)
result = compss_wait_on(result)
print("Elapsed Time(s)")
print(time.time() - startTime)
import pprint
pprint.pprint(result)
Elapsed Time(s)
3.44308185577
[(0.024594087944372567, 247),
(0.15829934318232952, 833),
(0.20351953951482715, 429),
(0.2411229869421292, 163),
(0.3449798392084097, 168),
(0.40180614568127493, 723),
(0.4131674972232263, 335),
(0.4900006613572737, 266),
(0.5370003033027633, 417),
(0.5572901245409135, 69),
(0.6360367344429025, 353),
(0.653407162593262, 533),
(0.6831094170770151, 534),
(0.8122494146592366, 273),
(0.8705321795897965, 648),
(0.8834590127098692, 15),
(0.9043691365994025, 756),
(0.9469655464060827, 576),
(0.9479567038949842, 432)]
[11]:
ipycompss.stop()
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Warning: some of the variables used with PyCOMPSs may
have not been brought to the master.
****************************************************
KMeans
KMeans is machine-learning algorithm (NP-hard), popularly employed for cluster analysis in data mining, and interesting for benchmarking and performance evaluation.
The objective of the Kmeans algorithm to group a set of multidimensional points into a predefined number of clusters, in which each point belongs to the closest cluster (with the nearest mean distance), in an iterative process.
[1]:
import pycompss.interactive as ipycompss
[2]:
import os
if 'BINDER_SERVICE_HOST' in os.environ:
ipycompss.start(graph=True, # trace=True
project_xml='../xml/project.xml',
resources_xml='../xml/resources.xml')
else:
ipycompss.start(graph=True, monitor=1000) # trace=True
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
* .-~~-.--. _____ ________ *
* : ) |____ \ |____ / *
* .~ ~ -.\ /.- ~~ . ___) | / / *
* > `. .' < / ___/ / / *
* ( .- -. ) | |___ _ / / *
* `- -.-~ `- -' ~-.- -' |_____| |_| /__/ *
* ( : ) _ _ .-: *
* ~--. : .--~ .-~ .-~ } *
* ~-.-^-.-~ \_ .~ .-~ .~ *
* \ \ ' \ '_ _ -~ *
* \`.\`. // *
* . - ~ ~-.__\`.\`-.// *
* .-~ . - ~ }~ ~ ~-.~-. *
* .' .-~ .-~ :/~-.~-./: *
* /_~_ _ . - ~ ~-.~-._ *
* ~-.< *
******************************************************
* - Starting COMPSs runtime... *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun! *
******************************************************
[3]:
from pycompss.api.task import task
[4]:
import numpy as np
[5]:
def init_random(numV, dim, seed):
np.random.seed(seed)
c = [np.random.uniform(-3.5, 3.5, dim)]
while len(c) < numV:
p = np.random.uniform(-3.5, 3.5, dim)
distance = [np.linalg.norm(p-i) for i in c]
if min(distance) > 2:
c.append(p)
return c
[6]:
#@task(returns=list) # Not a task for plotting
def genFragment(numV, K, c, dim, mode='gauss'):
if mode == "gauss":
n = int(float(numV) / K)
r = numV % K
data = []
for k in range(K):
s = np.random.uniform(0.05, 0.75)
for i in range(n+r):
d = np.array([np.random.normal(c[k][j], s) for j in range(dim)])
data.append(d)
return np.array(data)[:numV]
else:
return [np.random.random(dim) for _ in range(numV)]
[7]:
@task(returns=dict)
def cluster_points_partial(XP, mu, ind):
dic = {}
for x in enumerate(XP):
bestmukey = min([(i[0], np.linalg.norm(x[1] - mu[i[0]])) for i in enumerate(mu)], key=lambda t: t[1])[0]
if bestmukey not in dic:
dic[bestmukey] = [x[0] + ind]
else:
dic[bestmukey].append(x[0] + ind)
return dic
Found task: cluster_points_partial
[8]:
@task(returns=dict)
def partial_sum(XP, clusters, ind):
p = [(i, [(XP[j - ind]) for j in clusters[i]]) for i in clusters]
dic = {}
for i, l in p:
dic[i] = (len(l), np.sum(l, axis=0))
return dic
Found task: partial_sum
[9]:
@task(returns=dict, priority=True)
def reduceCentersTask(a, b):
for key in b:
if key not in a:
a[key] = b[key]
else:
a[key] = (a[key][0] + b[key][0], a[key][1] + b[key][1])
return a
Found task: reduceCentersTask
[10]:
def mergeReduce(function, data):
from collections import deque
q = deque(list(range(len(data))))
while len(q):
x = q.popleft()
if len(q):
y = q.popleft()
data[x] = function(data[x], data[y])
q.append(x)
else:
return data[x]
[11]:
def has_converged(mu, oldmu, epsilon, iter, maxIterations):
print("iter: " + str(iter))
print("maxIterations: " + str(maxIterations))
if oldmu != []:
if iter < maxIterations:
aux = [np.linalg.norm(oldmu[i] - mu[i]) for i in range(len(mu))]
distancia = sum(aux)
if distancia < epsilon * epsilon:
print("Distance_T: " + str(distancia))
return True
else:
print("Distance_F: " + str(distancia))
return False
else:
# Reached the max amount of iterations
return True
[12]:
def plotKMEANS(dim, mu, clusters, data):
import pylab as plt
colors = ['b','g','r','c','m','y','k']
if dim == 2 and len(mu) <= len(colors):
from matplotlib.patches import Circle
from matplotlib.collections import PatchCollection
fig, ax = plt.subplots(figsize=(10,10))
patches = []
pcolors = []
for i in range(len(clusters)):
for key in clusters[i].keys():
d = clusters[i][key]
for j in d:
j = j - i * len(data[0])
C = Circle((data[i][j][0], data[i][j][1]), .05)
pcolors.append(colors[key])
patches.append(C)
collection = PatchCollection(patches)
collection.set_facecolor(pcolors)
ax.add_collection(collection)
x, y = zip(*mu)
plt.plot(x, y, '*', c='y', markersize=20)
plt.autoscale(enable=True, axis='both', tight=False)
plt.show()
elif dim == 3 and len(mu) <= len(colors):
from mpl_toolkits.mplot3d import Axes3D
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
for i in range(len(clusters)):
for key in clusters[i].keys():
d = clusters[i][key]
for j in d:
j = j - i * len(data[0])
ax.scatter(data[i][j][0], data[i][j][1], data[i][j][2], 'o', c=colors[key])
x, y, z = zip(*mu)
for i in range(len(mu)):
ax.scatter(x[i], y[i], z[i], s=80, c='y', marker='D')
plt.show()
else:
print("No representable dim or not enough colours")
MAIN
[13]:
%matplotlib inline
import ipywidgets as widgets
from pycompss.api.api import compss_wait_on
w_numV = widgets.IntText(value=10000) # Number of Vectors - with 1000 it is feasible to see the evolution across iterations
w_dim = widgets.IntText(value=2) # Number of Dimensions
w_k = widgets.IntText(value=4) # Centers
w_numFrag = widgets.IntText(value=16) # Fragments
w_epsilon = widgets.FloatText(value=1e-10) # Convergence condition
w_maxIterations = widgets.IntText(value=20) # Max number of iterations
w_seed = widgets.IntText(value=8) # Random seed
def kmeans(numV, dim, k, numFrag, epsilon, maxIterations, seed):
size = int(numV / numFrag)
cloudCenters = init_random(k, dim, seed) # centers to create data groups
X = [genFragment(size, k, cloudCenters, dim, mode='gauss') for _ in range(numFrag)]
mu = init_random(k, dim, seed - 1) # First centers
oldmu = []
n = 0
while not has_converged(mu, oldmu, epsilon, n, maxIterations):
oldmu = mu
clusters = [cluster_points_partial(X[f], mu, f * size) for f in range(numFrag)]
partialResult = [partial_sum(X[f], clusters[f], f * size) for f in range(numFrag)]
mu = mergeReduce(reduceCentersTask, partialResult)
mu = compss_wait_on(mu)
mu = [mu[c][1] / mu[c][0] for c in mu]
while len(mu) < k:
# Add new random center if one of the centers has no points.
indP = np.random.randint(0, size)
indF = np.random.randint(0, numFrag)
mu.append(X[indF][indP])
n += 1
clusters = compss_wait_on(clusters)
plotKMEANS(dim, mu, clusters, X)
print("--------------------")
print("Result:")
print("Iterations: ", n)
print("Centers: ", mu)
print("--------------------")
widgets.interact_manual(kmeans, numV=w_numV, dim=w_dim, k=w_k, numFrag=w_numFrag, epsilon=w_epsilon, maxIterations=w_maxIterations, seed=w_seed)
[13]:
<function __main__.kmeans>
[14]:
ipycompss.stop()
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Warning: some of the variables used with PyCOMPSs may
have not been brought to the master.
****************************************************
KMeans with Reduce
KMeans is machine-learning algorithm (NP-hard), popularly employed for cluster analysis in data mining, and interesting for benchmarking and performance evaluation.
The objective of the Kmeans algorithm to group a set of multidimensional points into a predefined number of clusters, in which each point belongs to the closest cluster (with the nearest mean distance), in an iterative process.
[1]:
import pycompss.interactive as ipycompss
[2]:
import os
if 'BINDER_SERVICE_HOST' in os.environ:
ipycompss.start(graph=True, # trace=True
project_xml='../xml/project.xml',
resources_xml='../xml/resources.xml')
else:
ipycompss.start(graph=True, monitor=1000) # trace=True
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
* .-~~-.--. _____ ________ *
* : ) |____ \ |____ / *
* .~ ~ -.\ /.- ~~ . ___) | / / *
* > `. .' < / ___/ / / *
* ( .- -. ) | |___ _ / / *
* `- -.-~ `- -' ~-.- -' |_____| |_| /__/ *
* ( : ) _ _ .-: *
* ~--. : .--~ .-~ .-~ } *
* ~-.-^-.-~ \_ .~ .-~ .~ *
* \ \ ' \ '_ _ -~ *
* \`.\`. // *
* . - ~ ~-.__\`.\`-.// *
* .-~ . - ~ }~ ~ ~-.~-. *
* .' .-~ .-~ :/~-.~-./: *
* /_~_ _ . - ~ ~-.~-._ *
* ~-.< *
******************************************************
* - Starting COMPSs runtime... *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun! *
******************************************************
[3]:
from pycompss.api.task import task
[4]:
import numpy as np
[5]:
def init_random(numV, dim, seed):
np.random.seed(seed)
c = [np.random.uniform(-3.5, 3.5, dim)]
while len(c) < numV:
p = np.random.uniform(-3.5, 3.5, dim)
distance = [np.linalg.norm(p-i) for i in c]
if min(distance) > 2:
c.append(p)
return c
[6]:
#@task(returns=list) # Not a task for plotting
def genFragment(numV, K, c, dim, mode='gauss'):
if mode == "gauss":
n = int(float(numV) / K)
r = numV % K
data = []
for k in range(K):
s = np.random.uniform(0.05, 0.75)
for i in range(n+r):
d = np.array([np.random.normal(c[k][j], s) for j in range(dim)])
data.append(d)
return np.array(data)[:numV]
else:
return [np.random.random(dim) for _ in range(numV)]
[7]:
@task(returns=dict)
def cluster_points_partial(XP, mu, ind):
dic = {}
for x in enumerate(XP):
bestmukey = min([(i[0], np.linalg.norm(x[1] - mu[i[0]])) for i in enumerate(mu)], key=lambda t: t[1])[0]
if bestmukey not in dic:
dic[bestmukey] = [x[0] + ind]
else:
dic[bestmukey].append(x[0] + ind)
return dic
Found task: cluster_points_partial
[8]:
@task(returns=dict)
def partial_sum(XP, clusters, ind):
p = [(i, [(XP[j - ind]) for j in clusters[i]]) for i in clusters]
dic = {}
for i, l in p:
dic[i] = (len(l), np.sum(l, axis=0))
return dic
Found task: partial_sum
[9]:
def reduceCenters(a, b):
"""
Reduce method to sum the result of two partial_sum methods
:param a: partial_sum {cluster_ind: (#points_a, sum(points_a))}
:param b: partial_sum {cluster_ind: (#points_b, sum(points_b))}
:return: {cluster_ind: (#points_a+#points_b, sum(points_a+points_b))}
"""
for key in b:
if key not in a:
a[key] = b[key]
else:
a[key] = (a[key][0] + b[key][0], a[key][1] + b[key][1])
return a
[10]:
@task(returns=dict)
def reduceCentersTask(*data):
reduce_value = data[0]
for i in range(1, len(data)):
reduce_value = reduceCenters(reduce_value, data[i])
return reduce_value
Found task: reduceCentersTask
[11]:
def mergeReduce(function, data, chunk=50):
""" Apply function cumulatively to the items of data,
from left to right in binary tree structure, so as to
reduce the data to a single value.
:param function: function to apply to reduce data
:param data: List of items to be reduced
:return: result of reduce the data to a single value
"""
while(len(data)) > 1:
dataToReduce = data[:chunk]
data = data[chunk:]
data.append(function(*dataToReduce))
return data[0]
[12]:
def has_converged(mu, oldmu, epsilon, iter, maxIterations):
print("iter: " + str(iter))
print("maxIterations: " + str(maxIterations))
if oldmu != []:
if iter < maxIterations:
aux = [np.linalg.norm(oldmu[i] - mu[i]) for i in range(len(mu))]
distancia = sum(aux)
if distancia < epsilon * epsilon:
print("Distance_T: " + str(distancia))
return True
else:
print("Distance_F: " + str(distancia))
return False
else:
# Reached the max amount of iterations
return True
[13]:
def plotKMEANS(dim, mu, clusters, data):
import pylab as plt
colors = ['b','g','r','c','m','y','k']
if dim == 2 and len(mu) <= len(colors):
from matplotlib.patches import Circle
from matplotlib.collections import PatchCollection
fig, ax = plt.subplots(figsize=(10,10))
patches = []
pcolors = []
for i in range(len(clusters)):
for key in clusters[i].keys():
d = clusters[i][key]
for j in d:
j = j - i * len(data[0])
C = Circle((data[i][j][0], data[i][j][1]), .05)
pcolors.append(colors[key])
patches.append(C)
collection = PatchCollection(patches)
collection.set_facecolor(pcolors)
ax.add_collection(collection)
x, y = zip(*mu)
plt.plot(x, y, '*', c='y', markersize=20)
plt.autoscale(enable=True, axis='both', tight=False)
plt.show()
elif dim == 3 and len(mu) <= len(colors):
from mpl_toolkits.mplot3d import Axes3D
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
for i in range(len(clusters)):
for key in clusters[i].keys():
d = clusters[i][key]
for j in d:
j = j - i * len(data[0])
ax.scatter(data[i][j][0], data[i][j][1], data[i][j][2], 'o', c=colors[key])
x, y, z = zip(*mu)
for i in range(len(mu)):
ax.scatter(x[i], y[i], z[i], s=80, c='y', marker='D')
plt.show()
else:
print("No representable dim or not enough colours")
MAIN
[14]:
%matplotlib inline
import ipywidgets as widgets
from pycompss.api.api import compss_wait_on
w_numV = widgets.IntText(value=10000) # Number of Vectors - with 1000 it is feasible to see the evolution across iterations
w_dim = widgets.IntText(value=2) # Number of Dimensions
w_k = widgets.IntText(value=4) # Centers
w_numFrag = widgets.IntText(value=16) # Fragments
w_epsilon = widgets.FloatText(value=1e-10) # Convergence condition
w_maxIterations = widgets.IntText(value=20) # Max number of iterations
w_seed = widgets.IntText(value=8) # Random seed
def kmeans(numV, dim, k, numFrag, epsilon, maxIterations, seed):
size = int(numV / numFrag)
cloudCenters = init_random(k, dim, seed) # centers to create data groups
X = [genFragment(size, k, cloudCenters, dim, mode='gauss') for _ in range(numFrag)]
mu = init_random(k, dim, seed - 1) # First centers
oldmu = []
n = 0
while not has_converged(mu, oldmu, epsilon, n, maxIterations):
oldmu = mu
clusters = [cluster_points_partial(X[f], mu, f * size) for f in range(numFrag)]
partialResult = [partial_sum(X[f], clusters[f], f * size) for f in range(numFrag)]
mu = mergeReduce(reduceCentersTask, partialResult, chunk=4)
mu = compss_wait_on(mu)
mu = [mu[c][1] / mu[c][0] for c in mu]
while len(mu) < k:
# Add new random center if one of the centers has no points.
indP = np.random.randint(0, size)
indF = np.random.randint(0, numFrag)
mu.append(X[indF][indP])
n += 1
clusters = compss_wait_on(clusters)
plotKMEANS(dim, mu, clusters, X)
print("--------------------")
print("Result:")
print("Iterations: ", n)
print("Centers: ", mu)
print("--------------------")
widgets.interact_manual(kmeans, numV=w_numV, dim=w_dim, k=w_k, numFrag=w_numFrag, epsilon=w_epsilon, maxIterations=w_maxIterations, seed=w_seed)
[14]:
<function __main__.kmeans>
[15]:
ipycompss.stop()
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Warning: some of the variables used with PyCOMPSs may
have not been brought to the master.
****************************************************
Cholesky Decomposition/Factorization
Given a symmetric positive definite matrix A, the Cholesky decomposition is an upper triangular matrix U (with strictly positive diagonal entries) such that:
[1]:
import pycompss.interactive as ipycompss
[2]:
# Start PyCOMPSs runtime with graph and tracing enabled
import os
if 'BINDER_SERVICE_HOST' in os.environ:
ipycompss.start(graph=True, trace=True,
project_xml='../xml/project.xml',
resources_xml='../xml/resources.xml')
else:
ipycompss.start(graph=True, monitor=1000, trace=True)
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
* .-~~-.--. _____ ________ *
* : ) |____ \ |____ / *
* .~ ~ -.\ /.- ~~ . ___) | / / *
* > `. .' < / ___/ / / *
* ( .- -. ) | |___ _ / / *
* `- -.-~ `- -' ~-.- -' |_____| |_| /__/ *
* ( : ) _ _ .-: *
* ~--. : .--~ .-~ .-~ } *
* ~-.-^-.-~ \_ .~ .-~ .~ *
* \ \ ' \ '_ _ -~ *
* \`.\`. // *
* . - ~ ~-.__\`.\`-.// *
* .-~ . - ~ }~ ~ ~-.~-. *
* .' .-~ .-~ :/~-.~-./: *
* /_~_ _ . - ~ ~-.~-._ *
* ~-.< *
******************************************************
* - Starting COMPSs runtime... *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun! *
******************************************************
[3]:
from pycompss.api.task import task
from scipy import linalg
from scipy import random
import numpy as np
import ctypes
Task definitions
[4]:
@task(returns=list)
def createBlock(BSIZE, MKLProc, diag):
import os
os.environ["MKL_NUM_THREADS"]=str(MKLProc)
block = np.array(np.random.random((BSIZE, BSIZE)), dtype=np.double,copy=False)
mb = np.matrix(block, dtype=np.double, copy=False)
mb = mb + np.transpose(mb)
if diag:
mb = mb + 2*BSIZE*np.eye(BSIZE)
return mb
@task(returns=np.ndarray)
def potrf(A, MKLProc):
from scipy.linalg.lapack import dpotrf
import os
os.environ['MKL_NUM_THREADS']=str(MKLProc)
A = dpotrf(A, lower=True)[0]
return A
@task(returns=np.ndarray)
def solve_triangular(A, B, MKLProc):
from scipy.linalg import solve_triangular
from numpy import transpose
import os
os.environ['MKL_NUM_THREADS']=str(MKLProc)
B = transpose(B)
B = solve_triangular(A, B, lower=True) # , trans='T'
B = transpose(B)
return B
@task(returns=np.ndarray)
def gemm(alpha, A, B, C, beta, MKLProc):
from scipy.linalg.blas import dgemm
from numpy import transpose
import os
os.environ['MKL_NUM_THREADS']=str(MKLProc)
B = transpose(B)
C = dgemm(alpha, A, B, c=C, beta=beta)
return C
Found task: createBlock
Found task: potrf
Found task: solve_triangular
Found task: gemm
Auxiliar functions
[5]:
def genMatrix(MSIZE, BSIZE, MKLProc, A):
for i in range(MSIZE):
A.append([])
for j in range(MSIZE):
A[i].append([])
for i in range(MSIZE):
mb = createBlock(BSIZE, MKLProc, True)
A[i][i]=mb
for j in range(i+1,MSIZE):
mb = createBlock(BSIZE, MKLProc, False)
A[i][j]=mb
A[j][i]=mb
[6]:
def cholesky_blocked(MSIZE, BSIZE, mkl_threads, A):
import os
for k in range(MSIZE):
# Diagonal block factorization
A[k][k] = potrf(A[k][k], mkl_threads)
# Triangular systems
for i in range(k+1, MSIZE):
A[i][k] = solve_triangular(A[k][k], A[i][k], mkl_threads)
A[k][i] = np.zeros((BSIZE,BSIZE))
# update trailing matrix
for i in range(k+1, MSIZE):
for j in range(i, MSIZE):
A[j][i] = gemm(-1.0, A[j][k], A[i][k], A[j][i], 1.0, mkl_threads)
return A
MAIN Code
Parameters (that can be configured in the following cell): * MSIZE: Matrix size (default: 8) * BSIZE: Block size (default: 1024) * mkl_threads: Number of MKL threads (default: 1)
[7]:
import ipywidgets as widgets
from pycompss.api.api import compss_barrier
import time
w_MSIZE = widgets.IntText(value=8)
w_BSIZE = widgets.IntText(value=1024)
w_mkl_threads = widgets.IntText(value=1)
def cholesky(MSIZE, BSIZE, mkl_threads):
# Generate de matrix
startTime = time.time()
# Generate supermatrix
A = []
res = []
genMatrix(MSIZE, BSIZE, mkl_threads, A)
compss_barrier()
initTime = time.time() - startTime
startDecompTime = time.time()
res = cholesky_blocked(MSIZE, BSIZE, mkl_threads, A)
compss_barrier()
decompTime = time.time() - startDecompTime
totalTime = decompTime + initTime
print("---------- Elapsed Times ----------")
print("initT:{}".format(initTime))
print("decompT:{}".format(decompTime))
print("totalTime:{}".format(totalTime))
print("-----------------------------------")
widgets.interact_manual(cholesky, MSIZE=w_MSIZE, BSIZE=w_BSIZE, mkl_threads=w_mkl_threads)
[7]:
<function __main__.cholesky>
[8]:
ipycompss.stop()
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Warning: some of the variables used with PyCOMPSs may
have not been brought to the master.
****************************************************
Wordcount Exercise
Sequential version
[1]:
import os
[2]:
def read_file(file_path):
""" Read a file and return a list of words.
:param file_path: file's path
:return: list of words
"""
data = []
with open(file_path, 'r') as f:
for line in f:
data += line.split()
return data
[3]:
def wordCount(data):
""" Construct a frequency word dictorionary from a list of words.
:param data: a list of words
:return: a dictionary where key=word and value=#appearances
"""
partialResult = {}
for entry in data:
if entry in partialResult:
partialResult[entry] += 1
else:
partialResult[entry] = 1
return partialResult
[4]:
def merge_two_dicts(dic1, dic2):
""" Update a dictionary with another dictionary.
:param dic1: first dictionary
:param dic2: second dictionary
:return: dic1+=dic2
"""
for k in dic2:
if k in dic1:
dic1[k] += dic2[k]
else:
dic1[k] = dic2[k]
return dic1
[5]:
# Get the dataset path
pathDataset = os.getcwd() + '/dataset'
# Read file's content execute a wordcount on each of them
partialResult = []
for fileName in os.listdir(pathDataset):
file_path = os.path.join(pathDataset, fileName)
data = read_file(file_path)
partialResult.append(wordCount(data))
# Accumulate the partial results to get the final result.
result = {}
for partial in partialResult:
result = merge_two_dicts(result, partial)
[6]:
print("Result:")
from pprint import pprint
pprint(result)
print("Words: {}".format(sum(result.values())))
Result:
{'Adipisci': 227,
'Aliquam': 233,
'Amet': 207,
'Consectetur': 201,
'Dolor': 198,
'Dolore': 236,
'Dolorem': 232,
'Eius': 251,
'Est': 197,
'Etincidunt': 232,
'Ipsum': 228,
'Labore': 229,
'Magnam': 195,
'Modi': 201,
'Neque': 205,
'Non': 226,
'Numquam': 253,
'Porro': 205,
'Quaerat': 217,
'Quiquia': 212,
'Quisquam': 214,
'Sed': 225,
'Sit': 220,
'Tempora': 189,
'Ut': 217,
'Velit': 218,
'Voluptatem': 235,
'adipisci': 1078,
'aliquam': 1107,
'amet': 1044,
'consectetur': 1073,
'dolor': 1120,
'dolore': 1065,
'dolorem': 1107,
'eius': 1048,
'est': 1101,
'etincidunt': 1114,
'ipsum': 1061,
'labore': 1070,
'magnam': 1096,
'modi': 1127,
'neque': 1093,
'non': 1099,
'numquam': 1094,
'porro': 1101,
'quaerat': 1086,
'quiquia': 1079,
'quisquam': 1144,
'sed': 1109,
'sit': 1130,
'tempora': 1064,
'ut': 1070,
'velit': 1105,
'voluptatem': 1121}
Words: 35409
Wordcount Solution
Complete version
[1]:
import os
[2]:
import pycompss.interactive as ipycompss
[3]:
from pycompss.api.task import task
[4]:
from pycompss.api.parameter import *
[5]:
if 'BINDER_SERVICE_HOST' in os.environ:
ipycompss.start(graph=True, trace=True, debug=False,
project_xml='../xml/project.xml',
resources_xml='../xml/resources.xml')
else:
ipycompss.start(graph=True, monitor=1000, trace=True, debug=False)
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
* .-~~-.--. _____ ________ *
* : ) |____ \ |____ / *
* .~ ~ -.\ /.- ~~ . ___) | / / *
* > `. .' < / ___/ / / *
* ( .- -. ) | |___ _ / / *
* `- -.-~ `- -' ~-.- -' |_____| |_| /__/ *
* ( : ) _ _ .-: *
* ~--. : .--~ .-~ .-~ } *
* ~-.-^-.-~ \_ .~ .-~ .~ *
* \ \ ' \ '_ _ -~ *
* \`.\`. // *
* . - ~ ~-.__\`.\`-.// *
* .-~ . - ~ }~ ~ ~-.~-. *
* .' .-~ .-~ :/~-.~-./: *
* /_~_ _ . - ~ ~-.~-._ *
* ~-.< *
******************************************************
* - Starting COMPSs runtime... *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun! *
******************************************************
[6]:
@task(returns=list)
def read_file(file_path):
""" Read a file and return a list of words.
:param file_path: file's path
:return: list of words
"""
data = []
with open(file_path, 'r') as f:
for line in f:
data += line.split()
return data
Found task: read_file
[7]:
@task(returns=dict)
def wordCount(data):
""" Construct a frequency word dictorionary from a list of words.
:param data: a list of words
:return: a dictionary where key=word and value=#appearances
"""
partialResult = {}
for entry in data:
if entry in partialResult:
partialResult[entry] += 1
else:
partialResult[entry] = 1
return partialResult
Found task: wordCount
[8]:
@task(returns=dict, priority=True)
def merge_two_dicts(dic1, dic2):
""" Update a dictionary with another dictionary.
:param dic1: first dictionary
:param dic2: second dictionary
:return: dic1+=dic2
"""
for k in dic2:
if k in dic1:
dic1[k] += dic2[k]
else:
dic1[k] = dic2[k]
return dic1
Found task: merge_two_dicts
[9]:
from pycompss.api.api import compss_wait_on
# Get the dataset path
pathDataset = os.getcwd() + '/dataset'
# Read file's content execute a wordcount on each of them
partialResult = []
for fileName in os.listdir(pathDataset):
file_path = os.path.join(pathDataset, fileName)
data = read_file(file_path)
partialResult.append(wordCount(data))
# Accumulate the partial results to get the final result.
result = {}
for partial in partialResult:
result = merge_two_dicts(result, partial)
# Wait for result
result = compss_wait_on(result)
[10]:
print("Result:")
from pprint import pprint
pprint(result)
print("Words: {}".format(sum(result.values())))
Result:
{'Adipisci': 227,
'Aliquam': 233,
'Amet': 207,
'Consectetur': 201,
'Dolor': 198,
'Dolore': 236,
'Dolorem': 232,
'Eius': 251,
'Est': 197,
'Etincidunt': 232,
'Ipsum': 228,
'Labore': 229,
'Magnam': 195,
'Modi': 201,
'Neque': 205,
'Non': 226,
'Numquam': 253,
'Porro': 205,
'Quaerat': 217,
'Quiquia': 212,
'Quisquam': 214,
'Sed': 225,
'Sit': 220,
'Tempora': 189,
'Ut': 217,
'Velit': 218,
'Voluptatem': 235,
'adipisci': 1078,
'aliquam': 1107,
'amet': 1044,
'consectetur': 1073,
'dolor': 1120,
'dolore': 1065,
'dolorem': 1107,
'eius': 1048,
'est': 1101,
'etincidunt': 1114,
'ipsum': 1061,
'labore': 1070,
'magnam': 1096,
'modi': 1127,
'neque': 1093,
'non': 1099,
'numquam': 1094,
'porro': 1101,
'quaerat': 1086,
'quiquia': 1079,
'quisquam': 1144,
'sed': 1109,
'sit': 1130,
'tempora': 1064,
'ut': 1070,
'velit': 1105,
'voluptatem': 1121}
Words: 35409
[11]:
ipycompss.stop()
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Warning: some of the variables used with PyCOMPSs may
have not been brought to the master.
****************************************************
Wordcount Solution (With reduce)
Complete version
[1]:
import os
[2]:
import pycompss.interactive as ipycompss
[3]:
from pycompss.api.task import task
[4]:
from pycompss.api.parameter import *
[5]:
if 'BINDER_SERVICE_HOST' in os.environ:
ipycompss.start(graph=True, trace=True, debug=False,
project_xml='../xml/project.xml',
resources_xml='../xml/resources.xml')
else:
ipycompss.start(graph=True, monitor=1000, trace=True, debug=False)
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
* .-~~-.--. _____ ________ *
* : ) |____ \ |____ / *
* .~ ~ -.\ /.- ~~ . ___) | / / *
* > `. .' < / ___/ / / *
* ( .- -. ) | |___ _ / / *
* `- -.-~ `- -' ~-.- -' |_____| |_| /__/ *
* ( : ) _ _ .-: *
* ~--. : .--~ .-~ .-~ } *
* ~-.-^-.-~ \_ .~ .-~ .~ *
* \ \ ' \ '_ _ -~ *
* \`.\`. // *
* . - ~ ~-.__\`.\`-.// *
* .-~ . - ~ }~ ~ ~-.~-. *
* .' .-~ .-~ :/~-.~-./: *
* /_~_ _ . - ~ ~-.~-._ *
* ~-.< *
******************************************************
* - Starting COMPSs runtime... *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun! *
******************************************************
[6]:
@task(returns=list)
def read_file(file_path):
""" Read a file and return a list of words.
:param file_path: file's path
:return: list of words
"""
data = []
with open(file_path, 'r') as f:
for line in f:
data += line.split()
return data
Found task: read_file
[7]:
@task(returns=dict)
def wordCount(data):
""" Construct a frequency word dictorionary from a list of words.
:param data: a list of words
:return: a dictionary where key=word and value=#appearances
"""
partialResult = {}
for entry in data:
if entry in partialResult:
partialResult[entry] += 1
else:
partialResult[entry] = 1
return partialResult
Found task: wordCount
[8]:
@task(returns=dict, priority=True)
def merge_dicts(*dictionaries):
import Queue
q = Queue.Queue()
for i in dictionaries:
q.put(i)
while not q.empty():
x = q.get()
if not q.empty():
y = q.get()
for k in y:
if k in x:
x[k] += y[k]
else:
x[k] = y[k]
q.put(x)
return(x)
Found task: merge_dicts
[9]:
from pycompss.api.api import compss_wait_on
# Get the dataset path
pathDataset = os.getcwd() + '/dataset'
# Construct a list with the file's paths from the dataset
partialResult = []
for fileName in os.listdir(pathDataset):
p = os.path.join(pathDataset, fileName)
data=read_file(p)
partialResult.append(wordCount(data))
# Accumulate the partial results to get the final result.
result=merge_dicts(*partialResult)
# Wait for result
result = compss_wait_on(result)
[10]:
print("Result:")
from pprint import pprint
pprint(result)
print("Words: {}".format(sum(result.values())))
Result:
{'Adipisci': 227,
'Aliquam': 233,
'Amet': 207,
'Consectetur': 201,
'Dolor': 198,
'Dolore': 236,
'Dolorem': 232,
'Eius': 251,
'Est': 197,
'Etincidunt': 232,
'Ipsum': 228,
'Labore': 229,
'Magnam': 195,
'Modi': 201,
'Neque': 205,
'Non': 226,
'Numquam': 253,
'Porro': 205,
'Quaerat': 217,
'Quiquia': 212,
'Quisquam': 214,
'Sed': 225,
'Sit': 220,
'Tempora': 189,
'Ut': 217,
'Velit': 218,
'Voluptatem': 235,
'adipisci': 1078,
'aliquam': 1107,
'amet': 1044,
'consectetur': 1073,
'dolor': 1120,
'dolore': 1065,
'dolorem': 1107,
'eius': 1048,
'est': 1101,
'etincidunt': 1114,
'ipsum': 1061,
'labore': 1070,
'magnam': 1096,
'modi': 1127,
'neque': 1093,
'non': 1099,
'numquam': 1094,
'porro': 1101,
'quaerat': 1086,
'quiquia': 1079,
'quisquam': 1144,
'sed': 1109,
'sit': 1130,
'tempora': 1064,
'ut': 1070,
'velit': 1105,
'voluptatem': 1121}
Words: 35409
[11]:
ipycompss.stop()
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Warning: some of the variables used with PyCOMPSs may
have not been brought to the master.
****************************************************
Demos
Here you will find the demonstration notebooks used in the tutorials.
Accelerating parallel code with PyCOMPSs and Numba
Demo Supercomputing 2019
What is mandelbrot?
The mandelbrot set is a fractal, which is plotted on the complex plane. It shows how intrincate can be formed from a simple equation.
It is generated using the algorithm:
Where Z and A are complex numbers, and n represents the number of iterations.
First, import time to measure the elapsed execution times and create an ordered dictionary to keep all measures -> we are going to measure and plot the performance with different conditions!
[1]:
import time
from collections import OrderedDict
times = OrderedDict()
And then, all required imports
[2]:
from numpy import NaN, arange, abs, array
Mandelbrot set implementation:
[3]:
def mandelbrot(a, max_iter):
z = 0
for n in range(1, max_iter):
z = z**2 + a
if abs(z) > 2:
return n
return NaN
[4]:
def mandelbrot_set(y, X, max_iter):
Z = [0 for _ in range(len(X))]
for ix, x in enumerate(X):
Z[ix] = mandelbrot(x + 1j * y, max_iter)
return Z
Main function to generate the mandelbrot set. It splits the space in vertical chunks, and calculates the mandelbrot set of each one, generating the result Z.
[5]:
def run_mandelbrot(X, Y, max_iter):
st = time.time()
Z = [[] for _ in range(len(Y))]
for iy, y in enumerate(Y):
Z[iy] = mandelbrot_set(y, X, max_iter)
elapsed = time.time() - st
print("Elapsed time (s): {}".format(elapsed))
return Z, elapsed
The following function plots the fractal inline (the coerced parameter ** is used to set NaN in coerced elements within Z).
[6]:
%matplotlib inline
def plot_fractal(Z, coerced):
if coerced:
Z = [[NaN if c == -2**63 else c for c in row] for row in Z]
import matplotlib.pyplot as plt
Z = array(Z)
plt.imshow(Z, cmap='plasma')
plt.show()
Define a benchmarking function:
[7]:
def generate_fractal(coerced=False):
X = arange(-2, .5, .01)
Y = arange(-1.0, 1.0, .01)
max_iterations = 2000
Z, elapsed = run_mandelbrot(X, Y, max_iterations)
plot_fractal(Z, coerced)
return elapsed
Run the previous code sequentially:
[8]:
times['Sequential'] = generate_fractal()
Elapsed time (s): 48.8038640022
Paralellization with PyCOMPSs
After analysing the code, each mandelbrot set can be considered as a task, requiring only to decorate the mandelbrot_set function. It is interesting to observe that all sets are independent among them, so they can be computed completely independently, enabling to exploit multiple resources concurrently.
In order to run this code with we need first to start the COMPSs runtime:
[9]:
import os
import pycompss.interactive as ipycompss
if 'BINDER_SERVICE_HOST' in os.environ:
ipycompss.start(project_xml='../xml/project.xml',
resources_xml='../xml/resources.xml')
else:
ipycompss.start(graph=False, trace=True, monitor=1000)
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
* .-~~-.--. _____ ________ *
* : ) |____ \ |____ / *
* .~ ~ -.\ /.- ~~ . ___) | / / *
* > `. .' < / ___/ / / *
* ( .- -. ) | |___ _ / / *
* `- -.-~ `- -' ~-.- -' |_____| |_| /__/ *
* ( : ) _ _ .-: *
* ~--. : .--~ .-~ .-~ } *
* ~-.-^-.-~ \_ .~ .-~ .~ *
* \ \ ' \ '_ _ -~ *
* \`.\`. // *
* . - ~ ~-.__\`.\`-.// *
* .-~ . - ~ }~ ~ ~-.~-. *
* .' .-~ .-~ :/~-.~-./: *
* /_~_ _ . - ~ ~-.~-._ *
* ~-.< *
******************************************************
* - Starting COMPSs runtime... *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun! *
******************************************************
It is necessary to decorate the mandelbrot_set function with the @task decorator.
Note that the mandelbrot_set function returns a list of elements.
[10]:
from pycompss.api.task import task
[11]:
@task(returns=list)
def mandelbrot_set(y, X, max_iter):
Z = [0 for _ in range(len(X))]
for ix, x in enumerate(X):
Z[ix] = mandelbrot(x + 1j * y, max_iter)
return Z
Found task: mandelbrot_set
And finally, include the synchronization of Z with compss_wait_on.
[12]:
from pycompss.api.api import compss_wait_on
[13]:
def run_mandelbrot(X, Y, max_iter):
st = time.time()
Z = [[] for _ in range(len(Y))]
for iy, y in enumerate(Y):
Z[iy] = mandelbrot_set(y, X, max_iter)
Z = compss_wait_on(Z)
elapsed = time.time() - st
print("Elapsed time (s): {}".format(elapsed))
return Z, elapsed
Run the benchmark with PyCOMPSs:
[14]:
times['PyCOMPSs'] = generate_fractal()
Elapsed time (s): 21.9165420532
Accelerating the tasks with Numba
To this end, it is necessary to either use: 1. the Numba’s @jit decorator under the PyCOMPSs @task decorator 2. or define the numba=True within the @task decorator.
First, we decorate the inner function (mandelbrot) with @jit since it is also a target function to be optimized with Numba.
[15]:
from numba import jit
@jit
def mandelbrot(a, max_iter):
z = 0
for n in range(1, max_iter):
z = z**2 + a
if abs(z) > 2:
return n
return NaN # NaN is coerced by Numba
Option 1 - Add the @jit decorator explicitly under @task decorator
Option 2 - Add the numba=True flag within @task decorator
[16]:
@task(returns=list, numba=True)
def mandelbrot_set(y, X, max_iter):
Z = [0 for _ in range(len(X))]
for ix, x in enumerate(X):
Z[ix] = mandelbrot(x + 1j * y, max_iter)
return Z
Found task: mandelbrot_set
Run the benchmark with Numba:
[17]:
times['PyCOMPSs + Numba'] = generate_fractal(coerced=True)
Elapsed time (s): 5.50196003914
Plot the times:
[18]:
import matplotlib.pyplot as plt
plt.bar(*zip(*times.items()))
plt.show()
Stop COMPSs runtime
[19]:
ipycompss.stop()
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Warning: some of the variables used with PyCOMPSs may
have not been brought to the master.
****************************************************
Hint
These notebooks can be used within MyBinder, with the PyCOMPSs Player, within Docker, within Virtual Machine (recommended for Windows) provided by BSC, or locally.
- Prerequisites
Using MyBinder:
- Open
Caution
Sometimes it may take a while to deploy the COMPSs infrastructure.
- Open
Using PyCOMPSs Player:
pycompss-player(see Requirements and Installation)
Using Docker:
- Docker
- Git
Using Virtual Machine:
- VirtualBox
For local execution:
- Python 2 or 3
- Install COMPSs requirements described in Dependencies.
- Install COMPSs (See Building from sources)
- Jupyter (with the desired ipykernel)
- ipywidgets (only for some hands-on notebooks)
- numpy (only for some notebooks)
- dislib (only for some notebooks)
- numba (only for some notebooks)
- Git
- Instructions
Using MyBinder:
Just explore the folders and run the examples (they have the same structure as this documentation).
Using
pycompss-player:Check the
pycompss-playerusage instructions (see Usage)Get the notebooks:
$ git clone https://github.com/bsc-wdc/notebooks.git
Using Docker:
Run in your machine:
$ git clone https://github.com/bsc-wdc/notebooks.git $ docker pull compss/compss:2.7 $ # Update the path to the notebooks path in the next command before running it $ docker run --name mycompss -p 8888:8888 -p 8080:8080 -v /PATH/TO/notebooks:/home/notebooks -itd compss/compss:2.7 $ docker exec -it mycompss /bin/bash
Now that docker is running and you are connected:
$ cd /home/notebooks $ /etc/init.d/compss-monitor start $ jupyter-notebook --no-browser --allow-root --ip=172.17.0.2 --NotebookApp.token=
From local web browser:
Open COMPSs monitor: http://localhost:8080/compss-monitor/index.zul Open Jupyter notebook interface: http://localhost:8888/
Using Virtual Machine:
Download the OVA from: https://www.bsc.es/research-and-development/software-and-apps/software-list/comp-superscalar/downloads (Look for Virtual Appliances section)
Import the OVA from VirtualBox
Start the Virtual Machine
- User: compss
- Password: compss2019
Open a console and run:
$ git clone https://github.com/bsc-wdc/notebooks.git $ cd notebooks $ /etc/init.d/compss-monitor start $ jupyter-notebookOpen the web browser:
* Open COMPSs monitor: http://localhost:8080/compss-monitor/index.zul * Open Jupyter notebook interface: http://localhost:8888/
Using local installation
Get the notebooks and start jupyter
$ git clone https://github.com/bsc-wdc/notebooks.git $ cd notebooks $ /etc/init.d/compss-monitor start $ jupyter-notebookThen
* Open COMPSs monitor: http://localhost:8080/compss-monitor/index.zul * Open Jupyter notebook interface: http://localhost:8888/ * Look for the application.ipynb of interest.
Important
It is necessary to RESTART the python kernel from Jupyter after the execution of any notebook.
- Troubleshooting
ISSUE 1: Cannot connect using docker pull.
REASON: The docker service is not running:
$ # Error messsage: $ Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? $ # SOLUTION: Restart the docker service: $ sudo service docker start
ISSUE 2: The notebooks folder is empty or contains other data using docker.
REASON: The notebooks path in the docker run command is wrong.
$ # Remove the docker instance and reinstantiate with the appropriate notebooks path $ exit $ docker stop mycompss $ docker rm mycompss $ # Pay attention and UPDATE: /PATH/TO in the next command $ docker run --name mycompss -p 8888:8888 -p 8080:8080 -v /PATH/TO/notebooks:/home/notebooks -itd compss/compss-tutorial:2.7 $ # Continue as normal
ISSUE 3: COMPSs does not start in Jupyter.
REASON: The python kernel has not been restarted between COMPSs start, or some processes from previous failed execution may exist.
$ # SOLUTION: Restart the python kernel from Jupyter and check that there are no COMPSs' python/java processes running.ISSUE 4: Numba is not working with the VM or Docker.
REASON: Numba is not installed in the VM or docker
$ # SOLUTION: Install Numba in the VM/Docker $ # Open a console in the VM/Docker and follow the next steps. $ # For Python 2: $ sudo python2 -m pip install numba $ # For Python 3: $ sudo python3 -m pip install numba
ISSUE 5: Matplotlib is not working with the VM or Docker.
REASON: Matplotlib is not installed in the VM or docker
$ # SOLUTION: Install Matplotlib in the VM/Docker $ # Open a console in the VM/Docker and follow the next steps. $ # For Python 2: $ sudo python2 -m pip install matplotlib $ # For Python 3: $ sudo python3 -m pip install matplotlib
- Contact
- support-compss@bsc.es
Troubleshooting
This section provides answers for the most common issues of the execution of COMPSs applications and its known limitations.
For specific issues not covered inthis section, please do not hesitate to contact us at: support-compss@bsc.es .
How to debug
When an error/exception happens during the execution of an application, the first thing that users must do is to check the application output:
- Using
runcompssthe output is shown in the console. - Using
enqueue_compssthe output is in thecompss-<JOB_ID>.outandcompss-<JOB_ID>.err
If the error happens within a task, it will not appear in these files. Users must check the log folder in order to find what has failed. The log folder is by default in:
- Using
runcompss:$HOME/.COMPSs/<APP_NAME>_XX(where XX is a number between 00 and 99, and increases on each run). - Using
enqueue_compss:$HOME/.COMPSs/<JOB_ID>
This log folder contains the jobs folder, where all output/errors of the
tasks are stored. In particular, each task produces a JOB<TASK_NUMBER>_NEW.out
and JOB<TASK_NUMBER>_NEW.err files when a task fails.
Tip
If the user enables the debug mode by including the -t flag into
runcompss or enqueue_compss command, more information will be
stored in the log folder of each run easing the error detection.
In particular, all output and error output of all tasks will appear
within the jobs folder.
In addition, some more log files will appear:
runtime.logpycompss.log(only if using the Python binding).pycompss.err(only if using the Python binding and an error in the binding happens.)resources.logworkersfolder. This folder will contain four files per worker node:worker_<MACHINE_NAME>.outworker_<MACHINE_NAME>.errbinding_worker_<MACHINE_NAME>.outbinding_worker_<MACHINE_NAME>.err
As a suggestion, users should check the last lines of the runtime.log.
If the file-transfers or the tasks are failing an error message will appear
in this file. If the file-transfers are successfully and the jobs are
submitted, users should check the jobs folder and look at the error
messages produced inside each job. Users should notice that if there are
RESUBMITTED files something inside the job is failing.
If the workers folder is empty, means that the execution failed and
the COMPSs runtime was not able to retrieve the workers logs. In this case,
users must connect to the workers and look directly into the worker logs.
Alternatively, if the user is running with a shared disk (e.g. in a
supercomputer), the user can define a shared folder in the
--worker_working_directory=/shared/folder where a tmp_XXXXXX folder
will be created on the application execution and all worker logs will be
stored.
The following subsections show debugging examples depending on the choosen flavour (Java, Python or C/C++).
Java examples
Exception in the main code
TODO
Missing subsection
Exception in a task
TODO
Missing subsection
Python examples
Exception in the main code
Consider the following code where an intended error in the main code has been introduced to show how it can be debugged.
from pycompss.api.task import task
@task(returns=1)
def increment(value):
return value + 1
def main():
initial_value = 1
result = increment(initial_value)
result = result + 1 # Try to use result without synchronizing it: Error
print("Result: " + str(result))
if __name__=='__main__':
main()
When executed, it produces the following output:
$ runcompss error_in_main.py
[ INFO] Inferred PYTHON language
[ INFO] Using default location for project file: /opt/COMPSs//Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs//Runtime/configuration/xml/resources/default_resources.xml
[ INFO] Using default execution type: compss
----------------- Executing error_in_main.py --------------------------
WARNING: COMPSs Properties file is null. Setting default values
[(377) API] - Starting COMPSs Runtime v2.7 (build 20200519-1005.r6093e5ac94d67250e097a6fad9d3ec00d676fe6c)
[ ERROR ]: An exception occurred: unsupported operand type(s) for +: 'Future' and 'int'
Traceback (most recent call last):
File "/opt/COMPSs//Bindings/python/2/pycompss/runtime/launch.py", line 204, in compss_main
execfile(APP_PATH, globals()) # MAIN EXECUTION
File "error_in_main.py", line 16, in <module>
main()
File "error_in_main.py", line 11, in main
result = result + 1 # Try to use result without synchronizing it: Error
TypeError: unsupported operand type(s) for +: 'Future' and 'int'
[ERRMGR] - WARNING: Task 1(Action: 1) with name error_in_main.increment has been cancelled.
[ERRMGR] - WARNING: Task canceled: [[Task id: 1], [Status: CANCELED], [Core id: 0], [Priority: false], [NumNodes: 1], [MustReplicate: false], [MustDistribute: false], [error_in_main.increment(INT_T)]]
[(3609) API] - Execution Finished
Error running application
It can be identified the complete trackeback pointing where the error is, and
the reason. In this example, the reason is
TypeError: unsupported operand type(s) for +: 'Future' and 'int'
since we are trying to use an object that has not been synchronized.
Tip
Any exception raised from the main code will appear in the same way, showing the traceback helping to idenftiy the line which produced the exception and its reason.
Exception in a task
Consider the following code where an intended error in a task code has been introduced to show how it can be debugged.
from pycompss.api.task import task
from pycompss.api.api import compss_wait_on
@task(returns=1)
def increment(value):
return value + 1 # value is an string, can not add an int: Error
def main():
initial_value = "1" # the initial value is a string instead of an integer
result = increment(initial_value)
result = compss_wait_on(result)
print("Result: " + str(result))
if __name__=='__main__':
main()
When executed, it produces the following output:
$ runcompss error_in_task.py
[ INFO] Inferred PYTHON language
[ INFO] Using default location for project file: /opt/COMPSs//Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs//Runtime/configuration/xml/resources/default_resources.xml
[ INFO] Using default execution type: compss
----------------- Executing error_in_task.py --------------------------
WARNING: COMPSs Properties file is null. Setting default values
[(570) API] - Starting COMPSs Runtime v2.7 (build 20200519-1005.r6093e5ac94d67250e097a6fad9d3ec00d676fe6c)
[ERRMGR] - WARNING: Job 1 for running task 1 on worker localhost has failed; resubmitting task to the same worker.
[ERRMGR] - WARNING: Task 1 execution on worker localhost has failed; rescheduling task execution. (changing worker)
[ERRMGR] - WARNING: Job 2 for running task 1 on worker localhost has failed; resubmitting task to the same worker.
[ERRMGR] - WARNING: Task 1 has already been rescheduled; notifying task failure.
[ERRMGR] - WARNING: Task 'error_in_task.increment' TOTALLY FAILED.
Possible causes:
-Exception thrown by task 'error_in_task.increment'.
-Expected output files not generated by task 'error_in_task.increment'.
-Could not provide nor retrieve needed data between master and worker.
Check files '/home/user/.COMPSs/error_in_task.py_01/jobs/job[1|2'] to find out the error.
[ERRMGR] - ERROR: Task failed: [[Task id: 1], [Status: FAILED], [Core id: 0], [Priority: false], [NumNodes: 1], [MustReplicate: false], [MustDistribute: false], [error_in_task.increment(STRING_T)]]
[ERRMGR] - Shutting down COMPSs...
[(4711) API] - Execution Finished
Shutting down the running process
Error running application
The output describes that there has been an issue with the task number 1. Since the default behaviour of the runtime is to resubmit the failed task, task 2 also fails.
In this case, the runtime suggests to check the log files of the tasks:
/home/user/.COMPSs/error_in_task.py_01/jobs/job[1|2]
Looking into the logs folder, it can be seen that the jobs folder contains
the logs of the failed tasks:
$HOME/.COMPSs
└── error_in_task.py_01
├── jobs
│ ├── job1_NEW.err
│ ├── job1_NEW.out
│ ├── job1_RESUBMITTED.err
│ ├── job1_RESUBMITTED.out
│ ├── job2_NEW.err
│ ├── job2_NEW.out
│ ├── job2_RESUBMITTED.err
│ └── job2_RESUBMITTED.out
├── resources.log
├── runtime.log
├── tmpFiles
└── workers
And the job1_NEW.err contains the complete traceback of the exception that
has been raised (TypeError: cannot concatenate 'str' and 'int' objects as
consequence of using a string for the task input which tries to add 1):
[EXECUTOR] executeTask - Error in task execution
es.bsc.compss.types.execution.exceptions.JobExecutionException: Job 1 exit with value 1
at es.bsc.compss.invokers.external.piped.PipedInvoker.invokeMethod(PipedInvoker.java:78)
at es.bsc.compss.invokers.Invoker.invoke(Invoker.java:352)
at es.bsc.compss.invokers.Invoker.processTask(Invoker.java:287)
at es.bsc.compss.executor.Executor.executeTask(Executor.java:486)
at es.bsc.compss.executor.Executor.executeTaskWrapper(Executor.java:322)
at es.bsc.compss.executor.Executor.execute(Executor.java:229)
at es.bsc.compss.executor.Executor.processRequests(Executor.java:198)
at es.bsc.compss.executor.Executor.run(Executor.java:153)
at es.bsc.compss.executor.utils.ExecutionPlatform$2.run(ExecutionPlatform.java:178)
at java.lang.Thread.run(Thread.java:748)
Traceback (most recent call last):
File "/opt/COMPSs/Bindings/python/2/pycompss/worker/commons/worker.py", line 265, in task_execution
**compss_kwargs)
File "/opt/COMPSs/Bindings/python/2/pycompss/api/task.py", line 267, in task_decorator
return self.worker_call(*args, **kwargs)
File "/opt/COMPSs/Bindings/python/2/pycompss/api/task.py", line 1523, in worker_call
**user_kwargs)
File "/home/javier/temp/Bugs/documentation/error_in_task.py", line 6, in increment
return value + 1
TypeError: cannot concatenate 'str' and 'int' objects
Tip
Any exception raised from the task code will appear in the same way, showing the traceback helping to identify the line which produced the exception and its reason.
C/C++ examples
Exception in the main code
TODO
Missing subsection
Exception in a task
TODO
Missing subsection
Common Issues
Tasks are not executed
If the tasks remain in Blocked state probably there are no existing resources matching the specific task constraints. This error can be potentially caused by two facts: the resources are not correctly loaded into the runtime, or the task constraints do not match with any resource.
In the first case, users should take a look at the resouces.log and
check that all the resources defined in the project.xml file are
available to the runtime. In the second case users should re-define the
task constraints taking into account the resources capabilities defined
into the resources.xml and project.xml files.
Jobs fail
If all the application’s tasks fail because all the submitted jobs fail, it is probably due to the fact that there is a resource miss-configuration. In most of the cases, the resource that the application is trying to access has no passwordless access through the configured user. This can be checked by:
- Open the
project.xml. (The default file is stored under/opt/COMPSs/ Runtime/configuration/xml/projects/project.xml) - For each resource annotate its name and the value inside the
Usertag. Remember that if there is noUsertag COMPSs will try to connect this resource with the same username than the one that launches the main application. - For each annotated resourceName - user please try
ssh user@resourceName. If the connection asks for a password then there is an error in the configuration of the ssh access in the resource.
The problem can be solved running the following commands:
compss@bsc:~$ scp ~/.ssh/id_rsa.pub user@resourceName:./myRSA.pub
compss@bsc:~$ ssh user@resourceName "cat myRSA.pub >> ~/.ssh/authorized_keys; rm ./myRSA.pub"
These commands are a quick solution, for further details please check the Additional Configuration Section.
Exceptions when starting the Worker processes
When the COMPSs master is not able to communicate with one of the COMPSs workers described in the project.xml and resources.xml files, different exceptions can be raised and logged on the runtime.log of the application. All of them are raised during the worker start up and contain the [WorkerStarter] prefix. Next we provide a list with the common exceptions:
- InitNodeException
- Exception raised when the remote SSH process to start the worker has failed.
- UnstartedNodeException
- Exception raised when the worker process has aborted.
- Connection refused
- Exception raised when the master cannot communicate with the worker process (NIO).
All these exceptions encapsulate an error when starting the worker process. This means that the worker machine is not properly configured and thus, you need to check the environment of the failing worker. Further information about the specific error can be found on the worker log, available at the working directory path in the remote worker machine (the worker working directory specified in the project.xml} file).
Next, we list the most common errors and their solutions:
- java command not found
- Invalid path to the java binary. Check the JAVA_HOME definition at the remote worker machine.
- Cannot create WD
- Invalid working directory. Check the rw permissions of the worker’s working directory.
- No exception
- The worker process has started normally and there is no exception.
In this case the issue is normally due to the firewall configuration
preventing the communication between the COMPSs master and worker.
Please check that the worker firewall has in and out permissions for TCP
and UDP in the adaptor ports (the adaptor ports are specified in the
resources.xmlfile. By default the port rank is 43000-44000.
Compilation error: @Method not found
When trying to compile Java applications users can get some of the following compilation errors:
error: package es.bsc.compss.types.annotations does not exist
import es.bsc.compss.types.annotations.Constraints;
^
error: package es.bsc.compss.types.annotations.task does not exist
import es.bsc.compss.types.annotations.task.Method;
^
error: package es.bsc.compss.types.annotations does not exist
import es.bsc.compss.types.annotations.Parameter;
^
error: package es.bsc.compss.types.annotations.Parameter does not exist
import es.bsc.compss.types.annotations.parameter.Direction;
^
error: package es.bsc.compss.types.annotations.Parameter does not exist
import es.bsc.compss.types.annotations.parameter.Type;
^
error: cannot find symbol
@Parameter(type = Type.FILE, direction = Direction.INOUT)
^
symbol: class Parameter
location: interface APPLICATION_Itf
error: cannot find symbol
@Constraints(computingUnits = "2")
^
symbol: class Constraints
location: interface APPLICATION_Itf
error: cannot find symbol
@Method(declaringClass = "application.ApplicationImpl")
^
symbol: class Method
location: interface APPLICATION_Itf
All these errors are raised because the compss-engine.jar is not
listed in the CLASSPATH. The default COMPSs installation automatically
inserts this package into the CLASSPATH but it may have been overwritten
or deleted. Please check that your environment variable CLASSPATH
containts the compss-engine.jar location by running the following
command:
$ echo $CLASSPATH | grep compss-engine
If the result of the previous command is empty it means that you are
missing the compss-engine.jar package in your classpath.
The easiest solution is to manually export the CLASSPATH variable into the user session:
$ export CLASSPATH=$CLASSPATH:/opt/COMPSs/Runtime/compss-engine.jar
However, you will need to remember to export this variable every time
you log out and back in again. Consequently, we recommend to add this
export to the .bashrc file:
$ echo "# COMPSs variables for Java compilation" >> ~/.bashrc
$ echo "export CLASSPATH=$CLASSPATH:/opt/COMPSs/Runtime/compss-engine.jar" >> ~/.bashrc
Warning
The compss-engine.jar is installed inside the COMPSs
installation directory. If you have performed a custom installation,
the path of the package may be different.
Jobs failed on method reflection
When executing an application the main code gets stuck executing a task.
Taking a look at the runtime.log users can check that the job
associated to the task has failed (and all its resubmissions too). Then,
opening the jobX_NEW.out or the jobX_NEW.err files users find
the following error:
[ERROR|es.bsc.compss.Worker|Executor] Can not get method by reflection
es.bsc.compss.nio.worker.executors.Executor$JobExecutionException: Can not get method by reflection
at es.bsc.compss.nio.worker.executors.JavaExecutor.executeTask(JavaExecutor.java:142)
at es.bsc.compss.nio.worker.executors.Executor.execute(Executor.java:42)
at es.bsc.compss.nio.worker.JobLauncher.executeTask(JobLauncher.java:46)
at es.bsc.compss.nio.worker.JobLauncher.processRequests(JobLauncher.java:34)
at es.bsc.compss.util.RequestDispatcher.run(RequestDispatcher.java:46)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoSuchMethodException: simple.Simple.increment(java.lang.String)
at java.lang.Class.getMethod(Class.java:1678)
at es.bsc.compss.nio.worker.executors.JavaExecutor.executeTask(JavaExecutor.java:140)
... 5 more
This error is due to the fact that COMPSs cannot find one of the tasks declared in the Java Interface. Commonly this is triggered by one of the following errors:
- The declaringClass of the tasks in the Java Interface has not been correctly defined.
- The parameters of the tasks in the Java Interface do not match the task call.
- The tasks have not been defined as public.
Jobs failed on reflect target invocation null pointer
When executing an application the main code gets stuck executing a task.
Taking a look at the runtime.log users can check that the job
associated to the task has failed (and all its resubmissions too). Then,
opening the jobX_NEW.out or the jobX_NEW.err files users find
the following error:
[ERROR|es.bsc.compss.Worker|Executor]
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at es.bsc.compss.nio.worker.executors.JavaExecutor.executeTask(JavaExecutor.java:154)
at es.bsc.compss.nio.worker.executors.Executor.execute(Executor.java:42)
at es.bsc.compss.nio.worker.JobLauncher.executeTask(JobLauncher.java:46)
at es.bsc.compss.nio.worker.JobLauncher.processRequests(JobLauncher.java:34)
at es.bsc.compss.util.RequestDispatcher.run(RequestDispatcher.java:46)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at simple.Ll.printY(Ll.java:25)
at simple.Simple.task(Simple.java:72)
... 10 more
This cause of this error is that the Java object accessed by the task has not been correctly transferred and one or more of its fields is null. The transfer failure is normally caused because the transferred object is not serializable.
Users should check that all the object parameters in the task are either implementing the serializable interface or following the java beans model (by implementing an empty constructor and getters and setters for each attribute).
Tracing merge failed: too many open files
When too many nodes and threads are instrumented, the tracing merge can fail due to an OS limitation, namely: the maximum open files. This problem usually happens when using advanced mode due to the larger number of threads instrumented. To overcome this issue users have two choices. First option, use Extrae parallel MPI merger. This merger is automatically used if COMPSs was installed with MPI support. In Ubuntu you can install the following packets to get MPI support:
$ sudo apt-get install libcr-dev mpich2 mpich2-doc
Please note that extrae is never compiled with MPI support when building it locally (with buildlocal command).
To check if COMPSs was deployed with MPI support, you can check the installation log and look for the following Extrae configuration output:
Package configuration for Extrae VERSION based on extrae/trunk rev. 3966:
-----------------------
Installation prefix: /gpfs/apps/MN3/COMPSs/Trunk/Dependencies/extrae
Cross compilation: no
CC: gcc
CXX: g++
Binary type: 64 bits
MPI instrumentation: yes
MPI home: /apps/OPENMPI/1.8.1-mellanox
MPI launcher: /apps/OPENMPI/1.8.1-mellanox/bin/mpirun
On the other hand, if you already installed COMPSs, you can check
Extrae configuration executing the script
/opt/COMPSs/Dependencies/extrae/etc/configured.sh. Users should
check that flags --with-mpi=/usr and --enable-parallel-merge are
present and that MPI path is correct and exists. Sample output:
EXTRAE_HOME is not set. Guessing from the script invoked that Extrae was installed in /opt/COMPSs/Dependencies/extrae
The directory exists .. OK
Loaded specs for Extrae from /opt/COMPSs/Dependencies/extrae/etc/extrae-vars.sh
Extrae SVN branch extrae/trunk at revision 3966
Extrae was configured with:
$ ./configure --enable-gettimeofday-clock --without-mpi --without-unwind --without-dyninst --without-binutils --with-mpi=/usr --enable-parallel-merge --with-papi=/usr --with-java-jdk=/usr/lib/jvm/java-7-openjdk-amd64/ --disable-openmp --disable-nanos --disable-smpss --prefix=/opt/COMPSs/Dependencies/extrae --with-mpi=/usr --enable-parallel-merge --libdir=/opt/COMPSs/Dependencies/extrae/lib
CC was gcc
CFLAGS was -g -O2 -fno-optimize-sibling-calls -Wall -W
CXX was g++
CXXFLAGS was -g -O2 -fno-optimize-sibling-calls -Wall -W
MPI_HOME points to /usr and the directory exists .. OK
LIBXML2_HOME points to /usr and the directory exists .. OK
PAPI_HOME points to /usr and the directory exists .. OK
DYNINST support seems to be disabled
UNWINDing support seems to be disabled (or not needed)
Translating addresses into source code references seems to be disabled (or not needed)
Please, report bugs to tools@bsc.es
Important
Disclaimer: the parallel merge with MPI will not bypass the system’s maximum number of open files, just distribute the files among the resources. If all resources belong to the same machine, the merge will fail anyways.
The second option is to increase the OS maximum number of open files. For instance, in Ubuntu add `` ulimit -n 40000 `` just before the start-stop-daemon line in the do_start section.
Known Limitations
The current COMPSs version has the following limitations:
Global
- Exceptions
- The current COMPSs version is not able to propagate exceptions raised from a task to the master. However, the runtime catches any exception and sets the task as failed.
- Use of file paths
- The persistent workers implementation has a unique Working Directory per worker. That means that tasks should not use hardcoded file names to avoid file collisions and tasks misbehaviours. We recommend to use files declared as task parameters, or to manually create a sandbox inside each task execution and/or to generate temporary random file names.
With Java Applications
- Java tasks
- Java tasks must be declared as public. Despite the fact that tasks can be defined in the main class or in other ones, we recommend to define the tasks in a separated class from the main method to force its public declaration.
- Java objects
- Objects used by tasks must follow the java beans model (implementing an empty constructor and getters and setters for each attribute) or implement the serializable interface. This is due to the fact that objects will be transferred to remote machines to execute the tasks.
- Java object aliasing
If a task has an object parameter and returns an object, the returned value must be a new object (or a cloned one) to prevent any aliasing with the task parameters.
// @Method(declaringClass = "...") // DummyObject incorrectTask ( // @Parameter(type = Type.OBJECT, direction = Direction.IN) DummyObject a, // @Parameter(type = Type.OBJECT, direction = Direction.IN) DummyObject b // ); public DummyObject incorrectTask (DummyObject a, DummyObject b) { if (a.getValue() > b.getValue()) { return a; } return b; } // @Method(declaringClass = "...") // DummyObject correctTask ( // @Parameter(type = Type.OBJECT, direction = Direction.IN) DummyObject a, // @Parameter(type = Type.OBJECT, direction = Direction.IN) DummyObject b // ); public DummyObject correctTask (DummyObject a, DummyObject b) { if (a.getValue() > b.getValue()) { return a.clone(); } return b.clone(); } public static void main() { DummyObject a1 = new DummyObject(); DummyObject b1 = new DummyObject(); DummyObject c1 = new DummyObject(); c1 = incorrectTask(a1, b1); System.out.println("Initial value: " + c1.getValue()); a1.modify(); b1.modify(); System.out.println("Aliased value: " + c1.getValue()); DummyObject a2 = new DummyObject(); DummyObject b2 = new DummyObject(); DummyObject c2 = new DummyObject(); c2 = incorrectTask(a2, b2); System.out.println("Initial value: " + c2.getValue()); a2.modify(); b2.modify(); System.out.println("Non-aliased value: " + c2.getValue()); }
With Python Applications
- Python constraints in the cloud
- When using python applications with constraints in the cloud the minimum number of VMs must be set to 0 because the initial VM creation does not respect the tasks contraints. Notice that if no contraints are defined the initial VMs are still usable.
- Intermediate files
- Some applications may generate intermediate files that are only used among tasks and are never needed inside the master’s code. However, COMPSs will transfer back these files to the master node at the end of the execution. Currently, the only way to avoid transferring these intermediate files is to manually erase them at the end of the master’s code. Users must take into account that this only applies for files declared as task parameters and not for files created and/or erased inside a task.
- User defined classes in Python
- User defined classes in Python must not be declared in the same file
that contains the main method (
if __name__==__main__') to avoid serialization problems of the objects. - Python object hierarchy dependency detection
Dependencies are detected only on the objects that are task parameters or outputs. Consider the following code:
# a.py class A: def __init__(self, b): self.b = b # main.py from a import A from pycompss.api.task import task from pycompss.api.parameter import * from pycompss.api.api import compss_wait_on @task(obj = IN, returns = int) def get_b(obj): return obj.b @task(obj = INOUT) def inc(obj): obj += [1] def main(): my_a = A([5]) inc(my_a.b) obj = get_b(my_a) obj = compss_wait_on(obj) print obj if __name__ == '__main__': main()
Note that there should exist a dependency between
AandA.b. However, PyCOMPSs is not capable to detect dependencies of that kind. These dependencies must be handled (and avoided) manually.- Python modules with global states
- Some modules (for example
logging) have internal variables apart from functions. These modules are not guaranteed to work in PyCOMPSs due to the fact that master and worker code are executed in different interpreters. For instance, if aloggingconfiguration is set on some worker, it will not be visible from the master interpreter instance. - Python global variables
- This issue is very similar to the previous one. PyCOMPSs does not guarantee that applications that create or modify global variables while worker code is executed will work. In particular, this issue (and the previous one) is due to Python’s Global Interpreter Lock (GIL).
- Python application directory as a module
If the Python application root folder is a python module (i.e: it contains an
__init__.pyfile) thenruncompssmust be called from the parent folder. For example, if the Python application is in a folder with an__init__.pyfile namedmy_folderthen PyCOMPSs will resolve all functions, classes and variables asmy_folder.object_nameinstead ofobject_name. For example, consider the following file tree:my_apps/ └── kmeans/ ├── __init__.py └── kmeans.pyThen the correct command to call this app is
runcompss kmeans/kmeans.pyfrom themy_appsdirectory.- Python early program exit
- All intentional, premature exit operations must be done with
sys.exit. PyCOMPSs needs to perform some cleanup tasks before exiting and, if an early exit is performed withsys.exit, the event will be captured, allowing PyCOMPSs to perform these tasks. If the exit operation is done in a different way then there is no guarantee that the application will end properly. - Python with numpy and MKL
- Tasks that invoke numpy and MKL may experience issues if tasks use a different number of MKL threads. This is due to the fact that MKL reuses threads along different calls and it does not change the number of threads from one call to another.
With Services
- Services types
- The current COMPSs version only supports SOAP based services that implement the WS interoperability standard. REST services are not supported.