Quickstart

Install COMPSs

  • Choose the installation method:

Requirements:

  • Ensure that the required system Dependencies are installed.

  • Check that your JAVA_HOME environment variable points to the Java JDK folder, that the GRADLE_HOME environment variable points to the GRADLE folder, and the gradle binary is in the PATH environment variable.

  • Enable SSH passwordless to localhost. See Configure SSH passwordless.

COMPSs will be installed within the $HOME/.local/ folder (or alternatively within the active virtual environment).

$ pip install pycompss -v

Important

Please, update the environment after installing COMPSs:

$ source ~/.bashrc  # or alternatively reboot the machine

If installed within a virtual environment, deactivate and activate it to ensure that the environment is propperly updated.

Warning

If using Ubuntu 18.04 or higher, you will need to comment some lines of your .bashrc and do a complete logout. Please, check the Post installation Section for detailed instructions.

See Installation and Administration section for more information

Warning

For macOS distributions, only installations local to the user are supported (both with pip and building from sources). This is due to the System Integrity Protection (SIP) implemented in the newest versions of macOS, that does not allow modifications in the /System directory, even when having root permissions in the machine.

Write your first app

Choose your flavour:

Application Overview

A COMPSs application is composed of three parts:

  • Main application code: the code that is executed sequentially and contains the calls to the user-selected methods that will be executed by the COMPSs runtime as asynchronous parallel tasks.

  • Remote methods code: the implementation of the tasks.

  • Task definition interface: It is a Java annotated interface which declares the methods to be run as remote tasks along with metadata information needed by the runtime to properly schedule the tasks.

The main application file name has to be the same of the main class and starts with capital letter, in this case it is Simple.java. The Java annotated interface filename is application name + Itf.java, in this case it is SimpleItf.java. And the code that implements the remote tasks is defined in the application name + Impl.java file, in this case it is SimpleImpl.java.

All code examples are in the /home/compss/tutorial_apps/java/ folder of the development environment.

Main application code

In COMPSs, the user’s application code is kept unchanged, no API calls need to be included in the main application code in order to run the selected tasks on the nodes.

The COMPSs runtime is in charge of replacing the invocations to the user-selected methods with the creation of remote tasks also taking care of the access to files where required. Let’s consider the Simple application example that takes an integer as input parameter and increases it by one unit.

The main application code of Simple application is shown in the following code block. It is executed sequentially until the call to the increment() method. COMPSs, as mentioned above, replaces the call to this method with the generation of a remote task that will be executed on an available node.

Code 1 Simple in Java (Simple.java)
package simple;

import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import simple.SimpleImpl;

public class Simple {

  public static void main(String[] args) {
    String counterName = "counter";
    int initialValue = args[0];

    //--------------------------------------------------------------//
    // Creation of the file which will contain the counter variable //
    //--------------------------------------------------------------//
    try {
       FileOutputStream fos = new FileOutputStream(counterName);
       fos.write(initialValue);
       System.out.println("Initial counter value is " + initialValue);
       fos.close();
    }catch(IOException ioe) {
       ioe.printStackTrace();
    }

    //----------------------------------------------//
    //           Execution of the program           //
    //----------------------------------------------//
    SimpleImpl.increment(counterName);

    //----------------------------------------------//
    //    Reading from an object stored in a File   //
    //----------------------------------------------//
    try {
       FileInputStream fis = new FileInputStream(counterName);
       System.out.println("Final counter value is " + fis.read());
       fis.close();
    }catch(IOException ioe) {
       ioe.printStackTrace();
    }
  }
}

Remote methods code

The following code contains the implementation of the remote method of the Simple application that will be executed remotely by COMPSs.

Code 2 Simple Implementation (SimpleImpl.java)
package simple;

import  java.io.FileInputStream;
import  java.io.FileOutputStream;
import  java.io.IOException;
import  java.io.FileNotFoundException;

public class SimpleImpl {
  public static void increment(String counterFile) {
    try{
      FileInputStream fis = new FileInputStream(counterFile);
      int count = fis.read();
      fis.close();
      FileOutputStream fos = new FileOutputStream(counterFile);
      fos.write(++count);
      fos.close();
    }catch(FileNotFoundException fnfe){
      fnfe.printStackTrace();
    }catch(IOException ioe){
      ioe.printStackTrace();
    }
  }
}

Task definition interface

This Java interface is used to declare the methods to be executed remotely along with Java annotations that specify the necessary metadata about the tasks. The metadata can be of three different types:

  1. For each parameter of a method, the data type (currently File type, primitive types and the String type are supported) and its directions (IN, OUT, INOUT, COMMUTATIVE or CONCURRENT).

  2. The Java class that contains the code of the method.

  3. The constraints that a given resource must fulfill to execute the method, such as the number of processors or main memory size.

The task description interface of the Simple app example is shown in the following figure. It includes the description of the Increment() method metadata. The method interface contains a single input parameter, a string containing a path to the file counterFile. In this example there are constraints on the minimum number of processors and minimum memory size needed to run the method.

Code 3 Interface of the Simple application (SimpleItf.java)
package simple;

import  es.bsc.compss.types.annotations.Constraints;
import  es.bsc.compss.types.annotations.task.Method;
import  es.bsc.compss.types.annotations.Parameter;
import  es.bsc.compss.types.annotations.parameter.Direction;
import  es.bsc.compss.types.annotations.parameter.Type;

public interface SimpleItf {

  @Constraints(computingUnits = "1", memorySize = "0.3")
  @Method(declaringClass = "simple.SimpleImpl")
  void increment(
      @Parameter(type = Type.FILE, direction = Direction.INOUT)
      String file
  );

}

Application compilation

A COMPSs Java application needs to be packaged in a jar file containing the class files of the main code, of the methods implementations and of the Itf annotation. This jar package can be generated using the commands available in the Java SDK or creating your application as a Apache Maven project.

To integrate COMPSs in the maven compile process you just need to add the compss-api artifact as dependency in the application project.

<dependencies>
    <dependency>
        <groupId>es.bsc.compss</groupId>
        <artifactId>compss-api</artifactId>
        <version>${compss.version}</version>
    </dependency>
</dependencies>

To build the jar in the maven case use the following command

$ mvn package

Next we provide a set of commands to compile the Java Simple application (detailed at Java Sample applications).

$ cd tutorial_apps/java/simple/src/main/java/simple/
$~/tutorial_apps/java/simple/src/main/java/simple$ javac *.java
$~/tutorial_apps/java/simple/src/main/java/simple$ cd ..
$~/tutorial_apps/java/simple/src/main/java$ jar cf simple.jar simple/
$~/tutorial_apps/java/simple/src/main/java$ mv ./simple.jar ../../../jar/

In order to properly compile the code, the CLASSPATH variable has to contain the path of the compss-engine.jar package. The default COMPSs installation automatically add this package to the CLASSPATH; please check that your environment variable CLASSPATH contains the compss-engine.jar location by running the following command:

$ echo $CLASSPATH | grep compss-engine

If the result of the previous command is empty it means that you are missing the compss-engine.jar package in your classpath. We recommend to automatically load the variable by editing the .bashrc file:

$ echo "# COMPSs variables for Java compilation" >> ~/.bashrc
$ echo "export CLASSPATH=$CLASSPATH:/opt/COMPSs/Runtime/compss-engine.jar" >> ~/.bashrc

Application execution

A Java COMPSs application is executed through the runcompss script. An example of an invocation of the script is:

$ runcompss --classpath=/home/compss/tutorial_apps/java/simple/jar/simple.jar simple.Simple 1

A comprehensive description of the runcompss command is available in the Executing COMPSs applications section.

In addition to Java, COMPSs supports the execution of applications written in other languages by means of bindings. A binding manages the interaction of the no-Java application with the COMPSs Java runtime, providing the necessary language translation.

Useful information

Choose your flavour: