1.10. PyCOMPSs: Other decorators - Binary

In this example we will how to invoke binaries as tasks with PyCOMPSs.

Import the PyCOMPSs library

[1]:
import pycompss.interactive as ipycompss

Start the runtime

Initialize COMPSs runtime Parameters indicates if the execution will generate task graph, tracefile, monitor interval and debug information.

[2]:
import os
if 'BINDER_SERVICE_HOST' in os.environ:
    ipycompss.start(debug=False,
                    project_xml='../xml/project.xml',
                    resources_xml='../xml/resources.xml')
else:
    ipycompss.start(graph=True, monitor=1000, trace=True, debug=True)
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
*          .-~~-.--.           _____       ________  *
*         :         )         |____ \     |____   /  *
*   .~ ~ -.\       /.- ~~ .     ___) |        /  /   *
*   >       `.   .'       <    / ___/        /  /    *
*  (         .- -.         )  | |___   _    /  /     *
*   `- -.-~  `- -'  ~-.- -'   |_____| |_|  /__/      *
*     (        :        )           _ _ .-:          *
*      ~--.    :    .--~        .-~  .-~  }          *
*          ~-.-^-.-~ \_      .~  .-~   .~            *
*                   \ \ '     \ '_ _ -~              *
*                    \`.\`.    //                    *
*           . - ~ ~-.__\`.\`-.//                     *
*       .-~   . - ~  }~ ~ ~-.~-.                     *
*     .' .-~      .-~       :/~-.~-./:               *
*    /_~_ _ . - ~                 ~-.~-._            *
*                                     ~-.<           *
******************************************************
* - Starting COMPSs runtime...                       *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun!            *
******************************************************

Importing task and binary modules

Import task module before annotating functions or methods

[3]:
from pycompss.api.task import task
from pycompss.api.binary import binary
from pycompss.api.parameter import *

Declaring tasks

Declare functions and decorate with @task those that should be tasks and with @binary the ones that execute a binary file

[4]:
@binary(binary="sed")
@task(file=FILE_INOUT)
def sed(flag, expression, file):
    # Equivalent to: $ sed flag expression file
    pass
Task definition detected.
Found task: sed
[5]:
@binary(binary="grep")
@task(infile={Type:FILE_IN, StdIOStream:STDIN}, result={Type:FILE_OUT, StdIOStream:STDOUT})
def grep(keyword, infile, result):
    # Equivalent to: $ grep keyword < infile > result
    pass
Task definition detected.
Found task: grep

Invoking tasks

[6]:
from pycompss.api.api import compss_open

finout = "inoutfile.txt"
with open(finout, 'w') as finout_d:
    finout_d.write("Hi, this a simple test!")
    finout_d.write("\nHow are you?")

sed('-i', 's/Hi/Hello/g', finout)
fout = "outfile.txt"
grep("Hello", finout, fout)

Accessing data outside tasks requires synchronization

[7]:
# Check the result of 'sed'
with compss_open(finout, "r") as finout_r:
    sedresult = finout_r.read()
print(sedresult)
Hello, this a simple test!
How are you?
[8]:
# Check the result of 'grep'
with compss_open(fout, "r") as fout_r:
    grepresult = fout_r.read()
print(grepresult)
Hello, this a simple test!

Stop the runtime

[9]:
ipycompss.stop(sync=True)
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Synchronizing all future objects left on the user scope.
****************************************************