1.8. PyCOMPSs: Using files

In this example we will how files can be used with PyCOMPSs.

Import the PyCOMPSs library

import pycompss.interactive as ipycompss

Start the runtime

Initialize COMPSs runtime Parameters indicates if the execution will generate task graph, tracefile, monitor interval and debug information.

import os
if 'BINDER_SERVICE_HOST' in os.environ:
    ipycompss.start(graph=True, debug=False,
                    project_xml='../xml/project.xml',
                    resources_xml='../xml/resources.xml')
else:
    ipycompss.start(graph=True, monitor=1000, trace=False, debug=False)

********************************************************
**************** PyCOMPSs Interactive ******************
********************************************************
*          .-~~-.--.           ______         ______   *
*         :         )         |____  \       |____  \  *
*   .~ ~ -.\       /.- ~~ .      __) |          __) |  *
*   >       `.   .'       <     |__  |         |__  |  *
*  (         .- -.         )   ____) |   _    ____) |  *
*   `- -.-~  `- -'  ~-.- -'   |______/  |_|  |______/  *
*     (        :        )           _ _ .-:            *
*      ~--.    :    .--~        .-~  .-~  }            *
*          ~-.-^-.-~ \_      .~  .-~   .~              *
*                   \ \ '     \ '_ _ -~                *
*                    \`.\`.    //                      *
*           . - ~ ~-.__\`.\`-.//                       *
*       .-~   . - ~  }~ ~ ~-.~-.                       *
*     .' .-~      .-~       :/~-.~-./:                 *
*    /_~_ _ . - ~                 ~-.~-._              *
*                                     ~-.<             *
********************************************************
* - Starting COMPSs runtime...                         *
* - Log path : /home/user/.COMPSs/Interactive_08/
* - PyCOMPSs Runtime started... Have fun!              *
********************************************************

Importing task and parameter modules

Import task module before annotating functions or methods

from pycompss.api.task import task
from pycompss.api.parameter import FILE_IN, FILE_OUT, FILE_INOUT
from pycompss.api.api import compss_wait_on, compss_open

Declaring tasks

Declare functions and decorate with @task those that should be tasks

@task(fout=FILE_OUT)
def write(fout, content):
    with open(fout, 'w') as fout_d:
        fout_d.write(content)

@task(finout=FILE_INOUT)
def append(finout):
    finout_d = open(finout, 'a')
    finout_d.write("\n===> INOUT FILE ADDED CONTENT")
    finout_d.close()

@task(fin=FILE_IN, returns=str)
def readFile(fin):
    fin_d = open(fin, 'r')
    content = fin_d.read()
    fin_d.close()
    return content

Invoking tasks

f = "myFile.txt"
content = "OUT FILE CONTENT"
write(f, content)

Found task: write

append(f)

Found task: append

readed = readFile(f)

Found task: readFile

append(f)

Accessing data outside tasks requires synchronization

readed = compss_wait_on(readed)
print(readed)

OUT FILE CONTENT
===> INOUT FILE ADDED CONTENT

with compss_open(f) as fd:
    f_content = fd.read()
print(f_content)

OUT FILE CONTENT
===> INOUT FILE ADDED CONTENT
===> INOUT FILE ADDED CONTENT

Stop the runtime

ipycompss.stop(sync=True)

********************************************************
***************** STOPPING PyCOMPSs ********************
********************************************************
Checking if any issue happened.
Synchronizing all future objects left on the user scope.
********************************************************