1.7. PyCOMPSs: Using files

In this example we will how files can be used with PyCOMPSs.

Import the PyCOMPSs library

[1]:
import pycompss.interactive as ipycompss

Start the runtime

Initialize COMPSs runtime Parameters indicates if the execution will generate task graph, tracefile, monitor interval and debug information.

[2]:
import os
if 'BINDER_SERVICE_HOST' in os.environ:
    ipycompss.start(graph=True, debug=False,
                    project_xml='../xml/project.xml',
                    resources_xml='../xml/resources.xml')
else:
    ipycompss.start(graph=True, monitor=1000, trace=False, debug=False)
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
*          .-~~-.--.           _____       ________  *
*         :         )         |____ \     |____   /  *
*   .~ ~ -.\       /.- ~~ .     ___) |        /  /   *
*   >       `.   .'       <    / ___/        /  /    *
*  (         .- -.         )  | |___   _    /  /     *
*   `- -.-~  `- -'  ~-.- -'   |_____| |_|  /__/      *
*     (        :        )           _ _ .-:          *
*      ~--.    :    .--~        .-~  .-~  }          *
*          ~-.-^-.-~ \_      .~  .-~   .~            *
*                   \ \ '     \ '_ _ -~              *
*                    \`.\`.    //                    *
*           . - ~ ~-.__\`.\`-.//                     *
*       .-~   . - ~  }~ ~ ~-.~-.                     *
*     .' .-~      .-~       :/~-.~-./:               *
*    /_~_ _ . - ~                 ~-.~-._            *
*                                     ~-.<           *
******************************************************
* - Starting COMPSs runtime...                       *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun!            *
******************************************************

Importing task and parameter modules

Import task module before annotating functions or methods

[3]:
from pycompss.api.task import task
from pycompss.api.parameter import FILE_IN, FILE_OUT, FILE_INOUT
from pycompss.api.api import compss_wait_on, compss_open

Declaring tasks

Declare functions and decorate with @task those that should be tasks

[4]:
@task(fout=FILE_OUT)
def write(fout, content):
    with open(fout, 'w') as fout_d:
        fout_d.write(content)
Found task: write
[5]:
@task(finout=FILE_INOUT)
def append(finout):
    finout_d = open(finout, 'a')
    finout_d.write("\n===> INOUT FILE ADDED CONTENT")
    finout_d.close()
Found task: append
[6]:
@task(fin=FILE_IN, returns=str)
def readFile(fin):
    fin_d = open(fin, 'r')
    content = fin_d.read()
    fin_d.close()
    return content
Found task: readFile

Invoking tasks

[7]:
f = "myFile.txt"
content = "OUT FILE CONTENT"
write(f, content)
[8]:
append(f)
[9]:
readed = readFile(f)
[10]:
append(f)

Accessing data outside tasks requires synchronization

[11]:
readed = compss_wait_on(readed)
print(readed)
OUT FILE CONTENT
===> INOUT FILE ADDED CONTENT
[12]:
with compss_open(f) as fd:
    f_content = fd.read()
print(f_content)
OUT FILE CONTENT
===> INOUT FILE ADDED CONTENT
===> INOUT FILE ADDED CONTENT

Stop the runtime

[13]:
ipycompss.stop(sync=True)
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Synchronizing all future objects left on the user scope.
****************************************************