1.5. PyCOMPSs: Using objects, lists, and synchronization. Using collections.

In this example we will see how classes and objects can be used from PyCOMPSs, and that class methods can become tasks. The example also illustrates the use of collections

Import the PyCOMPSs library

[1]:
import pycompss.interactive as ipycompss

Start the runtime

Initialize COMPSs runtime Parameters indicates if the execution will generate task graph, tracefile, monitor interval and debug information.

[2]:
import os
if 'BINDER_SERVICE_HOST' in os.environ:
    ipycompss.start(graph=True, debug=True,
                    project_xml='../xml/project.xml',
                    resources_xml='../xml/resources.xml')
else:
    ipycompss.start(graph=True, monitor=1000, debug=True, trace=False)
******************************************************
*************** PyCOMPSs Interactive *****************
******************************************************
*          .-~~-.--.           _____       ________  *
*         :         )         |____ \     |____   /  *
*   .~ ~ -.\       /.- ~~ .     ___) |        /  /   *
*   >       `.   .'       <    / ___/        /  /    *
*  (         .- -.         )  | |___   _    /  /     *
*   `- -.-~  `- -'  ~-.- -'   |_____| |_|  /__/      *
*     (        :        )           _ _ .-:          *
*      ~--.    :    .--~        .-~  .-~  }          *
*          ~-.-^-.-~ \_      .~  .-~   .~            *
*                   \ \ '     \ '_ _ -~              *
*                    \`.\`.    //                    *
*           . - ~ ~-.__\`.\`-.//                     *
*       .-~   . - ~  }~ ~ ~-.~-.                     *
*     .' .-~      .-~       :/~-.~-./:               *
*    /_~_ _ . - ~                 ~-.~-._            *
*                                     ~-.<           *
******************************************************
* - Starting COMPSs runtime...                       *
* - Log path : /home/user/.COMPSs/Interactive_01/
* - PyCOMPSs Runtime started... Have fun!            *
******************************************************

Importing task and arguments directionality modules

Import task module before annotating functions or methods

[3]:
from pycompss.api.api import compss_barrier
from pycompss.api.api import compss_wait_on
from pycompss.api.task import task
from pycompss.api.parameter import *

Declaring a class

[4]:
%%writefile my_shaper.py

from pycompss.api.task import task
from pycompss.api.parameter import IN

class Shape(object):
    def __init__(self,x,y):
        self.x = x
        self.y = y
        description = "This shape has not been described yet"

    @task(returns=int, target_direction=IN)
    def area(self):
        import time
        time.sleep(4)
        return self.x * self.y

    @task()
    def scaleSize(self,scale):
        import time
        time.sleep(4)
        self.x = self.x * scale
        self.y = self.y * scale

    @task(returns=int, target_direction=IN)
    def perimeter(self):
        import time
        time.sleep(4)
        return 2 * self.x + 2 * self.y

    def describe(self,text):
        self.description = text

    @task(target_direction=IN)
    def infoShape(self):
        import time
        time.sleep(1)
        print('Shape x=', self.x, 'y= ', self.y)
Overwriting my_shaper.py
[5]:
#Operations with collections: previous to release 2.5
@task(returns=1)
def addAll(*mylist):
    import time
    time.sleep(1)
    sum = 0
    for ll in mylist:
        sum = sum + ll
    return sum
Task definition detected.
Found task: addAll
[6]:
@task(returns=int, mylist = COLLECTION_IN)
def addAll_C(mylist):
    import time
    time.sleep(4)
    sum = 0
    for ll in mylist:
        sum = sum + ll
    return sum
Task definition detected.
Found task: addAll_C
[7]:
@task(returns=2, mylist = COLLECTION_IN, my_otherlist = COLLECTION_IN)
def addAll_C2(mylist, my_otherlist):
    import time
    time.sleep(4)
    sum = 0
    sum2 = 0
    for ll in mylist:
        sum = sum + ll
    for jj in my_otherlist:
        sum2 = sum2 + jj
    return sum, sum2
Task definition detected.
Found task: addAll_C2
[8]:
@task (mylist = COLLECTION_INOUT)
def scale_all (mylist, scale):
    import time
    time.sleep(4)
    for ll in mylist:
        ll.x = ll.x * scale
        ll.y = ll.y * scale
Task definition detected.
Found task: scale_all

Invoking tasks

[9]:
from my_shaper import Shape
[10]:
my_shapes = []
my_shapes.append(Shape(100,45))
my_shapes.append(Shape(50,50))
my_shapes.append(Shape(10,100))
my_shapes.append(Shape(20,30))
[11]:
all_areas = []
[12]:
for this_shape in my_shapes:
    all_areas.append(this_shape.area())

Synchronizing results from tasks

[13]:
all_areas = compss_wait_on(all_areas)
print(all_areas)
[4500, 2500, 1000, 600]
[14]:
rectangle = Shape(200,25)
rectangle.scaleSize(5)
area_rectangle = rectangle.area()
rectangle = compss_wait_on(rectangle)
print('X =', rectangle.x)
area_rectangle = compss_wait_on(area_rectangle)
print('Area =', area_rectangle)
('X =', 1000)
('Area =', 125000)

Accessing data in collections

[15]:
all_perimeters=[]
my_shapes.append(rectangle)
for this_shape in my_shapes:
    all_perimeters.append(this_shape.perimeter())
[16]:
mysum = addAll_C(all_perimeters)
mysum = compss_wait_on(mysum)
print(mysum)
3060
[17]:
# Previous version without collections
# mysum = addAll(*all_perimeters)
# mysum = compss_wait_on(mysum)
# print(mysum)

Accessing two collections

[18]:
all_perimeters=[]
all_areas=[]
for this_shape in my_shapes:
    all_perimeters.append(this_shape.perimeter())
    all_areas.append(this_shape.area())
[19]:
[my_per, my_area] = addAll_C2(all_perimeters, all_areas)
[my_per, my_area] = compss_wait_on([my_per, my_area])
print([my_per, my_area])
[3060, 133600]

Scattering data from a collection

[20]:
scale_all(my_shapes,2)
scaled_areas=[]
for this_shape in my_shapes:
    scaled_areas.append(this_shape.area())

scaled_areas = compss_wait_on(scaled_areas)
print(scaled_areas)
[18000, 10000, 4000, 2400, 500000]

Stop the runtime

[21]:
ipycompss.stop(sync=True)
****************************************************
*************** STOPPING PyCOMPSs ******************
****************************************************
Synchronizing all future objects left on the user scope.
Found an object to synchronize: this_shape
****************************************************