1.12. PyCOMPSs: Integration with Numba

In this example we will how to use Numba with PyCOMPSs.

Import the PyCOMPSs library

import pycompss.interactive as ipycompss

Starting runtime

Initialize COMPSs runtime Parameters indicates if the execution will generate task graph, tracefile, monitor interval and debug information.

import os
if 'BINDER_SERVICE_HOST' in os.environ:
    ipycompss.start(graph=True, debug=False,
                    project_xml='../xml/project.xml',
                    resources_xml='../xml/resources.xml')
else:
    ipycompss.start(graph=True, monitor=1000, trace=True, debug=False)

********************************************************
**************** PyCOMPSs Interactive ******************
********************************************************
*          .-~~-.--.           ______         ______   *
*         :         )         |____  \       |____  \  *
*   .~ ~ -.\       /.- ~~ .      __) |          __) |  *
*   >       `.   .'       <     |__  |         |__  |  *
*  (         .- -.         )   ____) |   _    ____) |  *
*   `- -.-~  `- -'  ~-.- -'   |______/  |_|  |______/  *
*     (        :        )           _ _ .-:            *
*      ~--.    :    .--~        .-~  .-~  }            *
*          ~-.-^-.-~ \_      .~  .-~   .~              *
*                   \ \ '     \ '_ _ -~                *
*                    \`.\`.    //                      *
*           . - ~ ~-.__\`.\`-.//                       *
*       .-~   . - ~  }~ ~ ~-.~-.                       *
*     .' .-~      .-~       :/~-.~-./:                 *
*    /_~_ _ . - ~                 ~-.~-._              *
*                                     ~-.<             *
********************************************************
* - Starting COMPSs runtime...                         *
* - Log path : /home/user/.COMPSs/Interactive_12/
* - PyCOMPSs Runtime started... Have fun!              *
********************************************************

Importing task and arguments directionality modules

Import task module before annotating functions or methods

from pycompss.api.task import task
from pycompss.api.parameter import *
from pycompss.api.api import compss_barrier
from pycompss.api.api import compss_wait_on

Importing other modules

Import the time and numpy modules

import time
import numpy as np

Declaring tasks

Declare functions and decorate with @task those that should be tasks – Note that they are exactly the same but the “numba” parameter in the @task decorator

@task(returns=1, numba=False)  # Default: numba=False
def ident_loops(x):
    r = np.empty_like(x)
    n = len(x)
    for i in range(n):
        r[i] = np.cos(x[i]) ** 2 + np.sin(x[i]) ** 2
    return r

@task(returns=1, numba=True)
def ident_loops_jit(x):
    r = np.empty_like(x)
    n = len(x)
    for i in range(n):
        r[i] = np.cos(x[i]) ** 2 + np.sin(x[i]) ** 2
    return r

Invoking tasks

size = 1000000
ntasks = 8

# Run some tasks without numba jit
start = time.time()
for i in range(ntasks):
    out = ident_loops(np.arange(size))
compss_barrier()
end = time.time()

# Run some tasks with numba jit
start_jit = time.time()
for i in range(ntasks):
    out_jit = ident_loops_jit(np.arange(size))
compss_barrier()
end_jit = time.time()

# Get the last result of each run to compare that the results are ok
out = compss_wait_on(out)
out_jit = compss_wait_on(out_jit)

print("TIMING RESULTS:")
print("* ident_loops      : %s seconds" % str(end - start))
print("* ident_loops_jit  : %s seconds" % str(end_jit - start_jit))
if len(out) == len(out_jit) and list(out) == list(out_jit):
    print("* SUCCESS: Results match.")
else:
    print("* FAILURE: Results are different!!!")

Found task: ident_loops
Found task: ident_loops_jit
TIMING RESULTS:
* ident_loops      : 12.379663944244385 seconds
* ident_loops_jit  : 4.676412582397461 seconds
* SUCCESS: Results match.

Stop the runtime

ipycompss.stop(sync=False)

********************************************************
***************** STOPPING PyCOMPSs ********************
********************************************************
Checking if any issue happened.
Warning: some of the variables used with PyCOMPSs may
         have not been brought to the master.
********************************************************