Python examples
Exception in the main code
Consider the following code where an intended error in the main code has been introduced to show how it can be debugged.
from pycompss.api.task import task
@task(returns=1)
def increment(value):
return value + 1
def main():
initial_value = 1
result = increment(initial_value)
result = result + 1 # Try to use result without synchronizing it: Error
print("Result: " + str(result))
if __name__=='__main__':
main()
When executed, it produces the following output:
$ runcompss error_in_main.py
[ INFO] Inferred PYTHON language
[ INFO] Using default location for project file: /opt/COMPSs//Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs//Runtime/configuration/xml/resources/default_resources.xml
[ INFO] Using default execution type: compss
----------------- Executing error_in_main.py --------------------------
WARNING: COMPSs Properties file is null. Setting default values
[(377) API] - Starting COMPSs Runtime v3.3
[ ERROR ]: An exception occurred: unsupported operand type(s) for +: 'Future' and 'int'
Traceback (most recent call last):
File "/opt/COMPSs//Bindings/python/3/pycompss/runtime/launch.py", line 204, in compss_main
execfile(APP_PATH, globals()) # MAIN EXECUTION
File "error_in_main.py", line 16, in <module>
main()
File "error_in_main.py", line 11, in main
result = result + 1 # Try to use result without synchronizing it: Error
TypeError: unsupported operand type(s) for +: 'Future' and 'int'
[ERRMGR] - WARNING: Task 1(Action: 1) with name error_in_main.increment has been cancelled.
[ERRMGR] - WARNING: Task canceled: [[Task id: 1], [Status: CANCELED], [Core id: 0], [Priority: false], [NumNodes: 1], [MustReplicate: false], [MustDistribute: false], [error_in_main.increment(INT_T)]]
[(3609) API] - Execution Finished
Error running application
It can be identified the complete trackeback pointing where the error is, and
the reason. In this example, the reason is
TypeError: unsupported operand type(s) for +: 'Future' and 'int'
since we are trying to use an object that has not been synchronized.
Tip
Any exception raised from the main code will appear in the same way, showing the traceback helping to idenftiy the line which produced the exception and its reason.
Exception in a task
Consider the following code where an intended error in a task code has been introduced to show how it can be debugged.
from pycompss.api.task import task
from pycompss.api.api import compss_wait_on
@task(returns=1)
def increment(value):
return value + 1 # value is an string, can not add an int: Error
def main():
initial_value = "1" # the initial value is a string instead of an integer
result = increment(initial_value)
result = compss_wait_on(result)
print("Result: " + str(result))
if __name__=='__main__':
main()
When executed, it produces the following output:
$ runcompss error_in_task.py
[ INFO] Inferred PYTHON language
[ INFO] Using default location for project file: /opt/COMPSs//Runtime/configuration/xml/projects/default_project.xml
[ INFO] Using default location for resources file: /opt/COMPSs//Runtime/configuration/xml/resources/default_resources.xml
[ INFO] Using default execution type: compss
----------------- Executing error_in_task.py --------------------------
WARNING: COMPSs Properties file is null. Setting default values
[(570) API] - Starting COMPSs Runtime v3.3
[ERRMGR] - WARNING: Job 1 for running task 1 on worker localhost has failed; resubmitting task to the same worker.
[ERRMGR] - WARNING: Task 1 execution on worker localhost has failed; rescheduling task execution. (changing worker)
[ERRMGR] - WARNING: Job 2 for running task 1 on worker localhost has failed; resubmitting task to the same worker.
[ERRMGR] - WARNING: Task 1 has already been rescheduled; notifying task failure.
[ERRMGR] - WARNING: Task 'error_in_task.increment' TOTALLY FAILED.
Possible causes:
-Exception thrown by task 'error_in_task.increment'.
-Expected output files not generated by task 'error_in_task.increment'.
-Could not provide nor retrieve needed data between master and worker.
Check files '/home/user/.COMPSs/error_in_task.py_01/jobs/job[1|2'] to find out the error.
[ERRMGR] - ERROR: Task failed: [[Task id: 1], [Status: FAILED], [Core id: 0], [Priority: false], [NumNodes: 1], [MustReplicate: false], [MustDistribute: false], [error_in_task.increment(STRING_T)]]
[ERRMGR] - Shutting down COMPSs...
[(4711) API] - Execution Finished
Shutting down the running process
Error running application
The output describes that there has been an issue with the task number 1. Since the default behaviour of the runtime is to resubmit the failed task, task 2 also fails.
In this case, the runtime suggests to check the log files of the tasks:
/home/user/.COMPSs/error_in_task.py_01/jobs/job[1|2]
Looking into the logs folder, it can be seen that the jobs
folder contains
the logs of the failed tasks:
$HOME/.COMPSs
└── error_in_task.py_01
├── jobs
│ ├── job1_NEW.err
│ ├── job1_NEW.out
│ ├── job1_RESUBMITTED.err
│ ├── job1_RESUBMITTED.out
│ ├── job2_NEW.err
│ ├── job2_NEW.out
│ ├── job2_RESUBMITTED.err
│ └── job2_RESUBMITTED.out
├── resources.log
├── runtime.log
├── tmpFiles
└── workers
And the job1_NEW.err
contains the complete traceback of the exception that
has been raised (TypeError: cannot concatenate 'str' and 'int' objects
as
consequence of using a string for the task input which tries to add 1):
[EXECUTOR] executeTask - Error in task execution
es.bsc.compss.types.execution.exceptions.JobExecutionException: Job 1 exit with value 1
at es.bsc.compss.invokers.external.piped.PipedInvoker.invokeMethod(PipedInvoker.java:78)
at es.bsc.compss.invokers.Invoker.invoke(Invoker.java:352)
at es.bsc.compss.invokers.Invoker.processTask(Invoker.java:287)
at es.bsc.compss.executor.Executor.executeTask(Executor.java:486)
at es.bsc.compss.executor.Executor.executeTaskWrapper(Executor.java:322)
at es.bsc.compss.executor.Executor.execute(Executor.java:229)
at es.bsc.compss.executor.Executor.processRequests(Executor.java:198)
at es.bsc.compss.executor.Executor.run(Executor.java:153)
at es.bsc.compss.executor.utils.ExecutionPlatform$2.run(ExecutionPlatform.java:178)
at java.lang.Thread.run(Thread.java:748)
Traceback (most recent call last):
File "/opt/COMPSs/Bindings/python/2/pycompss/worker/commons/worker.py", line 265, in task_execution
**compss_kwargs)
File "/opt/COMPSs/Bindings/python/2/pycompss/api/task.py", line 267, in task_decorator
return self.worker_call(*args, **kwargs)
File "/opt/COMPSs/Bindings/python/2/pycompss/api/task.py", line 1523, in worker_call
**user_kwargs)
File "/home/user/temp/Bugs/documentation/error_in_task.py", line 6, in increment
return value + 1
TypeError: cannot concatenate 'str' and 'int' objects
Tip
Any exception raised from the task code will appear in the same way, showing the traceback helping to identify the line which produced the exception and its reason.