1.14. Machine Learning with dislib

This tutorial will show the different algorithms available in dislib.

Setup

First, we need to start an interactive PyCOMPSs session:

import os
os.environ["ComputingUnits"] = "1"

import pycompss.interactive as ipycompss
if 'BINDER_SERVICE_HOST' in os.environ:
    ipycompss.start(graph=True,
                    project_xml='../xml/project.xml',
                    resources_xml='../xml/resources.xml')
else:
    ipycompss.start(graph=True, monitor=1000)

********************************************************
**************** PyCOMPSs Interactive ******************
********************************************************
*          .-~~-.--.           ______         ______   *
*         :         )         |____  \       |____  \  *
*   .~ ~ -.\       /.- ~~ .      __) |          __) |  *
*   >       `.   .'       <     |__  |         |__  |  *
*  (         .- -.         )   ____) |   _    ____) |  *
*   `- -.-~  `- -'  ~-.- -'   |______/  |_|  |______/  *
*     (        :        )           _ _ .-:            *
*      ~--.    :    .--~        .-~  .-~  }            *
*          ~-.-^-.-~ \_      .~  .-~   .~              *
*                   \ \ '     \ '_ _ -~                *
*                    \`.\`.    //                      *
*           . - ~ ~-.__\`.\`-.//                       *
*       .-~   . - ~  }~ ~ ~-.~-.                       *
*     .' .-~      .-~       :/~-.~-./:                 *
*    /_~_ _ . - ~                 ~-.~-._              *
*                                     ~-.<             *
********************************************************
* - Starting COMPSs runtime...                         *
* - Log path : /home/user/.COMPSs/Interactive_14/
* - PyCOMPSs Runtime started... Have fun!              *
********************************************************

Next, we import dislib and we are all set to start working!

import dislib as ds

Load the MNIST dataset

x, y = ds.load_svmlight_file('/tmp/mnist/mnist', # Download the dataset
                             block_size=(10000, 784), n_features=784, store_sparse=False)

x.shape

(60000, 784)

y.shape

(60000, 1)

y_array = y.collect()
y_array

array([5., 0., 4., ..., 5., 6., 8.])

img = x[0].collect().reshape(28,28)

%matplotlib inline
import matplotlib.pyplot as plt
plt.imshow(img)

<matplotlib.image.AxesImage at 0x7ff14e688e20>

int(y[0].collect())

5

dislib algorithms

Preprocessing

from dislib.preprocessing import StandardScaler
from dislib.decomposition import PCA

Clustering

from dislib.cluster import KMeans
from dislib.cluster import DBSCAN
from dislib.cluster import GaussianMixture

Classification

from dislib.classification import CascadeSVM
from dislib.classification import RandomForestClassifier

Recommendation

from dislib.recommendation import ALS

Model selection

from dislib.model_selection import GridSearchCV

Others

from dislib.regression import LinearRegression
from dislib.neighbors import NearestNeighbors

Examples

KMeans

kmeans = KMeans(n_clusters=10)
pred_clusters = kmeans.fit_predict(x).collect()

Get the number of images of each class in the cluster 0:

from collections import Counter
Counter(y_array[pred_clusters==0])

Counter({8.0: 3499,
         5.0: 1209,
         3.0: 1058,
         2.0: 323,
         0.0: 121,
         9.0: 54,
         6.0: 45,
         7.0: 21,
         4.0: 16,
         1.0: 9})

GaussianMixture

Fit the GaussianMixture with the painted pixels of a single image:

import numpy as np
img_filtered_pixels = np.stack([np.array([i, j]) for i in range(28) for j in range(28) if img[i,j] > 10])
img_pixels = ds.array(img_filtered_pixels, block_size=(50,2))
gm = GaussianMixture(n_components=7, random_state=0)
gm.fit(img_pixels)

Get the parameters that define the Gaussian components:

from pycompss.api.api import compss_wait_on
means = compss_wait_on(gm.means_)
covariances = compss_wait_on(gm.covariances_)
weights = compss_wait_on(gm.weights_)

Use the Gaussian mixture model to sample random pixels replicating the original distribution:

samples = np.concatenate([np.random.multivariate_normal(means[i], covariances[i], int(weights[i]*1000))
                    for i in range(7)])
plt.scatter(samples[:,1], samples[:,0])
plt.gca().set_aspect('equal', adjustable='box')
plt.gca().invert_yaxis()
plt.draw()

PCA

pca = PCA()
pca.fit(x)

PCA()

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Calculate the explained variance of the 10 first eigenvectors:

explained_variance = pca.explained_variance_.collect()
sum(explained_variance[0:10])/sum(explained_variance)

0.48814980354933996

Show the weights of the first eigenvector:

plt.imshow(np.abs(pca.components_.collect()[0]).reshape(28,28))

<matplotlib.image.AxesImage at 0x7ff144982fe0>

RandomForestClassifier

rf = RandomForestClassifier(n_estimators=5, max_depth=3)
rf.fit(x, y)

RandomForestClassifier(max_depth=3, n_estimators=5)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Use the test dataset to get an accuracy score:

x_test, y_test = ds.load_svmlight_file('/tmp/mnist/mnist.test', block_size=(10000, 784), n_features=784, store_sparse=False)
score = rf.score(x_test, y_test)
print(compss_wait_on(score))

0.6152

Close the session

To finish the session, we need to stop PyCOMPSs:

ipycompss.stop()

********************************************************
***************** STOPPING PyCOMPSs ********************
********************************************************
Checking if any issue happened.
Warning: some of the variables used with PyCOMPSs may
         have not been brought to the master.
********************************************************