NumPy/SciPy Cheat Sheet
This cheat sheet is a quick reference for NumPy / SciPy beginners and gives an overview about the most important commands and functions of NumPy and SciPy that you might need on solving the exercise sheets about Linear Algebra in Information Retrieval. It doesn't claim to be complete and will be extended continuously. If you think that some important thing is missing or if you find any errors, please let us know.
General
What is NumPy?
A library that allows to work with arrays and matrices in Python.
What is SciPy?
Another library built upon NumPy that provides advanced Linear Algebra stuff.
Install
The routine to install NumPy and SciPy depends on your operating system.
Linux (Ubuntu, Debian)
apt-get install python-numpy python-scipy
Other systems (Windows, Mac, etc.)
For all other systems (Windows, Mac, etc.) see the instructions given on the offical SciPy website.
Matrix construction
We distinguish between dense matrices and sparse matrices. Dense matrices store every entry in the matrix, while sparse matrices only store the non-zero entries (together with their row and column index). Dense matrices are more feature-rich, but may consume more memory space than sparse matrices (in particular if most of the entries in a matrix are zero).
Dense matrices
In NumPy, there are two concepts of dense matrices: matrices and arrays. Matrices are strictly 2-dimensional, while arrays are n-dimensional (the term array is a bit misleading here).
Construct a matrix:
numpy.matrix(arg, dtype=None)
arg:
The data to construct the matrix from, given as
* a standard Python array; or
* a string with columns separated by commas or spaces and rows separated by semicolons.
dtype (str, optional):
The type of the entries in the matrix (e.g., 'integer', 'float', 'string', etc.).
----------
Examples:
>>> numpy.matrix("1 2; 3 4")
[[1 2]
[3 4]]
>>> numpy.matrix([[1, 2], [3, 4]], dtype='float')
[[1.0 2.0]
[3.0 4.0]]Construct an array:
numpy.array(arg, dtype=None, ndmin=0)
arg:
The data to construct the matrix from, given as
* a standard array; or
* a function that returns an array.
dtype (str, optional):
The type of the entries in the matrix ('integer', 'float', 'string', etc.).
ndmin (int, optional):
The minimum number of dimensions that the array should have.
----------
Examples:
>>> numpy.array([[1, 2], [3, 4]])
[[1 2]
[3 4]]
>>> numpy.array([[1, 2], [3, 4]], dtype='float')
[[1.0 2.0]
[3.0 4.0]]
>>> numpy.array([[1, 2], [3, 4]], ndmin=3)
[[[1 2]
[3 4]]]There are some utility functions to create special-structured arrays:
(1) Construct an array filled with zeros:
numpy.zeros(shape, dtype=float)
shape (int or sequence of ints):
The dimensions of the array to create.
dtype (str, optional):
The type of the entries in the matrix ('integer', 'float', 'string', etc.).
----------
Examples:
>>> numpy.zeros(3)
[0.0, 0.0, 0.0]
>>> numpy.zeros([3, 2], dtype='int')
[[0 0]
[0 0]
[0 0]](2) Construct an array without initializing the entries (an array with random entries):
numpy.empty(shape, dtype=float)
shape (int or sequence of ints):
The dimensions of the array to create.
dtype (str, optional):
The type of the entries in the matrix ('integer', 'float', 'string', etc.).
----------
Examples:
>>> numpy.empty(3)
[6.95052181e-310 1.74512682e-316 1.58101007e-322]
>>> numpy.empty([3, 2], dtype='int')
[[140045355821992 140045355821992]
[140045136216840 140045136244784]
[140045125643544 140045153116544]]
Sparse matrices
There are two principle concepts of sparse matrices:
Compressed Sparse Row matrix (CSR matrix): entries are stored row by row (sorted by row index first)
Compressed Sparse Column matrix (CSC matrix): entries are stored column by column (sorted by column index first)
Construct a CSR matrix:
scipy.sparse.csr_matrix(arg, shape=None, dtype=None)
arg:
The data to create the CSR matrix from, given as
* a dense matrix; or
* another sparse matrix; or
* a tuple (m, n), to construct an empty matrix with shape (n, m); or
* a tuple (data, (rows, cols), to construct a matrix A where A[rows[k], cols[k]] = data[k]; or
* a tuple (data, indices, indptr)
shape (int or sequence of ints):
The dimensions of the matrix to create.
dtype (str, optional):
The type of the entries in the matrix ('integer', 'float', 'string', etc.).
----------
Examples:
>>> scipy.sparse.csr_matrix([[1, 2, 3], [0, 0, 1], [0, 1, 3]])
[[1 2 3]
[0 0 1]
[0 1 3]] # transformed to dense matrix.
>>> values = [1, 2, 3]
>>> rows = [0, 0, 1]
>>> cols = [0, 1, 3]
>>> scipy.sparse.csr_matrix((values, (rows, columns)), shape=[5, 5], dtype=int)
[[1 1 0 0]
[0 0 0 3]
[0 0 0 0]
[0 0 0 0]] # transformed to dense matrix.
Accessing elements
TODO (Hannah): crazy element access magic, single elements, entire rows, sub-matrices
Matrix operations
Constant addition
Addition of a constant adds it to every element of the matrix (only for dense matrices)
>>> B_dense = numpy.matrix([[2, 1], [3, 4]], dtype=float)
>>> B_dense + 10
matrix([[ 12., 11.],
[ 13., 14.]])
Multiplication by a constant
Multiplication by a constant multiplies every element of the matrix by that constant (both for sparse and dense matrices)
>>> A_sparse = csr_matrix([[1, 0], [0, 1], [3, 2]], dtype=float)
>>> (A_sparse * 10).todense()
matrix([[ 10., 0.],
[ 0., 10.],
[ 30., 20.]])
Multiplication
* produces the normal matrix multiplication between a csr_matrix (sparse) and a numpy matrix (dense).
* produces the element-wise matrix multiplication for numpy arrays (also dense). In these cases Python broadcasts the operands in case their dimensions mismatch.
matrix.dot() produces the normal matrix multiplication between a csr_matrix and a numpy matrix except in the case of a dense.dot(sparse) matrix multiplication.
The result of a matrix multiplication between:
- a sparse and a sparse matrix is sparse
- a sparse and a dense matrix is dense
- a dense and a dense matrix is dense
https://docs.scipy.org/doc/scipy/reference/sparse.html
http://www.scipy-lectures.org/intro/numpy/operations.html
>>> A_sparse = csr_matrix([[1, 0], [0, 1], [3, 2]], dtype=float)
>>> B_dense = numpy.matrix([[2, 1], [3, 4]], dtype=float)
>>> A_dense = A_sparse.todense()
>>> B_sparse = csr_matrix(B_dense)
## Sparse with sparse
>>> C_sparse = A_sparse * B_sparse #(Normal Matrix multiplication, 3x2 matrix with 2x2 matrix)
>>> C_sparse.todense()
matrix([[ 2., 1.],
[ 3., 4.],
[ 12., 11.]])
>>> C_sparse = A_sparse.dot(B_sparse) #(Normal Matrix multiplication, 3x2 matrix with 2x2 matrix)
>>> C_sparse.todense()
matrix([[ 2., 1.],
[ 3., 4.],
[ 12., 11.]])
## Sparse with dense
>>> C_dense = A_sparse * B_dense #(Normal Matrix multiplication, 3x2 matrix with 2x2 matrix)
>>> C_dense
matrix([[ 2., 1.],
[ 3., 4.],
[ 12., 11.]])
>>> C_dense = A_sparse.dot(B_dense) #(Normal Matrix multiplication, 3x2 matrix with 2x2 matrix)
>>> C_dense
matrix([[ 2., 1.],
[ 3., 4.],
[ 12., 11.]])
## Dense with sparse
>>> C_dense = A_dense * B_sparse
>>> C_dense
matrix([[ 2., 1.],
[ 3., 4.],
[ 12., 11.]])
>>> A_dense.dot(B_sparse)
matrix([[ <2x2 sparse matrix of type '<class 'numpy.float64'>'
with 4 stored elements in Compressed Sparse Row format>,
<2x2 sparse matrix of type '<class 'numpy.float64'>'
with 4 stored elements in Compressed Sparse Row format>],
[ <2x2 sparse matrix of type '<class 'numpy.float64'>'
with 4 stored elements in Compressed Sparse Row format>,
<2x2 sparse matrix of type '<class 'numpy.float64'>'
with 4 stored elements in Compressed Sparse Row format>],
[ <2x2 sparse matrix of type '<class 'numpy.float64'>'
with 4 stored elements in Compressed Sparse Row format>,
<2x2 sparse matrix of type '<class 'numpy.float64'>'
with 4 stored elements in Compressed Sparse Row format>]], dtype=object)
## Dense with dense
>>> C_dense = A_dense.dot(B_dense) #(Normal Matrix multiplication, 3x2 matrix with 2x2 matrix)
>>> C_dense
matrix([[ 2., 1.],
[ 3., 4.],
[ 12., 11.]])
>>> C_dense = A_dense * B_dense #(Normal Matrix multiplication, 3x2 matrix with 2x2 matrix)
>>> C_dense
matrix([[ 2., 1.],
[ 3., 4.],
[ 12., 11.]])## numpy.ndarray
>>> A_ndarray = numpy.array([[1, 0], [0, 1], [3, 2]])
>>> B_ndarray = numpy.array([[2, 1], [3, 4]])
>>> C_ndarray = numpy.array([2, 1])
>>> B_ndarray * B_ndarray #(Element-wise Matrix multiplication, 2x2 matrix with 2x2 matrix)
array([[ 4, 1],
[ 9, 16]])
>>> B_ndarray.dot(B_ndarray) #(Normal Matrix multiplication, 2x2 matrix with 2x2 matrix)
array([[ 7, 6],
[18, 19]])
>>> A_ndarray.dot(B_ndarray) #(Normal Matrix multiplication, 3x2 matrix with 2x2 matrix)
array([[ 2, 1],
[ 3, 4],
[12, 11]])
>>> C_ndarray * B_ndarray #(Broadcasting)
array([[4, 1],
[6, 4]])TODO (Claudius): Element-wise operations like taking log, sqrt. Multiplying two m*n matrices element-wise (for example, to square the entries in a matrix etc...)
Row- or column-wise operations
TODO (Claudius): summing of rows or columns, sorting rows / columns etc
Useful methods
numpy.round
Takes an array and rounds its values to the given number of decimals. Note that for values exactly halfway between rounded decimal values, Numpy rounds to the nearest even value. numpy.around
>>> numpy.round([1.98, 2.34, 4.76], 1) [ 2. 2.3 4.8]
>>> numpy.round([1.5, 0.5, 3.5, 4.5], 0) [ 2. 0. 4. 4.]
numpy.min
Takes an array and returns its minimum value. If an axis is specified, returns the minimum along the axis. numpy.amin
>>> numpy.min([[5, 0, 1], [4, 3, 2]]) 0
>>> numpy.min([[5, 0, 1], [4, 3, 2]], axis=0) [4 0 1]
numpy.argmin
Takes an array and returns the index of the minimum value of the flattened array. If an axis is specified, returns the indices of the minimum values along the axis. numpy.argmin
>>> numpy.argmin([[5, 0, 1], [4, 3, 2]]) 1
>>> numpy.argmin([[5, 0, 1], [4, 3, 2]], axis=0) [1 0 0]
numpy.argsort
Takes an array a and returns an array of indices that sort a. Optionally, you can specify the axis along which a will be sorted. By default the axis is -1. numpy.argsort
>>> numpy.argsort([[0, 4, 0], [4, 3, 2]], axis=0) [[0 1 0] [1 0 1]]
>>> numpy.argsort([[0, 4, 0], [4, 3, 2]], axis=1) [[0 2 1] [2 1 0]]
numpy.where
Takes a condition and optionally two array-like objects x and y. If x and y are specified, returns an array that contains elements from x where condition is true and elements from y elsewhere. numpy.where
>>> x = numpy.array([[5, 4, 3], [2, 1, 0]]) >>> y = numpy.array([[0, 1, 2], [3, 4, 5]]) >>> numpy.where(x > 3, x, y) [[5 4 2] [3 4 5]]
Special matrices
Diagonal matrix
Matrix (usually square) in which all entries are zero, except on the main diagonal. Use numpy.diag to either create a diagonal matrix from a given main diagonal, or extract the diagonal matrix from a given matrix.
>>> numpy.diag([1,2,3])
array([[1, 0, 0],
[0, 2, 0],
[0, 0, 3]])>>> numpy.diag([[1, 5, 4],
[7, 2, 4],
[4, 7, 3]])
array([1, 2, 3])For a sparse matrix, use scipy.spare.spdiags.
Identity matrix
Special diagonal m*m matrix where all elements on the main diagonal are 1. Read as the '1' of matrix world. For example, a n*m matrix A multiplied with an m*m identity matrix yields A again. Use numpy.identity(k) to create a k*k identity matrix.
>>> numpy.identity(4)
array([[ 1., 0., 0., 0.],
[ 0., 1., 0., 0.],
[ 0., 0., 1., 0.],
[ 0., 0., 0., 1.]])>>> numpy.array([[1, 2, 3],
[3, 4, 3]]).dot(numpy.identity(3))
array([[ 1., 2., 3.],
[ 3., 4., 3.]])For a sparse matrix, use scipy.sparse.identity.
Triangular matrix
A (square) matrix where all elements below (upper triangle) or above (lower triangle) the main diagonal are zero. numpy.triu creates the upper (u), numpy.tril the lower (l) triangular matrix from a given matrix.
>>> numpy.triu([[1, 5, 4],
[7, 2, 4],
[4, 7, 3]])
array([[1, 5, 4],
[0, 2, 4],
[0, 0, 3]])>>> numpy.tril([[1, 5, 4],
[7, 2, 4],
[4, 7, 3]])
array([[1, 0, 0],
[7, 2, 0],
[4, 7, 3]])For a sparse matrix, use scipy.sparse.triu and scipy.sparse.tril.
Matrix decomposition
Singular Value Decompostion (SVD)
Factorize a matrix A (m*n) into three matrices U (m * r), S (r * r) and V (r * n) such that A = U * S * V. Here r is the rank of A.
Use numpy.linalg.svd to do a singular value decomposition for a dense matrix. Use scipy.sparse.linalg.svds for sparse matrices (computes the largest k singular values for a sparse matrix).
>>> Uk, Sk, Vk = svds(csr_matrix([[1, 2, 3], [3, 4, 5], [5, 6, 4]], dtype=float), 2)
>>> print("Uk:\n", Uk, "\nSk:\n", Sk, "\nVk:\n", Vk)
Uk:
[[ 0.56475636 -0.30288472]
[ 0.51457155 -0.59799935]
[-0.64518709 -0.74206309]]
Sk:
[ 2.13530566 11.67829513]
Vk:
[[-0.52332762 -0.32001209 0.78975975]
[-0.49726421 -0.63794803 -0.58800563]]