May 20, 2013

Numpy masked arrays

An array with invalid data can alter the results of operations to the array in unintended ways. For example, consider the following case where one invalid value(NaN) pushes the mean to NaN.
In [1]: a = np.array([1,2,nan,4])

In [2]: a
Out[2]: array([  1.,   2.,  nan,   4.])

In [3]: np.mean(a)
Out[3]: nan

In such a scenario, the incorrect value can be masked as illustrated below.
In [6]: b
Out[6]: 
masked_array(data = [1.0 2.0 -- 4.0],
             mask = [False False  True False],
       fill_value = 1e+20)

With the masked array, the mean is computed without considering the masked value.
In [7]: np.mean(b)
Out[7]: 2.3333333333333335

References: http://docs.scipy.org/doc/numpy/reference/maskedarray.html