Scipy教程 - 距离计算库scipy.spatial.distance

港控/mmm° 2022-08-09 00:52 209阅读 0赞

http://[blog.csdn.net/pipisorry/article/details/48814183][blog.csdn.net_pipisorry_article_details_48814183][  
][Link 1]

在scipy.spatial中最重要的模块应该就是距离计算模块distance了。

from scipy import spatial

# 距离计算 #

## 矩阵距离计算函数 ##

矩阵参数每行代表一个观测值，计算结果就是每行之间的metric距离。Distance matrix computation from a collection of raw observation vectors stored in a rectangular array.

### 向量距离计算函数Distance functions between two vectors u and v    ###

Distance functions between two vectors u and v. Computingdistances over a large collection of vectors is inefficient for thesefunctions. Use pdist for this purpose.

输入的参数应该是向量，也就是维度应该是(n, )，当然也可以是(1, n)它会使用squeeze自动去掉维度为1的维度；但是如果是多维向量，至少有两个维度>1就会出错。

e.g. spatial.distance.correlation(u, v)    \#计算向量u和v之间的相关系数（pearson correlation coefficient, Centered Cosine）

Note: 如果向量u和v元素数目都只有一个或者某个向量中所有元素相同（分母norm(u - u.mean())为0），那么相关系数当然计算无效，会返回nan。

<table> 
 <tbody> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.braycurtis.html#scipy.spatial.distance.braycurtis" title="scipy.spatial.distance.braycurtis" rel="nofollow">braycurtis</a>(u,&nbsp;v)</td> 
 <td>Computes the Bray-Curtis distance between two 1-D arrays.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.canberra.html#scipy.spatial.distance.canberra" title="scipy.spatial.distance.canberra" rel="nofollow">canberra</a>(u,&nbsp;v)</td> 
 <td>Computes the Canberra distance between two 1-D arrays.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.chebyshev.html#scipy.spatial.distance.chebyshev" title="scipy.spatial.distance.chebyshev" rel="nofollow">chebyshev</a>(u,&nbsp;v)</td> 
 <td>Computes the Chebyshev distance.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.cityblock.html#scipy.spatial.distance.cityblock" title="scipy.spatial.distance.cityblock" rel="nofollow">cityblock</a>(u,&nbsp;v)</td> 
 <td>Computes the City Block (Manhattan) distance.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.correlation.html#scipy.spatial.distance.correlation" title="scipy.spatial.distance.correlation" rel="nofollow">correlation</a>(u,&nbsp;v)</td> 
 <td>Computes the correlation distance between two 1-D arrays.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.cosine.html#scipy.spatial.distance.cosine" title="scipy.spatial.distance.cosine" rel="nofollow">cosine</a>(u,&nbsp;v)</td> 
 <td>Computes the Cosine distance between 1-D arrays.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.dice.html#scipy.spatial.distance.dice" title="scipy.spatial.distance.dice" rel="nofollow">dice</a>(u,&nbsp;v)</td> 
 <td>Computes the Dice dissimilarity between two boolean 1-D arrays.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.euclidean.html#scipy.spatial.distance.euclidean" title="scipy.spatial.distance.euclidean" rel="nofollow">euclidean</a>(u,&nbsp;v)</td> 
 <td>Computes the Euclidean distance between two 1-D arrays.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.hamming.html#scipy.spatial.distance.hamming" title="scipy.spatial.distance.hamming" rel="nofollow">hamming</a>(u,&nbsp;v)</td> 
 <td>Computes the Hamming distance between two 1-D arrays.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.jaccard.html#scipy.spatial.distance.jaccard" title="scipy.spatial.distance.jaccard" rel="nofollow">jaccard</a>(u,&nbsp;v)</td> 
 <td>Computes the Jaccard-Needham dissimilarity between two boolean 1-D arrays.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.kulsinski.html#scipy.spatial.distance.kulsinski" title="scipy.spatial.distance.kulsinski" rel="nofollow">kulsinski</a>(u,&nbsp;v)</td> 
 <td>Computes the Kulsinski dissimilarity between two boolean 1-D arrays.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.mahalanobis.html#scipy.spatial.distance.mahalanobis" title="scipy.spatial.distance.mahalanobis" rel="nofollow">mahalanobis</a>(u,&nbsp;v,&nbsp;VI)</td> 
 <td>Computes the Mahalanobis distance between two 1-D arrays.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.matching.html#scipy.spatial.distance.matching" title="scipy.spatial.distance.matching" rel="nofollow">matching</a>(u,&nbsp;v)</td> 
 <td>Computes the Matching dissimilarity between two boolean 1-D arrays.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.minkowski.html#scipy.spatial.distance.minkowski" title="scipy.spatial.distance.minkowski" rel="nofollow">minkowski</a>(u,&nbsp;v,&nbsp;p)</td> 
 <td>Computes the Minkowski distance between two 1-D arrays.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.rogerstanimoto.html#scipy.spatial.distance.rogerstanimoto" title="scipy.spatial.distance.rogerstanimoto" rel="nofollow">rogerstanimoto</a>(u,&nbsp;v)</td> 
 <td>Computes the Rogers-Tanimoto dissimilarity between two boolean 1-D arrays.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.russellrao.html#scipy.spatial.distance.russellrao" title="scipy.spatial.distance.russellrao" rel="nofollow">russellrao</a>(u,&nbsp;v)</td> 
 <td>Computes the Russell-Rao dissimilarity between two boolean 1-D arrays.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.seuclidean.html#scipy.spatial.distance.seuclidean" title="scipy.spatial.distance.seuclidean" rel="nofollow">seuclidean</a>(u,&nbsp;v,&nbsp;V)</td> 
 <td>Returns the standardized Euclidean distance between two 1-D arrays.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.sokalmichener.html#scipy.spatial.distance.sokalmichener" title="scipy.spatial.distance.sokalmichener" rel="nofollow">sokalmichener</a>(u,&nbsp;v)</td> 
 <td>Computes the Sokal-Michener dissimilarity between two boolean 1-D arrays.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.sokalsneath.html#scipy.spatial.distance.sokalsneath" title="scipy.spatial.distance.sokalsneath" rel="nofollow">sokalsneath</a>(u,&nbsp;v)</td> 
 <td>Computes the Sokal-Sneath dissimilarity between two boolean 1-D arrays.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.sqeuclidean.html#scipy.spatial.distance.sqeuclidean" title="scipy.spatial.distance.sqeuclidean" rel="nofollow">sqeuclidean</a>(u,&nbsp;v)</td> 
 <td>Computes the squared Euclidean distance between two 1-D arrays.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.wminkowski.html#scipy.spatial.distance.wminkowski" title="scipy.spatial.distance.wminkowski" rel="nofollow">wminkowski</a>(u,&nbsp;v,&nbsp;p,&nbsp;w)</td> 
 <td>Computes the weighted Minkowski distance between two 1-D arrays.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.yule.html#scipy.spatial.distance.yule" title="scipy.spatial.distance.yule" rel="nofollow">yule</a>(u,&nbsp;v)</td> 
 <td>Computes the Yule dissimilarity between two boolean 1-D arrays.</td> 
 </tr> 
 </tbody> 
</table>

\[ [距离和相似度计算 ][Link 2] \]

### scipy.spatial.distance.pdist(X, metric=’euclidean’, p=2, w=None, V=None, VI=None) ###

[pdist][](X\[, metric, p, w, V, VI\])Pairwise distances between observations in n-dimensional space.观测值（n维）两两之间的距离。Pairwise distances between observations in n-dimensional space.距离值越大，相关度越小。

注意，距离转换成相似度时，由于自己和自己的距离是不会计算的默认为0，所以要先通过dist = spatial.distance.squareform(dist)转换成dense矩阵，再通过1 - dist计算相似度。

**metric：**

1 距离计算可以使用自己写的函数。Y = pdist(X, f) Computes the distance between all pairs of vectors in Xusing the user supplied 2-arity function f.

如欧式距离计算可以这样：

dm = pdist(X, lambda u, v: np.sqrt(((u-v)\*\*2).sum()))

但是如果scipy库中有相应的距离计算函数的话，就不要使用dm = pdist(X, sokalsneath)这种方式计算，sokalsneath调用的是python自带的函数，会调用c(n, 2)次；而应该使用scipy中的optimized C version，使用dm = pdist(X, 'sokalsneath')。

再如矩阵行之间的所有cause effect值的计算可以这样：

def causal_effect(m):
        effect = lambda u, v: u.dot(v) / sum(u) - (1 - u).dot(v) / sum(1 - u)
        return spatial.distance.squareform(spatial.distance.pdist(m, metric=effect))

2 这里计算的是两两之间的距离，而不是相似度，如计算cosine距离后要用1-cosine才能得到相似度。从下面的consine计算公式就可以看出。

Y = pdist(X, ’euclidean’)    \#d=sqrt((x1-x2)^2+(y1-y2)^2+(z1-z2)^2)

Y = pdist(X, ’minkowski’, p)

![Center][]

### scipy.spatial.distance.cdist(XA, XB, metric=’euclidean’, p=2, V=None, VI=None, w=None) ###

[cdist][](XA, XB\[, metric, p, V, VI, w\])Computes distance between each pair of the two collections of inputs.

当然XA\\XB最简单的形式是一个二维向量（也必须是，否则报错ValueError: XA must be a 2-dimensional array.），计算的就是两个向量之间的metric距离度量。

### scipy.spatial.distance.squareform(X, force=’no’, checks=True) ###

[squareform][](X\[, force, checks\])Converts a vector-form distance vector to a square-form distance matrix, and vice-versa.

将向量形式的距离表示转换成dense矩阵形式。Converts a vector-form distance vector to a square-form distance matrix, and vice-versa.

注意：Distance matrix 'X' must be symmetric&diagonal must be zero.

[皮皮blog][blog]

# 矩阵距离计算示例    #

### **示例1**    ###

x  
array(\[\[0, 2, 3\],  
       \[2, 0, 6\],  
       \[3, 6, 0\]\])  
y=dis.pdist(x)  
Iy  
array(\[ 4.12310563,  5.83095189,  8.54400375\])  
z=dis.squareform(y)  
z  
array(\[\[ 0.        ,  4.12310563,  5.83095189\],  
       \[ 4.12310563,  0.        ,  8.54400375\],  
       \[ 5.83095189,  8.54400375,  0.        \]\])  
type(z)  
numpy.ndarray  
type(y)  
numpy.ndarray

### **示例2** ###

print(sim)
    print(spatial.distance.cdist(sim[0].reshape((1, 2)), sim[1].reshape((1, 2)), metric='cosine'))
    print(spatial.distance.pdist(sim, metric='cosine'))

\[\[-2.85 -0.45\]  
 \[-2.5   1.04\]\]

\[\[ 0.14790689\]\]

\[ 0.14790689\]

[皮皮blog][blog]

# 检验距离矩阵有效性Predicates for checking the validity of distance matrices #

Predicates for checking the validity of distance matrices, bothcondensed and redundant. Also contained in this module are functionsfor computing the number of observations in a distance matrix.

<table> 
 <tbody> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.is_valid_dm.html#scipy.spatial.distance.is_valid_dm" title="scipy.spatial.distance.is_valid_dm" rel="nofollow">is_valid_dm</a>(D[,&nbsp;tol,&nbsp;throw,&nbsp;name,&nbsp;warning])</td> 
 <td>Returns True if input array is a valid distance matrix.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.is_valid_y.html#scipy.spatial.distance.is_valid_y" title="scipy.spatial.distance.is_valid_y" rel="nofollow">is_valid_y</a>(y[,&nbsp;warning,&nbsp;throw,&nbsp;name])</td> 
 <td>Returns True if the input array is a valid condensed distance matrix.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.num_obs_dm.html#scipy.spatial.distance.num_obs_dm" title="scipy.spatial.distance.num_obs_dm" rel="nofollow">num_obs_dm</a>(d)</td> 
 <td>Returns the number of original observations that correspond to a square, redundant distance matrix.</td> 
 </tr> 
 <tr> 
 <td><a href="http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.num_obs_y.html#scipy.spatial.distance.num_obs_y" title="scipy.spatial.distance.num_obs_y" rel="nofollow">num_obs_y</a>(Y)</td> 
 <td>Returns the number of original observations that correspond to a condensed distance matrix.</td> 
 </tr> 
 </tbody> 
</table>

from: [http://blog.csdn.net/pipisorry/article/details/48814183][blog.csdn.net_pipisorry_article_details_48814183]

ref: Distance computations ([scipy.spatial.distance][])

Spatial algorithms and data structures ([scipy.spatial][])

scipy-ref-0.14.0-p933

[blog.csdn.net_pipisorry_article_details_48814183]: http://blog.csdn.net/pipisorry/article/details/48814183
[Link 1]: http://blog.csdn.net/pipisorry/article/details/43277755
[Link 2]: http://blog.csdn.net/pipisorry/article/details/45651315
[pdist]: http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.pdist.html#scipy.spatial.distance.pdist
[Center]: /images/20220731/419db96a7dd04208b36539c1ee4467a8.png
[cdist]: http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.cdist.html#scipy.spatial.distance.cdist
[squareform]: http://docs.scipy.org/doc/scipy-0.17.1/reference/generated/scipy.spatial.distance.squareform.html#scipy.spatial.distance.squareform
[blog]: http://blog.csdn.net/pipisorry
[scipy.spatial.distance]: http://docs.scipy.org/doc/scipy-0.17.1/reference/spatial.distance.html#module-scipy.spatial.distance
[scipy.spatial]: http://docs.scipy.org/doc/scipy-0.17.1/reference/spatial.html#module-scipy.spatial