Distance and Similarity Measures

Distance measures quantify dissimilarity; similarity measures quantify closeness.

1) Euclidean Distance (L2)

For points $(x_1,y_1)$ and $(x_2,y_2)$:

$$d_E = \sqrt{(x_2-x_1)^2 + (y_2-y_1)^2}$$

For $n$ dimensions:

$$d_E(\mathbf{x},\mathbf{y}) = \sqrt{\sum_{i=1}^{n}(x_i-y_i)^2}$$

Example for points $(1,2)$ and $(5,6)$:

$$d_E = \sqrt{(5-1)^2 + (6-2)^2} = \sqrt{32} = 4\sqrt{2}$$

2) Manhattan Distance (L1)

For points $(x_1,y_1)$ and $(x_2,y_2)$:

$$d_M = |x_2-x_1| + |y_2-y_1|$$

Example from $(1,2)$ to $(3,5)$:

$$d_M = |3-1| + |5-2| = 2 + 3 = 5$$

3) Worked Dataset

Manhattan Distance Matrix

Euclidean Distance Matrix

4) Cosine Similarity and Cosine Distance

Cosine similarity compares direction, not magnitude:

$$\cos(\theta)=\frac{\mathbf{A}\cdot\mathbf{B}}{\|\mathbf{A}\|\|\mathbf{B}\|}$$

Cosine distance:

$$d_{\cos}=1-\cos(\theta)$$

Special cases:

• $\cos(0^\circ)=1$ means maximum similarity.
• $\cos(90^\circ)=0$ means orthogonal (no directional similarity).

Example with $\mathbf{A}=[1,2,0]$ and $\mathbf{B}=[2,1,1]$:

$$\mathbf{A}\cdot\mathbf{B}=4, \quad \|\mathbf{A}\|=\sqrt{5}, \quad \|\mathbf{B}\|=\sqrt{6}$$ $$\cos(\theta)=\frac{4}{\sqrt{30}}\approx 0.73, \quad d_{\cos}\approx 0.27$$

5) Jaccard Similarity and Distance

For sets $A$ and $B$:

$$J(A,B)=\frac{|A\cap B|}{|A\cup B|}$$ $$d_J(A,B)=1-J(A,B)$$

Given:

$$A=\{1,2,4,5\},\quad B=\{2,3,5,7\}$$ $$A\cap B=\{2,5\},\quad A\cup B=\{1,2,3,4,5,7\}$$ $$J(A,B)=\frac{2}{6}=0.333\ldots, \quad d_J(A,B)=0.666\ldots$$

6) Which Measure to Use

• Euclidean: dense numeric features where straight-line geometry is meaningful.
• Manhattan: grid-like movement or absolute coordinate differences.
• Cosine: text vectors and high-dimensional sparse vectors.
• Jaccard: set or binary-presence features.

These metrics are commonly used in KNN, clustering, recommendation, and retrieval.