Overview of Shepard’s method
Shepard’s method is one of the non-linear dimensionality reduction methods, used in particular as part of MDS, which is also described in ‘Multidimensional Scaling (MDS)’, and is mainly used to effectively map distances or similarities between data in low-dimensional space. Shepard’s method can be characterised as a non-linear distance reduction method, an approach that is particularly capable of successfully representing diverse relationships between data.
Shepard’s method is defined as follows.
- Input data: first, the distance or similarity between each point in the original data set is calculated. Different distance measures can be used for this, e.g. Euclidean distance or cosine similarity.
- Aim: The aim is to map the distance (or similarity) relationships in the higher dimensional space to the lower dimensional space, where the distances between points in the lower dimensional space are arranged to be as close as possible to the distances in the higher dimensional space.
- Preservation of distances: Shepard’s method requires that the distances (or similarities) between points should be reflected in the low-dimensional space as much as possible. The placement of points in the low-dimensional space is therefore optimised to preserve the distance relationships in the original data space.
- Non-linearity: unlike other linear methods (e.g. principal component analysis), Shepard’s method is **non-linear**. This non-linearity allows it to adequately reflect complex patterns and non-linear relationships in the data.
Shepard’s method uses the original distance matrix \( D \) (distances in high-dimensional space) and optimises the placement of points in low-dimensional space. This achieves the following objectives.
- Maintain similarity or distance relationships in high-dimensional space in low-dimensional space.
- Determine the low-dimensional arrangement so that the distance difference between each point is as small as possible.
In a specific mathematical expression, the distance \( D_{ij} \) between the points \( \mathbf{x}_i \) and \( \mathbf{x}_j \) in the original higher-dimensional space is compared with the distance \( d_{ij} \) between the points \( \mathbf{y}_i \) and \( \mathbf{y}_j \) in the lower-dimensional space and the following relation is optimised to minimise.
\[
\min \sum_{i < j} w_{ij} ( D_{ij} – d_{ij})^2
\]
where \( w_{ij} \) represents the similarity weights. The method uses an optimisation technique (e.g. steepest descent method) to minimise the conservation of distances and to determine the placement of points in a low-dimensional space.
Features and advantages of Shepard’s method include
- Non-linearity: Shepard’s method can deal with non-linear data relationships, making it useful even when linear dimensionality reduction methods do not work.
- Distance preservation: the distances between data points are preserved as faithfully as possible in low-dimensional space.
- Visual representation: widely used in high-dimensional data visualisation as a tool to facilitate visual understanding of relationships between data.
Shepard’s method is particularly effective when non-linear dimensionality reduction is desired, and has become one of the approaches used for visualising high-dimensional data and representing it in low-dimensional space while preserving complex similarity relationships between data.
Implementation example
An example implementation of non-linear dimensionality reduction using Shepard’s method is shown using Python. The code mainly uses scikit-learn and matplotlib to perform Shepard’s method on a simple 2D dataset.
Example implementation.
import numpy as np
import matplotlib.pyplot as plt
from sklearn.metrics import pairwise_distances
from sklearn.manifold import MDS
# Sample data set (5 data points)
data = np.array([[1, 2],
[2, 3],
[3, 4],
[5, 6],
[6, 7]])
# Calculate the distance between data (Euclidean distance).
distances = pairwise_distances(data)
# Mapping to two dimensions using MDS based on Shepard's method
mds = MDS(n_components=2, dissimilarity="precomputed", random_state=42)
embedding = mds.fit_transform(distances)
# Plotting the results
plt.scatter(embedding[:, 0], embedding[:, 1], c='blue', label='Transformed Data')
plt.title("Shepard's Method (MDS)")
plt.xlabel("Dimension 1")
plt.ylabel("Dimension 2")
plt.grid(True)
plt.show()Code description.
- Creating a dataset:
- Define a two-dimensional sample dataset called data. Each row represents a data point.
- Calculate distance matrix:
- Calculate the Euclidean distances between the data points using the pairwise_distances function. This distance matrix is the input to Shepard’s method.
- Dimensionality reduction using Multidimensional Scaling (MDS):
- Use sklearn.manifold.MDS to embed data in a lower dimensional space based on the distance matrix. n_components=2 is specified to map the results to a two-dimensional space.
- By setting dissimilarity=‘precomputed’, the calculated distance matrix is input directly into the MDS.
- Plotting the results:
- Use matplotlib to visualise the data points in 2D after dimensionality reduction. It is possible to visually see how the data points are arranged.
Running results: running this code shows how the original data is mapped to a lower dimensional space (2D); Shepard’s method allows high dimensional relationships to be stored in the lower dimensional space and the structure of the data to be visualised.
Application examples
Use cases for applying Shepard’s method (or MDS) are widely used in the following data analysis and visualisation scenarios.
1. sensory evaluation in psychology
APPLICATION: Shepard’s method is used to visualise the perceptual distance between different items in the evaluation of human sensations (colour, sound, shape, etc.) in low dimensions.
Case study (music genre classification): similarity between music genres is calculated to create a 2D plot showing how close different music genres are. This allows visual confirmation of how similar music genres are grouped.
Method: create a distance matrix using sonic features between music genres (e.g. sound frequency spectrum or rhythmic features); apply Shepard’s method and plot the genres in a low-dimensional (2D or 3D) space to visually identify similar genres.
2. clustering of gene expression data
APPLICATION: used to visualise genetic relatedness between samples based on similarities between different samples in gene expression data.
Case study (cancer sample analysis): using gene expression data from cancer patients, the genetic distance between samples is calculated and then the genes are clustered in a low-dimensional space using Shepard’s method. This method allows visualisation of groups of samples with similar gene expression.
Method: compute the correlation of gene expression between samples and create a distance matrix; apply Shepard’s method and plot the samples in low-dimensional space (2D), placing samples with similar expression patterns close together.
3. feature visualisation in image processing
APPLICATION: map high-dimensional image features into low-dimensional space to visually check similarities between images. Especially often used in face and object recognition.
Case study (visualisation of face image datasets): compute a distance matrix of features (e.g. facial recognition features) in face recognition and visualise different face images in low dimension using Shepard’s method. This allows for the similarity of faces and grouped face patterns to be identified.
Method: distance matrices are created using features from the face images (e.g. PCA or deep learning feature extraction); Shepard’s method is applied to place the face images in 2D or 3D space and clustering is visualised based on the similarity of the face images.
4. consumer behaviour analysis in marketing
APPLICATION: used to analyse how similar consumers are to different products and brands and to visualise the positioning of products in the market.
Case study (analysis of brand similarity): based on how consumers perceive different brands (e.g. luxury, quality, price, etc.), the similarity between brands is calculated and the positioning of the brands is visualised using Shepard’s method. This enables an understanding of the relative position of brands in the market.
Method: distance (similarity) between brands is calculated based on consumer questionnaires and purchase histories; Shepard’s method is applied to plot brands in a low-dimensional space to visualise the similarity between brands based on consumers’ perceptions.
5. word embedding in linguistic processing
APPLICATION: utilising Shepard’s method in visualising semantic similarities between words using word embeddings (word embeddings).
Case study (visualisation of Word2Vec embeddings): visualise words in 2D space using Shepard’s method by calculating the distance matrix between word vectors obtained with techniques such as Word2Vec and GloVe. This allows one to see how words with similar meanings are placed in close proximity.
Method: use high-dimensional word embedding vectors obtained from Word2Vec and GloVe to calculate the similarity between words and create a distance matrix; use Shepard’s method to map word embeddings into 2D or 3D space and visualise how semantically similar words are placed close together.
Shepard’s method (MDS) is a powerful tool in embedding similarities and distances between data in low-dimensional space and has been used to visually identify patterns in data in a variety of fields, including psychology, genetics, image processing, marketing and language processing, as in the application examples above It has been used in a number of fields, including psychology, genetics, image processing, marketing and language processing. This technique can be very useful for exploratory analysis of data, as it preserves complex relationships between data while making them visually comprehensible.
reference book
Reference books related to Shepard’s method (or MDS) are described.
1. ‘Multidimensional Scaling’ by I. R. Shepard and J. B. Kruskal
– Abstract: This book describes the basic theory and implementation of MDS; based on Shepard’s original work, it provides the mathematical background and various algorithms of MDS and is suitable for learning about the basic approach of MDS and its applications.
2. ‘Modern Multidimensional Scaling: Theory and Applications’ by Ingwer Borg and Patrick J. F. Groenen
– Abstract: This comprehensive book describes the latest theory and practical applications of MDS. The book describes the different algorithms of MDS and their computational methods, as well as a wealth of examples of their application to real data sets. Recommended for those who want a thorough understanding of the formulas and algorithms.
3. ‘Data Science for Business: What You Need to Know about Data Mining and Data-Analytic Thinking’ by Foster Provost and Tom Fawcett
– Summary: A book explaining data mining techniques from a business analysis perspective, touching on how MDS and other dimensionality reduction techniques apply to business, useful for those wishing to apply Shepard’s method to business analysis.
4. ‘Principles of Multivariate Analysis: A User’s Perspective’ by N. T. Longford
– Abstract: A practical guide to multivariate analysis methods, including MDS, with particular emphasis on how to implement MDS and apply it to real data. It focuses on MDS as a statistical analysis method.
5. ‘Applied Multivariate Statistical Analysis’ by Richard A. Johnson and Dean W. Wichern
– Abstract: This book provides comprehensive coverage of multivariate analysis methods in statistics; it explains the theory of MDS and other dimensionality reduction techniques and their practical use, and is useful for an in-depth understanding of dimensionality reduction methods in data analysis.
6. ‘The Art of Data Science’ by Roger D. Peng and Elizabeth Matsui
– Abstract: This book introduces thinking and approaches to problem solving in data science, including discussions on dimensionality reduction and visualisation techniques, and is suitable for developing thinking about data analysis using MDS.
7. “Methods for Multidimensional Scaling Part 1: Overview”
Online resource.
– Statistical Learning and Multidimensional Scaling: free online courses offered by Stanford University and MIT, among others, to learn about the theory and implementation of MDS. These courses help to gain an in-depth understanding of its application to real-world datasets.
コメント