Non-metric Multidimensional Scaling (Non-metric MDS) Overview
Non-metric MDS becomes a method of embedding data into a low-dimensional space based on the similarity or dissimilarity of the data (often given as ‘ordinal data’ or ‘ranking’).’ Whereas metric MDS, described in ‘Overview of metric MDS and examples of algorithms and implementations’, attempts to preserve absolute values of distances (e.g. Euclidean distances), non-metric MDS prioritises the ‘ordinal relationship’ of the data.
Features of non-metric MDS include the following
- Order-oriented: it is not the distance matrix that is given as input, but similarity and ordinal information. Therefore, the main focus is on preserving the order between the data (e.g. closest data, furthest data).
- Non-linear scaling: it is possible to apply non-linear transformations to satisfy the ordinal relationships of the input data.
- Flexibility: particularly useful when the distances themselves cannot be calculated or when data are given in ranking form.
The goal of non-metric MDS is to preserve the order of ‘similarity’ or ‘dissimilarity’ between input data and represent it in a low-dimensional space, where the input is a value (non-negative) indicating the similarity or order between the data and the output is a coordinate placing the data in a low-dimensional space (usually 2D or 3D).
Non-metric MDS proceeds as follows.
- Record the dissimilarities between the data as ordinal (e.g. in decreasing order).
- Placement of points in low-dimensional space so that the ordinal relationship is satisfied.
- The placement is optimised using a method to minimise an evaluation metric called the stress function.
The following algorithms are commonly used for the optimisation of non-metric MDS.
1. definition of the stress function: the stress function evaluates the degree of agreement between the order of the input data and the distance in the low-dimensional space. Example: evaluation using a Shepard diagram.
2. iterative optimisation: an initial configuration is generated based on the order of the input data and the stress function is iteratively minimised to find the optimal configuration. Usually non-linear optimisation methods (e.g. SMACOF algorithm) are used.
The differences with metric MDS are summarised below.
| item | metric MDS | Non-metric MDS |
| Nature of input data. | Distance (absolute) | Order and ranking. |
| Data accuracy | Focus on quantitative accuracy | Focus on accuracy of sequence |
| distance model | Assumption of a definite distance measure, e.g. Euclidean distance | Non-linear measures can also be applied. |
| usage scenario | Where clear distance data exist. | Where distances are ambiguous or only ordinal information is available |
Non-metric MDS is particularly powerful when the ‘order of the data’ itself is important. This makes it a particularly popular approach in psychology and marketing research.
Relevant algorithms
There are several algorithms related to non-metric multidimensional scaling (non-metric MDS), including the following methods. These are used for embedding data into low-dimensional spaces based on their ordinal relationships and similarities.
1. the SMACOF (Scaling by Majorizing a Complex Function) algorithm
Abstract: SMACOF, described in ‘Overview of SMACOF (Scaling by Majorizing a Complex Function) and examples of algorithms and implementations’, is the most widely used optimisation algorithm for non-metric MDS, which iteratively minimises the stress function by The method involves moving data points iteratively to minimise the stress function. The minimisation problem is solved iteratively to measure the consistency of the ordinal distances.
Key features:
– Minimisation of the stress function: the stress function assesses the degree of agreement between the ordinal information and the arrangement in the low-dimensional space.
– Efficient optimisation: non-linear optimisation methods are used to search for suitable solutions.
– Computational cost: optimisation in multidimensional spaces can be computationally expensive, but convergence speed is relatively fast.
2. Shepard’s Method
Abstract: Shepard’s method, described in ‘Overview of Shepard’s method and examples of algorithms and implementations’, is one of the methods used to generate initial configurations in non-metric MDS, where the input distances are given in an ordered manner. If the input distances are given in order, the placement is created while maintaining that order.
Main features:
– Distance estimation: when estimating the initial placement based on the given ordinal information, the placement is made in such a way that the approximate distance relation is satisfied.
– Direct optimisation: optimisation is performed by directly reflecting the ordinal information.
3. bias correction method
Abstract: In non-metric MDS, as described in ‘Overview, algorithms and implementation examples of the Bias Correction Method in non-metric MDS’, the order relationship between the measured data may be incomplete. In such cases, the Bias Correction Method is used for optimisation so that the order of the data is maintained more accurately.
Key features:
– Order bias correction: corrects for out-of-order relationships and minimises errors in the data.
– Correction methods: use non-linear optimisation techniques to improve data alignment.
4. self-organising map (SOM)
Abstract: Self-organising maps (SOM), described in ‘Overview of self-organising maps (SOM) and examples of algorithms and implementations’, is a neural network-based method that maps a multidimensional space of data into a 2D grid, but can also be used to place data while preserving similarity in an approach similar to non-metric MDS. It can also be used for placement.
Key features:
– Data clustering: SOM clusters data and maps them to a low-dimensional space to reflect the distances between clusters.
– Order preservation: relative distances and order between data can be maintained.
5. t-SNE (t-Distributed Stochastic Neighbour Embedding)
Abstract: t-SNE, also described in ‘About t-SNE (t-distributed Stochastic Neighbour Embedding)’, is a dimensionality reduction technique that differs from non-metric MDS but has similar objectives, in particular, to ensure similarity between the data while embedding them in a low-dimensional space. t-SNE uses a probabilistic distance function and preserves the local structure of the data.
Key features:
– Probabilistic approach: similarities between data are taken as probability distributions and placed in a low-dimensional space.
– Preservation of local structure: preserves relationships between data that are close to non-linear.
– Visualisation: mainly used to visualise high-dimensional data, visualising ordinal relationships, as in non-metric MDS.
6. ISOMAP
Abstract: ISOMAP, described in ‘ISOMAP (Isometric Mapping) Overview, Algorithms and Implementation Examples’, is a non-linear dimensionality reduction method that calculates geodesic distances (spatial shortest paths) between data and embeds them in a low-dimensional space on that basis, unlike non-metric MDS, It is characterised by the fact that the structure of the distances is preserved.
Key features:
– Non-linear dimensionality reduction: maps non-linear data structures efficiently into low-dimensional space.
– Geodesic distance: uses metric-based distances to place data while preserving order.
7. local linear embedding (LLE)
Abstract: LLE, also described in ‘About LLE (Locally Linear Embedding)’, is a non-linear dimensionality reduction technique that embeds data into a low-dimensional space while preserving local linear relationships, which, similar to non-metric MDS, can reflect similarity and distance relationships between data. This will be the one that can be used.
Key features:
– Preservation of local linearity: local linear structures in high-dimensional space are preserved in low-dimensional space.
– Capturing non-linear structures: represents non-linear data structures in lower dimensions.
Implementation example
An example of the application of non-metric MDS (Non-metric Multidimensional Scaling) to a simple dataset, using the scikit-learn library in Python.
Example implementation of non-metric MDS in Python
Installing the required libraries: first, if scikit-learn has not been installed, install it using the following command.
pip install scikit-learn
Code implementation: the following code uses scikit-learn’s MDS class to apply non-metric MDS.
import numpy as np
import matplotlib.pyplot as plt
from sklearn.manifold import MDS
from sklearn.metrics import pairwise_distances
# Sample data (any data set)
# We assume a distance matrix between the four data points
data = np.array([
[0, 1, 2],
[1, 0, 3],
[2, 3, 0],
[3, 2, 1]
])
# Calculate Euclidean distances between data.
distances = pairwise_distances(data, metric='euclidean')
# Application of non-metric MDS.
# dissimilarity=‘precomputed’ to specify that an already calculated distance matrix should be used
mds = MDS(n_components=2, dissimilarity='precomputed', n_init=4, max_iter=300)
embedding = mds.fit_transform(distances)
# Display of results
plt.figure(figsize=(6, 6))
plt.scatter(embedding[:, 0], embedding[:, 1], color='blue')
# Label each point.
for i, (x, y) in enumerate(embedding):
plt.text(x, y, f'Point {i+1}', fontsize=12, ha='right')
plt.title("Non-metric MDS Example")
plt.xlabel("Dimension 1")
plt.ylabel("Dimension 2")
plt.grid(True)
plt.show()- The embedding contains the embedded data points in 2D space.
- The embedded data points are plotted in plt.scatter and labelled in plt.text.
EXECUTION RESULTS: When this code is executed, the four data points are visualised in a two-dimensional space computed by non-metric MDS. The data are plotted in the 2D space while maintaining the order relationship between the data, so that the arrangement reflects the relative similarity and distance between the data.
Parameters of non-metric MDS
- n_components: number of dimensions to embed. Usually embedded in two or three dimensions.
- dissimilarity: specifies the type of distance matrix.’ precomputed’ is used if the distance matrix has been precomputed.
- n_init: number of initial placement attempts. The optimisation is tried several times and the best result is selected.
- max_iter: maximum number of optimisation iterations.
Reference.
This code example is the most basic implementation of non-metric MDS. It can be adapted for various applications by changing the input data and parameters according to the nature and purpose of the data.
The scikit-learn MDS implementation uses the SMACOF algorithm in non-metric MDS optimisation.
Application examples
Non-metric Multidimensional Scaling (Non-metric MDS) is a method for embedding order relationships and similarities between data in a low-dimensional space, which is particularly effective when dealing with human subjective evaluations and non-linear relationships. Specific applications are described below.
1. analysing consumer preferences in psychology and marketing
BACKGROUND: Consumer preferences and opinions about products are often expressed as ‘ordinal relationships’ that cannot be quantitatively measured. For example, information such as a consumer’s preference for product A over product B but product B over product C cannot be measured in terms of distance or angle. Non-metric MDS can map such ordinal information into a low-dimensional space to visualise patterns of consumer preferences.
Application examples:
– Consumer preference mapping: companies use non-metric MDS to analyse consumers’ evaluations of products (preferred order) and visually represent the relative distances between products. This enables an understanding of how similar product groups are and which products consumers are likely to choose.
Procedure:
1. ask consumers to rate pairs of products and answer in order which they prefer.
2. calculate the ‘ordinal distance’ between the products based on the results of their ratings.
3. use non-metric MDS to map the products in a low-dimensional space and visualise patterns of consumer preferences
2. phylogenetic tree analysis in genetics
BACKGROUND: In genetics research, genetic distances (e.g. DNA similarity) between different species can vary depending on the measurement method. Non-metric MDS helps to map genetic distances in a lower dimension and to visually understand the relationships between species.
APPLICATIONS:
– Genetic distance analysis between species: calculate the genetic distances between various species and apply non-metric MDS based on these distances to visually represent the phylogenetic tree between species. This allows the grouping of genetically closest species and the visualisation of evolutionary relationships.
Procedure:
1. calculate the genetic distance between species (e.g. differences in DNA sequence).
2. embed this genetic distance in a lower dimension (usually 2D or 3D) using non-metric MDS.
3. visualise the results to understand the relationships between species
3. analysing similarities between musical genres
BACKGROUND: When measuring similarity between musical genres, the similarity of musical features (e.g. rhythm, melody, chords) can be calculated, but this similarity may not be linear. Non-metric MDS can be used to visualise the ordinal relationships between music genres and place similar genres in close proximity.
APPLICATIONS:
– Visualisation of music genres: use non-metric MDS to visually represent similarities between different music genres. A distance matrix is created using musical features (e.g. intensity of sound, tempo, chord structure, etc.), which is then mapped onto a low-dimensional space.
Procedure:
1. calculate the feature values (e.g. sound rhythm, tempo, use of chords, etc.) between musical genres.
2. calculate the distances between musical genres based on them.
3. use non-metric MDS to embed the distances between musical genres into the low-dimensional space.
4. plot the results to visualise the relationships between musical genres.
4. social network analysis
BACKGROUND: Social network analysis involves calculating similarities between different persons in order to understand people’s relationships and patterns of interaction. Non-metric MDS can be used to map the people in a network into a low-dimensional space and place similar people close to each other.
APPLICATIONS:
– Social network visualisation: using non-metric MDS, the intensity of interaction between users (e.g. between users who interact frequently) is represented as a distance, which is then embedded and visualised in a low-dimensional space. This provides a visual understanding of the people being grouped.
Procedure:
1. measure the degree of interaction or similarity between users (e.g. number of messages exchanged, degree of agreement in interests, etc.).
2. create a distance matrix based on similarities.
3. map this distance matrix to a low-dimensional space using non-metric MDS.
4. visualise the results and analyse the relationships between users.
5. product recommendation systems
BACKGROUND: Recommendation systems suggest suitable products to users based on their preferences and product features. Non-metric MDS can be used to map similarities between product features in a low-dimensional space and group similar products.
APPLICATIONS:
– Product similarity analysis: using non-metric MDS, similarities between products can be calculated based on product features (price range, category, user ratings, etc.) and embedded in a low-dimensional space to facilitate finding similar products.
Procedure:
1. calculate the similarity between products based on the product features.
2. use non-metric MDS to embed the similarity of the products into the low-dimensional space.
3. visually group similar products and incorporate them into the recommendation system.
reference book
Relevant reference books on Non-metric Multidimensional Scaling (Non-metric MDS) are listed below.
2. ‘A Comparison Study on Similarity and Dissimilarity Measures in Clustering Continuous Data’
3. ‘Modern Multidimensional Scaling: Theory and Applications’ by Ingwer Borg and Patrick J. F. Groenen
– Abstract: This book is a comprehensive review of MDS algorithms, providing a detailed description of metric and non-metric MDS, with a particular focus on computational techniques and their applications.
– Contents: provides theoretical and practical approaches to implementing modern MDS methods, including non-metric MDS. Case studies and implementation codes are also included in abundance.
4. “Nonmetric multidimensional scaling: Neural networks versus traditional techniques”
6. ‘Applied Multidimensional Scaling’
Papers and online resources.
1. “Review of the Development of Multidimensional Scaling Methods”
2. online tutorials and repositories:.
– Examples of implementations and tutorials of MDS algorithms published on platforms such as GitHub, for example, are also helpful. In particular, reviewing Python implementations of non-metric MDS and examples of the use of R packages (such as `smacof`) can provide exposure to real code.
コメント