Overview of self-organising maps (SOM) and examples of algorithms and implementations.

Self-organising map (SOM) overview

Self-Organising Map (SOM) is a type of artificial neural network that maps high-dimensional data into a low-dimensional (usually two-dimensional) space for visualisation. The maps are constructed in such a way that similar data are brought into close proximity, while preserving the features of the data. This technique is useful for data clustering, dimensionality reduction and visualisation.

The basic structure of a SOM is an input layer, which receives high-dimensional input data, and a mapping layer (output layer), which consists of a low-dimensional grid structure (usually a two-dimensional grid). Each cell (neuron) in the grid of the output layer is a structure with a weight vector of the same number of dimensions as the input data.

The learning algorithm takes the following steps

1. Initialisation: the neurons of the grid are assigned a weight vector at random or in a specific way.
2. Data input: each data point is entered into the algorithm in turn.
3. Selection of competing layers (Best Matching Unit, BMU): select the neuron with the weight vector closest to the input data. \[\text{BMU} = \arg\min_i \| \mathbf{x} – \mathbf{w}_i \|\], where \(\mathbf{x}\) is the input data and \(\mathbf{w}_i\) the weight vector of the neuron.
4. Update weights: update the weights of the BMU and its neighbouring neurons. The update is based on the following rule: \[\mathbf{w}_i(t+1) = \mathbf{w}_i(t) + \eta(t) h_i(t) (\mathbf{x} – \mathbf{w}_i(t))\]- \(\eta(t)\): learning rate (decreases with time).
   – \(h_i(t)\): neighbourhood function (determined based on distance from the BMU).
5. Iterate until convergence: the data is trained multiple times (epochs) and the above procedure is repeated until the weights converge.

The main features include the following

• Unsupervised learning: useful for pattern recognition and clustering of unknown data, as labelling is not required.
• Topology preservation: preserves the similarity of the input space in a low-dimensional space.
• Dimensionality reduction: makes high-dimensional data intuitive and understandable.

Advantages and challenges of using SOM include

• Advantages.
  • Easier analysis of high-dimensional data.
  • Preservation of topology allows visualisation of relationships between data.
  • Concise algorithmic structure.
• Challenges.
  • Sensitive to initialisation and parameter settings (e.g. learning rate, neighbourhood function).
  • Increased computational cost when too much high-dimensional data is involved.
    Need to select an appropriate grid size.

SOM has become a widely used approach as a fundamental method in data analysis and machine learning, especially in situations where visualisation is important.

Implementation example

Below is a simple example of a self-organising map (SOM) implementation using Python and the library MiniSom. The library provides tools to easily perform the SOM construction and learning process.

Procedure.

1. Install the required libraries.
2. Prepare data.
3. Build and train the SOM.
4. Visualise the learning results.

Code example.

# Installing the library
# pip install minisom

import numpy as np
from minisom import MiniSom
import matplotlib.pyplot as plt
from sklearn.datasets import load_iris
from sklearn.preprocessing import MinMaxScaler

# data preparation
data = load_iris()
X = data.data  # feature value
y = data.target  # Labels (used for visualisation)

# data normalisation
scaler = MinMaxScaler()
X = scaler.fit_transform(X)

# SOM settings.
som = MiniSom(x=10, y=10, input_len=X.shape[1], sigma=1.0, learning_rate=0.5)

# Initialisation of weights
som.random_weights_init(X)

# Learning the SOM
print("Start learning about SOM ...")
som.train_random(data=X, num_iteration=1000)
print("The SOM study has been completed!")

# Visualisation of learning results
plt.figure(figsize=(10, 10))

# Plot data points for each neuron on the map
for i, x in enumerate(X):
    w = som.winner(x)  # Nearest neuron to input data x
    plt.text(w[0] + 0.5, w[1] + 0.5, str(y[i]),
             color=plt.cm.tab10(y[i] / 10),
             fontdict={'weight': 'bold', 'size': 9})

plt.title("Self-Organizing Map")
plt.xlim([0, som.x])
plt.ylim([0, som.y])
plt.grid(True)
plt.show()

Description.

1. Dataset preparation: the Iris dataset is used as a sample. This data is classified into four features and three classes.
2. Data normalisation: the SOM typically uses a ‘MinMaxScaler’ as the input data needs to be scaled to a range between 0 and 1.
3. SOM construction.
   • x=10, y=10 is the size of the mapping layer (10×10 grid).
   • input_len=X.shape[1] is the number of dimensions of the input data (4 dimensions).
   • sigma and learning_rate are parameters of the SOM that control the range of the neighbourhood function and the learning rate.
4. Learning: the train_random method inputs data to the SOM in a random order and iteratively trains it.
5. Visualisation: each data point is mapped to a neuron on the corresponding grid and displayed in a different colour for each class.

Running results.

• The data points (classes of the Iris dataset) are plotted on a 10×10 grid.
• Clustering is visualised by mapping data of the same class in close proximity.

Hints for improvement

• Changing the data size: tune the size of the mapping layers (x and y).
• Parameter tuning: optimise sigma and learning_rate.
• Different data sets: try applying to time series data or image data.

The implementation of SOM is very flexible and can be widely applied to data analysis and clustering.

Application examples

Self-organising maps (SOM) are used in many fields as a method for visualising and clustering high-dimensional data. Specific applications are described below.

1. customer segmentation

Case study: marketing analysis
Objective: to analyse customer data (e.g. purchase history, behavioural data, demographic information) and classify customers into segments.
Process:
– Visualisation of customer data with SOM.
– Customers with similar characteristics are mapped onto the same grid.
– Optimise targeted advertising and campaigns based on each segment.
Benefits:
– Intuitive understanding of high-dimensional customer data.
– Effective marketing strategies can be developed.

2. analysis of medical data

Case study: disease classification and prediction
Objective: to use patient data (e.g. blood test results, symptoms, diagnosis) to classify and predict diseases.
Process:
– Patient data is entered into the SOM and clustered according to disease type.
– Abnormal values are detected and potential disease risks are identified.
Benefits:
– Physicians have a visual understanding of the relationships between data.
– Facilitates the discovery of new diagnostic guidelines and treatments.

3. financial risk management

Case study: credit scoring
Objective: to analyse customer credit information and assess credit risk.
Process:
– Credit card usage data, payment history and annual income are mapped by SOM.
– High-risk customers are identified and appropriate credit limits are set.
Benefits:
– Visual detection of high-risk customers.
– Reduced risk of bad debts.

4. classification of image data

Case study: feature extraction of photos and videos
Objective: cluster image data and classify similar images.
Process:
– Convert each image into a feature vector (colour, shape, texture, etc.).
– Clustering with SOM to bring similar images closer together.
Benefits:
– Used for image retrieval systems and tagging.
– Detection of anomalous images (e.g. tumours in medical images).

5. genetic data analysis

Case study: clustering of gene expression patterns
Objective: To analyse gene expression data and group genes with similar expression patterns.
Process:
– Gene data is mapped to a lower dimension using SOM.
– Genes with similar functions and regulatory patterns are identified.
Benefits:
– Better understanding of biological processes.
– Visualisation of relationships between genes.

6. anomaly detection in IoT data.

Case study: analysing data from sensors
Objective: to analyse time-series data from IoT sensors to detect anomalies.
Process:
– Data on temperature, vibration, power consumption, etc. are input into the SOM.
– Anomalous patterns (isolated points on the grid) are identified.
Benefits:
– Early response to equipment failures and preventive maintenance.
– Efficient management in data-intensive IoT systems.

7. text analysis and natural language processing

Case study: document classification and thematic extraction
Objective: classify and organise vast amounts of text data into themes.
Process:
– Vectorisation of documents (e.g. TF-IDF and Word2Vec).
– Cluster similar documents using SOM.
– Each cluster is interpreted as a theme.
Benefits:
– Useful for topic modelling and customer review analysis.
– Improved efficiency of information retrieval.

8. robot control and action planning

Case study: learning behaviour patterns in reinforcement learning.
Objective: to analyse robot behaviour data and develop environmentally adaptive behaviour plans.
Process:
– Sensor information is input to the SOM and clusters are formed according to the environmental situation.
– The robot’s behaviour is adjusted based on each cluster.
Benefits:
– Improved performance of autonomous robots.
– Flexible response to changes in the environment.

SOM is suitable for intuitive understanding and classification of high-dimensional data and has proven effective in a wide range of fields.

reference book

Reference books on the theory and implementation of self-organising maps (SOM) are described below.

1. basic theory and overview
Books:
1. ‘Self-Organising Maps’.
Author: Teuvo Kohonen
Year of publication: 2001 (3rd Edition)
Publisher: Springer
– Written by Kohonen himself, the founder of SOM, this book covers the theoretical foundations and application examples.
– It details the principles of SOM in high-dimensional data visualisation and clustering.

2. ‘Neural Networks and Learning Machines’.
Author: Simon Haykin
Year of publication: 2008 (3rd Edition)
Publisher: Pearson
– Explains neural networks in general, including SOM.
– Includes a comparison of the theory of SOM with other learning models. 2.

2. applications and practice
Books:
1. ‘Data Clustering: Theory, Algorithms, and Applications’.
Author(s): Charu C. Aggarwal, Chandan K. Reddy
Year of publication: 2013
Publisher: Chapman and Hall/CRC
– Covers a wide range of data clustering techniques and provides practical applications of SOM.
– Focuses on large data sets and application areas.

2. ‘Advances in Self-Organising Maps and Learning Vector Quantization’.
Author(s): Thomas Villmann, Frank-Michael Schleif
Year of publication: 2016
Publisher: Springer
– Describes the latest SOM research results and their relationship to learning vector quantisation (LVQ).
– Includes application examples for medical data, image data and time series data.

3. useful books for Python and implementation.
Books:
1. ‘Python Machine Learning’.
Author(s): Sebastian Raschka, Vahid Mirjalili
Year of publication: 2019 (3rd Edition)
Publisher: Packt
– Contains concrete examples of implementing SOM in Python.
– In particular, practical examples can be learned by utilising the MiniSom library.

2. ‘Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow’.
Author(s): Aurélien Géron
Year of publication: 2019
Publisher: O’Reilly Media
– Useful for learning about pre-processing and visualisation methods for high-dimensional data, although SOM itself is not well explained.
– Gives hints on using SOM in combination with other algorithms.

4.Refrerence:.
1. ‘Introduction to Self-organising Maps’

2. ‘Neural Networks and Deep Learning

5. paper (as supplementary material).
– ‘Self-Organising Map’

– ‘The Self-Organising Map: recent Advances and Applications’.

Online resources.
1. documentation of the MiniSom library
– A useful library for implementing SOM in Python.

2. research page of Teuvo Kohonen
– A wealth of research material and resources on SOM.