Overview of Automatic Machine Learning (AutoML) and its algorithms and various implementations

AutoML

Automatic machine learning (AutoML) refers to methods and tools for automating the process of designing, training, and optimizing machine learning models.AutoML is particularly useful for users with limited machine learning expertise or those seeking to develop efficient models, with the following main goals.

• Simplification and efficiency: AutoML reduces the time and effort users spend designing models and adjusting hyperparameters. This increases the overall efficiency of the machine learning project.
• No Expertise Required: With AutoML, even users who lack machine learning expertise may be able to build suitable models; AutoML tools are designed so that users do not have to worry about the technical details of the model.
• Searching for best practices: AutoML helps users find the best model by trying multiple algorithms and combinations of hyperparameters. This saves users the time and effort of manual trial and error.

AutoML can be applied to a variety of tasks. Typical automatic machine learning methods and tools include the following

• Hyperparameter optimization: a method for searching for optimal values for the hyperparameters of a model (learning rate, number of layers, etc.). Tools include Hyperopt, Optuna, and Tune.
• Model selection: a method for automatically selecting the best algorithm or architecture for the data; tools include AutoSklearn, H2O.ai, Google AutoML, etc.
  Feature engineering: methods for automatically extracting and transforming data features, such as Featuretools.
• Model architecture exploration: methods for automatically exploring neural network architectures, such as AutoKeras and Google AutoML.

While AutoML is used to improve model quality and efficiency, it may need to be adjusted depending on the task and data, and while AutoML has tools to help users, it is important to apply it appropriately to the right task.

Algorithms used in automatic machine learning (AutoML)

Various algorithms and methods are used in automatic machine learning (AutoML). These algorithms are applied to various aspects of AutoML, including model hyperparameter tuning, algorithm selection, feature engineering, and model architecture exploration. Examples of algorithms commonly used in AutoML are described below.

• Hyper-Parameter Optimization:
  • Random Search: A method that randomly selects and evaluates combinations of hyperparameters to find the optimal combination.
  • Grid Search: A method that tries all combinations of hyperparameter values specified in advance.
  • Bayesian Optimization: a method to predict the next combination to try based on the results of hyperparameter evaluation, e.g. Optuna or Hyperopt.
• Model Selection:
  • AutoSklearn: A tool that trains multiple algorithms and automatically selects the best model.
  • TPOT: A method that uses genetic programming to evolve the architecture and hyperparameters of a model.
• Feature Engineering:
  • Featuretools: Tools for automatically generating features and finding new features.
• Model architecture exploration:
  • AutoKeras: tool for automatically exploring neural network architectures.
  • Google AutoML: A Google service that automates the design, training, and evaluation of custom models.

These algorithms are selected according to the purpose of AutoML. This means, for example, that hyperparameter optimization algorithms may be used to find the best hyperparameters for a particular task, or tools such as AutoSklearn and AutoKeras may be helpful in algorithm selection and model architecture exploration.

Importantly, AutoML is by no means a panacea; it should be applied appropriately to the right tasks and data, and it is important to choose which algorithms and tools are best suited depending on the task characteristics and data quality.

Libraries and platforms used for automated machine learning (AutoML)

Various libraries and platforms exist for automated machine learning (AutoML). These tools are used to automate tasks such as model hyperparameter optimization, algorithm selection, feature engineering, and model architecture exploration. The following is a list of typical AutoML libraries and platforms.

• AutoSklearn: This is a Python-based AutoML library that selects the best model among multiple machine learning algorithms and also performs hyperparameter optimization. It integrates with Scikit-learn and is easy to use.
• TPOT: Uses genetic programming to optimize feature engineering, model architecture, and hyperparameters. It is also integrated with Scikit-learn, allowing for flexible customization.
• AutoKeras: An AutoML library based on Keras that automatically adjusts the architecture and hyperparameters of neural networks. It is primarily used for image data.
• Google AutoML: A platform provided by Google that automates the design, training, and deployment of custom machine learning models. It has versions for various domains, including AutoML Vision, AutoML Tables, and AutoML Natural Language.
• H2O.ai: H2O’s AutoML functionality will automate model training, stacking, and hyperparameter optimization. It is available in both Python and R.
• Optuna: A Python-based hyperparameter optimization library that uses Bayesian optimization to perform efficient hyperparameter search.
• Featuretools: An automated feature generation library that generates new features from data and automates feature engineering.

Application Examples of Automatic Machine Learning (AutoML)

Automatic machine learning (AutoML) has been applied in a variety of tasks and domains. The following are examples of AutoML applications.

• Image Classification: AutoML is also widely used in image classification tasks. In particular, it is useful for automatically exploring neural network architectures and hyperparameters suitable for specific data sets, e.g., for detecting defects in car parts or classifying food products.
• Text Analysis: AutoML can also be useful in natural language processing tasks on textual data. It is used to optimize model architecture, tokenization methods, and hyperparameters for tasks such as text classification, sentiment analysis, and summarization.
• Tabular Data: AutoML is especially important for tasks involving tabular data (data in tabular form). It helps to automatically perform feature engineering and algorithm selection to extract patterns in the data, for example, for customer segmentation and financial risk prediction.
• Speech processing: AutoML is also used for tasks such as speech recognition and music analysis on audio data. It automatically extracts features and builds models to assist in the analysis of speech data.
• Motion Recognition: AutoML is also applied to tasks that use sensor data to recognize actions and behaviors. For example, there are situations where walking or running is identified from accelerometer data from a wearable device.
• Time-series data: For tasks involving time-series data, AutoML has been used to optimize the architecture and hyperparameters of time-series models, for example, for forecasting financial data or energy consumption.

AutoML is in fact used in a wide variety of tasks and domains, and the use of AutoML simplifies and streamlines the machine learning process for users who lack specialized knowledge or who seek to build efficient models.

Example implementation for image classification using AutoML

As an example of an implementation of image classification using AutoML, we describe how to use Google’s AutoML Vision, a service that allows users to train and deploy custom image classification models.

The following is a brief description of how to train an image classification model using AutoML Vision. For detailed instructions, please refer to the official Google Cloud documentation.

1. Configure Google Cloud Platform (GCP):
   • Create a Google Cloud Platform account and create a project.
   • Enable AutoML Vision API from the GCP dashboard.
2. Upload data:
   • Prepare image data and create a folder for each class.
   • From the AutoML Vision screen in GCP, create a dataset and upload the images.
3. Train the model:
   • Train the model using the dataset; AutoML Vision will automatically select the best hyperparameters and train the model.
   • Once training is complete, evaluate model performance.
4. Deployment and Prediction:
   • After successful training, the model is deployed and ready for use.
   • The deployed model is used to make predictions on new image data.

For detailed instructions and configuration of Google AutoML Vision, please refer to the official Google Cloud documentation. Using this approach, training and deploying image classification models is automatic, making it easy to create custom image classification models even if you have limited machine learning expertise.

Example implementation for text analysis using AutoML

As an example of an implementation of text analysis using AutoML, we describe how to use Google’s AutoML Natural Language, a service for training custom text classification models and classifying text data. AutoML Natural Language is a service for training custom text classification models and classifying text data.

Below we describe the steps for training a text analysis model using AutoML Natural Language. For detailed instructions, please refer to the official Google Cloud documentation.

1. Configure Google Cloud Platform (GCP):
   • Create an account on Google Cloud Platform and create a project.
   • Enable the AutoML Natural Language API from the GCP dashboard.
2. Data Preparation and Uploading:
   • Prepare text data and organize the data for each class.
   • Create datasets and upload text data from the AutoML Natural Language screen in GCP.
3. Train the model:
   • Use the dataset to train your model; AutoML Natural Language will automatically select the best architecture and hyperparameters to train your model.
   • Once training is complete, the performance of the model is evaluated.
4. Deployment and Prediction:
   • After successful training, the model is deployed and made available for use.
   • The deployed model is then used to perform classification on the new text data.

For detailed instructions and configuration of Google AutoML Natural Language, please refer to the official Google Cloud documentation. Using this methodology, you can easily create custom classification models for text data and perform text analysis even if you have limited expertise.

Example implementation of a task that uses AutoML to handle tabular data

As an example of how to implement the task of working with tabular data using AutoML, we describe how to use H2O.ai’s AutoML library. H2O.ai’s AutoML is a tool to automatically train models on tabular data (tabular data) and select the best model.

The following is a brief description of the procedure for working with tabular data using H2O.ai’s AutoML.

• Installation of H2O.ai:
  • Install the H2O.ai library; if using Python, it can be installed as follows

pip install h2o

• Data Preparation:
  • Prepare tabular data and save it in CSV or other formats.
  • Start H2O Cluster: Start H2O Cluster and load data.
    Start H2O cluster and load data.

import h2o
h2o.init()
data = h2o.import_file("your_data.csv")

• AutoML Execution:
  • Use AutoML to train your model; H2O AutoML tries multiple algorithms and selects the best model.

from h2o.automl import H2OAutoML
aml = H2OAutoML(max_runtime_secs=3600)
aml.train(y="target_column", training_frame=data)

• Selecting and Evaluating the Best Models:
  • Once AutoML is complete, select and evaluate the best model.

best_model = aml.leader
test_data = h2o.import_file("test_data.csv")
predictions = best_model.predict(test_data)

Using H2O.ai’s AutoML, model training and evaluation are automatically performed on tabular data. Simply specify the target variables, load the data, and the best model is automatically selected.

Example implementation of speech processing using AutoML

AutoML Speech-to-Text is a service that trains custom models to convert speech data into text for speech recognition. AutoML Speech-to-Text is a service to train custom models to convert speech data to text for speech recognition.

Below are the steps for training a speech recognition model using AutoML Speech-to-Text. For detailed instructions, please refer to the official Google Cloud documentation.

1. Configure Google Cloud Platform (GCP):
   • Create an account on Google Cloud Platform and create a project.
   • Enable the AutoML Speech-to-Text API from the GCP dashboard.
2. Upload data:
   • Prepare your audio data and upload it to GCP storage. The audio file format must be a supported format (e.g., WAV, FLAC).
3. Data preprocessing:
   1. The audio data must be mapped to labeled text. The labels should correspond to the text conversion results of the audio data.
4. Model training:
   • Create a dataset by uploading data from GCP’s AutoML Speech-to-Text screen.
   • AutoML Speech-to-Text automatically trains speech recognition models.
5. Evaluate the model:
   • Once training is complete, evaluate the model. Evaluate model performance using test data.
6. Deployment and Speech Recognition:
   • After successful training, the model is deployed and ready for use.
   • The deployed model is used to perform speech recognition on the new speech data.

For detailed instructions and configuration of Google AutoML Speech-to-Text, please refer to the official Google Cloud documentation. Using this method, custom speech recognition models can be easily created and used to convert speech data to text.

Example implementation using AutoML for motion recognition

As an example of an implementation of motion recognition using AutoML, we describe a task that identifies actions (behaviors) using accelerometer data from a wearable device. In this example, the motion recognition model is trained using H2O.ai’s AutoML.

The following is a brief description of the procedure for training the behavior recognition model.

• Installation of H2O.ai:
  • Install the H2O.ai library; if using Python, it can be installed as follows

pip install h2o

• Data Preparation:
  • Prepare accelerometer data collected from wearable devices. Labels corresponding to each movement are also needed.
• Start H2O cluster:
  • Start the H2O cluster and load the data.

import h2o
h2o.init()
data = h2o.import_file("acceleration_data.csv")

• Feature Engineering:
  • Perform preprocessing to extract features from accelerometer data. For example, moving average and FFT (Fast Fourier Transform) are used to extract features.
• AutoML execution:
  • Train a motion recognition model using AutoML; H2O AutoML tries multiple algorithms and selects the best model.

from h2o.automl import H2OAutoML
aml = H2OAutoML(max_runtime_secs=3600)
aml.train(y="label_column", training_frame=data)

• Selecting and Evaluating the Best Model:
  • Once AutoML is complete, select the best model and evaluate it with test data.

best_model = aml.leader
test_data = h2o.import_file("test_acceleration_data.csv")
predictions = best_model.predict(test_data)

Using this technique, motion recognition models can be trained using accelerometer data collected from wearable devices. However, it is important to make appropriate feature engineering and model adjustments for the specific data and task.

Example implementation for time series data analysis using AutoML

As an example of an implementation of time-series data analysis using AutoML, we describe how to train a predictive model for time-series data using H2O.ai’s AutoML.

The following is a brief description of the procedure for training a predictive model for time-series data.

• Installation of H2O.ai:
  • Install the H2O.ai library.

pip install h2o

• Data Preparation:
  • Prepare time series data and save in CSV or other format. Data must include time and corresponding target variables.
• Start H2O cluster:
  • Start the H2O cluster and load the data.

import h2o
h2o.init()
data = h2o.import_file("time_series_data.csv")

• Running AutoML:
  • Use AutoML to train predictive models for time series data; H2O AutoML will try multiple algorithms and select the best model.

from h2o.automl import H2OAutoML
aml = H2OAutoML(max_runtime_secs=3600)
aml.train(y="target_column", training_frame=data)

• Selecting the best model and forecasting:
  • Once AutoML is complete, select the best model and make predictions for future time series data.

best_model = aml.leader
future_data = h2o.import_file("future_time_series_data.csv")
predictions = best_model.predict(future_data)

This method can be used to automatically train prediction models for time-series data. It is also possible to build more accurate forecast models by pre-processing the data and adjusting the models.

Reference Information and Reference Books

For more information on machine learning in general, see “Machine Learning Technology.

AutoML Standard Requirements” for reference.

“Automated Machine Learning in Action”

“Hands-On Automated Machine Learning: A beginner’s guide to building automated machine learning systems using AutoML and Python”

“Automated Machine Learning: Methods, Systems, Challenges”

“Automated Machine Learning with Microsoft Azure: Build highly accurate and scalable end-to-end AI solutions with Azure AutoML”