Automatic machine learning is a method that can automatically establish machine learning model, which mainly includes three aspects: first, hyperparameter optimization; Aspect two, automatic feature engineering and machine learning algorithm automatic selection; Aspect three, neural network structure search. This paper focuses on aspect three, divine network structure search.

The first two parts of automated machine learning have one characteristic: they search for existing algorithms instead of creating new ones. Generally speaking, when machine learning experts develop machine learning applications or build machine learning models, they are unlikely to create a new algorithm from scratch. But when it comes to deep neural networks, something happens. Strictly speaking, the basic structure of neural networks is fixed and finite. But every time we recreate a model, a different combination of basic units can be seen as creating a new neural network model. Under this basic reality, a third automated machine learning technique is applied, namely neural network Architecture Search Overview (NAS), which creates a new neural network structure based on the basic neural network basic units. The end result is a very powerful neural network.

A large number of scientists have done in-depth research in this area. However, as its basic assumption is deep neural network, and the architecture search is carried out on it, so the requirements for computing power are often relatively high. At present, the most powerful algorithm that can run on a single card is ENAS, and the author of ENAS opened source his work based on ENAS, Autokeras, with the release of the paper, and anyone can download and use this library.

The bottom layer of autokeras is ENAS algorithm, but its existence can be divided into the following types according to different applications:

• Image classification
• Image regression
• Text classification
• Text regression
• Structured data classification
• Structured data regression we directly carry out more complex operations, using Autokeras for automatic image classification.

Then we convert the data reading into TFDS format, which will be much more efficient. The specific implementation is as follows:

import warnings
warnings.filterwarnings("ignore")

print("-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- parameters -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --")

BATCH_SIZE = 256
IMG_HEIGHT = 224
IMG_WIDTH = 224
STEPS_PER_EPOCH = np.ceil(image_count/BATCH_SIZE)

print("----------------- start tfds -----------------")
list_ds = tf.data.Dataset.list_files(str(data_dir/'* / *'))

def get_label(file_path) :
  # convert the path to a list of path components
  parts = tf.strings.split(file_path, os.path.sep)
  # The second to last is the class-directory
  return parts[-2] == CLASS_NAMES

def decode_img(img) :
  # convert the compressed string to a 3D uint8 tensor
  img = tf.image.decode_jpeg(img, channels=3)
  Floats in the [0,1] range. # Use 'convert_image_dtype' to convert to floats in the [0,1] range.
  img = tf.image.convert_image_dtype(img, tf.float32)
  # resize the image to the desired size.
  return tf.image.resize(img, [IMG_HEIGHT, IMG_WIDTH])

def process_path(file_path) :
  label = get_label(file_path)
  # load the raw data from the file as a string
  img = tf.io.read_file(file_path)
  img = decode_img(img)
  return img, label

# Set `num_parallel_calls` so multiple images are loaded/processed in parallel.
labeled_ds = list_ds.map(process_path, num_parallel_calls=AUTOTUNE)
print("type(labeled_ds): ".type(labeled_ds))
Copy the code

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- parameters -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- start TFDS -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- type (labeled_ds) : <class 'tensorflow.python.data.ops.dataset_ops.ParallelMapDataset'>Copy the code

The training process of the model outputs a lot of data, but only requires a few lines of code to use. In addition, the same is true for model prediction and export. In the training process, the training process has been saved, but the export_model method is needed to output the best trained model.

print("----------------- Predict with the best model -----------------")
# Predict with the best model.
predicted_y = clf.predict(labeled_ds)
# predicted_y = clf.predict(train_ds)
# Evaluate the best model with testing data.
print(clf.evaluate(labeled_ds))
# print(clf.evaluate(train_ds))
Copy the code

----------------- Predict with the best model ----------------- WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...) .expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details. WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...) .expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics 115/115 [for the details. = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =] - ETA: 0 s - loss: 1.6073 accuracy: ETA: 0.28-3 s - loss: 1.6060... Loss: 1.6052-accuracy: 0.24-5s 42ms/ step-loss: 1.6052-accuracy: 0.2455 [1.6052242517471313, 0.24550408124923706]Copy the code