In this Tutorial, Raywenderlich writes a Tutorial called Create ML :Getting Started on the MAC /iOS platform. Because the translator does not have a deep understanding of machine learning, it is inevitable that some mistakes are made in translation. Please forgive me.

Download this document

Create ML is proof of Apple’s determination to make it easier for you to use machine learning models in your apps. In this tutorial, you’ll learn how to use Create ML to speed up your workflow: to improve your model by improving your data, and to reduce the learning curve by using Xcode and Swift.

At the same time, you’ll become more familiar with ML toolsets and terminology. Without introducing mathematics! You don’t need to know how to write a compiler to use Swift, and you don’t need to write a new ML algorithm to use the classifier. With Create ML, you have no reason not to start learning machine learning!

A brief history of Apple ML:

  • Core ML: Announced at WWDC 2017, Core ML is now supported by every major ML platform and can export existing models. But the existing models are always too large and/or too general.
  • Turi Create: Acquired after WWDC 2017, Turi Create allows you to customize existing models with your own data. It’s just using… Python :[.
  • IBM Watson Services: Announced in March 2018. You can customize IBM Watson’s visual recognition model to identify your own data. Just drag and drop your data, no code, but you need to use the IBM Cloud, and the Core ML model is wrapped in the Watson API.
  • Create ML: Announced at WWDC 2018. Using Xcode and Swift ML! Currently, only two of the seven Turi Create focus toolsets are included, plus a general classifier and regression, and data tables. In my opinion, this is a decoy to lead you to a Turi Create palace inhabited by a “good dog” instead of an evil witch! (The Turi Create logo is a dog.)

In this tutorial, you’ll start with a Create ML tip: Create an image classifier with a GUI, using images from the Kaggle Cats and Dogs Dataset. You then compare this to the Turi Create example using the same data set. As you can see,Turi Create is much more manual, but it’s also much more flexible and not at all difficult to understand! For the code-based example, you will compare the Create ML and Turi Create text classifier codes.

Then I’ll show you how to quickly Create an environment to use Turi Create. Apple has even changed Xcode’s PlayGrounds to make it closer to the Jupyter Notebook (previously known as the IPython Notebook, which is an interactive Notebook that runs in over 40 programming languages), so that the code environment will be more familiar! Try it out, you’re going to use Turi Create to Create an image similarity model in the Jupyter Notebook for the same number of cats and dogs in the data set.

You can wait and hope Apple moves the rest of Turi Create into Create ML, but you’ll find it easy to use Create ML as a stepping stone to use Turi Create directly. If you need more information, we also have tutorials to teach you how to use Keras, Scikit-Learn and Caffe (available later). Once you are familiar with the development environment, there are a number of tutorials in ML that you can choose from.

Note: What about Swift for TensorFlow? Create ML is an ML tool for Swift users, and Swift for TensorFlow is a Swift language for ML users – the goal of this project is to provide a better programming language, and compiler.

start

To follow this tutorial, you need:

  • A Mac, running macOS 10.14 Mojave Beta
  • Xcode 10.x beta

Click the Download material button at the beginning or end of the article. The starter folder contains:

  • Pets-100, PetS-1000, and pets-testing: These include pictures of cats and dogs; You’re going to use these to train and validate this cat-dog classifier.
  • ClassifyingImagesWithVisionAndCoreML: apple CoreML sample program; You will replace the MobileNet model with the one you trained in Create ML.
  • Good-dog-. PNG: An additional picture of a dog.
  • Turienv.yaml: You will use this file to Create an environment to run the Turi Create code.

Create ML image classifier

First, prepare your data — you’ll use it to train a picture classifier model to recognize pictures of cats and dogs. When you show an image, it will return the tag “Cat” or “Dog”. To train this model, you need a Cat folder and a Dog folder. Ideally, the two folders should have the same number of pictures — if you have 30 pictures of cats and 200 pictures of dogs, the model will be biased to recognize the pictures as dogs. Do not include any pictures of more than one animal at a time.

How many images are needed for each class? At least 10, but more images will make the model more accurate. Each class in the Kaggle Cats and Dogs Dataset has 12,500 images, but you don’t need all of them! The training time increases with the number of pictures, and roughly doubles as the number of pictures doubles.

To train a Create ML image classifier, you need to give it a Training dataset — a folder containing the classification folders. In fact, the Starter folder contains two data sets THAT I have prepared,Pets-100 and PetS-1000.

Testing dataset
Pets-Testing

You’ll start by training the PetS-100 model and then test it with Pets-testing. Then train with petS-1000 and test with Pets-testing.

Apple’s special trick

In Xcode 10, create a new macOS version of playground and type in the following code:

import CreateMLUI

let builder = MLImageClassifierBuilder()
builder.showInLiveView()
Copy the code

To display the auxiliary editing area, click the “Run” button:

You will create and present an interactive view to train and validate an image classifier. It magically lets you experience different data sets — because it’s not about who has the best algorithm, it’s about who has the best data. The algorithm is so good right now, you can get data scientists to work on it to make it better. But garbage data goes in, garbage model comes out; Most of the time, most of the effort, is spent planning data sets for machine learning. Now, this GUI image classifier helps you improve your data planning skills! You can download the Kaggle Cats and Dogs Dataset and create your own Dataset. After you see the output of my data set, you may want to be more careful about picking pictures from the pile.

Drag the petS-100 folder into the view. The training process began immediately. After some Time, a table appears in the Debug field showing Images Processed, Elapsed Time, and Percent Complete:

What’s going on here? This is called Transfer Learning. The underlying model — VisionFeaturePrint_Screen, which supports the Vision Framework — is pre-trained with massive data sets to recognize a large number of different types of images. It classifies images by learning which features to look for and how to combine them. So, the training time for your data set is actually the time to extract about 1000 features. These features may include lower-level shapes and textures, as well as higher-level shapes such as ears, eye distance, and muzzle shape. And then it takes a very small amount of time to train a logistic regression model to sort your images into two categories. It’s like dividing a line into discrete points, only on a scale of 1,000 instead of 2. But it’s still very fast: I took 1m 15s for feature extraction and 0.177886 seconds for training and logistic regression.

Transfer learning can only work successfully if your data set feature points are very similar to the data set feature points of the training model. A model pretrained on ImageNet — trained with a lot of real photos — may not migrate to a pencil drawing or photomicrograph.

You might want to check out two more interesting articles about features in Google Brain/Research:

  • Feature Visualization
  • Building Blocks of Interpretability

Note: I’m running Create ML on an early-2016 MacBook with a 1.1ghz CPU. Your time may be faster, especially if your Mac is the latest model, or an updated macOS,Xcode version, etc. on the 2017 MacBook Pro with a 2.9ghz I7 CPU, The feature extraction time dropped to 11.27s and the training only took 0.154341 seconds.

Training & Validation accuracy

When the Training is complete, the view displays metrics for Training and Validation accuracy, along with details for the Debug area.

Validation set

So what is validation? What does accuracy mean? Training accuracy is simple: Training involves guessing the weight of each feature. Because you label the picture “Cat” or “Dog”, the training algorithm can check its answer and calculate the correct percentage. The correct or incorrect information is then fed back to the next iteration to refine the weights. Validation accuracy is similar: Before the training begins, 10% of the data set is randomly selected and put into the Validation data. Like a training set, features are extracted and answers are calculated and weighted. But the results don’t have to be directly reweighted. Their aim is to prevent overfitting — focusing too much on unimportant features like background color or lighting. If the verification accuracy is very different from the training accuracy, the algorithm will adjust automatically. Therefore, the selection of verification images affects both verification accuracy and training accuracy. Turi Create lets you provide a fixed validation data set if you have created a similar feature parameter for your test data. And your test data set is a representation of the photos that users provide to the app.

Evaluation (Evaluation)

The real question is: how does the model classify untrained images?

The view prompts you to drag and drop images to start Testing: Drag the pets-Testing folder into the view. Soon, the debug area displays the accuracy of the Evaluation and its details:

In the image above, I click the Show button to see the confidence: the model is 100% convinced that this is a dog! But a closer look at the other photos showed that the model still got the right answer for the other poor quality photos.

Improved accuracy

The PETS-100 training data set has only 50 photos per class. Create ML makes it easy to experiment with different data sets to see how more data can improve accuracy.

Click the Stop button for playground, and then click again when it becomes run. This loads a new view, ready to receive the training data.

Pets-1000
Improving Your Model’s Accuracy

Improved training accuracy

  • Increase the Max iterations of the image classifier.(This option is unavailable in the first Xcode beta, but will be available in Beta 2)
  • Use different algorithms for text classifiers.
  • Use different models for general classifiers or regressors.

Improved verification accuracy

  • Increase data: For the image classifier, you can add parameters, such as flipping flipping, rotating, oblique shearing, or changing exposure. seeApple’s illustrationof data augmentation:
  • Possible overfitting:To reduceMaximum Iterations.You probably don’t need to worry about this, because my training only runsLess thanAfter 10 times, he was satisfied and stopped.

Improved assessment accuracy

Make sure your training data is diverse enough to match your test data, and that the two data sets are similar enough to the user-uploaded images on your app.

Back to the playground

We trained 1000 images and got 100% training accuracy, but only 96% validation accuracy. We can run it several times and sometimes get 99% validation accuracy.

Pets-Testing

Increase the maximum iterations?

The sample program is fairly accurate — the underlying model probably already knows about cats and dogs. However, if you are training different types and get a low rate of iterations, you may try to set the Max Iterations to 20.

Stop and restart playground, then click on the expansion symbol in the upper right corner, change 10 to 20, and press Enter:

Click the expand symbol to close the setting, then open it and check once to make sure it’s still 20.

Note: Creativity is a problem in both Create ML and Turi Create – you can’t train models without creating them. To increase the number of iterations, you have to start from scratch and extract features as before. The Create ML GUI does not provide an option to save features. In more manual frameworks, such as Keras, the model can be constructed, compiled, and then adjusted, so the adjustment only affects the subsequent operations. It is possible, in fact, to map the Turi Create source code to the low-level code that extracts the feature points of the image — this is the part that is most useful in many cases. You can then save the extracted features and reload them when you want more iterations! Hopefully this will make you even more interested in Turi Create and Keras!

Use the picture classifier

Here’s another apple trick. Create ML GUI outputs a Core ML model, then drag and drop your model directly into the old Core ML project, change one word in the code, and you’re ready to run!

Click the expand symbol in the upper right corner to see a different setting. Click the text and change it to PetsClassifier. Change Where to the starter folder and click Save:

Open the starter folder ClassifyingImagesWithVisionAndCoreML project. This is apple’s 2017 project, and I’ve upgraded it to Swift 4.2 and fixed the photo reading issue. This project uses mobilenet.mlModel, which is 17.1MB in size:

Drag the petsclassifier. mlModel into the navigation area of the project. Its size is 17KB:

Search for MobileNet in the project:

let model
MobileNet
PetsClassifier 

let model = try VNCoreMLModel(for: PetsClassifier().model)
Copy the code

Click on the camera icon to go to the Photo selector, then drag some dog and cat images into Photos:

Turi Create image classifier

This code is from the Turi Create Image classifier example, which uses the same dataset — the complete dataset of 25000 images:

import turicreate as tc

# 1. Load images (Note: you can ignore 'Not a JPEG file' errors)
data = tc.image_analysis.load_images('PetImages', with_path=True)

# 2. From the path-name, create a label column
data['label'] = data['path'].apply(lambda path: 'dog' if '/Dog' in path else 'cat')

# Note: If you have more than two classes, extract the folder names like this:
# train_data["label"] = train_data["path"].apply(lambda path: os.path.basename(os.path.split(path)[0]))

# 3. Make a train-test split
train_data, test_data = data.random_split(0.8)

# 4. Create the model
model = tc.image_classifier.create(train_data, target='label')

# 5. Save predictions to an SArray
predictions = model.predict(test_data)

# 6. Evaluate the model and save the results into a dictionary
metrics = model.evaluate(test_data)
print(metrics['accuracy'])

# 7. Save the model for later use in Turi Create
model.save('mymodel.model')

# 8. Export for use in Core ML
model.export_coreml('MyCustomImageClassifier.mlmodel')
Copy the code

This is a lot more code than you’ve written in playground, but you’ll soon see that it’s similar to the Create ML text classifier code.

Similar to what you did in Create ML:

  • Steps 1 through 4 correspond to creating the training and test folders, and then dragging the training folders into the view. Turi Create must extract the class tags from the image path, but step 3 randomly assigns 20% of the data set to test_Data, saving you the work of creating training and test folders, and getting a different test data set every time you run the code.

Note: In step 2, extracting only the labels of two classes is a special case. I’ve added a comment to the code to show the normal case. First,os.path.split() splits the path into two parts: the file name (such as 42.jpg), and the rest. Then os.path.basename() is the name of the last folder, which is also the name of the class.

  • Steps 5 and 6 correspond to dragging the test folder into the view. The Jupyter Notebook can easily display the Predictions array as a Create ML view. You can also filter the data to find incorrect categories without having to iterate through the entire test gallery.
  • Step 7, save the model for use, you can reload it and run it on a different set of test data.
  • Step 8, output the Core ML model.

So,Turi Create image categorization is more manual than Create ML, but more flexible. The documentation for Turicreate.create() lists some optional parameters. You can specify the underlying model to match Create ML. Look at the size of the Core ML model! You can also set a fixed validation_set if you have created a real test data and don’t want the model to use test data randomly selected from your training data.

In the Create ML image classification is a very special case: MLImageClassifierBuilder GUI interface makes the code is no longer necessary. But in the next section, you’ll see that other Create ML models still require a lot of code.

Text classifier

Now compare how Create ML and Turi Create train and test text classification models. The Turi Create model needs to convert the test text into the Bag of Words — in the Create ML model, this conversion is built directly into the Create ML model, so you just receive the test text folder directly.

Create ML

Here is an example of a Create ML text classifier:

import CreateML

// 1. Load data from a JSON file
let data = try? MLDataTable(contentsOf: URL(fileURLWithPath: "<#/path/to/read/data.json#>"))

// 2. Make a train-test split
let (trainingData, testingData) = data.randomSplit(by: 0.8, seed: 5)

// 3. Create the model
let sentimentClassifier = try? MLTextClassifier(trainingData: trainingData,
  textColumn: "text", labelColumn: "label")
  
// 4. Training accuracy as a percentage
let trainingAccuracy = (1.0 - sentimentClassifier.trainingMetrics.classificationError) * 100

// 5. Validation accuracy as a percentage
let validationAccuracy = (1.0 - sentimentClassifier.validationMetrics.classificationError) * 100

// 6. Evaluation accuracy as a percentage
let evaluationMetrics = sentimentClassifier.evaluation(on: testingData)
let evaluationAccuracy = (1.0 - evaluationMetrics.classificationError) * 100

// 7. Add metadata
let metadata = MLModelMetadata(author: "John Appleseed",
  shortDescription: "A model trained to classify movie review sentiment", version: "1.0")

// 8. Export for use in Core ML
try? sentimentClassifier.write(to: URL(fileURLWithPath: "<#/path/to/save/SentimentClassifier.mlmodel#>"),
    metadata: metadata)
Copy the code
  • Step 1: Load text and tag columns into the table. The tag values are positive, negative, and neutral. In WWDC 2018 Session 703 Video, we showed another way to load separate saved text files named Positive and Negative, similar to loading images into an image classifier. But it only works for Create ML; Does not apply to Turi Create.

WWDC 2018 Session 703 provides another way to load tag text data:

let trainDirectory = URL(fileURLWithPath: “/Users/createml/Desktop/ "train")let testDirectory = URL(fileURLWithPath: “/Users/createml/Desktop/ test ")// Create Model
let classifier = try MLTextClassifier(trainingData: .labeledDirectories(at: trainDirectory))
Copy the code

Back to the text classifier main code:

  • In the second step, as in Turi Create random_split(), 20% of the data is randomly assigned to testingData. The optional seed parameter is used to set the seed of the random number generator.
  • Step 3: Do the same thing as in Turi Create’s sentence_classifier.create().
  • In steps 4-6, calculate training, validate, and evaluate accuracy metrics.
  • Steps 7 and 8 output the Core ML model with metadata.

Turi Create

This code comes from our tutorial Natural Language Processing on iOS with Turi Create. It trained a sentence classifier using poems from 10 poets to predict the authors of the test texts.

import turicreate as tc

# 1. Load data from a JSON file
data = tc.SFrame.read_json('corpus.json', orient='records')

# 2. Create the model
model = tc.sentence_classifier.create(data, 'author', features=['text'])

# 3. Export for use in Core ML
model.export_coreml('Poets.mlmodel')
Copy the code
  • Step 1: As with Create ML, you can load data from JSON or CSV files.
  • Step 2: Train the model.
  • Step 3, output the Core ML model.

Included in the Turi Create tutorial materials is an iOS app where you can paste text into a TextView to test the model. This app uses a wordCounts(text:) function similar to the bag of words function at the end of Turi Create text classification example.

The Turi Create text classifier expects the input to be a dictionary of words and word counts. The Create ML text classifier receives text input directly and creates its own bag of words.

Turi Create image similarity

Now take a deep breath — you’re in for a Turi Create adventure!

Turi Create has five other task-focused Toolkits that are not currently included in Create ML:

  • Recommendation system
  • Image similarity
  • Object detection
  • Style migration
  • Classification of activities

Pictures of dogs and cats look very interesting, so you will train a model to find similar pictures.

Yes, you need to write some Python now. The development environment uses Jupyter Notebook and you’ll be familiar with it — it’s like an Xcode playground, but it runs in your browser.

The easiest way to do this is to use Anaconda — which was created by the ML community to clean up all versions of Python and ML libraries and manage them in separate environments.

Anaconda & Notebooks

Download Python 3.6 Version of Anaconda for macOS and install it in your home directory, not the root directory:

Install on a specific Disk…Button and then click back toHome

Note: It takes a few minutes to install Anaconda and Create the Tuti Create environment. While you’re waiting, check out Michael Kennedy’s Comparison of Python and Swift Syntax (November 2014) and Jason Brownlee’s Crash Course in (May 2016) Python for Machine Learning Developers.Brownlee’s article includes examples using the data science libraries NumPy, Matplotlib, and Pandas. The biggest difference between Swift and Python syntax is that when you define closures, functions, and classes, you use indentation instead of {… }.

Create the Turi Create environment

You can use the Anaconda Navigator GUI or the Terminal command to Create the environment to run the Turi Create code.

GUI: Open the Anaconda Navigator, switch to the Environments TAB page, and import starter/ Turienv.yaml — click on the folder icon and set the file location in the Finder. Anaconda Navigator will populate the environment name from the file:

Terminal
Terminal 

conda env create -f <drag starter/turienv.yaml file from Finder>
Copy the code

Start the Jupyter Notebook

In the Turienv environment, start the Jupyter Notebook with the GUI or Terminal command.

First, in Finder, create a local folder named Notebooks.

If you have the latest Mac, download and unpack the Kaggle Cats and Dogs Dataset, then move the PetImages folder back to notebook so you can easily load it into the notebook.

The complete Kaggle data set contains 25,000 images, which would take a long time to process on older Macs. You can use the PETS-1000 file instead, or create your own data set.

GUI: If you’re using Anaconda Navigator, go to the Home TAB, click On Turienv in Applications on, and then click on Jupyter Launch:

notebooks
Terminal
Terminal
turienv 

source activate turienv
Copy the code

The Command line prompt is now started with turienv. Enter the following command to launch the Jupyter server in the Notebooks folder and display the browser window:

jupyter notebook <drag notebooks folder from the Finder>
Copy the code

Training model

Create a new Python 3.6 Notebook:

Note: This example is similar to Apple’s Image Similarity example, but uses the cat and dog datasets.

We have a blank cell in the notebook. Enter the following line in the cell, then shift-Enter to run the cell:

import turicreate as tc
Copy the code

Note :Shift-Enter is also available in the Xcode playground if you just want to run a piece of code.

Another new cell appears. Enter the following and run:

reference_data = tc.image_analysis.load_images('./PetImages')
reference_data = reference_data.add_row_number()
reference_data.save('./kaggle-pets.sframe')
Copy the code

Now you are loading the image into the table, adding row numbers to the table, and saving it for future use. Ignore JPEG decode Failure information.

Note: Use the TAB key to auto-complete when typing Python code.

In the next cell, run the following statement to retrieve the data:

reference_data.explore()
Copy the code

A new window opens showing the ID, path, and picture columns. Move the cursor over the line to show the picture:

Next, run the following statement:

model = tc.image_similarity.create(reference_data)
Copy the code

This will take a while – show it running in [*]. While you wait, read about unsupervised Learning.

Note: If you need to manually Stop the cell before completing, click the Stop button (next to Run in the toolbar). You can remove images from PetImages, or just load petS-1000. While my 2015 MacBook Pro was running, I went out for lunch and returned 90 minutes later to finish running it.

Unsupervised learning

Providing tag data to the image classifier allows it to measure accuracy by detecting the difference between its predictions and the tags. Supervised learning.

However, sometimes even though you provide the same set of labels to an image-like trainer, it doesn’t use those labels: this model uses unsupervised learning. The base model looks at a very large number of images and teaches itself which combinations of pixels make up features that can be used to cluster “similar” images. So just like with picture classifiers, most of the training time is spent extracting features from your data set. Then, it trains the “violent” nearest Neighbors model: for each image, it calculates its distance to other images and sorts the other images by radius. And then, again, compared to the extracted feature points, this is a quick step.

The query model

When the model is ready, run the following lines:

query_results = model.query(reference_data[0:10], k=10)
query_results.head()
Copy the code

So, you pass in an array of 10 reference_data images, calculate the similarity of each of the 10 images, and show the first 10 rows of query_Results.

Let’s say you want to look at a similar picture in number 10. First, let’s see what it is:

reference_data[9] ['image'].show()
Copy the code

So, run these lines of code:

similar_rows = query_results[query_results['query_label'] = =9] ['reference_label']
reference_data.filter_by(similar_rows, 'id').explore()
Copy the code

Congratulations to you! You have now created an image similarity model in Python! Your Mac didn’t explode. Hopefully, you’ll also try running other Turi Create examples on your own data.

Shut down

Log out of the jupyter browser window.

In the Terminal window, the Jupyter server is running. Press Control-C-C to stop the server.

If your command line prompt is started with turienv, enter the following command to exit:

source deactivate
Copy the code

If you really don’t want to use Anaconda anymore, type the following command:

rm -rf ~/anaconda3
Copy the code

And then what do we do?

The full Turi Create Notebook and iOS Project are in the Finished folder in this article’s resources. You can download it at the beginning or end.

Now that you have learned to experiment with data sets in Create ML, hopefully you will continue to learn about Turi Create.

Explore Create ML and its official documentation, but also spend some time browsing the Turi Create User Guide, even if you don’t want to use Python. In Turi Create’s How It Works documentation, a wealth of information is provided without the need for deep mathematical knowledge. To learn more, see their academic citation links.

Here are some resources for continuing your study:

Our tutorial

This article is the latest in our series of ML tutorials. Others include:

  • Core ML and Vision: Machine Learning in iOS 11 Tutorial
  • Beginning Machine Learning with Keras & Core ML
  • Beginning Machine Learning with scikit-learn
  • IBM Watson Services for Core ML Tutorial
  • Natural Language Processing on iOS with Turi Create
  • Caffe (to be launched later).

ML community

  • Kaggle is a data set that is contributed by members, typically using notebook collaboration to analyze and visualize the data. It runs the model to predict competition, as shown in the link below.
  • Machine Learning Zero-to-Hero: Everything you need in order to compete on Kaggle for the first time, step-by-step!

I hope you enjoyed this Create ML tutorial, and if you have any questions or comments, please comment below. Especially remember to tell us what you do with Create ML and Turi Create!

Download this document