Abstract: For AI beginners with a certain AI foundation, ModelArts provides preset algorithm based on the mainstream engine of the industry, without paying attention to the model development process, directly use the preset algorithm to train the existing data, and quickly deploy as a service. The preset algorithm can be used for object category and location, image classification and other scenes. This experiment provides a flower image classification application sample, to help you quickly familiar with the process of using ModelArts preset algorithm to build a model.

This sample annotates the existing image data for the preset flower image data set, and then trains the data with the preset “ResNet_v1_50” algorithm to get an available model. Finally, this model is deployed as an online service. After the deployment is complete, users can identify the type of flowers in the input picture through the online service. The steps to complete the model construction using the preset algorithm are as follows:

  • Step 1: Prepare data
  • Step 2: Train the model
  • Step 3 :(optional) create a visualization job and view the model training process
  • Step 4: Import the model
  • Step 5: Deployment online
  • Step 6: Test the service
  • Step 7: Clear resources to avoid costs

The preparatory work

  • You have registered a Huawei cloud account and checked the account status before using ModelArts. The account cannot be in arrears or frozen state.
  • Access authorization has been configured for the current account. If not, refer to Using Delegation for Authorization. You are advised to clear the authorization of users who have previously used the access key, and then authorize users by proxy.
  • Buckets and folders have been created in the OBS service to hold sample datasets and models. In the following example, create a bucket named test-ModelArts and create folders as shown in Table 1.
  • For details about how to create OBS buckets and folders, see Creating A Bucket and Creating a Folder. Make sure that the OBS directory you use is in the same area as ModelArts.

Step 1: Prepare data

ModelArts provides a sample Data Set for Flowers in a public OBS bucket named ** “flowers-data-set **”, so the operation examples in this article use this Data Set for model building. You need to perform the following operations to upload data sets to your OBS directory, that is, the test-ModelArts/Dataset -flowers OBS directory that you created during the preparation.

Description:

  • The flows-data-set Data Set has been annotated, and the. TXT file in the Data Set is the annotation file of the corresponding image. Therefore, Data annotation is ignored in this procedure.
  • If you want to use your own data set, skip this step, upload the data directly to the OBS folder, and select this directory directly in the Step 2: Training Model operation. If the data set you are using is not annotated, choose Data Management > Data Set to create a data set and manually annotate the data set before creating a training job.
  1. Click the download link to download the ** “flows-data-set **” dataset to your local PC.
  2. On the local PC, decompress the ** flows-data-set. zip package. For example, decompress the package to the flows-data-set ** folder.
  3. Upload all the files in the flows-data-set folder to the test-ModelArts /dataset- Flowers OBS by referring to uploading files in batches.

Step 2: Train the model

Once the data is ready, you can create a training job using the preset algorithm “ResNet_v1_50” and eventually generate a usable model.

“ResNet_v1_50” algorithm is based on “TensorFlow, TF-1.8.0-PYTHon2.7” engine, which is used for image classification. If you want to obtain more introduction to the preset algorithm, you can click the Preset Algorithm TAB on the Training Job page to learn the algorithm information provided by ModelArts, such as purpose, engine type, and accuracy.

1) In the ModelArts management console, choose “Training Management > Training Job” in the left navigation bar to enter the “Training Job” management page.

2) Click Create. The Create Training Job page is displayed.

3) On the Create Training Job page, set related parameters. For details, see the following steps.

  • Algorithm Source: Click Select, and select ResNet_v1_50 from the Preset Algorithm list.
  • Data Source: Because the imported data set has been annotated, it can be imported directly from the data store location. Click Data Store Location, click Select on the right of the text box, and select the OBS path where the dataset resides, for example, / test-modelArts /dataset-flowers/.
  • Training Output Location: Select the model and forecast file storage path from the existing OBS bucket. Use the **model-test ** folder that you created in preparation. If no folder is available, click Select to create a folder in the dialog box that is displayed.
  • Running Parameters: If ResNet_v1_50** is selected, max_EPOches is included by default. The default value is 100. For this example, you are advised to change the value of max_epoches to 10. One epoch indicates that the entire data set is trained once, and the running parameter indicates that 10 epoches are trained. The larger the value of max_epoches is, the longer the training time will be.
  • Job Log Path: Select a path for storing logs from existing OBS buckets. Use the “train-log” folder that you created in preparation. If no folder is available, click Select to create a folder in the dialog box that is displayed.
  • In the basic information area, Billing Mode and Version are automatically generated by the system and do not need to be changed. Set Name and Description as prompted.

Figure 1 Fill in the name and description

  • In the parameter configuration area, select Algorithm Source, and set Data Source, Training Output Location, Running Parameters, and Job Log Path.

Figure 2 Parameter configuration

  • In the resource Settings area, select Common Resource Pool and set Specifications and Number of Compute Nodes.

If you choose to use the free spec, please read the prompt carefully and check “I have read and agree to the above”.

Figure 3. Set up the resources used

  • Click Next to complete the information.

4) On the “Specification Confirmation” page, confirm the parameter information of the training operation and click “Submit” after confirming all information.

5) On the “Training Job” management page, you can view the status of the new training job. It takes about 10 minutes to create and run a training job. When the status changes to Run Succeeded, the training job is created.

You can click the name of a training job to view its configuration, Log, and Resource Usage information on the job details page. The generated model file can be obtained in the OBS path of “training output location”, that is, “/test-modelarts/model-test/”.

FIG. 4 Details of training operations

Step 3 :(optional) create a visualization job and view the model training process

Currently, visualization jobs provided by ModelArts default to TensorBoard type jobs. TensorBoard is a visual tool that can effectively show the calculation diagram of TensorFlow or MXNet during operation, the variation trend of various indicators over time, and the data information used in training. Visualization jobs currently only support training jobs based on TensorFlow and MXNet engines.

If the details in the training details page are sufficient for you to determine whether the model is good or not and build the model, you can skip this step and go directly to Step 4: Importing the model.

1) In the ModelArts management console, choose “Training Management > Training Job” in the left navigation bar, and then click the “Visual Job” TAB to enter the “Visual Job” management page.

2) On the Visual Job management page, click Create.

3) On the Create Visual Job page, set related parameters and click Next.

The default visual job type is Visual Job and cannot be changed. Set the name of the visualization job, and the Training Output Location. Training Output Position must be set to Training Output Position in the training task. In the preceding steps, Training Output Position is /test-modelarts/model-test/. Automatic Stop is enabled and set to stop after 1 hour to avoid unnecessary costs.

Figure 5 setting visual job parameter information

4) On the Confirm Specifications page, click Submit after confirming the information.

5) Go to the Visual Job management page and wait for a period of time. When the visual job status changes to Running, it is created successfully.

For a running visual job, you can click the name of the visual job to jump to its visual interface. You can learn about the training process of the model on the page. If the model training process and parameters meet the requirements, you can proceed to Step 4: Import the model operation.

Figure 6 Visual interface

Step 4: Import the model

The trained model is still stored in the OBS path, which you can import into ModelArts for management and deployment.

1) In The ModelArts Management Console, click “Model Management > Model” in the left navigation bar to enter the “Model” page.

2) On the Model page, click Import.

3) On the Import Model page, set related parameters and click Create Now.

Set the “name” and “version” of the model, and select “Select from Training” in the parameter “Metamodel Source”. The system will automatically select the training job you created. You can select the training job available in the system from the drop-down box. Because the data in this example is simple, use the default values for other parameters.

Figure 7 Import model

4) After the model is imported, the system will automatically jump to the model list page. You can view imported models and their versions on the Models list page.

Figure 8 Model list

Step 5: Deployment online

After the model is imported, if the model status is Normal, you can deploy the model online or to Online service, Batch Service, or Edge Service. The following describes how to deploy an online service.

1) Choose Model Management > Models, and click the triangle to the left of the model name to open all versions of the model. Locate the row that contains the target version, click Deploy in the operation column, and select Online Service from the drop-down list. The Deployment page is displayed.

2) On the Deployment page, set related parameters and click Next.

Set Name for an online service to enable automatic Stop. In the Select Model and Configuration area, the system automatically selects the model and its version in Step 4: Import model. From the drop-down list on the right of Compute Node Specifications, select the resource to be used. In this example, CPU: 2-core 8GiB is used to deploy online services. Use the default values for other parameters.

You are advised to use the default values for Data Collection and Difficult Case Filtering.

Figure 9 Deploying as an online service

3) On the Confirm Specifications page, click Submit after confirming the information.

4) Choose Deployment Online > Online Service to view information about online services. Because it takes some time for the model to be deployed online, please be patient for a few minutes. When the online service status is Running, the online service has been deployed.

Step 6: Test the service

  1. After the online service is successfully deployed, you can enter the online service and initiate a predictive request for testing.
  2. On the Online Service management page, click the name of the online service to go to the details page.
  3. On the online service details page, click the Forecast TAB to access the forecast page.
  4. To the right of Select forecast picture file, click the Upload button, upload a picture with flowers, and then click Forecast.
  5. After the prediction is completed, the prediction result will be displayed in the prediction result display area. According to the score of the prediction result, the flower in this picture can be identified as “Daisy”.

Note: Since this is a test service, to ensure the test effect, it is recommended not to use the existing images in the sample data set when selecting images.

FIG. 10 Prediction results

Step 7: Clear resources to avoid costs

To avoid unnecessary costs, it is recommended that you delete related resources, such as online services, visualization jobs, training jobs, data and their OBS directories, after completing the trial.

  • Deleting an online service: On the Online Service page, click More in the operation column, and select Delete.
  • Deleting a Visual Job: Choose Training Job > Visual Job, and click Delete in the operation column.
  • Deleting a training Job: On the Training Job page, click Delete in the operation column.
  • Delete data: Go to OBS, delete the uploaded data, and then delete the folder and OBS bucket.

This article is shared from huawei cloud community “Using preset algorithm to build a model experience”, the original author: Luck boy.

Click to follow, the first time to learn about Huawei cloud fresh technology ~