Rising college offers online courses “pick up open source framework MindSpore lesson 6 complete, king jumps over the lecturer to bring us the topic to share the MindSpore visualization tools use guide, video lesson 6 review is as follows: www.bilibili.com/video/BV1et…
Lesson 6 Questions and answers
In the QA section of lesson 6, some questions have been widely mentioned, and The lecturer Wang Yue has made selection and editing again for your reference.
Q: Momentum is SGD?
Momentum is an optimization method of SGD. The main idea is to introduce an information momentum with accumulated historical gradient to accelerate SGD. In my opinion, it mainly solves two problems of SGD, one is the noise introduced by the random gradient method, and the other is that in the process of convergence of SGD, compared with the gradient we want, there will be a relatively large swing.
Currently mindspore offers both SGD and Momentum.
Q: How is it different from TensorBoard?
I think the main differences are as follows:
From the point of view of design, TensorBoard is mainly built in the form of plug-in. Its advantages are convenient development and clear functional decoupling. Develop a new feature to quickly add a new TAB to TensorBoard. However, I think the disadvantage of this mode is that it keeps adding individual functions and lacks overall guidance to guide users on how to network step by step. MindInsight is designed to provide users with a methodology for network debugging and tuning. As you can see, the entry to MindInsight starts with the training list, and when you click on a training, you want to give the user a very clear indication of what functionality to use in the event of a problem during the tuning phase.
Component-wise, I think MindInsight now has some features that TensorBoard doesn’t have, such as traceability, data graph presentation, etc. Of course, MindInsight is still under rapid construction and development, with many new components coming online.
Ecologically, TensorBoard and Tensorflow are currently focused on GPU/TPU, while MindInsight and MindSpore need Ascend. Different chips can lead to differences in functionality, such as Profiling, MindInsight needs to consider performance presentation for training scenarios such as data sink.
Q: Does Mindspore support dynamic graphs?
MindSpore currently supports dynamic maps, which we call Pynative. How to use dynamic maps for training and debugging can be referred to lecture 3. MindInsight will release the Debugger component later. You can debug the script in Pynative mode, and use the MindInsight Debugger if an exception occurs during graph mode execution.
Q: Does the source of the data say that all training is the same data?
In the previous video, we saw a straight line of data traceability, indicating that training tasks all used the same data pipeline flow. If the user finds that the training may be abnormal due to a data enhancement operation and changes the data enhancement operation, the difference between the two training will be reflected in the data traceability.
Q: Are there any other profiles available?
Currently, in Ascend chip training or reasoning, you can use the components in the RUN package to profile, but using them requires configuration items, and parsing results depends on scripts. MindInsight will initially provide easy-to-use profile capabilities and result presentations based on the Ascend chip, and gradually migrate capabilities to gpus and cpus.
Q: Can NVProf work with Mindspore?
I personally haven’t tried it, but it should work from principle; If you are interested, you can have a try. We can discuss in the group.
Q: Training middle layer visualization?
If mid-tier visualization means mid-tier information about a compute diagram, you can use the compute diagram visual component in MindInsight to view this. You can click on the graph to find the layer you are interested in and view the node name, type and other information.
If mid-layer visualization refers to the trend of mid-layer weights, it can be recorded using HistogramSummary and viewed using the MindInsight parameter distribution function.
If the visualization of the middle layer means that you want to see the calculation results of the middle layer during training, PyNative mode can be used in conjunction with PDB to directly view the results when the operator returns, and graph mode can be viewed by relying on the Debugger of MindInsight subsequent release. The Debugger can suspend training after a step is completed. Users can find the middle layer they are interested in on the calculation diagram and click the node to view the corresponding operator output.
Q: Is MindinSight currently available in the cloud?
MindInsight is a Python package that naturally supports use in the cloud; We will provide official support and instructions for scenarios on the cloud later.
Q: Can Minsight output optimized graphics?
Yes, the calculation graphs recorded in the summary file are the graphs optimized in the front end of MindSpore. If you want to view more graphs at different stages, you can turn on the save_Graphs switch in the context to obtain some calculation graphs at different stages (the suffix is.pb). You can visualize these diagrams in MindInsight.
Q: Can you give an example of loading GE graphs in the future?
At present, the function of loading GE diagram (ge_ir.proto) is still in the trial stage. After the function is mature, we will consider open source, and the corresponding document will be provided at that time.
Lesson 6 PPT is as follows:
