The 7th Engineering Training Competition
On September 9, 2021.21, Zhejiang Province held the 7th Engineering Training Competition. Our group participated in the project of garbage classification, and our group successfully advanced to the final. However, we saw that the rules of the final did not label multiple garbage classification, so we were not fully prepared to deal with multiple garbage classification, so we missed the national competition!
preface
With the continuous development of artificial intelligence, the technology of machine learning is becoming more and more important. Many people have started to learn machine learning. This paper introduces the basic content of machine learning through the competition, and the interface design needed to participate in the competition
The rules of our preliminary competition are as follows, but I am a little confused about the display of garbage names. It would be better to divide four kinds of garbage, but why distinguish vegetable leaves and orange peels?
I introduce myself first, I’m from computer science and technology a sophomore year undergraduate, I and two of my teammates for the 7th engineering training series, I have in the team as a task is to image processing, and model training, stm322F4 serial port communication and interface design, so I focus on the next implementation code, I speak is not good, hope to understanding!
When we play, the garbage is like this!
(1) Software installation preparation
Software housekeeper have these software download, you might as well go to WeChat public number to pay attention to, download the steps are really detailed!!
I installed Anaconda PyCharm Python 3.6(not too high version) Qt interface design software
Download TensFlow via Python 3.6
Other library downloads require you to set up the environment and then download it from the following location in PyCharm
Format: PIP install numpy
We use the laptop computer for training, and then put the model training on the computer Win10 for running the interface display (note: the training machine must be better!! If possible, direct access to desktop computers)
(2) Training model of garbage classification
First, I show my final code for the competition in the file below
Then we look at the garbage classification itself, it seems to be very understandable, that is, to distinguish garbage, but how to achieve it? Maybe not at all. It would be much faster to just start with the file above.
Like how we categorize things, is it a learning process? But what is the basic? It’s our ability to learn, and that’s the framework of the program model
The first.py file is the training model, the framework is using the RESNET model in Keras, and then we train this model into a targeted model to deal with image recognition through training.
This is the training set, which is to put the pictures in the “DataSet1” folder (I was classifying them into 10 kinds, each 420 pictures) for training. However, if the hardware condition allows, the more pictures the better.
However, be careful not to put images in the Chinese path, and the number of images in each file as much as possible
# Processes the 224*224 folder in the program with the directory... train_path_A = './dataset1/train/A/' train_path_B = './dataset1/train/B/' train_path_C = './dataset1/train/C/' train_path_D = './dataset1/train/D/' train_path_E = './dataset1/train/E/' train_path_F = './dataset1/train/F/' train_path_G = './dataset1/train/G/' train_path_H = './dataset1/train/H/' train_path_I = './dataset1/train/I/' /dataset1/train/J/' mglist_train_A = os.listdir(train_path_A) # Imglist_train_b = ' os.listdir(train_path_B) imglist_train_C = os.listdir(train_path_C) imglist_train_D = os.listdir(train_path_D) imglist_train_E = os.listdir(train_path_E) imglist_train_F = os.listdir(train_path_F) imglist_train_G = os.listdir(train_path_G) imglist_train_H = os.listdir(train_path_H) imglist_train_I = os.listdir(train_path_I) imglist_train_J = os.listdir(train_path_J)Here we define two numpy objects, the X_test input array and the Y_test tag array, and create an empty multidimensional array for np.empty.
3 is the number of channels of the image (RGB tricolor)
Because there are ten images, the second item of Y_train() is set to 10
X_train = np.empty((len(imglist_train_A) + len(imglist_train_B) + len(imglist_train_C) + len(imglist_train_D) + len(imglist_train_E)
+ len(imglist_train_F) + len(imglist_train_G) + len(imglist_train_H) + len(imglist_train_I) + len(imglist_train_J), 224, 224, 3))
Y_train = np.empty((len(imglist_train_A) + len(imglist_train_B) + len(imglist_train_C) + len(imglist_train_D) + len(imglist_train_E)
+ len(imglist_train_F) + len(imglist_train_G) + len(imglist_train_H) + len(imglist_train_I) + len(imglist_train_J), 10))The trained models are saved on the following models
H5 model.save(' my_resnet_model_abcd.h5 ') model.save_weights(' my_resnet_weights_model_abcd.h5 ') # Save variable training file H5 model = tf.keras.models.load_model(' my_resnet_model_abcd.h5 ') model.load_weights('my_resnet_weights_model_ABCD.h5')If you change the training type of the data, these parameters need to be corresponding to the change of the arrow pointing to the data need to focus on the review, the annotation above is detailed, that is, batch_size preferably even number
The training model is available, but we want it to give us an accurate value of feedback. At this time, we need the test set. The ratio between the training set and the test set picture is about 10:3
# predict(img) for I in range(10): Img =X_test[I] # print(' X_test[I] '); img=X_test[I] # print(' X_test[I] '); Img. Shape) img = (np) expand_dims (img, 0)) at position 0 # said to add a dimension data [[[...], [...], [...],..., [...]]]. # tf keras model input form to add a dimension,28,28 (1) Print (" IMG. SHAPE ", Predictions = Model.predict (img) # Print (" IMG. SHAPE ", Predictions = Model.predict (img Print (" Predictions [I]: ") # The maximum probability of the 0th item of the picture, Print (" The predictions ",np. Argmax (phoresis)) # print(" The predictions ",Y_test[I]) # print(" The predictions ",Y_test[I]) plt.figure() plt.imshow(X_test[i]) plt.colorbar() plt.grid(False) plt.show()(3) Qt interface design
When I was designing the interface, I tried to arrange the content according to the size of the screen, so the content was relatively compact and the background was green, which was not difficult to think about. It was related to this theme.
How do I display text on TextEdit? How do I trigger buttons?
# write_ui... The interface of textEdit... Text_print1.connect (self.write_ui1) self.ms. Text_print2.connect (self.write_ui2) self.ms. Text_print2.connect (self.write_ui2) self.ms.text_print3.connect(self.write_ui3) self.ms.text_print4.connect(self.write_ui4) Self. Ms. Text_print5. Connect (self. Write_ui5) # initialization thread parameter self. The UI. The pushButton. Clicked. Connect (self. HandlePlay) # PushButton_Clicked. Connect (self.handlecircle) # PushButton_7. Clicked. Connect (self.handleStopPlay) PushButton_Clicked. Connect (self.handlestart) # Stop playing self.ui.pushButton_Clicked. Connect (self.handlestart) # Check starts PushButton_4.clicked. Connect (self.ui.pushButton_5.clicked. Connect (self.handleshow) # Displays image self.ui.pushButton_6.Clicked. Connect (self.handlequit) # Close image # on each TextEdit... Self, str1: self.ui.TextEdit. append(str1 + '\n') Self.ui.textedit2.append (str2 + '\n') # Textedit_2 STR def write_ui3(self, str3) Self.ui.textedit_3.append (str3 + '\n') # Textedit_3: Self.ui.textedit_4.append (str4 + '\n') # Textedit_4 with STR def write_ui5(self, str5) Self.ui.textedit_5.append (str5 + '\n') # Write STR to textEdit_5How does looping and stopping play work? I loop play by triggering the button and close the flag bit to stop play. The code is as follows
Def handleCircle(self): Global ThreadFlag1 = 0 for I in range(20): j = 0 cap = cv2.VideoCapture('./refuse classification video.mp4') while (cap.isOpened()): Info = "info += f'\t-- garbage collection promo loops --\n' self.ms. Text_print1.emit (info) # str1 ret in TextEdit frame = cap.read() cv2.imshow('refuse classification video.mp4', frame) j += 1 if (j == 835): If (ThreadFlag1 == 1): Self.ms. Text_print1.emit (f'\t-- Garbage collection promo stop playing --\n') break k = cv2.waitKey(20) # Close window cap.release() CV2.destroyAllWindows () # def handleStopPlay(self) # def handleStopPlay(self) # def handleStopPlay(self) If (threadFlag1 ==0 &cap.isopened ()): threadFlag1 = 1A few weeks before the competition, I wanted to switch the camera on and off to see the environment inside the bin, because the top half of our bin is black, so it’s hard to see directly
Self = 0 self.ms. Text_print3. Emit (f') = 0 self.ms. --\n') while 1: # get a frame ret, frame = cap.read() # show a frame cv2.imshow("capture", frame) if ThreadFlag3 == 1: Self.ms. Text_print3. emit(f' -- The camera has been turned off, start to mark! --\n') break cv2.waitKey(1) # def handleQuit(self): global threadFlag3 # if threadFlag3 == 0: threadFlag3 = 1And then we do that by receiving the serial port when do we start image recognition? When do you start transmitting messages to the downloader, turning the steering gear and the garbage can
Def handleRecv(self): Global final global no global ThreadFlag2 = 0 ser.flushInput( download(): while 1: Self.ms. Text_print2.emit (f'\t-- Listen to SerialData --\n') MCU = ser.read(1) Print (" print ") print(" print ") print(" print ") print(" print ") print(" print ") print(" print ") print(" print ") print(" print ") print(" print ") print(" print ") print(" print ") print(" print ") print(" print ") print(" print ") print(" print ") print(" print ") print(" print ") If MCU == b'5' if MCU == b'5' if MCU == b'5': # If you receive data/characters sent by the lower computer, Self-m.text_print2.emit (f'\t-- start taking picture --') MCU = frameOne,frame = cap.read() # Cv2.imwrite ("D:\ Project_garbage \\picture\\0.jpg", cv2.imwrite("D:\ Project_garbage \\picture\\0.jpg", Create picture self.ms. Text_print2.emit (f'\t-- save photo --') self.predict() # create picture self.msFinally we start to predict the realization of image recognition.
I encountered many problems when I made this interface design, and I will talk about them here
First questionThat’s the difference between 0 and 1
Zero refers to the fact that the computer’s display doesn’t have a camera of its own, but instead uses a peripheral camera to take pictures
1 refers to a laptop, for example, which has its own camera and can switch between its own camera and peripheral camera
In the beginning of microcomputer debugging did not find this problem, delayed my time for a while
The second problem is that the path of the image import is very noticeable / \ difference between importing the picture and reading the picture
Cv2.imwrite ("D:\ project_garbage\\picture\ 0.jpg", frame) # Save the image, Create your own picture folder img_path = "D:/project_garbage/picture/0.jpg"Third questionThis problem has many kinds and can be solved more through Baidu. We are Python and SCM 32 lower computer to carry out serial port communication. It is crucial to add \r\n to the end
conclusion
Memorable video of the preparation
Although the 7th Engineering Training Competition failed to reach the national competition, I still remember the scene of our team members struggling together in the laboratory. I think this memory is very precious. We have worked together! To struggle! I think it’s worth it. Friendship outside competition is very rare!
First of all, I would like to share with you that the night before the competition, we protected our precious garbage can into the laboratory building (two teammates).
Next, I would like to share some photos from our trip to Hangzhou Bay in Ningbo
Ningbo Institute of Engineering Bird’s Nest type book bar, I really like!
This is before the game, when the wind is very strong, my teammates are taking photos, I am retraining the model, presenting my garbage can, full of mystery!!
