This article was first published on Jizhi

Resources: towardsdatascience.com

How time flies, the 2018 World Cup ended in the blink of an eye, and this summer was given more meaning by the World Cup, leaving fans with endless stories in the smell of beer. Who would have thought the defending German champions would be sent home with their potato and sausage unfinished? All four favourites knocked out?

You may have noticed a lot of people using data science or artificial intelligence to analyze the World Cup and come up with some fun projects lately. Have you ever considered using machine learning to analyze a World Cup game yourself? Today, I’m going to share a practical guide to analyzing the World Cup with deep learning techniques and some machine learning frameworks, so you can try your hand at it. See the code at the end of this article.

For data scientists and machine learning researchers, the benefit of the job is to use their knowledge to do some interesting analysis of the World Cup. For example, as someone who studies deep learning, I use deep learning and OpenCV to get interesting results from game videos. Here are some of the results of my analysis of the Australia VS Peru video:

We can identify all the players and referees, the ball, and we can predict the team based on the color of the shirt, all in real time.

Step overview

The TensorFlow Object Detection API is a very powerful tool for quickly creating a target Detection model. If you’re not familiar with the API, check out the following three articles we shared about detecting objects with it in more detail:

How to use TensorFlow to track the Millennium Falcon in Star Wars?

How to write an App to identify Taylor Swift using TensorFlow and Swift?

How to identify Pikachu on an Android phone using TensorFlow?

How to create a custom model using the TensorFlow Object Detection API

The Object Detection API provides pre-trained Object Detection models trained from COCO datasets. The COCO dataset contains 90 common objects. Here are images of some of the objects in the COCO dataset:

In the case of our World Cup analysis, we focused on categories — people and football — that were included in the COCO dataset.

The API also supports a number of machine learning models, as shown below:

However, the speed and accuracy of these models are not compatible, so there is a trade-off. Since I’m interested in analyzing the game in real time, I chose SSDLite Mobilenet V2.

OpenCV, a powerful image processing library, can be used to identify players and predict their national teams using the TensorFlow Object Detection API. If you’re not familiar with OpenCV, check out the following two tutorials:

How to implement face recognition in less than 25 lines of Python code

OpenCV – Python tutorial

OpenCV allows us to identify a mask of a specific color, which can be used to identify red and yellow players. The following is an example of how OpenCV mask detects red color in an image:

Detail the main steps

Now let’s talk more about how to do it.

If you are using the TensorFlow Object Detection API for the first time, please download GitHub for this link

Install all environment dependencies using the instructions in this link.

If you haven’t already installed OpenCV, follow this tutorial to install it:

The main steps I took are as follows, and follow my Jupyter Notebook on GitHub:

  • Load the SSDLite Mobilenet model into the Graph and load the list of classes in the COCO dataset that contain characters and soccer balls.
  • Open the game video with cv2.VideoCapture (file name) and read the video frame one by one.
  • Target detection is performed for each frame using the loaded Graph.
  • The result returned from SSDLite is each identified class with its confidence and bounding box predictions. I cropped out all the people with an identification confidence greater than 0.6.
  • Now we have extracted every player. We need to read their shirt color to predict whether they belong to Australia or Peru. This step can be done by code block detection. First we define the red and blue color ranges and then create a mask for the colors with cv2.inRange and Cv. bitwise. To detect the team, I calculated how many red and yellow pixels to detect, as well as the ratio to the total number of pixels in the cropped image.
  • Finally, merge all the code blocks together and run them simultaneously, showing the results with Cv.2imshow.

conclusion

If you follow the steps above, you should eventually be able to detect who is playing at the World Cup and which team they belong to.

Lo and behold, we can create interesting things by simply combining deep learning with OpenCV. When you practice on your own, you can take it a step further and try other tricks:

  • As the camera follows Australia’s scoring area, you can work out how many Peruvians and how many Australians are playing in that area.
  • You can plot the footprints of each team, such as the areas where Peruvian players appear most frequently.
  • You can also plot the trajectories of the scoring players on both teams.

If you are interested in this year’s World Cup, you can get the Croatia VS France final trainer, welcome to share the results of the practice with us (send it to the community for a chance to get a gift card).

Attached project GitHub code address

0806 ai – From Scratch to Master – on sale for a limited time!

Click here for details

What about online programming?

(The first 25 students can also get ￥200 coupon)