Use OpenCV for real-time lane detection

The author | ABHISHEK SHARMA, compile | source of vitamin k | Analytics Vidhya

About a decade ago, when the guys at Google were still experimenting with a prototype, I envisioned my first self-driving car. I was immediately fascinated by the idea. Admittedly, I had to wait a while for these concepts to open up to the community, and now it looks like the wait was really worth it!

I recently experimented with some computer vision related self-driving car concepts, including lane detection. When you think about it, it’s actually the core concept for designing any autonomous car.

Here is the lane detection system we will build in this video: Youtu.be/SYhzBHT-SMW

Cool, right? In this tutorial, I will use the OpenCV library for lane detection and self-driving cars. Of course, in this tutorial, we’ll also introduce Python code.

Understand the concept of lane detection

So what is lane detection? Here’s how Baidu Baike defines a lane:

Lane, also known as lane, roadway, is used for vehicles on the road. They are set up on both highways and motorways, which have legal rules for lane use, such as carriageways and overtaking lanes.

Defining it is important because it allows us to continue with the concept of lane detection. We can’t have any ambiguity in building a system.

As I mentioned earlier, lane detection is a key component of self-driving cars and autonomous vehicles. This is one of the important research topics of driving scene understanding. Once the lane position is obtained, the vehicle knows where to go and avoids hitting other lanes or leaving the road. This prevents the driver/vehicle system from deviating from the lane.

Here are some random road images (line 1) and the lanes detected (line 2) :

Understand the problem statement

The task we want to perform is to detect lanes in the video in real time. Lane detection can be done in a number of ways. We can use learning-based approaches, such as training deep learning models on annotated video datasets, or using pre-trained models.

However, there are simpler ways to perform lane detection. In this article, I’ll show you how to do this without using any in-depth learning models. But we will use the popular OpenCV library in Python.

Here’s a frame of the video we’ll be working on:

As we can see in this picture, we have four lanes separated by white lane markers. So, to detect lanes, we have to detect white markings on both sides of lanes. This raises a key question — how do we detect lane markings?

In addition to lane markings, there are many other objects in the scene. There are vehicles, roadside guardrails and street lamps on the road. In the video, every frame will have scene changes. This is a good reflection of real driving.

Therefore, before we can solve the problem of lane detection, we must find a way to ignore unwanted objects in driving scenarios.

One thing we can do now is narrow down our areas of interest. Instead of using the entire frame, use only a portion of the frame. In the image below, everything is hidden except the markings of the lanes. As the vehicle moves, lane markers will more or less fall within the area:

In the next section, I’ll show you how to edit a video frame to select a specific region. You will also learn some of the necessary image preprocessing operations.

What is a Frame Mask?

In this case, the frame mask is just a NumPy array. When we want to apply a mask to an image, we simply change the pixel value of the desired region of the image to 0, 255, or any other number. An example of image masking is given below. A region of the image has its pixel value set to 0:

This is a very simple but effective way to remove unwanted areas and objects from an image.

Image preprocessing for lane detection

We will first apply masks to all frames in the input video. Then, we apply image thresholding and Hough line transform to detect lane marking.

Image thresholding

In this method, the pixel value of a grayscale image is specified according to a threshold as one of two values representing black and white colors. Thus, if a pixel’s value is greater than a threshold, it is assigned one value, otherwise it is assigned another value.

As shown above, after applying the threshold value to the Mongolian map image, we only get the lane markers in the output image. These markers can now be easily detected by hough line transformation.

Hough line transformation

Hough line transformation is a method of detecting any shape that can be represented mathematically.

For example, it can detect shapes such as rectangles, circles, triangles or lines. We are interested in detecting lane markings that can be represented as straight lines.

This is the related documents: opencv – python – tutroals. Readthedocs. IO/en/latest/p…

Applying hough line transform to the image after performing image thresholding will provide the following output:

We need to do this for all frames and then stitch the resulting frames into the new video.

Use OpenCV to implement lane detection in Python

It’s time to implement this lane detection project in Python! I recommend Using Google Colab because it takes computing power to build a lane detection system.

First import the required libraries:

import os
import re
import cv2
import numpy as np
from tqdm import tqdm_notebook
import matplotlib.pyplot as pltCopy the code

Read video frame

I’ve pulled some clips from this YouTube video. You can download it from this link: drive.google.com/file/d/1e4c…

Col_frames = os.listdir('frames/') col_frames. Sort (key=lambda f: Int (re.sub('\D', ', f)) # col_images=[] for I in tqdm_notebook(col_frames): img = cv2.imread('frames/'+i) col_images.append(img)Copy the code

Let’s draw a frame:

# plot frame plt.figure(figsize=(10,10)) plt.imshow(col_images[idx][:,:,0], cmap= "gray") plt.show() # plot frame plt.figure(figsize=(10,10)) plt.imshow(col_images[idx][:,:,0], cmap= "gray") plt.show()Copy the code

Frame mask created

The region of interest is a polygon. We’re trying to cover up everything except this area. Therefore, we must first specify the coordinates of the polygon and then use it to prepare the frame mask:

Polygon = np.array([[50,270], [220,160], [360,160], [480,270]] # fillConvexPoly(stencil, polygon, 1)Copy the code

Figure (figsize=(10,10)) plt.imshow(stencil, cmap= "gray") plt.show()Copy the code

# apply the polygon as a mask. Img = cv2 bitwise_and (col_images [independence idx] [0] :, :,, col_images [independence idx] [0] :, :,, Mask =stencil) # plot masked frame plt.figure(figsize=(10,10)) plt.imshow(img, cmap= "gray") plt.show()Copy the code

Image preprocessing

We have to perform some image preprocessing on the video frames to detect the desired lanes. Pre-processing operations include:

Image thresholding
Hough line transformation

1. Image thresholding

Thresh = cv2.threshold(img, 130, 145, Plt.figure (figsize=(10,10)) plt.imshow(thresh, cmap= "gray") plt.show()Copy the code

2. Hough line transformation

lines = cv2.HoughLinesP(thresh, 1, np.pi/180, 30, Dmy = col_images[idx][:,:,0]. Copy () # hof for line in lines: x1, y1, x2, y2 = line[0] cv2.line(dmy, (x1, y1), (x2, y2), (255, 0, 0), Figure (figsize=(10,10)) plt.imshow(dmy, cmap= "gray") plt.show()Copy the code

Now we will apply all of these operations to each frame. We also save the resulting frame in a new directory:

cnt = 0 for img in tqdm_notebook(col_images): Bitwise_and (img[:,:,0], img[:,:,0], mask=stencil), thresh = cv2.threshold(masked, HoughLinesP(ThRESH, 1, Np.pi /180, 30, MaxLineGap =200) dmy = img.copy() # try: for line in lines: x1, y1, x2, y2 = line[0] cv2.line(dmy, (x1, y1), (x2, y2), (255, 0, 0), 3) cv2.imwrite('detected/'+str(cnt)+'.png',dmy) except TypeError: cv2.imwrite('detected/'+str(cnt)+'.png',img) cnt+= 1Copy the code

Video to prepare

# pathIn= 'detected/' # Output video path pathOut = 'roads_v2.mp4' # Video frames per second FPS = 30.0Copy the code

File = [f for f in os.listdir(pathIn) if isfile(join(pathIn, f))] files.sort(key=lambda f: int(re.sub('\D', '', f)))Copy the code

Next, we will put all the detected frames on the lane into a list:

frame_list = [] for i in tqdm_notebook(range(len(files))): Img = cv2.imread(filename) height, width, Shape size = (width,height) # Insert frame into image array frame_list.append(img)Copy the code

Finally, we can now combine frames into video using the following code:

Out = cv2.VideoWriter(pathOut,cv2.VideoWriter_fourcc(*'DIVX'), FPS, size) for I in range(len(frame_array)): out.write(frame_array[i]) out.release()Copy the code

This completes the lane detection system in Python.

At the end

In this tutorial, we introduce a simple lane detection technique. We didn’t use any models or complex image features. Instead, our solution is entirely based on certain image preprocessing operations.

However, in many cases, this solution does not work. For example, the system will fail when there are no lane markings or too many cars on the road. There are more sophisticated ways to overcome these problems in lane detection.

The original link: www.analyticsvidhya.com/blog/2020/0…

Welcome to panchuangai blog: panchuang.net/

Sklearn123.com/

Welcome to docs.panchuang.net/