directory

  • directory
  • Application platform
  • Screen recording
  • Calculate the optimal FPS for video and use NUMpy to calculate the intermediate frame array
  • Listen for keyboard keys using PyNput
  • How to save video in MP4 format
  • The source code
  • conclusion

Recently, I was using the screen recording software to record the desktop, and in the process of using it, I suddenly thought that python could be used as a screen recording tool, and also exercise my hands-on ability. Next in a series of articles on how to make a screen recording tool in Python:

  • Record screen to make video
  • The audio recording
  • Synthesizing video, audio
  • Create visual window based on PyQt5

I hope I can improve the above four parts as soon as possible. Next, I will use Python to make the screen recording part.

Application platform

  • windows 10
  • Python 3.7

Screen recording

Screen recording can be simply understood as playing screen snapshots in the form of ANIMATED images. Here, I choose ImageGrab under PIL to capture screen images. First, after PIP Install Pillow, I need to combine the captured snapshots into videos. The ImageGrab class can not be stored as video directly. Numpy module is used for number composition, and cV2.COLOR_BGR2RGB is converted to CV2 color channel. pip install numpy

Screen recording main code:

import numpy as np from PIL import ImageGrab import cv2 im = ImageGrab.grab() width, Fourcc = cv2.VideoWriter_fourcc(*'I420') # set the video encoding format FPS = 15 # Set the frame rate video = cv2.VideoWriter('test.avi', fourcc, fps, (width, high)) while True: Im_cv = cv2.cvtColor(Np.array (im), cv2.color_bgr2RGB) # image.write (im_cv) if xx: Break video.release() # Release cache, persist videoCopy the code

A test run can save screenshots as videos, but it’s not elegant or conducive to subsequent operations.

Encapsulate into class, inherit thread parent class, convenient to use the keyboard to control the end of video recording.

From threading import Thread class ScreenshotVideo(Thread): def __init__(self): """ "" super().__init__().Copy the code

The detailed code will be given at the end of this article.

Calculate the optimal FPS for video and use NUMpy to calculate the intermediate frame array

In practice, different computers may have different frame rates for video recording, resulting in fast or slow video playback. Therefore, you need to calculate the optimal FPS value based on different computers.

def video_best_fps(self, path): Video = v2.videocapture (path) # Read video FPS = video.get(cv2.cap_prop_fps) # get the current video frame rate count = Video.get (cv2.cap_prop_frame_count) # Self.best_fps = int(FPS * (int(count)/FPS)/self.spend_time))Copy the code

Adjusting the frame rate parameter to record the video attenuates whether the video is too fast or too slow. You can also increase the number of frames to prolong the playback time, here I use a very simple method to increase video frames, just for reference.

@jit(nopython=True) def average_n(x, y): "" "Numpy calculation approach value "" "return ((x + y + y) / / 3). The astype (x.d type)Copy the code

This method only applies to the case that the FPS is higher than the optimal FPS, and the video after processing is still short, but the number of detailed frames increases, so the playing time will be longer than that before processing, with slightly residual image.

Listen for keyboard keys using PyNput

In video recording, we do not know when the video ends, so we wrap the recording code with a while loop, but it is impossible to make the code run endlessly, so we interrupt the recording code by listening to the keyboard module.

from pynput import keyboard # pip install pynput def hotkey(self): Join () def on_press(self, key): try: self.join () def on_press(self, key): try: self.join () def on_press(self, key): try: If key.char == 't': # save the video self.flag = True elif key.char == 'k': Self. flag = True self.kill = True except Exception as e: print(e)Copy the code

Press the “T” key to end recording and save the video. The “K” key stops recording and deletes cached files.

How to save video in MP4 format

The video encoding format should be (‘ A ‘, ‘V ‘,’ C ‘, ‘1’), and the file suffix should be ‘.mp4’. Before recording, download the dL. bz2 file of the corresponding platform, decompress the compressed package and put it in the project folder. ! OpenH264 Video Codec provided by Cisco Systems, Inc.

The source code

The source code of this article is as follows:

import time from PIL import ImageGrab import cv2 from pathlib import Path import numpy as np from numba import jit from pynput import keyboard from threading import Thread @jit(nopython=True) def average_n(x, y): """ return ((x + y + y) // 3). Astype (x.dtype) class ScreenshotVideo(Thread): def __init__(self, width, high, path='', fps=15): Super ().__init__() self.save_file = path self.best_fps = FPS self.fps = FPS self.width = width self.high = high self.spend_time = 1 self.flag = False self.kill = False self.video = None def __call__(self, path): """ Reload video path, "" self.save_file = Path(Path) self.video = self.init_videowriter(self.save_file) @staticMethod def Screenshot (): """ staticmethod, screenshot, and transform to np.array array """ return np.array(imagegrab.grab ()) @staticmethod def get_fourcc(name): "" "" video coding dictionary "" fourcc_maps = {'. Avi ':' I420 ', 'm4v' : 'mp4v', '. Mp4 ':' avc1 ', 'ogv' : 'THEO', 'FLV' : 'FLV1',} return fourcc_maps.get(name) def init_videowriter(self, path): """ Raise Exception(' Video path is not set, \ nVideo = ScreenshotVideo(FPS,width,high)\ nVideo = video(video_path)') path = path (path) if isinstance(path, str) else path fourcc = cv2.VideoWriter_fourcc(*self.get_fourcc(path.suffix)) return cv2.VideoWriter(path.as_posix(), fourcc, self.fps, (self.width, self.high)) def video_record_doing(self, img): Im_cv = cv2.cvtcolor (img, cv2.color_bgr2rgb) self.video.write(im_cv) def video_record_end(self): """ self.video.release() cv2.destroyallWindows () if self.save_file and self.kill: Path(self.save_file).unlink() def video_best_fps(self, path): Video = cv2.videocapture (path) FPS = video.get(cv2.cap_prop_fps) count = video.get(cv2.CAP_PROP_FRAME_COUNT) self.best_fps = int(fps * ((int(count) / fps) / self.spend_time)) video.release() def pre_video_record(self): Self.video = self.init_videowriter('test.mp4') start_time = time.time() for _ in range(10): self.video = self.init_videowriter('test.mp4') start_time = time.time() for _ in range(10): im = self.screenshot() self.video_record_doing(im) self.spend_time = round(time.time() - start_time, 4) self.video_record_end() time.sleep(2) self.video_best_fps('test.mp4') Path('test.mp4').unlink() def Insert_frame_array (self, frame_list): """ "fps_n = round(self.fps/self.best_fps) if fps_n <= 0: Return frame_list times = int(np.log2(fps_n)) # multiplier for _ in range(times): frame_list2 = map(average_n, [frame_list[0]] + frame_list[:-1], frame_list) frame_list = [[x, y] for x, y in zip(frame_list2, frame_list)] frame_list = [j for i in frame_list for j in i] return frame_list def frame2video_run(self): Self.video = self.init_videoWriter (self.save_file) start_time = time.time() frame_list = [] while True: frame_list.append(self.screenshot()) if self.flag: break self.spend_time = round(time.time() - start_time, 4) if not self.kill: Frame_list = self.insert_frame_array(frame_list) for im in frame_list: self.video_record_doing(im) self.video_record_end() def hotkey(self): Join () def on_press(self, key): try: self.join () def on_press(self, key): try: self.join () def on_press(self, key): try: If key.char == 't': # save the video self.flag = True elif key.char == 'k': Self. flag = True self.kill = True except Exception as e: print(e) def run(self): Thread(target=self.hotkey, Self.frame2video_run () screen = imagegrab.grab () width, high = screen.size video = ScreenshotVideo(width, high, FPS =60) video.pre_video_record() # Video.run ()Copy the code

conclusion

At present, this paper uses OpencV and related modules to record the screen and convert it to video for saving. It also learns to encapsulate multiple functions into classes for the convenience of subsequent function development. The road to learning is endless, take bold steps!

Friends, quickly use the practice! If you have any problems during the learning process, please follow me and I will invite you to join the Python Learning Exchange group to discuss and learn together.