This section describes how to collect videos in Windows

There are two main ways to capture video on Windows: Media Foundation and DirectShow.

Meida Foundation is a new set of multimedia SDK for Windows launched after Vista. It is simple and convenient, and matured from Windows 7.

The other is DirectShow, which is mainly used to capture video before Windows 7. Writing code with DirectShow is a bit of a hassle, mainly because Windows engineers designed the DirectsShow development interface in a logical circuit way of thinking, introducing concepts such as filter and PIN. These are old things that few people can understand these days, unless you are from that era, haha.

This also explains why few people learn to develop Windows programs these days because they can’t keep up with The Times. You see others Android/iOS do video collection how easy, you whole such trouble, who is willing to learn!

Some concepts of the Media Foundation

DirectShow program we will leave to later analysis, today we mainly talk about how to do video collection MediaFoundation.

Before we get there, we need to fill in some basic concepts. You can find these concepts in the Media Foundation Programming Guide. The following text is basically a translation of Windows official documentation.


The overall structure of MF(MediaFoundation) is as follows:






The Primitives and Platfrom

Primitives at the bottom are some auxiliary apis:

  • Attributes: a Map consisting of keys and values.
  • Media Type: format of Media data flow.
  • Media Buffers: Stores a period of Media data.
  • Media Samples: Container for Media Buffers, equivalent to a Buffter List.

The MF Platform provides apis for some of the core functions. Examples are asynchronous calls, work queues.

Media Pipeline

Media Pipeline includes three types of objects: Media Sources, MFTs(Media Foundation Transfors), and Media Sink.

  • Media Sources: Bring data into the pipeline. Data can come from local files, network streams, or both hardware devices.
  • MFTs: Processes stream data. Codecs are implemented in MFTs.
  • Media Sink: Consume data. Display video to display screen, play sound or write data to media file.

Media Session controls data flow through pipes. Such as quality control, audio/video synchronization, format changes.

Source Reader and Sink Writer

Source Readers and Sink Writers provide another way to use Media Foundation (as opposed to Media Sources, transforms, and Media sinks).

  • The Source Reader controls the media Source and multiple decoders.
  • Sink Writer Controls media sinks and multiple encoders.

You can use the Source Reader to get compressed or uncompressed data from the media Source and use the Sinker Writer to encode the data and send it to the Media Sink.

Here’s how MF collects video data.

Video data collection

Through the above introduction, we can basically know that MF uses the architecture of data collection from source, encoding and decoding, and output rendering to process multimedia. This method is easy to understand and very convenient to use.

Basic steps of MF video collection

  • Initialize the COM component.
  • Gets the video device list.
  • Activate a video device and obtain the Media Source of the device.
  • Create a Source Reader based on the request command and Media Source.
  • Set the Media Type for the Source Reader.
  • Read video data in Media Type format from the device using the Source Reader.

The above are the basic steps for MF to collect data from video equipment, and we will introduce each step in detail below.

A detailed analysis

Since the code for each step is really simple, I won’t go into too much text description here, but you can easily understand each step through the following code and its comments.


Initialize the COM component and start the MF



. CoInitializeEx(NULL, COINIT_APARTMENTTHREAD | COINIT_DISABLE_OLEDDE) MFStartup(MF_VERSION) ...Copy the code



Get all the video equipment



IMFAttributes *videoCmd = NULL; IMFActivate **videoDevices = NULL; UINT32 videoDeviceCount = 0; . // set the command to get the video device MFCreateAttributes(videoCmd, 1/* indicates only one */); videoCmd->setGUID( MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE, //key MF_DEVSOURCE_ATTRIBUTE_SOURCE_TYPE_VIDCAP_GUID); MFEnumDeviceSources(videoCmd, &videoDevices, // Here is the device list &videoDeviceCount); // This is the number of devices...Copy the code



Activate a video device



IMFMediaSource *mediaSource = NULL; . VideoDevices [0]->ActivateObject(IID_PPV_ARGS(&mediaSource)); // Activate the first video device and generate a logical Media Source for the setup. .Copy the code



Create the Source Reader



IMFSourceReader *soureReader = NULL; . // The source reader can be obtained from the media source and the request command. (the second development model) MFCreateSourceReaderFromMediaSource (mediaSource videoCmd, & sourceReader); .Copy the code



Set the Media Type



IMFMediaType *mediaType = NULL; . MFCreateMediaType(&mediaType); // Set media to video mediaType->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video); // Set media to video mediaType->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video); //YUV format is I420 mediaType->SetGUID(MF_MT_SUBTYPE, WMMEDIASUBTYPE_I420); // Each video frame is 640 * 480 MFSetAttributeSize(mediaType, MF_MT_FRAME_SIZE, 640, 480); sourceReader->SetCurrentMediaType(MF_SOURCE_READER_FIRST_VIDEO_STREAM, NULL, mediaType);Copy the code



Read the data



IMFSample *sample = NULL; DWORD index, flags; LONGLONG llVideoTs; . While (runing){sourceReader->ReadSample(MF_SOURCE_READER_FIRST_VIDEO_STREAM, 0, &index, // staUS flags &llVideoTs, // timestamp &sample); // Store the collected video data}Copy the code


By following the simple steps above, you can easily retrieve the video data from the video device. MF is so much simpler than DirectShow.

The above introduction is to use the synchronous way to collect video data using MF. MF also provides a more efficient asynchronous way to obtain video data. Interested friends can learn its asynchronous way based on this article.

summary

Today, I introduced how to collect video using MF under Windows. You can do this in the following 6 steps:

  • Initialize the COM component.
  • Gets the video device list.
  • Activate a video device and obtain the Media Source of the device.
  • Create a Source Reader based on the request command and Media Source.
  • Set the Media Type for the Source Reader.
  • Read video data in Media Type format from the device using the Source Reader.

In addition, MF acquisition scheme is only applicable to Win7 after the system, for the previous system or to use DirectShow scheme. I will also show you how to use DirectShow to capture video later.