IOS & OpenGL & OpenGL ES & Metal

I. Introduction to Metal

1, the Metal

Back in 2014 at WWDC, Apple introduced Metal, a new platform for game developers that can provide 3D graphics with 10x better rendering performance and support for familiar game engines and companies.

Prior to 2018,OpenGL ES could only use gpus for graphics processing and could not schedule Gpus for highly customized concurrent operations in projects. But in Metal, Apple has given such a portal that you can fully invoke the GPU to do this.

In 2018, apple made the decision to migrate the CoreAnimation kernel from OpenGL ES to Metal. At this point, we can use Metal to process business on iOS devices and maximize the performance of its GPU.

In other words, when we need to do high concurrency, we also use Metal to do it. This is because Metal has a portal to call the GPU to perform calculations. (For example, AVFoundation’s face recognition function, audio and video encoding (compression) and decoding (decompression) all require GPU to achieve the best effect of high concurrency)

Note: Most Metal programs do not support emulator execution and need to be executed on a real machine. Therefore, there is also a requirement for real phones: processors above A7, which means 6s and later phones.

2, the characteristics of

  1. CPU overhead is very low
  2. Get the best out of the GPU (mostly apple is more comfortable calling their own hardware)
  3. Maximize CPU/GPU concurrency
  4. Manage our resources more effectively

3. Graphics pipeline

It’s basically the same as OpenGL ES, except Shader has changed its name to Processing.

CPU: Processing vertex data, passing it to vertex program (shader)

GPU part:

  1. Vertex shaders process vertex data from the CPU and perform a series of coordinate transformations and clipping ↓
  2. Assemble primitives ↓
  3. Rasterizer left
  4. Chip program (shader) to handle texture, transparency, depth, etc. ↓
  5. The final data is stored in the frame buffer and displayed on the screen

Note that there are 9 ways to connect primions in OpenGL ES: point, line, line segment, line ring, quadrilateral, quadrilateral band, triangle, triangle band, triangle fan

Metal has only five types: point, line segment, line ring, triangle, and triangle fan

4. Suggestions on using Metal

  1. Apple doesn’t want Rendering to be done in VC, they want us to Separate Your Rendering loops for Metal into a Separate class

  2. That is, we will implement two delegate methods following the MTKViewDelegate protocol in a separate rendering class

  3. MTLCommandQueue = MTLCommandQueue = MTLCommandQueue = MTLCommandQueue = MTLCommandQueue = MTLCommandQueue = MTLCommandQueue = MTLCommandQueue = MTLCommandQueue = MTLCommandQueue = MTLCommandQueue = MTLCommandQueue = MTLCommandQueue = MTLCommandQueue = MTLCommandQueue = MTLCommandQueue

5, The relationship between Metal command objects

  1. The command buffer is created from the command queue
  2. Command encoders encode commands into the command cache
  3. Commit the command cache and send it to the GPU
  4. The GPU executes the command and renders the result as drawable

2. Metal API

1, MTKView

Official document MTKView

Similar to GLKView provided in GLKit. Metal provides us with MTKView, inherited from UIView, which handles the details of Metal drawing and displaying to the screen.

MTKView *view = [[MTKView alloc] init];
Copy the code

2, MTLDevice

The MTLDevice object indicates the GPU that can execute commands. The MTLDevice protocol has:

Methods for creating new command queues, allocating buffers from memory, creating textures, and querying device functionality.

To get on the system of choice for system equipment, please call MTLCreateSystemDefaultDevice function.

The Device Object Represents The GPU.

A MTLDevice object with a GPU, means that we can call the method usually MTLCreateSystemDefaultDevice () to obtain a single object represents the default GPU. In fact, it is equivalent to obtaining a permission to operate the GPU.

Note: MTKView must be set to MTLDevice.

/ / create a default device the device = MTLCreateSystemDefaultDevice (); // If (! view.device) { NSLog(@"Metal is not supported on this device"); return; }Copy the code

3, MTLCommandQueue

Official Document Command Queue

If the MTLDevice is created successfully, after the GPU is acquired, a rendering queue MTLCommandQueue is required. This queue is the first object that interacts with the GPU. The MTLCommandQueue stores the command MTLCommandBuffer that will be rendered.

Each commandQueue has a long lifetime, so commandqueues can be reused rather than being created and destroyed frequently.

// Create an MTLCommandQueue ID <MTLCommandQueue> commandQueue = [view.device newCommandQueue]; // Create an MTLCommandQueue ID <MTLCommandQueue> commandQueue = [view.device newCommandQueue];Copy the code

4, MTLCommandBuffer

Official document command Buffer

A command buffer is primarily a command used to store encoding, and its life cycle is until the buffer is committed to the GPU for execution. A single command buffer can contain different encoding commands, depending on the type and number of encoders used to build it.

// Create an MTLCommandBuffer id<MTLCommandBuffer> commandBuffer = [commandQueue commandBuffer]; // Give commandBuffer a name commandBuffer.label = @"MyCommand";Copy the code

The MTLCommandBuffer object is submitted to the MTLCommandQueue object. It can only be executed after submission, by enlisting order. There are two ways to do this:

  • Enqueue: Execution in sequence
  • Commit: Execute queue as soon as possible. If there is a commit ahead, you still need to wait in line

5, MTLRenderCommandEncoder

Official document Command Encoder

Command encoders represent render states and render commands associated with a single render process and have the following functions:

  • Specify graphic Resources, such as cache and texture objects, which contain vertex, slice, and texture picture data
  • Specify an MTLRenderPipelineState objectRepresents the compiled render state, including compile-link conditions for vertex shaders and fragment shaders
  • Specify fixed functions, including viewport, triangle fill mode, scissor rectangle, depth, template test, and other values
  • Draw 3D primions

MTLRenderCommandEncoder is created, and the rendering descriptor MTLRenderPassDescriptor is needed

/ / 1. From the view map, for rendering descriptor MTLRenderPassDescriptor * renderPassDescriptor = the currentRenderPassDescriptor; //2. Determine whether the renderPassDescriptor is successfully created; otherwise, skip any rendering. Rendercommandencoder <MTLRenderCommandEncoder> renderEncoder = [commandBuffer) {//3 renderCommandEncoderWithDescriptor:renderPassDescriptor]; RenderEncoder. Label = @"MyRenderEncoder"; //5. Draw some Metal files //... //6. End work [renderEncoder endEncoding]; }Copy the code

6, MTKViewDelegate

// Set MTKView's delegate method (CustomRender is used to implement MTKView's delegate method) view.delegate = render; / / view can be set according to the view attributes on frame rate (designated time to call drawInMTKView method - the view to render call) is the screen refresh every 60 frames the preferredFramesPerSecond = 60; - (void)drawInMTKView:(nonnull MTKView *)view; - (void) MTKView :(nonnull MTKView *)view drawableSizeWillChange:(CGSize)size;Copy the code

Three, small cases

Here is a small example – render random background color to show what Metal syntax looks like in action.

First, we followed Apple’s advice and created a separate render class, CustomRender

1, CustomRender. H

#import <Foundation/Foundation.h>
#import <MetalKit/MetalKit.h>

NS_ASSUME_NONNULL_BEGIN

@interface CustomRender : NSObject<MTKViewDelegate>

-(id)initWithMetalKitView:(MTKView *)mtkView;

@end

Copy the code

2, CustomRender. M

#import "CustomRender.h" @implementation CustomRender { id<MTLDevice> _device; id<MTLCommandQueue> _commandQueue; } // typedef struct {float red, green, blue, alpha; } Color; // init method - (id)initWithMetalKitView:(MTKView *) MTKView {self = [super init]; If (self) {// Get the device passed in. _device = mtkView.device; _commandQueue _commandQueue = [_device newCommandQueue]; } return self; } // Set the Color - (Color)makeFancyColor { Static BOOL growing = YES; static BOOL growing = YES; //2. Color channel value (0~3) static NSUInteger primaryChannel = 0; Static float colorChannels[] = {1.0, 0.0, 0.0, 1.0}; //4. Const float DynamicColorRate = 0.015; NSUInteger dynamicChannelIndex = (primaryChannel+1)%3; // if(growing) {// dynamicChannelIndex = (primaryChannel+1)%3; 0.015 colorChannels[dynamicChannelIndex] += DynamicColorRate; If (colorChannels[dynamicChannelIndex] >= 1.0) {// Set NO to NO; // Change the channel to a dynamic channel primaryChannel = dynamicChannelIndex; NSUInteger dynamicChannelIndex = (primaryChannel+2)%3; ColorChannels [dynamicChannelIndex] -= dynamicColorChannel; If (colorChannels[dynamicChannelIndex] <= 0.0) {if(colorChannels[dynamicChannelIndex] <= 0.0) { }} // create Color Color Color; // Change the color RGBA value color.red = colorChannels[0]; color.green = colorChannels[1]; color.blue = colorChannels[2]; color.alpha = colorChannels[3]; // return color; } //MTKViewDelegate delegate method -(void)drawInMTKView (MTKView *)view{//1. Color = [self makeFancyColor]; ClearColor = MTLClearColorMake(color.red, color.green, color.blue, color.alpha); Id <MTLCommandBuffer> commandBuffer = [_commandQueue commandBuffer]; / / 2, rendering the descriptor MTLRenderPassDescriptor * renderPassDescriptor = the currentRenderPassDescriptor; RenderPassDescriptor = renderPassDescriptor! RenderEncoder <MTLRenderCommandEncoder> renderEncoder = [commandBuffer renderCommandEncoderWithDescriptor:renderPassDescriptor]; RenderEncoder. Label = @"myEncoder"; RenderEncoder endEncoding [renderEncoder endEncoding]; /* When the encoder finishes, the command cache will receive two commands: 1) present 2) commit Because the GPU does not draw directly to the screen, you do not give instructions. There will not be any content rendering to the screen. * / / / 8. Add a command to display the last clear can map screen [commandBuffer presentDrawable: the currentDrawable]; } //9, finish rendering here and commit commandBuffer to GPU [commandBuffer commit]; } - (void)mtkView:(MTKView *)view drawableSizeWillChange:(CGSize)size { } @endCopy the code

3, ViewController. M

#import "ViewController.h" #import "CustomRender.h" @interface ViewController () { MTKView *_view; CustomRender *_render; } @end @implementation ViewController - (void)viewDidLoad { [super viewDidLoad]; / / 1, _view = [[MTKView alloc] initWithFrame: CGRectMake (0, 0, the self. The frame. The size, width, the self. The frame. The size, height)); UILabel *label = [[UILabel alloc]initWithFrame:CGRectMake(0, 200, self.view.frame.size.width, 200)]; label.text = @"Hello"; label.font = [UIFont fontWithName:@"AmericanTypewriter" size:50]; label.textAlignment = NSTextAlignmentCenter; [_view addSubview:label]; [self.view addSubview:_view]; / / 2, create a default device _view. The device = MTLCreateSystemDefaultDevice (); // if (! _view.device) { NSLog(@"Metal is not supported on this device"); return; } _render = [[CustomRender alloc]initWithMetalKitView:_view]; // if (! _render) { NSLog(@"Renderer failed initialization"); return; } //6, set the MTKView delegate (CustomRender) _view.delegate = _render; / / 7, set the frame rate, that is, how many frames _view. Call a rendering agent method preferredFramesPerSecond = 60; }Copy the code