OpenGL ES case – Implement split screen filter

Split screen filter is very common in short video now, today we will use OpenGL ES to achieve the effect of split screen filter.

1. Realization results

Two, split screen principle:

Since it is a filter effect, it must be implemented in the shader, the split-screen effect only needs to modify the source shader code, the source shader code does not need to change.

1. Vertex shader and source shader codes in normal mode:

• Vertex shader

attribute vec4 Position;
attribute vec2 TextureCoords;
varying vec2 TextureCoordsVarying;

void main (void) {
    TextureCoordsVarying = TextureCoords;
    gl_Position = Position;
    
}
Copy the code

• Source shader:

precision highp float; uniform sampler2D Texture; varying vec2 TextureCoordsVarying; void main (void) { vec4 mask = texture2D(Texture, TextureCoordsVarying); Gl_FragColor = vec4 (mask. RGB, 1.0); }Copy the code

2. Binary screen

A bisection screen is essentially taking a middle part of an image and showing it up and down again:Source shader code:

precision highp float; uniform sampler2D Texture; varying highp vec2 TextureCoordsVarying; void main() { vec2 uv = TextureCoordsVarying.xy; float y; If (uv) > = 0.0 && y uv) < = 0.5) {y y = uv. Y + 0.25; } else {y = uv. y-0.25; } gl_FragColor = texture2D(Texture, vec2(uv.x, y)); }Copy the code

3. Three-split screen

3 split screen is similar to 2 split frequency, capturing only the middle 1/3 of the image:Source shader code:

precision highp float; uniform sampler2D Texture; varying highp vec2 TextureCoordsVarying; void main() { vec2 uv = TextureCoordsVarying.xy; If (uv.y < 1.0/3.0) {uv.y = uv.y + 1.0/3.0; } else if (uv) y > 2.0/3.0) {uv. Y = uv. Y - 1.0/3.0. } gl_FragColor = texture2D(Texture, uv); }Copy the code

4. Quad screen

The quad-split screen is different from the previous two. Because the picture in our case is square, the position can be directly adjusted without interception. In actual development, it can be flexibly configured according to requirements.Source shader code:

precision highp float; uniform sampler2D Texture; varying highp vec2 TextureCoordsVarying; void main() { vec2 uv = TextureCoordsVarying.xy; If (uv.x <= 0.5){uv.x = uv.x * 2.0; }else{uv.x = (uv.x -0.5) * 2.0; } if (uv.y<= 0.5) {uv.y = uv.y * 2.0; }else{uv.y = (uV. y-0.5) * 2.0; } gl_FragColor = texture2D(Texture, uv); }Copy the code

5. Six-split screen

Sextet, also part of a screenshotSource shader code:

precision highp float; uniform sampler2D Texture; varying highp vec2 TextureCoordsVarying; void main() { vec2 uv = TextureCoordsVarying.xy; If (uv.x <= 1.0/3.0){uv.x = uv.x + 1.0/3.0; } else if (uv) x > = 2.0/3.0) {uv. X = uv. X - 1.0/3.0. } if(uv.y <= 0.5){uv.y = uv.y + 0.25; }else {uv.y = uv.y -0.25; } gl_FragColor = texture2D(Texture, uv); }Copy the code

6. Nine-point screen

Nine split screen is similar to four split screen, no interception is requiredSource shader code:

precision highp float; uniform sampler2D Texture; varying highp vec2 TextureCoordsVarying; void main() { vec2 uv = TextureCoordsVarying.xy; If (uv.x < 1.0/3.0) {uv.x = uv.x * 3.0; } else if (uv) x < 2.0/3.0) {uv. X = (uv) x - 1.0/3.0) * 3.0; } else {uv.x = (uv.x-2.0/3.0) * 3.0; } if (uv.y <= 1.0/3.0) {uv.y = uv.y * 3.0; } else if (uv) y < 2.0/3.0) {uv. Y = (uv) y - 1.0/3.0) * 3.0; } else {uv.y = (uv.y-2.0/3.0) * 3.0; } gl_FragColor = texture2D(Texture, uv); }Copy the code

Three, the code part

1. Project Structure:

2, ViewController

#import "ViewController.h"
#import <GLKit/GLKit.h>
#import "FilterBar.h"

typedef struct {
    GLKVector3 positionCoord; // (X, Y, Z)
    GLKVector2 textureCoord; // (U, V)
} SenceVertex;
@interface ViewController ()<FilterBarDelegate>
@property (nonatomic, assign) SenceVertex *vertices;
@property (nonatomic, strong) EAGLContext *context;
// 用于刷新屏幕
@property (nonatomic, strong) CADisplayLink *displayLink;
// 开始的时间戳
@property (nonatomic, assign) NSTimeInterval startTimeInterval;
// 着色器程序
@property (nonatomic, assign) GLuint program;
// 顶点缓存
@property (nonatomic, assign) GLuint vertexBuffer;
// 纹理 ID
@property (nonatomic, assign) GLuint textureID;
@end

@implementation ViewController

- (void)viewDidLoad {
    [super viewDidLoad];
    //设置背景颜色
    self.view.backgroundColor = [UIColor blackColor];
       //创建滤镜工具栏
    [self setupFilterBar];
       //滤镜处理初始化
    [self filterInit];
       //开始一个滤镜动画
    [self startFilerAnimation];
}
- (void)viewWillDisappear:(BOOL)animated{
    [super viewWillDisappear:animated];
    // 移除 displayLink
    if (self.displayLink) {
        [self.displayLink invalidate];
        self.displayLink = nil;
    }
}
- (void)dealloc {
    //1.上下文释放
    if ([EAGLContext currentContext] == self.context) {
        [EAGLContext setCurrentContext:nil];
    }
    //顶点缓存区释放
    if (_vertexBuffer) {
        glDeleteBuffers(1, &_vertexBuffer);
        _vertexBuffer = 0;
    }
    //顶点数组释放
    if (_vertices) {
        free(_vertices);
        _vertices = nil;
    }
}
// 创建滤镜栏
- (void)setupFilterBar{
    CGFloat filterBarWidth = [UIScreen mainScreen].bounds.size.width;
       CGFloat filterBarHeight = 60;
       CGFloat filterBarY = [UIScreen mainScreen].bounds.size.height - filterBarHeight - 50;
       FilterBar *filerBar = [[FilterBar alloc] initWithFrame:CGRectMake(0, filterBarY, filterBarWidth, filterBarHeight)];
       filerBar.delegate = self;
       [self.view addSubview:filerBar];
       
       NSArray *dataSource = @[@"无",@"分屏_2",@"分屏_3",@"分屏_4",@"分屏_6",@"分屏_9"];
       filerBar.itemList = dataSource;
}
- (void)filterInit{
    self.context = [[EAGLContext alloc]initWithAPI:kEAGLRenderingAPIOpenGLES2];
    [EAGLContext setCurrentContext:self.context];
    
    self.vertices = malloc(sizeof(SenceVertex) * 4);
    
    self.vertices[0] = (SenceVertex){{-1, 1, 0}, {0, 1}};
    self.vertices[1] = (SenceVertex){{-1, -1, 0}, {0, 0}};
    self.vertices[2] = (SenceVertex){{1, 1, 0}, {1, 1}};
    self.vertices[3] = (SenceVertex){{1, -1, 0}, {1, 0}};
    
    CAEAGLLayer *layer = [[CAEAGLLayer alloc]init];
    layer.frame = CGRectMake(0, 100, self.view.bounds.size.width, self.view.bounds.size.width);
    layer.contentsScale = [UIScreen mainScreen].scale;
    [self.view.layer addSublayer:layer];
    [self bindRenderLayer:layer];
    
    NSString *imagePath = [[NSBundle mainBundle]pathForResource:@"jessica" ofType:@"jpg"];
    
    UIImage *image = [UIImage imageWithContentsOfFile:imagePath];
    
    GLuint textureID = [self createTextureWithImage:image];
    
    self.textureID = textureID;
    
    glViewport(0, 0, [self drawableWidth], [self drawableHeight]);
    
    GLuint vertexBuffer;
    glGenBuffers(1, &vertexBuffer);
    glBindBuffer(GL_ARRAY_BUFFER, vertexBuffer);
    GLsizeiptr bufferSizeBytes = sizeof(SenceVertex) * 4;
    glBufferData(GL_ARRAY_BUFFER, bufferSizeBytes, self.vertices, GL_STATIC_DRAW);
    
    [self setupNormalShaderProgram];
    
    self.vertexBuffer = vertexBuffer;
    
}
- (void)startFilerAnimation{
    //1.判断displayLink 是否为空
    //CADisplayLink 定时器
    if (self.displayLink) {
        [self.displayLink invalidate];
        self.displayLink = nil;
    }
    //2. 设置displayLink 的方法
    self.startTimeInterval = 0;
    self.displayLink = [CADisplayLink displayLinkWithTarget:self selector:@selector(timeAction)];
    
    //3.将displayLink 添加到runloop 运行循环
    [self.displayLink addToRunLoop:[NSRunLoop mainRunLoop]
                           forMode:NSRunLoopCommonModes];
}

//2. 动画
- (void)timeAction {
    //DisplayLink 的当前时间撮
    if (self.startTimeInterval == 0) {
        self.startTimeInterval = self.displayLink.timestamp;
    }
    //使用program
    glUseProgram(self.program);
    //绑定buffer
    glBindBuffer(GL_ARRAY_BUFFER, self.vertexBuffer);
    
    // 传入时间
    CGFloat currentTime = self.displayLink.timestamp - self.startTimeInterval;
    GLuint time = glGetUniformLocation(self.program, "Time");
    glUniform1f(time, currentTime);
    
    // 清除画布
    glClear(GL_COLOR_BUFFER_BIT);
    glClearColor(1, 1, 1, 1);
    
    // 重绘
    glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
    //渲染到屏幕上
    [self.context presentRenderbuffer:GL_RENDERBUFFER];
}

#pragma mark -- render
- (void)render{
    
    // 清除画布
    glClear(GL_COLOR_BUFFER_BIT);
    glClearColor(1, 1, 1, 1);
    
    //使用program
    glUseProgram(self.program);
    //绑定buffer
    glBindBuffer(GL_ARRAY_BUFFER, self.vertexBuffer);
    
    // 重绘
    glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
    //渲染到屏幕上
    [self.context presentRenderbuffer:GL_RENDERBUFFER];
    
}


//绑定渲染缓存区和帧缓存区
- (void)bindRenderLayer:(CALayer <EAGLDrawable> *)layer {
    GLuint renderBuffer;
    GLuint frameBuffer;
    
    glGenRenderbuffers(1, &renderBuffer);
    glBindRenderbuffer(GL_RENDERBUFFER, renderBuffer);
    
    [self.context renderbufferStorage:GL_RENDERBUFFER fromDrawable:layer];
    
    glGenFramebuffers(1, &frameBuffer);
    glBindFramebuffer(GL_FRAMEBUFFER, frameBuffer);
    glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_RENDERBUFFER, renderBuffer);
    
}
#pragma mark -- 纹理
- (GLuint)createTextureWithImage:(UIImage *)image {
    
    CGImageRef cgImageRef = image.CGImage;
    if (!cgImageRef) {
        NSLog(@"Failed to load image");
        exit(1);
    }
    //2、读取图片的大小，宽和高
    GLuint width = (GLuint)CGImageGetWidth(cgImageRef);
    GLuint height = (GLuint)CGImageGetHeight(cgImageRef);
    //获取图片的rect
    CGRect rect = CGRectMake(0, 0, width, height);
    
    //获取图片的颜色空间
    CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
    //3.获取图片字节数 宽*高*4（RGBA）
    void *imageData = malloc(width * height * 4);
    
    CGContextRef context = CGBitmapContextCreate(imageData, width, height, 8, width * 4, colorSpace, kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big);
       
       //将图片翻转过来(图片默认是倒置的)
       CGContextTranslateCTM(context, 0, height);
       CGContextScaleCTM(context, 1.0f, -1.0f);
       CGColorSpaceRelease(colorSpace);
       CGContextClearRect(context, rect);
       
       //对图片进行重新绘制，得到一张新的解压缩后的位图
       CGContextDrawImage(context, rect, cgImageRef);
       
       //设置图片纹理属性
       //5. 获取纹理ID
       GLuint textureID;
       glGenTextures(1, &textureID);
       glBindTexture(GL_TEXTURE_2D, textureID);
    glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE, imageData);
    
    //7.设置纹理属性
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
    
    //8.绑定纹理
    /*
     参数1：纹理维度
     参数2：纹理ID,因为只有一个纹理，给0就可以了。
     */
    glBindTexture(GL_TEXTURE_2D, 0);
    
    //9.释放context,imageData
    CGContextRelease(context);
    free(imageData);
    
    //10.返回纹理ID
    return textureID;
}
#pragma mark - Shader
// 默认着色器程序
- (void)setupNormalShaderProgram {
    //设置着色器程序
    [self setupShaderProgramWithName:@"normal"];
}

// 初始化着色器程序
- (void)setupShaderProgramWithName:(NSString *)name {
    GLuint program = [self programWithShaderName:name];
       
       //2. use Program
       glUseProgram(program);
       
       //3. 获取Position,Texture,TextureCoords 的索引位置
       GLuint positionSlot = glGetAttribLocation(program, "Position");
       GLuint textureSlot = glGetUniformLocation(program, "Texture");
       GLuint textureCoordsSlot = glGetAttribLocation(program, "TextureCoords");
       
       //4.激活纹理,绑定纹理ID
       glActiveTexture(GL_TEXTURE0);
       glBindTexture(GL_TEXTURE_2D, self.textureID);
       
       //5.纹理sample
       glUniform1i(textureSlot, 0);
       
       //6.打开positionSlot 属性并且传递数据到positionSlot中(顶点坐标)
       glEnableVertexAttribArray(positionSlot);
       glVertexAttribPointer(positionSlot, 3, GL_FLOAT, GL_FALSE, sizeof(SenceVertex), NULL + offsetof(SenceVertex, positionCoord));
       
       //7.打开textureCoordsSlot 属性并传递数据到textureCoordsSlot(纹理坐标)
       glEnableVertexAttribArray(textureCoordsSlot);
       glVertexAttribPointer(textureCoordsSlot, 2, GL_FLOAT, GL_FALSE, sizeof(SenceVertex), NULL + offsetof(SenceVertex, textureCoord));
       
       //8.保存program,界面销毁则释放
       self.program = program;
}
//link Program
- (GLuint)programWithShaderName:(NSString *)shaderName {
    //1. 编译顶点着色器/片元着色器
    GLuint vertexShader = [self compileShaderWithName:shaderName type:GL_VERTEX_SHADER];
    GLuint fragmentShader = [self compileShaderWithName:shaderName type:GL_FRAGMENT_SHADER];
    
    //2. 将顶点/片元附着到program
    GLuint program = glCreateProgram();
    glAttachShader(program, vertexShader);
    glAttachShader(program, fragmentShader);
    
    //3.linkProgram
    glLinkProgram(program);
    
    //4.检查是否link成功
    GLint linkSuccess;
    glGetProgramiv(program, GL_LINK_STATUS, &linkSuccess);
    if (linkSuccess == GL_FALSE) {
        GLchar messages[256];
        glGetProgramInfoLog(program, sizeof(messages), 0, &messages[0]);
        NSString *messageString = [NSString stringWithUTF8String:messages];
        NSAssert(NO, @"program链接失败：%@", messageString);
        exit(1);
    }
    //5.返回program
    return program;
}

//编译shader代码
- (GLuint)compileShaderWithName:(NSString *)name type:(GLenum)shaderType {
    
    //1.获取shader 路径
    NSString *shaderPath = [[NSBundle mainBundle] pathForResource:name ofType:shaderType == GL_VERTEX_SHADER ? @"vsh" : @"fsh"];
    NSError *error;
    NSString *shaderString = [NSString stringWithContentsOfFile:shaderPath encoding:NSUTF8StringEncoding error:&error];
    if (!shaderString) {
        NSAssert(NO, @"读取shader失败");
        exit(1);
    }
    
    //2. 创建shader->根据shaderType
    GLuint shader = glCreateShader(shaderType);
    
    //3.获取shader source
    const char *shaderStringUTF8 = [shaderString UTF8String];
    int shaderStringLength = (int)[shaderString length];
    glShaderSource(shader, 1, &shaderStringUTF8, &shaderStringLength);
    
    //4.编译shader
    glCompileShader(shader);
    
    //5.查看编译是否成功
    GLint compileSuccess;
    glGetShaderiv(shader, GL_COMPILE_STATUS, &compileSuccess);
    if (compileSuccess == GL_FALSE) {
        GLchar messages[256];
        glGetShaderInfoLog(shader, sizeof(messages), 0, &messages[0]);
        NSString *messageString = [NSString stringWithUTF8String:messages];
        NSAssert(NO, @"shader编译失败：%@", messageString);
        exit(1);
    }
    //6.返回shader
    return shader;
}


//获取渲染缓存区的宽
- (GLint)drawableWidth {
    GLint backingWidth;
    glGetRenderbufferParameteriv(GL_RENDERBUFFER, GL_RENDERBUFFER_WIDTH, &backingWidth);
    return backingWidth;
}
//获取渲染缓存区的高
- (GLint)drawableHeight {
    GLint backingHeight;
    glGetRenderbufferParameteriv(GL_RENDERBUFFER, GL_RENDERBUFFER_HEIGHT, &backingHeight);
    return backingHeight;
}

#pragma mark -- FilterBarDelegate
- (void)filterBar:(FilterBar *)filterBar didScrollToIndex:(NSUInteger)index{
    if (index == 0) {
         [self setupNormalShaderProgram];
    }else{
        [self setupShaderWithIndex:index];
    }
    
    
     // 重新开始滤镜动画
      [self startFilerAnimation];
     
     //渲染
//     [self render];
}
- (void)setupShaderWithIndex:(NSUInteger )index{
    switch (index) {
        case 1:
            [self setupShaderProgramWithName:@"SplitScreen_2"];
            break;
        case 2:
            [self setupShaderProgramWithName:@"SplitScreen_3"];
            break;
        case 3:
            [self setupShaderProgramWithName:@"SplitScreen_4"];
            break;
        case 4:
            [self setupShaderProgramWithName:@"SplitScreen_6"];
            break;
        case 5:
            [self setupShaderProgramWithName:@"SplitScreen_9"];
            break;
            
        default:
            break;
    }
}

@end

Copy the code

Remember to like it if you think it’s good! I heard that the people who read the praise will meet the test, meet the prize will be in. ღ(´ · ᴗ · ‘)