Metal supports GPU hardware acceleration, advanced 3D graphics rendering, and big data parallel computing. It also provides advanced and streamlined apis to ensure fine-grain framework and support low-level control in organizational architecture, program processing, graphical presentation, operation instructions, and management of instruction related data resources. Its core purpose is to reduce CPU overhead as much as possible, while transferring most of the load generated at runtime to the GPU.

Harbeth is a small collection of utilities and extensions on Apple’s Metal framework dedicated to making your Swift GPU code simpler and allowing you to prototype pipes faster. This article will introduce and design based on GPU filter, graphics processing and filter production… 👒 👒 👒

Characteristics of the

🟣 Currently, Harbeth’s most important features can be summarized as follows:

  • OpenCV module:

    • Hough line detects and corrects text
    • Feature extraction processing
    • Restoring old photos
    • Fixed image removing watermark
    • Maximum area cutting
    • Morphological operation
    • Fuzzy skin whitening
    • Picture perspective or blending
    • Modify brightness and contrast

  • The Metal module

    • Blend: Image fusion technology
    • BlurBlur effect
    • ColorProcess: Basic pixel color processing of an image
    • Effect: Effect processing
    • Lookup: Find table filters
    • Matrix: matrix convolution filter
    • Shape: Image shape and size are related

There are currently 90+ filters for you to use.

rendering

The main part of

  • Core core plate

    • C7FilterProtocol: Filter design must follow this protocol.
      • Modifier: Encoder type and corresponding function name.
      • factors: Set modify parameter factor, need to convert toFloat.
      • otherInputTextures: Multiple input sources, includingMTLTextureAn array of
      • OutputSize: Changes the size of the output image.

  • Output, output plate

    • C7FilterSerializer: Output content protocol. All outputs must implement this protocol.
      • MakeMTLTexture: Create a new texture based on the filter content, please note that the order in which the filters are added may affect the result of the image generation.
      • MakeImage: Generates data based on filter processing.
      • MakeGroup: Multiple filter combinations. Note that the order in which filters are added may affect the result of image generation.
    • C7FilterImage: Image input source based on C7FilterSerializer. The following modes only support encoders based on parallel computing.

Design a filter

  • Now let’s share how to design and deal with the first filter
  1. Implementation agreementC7FilterProtocal
Public protocol C7FilterProtocol {/// Encoder type and corresponding function name /// /// Calculate the corresponding 'kernel' function name /// Render requires a 'vertex' shader function name and a 'fragment' shader function name var modifier: modifier {get} /// create buffer /// set modify parameter factor, need to convert to 'Float'. Var factors: [Float] {get} /// multiple input source extension /// array containing 'MTLTexture' var otherInputTextures: C7InputTextures {get} func outputSize (input size: C7Size) -> C7Size}Copy the code
  1. Write a kernel shader based on parallel computing.
  2. Configure the transfer parameter factor. Only supportedFloatType.
  3. Configure additional textures as required.

For example

Designing an out-of-body filter,

public struct C7SoulOut: C7FilterProtocol {/// The adjusted soul, from 0.0 to 1.0, with a default of 0.5 public var soul: Float = 0.5 public var maxScale: Float = 1.5 public var maxAlpha: Float = 0.5 public var modifier: Modifier { return .compute(kernel: "C7SoulOut") } public var factors: [Float] { return [soul, maxScale, maxAlpha] } public init() { } }Copy the code

  • This filter takes three parameters:

    • soul: Adjusted soul, from 0.0 to 1.0, defaults to 0.5
    • maxScale: Maximum ratio of souls
    • maxAlpha: maximum transparency of soul

  • Write kernel functions based on parallel computation

kernel void C7SoulOut(texture2d<half, access::write> outputTexture [[texture(0)]], texture2d<half, access::sample> inputTexture [[texture(1)]], constant float *soulPointer [[buffer(0)]], constant float *maxScalePointer [[buffer(1)]], constant float *maxAlphaPointer [[buffer(2)]], uint2 grid [[thread_position_in_grid]]) { constexpr sampler quadSampler(mag_filter::linear, min_filter::linear); const half4 inColor = inputTexture.read(grid); const float x = float(grid.x) / outputTexture.get_width(); const float y = float(grid.y) / outputTexture.get_height(); const half soul = half(*soulPointer); const half maxScale = half(*maxScalePointer); const half maxAlpha = half(*maxAlphaPointer); Const half alpha = maxAlpha * (1.0h-soul); Const half scale = 1.0h + (maxscale-1.0h) * soul; Const half soulX = 0.5h + (x-0.5h)/scale; Const half soulY = 0.5h + (y-0.5h)/scale; const half4 soulMask = inputTexture.sample(quadSampler, float2(soulX, soulY)); Const half4 outColor = inColor * (1.0h-alpha) + soulMask * alpha; outputTexture.write(outColor, grid); }Copy the code
  • Simple to use, because my design is based on parallel computing pipeline, so you can directly generate pictures
Var filter = C7SoulOut() filter.soul = 0.5 filter.maxscale = 2.0 /// directly display in ImageView imageView.image = originImage.makeImage(filter: filter)Copy the code
  • As for the motion effects above, it’s easy to add a timer and changesoulIt’s worth it. It’s easy.

CocoaPods

  • To import the Metal module, you need to do so in your Podfile:
pod 'Harbeth'
Copy the code
  • To import the OpenCV image module, you need to do this in your Podfile:
pod 'OpencvQueen'
Copy the code

The last

  • So much for the introduction and design of filter frame.
  • Slowly add other related filters.
  • Filter Demo address, currently contains more than 90 kinds of filters, of course, most of the filter algorithm is designed by referring to GPUImage.
  • There is also a development accelerator KJCategoriesDemo at 🎷 and if you like, you can click 🌟

✌ ️.