[ARKit]4- The application of Shader

instructions

Code address for this article

ARKit series of articles directory

Last time we showed you how to read a shader String and load it using the shaderModifiers. Shader types have SCNShaderModifierEntryPoint geometry, surface, lightingModel, fragments.

Use the sample

Let’s take the surface type as an example. How does swift pass parameters to a shader? Direct use of KVC…

private func setupShader(a) {
    guard let path = Bundle.main.path(forResource: "skin", ofType: "shaderModifier", inDirectory: "art.scnassets"),
    let shader = try? String(contentsOfFile: path, encoding: String.Encoding.utf8) else {
        return
    }
    
    skin.shaderModifiers = [SCNShaderModifierEntryPoint.surface: shader]
    
    skin.setValue(Double(0), forKey: "blendFactor")
    skin.setValue(NSValue(scnVector3: SCNVector3Zero), forKey: "skinColorFromEnvironment")
    
    let sparseTexture = SCNMaterialProperty(contents: UIImage(named: "art.scnassets/textures/chameleon_DIFFUSE_BASE.png")! skin.setValue(sparseTexture, forKey:"sparseTexture")}Copy the code

OC

    NSURL *url = [[NSBundle mainBundle] URLForResource:@"skin" withExtension:@"shaderModifier"];
    NSError *error;
    NSString *shaderSource = [[NSString alloc] initWithContentsOfURL:url
                                                            encoding:NSUTF8StringEncoding
                                                               error:&error];
    if(! shaderSource) {// Handle the error
        NSLog(@"Failed to load shader source code, with error: %@", [error localizedDescription]);
    } else {
        skin.shaderModifiers = @{ SCNShaderModifierEntryPointSurface : shader };
    }

    [skin setValue:@(0) forKey:@"blendFactor"];
    [skin setValue:[NSValue valueWithSCNVector3:SCNVector3Zero] forKey:@"skinColorFromEnvironment"];
    SCNMaterialProperty *sparseTexture = [SCNMaterialProperty materialPropertyWithContents:[NSImage imageNamed:@"art.scnassets/textures/chameleon_DIFFUSE_BASE.png"]];
    [skin setValue:sparseTexture forKey:@"sparseTexture"];
Copy the code

Details about the shader type

• SCNShaderModifierEntryPointGeometry: used to handle geometry deformation.

The input is the structure Geometry:

struct SCNShaderGeometry {float4 position; float3 normal; float4 tangent; float4 color; float2 texcoords[kSCNTexcoordCount]; } _geometry; Access: ReadWrite Access: read and write Stages: Vertex Shader only Only in Vertex shadersCopy the code

KSCNTexcoordCount is an integer constant indicating the number of vertex coordinates used. The vector and coordinate (position, Normal and tangent) is the model space. Shader example, sinusoidal deformation:

GLSL
 uniform float Amplitude = 0.1;

 _geometry.position.xyz += _geometry.normal * (Amplitude * _geometry.position.y * _geometry.position.x) * sin(u_time);

Metal Shading Language
 #pragma arguments
 float Amplitude;
 
 #pragma body
 _geometry.position.xyz += _geometry.normal * (Amplitude * _geometry.position.y * _geometry.position.x) * sin(scn_frame.time);
Copy the code

• SCNShaderModifierEntryPointSurface: used to process the material in front of the lighting applications.

Input is the structure surface:

struct SCNShaderSurface {
       float3 view;                     // Direction from the point on the surface toward the camera (V)
       float3 position;                 // Position of the fragment
       float3 normal;                   // Normal of the fragment (N)
       float3 geometryNormal;           // Geometric normal of the fragment (normal map is ignored)
       float3 tangent;                  // Tangent of the fragment
       float3 bitangent;                // Bitangent of the fragment
       float4 ambient;                  // Ambient property of the fragment
       float2 ambientTexcoord;          // Ambient texture coordinates
       float4 diffuse;                  // Diffuse property of the fragment. Alpha contains the opacity.
       float2 diffuseTexcoord;          // Diffuse texture coordinates
       float4 specular;                 // Specular property of the fragment
       float2 specularTexcoord;         // Specular texture coordinates
       float4 emission;                 // Emission property of the fragment
       float2 emissionTexcoord;         // Emission texture coordinates
       float4 multiply;                 // Multiply property of the fragment
       float2 multiplyTexcoord;         // Multiply texture coordinates
       float4 transparent;              // Transparent property of the fragment
       float2 transparentTexcoord;      // Transparent texture coordinates
       float4 reflective;               // Reflective property of the fragment
       float  metalness;                // Metalness property of the fragment
       float2 metalnessTexcoord;        // Metalness texture coordinates
       float  roughness;                // Roughness property of the fragment
       float2 roughnessTexcoord;        // Roughness texture coordinates
       float4 selfIllumination;         // Self Illumination property of the fragment. Available since macOS 10.13, iOS 11, tvOS 11 and watchOS 4. Available as `emission` in previous versions.
       float2 selfIlluminationTexcoord; // Self textures coordinates Available since macOS 10.13, iOS 11, tvOS 11 and watchOS 4. Available as `emissionTexcoord` in previous versions.
       float  ambientOcclusion;         Ambient Occlusion property of the fragment. Available macOS 10.13, iOS 11, tvOS 11 and watchOS 4. Available as `multiply` in previous versions.
       float2 ambientOcclusionTexcoord; // Ambient texture coordinates Available since macOS 10.13, iOS 11, tvOS 11 and watchOS 4. Available as `multiplyTexcoord` in previous versions.
       float  shininess;                // Shininess property of the fragment
       float  fresnel;                  // Fresnel property of the fragment} _surface; Access: ReadWrite Access: read and write Stages: Fragment Shader only This parameter is limited to the Fragment shaderCopy the code

The vectors and their coordinates are the view space. Example shader to produce black and white stripes:

GLSL
 uniform float Scale = 12.0;
 uniform float Width = 0.25;
 uniform float Blend = 0.3;

 vec2 position = fract(_surface.diffuseTexcoord * Scale);
 float f1 = clamp(position.y / Blend, 0.0.1.0);
 float f2 = clamp((position.y - Width) / Blend, 0.0.1.0);
 f1 = f1 * (1.0 - f2);
 f1 = f1 * f1 * 2.0 * (3. * 2. * f1);
 _surface.diffuse = mix(vec4(1.0), vec4(vec3(0.0),1.0), f1);

Metal Shading Language
 #pragma arguments
 float Scale;
 float Width;
 float Blend;

 #pragma body
 float2 position = fract(_surface.diffuseTexcoord * Scale);
 float f1 = clamp(position.y / Blend, 0.0.1.0);
 float f2 = clamp((position.y - Width) / Blend, 0.0.1.0);
 f1 = f1 * (1.0 - f2);
 f1 = f1 * f1 * 2.0 * (3. * 2. * f1);
 _surface.diffuse = mix(float4(1.0), float4(float3(0.0),1.0), f1);
Copy the code

• SCNShaderModifierEntryPointLightingModel: providing custom lighting equation.

Input has more than one, all of the structure of the SCNShaderModifierEntryPointSurface structure lightingContribution, structure light:

All the structures available from the SCNShaderModifierEntryPointSurface entry point SCNShaderModifierEntryPointSurface all the structure of Access: ReadOnly Access: read-only Stages: Vertex shaderandFragment shader vertex shader and fragment shaderstruct SCNShaderLightingContribution {float3 ambient; float3 diffuse; float3 specular; } _lightingContribution; Access: ReadWrite: reads and writes Stages: Vertex ShaderandFragment shader vertex shader and fragment shaderstruct SCNShaderLight {
       float4 intensity;
       float3 direction; // Direction from the point on the surface toward the light (L)} _light; Access: ReadOnly Indicates the Access permission. Read-only Stages: Vertex ShaderandFragment shader vertex shader and fragment shaderCopy the code

Example shader for diffuse illumination:

GLSL
uniform float WrapFactor = 0.5;

float dotProduct = (WrapFactor + max(0.0, dot(_surface.normal,_light.direction))) / (1 + WrapFactor);
_lightingContribution.diffuse += (dotProduct * _light.intensity.rgb);
vec3 halfVector = normalize(_light.direction + _surface.view);
dotProduct = max(0.0.pow(max(0.0, dot(_surface.normal, halfVector)), _surface.shininess));
_lightingContribution.specular += (dotProduct * _light.intensity.rgb);

Metal Shading Language
#pragma arguments
float WrapFactor;

#pragma body
float dotProduct = (WrapFactor + max(0.0, dot(_surface.normal,_light.direction))) / (1 + WrapFactor);
_lightingContribution.diffuse += (dotProduct * _light.intensity.rgb);
float3 halfVector = normalize(_light.direction + _surface.view);
dotProduct = max(0.0.pow(max(0.0, dot(_surface.normal, halfVector)), _surface.shininess));
_lightingContribution.specular += (dotProduct * _light.intensity.rgb);

Copy the code

• SCNShaderModifierEntryPointFragment: used to last modified color.

Enter the structure in SCNShaderModifierEntryPointSurface, and structure the output:

All the structures available from the SCNShaderModifierEntryPointSurface entry point SCNShaderModifierEntryPointSurface all the structure of Access: ReadOnly Access: read-only Stages: Struct SCNShaderOutput {Fragment shader only struct SCNShaderOutputfloat4 color; } _output; Access: ReadWrite Access: read and write Stages: Fragment Shader only This parameter is limited to the Fragment shaderCopy the code

Example shader to reverse the final color:

GLSL
_output.color.rgb = vec3(1.0) - _output.color.rgb;

Metal Shading Language
_output.color.rgb = 1.0 - _output.color.rgb;
Copy the code

Shader format and parameters

shaderModifiers

// Custom GLSL uniforms declarations are of the form: [uniform type uniformName [= defaultValue]]
// Uniform global variable name [= default]]
uniform float myGrayAmount = 3.0;

// Custom Metal uniforms declarations require a #pragma and are of the form: [type name]
// Customize the Metal Uniforms declaration format: #pragma, format :[type global variable name]
#pragma arguments
float myGrayAmount;

// In Metal, you can also transfer varying values from the vertex shader (geometry modifier) to the fragment shader (surface/fragment  modifier)
// In Metal, you pass variable values from the vertex modifier to the fragment shader
// In one (or both) of the modifier, declare the varying values
// In one (or all) modifier, declare variable values
#pragma varyings
half3 myVec;

// Output varying values in the geometry modifier
// Output the variable value in the geometry modifier
out.myVec = _geometry.normal.xyz * 0.5h + 0.5h;

// And use them in the fragment modifier
// Use these values in the fragment modifier
_output.color.rgb = saturate(in.myVec);

// Optional global function definitions (for Metal: references to uniforms in global functions are not supported).
// Optional global function definition (Metal does not support references to global functions)
float mySin(float t) {
       return sin(t);
}

[#pragma transparent | opaque]
[#pragma body]

// the shader modifier code snippet itself
// 
float3 myColor = myGrayAmount;
_output.color.rgb += myColor;
Copy the code

Pragma body directive: must be used when declaring a function that is not in the shader code.

#pragma transparent: _output.color. RGB + (1 – _output.color.a) * dst.rgb; To render blend mode; Where DST represents the current fragment color, and the RGB component must be multiplied from the left.

The #pragma opaque directive forces render to be opaque. Ignore the alpha component of the fragment.

The SCNGeometry and SCNMaterial classes are key-value encoding compatible, which means you can assign key myAmplitude using KVC, even if it is not declared in the current class. SceneKit listens to the recipient’s myAmplitude Key when myAmplitude Uniform is declared in the Shader modifier. When this value changes,SceneKit binds uniform to a new value. Normal title types wrapped with NSValue are supported.

For the Metal:

• MTLBuffer is also supported as values.
• Complex data types (structures) declared in Metal Shader are also supported.
  • You can set them together with NSData.
  • Or you can use individual struct members, with the name of the member as the key and the value of the member of the corresponding type as the value.

Custom frames can be explicitly animated. The following types are available when declaring or binding custom frames:

The following prefixes are reserved by SceneKit by default and cannot be used in custom variable names: u_ a_ v_

SceneKit’s stated internal uniforms were:

Shader code sample

First, we create a new AR project, and Xcode will automatically generate the default project, showing a small plane. We changed the interface a little and added a few switches to control the Shader.

The setupShader() method is then implemented to load the four shaders and is called in viewDidLoad(). Here we modify the shader a little and add an effect factor to control whether the effect is displayed:

private func setupShader() { let shipNode = sceneView.scene.rootNode.childNode(withName: "shipMesh", recursively: true) let skin = shipNode? .geometry? .firstMaterial; // To make it easier to see the mixing effect, I added a factor in each sample shader. I set it to 0.0 and 1.0 to control the effect on and off. Default to 0.0-- off; Let geometryShader = """ Uniform float Amplitude = 0.1; Uniform Float GeometryFactor = 0.0; _geometry.position.xyz += _geometry.normal * (Amplitude * _geometry.position.y * _geometry.position.x) * sin(u_time) * GeometryFactor; """ let surfaceShader = """ uniform float Scale = 12.0; Uniform float Width = 0.25; Uniform Float Blend = 0.3; Uniform float SurfaceFactor = 0.0; vec2 position = fract(_surface.diffuseTexcoord * Scale); Float f1 = clamp(position.y/Blend, 0.0, 1.0); Float f2 = clamp((position.y-width)/Blend, 0.0, 1.0); float f2 = clamp((position.y-width)/Blend, 0.0, 1.0); F1 = f1 * (1.0-f2); F1 = f1 * F1 * 2.0 * (3. * 2. * f1); _surface.diffuse = _surface.diffuse * (1-surfacefactor) + mix(Vec4 (1.0), vec4(0.0),1.0) * SurfaceFactor; """ let lightShader = """ uniform float WrapFactor = 0.5; Uniform float LightFactor = 0.0; Float dotProduct = (WrapFactor + Max (0.0, dot(_surface.normal,_light.direction))/(1 + WrapFactor); float dotProduct = (WrapFactor + Max (0.0, dot(_surface.normal,_light.direction))/(1 + WrapFactor); _lightingContribution.diffuse += (dotProduct * _light.intensity.rgb) * LightFactor; vec3 halfVector = normalize(_light.direction + _surface.view); DotProduct = Max (0.0, pow(Max (0.0, dot(_surface.normal, halfVector)), _surface.shininess)); _lightingContribution.specular += (dotProduct * _light.intensity.rgb) * LightFactor; """ let float fragmentShader = """ uniform float FragmentFactor = 0.0; _output.color.rgb = (vec3(1.0) -_output.color.rgb) * FragmentFactor + (1-fragmentfactor) * _output.color.rgb; """ skin? .shaderModifiers = [SCNShaderModifierEntryPoint.geometry: geometryShader, SCNShaderModifierEntryPoint.surface: surfaceShader, SCNShaderModifierEntryPoint.lightingModel: lightShader, SCNShaderModifierEntryPoint.fragment: fragmentShader ] }Copy the code

Finally, as long as the switch changes, use KVC to set the factor value of the corresponding shader:

@IBAction func switchChange(_ sender: UISwitch) {
    let shipNode = sceneView.scene.rootNode.childNode(withName: "shipMesh", recursively: true)
    letskin = shipNode? .geometry? .firstMaterial;let factor = sender.isOn ? 1.0 : 0.0
    switch sender.tag {
    case 10: skin? .setValue(Double(factor), forKey: "GeometryFactor")
    case 11: skin? .setValue(Double(factor), forKey: "SurfaceFactor")
    case 12: skin? .setValue(Double(factor), forKey: "LightFactor")
    case 13: skin? .setValue(Double(factor), forKey: "FragmentFactor")
    default:
        print("switch")}}Copy the code

The effect is as follows: