“This is the 10th day of my participation in the Gwen Challenge in November. Check out the details: The Last Gwen Challenge in 2021.”

This is to share a video reference foreigners, I hope I am not only the porter of translation, but also the basis of the original share into their own content. Bring more and more fun AI projects to everyone.

<script src="https://cdn.jsdelivr.net/npm/[email protected]/lib/p5.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/[email protected]/dist/tf.min.js"></script>
Copy the code

Reinforcement learning is usually used in games to make intelligent agents play games. Because the game environment is relatively simple and fixed, it is suitable for studying reinforcement learning.

Design ideas

Here, we design a neural network, the input is not a picture, but as a vector image information data, then this can reflect the current game state information (5 dimensional vector, specific what information you can refer to the following input), and then through a hidden layer (8) unit of neurons in hidden layer after the output probability of up and down. It’s a classification problem

The input

  • Bird y.
  • Bird y vel
  • Top pipe Position closest to the upper pipe
  • Bottom pipe The location of the nearest lower pipe
  • X distance from the pipe
let inputs = [];
inputs[0] = this.y / height;
inputs[1] = closest.top / height;
inputs[2] = closest.bottom / height;
inputs[3] = closest.x / width;
inputs[4] = this.velocity / 10;
Copy the code

In fact, the ideal is to input an image, and then extract the features of the image through the convolutional neural network.

The output

Outputs the probability that two actions are down and up

  • down
  • upward
if (output[0] > output[1]) {
    this.up();
}
Copy the code

Code implementation

The introduction of tensorflow. Js

tf.memory()
{unreliable: true.reasons: Array(1), numTensors: 1000.numDataBuffers: 1000.numBytes: 66000}
Copy the code

Create a nn. Js

this.brain = new NeuralNetwork(5.8.2);
Copy the code

Here the constructor receives 3 with parameters A, b, and c representing the number of inputs, hidden layers, and output nodes, respectively

class NeuralNetwork{
    constructor(a,b,c){}}Copy the code
class NeuralNetwork{
    constructor(a,b,c){
        this.input_nodes = a;
        this.hidden_nodes = b;
        this.output_nodes = c; }}Copy the code

Define the model

Next we create a model this.createmodel () to create the model, which uses tF.sequential to create a container. This API should be familiar if used with Keras.

class NeuralNetwork{
    constructor(a,b,c){
        this.input_nodes = a;
        this.hidden_nodes = b;
        this.output_nodes = c;

        this.createModel();
    }

    createModel(){
        this.model = tf.sequential()
    }
}
Copy the code

Defining the hidden layer

    createModel(){
        this.model = tf.sequential();
        const hiddn = tf.layers.dense({
            units:this.hidden_nodes,
            inputShape: [this.input_nodes],
            activation:'sigmoid'
        })
        this.model.add(hiddn)
    }
Copy the code

Here a hidden layer is defined as the fully connected layer, tf.layers.dense receives an object,

  • Units: Specifies the number of neurons at this layer
  • InputShape: Enter the number of nodes
  • Activation: Specifies the activation function

If you’re not familiar with neural networks, the activation function is to add nonlinear transformations between the layers of the neural network

Define the output layer

const output = tf.layers.dense({
    units:this.output_nodes,
    activation:'softmax'
});
this.model.add(output)
Copy the code

To predict

In the prediction process, you need the component to translate JavaScript array objects into tensorflow tensor,

predict(inputs){
    const xs = tf.tensor2d([inputs]);
    const ys = this.model.predict(xs);
    const outputs = ys.dataSync();
    console.log(outputs);
    return outputs;

}
Copy the code