WebAudio develops 8th Note Sauce for H5

Welcome to Tencent cloud technology community, get more Tencent mass technology practice dry goods oh ~

The author:
QQ Music Technology team

Game demo source background

During the Spring Festival of 2017, a game from Toyo Neon became popular on Weibo and wechat moments. The game is “Don’t Stop! Eighth Note Sauce”. Quaver sauce has become popular because it doesn’t use manual manipulation, but sounds to control the game’s walking and jumping, which makes it feel new to users. The fun of the game has spawned many jokes on the Internet, such as “I would have beaten it if my neighbor hadn’t knocked on my door” and so on, and now people are playing the game with Musical Instruments.

At first, quaver sauce was only available on PC, but now it seems to be available on ios and Android. You can search and download it yourself. This paper tries to use JS, combined with web audio processing interface webAudio, to achieve an H5 version of “Don’t stop! Eighth note sauce” demo. I am also the first time to write a small game, the shortcomings of the article (such as game modeling, code implementation) also trouble readers to criticize and correct, learn together.

start

First look at a map of the game, experience, address (due to system compatibility issues, it is recommended that the copy address within WeChat webview open) zhazhaxia. Making. IO/server/publ…

play

After connecting the headset, it is best to open this page on wechat or mobile Q (system needs android5.0+) and agree to obtain the microphone permission. Then shout something into the microphone like, “Ahhh…” “, then the doge in the game will start to walk, the sound is loud enough, Doge will jump, fall pit will lose.

The game model

This is essentially a collision model game, and the main concepts in collision models are

• Target object: Doge block in the game
• Colliding Objects: Pits in the game
• Winning and losing conditions: if the volume of the target object overlaps with that of the collision object, it will be judged as losing

Based on these concepts, we can start designing the game.

Game design

Let’s take a look at the initial design

• The target object

The brown object in the picture is the target object, which is the operation object in our vision and can walk or jump

• The carrier of the target object

The blue box in the picture is the road in the game, carrying objects to walk. The road in the game is a whole, and the object we are actually manipulating in the code can move as a whole on the road below, visually as the movement of the target object. After the move, the picture is as follows

• Collision objects

Colliding objects are essentially potholes in the game path. As the object moves, the game places obstacles in the way, and the object must jump over these pits or the game fails and starts again.

The game to achieve

Game modeling design can start to realize, because this is a single page and relatively simple action, so the use of a single design mode to achieve.

The implementation section is divided into two parts. The first part introduces the overall implementation idea of the game, which is relatively easy. The second part introduces the webAudio voice control part of the game, which is the core of the game.

Implementation approach

• Parameter configuration

The game involves the configuration of some parameters to control the state of the game, the specific configuration can be generated at the time of writing, here is the configuration information of part of this article.

Config :{barrierWidth:80,// containerWidth:$('.container').width(),// number of barriers :0,// number of barriers: 1,2, timer:null, lockMove:false,// Lock mobile lockLost:false// dangerArea:[null,null],// dangerArea, collision area$tmpBarrier: $(".barrier-low"),
    lockConsole:true, volSum:0,// volSum:0, score:0, gameEnd:false, walkValue:1,// Volume threshold jumpValue:~~$('.threshold').val()// The volume threshold for the jump}Copy the code

• Initialize the

Initialization is mostly about generating vectors to populate the page. This paper mainly generates the number of initial carriers according to the width of the game container and fills them into the container.

initStat:function() {// Initialize the width of the obstacle, initialize the carrier $('.barrier').width(exports.config.barrierWidth);
      exports.config.numberOfBarrier = Math.ceil(exports.config.containerWidth / exports.config.barrierWidth) + 2;
      $('.bottom-barrier').width(exports.config.numberOfBarrier * exports.config.barrierWidth); / / barrier container wide exports. CreateBarrier (exports. Config. NumberOfBarrier); // create and populate}Copy the code

• Create a carrier

The various object designs in this article are implemented using DOM, but can also be implemented using Canvas or other implementations. When the carrier moves to a certain distance, a carrier is inserted behind the container, which may be a road or a pit. After insertion, delete the previously moved carrier to avoid excessive DOM performance problems.

  createBarrier:function(num) {// Create an obstacle, num number... // Other code$bc.append(exports.getBarrier(num,type));
  },
  getBarrier:function (num,type) {// Get obstacles var HTML ="";
      for(var i = 0; i < num; i++){
          html += '<div class="barrier '+ (type= = = 1?"barrier-high" : "barrier-low") +'" data-id="'+new Date().getTime()+'" > "< / div >'
      }
      return html;
  }
Copy the code

• The target object moves and jumps

When the volume reaches a certain condition, the target object starts to move visually, but what we are actually moving is the carrier below the target object.

letsGo:function(vol) {// Reach the condition to walk or jumpif(vol > exports.config.walkvalue) {// go exports.movebarrier ();if(vol > exports.config.jumpvalue) {// jump exports.jumpNotes(); }}else{/ / stop exports. StopBarrier (); }}Copy the code

• Collision detection

Collision detection is the detection of the distance between the target object and the collision object. In this game, we use an array to update colliding objects, adding them as they come in and updating them again as they leave. Detect as you move.

judgeLost:function(){// If failed, collision detection.... // Other codeif(exports.config.$tmpBarrier.attr('data-id')! = =$barrier.attr("data-id"){// Update colliding object... // Other code, update, score}... //if (parseInt($notes.css("bottom")) < = 200) {var / / determine whether in interval left = exports. Config. DangerArea [0], right = exports. Config. DangerArea [1].if(left <= 80&&right >= 160){// Whether the collision condition is exports.lost(); }}}Copy the code

• Failure reset

If the game fails, it reinitializes the Settings and repeats the above steps

lost:function() {// Lost the pit $()'.title').text('ah! Drop pit! Let's do it again! '); // reset part of exports.stopbarrier (); exports.config.gameEnd =true;
    $notes.stop(true).animate({bottom:0},500)
    setTimeout(function () {
        exports.config.gameEnd = false;
        $('.bottom-barrier').html("");
        exports.initStat();
        $notes.css("bottom"$(200),'.title').text('Speak up! Don't stop! Eighth note sauce ')
        exports.config.score = 0;
        $('.j_score').text(exports.config.score); }}, 3000)Copy the code

Use webAudio to control the walk and jump of the game

• Get microphone and volume

Getting the microphone on the web can be obtained through navigator.getusermedia, but this function is only available on the mobile end of android5.0+, iPhone currently does not provide this interface for JS calls. At present, the default browser of some domestic mobile phone manufacturers also has restrictions on this permission, or there are compatibility problems, it is suggested to use Wechat, Handq and other webviews to experience QQ browser X5 core APP (sold an advertisement).

The compatibility of Navigator.getUserMedia on PC is generally

navigator.getUserMedia = (navigator.getUserMedia ||
           navigator.webkitGetUserMedia ||
           navigator.mozGetUserMedia ||
           navigator.msGetUserMedia);
Copy the code

webAudioApi

To obtain the size of the microphone, you need to use the relevant interface of webAudioApi (for more information about webAudioApi, please refer to the introduction written by the author earlier github.com/zhazhaxia/w…).

• A brief introduction to

WebAudioApi is a W3C specification for handling Web audio. The core is the AudioContext, which is the core object for processing Web audio. All the processing interfaces are connected as nodes. The figure below depicts a source-to-target node Web audio processing process.

• Recording audio returns to the ear

Audio reflux refers to the real-time feedback of the audio received by the microphone during recording.

The scriptProcessNode of webAudioApi can be used to obtain the audio data of the microphone, and then output the audio data, and there will be a backear effect.

Realization process: After webAudio gets the microphone audio source, it is connected to ScriptProcess node, which can obtain the audio input data and output the audio in real time, so as to achieve the earback effect.

var source=exports.audioContext.createMediaStreamSource(stream); / / used for recording the processor nodes var recorder = exports. AudioContext. CreateScriptProcessor,1,1 (1024); source.connect(recorder); // Node connection recorder. Onaudioprocess =functionVar inputBuffer = e.inputBuffer; var outputBuffer = e.outputBuffer;for(var channel = 0; channel < outputBuffer.numberOfChannels; channel++) { var inputData = inputBuffer.getChannelData(channel); / / audio input var outputData = outputBuffer. GetChannelData (channel);for(var sample = 0; sample < inputBuffer.length; sample++) { outputData[sample] = inputData[sample]; }}};Copy the code

• Audio amplitude information

Getting audio amplitude can be understood as getting the volume of the audio.

The Analyser interface of webAudioApi can be used to obtain the audio data after Fourier transform, which contains the audio amplitude and other information. To get the audio amplitude in real time, you need to get the data in onAudioProcess. Since the audio noise component obtained by the microphone is a little high, a weighted processing is made here, and the average value is used as the target amplitude value. Finally, according to the processed audio amplitude of the game to walk and jump.

var analyser = exports.audioContext.createAnalyser(); // Audio parser recorder. Connect (analyser); analyser.connect(exports.audioContext.destination); // Set data analyser.fftSize = 1024; // Number of channels bufferLength = analyser.fftSize; dataArray = new Float32Array(bufferLength); // The frequency of each channel is recorder. Onaudioprocess =function(e){ analyser.getFloatTimeDomainData(dataArray); // get amplitude information exports.getVolume(dataArray); // Weighted amplitude}};Copy the code

1. Due to the difference between different hardware, the delay of earback effect is different

2. Due to the difference between PC and mobile phone hardware, the actual amplitude of PC will be significantly higher than that of mobile phone

The above is the main design of this game related ideas.

conclusion

Based on the inspiration of the PC game “Don’t stop? Eighth note Sauce”, this article describes the development ideas of its H5 simple version. There are many deficiencies in the design of the game, please criticize and correct the readers.

The intention of the author to develop the H5 version of eighth note Sauce is not only to realize the PC game with H5, but also to complete a webAduio demo through such a game with some innovation in gameplay. At present, Web is booming, W3C has also produced many new Web standards, such as webAudioApi, webAssembly, webAR, webGL and so on, which are in the development stage and have not been widely used in practical applications. So hopefully with a demo like this, we can get more ideas and make more fun applications with webAudio.

• Game address zhazhaxia. Making. IO/server/publ…

• Code github.com/zhazhaxia/z…

• reference

The W3C webAudioApi www.w3.org/TR/webaudio…

WebAudioApi and application case analysis github.com/zhazhaxia/w…

reading