The original link

Recently, I made a piano web application with Vue + tone.js, named AutoPiano. Life is like music, cheerful and free. This article is the summary and share of the project

Project introduction

AutoPiano is an online piano application developed using HTML5 technology. It aims to provide an elegant and simple platform for piano lovers, music lovers and all other creators to enjoy piano and music after learning and working. Similar to the piano games developed by Flash many years ago, Free Piano has changed the technology of H5 and also supports autoplay of piano music.

AutoPiano supports keyboard and mouse click playback, and there will be key and sound name prompts on the keys. In addition, AutoPiano also has teaching functions. One way is to get started quickly and play the music with simple keys, and the other way is to play examples and play the piano music automatically to achieve the purpose of demonstration. At present, both of these functions are under continuous improvement, as shown in the figure below:

Experience address: www.autopiano.cn

Does it take music theory to develop such an app?

Of course. Basic knowledge of music theory should be known, such as CDEFGAB name, staff, mode, rhythm and so on. I don’t have space to discuss it here. I recommend two websites:

  • www.bilibili.com/video/av121…
  • www.cnblogs.com/devymex/p/3…

The rest is programming, and how to translate music theory into program logic. The current technical architecture of AutoPiano is vue framework + tone.js.

Piano interface effect is how to write?

You can use CSS or maps. The author directly uses CSS to implement it. Considering that the piano has black keys and white keys, and the black keys and white keys are arranged in an orderly 7:5 pattern, it is not complicated to implement.

<div class="piano-key-wrap">
  <div class="piano-key wkey" v-for="note in Notes" :key="note.keyCode" :data-keyCode = "note.keyCode" v-if="note.type=='white'" @click="clickPianoKey($event, note.keyCode)"></div>
  <div class="bkey-wrap bkey-wrap1">
    <div class="piano-key bkey" v-for="note in Notes" :key="note.keyCode" :data-keyCode = "note.keyCode" v-if="note.type=='black' && note.id >= 36 && note.id <= 40" @click="clickPianoKey($event, note.keyCode)"></div>
  </div>
</div>
Copy the code
.piano-wrap { width: 90%; margin: 20px auto; .piano-key-wrap { width: 100%; background: @dark; overflow: hidden; position: relative; .wkey { display: inline-block; Width: 2.775%; height: 100%; margin: 0 auto; background: linear-gradient(white 10%, rgb(251, 251, 251) 92%, rgb(220, 220, 220) 93%, white 97%); border: solid 1px @dark; border-radius: 0 0 5px 5px; position: relative; &:active { background: linear-gradient(#eee 10%, #ddd 60%, #bbb 93%, #ccc 97%); } } .wkey-active { background: linear-gradient(#eee 10%, #ddd 60%, #bbb 93%, #ccc 97%); } .bkey-wrap { width: 20%; height: 0; position: absolute; top: 0; } .bkey-wrap1 {left: 0; }. Bkey - wrap2 {left: 19.5%; } .bkey-wrap3 {left: 39%; }. Bkey - wrap4 {left: 58.3%; }. Bkey - wrap5 {left: 77.7%; } .bkey { display: inline-block; width: 10%; height: 70%; background: linear-gradient(#000 10%, rgb(86, 86, 86) 85%, #000 90%); border-radius: 0 0 3px 3px; position: absolute; top: 0; overflow: hidden; &:active { background: linear-gradient(rgb(86, 86, 86) 10%, #000 90%, #222 100%); } } .bkey-active { background: linear-gradient(rgb(86, 86, 86) 10%, #000 90%, #222 100%); } .bkey:nth-child(1) {left: 9%; } .bkey:nth-child(2) {left: 23%; } .bkey:nth-child(3) {left: 50%; } .bkey:nth-child(4) {left: 65%; } .bkey:nth-child(5) {left: 79%; }}}Copy the code

Codepen also provides a number of examples that do not necessarily use the above implementation:

Codepen. IO/search/pens…

With proper control of CSS variables and values, you can make a better Piano interface.

How to achieve a single note playback?

The easiest way to implement audio playback is to use the AUDIO tag in HTML5 to control the audio by triggering the play and pause methods of audio. This is what I did at the beginning.

// <div class="audios-wrap" id="audios-wrap">
// 
      
// </div>

// Pre-create an audio element for each note
initAudioDom() {
  var vm = this
  for (let i = 0; i< vm.Notes.length; i++) {
    var note = vm.Notes[i]
    $('.audios-wrap').append(`<audio src='${note.url}' hidden='true' data-id='audio${i}' class='audioEle'>`); }},// Triggers playback of an audio element
playNote(url) {
  var vm = this
  if(! url ||typeofurl ! ='string') return;
  var audios = $('.audioEle');
  for (let i = 0; i< audios.length; i++) {
    let audio = audios[i];
    if (audio.src.indexOf(url) > - 1) {
      var cloneAudioNode = audio.cloneNode()
      cloneAudioNode.play()
      cloneAudioNode.remove()
      break; }}}Copy the code

This is my first implementation, where different notes trigger different audio. Then, perhaps out of curiosity, I tried tone.js, which provides more efficient control over audio playback through tone.js + built-in sampler, but of course, many of its complex features are not yet available…

// Initialize the synthesizer
this.synth = SmapleLibrary.load({
  instruments: "piano"
}).toMaster()

// Synthesizer triggers audio release
playNote(notename = 'C4', duration = '2n') {
  if (!this.synth) return
  this.synth.triggerAttackRelease(notename, duration);
}
Copy the code

Well, now the code conforms to the music aesthetics and code aesthetics, deliciously. Of course, THE author also expects tone.js to improve the Chinese document as soon as possible, otherwise it is still very difficult to get started. Interested partners can go to its official website for research.

About the autoplay of piano music

This part should be the most difficult part to develop the whole application, because the music or the score itself is quite complicated. According to baidu Baike, the staff originated from Greece and became the standard of music after thousands of years of continuous improvement. The emergence of simplified music is much later, but it is still complete, it can be said that simplified music is not simple.

The author’s realization idea is to take a musical score format as the carrier, convert the musical score into a format that can be recognized by the program, and then import it into the program for playing. The recognizable format is as follows, which is also used at present:

  {
    name: 'Little Star',
    step: 'C',
    speed: '100',
    playState: ' ',
    mainTrack: ['1 (1)'.' 1(1)'.' 5(1)'.' 5(1)'.' 6(1)'.' 6(1)'.' 5(2)'.' 4(1)'.' 4(1)'.' 3(1)'.' 3(1)'.' 2(1)'.' 2(1)'.' 1(2)'.' 5(1)'.' 5(1)'.' 4(1)'.' 4(1)'.' 3(1)'.' 3(1)'.'two (2)'.' 5(1)'.' 5(1)'.' 4(1)'.' 4(1)'.' 3(1)'.' 3(1)'.'two (2)'.' 1(1)'.' 1(1)'.' 5(1)'.' 5(1)'.' 6(1)'.' 6(1)'.' 5(2)'.' 4(1)'.' 4(1)'.' 3(1)'.' 3(1)'.' 2(1)'.' 2(1)'.' 1(2)'.'1 < (1)'.'1 < (1)'.'5 "(1)'.'5 "(1)'.'6 "(1)'.'6 "(1)'.'5 "(2)'.'4 "(1)'.'4 "(1)'.'< 3 (1)'.'< 3 (1)'.'< 2 (1)'.'< 2 (1)'.'1 < (2)'.'5 "(1)'.'5 "(1)'.'4 "(1)'.'4 "(1)'.'< 3 (1)'.'< 3 (1)'.'2 < (2)'.'5 "(1)'.'5 "(1)'.'4 "(1)'.'4 "(1)'.'< 3 (1)'.'< 3 (1)'.'2 < (2)'.'1 < (1)'.'1 < (1)'.'5 "(1)'.'5 "(1)'.'6 "(1)'.'6 "(1)'.'5 "(2)'.'4 "(1)'.'4 "(1)'.'< 3 (1)'.'< 3 (1)'.'< 2 (1)'.'< 2 (1)'.'1 < (2)'],
    backingTrack: ['1 > (0.5)'.5 > '(0.5)'.' '3 > (0.5).5 > '(0.5)'.'1 > (0.5)'.5 > '(0.5)'.' '3 > (0.5).5 > '(0.5)'.'1 > (0.5)'.6 > '(0.5)'.'4 > (0.5)'.6 > '(0.5)'.'1 > (0.5)'.5 > '(0.5)'.' '3 > (0.5).5 > '(0.5)'.'1 > (0.5)'.6 > '(0.5)'.'4 > (0.5)'.6 > '(0.5)'.'1 > (0.5)'.5 > '(0.5)'.' '3 > (0.5).5 > '(0.5)'.'7 > > (0.5)'.5 > '(0.5)'.'2 > (0.5)'.5 > '(0.5)'.'1 > (0.5)'.5 > '(0.5)'.' '3 > (0.5).5 > '(0.5)'.'1 > (0.5)'.' '3 > (0.5).5 > '(0.5)'.'1' (0.5).'1 > (0.5)'.'4 > (0.5)'.6 > '(0.5)'.'1' (0.5).'1 > (0.5)'.' '3 > (0.5).5 > '(0.5)'.'1' (0.5).'5 > > (0.5)'.'7 > > (0.5)'.'2 > (0.5)'.5 > '(0.5)'.'1 > (0.5)'.' '3 > (0.5).5 > '(0.5)'.'1' (0.5).'1 > (0.5)'.'4 > (0.5)'.6 > '(0.5)'.'1' (0.5).'1 > (0.5)'.' '3 > (0.5).5 > '(0.5)'.'1' (0.5).'5 > > (0.5)'.'7 > > (0.5)'.'2 > (0.5)'.5 > '(0.5)'.'1 > (0.5)'.5 > '(0.5)'.' '3 > (0.5).5 > '(0.5)'.'1 > (0.5)'.5 > '(0.5)'.' '3 > (0.5).5 > '(0.5)'.'1 > (0.5)'.6 > '(0.5)'.'4 > (0.5)'.6 > '(0.5)'.'1 > (0.5)'.5 > '(0.5)'.' '3 > (0.5).5 > '(0.5)'.'1 > (0.5)'.6 > '(0.5)'.'4 > (0.5)'.6 > '(0.5)'.'1 > (0.5)'.5 > '(0.5)'.' '3 > (0.5).5 > '(0.5)'.'7 > > (0.5)'.5 > '(0.5)'.'2 > (0.5)'.5 > '(0.5)'.'1 > (0.5)'.5 > '(0.5)'.' '3 > (0.5).5 > '(0.5)'.'1' (0.75).'5 (0.25).'3' (0.5).'5 (0.5).'1' (0.75).'5 (0.25).'3' (0.5).'5 (0.5).'1' (0.75).'6 (0.25).'4 (0.5).'6 (0.5).'1' (0.75).'5 (0.25).'3' (0.5).'5 (0.5).'1' (0.75).'6 (0.25).'4 (0.5).'6 (0.5).'1' (0.75).'5 (0.25).'3' (0.5).'5 (0.5).'7 > (0.75)'.'5 (0.25).'2 (0.5).'5 (0.5).'1' (0.75).'5 (0.25).'3' (0.5).'5 (0.5).'1' (0.75).'3' (0.25).'5 (0.5).'1 < (0.5)'.'1' (0.75).'4 (0.25).'6 (0.5).'1 < (0.5)'.'1' (0.75).'3' (0.25).'5 (0.5).'1 < (0.5)'.5 > '(0.75)'.'7 > (0.25)'.'2 (0.5).'5 (0.5).'1' (0.75).'3' (0.25).'5 (0.5).'1 < (0.5)'.'1' (0.75).'4 (0.25).'6 (0.5).'1 < (0.5)'.'1' (0.75).'3' (0.25).'5 (0.5).'1 < (0.5)'.5 > '(0.75)'.'7 > (0.25)'.'2 (0.5).'5 (0.5).'1' (0.75).'5 (0.25).'3' (0.5).'5 (0.5).'1' (0.75).'5 (0.25).'3' (0.5).'5 (0.5).'1' (0.75).'6 (0.25).'4 (0.5).'6 (0.5).'1' (0.75).'5 (0.25).'3' (0.5).'5 (0.5).'1' (0.75).'6 (0.25).'4 (0.5).'6 (0.5).'1' (0.75).'5 (0.25).'3' (0.5).'5 (0.5).'7 > (0.75)'.'5 (0.25).'2 (0.5).'5 (0.5).'1 > (2)']}Copy the code

Well, isn’t it complicated, bloated… It takes the simple spectrum as the carrier and marks the pitch and duration through special symbols, thus producing mainTrack and backingTrack two tracks, and then playing synchronously. This implementation, while simple, has a number of fatal drawbacks:

  1. Incompatible with common computer music notation formats such as MusicXML
  2. Cannot represent all dimensions of music, for example, many piano scores have more than two tracks
  3. It is too abstract and complex to be practical to make this recognition format
  4. Music professional: What are you doing?

So I turned to another implementation, parsing MusicXML, but this process is time-consuming and only half completed, and some details are not fully parsed. If you have any good ideas, please leave them in the comments section.

Welcome to contribute and cooperate

  • Contribute random lyrics shown on the home page: github.com/WarpPrism/A…
  • Contribute a quick primer on how to play: github.com/WarpPrism/A…

When forking, follow the GPL open source license.

The last

Finally, post the experience address: www.autopiano.cn

Welcome to experience and share.

Parsing MusicXML is still a work in progress, and if it works, there will be tons of songs added to the demo to learn from, but if it doesn’t, well, it’s because life got in my way…

Original is not easy, reprint to share when please indicate the source ~