Hiqh quality DPCM

Hi all, it’s been some time since I last wrote anything on this blog. This time around I’d like to share some experience that I’ve had with my recent hobby project: high quality time domain audio compression using DPCM (Differential Pulse-Code Modulation).

Be warned: this is quite a lengthy post.

tl;dr version: A Dynamic Differential Pulse-Code Modulation (DDPCM) method is presented. The method generally gives CD quality audio with about 8 bits / sample (compression ratio 2:1), and close-to CD quality with 4 bits / sample (compression ratio 4:1), using a super-simple decoder. Sound examples here, and a C++ reference implementation here.

Continue reading

Posted in Audio, Compression | Leave a comment

Working on SoundBox

I’ve done some more work on the SoundBox tool, and as some of you may have noticed that it’s now live on the dedicated site: sb.bitsnbites.eu.

Compared to the Lite version (posted to the Mozilla Dev Derby), a few new additions have been made:

  • You can now import/export binary song files from/to your local computer.
  • You can import Sonant songs into the editor.
  • WAV export should now work much better in most browsers.
  • You get real-time audio playback when you enter notes (requires the Web Audio API, tested in Firefox Nightly and Chrome).
  • You can even enter notes using a MIDI input device (currently requires the Jazz-Plugin, in anticipation of the Web MIDI API).
  • There’s a stand-alone player routine: player-small.js.

It keeps getting better – hope you like it!

Posted in JavaScript | 4 Comments

The SoundBox synth

After releasing the 4k demo SWAY, I’ve received much positive feedback regarding the synth that was used to produce the music. The synth is based on a new work-in-progress project that I have not released yet. However, my colleges convinced me to participate in the Mozilla Dev Derby (April 2012 is all about audio), so I submitted a slightly stripped down version yesterday (I had to remove the PHP/server parts). So, now you can try it out at least (it’s GPL, so all source is available): I call it SoundBox Lite.

The background to the tool is obviously Sonant Live, but with some useful changes:

  • The soft synth core was rewritten, generally giving better sound quality (better filter logic, supporting sharp transients etc) and faster sound generation.
  • There are now four note tracks per pattern, so you can do proper chords.
  • An “effects track” was added per pattern, which gives you the opportunity to change any instrument / fx parameter dynamically (quite useful!).
  • A new distortion effect was added, giving that powerful dirty sound.
  • Also, some functions that I personally did not use that much, or that were just redundant, were removed (for a more compact player routine).

I will continue working on a more public release. In the meantime, enjoy the demo.

EDIT: You can find the official tool here: SoundBox.

Posted in JavaScript | 5 Comments

Targeting the mobile platform

The latest and greatest version of Opera Mobile (version 12) has just been released, which includes support for lots of HTML 5 goodies, including WebGL. While it usually works great, I noticed that many sites, HTML apps and demos have not really been designed for the mobile platform. In this post I will try to cover two important factors to keep in mind if you want your HTML5 / WebGL app to work well on a mobile device:

  1. Adapt the viewport size/scale to the mobile screen.
  2. Support touch input.

(I was originally planning to write up some notes on WebGL performance and precision considerations too, but I’ll save that for a later post).
Continue reading

Posted in JavaScript, Mobile, WebGL | 1 Comment

Frank 4k WebGL demo – Lessons learned

Frank 4k So, I’ve just released my first proper 4k WebGL demo: Frank 4k! It’s a neat multi-part demo with 3D graphics and a low-fi synth tune, all written entirely in JavaScript (as far as I know, it’s the first of its kind).

Try it here (compressed) or here (“safe mode”, no compression, with error checks etc).

I have to admit that the design is kind of simple. One of the reasons is that I mostly focused on fitting it all into less than 4096 bytes.

If you want to know more about how I did it, read on… I hope that this post will inspire more people to try out the slightly odd art of 4k JavaScript demo programming.

Continue reading

Posted in Compression, JavaScript, WebGL | 20 Comments

Muon Baryon in 6k JavaScript

I present to you my latest hack: Muon Baryon for JavaScript/WebGL. (Note: requires an HTML5/WebGL browser, a decent graphics card, and the pre-calculation time before the demo actually starts is quite long).

The Original

So, what is Muon Baryon? It’s a Windows 4k demo originally created by Youth Uprising, Ümlaüt Design and Outracks. More precisely, it’s the winner of the Assembly 2009 4k contest (so it’s a bit old – but still one of my absolute favorites). You can find out more here.

The HTML5 Port

Now, Muon Baryon was low hanging fruit for an HTML5 port, since:

  1. Most of it is written in GLSL, which is available through WebGL.
  2. The most complicated non-GLSL part was the music synth, and it has already been ported to JavaScript.

So, to begin with I started out converting the remaining C code to JavaScript, which mostly consisted of replacing things like glUseProgram() with gl.useProgram(), etc (a no-brainer). I also had to replace the handy desktop OpenGL call glRect with slightly more bloated vertex buffer code. When trying it out for the first time, nothing worked (black screen)… of course. I suspected the GLSL code and inserted the following lines after compiling the shaders:

if (!gl.getShaderParameter(shader, gl.COMPILE_STATUS))
    alert(gl.getShaderInfoLog(shader));

I quickly realized that the original GLSL code was written for the older desktop GLSL 1.x (using ftransform etc), while WebGL uses GLSL ES, which has dropped a few functions. Here are a few things I had to do to convert the GLSL code to GLSL ES:

  1. Replace ftransform with a “custom” position attribute (quite simple, since the demo didn’t use any transformations).
  2. Include the line precision highp float; in the fragment shaders.
  3. Replace the while-loop with a fixed length for-loop, i.e:
while(g<1.){ ... }

becomes:

for(int j=0; j<999; j++){if(g>=1.)break; ... }

Not sure about how good this solution is, but at least Firefox and Chrome do not complain. Another thing I did was to make the shader resolution independent (the original was hard-coded to a certain screen resolution). Other than that, it was quite straight forward work.

The Result

The size of the original Windows demo was 3977 bytes. I did not expect to be able to squeeze the JavaScript port down to 4k, but I hoped to get it below 8k at least. And the final result, after using Google Closure Compiler and CrunchMe to compress the code, is: 5846 bytes!!

The un-packed code can be found here: demo-unpacked.js

…and the (unreadable) packed code can be found here: demo.js

Getting it to run

Ok, I have tried this in Firefox 5 and Chromium 13+ under Ubuntu with an NVIDIA 9600 card (fairly old by now), and it works just fine.

Windows users may find that it will not work by default (seems to be an issue in the ANGLE GLSL->HLSL conversion). To make it work, enable the OpenGL back-end.

In Firefox, go to about:config, filter out “webgl” and change webgl.prefer-native-gl to true.

With Chrome, you have to start the program with an extra argument, like this: chrome.exe –use-gl=desktop.

Posted in JavaScript, WebGL | 16 Comments

Let’s make some music – online!

Music trackers for PCs have been around for decades, and I’ve personally used quite a few of them, starting with SoundTracker on the Amiga, and then various version and ProTracker, OctaMED, MadTracker and ReNoise, to name the ones I’ve used the most.

Some time ago, I came across Sonant, which is a very specialized tracker. Its main purpose is to create songs that can be used in tiny demos, where the size of the song data plus the player software must not exceed a couple of kilobytes (for comparison, an MP3 file is usually several megabytes).

For various reasons, I decided to make a JavaScript port of Sonant, and the result is:

Sonant Live

Feel free to try it out!

Continue reading

Posted in JavaScript | 17 Comments

Sonant Live – sneak peek

Have been working on a web tool that I think is a bit different from the rest: It’s a music tracker that runs entirely in your browser!

It’s currently a beta-release, but with some patience and training you can get it to do wonderful things. So try it out now: http://sonantlive.bitsnbites.eu/

The most annoying limitation right now is the lack of a progress bar (you get kind of a progress indicator in Opera – but not in other browsers). Haven’t solved how to do the music generation in a web worker yet (for instance, I use a 2D canvas context in the synth, which I think isn’t possible in a Web Worker).

Posted in JavaScript | Leave a comment

Benchmarking different browsers with the synth

In an earlier post I mentioned that Opera performs very well with my JavaScript port of the software synth Sonant.

Out of curiosity, I did some benchmarking of other browsers as well (this time on a quad core Xeon running Ubuntu 11.04 64-bit). Again, I tested the tune from the Muon Baryon demo (you can get the song file here if you wish to try it yourself in the online player/converter).

For reference, I ran the original demo (assembler optimized) in Wine (I believe it should perform as in native Windows, but it uses 32-bit machine code instead of the 64-bit machine code in the browser JIT:s).

Results

So, here are the results (lower is better):

Opera 11.11 35 s
Chromium 13.0.774.0 44 s
Firefox 4.0.1 70 s
Original demo (native x86, running in Wine) 42 s

Conclusions

Well, Opera is ahead of the competition here (it’s twice as fast as Firefox 4!), but Chromium isn’t far behind.

Also worth noting is that Opera is even faster than the native original synth. While this may sound amazing (yes, I think it is!), please remember that the comparison is not really fair. For instance, the original was optimized for size, not for speed (the JIT compilers in the browsers probably optimize more for speed), and as I mentioned, the original runs in 32-bit mode, while the browsers run in 64-bit mode.

Posted in JavaScript | Leave a comment

Impressive JavaScript performance!

I just compared the song generation time in the JavaScript port of Sonant to the hand optimized x86 assembler version of the same synth, as used in the original Muon Baryon 4k demo.

I found the SNT tune here, and ran it in Opera in my new online Sonant player (check “Play Now!” if you wish to play the song in your browser), and guess what:

Native asm synth: 43s
JavaScript synth: 47s

That means that the JavaScript synth is less than 10% slower than the hand optimized native assembly language synth!

Posted in JavaScript | 1 Comment