{"rowid": 209, "title": "Feeding the Audio Graph", "contents": "In 2004, I was given an iPod.\nI count this as one of the most intuitive pieces of technology I\u2019ve ever owned. It wasn\u2019t because of the the snazzy (colour!) menus or circular touchpad. I loved how smoothly it fitted into my life. I could plug in my headphones and listen to music while I was walking around town. Then when I got home, I could plug it into an amplifier and carry on listening there.\nThere was no faff. It didn\u2019t matter if I could find my favourite mix tape, or if my WiFi was flakey - it was all just there.\nNowadays, when I\u2019m trying to pair my phone with some Bluetooth speakers, or can\u2019t find my USB-to-headphone jack, or even access any music because I don\u2019t have cellular reception; I really miss this simplicity.\nThe Web Audio API\nI think the Web Audio API feels kind of like my iPod did.\nIt\u2019s different from most browser APIs - rather than throwing around data, or updating DOM elements - you plug together a graph of audio nodes, which the browser uses to generate, process, and play sounds.\nThe thing I like about it is that you can totally plug it into whatever you want, and it\u2019ll mostly just work.\nSo, let\u2019s get started. First of all we want an audio source.\n\n\n(Song - Night Owl by Broke For Free)\nThis totally works. However, it\u2019s not using the Web Audio API, so we can\u2019t access or modify the sound it makes.\nTo hook this up to our audio graph, we can use an AudioSourceNode. This captures the sound from the element, and lets us connect to other nodes in a graph.\nconst audioCtx = new AudioContext()\n\nconst audio = document.querySelector('audio')\nconst input = audioCtx.createAudioSourceNode(audio)\n\ninput.connect(audioCtx.destination)\n\nGreat. We\u2019ve made something that looks and sounds exactly the same as it did before. Go us.\nGain\nLet\u2019s plug in a GainNode - this allows you to alter the amplitude (volume) of an an audio stream.\nWe can hook this node up to an element by setting the gain property of the node. (The syntax for this is kind of weird because it\u2019s an AudioParam which has options to set values at precise intervals).\nconst node = audioCtx.createGain()\n\nconst input = document.querySelector('input')\ninput.oninput = () => node.gain.value = parseFloat(input.value)\n\ninput.connect(node)\nnode.connect(audioCtx.destination)\n\nYou can now see a range input, which can be dragged to update the state of our graph. This input could be any kind of element, so now you\u2019ll be free to build the volume control of your dreams.\nThere\u2019s a number of nodes that let you modify/filter an audio stream in more interesting ways. Head over to the MDN Web Audio page for a list of them.\nAnalysers\nSomething else we can add to our graph is an AnalyserNode. This doesn\u2019t modify the audio at all, but allows us to inspect the sounds that are flowing through it. We can put this into our graph between our AudioSourceNode and the GainNode.\nconst analyser = audioCtx.createAnalyser()\n\ninput.connect(analyser)\nanalyser.connect(gain)\ngain.connect(audioCtx.destination)\nAnd now we have an analyser. We can access it from elsewhere to drive any kind of visuals. For instance, if we wanted to draw lines on a canvas we could totally do that:\nconst waveform = new Uint8Array(analyser.fftSize)\nconst frequencies = new Uint8Array(analyser.frequencyBinCount)\nconst ctx = canvas.getContext('2d')\n\nconst loop = () => {\n requestAnimationFrame(loop)\n analyser.getByteTimeDomainData(waveform)\n analyser.getByteFrequencyData(frequencies)\n\n ctx.beginPath()\n waveform.forEach((f, i) => ctx.lineTo(i, f))\n ctx.lineTo(0,255)\n frequencies.forEach((f, i) => ctx.lineTo(i, 255-f))\n ctx.stroke()\n}\nloop()\n\nYou can see that we have two arrays of data available (I added colours for clarity):\n\nThe waveform - the raw samples of the audio being played.\nThe frequencies - a fourier transform of the audio passing through the node.\n\nWhat\u2019s cool about this is that you\u2019re not tied to any specific functionality of the Web Audio API. If it\u2019s possible for you to update something with an array of numbers, then you can just apply it to the output of the analyser node.\nFor instance, if we wanted to, we could definitely animate a list of emoji in time with our music.\nspans.forEach(\n (s, i) => s.style.transform = `scale(${1 + (frequencies[i]/100)})`\n)\n\n\ud83d\udd08\ud83c\udfa4\ud83c\udfa4\ud83c\udfa4\ud83c\udfba\ud83c\udfb7\ud83d\udcef\ud83c\udfb6\ud83d\udd0a\ud83c\udfb8\ud83c\udfba\ud83c\udfa4\ud83c\udfb8\ud83c\udfbc\ud83c\udfb7\ud83c\udfba\ud83c\udfbb\ud83c\udfb8\ud83c\udfbb\ud83c\udfba\ud83c\udfb8\ud83c\udfb6\ud83e\udd41\ud83c\udfb6\ud83c\udfb5\ud83c\udfb5\ud83c\udfb7\ud83d\udcef\ud83c\udfb8\ud83c\udfb9\ud83c\udfa4\ud83c\udfb7\ud83c\udfbb\ud83c\udfb7\ud83d\udd08\ud83d\udd0a\ud83d\udcef\ud83c\udfbc\ud83c\udfa4\ud83c\udfb5\ud83c\udfbc\ud83d\udcef\ud83e\udd41\ud83c\udfbb\ud83c\udfbb\ud83c\udfa4\ud83d\udd09\ud83c\udfb5\ud83c\udfb9\ud83c\udfb8\ud83c\udfb7\ud83d\udd09\ud83d\udd08\ud83d\udd09\ud83c\udfb7\ud83c\udfb6\ud83d\udd08\ud83c\udfb8\ud83c\udfb8\ud83c\udfbb\ud83c\udfa4\ud83e\udd41\ud83c\udfbc\ud83d\udcef\ud83c\udfb8\ud83c\udfb8\ud83c\udfbc\ud83c\udfb8\ud83e\udd41\ud83c\udfbc\ud83c\udfb6\ud83c\udfb6\ud83e\udd41\ud83c\udfa4\ud83d\udd0a\ud83c\udfb7\ud83d\udd0a\ud83d\udd08\ud83c\udfba\ud83d\udd0a\ud83c\udfbb\ud83c\udfb5\ud83c\udfbb\ud83c\udfb8\ud83c\udfb5\ud83c\udfba\ud83c\udfa4\ud83c\udfb7\ud83c\udfb8\ud83c\udfb6\ud83c\udfbc\ud83d\udcef\ud83d\udd08\ud83c\udfba\ud83c\udfa4\ud83c\udfb5\ud83c\udfb8\ud83c\udfb8\ud83d\udd0a\ud83c\udfb6\ud83c\udfa4\ud83e\udd41\ud83c\udfb5\ud83c\udfb9\ud83c\udfb8\ud83d\udd08\ud83c\udfbb\ud83d\udd09\ud83e\udd41\ud83d\udd09\ud83c\udfba\ud83d\udd0a\ud83c\udfb9\ud83e\udd41\ud83c\udfb7\ud83d\udcef\ud83c\udfb7\ud83c\udfb7\ud83c\udfa4\ud83c\udfb8\ud83d\udd09\ud83c\udfb9\ud83c\udfb7\ud83c\udfb8\ud83c\udfba\ud83c\udfbc\ud83c\udfa4\ud83c\udfbc\ud83c\udfb6\ud83c\udfb7\ud83c\udfa4\ud83c\udfb7\ud83d\udcef\ud83d\udcef\ud83c\udfbb\ud83c\udfa4\ud83c\udfb7\ud83d\udcef\ud83c\udfb9\ud83d\udd08\ud83c\udfb5\ud83c\udfb9\ud83c\udfbc\ud83d\udd0a\ud83d\udd09\ud83d\udd09\ud83d\udd08\ud83c\udfb6\ud83c\udfb8\ud83e\udd41\ud83c\udfba\ud83d\udd08\ud83c\udfb7\ud83c\udfb5\ud83d\udd09\ud83e\udd41\ud83c\udfb7\ud83c\udfb9\ud83c\udfb7\ud83d\udd0a\ud83c\udfa4\ud83c\udfa4\ud83d\udd0a\ud83c\udfa4\ud83c\udfa4\ud83c\udfb9\ud83c\udfb8\ud83c\udfb9\ud83d\udd09\ud83c\udfb7\n\n\nGenerating Audio\nSo far, we\u2019ve been using the