In our final project of the semester, we decided to create an LED light show. We would take audio as input, perform some processing to determine the LED strip colors, then send the data to the LED strip to make the dynamic light show. Tools used include BlackHole for audio input, output, and routing, Python along with several libraries for audio processing, and a Raspberry Pi pico for communication between the computer program and the LED strip.

The first step involved obtaining the audio in a format we could process. We were able to read from an existing .wav file, take audio input from an external microphone, and even obtain audio input from other apps on the computer. To read from an existing .wav file, we used the function scipy.io.wavfile.read function in the Python library scipy, passing in the path to the file as an argument. This would obtain the raw audio data along with the sample rate.

However, this only worked with preexisting wav files, so the amount of music that could actually be played was fairly limited. In order to generalize this process, we used BlackHole Audio, a loopback driver which reroutes the sound playing from a device back into its own microphone.

Once this is done, we can use another library, pyaudio, to listen to the microphone input.

Next, we had to determine how to transform the audio data into an array of colors. We first obtained the Fourier transform of the data to obtain more meaningful data to work with. The resulting light sequence had three components: a volume slider at the left side representing the current volume of the audio, a background hue of which its intensity corresponded to the current volume of the low passed audio, and bright patches that moved to the right, spawning when the low passed audio had peaks in volume.

At first, we wanted to use advanced techniques in signal processing to identify each instrument, but some researching quickly revealed the complexity of the task, which meant we had to come up with a simpler approximation. While most percussion sounds are largely comprised of a transient filled with higher frequencies, there was enough of a spike on the low end of the frequency spectrum that we figured looking for spikes in the low passed data would suffice for a first attempt. To detect spikes, we compared the current volume to an estimate of the current peak volume. To ensure the current peak volume remained reasonably accurate even in cases of a loud section immediately followed by a quiet section, we had the previous estimated peak decay over time, allowing the volume to eventually reach the correct value. Here is a demonstration of what this looks like! Well, kinda. Given the dimensions of the recording, not sure how visible this is.

We also needed to set up the LED strip by connecting it to a circuit with a Raspberry Pi pico. The LED strip we used is made up of about 300 neopixels, small individually addressable RGB light elements in parallel. The pico pinout connects to the control wire of the strip, sending each light direction about the coloring it should take on. These each can consume a fairly decent 60 milliamps of power, so the whole setup needs a pretty significant power supply.

Finally, we linked took the lightshow array and broadcast it to the Raspberry Pi and LED strip. This ended up taking much longer than initially intended. We originally tried to make this connection over bluetooth. The data transmission rate was more than fast enough for our purposes. However, it seems that the pico has some hardware issues that prevent it from writing to pinout while simultaneously maintaining a bluetooth connection, meaning the lights would flicker constantly when connected. So, we instead switched to wifi connection by having the pico subscribe to an mqtt server that is published to by the computer.

Putting it all together, we had a very flashy project which combined mathematics, coding, and engineering to create a work of art!