A sound playground of your very own.

Holofunk is an improvisational way to make music with your voice and your body, almost like a symphony that you both create and conduct.

Holofunk integrates Kinect body tracking for gesturing, a Wiimote for clicking and picking, the BASS audio library for audio processing, an ASIO sound driver for low-latency recording, the Turnado VST plugin for ridiculously powerful sound effects, and SharpDX for a fast and attractive interface.

Go here to see an early version in action.

Current Features

  • Low-latency pass-through of microphone audio.
  • Squeeze Wiimote trigger to start a loop; release it to let the loop start running.
  • Loops appear as colored circles, that throb according to the loop's volume.
  • Loops are quantized to fit the current tempo, so everything syncs up.
  • Press the Wiimote minus button and wave over sounds to mute them and turn them grey.
  • Press the plus button and wave over sounds to unmute and re-colorize them.
  • Press minus over a muted loop to delete it altogether.
  • Two-player support -- jam with a friend!
  • Two-monitor support -- show the audience an audience-eye view (on a projector or other monitor), while you watch a mirrored view of yourself on your laptop.
  • SharpDX integration -- Windows 8-friendly access to the full DirectX 11 API (which I do criminally little with, yet!).
  • VST support with Turnado integration and gestural effects control -- stupendous audio processing power, the surface of which I've again barely scratched.
  • Hold Home to pop up a menu with various commands (wipe all sounds, switch players, etc.).

Background and Contact Info

For more background on Holofunk's inspirations and ambitions, see this blog post: http://robjsoftware.org/2011/07/12/first-steps-on-the-road-to-holofunkiness/.

If you are interested in future developments, please consider joining the Holofunk facebook group.

Click here to browse the latest source. 

This project is licensed under the quite permissive Microsoft Public License, and I am very much seeking collaborators!  The potential of this project is pretty near limitless, and it is just getting started. 

In fact, it is so much just getting started that it currently uses various non-commercial components that I am not able to redistribute in product form -- specifically, the BASS audio library, which I am currently using under noncommercial terms, and the Turnado VST plugin, which I paid for.  So you may download this, play with it, and hack on it, but not sell it (without violating the license terms of some components)

Not daunted?  Check out the Setup and Playing instructions.  I have yet to put up binaries.

I am a Microsoft employee, but this project was done purely on my own time with no Microsoft corporate resources.

For a meta perspective, consider that Holofunk is a synesthizer.

What's Next

Oh, basically an infinite number of things.  Some obvious features that should be added:

  • Ditching the Wiimote, and using Kinect gestures for pure hands-free looping.  Kinect 2 looks like it supports enough hand poses to make this feasible; I will be trying it the second I can get my hands on a Kinect 2 beta SDK for Windows, HINT HINT GUYS ;-)
  • Ditching the USB audio interface and using the Kinect mike array.  The latency issues here are serious, but it could be a fun mode for fooling around, though latency is surprisingly critical even for beginning users....
  • Multiple skins because not everyone likes hippy dippy rainbows :-D
  • Converting the loops to use an FFT-driven frequency display.
  • Integrating the Kinect video stream so you see yourself, not just your shadow.
  • Gesture support -- warp your sounds with your body.
  • Collapsing sounds together, or splitting them apart.
  • Side-chaining and integrated compression and EQ throughout.  (I know very little about this, another massive learning area....)
  • Multiple interfaces for time distortion/scratching, MIDI sequencing, full-body capture, keyboard support, foot position support for guitar players....
  • Implement reusable sound buffers -- at this point, the amount of sound you could get into a single performance would be nigh limitless.
  • Importing multiple sound sources, integrating other instruments, integrating MIDI, bringing in multiple visualizers.
  • AND THEN SOME.

Now you can see why I'd love your help!

 

Last edited Jun 6, 2013 at 6:42 PM by RobJellinghaus, version 19