Project Description
A simple yet highly configurable Kinect-based MIDI controller with MIDI playback, gesture recognition and voice control.

This is a simple Kinect-based MIDI controller inspired by such projects as KinectAR, Kinect FAAST and Peripheral MIDI controller.

Current features
  • Four simple gestures for each hand
  • Voice command control
  • MIDI files playback
  • Basic MIDI messages: Note on/off, volume change, pitch bend
  • Configurable MIDI output
  • Configuration files which can change gesture and voice recognition behavior, MIDI messages, note scales and velocity.
  • Fast save/load functions for configuration and MIDI files

Important notes
  • For MIDI playback, MIDI file should contain two instrument tracks + one tempo track and be converted to CSV file via midicsv command line tool
  • Velocity change commands are available only for MIDI playback
  • To rig the controller to any DAW, you can use virtual MIDI port, for example LoopBe or MIDI Yoke. I used LoopBe

Libraries and frameworks
  • Gesture and voice recognition are provided by KinectToolbox (included in project)
  • MIDI library: midi-dot-net (included in project)
  • Official Kinect SDK v1 and higher (not compatible with beta versions)

Examples and demos
  • Video demos are available on my YouTube channel
  • Class reference and guides will be ready a bit later

and here's how it looks:


This project is currently under development and most likely contains many bugs. There's no thorough documentation right now and the code is a bit messy, but I'll be happy to answer all your questions regarding this project :)

Also, initial repository on github:

Last edited May 19, 2012 at 7:06 AM by apolotary, version 10