Refer to Teensyduino instructions to set up the development environment.
When compiling using Arduino IDE, remember to set the mode to "Serial + MIDI".
Note: the "Serial + MIDI" mode was added recently and as of this writing is only available in the beta version of Teensyduiono.
Also check out this template for using make and a text editor of your choice for comiling TeensyUSB code.