Perfect timing. I'm expecting to get my hands on a MOTU MIDI Express XT from my local Guitar Center within the next couple days (I paid for it when it arrived there a couple weeks ago, but they have a mandatory waiting period on used equipment to make sure it ain't stolen), which unfortunately uses some weird proprietary protocol instead of class-compliant MIDI-over-USB — so any use over USB from my PCs (nearly all of which are running Linux, OpenBSD, Haiku, or something other than Windows or macOS) is a no-go. This is okay for my immediate use cases (I just need it to route between some synth modules and controllers, without necessarily needing the PC to do any processing in-between), but it'd be cool to get the PC side of things working, too.
There's an existing out-of-tree Linux driver¹ that looks promising, but AFAICT it only does the bare minimum of exposing the MIDI ports for use with e.g. JACK, and it's also unclear how stable it is and whether it really does support the XT (the README says the kernel panic got fixed, but there are open issues about it; the README says the XT's supported, but there are open issues about that, too). I'd like to be able to create new routing presets and stuff like what the proprietary companion app can do, and I'd also like to be able to use the thing without needing to shove extra drivers into my kernel, and I'd also like to be able to use the thing on my OpenBSD and Haiku boxen, so I've been perusing LibUSB docs since a userspace USB driver that then presents the relevant MIDI ports and some tooling to reroute the MIDI ports as desired seems like something useful. This article happens to be exactly what I've been looking for w.r.t. a starting point for such a userspace driver.
I packaged that driver on the AUR, since I’ve got the same device. I can’t get the binary blob to work (admittedly, I haven’t tried very hard) and it’s not high on my list of priorities so I’m okay using it as a dumb MIDI tool.
But this kinda expects that your USB driver is the application code too, no? This is less of a driver and more of a library + program. If I have, say, a USB to Ethernet device, how do I hook this into the ethernet adapter subsystem?
Things which are relatively standard tend to get good generic support: Ethernet devices will generally be USB/CDC/ECM or RNDIS, for example. That may Just Work (tm) if it has the right descriptors.
The userland approach is much more useful for weird or custom devices. In particular, on Windows you can do one of these user space "drivers" without having to bother with driver signing, and if you use libusb it will be portable too.
DFU - great example. If you have a USB device that has a DFU class that needs a custom driver, can dfu-util and the like hook into these userspace drivers? Or do you also need to maintain the application part?
Dfu-util is one of those "user space drivers", so if you have a nonstandard protocol you'd have to add it directly to dfu-util. There's no intermediate API.
It's not easy to set up a fake or "remapped" USB device on most OS as far as I'm aware, if you were trying to write an adapter program that modified USB packets.
dfu-util actually also just uses libusb under the hood!
Any class or device that doesn't have a driver baked into the OS can be implemented like this. And if you'd need the DFU functionality in a different application, you may be able to just simply link parts of the dfu-util tool into your application
On Linux you could create a tun/tap device from your application and translate data sent over that to requests sent to the ethernet adapter.
Of course, when you're doing these things in userspace you either need some way of communicating with the Kernel or for the other subsystems to be in userspace as well.
Not to be too facetious but a great place for communicating with the kernel where there are a ton of other driver subsystems is... the kernel.
Possibly a good addition to the article would be parallel development of an lkm. I guess it wouldn't have that windows interop but I would also be interested to see how this driver would be implemented on Windows. If it's idk 10x as many lines in the kernel vs userspace, that's a great benefit to the userspace approach.
In HFT user-space networking drivers have a long history - there is too much latency induced by switching from kernel to user space to handle networking.
> OpenOnload: A user-space network stack that intercepts socket calls to bypass the kernel network stack, accelerating standard socket operations for faster networking.
> Netmap: A framework providing a simple API for high-speed packet I/O in user space, bypassing much of the kernel overhead for efficient packet forwarding and filtering.
Really useful introduction! Working with low-level hardware APIs is challenging but rewarding. The abstraction layers in modern OS make it easier but understanding what's underneath is invaluable.
Ages ago when I was trying to create a simple USB device, I found that there is very much zero information how to do it - e.g. how to correctly write descriptors and so on. The typical advice was: find similar device to what you want to make, copy its descriptors and adapt to your own device using trial and error.
Sounds like USB is a wonderful standard. Am I wrong?
I use this to access UVC devices correspondingly without cgo: https://github.com/kevmo314/go-uvc
There's an existing out-of-tree Linux driver¹ that looks promising, but AFAICT it only does the bare minimum of exposing the MIDI ports for use with e.g. JACK, and it's also unclear how stable it is and whether it really does support the XT (the README says the kernel panic got fixed, but there are open issues about it; the README says the XT's supported, but there are open issues about that, too). I'd like to be able to create new routing presets and stuff like what the proprietary companion app can do, and I'd also like to be able to use the thing without needing to shove extra drivers into my kernel, and I'd also like to be able to use the thing on my OpenBSD and Haiku boxen, so I've been perusing LibUSB docs since a userspace USB driver that then presents the relevant MIDI ports and some tooling to reroute the MIDI ports as desired seems like something useful. This article happens to be exactly what I've been looking for w.r.t. a starting point for such a userspace driver.
----
¹: https://github.com/vampirefrog/motu
The userland approach is much more useful for weird or custom devices. In particular, on Windows you can do one of these user space "drivers" without having to bother with driver signing, and if you use libusb it will be portable too.
(I maintain a small USB DFU based tool for work)
It's not easy to set up a fake or "remapped" USB device on most OS as far as I'm aware, if you were trying to write an adapter program that modified USB packets.
Of course, when you're doing these things in userspace you either need some way of communicating with the Kernel or for the other subsystems to be in userspace as well.
Possibly a good addition to the article would be parallel development of an lkm. I guess it wouldn't have that windows interop but I would also be interested to see how this driver would be implemented on Windows. If it's idk 10x as many lines in the kernel vs userspace, that's a great benefit to the userspace approach.
Much easier to design the device to avoid that. E.g. by abusing USB-HID. The desktop USB missile launcher toy is USB HID, for example.
There are quite a few benefits to doing these things in userspace over the Kernel, not really necessarily just because of the code size:
- The code is much easier to write and debug, you just write code like you always would.
- Bugs don't have the possibility to taking down your entire system or introduce vulnerabilities
- Especially on Windows, everyone can do this without requiring an impossible to get driver signing certificate
> OpenOnload: A user-space network stack that intercepts socket calls to bypass the kernel network stack, accelerating standard socket operations for faster networking.
> Netmap: A framework providing a simple API for high-speed packet I/O in user space, bypassing much of the kernel overhead for efficient packet forwarding and filtering.
https://dysnix.com/blog/high-frequency-trading-infrastructur...
Sounds like USB is a wonderful standard. Am I wrong?
Hand it back, with a request to prove that it can't be done with one of the devices that the OS's already recognize as virtual COM ports.