Vtubers are a phenomenon that has intrigued and perplexed me over the years. Cards on the table, I do enjoy the odd Ironmouse song (I’ve had Cry for Me WA WA WA consistently in my rotation since it dropped), but I’m hardly an expert. The tech often intrigues me more than the personalities—and now that a YouTuber has seemingly created the best Vtuber rig ever, I think I’m about to become the anime cat girl of my dreams.
Before I start shopping for 3D artists and riggers to commission, let me back up a bit. Rather than using AI to track a body in space, DIY hardware creator Made By Dennis went all in on optical motion capture, setting out to build ‘Hollywood Motion Capture from Scratch‘ (via Hackaday). Obviously, this technique is also heavily used in games, as you can see in the behind-the-scenes of God of War Laufey, where they did motion capture on a cube.
Actor Jack Quaid’s obvious professionalism aside, Made By Dennis briefly explains how multiple cameras are used in a mocap setup to track retroreflective markers through space via a process called Triangulation. Those multiple mocap cameras are often pretty pricey, but the YouTuber ends up making their own from scratch for less than $200 each.
Made by Dennis deploys a number of DIY touches to cut costs. Each camera uses the relatively inexpensive monochrome variant of the AR0234 sensor. As these cameras are going to be paired with an infrared light ring and only used to track highly reflective white dots wiggling around in space, a monochrome sensor makes sense for capturing clean motion data. Using one of these teeny tiny sensors has the added perk of pairing well with the cheap, mass-produced lenses made for surveillance cameras. The 6.8 mm sensor, plus this lens, costs about $40 total.
The YouTuber struggled to find an appropriate board to pair with the monochrome sensor for a price that wasn’t heavily marked up—so they ended up simply designing and making their own PCBs. Okay, I may need to put my Vtuber plans on hold while I go back to Design Tech class…either that or I crib from the PCB design Made by Dennis has shared via their GitHub.
(Image credit: Ironmouse)
These cameras are only one small part of the entire project, with Made by Dennis also going on to explain how images captured by the 16 budget cameras can then be processed into precise motion data. That’s a little over my head, but Made by Dennis has also shared their own custom solution on GitHub, and they say it’s 300 times faster than the leading open-source option, OpenCV.
As open source projects go, this is definitely one of the most labour-intensive builds I’ve seen in a bit. But considering you get an IR camera rig that captures 4 billion pixels per second at 240 fps, capable of capturing motion data with a less than 0.5 mm margin for error, I could maybe be persuaded to pick up the soldering iron (not to mention hassle my buddy with the 3D printer). Alternatively, Made By Dennis is also scoping out demand for prebuilt versions of the IR cameras via this Google Form. With tariffs and so on factored in, each individual camera would cost around $300—that would be one way to support an independent creator.
