Cover

TL;DR

  • The Open DMG Display firmware now supports video output via USB Video Class (UVC).
  • Utilizes an MJPEG approach adapted from the GB Interceptor.
  • New firmware available at link.

Introduction

It has been a while since I published the Open DMG Display. Since then, I’ve tinkered with the display signals of the entire Game Boy lineup, which resulted in an open display for the Game Boy Advance. I don’t know who it was, but someone asked whether it would be possible to add capturing to this device, and that got me thinking about an implementation.

I remembered the talk by Sebastian Staacks at 37C3 about his GB Interceptor. I came to the conclusion that it wouldn’t be worth trying to implement this directly on the RP2350 (especially after asking Sebastian whether he thought it was possible). So I moved on to experimenting with an ESP32P4, which worked well. Since I also wanted to get familiar with the TinyUSB library, I tried getting capturing to run on a cheap CH32V305, which also worked.

What I totally overlooked was that the GB Interceptor approach could be adapted to the Open DMG Display, something I’m hereby catching up on.

Implementation

Until now, the Open DMG Display only used one of the RP2350’s cores, leaving the other free for USB video. The implementation relies heavily on the TinyUSB library for the core USB functionality, while the rest adapts the MJPEG approach from the GB Interceptor.

RAW frames are captured using the existing code. Once a frame is complete, it’s copied into a separate memory region for further processing, which ties USB output to the Game Boy’s frame timing. In 30 FPS mode, every second frame is skipped. Lookup tables convert the 2-bit-per-pixel raw data into 3-bit-per-pixel data as a preliminary step before JPEG encoding.

Unlike Sebastian Staacks’ approach, no PIOs are used, the dedicated core has enough resources to handle everything in software, including JPEG generation via lookup tables.

The adapted method from Sebastian and Ikadzuchi reduces JPEG quantization to the lowest frequencies, keeping mostly luminance. Huffman tables and other parameters are tuned to efficiently encode 3bpp grayscale images using 5 bits per pixel plus JPEG overhead, which includes unused chroma. This allows 30 FPS over a 12 Mbit USB link.

For 60 FPS mode, chroma is skipped entirely, transferring only luminance. This can cause compatibility issues with some software.

For a deeper dive, check out Sebastian’s talk.

Usage

Config menu of the Open DMG Display showing options for USB Video output.

The USB video output is disabled by default. To use it, you need to enable it in the menu. Look for “USB Video Mode” and choose between:

  • OFF
    • The system will not initialize the USB port, and the second core will remain inactive.
  • 30fps
    • This mode produces approximately 29.85 FPS. Every second frame is skipped to keep the data rate well below 12 Mbit.
  • 60fps
    • This mode may have compatibility issues. It has been tested with QV4L2 and OBS, only OBS handled the signal correctly.

After selecting a mode, restart the device. It should now be detected like a webcam or a USB microscope.

Thanks

Files