Some Limitations of the Game Boy

using C and GBDK

Jeff Gensler
10 min readSep 27, 2024

A few months back, I participated in the Game Boy Advanced Jam (2024) and created a visual novel as my submission. This was a fun time to build a game as I had never done something like that before. I had dabbled in a piece of software that is used to create simplified RPG-style games (RPGMaker 2003) and I understood how hard the art is (at least for a beginner). I figured the Visual Novel would simplify a lot of the technical requirements and would also allow me to focus on an engaging story, something that would be reminiscent of the anime I currently watch. I was able to finish the game largely because of a cool C++ game framework build specifically for the GBA: butano. On the tails of this hackathon, I was curious about going one step lower: the original Game Boy! fortunately, the Game Boy jam was just around the corner. What a great opportunity! I took a look at the some of the previous years submissions. Although many were built on GB Studio (a popular tool that eases the game design a build process specifically for Game Boy), there were many, many games built on Unreal, Godot, and Unity. The game jam requirements were only in screen size (160x144 px) and color palette (4 colors MAX!). This is a great limitation and enables many more people to contribute their ideas. Surely, there would be something as easy as butano for the Gameboy! How hard could it be?

Gameboy Technicals

Unlike the GBA (which uses an ARM processor), the Game Boy was built on a processor similar to the Z80. From Wikipedia:

Within the DMG-CPU, the main processor is a Sharp SM83,[22] a hybrid between two other 8-bit processors: the Intel 8080 and the Zilog Z80.

What this means in practice: I can’t use the devkitARM compiler or butano to compile code for the Game Boy. I’m on the hunt for another tool chain!

Now, there are people who will write their game in asembly (both for the GBA and for GB). However, I don’t have much experience with that and I can’t imagine being very fast with it (or essentially just re-inventing C calling conventions). Fortunately, there is a project (with quite a long history!) called GBDK. This project includes a compiler, linker, and a few other very helpful tools for compiling our C code into a Game Boy Rom. There is even a way to compile the code to other game systems: “Analogue Pocket, Mega Duck, Master System and Game Gear and NES”. Impressive!

Hello World

Naturally, the first thing to do is get the examples working and verifying our toolchain is setup correctly. There are a mix of examples: some “cross-platform” ones that show functionality common to all systems, and some “gb” ones specific to the Game Boy.

I think the two that come to mind are the following:

  • font example (link): you’ll note that the font is installed in the background data. (in Butano, this is done on the sprite layer)
  • galaxy (link): some code loading an image

Check out the limitation source code!

The following examples are generated using the following code:

Limitation: Sprites and Animation

Knowing that I wanted to create a story game similar to “Work Life,” I started by creating some simple (large) character sprites in Aseprite. I knew that the total number of sprites was limited so I was curious if I could animate using background tiles and leave sprites for menus and such. The short answer is no, you can’t. The long answer is “it depends” or “sometimes.” Here is an example of what happens when you try to animate (via loading a whole new set of data + rewriting the bg map every animation frame):

left from OBS, right from mGBA internal tool. The screen tearing happens on my GB emulator on my DSi, too
tile data on left, tile map on right. on mGBA: go to “Tools” > “Game state views” > “View tiles”

As you can see, there is a nasty “tear” at the bottom of the image. Now, the code I wrote isn’t sophisticated. Usually, you wait until after a Vertical Blank Interrupt (see this function) happens to begin changing any video buffer data (in the GBAs case: data itself or map values). Adding the vysnc call before our switch statement doesn’t seem to help much. So what is going on here? I believe most of the issue (and also partly why it “works”) is that a portion of the tile data “moves” back and forth and cases a large mapping discontinuity (a tile that used to point to a shoulder now points elsewhere, etc). this would explain why the top or beginning part of the image appears to animate smoothly. If a unique tile ends up with the same tile index/data location, this lets the animation appear quite smooth (likely because tears between the top and bottom of the picture could still happen but not necessarily cause visual discontinuity). Would I recommend this approach even if you could find a way organize tile data to accommodate “pleasant” screen tears? no, not really. Is handling this on the sprite layer any better, technically speaking? probably not, if we are just copying data from ROM to RAM, what difference does it make if its BG RAM or Sprite RAM? well, I’m no expert, but I’d say the moral of the story is:

  • keep the data copying to a minimum
  • try to optimize for unqiue tiles, save some space
  • let backgrounds be backgrounds for the sake of simplicity

Here are some more docs from the GBDK Wiki:

Ensuring Safe Access to Graphics Memory

There are certain times during each video frame when memory and registers relating to graphics are “busy” and should not be read or written to (otherwise there may be corrupt or dropped data). GBDK handles this automatically for most graphics related API calls. It also ensures that ISR handlers return in such a way that if they interrupted a graphics access then it will only resume when access is allowed.

The ISR return behavior can be turned off using the nowait_int_handler.

Limitation: ROM Banking

If you’re making a visual novel like I did, you’ll split the character sprites into the background base and sprite (aside: these are sometimes called lip or mouth flaps in animation link1 link2 ). Great!

The next problem you might run into is the size of your game. By default, the GBDK toolkit will not use any Memory Bank Controller (MBC) and you’ll get 32K bytes of ROM to put your code and assets. For reference, the five frame animation above uses the following (about 40% of that space):

$ romusage.exe .\bazel-bin\1_bg_animation\rom_2.gb

Bank Range Size Used Used% Free Free%
-------- ---------------- ------- ------- ----- ------- -----
ROM_0 0x0000 -> 0x3FFF 16384 944 6% 15440 94%
ROM_1 0x4000 -> 0x7FFF 16384 11898 73% 4486 27%

If you have an ultra small game, maybe you can get away with this default. It would also be cheaper to build and sell physical copies of your game if it could fit in that size! However, it is useful to plan for using an MBC earlier on because all of the code that ended up in ROM1 will not be (directly) available from code in the following banks (1–255). I won’t go into detail on how the memory banks work. All you need to know is that banks 1–255 are addressed with the same range (0x4000 -> 0x7FFF) and are “swapped” to and from using SWITCH_ROM(bank_num) .

ROM0 is special, cherish it! I ended up pushing as much rending logic into functions that would live in various ROM banks to make space in ROM0. I STILL didn’t have enough space to fit hUGETracker. Probably a skill issue or “bad” code, but if it can happen to me, it can happen to you!

Aside: music with hUGETracker

The recommended music/tracker library with GBDK is hUGETracker. By default hUGETracker uses ROM0! here was the before/after when I removed the music feature from my app to make space in ROM for more application code:

before (hUGETracker lib and one 1-bar song):

ROM_0        0x0000 -> 0x3FFF    16384    12640    77%     3744    23%

after:

ROM_0        0x0000 -> 0x3FFF    16384     9486    58%     6898    42%

As you can see, this used around 19% of ROM0! For the feature set that hUGETracker provides (like various and multiple sound effects), this may be extremely impressive. However, it is worth noting and keeping an eye on.

It is possible to build this from source (I did when I was debugging some music issues) so it may be possible to bank it. Part of their codebase is assembly so don’t expect to rely on the GBDK C macros to fix it. I am not sure how much space the individual song takes up but banking their code would also mean banking ALL music tracks alongside it. I suppose you could have multiple copies of hUGETracker and songs in separate banks… well, you can decide!

Limitation: Large Screen Swaps

In the case of a visual novel, you may not be able to get around large screen changes. This is especially challenging when the screen change is not a cutscene (so fades aren’t really an option). Example: two characters talking to one another.

My cheap trick for large screen swaps to show the window over the background during the swap. Note that sprites appear on TOP of the window so now the eyes, mouth, hair are without their body. spooky! you can use HIDE_SPRITES which turns off the entire sprite layer. You may also consider hide_sprite(…) though it isn’t the best tool in this case.

Here are some of the effects in action:

top left: no technique (fast, some discontinuity), top right: window flick, bottom left: fill entire bkg map 254, no window usage

Note that the window effect feels a bit less pronounced when you are pressing the button that triggers the screen change. I figure this is kind of like driving a car vs riding a car and feeling car sickness. Obviously, we would love to do without the flicker at all, so keeping all or most of the background tiles between scenes to speed up render is ideal.

Limitation: Sprite Limit

left: 8x8 mode + all on same line, center: 8x8x mode + move every other 2nd down by one pixel, right: 8x16 mode + move

I think the pictures do this one justice but you only get 10 sprites per scanline. this is also the case in 8x16 sprite mode. the example can also help illustrate how the 8x16 sprite mode works when interpreting sprite data in RAM. I made use of this mode as I had a LOT of sprite pixels to draw for character animations.

Aside: Sprite Optimization

The other problem I never got around to solving was removing empty tiles entirely. Although the map reduces a single empty tile to one place in ROM, our code still need to be smart enough to never render empty sprites and use up a vital sprite slot. By default, -map will export sprites in 8x8 mode so you don’t get fancy de-duplication for 8x16 tiles. Your custom sprite loader will need to read the 8x8 map and assemble the 8x8 data in 8x16.

This thinking animation uses up 25 of the 40 sprite slots, though a good portion are empty. Moreover, there may be many sprite in one scene that use the same tile (think a solid green tile) so the optimizations could span across entire sprites (like two characters in the same scene).

8x16 sprite mode: showing “Empty” sprites

I didn’t explore some of the Metasprite framework. Perhaps this is fixed with that tooling (not using the -map command in png2asset ). I also wonder if the empty sprites, like the background tile example above, allow me “gracefully tear” the sprite layer if a render pass happens while I am mid sprite frame copy.

Limitation: Build Times

Near the end of my game, I had close to 300 pngs each with its own C file. I believe this was causing some sort of slow down in the compilation process as the build time started to reach 1–2 minutes! I never really nailed the culprit, though I do know my Bazel shell build process definitely would recompile unchanged files (like unchanged images). This is usually fixed by having separate targets for intermediate (.o) files but lcc handles more than just compiling and linking, so I never went back to fix this. You might consider copying the Makefiles that include intermediate compilation if you want to avoid having this issue.

Final Remarks

Overall, this was a fun jam to learn about the Game Boy and build a story-like game. It was interesting to see and feel the limitations of the Game Boy. Overall, I am extremely impressed with the Game Boy even with these limitations. To think one of the first handheld gaming devices could STILL be used to render something rich like a pseudo-visual novel is really impressive. I am glad there are open source projects that let me learn and build on the Game Boy. There is really good documentation out there describing more hardware specifications and additional MBC considerations.

--

--

No responses yet