Double Buffering with UP Video IP Cores

I’ve been playing around some more with the Altera University Program Video IP cores, and one of the things I had a lot of trouble with was getting double buffering to work properly. The concept is simple enough, but there are a couple subtleties that made it a little difficult.

Requirements

Quartus II, Nios SBT, and the Altera University Program files will be required. For the FPGA, I used a DE2 (Cyclone II), but you can use a DE0 for this just as well, or any board with VGA output. Note that the DE0 video DAC is limited to 4 bits per color plane, while the DE2 supports 10 bits per pixel. Finally, you’ll need an old VGA monitor to view the output.

Background

Before we begin, I’ll talk a little about double buffering and why it’s important. In video applications, a frame buffer is a dedicated region of memory that stores the data (pixel values) that will be output to the screen. The minimum size of the frame buffer depends on  the resolution and color depth: multiply the [#rows] * [#columns] * [bits per pixel]. For example, a full HD display, 1920×1080, at 32 bits per pixel, is around 8MB. Of course, modern GPUs have far more memory than this, on the order of Gigabytes, and “frame buffer” is often loosely defined as general video memory.

Let’s stick with VGA resolution, say 640×480. Chances are, your system is running a little faster than the VGA controller (25MHz), so if you’re writing directly to the frame buffer while it’s read & being displayed, you’re going to run into some problems like flickering or other drawing artifacts, since you’re writing new info faster than the old can be displayed! Also, don’t forget, you’ll have to clear the screen between drawings if you want any kind of animation, and setting 640×480 32-bit words to 0x00000000 will take quite some time, making the problem worse.

For this reason, you should not draw directly to the frame buffer. If you want animations, and have the memory for it, consider using a second buffer. Draw to this “back buffer”, then instruct the VGA controller to read from it once you’ve finished drawing to it. Meanwhile, begin clearing the other buffer, draw to it, and once you’re done, flip it again. That way, you’re never drawing directly to the screen,  and you’re pretty much in the clear.

Configuration

With the UP Video cores, there is support for double buffering, but I found it to be a little less-than-intuitive. The Pixel Buffer DMA core has a few options when instantiated, including  addressing mode, primary buffer base address, back buffer base address, x resolution, and y resolution, and color space.

First of all, be careful when setting up the primary and back buffer addresses. Make sure to assign base addresses to your Nios system first, and then use the assigned Pixel Buffer values. If you don’t have any other large memory cores (i.e. SDRAM Controller), Qsys will assign a base address of 0x000000 to the primary buffer, and by default, the same value for the back buffer. In other words, there is no back buffer by default. If you want one, assign another base address now, before things get messy.

In my case, I had the pixel buffer memory using the SRAM on the DE2. Since I also declared an SDRAM controller, the base address of the SRAM was 0x01000000, so I use this for the primary buffer base address. I then declare the back buffer base address at the halfway point, 0x0103FFFF (the SRAM span is 0x7FFFF, or 512KB). Generate your system, and remember, if you ever reassign base addresses, double check that the primary buffer base address is still valid.

Double Buffering with the HAL

In your program, the following code will allow you to write to a back buffer and flip it to the screen:

Note: this assumes you’ve successfully called open_dev on the pixel buffer device.

while (1) {

// clear the back buffer
alt_up_pixel_buffer_dma_clear_screen(pixel_buffer_dev, 1);		
// draw something to the back buffer
alt_up_pixel_buffer_dma_draw_box(pixel_buffer_dev, x0, y0, x1, y1, color, 1);

// call swap buffers
alt_up_pixel_buffer_dma_swap_buffers(pixel_buffer_dev);

// wait for the operation to complete
while(alt_up_pixel_buffer_dma_check_swap_buffers_status(pixel_buffer_dev) == 1);

}

The trick is, always draw to the back buffer by specifying the same number (0 or 1) in the last argument of the function calls. The Swap Buffers call will simply swap the base addresses of the primary and secondary buffers:

Directly from altera_up_avalon_video_pixel_buffer_dma.c:

int alt_up_pixel_buffer_dma_swap_buffers(alt_up_pixel_buffer_dma_dev *pixel_buffer)
/* This function swaps the front and back buffers. At the next refresh cycle the back buffer will be drawn on the screen
 * and will become the front buffer. */
{
register unsigned int temp = pixel_buffer->back_buffer_start_address;
IOWR_32DIRECT(pixel_buffer->base, 0, 1);
pixel_buffer->back_buffer_start_address = pixel_buffer->buffer_start_address;
pixel_buffer->buffer_start_address = temp;
return 0;
}

While this gets the job done, it can also lead to a problem that was a little tough to diagnose. There’s another function, alt_up_pixel_buffer_dma_change_back_buffer_address, which (as the name implies) allows you to switch the back buffer address. However, if you do this, then call swap buffers, the register containing the primary buffer address will be set to the new back buffer address, and subsequent program launches may result in both buffers having the same address!

To illustrate, consider the following. Say you neglected to set the back buffer in Qsys, but decide in your application to create one by changing the back buffer address:

Original Primary address: 0x00000000
Original Back buffer address: 0x00000000

1. Change the back buffer address to 0x0003FFFF
2. Clear/Draw to back buffer
3. Call swap buffers.

New Primary address: 0x0003FFFF
New Back buffer address: 0x00000000

At this point, all is well. However, the next time you run your application, your buffer address changes may persist, and as you are running the same code, the following happens:

Original Primary address: 0x0003FFFF
Original Back buffer address: 0x00000000

1. Change the back buffer address to 0x0003FFFF
2. Clear/Draw to back buffer
3. Call swap buffers.

Primary address: 0x0003FFFF
Back buffer address: 0x0003FFFF

Re-programming the board in Quartus fixes the issue for the first run, but after running the code twice, the issue arises again. I’d guess there’s a problem with the registers not resetting (hey, maybe it’s just my hardware, too!), and it’s simple enough to fix, but can be quite frustrating until you figure out what’s going on.

The easiest way to fix this is, of course, to regenerate your Nios II system, this time with the correct back buffer base address. That avoids the problem entirely, since you won’t have to call change_back_buffer_address.

If you’re lazy, and don’t mind a few extra lines of code at initialization, you can change the primary buffer address to what it should be before changing the back buffer address. However, the primary buffer address is (supposed to be) read only. You can, however, use the following work-around, which capitalizes on the source of the problem in the first place!

// check if the pixel buffer device has the correct primary buffer base address
if(pixel_buffer_dev->buffer_start_address != PXL_BUFF_BASE) {
// if not, write the correct primary buffer base address to the back buffer base address, since this register is writable
alt_up_pixel_buffer_dma_change_back_buffer_address(pixel_buffer_dev, PXL_BUFF_BASE);
// call swap buffers, which writes the back buffer base address to the primary buffer base address
alt_up_pixel_buffer_dma_swap_buffers(pixel_buffer_dev);
// wait for the operation to complete before moving on
while(alt_up_pixel_buffer_dma_check_swap_buffers_status(pixel_buffer_dev) == 1);

}

Now, you’re ready to assign your new back buffer base address and double buffer away. Again, it’s much simpler to just regenerate your Nios system, but this works just as well.

Of course, sometimes you might not want a back buffer at all, and you might not want to clear the screen between drawings:

But that’s just for fun :)