Blog

JUCE 8 Feature Overview: Direct 2D

Direct2D is the new default renderer for JUCE on Windows. The Direct2D renderer was nursed back to life by Matt Gonzalez and brought over the finish line in a collaboration with the JUCE team.

What's new in JUCE 8

  • The Direct2D renderer is now the default JUCE renderer on Windows.
  • Both JUCE desktop windows and JUCE images are now GPU backed on Windows.
  • The old "software renderer" remains an alternative on Windows.
  • Built on modern native platform APIs, Direct2D brings significant rendering and performance improvements to JUCE 8.

Breaking changes and deprecations

  • JUCE 8 now only supports Windows 10 and higher.
  • When constructing new juce::Graphics objects, note that the Direct2D renderer (like the OpenGL renderer) will not write to the image or window until the Graphics object goes out of scope.
  • When manually writing data to an image with juce::Image::BitmapData, the actual pixels will not change (and therefore cannot be read) until the BitmapData goes out of scope and the GPU writes to the image.

See "Working directly with Graphics objects" below for more details on the last two items.

Definitions

  • Discrete GPU. A GPU separate from the CPU with its own dedicated video memory not shared with the CPU. For example an NVIDIA GPU card.
  • Integrated GPU. A GPU embedded alongside the processor, does not come on its own card and typically shares RAM with the CPU.
  • Renderer. The entity responsible for converting JUCE's drawing commands into bitmaps. Depending on the platform and renderer chosen, it may or may not be hardware accelerated by the GPU. For example, there's the software renderer and the OpenGL renderer.
  • Rasterization. Taking vector graphics and converting them into a bitmap-like format (a 2D image with pixels).
  • Tessellation. The process of covering a surface with geometric shapes (tiles) with no overlaps and no gaps. In Direct2D, tessellation is the process of decomposing a 2D area into triangles.
  • Texture. 2D Image or Bitmap living on the GPU.
  • VRAM or Video Ram. Stores textures and other graphics data to be used by the GPU.

What is Direct2D

This is how Microsoft describes Direct2D:

Direct2D is a hardware-accelerated, immediate-mode, 2-D graphics API that provides high performance and high-quality rendering for 2-D geometry, bitmaps, and text. 

In layperson's terms: it takes the stuff you want to draw, preps and translates it into queued calls for later rasterization on GPU. For example, it might take a line or font shape, chop it up into triangles and send that to the GPU.

Source: Charles Petzold's article on DirectX triangle tessellation.

Direct2D is a black box in the sense that the dividing line between GPU and CPU work is blurry. in JUCE, lots of effort went into optimizing things that seemed harmless (such as Graphics#setTransform) which initially happened to be CPU heavy in Direct2D's paradigm.

JUCE Windows and Images are now GPU Backed on Windows

Issuing commands like g.fillAll() take immediate effect when using the software renderer. The pixels on the context's image are immediately modified.

The new Direct2D renderer on Windows works differently. It preps and queues a list of deferred draw commands to be later executed on the GPU. The resulting image is then stored as a texture (living temporarily on the GPU in the VRAM).

Code running on the CPU cannot directly access the individual pixels in VRAM. There's a cost incurred when taking pixels from the GPU and syncing to the CPU, so this is done as infrequently as possible.

Using juce::Image::BitmapData also syncs the pixels between the GPU and CPU, and therefore its usage comes with a cost. For this reason, avoid using Image#getPixelAt in a loop, as it will sync per-iteration.

Working directly with Graphics objects

If you are manually constructing juce::Graphics objects, be advised that the image data is stored on the GPU when using the Direct2D renderer.

Let's take a look at filling an image with white pixels:

juce::Image result (juce::Image::ARGB, 9, 9, true);
juce::Graphics g (result);
g.fillAll (juce::Colours::white);
jassert (result.getPixelAt (2, 2).toDisplayString (true) == "FFFFFFFF");

The fill occurs immediately on the software renderer. However, the above code will assert on the Direct2D renderer: fillAll queues instructions for the GPU, but those instructions won't run until juce::Graphics goes out of scope, at which point the whole draw call queue runs.

In tests, or other situations where you need immediate pixel access, you can wrap the graphics call in a scope to force the render:

juce::Image result (juce::Image::ARGB, 9, 9, true);
{
    juce::Graphics g (result);
    g.fillAll (juce::Colours::white);
}

The juce::Graphics object passed to your components stay "in scope" throughout all component painting. That means drawing setup occurs during paint calls, but rasterization does not.

It's always recommended to destroy the Graphics context before accessing the underlying image. It's not recommended to use the SoftwareImageType as an alternative to scoping — its behavior (immediately modifying the underlying image) isn't guaranteed to remain true in the future.

Specifying SoftwareImageType

As we have seen, images are GPU backed by the Direct2D renderer.

There are rare times when you might want to explicitly create a juce::Image stored on the CPU (not GPU). For example, when working with images not shown onscreen or modified by a Graphics context (in which case the benefits of being GPU accelerated are minimal). Or when needing to guarantee pixel-perfect image modifications across platforms (at the cost of performance).

To accomplish this, you can specify juce::SoftwareImageType() in the constructor for a JUCE image. You can also convert between image types with juce::ImageType::convert, like so:

myImage = juce::SoftwareImageType().convert (myImage); 

Toggling between renderers

While migrating to the new renderer, you might find it advantageous to toggle between the Direct2D and the old software renderer. You can accomplish this on a per-window basis at runtime with getPeer()->setCurrentRenderingEngine.

Note that getPeer returns nullptr until the component has been added to a platform window via addToDesktop(). Therefore it's often not possible to configure the renderer from within a component's constructor. Instead, wait until the component has been added to a peer, which can be detected by overriding the parentHierarchyChanged callback. An example can be seen here.

Direct2D Performance

One of Direct2D's primary advantages over the software renderer is raw performance. In a large part, this is because rasterization has been moved off the CPU and onto the more efficient GPU.

This also means GPU performance now becomes relevant on Windows. One can track this with Window's Task Manager, or download Microsoft's official Process Explorer app which lets one inspect texture memory usage.

It's up to the operating system which GPU (integrated or discrete) will be used when multiple are present. Please measure on your own machine. At this time there's no way to guarantee a computer's discrete GPU (vs. integrated) will be used by JUCE's draw calls.

Performance should improve on the vast majority of machines. Note that it is possible for some older machines with integrated GPUs only (no dedicated VRAM) to not perform as well on Direct2D as the software renderer.

To see the Direct2D renderer's performance improvements in action, check out the GraphicsDemo.h in JUCE's DemoRunner. It features the ability to switch between renderers in Settings and reports on timings. Both the component transforms and tiled image fills should demonstrate the biggest wins that JUCE's Direct2D renderer brings.

Direct2D "Golden Rules"

  • In general (with any renderer), when drawing at high FPS, the same principles as real-time audio tend to apply — don't lock, don't allocate.
  • Direct2D resources are expensive to allocate and cheap to render. Preallocate and reuse images and gradients. For example, not only does ColourGradient use an Array internally (which allocates heap memory) but Direct2D gradients are cached internally by the renderer itself.
  • Avoid manipulating image data directly (Image#getPixelAt, Image#setPixelAt, Image::BitmapData) as much as possible. Bandwidth within the GPU is enormous; bandwidth between the CPU and the GPU is narrow. Try to do as much as possible with GPU drawing operations. 

Direct2D technical esoterica

  • Direct2D backed images are ephemeral. This is because the graphics device could be disconnected or switched at runtime, at which point access to the underlying GPU image in VRAM is lost. To address these edge cases, JUCE stores a software backup image internally for every Direct2D backed image (not Direct2D backed windows). Having a backup image incurs a small cost but allows for the recreation of the image on the GPU.

More

linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram