At the heart of every video game is a graphics library. In order to find out what it does, it is worth having some background knowledge of hardware accelerated rendering. This can serve as an introduction to what we are going to talk about in Game Development Days.
Real-time rendering
Modern games are massive software solutions where monstrous pieces of information are processed and composed into images in extremely short periods. Drawing of sophisticated characters and scenes in a game requires a lot of processing from the computer’s perspective, and a vast knowledge of the mathematics involved by the developer. The process of converting definitions of a scene (objects, lights, etc) into images is called rendering. Rendering can be a lengthy task. You might have seen stunning photos on the internet that, although they are created using graphical software, but look surprisingly realistic. Take a look at the following example:
Knowing that this is a synthetic image and not a photo may fascinate the observer. What cannot be grasped from these images is that some of them might have taken hours to “render”, depending on their quality, resolution, and the power of the hardware that rendered them. Now you can imagine how long the rendering of a Disney animated movie might have taken. For applications like still art and animation, time is not that crucial. What is required is only the final picture or video, regardless of how long it takes to be produced. But when it comes to video games, images need to be rendered immediately. If you are a gamer, you know that a desirable rendering rate is 60 frames per second and anything less than this does not feel completely natural. But this means 60 images rendered in a single second! This is called real-time rendering. In other words, when the rendering of images is fast enough that a user can actually interact with the scene and see the results immediately (as in a video game), it is called “real-time”. Real-time rendering is a challenging task. Remember that the above photo could take a long time to render by a piece of software. So, the question is:
How can a game scene with many objects and effects be rendered in one sixtieth of a second?
Part of the answer to the above question is in the design of a rendering technique. The other part is in the capabilities of the hardware that renders. We will get to it very soon. You might now realize why characters used to be very spiky in games even 5 years ago and you can still see simplicity in various parts of a game’s world today. That is mainly to reduce the amount of information to be processed, in order to speed up rendering. But these requirements are getting dimmer and dimmer as gaming hardware grows more powerful. Nevertheless, the choice of how and when to simplify parts of a scene to speed up rendering, while keeping the scene natural, is a huge part of game design, and it is still considered even in state-of-the-art games. What you see at the horizon in World of Warcraft is not a city. It is a flat shape mimicking the shape of the city from far far away. Even when you get a bit far from a point, it is no longer rendered the same way it would if you were close to it. As you play a game, objects are constantly getting simplified or detailed for one very reason: keep frame rate as high as possible while giving you an acceptable image of the scene.
Hardware acceleration
As mentioned in the previous section, real-time rendering is an essence for a game. You cannot have a game that takes one hour to render each frame. Even one tenth of a second is unacceptable by gamers. Software applications have some limitations. First of all, software is run on a local CPU, which is already busy doing other tasks. It is the heart of your computer, so of course it has more work than to render your game’s frames. The memory on your machine is also limited. When it comes to mobile phones, this is even more critical. When your game loads, probably most of the memory on your machine is already taken by the assets of the game (graphics, sounds, etc) and there is very little space left for rendering computations.
Games are getting more and more demanding. It has been quite a while since hardware manufacturers realized that the global resources of a computer are not enough for games. They have created another set of processing resources dedicated to real-time rendering. Your graphics card today has a CPU of its own. They call it the GPU (Graphics Processing Unit). When you run a game, the GPU does all the massive rendering computations, while your CPU is busy handling artificial intelligence or physics. Some graphics adapters even have physics processors to remove even more load from the CPU. NVIDIA was the first to introduce GPU. Your graphics adapter has its own RAM too. My first personal computer had 128 megabytes of RAM. My current computer’s graphics card alone has 2 gigabytes of RAM. 2 gigabytes of memory dedicated only on taking my game objects and producing images. My GPU is more powerful than the CPU on my first PC. Of course one could still use the CPU and the machine’s RAM to do everything. But considering what graphics adapters can do, it is a waste of resources.
This is were the meaning of hardware acceleration reveals itself. It is simply the handing the tasks that the GPU can do the best to the GPU and thus save loads of memory and CPU time. Today, hardware acceleration is not limited to desktop machines or even laptops any more. Some smartphones today have GPUs or hardware accelerated rendering capabilities as part of their CPU. The development of such hardware shows how huge the gaming industry has become and can be a motivation for anyone to enter this business.
Today, many hardware manufacturers produce graphics adapters and there are many in the market. Intel, NVIDIA and AMD are prominent examples. Each of the different products, while having the same purpose, have different capabilities and characteristics. One should be familiar with their GPU specifications to be able to program with them. To put it simple, handing your task to an NVIDIA video card can be completely different than to an AMD video card. It is like accountants that speak different languages. They all do the same thing, but you need to know the language to speak to all of them. This could introduce a challenge to game developers. Because they want their game to work on all computers, regardless of what hardware is installed on them.
Open Graphics Library
Having said what we have said, it is now evident why we would need a graphics library. Real-time rendering is a tricky task and requires highly efficient programming. The programmer not only needs to know how to write a code that is the fastest, they often need to step further and manipulate the assembly code to use specific CPU instructions that are slightly faster than the ones automatically generated by compilers. After all, we are talking real-time, and you do not want the bottleneck to be your code. In this regard, all games have similar requirements. At the lowest level, they all need to combine pictures and models into scenes and render frames. These tasks can be pre-written once in the most efficient way possible, and then reused for all games.
Besides, hardware compatibility is a bigger pain. Imagine you have implemented the most efficient way of rendering your scene on an NVIDIA video adapter. This code not only does not work on an AMD card efficiently, it most probably won’t work at all! It would be a relief to have a single agent that knows all graphics cards and can communicate with all of them. Then you can simply interact with this agent and be sure that your code runs similarly on all the hardware, because you know that end is covered by the agent. This agent is the graphics library. Examples of graphics libraries are OpenGL by SGI, and Direct3D by Microsoft.
Open Graphics Library (OpenGL) was first released in 1992. It is an abstract application programming interface (API) for 2D and 3D rendering. It is designed in a way that it can be run both on software (through CPU) or completely on hardware (GPU). The API described in OpenGL provides a complete set of functions and definitions one needs for real-time rendering. It even provides some features to simplify images when needed in order to increase frame rate.
OpenGL is language independent and platform independent. Once you get familiar with OpenGL, you can code in any programming language for which OpenGL is available (which is the case for almost all of the prominent programming languages). Besides, the platform independence means that it is available on all software platforms (Windows, Linux, …).
OpenGL ES
As we saw previously, OpenGL is a cross-platform API. OpenGL ES, which stands for OpenGL for Embedded Systems, is a subset of the OpenGL library that is designed for smartphones, tablets and other embedded systems. It has additional definitions to support the shortcomings of CPUs on those devices, and lacks some of the features available to OpenGL for personal computers. However, the API is powerful enough to render demanding 3-dimensional games on embedded devices. As an example, here is a screenshot of the game Asphalt running on a Nexus series tablet, using GPU acceleration:
I will not go into more detail today. But if you find something not clear enough, please feel free to leave a comment. And if you are ready to proceed, please click on the link below to go to Day 2!
