Python 3 and OpenGL woes ..

Now that I have spent many hours researching and messing about to get some current OpenGL code working with the PyOpenGL library in an SDL 2.0 context I will better write it all down before it all falls apart again :) or I simply forget.

Prerequisites

  • GNU/Linux (as of now untested on Windows)
  • Python 3 (I used 3.2.5 on Gentoo/amd64)
  • Somewhat recent OpenGL driver for your graphics card.

Better make sure your setup is fine to avoid pitfalls later on.

GNU/Linux

This example is not tested on Windows yet. I used Gentoo, but any decent Linux distribution should do.

Python 3

Make sure you have Python 3 installed properly and it is the default. When type "python" in this tutorial it will always mean "python3".

Virtualenv and Virtualenvwrapper

I highly recommend using virtualenv and virtualenvwrapper. This tutorial uses both.

OpenGL

Make sure you have a driver with good OpenGL support installed. On a current system OpenGL support should be well above 2.1. Check with glxinfo from mesa-progs.

glxinfo | grep -e "direct rendering:" -e "OpenGL renderer string:" \
-e "OpenGL version string:"

On my system this shows:

direct rendering: Yes
OpenGL renderer string: AMD Radeon HD 6670
OpenGL version string: 4.2.12337 Compatibility Profile Context 12.104

So we are good to go here. Note: I am using the ATI drivers. The Open Source radeon driver was at OpenGL version 2.1 at the time of writing.

SDL 2.0

We need something to give us a window with a OpenGL context. Since I plan to use OpenGL in games SDL comes to mind. However the 2.0 Version is in release candidate phase [EDIT 29.09.2013: it has been released now. The link has been updated]. So we will compile it from source. "openglexample" is a project folder in which we will do our tests.

[openglexample] $ curl -sS "http://www.libsdl.org/release/SDL2-2.0.0.tar.gz" | tar xvz
[openglexample] $ cd SDL2-2.0.0
[SDL2-2.0.0] $ ./autogen.sh
[SDL2-2.0.0] $ ./configure
[SDL2-2.0.0] $ make

Now install SDL2. The library will then go into /usr/local/lib.

[SDL2-2.0.0] $ sudo make install
[SDL2-2.0.0] $ cd ..

Remove the source if you do not need it anymore.

[openglexample] $ rm -rf SDL2-2.0.0

Of course you may download manually or check out news about SDL first here www.libsdl.org.

PySDL2

At the time of writing the latest PySDL2 version was 0.4.1 and available on PyPI. We will install it in a virtualenv for our example.

[openglexample] $ mkvirtualenv pysdl2
(pysdl2)[openglexample] $ pip install pysdl2

Bookmark the PySDL2 Documentation.

Let's do a quick test to see if SDL 2.0 and PySDL2 work. 

(pysdl2)[openglexample] $ python ~/.virtualenvs/pysdl2/lib64/python3.2/site-packages/sdl2/examples/sdl2hello.py

This should show a black window with "Hello World" in the title. You may have to adapt that path. Just note that there are some examples in site-packages/sdl2/examples. They will not all work right away. To make them work you will have to compile/install SDL_image, SDL_mixer, SDL_ttf, SDL_gfx and add PyOpenGL (see below)

PyOpenGL

This is the center piece of our attention. You can find out more about it on the PyOpenGL Homepage. Please note that Python 3 support is still considered experimental. So be prepared and help improving it. There is also a package named OpenGLContext for teaching and testing purposes as part of the PyOpenGL project. However I found that it abstracts away some parts and seems to be Python 2 only. Since all of this is a means of learning current OpenGL with GLSL and such for me it's best to have as little abstraction as possible from the pure OpenGL structures and interfaces.

Let's go ahead and install PyOpenGL in our virtualenv.

(pysdl2)[openglexample] $ pip install pyopengl

At the time of writing this installed version 3.1.0a1 of PyOpenGL. Now the little opengl example, that came with PySDL2 should work. Check it out.

(pysdl2)[openglexample] $ python ~/.virtualenvs/pysdl2/lib64/python3.2/site-packages/sdl2/examples/opengl.py

You should see a spinning triangle. Now our environment is set up and we can finally get started with our custom OpenGL code.

OpenGL with GLSL and Arrays

The example code we will produce now has one simple goal: Show a white triangle on an orange background. Orange is the background color of choice because black always has the notion that nothing worked. But we will do so using the new way of doing things in OpenGL. Namely shaders and arrays.

First things first. Here is the current OpenGL Reference.

And at this point I owe special thanks to two distinctive sources without which I would have probably thrown in the towel.

  1. open.gl Looks current and an excellent starting point. It's in targeted at C++, but I found it extremely helpful.
  2. Python examples from Sean J. McKiernan Even though the examples are targeted at Python 2.7 and use Pygame they were the help I needed to weed out a nasty bug that gave me no errors but no triangle either. I will point this out again further below.

Now a quick and dirty overview how this new style of OpenGL programming is organized would certainly be helpful. And here I will try not to disappoint. It very much all boils down to:

  1. You create arrays of data such as vertices, colors, and pretty much anything you like. These are called: Vertex Buffer Objects (VBOs).

  2. You set up small programs in a C like language, the GLSL, which work on these VBOs one element (vertex, color, whatever) at a time. But quite likely in a massively parallel fashion. These are called Shaders.

  3. You set up a mapping between these array elements of the VBOs and the respective, possibly self-defined, attribute names in the shaders. This is stored in Vertex Array Objects (VAOs)

Most of the setup will be done before anything is rendered. The main point is that you move as much of the data as possible over to the graphics card before any rendering or animation is done. Then at runtime, when for example a game is actually being played, you will mostly just bind and unbind VBOs and VOAs, switch the shaders which are to be used and influence some global parameters of the shaders, so called uniforms, from your program. This way you only make very light calls to the graphics card. You will only quite rarely work on arrays of stuff (vertices, colors and so forth) at runtime and almost never on actual pixel data. With this in mind I am fairly certain that Python should be fine for most applications.

The Code

We will now build our small example step-by-step. You can also find the complete file near the bottom of this post. But I recommend playing it through from the start.

Open up your favorite editor:

(pysdl2)[openglexample] gvim example1.py

We will not do a * wildcard import here although it may seem convenient at first. But this way there will be no doubt about from where a name originates.

"""Example 1: white triangle on orange background

setting up a window with OpenGL context and rendering one white triangle on an
orange background using shaders

"""

import ctypes

import sdl2

from OpenGL import GL

For this first example we will fit everything in one function. 

def run():
    if sdl2.SDL_Init(sdl2.SDL_INIT_VIDEO) != 0:
        print(sdl2.SDL_GetError())
        return

SDL_Init just initializes the SDL library. Since PySDL mainly just a ctypes wrapper around SDL we have to check for errors manually.

    window = sdl2.SDL_CreateWindow(
        b"Example 1", 
        sdl2.SDL_WINDOWPOS_UNDEFINED, sdl2.SDL_WINDOWPOS_UNDEFINED, 640, 480,
        sdl2.SDL_WINDOW_OPENGL)

This SDL_CreateWindow call gives us an OpenGL enabled 640x480 pixel window.

Note: The title is specified as a bytestring here. The encoding is UTF-8. But since this string only uses ASCII it works fine. For more complicated unicode strings you can use something like: "Ɩ ǝldɯɐxƎ".encode('utf-8')

Now lets create an OpenGL context for the window.

    context = sdl2.SDL_GL_CreateContext(window)

Note that we will not really need the context reference SDL_GL_CreateContext gives us except for its deletion later on. This is the case because we only work with one context. All following OpenGL functions will automatically apply to this context. If you had more than one OpenGL enabled window you would need to switch them around with SDL_GL_MakeCurrent.

We are now diving head on into OpenGL. Don't worry if you don't understand everything at once. It will become clearer when you see the whole picture.

    # get Vertex Array Object name
    vao = GL.glGenVertexArrays(1)
    # set this new VAO to the active one
    GL.glBindVertexArray(vao)

glGenVertexArrays gives us a name for one so called Vertex Array Object (PyOpenGL Reference / OpenGL Reference) which will store the mapping between element positions in the VBO data and the attribute names in the shaders. In this context you should think of 'name' just as a unique identifier. It basically is just an integer number. Binding it sets it up to store the mapping between the VBOs and shader attributes we specify below. This example would default to work without it but let's work with this concept right away since it is extremely powerful. With one call to glBindVertexArray switching from one to another VAO the active VBOs and the associated attribute mappings change.

Note: We asked for exacly one VAO. glGenVertexArrays therefore gives us the name directly. If we had asked for 2 we would have to refer to them with vao[0] and vao[1].

Also note: The PyOpenGL calls often use return values while the original C specification expects references to buffers for return values.

Of course we also need some vertices to define the one triangle we want to display.

    # vertex data for one triangle 
    triangle_vertices = [0.0, 0.5, 0.5, -0.5, -0.5, -0.5]
    # convert to ctypes c_float array
    triangle_array = ((ctypes.c_float * len(triangle_vertices))
                      (*triangle_vertices))

The VBO needs a ctypes array. For the ease of use and easy experimentation we set up a regular Python list first and then generate a ctypes array from it. Later on you will probably want to use NumPy arrays.

If you are not familiar take a peek at the documentation for ctypes arrays.

Now we need to get a new VBO object from the graphics card and fill it with our data.

    # get a VBO name from the graphics card
    vbo = GL.glGenBuffers(1)
    # bind our vbo name to the GL_ARRAY_BUFFER target
    GL.glBindBuffer(GL.GL_ARRAY_BUFFER, vbo)
    # move the vertex data to a new data store associated with our vbo
    GL.glBufferData(GL.GL_ARRAY_BUFFER, ctypes.sizeof(triangle_array),
                    triangle_array, GL.GL_STATIC_DRAW)

You can see that it is somewhat similar to the VAO creation. There are however different types of buffers. So you have to specify which buffer should be represented by our vbo. The most interesting call is glBufferData. The last parameter GL_STATIC_DRAW is just a hint on how the data will be used.

After we have specified the vertex data we need to move on to the shaders to specify how our triangle will be rendered. The 2 different types of shaders you need to think about now are:

  1. Vertex Shaders : compute the position and additional data for each vertex. You would do 3D transformation here.
  2. Fragment Shaders : computes the color of each pixel. You would for example apply your texture data here.

The vertex shader in our code is assembled like this:

    # vertex shader
    vertexShaderProgram = """#version 100
        in vec2 position;
        void main() {
            gl_Position = vec4(position, 0.0, 1.0);
        }"""
    vertexShader = GL.glCreateShader(GL.GL_VERTEX_SHADER)
    GL.glShaderSource(vertexShader, vertexShaderProgram)
    GL.glCompileShader(vertexShader)

The shader program is just represented as a string, then we get a new name for it (i.e. create it), attach the source string and compile it. The shader source is written in GLSL the OpenGL Shading Language. You can see the full reference here: OpenGL Shading Language. Also check out the quick reference guide and have a look at the pages with the OpenGL Pipeline. This shows you that each shader has its position in the graphics pipeline. It has some inputs, modifies the data and outputs it to the next shader.

You can see the vertex shader in our example has a 2-component vector input. Our triangle vertices will go through here. We only use 2D for now so only the x and y coordinate are used. z is set to 0.0. You can also use position.x, position.y to access the components. The coordinates x,y and z are automatically divided by the 4th component. Don't worry about this now. Just set it to 1.0.

Note: gl_Position is a builtin output variable for vertex shaders

The version preprocessor directive indicates which version of the GLSL the shader will use. Version 1.00 which came with OpenGL 2.0 is good enough for this small example.

Furthermore it is important to know that the compilation step will not raise an exception in case compilation fails. For example due to a syntax error. The later call to glUseProgram will however fail. For debugging shaders have a look at the functions glGetShader and glGetShaderInfoLog.

Now we move on to the fragment shader, which will deal with the color of each pixel of our triangle. 

    # fragment shader
    fragmentShaderProgram = """#version 100
        out vec4 outColor;
        void main() {
            outColor = vec4(1.0, 1.0, 1.0, 1.0);
        }"""

    fragmentShader = GL.glCreateShader(GL.GL_FRAGMENT_SHADER)
    GL.glShaderSource(fragmentShader, fragmentShaderProgram)
    GL.glCompileShader(fragmentShader)

The code is pretty similar to what we had for our vertex shader. Instead of a position the fragment shader does however output a color. Here the color is fixed to white (1.0 for all RGBA components). But it could just as well be sampled from a texture which I will show in later examples.

Note: You won't find outColor in any documentation. However the first user defined output variable will be output to the default framebuffer's default color buffer (GL_BACK here). See Default Framebuffer Wiki for more information. Or you expicitly name the fragment output with glBindFragDataLocation as we will do for completeness further below.

At this point we have the source for our 2 shaders ready and compiled. We will now attach them to a shader program. You can have a lot of different programs and switch between them at runtime.

    # shader program 
    shaderProgram = GL.glCreateProgram()
    GL.glAttachShader(shaderProgram, vertexShader)
    GL.glAttachShader(shaderProgram, fragmentShader)

This code gives us a new empty program and then the vertex and fragment shaders are attached to it.

Next we specify where the fragment output color will come from.

    # color output buffer assignment
    GL.glBindFragDataLocation(shaderProgram, 0, b"outColor")

As stated before this is not strictly necessary since the first user defined output will be taken as default. However we explicitly specify it here for completeness. 0 is the index into the array of color buffers. Check glDrawBuffers for more information. Basically you can render to more than one buffer, possible off-screen, to a texture and reuse the output elsewhere in your scene.

After this we move on to linking, validating and activating our program. 

    # link the program
    GL.glLinkProgram(shaderProgram)

    # validate the program
    GL.glValidateProgram(shaderProgram)

    # activate the program
    GL.glUseProgram(shaderProgram)

This very much like you would expect it from a programming language such as C. First you compile all the parts (here the shaders) then you build a program from all or a subset of the parts. Imagine how you would use this in a complicated setup with a lot of prepared shaders and then stick together a handful of programs between which you switch back and forth. For example one program for all objects in your scene with matte surface and another for all objects with a glossy surface. The validation step checks whether the program can possible run in the current OpenGL state at all.

We are getting much closer to actually getting something on the screen now. The next step is to define what parts of our vertex data, the triangle corners, will flow where into the shaders. 

    # specify the layout of our vertex data
    posAttrib = GL.glGetAttribLocation(shaderProgram, b"position")
    GL.glEnableVertexAttribArray(posAttrib)
    GL.glVertexAttribPointer(posAttrib, 2, GL.GL_FLOAT, False, 0,
                             ctypes.c_voidp(0))

In the first line with glGetAttribLocation we get a handle for the input variable position in our shader program. This is the point where our vertex data will flow into the pipeline. Think of it as long streams of data. In the second line we activate this input with glEnableVertexAttribArray . It is off by default. Try to comment it out. The program will run without error but not show the triangle. The last line with glVertexAttribPointer explains the format of the vertex data. Here it is defined as consisting of pairs of GL_FLOAT type items with no other items between them (as indicated by the 0 stride parameter) starting at offset 0 in the buffer This function refers to the currently bound GL_ARRAY_BUFFER, which is our vbo with the corner coordinates of the triangle. However it is very important to realize that this buffer is not just for vertices. It is quite common to put other data in it which is associated with each vertex. For example color values or texture coordinates. This will become clearer in later examples. For now it is important to see that the offset needs to be a pointer value. If you just put in 0 it won't work. None would work but only for the case that you do want an offset of 0. This is the nasty bug I had in my example code when trying it out first.

Finally we are now about to start the drawing itself.

    # do the actual drawing
    GL.glClearColor(1.0, 0.5, 0.0, 1.0)
    GL.glClear(GL.GL_COLOR_BUFFER_BIT)
    GL.glDrawArrays(GL.GL_TRIANGLES, 0, int(len(triangle_vertices) / 2))

The glClearColor specifies our lovely orange background color. The we start the clearing process with glClear and select all color buffers currently enabled for writing with GL_COLOR_BUFFER_BIT. The glDrawArrays then draws our triangle by telling the graphics card to walk 3 elements, i.e. vertices though the vertex array starting right at the beginning 0. I choose int(len(triangle_vertices) / 2) instead of the explicit vertex count just in case you want to mess around and add another triangle.

If you were to stop here you would not see anything since the OpenGL context SDL has prepared for you uses double buffering by default. Therefore you now need to set it to the front. 

    # show the back buffer
    sdl2.SDL_GL_SwapWindow(window)

Last but not least we have to make the program wait a bit or we will not see anything. For this we could just use a standard python time.sleep(5) but we rather move directly to SDL since we will be needing it more and more anyway.

    # wait for somebody to close the window
    event = sdl2.SDL_Event()
    while sdl2.SDL_WaitEvent(ctypes.byref(event)):
        if event.type == sdl2.SDL_QUIT:
            break

Remember that PySDL2 is a thin ctypes wrapper so we have to call it a bit C-ish. First we get an empty event and then start an infinite loop which will incoming wait for events with SDL_WaitEvent. If the window with our triangle is closed it will signal SDL_QUIT and we break out of the loop.

Even though it would probably work without it in this small example we add some cleanup code 

    # cleanup
    GL.glDisableVertexAttribArray(posAttrib)
    GL.glDeleteProgram(shaderProgram)
    GL.glDeleteShader(fragmentShader)
    GL.glDeleteShader(vertexShader)
    GL.glDeleteBuffers(1, [vbo])
    GL.glDeleteVertexArrays(1, [vao])
    sdl2.SDL_GL_DeleteContext(context)
    sdl2.SDL_Quit()


if __name__ == "__main__":
    run()

Most things are pretty obvious here. glDeleteBuffers and glDeleteVertexArrays need some extra attention. They expect the number of buffers/arrays to delete from a given list. Therefore we have to make our vbo and vao objects lists.

Now run the whole thing.

(pysdl2)[openglexample] $ python example1.py

You can also download the complete script here example1.py.

The Final Result

This is what you should see when you run the program.

Alt Text

Not that impressive you say? But believe me it is a solid start :).

Debugging

While I will not go into explicitly OpenGL related debugging here I recommend running the example in the Python debugger. Have a look at what you get back from the OpenGL calls.

(pysdl2)[openglexample] $ python -m pdb example1.py

A quick reminder. Press n[enter] to execute the currently displayed line and step to the next line or press s[enter] to execute the current line and step into a function call. Press just [enter] to repeat the last command. So in this case n[enter] and a bunch of times just [enter] until you see:

-> run()
(Pdb)

Step into with s[enter] and then move with n[enter]. Check out the some of the fancy sounding things like the vbo.

-> GL.glBindBuffer(GL.GL_ARRAY_BUFFER, vbo)
(Pdb) pp vbo
1
(Pdb)

You can see that things like the vbo are just identifiers or names for the objects themselves. Just simple integer numbers.

Discussion / Next Topics ?

We have barely touched the surface here. To be honest we just made sure everything works. There is a lot to explore and many directions to go in. I suggest you mess around with the code a bit to get familiar with the constructs.

Assuming that Murphy's law holds true once more you will probably be disappointed that something did not work and you do not know where it fails.

With that in mind I plan my next post to be on debugging the OpenGL state in this very example step-by-step.

social