Programming DieselEngine 2. Using Diesel3D

By Jani Immonen, February 18, 2001.
CTO, Co-Founder www.inmarsoftware.com
Print version

Introduction

DieselEngine is a software development kit by Inmar Software Ltd. It is multi platform SDK that allows producing real-time 2D and 3D graphics, along with input mapping and sound engine with 3D sound. DieselEngine SDK is free for non-commercial use, and its demo version downloadable at: www.inmarsoftware.com. Demo version contains all features that licensed version, and works equally well when following this set of articles.

By writing this article, newest version of DieselEngine SDK is version 1.10. If you are using older version, please update before proceeding.

This is second article about programming DieselEngine. I strongly suggest that you read the first one Close up and personal with DieselEngine before proceeding, it contains lots of required information. The same requirements apply to this article as with first article.

In addition to requirements I expect reader to understand matrix transformations and basic 3D renderer concepts. And as you will notice, the knowledge of Microsoft Direct3D helps a lot.

What is Diesel3D

Diesel3D is one component of DieselEngine, and actually is just a 'add on' component to DieselGraphics. Diesel3D cannot work without DieselGraphics objects; it uses CDieselSurface objects as render targets and textures.

The very heart of Diesel3D is the CDiesel3DDevice class, which handles all rendering. CDiesel3DDevice class uses some other classes as input objects, such as vertex buffers, index buffers, materials, lights etc. but basically all other Diesel3D classes are higher level objects and not necessarily needed for rendering 3D graphics. In this article I'm not going to use higher-level objects, like 3D particle systems. I will create application that has CDiesel3DDevice object, and another object called 'the scene' object.

Design considerations

If you are about to start developing a 3D application for current PDA devices, there are some things to consider. The fact is, that with current PDA devices, the processing power and memory are limited, and 3D applications are known to be heavy and memory consuming. But, with careful planning, design, and knowledge of the limitations, it is possible to develop a mind blowing 3D app for even the modest PDA device.

First thing to consider is: "does it have to be real time?" many times you can change your game idea, just a little bit, to make it use static backgrounds. And by static I don't mean that they cannot change. Just use the 3D engine to render the background, even with more detail than usually, crab the z-buffer after background render, and then render all moving objects in top of static scene. Now, whenever something moves, only they are rendered, but before rendering, the z-buffer from static scene rendering is set to 3D device. Imagine adventure games with rooms, which can be in isometric view, or even on real perspective. If game is designed properly, no one cares if moving into another room takes a second or two. And within a second, you can render huge amount of triangles. Our game 'Diesel Minigolf' (available at: www.handango.com or www.inmarsoftware.com) takes this approach when using fixed camera setting.

If your application design requires complete real time rendering, first thing to consider is polygon count per frame. I recommend that you keep your per frame polygon count to less than 1000 polygons. It is possible to push more, but then it leaves little room for sound, input and application logics. If possible, disable the specular lighting from 3D device; the specular lighting is computationally really expensive. Another thing is to disable alpha blending, whenever it is not needed. If using textured objects, try to disable lighting. The result might be satisfactory, even that all polygons are rendered flat. The transformation stage of Diesel3D is the bottleneck of the pipeline. If your application uses lots of 3D objects, consider implementing powerful visibility algorithm for your objects.

One last thing to consider is: "is your application meant for multiple platforms?" if it is, then you need to design your objects and textures to be scalable. Using Diesel3D with Microsoft Direct3D renderer pushes up more polygons than with PocketPC (that should not be a surprise (??!))

Creating 3D application

Creating 3D application is as straight forward as creating the 2D application on previous article. Use Diesel Application Wizard to generate project for you, and at step 4, select 'Use Diesel3D' and 'Create Z - Buffer', but leave 'Create Spinning Cube' unchecked at this time. Selecting spinning cube option creates cube object and rotates it around on screen. That is not necessary here, as I'll try to give information from other side of the Diesel3D programming. Feel free to experiment with spinning cube, as it also generates lots of useful code to experiment.

I created example project called 'Diesel3DApp' and named the application class to 'CDiesel3DApp'. You can use whatever names you want, but following this article is easier with same names. Also remember to select the OnKeyDown and OnSize virtual functions to override.

Now, if you browse thru generated source and header files, you'll notice that it is very similar to application created on first article. Only two new member objects are added to application object, CDiesel3DDevice and CDiesel3DScene. I'll describe those objects later, but first we have to add code to shut the application down. Like in first article, add code to OnKeyDown handler:

Unlike in previous article, this code works with all supported platforms. Code needs to have defines for different platforms, OnKeyDown is platform depended handler, which just sends the operating system key code into it. DieselInput component of DieselEngine maps all key presses to common values, but we'll get back to DieselInput later. Now the return or escape key will exit the application (on Compaq iPaq: centre on directional button).

Now look at the OnSize handler. The wizard has generated some code to handle the window size changes. This application will be full screen app, so we really don't need those, but they do absolutely no harm, so leave them as they are. What handler does is, it calls implementation on IDieselApplication, which effectively resizes the application back buffer surface, then sets the new resized back buffer as render target to CDiesel3DDevice object, and sets the rendering viewport to entire back buffer surface.

Go to the OnInitDone handler, there you'll find initialisation of 3D device and scene objects.

The OnExit handler contains wizard generated releasing of CDiesel3DDevice and CDiesel3DScene objects.

Initialisation of CDiesel3DDevice object

The CDiesel3DDevice object is responsible for rendering the 3D primitives to screen. Normally, CDiesel3DDevice object is a member of IDieselApplication, but it is also possible to use multiple inheritance, if it better suits your needs.

Starting the device object is simple: only one parameter, which is the start-up flags. And only flag supported with PocketPC devices is DE_CREATEZBUFFER. That flag instructs the device to create automatic z-buffer surface. Generally you should always use automatic z-buffer, when there is any possibility to have objects behind other objects. Z-buffer is implemented with CDieselSurface object, and can be retrieved by calling CDiesel3DDevice::GetZBuffer function.

Second step on device start-up is the setting of transformation matrices. Device object sets some default matrices to itself, but normally they are nothing like your application needs.

First the camera matrix is constructed by making the identity matrix, and setting the position to Z = -50. This moves camera 50 units back on z-axis.

Diesel3D uses left handed coordinate system, where x-axis increases right, y-axis increases up and z-axis increases away from viewer. Same coordinate system and matrix layout is also used with Microsoft Direct3D:

Matrix ordering on Diesel3D:

M00	M01	M02	M03	M00, M01, M02 - The 'right' vector
M10	M11	M12	M13	M10, M11, M12 - The 'up' vector
M20	M21	M22	M23	M20, M21, M22 - The 'direction' vector
M30	M31	M32	M33	M30, M31, M32 - The 'position' vector

Although matrix ordering and coordinate system are same as Direct3D, the way to use camera matrix is not. In Direct3D camera is set by 'view' matrix, Diesel3D uses matrix ordered exactly like world matrices for 3D objects, so setting the camera orientation and position matrix is same as setting the world matrix for rendering 3D objects. This way same algorithms can be used to animate cameras and objects. The Diesel3D camera matrix can be converted to Direct3D 'view' matrix by inverting it, and of course, it also works another way.

Second matrix to set is the projection matrix. This matrix defines how rendered primitives 'projects' into the render target surface. CDieselMatrix4 class has function CDieselMatrix4::MakePerspective which helps building the projection matrix.

First parameter to function is view field of view angle in radians, second is view aspect ratio. Aspect ratio is normally render target surface height divided by render target surface width; this gets rectangular pixel regardless of the render target surface size. Third and fourth parameters are projection near and far clipping planes. Any triangle closer than near plane, or further than far plane will be clipped away.

The projection matrix used by Diesel3D is compatible 'as is' with Microsoft Direct3D, and it is possible to make 'non-perspective' projections with projection matrix, for example, isometric views.

That was all (and more) needed to start the 3D device object. It is now ready to render!

What is CDiesel3DScene object?

The main operation of CDiesel3DScene object is to store all resources that can be used with multiple 3D objects. Those include vertex buffers, index buffers, textures, materials, lights etc. Scene object can also contain hierarchical tree of 3D objects. The fundamental operation of scene bases on CDieselID derived classes. All classes derived from CDieselID have functions for setting and retrieving the name. Scene object uses those names to identify the resources inside scene. For example trying to add texture named 'basetexture.jpg' to scene that already contains texture with same name, fails. The same system is used with all resources, vertex buffers, materials etc

Why is all this storing of resources necessary? Consider the classic example; you have space shooter game, where horde of enemy ships attacks you. You may have 10 ships flying around you, and all have same 'form': that is, are made from same vertices. Some of them might have different texture, but still they all are based on same vertex mesh. The scene object has only one vertex buffer and one, or maybe few textures. The 3D objects then just have pointers to that data inside scene, and multiple 3D objects can use same vertices without need to duplicate data.

The CDiesel3DScene object is somewhat restrictive, and some people like it some don't, but fortunately there are many ways to use it, and it is not necessary to exist at all. Now I'll describe few ways the scene object can be used.

1. Scene object can be used as storage for shared resources, and also contain 3D objects. Scene object can load itself from file, and contain object animations and hierarchies. Single call to CDiesel3DScene::Update updates all objects in scene, and single call to CDiesel3DScene::Render renders all objects in scene. This way to use scene is somewhat the easiest, but also very restrictive when using 3D objects.
2. Just like 1., but you'll never call Update or Render. Pointer to 3D objects inside the scene can be obtained with call to CDiesel3DScene::FindObject, then you can use the obtained pointers to update and render your 3D objects. This way you have more control over the updating and rendering of 3D objects inside the scene object.
3. Derive your own class from CDiesel3DScene class, and override any functionality you don't want, or want to work differently.
4. Scene can be used as storage for shared resources, and contain no 3D objects.
5. Don't use it at all; store textures etc. where ever you want. The CDiesel3DDevice object doesn't care where textures or material come from.

In this article we'll use the scene object as storage for resources, and it will not contain any objects (way number 4). There are plenty of example applications on DieselEngine SDK that shows the usage of scene object.

Only thing needed to start the scene object is to call it's Startup function. The Startup must be called before doing anything with scene object, including loading the scene from file. Multiple scene files can be combined by loading them in sequence, but then you must make sure that none of scene files have objects or resources with same names. Those objects and resources will not be loaded into the scene, the scene object assumes them to be same objects or resources.

The Basic Rendering

Rendering with CDiesel3DDevice object is quite simple, but first we have to create some vertices. Lets create a vertex buffer that will form single quad. Go to the application OnInitDone function, and add following code after m_Scene.Startup() function call:

Code creates new CDiesel3DVertexBuffer object, that object will contain all our vertices. Next, the name of vertex buffer object is set with SetName function.

The CDiesel3DVertexBuffer::Startup function allocates memory for vertices. First parameter is pointer to initialisation vertices, or NULL if data is left uninitialised. If this parameter is not NULL, the data from that pointer is copied into the vertex buffer. So it is safe to delete the initialisation vertices after call to Startup. Second parameter is number of vertices this buffer contains, 4 is enough for creating a quad. Last parameter defines the vertex type this buffer contains; the eDE_VERTEXTYPE_TC1 is basic vertex with one set of texture coordinates.

The access to vertex buffer data is provided with Lock function. Because vertex buffers can contain different kind of vertices, Lock returns a void pointer, so casting is necessary. The eDE_VERTEXTYPE_TC1 equals to CDiesel3DVertex object.

After the accessing buffer data, code fills the vertices member by member. It is possible to use CDiesel3DVertex class overridden constructors to build the vertex, but I wanted to demonstrate the members of vertex class. Members x, y and z are vertex position, members nx, ny and nz are vertex normal vector, and members tu and tv are texture coordinates for vertex.

After all four vertices are initialised; the direct access to vertex buffer data must be released with call to Unlock. Generally you always need to match every call to Lock with call to Unlock.

Last thing to do with vertex buffer is to add it to CDiesel3DScene object. By adding the buffer to scene, we don't have to worry about it anymore. The scene object releases vertex buffer on its Shutdown function.

Next, add some rendering states to 3D device object, go to the end of application OnInitDone function:

In code above we disable alpha blending and specular lighting, enable lighting and set ambient lighting level to 0.

The rendering of quad is straightforward. Go to the OnFlip function, and first remove the m_Scene.Update() and m_Scene.Render() function calls. We are using the scene object as resource container, so there is no need to update or render the scene object. Next add following code after call to m_3DDevice.BeginScene()

All rendering must happen in between CDiesel3DDevice::BeginScene and CDiesel3DDevice::EndScene function calls. We call device's DrawPrimitiveVB function, first parameter is a primitive type, here we have a triangle fan, which can be used to render a quad from two triangles. Second parameter is a pointer to vertex buffer, which we ask from the scene object. We could make the vertex buffer pointer to member variable of our application class, but I wanted to demonstrate the retrieval of vertex buffer from scene object. In this example we ask the first vertex buffer (index 0) from scene object, but it is also possible to retrieve it by name. Retrieving by name is done with CDiesel3DScene::GetVertexBuffer function. Last parameter to DrawPrimitiveVB is a draw flags. Last parameter is not currently used, so set it to 0.

Build and execute the application to see the results. Not very exiting yet, just a rectangle sitting in middle of the screen. Lets make it rotate, add code to OnFlip function, before the call to m_3DDevice.BeginScene:

Ok, the code above is 'quick and dirty', and uses static variable to store the rotation angle, but it will do for now. Code builds matrix that rotates around Y axis, sets it to 3D device as world matrix, and adds rotation angle PI radians per second.

Now you should see spinning quad:

Note, when our primitive is spinning, it is not rendered when it is facing away. This is default behaviour of CDiesel3DDevice object, but can be changed by calling CDiesel3DDevice::SetCullMode function.

The texturing

Texturing, even the multitexturing, with Diesel3D is quite simple. All textures are just CDieselSurface objects, created with one additional flag. On SDE_SURFACEDESC structure, the dwFlags member must have a DE_TEXTURE flag set when creating a surface.

There are restrictions with texture surfaces. The size of texture surface must be rectangular powers of 2 sized image. When loading surfaces with DE_TEXTURE flag, CDieselSurface object automatically re-sizes the surface to nearest possible powers of 2 size, if necessary. You should consider this when designing the textures, try to make them all sized something like: 64x64, or 128x128, or 256x256 etc.

Lets add texture to our quad. Maybe you noticed that we already set the texture coordinates to vertices, so lets load a texture from file. Go to the OnInitDone function:

First we create new CDieselSurface object, the fill the SDE_SURFACEDESC structure with DE_TEXTURE flag. The sizes 0 and 0, loads the texture as is at file, unless the size is not powers of 2. In that case texture surface is resized to nearest possible size compatible with 3D device.

We load texture from jpg file using BuildFilepath helper function, and CDieselSurface::Load function. After loading is successful, we set the name to texture and add it to scene object. Again, after texture is added to scene object, we don't have to worry about its release. It is done by scene object.

Next, go to the OnFlip function, and add following code just before call to DrawPrimitiveVB:

Now we set the texture into the 3D device object, by retrieving it from scene object. This example shows how to retrieve the resource from scene by name. We could retrieve it by index also, but again, I wanted to demonstrate different ways to do it. The first parameter to CDiesel3DDevice::SetTexture is texture stage index, in this case 0 (first texture stage). Second parameter is pointer to CDieselSurface object used as texture.

Build & execute, and you'll see something like this:

Now it becomes apparent, that Diesel3D do not have perspective corrected texture mapping on software renderer (PocketPC, EPOC). This is known decision; there are always compromises between speed and quality. The lack of perspective correction can be compensated by not using large triangles as textured primitives. Divide your large primitive surfaces to multiple triangles.

Multitexturing

Multitexturing differs only slightly from single texture texturing, just add another texture to texture stage 1, and set some texturing types and you're all set. Lets load another texture at OnInitDone:

Code above is basically the same as loading the first texture. If this example I use 64x64 sized texture, which looks like this:

Now go to the OnFlip function, and add code just after previous SetTexture function call:

Here we set the texture into the texture stage 1 (second stage). All texture stages from stage 1 up, are disabled by default, so we need to set how we want Diesel3D to blend the textures together. The SetTextureState function does that. Here we set the stage 1 to eDE_TEXTURESTATE_ADD which effectively adds texture with previous state.

Now build & execute and your program should look like this (with your own textures of course):

Experiment with different texture state values. Try eDE_TEXTURESTATE_SUB instead of eDE_TEXTURESTATE_ADD, and so on. Possible values for SetTextureState are:

Go ahead and experiment them all.

In addition, Diesel3D supports some special effects with texture coordinate manipulation. Try to add following line just after the last SetTextureState function

This makes Diesel3D to compute the texture coordinates for texture state 1, using current camera matrix. It produces quite nice environment mapping effect. Another value eDE_TEXTURINGTYPE_ENV1 is also available, but it is better suited for rounded objects. Default value for all texture states is eDE_TEXTURINGTYPE_VERTEXUV, which uses texture coordinates from vertices. Feel free to experiment with different texturing types.

Loading 3D objects from files

Building all 3D objects vertex-by-vertex won't get you far. Fortunately Diesel3D can load entire 3D scenarios with animations etc. from files. The format supported natively is 'DSC' format, which is Diesel3D's own 3D scenario file format.

Now I hear you cry: "Why another 3D object format !!??", there is a reason for that. Making a well working loader to any of currently widespread format is difficult, and code for it will be VERY long. Now, we are running our applications on mobile devices, can we afford to add another 100kb to our executable size? Also, normally those common formats contain lots of unnecessary information, so they are quite big. And just another thing, some people insists to have native support for 3D Studio MAX, and some absolutely require support for Maya or Lightwave.

Diesel3D tries to go around that problem by providing its own, quite compact format, and offering converters as a plug-in applications to Diesel Media Packer application. Now, nothing is keeping you from writing your own plug-ins to Diesel Media Packer to support any format you like. Bundled with DieselEngine SDK is a converter plug-in, that can convert into the 'DSC' file from following formats: 3D Studio MAX ASE export (*.ASE), 3D Studio files (*.3DS), Quake2 objects (*.MD2), and from Quake3 objects (*.MD3). Now, only conversion that properly supports all features of Diesel3D is conversion from 3D Studio ASE exported files.

Here I'm going to give step-by-step info on how to get your 3D models from 3D Studio MAX into the Diesel3D application.

1. Create your 3D model at 3D Studio MAX. Pay close attention to object, material etc. names, because those same names will be at converted 'DSC' file.
2. Select menu File->Export. Select the export file type to ASE. From ASE export dialog, select what ever you want to export. All but helper objects and IK joints are supported.
3. Open the Diesel Media Packer application, and add your exported ASE file to pack. This is done by selecting menu Edit->Add Files to pack:after file is added to pack, select it from tree view.
4. With your ASE file selected at tree view, select menu Plugins->3D plugins->Inmar Software Ltd.->Diesel3D scene converter plugin. After conversion is complete, new item appears into the tree view. Rename this to what ever you like, preferably with 'DSC' extension.
5. Select the new item created by plug-in application, and right click it. From pop-up menu select: Export items:Select folder where file will be exported.
6. Create Diesel3D application, and at OnInitDone handler function, just after call to CDiesel3DScene::Startup call scene objects Load function, with file name as only parameter.

Summary

Here I provided some 'low-level' information about using Diesel3D. This should be enough to get you started. Remember, best sources for more information are DieselEngine SDK example programs and documentation. Have fun!

Related resources:

• Section: GAPI
• Article: Programming DieselEngine 1. Close up and personal with DieselEngine, Using DieselGraphics
• Library: DieselEngine
• Article: Drawing images and other graphics with GAPI
• Article: GapiDraw from a DirectDraw developers perspective
• Library: STGapiBuffer
• Library: EasyCE
• Library: GAPI for Emulator
• Library: Pocket GL (iPAQ and Cassiopeia)
• Library: GapiTools
• Library: GapiDraw
• Control: ASpriteCE Game Control
• Article: Getting Started with the Game API
• Article: Here Comes GAPI!
• Article: PocketPC Advanced Graphics & Games Programming Tutorial. Part 1 - Getting started with EasyCE
• Article: Pocket PC Advanced Graphics & Games Programming Tutorial. Part 2 - Advanced tricks with pixels
• Article: Pocket PC Advanced Graphics & Games Programming Tutorial. Part 3 - Pixel warps and displacement tables
• Article: Pocket PC Advanced Graphics & Games Programming Tutorial. Part 4 - Buffers, pointers and sprites
• Article: Pocket PC Advanced Graphics & Games Programming Tutorial. Part 5 - Rotozooming and fixed point arithmetic
• Article: Sprite
• Article: Animation
• Article: Timer call back
• Article: Scrolling background
• Article: Object collision
• Article: Off-screen technique

Discuss