Create a 3D Flight Simulator App for iOS and Android - Theoretical Overview

In this tutorial series, we will create a flight simulator application using ShiVa3D Suite. A flight simulator is an interesting example of 3D programming. In order to build this application, we will cover all the topics necessary for creating 3D games, educational applications, and novelty 3D apps. The simulator built in this series will be deployed and tested on a Motorola Droid with Android 2.2, an iPod Touch with iOS 4.3, and an iPad 2 with iOS 4.3.

Also available in this series:

Create a 3D Flight Simulator App for iOS and Android - Theoretical Overview
Create a 3D Flight Simulator App for iOS and Android - Environment Creation
Create a 3D Flight Simulator App for iOS and Android - Simulator Programming
Create a 3D Flight Simulator App for iOS and Android – Project Export

Series Overview

With Shiva3D Suite, a programmer does not need to write any Android or iOS specific code. There are two main steps in creating a cross-platform 3D application with Shiva3D Suite: (1) Develop the visual aspects of the application and the corresponding code using ShiVa Editor. (2) Convert the resulting binary file to a platform specific executable (e.g. an apk file for Android) or a project file (e.g. an Xcode project for iPad) using ShiVa Authoring Tool.

ShiVa Editor is a powerful tool to develop 3D applications. Among the available features in the tool, this series focuses on some basic functions, such as creating a Game, a Scene, Terrain, a Heads Up Display (user interface components), AIModel (behavioral model for an object), as well as coding in the Lua scripting language. After developing the Simulator in the ShiVa Editor as a 3D application, we will export it for both the Android OS and iOS platforms. For the Android platform, the file exported from the ShiVa Editor will be imported into the ShiVa Authoring Tool. Then, it will be converted to an apk file for deployment into a Motorola Droid with Android 2.2. For iPad and iPhone platforms, the file exported from the ShiVa Editor will be imported into the ShiVa Authoring Tool for conversion into Xcode projects. Then, one can build the projects in Xcode and deploy the resulting executables into connected devices.

Organization Of This Series

In part 1 of this series, we introduce the flight simulator application named Simulator and explain the development process we followed while creating the Simulator. We also described the contents of the download archive accompanying this series. Finally, we introduced the basic flight maneuvers simulated in the application.

In part 2, we start describing how to develop the game using the ShiVa Editor. Our main focus in part 2 is creating the visual components of the application.

In part 3, we continue developing the game using the ShiVa Editor. Primarily, part 3 is involved with coding and unit testing the application.

In part 4, the last installment of the series, we show how to export the application from the ShiVa Editor and import into the ShiVa Authoring Tool. Then, we explain how to convert the application into a platform specific executable via the ShiVa Authoring Tool. The platforms we will consider are Android, iPad, and iPhone. We give screen images of the Simulator from actual devices used for testing, a Motorola Droid phone with Android OS 2.2, an iPad 2 with iOS 4.3, and an iPod Touch with iOS 4.3. Finally, in 'Concluding Remarks', we provide a conclusion to the series.

Credits

The skybox images back-crop.jpg, front-crop.jpg, left-crop.jpg, right-crop.jpg and top-crop.jpg have been edited from a panoramic image in Wikipedia, which is used under the Creative Commons Attribution-ShareAlike 3.0 License.
Various P51 Mustang images used throughout this series have been obtained from a 3D Collada model in Google 3D Warehouse, which was originally developed by user Tom. The model is used under the license terms described in http://sketchup.google.com/intl/en/3dwh/tos.html.
The author has benefited from Chapter 16 of Shando's Shiva Book and a tutorial on terrain creation at ShiVa3D developer site, named Terrain Creation, while developing the terrain. In addition, the multitouch event handling code used in the series borrowed a technique introduced in a tutorial at the ShiVa3D developer site, named Multitouch Management.

Flight Simulator Screens

The simulator provides a view of the surroundings from the cockpit as a pilot would see them. The plane itself, whose dynamics are simulated, is never visible. The user is going to see a terrain below and the sky above the terrain. There are four simple maneuvers to simulate: the pitch maneuver to go up or down and the banked turn maneuver to go left or right. Technical descriptions of those maneuvers are given in the next section. Here, we will review the screens during the maneuvers and discuss how the maneuvers are performed by the user.

To play the Simulator, the mobile device needs to be held on its side and rotated 90 degrees counter-clockwise. In that position, there are five buttons atop the screen (see below):

L: initiates a banked turn maneuver to left
U: initiates a pitch maneuver to go up
Re: restarts the simulator
D: initiates a pitch maneuver to go down
R: initiates a banked turn maneuver to right

Each of those four maneuvers, once initiated, takes a certain amount of time to complete. Once a maneuver starts, no other maneuver could be initiated until the current one completes. At all times, the plane keeps moving with constant speed. It changes altitude only during a pitch maneuver and changes direction only during a turn maneuver.

Before any maneuver starts, the plane is parallel to the surface (terrain).

Banked Turn Maneuvers

During a banked left turn, the tip of the plane's left wing goes down and the tip of its right wing goes up. To simulate this behavior, the horizon will start rotating clockwise. The terrain will shift to the right, because the plane is turning left, and a message will be displayed, 'Banking left', as follows:

Once the plane reaches a maximum roll (bank) angle, the horizon will start rotating counter-clockwise and stop rotating when the plane becomes parallel again to the surface. During the entire cycle of the left turn, the terrain will keep shifting to right to simulate a left turn of the plane.

A banked right turn is symmetrical to a banked left turn above: the horizon will start rotating counter-clockwise to simulate the roll of the plane, the terrain will shift to the left, and a message will be displayed, 'Banking right', as follows:

Once the plane reaches a minimum roll angle, the horizon will start rotating clockwise and stop rotating when the plane becomes parallel again to the surface. During the entire cycle of the right turn, the terrain will keep shifting to the left to simulate a right turn of the plane.

Pitch Maneuvers

During a pitch maneuver to go up, the horizon will first go down to simulate that the plane's nose goes up. In addition, a message will be displayed, 'Pitching up', as follows:

Once the plane reaches a maximum pitch angle, the plane's nose starts going down. To simulate this behavior, horizon will start going up until the plane becomes parallel again to the surface. As the horizon goes up, altitude will be displayed next to the message 'Pitching up' as shown below:

The pitch maneuver to go down is symmetrical to going up. The horizon will start going up to simulate that the plane's nose goes down and a message will be displayed, 'Pitching down', as follows:

Once the plane reaches a minimum pitch angle, the horizon will start going down until the plane becomes parallel again to the surface. As the horizon goes down, the altitude will be displayed next to the message 'Pitching down' as shown below.

During the entire cycle of a pitch maneuver, the plane gains altitude, if pitching up, or loses altitude, if pitching down.

Restarting The Simulator

When the user restarts the simulator, a message, 'Simulation restarting', will be displayed on the screen for a few seconds before the application restarts. This is shown below.

Thumb Actions To Control The Simulator

In addition to buttons, the user can initiate turn and pitch maneuvers by a single touch action on the screen.

Moving a finger to left initiates a banked turn maneuver to left.
Moving a finger to right initiates a banked turn maneuver to right.
Moving a finger up initiates a pitch maneuver to go up.
Moving a finger down initiates a pitch maneuver to go down.

The turn maneuvers via thumb actions are shown in the figure below.

The pitch maneuvers via thumb actions are shown in the figure below.

The user can also restart the simulator by moving two fingers on the screen simultaneously in the same direction, either up or down, as indicated below.

Out Of Bounds

The terrain has finite dimensions. If the plane goes out of bounds a message, 'Out of bounds', is displayed, as shown below, and the simulation restarts.

Development Process With Shiva3D Suite

ShiVa3D Concepts

In this section, we will discuss basic concepts of 3D game development with ShiVa3D. Most of the discussion here is borrowed from the original ShiVa3D documentation. For more information, please see http://www.stonetrip.com/developer/doc/ and the user manual that comes with the ShiVa editor.

Game represents a game, the main entity of the application. It encapsulates everything else in the application such as cameras, scenes, etc. Game is a stand alone deployment unit.

Scene represents a place or view associated with the game. There is a set of objects or models associated with a scene. A game can have more than one scene. The game in the Simulator application has a single scene.

Camera represents a viewpoint in the game. The user will see the game through the camera. A camera can move from one position to another or its direction can be changed. In this series, the camera will capture the flight from the view of a pilot. We will assume that the camera is placed in front of the plane pointed straight ahead. The plane itself will not be visible.

Model can be an object or a set of objects with various attributes such as shape, light, or sensor. The only model we will have in the Simulator is a default light set to illuminate the terrain.

Terrain is the ground at a scene. In the Simulator, we will create a terrain to simulate what a pilot would see during a flight.

AIModel implies 'artificial intelligence' and represents behavior. An AIModel can have functions, handlers, states, and variables. In the Simulator, we will use an AIModel to describe the simplified dynamics of a plane.

Script contains the code in an AIModel, for example, the code for a function or handler. The scripting language in ShiVa3D is Lua (http://www.lua.org).

HUD stands for Head Up Display and is a term to represent various user interface widgets such as the button, label, list, slider, etc. allowing the user to interact with the game. The HUD components we are going to use in the Simulator are the text labels and buttons.

ShiVa3D Tools

To develop the Simulator we will use the free version of ShiVa3D Suite (http://www.stonetrip.com/download.html), which includes ShiVa Editor PLE (personal learning edition) and ShiVa Authoring Tool. The diagram below gives an overview of the development process we used with those tools.

Let us discuss individual steps of that process.

ShiVa Editor

The ShiVa Editor has various modules to develop and test a 3D application, typically a game, from the ground up. An important feature of the ShiVa Editor is that an application developed with that tool can be deployed (after authored in ShiVa Authoring Tool) in different devices with different operating systems, e.g. a PC running Windows OS, a mobile phone with Android OS or iPhone etc. Some of the basic actions you can perform with the ShiVa Editor are as follows.

Develop:

Create a new game.
Create a new scene and associate it with the game.
Create a terrain for a scene.
Create AIModels and write scripts for the AIModels.
Compile the scripts in the game.

Test: You can perform an initial test of the 3D application in the development environment before deploying it in a target device. Testing is performed via the Animate or Preview feature of ShiVa Editor. During testing, you can change settings for the screen size to see how the application will be displayed in the actual target device. To test button events, we used mouse clicks. To test touch events, we exported and deployed the application in an actual device.

Export: After testing is complete, export the application. This will generate a file with stk extension. The exported application will be used by the ShiVa Authoring tool, which is discussed next.

ShiVa Authoring Tool

The main purpose of the ShiVa Authoring tool is to convert an application, created via the ShiVa Editor, into a platform specific application that can be deployed in a particular device (e.g. iPhone, iPad or Android phone). Certain restrictions apply in regard to the operating system of the machine running the ShiVa Authoring tool and the target device for conversion. For example, the ShiVa Authoring tool running on a Windows machine cannot generate an iPad or iPhone application. While developing the Simulator, we used a Windows machine for The ShiVa Editor. To create the Android Simulator application, we also used a Windows machine to run the ShiVa Authoring Tool. On the other hand, to create the iOS Simulator applications, for iPhone and iPad, we used a Mac OS machine to run the ShiVa Authoring Tool.

Some of the basic actions one may perform with the ShiVa Authoring tool are the following.

Select Authoring Platform: For the purposes of this series, the three platform options we are interested in are iPhone, iPad and Android. Note that, for iPhone and iPad, you need to have the ShiVa Authoring Tool running in a Mac OS machine.

Import: Import the application (a file with stk extension) which was exported via the ShiVa Editor.

Build: When authoring for Android, we will configure the ShiVa Authoring Tool to generate an Android apk file that can be directly deployed in a compatible Android device. When authoring for iPhone, we will configure the ShiVa Authoring Tool to generate an Xcode project for iPhone development. Then, one can build the project in Xcode and deploy into a connected device (We actually used an iPod touch device for testing instead of an iPhone). Similarly, when authoring for iPad, we will configure the ShiVa Authoring Tool to generate an Xcode project for iPad development. Then, one can build the project in Xcode and deploy it into a connected iPad.

For each platform, the ShiVa Authoring Tool offers slightly different options to generate its end product. For example, when authoring for Android, you could generate an Eclipse project instead of an Android executable (as mentioned above, we chose to generate an Android executable in this series).

Similarly, for iPhone or iPad, one could generate an executable for distribution instead of generating an Xcode project (As mentioned previously, we chose to generate Xcode projects in this series). Please consult the ShiVa Authoring Tool's user manual for further details.

Software Configurations

The following is a list of configurations we used while developing the Simulator app for this series. For additional requirements and prerequisites, see the Shiva3D documentation described in http://www.stonetrip.com/developer/doc/editor/information/hardware and http://www.stonetrip.com/developer/doc/authoringtool/installation.

Configuration For the ShiVa Editor

OS of development machine: Windows XP

Configuration For ShiVa Authoring Tool

Android:
- OS of development machine: Windows XP
- Software Versions:
  - Android SDK revision 8 (Android 2.2)
  - Android NDK revision 5b
  - Cygwin, GNU make package version 3.81
  - Apache Ant version 1.7.0
  - Java SDK version 1.5.21
  - Eclipse 3.5
  - ADT for Eclipse version 8.0.1
- Test device: Android 2.2.2 mobile phone.
iPhone and iPad
- OS of development machine: Mac OS 10.6.7
- Software Versions:
  - Xcode 3.2.6
  - iOS SDK 4.3
- Test devices: iPod Touch device with iOS 4.3.3, iPad2 with iOS 4.3.1

Files In Download Archive

In this section, we will give a description of the files in the archive file accompanying this article.

The folder named sky consists of skybox images back-crop.jpg, front-crop.jpg, left-crop.jpg, right-crop.jpg and top-crop.jpg. Those image files are needed to construct the sky surrounding the terrain (see the section named 'Step-By-Step Instructions' in part 2 of this series.)
The folder named author consists of flight-114-114.png, flight-48-48.png, flight-480-800.png, flight-640-960.png, flight-72-72.png, and flight-768-1024.png. Those files are needed to author the Simulator for Android and iOS environments (see the section named 'ShiVa Authoring Tool' in this tutorial.) In particular:
- flight-48-48.png is needed as the launch icon for Android
- flight-480-800.png is needed for the splash screen in Android
- flight-114-114.png is needed as the launch icon for iPhone/iPod Touch
- flight-640-960.png is needed for the splash screen in iPhone/iPod Touch
- flight-72-72.png is needed as the launch icon for iPad
- flight-768-1024.png is needed for the splash screen in iPad
Simulator.ste is an export of the Simulator from the ShiVa Editor. It contains the application with all its resources, including the code as well as the skybox images back-crop.jpg, front-crop.jpg, left-crop.jpg, right-crop.jpg, and top-crop.jpg. It represents the final state of the Simulator in the section 'Developing The Simulator In ShiVa Editor'. While following the instructions in that section if you experience an issue, Simulator.ste should be particularly useful. You can open two separate instances of the ShiVa Editor, import Simulator.ste in one instance as a reference and work on the other instance to carry out the instructions. (In order to import Simulator.ste in ShiVa Editor, from Main -> Projects bring up the Settings dialog and click Add to add a new project. Then, in that project open up Data Explorer and select Import -> Archive. In Import Archive text field specify the full path to Simulator.ste.)

Flight Dynamics

This section will give an overview of flight dynamics that will be simulated in the application.

The overall setting for the flight is a square terrain 4,096 x 4,096 units long on each side (for a discussion on the term 'unit' see the section named 'Units And Variables' in part 4 of this series). The three dimensional coordinates are at the terrain's center. A top view of the terrain is shown below. Note that the arrow indicates the positive section of an axis. Also, the dot inside the circle representing the Y axis indicates that the positive section of the Y axis is opposite of, i.e. away from, the terrain. From the figure, observe that the Y axis is the vertical axis whereas the X and Z axes are the horizontal axes representing the 2-dimensional plane on which the terrain lies.

Initially, the plane we simulate is at (x,y,z)=(0,50,2050) and its direction of travel is straight ahead.

Note that the simulation will not display the plane. Instead, the user will be 'sitting' at the cockpit and see the surroundings (terrain and sky) from the cockpit like a pilot would see. The figure below represents the surroundings from the pilot's perspective at the beginning of the simulation (In the simulation, the coordinate system is not displayed to the user). Elevations on the terrain change between, approximately, a maximum of 34 units and a minimum of -34 units.

The Simulator will perform four simple maneuvers: a pitch maneuver to go up or down and a banked turn maneuver to go left or right.

Pitch Maneuver

Let us first explain the pitch maneuver. As shown below, the pitch angle is the angle of rotation around the lateral axis that passes through the plane from wingtip to wingtip (http://en.wikipedia.org/wiki/Aircraft_principal_axes).

The figure below shows the sequence of pitch maneuver when the plane goes up.

Initially, the plane is traveling straight ahead parallel to the ground, which is a steady state condition. When the pitch starts, the nose of the plane starts going up. At the same time, the plane starts gaining altitude. At some point the pitch angle reaches a maximum. From that point on the nose of the plane starts going down, however, the plane continues to gain altitude. The nose of the plane keeps going down until the pitch angle reaches 0, i.e. the plane reaches the steady state again. From that point on, the plane will maintain its altitude. As a result of this maneuver, the plane has gained altitude between the two steady state conditions as shown below.

The pitch maneuver when the plane goes down is symmetric to the above. Initially, the plane is traveling straight ahead. When the pitch starts, the nose of the plane starts going down and the plane starts losing altitude. At some point the pitch angle reaches a minimum. From that point on the nose of the plane starts going up, however, the plane continues to lose altitude. The nose of the plane keeps going up until the pitch angle reaches 0, i.e. the plane reaches the steady-state again. From that point on, the plane will maintain its altitude. As a result of this maneuver, the plane has lost altitude between the two steady-state conditions.

Banked Turn Maneuver

Let us now describe the banked turn. As shown below, the roll (equivalent to the bank) angle is the angle of rotation around the longitudinal axis that passes through the plane from nose to tail. (http://en.wikipedia.org/wiki/Aircraft_principal_axes)

The figure below shows the sequence of a banked left turn maneuver.

Initially, the plane is traveling straight ahead, which is a steady state condition. When the maneuver starts, the bank angle starts increasing and the plane starts turning left. At some point the bank angle reaches a maximum. From that point on the bank angle starts decreasing, however, the plane continues to turn left. The bank angle keeps decreasing until reaching 0, i.e. it reaches the steady state again. From that point on, the plane will keep going straight ahead. As a result of this maneuver, the plane has changed direction to left between the two steady state conditions as shown below.

The banked turn to right is symmetric to the above. Initially, the plane is traveling straight ahead, which is a steady state condition. When the maneuver starts, the bank angle starts decreasing and the plane starts turning right. At some point the bank angle reaches a minimum. From that point on the bank angle starts increasing, however, the plane continues to turn right. The bank angle keeps increasing until reaching 0, i.e. reaching the steady state again. From that point on, the plane will keep going straight ahead. As a result of this maneuver, the plane has changed direction to right between the two steady state conditions.

In reality, planes could perform more sophisticated maneuvers such as banking and pitching at the same time. For simplicity, our plane will perform only one of those at a given time. Adding advanced maneuvers to the simulator is not a difficult task and is left to the interested reader as an exercise.

Motion Along The X,Y,Z Axes

To simulate the motion of the plane in three dimensional coordinates, we utilize a simplified model as described below.

During the pitch maneuver, as the plane goes up (down, respectively) the application code will increment (decrement, respectively) the value of Y coordinate by a fixed amount at each frame of the simulation independent of the pitch angle. The Y coordinate of the plane is fixed at all times except during a pitch maneuver.

During the banked turn maneuver, as a plane turns left (right, respectively) the application code will increment (decrement, respectively) the rotation of the plane around the Y axis by a fixed amount at each frame of the simulation.

Then, the X and Z coordinates of the plane are calculated at each frame using two variables, V, velocity of the plane, and Yg, rotation of the plane around the Y axis, shown below:

It is easy to see from the above figure that the velocity of the plane along the X and Z axes are -V*sin(Yg) and -V*cos(Yg), respectively. At each frame of the simulation, the application code obtains Yg via an API call and calculates those velocities. Then, it obtains the current X and Z coordinates of the plane and sets the new X and Z coordinates based on the calculated velocities. The magnitude of the velocity is constant throughout the simulation.

Closing Remarks For Part 1 Of This Series

In part 1 of this series, we introduced the flight simulator application named Simulator and explained the development process we followed while creating the Simulator. We also described the contents of the download archive accompanying this article. Finally, we introduced the basic flight maneuvers simulated in the application. In part 2, we will start describing how to develop the game using the ShiVa Editor.

HIGHLIGHTS OF THE DAY