Practical Application and Usage of the W3C Device Orientation API

Contents

• Introduction
• Our coordinate system revisited
• Limitations of using Euler angles
• Practical considerations for using device orientation in web applications
  • Avoiding gimbal lock
  • Screen orientation
  • World orientation
• Conversion to alternate device orientation representations
  • Using Rotation Matrices
  • Using Quaternions
• Example: An orientation-aware virtual reality viewer
• Cross browser compatibility
• Summary

Introduction

The W3C Device Orientation specification allows developers to plug in and use a device’s gyroscope and accelerometer data. Such capabilities could be used to build virtual and augmented reality experiences right in to today’s web browsers. Yet the learning curve required for web developers to utilize this raw information to date has been large.

In this article we’ll break down and explain practical usage of device orientation event data so web developers can incorporate deviceorientation event data in to their web applications in a practical way.

Our coordinate system revisited

In our previous article in this series we introduced the coordinate system in use within the W3C Device Orientation specification.

A full description of the coordinate system is provided in our previous article but to recap, here is the standard W3C Device Orientation coordinate frame:

Device orientation defines three types of rotation, which are are follows:

Alpha: The amount of rotation around the Z axis is known as alpha. The range is from 0 to 360 degrees and the current orientation in this axis is denoted with z.

Beta: The amount of rotation around the X-axis is known as beta. The range is from -180 to 180 degrees and the current orientation in this axis is denoted with x.

Gamma: The amount of rotation around the Y-Axis is known as gamma. The range is from -90 to 90 degrees and the current orientation in this axis is denoted with y.

The values provided back to us from the Device Orientation API are in the form of Tait-Bryan angles. Tait-Bryan angles are special forms of Euler angles that require 3 rotations around each of the 3 axis. Normal Euler angles require 3 rotations around only 2 axis (where one axis of rotation is repeated). The full set of Tait-Bryan rotations we could potentially use to describe the rotation of a device is therefore as follows:

x-y-z, x-z-y, y-x-z, y-z-x, z-x-y, z-y-x

We can obtain the current orientation of the device by registering a deviceorientation event listener against the window object in our web application. Ideally, we should aim to keep JavaScript processing we do in this event listener to a minimum and aim to manipulate the device orientation event data values only during a canvas paint function or while running a program loop such as requestAnimationFrame.

Here’s a simple event handler that captures the given deviceorientation event data values and stores them for us to use at a later time within our application:

var deviceOrientationData = null;

window.addEventListener('deviceorientation', function( event ) {
	deviceOrientationData = event;
}, false);

Limitations of using Euler angles

Tait-Bryan angles, and Euler angles in general, give us a good way to visualize the rotations of a device as it moves around in physical space. However, using Euler angles is known to introduce an issue when trying to model 3-axis rotations called gimbal lock.

If we imagine each of the rotation planes described above as a single circular rotation plane — a gimbal — then each of the three rotation planes work together to define device orientation movement.

As two of our gimbals start approaching the same parallel direction they start to have difficulty knowing which way to turn and they start spinning. When this occurs, we effectively lose one degree of freedom in our 3-axis modelling of device orientation. This creates a situation where we can no longer accurately model device rotation as the axis’ approach the extremes of their gimbal rotation.

Fortunately, other device orientation representations exist that can remove the effects of gimbal lock. In order to remove gimbal lock as an issue we need to represent device rotations in an alternative rotation system such as matrix-based or quaternion-based device orientation representations. We will run through both of these alternate orientation representations in a subsequent section.

Practical considerations for using device orientation in web applications

Let’s run through some considerations that can affect how device orientation data will behave in our web application.

Avoiding gimbal lock

Firstly, it makes sense for us to avoid gimbal lock as we discussed in the previous section. To do this we can convert the Tait-Bryan angles collected in our deviceorientation event listener to a different rotational representation such as a rotation matrix or a quaternion. We will demonstrate how to make that conversion in the alternative device orientation representations section below.

Matching the current screen orientation

Once we have an alternate orientation that can avoid gimbal lock such as a rotation matrix or a quaternion, we need to transform our new rotational representation to account for the current screen orientation of a device. Screen orientation changes can be defined as a Z-axis based transformation by the current screen orientation angle.

The W3C Device Orientation Events specification states that if the orientation of the screen changes (e.g. from portrait to landscape or vice versa) when the device is rotated that this does not affect the orientation of the coordinate frame relative to the device. However, because our user is now holding the device with a different screen orientation, the coordinate frame relative to the device will no longer match the current coordinate frame relative to the screen.

Web developers are required to fix this physical anomaly themselves according to the specification. Thankfully there are easy ways for web developers to do this based on registering an orientationchange event handler against the window object in our application.

Known collectively as the window.orientation API, by registering for orientationchange events a web developer can be informed whenever the device’s screen itself rotates.

Let’s go ahead and add an orientationchange event listener to our web application code to record screen orientation changes:

var currentScreenOrientation = window.orientation || 0; // active default

window.addEventListener('orientationchange', function() {
	currentScreenOrientation = window.orientation;
}, false);

We will discuss how we can apply this screen orientation data against different alternative device orientation representations in the next section.

Matching the application world orientation

The final consideration we may need to account for is how we want our orientation to behave relative to the world coordinate frame that we want to use in our final web application model.

For virtual reality or augmented reality applications we want our world coordinate frame to point out the back of the screen in the direction that our device (and therefore, our user) is currently “looking at”. More precisely, we want our world coordinate frame to reflect what can be “seen” behind the device’s screen when the user rotates their device around in physical space.

To do this we will need to make adjustments to our accumulated rotational representation that we can then finally apply to our web application. We will discuss how this can be done in more depth for each of the different alternative device orientation representations presented in the following section.

Conversion to alternate device orientation representations

In the limitations of using Euler angles section above we discussed how Euler angles tend to introduce gimbal lock in to our rotational coordinate system. Let’s take the device orientation data and convert it to one of the two alternative representations below to avoid this issue entirely.

You can decide to use either a Rotation Matrix or a Quaternions representation for your needs. For the sake of completion, we will cover both methods below.

The methods provided below expect that each of the alpha, beta and gamma values provided from the deviceorientation event are defined and are not null.

Using Rotation Matrices

A rotation matrix is a matrix that can be used to represent the physical rotation of our device within 3-dimensional space. In order to build a rotation matrix we need a way to represent matrix-based rotation for each of the x, y and z axis rotations. We shall call each of these axis matrices a component rotation matrix and we will then multiply these together to generate a combined rotation matrix representing the full 3-axis rotation of our device.

Based on the practical considerations we discussed above, we need to follow three steps to create a suitable rotation matrix we can use in our web applications.

1. Convert Euler angles to a Rotation Matrix representation
2. Calculate a screen coordinate frame transformation for our Rotation Matrix
3. Calculate a world coordinate frame transformation for our Rotation Matrix (optional)
4. Put it all together to compute a screen-adjusted and world-adjusted Rotation Matrix representation

R.1: Converting deviceorientation angles to a Rotation Matrix representation

In order to rotate the device by β degrees around the X axis we can use the following component rotation matrix:

In order to rotate the device by γ degrees around the Y axis we can use the following component rotation matrix:

In order to rotate the device by α degrees around the Z axis we can use the following component rotation matrix:

The combined rotation matrix can then be constructed by multiplying each of the component rotation matrices above in any of the Tait-Bryan rotation order combinations we described previously in this article. For example, we can represent a combined rotation matrix R when using a device rotation order of z-x-y as follows:

Multiplying each Z, X and Y component rotation matrix together we arrive at the following combined rotation matrix R:

Let’s add this to our codebase as follows:

var degtorad = Math.PI / 180; // Degree-to-Radian conversion

function getBaseRotationMatrix( alpha, beta, gamma ) {
	var _x = beta  ? beta  * degtorad : 0; // beta value
	var _y = gamma ? gamma * degtorad : 0; // gamma value
	var _z = alpha ? alpha * degtorad : 0; // alpha value

	var cX = Math.cos( _x );
	var cY = Math.cos( _y );
	var cZ = Math.cos( _z );
	var sX = Math.sin( _x );
	var sY = Math.sin( _y );
	var sZ = Math.sin( _z );

	//
	// ZXY-ordered rotation matrix construction.
	//

	var m11 = cZ * cY - sZ * sX * sY;
	var m12 = - cX * sZ;
	var m13 = cY * sZ * sX + cZ * sY;

	var m21 = cY * sZ + cZ * sX * sY;
	var m22 = cZ * cX;
	var m23 = sZ * sY - cZ * cY * sX;

	var m31 = - cX * sY;
	var m32 = sX;
	var m33 = cX * cY;

	return [
		m11,    m12,    m13,
		m21,    m22,    m23,
		m31,    m32,    m33
	];
};

So now we have a way to obtain a combined rotation matrix representation that matches the Tait-Bryan representation originally provided by the deviceorientation event. Next we need to apply device-specific transformations to this rotation matrix to match it up with both the current screen orientation and world orientation.

R.2: Fixing our rotation matrix frame relative to the current screen orientation

As we discussed in the practical considerations section above, any orientation representation we use must be modified to match the current screen orientation. Therefore we need to rotate our rotation matrix by the current screen orientation in order for our coordinate frame to match the current screen orientation and not the default screen orientation.

To obtain our screen-adjusted rotation matrix (Rs) we need to multiple the rotation matrix (R) we constructed in step 1 above by a Z-axis based transformation representing the current screen orientation angle (θ) offset from 0:

We construct our screen orientation transformation matrix (rs) as follows where θ is the value of currentScreenOrientation we collected above in our application code:

The construction of our screen orientation transform matrix (rs) can be represented in JavaScript as follows:

function getScreenTransformationMatrix( screenOrientation ) {
	var orientationAngle = screenOrientation ? screenOrientation * degtorad : 0;

	var cA = Math.cos( orientationAngle );
	var sA = Math.sin( orientationAngle );

	// Construct our screen transformation matrix
	var r_s = [
		cA,    -sA,    0,
		sA,    cA,     0,
		0,     0,      1
	];

	return r_s;
}

R.3: Fixing our rotation matrix frame relative to our application’s world orientation

In the same way we “fixed” our rotation matrix frame to account for the current screen orientation we may need to adjust it again to account for the configuration of our application’s “world” space. Depending on how the coordinate frame of your application is constructed it may be necessary to, for example, flip the rotation matrix so it points out of the back of the screen.

In our example we will again transform our rotation matrix for a virtual reality and augmented reality use case to point out the back of the screen for use in the three.js world space. More formally, we will perform an X-axis based transformation by 90 degrees to our screen-adjusted rotation matrix to fit three.js world space.

To obtain our world-adjusted rotation matrix (Rw) we need to multiple the screen-adjusted rotation matrix (Rs) we constructed in step 2 above by an X-axis based transformation (θ) of 90 degrees (converted to radians):

We construct our world orientation transformation matrix (rw) as follows:

The construction of our world orientation transform matrix (rw) can be represented in JavaScript as follows:

function getWorldTransformationMatrix() {
	var x = 90 * degtorad;

	var cA = Math.cos( x );
	var sA = Math.sin( x );

	// Construct our world transformation matrix
	var r_w = [
		1,     0,    0,
		0,     cA,   -sA,
		0,     sA,   cA
	];

	return r_w;
}

R.4: Computing our final rotation matrix representation

In this section we worked through and performed the following:

1. Construction of a rotation matrix from Tait-Bryan angles provided in the deviceorientation event.
2. Adjustment of our rotation matrix to account for the current screen orientation
3. Adjustment of our rotation matrix to account for our application’s world orientation

Now we need to put all of this code together so it can be called on each loop of our application’s execution.

Let’s define a two functions: matrixMultiply(a, b) that provides generic matrix multiplication and computeMatrix() that applies all the multiplications defined above to obtain a final rotation matrix we can finally use in our web application:

function matrixMultiply( a, b ) {
	var final = [];

	final[0] = a[0] * b[0] + a[1] * b[3] + a[2] * b[6];
	final[1] = a[0] * b[1] + a[1] * b[4] + a[2] * b[7];
	final[2] = a[0] * b[2] + a[1] * b[5] + a[2] * b[8];

	final[3] = a[3] * b[0] + a[4] * b[3] + a[5] * b[6];
	final[4] = a[3] * b[1] + a[4] * b[4] + a[5] * b[7];
	final[5] = a[3] * b[2] + a[4] * b[5] + a[5] * b[8];

	final[6] = a[6] * b[0] + a[7] * b[3] + a[8] * b[6];
	final[7] = a[6] * b[1] + a[7] * b[4] + a[8] * b[7];
	final[8] = a[6] * b[2] + a[7] * b[5] + a[8] * b[8];

	return final;
}

function computeMatrix() {
	var rotationMatrix = getBaseRotationMatrix(
		deviceOrientationData.alpha,
		deviceOrientationData.beta,
		deviceOrientationData.gamma
	); // R

	var screenTransform = getScreenTransformationMatrix( currentScreenOrientation ); // r_s

	var screenAdjustedMatrix = matrixMultiply( rotationMatrix, screenTransform ); // R_s

	var worldTransform = getWorldTransformationMatrix(); // r_w

	var finalMatrix = matrixMultiply( screenAdjustedMatrix, worldTransform ); // R_w

	return finalMatrix; // [ m11, m12, m13, m21, m22, m23, m31, m32, m33 ]
}

We can now call the computeMatrix() function whenever we like, typically during a loop in our application for example using requestAnimationFrame.

Using Quaternions

A quaternion can be used as another alternative representation for device orientation. A quaternion itself consists of two things. Firstly, every quaternion has an x, y and z component that represents the axis about which a device rotation occurs. Secondly, every quaternion has a w component that represents the amount of rotation that will occur about this axis. With these four numbers it is possible to describe device orientation perfectly while also avoiding introducing the problem of gimbal lock.

Based on the practical considerations we explained above, we need to follow three steps to create a quaternion we can use in our web application.

1. Convert Euler angles to a Unit Quaternion representation
2. Calculate a screen coordinate frame transformation for our Quaternion
3. Calculate a world coordinate frame transformation for our Quaternion (optional)
4. Put it all together to compute a screen-adjusted and world-adjusted Quaternion representation

Q.1: Converting deviceorientation angles to a Unit Quaternion representation

We can convert the Tait-Bryan alpha (α), beta (β) and gamma (γ) representation in to a Unit Quaternion (q) using the following formula:

This can be represented in JavaScript as follows:

var degtorad = Math.PI / 180; // Degree-to-Radian conversion

function getBaseQuaternion( alpha, beta, gamma ) {
	var _x = beta  ? beta- degtorad : 0; // beta value
	var _y = gamma ? gamma * degtorad : 0; // gamma value
	var _z = alpha ? alpha * degtorad : 0; // alpha value

	var cX = Math.cos( _x/2 );
	var cY = Math.cos( _y/2 );
	var cZ = Math.cos( _z/2 );
	var sX = Math.sin( _x/2 );
	var sY = Math.sin( _y/2 );
	var sZ = Math.sin( _z/2 );

	//
	// ZXY quaternion construction.
	//

	var w = cX * cY * cZ - sX * sY * sZ;
	var x = sX * cY * cZ - cX * sY * sZ;
	var y = cX * sY * cZ + sX * cY * sZ;
	var z = cX * cY * sZ + sX * sY * cZ;

	return [ w, x, y, z ];
}

Q.2: Fixing our quaternion frame relative to the current screen orientation

As we discussed in the practical considerations section above, any orientation representation we use must be modified to match the current screen orientation. Therefore we need to rotate our quaternion by the current screen orientation in order for our quaternion frame to match the current screen orientation and not only the default screen orientation.

To obtain our screen-adjusted quaternion (q’s) we need to multiple the quaternion (q) we constructed in step 1 above by a Z-axis based transformation quaternion (qs) representing the current screen orientation angle (θ) offset from 0 degrees:

We construct our quaternion transformation (qs) as follows where θ is the value of currentScreenOrientation we collected above in our application code:

The construction of our quaternion transform (qs) can be represented in JavaScript as follows:

function getScreenTransformationQuaternion( screenOrientation ) {
	var orientationAngle = screenOrientation ? screenOrientation * degtorad : 0;

	var minusHalfAngle = - orientationAngle / 2;

	// Construct the screen transformation quaternion
	var q_s = [
		Math.cos( minusHalfAngle ),
		0,
		0,
		Math.sin( minusHalfAngle )
	];

	return q_s;
}

Q.3: Fixing our quaternion frame relative to our application’s world orientation

In the same way we “fixed” our quaternion frame to account for the current screen orientation we may need to adjust it again to account for the configuration of our application’s “world” space. Depending on how the coordinate frame of your application is constructed it may be necessary to, for example, flip the quaternion so it points out of the back of the screen.

In our example we will again transform our quaternion for a virtual reality and augmented reality use case to point out the back of the screen for use in the three.js world space. More formally, we will perform an X-axis based transformation by 90 degrees to our screen-adjusted quaternion to fit our predefined world space.

To obtain our world-adjusted quaternion (q’w) we need to multiple the screen-adjusted quaternion (q’s) we constructed in step 2 above by an X-axis based quaternion transformation (θ) of 90 degrees (converted to radians):

We construct our world orientation transformation quaternion (qw) as follows:

The construction of our world orientation transform quaternion (qw) can be represented in JavaScript as follows:

function getWorldTransformationQuaternion() {
	var worldAngle = 90 * degtorad;

	var minusHalfAngle = - worldAngle / 2;

	// Construct the world transformation quaternion
	var q_w = [
		Math.cos( minusHalfAngle ),
		Math.sin( minusHalfAngle ),
		0,
		0
	];

	return q_w;
}

Q.4: Computing our final quaternion representation

In this section we worked through and performed the following:

1. Construction of a quaternion from Tait-Bryan angles provided in the deviceorientation event.
2. Adjustment of our quaternion to account for the current screen orientation
3. Adjustment of our quaternion to account for our application’s world orientation

Now we need to put all of this code together so it can be called on each loop of our application’s execution.

Let’s define a two functions: quaternionMultiply(a, b) that provides generic quaternion multiplication and computeQuaternion() that applies all the multiplications defined above to obtain a final quaternion representation we can finally use in our web application:

function quaternionMultiply( a, b ) {
	var w = a[0] * b[0] - a[1] * b[1] - a[2] * b[2] - a[3] * b[3];
	var x = a[1] * b[0] + a[0] * b[1] + a[2] * b[3] - a[3] * b[2];
	var y = a[2] * b[0] + a[0] * b[2] + a[3] * b[1] - a[1] * b[3];
	var z = a[3] * b[0] + a[0] * b[3] + a[1] * b[2] - a[2] * b[1];

	return [ w, x, y, z ];
}

function computeQuaternion() {
	var quaternion = getBaseQuaternion(
		deviceOrientationData.alpha,
		deviceOrientationData.beta,
		deviceOrientationData.gamma
	); // q

	var worldTransform = getWorldTransformationQuaternion(); // q_w

	var worldAdjustedQuaternion = quaternionMultiply( quaternion, worldTransform ); // q'_w

	var screenTransform = getScreenTransformationQuaternion( currentScreenOrientation ); // q_s

	var finalQuaternion = quaternionMultiply( worldAdjustedQuaternion, screenTransform ); // q'_s

	return finalQuaternion; // [ w, x, y, z ]
}

We can now call the computeQuaternion() function whenever we like, typically during a loop in our application for example using requestAnimationFrame.

Example: An orientation-aware virtual reality viewer

Applying everything we have covered in this article we can now create virtual reality and augmented reality experiences right in the web browser!

We have created a demonstration virtual reality viewer web application that utilizes both quaternions and rotation matrix rotation representations and uses the three.js JavaScript library to render a cubemap-based scene.

Here are a couple of screenshots from our demo virtual reality viewer running in Opera 20 for Android:

You can find a live version of this virtual reality demonstration here (best viewed on mobile) and the source code along with a helper library can be found on Github.

Cross browser compatibility

Since the publication of our previous article in this series the cross-browser compatibility of deviceorientation data values has improved considerably between different web browsers.

At the time of writing the above provided rotation transformations works correctly on most Android and iOS-based browsers with a few important caveats that web developers need to keep in mind:

1. The functions provided above expect deviceOrientationData.alpha, deviceOrientationData.beta and deviceOrientationData.gamma data values provided from the deviceorientation event to be defined and not null. Web developers should check that each property is provided and is not null otherwise the calculations will not work correctly and the developers should provide some alternative fallback functionality (e.g. manual orientation controls).
2. The window.orientation API is not currently supported in Gecko-based browsers (e.g. Firefox). There is a proposal to add screen orientation change detection (in degrees from the default screen orientation position) to the W3C Screen Orientation API but at the time of writing this is not available.
3. iOS-based browsers currently return deviceOrientationData.alpha as an arbitrary non-compass-based value. Therefore, in iOS-based browsers if you need world-accurate values you will need to replace the deviceOrientationData.alpha value stored with (360 - deviceOrientationData.webkitCompassHeading). See this bug for further details.

Summary

In this article we have presented two methods we can use to model a device’s 3-dimensional movement without introducing gimbal lock: rotation matrixes and quaternions.

Using the provided alternative device rotation representations we can easily account for additional usage considerations such as screen orientation changes and world orientation frames to produce rotational models that can be used within advanced real-world web applications.

Device Orientation is an exciting capability that is already shipping and available in the majority of mobile web browsers. Where it has been previously difficult to determine how to use this API in advanced 3D modelling applications we hope this article helps web developers in understanding more about device orientation and we hope you will consider using it in your next project!