An overview of roomscale tracking technologies – Oct 2018

Some will argue (and I am one them), that the best feature of VR is roomscale. The freedom to move around and explore your virtual surrounding contributes to the sense of presence and agency, which really sets VR apart from any other media. This is often referred to as 6 degrees of freedom (or 6DOF). I explain why 6DOF is so important in another blog post.

In this post I want to dive into the different methods used to achieve 6DOF tracking in VR headsets and peripherals, the pros and cons of each method and how they work. There are three major VR tracking methodologies currently on the market:

• Outside-in tracking
• Marker-based inside-out tracking
• Markerless inside-out tracking

All of the major VR headsets fall into one of those categories, but they all implement the design in different ways. Lets dive into this.

Outside-in tracking

In this setup, the sensors are placed outside of the tracking area, and are looking in.

There are two good examples of systems that use this technique.

Outside-in tracking: Oculus Rift

The Oculus Rift uses cameras, placed on the edges of the tracking area, and facing inward.

The headset and controllers have IR LEDs, to allow the cameras to track their position. The exact shape and size of the LED pattern is known, so by analyzing the camera frames the software can deduce the position and rotation of the headset in space. Adding cameras helps prevent dead spots and increases accuracy.

Disadvantages:

• Analyzing the camera frames requires a lot of processing
• The processing power required is related to the number of cameras. Adding cameras means more processing is needed to analyze the images.
• The system loses accuracy with distance, as the LED pattern takes less and less pixels in the camera image, the further it is from the camera.
• The cameras need to be connected to the computer (with USB cables).

That last disadvantage is a big one. For example, it completely rules out backpack computers. Also, running USB cables over long distances is complicated and expensive, so big spaces are also a problem.

Outside-in tracking: Optitrack

Optitrack take this approach to it’s extreme. This high end system originally was developed for motion capture and today is also used for high-end and large scale VR system. The Optitrack system places cameras and IR beamers on the perimeter of the tracking area. A lot of cameras. Dozens of them. Any object that needs to be tracked has reflective little balls placed on it. The software will look for these balls / dots and will extract the 3D position and rotation of objects from the camera frames.

Advantages:

• The tracked objects can be passive, no power source required (because the cameras have built-in IR lights)
• Very accurate
• Can cover spaces of any size or shape

Disadvantages:

• Very expensive. A modest setup can easily cost $50k.

Marker-based Inside-out tracking

In this setup the sensors are inside the tracked area, on the object being tracked (in our case, the VR headset) and pointing outwards. Fixed markers are placed outside the tracked area to assist the system to perform tracking and also to anchor it to a known real-world coordinate system.

Marker-based Inside-out tracking: HTC Vive and SteamVR

The tracking system used by the Vive was not developed by HTC, but rather Valve (makers of Steam). It’s called SteamVR. This might surprise some of you, but SteamVR is a Marker-based Inside-out system.

The most important thing to understand is that the SteamVR basestations are not sensors, despite them often (and wrongfully) referred to as such. The basestations are IR markers, they emit an IR laser that is picked up by the headset. The sensors are on the headset and the controllers.

Advantages:

• The basestations are passive, in the sense that they do not communicate with the computer. They only require power. No need to run a usb cable to the computer. This allows the use of backpack computers and makes setup a lot easier.
• Many headsets can share a single set of basestations.
• Tracking is very efficient and requires very little processing, and many objects can be tracked without losing performance or accuracy.

Disadvantages:

• One or more basestations are required.

The SteamVR tracking system has an open-licencing, meaning that anyone can build a sell a VR headset that uses it. Pimax have done exactly that, and are the second SteamVR headset to come to market. I personally hope that more will follow. IMHO, the SteamVR tracking system is superior to all the others.

It’s also worth mentioned that the new SteamVR V2.0 supports up to four basestations in the same space, and a much large tracked area. There are indications more than four basestations might be possible, and will be enabled in the future with software updates.

Marker-based Inside-out tracking: Honorable Mention

There are few other tracking system that are Marker-based Inside-out, they are either very new or have not gained much traction, but are still worth mentioning as they do serve niche markets (specifically, very large tracking areas).

• Antilatency.com – Uses a camera mounted on the headset and a strips of LED lights placed on the floor, that serve as the markers.
• Startracker.xyz – A camera is mounted on the headset, pointing upwards. LED lights are placed on the ceiling. The system is designed to cover large tracking areas.

Markerless Inside-out tracking: Windows Mixed Reality headsets (Acer, Samsung, Lenovo, etc)

Markerless Inside-out tracking is similar to Marker-based, just without any markers installed (duh). Sensors (usually cameras) are inside the tracked area (usually on the headset) and are looking outwards. A good example for this tracking system are the new Windows Mixed Reality headsets (which are really just VR headsets and not Mixed Reality at all).

These headsets have two (or more) cameras, usually facing slightly outwards and down. Using complex computer vision algorithms, these cameras act as a 3D scanner, mapping the environment around the headset and using that information to calculate position and rotation in space.

The same cameras are also used to track the controllers, which have white LED lights on them. The controllers use Outside-in tracking, while the headset uses Inside-out tracking. It’s quite neat, actually.

The upcoming Oculus Quest, will have 4 cameras on the headset.

Advantages:

• Setup is very easy, no need to install basestations or markers, almost no calibration is required. Put on the headset and you’re good to go.
• Reduce cost, less parts are needed
• No need to run cables (power or USB)

Disadvantages:

• The cameras are not IR and operate in the visible light spectrum. These headsets don’t work in the dark, or when the lights are changing quickly (like a party or event).
• The system needs fixed objects in the environment to lock on to. If there are many people around, for example at a trade show or other events, tracking will stutter and fail.
• It’s not locked to a real-world coordinate system. The headsets can track where they are related to the starting position. The problem is that they don’t know this absolutely but rather relatively. This is not a big deal for one headset, but if you want people to see each-other in a multi-user experience, this is a big problem.

Conclusion

Each system has it’s own set of pluses and minuses. My own personal favorite is SteamVR, which I believe is superior to all the others. To explain why, I will use an example.

A few months ago I was invited to demo my VR Art Gallery at Creative Tech Week in NYC. The venue was small, dark and packed to the brim with people. I used my HTC Vive, and installed one Basestation on a tripod standing by the wall. I clamped the other basestation to a light fixture all the way across the room, and powered it with a 12V battery. It took 5 minutes to setup and worked perfectly the entire night. No other tracking technology would have worked in that environment. WMR headset would have failed due to lack or light and too many people around. The Rift camera would not reach my laptop from the other side of the room, and running a USB cable would have been impossible.

Back to blog posts