Ex-Valve Engineers Explain the CastAR's Benefits Over the Competition

This article first appeared on USgamer, a partner publication of VG247. Some content, such as this article, has been migrated to VG247 for posterity after USgamer's closure - but it has not been edited or further vetted by the VG247 team.

"We can do everything Oculus can do and more," claims ex-Valve inventor Jeri Ellsworth as she tells me about her upcoming CastAR headset over Skype.

That's a bold claim, but Ellsworth and her software programming colleague Rick Johnson (also a former Valve staffer) at Technical Illusions are confident that they can deliver the same experience as the Oculus Rift, only better, in addition to a host of other potentially revolutionary features.

How does it work, you ask? The successfully funded CastAR Kickstarter campaign has a rather detailed breakdown, but the basic gist is that it's a pair of spectacles (built to fit over prescription glasses) that projects an image onto a retro-reflective grey surface, so the wearer sees what appears to be an augmented reality hologram. We call this "Projected AR."

Additionally, an optional shutter clip-on attachment will block out other light, bouncing the image right back to the player's eyeballs without the need to lay out any retro-reflective material. This VR peripheral essentially makes it like an Oculus Rift, with the game occupying your entire field of view. Furthermore, it offers "True AR," which overlays images in front of the rest of the world.

To see this content please enable targeting cookies. Manage cookie settings

So what's its leg up on Oculus, then? The purported advantages are numerous, but one of Ellsworth's chief selling points is that CastAR eliminates all the nausea and motion sickness issues that arose from the Oculus.

"We have the projected AR system where you can project out to a retro-reflective screen that you roll out on the table or on your wall in different configurations. This is by far the most comfortable way to do any AR or VR experience because your eyes are focusing on a nice comfortable distance, and it's immune to all of the issues of nausea and headaches due to displays," Ellsworth explains.

"Motion sickness is a very complicated mechanism and not the same for all. We did a lot of research into this at the previous place I worked and it mostly comes down to conflicts between the inner ear and what is being presented to the user," she explains. "Very high percentages of users -- almost 100 per cent -- reported sickness, feeling dizzy and/or headaches if they were cut off from the real world and present motion in the graphics that does not match the inner ear. We found with CastAR that almost no users complain [of] these issues, because we have a very accurate head tracker that allows us to generate one-to-one graphics -- you move 10 centimeters and the graphics are adjusted to look 10 centimeters offset -- and the user can see the real world at anytime or during use."

Ellsworth claims she's shown the CastAR to somewhere in the neighborhood of 5,000-6,000 users at various trade shows and events over the last year or so and not a single person has complained of motion sickness. "One of the issues when you have a display jammed right against your face is that an error or misalignment on the lenses is magnified greatly, which can cause headaches. When you turn everything inside out with this reflective sheen that we put out, those errors pretty much go away because of distances are so long."

She also notes that the CastAR's shutter clip-on will produce cleaner and crisper images than that of the Oculus Rift's near-to-eye screens. "We're pretty proud of our VR clip-on because they're extremely low distortions," Ellsworth says. "Oculus has a lot of pixel-warping on the end and they have to throw away a lot of pixels on their display. We get to use pixels clear out to the edge of the display, so it makes it much easier for game developers to do things like put text up or have HUDs because there's not a lot of warping."

Another advantage CastAR allegedly has over Oculus is its head-tracking tech. "Our tracking system's probably the gold standard out there for consumer level tracking," Ellsworth boasts. "It's seven millimeter accurate, you can move anywhere within a 3D volume and we'd know exactly where your head position is. You can move laterally, you can step back, head rotation -- we can detect it all. That's something that Oculus can't do. They just have a gyro."

"Gyros drift. They're not that accurate over time," Johnson adds, before noting that the glasses themselves are extremely lightweight. "They're under 100 grams, so they're slightly heavier than a pair of sunglasses and they're very comfortable. They're also designed to fit on top of prescription glasses. There's no calibration step or trying to pack your head and your glasses into an Oculus Rift-type thing which adds some discomfort."

While the VR clip-on shutters will be subject to some of the Oculus' follies, it won't be as harsh to the user's system due to its light weight and low latency. "Generally it's the latency of tracking that is the additional problem with head tracking technologies and nausea," explains technical director Jenesee Grey. "It was discovered, in the 80s, that 100ms was the most latency (on average) that a user could accept before the lag between movement and updating the visuals started to make people feel dizzy and disturbed. This is subjective, of course."

"The VR aspect clip will likely have many of the same issues Oculus has, except the latency is better and it is much lighter, so you might avoid the headache aspect."

While Technical Illusions is confident that its device is an improvement over the Oculus, what it's really interested in is what CastAR does that isn't like the Rift. Where Palmer Luckey's cyberpunk apparatus seeks to replace your world with that of a virtual one, CastAR seeks to transplant the digital landscape into our reality.

One of the problems with the Rift -- besides motion sickness -- is that it's an incredibly isolating experience. Since its debut people have likened it to The Matrix, forgetting that those films were actually about people living their whole lives in disgusting robo-insect sacs. And hey, there's a place for that. Some games are designed to evoke loneliness, after all. The Amnesia series springs to mind, for example. But sometimes, we want our human interaction too, and that's where CastAR's augmented reality angle comes into play.

By using the CastAR's reflective drapery players will be able to see each other -- and the rest of the world -- just fine, but all CastAR wearers can see a different image by looking at the retro-reflective surface. "The property of the retro-material is that it bounces most of the light back towards the wearer of the glasses with very little scatter i.e. once you get shoulder to shoulder, there is not enough light scattering that you can make out what the other person is seeing/projecting," explains Johnson. "I could be projecting a scene from a racing game, you could be next to me playing an FPS, a third person could be playing an RPG, and none of us would be the wiser." If this takes off, it could effectively make splitscreen a thing of the past.

"The other advantage of using the surfaces is if you want to do more social type games," Johnson states. "Having your vision obscured by virtual reality, you're not going to see the people you're playing across from. You're not going to be able to interact with the physical world... With projected augmented reality we're bringing the social back into gaming where if you do something vicious to your opponent across the table you can see his expressions and look at him and stare him straight down the face. You can stand next to each other and become a little more physical as you're playing games."

Ellsworth says that for certain types of games people wanted to have a greater awareness for their actual body rather than their digital avatar. "There's actually a lot of interest in our system for racing cockpits and flight simulators," she says, noting that she doesn't just mean flight sim games, but actual real world flight simulators. "They [players, potential pilots] want to be able to see their hands to control flight knobs," she explains. Space Camp may never be the same again.

Part of this is due to CastAR's low cost. Right now the plan is to deliver a headset and retro-reflective screen for $189, but Johnson notes this could go down in price if there's enough demand. "The actual reflective material is pretty cheap," Johnson says. "For a square meter it's not that much more expensive than wallpaper that you buy in bulk. So if a person wanted to do a 10[foot] x 10 room or 20 x 20 room, it's not that unheard of versus how much you'd spend buying a normal flatscreen TV."

Of course, few people have an entire unused spare room to turn into a holodeck, but the potential is there. When asked what game ideas the pair could foresee CastAR delivering, Johnson explains a concept that sounds like a cross between Tron's light bikes and Johann Sebastian Joust (or more accurately B.U.T.T.O.N. by the same developer). "One of the ideas we're going to explore originally came from an old arcade game where two players are running down a corridor and there's only a doorway in the center where one person's going to fit. So instead of using a joystick and trying to navigate it, now you can actually physically use your body to push other people around."

"We're finding ways to bridge the physical world with the virtual world. Nobody else is quite doing that with an augmented reality headset."

So if the CastAR is so great, why didn't Valve stick with it, you ask? No one knows for sure, but Ellsworth says Valve just didn't get it. "Valve is an interesting place with not great communication sometimes and so getting our message out within the company was nearly impossible," she says. "There was a lot of misunderstandings about what we were trying to do. Even though there was very much a lot of interest in VR, people couldn't really get over the fact that the system I was developing used a surface also. They had a hard time believing that I would be able to make the VR clip-ons to make it do everything. So it just didn't get much traction. It just wasn't highly valued, I guess."

That being said, she did get some support from her colleagues while she was tinkering with the prototype. "We had to come up with a system that could easily test out these ideas and iterate upon them, so often we had people playing in the hardware lab late at night," Johnson remembers.

"It got so bad that people wouldn't leave the hardware lab. We couldn't get our work done," Ellsworth adds. "We knew we were onto something at that point."