VR prototypes reveal Facebook’s surprisingly critical research directions

Recently, Checked posted a video about Meta’s (meaning Facebook) hands-on time with prototype virtual reality (VR) headsets, and there are some really interesting bits in it. The video itself is over an hour long, but if you’re mainly interested in the technical angles and why they’re important to VR, read on as we’ll be highlighting each of the main research points.

As absurd as it may seem to many of us to have a social network spearheading meaningful VR development, it can’t be said that they don’t take it seriously. It’s also refreshing to see each of the prototypes presented by a researcher who is clearly excited to talk about their work. The big dream is to figure out what it takes to pass the “Visual Turing Test”, which means delivering images that match physical reality. Some of these critical elements may come as a bit of a surprise as they go in directions beyond resolution and field of view.

Demo of a solid-state varifocal lens with 32 discrete focus levels.

At 9:35 in the video, [Douglas Lanman] shows [Norman Chan] how important variable focus is to deliver a good visual experience followed by a tour of all the different prototypes they used to achieve that. Currently, VR headsets only display images in one focal plane, but that means, among other things, that it gets blurry when you bring a virtual object close to your eyes. (By the way, older people don’t find this part very strange since it’s a common accompaniment of aging.)

The solution is to change the focus based on where the user is looking and [Douglas] shows all the different ways this has been explored: from motors and actuators that mechanically change the focal length of the display, to a solid-state solution made up of stacked elements that can selectively converge or diverge light based on its polarization. [Doug]s pride and excitement is palpable and he really goes into great detail.

At 30:21 p.m [Yang Zhao] explains the importance of higher resolution displays and also talks about lenses and optics. Interestingly, the ultra-clear text rendering made possible by a high-resolution display isn’t the result of capture [Norman]the most attention. When high definition was combined with variable focus, it was the textures on pillows, the vibrancy of wall art and the patterns on walls [Norman] found he just couldn’t stop researching.

Next up at 39:40 is something really interesting presented by [Phillip Guan]. A VR headset must apply software corrections for distortion, and it turns out those corrections can be complex. An image not only gets slightly distorted when it goes through a lens, but this distortion changes in nature depending on the perspective. All of this needs to be corrected in software for a high-fidelity experience, but a real bottleneck is waiting for a physical prototype to be built, and this is complicated by the fact that different people have slightly different subjective experiences of distortion. To mention that [Phillip] shows a device whose purpose is to accurately simulate different physical headset designs (including different lenses and users) in software, allowing the study of different designs without actually building anything.

The final prototype – dubbed Starburst for reasons that will soon become clear – will be screened at 44:30, demonstrating the power of true High Dynamic Range. It looks the bulkiest, but that’s mainly because it essentially uses car headlights as the backlight. The purpose is not to blind users, but to deliver something important and missing. Why is high brightness so important? The answer is simple: real-world lighting levels far exceed anything a modern monitor (or VR headset) can deliver. This means that a headlight in VR actually only ever looks like one picture a headlight. It will never look real bright, not in the way your eyes and brain actually perceive the word. If headsets can deliver a true HDR experience, that’s about to change, and that’s exactly what this prototype delivers.

It’s clear that this direction is being taken very seriously, and it may come as a surprise that delivering a compelling visual experience goes well beyond higher resolution and a wider field of view. All really good VR ideas may have come from the 1960s, but this video is a great example of what goes into the basic scientific work of figuring out how to solve a problem.