The next generation of 360-degree cameras will revolutionize multimedia—allowing you to create your own high-resolution panoramic videos and VR gaming machinery like a pro.
The problem is, many current models suffer from bulkiness, poor video quality, low battery life, and painfully slow render times.
To begin solving these problems, developers created the award-winning Kandao Obsidion camera.
“A good-quality 3D, 360-degree camera is still too expensive for mass adoption, costing as much as $20,000,” say the authors of “360-Degree Virtual-Reality Cameras for the Masses” in the January-March 2018 issue of IEEE MultiMedia.
“To make the technology more accessible to the public, a device must be affordable, portable, reliable, high quality, and user friendly. We describe the challenges in meeting these goals and the techniques that Kandao—a VR startup company based in China—used to conquer them when designing its Obsidian cameras.”
Newer model: Kandao QooCam, a switchable 4K 360° & 3D camera
Jack Tan, co-founder of Kandao, and Rui Ma, a Kandao researcher fellow, teamed up with Gene Cheung of the National Institute of Informatics in Tokyo, an inter-university research institute corporation and a research organization of information and systems, to design the Kandao Obsidian, which won the German Design Award 2018, one of the most prestigious design competitions in the world, for the Excellent Product Design category.
Capturing better stereoscopic panoramas
In 1992, omni-directional stereo (ODS) technology relied on a slit camera that rotated around an inter-pupil-distance (IPD) circle to scan the real world.
“The image slices are stacked to form a rectangular panoramic image. An inherent drawback of this and other early methods is that they do not work on dynamic scenes with moving objects,” the authors say.
An updated version of ODS takes a computational photography approach.
Instead of actual scanning, the rotating slit camera is synthesized using computational imaging and vision techniques.
Building a user-friendly camera
The Kandao Obsidian camera also has a rich, compact interface and shape, making it easy to rotate and hold in your hand or mount on a stand.
With six fisheye lenses, the Kandao Obsidian camera can see everything in a seamless panoramic view.
Finding an easy way to create a synthetic dataset
The authors say that building a dataset for computational ODS is very difficult to do, because “one would need to accurately map every point densely in 3D and generate two slit cameras scanning the scene by rotating around an IPD circle.”
Instead, they took a computational approach by constructing a synthetic dataset with rich information: 3D models (mesh and texture), simulated virtual cameras, pixel correspondences, depth data, object labels, and segmentation boundaries.
Finding a new optical flow algorithm for six fisheye lenses
To build a high-quality ODS with six lenses, a highly accurate optical flow algorithm is necessary. The figure below shows how difficult this is to do.
The authors have proposed a neural network that is first trained on popular open-source datasets and further fine-tuned on their own synthetic dataset.
The deep-learning-based method proves to be a full 100 times faster than traditional optimization-based optical flow. The Kandao team compared their neural network with FlowNet 2.0, a state-of-the-art deep learning method.
Kandao’s CNN method is more accurate, efficient, and compact than FlowNet 2.0, the authors contend.
The challenges ahead
Several challenges remain before 360-degree VR cameras become more widely used. One challenge involves improving optimal flow for more temporal-consistent dense pixel correspondence. Another involves finding a way to encode and package VR video into sub-streams for faster transmission over bandwidth-limited networks.
“Many research groups are currently addressing these and other challenges, and the day will soon arrive when VR video becomes truly ubiquitous—easily produced, consumed, and enjoyed by the masses,” the authors say.
Related research on photography in the Computer Society Digital Library:
- The Future of Smart Photography
- Experimental Platforms for Computational Photography
- From Photography to Ubiquitous Capture Systems
- Shedding Light on Creativity: The History of Photography
- Social Snapshot: A System for Temporally Coupled Social Photography