Facebooktwittergoogle_plusredditlinkedintumblrmail

By Lori Cameron

While eyeglasses have improved the lives of millions of people, many wearers are painfully aware of how weird eyeglasses can make their eyes look—something sample frames at the eyeglass store don’t reveal.

Now, researchers have developed a one-of-a-kind virtual try-on system that shows wearers exactly how they will look in their new glasses.

“We present a system for virtually trying on prescription eyeglasses. Our system acts as a virtual mirror, allowing users to try on a variety of eyeglasses with corrective lenses based on their prescription,” write Qian Zhang of Nanyang Technological University in Singapore and her team in their article “A Virtual Try-On System for Prescription Eyeglasses,” which appears in IEEE Computer Graphics & Applications.

The system also differs from other virtual systems, which only duplicate the in-store experience, by providing a more accurate and realistic view of how the lenses will change the appearance of the wearer.

How eyeglasses can distort a wearer’s eyes

As can be seen in the photos below, the eyes of a near-sighted wearer appear smaller through the lens, while the eyes of a far-sighted wearer appear larger.

eyeglasses1

Prescription eyeglasses introduce refraction effects that change the wearer’s appearance. The eyes of a person wearing corrective lenses for nearsightedness (a) will appear smaller compared with (b) wearing nonprescription lenses, whereas the eyes of a person wearing lenses for farsightedness (c) will appear larger.

Trying on in-store frames doesn’t help because the lenses are not corrective.

According to the authors, the problem is that “the traditional process of trying on and picking new eyeglasses in a brick-and-mortar shop has a significant shortcoming: eyeglasses on the display are equipped with demo lenses that have zero corrective power, and thus refraction does not deform the eyes.”

The authors also point to current virtual try-on systems—such as GlassesUSA, JCPenney, FittingBox, and SmartBuyGlasses—that don’t show how a person’s eyes will distort through prescription lenses.

“To the best of our knowledge, all available virtual try-on solutions ignore the refraction effects caused by eyeglasses lenses,” the authors say.

The system developed by Zhang and her team gives users a more realistic experience by modifying an input video and virtually inserting prescription eyeglasses—allowing users to see themselves in a “virtual mirror.”

How the virtual eyeglass try-on system works

The virtual try-on system places prescription eyeglasses onto the user’s face and simulates important changes to the eyes after refraction, reflection, or shadows cast on the face. The system uses three simple pieces of information to render the final image:

  • Image sequence – video snapshots of the user without eyeglasses captured with a color camera.
  • Eyeglass prescription – the value of all parameters in the user’s eyeglass prescription needed to correct blurred vision due to refractive errors, including myopia, hyperopia, presbyopia, and astigmatism.
  • Eyeglass frames – the researchers purchased 3D models for six different commercially available eyeglass frames from TurboSquid.

Duplicating a process that opticians already use

The authors say their system was inspired by the traditional eyeglass manufacturing pipeline followed by opticians.

eyeglasses5

Overview of the pipeline for virtual try-on of prescription eyeglasses.

The proposed system requires a video of the user, the user’s eyeglass prescription, and a 3D model of the desired eyeglass frame.

In the virtual eyeglass generation stage shown above, the authors first create a virtual 3D representation of the desired prescription eyeglasses, by (a) positioning the eyeglasses frame on the user’s face, (b) creating a parametric lens model based on the user’s eyeglasses prescription, and (c) cutting the corrective lenses to size before mounting them into the frame.

In the second stage, video synthesis, the authors use image-based rendering to generate a synthetic image sequence where the prescription eyeglasses are virtually inserted, showing the effects of refraction, reflection, and shadows.

Read research on augmented reality for eyeglasses here

The proposed system generates a 3D representation of the corrective lenses mounted into the eyeglass frame, giving wearers an accurate picture of how the glasses make them look.

A view of how light reflects off the lenses and how the frames will cast shadows on the wearer’s face help round out the resulting images.

eyeglasses7

The authors use the Monte Carlo integration process for shading computation. (a) To estimate the shading at a point p, rays are traced from p to several directions of the viewing hemisphere. (b) Once virtual eyeglasses are inserted, the eyeglasses frame blocks some of these rays (rays in magenta). This results in shadows cast on the user’s face, which the approach is able to simulate.

Implementing and testing the virtual try-on system

In the illustration below, the authors compare their system with another online store try-on system to show how much more accurately their results compare to real images.

eyeglasses6

Comparison of eyeglasses virtual try-on results.

Extensive testing was done on 20 subjects who were then asked to rate the video sequences by how realistic they were. The study showed that subjects found the results that included refraction and reflection to be most accurate.

The refined approach renders the refraction caused by the corrective lens and adds reflections and shading, ultimately allowing wearers to make more informed eyeglass purchasing decisions.

The other writers of the research are Yu Guois of University of California, Irvine; Pierre-Yves Laffontis of Lemnis Technologies; Tobias Martinis of VirtaMed; and Markus Grossis of ETH Zurich.

 

Research related to vision, visual analytics, and eye health in the Computer Society Digital Library: