By Lori Cameron
While eyeglasses have improved the lives of millions of people, many wearers are painfully aware of how weird eyeglasses can make their eyes look—something sample frames at the eyeglass store don’t reveal.
Now, researchers have developed a one-of-a-kind virtual try-on system that shows wearers exactly how they will look in their new glasses.
“We present a system for virtually trying on prescription eyeglasses. Our system acts as a virtual mirror, allowing users to try on a variety of eyeglasses with corrective lenses based on their prescription,” write Qian Zhang of Nanyang Technological University in Singapore and her team in their article “A Virtual Try-On System for Prescription Eyeglasses,” which appears in IEEE Computer Graphics & Applications.
The system also differs from other virtual systems, which only duplicate the in-store experience, by providing a more accurate and realistic view of how the lenses will change the appearance of the wearer.
How eyeglasses can distort a wearer’s eyes
As can be seen in the photos below, the eyes of a near-sighted wearer appear smaller through the lens, while the eyes of a far-sighted wearer appear larger.
Trying on in-store frames doesn’t help because the lenses are not corrective.
According to the authors, the problem is that “the traditional process of trying on and picking new eyeglasses in a brick-and-mortar shop has a significant shortcoming: eyeglasses on the display are equipped with demo lenses that have zero corrective power, and thus refraction does not deform the eyes.”
The authors also point to current virtual try-on systems—such as GlassesUSA, JCPenney, FittingBox, and SmartBuyGlasses—that don’t show how a person’s eyes will distort through prescription lenses.
“To the best of our knowledge, all available virtual try-on solutions ignore the refraction effects caused by eyeglasses lenses,” the authors say.
The system developed by Zhang and her team gives users a more realistic experience by modifying an input video and virtually inserting prescription eyeglasses—allowing users to see themselves in a “virtual mirror.”
How the virtual eyeglass try-on system works
The virtual try-on system places prescription eyeglasses onto the user’s face and simulates important changes to the eyes after refraction, reflection, or shadows cast on the face. The system uses three simple pieces of information to render the final image:
- Image sequence – video snapshots of the user without eyeglasses captured with a color camera.
- Eyeglass prescription – the value of all parameters in the user’s eyeglass prescription needed to correct blurred vision due to refractive errors, including myopia, hyperopia, presbyopia, and astigmatism.
- Eyeglass frames – the researchers purchased 3D models for six different commercially available eyeglass frames from TurboSquid.
Duplicating a process that opticians already use
The authors say their system was inspired by the traditional eyeglass manufacturing pipeline followed by opticians.
The proposed system requires a video of the user, the user’s eyeglass prescription, and a 3D model of the desired eyeglass frame.
In the virtual eyeglass generation stage shown above, the authors first create a virtual 3D representation of the desired prescription eyeglasses, by (a) positioning the eyeglasses frame on the user’s face, (b) creating a parametric lens model based on the user’s eyeglasses prescription, and (c) cutting the corrective lenses to size before mounting them into the frame.
In the second stage, video synthesis, the authors use image-based rendering to generate a synthetic image sequence where the prescription eyeglasses are virtually inserted, showing the effects of refraction, reflection, and shadows.
The proposed system generates a 3D representation of the corrective lenses mounted into the eyeglass frame, giving wearers an accurate picture of how the glasses make them look.
A view of how light reflects off the lenses and how the frames will cast shadows on the wearer’s face help round out the resulting images.
Implementing and testing the virtual try-on system
In the illustration below, the authors compare their system with another online store try-on system to show how much more accurately their results compare to real images.
Extensive testing was done on 20 subjects who were then asked to rate the video sequences by how realistic they were. The study showed that subjects found the results that included refraction and reflection to be most accurate.
The refined approach renders the refraction caused by the corrective lens and adds reflections and shading, ultimately allowing wearers to make more informed eyeglass purchasing decisions.
Research related to vision, visual analytics, and eye health in the Computer Society Digital Library:
- Third Eye: A Shopping Assistant for the Visually Impaired
- Driver Gaze Region Estimation without Use of Eye Movement
- Forget the Funny Glasses
- Tracking Developers’ Eyes in the IDE
- Eye-Controlled Interfaces for Multimedia Interaction
- Eye Tracking in Computer-Based Visualization
- Eye Tracking for Personal Visual Analytics