Makeup detection Preprocessing Makeup decomposition Dataset ...
Graduate Category: Engineering and Technology Degree Level: Ph.D. Abstract ID# 643
SMILER: Faces Behind The Makeup Handong Zhao, Shuyang Wang, Joseph Robinson, Advisor: Professor Yun Fu Technical Details
Abstract Digital technology capable of manipulating facial makeup has grown to be a hot commodity in the beauty industry. Nonetheless, preexisting technologies only process bare faces, i.e., can apply products, but not remove. We have developed the technology to solve this inverse problem utilizing the cuttingedge computer vision and machine learning algorithms. In response to the abundance of applications and novelty of our framework, we have claimed patent rights on our system and published technical papers, respectively. With this, we are developing a startup based on our found technology.
DISTRIBUTION OF MAKEUP STATUS IN OUR DATASET
234 147 163
Background Current digital facial makeup Apps, e.g. L’Oréal’s Makeup Genius, allows customers to “try on” their makeup products digitally. In other words, the app enables them to add various types of makeup (e.g., eye-shadow, lipstick, etc.) using manually visual simulations. However, the technology has a serious limitation: it can only apply make-up to a bare face to accurately predict its resulting appearance.
Business Plan SMILER technologies are highly generalizable, i.e., applicable to a widearray of applications. To name a few:
120 -100
100
Conclusion Conclusion Having the capability to remove makeup, unlike any previous technologies, was discovered by a novel machine vision framework by the researchers of SMILE lab. This technology, SMILER, serves as the basis for our technical startup business plans.
Reference Wang, Shenlong, et al. "Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis." CVPR, 2012.
91
• • • •
Facial skin Mouth Eyebrow Eye
186
: : : :
foundation, highlight lipstick pencil shadow, liner, mascara
Before
After
389 300
Thin Plate Spline
Patches 500
700
Makeup decomposition
Makeup detection dictionary learning
Sequential dictionary learning Eyebrow Eyeliner
𝑃
Cosmetics recommendations for consumers. Security applications, which in itself carries many practical uses.
455
Fiducial landmarks location, which were first used to warp images into canonical form and then split the face into different patches.
Aligned face
User-specified makeup tutorial. A much richer qualitative analysis.
Preprocessing Alignment
Dataset collected
Status 𝑖 + 1
𝐴𝑖+1 −
2 𝑃𝐴𝑖 𝐹
Status 𝑖
Learn a sequential of sub-dictionaries of different makeup statuses, in which the coefficients of all identities between two status share a projection matrix P. Through those sub-dictionaries and projection matrix one can decompose the makeup.
Highlight ……
Non makeup
False eyelashes
Lipstick ……
Makeup
SMILER: Faces Behind The Makeup Handong Zhao, Shuyang Wang, Joseph Robinson, Advisor: Professor Yun Fu Technical Details
Abstract Digital technology capable of manipulating facial makeup has grown to be a hot commodity in the beauty industry. Nonetheless, preexisting technologies only process bare faces, i.e., can apply products, but not remove. We have developed the technology to solve this inverse problem utilizing the cuttingedge computer vision and machine learning algorithms. In response to the abundance of applications and novelty of our framework, we have claimed patent rights on our system and published technical papers, respectively. With this, we are developing a startup based on our found technology.
DISTRIBUTION OF MAKEUP STATUS IN OUR DATASET
234 147 163
Background Current digital facial makeup Apps, e.g. L’Oréal’s Makeup Genius, allows customers to “try on” their makeup products digitally. In other words, the app enables them to add various types of makeup (e.g., eye-shadow, lipstick, etc.) using manually visual simulations. However, the technology has a serious limitation: it can only apply make-up to a bare face to accurately predict its resulting appearance.
Business Plan SMILER technologies are highly generalizable, i.e., applicable to a widearray of applications. To name a few:
120 -100
100
Conclusion Conclusion Having the capability to remove makeup, unlike any previous technologies, was discovered by a novel machine vision framework by the researchers of SMILE lab. This technology, SMILER, serves as the basis for our technical startup business plans.
Reference Wang, Shenlong, et al. "Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis." CVPR, 2012.
91
• • • •
Facial skin Mouth Eyebrow Eye
186
: : : :
foundation, highlight lipstick pencil shadow, liner, mascara
Before
After
389 300
Thin Plate Spline
Patches 500
700
Makeup decomposition
Makeup detection dictionary learning
Sequential dictionary learning Eyebrow Eyeliner
𝑃
Cosmetics recommendations for consumers. Security applications, which in itself carries many practical uses.
455
Fiducial landmarks location, which were first used to warp images into canonical form and then split the face into different patches.
Aligned face
User-specified makeup tutorial. A much richer qualitative analysis.
Preprocessing Alignment
Dataset collected
Status 𝑖 + 1
𝐴𝑖+1 −
2 𝑃𝐴𝑖 𝐹
Status 𝑖
Learn a sequential of sub-dictionaries of different makeup statuses, in which the coefficients of all identities between two status share a projection matrix P. Through those sub-dictionaries and projection matrix one can decompose the makeup.
Highlight ……
Non makeup
False eyelashes
Lipstick ……
Makeup
