How to conduct an Experimental Design?

Digital Innovation — Virtual Try-On

PART ONE: Introduction: What is Virtual Try-On?

A virtual try-on is where a customer can “try on” a product through mobile or other devices equipped with a camera. Future customers can see themselves in a beloved product on their smartphone screen thanks to the underlying AR technology. The virtual try-on concept has been on the market for some time now. We owe its recent boom to the filters we know from social media (yes, those funny add-ons to our pictures on Snapchat, Instagram, or Messenger), which have helped develop better facial recognition, machine learning, and 3D modeling on images and video. The filters were a fantastic way to test new palettes of eye shadows, glasses, earrings, and other small accessories.

Types of Virtual Try-Ons?

1. Snapchat Virtual Try-Ons: You may have thought Snapchat was on its way out when Instagram introduced Stories and WhatsApp added disappearing photos. But Snapchat is killing it in the AR realm, which may keep them alive. Farfetch and other brands, such as Prada, have partnered with Snapchat to introduce virtual try-on.
2. YouTube Try-Ons: YouTube does AR try-on mainly through paid ads and influencer content. Google has partnered with several beauty tech companies, such as Perfect Corp., to make this possible. e.l.f Cosmetics has seen great success with its AR campaigns on YouTube. The results: increased traffic, a reduced bounce rate, and a 63 percent lower cost per unique user than previous campaigns. There’s a button that says “Try It On” beneath e.l.f. Cosmetics’ in-stream ads.
3. Brand Web/Mobile Application: If you have high-end, expensive products, it makes sense to offer a virtual try-on since customers don’t make such purchase decisions lightly, given the hefty price tag. So, anything you can do to help them along their journey will be beneficial. Luxury watch company Baume & Mercier partnered with Haptic Media to create an unbelievable virtual try-on experience on their website. Even Dior gives you a web application to virtually try out the products.
4. Meta Platform: Facebook (or Meta has partnered with L’Oreal’s AR and AI company ModiFace to introduce virtual try-on for makeup companies on Instagram Shopping.

Pros of Virtual Try-Ons:

Virtual try-on helps consumers in the decision-making process. They get to experiment with products at their own pace until they find the right choice for them. For instance, they might play around with accessories in assorted colors to see what works with their outfits. Consumers get to try before buying, making them less likely to return items. They are more likely to pick the right product in the first instance. Also, trying something on is a fun experience that consumers want to share. Picture somebody using Snapchat to try on your products and then send their snazzy snap to their friends. That is excellent marketing for your brand. Another significant benefit of virtual try-on is that it makes cross-selling easier. For example, a user might mix and match different makeup products using your virtual try-on tool. In addition, fashion brands are deciding to add augmented reality to their businesses. It helps them offer better customer service, observe customers’ preferences, customize products, and build interest in the product.

Cons of Virtual Try-Ons:

So far, augmented reality works well with small fashion accessories, shoes, and makeup. Larger pieces still must wait to improve 3D modeling and adjustment techniques so that the customer can see himself in well-fitted jackets and trousers that will not belong. We already see intense development works on AR in intelligent mirrors and other devices.

How has the Pandemic accelerated this trend?

Virtual try-on has offered a sweet sense of relief throughout the Pandemic to fashion and beauty brands unable to demo their products offline. From seeing how a new pair of sunglasses might look on your face to testing new lipstick shades out in anticipation of a mask-free future, these AR- and AI-enabled experiences have brought a small part of the traditional retail experiences to lockdown. Even retailers like Wayfair and IKEA rolled out 3D and augmented reality furniture visualization tools so shoppers could try them before they buy.

Brands Innovating in this space?

Michael Kors: virtual try-on of sunglasses on Facebook

World-famous fashion brand, Michael Kors, used Facebook’s AR ads in the news feed. They were the first to test out this new functionality. Facebook users could click the ads’ “tap to try it on” button to superimpose a pair of Michael Kors sunglasses on their faces. They had a straightforward way to purchase the sunglasses directly in the Facebook app.

L’Oreal: virtual makeup and Hair color try-on

L’Oréal Group has teamed up with Facebook to integrate its ModiFace augmented-reality technology and enable AR-powered virtual try-on to Instagram shopping. Spark AR, Facebook’s owned augmented reality platform, and ModiFace is working together to bring the quality and accuracy of the AR makeup experience to Instagram Shops. For salons, L’Oréal is launching an AR-connected hair color system that uses Virtual Try-on to project shades and an algorithm that offers hair color in more than 1,500 custom options.

Diamond Hedge: trying jewelry with a mobile app

The global diamond seller launched a virtual try-on of their gemstones directly on their webpage. By taking or uploading a picture of a hand, their customers can see themselves in rings decorated with different shapes, cuts, and colors of diamonds.

Gucci: Virtual try-on of the Cruise collection via mobile app

Gucci demonstrated a new AR function in its application. The new feature allows buyers to try the Ace sneakers line. Gucci’s main goal was to present the latest footwear models to customers as soon as possible. The application tracks the movements of the feet to view virtual footwear in real-time and at different angles.

PART TWO: Experimental Design

2.1 Assumptions:

Research Question and corresponding Hypothesis:

Before the experiment, we will collect and analyze the transaction data and conversion rate on the L’Oréal group e-commerce website. In this experiment, we will leverage the L’Oréal group e-commerce function ‘Virtual Makeup Try-On’; Augmented Reality technology is used for this function. L’Oréal Virtual Makeup Try-On lets potential consumers wear their products, such as lipsticks and hair colors, and virtually see the effect on the users’ faces and hair.

How it works — L’Oréal Virtual Makeup Try-On:

1. Select one of the products you would like to try on

2. Click Live Try On and see your latest look live or upload a selfie

3. Flick through the shades to find the product that best suits you

Methodology — A/B testing

In general, A/B testing is a research method you can use to understand what people do when given options. This method generates quantitative, behavioral data; use it when you want to understand what users will do. If you’re going to know why people do it, you should leverage other methods.

In practice, A/B testing is a form of experimentation where two or more variants of an experience are shown to users at random. Statistical analysis determines which version performs better for a given conversion goal.

A/B/n testing means you want to test three or more variants at once. “A” represents your control experience, “B” represents the hypothesis you want to try, and “n” represents additional hypotheses you would like to test.

Our experiment will be designed to address the following research questions:

1. Will the Augmented Reality virtual try-on add on the homepage (treatment) perform better than without virtual try-on (control); Does the virtual try-on drive more sales?
2. The target groups that upload their picture or try live videos, will they purchase the most by using the virtual try-on?

Our hypothesis corresponds to the questions:

1. Virtual try-on will perform better than without virtual try-on function; virtual try-on will boost more sales compared with no virtual try-on.
2. Own live video group will purchase the most among the two target groups when the users try the virtual try-on.

2.2 How to Select Experiment Targets and Do Randomization

To find out the suitable attributes, a summary on the features of the experiment products is provided:

L’Oréal itself does not provide any direct online purchase function. Once customers click the purchase button, they will be redirected to the online pharmacy store, which enlarges the pool for selection. For illustration purposes, we will use CVS, one of the pharmacy websites L’Oréal directed to, to explain our selection attributes.

L’Oréal Hair dye products

1. To corporate with CVS and run tests on new customers to L’Oréal’s.
2. To obtain a customer list from CVS and identify experimental targets who satisfy the below attributes, or ask CVS to provide the customer’s list who meet the below conditions together with their email address or phone number; The experimental targets should fulfill the below attributes:

• New customers: never bought L’Oréal hair dye products in the past two years.
• Frequent user of hair dye products: bought hair dye products more than 4 times in the past two years.
• Age range: 20–33; customers fall into this age range are assumed to be more fashion-oriented and would like to try different colors
• Not stick to only one brand or color: bought more than 2 brands of hair dye products and colors in the past two years, so the result would not be affected by customer’s loyalty to a specific brand and color.
• Price range USD11- USD13: the products they bought are in a similar price range to the experimental product, so the result would not be affected by the targets’ income and consumption level.

L’Oréal Lipsticks

To corporate with CVS and run tests on new and existing customers who are likely to be lipsticks users:

1. To obtain a list from CVS on customers and identify experimental targets from the list; or ask CVS to provide the customer’s list who meet the below attributes together with their email address or phone number;
2. The targets should fulfill the below attributes:

• Female: compared to male, female is much more likely to be lipstick user and try-on feature user.
• Age: 20 -35; a female who falls into this age range usually prefers purchasing lipsticks in a different color.
• Cosmetic product users: Bought cosmetic products in the past 1 year. Customers who do make-up are more likely to be lipstick users.
• If the user has brought L’Oréal lipstick avoid targeting them — as they may not buy due to repetitive purchases.

The above selections would be suitable for both Stage 1 and Stage 2 experiments.

2.3 First Stage Null Hypothesis:

Currently, L’Oreal’s Main Page shows some seasonal products; however, the Try-On Feature is hidden inside each one of the product categories. For instance, for Hair Products, one would have to click on Hair Colour on the Main Menu on the Homepage as shown by step 1 in the illustration below, then click on Hair Colour Try-On, as shown by step 2. L’Oreal can’t unleash its full potential by hiding this feature in each category.

Through this A/B Testing, we would like to understand whether the Augmented Reality virtual try-on add on the homepage (treatment) performs better than without virtual try-on (control); does the virtual try-on drive more sales? And result in a higher conversion rate.

“Conversion” is a generic term used to describe what happens when a user completes your desired action, like filling out a form, enrolling in a new feature, completing an onboarding flow, signing up for a new product, etc.

Adding a variant to the Try-On Main page would help validate if diverging from a very straightforward UI where a transparent Try-On Feature is displayed (treatment B) makes any difference from a classic hair product ad (variant n). This A/Bn Testing would help us determine if directly advertising The Try-On feature on the homepage leads to more online sales or virtual traffic. This leads us to the second part of our first stage. Once the users click on the Try-On feature, they can try their choices and order them online.

2.4 Second Stage Null Hypothesis:

Specify the problem to solve — 2 ways of try-on by uploading pictures and real-time try-on; the targets with a lower cart abandonment tendency with try-on functions will prefer which types of virtual try-on?

Second Stage Null Hypothesis — We believe that among the two ways of try on by uploading pictures and real-time requests for targets to turn on the camera, utilizing 2D/3D point detection to track the body (hairs/lips) and post-rendering can gain the most conversion rate.

Hair Products –Second Stage Experiment

Targets & Sample size — Subjects who have checked out hair products with the virtual try-on functions. We assume 50 subjects in the second stage experiment.

Variations: 100 subjects will be divided into two groups, with 50 subjects in Group A and 50 subjects in Group B. Group A and B will try on the same hair product — Préférence Vivids (Colorista) Permanent Gel Hair Dye, Rose Gold 9–214. Subjects in Group A will have the live try-on, whereas subjects in Group B will upload their photos to have the online try-on. Participants in Group A and Group B will decide if they will check out.

Lipsticks Products –Second Stage Experiment

Targets & Sample size: Subjects who have checked out lipstick products with the virtual try-on functions. We assume 100 subjects in the second stage experiment.

Variations: 100 subjects will be divided into two groups, with 50 subjects in Group A and 50 subjects in Group B. Group A and B will try on the same hair product — Rouge Signature Matte Liquid Colour Ink Lipstick 115 I Am Worth It. Subjects in Group A will have the live try-on, whereas subjects in Group B will upload their photos to have the online try-on. Participants in Group A and Group B will decide if they will check out.

Limitations, Considerations, and Even Better

• Since this experiment only tests one hair product, we rightfully think there are some considerations to consider seeing the limitations of the Try-On feature
• Under complex situations such as different light intensities and reflections, one product may not have a robust visualization. Hence, the experiment results will be more biased towards one or more products.
• For cosmetics and lipsticks, several pitfalls could happen once the product has been bought and used, such as Allergies and medical conditions that could not be found without personal trials.

Pragmatic approaches for marketing & recommendations

We believe that the Try-on Feature can be enhanced with the aid of ad-ones that will make the whole virtual experience more enticing. The links of these ad-ons can be displayed on the main page while the virtual try-on occurs. The examples we produced are:

- Book a virtual consultation: Upon using try-on, you could book a virtual consultation with a second party. In this case, a stylist licensed by L’Oreal would be able to provide an external opinion for those users who would like to get a suggestion.

- Upon selecting the hair product, one could find the address of the closest L’Oreal-licensed stylist on the map.

- Lastly, Pop-up products are based on the skin tones of the customers; this can be done in two ways:

• Real-time: Products can pop out on the left side of the Try-on view. This would constantly be moving and will be personalized to match your skin tone.
• Link-based: Just as shown in the picture below, once the user’s skin tone has been determined from the Try-On experience, the user will be able to access an array of products that could be customized to match their skin tone.

Team Credit: Annie Lu, Tomo Mikami, Alci Gonzalez, Ann Yang