FAQ
Affdex FAQ Dashboard Components What are the traces on the dashboard showing me? The dashboard shows traces for 7 measures for all respondents with usable data for that viewing of the ad (1st or 2nd). Dashboard metrics are aggregates of all individual sessions. The Affdex traces include two categories of emotion metrics: dimensions of emotion and discrete emotions: Dimensions of emotion are used to characterize the participants emotional responses while discrete emotions are used to describer the specific emotional state. The dimensions of emotion that Affdex measures includes: • Valence – A measure of the positive (or negative) nature of the participant’s experience with the content. If participants tend to be displaying more positive emotions Valence increases on the positive side or if more negative emotions are expressed Valence decreases toward the negative side. • Attention – A measure of the participant’s attention to the screen, using the orientation of the face to assess if they are looking directly at the screen or if they are distracted (turning away) while viewing content. • Expressiveness – A measure of how emotionally engaging content is, computed by accumulating the frequency and intensity of the discrete emotions including smile, dislike/disgust, surprise and concentration/frown. Unlike valence, expressiveness is independent of the positive or negative aspect of the facial expressions. The discrete emotion measures includes: • Smile – The degree to which the participant is displaying a smile. The smile classifier looks at the full face rather than just the mouth/lip area, incorporating other facial cues, like the eyes, to accurately indicate a true smile. Smirks and grimaces are trained not to trigger our smile classifier. • Concentration/Frown – The degree to which the participant is frowning (displaying a brow furrow) in the absence of a Dislike/Disgust classifier response. Concentration/Frown responses during ad test are most often negative in valence. They are usually attributable to mental effort, focus, or confusion. • Surprise – The degree to which the participant is showing a face of surprise, indicated by raised eyebrows. • Dislike/Disgust – The degree to which the participant is showing expressions of dislike or even disgust. These expressions include nose wrinkles and/or grimaces. 1
Confidential, Proprietary to Affectiva
What does the Indicator mean? On May 6, 2013 Affectiva released an upgrade to our dashboard metrics. The alert indicates which dashboards/summary metrics pages are “old” or delivered prior to this upgrade. This is important to note, as with the significant updates to the new metrics the data is not comparable between “old” and “new” dashboards & summary metrics. What is the ‘Bubble’ on the moving trace? The number in the bubble indicates the average moment by moment degree to which respondents are displaying a particular emotion (smiling, surprised etc.), taking into account both frequency (e.g. the number of the sample displaying a smile) and intensity (e.g. the strength of smiles). Therefore, a peak in the line for smile could represent a lot of the sample smiling a small amount, or a small number of the sample smiling broadly. The theoretical range for the discrete emotion measures (smile, surprise, concentration/frown & dislike/disgust) is 0-‐100 but in reality the vast majority of scores typically fall between 0 and 20. For the dimensions of emotion the ranges can vary as follows: • • •
For Attention -‐ The theoretical range for these scores is 0-‐100 but in reality the vast majority of scores typically fall between 70 and 100. For expressiveness-‐ the theoretical range for these scores is 0-‐100 but in reality the vast majority of scores typically fall between 0 and 30. For Valence -‐ The theoretical range for these scores is -‐100 to +100 but in reality the vast majority of scores typically fall between -‐10 to +20.
How can I view different segments of the data? The dashboard data can be segmented by using combinations of the follow controls: • Ad Selector: allows viewing data for the ads tested in the project o Clicking the arrows to the left/right of the ad will allow you to navigate through the ads in the project. o Hovering over the ad selector will display a downward arrow that when selected will display all of the ads in the project for selection. • New Select Exposure: allows viewing of the data of one of more exposures. An exposure is defined as the number of times the participant viewed the media, so in most cases there will be one or two of these. This control will only display on the dashboard if more than one exposure was collected in the project. o To select exposures, click on the control and check the box(s) to choose which exposures to view on the dashboard.
2
Confidential, Proprietary to Affectiva
o
o
o
A minimum of one and up to three exposures can be viewed on the dashboard at one time. You can select to look at each exposure individually or combine exposures for comparison. The first exposure is selected by default, which means that the other controls on the dashboard (participant group, question selector) are referencing exposure one. To view data specific to the second or third exposure select them individually. Each exposure is represented by a different line style on the dashboard: !
Exposure 1 – Solid colored line
!
Exposure 2 – Dotted colored line
! Exposure 3 – Dashed colored line Viewing Segment Breaks and Participant Groups – When viewing segment breaks and/or changing the participant groups in view the exposure selector will persist and show the traces for each exposure selected. o Known Limitation with Bookmarks: When viewing multiple exposures the available bookmarks will only be displayed for the 1st exposure. To see bookmarks on subsequent exposures you will need to view them individually Participant Groups: allows viewing data specific to the following groups: o All Participants -‐ includes data for all survey participants that have usable data. o Most Expressive Participants -‐ includes data for participants that were emotive in any of our captured emotions. o Most Expressive in Selected Emotion -‐ includes data for participants that were emotive in the selected emotion (i.e. selecting this participant group and the “Smile” measure will only show data for participants that smiled during the ad). *Note: this participant group is not applicable when viewing data for Valence, Attention and Expressiveness, thus when these combinations are chosen no data tracks will display and the user will be prompted to change their selections. Survey Questions: allows viewing data for specific data survey questions. The questions displayed are determined prior to beginning a study and generally consists of a standard set (age, gender, brand appeal etc.) and occasionally some client-‐specific custom questions. Emotions Metrics: clicking on any emotion will show data specific to the participant’s reaction in the selected emotion. (See details in “What are the traces on the dashboard showing me “ question above) Settings> Suppress Below – data traces for samples below 10 participants are suppressed from the dashboard display. Without an adequate number of respondents, the ability to break down the data and make meaningful conclusions is compromised. Data that has been suppressed is annotated with (Low Data Sample). o
•
•
•
•
3
Confidential, Proprietary to Affectiva
How do I use the Bookmark Feature? This dashboard feature allows you to save certain views of the Facial Coding data, making it easy to re-‐ visit and navigate between these views when analyzing and presenting your data. The Bookmarks feature is turned on by default, allowing you to view and add bookmarks right away. You may temporarily turn off the Bookmarks feature, but it will reset to On each time you return to the dashboard. • To turn the Bookmark feature on or off: o Go to the ‘settings’ menu by clicking on the gear icon. o Click the ‘bookmarks’ setting to on/off With the Bookmarks feature, you can: • View any bookmark that might have been added by the Affectiva team to highlight possible points of interest in the data lines. o To view these bookmarks, select them from the drop-‐down list or click the pin icon directly • Create new bookmarks: o Select the view you want to bookmark – the relevant ad, exposure, metric, Link question split and time o Right click, choose a bookmark title and description and click save • Delete bookmarks you have created: o Select the relevant bookmark from the drop-‐down list o Click on the pin for that bookmark which shows on the data line o Click ‘delete’ in the box • Hide bookmarks create by others: o Left click the ‘hide bookmark’ button • Unhide bookmarks created by others: o Go to the ‘settings’ menu by clicking on the gear icon o Click the ‘hidden bookmarks’ setting to ‘show’ o Select the bookmark you desire to unhide from the drop-‐down list o Click the “unhide bookmark” link. o Turning the ‘hidden bookmarks’ setting back to “hide” will hide all other hidden bookmarks
Interpreting results How should valence be interpreted? What is the meaning for us? -‐ Valence is a measure of perceived positivity or negativity to the stimulus. It is a derived metric blending elements of Smile and Concentration/Frown, yet is measured separately; i.e., Affdex is trained to detect Valence discretely. Beyond providing an assessment of positive and negative emotional impact, variability in Valence curves indicates content that contains emotional ebbs and flows (usually a desirable aspect of ads). Further, ads that produce a Valence curve that climbs "up and to the right", especially through brand reveal, are indicative of impactful ads. Are higher degrees related to the intensity of the emotion? -‐ The trace is a representation of the degree to which of a particular expression occurring at that moment. It is influenced both by the 4
Confidential, Proprietary to Affectiva
existence of the expression and by intensity. For example a point on the Smile trace of, 50 means either that each respondent reflected at that point is smiling with a degree of exactly 50-‐-‐or, more likely, that some respondents are smiling at a higher degree (some lower), but all averaging at 50.
Summary Metrics How do I use the Summary Metrics page? Delivered with each project’s dashboard is a view of projects summary metrics for quick comparison including how they compare to the Affectiva Ad Testing Norms. These are accessible from the project list page in the portal by clicking the icon. The metrics included in the Summary Metrics tab help you to answer different questions about the facial expressions included in the dashboard, for example using smile: •
Mean is the average of the aggregate line for the metric. •
•
Variability is a measure of the deviation of the aggregated data line for the metric. •
•
Questions you can consider: Does the ad elicit a lot of smiles? Does it bring about more or less smiles than other ads in the region?
Questions you can consider: How much do smiles change during the course of the ad? Higher levels of variability would suggest that viewers are responding to the ad by smiling at different points rather than simply smiling throughout.
Max/Min is the highest point of the aggregate line for the metric •
You can use max and min values to work out the range over which the smile varies
How do my projects summary metrics compare to the Affectiva Ad Testing Norms? Metrics for Mean, Variability, Maximum (Max) & Minimum (Min) are compared against the Affectiva norm data for the region the ad was tested in. You can determine which region your ads are being compared against by looking in the top left corner of the metrics page. The market to region mapping as deployed in our product today is below. This mapping is updated periodically through the calendar year, as we acquire more data. The goal is to get each market to have its own norm for comparison, needing approximately 30 projects to get there. 5
Confidential, Proprietary to Affectiva
View when comparing to the regional norm:
View when comparing to a market norm:
Norms are also calculated separately according to project type: online or offline. In the example below, you can see that the norms for an online project from the Mexico market were calculated based on a comparison to 144 ads from other online projects in that market.
The norms for an offline project from the Mexico market were calculated based on a comparison to 85 ads from other offline projects in that market.
The ad testing norm appears to the right of the ad metric norm in parentheses, with an arrow indicating whether the ad is significantly above or below the norm. As noted above, the norm values displayed on the summary metrics page are calculated separately by online or offline project type. NOTE: the metrics for max, min and % expressive do not display the arrow indicators as are they are not subject to significance testing. Project values are displayed against the ad norms only.
6
Confidential, Proprietary to Affectiva
Current Market to Region mapping: Country Angola Cameroon Ghana Kenya Madagascar Mauritius Mozambique Nigeria Senegal South Africa -‐ LSMA (7-‐10) -‐ F2F South Africa -‐ LSMB (5-‐6) -‐ F2F Tanzania Uganda Zambia Zimbabwe 7
Mapped to Region AMAP_Africa AMAP_Africa AMAP_Africa AMAP_Africa AMAP_Africa AMAP_Africa AMAP_Africa AMAP_Africa AMAP_Africa AMAP_Africa AMAP_Africa AMAP_Africa AMAP_Africa AMAP_Africa AMAP_Africa
Confidential, Proprietary to Affectiva
China -‐ Unspecified Hong Kong China -‐ Beijing China -‐ Secondary Cities Central and West China -‐ Secondary Cities East China -‐ Secondary Cities North China -‐ Secondary Cities South and Guangzhou China -‐ Shanghai China -‐ Tertiary Cities Taiwan Algeria Egypt Iran Kuwait Lebanon Morocco Saudi Arabia Syria Tunisia United Arab Emirates Australia New Zealand Indonesia Japan Korea Malaysia Malaysia -‐ Chinese sample Malaysia -‐ Malay sample Philippines Singapore Thailand Vietnam Vietnam -‐ Hanoi Vietnam -‐ Ho Chi Minh Myanmar India -‐ Unspecified Bangladesh India -‐ East India -‐ North India -‐ Rural 8
AMAP_Eastern-‐Asia AMAP_Eastern-‐Asia AMAP_Eastern-‐Asia AMAP_Eastern-‐Asia AMAP_Eastern-‐Asia AMAP_Eastern-‐Asia AMAP_Eastern-‐Asia AMAP_Eastern-‐Asia AMAP_Eastern-‐Asia AMAP_Eastern-‐Asia AMAP_Middle-‐East AMAP_Middle-‐East AMAP_Middle-‐East AMAP_Middle-‐East AMAP_Middle-‐East AMAP_Middle-‐East AMAP_Middle-‐East AMAP_Middle-‐East AMAP_Middle-‐East AMAP_Middle-‐East AMAP_Oceania AMAP_Oceania AMAP_South-‐Eastern-‐Asia AMAP_South-‐Eastern-‐Asia AMAP_South-‐Eastern-‐Asia AMAP_South-‐Eastern-‐Asia AMAP_South-‐Eastern-‐Asia AMAP_South-‐Eastern-‐Asia AMAP_South-‐Eastern-‐Asia AMAP_South-‐Eastern-‐Asia AMAP_South-‐Eastern-‐Asia AMAP_South-‐Eastern-‐Asia AMAP_South-‐Eastern-‐Asia AMAP_South-‐Eastern-‐Asia AMAP_South-‐Eastern-‐Asia AMAP_Southern-‐Asia AMAP_Southern-‐Asia AMAP_Southern-‐Asia AMAP_Southern-‐Asia AMAP_Southern-‐Asia
Confidential, Proprietary to Affectiva
India -‐ South India -‐ West Nepal Pakistan Sri Lanka Denmark -‐ mixed methodology Germany France Italy Netherlands Israel Poland Turkey United Kingdom Albania Austria Azerbaijan Belgium Belgium -‐ North (Flemish) Belgium -‐ South (French) Bosnia And Herzegovina Bulgaria Croatia Cyprus Czech Republic FYR Macedonia Finland -‐ mixed methodology Greece Hungary Iceland Ireland Kazakhstan Latvia Lithuania Luxembourg Malta Norway -‐ mixed methodology Portugal Romania Russia 9
AMAP_Southern-‐Asia AMAP_Southern-‐Asia AMAP_Southern-‐Asia AMAP_Southern-‐Asia AMAP_Southern-‐Asia Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe Europe
Confidential, Proprietary to Affectiva
Serbia Slovakia Slovenia Spain Sweden -‐ mixed methodology Switzerland Ukraine Brazil -‐ Unspecified Mexico Argentina Bolivia Brazil (CAWI Only) Brazil -‐ North East Brazil -‐ Rio de Janeiro Brazil -‐ Sao Paulo Brazil -‐ South Chile Colombia Costa Rica Dominican Republic Ecuador El Salvador Guatemala Honduras Jamaica Nicaragua Panama Paraguay Peru Puerto Rico Trinidad Uruguay Venezuela USA -‐ English Canada -‐ English Speaking Canada -‐ French Speaking USA -‐ Acculturated Hispanics USA -‐ African American USA -‐ Bicultural Hispanics USA -‐ Spanish 10
Europe Europe Europe Europe Europe Europe Europe LatAm LatAm LatAm LatAm LatAm LatAm LatAm LatAm LatAm LatAm LatAm LatAm LatAm LatAm LatAm LatAm LatAm LatAm LatAm LatAm LatAm LatAm LatAm LatAm LatAm LatAm NA NA NA NA NA NA NA
Confidential, Proprietary to Affectiva
