These are the best places to start to learn about text analysis in Displayr.

• Webinar: The complete “how-to” guide for analyzing market research text data
• Ebook: Using Machine Learning to Automate Text Coding
• Fully-integrated Text Analysis with Displayr
• How to Set up Your Text Analysis in Displayr
• Technical Documentation: Text Analysis
• At Last, Machine Learning Can Accurately Categorize Text Data
• Automatic Language Translation of Text Variables
• How to Analyze Free-Form Text Data from Surveys

General Categorization (Coding) Resources

• Technical Documentation: Coding
• Efficient Coding of Spontaneous Awareness Data in Q and Displayr

Automatic Categorization

• Automatic List Categorization of Text Data with Displayr
• How to: Automatic Coding of Unstructured Text Data
• Automatically Classify New Text Data Using an Existing Categorization
• Automatically Extract Entities and Sentiment from Text
• Automatic Categorization of Unstructured Text Data

Manual & Semi-automatic Categorization

• Technical Documentation: Manual Categorization - Multiple Overlapping Categories
• Choosing the Right Type of Manual Categorization for Text Analysis

Sentiment analysis

• Technical Documentation: Sentiment Analysis
• Twitter Sentiment Analysis Example
• How to Calculate Sentiment Scores for Open-Ended Responses in Displayr

Word Clouds

• Technical Documentation: Creating and Combining Phrases in Word Clouds
• Using Text Analytics to Tidy a Word Cloud
• The Best Tool for Creating a Word Cloud

Other Uses of Text Data

• Principal Component Analysis of Text Data
• Using Text Data for Driver Analysis
• Text Analysis: Predicting Engagement from Tweets
• R Code & Text Analysis: Hooking up Your Term Document Matrix to Custom R Code

Manipulation of Text Variables

• How to Split Text Strings in Excel
• How to Convert Text Dates in Excel
• How to Split Text Strings in Displayr
• How To Convert Text Dates To Numeric

Introduction to Variables and Variable Sets:

What is a Variable

What is a Variable Set

Variable and Variable Set Structure

How To Create and Update Variables:

10 Ways to Create New Variables in Displayr

How to Set Value Attributes for a Binary-Multi and Binary-Grid

Merging and Splitting Questions

How to Recode into Existing or New Variables

Date/Time Variable Settings

How to Band Numeric Variables in Displayr

How to Calculate an Average Value from Categorical Data

How to Create a Top 2 Box in Displayr

Creating and Working with JavaScript Variables

Creating R Variables from Multiple Input Variables Using Code

Easy Functions for Automating Filters and Rebasing

Interactive Tutorials:

Understanding Variable Sets in Displayr

How to Compute Net Promoter Score (NPS)

Five Ways to Create a Filter in Displayr

Exporting a Document as a File

How to Automatically Export Multiple Reports with Different Filters

PowerPoint

Exporting to PowerPoint 1: 10 advantages of creating PowerPoint reports in Displayr

Exporting to PowerPoint 2: How to Create PowerPoint Reports in Displayr

Exporting to PowerPoint 3: How to make changes to existing PowerPoint reports

How to retrospectively automate an existing PowerPoint report using Displayr

See PowerPoint linking in action (Video)

PowerPoint Export

How to export editable charts to PowerPoint

Export charts using your own PowerPoint Chart templates via Displayr cloud drive

Updating Existing PowerPoint documents

Automate PowerPoint reporting

Aligning Text and Images in PowerPoint Exports

How to Automatically Update Your Reports

Setting up “branded” Power-Point-style templates in Displayr

Create and Update PowerPoint Reports using R

Excel

Excel Export

Exporting LDA Functions from Displayr into Excel

What is Data Filtering?

What is Rebasing?

5 Things to Consider Before Building Your Dashboard

Variables and Questions

5 Ways to Create a Filter in Displayr

How to Filter Data in Displayr

How to Use Basic R Code for Creating Filters

Easy Functions for Automating Filters and Rebasing

Filters in Displayr

Managing Filters and Weights

10 Ways to Create New Variables in Displayr

Using Support Vector Machines in Displayr

Control boxes

Adding a Combo Box to a Displayr Dashboard

How to Connect Filters to a Combo Box (Control)

Combo Boxes (Controls) With Dynamic Lists in Displayr

How to Use the Same Control on Multiple Pages

How to Dynamically Change a Question Based on a Control Box

Using Controls to Determine when to Show Outputs

How to Switch Logos and Images Based on User Selections

How to Customize the Sample Size Description Widget

Optimizing your Conjoint Analysis Simulator in Displayr

R Visualizations, Tables and Outputs

How to Filter Rows and Columns in Visualizations and Tables without Code

How to Remove a Row or Column using R in Displayr

Filtering a Subset of Tables and Visualizations on a Page in Displayr

Creating tables with multiple variables (filters and multiway tables)

Pages and Documents

Using Displayr to Filter Data, Analyses, and Whole Reports

How to Filter a Dashboard Based on User Logins

Allowing Users to Filter Pages in Dashboards

How to Automatically Export Multiple Reports with Different Filters

Troubleshooting Guide, Videos, Tutorials and Dashboards

Troubleshooting Guide and FAQ on Filtering

Using R in Displayr Video Series

Tutorial: Filtering Data

Dashboard Design: 8 Types of Online Dashboards

Dashboard Example Gallery

Introduction

What is Max Diff? Have you ever needed to know what your customers prefer? MaxDiff (otherwise known as Best-Worst Scaling) quite simply involves respondents indicating the ‘Best’ and the ‘Worst’ options out of a given set so that we can gain an understanding of consumers’ preference choices. This can be from how they feel about specific brands or products and to know which features make a difference and are of value to the consumer.

A Beginners Guide to MaxDiff MaxDiff is a research technique for measuring relative preferences.

How MaxDiff Analysis Works This post explains the basic mechanics of how preferences can be measured using the data collected in a MaxDiff experiment.

DIY MaxDiff

Make MaxDiff a piece of cake Quickly go from experimental design to analysis, to interactive report in one tool. Displayr automates all the painful manual steps and makes it easy to deliver best-in-class results, even for the novice.

The 4 easy steps that’ll make any good researcher proficient at MaxDiff This webinar is for market researchers and consumer insights people who analyze data (from novice to expert).

11 Tips for your own MaxDiff Analysis  This post shares 11 tips to help researchers be self-sufficient in MaxDiff analysis.

DIY MaxDiff eBook This ebook will take you from generating experimental designs to conducting advanced Hierarchical Bayes analysis.

Interactive Tutorials

Creating an Experimentation Design for MaxDiff This interactive tutorial demonstrates how to create a MaxDiff Experimental Design

Analyzing MaxDiff Data This interactive tutorial demonstrates the range of purpose-built tools for analyzing the data from MaxDiff experiments available in Displayr.

Experimental Design

How to Create a MaxDiff Experimental Design in Displayr This post describes how you can create and check the design yourself.

Advanced MaxDiff Experimental Designs A MaxDiff experimental design creates multiple lists of alternatives to be shown to respondents in multiple questions.

Pairwise Balanced MaxDiff Designs This post gives some background on how MaxDiff designs are created, including a new method for making multiple version designs that are pairwise balanced.

How To Check Experimental Design This post explains the basic process followed when doing a rough-and-ready check of the experimental design.

Data File Formats

Custom Data Files - Survey Gizmo MaxDiff This QScript makes it possible to analyze Alchemer (formerly Survey Gizmo) MaxDiff data in Displayr.

MaxDiff Data File Layouts There is no standard way of laying out the data from MaxDiff experiments. The following descriptions encapsulate some of the common ways.

Statistical Analysis

How to Analyze MaxDiff Data in Displayr This post discusses a number of options that are available in Displayr for analyzing data from MaxDiff experiments.

Setting Up a MaxDiff Experiment as a Ranking There are some more 'exotic' types of analysis of MaxDiff data where it is useful to have the data set up as a Ranking Structure.

Counting Analysis of MaxDiff Data This post explains how to do Counts analysis of MaxDiff data.

Comparing MaxDiff Models and Creating Ensembles in Displayr There are a variety of different models available in Displayr to perform MaxDiff analysis. This post describes how to easily compare the models. It also demonstrates how to create an ensemble that combines the models and potentially improves prediction accuracy.

The Accuracy of Hierarchical Bayes When the Data Contains Segments This post explores the implications of using Hierarchical Bayes with data that contains segments.

Using Hierarchical Bayes for MaxDiff in Displayr This post describes how to run Hierarchical Bayes for MaxDiff in Displayr, and explain the options and outputs available.

Checking Convergence When Using Hierarchical Bayes for MaxDiff This post discusses technical information about how to check for convergence in a Hierarchical Bayes MaxDiff model.

Comparing Tricked Logit and Rank-Ordered Logit with Ties for MaxDiff This post compares two ways in which MaxDiff data is treated in analyses such as latent class analysis and Hierarchical Bayes.

Using Cross-Validation to Measure MaxDiff Performance This post compares various approaches to analyzing MaxDiff data using a method known as cross-validation.

Comparing MaxDiff Results from Different Packages This post lists the main reasons why you may get different results with different packages.

MaxDiff Mixture Models This post discussed the main mixture models used to analyze the MaxDiff experiments.

Anchored MaxDiff  Anchored MaxDiff experiments supplement standard MaxDiff questions with additional questions designed to work out the absolute importance of the attributes.

Case Studies

MaxDiff Analysis in Displayr, a Case Study This case study illustrates an advanced analysis of experimental data in Displayr.

Case Study: MaxDiff - Presidential Traits This case study formed the 4th and final part of the webinar DIY Market Research Dashboards - Building 4 in 40 minutes (webinar).

Commander-in-Chief MaxDiff An alternative to PowerPoint, story-style dashboard showing an analysis of what Americans desire in their Commander-in-Chief.

Learn more

If you can't find something you can always ask the technical support team, who love to help. Just email support@displayr.com. Happy learning!

Webinar: DIY Market Mapping Using Correspondence Analysis

Ebook: DIY Correspondence Analysis

How Correspondence Analysis Works (A Simple Explanation)

Understanding the Math of Correspondence Analysis

How to Interpret Correspondence Analysis Plots

Correspondence Analysis Versus Multiple Correspondence Analysis

Principal Component Analysis

Principal Component Analysis (Wiki example)

How to Do Principal Components Analysis in Displayr

The Basic Mechanics of Principal Components Analysis

Principal Component Analysis of Text Data

Varimax Rotation

Component Score Coefficient Matrix

Kaiser Rule

Determining the Number of Components in Principal Components Analysis

Validating Principal Components Analysis

Common Misinterpretations of Principal Components Analysis

Text Analysis - Advanced - Principal Components Analysis (Text)

Saved Principal Components Analysis Variables

Multidimensional Scaling and t-SNE

What is Multidimensional Scaling (MDS)?

t-SNE

How t-SNE Works

Goodness of Fit in MDS and t-SNE wit Shepard Diagrams

These are the best places to start to learn about weighting in Displayr.

• For a general overview of weighting tools in Displayr, start here: Technical Documentation: Weights in Displayr
• Our weighting ebook goes into more detail both on the theory and process of weighting in general and how to build weights in Displayr: How to Weight Survey Data
• This post shows how to merge multiple weights, such as wave or geography/market based weights: Creating and Working with JavaScript Variables

These resources address specific aspects of working with weights

• Technical Documentation: Managing Filters and Weights
• Technical Documentation: Weight Calibration
• The Correct Treatment of Sampling Weights in Statistical Tests

From here, the resources below dig into more specific aspects and uses of weights.

Effective Sample Size

• What is Effective Sample Size?
• Technical Documentation: Design Effects and Effective Sample Size

Unique Uses for Weight Variables

• How to Create a Bump Chart (Ranking Plot) from a Table Using Displayr
• Technical Documentation: Writing Functions for the flip Project R Package

Shapley Regression and Johnson's Relative Weights

Johnson's Relative Weights isn't about weighting survey data, but the technique will come up in results when looking for information about weighting on our blog or in our technical documentation. The collected resources on this topic are below.

• You Can Now Run Shapley Regression in Displayr
• The Difference Between Shapley Regression and Relative Weights
• Comparing Partial Least Squares to Johnson’s Relative Weights
• When to Use Relative Weights Over Shapley

How to Make a Column Chart in Displayr

What is a Column Chart?

Customizing Colors Within a Series on a Bar, Column, or Pyramid Visualization

How To Add a Line to a Column Chart

Using Sparklines to Show Trends in Bar and Column Charts

Technical Documentation: Small Multiples Column Chart

Histogram

How to Make a Histogram in Displayr

Technical Documentation: Histogram

Bar Charts

How to Create a Bar Chart in Displayr

Customizing Colors Within a Series on a Bar, Column, or Pyramid Visualization

7 Alternatives to Word Clouds for Visualizing Long Lists of Data

Why Pie Charts Are Better Than Bar Charts

Where Pictographs Beat Bar Charts: Proportional Data

Where Pictographs Beat Bar Charts: Count Data

5 Ways to Visualize Relative Importance Scores from Key Driver Analysis

Using Sparklines to Show Trends in Bar and Column Charts

Technical Documentation: Bar Chart

Technical Documentation: Pictograph Bar Chart

Technical Documentation: Small Multiples Bar Chart

Pyramid Charts

How to Create a Pyramid Chart in Displayr

Customizing Colors Within a Series on a Bar, Column, or Pyramid Visualization

Technical Documentation: Pyramid Chart

Line Charts

How to Create a Line Chart in Displayr

Using Sparklines to Show Trends in Bar and Column Charts

How To Add a Line to a Column Chart

Technical Documentation: Small Multiples Line Chart

Time Series Graph

How to Create a Time Series Graph in Displayr

2 Ways to Smooth Time Series in Displayr

Smoothing Time Series Data

Technical Documentation: Time Series Graph

Area Chart

How to Make an Area Chart in Displayr

Technical Documentation: Small Multiples Area Chart

Pie and Donut Charts

How to Create a Pie Chart in Displayr

What's Better Than Two Pie Charts?

7 Alternatives to Word Clouds for Visualizing Long Lists of Data

5 Ways to Visualize Relative Importance Scores from Key Driver Analysis

A Pie Chart for Pi Day: The Data Scientist Pie Eating Challenge

Why Pie Charts Are Better Than Bar Charts

Technical Documentation: Pie Chart

Technical Documentation: Donut Chart

Technical Documentation: Number in Donut

Radar Charts

How to Make a Radar Chart in Displayr

Technical Documentation: Radar Chart

Technical Documentation: Small Multiples Radar Chart

Scatterplots and Bubble Charts

What is a Scatter Plot?

What is a Labeled Scatter Plot?

What is a Bubble Chart?

Using Scatterplots to Chart Trends in Displayr

Adding Logos to Scatter Plots in Displayr

Using Bubble Charts to Show Significant Relationships and Residuals in Correspondence Analysis

5 Ways to Visualize Relative Importance Scores from Key Driver Analysis

Labeled Scatter Plots and Bubble Charts in R

Technical Documentation: Scatterplot

Technical Documentation: Small Multiples Scatterplot

Density, Bean, and Violin Plots

How to Make a Density Plot in Displayr

How to Create a Violin plot in Displayr

Using Heatmap Coloring on a Density Plot Using R to Visualize Distributions

Technical Documentation: Density Plot

Technical Documentation: Bean Plot

Technical Documentation: Violin Plot

Geographic Maps (Cartograms, Choropleths)

How to Make a Geographic Map in Displayr

How to Set the Initial Zoom and Position of Geographic Maps

How to Plot Data on an Interactive Globe in Displayr

Building an Interactive Globe Visualization in R

7 Alternatives to Word Clouds for Visualizing Long Lists of Data

How to Build a Geographic Dashboard with Real-Time Data

Technical Documentation: Geographic Map

Technical Documentation: Small Multiples Geographic Map

Heatmaps

How to Create a Heatmap in Displayr

Too Hot to Handle? The Problem with Heatmaps

Making Your Data Hot: Heatmaps for the Display of Large Tables

Heatmap Shading on Tables and Charts

Using Heatmap Coloring on a Density Plot Using R to Visualize Distributions

Making Your Data Hot: Heatmaps for the Display of Large Tables

2 Rules for Coloring Heatmaps so That Nobody Gets Burnt

Technical Documentation: Heatmap

Box Plots

How to Create a Box Plot in Displayr

Technical Documentation: Box Plot

Venn Diagram

How to Make a Venn Diagram in Displayr

Technical Documentation: Venn Diagram

Sankey Diagram

How to Create a Sankey Diagram From a Table in Displayr

How to Create Sankey Diagrams From Tables (Data Frames) Using R

Creating Custom Sankey Diagrams Using R

Visualizing Response Patterns and Survey Flow With Sankey Diagrams

Using Colors Effectively in Sankey Diagrams

Decision Tree Visualizations using Sankey Diagrams or Charts

Technical Documentation: Sankey Diagram

Streamgraph

How to Create a Streamgraph in Displayr

Technical Documentation: Streamgraph

Bump Chart, or Ranking Plot

How to Create a Bump Chart (Ranking Plot) from a Table Using Displayr

Ranking Plots: Illustrating Data with Different Magnitudes

Word Clouds

How to Show Sentiment in Word Clouds using R

How to Show Sentiment in Word Clouds using Displayr

How to Show Sentiment in Word Clouds

7 Alternatives to Word Clouds for Visualizing Long Lists of Data

Using Text Analytics to Tidy a Word Cloud

The Best Tool for Creating a Word Cloud

Technical Documentation: Word Cloud

Pictographs

How to Create a Single Icon Pictograph in Displayr

How to Create a Repeated Icon Pictograph in Displayr

How to Create a Pictograph Bar Chart in Displayr

Where Pictographs Beat Bar Charts: Proportional Data

Where Pictographs Beat Bar Charts: Count Data

How to easily add custom icons in Displayr

Palm Tree Chart

How to Create a Palm Tree Chart in Displayr

Using Palm Trees to Visualize Performance Across Multiple Dimensions (Egypt's Scary Palm Tree)

Technical Documentation: Palm Trees

Sparklines

Using Sparklines to Show Trends in Bar and Column Charts

Moonplot

Moonplots: A Better Visualization for Brand Maps

Single Number Visualizations

How to Create a Single Icon Pictograph in Displayr

How to Create a Repeated Icon Pictograph in Displayr

Technical Documentation: Number

Tables

Customizing the Look and Feel of Tables in Displayr

How to Customize Tables You Can Format and Align at Will in Displayr

Make Beautiful Tables with the Formattable Package

Conjoint Analysis: The Basics

Main Applications of Conjoint Analysis

Webinar: Introduction to Conjoint

Design

Experimental Design for Conjoint Analysis: Overview and Examples

Writing a Questionnaire for a Conjoint Analysis Study

Sample Size for Conjoint Analysis

Algorithms to Create your Choice Model Experimental Design

The Efficient Algorithm for Choice Model Experimental Designs

The Partial Profiles Algorithm for Experimental Designs

How to Create Alternative-Specific Choice Model Designs in Displayr

How to Set Up a Choice-Based Conjoint Analysis in Qualtrics

How Good is your Choice Model Experimental Design?

How to Check an Experimental Design (MaxDiff, Choice Modeling)

Webinar: How to Create Experimental Designs for Conjoint

Analysis

Formatting Data for Running Conjoint in Displayr

How to do Choice Modeling in Displayr

How to Use Hierarchical Bayes for Choice Modeling in Displayr

How to Use Simulated Data to Check Choice Model Experimental Designs Using Displayr

How to Analyze Dual-Response ‘None of These’ Conjoint Models in Displayr

How to Check a Choice-Based Conjoint Model

Testing Whether an Attribute Should be Numeric or Categorical in Conjoint Analysis

Numeric Attributes in Choice-Based Conjoint Analysis in Displayr

Numeric versus Categorical Price Attributes in Conjoint Analysis

Reordering Attribute Levels in Conjoint Analysis Models in Displayr

Comparing HB Root-likelihood (RLH) Between Displayr and Sawtooth

Checking Convergence When Using Hierarchical Bayes for Conjoint Analysis

Performing Conjoint Analysis Calculations with HB Draws (Iterations)

Comparing Choice Models and Creating Ensembles in Displayr

12 Techniques for Increasing the Accuracy of Forecasts from Conjoint Analysis

Understanding Logit Scaling

Computing Willingness-To-Pay (WTP) in Displayr

Webinar: Statistical Analysis for Conjoint

Visualizations

Data Visualization for Conjoint Analysis

Using Indifference Curves to Understand Tradeoffs in Conjoint Analysis

Using Substitution Maps to Understand Preferences in Conjoint Analysis

Creating Demand Curves Using Conjoint Studies

Webinar: Reporting for Conjoint

Webinar: Discover the Top Six Techniques of Pricing Research

Simulators

Creating Online Conjoint Analysis Choice Simulators Using Displayr

Adjusting Conjoint Analysis Simulators to Better Predict Market Share

Optimizing your Conjoint Analysis Simulator in Displayr

How to Create an Online Choice Simulator by Hand

Using Choice-Based Conjoint in Pricing Research Studies

Using the Value Equivalence Line (VEL) with Conjoint Simulators

Webinar: Reporting for Conjoint

Webinar: Discover the Top Six Techniques of Pricing Research

Case Study: Eggs Choice Simulator

Case Study: Fast Food Simulator

Consider the utilities plot below, which quantifies the appeal of different aspects of home delivery. If you hover over the mouse plot you will see the utilities. For example, you can see that Mexican has a utility of 4.6 and Indian of 0. These values are logit scaled.

Converting logit-scaled values into utilities

When things are on a logit scale, it has a couple of profound implications. The first is that we can compute probabilities of preference from the difference. For example, we can see from the utilities that this person seems to prefer Mexican food to Indian food (i.e., 4.6 > 0). The difference between 4.6 - 0 = 4.6 (we are starting with an easy example!), and this means that given a choice between Indian and Mexican food, we compute there is a 99% chance they would prefer Mexican food. The actual conversion from logit-scaled values to utilities is a simple formula, which is easy to compute in either Excel or R. Note that there is a minus sign prior to the value of 4.6 that we are evaluating.

Comparing Mexican food to Italian, we can see that Italian food is preferred and the difference is 0.5. As the difference is smaller, the probability is closer to 50%. A logit of 0.5 translates to a 62% probability of preferring Italian food to Mexican food.

Summing logit-scaled utilities

Things get even cooler when we add together utilities and then compute differences. Mexican food at $10 has a utility of 4.6 + 3.3 = 7.9, whereas Italian food at $20 has a utility of 5.0 + 1.0 = 6.0. This tells us that people prefer Mexican food if it is $10 cheaper. Further, as the difference is on a logit scale, we can convert the difference 7.9 - 6.0 = 1.9 into a probability of 87%.

Percentages versus probabilities

Now for some of the ugliness. So far I have described the data as being for a single person, and interpreted the logit scales as representing probabilities. In many situations, the underlying data represents multiple people (or whatever else is being studied). For example, in a model of customer churn, we would interpret the logit in terms of the percentage of people rather than the probability of a person. Why is this ugly? There are two special cases:

• In many fields, our data may contain repeated measurements. For example, in a typical choice-based conjoint study we would have multiple measurements for multiple people, and this means that the logit is some kind of blend of differences between people and uncertainty about a person's preference. It is usually hard to know which, so common practice is to use whichever interpretation feels most appropriate.
• More modern models, such as hierarchical Bayes, compute logit-scaled values for each person. This is good in that it means that we can interpret the scaling as representing probabilities about individual people. But, a practical problem is that the underlying mathematics means we cannot interpret averages of coefficients as being on a logit scale, and instead need to perform the relevant calculations for each person, and compute the averages of these.

The Customer Effort Score is different from other common customer feedback metrics, like Customer Satisfaction and the Net Promoter Score, in that it seeks to measure a specific component of a product or service, rather than a customer's overall sentiment.

However, CES results are often found to be better predictors of customer loyalty and retention than the Customer Satisfaction score and Net Promoter Score. In many cases, the Customer Effort Score is a better metric for a customer's overall sentiment, even though it asks an extremely specific question. This is why so many companies love including the CES in their customer feedback surveys.

Calculating the Customer Effort Score

You can calculate the score by simply taking the mean or median of your responses. The distribution of effort scores can be used to identify clusters among the respondents. If the median score is high but you had a cluster of customers respond with “Very Difficult,” then it is worth finding out why some customers are having such a hard time. They could potentially be experiencing a bug or there has been some form of misunderstanding.

When is the best time to survey

The perfect time to prompt your customers for a Customer Effort Score is directly after they’ve used your product or service. The experience will still be fresh in their minds and they will be able to answer most accurately. It is also useful to prompt a Customer Effort Score after they seek customer support. That way you can measure how effective the support was. Collect Customer Effort Scores often and consistently to track how the scores change over time.

Benefits and Shortcomings

How to Improve your Customer Effort Score

Once you’ve started tracking your score, it’s time to start looking for ways to improve it. Here are some things to consider:

• Focus on negative feedback: Customers that rated their experience “very difficult” are most at risk of churn. Their problems may have a very simple solution that could quickly change their response to “very easy.”
• Ask open-ended questions in your surveys: Start asking “why” questions and find out the specific parts of the user journey that they are finding difficult.
• Make improving Customer Effort Score a key priority: By placing the score at the center of your customer-relationship strategy, you have a reliable metric to track progress.

Find out why customer feedback is so important.

Customer feedback can come in many different forms and from many different sources. It can be extremely well-structured, like a multiple-choice survey. Or it can be messy and difficult to interpret, like feedback from a website comment section.

Here are the most common ways you can collect customer feedback:

• Customer feedback survey: a structured questionnaire designed to collect feedback from customers
• Social media: comments, ratings, suggestions, and general sentiments left on social media sites
• Website prompts: a website pop-up that prompts users to quickly rate a feature, product, or experience
• In-product prompt: users can be called upon to quickly rate their experience while they are using a product.
• Third-party reviews and ratings: you can also find feedback on user-generated review sites and rating aggregators.

Open-ended and Closed-ended feedback

You can divide customer feedback broadly, into two categories: open-ended and closed-ended responses.

An open-ended response is one where the customer is able to respond in their own words. Users are free to describe specific issues they are having and suggest ways to improve a product. Closed-ended responses have a set list of options for users to choose from, like multiple-choice questions and ratings on a numeric scale. Although this form of feedback is significantly less detailed than open-ended answers, the results are a lot easier to interpret and analyze.

Common customer feedback metrics

Customer feedback metrics are a great way for you to gauge how satisfied, interested, and loyal your customers are. The common metrics – Customer Satisfaction, Net Promoter Score, and Customer Effort Score – are particularly useful when you can compare them over a period of time.

Customer Satisfaction

Customer Satisfaction is the most generic form of customer feedback metrics. You can gauge customer satisfaction by asking a simple question like, “On a scale of 0-10, how would you rate this product/company/brand?”

The most common way to measure customer satisfaction is with a Top 2 Box score. This is calculated by taking the share of the two top options (for example, 9-10 on a 0-10 scale) as a percentage of all responses. The average, median, or mode score are also adequate ways of measuring satisfaction.

Tutorial: Measure Customer Satisfaction in Displayr

Net Promoter Score (NPS)

The Net Promoter Score measures how likely a respondent is to recommend a product, company, or brand. It is based on responses to a single question: “On a scale of 0-10, how likely are you to recommend this product/company/brand to a colleague or friend?”

Then you can calculate your NPS with the following formula:

NPS = Percentage of Promoters – Percentage of Detractors

Where:

• Detractors: respondents who gave a score between 0-6
• Passives: respondents who gave a score between 7-8
• Promoters: respondents who gave a score between 9-10

The NPS is a measure of customer loyalty and a predictor of revenue growth.

Tutorial: Calculate Net Promoter Score in Displayr

Customer Effort Score (CES)

The Customer Effort Score measures ease of use by simply asking customers to rank their experience on a scale of “Very Difficult” to “Very Easy”. Respondents who found a product difficult to use are vulnerable to churn, so it’s important to identify the pain points in the user journey.

1. They have collected data in a longitudinal study (tracker) – a project in which an analyst collects data over a period of time and analyzes it as intervals.
2. They have collected data in a before-and-after project – where the analyst collects data before an event, and then again after.

Similarly, some analysts collect data for a specific set of variables, but may at a later stage augment it with either data in different variables, or with data that comes from a different source altogether. Thus, there are three situations that may necessitate merging data into an existing file: you can add new cases, new variables (or both new cases and variables), or data based on one or more look-up values.

Merging in New Cases

Merging in new cases, sometimes known as appending data (or in SQL, “unions”) or adding data by rows (i.e. you’re adding new rows of data to each column), assumes that the variables in the two files you’re merging are the same in nature. For instance, var1 in the example below should be numeric in both questions, and not a string (text) variable in one file and numeric in the other. Most software matches up the data based on the variable name, and so the same names should be used across the two files. “var1” in one file should be “var1” in the other.

This also assumes that the IDs for each case are different.  If it should happen that you have a variable in one file that doesn’t have a match in the other, then missing data (blank values) may be inserted for those rows that do not have data.

Merging in New Variables

Contrary to when you merge new cases, merging in new variables requires the IDs for each case in the two files to be the same, but the variable names should be different.  In this scenario, which is sometimes referred to as augmenting your data (or in SQL, “joins”) or merging data by columns (i.e. you’re adding new columns of data to each row), you’re adding in new variables with information for each existing case in your data file. As with merging new cases where not all variables are present, the same thing applies if you merge in new variables where some cases are missing – these should simply be given blank values.

It could also happen that you have a new file with both new cases and new variables.  The approach here will depend on the software you’re using for your merge. If the software cannot handle merging both variables and cases at the same time, then consider first merging in only the new variables for the existing sample (i.e. augment first), and then append the new cases across all variables as a second step to your merge.

Merging in Data using Look-ups

The above is all good and well if you have complete data sets to combine.  You can, however, augment your data with information from other sources. Consider, for instance, a data file where you have collected the zip-codes (or postcodes) of your respondents, and you want to attach some demographic data to your survey data – maybe the average income in each zip-code.

You will have your survey data on the one hand (left in the diagram below), and a list of zip-codes with corresponding income values on the other (right in the diagram).  Here, the zip-code would be referred to as a look-up code and function as the ID value did in our previous examples.

In other words, we use the look-up code as the identifier and add in the income values into a new variable in our data file.  In the diagram, observe how the data is matched up for each case by looking up the zip-code and then augmenting the original data with the income data for each matching zip-code.  For those familiar with Excel, for instance, the formula to perform this type of augmentation is =VLOOKUP().

The look-up code should be unique in the file that contains the additional data (in our example, each zip-code should only appear once, with a single associated income), but the same value can appear multiple times in the file you’re wanting to augment.  Think of it like this:  lots of people can share a zip-code, but there’s only one average income for each of those locations.

Don’t Forget…

As the process of merging files and appending data can be complex, it’s always handy to have software that does most of the hard work for you.  There are, however, three things to remember that will help ensure that it’s a smooth process:

1. If you’re appending data, then the IDs should be unique in both files and the variables should be exactly the same in set-up and structure.
2. When augmenting data, then the variables should be unique in both files, apart from the ID variable which should be exactly the same in both files.
3. If you’re looking up values across files then the look-up values can be non-unique in the target file but should be unique in the source file.

The Customer Satisfaction Score is the most straightforward measure of customer satisfaction. Customer satisfaction is calculated by asking customers a question, along the lines of "How satisfied were you with your experience today?" Responses are collected on a scale, usually 1-3, 1-5 or 1-10. The final CSAT score can be calculated using several different methods. If used correctly, collecting your customer satisfaction can help you identify key areas where your customers are less than satisfied. From here, you can make improvements accordingly. This is key to long-term customer retention and ensuring that your customers are happy with your service.

In order to get the best possible insights from your customer feedback survey, it's important to understand what it does and doesn't measure. Because of the nature of the customer satisfaction question - which asks about immediate satisfaction - responses are likely to indicate sentiment towards the customer's most recent interaction with your brand or service. This is not necessarily a disadvantage, as it means you can gauge sentiment towards individual touch points throughout the customer life cycle. However, it also means that customer satisfaction cannot generally be used to infer broader attitudes toward your brand or the customer's experience.

How should I use Customer Satisfaction?

A key advantage of customer satisfaction is its simplicity. The rating scale can be modified to suit your particular business needs, as can the question. For instance, you can ask how satisfied a customer was with their overall experience, or you can ask them about something more specific. For instance, a hotel might ask their customers how satisfied they were with their stay. However, they could also ask how satisfied they were with their room, the food, the spa, or the service. These more granular questions help to reveal if there are any specific aspects which are less satisfying than the others. For this reason, it's advantageous to ask customers to rate their overall satisfaction, as well as their satisfaction with multiple attributes. Otherwise, you will not know what changes you should make to increase customer satisfaction.

This illustrates how one of customer satisfaction's big advantages - its simplicity - is also one of its big disadvantages. In isolation, one customer satisfaction score tells you very little about how you're doing. It is most useful when you can differentiate between different aspects of your product or service, thus allowing you to make improvements. It can also be useful to track your customer satisfaction score over time, or against industry benchmarks. Customer satisfaction is ultimately most useful as a comparative measure. When used well, it allows you to use your customers' feedback to keep them coming back for more!

Tutorial: Measure Customer Satisfaction in Displayr

To understand predictive lead scoring, we first have to understand traditional lead scoring so let’s rewind a little bit.

What is Traditional Lead Scoring?

Lead scoring is the process of assigning scores to prospects and potential customers based on relevant data. Relevant data could include demographic information. For example, the area of work, role title, etc. Other data can include frequency of online engagement or viewing particular website pages that signal an interest in purchasing. In traditional lead scoring, marketers rank the significance of certain actions to gauge customer intent and qualify leads. For example, a visitor that finds the homepage through an organic search and fills out a form or subscribes for more information will most likely receive a higher score than someone who opened an email or read a single blog post before bouncing.

Marketers and salespeople rank potential customers against a scale that represents how likely that lead is to convert and its perceived value. The resulting score determines which leads markets will assign the highest priority to be contacted by a sales representative. In an ideal world, leads are scored accurately and marketers can pass the most valuable new leads onto the sales team along with some handy information about them.

Unfortunately, we don’t live an ideal world and leads are not always scored accurately. Marketers often depend on their own subjective judgment or past data patterns to evaluate and weigh actions they deem relevant to making a sale. Opportunities may slip through the cracks or sales teams may spend too much time chasing under-qualified or ill-suited leads based on inaccurate scores.

What about Predictive Lead Scoring?

Here’s where predictive lead scoring can come to the rescue. Predictive lead scoring takes out or reduces the element of human error and increases the accuracy of identifying quality leads. Predictive lead scoring uses predictive modeling, a common statistical technique used to predict future behavior based on past behavior. Advanced predictive modeling algorithms combine historical and current data to generate a model predicting future outcomes. Linked CRM and marketing automation solutions provide internal data for these algorithms.

Predictive modeling algorithms pull in all this data. They analyze successful and unsuccessful leads in order to find patterns in the data. It is these patterns that identify factors that are most relevant and useful in predicting sales. Predictive lead scoring may be able to come up with an ideal profile of a customer that is most likely to buy based on this combination of historical demographic and activity data and therefore be able to identify the warmest leads. It can also help to identify patterns or relationships in the data that were previously missed. Beyond just reducing the margin of human error in lead scoring, predictive lead scoring helps the marketing and sales team align with data-driven lead scoring qualifications.

Check out "What's in the Future for Predictive Lead Scoring?" next.

The amount of sampling error is determined by four things. These are the sample size, the sampling method, the inherent variability in the population, and the population size.

Worked example

Stack Overflow, a popular online question-and-answer forum for software developers, recently conducted a survey that found that about 10% of people on their site were female. How much sampling error is attached to such an estimate? To understand the extent of sampling error, we will start by investigating the amount of sampling error we might expect if:

1. We assume that it is true that 10% of people who use Stack Overflow are women.
2. We have a sample of ten people.

If we randomly select ten people and 10% of people are female, this means that each of the people we choose will have a 10% chance of being female (this is a heroic assumption, due to factors discussed later in this article and also to non-sampling error, but it greatly simplifies the explanation of sampling error).

So, the chance of ten out of ten people in our sample being female is 0.1*0.1*0.1*0.1*0.1*0.1*0.1*0.1*0.1*0.1 =  0.00000001%. That is, there is essentially no chance that if in truth 10% of Stack Overflow users were women, that we would observe this as being 100% in a survey of 10 people.

By contrast, the probability of there being no women in our sample is 0.9*0.9*0.9*0.9*0.9*0.9*0.9*0.9*0.9*0.9 = 34.9%, where 0.9 is the probability that a man is selected at random, where this is 1 minus the probability of being a woman.

With a bit more math we can compute the chart below, which shows the probability that in our sample of ten people we would get 0% women, 10% women, etc.

What we really want to know is the true value in the population. Is it 10%, 5%, or 20%? Without talking to everybody we cannot know for sure, but we can make some inferences. We can see from the chart that if the true figure is 10% (which is what was assumed when creating the chart), there is a 34.9% chance of observing 0% women, 38.7% chance of observing 10% women, and 19.4% chance of observing 20% (2 or fewer) women. If we add these up, we can say if the true value is 10%, there is 93% chance that we would observe either 0%, 10%, or 20% – and virtually no chance of observing a sample of more than 40% female.

The next chart shows the calculations again, this time for 100 people. This chart tells us that if it was true that 10% of Stack Overflow visitors were female, then we could be confident of observing a value in a sample of 100 between 0% and 20%.

The actual Stack Overflow survey question about gender was answered by 35,990 people. Using the same math again, we can be extremely confident that if the truth is that 10% of all people on Stack Overflow are women, sampling error will mean that we observe a value between 9.4% and 10.5%. “Extremely confident” here means that there is a 99.9% probability that we would observe a value between 9.4% and 10.5%. Here, 9.4% and 10.5% are called the 99.9% confidence interval.

The role of sample size

As has been illustrated above, the bigger the sample size, the smaller the sampling error. The sampling error increases in proportion to the square root of the sample size. For example, when sample size is increased from 10 to 100, the sampling error halves, all else being equal.

The sampling method

In the calculations above, it was implicitly assumed that the samples were selected randomly (i.e., a simple random sample). With other types of samples, the math works differently. For example, with cluster sampling, the degree of sampling error is larger.

The inherent variability in the population

The example above investigated the sampling error for a proportion (i.e., a percentage). When investigating the sampling error for a mean (i.e., the average), we need to factor into the calculation the inherent variability in the population. The more variability, the greater the sampling error. For example, just as people vary more in their weights than their heights, the sampling error for weights is bigger than for heights.

The population size

The calculations above implicitly assumed an infinite population size. This assumption is rarely correct. If we modify the math to take into account the population size, the math gets more complicated. We see that the smaller the population, the smaller the sampling error. However, this effect is negligible unless the sample size is greater than 10% of the population size, so the effect of sample size can be safely ignored in most analyses.

Need to know more market research terminology? Brush up with our "What is" guides. 

Approaches to inferring latent variables from data include: using a single observed variable, multi-item scales, predictive models, dimension reduction techniques such as factor analysis, structural equation models, and mixture models.

Using a single observed variable

The simplest approach to measuring a latent variable is to find a single observed variable that is believed to be a sufficiently accurate measurement of the latent variable.  For example, if wanting to ask people how much they will pay, you can ask directly; or if you want to gauge intelligence, you can present people with a difficult mathematical question.

The chief virtue of using a single observed variable is simplicity. In the two examples just mentioned, however, this approach is not good. When asking people how much they will pay, they have a disincentive to be honest, as telling anybody the most you will pay increases the likelihood that you will get charged this amount. In the case of intelligence, asking a single math question is highly unreliable, as some people may get it right due to having done the same question recently at school. Furthermore, people high in non-mathematical aspects of intelligence will be incorrectly concluded to be less intelligent.

Multi-item scales

The standard solution that psychologists take to measuring latent variables is to use a series of questions that are all designed to measure the latent variable. This is known as a multi-item scale, where an “item” is a question, and a “scale” is the resulting estimate of the latent variable. For example, an IQ test typically involves people asking people between 100 and 200 questions, counting up the number correct (and then rescaling them to be consistent with 100 being the average IQ for people of the same age; different IQ tests have different ways of combining the answers).  Multi-item scales often have sub-scales. The Wechsler Adult Intelligence Scale IV IQ test has four sub-indexes: verbal comprehension, perceptual reasoning, working memory, and processing speed, and these are in turn broken up into further groupings of questions.

Predictive models

Latent variables can also be estimated using predictive models. For example, if estimating the latent variable of likelihood to cancel a telephone contract, an analysis could tell us that:

• 60% of customers who have been with the company for less than 12 months and who queried their bill in the last three months cancelled,
• 40% of customers who had been with the ISP for more than 12 months and had also queried their bill in the previous three months cancelled, and
• 5% of customers who did not query their bill cancelled.

This model allows us to assign a value of the latent variable of likelihood to cancel to each customer (i.e., where everybody is assigned a value of 60%, 40%, or 5%).

Dimension reduction techniques (e.g., factor analysis)

When we use multi-item scales we assume that each of the variables is a measure of the thing we are trying to measure. This assumption may be incorrect. Such assumptions can be checked by assessing the extent to which variables are correlated. For example, if answers to a question in an IQ test are uncorrelated with answers to any of the other questions, the implication is that the question likely does not measure an aspect of intelligence.

Numerous techniques have been developed for assessing the correlation-based relationship between variables, including factor analysis, principal components analysis, multiple correspondence analysis, and HOMALs.

Structural equation modeling (SEM)

Structural equation models are hybrids of predictive modeling and dimension reduction. Their principle use is when theory suggests the existence of relationships between latent variables (e.g., that two latent variables may predict a third).

Random effects models

Random effects models are predictive models that simultaneously estimate predictive models and estimate latent variables describing differences between people. There are numerous variants of such models, developed for all different types of data and many different estimation techniques, including random parameter logit models, random effects ANOVA, and Hierarchical Bayes, to name just three.

Sign up for Displayr