ANALYSING CUSTOMER EXPERIENCE FEEDBACK USING TEXT MINING: A LINGUISTICS-BASED APPROACH
Car park and transfer case study at an airport

21 March 2014

Francisco Villarroel Dr Charalampos Theodoulidis Dr Jamie Burton Prof Thorsten Gruber Dr Mohamed Zaki

Content
• Customer Feedback and Value co-creation • (Text Mining and its applications) • Car park and transfer case study at an airport • Results • Managerial Implications • Further Research

Customer Feedback
“Customer feedback process” plays a key role in ensuring that information from complaints, compliments, market research and other sources are systematically collected, analysed, and disseminated in ways that will drive service improvements” (Lovelock and Wirtz, 2007).

“Service Quality (SQ) is a marketing stream that considers customer feedback as an opportunity for assessing customer (di)satisfaction. SQ can be “measured through the difference between customer expectations and their real experience with the service” (Parasuraman et al.1985).

Customer Feedback Process

Implicit

Explicit

Determined Actions (i.e. eye tracking, reading time, number of scrolling, etc.)

Platforms (e.g. surveys, e-mails, online review, blog, etc.)

Customer Feedback Process
• Many companies analyse explicit feedback using quantitative

methods because of simplicity in analysis
• Evaluating an entire service of quantitative measures will result in

an
• incomplete understanding of customer experience (Macdonald

et al. 2011; Vargo et al. 2007)
• only superficial information about the entire customer

experience (Caemmerer and Wilson 2010)
• not capture all the resources and activities involved (Gronroos

2012)

Compliments and Complaints NOTES
Compliments
• Affects positively front line employees • Promotes positive WOM across

Complaints
• Valuable information about what should

be improved.
• Delight in the case of good service

customers

• Provide information about core

recovery.
• Maintain long term value from

competences of the company

dissatisfied customers.
• Represent areas that does not need

improvements.
• Lack of Originality in their content. • Receive less attention from customer

• Can damage self confidence on front

line employees.
• Can generate negative WOM

managers.
Luthans, (2002); Kraft and Martin, (2001); Soderlund, (1998); Gruber, (2010); Chebat et al (2005); Buttle and Burton, (2002)

Co-creation
What’s Value co-creation?:
• It’s the process of interactions between

the customer and the company’s service proposition.
• It’s a form of understanding the customer and

http://www.brsglobal.com/tag/ikea/

the firm as sum of resources which constantly interact in order to generate value (Vargo and Lush, 2004)
• These interactions occurs across a

service process, through different

activities which start from the contact of the customer with the firm until the end of the service (Payne et al 2008)

http://freshome.com/2008/08/05/ikea-catalog-2009-now-available-online-here

Text Mining
• Process of analyzing collections of textual materials in order to

capture key concepts & themes & uncover hidden trends.
• 80% of firms information is stored in text format.(Ur-Rahman and

Harding 2011)
• The approaches covered in literature:
• Linguistic approach: consider the natural language characteristics of the text in the

documents (e.g., syntax, grammar)
• Non-linguistic approach: view documents as a series of characters, words, sentences, paragraphs. Counting the number of times specific words appear in a document

Objectives
1. Automate process of customer feedback analysis

through a text mining model.
2. Determine what are the most important resources and

activities for the customer when using this service.
3. Evaluate the potential of customer compliments and

complaints for improving their service experience.

Research Process
1. Understand the customer feedback process 2. Collect a sample of customer compliments and complaints. 3. Development and test of a text mining model 4. Present the results to the participant company and evaluate

Customer Feedback Process
• Daily online survey is sent to customers who parked

their car 2 days before.
• In the Survey, open question asking:

“What is the single most important factor you feel we can improve upon to enhance your car park experience”
• The company receives approximately 1000 comments

per week, 50,000 responses annually on average

Current Practice
1. Each comment is classified into just one category (despite often

including more than one compliment, complaint, or suggestion)
2.

Positive or negative sentiments are individual categories, with no relationship to a specific element of the service The classification of comments by means of manual annotation is not consistent (approximately 2 weeks to generate a report)

3.

Proposed Linguistic- based Text Mining
Sample 100 comments

Population

1092 comments

Sample Process
Preprocessing tasks

Sample of 100 comments: • Extract the sentences with more valuable information for The M.A.

Library of Concepts

Categorization of Concepts
Comment
Car Ratin Park g Single improvement factor Barrier did not recognise my prebooked credit card press buzzer but person very helpful bus going out was fine waiting 15mins for bus very poor

Pattern Development

Model

Results and Model Refinement

Barrier did not recognise my pre-booked credit card had to press buzzer but person very helpful. Bus going out was fine - after waiting 15mins for bus on return we walked - very poor

E

5

Text Mining Process
Preprocessing tasks

Library of Concepts

Categorization of Concepts

Sample 100 comments: • Extraction of the main concepts by sentence • Categorization of the concepts into 4 main Groups: Resource Company, Resource Customer, Activities, and Attributes

Pattern Development

Resource Resource company Customer

Barrier credit card Buzzer/ Person bus bus 15 mins

Activity 1

Activity 2 Opinion

C&C Complaint

did not recognise Pre-­‐booked press going out waiting

Model

very helpful Compliment fine Compliment Complaint very poor Complaint

Results and Model Refinement

Text Mining Process
Preprocessing tasks

Sample 100 comments: • Extraction of the most common sentences patterns for compliments and complaints

Library of Concepts

Categorization of Concepts
• Barrier did not recognize my pre-booked credit card
CR Act CuR

Pattern Development

“Complain about entrance”
• …had to press buzzer but person very helpful
CR CR ATP

Model
Act

“Compliment staff”
• Waiting 15 mins for bus
CuR CR

Results and Model Refinement

“Bus Complaint”

Car park-transfer service process

Based on Payne et al (2008); Vargo and Lush (2004)

Text Mining Process
Preprocessing tasks

• The model has in total : • 694 patterns arose from these comments • 47 Subcategories of parking and transfer service process • 678 concepts mapped to these subcategories • 92% overall accuracy

Library of Concepts

Categorization of Concepts

Pattern Development

Model

Evaluation and Refinement

Right Predictions
Compliments vs. Complaints
Complaints Compliments

Implications:
• Considering that the questions was asking about suggestions or complaints it was interesting to find complimenting customers.

14%

86%

Complaints
Complaints through the Service
Most of the complaints were when the customers were inside the Car Park trying to park their car. 87 54 Arriving Car Park Parking Car 298 111

Booking

Bus Service

Compliments
In the case of Bus Service Most of Compliments were Related with the bus driver Helpfulness and Friendliness of staff was found valuable for Customers For general compliments it would be possible to sub-divide into new categories

Compliments
Bus Service Staff 8% 22% 70% General

Overall Results
Service Process Booking
-Booking general -Price

Right Predictions 87 23 64 54 298 71 25 9 111 65 17 111 550

Arriving Car Park Parking Car
-Space

-Staff
-Facilities -Directions -Others Car Park -Others Customer Resources

Bus Service TOTAL

Service Process General Bus Staff TOTAL

Right Predictions 62 7 20 89

Complaints Wrong Predictions Total 3 1 2 1 38 3 4 0 3 12 16 3 45 Compliments Wrong Predictions Total 7 1 2 10

Accurancy 90 24 66 55 336 74 29 9 114 77 33 114 595 97% 96% 97% 98% 89% 96% 86% 100% 97% 84% 52% 97% 92%

Accurancy 69 8 22 99 90% 88% 91% 90%

Implications
• Use of this model helps close the gaps in the service process

from a customer-centric perspective.
• Concepts such as service blueprinting might be updated

and improved through text mining.
• Addressing gaps in customer-centric service blueprint

could enable organizations to modify service offerings
• How changes in service offerings affect service encounters • How activities and resources are affected.

• The importance of development of text mining patterns could

aid in developing better predictive models

Limitations and Further Research
• The proposed text mining model is domain specific • The proposed model requires work to improve data capture and

accuracy and to be tested for another dataset.
• However, the approach could be tested and adapted to other

domains.
• Further research could investigate how information gathers from text

mining can be integrated in company information systems