I was included by a Senior UX Researcher in some NPS review after a Construction Services Provider in the UK was performing poorly against it's competitors. They had recieved poor feedback across many platforms and their user retention rate was poor. They hired a User Research team (I was included for a part of this research as a Mentee) to understand what was causing this negative feedback.
A general overview
The NPS 5/10 score gave us an insight into our detractors and analysing their responses helped us to understand why their experience has lead them to give a 5/10.
The analysis we conducted were:
Sentiment Analysis
Thematic Analysis
Here we determined a user’s overall sentiment or feeling towards the service; positive or negative
Here we delved deeper into the qualitative data, understanding why a user has had a negative or positive experience
Entering into the research, the goal was to identify what were the main themes taken from amongst the feedback given towards the 5/10. This way we would be able to identify main pain points during the user journey in reaching their goal when using the product. This data can then be used in future design iterations. The data collected was from an NPS Survey given to users during the first part of 2022.
This is perhaps not surprising but 92.5% of users comments were negative, hence why they gave a 5/10 detractor score.
As the 5/10 score suggests, many of these users weren't likely to recommend the service to a friend or colleague. Their experience when using the product was overall poor.
Their language when providing feedback also reflected this, as it was generally negative.
Some key comments were as follows:
“Website is not user friendly and sometime getting responses to questions can take a lot of time.”
“there has been no advantages , hopefully this changes otherwise I see it as a cost that doesn't make sense”
Keywords: difficult, not easy to use, no advantages, very complicated, very frustrating
Users were generally confused by the timing of the NPS question.
“only just signed up so not sure how well it works yet”
Users struggled using the interface and lamented the confusing and difficult layout.
“Website is not user friendly”
“Website is really difficult to use”
Users sometimes didn’t feel the service provided them any benefits.
“Not sure it provides much in the way of business opportunity”
As a summary, users scoring 5/10 on the NPS question, generally felt the website was difficult to navigate and overall was frustrating to use.
Users also feel that the service doesn’t provide them much in the way of benefits, claiming sometimes they weren’t getting much work from or that they felt constrained to use the platform due to client requests or lack of choice in the field.
Finally, users feel the NPS survey was appearing too early into their experience and many wondered why they needed to answer.
Usability was a major pain point throughout much of the feedback. As the NPS is only a small part of the research process, further research could be carried out on the site itself, alongside a UX audit focusing on User Experience guidelines and accessibility guidelines. This focus on usability study should be done alongside the data we've analysed in the NPS study to find out where users are struggling on the platform and addressing these pain points.
These results and data can then be used when ideating with the design team to find potential solutions to these usability issues.
I was only focusing on a small portion of the feedback (5/10) so this only gave me a small fraction of the overall user feedback. This was a challenge as I wasn't able to have a full, complete picture as to how the users as a whole felt about the service. This could give a more distorted view as to how the users view the service and distort my feedback if not a part of a team.
Being able to collaborate with a Senior UX Researcher was really useful in my Research career progression. I was able to see the steps he took in analysing the data, with thematic tagging, to understand overall user feelings towards a service and I was able to make sense of a lot of feedback (over 800 pieces of feedback). This was a first for me, to work on such a big project and I hope I cant take and hone these skills further.