But I still haven’t found what I’m looking for…
All feedback provided through ‘Give feedback on care’ (GFC) is specific to a particular service. The first step to the form asks users to search for the relevant service. This way, we ensure the information is picked up by the appropriate inspector. However, there is a concern that not all users are able to locate their service and therefore unable to complete the form before even providing their feedback.
A successful search is where a user clicks on a result after making a search. Since Private Beta, our successful search rate has hovered underneath 60% for users. In addition to this, around 80% of successful search results display in the top 3 results. Whilst both of these statistics are fairly strong, it still leaves space to ask — what are the common reasons users are unable to find their service?
Sampling unsuccessful searches
Taking a sample of 1,800 unique searches from May — June 2020 I followed the journeys that users were experiencing to see if there were any common themes and trends we could unpick from the 40% of unsuccessful searches.
Using the test GFC environment, the CQC (Care Quality Commissions) main website and Google search, I attempted the same search query to understand what the user may have been looking for and placed each failed search into one of the following categories:
- Not specific enough — Users who weren’t providing specific enough detail, for us to identify what they were search for. This category includes general terms i.e. ‘care home’
- Provider name — User is searching for the provider rather than the individual service. GFC only includes service name — not provider name
- Top result(s) — Users search did produce the correct result, but the user didn’t continue their submission
- Not a real search — Users search term was irrelevant or incoherent
- Archived — User searched for a service that has been archived
- It’s Scotland/Wales/N.I — User searched for a provider in the respective country
- Not in list — service was not in the list
- Misspelt — Users search was misspelt and therefore didn’t give them the right result
- Different name — service is registered under a different name
- Person — User searched for a person’s name
- Acronym — User used an acronym for the service name
Searches by themes
The largest group was ‘not specific enough’ at 45.8%. These were users where I was unable to place their search into a specific group — essentially, an ‘unknown’ category. Despite this, there were common themes that emerged in the dataset.
Main themes
11.9% of users were searching for ‘provider names’, however the GFC search results only displays ‘service names’. Note that a provider may have multiple services registered. The user would therefore need to be aware of the registered ‘service name’. This information is available on the CQC main website, however the link between service and provider name is unclear for the average users wanting to provide feedback.
8.8% of users were looking for a service based in Wales, Scotland or Northern Ireland. The CQC only covers services based in England. Despite, the main CQC website clearly states that it is an independent regulator of health and social care in England, there are still a strong number of users looking for services in other regions of the UK. Note that each region has its own regulator.
5% of unsuccessful searches were for archived services. There are number of different reasons why a service might be archived i.e. no longer in service, registered under a new name. On the main CQC website, users are still able to search for a service even if it’s been archived. The GFC search does not currently display these results.
2.8% of searches include common acronyms. At present, the search function doesn’t take acronyms into account and so there is a disconnect in how users would refer to services and how our system displays these results. Although only appearing in a lowered number of searches, this could be an easier issue to resolve if we begin working on a list of acronyms.
Secondary themes
9.2% of searches displayed as the top result. However, at this point in the journey the user decided not to continue with the form.
A combination of misspelt words and erroneous searches came to 9.9% of results, which was always likely considering that we’re only looking at unsuccessful searches.
4.4% of results were not in the list. These services exist (confirmed by Google), however they were not in either the CQC main website or GFC search results
What’s next?
- Discuss further with the UX team on how we manage searches for provider names, archived results, and Scotland/Wales/Northern Ireland searches.
- Compile a list of common acronyms used in searches.
- In the last 3 weeks, the GFC conversion rate has maintained around 39%.
- Despite an increase in both the number of form starts and overall submissions, these numbers have not proportionally increased with each other hence a lower conversion rate for the GFC journey. I believe the ‘Because we all care’ campaign will have had an impact on visits and form starts, however I’ll need to investigate further.
- A/B tests for ‘Give your feedback’ and ‘Can we contact you?’ pages are drawing to a close and we’ll be able to summarise the results soon.
