Before we get started on the details, it’s helpful to establish a comprehensive definition of session length. At its most basic, session length is the amount of time that passes between the moment a user first interacts with a chatbot and the moment they navigate away from the chatbot or terminate the interaction.
Average session length is just as simple. If you take all of your chatbot conversations over a period of time (it could be a day, week, month or year – that’s up to you), add together the length of all interactions and then divide by the total number of interactions, you’ll get the average session length.
For instance, imagine you have three conversations in a day. The first lasts 2 mins 35 secs, the second 5 mins and 5 secs and the third just 25 secs. Your calculation for average session length would be as follows.
215 secs + 305 secs + 50 secs = 9 mins 30 secs
9 mins 30 secs / 3 interactions = 3 mins 10 secs
Average Session Length = 3 mins 10 secs
Now you have a means of working out your average session length, you’ll probably want to know what constitutes a good session length.
Unfortunately, it’s just not that easy. Generalisations can be made about session length that will, by and large, stand true. For instance, when analysing session length there’s generally an optimal ‘window’ that represents a ‘good’ session length.
If your chatbot is recording session lengths that are short of this window it may indicate that your bot is frustrating users and causing them to navigate away before their enquiry is resolved. On the other hand, long sessions may suggest that your bot is taking too much time to complete tasks. This will frustrate users too, making them feel as though the bot is wasting their time.
However, there are no established parameters for this optimal window. In other words, the ideal session length varies from bot to bot and will largely depend on what the chatbot’s primary function is and the context of the conversation.
Some chatbots are designed to complete enquiries in as short a period of time as possible. A good example of this would be a chatbot that helps users book train tickets. In this instance, the bot has a single, basic function and requires relatively little information to complete the task. The aim here is for the user to successfully complete the transaction as quickly as possible.
Other chatbots require more time. For example, a chatbot that helps users with shopping by recommending items will want to keep that user engaged for a prolonged period.
Ultimately, you’ll have to set the parameters for your chatbots optimal window. This can be achieved by looking at all of your interactions over a set period and marking them as either successfully resolved or incomplete. If your sample size is large enough, you should start recognising a general pattern. Successful interactions will sit within the window, while failures will mark the limits of your window.
It’s important to note that you’ll still have incomplete sessions that sit within your window and successful interactions that fall outside of it. These are outliers. What you’re looking for is a general pattern that makes it easier to identify when your chatbot is not performing as it should.
As well as marking the chatbot interactions as successful or unsuccessful yourself, it’s also a good idea to get customer feedback. Using customer satisfaction scores, you can assess whether your analysis of an interaction matches up with the customer’s view. After all, a chatbot can resolve a customer’s enquiry without that customer feeling as though the bot provided satisfactory service.
The most useful ways to leverage your session length metric as a way to improve service are to:
- Use session length to identify failed interactions – any interactions that fall outside the optimal window can be checked to see what went wrong
- Use session length as an alarm system – if you see an increase in session lengths outside of your optimal window, you know that users are interacting with your bot in a different way or that there’s an issue that needs to be resolved
By looking at those interactions that fall outside of the optimal session length window, you can identify where they went wrong and make changes to improve the service. In such cases, you want to ask:
- Should these interactions be classified as failures?
- Where does the bot fail and why?
- How could the flaw be resolved and the customer service improved?
Taken on its own, session length is a powerful warning system and a means of identifying problematic interactions. However, it really comes into its own when analysed in conjunction with other chatbot metrics. By pairing it with other metrics, it provides you with a more detailed picture of how your chatbot is performing, allowing you to make adjustments that maximise benefits to your organisation.