The Problem
Little experience = Large learning curves...
Most of boodleAI's customers had little experience with AI and predictive models prior to using the platform, so they faced significant learning curves. One such learning curve was the platform's terminology, including the term 'Predicted Fit', which is an important factor in determining a predictive model's effectiveness when applied to a group of leads.
I received direct feedback from a couple of clients who were confused about outputs that looked like the below...
The Existing Experience...
Does something look off to you?
A tech issue? A data issue? AÂ design issue?
After reporting this issue to the dev team, I learned that Predicted Fit was being calculated as an average for each category of leads, and this was causing anomalies in the output. I communicated this to the clients, who said they understood - but after some time, I noticed they still weren't using these particular models.
So, I decided to investigate this further...
Through the research, a couple of patterns emerged:
- There was a lack of understanding about what Predicted Fit even was, or how it was being calculated for leads scored by the predictive model
- Â Users were unclear on the relationship between Predicted Fit and the lead categories (Predicted Fit is used to categorize lead quality - i.e. distinguishing between those who the model categorized as A leads vs. B, C and D leads)
Ultimately, this was contributing to a larger issue: Users didn't trust the predictive model
Iteration after Iteration...
I had a project scoping meeting with the VP of Product (also the PM for this project). We both agreed that the Predicted Fit averages displayed below the chart should be completely removed, because they were essentially inaccurate.
That left us with a fundamental question: How can we display Predicted Fit in a more accurate way?
I drew inspiration from an existing experience within the platform - a chart on the admin side where client predictive models were validated for accuracy. The output was similar with a key difference. On its X-axis, there was a line that displayed what the Predicted Fit at any one point. This showed me it was technically possible to display a less static chart.
I started exploring how we could translate this to the customer facing side in an even more dynamic way, to enable users to select any segment of contacts they wanted and know the Predicted Fit for that segment, rather than a pre-selected segment.
I took a Double Diamond approach, going wide by designing every single possible solution I could think of, then narrowing down to the best ones with guidance from the VP of Product.
The Solution: A Dynamic Experience
My project objectives guided me to the solution:
- Include a user-friendly definition of Predicted Fit
- Display Predicted Fit as an aggregate for a selection of leads, instead of a static average tied to a lead category
- Give users more granular control over the chart elements to better demonstrate the relationship between the chart elements
More control, more engagement             Â
The first element of the design was a blue bar that expands and contracts as the user moves the red slider. The aggregated Predicted Fit for all leads at and above the selected score cut off is displayed above the slider. Displaying aggregate Predicted Fit for any selection of leads meant the tool did not have to display static averages, instead calculating Predicted Fit dynamically. The dynamic display also shows the relationship between all the elements more clearly. The blue bar clearly represents the score cut off selection at each point, while the display above it tells the user what the Predicted Fit is for the leads in their score selection, which is more relevant to the user's goal than a static average.
Tooltips Provide Extra Information
The second element of the design were the tooltips, which clearly and concisely define Predicted Fit for the user. For these tooltips, I consulted with the Chief Data Officer to ensure the explanation was accurate.
Collaboration and Handoff
Before I presented to the entire development team and the CTO, I wanted to ensure it was technically feasible. Because I had already worked with the Front End Developer on other projects, I knew he had very good insight into all potential constraints. I presented him with the ideal solution and one backup solution. While he thought the ideal solution would be harder to implement, he decided he'd rather spend more time making sure it can be implemented, because it felt like the right one. His support made it easier to get buy in from the rest of the development team.
I also took part in QA to ensure everything was working as intended during implementation.
Impact
After releasing the redesigned chart, we noticed users experiencing less friction points when interacting with the different elements on the chart. Users also became more self-guided in making decisions using the chart and were less reliant on customer success manager support.
An Enriched Analytics Platform for Sales, Marketing, and Fundraising Teams
boodleAI is a SaaS startup that specializes in enriched analytics for sales, marketing, and fundraising teams. Their mission is to democratize data, making it faster and easier for all organizations to locate their best leads and prospects in any contact list. By using advanced data enrichment and insightful predictive analytics, boodleAI enables its users to achieve lifts in conversion, engagement, and retention rates.
As an interaction designer at boodleAI, I designed a number of new features and platform enhancements based on customer feedback.