Improving Onboarding Flow by Refining Uncertainty Detection

Introduction

In the Breniapp/brenia project, ensuring a smooth user onboarding experience is critical. A key part of this is understanding and responding appropriately to user input, especially when users express uncertainty. We recently addressed an issue where our system was incorrectly flagging valid responses as uncertain, leading to a frustrating user experience.

The Problem

The onboarding assistant was designed to detect uncertainty in user responses and offer helpful suggestions. However, the initial implementation had a flaw: it used a simple length check to identify uncertain responses. Specifically, responses with two or fewer words and fewer than 15 characters were flagged as uncertain. This caused problems because valid short answers like "consultant", "Mexico", or "startup" were incorrectly flagged, causing the assistant to repeat its help message in a loop.

The Solution: Refined Uncertainty Detection

To address this, we removed the length-based check for uncertainty. Now, the system only flags a response as uncertain if it contains explicit phrases indicating doubt or confusion. This approach relies on identifying specific keywords or phrases that indicate the user is unsure of how to proceed.

Here's a simplified example of how you might implement such a check in PHP:

function isUncertainResponse(string $response): bool {
    $uncertaintyIndicators = [
        "I'm not sure",
        "I don't know",
        "maybe",
        "perhaps"
    ];

    $response = strtolower($response);

    foreach ($uncertaintyIndicators as $indicator) {
        if (strpos($response, strtolower($indicator)) !== false) {
            return true;
        }
    }

    return false;
}

$userResponse = "I'm not sure how to answer.";
if (isUncertainResponse($userResponse)) {
    echo "Please provide a more specific answer.";
} else {
    echo "Thank you for your response.";
}

Results

By removing the length-based check and focusing on explicit indicators of uncertainty, we significantly reduced the number of false positives. Users are no longer prompted unnecessarily, and the onboarding flow is smoother and more efficient. This change allows the onboarding assistant to focus on genuinely uncertain responses, providing more relevant and helpful assistance.

Getting Started

1. Identify common phrases or keywords that indicate uncertainty in user responses.
2. Implement a function to detect these phrases in user input.
3. Test your implementation thoroughly to minimize false positives.
4. Continuously refine your list of uncertainty indicators based on user feedback.

Key Insight

When designing interactive systems, it's crucial to avoid overly simplistic heuristics. Relying on explicit indicators, rather than arbitrary rules like length, leads to more accurate and user-friendly interactions. By focusing on the intent behind user input, we can create a more intuitive and effective onboarding experience.