To practice conversation design diagnostics, I stress-tested Lufthansa's airline chatbot, Elisa, with prompts designed to surface common failure modes in conversational AI: compound requests, mid-conversation corrections, and scoped/negated requests. The goal was to identify a repeatable, architectural weakness and reveal how the underlying design handles natural conversation.
Across three tests, a pattern emerged: once Elisa enters a state where it expects a specific input format (ex: a booking code), it becomes unable to process any other kind of input from the user.
Diagnosis: The bot correctly addressed the fare/change-fee question but completely ignored the baggage request. This points to single-intent parsing. The system likely classifies a message into one intent and discards anything beyond it rather than recognizing and queuing multiple intents within a single message.
Diagnosis: This illustrates the clearest failure of the three. The user explicitly corrected their stated intent in plain language ("actually," "I don't want to," "I just want to"). Instead of recognizing this as a correction, the bot repeated its previous prompt, treating the correction as invalid input. The bot has no mechanism to distinguish "user is correcting course" from "user gave malformed input" as both produce identical rejections.
Diagnosis: Same failure as Test 2. Once Elisa is in the "awaiting booking code" state, any input that isn't a booking code produces the identical rejection. A scoped clarification ("just the outbound leg") is conversationally normal and should be parsed and retained for later use, not automatically discarded.
The three failures trace back to two related design flaws:
-
Single input type. Once the bot enters a state expecting a booking code, it cannot accept or interpret any other input — including corrections, clarifications, or scope changes even when that input is clear and on-topic.
-
Single-intent parsing. The bot identifies and acts on only one intent per message, silently dropping any additional requests rather than acknowledging and sequencing them.
The user is left unsure whether they were understood at all. A generic "I didn't understand" response that is repeated, signals the bot isn't tracking conversation state which quickly erodes trust with the user especially in a high-stakes situation where .
This doesn't require natural language understanding even basic correction-phrase detection would catch a meaningful share of real-world corrections since phrases like "actually," "wait," and "I meant" are predictable.
Silently dropping part of a request is worse than declining it. The user has no way to know their request wasn't registered and may assume it's been handled when it hasn't. Naming and sequencing multi-part requests is small but meaningfully improves perceived competence and reduces repeated, frustrated responses.
A conversational system that treats anything outside its expected input format as an error rather than as potentially meaningful language from the user. Small, low-cost interventions like these compound into a system that understands the user and what they are looking for.
- Whether Elisa can recover from a second correction in the same conversation
- How it handles ambiguous references (ex: "that one") with no prior context