LLM Impact on Conversation Design, A layered Perspective

Large Language Models (LLMs) like ChatGPT will have a significant impact on conversation design, but how? Here, we decompose conversation design into layers, following the same principle as we do with GUI design, enabling us to identify specific areas where conversation design is likely to be affected in the future. To see that, let’s revisit the dichotomy of GUI design.

UX vs UI Design for GUI Applications

Most GUI applications are developed in two parts: the backend and frontend. The backend is responsible for managing data, processing requests, and executing the business logic of the application, while the frontend directly handles user interaction. In order to adapt the same GUI application to screens of different sizes, frontend design is further decomposed into UX design and UI design.

The main responsibility of any frontend is to collect the information needed to trigger backend APIs. Clearly, the information we need to collect from the user is entirely defined by service design and business logic. UX design, or simply interaction design, determines how we collect the rest of the information given what are collected from the user thus far. Interaction design decides things like whether to provide a dropdown list for the user to choose from, how to verify user input, and what to do if the verification fails. A good UX design can provide great value to users even when paired with a basic UI design. The early days of Google and Craigslist serve as excellent examples.

On the other hand, UI design deals with the visual aspects of the user interface, encompassing elements such as typography, color schemes, icons, buttons, and other visual elements. A good look-and-feel also is important for a positive user experience, as it makes interaction easier for users and instills confidence in them, thereby facilitating problem-solving.

Conversation design can also be decomposed

There are many definitions of conversation design on the web; the one I like is from the Conversation Design Institute:

Conversation design is the practice of making AI assistants more helpful and natural when they talk to humans. It combines an understanding of technology, psychology, and language to create human-centric experiences for chatbots, agent, copilot and voice assistants.

As with most definitions, conversation design typically focuses on two aspects: 1) being helpful, effectively providing information or services, and 2) being natural, allowing users to receive assistance with minimal effort.

In order to bring consistent conversational experiences to markets with different languages in a cost-effective way, we can decompose conversation design into interaction design (also known as dialog design) and language design, and reuse the same interaction design for consistency. In addition, we can further decompose language design into input related NLU design and output related response design.

These three design tasks are roughly mapping to the three stages that that a user’s query goes through from the moment it is inputted into a system until the system provides a response:

1. Understand user’ natural language input.
2. Figure out which dialog act best move the conversation to service.
3. Render the dialog act into natural text with desired style.

Of these three aspects, we predict:

1. NLU design will likely disappear completely.
2. Interaction(dialog) design will become the dominant concern.
3. Language(response) design will gradually reduce to simple style selection.

NLU Design will be killed by LLMs

Natural Language Understanding (NLU) plays a crucial role in enabling conversational systems to comprehend and interpret user inputs accurately, historically making NLU design a central component of conversation design. However, natural language understanding poses more of a technical challenge than a design challenge, as its importance is not dictated by essential business objectives but rather by the inadequate technology we use to solve it. Shallow machine learning-based NLU requires building every intent model from scratch, demanding significant attention and manual effort.

With the rapid industry transition to Large Language Model (LLM)-based systems, often zero-shot capable, the importance of NLU design in conversation design will quickly diminish — and this transition is already underway. Conversational User Interface (CUI) platforms with no mention of NLU training are already emerging.

Interaction Design will stay

To make chatbot useful, it needs to provide services and information to users with as little effort on their part as possible. To achieve this, interaction design needs to utilize all available information, including user goals, business objectives, and various contexts in guiding the user toward the desired service. To do this, interaction design need to decide what is the most effective dialog act to communicate to user.

Dialog acts are language-independent structural representations of the meaning that chatbots need to communicate to users. They typically consist of communicative functions of the meaning one wants to express, along with a payload dictating the content of the meaning. There are many different classifications of communicative functions, some coarse and some fine. Here are some examples:

1. INFORM — Inform the value for a slot to the user. The slot and values fields in the corresponding action are always non-empty.
2. REQUEST — Request the value of a slot from the user. The corresponding action always contains a slot, but values are optional. When values are present, they are used as examples for the user e.g, “Would you like to eat Indian or Chinese food or something else?”
3. CONFIRM — Confirm the value of a slot before making a transactional service call.
4. OFFER — Offer a certain value for a slot to the user. The corresponding action always contains a slot and a list of values for that slot offered to the user.

Let’s use movie ticketing as an example. We need to determine the user’s preferred showtime before issuing the ticket to them. When prompting the user to supply this information, it is often useful to provide a value recommendation (OFFER [showtimes]) based on the movie they want to watch as well as the available seating. This way, users do not need to give their choice and only then be told it is sold out already. Another example is always to confirm (CONFIRM showtime) with the user for high-impact occasions.

Good interaction design requires not only an understanding of conversation principles but also up-to-date access to various business signals, such as inventory. Moreover, in today’s highly competitive world, only these differentiated experiences will win. This implies that this kind of design will always be a manual process and will require insights gained through continuous experimentation. It is possible that Large Language Models (LLMs) can be useful as a co-pilot in this process, but it will be hard for LLMs to take the main responsibility for interaction design away from conversational designers. Ultimately, it will become an increasingly important aspect of conversation design as businesses start to build conversational experiences.

Language Design will be greatly simplified in long term

Response design focuses on crafting the language and linguistic elements used within a conversation to convey the meaning defined by the dialog act in the chosen language. It involves shaping the tone, style, vocabulary, grammar, and syntax to ensure that the conversation feels natural and engaging to users. Response design takes into account factors such as cultural nuances, language preferences of the target audience, and the clarity and simplicity of communication.

Sometimes, the response design can be specified by the persona that represents the brand to target users in the channel.

In the short term, crafting engaging messages like these for intended users will still requires human involvement. But as we start to provide more and more such examples with a style label, and it is likely that LLM/Gen AI can potentially do a great job in converting dialog act into responses like these created by professional per style instructions.

Parting Words

Large Language Models (LLMs) will have a huge impact on conversation design going forward. If you are conversation designers, these are the trends that you need to know:

1. The NLU design should already be based on LLMs, given the amount of text they used in pretraining. They have demonstrated that they can master NLU tasks with very few or even no labeled examples. This makes it much easier to ground conversations to new APIs.
2. Instead of crafting the response from scratch, start using prompt engineering to generate drafts and modify them. This offline usage leverages LLMs while retaining our control over the style and other aspects of messaging. It also creates labeled examples for mapping prompt instructions and dialog acts into engaging messages, and hopefully, eventually automates the response design altogether.
3. Start to focus on interaction level. As a matter of fact, start to think about service design. As we figured out how to build usable conversational experiences in a cost effective manner, business will need more help to translate the business objectives and ever changing business conditions into differentiated conversational services that benefit both users and businesses.

LLMs will release conversation designers from the tedious technical aspects, allowing them to focus their energy on more business-dependent and higher-impact interaction design. Furthermore, new low-code platforms often integrate LLM’s text-to-code capability, potentially making conversation designers omnipotent in chatbot development. Hopefully, we will start to see truly usable conversational experiences emerge.

Reference:

Rethink Chatbot Building for LLM era

3-Layer Agent Architecture

5 Levels of conversational UI