Because AI systems are probabilistic, your system will probably give an incorrect or unexpected output at some point.

This makes it critical that you help users calibrate their expectations about system functionality and output. Do this by being transparent about both its capabilities and limitations.

For example, indicating a prediction could be wrong may cause the user to trust that particular prediction less. However, in the long term, users may use or rely on your product more, because they’re less likely to over-trust your system and be disappointed.

Many AI systems rely on user data to personalize their recommendations or optimize the system.

If you are collecting user data, proactively ask users for permissions early on, and make it easy for them to adjust their settings.

Over time, you might also need to remind your users of the settings that they’ve chosen so that they can update to match their current preferences.

When prompting users to set or review their permissions, explain what you’re using their data for and why, and make sure your explanations are easily understandable.

In high stakes domains or when users are new, it can be useful to provide users with contextual information from other people to help them determine the value of your recommendations. Sources of this kind of contextual information can include:

• third-party experts, whose information can help users vet the quality of your output
• relevant community groups, who can help users calibrate their trust through human-to-human connection

This additional context, in tandem, with explanations and confidence displays, can help set user expectations about the quality of your recommendation.

When your AI-enabled system behaves in a way that a user doesn’t expect or want, make sure that they have an option to share feedback. And, as much as possible, use that feedback to improve your model.

Feedback in AI systems can take a range of forms, including,

• Giving a thumbs up or thumbs down on a recommendation
• Hiding unwanted recommendations
• Flagging or reporting problematic recommendations
• More traditional feedback flows, where a user manually reports a problem through a form or other mechanism

Once a user gives feedback, acknowledge that you received it. If possible, let them know how the system will respond to the feedback.

Allow your users to supervise automation and take back control when needed, rather than automating across the board.

This is helpful to users in a number of ways, including:

• Building comfort in new or high stakes situations with the support of controls to override the system if needed.
• Learning how the system works as they exercise their controls. For example, by manually confirming an appointment time recommendation, they become more familiar with the system’s appointment recommendations.
• Giving them a way to complete their task when the system doesn’t work as intended.

Even in cases where users may not frequently exercise the option to take back control, it can be helpful to let them know that they have that option, and to help them build confidence in the system.

When determining how much to automate your product flows, think about the stakes of your product, and the level of comfort that users may have with your type of product.

In low risk, well-established products, like content recommendation systems, you might choose to prioritize a more heavily automated product flow where user control is available but optional.

However, when onboarding to a new type of product, or in high-stakes situations, errors can be particularly problematic, and can corrode user trust and potentially cause dangerous situations. In such cases, design your system to give users more control over the system.

When an AI system fails or gives users a poor prediction, the easiest path forward is often to let the user take over in a non-automated way.

Make it as easy and intuitive as possible for users to quickly pick up where the system leaves off. Give them all the information they need to take the reins:

• Awareness of the situation
• What they need to do next
• How to take their next action

In difficult or high-stakes situations, you may even need to redirect users to a human for extra support.