Explainable Adaptive Assistants

Deborah L. McGuinness, Tetherless World Constellation, RPI Alyssa Glass, Stanford University Michael Wolverton, SRI International Paulo Pinheiro da Silva, University of Texas at El Paso Cynthia Chang, Tetherless World Constellation, RPI

Motivation

Li Ding, Tetherless World Constellation, RPI

Usable adaptive assistants need to be able to explain their recommendations if users are expected to trust them. Our

Use

work on ICEE -- the

Integrated Cognitive Explanation Environment

-- provides an extensible infrastructure for supporting explanations. We aim to improve trust in learning enabled agents by providing transparency concerning:

Update Ask Understand



Provenance



Information manipulation



Task processing



Learning

SPARK Wrapper and Database

Designed to The ICEE explainer includes:



Descriptions of question types and explanation strategies



Extract an explanation-relevant snapshot of execution state, including learning provenance



Communicate that information to the Explainer



Architecture for generating interoperable, machine interpretable, sharable justifications of answers containing enough information to generate explanations System utilizes introspective predicates to build a justification of current task(s), then outputs the justification structure via an RDBMS



Components capable of obtaining justification information from SPARK (a BDI agent architecture)



History of execution states as justifications



Database schema designed for easy, scalable storage and retrieval of relevant execution and provenance information

Explanation Foundation PML

: Provenance, Justification, and Trust Interlingua

Inference Web Toolkit

: Suite of tools for generating, browsing, searching, validating, and summarizing explanations

ICEE

: Explanation framework for explaining cognitive agents, with focus on task processing and learning



Designed to be reusable for other types of reasoning about execution and action

Dialogue

Given a PML justification of SPARK’s execution, ICEE provides a dialogue interface to explaining actions. Users can start a dialogue with any of several question types, for example: “Why are you doing ?”

Explanation Dispatcher

Architecture

CALO GUI TM Explainer Task Manager (TM) TM Wrapper

ICEE contains several explanation strategies for each question type and, based on context and a user model, chooses one strategy, for example: Strategy: Reveal task hierarchy “I am trying to do and is one subgoal in the process.”

PML Generator Task database

User Studies

1. Interviewed 13 adaptive agent users, focusing on trust, failures, surprises, and other sources of confusion. Identified themes on trusting adaptive agents, including: ICEE then parses the execution justification to present the portions relevant to the query and the explanation strategy. Strategies for explaining learned and modified procedures focus on provenance information and the learning method used. ICEE also suggests follow-up queries, enabling back and-forth dialogue between agent and the user.



Learning can make users feel ignored

Current Focus



Users want to ask questions when they perceive failures



Added strategies focused on conflicts

 

Granularity of feedback is vitally important



Users need transparency and access to knowledge provenance Users require explainable verification to build trust



Underlying infrastructure updates for combinations of justifications



Explanation generation and abstraction strategies based on improvement criteria including step minimization .

2. Study planned for 1Q09 focusing on conflicts and trust



Identifying integrity issues that require explanation Glass, A., McGuinness, D.L., and Wolverton, M. “Toward Establishing Trust in Adaptive Agents.” IUI 2008

Alyssa Glass, Deborah L. McGuinness, Paulo Pinheiro da Silva, and Michael Wolverton. Trustable Task Processing Systems. In Roth-Berghofer, T., and Richter, M.M., editors, KI Journal, Special Issue on Explanation, Kunstliche Intelligenz, 2008. Jiao Tao , Li Ding , Jie Bao , Deborah L. McGuinness .

Characterizing and Detecting Integrity Issues in OWL Instance Data

, In OWLED 2008 EU, 2008 Paulo Pinheiro da Silva , Deborah McGuinness , Li Ding , Nicholas Del Rio .

Inference Web in Action: Lightweight Use of Proof Markup Language

, In ISWC'08, October,2008 .

Paulo Pinheiro da Silva , Nicholas Del Rio , Deborah McGuinness , Li Ding , Cynthia Chang , Geoff Sutcliffe .

User Interfaces for Portable Proofs

, In UITP'08), August,2008

www.inference-web.org