| Abstract |
Coalition situational understanding (CSU) involves the use of artificial intelligence (AI) based assets to assist human decision-makers, operating at or near the edge of the network, in responding to rapidly-changing events. CSU exploits data collected from sensors of multiple modalities, for example, visual and acoustic. A user will often need to rely on an AI asset created by a different coalition partner; therefore, the assets must be capable of generating explanations for their outputs to engender user trust. Moreover, the edge setting requires explanations to be as specific and targeted as possible. We demonstrate a novel technique—selective audio visual relevance (SAVR)—designed to support CSU in real-time, edge settings, by providing effective and resource-efficient explanations for AI-based activity detection from videos with audio soundtracks. We illustrate the applicability of SAVR as part of a real-time CSU ‘dashboard’ for monitoring events in an urban terrain setting. |