| Abstract |
Deep learning models have been able to significantly outperform reasoning-based models for inferencing over transient features of a latent space, i.e., they learn data representations to make decisions based on the current state of the inputs over short periods of time. However, deep learning currently falls short for detecting events with complex spatial-temporal dependencies primarily due to insufficient data. In this paper, we propose DEEPCEP++, a framework that integrates the concepts of deep learning models with complex event processing engines to make inferences across distributed, multimodal information streams with complex spatial-temporal dependencies. DEEPCEP++ is an extension of DEEPCEP1, which utilizes deep learning to detect primitive events. A user can define a complex event to be detected as a particular spatial-temporal pattern of primitive events as well as any other logical predicates that constrain the definition of such an event. We illustrate how the uncertainty of a model can be propagated throughout the complex event detection pipeline. We demonstrate the efficacy and practicality of DEEPCEP++ in the context of human-machine teaming where human operators need to make complex decisions based on event streams from low-power autonomous sensors. Finally, we detail future directions of research with an emphasis on end-to-end learning of this hybrid structure. |