Semantic Mediation for Dynamic Fusion of Human Observations and Sensor Data

This paper addresses the problem of combining human observations and sensor data for entity tracking and identification in dynamic environments. The complexity of the track-and-detect task for realistic applications requires dynamic fusion of sensor data and observations, and a semantic mediation approach is adopted. Moving targets are detected and classified based on sensor data. Soft data in the form of short messages are automatically processed to identify relevant information, to be associated with entities detected by sensors. While sensor data provide rows of numerical features, observations convey finer descriptions of entities and contextual information that is intuitively included by human sources when reporting. A fusion system accommodates both sensor and soft input, and provides a unified framework for their effective integration. The system relies on semantic mediation to combine observations and sensor data and uses ontologies to create a bridge between two complementary representations of the same situation.