RaahiSense

A Temporally Structured Historical Vehicle-Intelligence Pipeline for Evidence-Grounded Vehicle Behaviour Analysis

RaahiSense is an MTech Data Science research project focused on building a temporally structured vehicle-intelligence pipeline for retrieving and explaining relevant historical vehicle-behaviour evidence from heterogeneous, timestamped data.

The project investigates whether temporal structure, contextual information, and evidence ranking can improve the retrieval of relevant vehicle-behaviour evidence and the grounding of explanations generated by a resource-constrained local language model.

---

Research Objective

To evaluate whether temporal and contextual evidence retrieval improves the retrieval of relevant vehicle-behaviour evidence and the grounding of resource-constrained language-model explanations.

RaahiSense focuses on historical vehicle intelligence, rather than autonomous vehicle control.

The system processes heterogeneous timestamped data, detects events and contexts, establishes temporal relationships, retrieves relevant historical evidence, and prepares evidence packages for a local SLM.

---

Research Questions

1. How does temporal structure affect retrieval of relevant vehicle-behaviour evidence?
2. Can temporal and contextual ranking improve evidence retrieval compared with basic retrieval methods?
3. Does structured evidence improve the grounding and traceability of SLM-generated explanations?
4. Which historical vehicle-intelligence query classes benefit most from structured temporal retrieval?
5. What are the performance and resource trade-offs when deploying the pipeline on constrained edge hardware?

---

System Architecture

Vehicle Data Sources
        |
        v
Data Normalization
        |
        v
Event & Context Detection
        |
        v
Temporal Relationship Engine
        |
        v
Historical Retrieval
        |
        v
Evidence Package
        |
        v
Local SLM
        |
        v
Grounded Explanation

---

Core Database Model

The current database architecture is based on six core relational tables:

sessions
signals
observations
events
contexts
event_relationships

All supported data sources are normalized into this common model instead of creating separate source-specific database schemas.

---

Repository Structure

RaahiSense/
|
|-- database/
|   `-- schema.sql
|
|-- config/
|   |-- signal_dictionary.json
|   |-- event_rules.yaml
|   `-- scenario_schema.yaml
|
|-- generator/
|   `-- synthetic_generator.py
|
|-- benchmark/
|   `-- benchmark.json
|
|-- data/
|   |-- raw/
|   |-- generated/
|   `-- database/
|
|-- tests/
|-- scripts/
|-- docs/
|-- requirements.txt
|-- .gitignore
`-- README.md

---

Database

RaahiSense currently uses SQLite for the initial historical storage and retrieval layer.

The database schema is defined in database/schema.sql.

The schema uses:

• UUIDv7 identifiers stored as TEXT
• Unix epoch milliseconds stored as INTEGER
• JSON stored as TEXT
• SQLite foreign-key constraints
• Indexed temporal queries
• Explicit data-origin tracking
• Explicit data-quality information
• Ground-truth and detected-event separation

Data Origins

synthetic
simulation
replay
physical

Synthetic data is explicitly distinguished from real-world measurements.

---

Signal Dictionary

Canonical signals are defined in config/signal_dictionary.json.

Examples include:

vehicle_speed
vehicle_acceleration
brake_pressure
steering_angle
throttle_position
battery_soc
battery_temperature
pedestrian_detected
pedestrian_distance
vehicle_detected
vehicle_distance
collision_warning
lane_departure_warning
gps_latitude
gps_longitude

---

Synthetic Scenario Generator

Controlled synthetic scenarios are generated using generator/synthetic_generator.py.

The generator is intended to produce controlled datasets containing:

• Vehicle signals
• Road-user signals
• Environmental conditions
• Vehicle behaviour
• Sensor noise
• Missing data
• Timestamp jitter
• Source delays
• Conflicting evidence
• Hidden ground truth

Synthetic data is primarily used for controlled experimentation and evaluation, particularly where precise ground truth is difficult to obtain from real-world data.

---

Event Detection

Events are intended to be detected from observations rather than being treated simply as manually inserted analytical labels.

Examples include:

SHARP_BRAKING
PEDESTRIAN_DETECTED
VEHICLE_DETECTED
COLLISION_WARNING
LANE_DEPARTURE_WARNING
EMERGENCY_BRAKING

Detected events contain information such as event type, timestamps, severity, confidence, detection method, detection version, trigger observation, and data origin.

---

Temporal Relationships

Supported relationships include:

PRECEDES
FOLLOWS
OVERLAPS
CO_OCCURS
WITHIN_WINDOW
ASSOCIATED_WITH

The system intentionally does not use a generic CAUSES relationship. Temporal proximity or association must not automatically be interpreted as causation.

The database therefore includes causal_claim_allowed as an explicit guard against unsupported causal explanations.

---

Example Historical Query

Why did the vehicle brake sharply at 10:42 AM?

The system should retrieve evidence such as:

Pedestrian Detection
        |
        v
Pedestrian Distance
        |
        v
Vehicle Speed
        |
        v
Rapid Deceleration
        |
        v
Brake Activation
        |
        v
Post-event Vehicle Speed

The resulting explanation must describe the observed temporal sequence without making unsupported causal claims.

---

Evaluation

RaahiSense evaluates multiple retrieval strategies:

Baseline 1: Raw / Keyword Retrieval

        VS.

Baseline 2: Structured Event Retrieval

        VS.

Proposed: Temporal + Contextual Evidence Retrieval

Evaluation areas include:

• Event detection: Precision, Recall, F1-score
• Retrieval: relevant evidence, temporal relevance, context relevance, ranking quality
• Grounding: evidence coverage, traceability, unsupported claims, grounded explanation quality
• Edge performance: latency, memory, storage, inference cost

---

Robustness and Data Quality

Evaluation scenarios include:

Missing Data
Timestamp Jitter
Source Delay
Measurement Noise
Conflicting Evidence
Outliers
Interpolated Values
Invalid Values

---

Ground Truth Policy

Ground truth is kept separate from the evidence available to the retrieval system during evaluation. This prevents the retrieval system from directly accessing the answer it is expected to retrieve.

---

Real-World Data Integration

The architecture is designed to support real-world multimodal vehicle datasets in addition to controlled synthetic data.

Potential sources include:

Camera
LiDAR
Radar
GNSS / GPS
IMU
Vehicle Telemetry
Replay Data

A common normalization layer maps supported sources into the six-table RaahiSense data model.

---

Local SLM

After the retrieval pipeline is evaluated, a resource-constrained local language model will be integrated.

The SLM is not intended to independently discover the evidence. Instead:

User Query
    |
    v
Historical Retrieval
    |
    v
Temporal / Contextual Ranking
    |
    v
Evidence Package
    |
    v
Local SLM
    |
    v
Grounded Explanation

---

Edge Deployment

The final system is intended to be evaluated on resource-constrained hardware, with a Raspberry Pi-class device considered as a target platform.

The edge evaluation will investigate the trade-off between accuracy, retrieval quality, latency, memory, storage, model size, and computational cost.

RaahiSense does not attempt to build a complete autonomous-driving system.

---

Research Scope

Included

• Heterogeneous timestamped vehicle data
• Temporal data representation
• Event detection
• Context extraction
• Temporal relationships
• Historical evidence retrieval
• Temporal/contextual ranking
• Evidence grounding
• Local SLM integration
• Edge performance evaluation

Excluded

• Full autonomous driving
• End-to-end autonomous vehicle control
• Full local foundation-model training
• Replacement of an ADAS/autonomous-driving stack
• Unsupported causal inference from temporal correlation

---

Development Roadmap

[1] Freeze Database Schema       [DONE]
        |
[2] Signal Dictionary           [DONE]
        |
[3] Scenario Definition         [DONE]
        |
[4] Synthetic Generator
        |
[5] Event Detector
        |
[6] Context Detector
        |
[7] Temporal Relationship Engine
        |
[8] Synthetic Validation
        |
[9] Controlled Benchmark
        |
[10] Real Dataset Import
        |
[11] Baseline Retrieval
        |
[12] Temporal + Contextual Retrieval
        |
[13] Retrieval Evaluation
        |
[14] Evidence-Grounded SLM
        |
[15] Edge Deployment
        |
[16] Final Evaluation

---

Technology Stack

Programming & Data

Python
SQLite
Pandas
NumPy
PyYAML

Simulation & Data Sources

Synthetic Scenario Generator
CARLA or equivalent simulation environment
Real-world multimodal datasets
Replay data

Backend

FastAPI

AI / ML

Event Detection
Temporal Ranking
Contextual Ranking
Local / Quantized SLM

Edge

Raspberry Pi-class hardware

Development

Git
GitHub
Python Virtual Environment

---

Installation

Clone the repository:

git clone https://github.com/jenishborah/RaahiSense.git
cd RaahiSense

Create a virtual environment.

Windows

python -m venv .venv
.venv\\Scripts\\Activate.ps1

Linux / macOS

python3 -m venv .venv
source .venv/bin/activate

Install dependencies:

pip install -r requirements.txt

---

Initialize the Database

The SQLite database is generated from database/schema.sql.

Example:

python -c "import sqlite3; from pathlib import Path; db=Path('data/database/raahisense.db'); db.parent.mkdir(parents=True, exist_ok=True); con=sqlite3.connect(db); con.executescript(Path('database/schema.sql').read_text()); con.close(); print('RaahiSense database created successfully')"

The database file is intentionally excluded from Git. The schema is version-controlled so that the database can be reproduced locally.

---

Reproducibility

RaahiSense is designed around reproducible experiments.

Synthetic scenarios use explicit random seeds, while experimental configurations are stored separately from generated data.

Important configuration files include:

config/signal_dictionary.json
config/event_rules.yaml
config/scenario_schema.yaml

Experiments should record:

Scenario
Seed
Generator Version
Schema Version
Dataset Variant
Data Origin

---

Project Status

Completed

• GitHub repository initialized
• Frozen v1 relational schema
• SQLite database initialized
• Six core tables implemented
• Database indexes implemented
• Signal dictionary structure
• Event-rule configuration
• Scenario schema structure
• Initial synthetic generator

In Progress

• Frequency-aware synthetic generation
• Epoch-based timestamp handling
• Separation of generated observations and detected events
• Hidden ground-truth generation
• Event detector
• Context detector
• Temporal relationship engine
• Synthetic validation
• Benchmark dataset

Planned

• Real-world multimodal dataset ingestion
• Baseline retrieval
• Temporal/contextual retrieval
• Evidence ranking
• Retrieval evaluation
• Grounded local SLM
• Raspberry Pi evaluation

---

Research Principle

Retrieve and structure the evidence first; generate the explanation second.

The language model should explain evidence retrieved from the historical vehicle-intelligence pipeline rather than independently generating an explanation from the query alone.

---

Author

Jenish Ali Borah

MTech Data Science
Department of Computer Science
Tezpur University, Assam, India

---

Repository

GitHub:
https://github.com/jenishborah/RaahiSense