Sentinelle - Multilingual Truth Engine

A full-stack mobile application for verifying video content credibility using AI analysis with Firebase AI Logic.

Overview

Sentinelle is a real-time credibility assessment layer for short-form video content in Singapore's four official languages (English, Chinese, Malay, Tamil). It uses AI to automatically analyze video content and provide instant credibility insights.

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                    Mobile App (React Native)                        │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐              │
│  │   Feed View  │  │ StatusBadge  │  │ Badge Modal  │              │
│  │  (tap badge) │→ │ (circular UI)│→ │ (show results)│              │
│  └──────────────┘  └──────────────┘  └──────────────┘              │
└─────────────────────────────┬───────────────────────────────────────┘
                              │
                              ↓
┌─────────────────────────────────────────────────────────────────────┐
│              Firebase AI Logic (Gemini 2.5 Flash)                   │
│  ┌────────────────────────────────────────────────────────────┐    │
│  │ firebaseAnalysis.ts                                         │    │
│  │  1. Upload video to Firebase Storage                       │    │
│  │  2. Transcribe with Gemini (multimodal)                    │    │
│  │  3. Extract claims (structured JSON)                       │    │
│  │  4. Verify claims with web search grounding                │    │
│  │  5. Calculate credibility score                            │    │
│  └────────────────────────────────────────────────────────────┘    │
└─────────────────────────────┬───────────────────────────────────────┘
                              │
                              ↓
┌─────────────────────────────────────────────────────────────────────┐
│                    Firebase Services                                │
│  ┌──────────────────┐  ┌──────────────────┐  ┌─────────────────┐  │
│  │  Firestore DB    │  │ Firebase Storage │  │   Firebase AI   │  │
│  │  (Posts/Users)   │  │   (Videos)       │  │     Logic       │  │
│  └──────────────────┘  └──────────────────┘  └─────────────────┘  │
└─────────────────────────────────────────────────────────────────────┘

Tech Stack

Frontend

• React Native + Expo 54
• Redux Toolkit
• TypeScript
• Firebase SDK v13+

Backend

• Firebase Firestore (Database)
• Firebase Storage (Video Storage)
• Firebase AI Logic (Gemini 2.5 Flash)

AI

• Firebase AI Logic with Gemini 2.5 Flash
• Multimodal video analysis
• Structured JSON output for claim extraction
• Web search grounding for verification

Project Structure

hackomania-mobile/
├── src/
│   ├── components/
│   │   └── general/post/overlay/
│   │       └── StatusBadge.tsx            # Circular badge with built-in modals
│   ├── services/
│   │   ├── firebaseAILogic.ts            # Gemini initialization
│   │   ├── firebaseAnalysis.ts           # Video analysis with Gemini
│   │   ├── posts.ts                      # Firestore posts service
│   │   ├── chat.ts                       # Firestore chat service
│   │   ├── user.ts                       # Firestore user service
│   │   └── storage.ts                    # Firebase Storage uploads
│   ├── hooks/                            # Custom React Hooks
│   ├── redux/                            # Redux store & slices
│   └── data/                             # Data & mapping
├── types/                                # TypeScript types
├── assets/
│   ├── dummy/                            # Dummy video files
│   └── demo_videos/                      # Demo video documentation
├── firebaseConfig.ts                     # Firebase configuration
└── package.json

Features

1. Multilingual Support: English, Chinese, Malay, Tamil
2. Client-Side Analysis: Video analysis directly on device using Firebase AI Logic
3. StatusBadge Component:
   • Circular badge icons with distinct designs for each status
   • ✅ Verified (green): check-decagram icon
   • ⚠ AI-generated (yellow): robot icon
   • ❌ Fake (red): alert-octagon icon
   • Built-in modals for detailed information
   • AI Nutrition Label for AI-generated content
   • Fact check details for fake content
   • Verification details for verified content
4. Credibility Categories:
   • ✓ Verified (80-100)
   • ⚠ Unverified (60-79)
   • ⚠ Misleading (40-59)
   • ✕ Disinformation (0-39)
5. Singapore Context: Verification against MOH, SPF, MAS, MOE, CNA, ST
6. Real-time Updates: Firestore onSnapshot listeners for live data
7. Demo Mode: Anonymous user mode with DEMO_USER_ID

Setup

Prerequisites

• Firebase project with Blaze plan (for AI Logic)
• Gemini Developer API enabled
• Node.js 18+
• Expo CLI

Quick Start

1. Clone and Install

git clone https://github.com/codeandconquersg/hackathon.git
cd hackathon
npm install

2. Firebase Configuration

   The project is configured with Firebase project: codeandconquer-fb59c

   Firebase services used:

   • ✅ Firestore Database
   • ✅ Firebase Storage
   • ✅ Firebase AI Logic (Gemini Developer API)

3. Run the App

npx expo start

4. Scan QR Code with Expo Go app or run on emulator

Firebase AI Logic Usage

Video Analysis Flow

1. Upload: Video uploaded to Firebase Storage
2. Transcribe: Gemini 2.5 Flash transcribes video (multimodal)
3. Extract Claims: Structured JSON output for claims
4. Verify Claims: Web search grounding for verification
5. Calculate Score: Aggregated credibility score
6. Store Results: Save to Firestore

Example Code

import { analyzeVideo } from "./services/firebaseAnalysis";

const result = await analyzeVideo(videoUri, videoId, {
  onProgress: (progress) => {
    console.log(`${progress.step}: ${progress.progress}%`);
  },
  onComplete: (result) => {
    console.log("Credibility Score:", result.status_info.score);
  },
});

API Endpoints

All data operations use Firebase Firestore and Storage directly from the client.

Firestore Collections

• post - Video posts with analysis results
• user - User profiles
• chats - Chat messages
• comments - Post comments

Firebase Storage

• videos/ - Uploaded video files
• images/ - Uploaded image files

Credibility Categories

Category	Score Range	Status Type	Badge
verified	80-100	verified	✓
unverified	60-79	ai-generated	⚠
misleading	40-59	fake	⚠
disinformation	0-39	fake	✕

Supported Languages

• English (en)
• Chinese (zh)
• Malay (ms)
• Tamil (ta)

Cost Estimation

Service	Usage	Est. Cost
Gemini 2.5 Flash	100 videos analyzed	~$2-5
Firestore Free Tier	50K reads/day	$0
Firebase Storage	10GB stored	~$0.26
Total		~$2.26-5.26/month

Demo Videos

Demo videos are provided in assets/dummy/ directory:

• clip1.mp4 through clip5.mp4
• img1.jpg through img10.jpg

Development

Run Development Server

npx expo start

Type Checking

npx tsc --noEmit

Build for Production

npx expo export --platform web

Git Repository

• GitHub: https://github.com/codeandconquersg/hackathon

Contributing

Contributions are welcome! Please follow these steps:

1. Fork the repository
2. Create a feature branch
3. Commit your changes
4. Push to the branch
5. Create a Pull Request

License

MIT License

Team

Built for Singapore Hackathon 2024

---

Note: This project uses Firebase AI Logic (Gemini Developer API) for all AI-powered features. No backend server required.