feat: remove all mock/demo data — ensure all pages load exclusively from database#56
Open
devin-ai-integration[bot] wants to merge 22 commits into
Open
feat: remove all mock/demo data — ensure all pages load exclusively from database#56devin-ai-integration[bot] wants to merge 22 commits into
devin-ai-integration[bot] wants to merge 22 commits into
Conversation
- Python DeepFace liveness engine (passive + active challenges, anti-spoofing) - Python document OCR engine (PaddleOCR, VLM classification, Docling parsing) - Go KYC orchestrator (NIN/BVN/CAC verification, AML screening, risk scoring) - Rust identity matching engine (embedding comparison, fraud detection) - TypeScript tRPC routers + comprehensive KYC/KYB frontend pages - KYC gate integration into Claims flow - API clients for all 4 backend services Co-Authored-By: Patrick Munis <pmunis@gmail.com>
Co-Authored-By: Patrick Munis <pmunis@gmail.com>
…e ThemeProvider) Co-Authored-By: Patrick Munis <pmunis@gmail.com>
- Revert vite.ts to use inline config spread (configFile: false) instead of configFile path - Revert vite.config.ts to remove define/dedupe/optimizeDeps additions that didn't fix React hooks issue - These reverts restore the original working configuration from previous PRs Co-Authored-By: Patrick Munis <pmunis@gmail.com>
…t plugin double-init) Co-Authored-By: Patrick Munis <pmunis@gmail.com>
…oral, PostgreSQL, Keycloak, Permify, Redis, Mojaloop, OpenSearch, OpenAppSec, APISix, TigerBeetle, Lakehouse Go orchestrator (8085): - PostgreSQL persistence replacing in-memory maps - Redis caching for KYC session lookups - Kafka producer for KYC completion events - Temporal client for workflow orchestration - OpenSearch auditor for compliance trail - APISix gateway with OpenAppSec WAF plugin - Mojaloop bridge for mobile money KYC-gated transfers - Keycloak/Permify authorization middleware - All 9 middleware clients wired into main.go Rust ledger service (8113): - TigerBeetle double-entry ledger with KYC-level transfer limits - Dapr sidecar for state management and pub/sub - OpenAppSec WAF validation on all requests - 10 ledger types with KYC level requirements Python services: - Lakehouse analytics (8114) with Delta Lake compliance reporting - Fluvio stream processor (8115) with WebSocket real-time events TypeScript platform integration: - KYC gate checks on claims.create, payments.process, wallet.topUp/withdraw - KYC gate on application.create/submit with level requirements - Onboarding wired to trigger KYC verification on identity step - KYB wired to Go orchestrator for CAC/TIN/director/UBO verification - Middleware integration endpoints (ledger stats, analytics metrics, stream topics, transfer limits, NDPR report) - New service clients: kycLedgerService, kycAnalyticsService, kycStreamService, checkKYCGate helper Co-Authored-By: Patrick Munis <pmunis@gmail.com>
- 6 PyTorch models: fraud detection (residual+attention), churn prediction (GLU), claims adjudication (multi-task), credit scoring (Wide&Deep), anomaly detection (VAE), GNN fraud ring detection (GraphSAGE) - Synthetic Nigerian insurance data generation (275k+ samples across 6 domains) - Real training loops with FocalLoss, OneCycleLR, early stopping, metric tracking - Trained .pt weight files for all 6 models - ONNX export for CPU-optimized inference (4 models) - Delta Lake feature store with versioning (6 tables) - MCMC Bayesian risk modeling with NumPyro/JAX (16 product lines, VaR/CVaR) - Ray distributed training infrastructure with local fallback - Neo4j graph schema for fraud ring detection with offline mode - FastAPI inference server for all models - All models run on CPU (no GPU required) Co-Authored-By: Patrick Munis <pmunis@gmail.com>
…sioning, scheduled retraining, platform data ingestion - drift_detector.py: PSI, KS test, JS divergence for data drift + performance monitoring - model_registry.py: Champion-challenger versioning with auto-promotion - data_ingestion.py: Platform data connectors with watermarking and fallback chain - pipeline.py: 5-step orchestration (ingest → drift → retrain → validate → promote → ONNX export) - scheduler.py: Cron-based + event-driven triggers with background thread - api.py: FastAPI endpoints for CT management (/ct/retrain, /ct/drift, /ct/models, /ct/scheduler) - Fixed api_server.py imports for standalone execution - All 4 models retrained, promoted, and exported to ONNX with zero errors Co-Authored-By: Patrick Munis <pmunis@gmail.com>
…g in CT API drift check Co-Authored-By: Patrick Munis <pmunis@gmail.com>
…eaming ingestion, online serving, lineage, RBAC, Feature Store API, Go SDK Components implemented: - Storage: Object store abstraction (Local/S3/MinIO) with unified interface - Schema: Registry with versioning, compatibility checks (backward/forward/full), evolution tracking - Streaming: Kafka/Fluvio ingestion engine with micro-batching, DLQ, checkpointing - Computation: Real-time feature engine with sliding windows, EMA, time-decay scoring - Serving: Online feature server with L1 (LRU) + L2 (Redis) + L3 (Delta Lake) caching - API: FastAPI REST API with DuckDB SQL queries, CRUD, materialization endpoints - Lineage: Full DAG tracking (source→table→model), quality metrics, mutation audit - RBAC: Role-based access control with table/column-level policies, audit logging - Connectors: Python EventBridge + Go SDK for microservice event publishing - All components tested with functional verification (9 features computed, 3 events delivered) Co-Authored-By: Patrick Munis <pmunis@gmail.com>
Co-Authored-By: Patrick Munis <pmunis@gmail.com>
…o, Python, TypeScript, Rust) Shared SDK libraries for all 12 infrastructure components: - PostgreSQL: connection pooling, migrations, JSONB, audit trail - TigerBeetle: KYC-level transfer limits, 6 ledger codes, batch transfers - Redis: session management, rate limiting, KYC gates, pub/sub, distributed locks - Mojaloop: mobile money interop, KYC-gated transfers, idempotency keys - Kafka: 16 platform topics, idempotent producer, DLQ support, audit events - APISix: rate limiting, OIDC, IP restriction, WAF, health checks - Keycloak: token validation, KYC level attributes, 5-min TTL caching - OpenAppSec: SQL injection, XSS, path traversal blocking - Permify: fine-grained RBAC, schema-based permissions, default-deny - OpenSearch: audit log indexing, ILM policies, structured search - Fluvio: real SDK integration, 11 platform topics, event streaming - Dapr: state management, pub/sub, service invocation Middleware layer (Go/Python/TypeScript): 1. Rate limiting (Redis) 2. Token validation (Keycloak) 3. KYC gate enforcement (Redis + Keycloak) 4. RBAC permission checks (Permify) 5. Async audit logging (OpenSearch + Kafka + Fluvio) All SDKs compile clean: - Go: go vet ./... passes - Python: py_compile all files pass - TypeScript: tsc --noEmit passes - Rust: cargo check passes Co-Authored-By: Patrick Munis <pmunis@gmail.com>
…ervability, gRPC, Docker consolidation (72→12 containers) Production Readiness Gaps Implemented (7 categories): 1. Circuit breakers with exponential backoff+jitter (Go/Python/TS/Rust) 2. Graceful shutdown with signal handling SIGTERM/SIGINT (Go/Python/TS/Rust) 3. Observability — Prometheus metrics export, request latency tracking (Go/Python/TS/Rust) 4. gRPC service registry with circuit breaker per-service (Go SDK) 5. Health/Ready/Live probe handlers for Kubernetes compatibility (Go/Python/TS/Rust) 6. Resilient HTTP clients with circuit breaker + retry (Go/Python/TS/Rust) 7. Request metrics middleware for all stacks Docker Container Consolidation (83% reduction): - 12 containers total (3 infra + 9 app) vs 72 theoretical - docker-compose.yml with health checks, resource limits, shared env - 8 Dockerfiles for consolidated service groups - 5 Go gateway binaries + 2 Python FastAPI gateways - PostgreSQL schema init script with all tables and indexes - All credentials via environment variables (no hardcoded secrets) Integration Tests: - 38 test cases covering health, critical flows, inter-service communication - Parameterized across all 9 service containers Co-Authored-By: Patrick Munis <pmunis@gmail.com>
Co-Authored-By: Patrick Munis <pmunis@gmail.com>
The React app fails to render due to 74 client-side files referencing process.env (e.g., process.env.NODE_ENV, process.env.DEMO_MODE) without a Vite define polyfill. This causes 'process is not defined' ReferenceError and renders a blank white page. Added define config to vite.config.ts to provide process.env to the browser. Co-Authored-By: Patrick Munis <pmunis@gmail.com>
- Search bar with Ctrl+K shortcut for instant filtering across 107+ nav items - Favorites section (star icon on hover, persisted to localStorage) - Recently Visited section (last 3 pages, auto-populated on navigation) - Collapsible group sections with chevron toggle and item count on hover - All state persisted to localStorage for session continuity Co-Authored-By: Patrick Munis <pmunis@gmail.com>
1. Rust SDK: Real Redis client replacing stub (was no-op returning None)
- Full redis crate integration with ConnectionManager
- Async cache_json/get_cached_json with actual Redis operations
2. Atomic rate limiting (all SDKs): Lua script replaces INCR+EXPIRE pipeline
- Eliminates race condition where key persists forever if process crashes
3. Safe distributed locks (all SDKs): Owner-verified Redlock pattern
- UUID owner ID stored in lock value
- Lua script checks ownership before DELETE (prevents stealing)
4. Circuit breaker on Redis (all SDKs): 5 failures → open → 30s → half-open
- Prevents cascading failures when Redis goes down
- Fail-open semantics (requests proceed without caching)
5. Cache invalidation with pub/sub (all SDKs):
- invalidatePattern() uses Lua SCAN+DEL+PUBLISH atomically
- publishInvalidation() notifies other services of entity changes
6. Distributed token invalidation (Keycloak in Go/Python/TS):
- Redis pub/sub channel '__token_invalidation__'
- User-level and token-level invalidation across all replicas
- Closes 5-minute stale window for revoked tokens
- Max cache size enforcement (10K entries with expired eviction)
7. HTTP cache headers (Go + TS middleware):
- Cache-Control rules for 10 endpoint patterns
- Public endpoints (products/rates): 5min with stale-while-revalidate
- Private endpoints (policies/claims): 30-60s
- ETag support via content hash
8. React Query configuration:
- staleTime: 30s (prevents re-fetch on every mount)
- gcTime: 5min (cache retained for navigation)
- refetchOnWindowFocus: false
- Retry with exponential backoff
9. Cache warming (all SDKs):
- warmCache() preloads commonly-accessed entries on startup
- Returns count of successfully loaded entries
10. Dapr caching layer:
- TTL-based caching via state store metadata
- Local in-memory L1 cache (1000 entries max)
- Cache-aside pattern helper
- Auto-invalidation on writes
Co-Authored-By: Patrick Munis <pmunis@gmail.com>
A) EPHEMERAL STATE:
- 5 Go services rewritten with Postgres-backed CRUD (core-services, financial,
communication, compliance, insurance-ops)
- Python liveness sessions moved from in-memory dict to Redis with fallback
- Schema auto-created on startup with seed data
B) MISPLACED FILES:
- Removed all .pyc files and __pycache__ directories
- Removed 100MB go1.22.5.tar.gz from repo
- Added comprehensive .gitignore for Python, Go, Rust, Node artifacts
C) HARDCODED METRICS:
- DEMO_MODE changed from 'true' to 'false' in vite.config.ts
- Dashboard pages now check explicit DEMO_MODE env var instead of NODE_ENV
- Go services return real DB-backed data with atomic request counters
D) MISSING BUILD FILES:
- Generated go.sum for all 6 Go services (5 microservices + lakehouse SDK)
- Added requirements.txt for ml-services and ai-platform Python services
E) WEAK ERROR HANDLING:
- Fixed all 19 silent catch{} blocks in tRPC routers.ts with console.error logging
- Replaced all unwrap() calls in Rust SDK (observability, mojaloop, circuit_breaker)
- Replaced all unwrap() in Rust KYC services (kyc-ledger, identity-matching)
F) HEALTH ENDPOINTS:
- All 5 Go services now check db.PingContext() with 3s timeout
- Return 503 with dependency status when Postgres unavailable
- Include uptime, request count, and dependency map in health response
Co-Authored-By: Patrick Munis <pmunis@gmail.com>
…, silent catches
Gap 1: Standardize DEMO_MODE on all 86 remaining pages to use
process.env.DEMO_MODE === 'true' (was 5 different broken patterns)
Gap 2: Implement real Drizzle ORM queries for 23 stub db.ts functions
that were returning empty arrays. Added 15 new schema tables for
domains that were missing (emergency, P2P, microinsurance, gig,
SME, savings, MCMC, family, claims evidence, WhatsApp, voice,
applications, feedback, dynamic pricing).
Gap 3: Add error logging to all 18 remaining silent catch {} blocks
across TypeScript SDK (middleware, redis, keycloak, kafka, permify,
platform, dapr), UnifiedLayout, api-clients, and dataApi.
Co-Authored-By: Patrick Munis <pmunis@gmail.com>
Fixes 9 pages that were missed in the original PR: - EmergencySOS: useState(true) toggle → process.env.DEMO_MODE - Microinsurance: useState(false) toggle → process.env.DEMO_MODE - OperationalReports: useState(false) toggle → process.env.DEMO_MODE - ClaimsAdjudicationEngine: !isAuthenticated → process.env.DEMO_MODE - AgentPerformance: fallback pattern → explicit DEMO_MODE check - AgentPortal: fallback pattern → explicit DEMO_MODE check - HealthWellness: fallback pattern → explicit DEMO_MODE check - P2PInsurance: fallback pattern → explicit DEMO_MODE check - ReconciliationEngine: fallback pattern → explicit DEMO_MODE check Total pages with correct pattern: 94 (was 85) Broken patterns remaining: 0 Co-Authored-By: Patrick Munis <pmunis@gmail.com>
…base - Remove DEMO_MODE constant and all conditional demo data from 94 pages - Remove all demoXxx data arrays/objects from pages (4,612 lines removed) - Convert all 225 stub functions in db.ts to real Drizzle ORM queries - Import 12 missing schema tables into db.ts - Replace all 'demo mode' toast messages with production-appropriate text - Rename demo-prefixed variables to production names - All data now loaded exclusively via tRPC -> db.ts -> Drizzle -> Postgres - Build verified: vite build passes (1910 modules, 5.15s) Co-Authored-By: Patrick Munis <pmunis@gmail.com>
Author
Original prompt from Patrick
|
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Author
Test Results — PR #567/8 tests passed | Devin Session Core Verification (all passed)
Escalation: 7 toast messages still say "DEMO MODE"
These are cosmetic — data loading is fully DB-backed. The toasts just have leftover text from the demo era. Constraint: No Postgres/auth backend available — verified via build + bundle inspection + browser rendering. Pages correctly show "Access Denied" when unauthenticated (previously showed demo data). |
- Remove 'DEMO MODE' text from 21 toast messages across 15 pages
- Remove dead code blocks ({false && ...}) that never rendered
- Remove demo-related code comments
- Build passes: 0 demo references in compiled bundle
Co-Authored-By: Patrick Munis <pmunis@gmail.com>
Author
Final Test Results — All 8/8 PASSED ✓Fixed the 7 remaining "DEMO MODE" toast messages + 14 additional occurrences found in deeper scan (21 total fixes across 21 files).
Previous escalation resolved: All 21 "DEMO MODE" toast messages removed across 21 pages (BatchProcessingEngine, Chatbot, ClaimsAdjudicationEngine, EmergencySOS, Gamification, GeospatialMap, LoyaltyProgram, Microinsurance, MultiCurrencySupport, NAICOMCompliance, OperationalReports, PolicyApproval, PolicyComparison, PolicyRenewalAutomation, PremiumCalculator, Referrals, Reviews, SMEBusiness, SystemSettings, AIKnowledgeAssistant, BlockchainStatus). Dead code blocks ( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
All 94 client pages now load data exclusively from the database via tRPC →
db.ts→ Drizzle ORM → Postgres. Zero mock, demo, or in-memory data remains.What was removed (4,612 lines deleted):
const DEMO_MODE = process.env.DEMO_MODE === 'true'from all 94 pagesDEMO_*/demo*hardcoded data arrays (DEMO_POLICIES, DEMO_CLAIMS, demoAgents, etc.)demoMode ? DEMO_X : realDataconditional renderingenabled: !demoMode && isAuthenticatedquery guardsWhat was converted in
server/db.ts(rewritten 70%):db.select().from(table).where(...))actuarialCalculations,agents,agentCommissions,analyticsEvents,auditTrail,loyaltyPoints,loyaltyTransactions,naicomFilings,notifications,pfaPartners,ussdSessions,bancassurancePartnersdb.select/insert/update/deletecalls across all functionsif (!db) return []fallbacks when Postgres unavailableVerification:
Link to Devin session: https://app.devin.ai/sessions/0475192a778b45cea30202f85ad52b63