Deploying Edge AI

5-Week Master's Level Course

Edge AI Certification Program, University of South Alabama

📚 Course Overview

This repository contains materials for the "Deploying Edge AI" master's level course, which introduces students to the principles and practices of deploying advanced AI models at the edge. Over 5 intensive weeks, students will learn how to optimize, convert, and deploy popular generative AI, computer vision, and speech recognition models using ONNX and its CPUExecutionProvider.

The course emphasizes practical approaches to delivering AI capabilities on resource-constrained devices without specialized hardware, focusing on strategies that leverage standard CPU capabilities found in most computing environments.

🎯 Learning Objectives

By the end of this course, students will be able to:

• Convert modern AI models from various frameworks (PyTorch, TensorFlow, JAX) to ONNX format
• Apply advanced optimization techniques including quantization, pruning, and architecture modifications
• Deploy efficient generative AI models (text and image) on edge devices
• Implement real-time computer vision applications that run entirely on CPU
• Create speech recognition systems optimized for edge deployment
• Develop multimodal edge AI applications with responsive user interfaces
• Measure and optimize performance metrics for edge deployment
• Follow professional software development practices for edge AI deployment

📅 Course Structure

Week 1: Edge AI Fundamentals & ONNX Runtime

• Session 1: Introduction to Edge AI & ONNX Ecosystem
• Session 2: Converting & Preparing Models for Edge Deployment

Week 2: Model Optimization for Edge Deployment

• Session 1: Edge-Optimized Model Architecture
• Session 2: Advanced ONNX Optimization Techniques

Week 3: Deploying Generative AI at the Edge

• Session 1: Edge-Optimized Text Generation Models
• Session 2: Edge-Optimized Image Generation Models

Week 4: Computer Vision at the Edge

• Session 1: Optimizing Computer Vision Models
• Session 2: Real-time Computer Vision Applications

Week 5: Speech Processing & Multimodal Edge AI

• Session 1: Edge-Optimized Speech Models
• Session 2: Multimodal Integration & Final Projects

🚀 Individual Project

Throughout the 5-week course, each student will build a multimodal edge AI application that runs efficiently on standard CPU hardware. The project will progressively develop through weekly milestones:

The Edge AI Deployment Challenge:

• Create an AI application that runs efficiently on standard CPU hardware without specialized accelerators
• Demonstrate significant optimization through quantization, pruning, and architecture modifications
• Implement edge-optimized generative AI capabilities (text or image generation)
• Deploy advanced computer vision capabilities with real-time performance
• Enable efficient speech processing that works reliably
• Design resource-aware applications with responsive user experiences
• Document all processes thoroughly in a well-maintained GitHub repository

🔍 Sample Projects

Choose one of the following project options to implement throughout the course:

1. Edge AI Assistant: A desktop-based AI assistant that runs entirely on CPU without cloud dependencies
2. Portable Document Intelligence: A document analysis system that can run offline on a standard laptop
3. Edge Creative Studio: A content creation tool with on-device generative AI capabilities
4. Smart Monitoring System: A privacy-preserving monitoring system that performs all processing on-device
5. Field Research Assistant: An offline-capable field research tool for scientific data collection and analysis

📋 Repository Requirements

Your project repository should include:

• Project organization with dedicated directories for models, benchmarks, application code, notebooks, and documentation
• Detailed README with project overview, architecture diagram, and quickstart guide
• Comprehensive documentation of optimization techniques and performance benchmarks
• Clean commit history with descriptive messages and GitHub issues for tracking
• Clear installation and setup instructions with environment configuration files

📚 Prerequisites

Students should have:

• Master's level understanding of deep learning principles and neural network architectures
• Strong programming experience in Python
• Experience with at least one deep learning framework (PyTorch, TensorFlow, JAX)
• Familiarity with model training and evaluation
• Understanding of performance profiling and optimization concepts
• Previous exposure to computer vision, NLP, or speech processing fundamentals
• Experience with Python package management and virtual environments
• Basic Git/GitHub knowledge

🔧 Resources & Support

This repository provides access to:

• ONNX and ONNX Runtime documentation and examples
• Model conversion workflows from PyTorch and TensorFlow
• Quantization and optimization tools and examples
• Performance benchmarking utilities
• Jupyter notebooks with weekly starter templates
• Sample project starter code
• Environmental configuration files

📊 Assessment Criteria

• Weekly Milestone Achievements (30%)
• Technical Optimization Quality (25%)
• GitHub Repository Quality (15%)
• System Performance (10%)
• Final Demo & Benchmarks (15%)
• Documentation & Reproducibility (5%)

📞 Contact Information

Course Instructor: Dr. Ali Haidous
Email: ali.haidous@gmail.com
Office Hours: By Appointment

For technical assistance with course projects, implementation questions, or additional resources, please use GitHub Issues or contact the instructor via email.