Vinsmoke2110/CoreML-Live-Vision-Showcase
Vinsmoke2110/CoreML-Live-Vision-Showcase has 56 GitHub stars and 0 forks. GitHot measured 0 stars in 24 hours. Last checked 2026-11-16 20:23 UTC. Last push 2026-11-13.
측정된 스타 증가
- in 24h
- 0
- in 7 days
- 업데이트 지연
- in 15 days
- 업데이트 지연
- in 1 month
- 업데이트 지연
서로 겹치는 기간의 순증가량이며 일별 합계가 아닙니다. 측정값이 없으면 비워 둡니다. 스타는 관심도를 나타내며 품질을 보장하지 않습니다.
소개
Build Real-Time CoreML Vision Apps: 2026 Swift Meetup Example Guide
머신러닝README.md
Original🌟 CoreVision Live - Real-Time Object Recognition for Apple Platforms
An innovative demonstration project showcasing the seamless integration of CoreML and Vision frameworks for both static image analysis and real-time video processing on Apple devices. Drawing inspiration from academic datasets and professional product photography, this repository serves as a comprehensive learning resource for developers eager to explore on-device machine learning capabilities.
📖 Table of Contents
- Project Overview
- The Story Behind CoreVision Live
- Why This Project Matters
- Feature Highlights
- Architecture Deep Dive
- Model Training Pipeline
- Real-Time Processing Engine
- Static Image Analysis Module
- User Interface Philosophy
- Multilingual Support System
- Performance Optimization Techniques
- Responsive Design Approach
- Continuous Availability Framework
- Getting Started Without Traditional Installation
- Project Structure Explained
- Contributing Guidelines
- Community and Support
- SEO and Discoverability
- License Information
- Disclaimer
🎯 Project Overview
CoreVision Live represents a thoughtfully crafted exploration into the world of on-device machine learning using Apple's powerful ecosystem. This repository demonstrates how developers can leverage CoreML and the Vision framework to create applications that understand visual information without requiring constant connectivity to external servers.
The project emerged from a desire to make machine learning accessible to iOS and macOS developers who want to integrate intelligent image recognition into their applications. Unlike cloud-based solutions that raise privacy concerns and introduce latency, CoreVision Live operates entirely on the user's device, ensuring both speed and confidentiality.
This implementation uses two distinct approaches to demonstrate the versatility of Apple's ML frameworks. The static image analysis component utilizes the Caltech-101 dataset, a well-known academic collection spanning 101 object categories, providing a robust foundation for understanding classification techniques. Meanwhile, the live video processing feature employs a curated collection of Apple product photographs, offering practical, real-world scenarios that resonate with developers and users alike.
The repository serves multiple purposes: it acts as an educational resource for those new to CoreML, provides reference implementations for experienced developers, and sparks inspiration for creative applications of machine learning technology on Apple platforms.
📚 The Story Behind CoreVision Live
The genesis of this project traces back to the vibrant developer community in Munich, where Swift enthusiasts gather regularly to share knowledge and explore emerging technologies. During one such meetup, the question arose: how can developers practically implement machine learning in their applications without becoming data scientists first?
This repository answers that question by providing a complete, working example that demonstrates the entire pipeline from model selection to user interface integration. The choice of Caltech-101 for static images was deliberate—it offers a standardized benchmark that allows developers to compare their results and understand classification accuracy in meaningful terms.
For the live processing component, the decision to use Apple product photography was both practical and symbolic. Practically, these images represent objects that users encounter daily and might want to recognize or categorize. Symbolically, they represent the ecosystem within which these technologies operate, creating a cohesive narrative about Apple's integrated approach to hardware and software.
The project has evolved through multiple iterations, incorporating feedback from workshop participants and adapting to updates in Apple's frameworks. Each version has refined the balance between simplicity for beginners and depth for advanced users.
💡 Why This Project Matters
In an era where artificial intelligence increasingly influences how we interact with technology, understanding the fundamentals of on-device machine learning becomes essential for developers. CoreVision Live addresses this need by providing a practical, hands-on introduction to concepts that might otherwise seem intimidating.
The privacy implications of on-device processing cannot be overstated. When image recognition happens locally, user data never leaves their device. This architectural choice aligns with Apple's broader commitment to privacy and provides developers with a template for building responsible AI applications.
Performance considerations also favor on-device processing. Network latency disappears, enabling real-time responses that feel instantaneous. The Vision framework's optimized implementations take full advantage of Apple's neural engine and GPU capabilities, delivering performance that cloud-based solutions struggle to match.
From an educational perspective, this repository fills a gap between theoretical machine learning courses and practical implementation guides. It bridges academic concepts with production-ready code, helping developers understand not just what works, but why it works.
✨ Feature Highlights
🖼️ Static Image Classification
The static analysis module processes photographs from the Caltech-101 dataset, identifying objects across 101 distinct categories with remarkable accuracy. This feature demonstrates the fundamental workflow of loading a model, preparing input data, and interpreting predictions.
🎥 Real-Time Video Recognition
The live processing engine captures frames from the device camera and performs classification at impressive speeds, enabling applications such as augmented reality overlays, accessibility tools, and interactive experiences.
🌍 Multilingual Interface
Recognition results and interface elements support multiple languages, making the application accessible to users worldwide. The localization system handles right-to-left languages gracefully and adapts to regional formatting preferences.
📱 Responsive Layout
The user interface adapts seamlessly across iPhone, iPad, and Mac form factors, taking advantage of larger screens on tablets and desktops while remaining functional on compact devices.
🔄 Continuous Operation Mode
The application maintains consistent functionality throughout extended usage sessions, with automatic recovery mechanisms ensuring reliable performance even under demanding conditions.
🎨 Custom Model Support
Developers can easily substitute their own CoreML models, extending the application's capabilities to specialized domains such as medical imaging, industrial inspection, or botanical identification.
⚡ Optimized Inference Pipeline
Careful attention to memory management, thread scheduling, and hardware acceleration ensures smooth performance without battery drain or thermal throttling.
🔒 Complete Privacy Protection
All processing occurs locally on the device, with no data transmission to external servers. Users can verify this through network monitoring tools, confirming the privacy-preserving architecture.
🏗️ Architecture Deep Dive
The project follows a modular architecture that separates concerns and facilitates testing. Understanding this structure helps developers adapt the code for their own applications.
The Model Layer encompasses the CoreML model files and their associated metadata. Models are loaded lazily to minimize startup time and memory footprint. Each model includes documentation about its training data, input requirements, and output format.
The Vision Layer wraps Apple's Vision framework, providing a simplified interface for common operations such as image classification, object detection, and feature extraction. This abstraction shields higher-level code from framework-specific details.
The Processing Layer manages the flow of data from input sources through the model to output consumers. It handles concerns such as frame dropping during high-load scenarios, result caching, and confidence threshold filtering.
The Presentation Layer constructs the user interface using SwiftUI, with careful attention to accessibility, internationalization, and responsive design. The declarative approach enables rapid iteration and clear separation from business logic.
The Utility Layer provides supporting functionality such as image preprocessing, result formatting, and logging. These utilities are designed for reuse across projects.
🧠 Model Training Pipeline
While this repository focuses on inference, understanding how models are created provides valuable context. The Caltech-101 dataset contains images across categories ranging from animals to vehicles to household objects.
Training involves presenting labeled examples to a neural network, which gradually learns to associate visual patterns with correct categories. Techniques such as data augmentation, transfer learning, and regularization improve generalization to unseen images.
The resulting model is converted to CoreML format using Apple's conversion tools, which optimize the network for efficient execution on Apple hardware. Quantization reduces model size while maintaining accuracy.
For the Apple product images used in live processing, a smaller, more specialized model was trained to recognize specific products with high confidence. This focused approach demonstrates how domain-specific models can outperform general-purpose alternatives within their niche.
⚡ Real-Time Processing Engine
The live processing feature represents the most technically challenging aspect of the project. Capturing video frames, running inference, and displaying results at 30 frames per second requires careful orchestration.
The capture pipeline uses AVFoundation to access the camera, with configuration options for resolution, frame rate, and focus mode. Frames are delivered to a processing queue where they undergo preprocessing before inference.
The inference step runs on a dedicated thread, preventing UI blocking and ensuring smooth animation. Results are passed back to the main thread for display, with confidence scores determining the visual presentation.
Temporal smoothing reduces flicker by considering previous frame results when classifying the current frame. This technique significantly improves perceived accuracy in video scenarios.
🖼️ Static Image Analysis Module
The static analysis feature provides a gentler introduction to the project, allowing developers to test models without camera hardware. Users can select images from their photo library or drag files onto the interface on macOS.
The analysis process begins with image preprocessing, including resizing, normalization, and format conversion. These steps ensure compatibility with model requirements and improve classification accuracy.
Multiple models can be evaluated for comparison purposes, with results displayed side by side. This feature helps developers understand how different architectures and training approaches affect performance.
Detailed result views show confidence scores for each category, not just the top prediction. This information proves valuable for applications that need to make nuanced decisions based on uncertainty.
🎨 User Interface Philosophy
The interface design prioritizes clarity and immediacy. Recognition results appear prominently, with visual indicators showing confidence levels. Color coding conveys certainty: green for high confidence, yellow for moderate, and red for uncertain predictions.
Accessibility considerations include VoiceOver support, dynamic type compatibility, and high contrast mode. The application remains fully functional for users with visual impairments.
Dark mode support extends to all screens, with carefully chosen colors that maintain readability and reduce eye strain during extended use. The interface respects system-wide appearance preferences.
Animations are purposeful rather than decorative, providing feedback about processing states and drawing attention to results. Subtle transitions maintain context as users navigate between screens.
🌍 Multilingual Support System
Internationalization goes beyond simple string translation. The application handles right-to-left languages, adjusts date and number formatting based on locale, and respects regional preferences for measurement units.
Currently supported languages include English, German, French, Spanish, Italian, Japanese, Korean, and Chinese. The translation files are structured to facilitate community contributions.
Category names from the Caltech-101 dataset are translated, ensuring that results make sense to non-English speakers. This localization extends to error messages and help content.
🚀 Performance Optimization Techniques
Achieving smooth performance required attention to multiple optimization areas. Model quantization reduces precision from 32-bit to 16-bit floating point, halving memory requirements with minimal accuracy loss.
Batch processing for static images allows the neural engine to process multiple images simultaneously, improving throughput for bulk analysis scenarios.
Memory pooling prevents allocation overhead during video processing, where frames are constantly created and discarded. Reusing buffers significantly reduces garbage collection pressure.
Thread priority adjustments ensure that inference tasks receive appropriate CPU time without starving UI updates. The quality of service settings balance responsiveness against efficiency.
📐 Responsive Design Approach
The layout system uses SwiftUI's adaptive capabilities to create interfaces that work well on any screen size. Grid layouts reflow based on available space, and text scales appropriately.
On iPad, the additional screen real estate enables side-by-side comparisons of model results, along with larger preview images. Multitasking support allows the application to share the screen with other apps.
Mac Catalyst brings the application to macOS with appropriate adaptations for mouse and keyboard input. Menu bar integration provides quick access to common commands.
🔄 Continuous Availability Framework
The application maintains consistent operation through various conditions that might otherwise cause failures. Model loading errors trigger fallback mechanisms that attempt alternative models or provide informative guidance.
Camera unavailability, whether due to permissions or hardware limitations, doesn't prevent static image analysis from functioning. The application gracefully degrades to available capabilities.
Network connectivity is never required, as all processing happens locally. This design decision ensures functionality in offline scenarios and eliminates dependency on external services.
🛠️ Getting Started Without Traditional Installation
This section describes how to begin working with CoreVision Live using modern development workflows. The approach emphasizes clarity and avoids assumptions about existing tooling.
Begin by obtaining the project source through your preferred version control interface. The repository structure follows standard conventions, with the Xcode project file at the root level.
Open the project in Xcode 15 or later, as earlier versions may not support all APIs used in the implementation. The project targets iOS 17 and macOS 14, taking advantage of recent framework improvements.
Select your target device or simulator from the scheme selector. Physical devices provide the best experience for testing live camera features, while simulators suffice for exploring static image analysis.
Build and run the application using the standard keyboard shortcut or menu command. The initial build may take several minutes as dependencies are resolved and models are processed.
📁 Project Structure Explained
Understanding the directory organization helps navigate the codebase efficiently. Each top-level folder contains related functionality with clear naming conventions.
The Models directory houses CoreML model files along with their metadata. Model cards describe training data, accuracy metrics, and intended use cases.
The Views directory contains SwiftUI view implementations, organized by feature area. Reusable components are separated into their own files for easy discovery.
The ViewModels directory implements the MVVM pattern, connecting views to underlying data and business logic. This separation facilitates testing and maintains clean architecture.
The Services directory provides abstractions over system frameworks, including Vision, AVFoundation, and CoreML. These services can be injected for testing purposes.
The Resources directory holds localization files, asset catalogs, and configuration files. This organization simplifies the process of adding new languages or customizing the application's appearance.
🤝 Contributing Guidelines
Contributions to CoreVision Live are welcomed and appreciated. Whether you're fixing bugs, adding features, improving documentation, or translating content, your efforts help the entire community.
Before beginning work on significant changes, please open an issue to discuss your plans. This conversation ensures alignment with project goals and prevents duplicate efforts.
Code contributions should follow Swift style conventions and include appropriate documentation. Unit tests for new functionality help maintain quality as the project evolves.
Pull requests should target the main branch and include a clear description of changes. Screenshots or recordings demonstrating new features are particularly helpful.
💬 Community and Support
The project maintains an active presence within the Apple developer community. Questions and discussions are welcome through the repository's issue tracker.
For developers in the Munich area, periodic meetups provide opportunities for in-person collaboration and knowledge sharing. These gatherings have proven valuable for building connections and exchanging ideas.
Documentation improvements are always appreciated. If you find a section unclear or discover missing information, consider submitting a pull request with enhancements.
🔍 SEO and Discoverability
This repository incorporates search engine optimization best practices to help developers find relevant resources. The README includes natural keyword integration without compromising readability.
Key terms such as "CoreML tutorial," "Vision framework example," "on-device machine learning," and "iOS image recognition" appear throughout the content in contextually appropriate ways.
The project structure and naming conventions follow conventions that search engines recognize as indicators of quality and relevance. Clear headings and descriptive content improve indexing.
📜 License Information
This project is released under the MIT License, a permissive license that allows use, modification, and distribution with minimal restrictions.
The full license text is available in the LICENSE file at the repository root. By using this software, you agree to the terms specified therein.
Copyright 2026 CoreVision Live Contributors
Permission is hereby granted, without charge, to any person obtaining a copy of this software and associated documentation files, to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
⚠️ Disclaimer
The information and code provided in this repository are for educational and demonstration purposes. While every effort has been made to ensure accuracy and reliability, the authors make no representations or warranties regarding the suitability of this software for any particular purpose.
Users are responsible for complying with all applicable laws and regulations when using this software. This includes, but is not limited to, privacy regulations such as GDPR and CCPA when processing personal data.
The Caltech-101 dataset is used in accordance with its original terms. Users who wish to use this dataset for their own projects should review and comply with the dataset's licensing requirements.
Apple product images used in demonstrations remain the property of Apple Inc. and are included solely for illustrative purposes. Their presence does not imply endorsement or affiliation.
Performance characteristics described in this document were measured under specific conditions and may vary based on hardware, software versions, and usage patterns. Developers should conduct their own testing to establish performance expectations for their target environments.
This project is not affiliated with, endorsed by, or sponsored by Apple Inc. CoreML, Vision, SwiftUI, and related technologies are trademarks of Apple Inc.