Apple Intelligence, perhaps the highlight of this year's WWDC, is tightly integrated into iOS 18, iPadOS 18, and macOS Sequoia, and includes advanced generative models specialized for everyday tasks like typing, refining text, summarizing notifications, creating images, and automating interactions. with applications. .
The system includes an on-device language model of 3 billion parameters and a larger server-based model that runs on Apple silicon servers through Private Cloud Compute (PCC). Apple says these basic models, along with an encoding model for Xcode and a broadcast model for visual expression, support a wide range of user and developer needs.
The company also adheres to responsible AI principles, ensuring tools empower users, represent diverse communities, and protect privacy through on-device processing and secure PCC. Apple says its models are trained on publicly available, authoritative data, with filters to remove personal information and low-quality content. The company employs a hybrid data strategy, combining synthetic and human-annotated data, and uses novel algorithms for post-training improvements.
Human raters
For inference performance, Apple says it optimized its models with techniques like clustered query attention, low-bit palletization, and dynamic adapters. The on-device models use a vocabulary size of 49 KB, while the server models use 100 KB, supporting additional languages and technical tokens. According to Apple, the on-device model achieves a generation rate of 30 tokens per second, with additional improvements stemming from token speculation.
Adapters, which are small neural network modules, tune models for specific tasks, maintaining the parameters of the base model, and specializing in specific functions. These adapters are loaded dynamically, ensuring efficient memory usage and responsiveness.
Security and usability are paramount at Apple Intelligence, the Cupertino-based tech giant insists, and the company evaluates its models through human evaluation, focusing on real-world indications in several categories. The company claims that its device model outperforms larger competitors such as Phi-3-mini and Mistral-7B, while the server model rivals DBRX-Instruct and GPT-3.5-Turbo. This competitive advantage is highlighted by Apple's claim that human testers prefer its models to established rivals across several benchmarks, some of which can be seen below.
