Deutsch
Español
Français
Italiano
Nederlands
Polski
Português
Русский
Türkçe
Svenska
Chinese
MagyarApple hints at on-device AI with an open-source language model
Last week, researchers at Apple introduced OpenELM, a series of “open-source efficient language models”, on the Hugging Face model library. The four variants range in size from 270 million parameters to 3 billion, and are the most likely candidates for on-device AI for Apple devices.
For context, Apple quietly launched a machine learning framework called MLX in December 2023. Up next was MLLM-Guided Image Editing (MGIE), followed by a succession of generative AI efforts including Keyframer, Ferret-UI, and AI code-completion in Xcode. For the most part, these projects harness the processing power of Apple silicon rather than offload the AI functionality to the cloud.
In the same vein, OpenELM represents Apple's on-device approach towards AI. Typically public LLMs utilize hundreds of billions (sometimes trillions) of variables to comprehend user input and decide on a suitable response. On the other hand, smaller language models such as Microsoft’s Phi-3 use as few as 3.8 billion parameters while Google Gemma boasts of 2 billion. However due to OpenELM’s unique approach to the architecture of the transformer model, the model bottoms out at merely 270 million parameters.
Obviously, there are some downsides to being small. For one, OpenELM is not multimodal, having too few parameters for that to be feasible. Also, its factual knowledge is quite low as demonstrated via the technical report. This problem besets all similarly-sized public LLMs. Yet, the small size allows the AI model to be hosted locally on phones or laptops instead of over the cloud.
Apple's public release of OpenELM is a departure from the company's typical practices. From the complete framework and evaluation of the model, to the training logs, pretraining configurations and the MLX inference code, every aspect of the language model is publicly available via Hugging Face for developers to tweak and repurpose for different use cases. Ostensibly, such an extensive release should strengthen Apple's stake in AI by inspiring researchers to toy with the possibilities on Apple devices.
But there are other players in this space. Microsoft's Phi-3 is a very competent rival, as are the other open LLM projects from Redmond. Google’s 2B - 3B Gemma is another. While all of the aforementioned models still run too slowly, the hardware and software are certainly moving in the right directions for small language models.
For now, edge devices such as Samsung Galaxy S24 (starting at $799 on Amazon), or the OnePlus 12R using the in-house Andes-GPT model, have to rely on cloud processing. Whether or not Apple incorporates OpenELM into the next iPhone, it is likely that the Cupertino-based company will collaborate with either Google or Open AI for heavier generative AI functions.
Source(s)
