Apple in early talks with PrismML over on-device AI shrinking tech

Apple in early talks with PrismML over on-device AI shrinking tech

Apple is in early discussions with PrismML, a startup specializing in the compression of large artificial intelligence models to enable them to run directly on smartphones. According to PrismML chief executive Babak Hassibi, Apple and other companies are currently evaluating the startup's technology, specifically measuring performance, energy efficiency, and speed on devices.

Hassibi told CNBC that the discussions are very early and it remains unclear where they will lead, though he noted that things are progressing nicely. Apple has not commented on the matter.

The timing of these talks coincides with the July 14, 2026, release of Bonsai 27B, a compressed build of Alibaba’s open-source Qwen model. PrismML used its technology to shrink the model from approximately 54GB to as little as 3.9GB. This release occurred one day after Apple opened the public beta of iOS 27, which includes a long-delayed overhaul of Siri.

Technical trade-offs and performance

PrismML, a Khosla Ventures-backed spinout from the California Institute of Technology, uses a method that reduces internal model values from 16 bits to either one or three possible values. This process allows the company to ship two versions under a free license:

• Ternary build: Designed for laptops, maintaining about 95% of full performance.
• 1-bit build: Approximately 3.9GB and designed to fit the memory budget of an iPhone 17 Pro, maintaining about 90% of full performance.

PrismML claims this compression cuts memory use by 10 to 15 times, speeds up responses by six to eight times, and lowers energy consumption by three to six times. However, Hassibi admitted that compressed models lose a few percentage points of performance. He noted that factual recall weakens first, followed by skills such as coding, mathematics, and reasoning.

Regarding raw speed, the 1-bit model reaches up to 163 tok/s on an NVIDIA GeForce RTX 5090 and up to 87 tok/s on an M5 Max. The Ternary version reaches 134 tok/s on the RTX 5090 and 58 tok/s on the M5 Max.

Hardware and economic implications

Fitting a model into a phone's memory is a strict requirement because devices do not expose full memory to apps. For instance, a 12 GB iPhone may only offer about 6 GB for a model, which must be shared with activations and its KV cache. At about 4 GB, the 1-bit Bonsai 27B is described as the first model of its size to pass through this memory gate with room to operate.

For Apple, increasing on-device AI processing would support its privacy pitch, reduce latency, allow features to work offline, and lower cloud costs. The move also carries financial weight; Morgan Stanley estimates that Apple's memory costs could rise sharply in the 2027 financial year, potentially leading the company to raise iPhone prices to protect margins. Smaller models would allow Apple to implement capable AI without paying for additional memory.

Industry analysts have expressed caution. Tarun Pathak of Counterpoint Research stated that the real test will be millions of queries across thousands of devices. Phil Solis of IDC identified power use as a primary open question, as frequent model operation could drain batteries.

The shift toward edge AI

The ability to move processing from the cloud to the device may impact the broader AI infrastructure. Gil Luria, an analyst at D.A. Davidson, suggested that while shrinking models would not remove the need for processors, it would shift them from data centers onto phones as part of a move toward edge AI.

PrismML, which raised a $16.25 million seed round in March, currently licenses patents owned by Caltech. Following the Qwen model, Hassibi said Google’s open-source Gemma model is next in the pipeline, followed by larger frontier models.