Apple's MR headset wants to open the "spatial computing" era, the strongest chip can run AI large models on a single device

Source: "Science and Technology Innovation Board Daily"

Editor: Qiu Siyu

"This is a revolutionary product," said Apple CEO Tim Cook. In the "One more thing" segment of WWDC23, after seven years of research and development, Apple's first MR headset, Vision Pro, which has been delayed for many times, finally made its finale debut. ** Cook said: "This marks the beginning of a journey."

The Vision Pro headset consists of a double-sided display, a battery, and a headset and mask. The headset ** adopts a dual-chip design, equipped with M2 and R1 chips**, the former provides computing performance, and the latter is a chip specially created for the headset, responsible for processing data from 12 cameras, 5 sensors, 6 The data signal of each microphone reduces the delay. **

In terms of display, the Vision Pro headset adopts a 3P Pancake+Micro OLED solution with a screen resolution exceeding 4K. **This head-mounted display is a relatively rare head-mounted display product in the industry that uses handle-free interaction. The supported interaction methods include eye movement interaction, voice interaction, and gesture interaction.

The Vision Pro is expected to be available in early 2024 at a price of $3,499, which is significantly higher than similar products from competitors. Taking Meta's Quest as an example, the initial price of the high-end Quest Pro is only $1499.99.

Explosive performance and high price, what is the positioning of Apple's Vision Pro?

From a functional point of view, Vision Pro covers at least four major usage scenarios, including movie viewing, gaming, office, home**, etc. In Apple's official demonstration video, users can watch "Star Wars" and other blockbuster special effects immersively, and can also watch multiple sports games at the same time. It is reported that Disney will provide streaming media services for it; in terms of office, Vision Pro can project a screen similar to a modern desktop to replace a computer monitor, and users can use keyboards and other devices to type.

In addition, during the entire press conference, Cook did not clearly state whether this product is called "head display" or "glasses", "VR" or "AR", but proposed "the first spatial computing system platform". The concept**, which is also Apple's positioning for Vision Pro.

“Just as the Mac brought us into the era of personal computing, and the iPhone brought us into the era of mobile computing, Vision Pro will bring us into the era of spatial computing.” Cook said.

In addition, Apple released VisionOS, its first Apple system based on spatial computing. The system is based on the innovations of Apple's macOS, iOS, and iPadOS systems, and supports 3D content engines.

Spatial computing refers to seamlessly mixing the digital world and the real world, so that the two worlds can perceive, understand and interact with each other, creating an unprecedented experience.

In addition to Apple, Qualcomm has repeatedly pointed out that spatial computing will be the focus of the next information age.

Qualcomm believes that the development of the XR industry and the realization of the metaverse vision cannot be separated from powerful connection technology and artificial intelligence technology. Based on the Snapdragon XR platform, Qualcomm is promoting the transformation of computing methods from two-dimensional display screens to spatial computing. Snapdragon Spaces provides developers with technologies and tools to jointly create an immersive XR experience.

Shen Jin, global vice president of Qualcomm, said in a recent speech that a new generation of mobile computing platforms with unlimited possibilities will connect the physical space and digital space of terminals, and accelerate the opening of a new era of spatial computing. **Spatial computing will be the future of computing. **

M2 Ultra: run AI large model on a single device

In addition to the MR headset, Apple also released new products such as the M2 Ultra chip and the 15-inch Macbook Air at WWDC23, and also updated the upgrades of the five major software platforms including iOS 17.

It's worth noting that the M2 Ultra is the last chip in Apple's M2 family, and Apple's most powerful chip to date.

The M2 Ultra uses the UltraFusion architecture, splicing two M2 Max chips together, with 134 billion transistors, 20 billion more than the previous generation M1 Ultra. In terms of manufacturing process, M2 Ultra adopts the second-generation 5nm process technology.

UltraFusion, Apple's custom packaging technology, is capable of delivering more than 2.5TB/s of low-latency inter-processor bandwidth by using a silicon interposer to connect chips with more than 10,000 signals.

In terms of performance, according to calculations, the M2 Ultra has a 24-core CPU, which is 20% faster** than the M1 Ultra; a maximum of 76-core GPU, which is 30% faster than the M1 Ultra; a 32-core neural network engine, Running 31.6 trillion times, 40% faster than M1 Ultra; the display engine supports up to 6 Pro display XDRs,** driving more than 100 million pixels. **

It is also worth noting that the M2 Ultra chip has a unified memory of up to 192GB and has a memory bandwidth of 800GB/s twice that of the M2 Max chip. Previously, due to insufficient memory, a single GPU could not handle large models.

Now, by integrating ultra-large memory bandwidth into a single SoC, Apple can finally realize that a single device can run huge machine learning workloads such as large Transformer models. **

At the same time, this also takes the performance of Mac computers to a new level. According to Apple, the Mac Studio** with the M2 Ultra is up to 6 times faster than the fastest Intel-based iMac;** the Mac Pro with the M2 Ultra is faster than the fastest Intel-powered Mac Pro. In various actual professional workflows such as video transcoding and 3D simulation, the maximum speed can be increased by up to 3 times. **