I built a MONSTER AI Pi with 8 Neural Processors!
TLDRIn this video, the creator shares his ambitious project of building a Raspberry Pi AI PC with 8 neural processors, achieving 55 TOPS of computing power. He explains the challenges of connecting the processors, including a PCI Express issue, and how he overcame them. The creator also details the hardware setup, including the use of a rare Alftel PCI Express expansion board and various TPUs. After troubleshooting, he successfully gets the Coral TPUs and Halo NPU working, demonstrating the system's capabilities and potential despite software limitations.
Takeaways
- ๐ฒ The creator built an AI PC with Raspberry Pi using 8 neural processors, achieving a combined performance of 55 TOPS.
- ๐ They faced challenges with power supply and PCI Express connectivity, which were critical for the system's operation.
- ๐ ๏ธ A Raspberry Pi 5 with Pineboard's UPCI-TY was utilized, featuring an additional 12-volt power supply for independent board powering.
- ๐พ The Alftel PCI Express 12-slot expansion board was a rare and crucial component, expanding the Pi's single PCI Express lane into 12.
- ๐ Two dual-edge TPUs were installed, but due to connectivity limitations, only one TPU per chip was operational.
- ๐ The system includes two Coral TPUs with a total of 16 TOPS and a Halo 8L plus a Halo 8, contributing 13 TOPS and 26 TOPS of Neural Compute respectively.
- ๐ A 12-volt power supply monitors the system's power consumption, which was approximately 8 watts at idle for the PCI Express board alone.
- ๐ ๏ธ Initially, the setup encountered a 'Failed to enable MSI' error, which was later resolved by following a detailed guide for Coral TPUs on PiOS 12 Bookworm.
- ๐ The creator successfully got the Coral TPUs working and also discovered the CodeProject.ai, a web GUI for AI tasks, enhancing the system's utility.
- ๐ง Despite the impressive setup, the creator advises against such complexity for most users, recommending simpler configurations for practical use.
Q & A
What is the main challenge the creator faced when building the AI PC with Raspberry Pi?
-The main challenge was a PCI Express issue that prevented the use of the pile of boards on the desk. The creator had to switch to using a Raspberry Pi 5 with Pineboard's UPCI-TY to resolve this.
How many total neural processor units (NPUs) does the creator's AI PC have?
-The creator's AI PC has a total of 8 neural processor units (NPUs), including two dual-edge TPUs, two Coral TPUs, a Halo 8L, and a Halo 8.
What is the purpose of the 12-volt power supply in the AI PC setup?
-The 12-volt power supply is used to power the board independently of the Raspberry Pi because the Pi's cable connector only supplies up to 5 watts, and the setup requires at least 7 to 10 watts.
What is the role of the PEX8619 switch in the Alftel PCI Express 12-slot expansion board?
-The PEX8619 switch expands the one lane of PCI Express Gen 2 on the Raspberry Pi into 12 lanes of PCI Express Gen 2, allowing for multiple devices to be connected.
Why did the creator initially encounter a 'Failed to enable MSI' error message?
-The 'Failed to enable MSI' error message was initially encountered due to a compatibility issue with the Halo software and the Raspberry Pi documentation guide.
How did the creator get the Coral TPUs working with PiOS 12 Bookworm?
-The creator followed a detailed guide posted by Midnight Link on their Pi PCI Express repo, which involved building the driver from source and adding the Pineboards Hat AI overlay.
What is the total theoretical neural compute power of the AI PC if all NPUs were fully utilized?
-The total theoretical neural compute power of the AI PC, if all NPUs were fully utilized, would be 63 TOPS (Tera Operations Per Second).
What is the current limitation of the AI PC in terms of software support?
-The current limitation is that software support does not assume the presence of multiple NPUs, similar to the early days of multi-core processors where most software assumed only one core was available.
What is the maximum neural compute power the creator could achieve if they were to fill all available slots with Halo 8s?
-If the creator were to fill all available slots with Halo 8s, they could potentially achieve a neural compute power of 200 to 300 TOPS.
What advice does the creator give regarding building a similar AI PC setup?
-The creator advises against building a similar AI PC setup due to the complexity and current software limitations, recommending instead to stick with one or two devices for practical use.
Outlines
๐ค Building a 55 TOPS Pi AI PC
The speaker discusses an ambitious project to create a Raspberry Pi AI PC with a total of 55 TOPS of processing power, surpassing that of contemporary processors from AMD, Intel, Qualcomm, and Apple. This is achieved by using multiple chips and expansion slots, including a rare Alftel PCI Express 12-slot expansion board and various AI accelerators like Coral and Halo TPUs. The project also involves overcoming technical challenges such as power supply limitations and PCI Express issues. The speaker details the process of setting up the system, including installing the necessary software and drivers, and mentions the discovery of a helpful project called CodeProject.ai for AI tasks.
๐ Testing and Expanding the AI PC
After successfully setting up the AI PC, the speaker tests the system with both Coral and Halo TPUs, noting that they function well despite initial errors. The speaker also explores the possibility of software support for multiple AI accelerators, drawing a parallel to the early days of multi-core processor support. They mention the potential for even greater processing power with additional hardware but acknowledge the current limitations due to software constraints and the single PCI Express Gen 2 lane connection to the Raspberry Pi. The speaker concludes by advising viewers to stick with simpler configurations for practical purposes but expresses their own enthusiasm for continuing such experimental projects.
Mindmap
Keywords
๐กRaspberry Pi
๐กNeural Processors
๐กTOPS
๐กPCI Express
๐กAlftel PCI Express 12-slot expansion board
๐กCoral TPUs
๐กHalo 8L and Halo 8
๐ก3rd Reality's smart outlet
๐กHome Assistant
๐กCodeProject.ai
Highlights
Creator built a 55-TOPS Pi AI PC with 8 Neural Processors, surpassing the latest AMD, Intel, Qualcomm, and Apple processors.
The AI PC has potential for expansion with 6 more slots available for additional chips.
Utilized Raspberry Pi 5 with Pineboard's UPCI-TY for a PCI Express slot and extra power supply.
Alftel PCI Express 12-slot expansion board was used, a rare and discontinued piece of hardware.
Broadcom PEX8619 switch expands one lane of PCI Express into 12 lanes, with some bandwidth limitations.
Two dual-edge TPUs were installed, but only one TPU per chip is functional due to PCI Express lane limitations.
Includes 16 TOPS of Coral TPUs and 13 TOPS from a Halo 8L, totaling 29 TOPS of neural compute power.
A Halo 8 with 26 TOPS of Neural Compute was also integrated into the system.
A 12-volt power supply monitors power usage through 3rd Reality's smart outlet integrated with Home Assistant.
Initial boot resulted in a 'Failed to enable MSI' error, similar to a previous issue.
Successfully got Coral TPUs working with PiOS 12 Bookworm following a detailed guide from Midnight Link.
Discovery of CodeProject.ai, a web GUI for AI tasks, during the setup process.
Coral TPUs and Halo were confirmed to be working after overcoming initial software hurdles.
YOLO model's tendency to identify rectangular objects as cell phones was observed during testing.
The system currently has 55 TOPS of available neural compute power, with 63 TOPS of hardware potential.
A potential software solution for utilizing all TPUs and NPUs together is compared to early multi-core processor support.
The creator advises against replicating this build due to software support limitations and recommends simpler setups.
The experiment, while fun and educational, is not recommended for practical use due to current software constraints.