I built a MONSTER AI Pi with 8 Neural Processors!

Level 2 Jeff
5 Jun 202408:29

TLDRIn this video, the creator shares his ambitious project of building a Raspberry Pi AI PC with 8 neural processors, achieving 55 TOPS of computing power. He explains the challenges of connecting the processors, including a PCI Express issue, and how he overcame them. The creator also details the hardware setup, including the use of a rare Alftel PCI Express expansion board and various TPUs. After troubleshooting, he successfully gets the Coral TPUs and Halo NPU working, demonstrating the system's capabilities and potential despite software limitations.

Takeaways

  • 😲 The creator built an AI PC with Raspberry Pi using 8 neural processors, achieving a combined performance of 55 TOPS.
  • 🔌 They faced challenges with power supply and PCI Express connectivity, which were critical for the system's operation.
  • 🛠️ A Raspberry Pi 5 with Pineboard's UPCI-TY was utilized, featuring an additional 12-volt power supply for independent board powering.
  • 💾 The Alftel PCI Express 12-slot expansion board was a rare and crucial component, expanding the Pi's single PCI Express lane into 12.
  • 🚀 Two dual-edge TPUs were installed, but due to connectivity limitations, only one TPU per chip was operational.
  • 🌟 The system includes two Coral TPUs with a total of 16 TOPS and a Halo 8L plus a Halo 8, contributing 13 TOPS and 26 TOPS of Neural Compute respectively.
  • 🔌 A 12-volt power supply monitors the system's power consumption, which was approximately 8 watts at idle for the PCI Express board alone.
  • 🛠️ Initially, the setup encountered a 'Failed to enable MSI' error, which was later resolved by following a detailed guide for Coral TPUs on PiOS 12 Bookworm.
  • 📈 The creator successfully got the Coral TPUs working and also discovered the CodeProject.ai, a web GUI for AI tasks, enhancing the system's utility.
  • 🔧 Despite the impressive setup, the creator advises against such complexity for most users, recommending simpler configurations for practical use.

Q & A

  • What is the main challenge the creator faced when building the AI PC with Raspberry Pi?

    -The main challenge was a PCI Express issue that prevented the use of the pile of boards on the desk. The creator had to switch to using a Raspberry Pi 5 with Pineboard's UPCI-TY to resolve this.

  • How many total neural processor units (NPUs) does the creator's AI PC have?

    -The creator's AI PC has a total of 8 neural processor units (NPUs), including two dual-edge TPUs, two Coral TPUs, a Halo 8L, and a Halo 8.

  • What is the purpose of the 12-volt power supply in the AI PC setup?

    -The 12-volt power supply is used to power the board independently of the Raspberry Pi because the Pi's cable connector only supplies up to 5 watts, and the setup requires at least 7 to 10 watts.

  • What is the role of the PEX8619 switch in the Alftel PCI Express 12-slot expansion board?

    -The PEX8619 switch expands the one lane of PCI Express Gen 2 on the Raspberry Pi into 12 lanes of PCI Express Gen 2, allowing for multiple devices to be connected.

  • Why did the creator initially encounter a 'Failed to enable MSI' error message?

    -The 'Failed to enable MSI' error message was initially encountered due to a compatibility issue with the Halo software and the Raspberry Pi documentation guide.

  • How did the creator get the Coral TPUs working with PiOS 12 Bookworm?

    -The creator followed a detailed guide posted by Midnight Link on their Pi PCI Express repo, which involved building the driver from source and adding the Pineboards Hat AI overlay.

  • What is the total theoretical neural compute power of the AI PC if all NPUs were fully utilized?

    -The total theoretical neural compute power of the AI PC, if all NPUs were fully utilized, would be 63 TOPS (Tera Operations Per Second).

  • What is the current limitation of the AI PC in terms of software support?

    -The current limitation is that software support does not assume the presence of multiple NPUs, similar to the early days of multi-core processors where most software assumed only one core was available.

  • What is the maximum neural compute power the creator could achieve if they were to fill all available slots with Halo 8s?

    -If the creator were to fill all available slots with Halo 8s, they could potentially achieve a neural compute power of 200 to 300 TOPS.

  • What advice does the creator give regarding building a similar AI PC setup?

    -The creator advises against building a similar AI PC setup due to the complexity and current software limitations, recommending instead to stick with one or two devices for practical use.

Outlines

00:00

🤖 Building a 55 TOPS Pi AI PC

The speaker discusses an ambitious project to create a Raspberry Pi AI PC with a total of 55 TOPS of processing power, surpassing that of contemporary processors from AMD, Intel, Qualcomm, and Apple. This is achieved by using multiple chips and expansion slots, including a rare Alftel PCI Express 12-slot expansion board and various AI accelerators like Coral and Halo TPUs. The project also involves overcoming technical challenges such as power supply limitations and PCI Express issues. The speaker details the process of setting up the system, including installing the necessary software and drivers, and mentions the discovery of a helpful project called CodeProject.ai for AI tasks.

05:01

🔍 Testing and Expanding the AI PC

After successfully setting up the AI PC, the speaker tests the system with both Coral and Halo TPUs, noting that they function well despite initial errors. The speaker also explores the possibility of software support for multiple AI accelerators, drawing a parallel to the early days of multi-core processor support. They mention the potential for even greater processing power with additional hardware but acknowledge the current limitations due to software constraints and the single PCI Express Gen 2 lane connection to the Raspberry Pi. The speaker concludes by advising viewers to stick with simpler configurations for practical purposes but expresses their own enthusiasm for continuing such experimental projects.

Mindmap

Keywords

💡Raspberry Pi

Raspberry Pi is a series of small single-board computers developed in the UK by the Raspberry Pi Foundation. In the context of the video, it is used as the base platform for creating a high-performance AI PC. The script mentions a Raspberry Pi 5 with Pineboard's UPCI-TY, which is an expansion board that adds a PCI Express slot to the Raspberry Pi, allowing for additional hardware to be connected.

💡Neural Processors

Neural processors, or NPUs, are specialized hardware designed to accelerate machine learning tasks, particularly those involving neural networks. The video discusses integrating multiple neural processors, such as Coral TPUs and Halo chips, into the Raspberry Pi setup to achieve a high TOPS (Tera Operations Per Second) count, indicating the computational power for AI tasks.

💡TOPS

TOPS stands for Tera Operations Per Second and is a measure of the performance of AI processors. It indicates how many trillion operations a processor can perform in one second. The video creator built a system with a combined TOPS that exceeds that of major processors from companies like AMD, Intel, Qualcomm, and Apple, showcasing the potential of their custom AI PC.

💡PCI Express

PCI Express (Peripheral Component Interconnect Express) is a high-speed serial computer expansion bus standard used for computer hardware devices. In the video, the creator uses a PCI Express slot and card to expand the capabilities of the Raspberry Pi, allowing for the connection of multiple neural processors.

💡Alftel PCI Express 12-slot expansion board

This is a specific type of hardware mentioned in the video, used to expand the Raspberry Pi's capabilities by providing additional slots for PCI Express devices. The board has a PEX8619 switch that allows the expansion of the single PCI Express lane from the Raspberry Pi into 12 lanes, enabling the connection of multiple neural processors.

💡Coral TPUs

Coral TPUs are AI accelerators designed to run TensorFlow Lite models for on-device machine learning. The video describes using two Coral TPUs, each with one TPU, to contribute to the overall TOPS of the AI PC. The creator had to build drivers from source to get these working with the latest version of the Raspberry Pi OS.

💡Halo 8L and Halo 8

Halo 8L and Halo 8 are neural compute sticks that provide AI acceleration. The Halo 8L has 13 TOPS, and the Halo 8 has 26 TOPS of Neural Compute. These devices are plugged into the custom AI PC, contributing to its total TOPS and enabling it to perform complex AI tasks more efficiently.

💡3rd Reality's smart outlet

This is a Zigbee-enabled smart outlet used by the video creator to monitor the power consumption of their AI PC setup. It is wired up to Home Assistant, a home automation platform, allowing the creator to track the power usage of the PCI Express board and the entire system, ensuring it operates within safe power limits.

💡Home Assistant

Home Assistant is an open-source home automation platform that室内 the video creator uses to manage their smart home devices, including the 3rd Reality smart outlet. It provides a dashboard for monitoring and controlling various aspects of the home, such as power usage in this case.

💡CodeProject.ai

CodeProject.ai is a project mentioned in the video that provides a web GUI for AI tasks. It is an example of software that can take advantage of the AI capabilities of the custom-built PC. The video creator suggests that while it's a cool project, it currently only supports single Coral TPU usage, indicating the need for further development to fully utilize the potential of multiple AI accelerators.

Highlights

Creator built a 55-TOPS Pi AI PC with 8 Neural Processors, surpassing the latest AMD, Intel, Qualcomm, and Apple processors.

The AI PC has potential for expansion with 6 more slots available for additional chips.

Utilized Raspberry Pi 5 with Pineboard's UPCI-TY for a PCI Express slot and extra power supply.

Alftel PCI Express 12-slot expansion board was used, a rare and discontinued piece of hardware.

Broadcom PEX8619 switch expands one lane of PCI Express into 12 lanes, with some bandwidth limitations.

Two dual-edge TPUs were installed, but only one TPU per chip is functional due to PCI Express lane limitations.

Includes 16 TOPS of Coral TPUs and 13 TOPS from a Halo 8L, totaling 29 TOPS of neural compute power.

A Halo 8 with 26 TOPS of Neural Compute was also integrated into the system.

A 12-volt power supply monitors power usage through 3rd Reality's smart outlet integrated with Home Assistant.

Initial boot resulted in a 'Failed to enable MSI' error, similar to a previous issue.

Successfully got Coral TPUs working with PiOS 12 Bookworm following a detailed guide from Midnight Link.

Discovery of CodeProject.ai, a web GUI for AI tasks, during the setup process.

Coral TPUs and Halo were confirmed to be working after overcoming initial software hurdles.

YOLO model's tendency to identify rectangular objects as cell phones was observed during testing.

The system currently has 55 TOPS of available neural compute power, with 63 TOPS of hardware potential.

A potential software solution for utilizing all TPUs and NPUs together is compared to early multi-core processor support.

The creator advises against replicating this build due to software support limitations and recommends simpler setups.

The experiment, while fun and educational, is not recommended for practical use due to current software constraints.

Casual Browsing

How I BUILT A YouTube Channel Using Only AI

2024-09-12 07:04:00

I Tried PI AI: A New AI Assistant And THIS Happened

2024-09-16 03:19:00

AI on a Pi? Believe it!

2024-09-16 05:49:00

I Built a Directory of FREE AI Tools and Apps (No Code Required)

2024-09-12 22:53:00

Human Detection Alarm System with Raspberry Pi AI Kit + Maker Pi PICO

2024-09-16 07:03:00

I Tested the New Pi AI, and This Happened ...

2024-09-16 06:45:00