EP-0180

Quad NVMe expansion board for Raspberry Pi 5

Description

The Raspberry Pi 5 Expansion Board is a versatile accessory designed to enhance the capabilities of the Raspberry Pi 5 through a PCIe interface. This board not only provides a power conversion solution but also serves as a high-speed storage expansion device. It is equipped with a 9V-20V Power Delivery (PD) input that converts the power supply to a stable 5.1V/5A output, ensuring reliable operation of the Raspberry Pi 5. The board requires a 9V-20V PD-compatible power source for activation and can power both the Raspberry Pi and attached storage devices, with the highest allowable voltage being 20V.

Features

• Power Delivery Support: It supports from 9V-20V PD and will automatically read the user's power supply PDO.
• Dual-Purpose Power Supply: Powers both the Raspberry Pi 5 and connected storage devices.
• Storage Expansion: Supports four M.2 NVMe SSDs for high-speed data storage and retrieval.
• Versatile Form Factor Compatibility: Accommodates M.2 SSDs in sizes 2230, 2242, 2260, and 2280.
• Application Flexibility: Ideal for use in Samba file systems or NAS setups, providing a robust disk expansion foundation.

Specifications

• Interface: PCIe
• Input Voltage: 9V- 20V Power Delivery (PD)
• Output Voltage & Current: 5.1V / 5A
• SSD Support: M.2 NVMe SSDs in 2230, 2242, 2260, and 2280 sizes
• Power Requirement: Powered by 9V-20V PD-compatible power source to initiate operation.
• Use Case: Suitable for Raspberry Pi 5 models, enhancing storage capabilities for applications such as Samba servers or Network Attached Storage (NAS).

Gallery

• Product outlook (frontal face)

• Product outlook (back face)

• Dimension

• Port definitions

• Compatibility

• The appearance from the front after assembly

• The appearance from the back after assembly

• Product details

• Application scenario

How to assmeble it?

• Please assemble it according to following figure:

Package Includes

• Please check following figure

How to enable PCIe on Raspberry Pi 5

• We assume that you are using Raspberry Pi OS (bookworm) 2024-03-15

Step 1

• Enable PCIe function

Edit /boot/firmware/config.txt file and adding following parameter in to the file. And the connection is certified for Gen 2.0 speed (5 GT/sec), but you can force it to Gen 3.0 (10 GT/sec) if you add the following line after: dtparam=pciex1_gen=3, but some SSD may not take effect, if not, please remove this line. The Raspberry Pi 5 is not certified for Gen 3.0 speeds. PCIe Gen 3.0 connections may be unstable.
Refer to: [ https://www.raspberrypi.com/documentation/computers/raspberry-pi.html#pcie-gen-3-0 ]
so that the dtparam=pciex1_gen=3 is optional.

NOTE: 
PCIe 4.0 (fourth-generation) solid-state drives (SSDs) are typically designed to be backward compatible, meaning they can operate on lower PCIe versions such as PCIe 2.0 or 3.0. This backward compatibility means that even if the Raspberry Pi 5's PCIe interface only supports up to Gen 2.0, a PCIe 4.0 SSD can still function on a Gen 2.0 interface, but its speed will be limited to the rates of Gen 2.0.

Specifically, the speed of a PCIe 4.0 SSD on a Gen 2.0 interface will be capped at about 5 GB/s (the theoretical maximum bandwidth of a PCIe 2.0 x1 interface), rather than the higher speeds achievable with a PCIe 4.0 x4 interface. Therefore, while PCIe 4.0 SSDs can be backward compatible with Gen 2.0, they cannot perform at their full potential on a Gen 2.0 interface.

dtparam=pciex1
dtparam=pciex1_gen=3

Save it and then do remember reboot your Raspberry Pi to take effect.

Step 2

• Check if the SSD drive has been recognized

sudo lspci 
sudo lsblk

Result be like:

Step 3

• Partitioning

Please divide the partition according to actual needs. Here I simply divided a partition and formatted it into ext4. Please refer to the configuration method related to the fdisk command to operate.

 
sudo fdisk /dev/nvme0n1
>d
>n 
>p
>1
> Enter > Enter 
> w
sudo partprobe /dev/nvme0n1

• Formating

sudo mkfs.ext4 /dev/nvme0n1p1

• Create mounting point and mount the partition to directory.

mkdir mynvme01
sudo mount -t ext4 /dev/nvme0n1p1 /home/pi/mynvme01 -v 

• Grant access permission

sudo chown -R pi:pi /home/pi/mynvme01 
sudo chmod -R 775 /home/pi/mynvme01

• Modify the /etc/fstab to enable automount function.

 
sudo nano /etc/fstab 

adding:

/dev/nvme0n1p1   /home/pi/mynvme01  ext4   defaults,noatime  0 0 

save it and execute:

sudo umount /dev/nvme0n1p1 
sudo mount -a
sudo systemctl daemon-reload
df -Th

 
NOTE: Same steps for the rest of NVMe SSDs.

How to setup Raid 0 on Raspberry Pi 5

• Prepare Hardware: Ensure you have multiple SD cards or USB flash drives, as well as a Raspberry Pi.

• Partitioning: Use the dd or fdisk command to clear the boot sector and partition table, then use fdisk or gdisk to create partitions of the same size,or just use all raw disk.

• Install mdadm: Install the RAID management tool by running:

sudo apt-get update && sudo apt-get install mdadm

• Create RAID Device: Use the mdadm command to create a RAID 0 device, for example:

sudo mdadm --create --verbose --level=0 --metadata=1.2 --raid-devices=4 /dev/md0 /dev/nvme{0..3}n1

• Format: Create a file system such as f2fs using:

sudo mkfs.ext4 /dev/md0

• Mount the RAID Device: Create a mount point and mount the RAID device, for example:

mkdir -pv /home/pi/myraid0
sudo mount -t ext4 /dev/md0 /home/pi/myraid0 

• Set Up Boot Mounting: Edit the /etc/fstab file to add the mount information for the RAID device and update the mdadm configuration.

sudo vim.tiny /etc/fstab

Adding following parameters into it.

/dev/md0    /home/pi/myraid0    ext4    defaults,noatime  0 0

Save it and quit.

• Update mdadm Configuration: After the RAID array is active, you can update the mdadm configuration file to save the current setup. Run the following command:

sudo mdadm --detail --scan | sudo tee -a /etc/mdadm/mdadm.conf

This command appends the configuration details of all assembled arrays to the mdadm.conf file.

• Update initramfs:After updating the mdadm.conf, you need to update the initial RAM filesystem to ensure that the RAID array is recognized and assembled at boot time:

sudo update-initramfs -u

• Reboot System: Finally, reboot your system to ensure that the RAID array is assembled automatically upon startup:

sudo reboot

• Verify RAID Array:After the system reboots, check that the RAID array is active and functioning as expected:

cat /proc/mdstat

This command will display the status of all RAID arrays. Remember to replace /dev/md0 and the device names with the actual names used in your configuration. It's also a good practice to back up the mdadm.conf file before making changes, so you can restore it if something goes wrong.

How to setup Raid 5 on Raspberry Pi 5

• Hardware Selection: Choose at least three storage devices, which can be USB hard drives or flash drives.

• Connect Storage Devices:Use at least 3 pcs of M.2 NVMe SSDs to N16 Quad NVMe expansion board to the Raspberry Pi.

• Install Operating System: Install the latest Raspberry Pi OS (bullseye 64bit) on the Raspberry Pi.

• Install mdadm: Same as for RAID 0 setup.

• Create RAID 5 Array: Use the mdadm to create a RAID 5 array, for example:

sudo mdadm --create /dev/md0 --level=5 --raid-devices=4 /dev/nvme{0..3}n1 
sudo mdadm --detail --scan | sudo tee -a /etc/mdadm/mdadm.conf 
sudo update-initramfs -u 

• Format and Mount: Format the RAID array to ext4 or another file system and mount it to the system.

sudo mkfs.ext4 /dev/md0

• Mount the Raid Device: Create a mount point and mount the Raid device.

mkdir -pv /home/pi/myraid5 
sudo mount -t ext4 /dev/md0 /home/pi/myraid5 
df -Th

• Set Up Boot Mounting:Edit the /etc/fstab file to add the mount information for the RAID device and update the mdadm configuration.

adding following parameters:

/dev/md0   /home/pi/myraid5  ext4   defaults,noatime 0 0

save it and execute following command:

sudo systemctl daemon-reload

• Waiting for rebuild stripping information:

sudo mdadm --detail /dev/md0

To check if a RAID 5 array has been synchronized, you can use the following methods:

• Check /proc/mdstat: This file provides information about the status of all active RAID devices. You can view the current status of your RAID devices by running the command:

cat /proc/mdstat

• Look for the RAID device in question and check for terms like "active", "resync", or "clean". If the status is "clean" and the device shows as "active", it typically means the RAID array is synchronized and operating normally.

Use mdadm Command: The mdadm utility is a tool for managing and monitoring RAID devices in Linux. To get detailed information about a RAID device, use:

mdadm --detail /dev/md0

or Check RAID Status with mdadm -D: This command provides a detailed description of the RAID device's status, including whether it is actively syncing or if the sync is complete:

mdadm -D /dev/md0

Look for the "State" line in the output. A "clean" state with no pending writes indicates synchronization is complete.

• Check for Resync Completion:If a RAID 5 array was resyncing, you can check if the resync has completed by looking for the absence of "resync" in the cat /proc/mdstat output or by using the mdadm --detail command. When the resync is complete, you should no longer see any mention of resync in progress.

• Check Array Events: You can also check for any events related to the RAID array by using:

mdadm --monitor --scan

This command will show the events related to the RAID devices, which can help you determine if there have been any recent issues or if the array is stable.

• Review System Logs:Sometimes, the system logs can provide information about the RAID array's synchronization status. You can use the dmesg command or check the logs in /var/log/ for any RAID-related messages.

• Configure Network Sharing:Install and configure network sharing services such as Samba or NFS to access the RAID storage over the network.

Please note that RAID 0 offers data striping to improve read and write speeds but has no redundancy; if one drive fails, all data is lost. RAID 5 provides data striping and parity, offering better redundancy, but write performance may be slightly lower. When building a RAID system, also consider factors such as power supply, cooling solutions, and network performance.

YOUTUBE VIDEOS

• Official Channel: [ https://www.youtube.com/watch?v=PgItO0vrNEc ]
• Leepsvideo Channel: [ https://www.youtube.com/watch?v=qXbtKbD8ZdY ]

NOTE

---

If a RAID 5 array has not completed synchronization, it is generally possible to write data to it, but there are several considerations and potential issues to be aware of:

• 1. Performance Impact: Writing to a RAID 5 array that is still in the process of resynchronization can significantly impact the performance of the array. The system may be slower to write data because it has to update both the data and the parity information across multiple disks.

• 2. Increased Wear on SSDs: If your RAID 5 array includes SSDs, the ongoing resynchronization process can contribute to increased write amplification and thus wear on the SSDs.

• 3. Risk of Data Loss: If the RAID 5 array is in the middle of a resync due to a previous disk failure or other issues, writing new data could potentially increase the risk of data loss in the event of another disk failure during the resync process. This is because the array is in a transitional state and may not have full redundancy.

• 4. Data Corruption: In rare cases, if there is a bug or hardware issue during the resynchronization process, there is a small risk that the new data written could become corrupted or could inadvertently corrupt existing data due to incorrect parity calculations.

• 5. System Stability: Writing to the array during resync might also affect the stability of the system, especially if the resync process is already consuming a significant amount of system resources.

• 6. Interruption of Resync: Some RAID controllers or software may pause the resynchronization process when new writes are detected, to avoid conflicts between the old and new data. This could effectively prolong the resync time.

• 7. Priority of Resync: On some systems, the RAID controller may prioritize the completion of the resync process over new write operations, which could delay the writing of new data until the resync is complete.

In summary, while it is usually possible to write to a RAID 5 array that has not finished resynchronizing, it is generally recommended to wait until the resync is complete to ensure optimal performance, minimize wear on SSDs, and reduce the risk of data loss or corruption. If it is necessary to write to the array during resync, it is important to monitor the system closely and ensure that there are robust backup procedures in place.

---

Keywords

• Quad NVMe SSD expansion board for Raspberry Pi 5, nas cluster expansion board for rpi5, pcie M.2 NVME SSD exppansion board, m.2 nvme ssd adapter board for Raspberry Pi 5.