Ever since I rebuilt my homelab during the covid pandemic, I was never happy with the performance I was getting for the hardware that I had. Homelab 2.0 as I dubbed it has changed to something different than I originally designed, moving from VMware vcenter to Hyper-V 2022 being one of the biggest changes! Frustratingly, I have rebuilt this lab about 3 or 4 times now to get things working as well as it should.
The final issue I’m battling is dreadful storage write and read speeds in storage spaces on server 2022.
If you have been using storage spaces in your homelab, you probably had performance issues. Google it and you will see there’s tons of discussion and blogs about this topic online.
Like most things designed for enterprise, things are not always as simple as they seem. When it comes to storage spaces and getting the most out of your hardware, there are some best practices that need to be followed, the problem is they aren't very well documented by MS and as a result most people have had issues and had to figure it out by themselves. Thankfully, there are other people like me who blog their findings and share knowledge online for others, filling that documentation void.
There's a severe lack of documentation on a lot of MS storage spaces stuff. for the most part I resorted to going back to basics, doing what we do in vendor support, creating a lab vm and testing things out !
This was accompanied by a LOT of research, I would recommend that you check out the following resources, these guys do a way better job of explaining this than I could:
- https://datacenteroverlords.com/2018/12/17/microsoft-storage-spaces-is-hot-garbage-for-parity-storage/
- https://storagespaceswarstories.com/storage-spaces-and-slow-parity-performance/#more-63
- https://wasteofserver.com/storage-spaces-with-parity-very-slow-writes-solved/
- https://jackharvest.com/index.php/2022/03/10/storage-spaces-parity-on-server-2022-is-finally-good-enough-to-use/
The summary of it is you have to carefully match your interleave, column and allocation unit size:
Allocation Unit size = Parity calculation: ($number_of_columns - 1) * $interleave
Double parity Calculation: ($number_of_columns - 2) * $interleave
Simple Calculation: ($number_of_columns) * $interleave
My original plan was to use SSD caching in my server as I thought that would solve my issues. I bought some SSDs on prime day and set out to configure it. Originally, I thought that storagebus cache would be the best solution, I already run server 2022 and do not have clustering, should be simple right.
In the end I decided against this... the severe lack of documentation and no information about this online make it a dangerous prospect. My original plan was again to lab this feature out and see how it works, but I had so many issues getting this to work in a VM that I decided it was too risky to put data in this configuration without knowing how I would handle drive failures and general DR related scenarios. I am not sure if my failures were due to me trying to do this in a VM or not, but I saw other people online who blogged about it and did have it working in a VM. I was confused to say the least.
New and unproven technology with little info online is a risky data storage solution. Furthermore, when comparing it with the old storage tiers type setup for SSD caching there was no real information about any benefits to using storagebus cache over tiers. Plus, there’s a lot more information out there about storage tiers configuration with PowerShell, so I decided that would be the best option and got to work writing a script in the lab to configure this.
It took a while to get working but I started out by taking the knowledge obtained from nils schimmelmann (https://nils.schimmelmann.us/post/153541254987/intel-smart-response-technology-vs-windows-10) and Joe Freeman (https://github.com/freemansoft/win10-storage-spaces) then building on that to create my own scripts.
I have provided a script below that you can use to create a tiered storage space and adjust the allocation unit size, columns and interleave sizes to suit your setup.
The beauty of it is that you can create SSD (performance) tier and a HDD (Capacity) tier and set different Resiliency settings, Interleave and Column sizes to suit how many disks you have of each type.
For example in my script below i configure a volume that has 2 HDD's and 1 SSD, therefore i needed to set different Interleave and Column sizes to match my Allocation unit size of 64K on REFS.
The beauty of it is that you can create SSD (performance) tier and a HDD (Capacity) tier and set different Resiliency settings, Interleave and Column sizes to suit how many disks you have of each type.
For example in my script below i configure a volume that has 2 HDD's and 1 SSD, therefore i needed to set different Interleave and Column sizes to match my Allocation unit size of 64K on REFS.
(I also created some scripts to extend or remove a tired storage space, as well as check the max size space you can create based on the type of array you have.)
Drop this in a .ps1 file and watch it go:
#start
write-output "
=================================================================
Welcome to the EASYSTOREAGESPACEWITHTIERS script.
By Luke Varley - www.phishandchips.dev
This script will allow you to easily create a Tiered Storage Spaces
Virtual Disk
Read the comments in the script for instructions
#Created with knowledge obtained from
nils.schimmelmann
#Credit: https://nils.schimmelmann.us/post/153541254987/intel-smart-response-technology-vs-windows-10
Joe Freeman
#Credit: https://github.com/freemansoft/win10-storage-spaces
=================================================================
"
#check if elevation is required
if (-Not ([Security.Principal.WindowsPrincipal] [Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] 'Administrator')) {
if ([int](Get-CimInstance -Class Win32_OperatingSystem | Select-Object -ExpandProperty BuildNumber) -ge 6000) {
write-output "
=================================================================
SCRIPT REQUIRES ELEVATION, PLEASE RUN FROM AN ADMINISTRATIVE
POWERSHELL WINDOW. RUN AS ADMINISTRATOR.
=================================================================
"
Pause
Exit
}
}
Pause
#start
#Pool that will suck in all drives
$StoragePoolName = "VMStoreTiered"
#Tiers in the storage pool
$SSDTierName = "SSDTier-VMStoreTiered"
$HDDTierName = "HDDTier-VMStoreTiered"
#Virtual Disk Name made up of disks in both tiers
$TieredDiskName = "VMStoreTieredVD"
#Set Mirror, Simple, or Parity Resiliency type. Simple = striped. Mirror only works if both can mirror AFIK
#https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-R2-and-2012/dn387076(v=ws.11)
$SSDTierResiliency = "Simple"
$HDDTierResiliency = "Simple"
#$ntfs_aus = Parity calculation: ($number_of_columns - 1) * $interleave
# Double parity Calculation: ($number_of_columns - 2) * $interleave
# Simple Calculation: ($number_of_columns) * $interleave
$SSDInterleave = "65536" #in Bytes
$HDDInterleave = "32768" #in Bytes
$AllocationUnitSize = "65536" #in Bytes
#
$NumberOfHDDColumns = "2"
$NumberOfSSDColumns = "1"
#Change to suit - drive later and the label name
$TieredDriveLetter = "H"
$TieredDriveLabel = "VMStoreTiered"
#Override the default sizing here - useful if have two different size SSDs or HDDs - set to smallest of pair
#These must be Equal or smaller than the disk size available in that tier SSD and HDD
#SSD:cache - HDD:data
#set to null so copy/paste to command prompt doesn't have previous run values
$SSDTierSize = $null
$HDDTierSize = $null
#Drives cannot always be fully allocated - probably broken for drives < 10GB
$UsableSpace = 0.99
#Uncomment and put your HDD type here if it shows up as unspecified with "Get-PhysicalDisk -CanPool $True
# If your HDDs show up as Unspecified instead of HDD
#$UseUnspecifiedDriveIsHDD = "Yes"
#List all disks that can be pooled and output in table format (format-table)
Get-PhysicalDisk -CanPool $True | ft FriendlyName, OperationalStatus, Size, MediaType
#Store all physical disks that can be pooled into a variable, $PhysicalDisks
# This assumes you want all raw / unpartitioned disks to end up in your pool -
# Add a clause like the example with your drive name to stop that drive from being included
# Example " | Where FriendlyName -NE "ATA LITEONIT LCS-256"
if ($UseUnspecifiedDriveIsHDD -ne $null){
$DisksToChange = (Get-PhysicalDisk -CanPool $True | where MediaType -eq Unspecified)
Get-PhysicalDisk -CanPool $True | where MediaType -eq Unspecified | Set-PhysicalDisk -MediaType HDD
# show the type changed
Get-PhysicalDisk -CanPool $True | ft FriendlyName, OperationalStatus, Size, MediaType
}
$PhysicalDisks = (Get-PhysicalDisk -CanPool $True | Where MediaType -NE UnSpecified)
if ($PhysicalDisks -eq $null){
throw "Abort! No physical Disks available"
}
#Create a new Storage Pool using the disks in variable $PhysicalDisks with a name of My Storage Pool
$SubSysName = (Get-StorageSubSystem).FriendlyName
New-StoragePool -PhysicalDisks $PhysicalDisks -StorageSubSystemFriendlyName $SubSysName -FriendlyName $StoragePoolName
#View the disks in the Storage Pool just created
Get-StoragePool -FriendlyName $StoragePoolName | Get-PhysicalDisk | Select FriendlyName, MediaType
#Create two tiers in the Storage Pool created. One for SSD disks and one for HDD disks
$SSDTier = New-StorageTier -StoragePoolFriendlyName $StoragePoolName -FriendlyName $SSDTierName -MediaType SSD -NumberOfColumns $NumberOfSSDColumns -Interleave $SSDInterleave -ResiliencySettingName $SSDTierResiliency
$HDDTier = New-StorageTier -StoragePoolFriendlyName $StoragePoolName -FriendlyName $HDDTierName -MediaType HDD -NumberOfColumns $NumberOfHDDColumns -Interleave $HDDInterleave -ResiliencySettingName $HDDTierResiliency
#Calculate tier sizes within this storage pool
#Can override by setting sizes at top
if ($SSDTierSize -eq $null){
$SSDTierSize = (Get-StorageTierSupportedSize -FriendlyName $SSDTierName -ResiliencySettingName $SSDTierResiliency).TierSizeMax
$SSDTierSize = [int64]($SSDTierSize * $UsableSpace)
}
if ($HDDTierSize -eq $null){
$HDDTierSize = (Get-StorageTierSupportedSize -FriendlyName $HDDTierName -ResiliencySettingName $HDDTierResiliency).TierSizeMax
$HDDTierSize = [int64]($HDDTierSize * $UsableSpace)
}
Write-Output "TierSizes: ( $SSDTierSize , $HDDTierSize )"
# you can end up with different number of columns in SSD - Ex: With Simple 1SSD and 2HDD could end up with SSD-1Col, HDD-2Col
New-VirtualDisk -StoragePoolFriendlyName $StoragePoolName -FriendlyName $TieredDiskName -StorageTiers @($SSDTier, $HDDTier) -StorageTierSizes @($SSDTierSize, $HDDTierSize) -WriteCacheSize 0GB
# initialize the disk, format and mount as a single volume
Write-Output "preparing volume"
Get-VirtualDisk $TieredDiskName | Get-Disk | Initialize-Disk -PartitionStyle GPT
# This will be Partition 2. Storage pool metadata is in Partition 1
Get-VirtualDisk $TieredDiskName | Get-Disk | New-Partition -DriveLetter $TieredDriveLetter -UseMaximumSize
Initialize-Volume -DriveLetter $TieredDriveLetter -FileSystem REFS -AllocationUnitSize $AllocationUnitSize -Confirm:$false -NewFileSystemLabel $TieredDriveLabel
Get-Volume -DriveLetter $TieredDriveLetter
$TieredDriveLetterPath = "$TieredDriveLetter" + ":\"
Write-Output "Setting REFS File integrity on:"
$TieredDriveLetterPath
Set-FileIntegrity $TieredDriveLetterPath -Enable $True
Write-Output "Operation complete"
No comments:
Post a Comment