Home / ubuntu / Questions / 1339175
Accepted
Sandbo
Asked: 2021-05-19 09:02:38 +0800 CST

NVMe SSDs performing much poorer on ubuntu 20.04.2 comparing to Windows

  • 772

On Windows, using AS-SSD, I found my Kingston A2000 to give read 2000 MB/s, write 1700 MB/s On ubuntu 20.04.2, using GNOME-DiskUtility benchmark, it gave read 2100 MB/s, but write is 430 MB/s

What am I missing? This is done actually in a Proxmox host, in the two guest OS, with the same SSD passed to them in each test.

ssd

1 Answers

  1. Best Answer

    If you truly want to test an NVMe drive you need a file 1/1000th of the drive size to flush the drive cache, otherwise you are just comparing cache speeds. So on a 1TB drive your test file size should be 1GB. Adjust as needed for your drive size. What program are you using to benchmark in Windows?

    I have a pair of 500gb PCIe 4.0 Samsung 980 Pro drives in Linux raid 0, with a 1GB file (1/1000th of the partition size) I'm getting 13.0 GB/s, right where I expect it to be. I get around 1.3 GB/s write speed, much lower than you would get on a Windows benchmark like Crystal Disk Mark. I am not certain why that is. However I have done some real world tests reading from a PCIe 3.0 1TB drive and writing to my raid, large files over 35GB and my raid is writing at the full read speed of the PCIe 3.0 drive, indicating the Disk Bench in Ubuntu is off and the real world performance is just fine.

    So here is what we know: Two completely different systems getting a similar read to write ratio using Gnome-Disk-Utility yet my independent testing indicates the Ubuntu test is somehow "flawed" or reporting some other sort of test for writes, like perhaps random writes.

    One thing you can do for a real real world test, if you have a second NVMe to read and write from is:

    fallocate -l 100G test.img

    This creates a fake 100GB (107.4GiB) file full of zeros very quickly, then time moving it back and fourth with a stopwatch. I get between 1.8 GB/s and 2.3 GB/s depending which way I am going. I have another pair of 4.0 drives on the way with intentions of putting my OS on a Linux Raid 5 NVMe array and when I get them, and the add in 4.0 card I am also waiting on I plan to set them up initially as a pair in Raid 0 and run this back and forth test as well as time some real files, like disk images. For ME benchmarking is neat, but I want to know the real world performance.

    I forgot to add to my comment. NVMe drives have fairly fast NAND flash but their real speed comes from fast cache on the drive, some tests give a false sense of speed by only filling the cache whereas a test like disk bench you've done in Ubuntu is writing 100 10MB files by default or 1GB total. On my Samsung drives the DDR4 SDRAM is 1 GB per 1TB drive size, so 512MB on my 500GB drives. Partitioned as a Raid 1 I have 1GB total cache. The first (1gb) file fills that, the next 99 are measuring the actual NAND speed. 100GB in less than a minute is impressive no matter how you slice it. On a read test it's just reading that 1GB file back directly from cache (in my opinion) while on a write test it has to actual put that file on the drive and therefore you are seeing the actual write speed of the drive. For this reason NVMe drives, while blazing fast, aren't truly what they are advertised as, again in my opinion, because rarely will you be reading files from cache, but small writes, less than the size of your cache will APPEAR to be super fast when in fact the drive will write that cache to NAND sooner or later.

    Edit: See below for Sandbo's suggestion of using KDiskMark which is available as a snap package

    sudo snap install kdiskmark

    KDiskMark works very well, much better than the built in disk bench.

    • 2