Sometimes we see customers on Linux wondering where their new NVMe capable SSD is on the Linux filesystem. It’s not in the standard place on the Linux filesystem in ‘/dev/sd*’ like all those scsi devices of the past 20+ years. So how come, where is it? For all of you new to the latest shipping Intel SSD’s for PCIe, they run on the NVMe storage controller protocol, and not the scsi protocol. That’s actually a big deal because that means efficiency and a protocol appropriate for “non-volatile memories” (NVM). Our newest P3700 and related drives will use the same, industry standard, and open source NVMe kernel driver. This driver drives I/O to the device and is part of the block driver subsystem of the linux kernel.
So maybe it is time to refresh on some not too familiar or oft-used linux administrative commands to see a bit more. The simple part is to look in “/dev/nvme*”. The devices will be numbered and the actual block device will have an n1 on the end, to support NVMe namespaces. So if you have one PCIe card or front-loading 2.5″ drive, you’ll have /dev/nvme0n1 as a block device to format, partition and use.
These important Data Center Linux distributions:
Red Hat 6.5/7.0
SUSE 11 SP2
Ubuntu 14.04 LTS
…all have in box nvme storage drivers, so you should be set if you are at these levels or newer.
Below are some basic Linux instructions and snapshots to give you a bit more depth. This is Red Hat/CentOS 6.5 distro relevant data below.
Are the drives in my system scan the pci and block devices:
[root@fm21vorc10 ~]$ lspci | grep 0953
04:00.0 Non-Volatile memory controller: Intel Corporation Device 0953 (rev 01)
05:00.0 Non-Volatile memory controller: Intel Corporation Device 0953 (rev 01)
48:00.0 Non-Volatile memory controller: Intel Corporation Device 0953 (rev 01)
49:00.0 Non-Volatile memory controller: Intel Corporation Device 0953 (rev 01)
[root@fm21vorc07 ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 372G 0 disk
├─sda1 8:1 0 10G 0 part /boot
├─sda2 8:2 0 128G 0 part [SWAP]
└─sda3 8:3 0 234G 0 part /
nvme0n1 259:0 0 372.6G 0 disk
└─nvme0n1p1 259:1 0 372.6G 0 part
Is the nvme driver built into my kernel:
[root@fm21vorc10 ~]$ modinfo nvme
author: Matthew Wilcox <email@example.com>
vermagic: 3.15.0-rc4 SMP mod_unload modversions
parm: io_timeout:timeout in seconds for I/O (byte)
Is my driver actually loaded into the kernel
[root@fm21vorc10 ~]$ lsmod | grep nvm
nvme 54197 0
Are my nvme block devices present:
[root@fm21vorc10 ~]$ ll /dev/nvme*n1
brw-rw—- 1 root disk 259, 0 Oct 8 21:05 /dev/nvme0n1
brw-rw—- 1 root disk 259, 1 Sep 25 17:08 /dev/nvme1n1
brw-rw—- 1 root disk 259, 2 Sep 25 17:08 /dev/nvme2n1
brw-rw—- 1 root disk 259, 3 Sep 25 17:08 /dev/nvme3n1
Run a quick test to see if you have a GB/s class SSD to have fun with.
[root@fm21vorc07 ~]# hdparm -tT –direct /dev/nvme0n1
Timing O_DIRECT cached reads: 3736 MB in 2.00 seconds = 1869.12 MB/sec
Timing O_DIRECT disk reads: 5542 MB in 3.00 seconds = 1847.30 MB/sec
Remember to consolidate and create parallelism as much as possible in your workloads.These drives will amaze you.