Here in Intel NVM and SSD group (NSG) we build and test Linux systems a lot and we’ve been working to mature the nvme driver stack on all kinds of operating systems. The Linux kernel is the innovation platform today, and it has come a long way now with NVMe stability. We always had a high level kernel build document but never in a blog (bad Intel, we are changing those ways). We also wanted to refresh it a bit as maturity is well along now with NVMe and Linux. Kernel 3.10 (*spring 2014) is when integration really happened, and the important data center Linux OS vendors are fully supporting the driver. In case you are on a 2.6 kernel and want to move up to a newer kernel, here are the steps to build a kernel for your testing platform and try out one of Intel’s Data Center SSD’s for PCIe and NVMe.. This assumes you want the latest and greatest for testing and are not interested in an older or vendor supported kernel. By the way, on those “6.5 distributions” you won’t be able to get a supported 3.x kernel, that’s one reason I wrote this blog. But it will run and allow you test with something newer. You may have your own reasons I am sure. As far as production goes you will probably want to make sure you work together with your OS vendor.
I run a 3.16.3 kernel on some of the popular 6.5 distros, you can too.
1. NVM Express background
The NVM express (NVMe) is optimized PCI Express SSD interface, NVM Express specification defines an optimized register interface, command set and feature set for PCI express (PCIe)-based Solid State Drives(SSD). Please refer towww.nvmexpress.org for background on NVMe.
The NVM Express Linux driver development utilizes the typical open-source process used by kernel.org. The development mailing list is email@example.com
The Linux NVMe driver intercepts kernel 3.10 and integrates to kernels above 3.10.
2. Development tools required (possible pre-requisites)
In order to clone, compile and build new kernel/driver, the following packages are needed
- build tools
- git (optional you could be using wget to get the Linux package)
You must be root to install these packages
apt-get install git-core build-essential libncurses5-dev
yum install git-core ncurses ncurses-develyum install groupinstall “Development Tools”
zipper install ncurses-devel git-core zipper install –type pattern Basis-Devel
3. Build new Linux kernel with NVMe driver
Pick up a starting distribution, it doesn’t matter from driver’s perspective which distribution you use since it is going to put a new kernel on top of it, so use whatever you are most comfortable with and/or has the tools required.Get kernel and driver
- Or you can download “snapshot” from the top commit (here’s an example)
tar –xvf linux-3.16.3.tar.xz
2. Build and install
Run menuconfig (which uses ncurses):
Confirm the NVMe Driver under Block is set to <M>
Device Drivers-> Block Devices -> NVM Express block device
This creates .config file in same directory.
Then, run as root these make commands (use the j flag as ½ your cores to improve make time)
make modules_install –j10
make install –j10
Depending on distribution you use, you may have to run update-initramfs and update-grub, but this is typically unnecessary. Once install is successful, reboot system to load new kernel and drivers. Usually the new kernel becomes default to boot which is the top line of menu.lst. Verify it with “uname –a” after booting, that the running kernel is what you expect. , Use “dmesg | grep –i error” and resolve any kernel loading issues.
4. NVMe Driver basic tests and tools
There are some basic open source nvme test programs you can use for checking nvme devices:
Git’ing source codes
git clone git://git.infradead.org/users/kbusch/nvme-user.git
Making testing programs
Add/modify Makefile with proper lib or header links and compile these programs
Example, check nvme device controller “identify”, “namespace” etc
>>sudo ./nvme_id_ctrl /dev/nvme0n1
>>sudo ./nvme_id_ns /dev/nvme0n1
Here are more commands you’ll find useful.
Zero out and condition a drive sequentially for performance testing:
dd if=/dev/zero of=/dev/nvme0n1 bs=2048k count=400000 oflag=direct
Quick test a drive, is it reading at over 2GB a second?
hdparm -tT –direct /dev/nvme0n1
Again enjoy these Gigabyte/s class SSD’s with low microsecond controller free performance!