File I/O stacks in operating systems are a great example for schools to teach Parkinson’s Law, which says that software expands to use the resources available. Last time I counted, the current Linux stack with an attached SSD has a total of six or seven address translations. That’s a huge software overhead, and you might be forgiven for wondering if the objective of the fastest I/O got lost in translation!
Fortunately, relief is at hand. The industry has created a new standard that enterprise SSD vendors are aggressively picking up: NVMe. The idea seems so simple that you wonder why we didn’t go there before, but IT is often very conservative, and this is a radical change.
NVMe stands for “Non-Volatile Memory express.” The hardware part of the solution takes away the disk interface chip that used to plug into PCIe on the motherboard, and this allows the drive to talk directly to the PCIe bus and so to system memory. This idea isn’t quite as radical as it sounds. Electrically, PCIe and SAS/SATA are very similar, so it’s more a protocol change than a completely new concept.