SATA, SAS, NVMe: Your SSD Is Only as Fast as the Architecture Around It

Buying the fastest drive doesn’t mean you’re building the fastest storage system.

Storage should be simple.

You have data.

You put it on a drive.

You read it back.

Unfortunately, somewhere along the way we accumulated enough acronyms to turn buying a storage drive into an architecture discussion.

SCSI.

SATA.

SAS.

NVMe.

PCIe.

AHCI.

U.2.

U.3.

M.2.

Then somebody brings up iSCSI, Fibre Channel and Ethernet, and suddenly we’re talking about an entirely different part of the storage stack.

I’ve worked with enough customers over the years to know how easily these technologies get mixed together.

One of the most common examples is the assumption that a SAS SSD and an NVMe SSD are basically two versions of the same thing.

They’re not.

They may both contain NAND flash.

They may both fit into something that looks like a 2.5-inch drive bay.

They may even plug into the same universal server backplane.

But underneath, they belong to very different architectures.

And understanding that difference matters because one of the easiest mistakes in storage architecture is buying the fastest component available and then putting it behind something that prevents you from using the performance you paid for.

So let’s go back to the beginning.

Before NVMe, There Was SCSI

If you’ve worked in enterprise infrastructure long enough, SCSI needs little introduction.

Small Computer System Interface became one of the foundational technologies of enterprise storage.

Traditional parallel SCSI connected disks and other peripherals through a shared bus. Multiple devices could exist on that bus, identified through SCSI IDs, with a controller communicating with them using the SCSI command set.

It was reliable.

It was mature.

It became deeply embedded in enterprise computing.

And importantly, SCSI wasn’t just about a connector.

It also established a command architecture for communicating with storage devices.

That distinction will become important later.

As drive performance increased, however, parallel buses became increasingly difficult to scale.

Running many electrical signals side by side at higher frequencies introduces challenges involving timing, signal integrity and cabling.

The industry gradually moved toward serial interfaces.

And that’s where two familiar technologies enter the story.

SATA and SAS.

SATA: Cheap, Simple and Everywhere

SATA stands for Serial ATA.

It evolved from the ATA world of desktop and commodity storage and became extraordinarily successful.

If you’ve built a desktop computer at any point during the last twenty years, you’ve probably used SATA.

SATA III tops out at a signaling rate of 6 Gb/s.

After encoding and protocol overhead, practical sequential SSD performance usually tops out somewhere around the mid-500 MB/s range.

That is why so many SATA SSD specifications look suspiciously similar.

550 MB/s.

560 MB/s.

Maybe a little more.

The NAND inside the SSD may be capable of moving data considerably faster.

The interface simply cannot.

This created an interesting problem as flash replaced spinning disks.

With hard drives, SATA wasn’t usually the limiting factor.

The mechanical disk was.

A spinning platter and moving actuator could only deliver data so quickly.

Then we removed the moving parts.

Flash got faster.

And suddenly the road connecting the drive to the system became narrower than the drive itself.

The storage device had caught up with its interface.

SAS: SCSI Goes Serial

Enterprise storage took a different path.

Serial Attached SCSI—or SAS—essentially brought the SCSI command ecosystem into the serial era.

SAS offered features enterprise environments cared about: dual-port connectivity, sophisticated expanders, multipath architectures, large device topologies and a mature enterprise command set.

SAS generations continued increasing bandwidth.

12G SAS became commonplace in enterprise servers and storage systems.

Now we have 24G SAS, technically signaling at 22.5 Gb/s per lane in current implementations.

Broadcom’s current 9600-series storage controllers, for example, support 22.5, 12 and 6 Gb/s SAS alongside SATA and NVMe connectivity.

And this is where people sometimes make the first mistake:

SAS SSD does not mean slow SSD.

Modern enterprise SAS SSDs can be extremely capable devices.

They also exist in an ecosystem built around enterprise availability and predictable storage behavior.

But SAS still carries architectural history originally designed around storage devices very different from modern flash.

And eventually the industry asked an obvious question:

Why are we making incredibly fast solid-state storage pretend to be a disk?

Enter NVMe

NVMe stands for Non-Volatile Memory Express.

The important part isn’t merely that NVMe SSDs are faster.

It’s why they’re faster.

NVMe was designed specifically for non-volatile memory and operates over PCI Express rather than routing storage through the traditional SATA/SAS architecture.

That dramatically changes the path between storage and CPU.

Instead of treating incredibly fast NAND flash like another descendant of a spinning disk, NVMe was designed around the parallelism and low latency of solid-state storage.

Modern NVMe supports enormous numbers of queues and commands in flight.

That matters because modern processors have many cores and modern applications perform many operations simultaneously.

NVMe allows storage architecture to behave much more like the technology underneath it.

And PCIe keeps getting faster.

PCIe Gen4.

Gen5.

Gen6.

Every generation dramatically expands the amount of bandwidth potentially available to storage.

This is why comparing a SATA SSD and an NVMe SSD purely because both use NAND flash misses most of the architecture.

It’s like saying a city street and an interstate are equivalent because both are paved with asphalt.

The material isn’t the important difference.

The transportation system is.

SATA SSD and NVMe SSD Are Not the Same Ballpark

This is probably the misconception I encounter most often.

Someone says:

“They’re both SSDs. What’s the difference?”

A lot.

A SATA SSD uses the SATA interface and traditionally communicates through AHCI.

A SAS SSD communicates through the SAS ecosystem using SCSI commands.

An NVMe SSD communicates using NVMe over PCI Express.

The flash media might be conceptually similar.

Everything around it is different.

The protocol.

The controller architecture.

The queue model.

The latency.

The available bandwidth.

The path to the CPU.

And potentially the availability architecture.

This is why an NVMe drive can deliver performance far beyond what a SATA SSD can achieve even when both contain very fast NAND.

But there is an enormous asterisk attached to that statement.

Your NVMe Drive Doesn’t Exist in a Vacuum

This is where storage architecture becomes more interesting than product specifications.

Let’s say a customer tells me:

“I want the fastest storage available.”

Fair enough.

So they buy NVMe drives.

Problem solved?

Not necessarily.

Where are those drives connected?

What PCIe generation?

How many lanes?

Is there a PCIe switch?

What backplane are we using?

Are the drives directly attached to CPU PCIe lanes?

Are they passing through a Tri-Mode controller?

Are we doing hardware RAID?

What RAID level?

How many drives?

What workload?

Sequential or random?

Read-heavy or write-heavy?

What block size?

How much queue depth?

What filesystem?

What CPU architecture?

Where are the NUMA boundaries?

And where is the application consuming the data?

Suddenly the label on the SSD is only one small piece of the performance question.

Storage performance is a path.

Your application doesn’t magically communicate with the NAND chips because the drive says NVMe on the front.

Every component between the application and that NAND matters.

Congratulations. You Bought a 7 GB/s Drive.

Here’s one of my favorite infrastructure problems.

A customer buys several extremely fast NVMe SSDs.

Each drive might advertise multiple gigabytes per second of sequential throughput.

Then they connect all of them through a storage architecture whose upstream path cannot carry the combined bandwidth of those drives.

Now imagine four drives capable of 7 GB/s each.

On paper:

28 GB/s.

Wonderful.

But if the controller, PCIe slot, switch, backplane or some other shared path can move only a fraction of that, you don’t have a 28 GB/s storage system.

You have a very expensive collection of drives waiting in traffic.

This isn’t unique to NVMe.

Storage architects have dealt with oversubscription forever.

SAS expanders.

SAN fabrics.

Ethernet uplinks.

PCIe switches.

RAID controllers.

Eventually many fast endpoints converge onto fewer shared resources.

The bottleneck moves.

That’s why quoting the speed of an individual drive doesn’t tell me the performance of your storage architecture.

Hardware RAID Changes the Conversation

Hardware RAID is a perfect example.

Traditional enterprise RAID controllers became incredibly mature around SAS and SATA.

You connect drives to the controller.

The controller handles RAID.

It may provide protected write cache.

It abstracts the physical drives and presents logical volumes to the operating system.

For years, this architecture made perfect sense.

Then NVMe arrived.

Early NVMe architectures often bypassed the traditional storage controller and connected drives much more directly through PCIe.

Great for performance.

Less convenient if your operational model depended on conventional hardware RAID.

The industry responded with technologies such as Tri-Mode controllers capable of handling SAS, SATA and NVMe devices.

Broadcom’s MegaRAID 9600 family, for example, supports NVMe, 24G SAS and SATA devices while providing traditional RAID levels including RAID 0, 1, 5, 6, 10, 50 and 60.

The newer MegaRAID 9760W-16i moves the host side to PCIe 5.0 x16 while supporting 24G SAS and PCIe/NVMe connectivity.

So modern hardware RAID and NVMe absolutely can coexist.

But the architecture still matters.

Put enough high-performance NVMe drives behind any shared controller and eventually something becomes saturated.

Maybe it’s the PCIe host interface.

Maybe it’s the controller.

Maybe it’s RAID parity processing.

Maybe it’s the backplane topology.

Maybe it’s somewhere else entirely.

The lesson isn’t that RAID controllers are slow.

The lesson is:

Know where your aggregate bandwidth goes.

Sometimes SAS Can Beat NVMe

Here’s another statement that sounds wrong until we add context:

A SAS storage system can outperform an NVMe storage system.

That does not mean SAS is inherently faster than NVMe.

It isn’t.

Given comparable modern devices and an unconstrained architecture, NVMe offers substantially greater potential bandwidth and lower latency.

But systems aren’t benchmarked by acronyms.

They’re benchmarked as systems.

A well-designed 24G SAS array using many drives across sufficient lanes and controllers can absolutely outperform a poorly designed NVMe architecture constrained by an undersized controller, PCIe uplink or RAID implementation.

Broadcom’s current 24G SAS architecture supports 22.5 Gb/s per SAS lane, while its controllers aggregate many ports behind PCIe host interfaces.

That is not slow storage.

The correct comparison isn’t:

NVMe versus SAS.

It’s:

This complete NVMe architecture versus this complete SAS architecture for this particular workload.

That’s a much less exciting answer.

It’s also the answer that matters.

And Please Don’t Confuse This With iSCSI

Now we get to another common source of confusion.

SAS.

SATA.

NVMe.

iSCSI.

Fibre Channel.

Ethernet.

These terms often end up in the same storage conversation even though they describe different layers of the architecture.

SATA, SAS and PCIe/NVMe describe how storage devices communicate locally within a server or storage system.

iSCSI and Fibre Channel generally describe how hosts communicate with storage systems across a storage network.

Ethernet is a networking technology over which several storage protocols can operate.

For example, you could build a storage array containing NVMe SSDs internally.

That array could expose block storage to servers using Fibre Channel.

Or iSCSI over Ethernet.

Or NVMe over Fabrics.

The drives inside the array don’t magically become “Fibre Channel drives.”

The host-facing storage network and the media-facing drive architecture are different parts of the system.

This distinction is fundamental.

Think in Layers

Here’s a simpler way to visualize it.

At the bottom you have the physical media:

NAND flash or magnetic disk.

Then the drive interface and command architecture:

SATA/AHCI, SAS/SCSI or PCIe/NVMe.

Then potentially a controller or aggregation layer:

HBA, RAID controller, PCIe switch, storage controller.

Then the storage system:

DAS, array, HCI node, software-defined storage platform.

Then potentially the network transport:

Ethernet or Fibre Channel.

Then the storage protocol presented to hosts:

iSCSI, FCP, NFS, SMB, NVMe/TCP, NVMe/FC and others.

Then the host.

Then the application.

Every layer can introduce a bottleneck.

And making one layer dramatically faster doesn’t automatically make the layers above it faster.

A 100GbE Port Doesn’t Make Your Disks Faster Either

The same mistake happens in the opposite direction.

Someone builds storage containing relatively slow media and connects it using 100Gb Ethernet.

Now the network is incredibly fast.

The storage isn’t.

Or perhaps the storage is extremely fast but the client accesses it over a 10GbE link.

Now the storage array spends much of its time waiting for the network.

Or perhaps both are fast, but the workload consists of tiny random writes protected by RAID 6.

Now throughput specifications aren’t telling you the interesting part of the story at all.

Maybe latency and IOPS are what matter.

Storage performance is always contextual.

That’s why I get nervous when someone tells me:

“Just give me the fastest drives.”

My next question is always:

Fastest at doing what?

Sequential Throughput Isn’t IOPS

This deserves its own discussion.

Storage vendors love enormous throughput numbers.

14 GB/s!

7 GB/s!

But those numbers usually represent favorable sequential workloads.

That might matter enormously if you’re moving giant files, streaming video, processing datasets or feeding large sequential workloads.

A transactional database may care far more about random IOPS and latency.

Virtual desktop infrastructure can produce a very different I/O profile.

AI training has another.

Backup has another.

OLTP has another.

Virtualization has another.

A drive optimized for enormous sequential bandwidth isn’t automatically the best drive for every workload.

You need to understand:

Throughput.

How much data can move?

IOPS.

How many operations can occur?

Latency.

How long does each operation take?

Queue depth.

How many operations are waiting?

And then something that enterprise buyers should never ignore:

Consistency.

I would often rather have predictable performance than a spectacular benchmark number that collapses under sustained load.

Enterprise SSDs Aren’t Just Expensive Consumer SSDs

This is another misconception worth killing.

An enterprise SSD isn’t simply a consumer SSD with a higher price.

Enterprise drives can differ in endurance, power-loss protection, firmware behavior, overprovisioning, error handling, telemetry, sustained performance and qualification.

Drive Writes Per Day—DWPD—matters.

Write endurance matters.

Power-loss protection matters.

Latency consistency matters.

Firmware validation matters.

If I’m storing someone’s Steam library, my priorities are different from a database processing financial transactions.

The fastest benchmark result isn’t necessarily the most appropriate enterprise drive.

Reliability and predictable behavior are performance characteristics too.

Then NVMe Escaped the Server

NVMe’s next trick was even more interesting.

Once we created a storage protocol designed around fast non-volatile memory, engineers naturally asked:

Why does the NVMe device need to be physically inside the server?

Enter NVMe over Fabrics, or NVMe-oF.

Now NVMe semantics can extend across a network using transports such as NVMe over Fibre Channel and NVMe over TCP.

This is where the distinction between drive technology and storage-network technology becomes especially important.

An application can potentially access remote NVMe storage across a fabric while preserving many of the architectural advantages of NVMe.

The drive is remote.

The storage is networked.

But we’re no longer necessarily translating everything back into the traditional SCSI model along the way.

That is a major architectural evolution.

What’s Coming Next?

Storage isn’t done evolving.

PCIe continues getting faster.

NVMe continues evolving.

Computational storage is moving some processing closer to the data rather than constantly dragging enormous datasets back to CPUs.

CXL—Compute Express Link—is beginning to blur traditional boundaries between memory and storage architectures by enabling new ways for processors and accelerators to access shared and expanded memory resources.

NVMe over Fabrics continues making high-performance disaggregated storage more practical.

QLC NAND continues pushing flash toward higher capacities and lower cost per terabyte, increasingly challenging hard drives in workloads where flash historically would have been economically unrealistic.

At the same time, hard drives aren’t disappearing.

HAMR and other advances are pushing HDD capacities higher because sometimes the requirement isn’t microseconds of latency.

Sometimes you just need an enormous amount of reasonably priced storage.

And AI is making all of this even more interesting.

AI infrastructure wants enormous throughput.

Huge datasets.

Fast checkpointing.

High-bandwidth local storage.

Massive shared storage.

GPUs that cost too much money to sit idle waiting for data.

In that environment, storage architecture becomes part of compute performance.

A $40,000 accelerator waiting on storage is an expensive way to discover that you designed the I/O path incorrectly.

Stop Buying Acronyms

If there’s one thing I want people to take away from this, it’s this:

Don’t design storage by acronym.

NVMe isn’t automatically the answer.

SAS isn’t obsolete simply because NVMe exists.

SATA isn’t useless because it’s slower.

Fibre Channel isn’t a drive technology.

iSCSI isn’t a type of SSD.

And buying the fastest drive available doesn’t make your application faster if everything around that drive is incapable of keeping up.

Start with the workload.

Determine the capacity.

Determine the performance requirements.

Determine the availability requirements.

Determine the endurance.

Determine the latency target.

Then follow the entire data path.

From the application.

Through the operating system.

Through the network if there is one.

Through the controller.

Across PCIe.

Through the backplane.

Into the drive.

And ultimately to the media itself.

Find the narrowest point.

Because that’s your storage performance.

Not the number printed on the SSD box.

The storage industry has spent decades moving from parallel SCSI to SAS and SATA, then from storage-specific buses toward PCIe and NVMe, and now toward increasingly disaggregated architectures such as NVMe over Fabrics.

The drives keep getting faster.

The interfaces keep getting faster.

The networks keep getting faster.

But one principle hasn’t changed:

Your storage system is only as fast as the slowest part of the path your data has to travel.