闪存技术与应用方法
Nand Flash
Nand Flash生产流程
正片(original die)，白片(good die)，黑片(ink die)
企业级SSD
SATA SAS PCIe NVMe NVDIMM
Nand 控制器
PCIE DMA
CPU0
CPU1
DDR3
Controller
4GB
72bit
ECC
DDR3
AXI4
AXI4-Lite
NAND CTRL NAND
Flash
NOR Flash I2C
Interrupt
Controller
Nand可靠性
• NAND Flash 错误模型
– Bad blocks
– Program disturb
– Read disturb
– Endurance
– Data retention
– Gross errors
ECC纠错技术
BCH
LDPC
RAID保护
aidsector
0
aidsector
5
aidsector
4
aidsector
3
aidsector
2
aidsector
1
aidsector
6
⊕ ⊕ ⊕
⊕ ⊕
⊕
初始化
aidsector
7
aidsector
0
aidsector
5
aidsector
4
aidsector
3
aidsector
2
aidsector
1
aidsector
7
aidsector
6
初始化后，受损，抛弃它
aidsector
0
aidsector
5
aidsector
4
aidsector
3
aidsector
2
aidsector
1
aidsector
7
aidsector
6
⊕ ⊕ ⊕
⊕ ⊕
⊕
aidsector
7
recover
replace
Read retry
Wear leveling
Garbage collection
schedule
SSD成本真的高？
$/IOPS, $/GB, latency
SSD应用考量
• 要不要用SSD?
• SLC/MLC/TCL/3D?
• SATA SSD还是PCIe SSD?
• 作为主存储还是Cache/Tier?
• 有无特殊要求? 如宽温、加固、加密、自毁
如何选择SSD？
• 容量/IOPS/带宽
• Latency/Jitter
• 写寿命
• 功耗/散热
• 成本或性价比
如何测试SSD？
@85℃ = 1year@40℃
Arrhenius equation (阿伦尼乌斯方程)
CPU的影响
• Intel, AMD, ARM, 龙芯,飞腾
• CPU主频
• 多socket/core
• NUMA问题
SSD系统优化
• I/O schedule算法
• CPU多核绑定
• 请求列队和最大请求数
• 禁用merge/rotational/read_ahead/barrier
• 分区/卷4KB对齐
• 文件系统开启SSD支持选项
软件定义闪存
Application Application
FileSystem+
IO Stack
Driver
Interface Protocol
FTL
NAND Management
Interface Protocol
NAND Management
Driver
Customized FS
Customized FTL
User Space
kernel
PCIe
Block devices
Pblaze Eblaze
闪存自定义
• API接口
• OP
• FTL
• RAID
• OOB
• Bad block
闪存替换内存计算
Good die/Ink die
Flash RAID
St
o
ra
ge
b
ac
kp
la
n
e
SA
SFlash RAID
SOC
SF
F
8
0
8
7
SF
F
8
0
8
7
SF
F
8
0
8
7
SF
F
8
0
8
7 inch
Flash Disk
SA
S
inch
Flash Disk
SA
S
inch
Flash Disk
SA
S
inch
Flash Disk
SA
S
PCIe
Server SAN
Node 1
SSDs HDDs
Node 2
SSDs HDDs
Node N
SSDs HDDs
Applications
Server SAN Controller Server SAN Controller Server SAN Controller
Applications Applications
Hypervisor I/O driver Hypervisor I/O driver Hypervisor I/O driver
Scale out
All Flash Array
CPU
CPU
NT Bridge
NVRAM
NVRAM
PCIe Switch
PCIe Switch
SSD
SSD
SSD
SSD
SSD
SSD
……
……
Controller0
Controller1
NVME
NVME
10GbE
10GbE
10GbE
10GbE
10GbE
10GbE
10GbE
10GbE
Failover
Eth
Switch
Client
Client
Client
Client
Client
Client
Q & A
