什么是SSD TRIM (by quqi99)_report_discard_supported

 

普通硬盘并不真正从硬盘里删除数据，只是在操作系统标记为删除，下次直接覆盖写；但SSD物理覆盖写却是很慢的(对于SSD，可以以4KB大小的页(128个page组成一个block)来读写数据，但却只能以512KB(128 page)大小来删除。要覆盖写一个4KB page的话，先要把512KB block全部复制到缓存里，然后删除这4kb page并替换成新的写数据，接下来清空SSD的这512KB block区域，并从缓存里把新的数据写回去）。所以当SSD的所有空闲块都被使用后，再有写的操作只能覆盖写到之前被操作系统标记为删除的区域，这也是速度下降的开始。特别是一系列小文件随机写入同时覆盖写很多的块，那样缓存一下子会快速过载而变得非常慢，所以SSD厂商都在新主控制器上加入了越来越大的外置缓存，这虽然可以一定程序上解决随机写入卡的问题，但是却不能解决SSD在覆盖写时速度下降的问题。如果操作系统在删除一个4KB page的时候不仅在操作系统记录里删除也控制SSD主控物理删除它的话，这就是TRIM干的事情，它将写时的延迟移到了删除时。需要内核、固件（SSD厂商在固件里放TRim算法）、驱动（如Intel的AHCI驱动）三者都支持TRIM才行。

可用fstrim命令回收一个已挂载的文件系统上所有未使用的块（sudo fstrim -v）；

也可使用smartctl -a /dev/sda 命令检测磁盘是否有错，从结果里找类似"177 Wear_Leveling_Count 0x0013 099 099 000 Pre-fail Always - 9"，那个9是SSD厂商估计的寿命，0代表新SSD盘，100代表可能明天就坏了。

 

下面是smartmontools自检工具的基本使用

1, 安装包

   sudo apt-get install smartmontools

2, 查看硬盘是否enable了SMART自检特性，若没enable,可使用命令”sudo smartctl --smart=on --offlineauto=on --saveauto=on /dev/sda“打开。
hua@node1:~$ sudo smartctl -i /dev/sda
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-4.4.0-9-generic] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org


=== START OF INFORMATION SECTION ===
Model Family:     Seagate Barracuda 7200.14 (AF)
Device Model:     ST2000DM001-9YN164
Serial Number:    W1E0S44Q
LU WWN Device Id: 5 000c50 052bb6b91
Firmware Version: CC4B
User Capacity:    2,000,398,934,016 bytes [2.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS T13/1699-D revision 4
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Mar 23 14:50:55 2016 CST


==> WARNING: A firmware update for this drive may be available,
see the following Seagate web pages:
http://knowledge.seagate.com/articles/en_US/FAQ/207931en
http://knowledge.seagate.com/articles/en_US/FAQ/223651en


SMART support is: Available - device has SMART capability.
SMART support is: Enabled

3, 检测
hua@node1:~$ sudo smartctl -H /dev/sda
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-4.4.0-9-generic] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org


=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED


4, 详细检测
hua@node1:~$ sudo smartctl -A /dev/sda
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-4.4.0-9-generic] (local build)
Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org


=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   117   099   006    Pre-fail  Always       -       141775376
  3 Spin_Up_Time            0x0003   095   094   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       561
  5 Reallocated_Sector_Ct   0x0033   100   100   036    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   084   060   030    Pre-fail  Always       -       306606505
  9 Power_On_Hours          0x0032   095   095   000    Old_age   Always       -       5237
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       667
183 Runtime_Bad_Block       0x0032   100   100   000    Old_age   Always       -       0
184 End-to-End_Error        0x0032   100   100   099    Old_age   Always       -       0
187 Reported_Uncorrect      0x0032   099   099   000    Old_age   Always       -       1
188 Command_Timeout         0x0032   100   099   000    Old_age   Always       -       0 0 1
189 High_Fly_Writes         0x003a   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0022   067   054   045    Old_age   Always       -       33 (Min/Max 20/36)
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       77
193 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       1907
194 Temperature_Celsius     0x0022   033   046   000    Old_age   Always       -       33 (0 8 0 0 0)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
240 Head_Flying_Hours       0x0000   100   253   000    Old_age   Offline      -       5111h+51m+54.467s
241 Total_LBAs_Written      0x0000   100   253   000    Old_age   Offline      -       12778926325684
242 Total_LBAs_Read         0x0000   100   253   000    Old_age   Offline      -       42981342412817

 

虚机中的discard

通过将trim与unmap指令提供的功能叫discard - search: DISCARD SUPPORT IN QEMU-KVM
对于SSD, trim指令允许操作系统在SSD上将不再使用的数据通知到SSD底层并在其内部将数据擦除掉。
虚机的vdisk与SSD类似, 也存在若不及时删除数据的话会造成数据越来越大, 这样virtio-scsi虚拟磁盘上出现了一个类似trim的指令叫unmap. 

1, 虚机定义增加额外的discard="unmap" (sda)
NOTE: 只对virtio-scsi驱动有效: glance image-update --property hw_scsi_model=virtio-scsi --property hw_disk_bus=scsi
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none' discard='unmap'/>
      <source file='/mnt/sda.img'/>
      <target dev='sda' bus='scsi'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none'/>
      <source file='/mnt/sdb.img'/>
      <target dev='sdb' bus='scsi'/>
    </disk>
2, 虚机内部挂载磁盘时增加对discard的支持: mount -o discard /dev/sda1 /sda
3, 测试脚本
$ for _ in {1..3}; do
    /bin/dd if=/dev/urandom of=/sda/file.$RANDOM bs=1M count=100 && sync
    /bin/dd if=/dev/urandom of=/sdb/file.$RANDOM bs=1M count=100 && sync
    /bin/rm -f /sda/file.* /sdb/file.* && sync
4, 使用fstrim命令手动一次性触发discard操作: /mntpoint. 或者如果使用上面脚试脚本后应该在host机器上用下列命令观察到自动trim
$ du -h sd{a,b}.img
34M  sda.img
138M sdb.img

另外, ceph驱动也支持trim, 除了使用virtio-scsi驱动(glance image-update --property hw_scsi_model=virtio-scsi --property hw_disk_bus=scsi), 也需要编辑nova.conf的[libvirt]段添加hw_disk_discard = unmap (注: cinder通过[ceph] section中report_discard_supported = True 来支持discard, so it is possible to configure Cinder per backend.), 这时虚机的libvirt xml配置文件应该有所反应, 或者 , 在虚机里面查看/sys/block/sdb/queue/discard_*, 见: 
https://ceph.io/geen-categorie/openstack-and-ceph-rbd-discard/
http://www.sebastien-han.fr/blog/2016/05/11/OpenStack-Cinder-discard-support-for-Ceph-in-Mitaka/

# $cinder/cinder/volume/manager.py
1759         # Add discard flag to connection_info if not set in the driver and
1760         # configured to be reported.
1761         if conn_info['data'].get('discard') is None:
1762             discard_supported = (self.driver.configuration
1763                                  .safe_get('report_discard_supported'))                                                                                                           
1764             if discard_supported:
1765                 conn_info['data']['discard'] = True
 
# $nova/nova/virt/libvirt/volume/volume.py
 93         # Configure usage of discard                                                                                                                                               
 94         if data.get('discard', False) is True:
 95             conf.driver_discard = 'unmap'
 
#./nova/virt/libvirt/imagebackend.py
 82 class Image(object):
  86     def __init__(self, path, source_type, driver_format, is_block_dev=False):
            ...
105         self.discard_mode = CONF.libvirt.hw_disk_discard 
 
 148     def libvirt_info(self, ...):
166         info.driver_discard = self.discard_mode
 
# $nova/nova/virt/libvirt/config.py
1009             if self.driver_discard is not None:
1010                 drv.set("discard", self.driver_discard)
 
# $nova/virt/libvirt/driver.py
 1765     def attach_volume(self, ...)
              ...
 1792         disk_info = blockinfo.get_info_from_bdm(
 1793             instance, CONF.libvirt.virt_type, instance.image_meta, bdm)
 1794         if disk_info['bus'] == 'scsi':
 1795             disk_info['unit'] = self._get_scsi_controller_max_unit(guest) + 1
 
 1797         conf = self._get_volume_config(connection_info, disk_info)
 1798 
 1799         self._check_discard_for_attach_volume(conf, instance)
 
 
 1748     def _check_discard_for_attach_volume(self, conf, instance):
 1749         """Perform some checks for volumes configured for discard support.
 1750 
 1751         If discard is configured for the volume, and the guest is using a
 1752         configuration known to not work, we will log a message explaining
 1753         the reason why.
 1754         """
 1755         if conf.driver_discard == 'unmap' and conf.target_bus == 'virtio':                                                                                                       
 1756             LOG.debug('Attempting to attach volume %(id)s with discard '
 1757                       'support enabled to an instance using an '
 1758                       'unsupported configuration. target_bus = '
 1759                       '%(bus)s. Trim commands will not be issued to '
 1760                       'the storage device.',
 1761                       {'bus': conf.target_bus,
 1762                        'id': conf.serial},
 1763                       instance=instance)