Redis基础知识(学习笔记17--持久化 (3))
3.4 参数优化
(1)appendfsync
# The fsync() call tells the Operating System to actually write data on disk # instead of waiting for more data in the output buffer. Some OS will really flush # data on disk, some other OS will just try to do it ASAP. # # Redis supports three different modes: # # no: don't fsync, just let the OS flush the data when it wants. Faster. ##这种模式比较快 # always: fsync after every write to the append only log. Slow, Safest. ##这种模式比较安全 # everysec: fsync only one time every second. Compromise. ##妥协或折中的方案 # # The default is "everysec"【默认值】, as that's usually the right compromise between # speed and data safety. It's up to you to understand if you can relax this to # "no" that will let the operating system flush the output buffer【输出缓存】 when # it wants, for better performances (but if you can live with【接受】 the idea of # some data loss consider the default persistence mode that's snapshotting), # or on the contrary【相反】, use "always" that's very slow but a bit safer than # everysec. # # More details please check the following article: # http://antirez.com/post/redis-persistence-demystified.html # # If unsure, use "everysec". # appendfsync always appendfsync everysec # appendfsync no
当客户端提交写命令后,该命令就会写入到aof_buf中,而aof_buf中的数据持久化到磁盘AOF文件的过程称为数据同步。
何时将aof_buf中的数据同步到AOF文件?采用不同的数据同步策略,同步的时机是不同的,有三种策略:
- alwasy:写操作命令写入aof_buf后会立即调用fsync()系统函数,将其追加到AOF文件。该策略效率较低,但是相对较安全,不会丢失太多数据。最多就是刚刚执行过的写操作在尚未同步时出现宕机或重启,将这一操作丢失。
- no:写操作命令写入aof_buf后什么也不做,不会调用fsync()函数。而将aof_buf中的数据同步到磁盘的操作由操作系统负责。Linux系统默认同步周期为30秒。效率较高。
- everysec:默认策略。写操作命令写入aof_buf后并不直接调用fsync(),而是每秒调用一次fsync()系统函数来完成同步。该策略兼顾到了性能与安全,是一种折中方案。
(2)no-appendfsync-on-rewrite
# When the AOF fsync policy is set to always or everysec, and a background # saving process (a background save or AOF log background rewriting) is # performing a lot of I/O against the disk, in some Linux configurations # Redis may block too long on the fsync() call. Note that there is no fix【没有修复】 for # this currently, as even performing fsync in a different thread will block # our synchronous write(2) call. # # In order to mitigate【缓和;减轻;缓解】 this problem it's possible to use the following option # that will prevent【阻塞;阻止;】 fsync() from being called in the main process while a # BGSAVE or BGREWRITEAOF is in progress. # # This means that while another child is saving, the durability【持久性】 of Redis is # the same as "appendfsync no". In practical terms, this means that it is # possible to lose up to 30 seconds of log in the worst scenario (with the # default Linux settings). # # If you have latency【延迟】 problems turn this to "yes". Otherwise leave it as # "no" that is the safest pick from the point of view of durability. no-appendfsync-on-rewrite no
该属性用于指定,当AOF fsync策略设置为always 或 everysec,当主进程创建了子进程正在执行bgsave或bgrewriteaof时,主进程是否不调用fsync()来做数据同步。设置为no,双重否定即肯定,主进程会调用fsync()做同步。而yes则不会调用fsync()做数据同步。
如果调用了fsync(),在需要同步的数据量非常大时,会阻塞主进程对外提供服务,即会存在延迟问题。如果不调用fsync(),则AOF fsync策略相当于设置为了no,可能会存在高达30秒日志的丢失。
(3)aof-rewrite-incremental-fsync 和 rdb-save-incremental-fsync
注意:这一步的定义在【########## ADVANCED CONFIG #######】部分
# When a child rewrites the AOF file, if the following option is enabled # the file will be fsync-ed every 4 MB of data generated. This is useful # in order to commit the file to the disk more incrementally and avoid # big latency【延迟】 spikes【尖刺】. aof-rewrite-incremental-fsync yes # When redis saves RDB file, if the following option is enabled # the file will be fsync-ed every 4 MB of data generated. This is useful # in order to commit the file to the disk more incrementally and avoid # big latency spikes. rdb-save-incremental-fsync yes
当bgrewriteaof在执行过程中也是先将rewrite计算的结果写入到了aof_rewrite_buf缓存中,然后当缓存中数据达到一定量后,就会调用fsync()进行刷盘操作,即数据同步,将数据写入到临时文件。该属性用于控制fsync()每次刷盘的数据量最大不超过4MB。这样可以避免由于单次刷盘量过大而引发长时间阻塞。
(4)aof-load-truncated
# An AOF file may be found to be truncated【截断的;缩减的】 at the end during the Redis # startup process, when the AOF data gets loaded back into memory. # This may happen when the system【指操作系统】 where Redis is running # crashes, especially when an ext4 filesystem is mounted without the # data=ordered option (however this can't happen when Redis itself # crashes or aborts but the operating system still works correctly). # # Redis can either exit with an error when this happens, or load as much # data as possible (the default now) and start if the AOF file is found # to be truncated at the end. The following option controls this behavior. # # If aof-load-truncated is set to yes, a truncated AOF file is loaded and # the Redis server starts emitting【发出;发射】 a log to inform the user of the event. # Otherwise if the option is set to no, the server aborts【终止】 with an error # and refuses to start. When the option is set to no, the user requires # to fix the AOF file using the "redis-check-aof" utility before to restart # the server. # # Note that if the AOF file will be found to be corrupted【损坏的】 in the middle # the server will still exit【退出】 with an error. This option only applies when # Redis will try to read more data from the AOF file but not enough bytes # will be found. aof-load-truncated yes
(5)aof-timestamp-enabled
# Redis supports recording timestamp annotations【注解】 in the AOF to support restoring # the data from a specific point-in-time. However, using this capability changes # the AOF format in a way that may not be compatible【兼容的】 with existing AOF parsers.【当前的AOF解析器】 aof-timestamp-enabled no
该属性设置为yes则会开启在AOF文件中增加时间戳的显示功能,可方便按照时间对数据进行恢复。但该方式可能会与AOF解析器不兼容,所以默认值为no,不开启。
3.5 AOF持久化过程
AOF详细的持久化过程如下:
1)Redis接受到写操作命令并不是直接追加到磁盘的AOF文件的,而是将每一条写命令按照redis通讯协议格式暂时添加到AOF缓冲区aof_buf。
2)根据设置的数据同步策略,当同步条件满足时,再将缓冲区中的数据一次性写入磁盘的AOF文件,以减少磁盘IO次数,提高性能。
3)当磁盘的AOF文件大小达到rewrite条件时,redis-server主进程会fork出一个子进程bgwriteaof,由该子进程完成rewrite过程。
4)子进程bgwriteaof首先对磁盘AOF文件进行rewrite计算,将计算结果写入到一个临时文件,全部写入完毕后,再rename该临时文件为磁盘文件的原名称,覆盖原文件。
5)如果在rewrite过程中又有写操作命令写入到临时文件后,那么这些数据会暂时写入aof_rewrite_buf缓冲区。等将全部rewrite计算结果写入临时文件后,会先将aof_rewrite_buf缓冲区中的数据写入临时文件,然后再rename为磁盘文件的原名称,覆盖原文件。
概况如下:
四. RDB 与 AOF 对比
4.4.1 RDB优势与不足
(1)RDB优势
- RDB文件较小(内存数据);
- 数据恢复较快。
(2)RDB不足
- 在一次性写入量较大的情况下,数据丢失风险较大;持久化过程相对较长,较重的磁盘IO;因此数据持久性相对差;
- 写时复制会降低性能;
- RDB文件可读性较差。
4.4.2 AOF优势与不足
(1)AOF优势
- 数据及时持久性强(数据丢失的可能性小);
- AOF文件可读性强。
(2)AOF不足
- AOF文件相对较大(日志型);
- 写操作会影响性能;
- 数据恢复相对慢。
4.4.3 持久化技术的选型
- 官方推荐使用RDB与AOF混合式持久化。
- 若对数据持久性要求不高,则推荐使用纯RDB持久化方式。
- 官方不推荐使用纯AOF持久化方式(文件大;恢复慢)。
- 若Redis仅用于缓存,则无需使用任何持久化技术。
学习参阅特别声明
【Redis视频从入门到高级】
【https://www.bilibili.com/video/BV1U24y1y7jF?p=11&vd_source=0e347fbc6c2b049143afaa5a15abfc1c】