02-dataType_api
pipelining 英 [ˈpaɪplaɪnɪŋ] n. 流水线；管道敷设（安装）；管路输送，v. 用管道输送；应用流水线技术设计（pipeline的ing形式）
aggregate 英 [ˈæɡrɪɡət] n. 总数，合计；骨料，混凝料，adj. 总计的，合计的；（种群）聚生的，v. 集合，聚集；总计达到；合计
1. redis-cli
➜ ~# redis-cli -h `ip` -p `port`
127.0.0.1:6379># auth default ******
OK
127.0.0.1:6379># ping
PONG
# 1. 环境变量已经配置，直接进入
redis-cli
# -h 查看帮助
# -p 端口
# -n 选择哪个库
redis-cli -
# 2. 退出redis-cli
exit
# 3. 查看redis进程
ps -ef | grep redis
# 1. 查询所有key
keys *
# 2. 清库
flushdb
flushall
127.0.0.1:6379># set ping pong
OK
127.0.0.1:6379># get k1
(nil)
1. 库
- 类比MySQL，有不同的库，库里有业务表，它们之间是相互隔离的。Redis默认16个库，相互隔离，相互独立
# 1. 指定库
redis-cli -n 8
# 2. 切换库
select 8
2. help
- @generic
- @string
- @list
- @set
- @sorted_set
- @hash
- @pubsub
- @transations
- @connection
- @server
- @scripting
- @hyperloglog
- @cluster
- @geo
- @stream
127.0.0.1:6379># help
redis-cli 6.2.6
To get help about Redis commands type:
"help @<group>" to get a list of commands in <group> # 1. 组
"help <command>" for help on <command> # 2. 命令
"help <tab>" to get a list of possible help topics # 3. tab补全
"quit" to exit
To set redis-cli preferences:
":set hints" enable online hints
":set nohints" disable online hints
Set your preferences in ~/.redisclirc
2. 二进制安全
- 在使用redis时，用户沟通好编码、解码即可
Redis和Hbase都是二进制安全
- Redis只存字节。不同Client取值，只要编解码相同，即可得到正确数据
- 不同的Client对数值有不同的类型，Redis没办法兼容所有类型。java：int float...
- Client编码影响存入字节大小。utf8中文3B；GBK偏向中文，中文2B，被压缩了
- Redis不会破坏你的数据。用什么编码存入，取出后就用什么编码解析。Redis完全不会管编码，
--raw用当前客户端编码显示数据，否则默认ASCII显示。超出ASCII的按照16进制显示
127.0.0.1:6379># set k 中
OK
# 1. 默认ASCII转出
127.0.0.1:6379># get k
"\xe4\xb8\xad"
# 2. raw触发客户端编码集的格式化
➜ redis git:(master) ✗# redis-cli --raw
127.0.0.1:6379> get k
中
1. ASCII
- ASCII
- American Standard Code for Information Interchange：美国信息交换标准代码
- ASCII为标准字符集，其他字符集叫做扩展字符集
- 扩展字符集不对ascii重编码
- 字符集即为底层字节与字符的mapping规则
- ASCII：以0开头，0xxxxxxx
- uft-8：1110开头，要读3个字节，剩余的拼接起来。显示字符
# 查看ASCII帮助文档
man ascii
---------------------------------------------------------------------------------------------------------
ASCII(7) Linux Programmer's Manual ASCII(7)
NAME
ascii - ASCII character set encoded in octal, decimal, and hexadecimal
DESCRIPTION
ASCII is the American Standard Code for Information Interchange. It is a 7-bit code. Many 8-bit
codes (such as ISO 8859-1, the Linux default character set) contain ASCII as their lower half.
The international counterpart of ASCII is known as ISO 646.
The following table contains the 128 ASCII characters.
C program '\X' escapes are noted.
# Hex 16进制41 —— 2进制 0100 0001：代表ASCII，大写A
Oct Dec Hex Char Oct Dec Hex Char
────────────────────────────────────────────────────────────────────────
000 0 00 NUL '\0' 100 64 40 @
001 1 01 SOH (start of heading) 101 65 41 A
002 2 02 STX (start of text) 102 66 42 B
003 3 03 ETX (end of text) 103 67 43 C
004 4 04 EOT (end of transmission) 104 68 44 D
3. String
127.0.0.1:6379># help set
SET key value [EX seconds|PX milliseconds|EXAT timestamp|PXAT milliseconds-timestamp|KEEPTTL] [NX|XX] [GET]
summary: Set the string value of a key
since: 1.0.0
group: string
# 1. 不存在才去设置，场景：分布式锁
set k1 v1 nx
# 2. 存在时，才能操作
set k1 v1 xx
# 3. 设置多个
mset k3 a k4 b
# 4. 获取多个
mget k3 k4
# 5. 追加
append k1 " hello"
# 6. 获取设置。减少一次IO通讯
getset k1 v2
# 7. 正反向索引，取字符串一部分"hello world"
127.0.0.1:6379># get k1
"hello world"
127.0.0.1:6379># getrange k1 6 10
"world"
127.0.0.1:6379># getrange k1 6 -1
"world"
# 8. 覆盖一部分
127.0.0.1:6379># setrange k1 6 songsong
(integer) 14
127.0.0.1:6379> get k1
"hello songsong"
# 9. 不存在则批量设置。原子性，一个失败，全部失败
msetnx k2 c k3 d
1. type
- key里记录数据的
type, encode, length - type（大类型）和encoding（小类型）。类型判断，类型错误直接报错
# 长度byte
strlen k1
# type（大类型）和encoding（小类型）。类型判断，类型错误直接报错
type k1
127.0.0.1:6379># help set
SET key value [EX seconds|PX milliseconds|EXAT timestamp|PXAT milliseconds-timestamp|KEEPTTL] [NX|XX] [GET]
summary: Set the string value of a key
since: 1.0.0
group: string # 命令是哪个分组，type就什么类型
127.0.0.1:6379># set k1 99
OK
127.0.0.1:6379># type k1
string
2. integer
应用场景
- 抢购
- 秒杀
- 详情页
- 点赞
- 评论
规避并发下，对数据库的事务操作，完全由Redis内存操作代替
127.0.0.1:6379># object help
1) OBJECT <subcommand> [<arg> [value] [opt] ...]. Subcommands are:
2) ENCODING <key> # 查看key的小类型
3) Return the kind of internal representation used in order to store the value
4) associated with a <key>.
5) FREQ <key>
6) Return the access frequency index of the <key>. The returned integer is
7) proportional to the logarithm of the recent access frequency of the key.
8) IDLETIME <key>
9) Return the idle time of the <key>, that is the approximated number of
10) seconds elapsed since the last access to the key.
11) REFCOUNT <key>
12) Return the number of references of the value associated with the specified
13) <key>.
14) HELP
15) Prints this help.
127.0.0.1:6379># set k2 hello
OK
# 查看key的encoding
# 1-embstr, 2-int, 3-raw
127.0.0.1:6379># OBJECT encoding k2
"embstr"
127.0.0.1:6379># set k1 99
OK
127.0.0.1:6379># OBJECT encoding k1
"int"
# +1
127.0.0.1:6379># INCR k1
(integer) 100
# +22
127.0.0.1:6379># INCRBY k1 22
(integer) 122
# +0.5
127.0.0.1:6379># INCRBYFLOAT k1 0.5
"122.5"
# -0.5
127.0.0.1:6379># INCRBYFLOAT k1 -0.5
"122"
# -1
127.0.0.1:6379># DECR k1
(integer) 121
# -22
127.0.0.1:6379># DECRBY k1 22
(integer) 99
127.0.0.1:6379># set k3 oo
OK
127.0.0.1:6379># OBJECT encoding k3
"embstr"
127.0.0.1:6379># APPEND k3 xx
(integer) 28
127.0.0.1:6379># OBJECT encoding k3
"raw" # 拼接的字符串即为row类型
3. bitmap
# 1. offset位的偏移量，value是0，1
127.0.0.1:6379># setbit k1 1 1
0
# 2. 长度展示value（字节长度）
127.0.0.1:6379># STRLEN k1
1
127.0.0.1:6379> setbit k1 7 1
0
127.0.0.1:6379> STRLEN k1
1
127.0.0.1:6379> get k1
A
127.0.0.1:6379> setbit k1 9 1
0
127.0.0.1:6379> STRLEN k1
2
127.0.0.1:6379> get k1
A@
# 3. 查询字节区间范围，第一个1出现的bit的offset
127.0.0.1:6379># help BITPOS
BITPOS key bit [start] [end] # 查找指定范围，bit值0|1的位置
summary: Find first bit set or clear in a string
since: 2.8.7
group: string
# 01000001 01000000
127.0.0.1:6379># BITPOS k1 1 0 0
1
127.0.0.1:6379> BITPOS k1 1 1 1
9
127.0.0.1:6379> BITPOS k1 1 0 1
1
# 4. 字节区间范围，1出现的count
127.0.0.1:6379># help BITCOUNT
BITCOUNT key [start end]
summary: Count set bits in a string
since: 2.6.0
group: string
127.0.0.1:6379># BITCOUNT k1 0 1
3
127.0.0.1:6379> BITCOUNT k1 0 0
2
# 01000001
127.0.0.1:6379> setbit k1 1 1
0
127.0.0.1:6379> setbit k1 7 1
0
127.0.0.1:6379> get k1
A
# 01000010
127.0.0.1:6379> setbit k2 1 1
0
127.0.0.1:6379> setbit k2 6 1
0
127.0.0.1:6379> get k2
B
# 5. bitop 进行位运算
127.0.0.1:6379># bitop and k3 k1 k2
1
127.0.0.1:6379> get k3
@
127.0.0.1:6379># bitop or k4 k1 k2
1
127.0.0.1:6379> get k4
C
1. 统计用户登录天数且窗口随机
(1G = 1024^3 * 1B = 2334w * 46B)。可以记录2000w人的登录情况
# 记录、统计用户在一年中哪天登录
127.0.0.1:6379> SETBIT ooxx 1 1
0
127.0.0.1:6379> SETBIT ooxx 7 1
0
127.0.0.1:6379> SETBIT ooxx 364 1
0
127.0.0.1:6379> strlen ooxx
46
127.0.0.1:6379> bitcount ooxx -2 -1
1
2. 活跃用户统计
# 统计（最近登录）活跃用户
127.0.0.1:6379> setbit 20230404 1 1
(integer) 0
127.0.0.1:6379> setbit 20230405 1 1
(integer) 0
127.0.0.1:6379> setbit 20230405 7 1
(integer) 0
127.0.0.1:6379> bitop or hotuser 20230404 20230405
(integer) 1
127.0.0.1:6379> bitcount hotuser 0 -1
(integer) 2
4. List
help @list 帮助文档
- L开头
- left
- list类型
- R开头：right
- B开头：block
底层
- 双向链表
- 正负索引
1. list, stack, queue
# 1.1.（stack）同向命令
lpush lpop
# 1.2.（queue）反向命令
lpush rpop
# 2. 获取所有（正负索引）
lrange k1 0 -1
# 3.1. index取value（array）
lindex k1 2
# 3.2. index设置value（array）
lset k1 3 v1
# 3.3. remove指定方向、指定个数的value
lrem k3 2 a
lrem k3 -2 a
# 4. 删除index范围外的元素
ltrim k1 1 -2
# 5. value前后插入
linsert k3 before 6 a
linsert k3 after 6 a
# 6. 统计元素个数
llen k3
# 1. 输入操作
127.0.0.1:6379># lpush k1 a b c d e f
(integer) 6
127.0.0.1:6379># rpush k2 a b c d e f
(integer) 6
# 2. 遍历操作。正负索引，打印所有
127.0.0.1:6379># lrange k1 0 -1
1) "f"
2) "e"
3) "d"
4) "c"
5) "b"
6) "a"
# 3. 栈操作
127.0.0.1:6379># lpop k1
"f"
127.0.0.1:6379># lpop k1 2
1) "e"
2) "d"
# 4. 队列操作
127.0.0.1:6379># rpop k1
"a"
127.0.0.1:6379># rpop k1 2
1) "b"
2) "c"
127.0.0.1:6379> LRANGE k2 0 -1
1) "f"
2) "e"
3) "d"
4) "c"
5) "b"
6) "a"
# 1. 获取、设置指定index元素。类比数组
127.0.0.1:6379># lindex k2 1
"e"
127.0.0.1:6379># lset k2 2 ooxx
OK
127.0.0.1:6379> LRANGE k2 0 -1
1) "f"
2) "e"
3) "ooxx"
4) "c"
5) "b"
6) "a"
127.0.0.1:6379> lpush k2 a
(integer) 7
127.0.0.1:6379> LRANGE k2 0 -1
1) "a"
2) "f"
3) "e"
4) "ooxx"
5) "c"
6) "b"
7) "a"
# 2. 删除指定个数"a"元素。个数<0，进行逆向删除
127.0.0.1:6379># lrem k2 2 a
(integer) 2
127.0.0.1:6379> LRANGE k2 0 -1
1) "f"
2) "e"
3) "ooxx"
4) "c"
5) "b"
# 3. "ooxx"元素前后插入"a"
127.0.0.1:6379># linsert k2 after ooxx a
(integer) 6
127.0.0.1:6379> LRANGE k2 0 -1
1) "f"
2) "e"
3) "ooxx"
4) "a"
5) "c"
6) "b"
127.0.0.1:6379># linsert k2 before ooxx a
(integer) 7
127.0.0.1:6379> LRANGE k2 0 -1
1) "f"
2) "e"
3) "a"
4) "ooxx"
5) "a"
6) "c"
7) "b"
# 4. 仅保留指定index范围元素
127.0.0.1:6379># LTRIM k2 3 6
OK
127.0.0.1:6379> LRANGE k2 0 -1
1) "ooxx"
2) "a"
3) "c"
4) "b"
2. BlockingQueue
- 3个客户端，2个blpop。1个push
- 先订阅的先pop
- FIFO先进先出
# redis-cli1。阻塞pop
127.0.0.1:6379># BLPOP ooxx 0
1) "ooxx"
2) "1"
(22.68s)
# redis-cli2。阻塞pop
127.0.0.1:6379># BLPOP ooxx 0
1) "ooxx"
2) "2"
(28.11s)
# redis-cli3。push
127.0.0.1:6379># lpush ooxx 1
(integer) 1
127.0.0.1:6379># lpush ooxx 2
(integer) 1
5. Hash
- h开头
- 简单理解：key-value，value是个map
- 应用场景
- 商品详情页的属性、浏览器次数、收藏次数、加入购物车次数
- 微博关注数、点赞数
# 1. 设置、获取一个
127.0.0.1:6379># hset user name listao
(integer) 1
127.0.0.1:6379> hset user age 18
(integer) 1
127.0.0.1:6379> hget user name
"listao"
# 2. 设置、获取多个
127.0.0.1:6379># hmget user name age
1) "listao"
2) "18"
# 3. 获取所有key
127.0.0.1:6379># hkeys user
1) "name"
2) "age"
# 4. 获取所有value
127.0.0.1:6379># hvals user
1) "listao"
2) "18"
# 5. 获取k1的key-value
127.0.0.1:6379># hgetall user
1) "name"
2) "listao"
3) "age"
4) "18"
# 6. 数值计算
127.0.0.1:6379># hincrbyfloat user age 0.5
"18.5"
6. Set_交, 并, 差
- s开头
- 唯一、无序
- 交、并、差集
# 一定要学会看help文档
# 1. 批量添加元素
127.0.0.1:6379># sadd k1 1 2 3 a 4 a 5
(integer) 6
127.0.0.1:6379> sadd k2 4 5 6 7 8
(integer) 5
# 2. 查询所有
127.0.0.1:6379># SMEMBERS k1
1) "a"
2) "3"
3) "5"
4) "2"
5) "1"
6) "4"
# 3. 删除所有a
127.0.0.1:6379># SREM k1 a
(integer) 1
127.0.0.1:6379> SMEMBERS k1
1) "5"
2) "2"
3) "1"
4) "4"
5) "3"
# 4.1. 交集
127.0.0.1:6379># SINTER k1 k2
1) "5"
2) "4"
# 4.2. 交集另存为k3
127.0.0.1:6379># SINTERSTORE k3 k1 k2
(integer) 2
127.0.0.1:6379> SMEMBERS k3
1) "4"
2) "5"
# 5. 并集。sunionstore
127.0.0.1:6379># SUNION k1 k2
1) "1"
2) "2"
3) "3"
4) "4"
5) "5"
6) "6"
7) "7"
8) "8"
# 6. 差集。sdiffstore
127.0.0.1:6379># sdiff k1 k2
1) "1"
2) "2"
3) "3"
# 7.1. 随机取值，不重复。正数：取出一个去重的结果集（不能超过已有集）
127.0.0.1:6379># SRANDMEMBER k1 3
1) "5"
2) "1"
3) "3"
# 7.2. 随机取值，可重复。负数：取出一个带重复的结果集，一定满足你要的数量
127.0.0.1:6379># SRANDMEMBER k1 -3
1) "5"
2) "2"
3) "5"
# 8. 弹出，年会抽奖
127.0.0.1:6379># spop k1
"1"
7. Sorted_Set
- z开头
- 物理内存左小右大，不随命令发生变化
- 场景：歌曲排行（分值计算）
# 1. 添加
127.0.0.1:6379># zadd k1 2 v2 1 v1 3 v3
(integer) 3
# 2. 遍历
127.0.0.1:6379># zrange k1 0 -1
1) "v1"
2) "v2"
3) "v3"
127.0.0.1:6379># ZREVRANGE k1 0 -1
1) "v3"
2) "v2"
3) "v1"
127.0.0.1:6379> ZRANGE k1 -1 -2
(empty array)
127.0.0.1:6379> ZRANGE k1 -2 -1
1) "v2"
2) "v3"
127.0.0.1:6379># ZRANGE k1 0 -1 withscores
1) "v1"
2) "1"
3) "v2"
4) "2"
5) "v3"
6) "3"
127.0.0.1:6379># ZRANGEBYSCORE k1 2 8
1) "v2"
2) "v3"
# 3. 获取权重
127.0.0.1:6379># ZSCORE k1 v2
"2"
# 4. 获取index
127.0.0.1:6379># ZRANK k1 v2
(integer) 1
# 5. 增加权重
127.0.0.1:6379># ZINCRBY k1 2.5 v2
"4.5"
127.0.0.1:6379># ZRANGE k1 0 -1 withscores
1) "v1"
2) "1"
3) "v3"
4) "3"
5) "v2"
6) "4.5"
1. 并集操作
# 1. 并集，默认weight为1，分值相加
127.0.0.1:6379> zadd k1 1 v1 2 v2 3 v3
(integer) 3
127.0.0.1:6379> zadd k2 2 v2 3 v3 4 v4
(integer) 3
127.0.0.1:6379># ZUNION 2 k1 k2
1) "v1"
2) "v2"
3) "v4"
4) "v3"
127.0.0.1:6379># ZUNIONSTORE k3 2 k1 k2
(integer) 4
127.0.0.1:6379> ZRANGE k3 0 -1 withscores
1) "v1"
2) "1"
3) "v2"
4) "4"
5) "v4"
6) "4"
7) "v3"
8) "6"
# 2. 并集，k1权重为1，k2权重为0.5，分值相加
127.0.0.1:6379># ZUNIONSTORE k4 2 k1 k2 weights 1 0.5
(integer) 4
127.0.0.1:6379> ZRANGE k4 0 -1 withscores
1) "v1"
2) "1"
3) "v4"
4) "2"
5) "v2"
6) "3"
7) "v3"
8) "4.5"
# 3. 并集，分值取最大
127.0.0.1:6379># ZUNIONSTORE k5 2 k1 k2 aggregate max
(integer) 4
127.0.0.1:6379> ZRANGE k5 0 -1 withscores
1) "v1"
2) "1"
3) "v2"
4) "2"
5) "v3"
6) "3"
7) "v4"
8) "4"
2. skip_list
- 排序是怎么实现的增、删、改、查的速度？ => 空间换时间
- update = delete + insert
- 跳表也叫类平衡树
- 数据量相对大，平均速度最优
8. Pipelining
- pipelining官网
- 管道可以让通信成本变低一点
# 没有Redis客户端，可以用nc来连接，mac可以直接用nc
yum install nc
# 和Redis建立socket连接
nc localhost 6379
# echo -e 可识别字符串中的换行符，也可以识别颜色
echo -e "jkjjkjkj\n kjkjjkj"
echo -e "\033[33mjkjjkjkjkjkjjkj\033[0m"
# 管道应用
echo -e "set k2 99\n incr k2\n get k2" | nc localhost 6379
1. 文件中批量插入
场景：Redis冷启动，预加载一些data
# redis-cli中只支持dos格式的换行符 \r\n
# Linux、Mac、Windows下创建的文件，最好都转个码
# mac安装unix2dos
brew install unix2dos
# linux安装unix2dos
yum install unix2dos
# 转码
unix2dos d1.txt
# 批量执行
cat data.txt | redis-cli --pipe
ruby proto.rb | redis-cli --pipe
9. Pub/Sub
- Pub/Sub
- 群聊天，一个人发布了消息，所有人都可以看到
# 1. sub端，先启动。持续阻塞
127.0.0.1:6379># SUBSCRIBE ooxx
Reading messages... (press Ctrl-C to quit)
1) "subscribe"
2) "ooxx"
3) (integer) 1
# 2. pub端
127.0.0.1:6379># PUBLISH ooxx hello
(integer) 1
1. Sub
- 场景：微信聊天记录
127.0.0.1:6379># help @sorted_set
ZREMRANGEBYRANK key start stop # 日期表示分值，已经排好序。只保留最近三天的
summary: Remove all members in a sorted set within the given indexes
since: 2.0.0
ZREMRANGEBYSCORE key min max
summary: Remove all members in a sorted set within the given scores
since: 1.2.0
# ...
2. PubClient挂机问题
- 完成了《发布订阅》还未完成《sorted_set》或者kafka，pub_client就挂了
10. Transaction
127.0.0.1:6379># help @transactions
DISCARD -
summary: Discard all commands issued after MULTI
since: 2.0.0
EXEC -
summary: Execute all commands issued after MULTI
since: 1.2.0
MULTI -
summary: Mark the start of a transaction block
since: 1.2.0
UNWATCH -
summary: Forget about all watched keys
since: 2.2.0
# 1. WATCH使得EXEC命令需要有条件地执行（乐观锁CAS），即watch_key发生变化，放弃本次Trx
# 2. WATCH命令可以被调用多次。对键的监视从WATCH执行之后开始生效，直到调用EXEC为止
WATCH key [key ...]
summary: Watch the given keys to determine execution of the MULTI/EXEC block
since: 2.2.0
# 1. 开启Trx
127.0.0.1:6379># MULTI
OK
127.0.0.1:6379(TX)> set k1 v1
QUEUED
127.0.0.1:6379(TX)> keys *
QUEUED
# 2. 执行Trx
127.0.0.1:6379(TX)># EXEC
1) OK
2) 1) "k1"
# 3. 监听k1
127.0.0.1:6379># WATCH k1
OK
127.0.0.1:6379> MULTI
OK
127.0.0.1:6379(TX)> set k2 v2
QUEUED
127.0.0.1:6379(TX)> keys *
QUEUED
# 4. 另一个Client，更改了k1
127.0.0.1:6379> set k1 vv
OK
# 5. Trx失败
127.0.0.1:6379(TX)> EXEC
(nil)
1. 不支持rollback
- Redis命令只会因为错误的语法而失败（并且这些问题不能在入队时发现），或是命令用在了错误类型的键上面
- 不需要对回滚进行支持，所以Redis内部可以保持简单且快速