05-击穿_穿透_雪崩_集成

lettuce 英 [ˈletɪs] n. 莴苣;生菜

• 重点：击穿，穿透，雪崩，分布式锁，API （jedis，lettce，SpringBoot：low/high level）

1. 击穿

1. 产生条件

1. Redis作缓存
2. 高并发一个过期key，击穿Redis请求到DB

2. 分布式锁

1. Redis
   1. setnx
   2. expire
   3. 多线程
      1. 事务线程
      2. 守护线程
2. redisson有现成的分布式锁
3. zookeeper专业分布式锁。既然已经是锁了，对效率要求其次，对准确和一致性要求第一

# redis分布式锁伪代码

# 1. 并请求业务key
get ooxx

# 2. get不到，尝试加锁
setnx k_lock 1

# 3.1. setnx成功，设置锁失效时间
expire k_lock 60

# 3.2. setnx失败，sleep(等待时间)，重复1

2. 穿透

1. 产生条件

1. Redis作缓存
2. 高并发大量不存在的key（Redis、DB都没有），穿透Redis请求到DB

2. bloomFilter

• Aim：解决缓存穿透问题
• 不支持删除

1. github_RedisBloom
2. Quick Start Guide for RedisBloom
3. github上的源码不能make成功

1. install

# linux下载
wget https://github.com/RedisBloom/RedisBloom/archive/refs/heads/master.zip

# github不能make，gitee可以
git clone https://gitee.com/listao/RedisBloom.git

# linux解压工具
yum install unzip
unzip ooxx

# 进入解压目录，Makefile文件，make得到.so拓展库
# windows拓展库是.dll
# linux拓展库是.so
make

# 启动redis并加载配置文件
# 顺序固定，路径必须为绝对路径
sudo ./redis-server /etc/redis/6379.conf --loadmodule /Users/list/soft/redis/RedisBloom/redisbloom.so

# 查看redis进程
ps -ef | grep redis

# 验证
redis-cli
bf命令已经加入

# cuckoo filter布谷鸟过滤器
cf.add

# 添加module（模块），绝对路径
loadmodule /path/to/other_module.so

2. 原理图

1. 穿透了，DB存在。BloomFilter添加
2. DB增加元素。BloomFilter添加

3. api

1. Redis只做缓存（Client + BloomFilter算法 + bitmap）
2. Redis + bitmap（Client + BloomFilter算法）
3. Redis + BloomFilter算法 + bitmap（Client）

127.0.0.1:6379># BF.ADD ooxx a
(integer) 1
127.0.0.1:6379># BF.EXISTS ooxx a
(integer) 1
127.0.0.1:6379># BF.EXISTS ooxx b
(integer) 0

3. 雪崩

1. 产生条件

1. Redis作缓存
2. 大量key同时过期，每个key有一定并发

2. 时点性

1. 无时点性，随机过期时间
2. 有时点性，强依赖击穿方案
3. 有时点性，业务层加判断随机sleep(ms)

4. 集成

1. 英文官网
2. 支持大量的语言。二进制安全的。java存，php取
3. Spring默认lettuce

1. Jedis(大数据用)

• github_jedis

<dependency>
    <groupId>redis.clients</groupId>
    <artifactId>jedis</artifactId>
    <version>4.3.0</version>
</dependency>

/**
 * 1. 线程不安全
 * 2. jedis对象不加lock，一个thread在jedis开启Trx，另一个thread通过这个jedis执行命令，脏请求
 */
@Test
void jedis() {
    Jedis one = new Jedis("127.0.0.1", 6379);
    String v1 = one.get("k1");
    System.out.println("v1 = " + v1);
}

/**
 * 1. pool连接池，解决线程不安全。github最新的readme，已经没有单独创建jedis对象了
 */
@Test
void jedisPool() {
    JedisPool pool = new JedisPool("localhost", 6379);

    try (Jedis jedis = pool.getResource()) {
        jedis.set("clientName", "Jedis");
        String v1 = jedis.get("k1");
        System.out.println("v1 = " + v1);
    }
}

• 线程不安全。client01创建Jedis对象，不加lock，并开启Trx，client02使用Jedis执行脏命令

2. lettuce(大数据用)

1. lettuce_官网
2. github_lettuce-core

<!-- spring-boot-starter-data-redis已经包含 -->
<dependency>
    <groupId>io.lettuce</groupId>
    <artifactId>lettuce-core</artifactId>
    <version>6.1.6.RELEASE</version>
</dependency>

/**
 * Basic Usage（支持同步）
 */
@Test
void lettuce_sync() {
    RedisClient client = RedisClient.create("redis://localhost:6379");
    StatefulRedisConnection<String, String> conn = client.connect();
    RedisStringCommands<String, String> sync = conn.sync();
    String value = sync.get("k1");
    System.out.println("value = " + value);
}

/**
 * Asynchronous API（支持异步）
 */
@Test
void lettuce_async() throws ExecutionException, InterruptedException {
    RedisClient client = RedisClient.create("redis://localhost:6379");
    StatefulRedisConnection<String, String> connection = client.connect();
    RedisStringAsyncCommands<String, String> async = connection.async();
    RedisFuture<String> get = async.get("key");

    String v = get.get();
    System.out.println("v = " + v);
}

3. Spring_data_redis(架构用)

1. SpringBoot_redis
2. SpringData_redis
3. SpringBoot_redis只是SpringData_redis的子集，它集成了Jedis、lettuce，直接使用这些对象
   1. 连接redis
   2. 高阶api，低阶api
   3. 序列化方式

1. redis_config

# 1.1. 获取redis所有配置
config get *
# 1.2. 获取redis所有配置
config get prote*

# 2. 允许远程访问，临时更改
config set protected-mode no

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-redis</artifactId>
    <version>2.6.3</version>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-json</artifactId>
    <version>2.6.3</version>
</dependency>

spring.redis.host=******
spring.redis.port=******
spring.redis.password=******

2. RedisTemplate

// 进行未知序列化
@Autowired
RedisTemplate<String, Object> redisTemplate;

@Test
void redis_Template() {
    // 高阶api
    // 1. key-value在redis里都乱码了，进行了未知序列化
    redisTemplate.opsForValue().set("k1", "v1");
    System.out.println(redisTemplate.opsForValue().get("k1"));

    // 低阶api
    // 2. 所有的redis命令，都可以获取到
    RedisConnection conn = redisTemplate.getConnectionFactory().getConnection();
    conn.set("k3".getBytes(), "v3".getBytes());
    System.out.println(new String(conn.get("k3".getBytes())));
}

3. StringRedisTemplate

• 源码key-value进行了StringRedisSerializer

public class StringRedisTemplate extends RedisTemplate<String, String> {

	/**
	 * Constructs a new <code>StringRedisTemplate</code> instance. {@link #setConnectionFactory(RedisConnectionFactory)}
	 * and {@link #afterPropertiesSet()} still need to be called.
	 */
	public StringRedisTemplate() {
		setKeySerializer(RedisSerializer.string());
		setValueSerializer(RedisSerializer.string());
		setHashKeySerializer(RedisSerializer.string());
		setHashValueSerializer(RedisSerializer.string());
	}

	/**
	 * Constructs a new <code>StringRedisTemplate</code> instance ready to be used.
	 *
	 * @param connectionFactory connection factory for creating new connections
	 */
	public StringRedisTemplate(RedisConnectionFactory connectionFactory) {
		this();
		setConnectionFactory(connectionFactory);
		afterPropertiesSet();
	}

	protected RedisConnection preProcessConnection(RedisConnection connection, boolean existingConnection) {
		return new DefaultStringRedisConnection(connection);
	}
}

@Autowired
private StringRedisTemplate stringRedisTemplate;

@Test
void string_RedisTemplate() {
    // 1. 默认进行了string序列化
    stringRedisTemplate.opsForValue().set("k2", "v2");
    System.out.println(stringRedisTemplate.opsForValue().get("k2"));
}

• StringRedisTemplate支持的数据类型

4. Obj

• Redis直接存取对象

1. opsForHash

@Test
void obj_hash() {
    // 存对象
    HashOperations<String, String, Object> hash = redisTemplate.opsForHash();
    hash.put("k1", "name", "listao");
    hash.put("k1", "age", 22);

    System.out.println(hash.entries("v1"));
}

2. 手动转hash

@Test
void flatten_true() {
    Address address = Address.builder()
            .country("china")
            .city("beijing")
            .build();
    Person p = Person.builder()
            .name("listao")
            .age(16)
            .address(address)
            .build();

    // pom.xml spring-boot-starter-json
    ObjectMapper objectMapper = new ObjectMapper();

    /*
     * 1 address.city       "beijing"
     * 2 name               "listao"
     * 3 age                18
     * 4 address.country    "china"
     */
    // flatten: 是否扁平
    Jackson2HashMapper jm = new Jackson2HashMapper(objectMapper, true);
    redisTemplate.opsForHash().putAll("p", jm.toHash(p));

    Map<String, Object> map = redisTemplate.<String, Object>opsForHash().entries("p");
    // 不能强转为Person
    Object per = jm.fromHash(map);
    // Person per = objectMapper.convertValue(map, Person.class);
    System.out.println("per = " + per);
}

@Test
void flatten_false() {
    Address address = Address.builder()
            .country("china")
            .city("beijing")
            .build();
    Person p = Person.builder()
            .name("listao")
            .age(16)
            .address(address)
            .build();

    ObjectMapper objectMapper = new ObjectMapper();

    /*
     * 1 name       "listao"
     * 2 age        16
     * 3 address    ["java.util.LinkedHashMap",{"country":"china","city":"beijing"}]
     */
    // flatten: 是否扁平
    Jackson2HashMapper jm = new Jackson2HashMapper(objectMapper, false);

    redisTemplate.opsForHash().putAll("p", jm.toHash(p));

    Map<String, Object> map = redisTemplate.<String, Object>opsForHash().entries("p");

    Person per = objectMapper.convertValue(map, Person.class);
    System.out.println("per = " + per);
}

3. jsonRedisTemplate

@Configuration
public class RedisConfig {

    /*
     * 1. 老版设置。没看源码前。redis中格式如下
     *   [
     *       "com.listao.redis.demo.Person",
     *       {
     *           "name": "listao",
     *           "age": 16,
     *           "address": [
     *               "com.listao.redis.demo.Address",
     *               {
     *                   "country": "china",
     *                   "city": "beijing"
     *               }
     *           ]
     *       }
     *   ]
     */
    // @Bean(name = "redisTemplateObj")
    public RedisTemplate<String, Object> redisTemplateObj(RedisConnectionFactory redisConnectionFactory) {
        RedisTemplate<String, Object> redisTemplate = new RedisTemplate<>();
        redisTemplate.setConnectionFactory(redisConnectionFactory);

        // 1. 使用Jackson2JsonRedisSerialize替换默认序列化
        Jackson2JsonRedisSerializer<Object> jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer<>(Object.class);

        ObjectMapper objectMapper = new ObjectMapper();
        objectMapper.setVisibility(PropertyAccessor.ALL, JsonAutoDetect.Visibility.ANY);
        objectMapper.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL);
        jackson2JsonRedisSerializer.setObjectMapper(objectMapper);

        StringRedisSerializer stringRedisSerializer = new StringRedisSerializer();

        // 2. 设置key, value的序列化规则
        redisTemplate.setKeySerializer(stringRedisSerializer);
        redisTemplate.setValueSerializer(jackson2JsonRedisSerializer);

        redisTemplate.setHashKeySerializer(stringRedisSerializer);
        redisTemplate.setHashValueSerializer(jackson2JsonRedisSerializer);
        redisTemplate.afterPropertiesSet();
        return redisTemplate;
    }

    /*
     * 2. 新版设置。看过源码后。redis中格式如下
     *   {
     *       "@class": "com.listao.redis.model.Person",
     *       "name": "listao",
     *       "age": 16,
     *       "address": {
     *           "@class": "com.listao.redis.model.Address",
     *           "country": "china",
     *           "city": "beijing"
     *       }
     *   }
     */
    @Bean(name = "jsonRedisTemplate")
    public RedisTemplate<String, Object> jsonRedisTemplate(RedisConnectionFactory redisConnectionFactory) {
        RedisTemplate<String, Object> redisTemplate = new RedisTemplate<>();
        redisTemplate.setConnectionFactory(redisConnectionFactory);

        // 1. 分别设置key, value序列化规则
        redisTemplate.setKeySerializer(RedisSerializer.string());
        redisTemplate.setValueSerializer(RedisSerializer.json());

        redisTemplate.setHashKeySerializer(RedisSerializer.string());
        redisTemplate.setHashValueSerializer(RedisSerializer.json());

        // 2. 将没有设置Serializer的key, value设置为默认的JdkSerializationRedisSerializer
        redisTemplate.afterPropertiesSet();
        return redisTemplate;
    }
}

@Test
void jsonRedisTemplate() {
    Address address = Address.builder()
            .country("china")
            .city("beijing")
            .build();
    Person p = Person.builder()
            .name("listao")
            .age(16)
            .address(address)
            .build();
    jsonRedisTemplateObj.opsForValue().set("k1", p);
    Person per = (Person) redisTemplate.opsForValue().get("k1");
    System.out.println("per = " + per);
}

5. pub_sub

1. 订阅端

/**
 * 1. 订阅端
 */
@Test
void sub() throws IOException {
    // 1. 订阅端
    RedisConnectionFactory connFactory = jsonRedisTemplate.getConnectionFactory();
    assert connFactory != null;

    RedisConnection rc = connFactory.getConnection();
    rc.subscribe(new MessageListener() {
        @Override
        public void onMessage(Message message, byte[] pattern) {
            byte[] body = message.getBody();
            System.out.println(new String(body));
        }
    }, "ooxx".getBytes());

    // 阻塞线程，等待发布消息
    System.in.read();
}

2. 发布端

/**
 * 2. 发布端
 */
@Test
void pub() {
    jsonRedisTemplate.convertAndSend("ooxx", "hello ooxx");
}