热Key与BigKey:发现、分析与解决方案

热Key问题 定义:访问频率极高的key,导致单个Redis节点负载过高 危害: 单节点CPU 100% 网络带宽打满 影响其他key访问 Cluster集群数据倾斜 发现热Key 方法1:redis-cli –hotkeys redis-cli --hotkeys # 统计访问频率最高的key 方法2:monitor命令 redis-cli monitor | head -n 100000 | awk '{print $4}' | sort | uniq -c | sort -rn | head -n 10 方法3:代码统计 @Aspect @Component public class RedisMonitorAspect { private ConcurrentHashMap<String, AtomicLong> accessCounter = new ConcurrentHashMap<>(); @Around("execution(* org.springframework.data.redis.core.RedisTemplate.opsFor*(..))") public Object around(ProceedingJoinPoint pjp) throws Throwable { Object result = pjp.proceed(); // 统计key访问次数 if (result != null) { String key = extractKey(pjp); if (key != null) { accessCounter.computeIfAbsent(key, k -> new AtomicLong()).incrementAndGet(); } } return result; } @Scheduled(fixedRate = 60000) // 每分钟 public void reportHotKeys() { List<Map.Entry<String, AtomicLong>> hotKeys = accessCounter.entrySet().stream() .sorted(Map.Entry.<String, AtomicLong>comparingByValue().reversed()) .limit(10) .collect(Collectors.toList()); log.info("热Key TOP 10: {}", hotKeys); // 清空统计 accessCounter.clear(); } } 解决方案 方案1:本地缓存 @Service public class ProductService { @Autowired private RedisTemplate<String, Object> redis; // 本地缓存热点数据 private Cache<String, Product> localCache = Caffeine.newBuilder() .maximumSize(1000) .expireAfterWrite(5, TimeUnit.MINUTES) .build(); public Product getProduct(Long productId) { String key = "product:" + productId; // 1. 本地缓存 Product product = localCache.getIfPresent(key); if (product != null) { return product; } // 2. Redis product = (Product) redis.opsForValue().get(key); if (product != null) { localCache.put(key, product); return product; } // 3. DB product = productMapper.selectById(productId); if (product != null) { redis.opsForValue().set(key, product, 3600, TimeUnit.SECONDS); localCache.put(key, product); } return product; } } 方案2:热Key备份 // 热Key复制多份,随机访问 public Object getHotKey(String key) { // 随机选择一个备份 int index = ThreadLocalRandom.current().nextInt(10); String backupKey = key + ":backup:" + index; Object value = redis.opsForValue().get(backupKey); if (value != null) { return value; } // 备份不存在,查询原key value = redis.opsForValue().get(key); if (value != null) { // 更新备份 redis.opsForValue().set(backupKey, value, 3600, TimeUnit.SECONDS); } return value; } 方案3:读写分离 // 读从节点,写主节点 @Configuration public class RedisConfig { @Bean public LettuceConnectionFactory writeConnectionFactory() { return new LettuceConnectionFactory(new RedisStandaloneConfiguration("master", 6379)); } @Bean public LettuceConnectionFactory readConnectionFactory() { return new LettuceConnectionFactory(new RedisStandaloneConfiguration("slave", 6379)); } } BigKey问题 定义:占用内存过大的key ...

2025-01-21 · maneng

如约数科科技工作室

浙ICP备2025203501号

👀 本站总访问量 ...| 👤 访客数 ...| 📅 今日访问 ...