How to monitor and operate Redis? What are the key metrics and tools? - 面试题

Redis monitoring and operations are important links to ensure stable Redis operation, requiring monitoring and management from multiple dimensions.

1. Redis Monitoring Metrics

Basic Metrics

Memory Usage:

bash
# View memory usage
INFO memory

# Key metrics
used_memory:1024000              # Used memory
used_memory_human:1.00M          # Used memory (human readable)
used_memory_rss:2048000          # OS allocated memory
used_memory_rss_human:2.00M      # OS allocated memory (human readable)
used_memory_peak:2048000          # Historical memory usage peak
used_memory_peak_human:2.00M      # Historical memory usage peak (human readable)
maxmemory:1073741824             # Maximum memory limit
maxmemory_human:1.00G             # Maximum memory limit (human readable)
mem_fragmentation_ratio:2.00        # Memory fragmentation ratio

Connection Status:

bash
# View connection status
INFO clients

# Key metrics
connected_clients:10               # Connected client count
blocked_clients:0                  # Blocked client count
client_longest_output_list:0       # Longest output list
client_biggest_input_buf:0         # Largest input buffer

Command Execution:

bash
# View command execution
INFO commandstats

# Key metrics
cmdstat_get:calls=1000,usec=5000,usec_per_call=5.00
cmdstat_set:calls=500,usec=2500,usec_per_call=5.00

Persistence Status:

bash
# View persistence status
INFO persistence

# Key metrics
rdb_last_save_time:1234567890    # Last RDB save time
rdb_changes_since_last_save:100    # Changes since last save
aof_enabled:1                      # AOF enabled
aof_rewrite_in_progress:0           # AOF rewrite in progress

Performance Metrics

QPS (Queries Per Second):

bash
# Calculate QPS
INFO stats

# Key metrics
instantaneous_ops_per_sec:1000     # Current operations per second
total_commands_processed:1000000    # Total commands processed
total_connections_received:10000      # Total connections received

Latency:

bash
# View latency
INFO stats

# Key metrics
instantaneous_input_kbps:100       # Current input bandwidth
instantaneous_output_kbps:200      # Current output bandwidth

Hit Rate:

bash
# Calculate hit rate
keyspace_hits:10000                # Hit count
keyspace_misses:1000               # Miss count
hit_rate = keyspace_hits / (keyspace_hits + keyspace_misses)

2. Redis Monitoring Tools

Redis Built-in Commands

INFO Command:

bash
# View all information
INFO

# View specific information
INFO memory
INFO stats
INFO replication
INFO persistence

SLOWLOG Command:

bash
# View slow queries
SLOWLOG GET 10

# View slow query count
SLOWLOG LEN

# Clear slow queries
SLOWLOG RESET

MONITOR Command:

bash
# Monitor Redis commands in real-time
MONITOR

Third-party Monitoring Tools

Redis Exporter:

bash
# Install Redis Exporter
docker run -d --name redis-exporter \
  -e REDIS_ADDR=redis://localhost:6379 \
  prom/redis-exporter

# Configure Prometheus to scrape data
scrape_configs:
  - job_name: 'redis'
    static_configs:
      - targets: ['localhost:9121']

Grafana + Prometheus:

bash
# Use Grafana to display Redis monitoring data
# Import Redis Dashboard
https://grafana.com/grafana/dashboards/11835-redis-dashboard/

Redis Insight:

bash
# Redis official visualization tool
# Download URL
https://redis.com/redis-enterprise/redis-insight/

3. Redis Operations

Backup and Recovery

RDB Backup:

bash
# Manually trigger RDB backup
SAVE
# or
BGSAVE

# Backup file location
/var/lib/redis/dump.rdb

# Restore RDB backup
# Stop Redis
redis-cli shutdown

# Copy backup file
cp dump.rdb /var/lib/redis/dump.rdb

# Start Redis
redis-server /etc/redis/redis.conf

AOF Backup:

bash
# Backup AOF file
cp /var/lib/redis/appendonly.aof /backup/appendonly.aof

# Restore AOF backup
# Stop Redis
redis-cli shutdown

# Copy backup file
cp /backup/appendonly.aof /var/lib/redis/appendonly.aof

# Start Redis
redis-server /etc/redis/redis.conf

Data Migration

Master-Slave Migration:

bash
# Configure master on slave node
redis-cli slaveof <master-ip> <master-port>

# Wait for sync completion
redis-cli info replication

# Cancel master-slave relationship
redis-cli slaveof no one

Using MIGRATE Command:

bash
# Migrate single key
MIGRATE <target-host> <target-port> <key> <target-database> <timeout>

# Migrate multiple keys
MIGRATE <target-host> <target-port> "" <target-database> <timeout> KEYS key1 key2 key3

Using Redis-Shake:

bash
# Install Redis-Shake
wget https://github.com/alibaba/RedisShake/releases/download/v2.0.3/redis-shake-v2.0.3.tar.gz
tar -xzf redis-shake-v2.0.3.tar.gz

# Configuration file
cat > shake.conf << EOF
source.type: standalone
source.address: source.redis.com:6379
source.password: source_password
target.type: standalone
target.address: target.redis.com:6379
target.password: target_password
EOF

# Start migration
./redis-shake.linux -type=sync -conf=shake.conf

Cluster Scaling

Add Node:

bash
# Add new node
redis-cli --cluster add-node <new-node-ip>:<new-node-port> <existing-node-ip>:<existing-node-port>

# Allocate hash slots
redis-cli --cluster reshard <existing-node-ip>:<existing-node-port> \
  --cluster-from <node-id> \
  --cluster-to <new-node-id> \
  --cluster-slots 1000

Remove Node:

bash
# Migrate hash slots
redis-cli --cluster reshard <existing-node-ip>:<existing-node-port> \
  --cluster-from <node-id> \
  --cluster-to <other-node-id> \
  --cluster-slots 1000

# Remove node
redis-cli --cluster del-node <existing-node-ip>:<existing-node-port> <node-id>

4. Redis Troubleshooting

Insufficient Memory

Problem Symptoms:

Redis error: OOM command not allowed when used memory > 'maxmemory'
Redis performance degradation

Troubleshooting Steps:

bash
# View memory usage
INFO memory

# View big keys
redis-cli --bigkeys

# View memory fragmentation ratio
INFO memory | grep mem_fragmentation_ratio

Solutions:

bash
# Adjust maximum memory limit
CONFIG SET maxmemory 2gb

# Enable memory eviction policy
CONFIG SET maxmemory-policy allkeys-lru

# Clean memory fragmentation
MEMORY PURGE

Too Many Connections

Problem Symptoms:

Redis error: max number of clients reached
Client connection failure

Troubleshooting Steps:

bash
# View connection count
INFO clients

# View maximum connection count
CONFIG GET maxclients

Solutions:

bash
# Adjust maximum connection count
CONFIG SET maxclients 10000

# Close idle connections
CONFIG SET timeout 300

Slow Queries

Problem Symptoms:

Slow Redis response
Increased slow query logs

Troubleshooting Steps:

bash
# View slow queries
SLOWLOG GET 10

# View slow query configuration
CONFIG GET slowlog-log-slower-than
CONFIG GET slowlog-max-len

Solutions:

bash
# Adjust slow query threshold
CONFIG SET slowlog-log-slower-than 10000

# Optimize slow query commands
# Avoid KEYS command, use SCAN
# Avoid O(n) complexity commands
# Use Pipeline to reduce network round trips

Master-Slave Sync Delay

Problem Symptoms:

Slave data lag
Reading old data

Troubleshooting Steps:

bash
# View master-slave sync status
INFO replication

# View sync delay
master_repl_offset:1000000
slave_repl_offset:999000

Solutions:

bash
# Adjust replication buffer size
CONFIG SET repl-backlog-size 10mb

# Adjust replication timeout
CONFIG SET repl-timeout 60

# Optimize network latency
# Use faster network
# Reduce distance between master and slave nodes

5. Redis Performance Optimization

Configuration Optimization

Memory Optimization:

bash
# Set maximum memory
maxmemory 2gb

# Set memory eviction policy
maxmemory-policy allkeys-lru

# Disable THP
echo never > /sys/kernel/mm/transparent_hugepage/enabled

Persistence Optimization:

bash
# RDB configuration
save 900 1
save 300 10
save 60 10000

# AOF configuration
appendonly yes
appendfsync everysec
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb

Network Optimization:

bash
# Set TCP backlog
tcp-backlog 511

# Set TCP keepalive
tcp-keepalive 300

# Set timeout
timeout 300

Operations Optimization

Regular Backup:

bash
# Regularly backup RDB file
0 2 * * * cp /var/lib/redis/dump.rdb /backup/dump_$(date +\%Y\%m\%d).rdb

# Regularly backup AOF file
0 3 * * * cp /var/lib/redis/appendonly.aof /backup/appendonly_$(date +\%Y\%m\%d).aof

Monitoring Alerts:

bash
# Configure monitoring alerts
# Alert when memory usage exceeds 80%
# Alert when QPS drops more than 50%
# Alert when latency exceeds 100ms
# Alert when connection count exceeds 80%

Summary

Redis monitoring and operations are important links to ensure stable Redis operation. Monitoring and management are required from multiple dimensions including basic metrics, performance metrics, monitoring tools, operations, troubleshooting, and performance optimization. In actual applications, it's necessary to establish a comprehensive monitoring system to discover and solve problems in time, ensuring Redis stability and performance.