What are the application scenarios of Consul's Key-Value Store (KV Store)? How to use it

Consul's Key-Value Store (KV Store) is a powerful distributed configuration center that provides dynamic configuration management, service coordination, leader election, and other features.

Basic Concepts

Consul KV Store is a distributed key-value storage system similar to etcd, with the following characteristics:

• Distributed: Data is replicated across multiple Consul Server nodes
• Strong consistency: Data consistency guaranteed through Raft protocol
• Hierarchical structure: Supports hierarchical key structure similar to file systems
• Atomic operations: Supports transactions and atomic operations
• Version control: Each key has a version number and modification time

CLI Operations

Basic Read/Write Operations

bash
# Set key-value
consul kv put config/app/name "myapp"

# Get key-value
consul kv get config/app/name

# Get detailed key-value information
consul kv get -detailed config/app/name

# Delete key
consul kv delete config/app/name

# Recursive delete
consul kv delete -recurse config/

List and Query

bash
# List all keys
consul kv get -recurse

# List keys with specified prefix
consul kv get -recurse config/

# Export all key-values
consul kv export > backup.json

# Import key-values
consul kv import < backup.json

HTTP API Operations

Basic Operations

bash
# Set key-value
curl -X PUT -d "value" http://localhost:8500/v1/kv/config/app/name

# Get key-value
curl http://localhost:8500/v1/kv/config/app/name

# Delete key
curl -X DELETE http://localhost:8500/v1/kv/config/app/name

# Recursive get
curl http://localhost:8500/v1/kv/config/?recurse=true

Advanced Operations

bash
# CAS (Compare-And-Set) operation
curl -X PUT -d "newvalue" \
  http://localhost:8500/v1/kv/config/app/name?cas=10

# Get key metadata
curl http://localhost:8500/v1/kv/config/app/name?keys=true

# Get data after specified index
curl http://localhost:8500/v1/kv/config/?recurse=true&index=100

Application Scenarios

1. Dynamic Configuration Management

bash
# Store application configuration
consul kv put config/app/database/host "localhost"
consul kv put config/app/database/port "5432"
consul kv put config/app/database/user "admin"

# Read configuration when application starts
curl http://localhost:8500/v1/kv/config/app/database/host?raw

2. Service Coordination

Distributed Lock:

bash
# Acquire lock
consul kv put -acquire lock/service1 "owner1"

# Release lock
consul kv put -release lock/service1 "owner1"

# Set lock timeout
consul kv put -acquire -ttl=30s lock/service1 "owner1"

Leader Election:

bash
# Participate in election
consul kv put -acquire leader/election "node1"

# Check current leader
consul kv get leader/election

3. Feature Flags

bash
# Enable new feature
consul kv put features/new_feature "true"

# Read feature flag
curl http://localhost:8500/v1/kv/features/new_feature?raw

4. Service Metadata

bash
# Store service metadata
consul kv put metadata/service1/version "1.0.0"
consul kv put metadata/service1/deploy_time "2024-01-01"

Watch Mechanism

Consul KV supports Watch mechanism to monitor key-value changes:

bash
# Watch key changes
consul watch -type=key -key=config/app/name /bin/cat

# Watch prefix changes
consul watch -type=keyprefix -prefix=config/ /bin/cat

Transaction Operations

Consul supports atomic transaction operations:

json
{
  "KV": [
    {
      "Verb": "set",
      "Key": "config/app/host",
      "Value": "aGVsbG8="
    },
    {
      "Verb": "set",
      "Key": "config/app/port",
      "Value": "ODA4MA=="
    }
  ]
}

bash
curl -X PUT -d @transaction.json http://localhost:8500/v1/txn

ACL Permission Control

Consul KV supports ACL (Access Control List) for permission control:

bash
# Create policy
consul acl policy create -name kv-policy -rules @kv-policy.hcl

# kv-policy.hcl content
key_prefix "config/" {
  policy = "read"
}

key_prefix "secrets/" {
  policy = "deny"
}

Best Practices

1. Key Naming Convention

shell
# Use hierarchical structure
config/{service}/{environment}/{key}
metadata/{service}/{key}
locks/{resource}/{id}

2. Data Version Management

bash
# Use version numbers
config/app/v1/database/host
config/app/v2/database/host

# Use timestamps
config/app/20240101/database/host

3. Data Backup

bash
# Regular backup
consul kv export > backup_$(date +%Y%m%d).json

# Restore data
consul kv import < backup_20240101.json

4. Monitoring and Alerting

bash
# Monitor key-value changes
consul watch -type=key -key=secrets/ /usr/local/bin/alert.sh

Integration with Other Tools

Spring Cloud Consul

java
@RefreshScope
@RestController
public class ConfigController {
    
    @Value("${app.name}")
    private String appName;
    
    @GetMapping("/config")
    public String getConfig() {
        return appName;
    }
}

Go Consul SDK

go
kv := client.KV()
pair, _, err := kv.Get("config/app/name", nil)
if err != nil {
    log.Fatal(err)
}
fmt.Printf("Value: %s\n", string(pair.Value))

Consul KV Store is a simple yet powerful distributed configuration center suitable for various microservice architecture configuration management needs.