SSH troubleshooting is an essential skill for operations engineers. Mastering common SSH connection problems and solutions enables quick problem identification and resolution.
Common Connection Problems
1. Connection Timeout
Symptoms: Connection request hangs without response
Possible Causes:
- Network unreachable
- Firewall blocking
- SSH service not running
- Port configuration error
Troubleshooting Steps:
bash# 1. Test network connectivity ping server.example.com # 2. Test if port is open telnet server.example.com 22 # Or use nc nc -zv server.example.com 22 # 3. Check local firewall sudo iptables -L -n | grep 22 # 4. Check server firewall ssh user@server "sudo iptables -L -n | grep 22" # 5. Check SSH service status ssh user@server "systemctl status sshd" # Or ssh user@server "service ssh status"
Solutions:
bash# Modify port configuration # /etc/ssh/sshd_config Port 2222 # Restart SSH service systemctl restart sshd # Configure firewall rules sudo iptables -A INPUT -p tcp --dport 2222 -j ACCEPT
2. Authentication Failure
Symptoms: "Permission denied (publickey,password)" message
Possible Causes:
- Key mismatch
- Key permission errors
- Server configuration issues
- Wrong username
Troubleshooting Steps:
bash# 1. Verbose debugging information ssh -vvv user@server # 2. Check local keys ls -l ~/.ssh/ cat ~/.ssh/id_rsa.pub # 3. Check server authorized keys ssh user@server "cat ~/.ssh/authorized_keys" # 4. Check key permissions ssh user@server "ls -l ~/.ssh/" # 5. Test key authentication ssh -i ~/.ssh/id_rsa user@server
Solutions:
bash# Fix local key permissions chmod 700 ~/.ssh chmod 600 ~/.ssh/id_rsa chmod 644 ~/.ssh/id_rsa.pub # Fix server permissions ssh user@server "chmod 700 ~/.ssh" ssh user@server "chmod 600 ~/.ssh/authorized_keys" # Re-add public key ssh-copy-id -i ~/.ssh/id_rsa.pub user@server # Check server configuration ssh user@server "grep -i pubkey /etc/ssh/sshd_config"
3. Host Key Verification Failed
Symptoms: "WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!" message
Possible Causes:
- Server reinstalled
- IP address reused
- Man-in-the-middle attack
Troubleshooting Steps:
bash# 1. View host keys ssh-keygen -l -f ~/.ssh/known_hosts # 2. View server host key ssh-keyscan -H server.example.com # 3. Compare key fingerprints ssh-keygen -lf /etc/ssh/ssh_host_rsa_key.pub
Solutions:
bash# Remove old host key ssh-keygen -R server.example.com # Or manually edit known_hosts vim ~/.ssh/known_hosts # Reconnect and accept new key ssh user@server
4. Connection Dropped
Symptoms: Connection suddenly drops during use
Possible Causes:
- Network instability
- Firewall timeout
- Server resource limits
- Keep-alive configuration issues
Troubleshooting Steps:
bash# 1. Check network stability ping -i 1 server.example.com # 2. Check server logs ssh user@server "tail -f /var/log/auth.log" # 3. Check system resources ssh user@server "free -h" ssh user@server "df -h"
Solutions:
bash# Client configuration # ~/.ssh/config Host * ServerAliveInterval 60 ServerAliveCountMax 3 # Server configuration # /etc/ssh/sshd_config ClientAliveInterval 300 ClientAliveCountMax 3 # Use autossh to maintain connection autossh -M 0 -o "ServerAliveInterval 60" -o "ServerAliveCountMax 3" user@server
Advanced Troubleshooting
1. Use Verbose Logging
bash# Client verbose logging ssh -vvv user@server # Server verbose logging # /etc/ssh/sshd_config LogLevel VERBOSE # View logs tail -f /var/log/auth.log
2. Test Specific Configuration
bash# Test configuration file syntax sshd -t # View effective configuration sshd -T | grep -i password # Test specific options ssh -o PreferredAuthentications=publickey user@server
3. Network Layer Diagnostics
bash# Trace route traceroute server.example.com # Check DNS resolution nslookup server.example.com dig server.example.com # Check MTU ping -M do -s 1472 server.example.com
4. Performance Analysis
bash# Measure connection time time ssh user@server "echo 'test'" # Analyze network latency ping -c 10 server.example.com # Check bandwidth iperf3 -c server.example.com
Common Troubleshooting Commands
Connection Testing
bash# Basic connection test ssh user@server # Test with specific port ssh -p 2222 user@server # Test with specific key ssh -i ~/.ssh/custom_key user@server # Test disabling specific authentication method ssh -o PreferredAuthentications=password user@server
Status Checking
bash# Check SSH service status systemctl status sshd service ssh status # Check listening ports netstat -tuln | grep :22 ss -tuln | grep :22 # Check processes ps aux | grep sshd
Log Analysis
bash# View authentication logs tail -f /var/log/auth.log tail -f /var/log/secure # View failed logins lastb -n 20 # View successful logins last -n 20 # Search for errors grep "sshd" /var/log/auth.log | grep -i error
Troubleshooting Flowchart
shellConnection Failed ↓ Test Network Connectivity (ping) ↓ Test Port Open (telnet/nc) ↓ Check SSH Service Status ↓ Check Firewall Rules ↓ Verbose Debugging (ssh -vvv) ↓ Check Authentication Configuration ↓ Check Key Permissions ↓ Check Server Logs ↓ Resolve Problem
Preventive Measures
1. Configure Monitoring
bash# Monitor SSH service systemctl enable sshd # Monitor logs tail -f /var/log/auth.log | grep sshd # Set up alerts # Use fail2ban for automatic banning
2. Regular Maintenance
bash# Regularly update SSH apt-get update && apt-get upgrade openssh-server # Regularly check configuration sshd -t # Regularly clean logs logrotate /etc/logrotate.d/ssh
3. Backup Configuration
bash# Backup configuration files cp /etc/ssh/sshd_config /etc/ssh/sshd_config.bak # Backup keys cp -r ~/.ssh ~/.ssh.bak # Backup known hosts cp ~/.ssh/known_hosts ~/.ssh/known_hosts.bak
Recommended Tools
1. Diagnostic Tools
- ssh-keyscan: Get host keys
- ssh-keygen: Key management
- autossh: Auto-reconnect
- mosh: Mobile SSH client
2. Monitoring Tools
- fail2ban: Prevent brute force attacks
- logwatch: Log analysis
- nagios: Service monitoring
- zabbix: Comprehensive monitoring
3. Network Tools
- tcpdump: Packet capture
- wireshark: Network analysis
- nmap: Port scanning
- mtr: Network diagnostics
Best Practices
- Enable Verbose Logging: Facilitate problem tracking
- Regularly Check Configuration: Ensure correct configuration
- Monitor Service Status: Detect anomalies early
- Backup Important Configuration: Quick recovery
- Use Version Control: Manage configuration changes
- Document Problems: Accumulate experience
- Automated Testing: Verify configuration
- Establish Emergency Plans: Quick response
Mastering SSH troubleshooting skills enables quick problem identification and resolution, improving work efficiency.