Easy Tutorial
❮ Nginx Setup Intro Verilog2 Real2Int ❯

Detailed Solutions for Slow SSH Connections

Category Programming Technology

Every time I use PuTTY to SSH into a remote Linux system for management, the login process is extremely slow—after entering the username, it takes about 30 seconds before the prompt to enter the password appears. This situation is truly unbearable, especially when quick response is needed in problem handling.

However, after some specific tests, I found that this is not a universal issue for all systems. The problematic machines are mainly concentrated on CentOS, while Debian systems connect remotely without any lag or hesitation. Could this be a CentOS issue?

Out of curiosity, I looked into the differences between the two systems during SSH

CentOS:

ssh -v [email protected]

The information displayed during SSH remote login is as follows:

OpenSSH_6.0p1 Debian-4, OpenSSL 1.0.1e 11 Feb 2013
...Some sensitive information...
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.3
debug1: match: OpenSSH_5.3 pat OpenSSH_5*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.0p1 Debian-4
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
...Some sensitive information...
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Next authentication method: gssapi-keyex
debug1: No valid Key exchange context
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure.  Minor code may provide more information
Cannot determine realm for numeric host address

debug1: Unspecified GSS failure.  Minor code may provide more information
Cannot determine realm for numeric host address

debug1: Unspecified GSS failure.  Minor code may provide more information

debug1: Unspecified GSS failure.  Minor code may provide more information
Cannot determine realm for numeric host address

debug1: Next authentication method: publickey
debug1: Trying private key: /home/mitchellchu/.ssh/id_rsa
debug1: Trying private key: /home/mitchellchu/.ssh/id_dsa
debug1: Trying private key: /home/mitchellchu/.ssh/id_ecdsa
debug1: Next authentication method: password

And Debian, using the same command, yields:

OpenSSH_6.0p1 Debian-4, OpenSSL 1.0.1e 11 Feb 2013
...Some sensitive information...
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.0p1 Debian-4
debug1: match: OpenSSH_6.0p1 Debian-4 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.0p1 Debian-4
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: sending SSH2_MSG_KEX_ECDH_INIT
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
...Some sensitive information...
debug1: ssh_ecdsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Trying private key: /home/mitchellchu/.ssh/id_rsa
debug1: Trying private key: /home/mitchellchu/.ssh/id_dsa
debug1: Trying private key: /home/mitchellchu/.ssh/id_ecdsa
debug1: Next authentication method: password

From the above, we can see that CentOS uses publickey, gssapi-keyex, gssapi-with-mic, and password for authentication (marked line 23), while Debian uses only Publickey and password. Connecting to CentOS takes a significant amount of time at line 23. Looking further down, we can clearly see the following information:

# Using GSSAPI-KEYEX for verification
debug1: Next authentication method: gssapi-keyex
# Error: No valid Key exchange context
debug1: No valid Key exchange context
# Next authentication method: GSSAPI-WITH-MIC
debug1: Next authentication method: gssapi-with-mic
# Unfortunately, GSSAPI-WITH-MIC also fails.
# Reason: Cannot determine realm for numeric host address
debug1: Unspecified GSS failure.  Minor code may provide more information
Cannot determine realm for numeric host address

debug1: Unspecified GSS failure.  Minor code may provide more information
Cannot determine realm for numeric host address

debug1: Unspecified GSS failure.  Minor code may provide more information

debug1: Unspecified GSS failure.  Minor code may provide more information
Cannot determine realm for numeric host address
# After several attempts, SSH authentication finally gives up on this verification. Proceeds to the next: Publickey
debug1: Next authentication method: publickey

Are there other methods? Of course, there are. CentOS actually provides us with a solution—disable GSSAPI authentication when using SSH for remote login. Additionally, if your machine has UseDNS enabled, it should also be disabled; see the final notes for details.

The error suggests that the issue is related to the host domain—presumably, the inability to confirm the domain corresponding to the IP address. GSSAPI is mainly based on Kerberos, so to solve this problem, the system would need to be configured with Kerberos. However, configuring Kerberos just to resolve a login delay issue might not be a wise decision, especially in a production environment. Minimizing the requirements to meet the need is the key.

Here are the methods to handle GSSAPI

Disabling GSSAPI authentication can be done in two ways: on the client side and the server side.

1. Client-side disablement

This is relatively simple and affects only the individual client user. You can achieve this with the following method:

ssh -o GSSAPIAuthentication=no your-server-username@serverIP

Using the above method to log in remotely will disable GSSAPIAuthentication.

If you find it cumbersome, you can directly configure the ssh client file /etc/ssh/ssh_config to permanently solve this issue:

vi /etc/ssh/ssh_config
### Find the line GSSAPIAuthentication yes in the ssh_config file

Modify to GSSAPIAuthentication no

Save the ssh_config file and exit

This modification affects all users on the machine. If you want to disable GSSAPIAuthentication for a specific user without affecting everyone, you can add a config file in the user's .ssh directory with the following line:

cat >>~/.ssh/config<&lt;EOF
GSSAPIAuthentication no
EOF

Using cat, this directly outputs to the file. Now, when you connect to a remote host using ssh, it will no longer use GSSAPI authentication.

These files are on the client side, not the server side. This means your client must also be Linux to modify these files.

If you are using a client tool like PuTTY on Windows, you should not use the above method. Instead, you can try setting this up before connecting:

>

PuTTY Configuration -> Connection -> SSH -> Auth -> GSSAPI -> (Uncheck) Attempt GSSAPI authentication (SSH-2 only)

If GSSAPIAuthentication is not disabled in PuTTY, you can right-click (or Ctrl + right-click) in the connection window to view the logs, which will show PuTTY attempting GSSAPI connections:

2014-05-18 23:46:54 Using SSPI from SECUR32.DLL
2014-05-18 23:46:54 Attempting GSSAPI authentication
2014-05-18 23:46:54 GSSAPI authentication request refused

These methods are generally applicable.

2. If you have configured Kerberos

You can also try the following client-side solution:

Add the remote host's hostname to your local hosts file (Linux: /etc/hosts, Windows: System Drive:/Windows/System32/drivers/etc/hosts). Both Linux and Windows can add the following line:

### Note: Replace IP-Addr with your remote machine's IP address, and HostName with the hostname
IP-Addr HostName

After adding, save and exit.

If you have not configured Kerberos, simply configuring the hosts file will not solve the problem. When logging in with ssh, you may see error logs similar to the following:

debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Next authentication method: gssapi-keyex
debug1: No valid Key exchange context
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure.  Minor code may provide more information
Credentials cache file '/tmp/krb5cc_0' not found

debug1: Unspecified GSS failure.  Minor code may provide more information
Credentials cache file '/tmp/krb5cc_0' not found

debug1: Unspecified GSS failure.  Minor code may provide more information

debug1: Unspecified GSS failure.  Minor code may provide more information
Credentials cache file '/tmp/krb5cc_0' not found

debug1: Next authentication method: publickey

This error is common and needs attention.

3. Disable GSSAPIAuthentication on the server side.

Go to /etc/ssh/sshd_config and change GSSAPIAuthentication yes to no. Also, note that you may need to change UseDNS to UseDNS no (this varies by system, here using CentOS 6 as an example):

sudo vi /etc/ssh/sshd_config
### Regular user permissions are not sufficient, root permissions are needed
### Find GSSAPIAuthentication yes, modify to
### GSSAPIAuthentication no
### Note, you also need to change UseDNS to no, as CentOS defaults to yes, even if this line is commented out, you need to add
### UseDNS no
### Some say UseDNS yes does not need to be changed to UseDNS no, but Mitchell's testing indicates it is necessary.
### Save the file, exit

After disabling, we need to restart the SSH service to ensure the new configuration file is correctly applied:

service sshd restart

Now, when you log in to this host using SSH, does it feel much faster?

Phew, finally finished this long article. It was challenging to write this while experimenting. However, this should cover the issue thoroughly. I hope you find it clear and informative. Feel free to discuss.

Explanation

  1. GSSAPI: Generic Security Services Application Program Interface, GSSAPI is a set of APIs standardized by IETF. Its most famous implementation is based on Kerberos. Generally, when mentioning GSSAPI, it implies the Kerberos implementation.

  2. UseDNS: This is a DNS lookup option on the OpenSSH server, which is turned on by default. When enabled, the server automatically performs a DNS PTR reverse lookup on the client's IP to find the corresponding hostname, then performs a DNS forward A record lookup based on the client's hostname to verify the IP matches the connecting client's IP. However, most of our machines dynamically obtain IP addresses, meaning this option is largely useless—even for static IP servers, if there is no reverse IP resolution, it is difficult to apply. If you fall into these categories, it is recommended to turn off UseDNS to speed up SSH remote login authentication.

Source: http://blog.useasp.net/archive/2014/05/19/solved-the-problem-of-ssh-client-such-as-putty-remote-login-linux-very-slowly.aspx

** Click to share notes

Cancel

-

-

-

❮ Nginx Setup Intro Verilog2 Real2Int ❯