Using the same SSH Key for All Servers and Devices; Securing Your Server from Insecure Logins

I was getting sick of the mess of SSH keys I had for all of my devices and servers, so I started on making all my devices use the same set of keys so I can have one key for everything.

You do NOT want to delete any of your old keys, if you have any, until you get all of them configured with the key you are trying to use.

Generating a New Key

First of all, you want to generate a key. I did it on a Linux PC to avoid any oddness with Windows, but it may work fine. To do this, you go into the Linux terminal and type:

ssh-keygen

It will prompt you for a file name. I used a different file to avoid conflicts with old keys. You want it to be in your home directory, so enter in:

~/.ssh/[keyname] #only write in a name; no file type

Next, it will prompt you to enter a passphrase. If you need the extra security, you can go ahead; otherwise, just hit Enter twice.

Moving the New Key to a Server

You should now have a new SSH key. To move it to your new servers, you have to copy the public key onto the server. To do so, run this command:

# if default file name
ssh-copy-id [server_user]@[server_domain]
# if custom file name
ssh-copy-id -i ~/.ssh/[keyname].pub [server_user]@[server_domain]

To make sure it works, you’ll want to specify the private key you created in the SSH command. This will be what you put in the .ssh folder for the devices you connect to the server with.

# if default file name
ssh -i ~/.ssh/id_rsa [server_user]@[server_domain]
# if custom file name
ssh -i ~/.ssh/[keyname] [server_user]@[server_domain]

If this works, you should be good to remove password logins on your server by following the section below.

Securing the Server from Unsecured Logins

To remove unsecured logins without an SSH key, you will have to edit the ssh config file. Make sure you are super user before adding the PermitRootLogin line, which you can test with sudo su. You will want to change or add these lines into your /etc/ssh/sshd_config:

PermitRootLogin no #if sudo user
PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM no
PubkeyAuthentication yes

From here, you can reboot your server, then reconnect to it with the ssh command above. If you are using the default id_rsa, you can just do ssh without the -i argument.