Reverse SSH

In this article, I will demonstrate how to bypass firewall to access the server behind the firewall and NAT

How does it work?

Reverse SSH provides you a method to access the server behind NAT and Firewall. Its theory is very simple, server make a connection to client, and client use the connection to establish SSH connection. That's it.

Given a scenario in which we have a host behind NAT and firewall, how can we access to in from internet?

Assume Client is the host machine behind the firewall and Server is a machine in the internet.

In this circumstance, the Client can make a reverse ssh call to Server and then setup a tunnel to let Server to make a callback ssh to Client via a reverse port.

First Step: Generate the Key

We need to generate a key for ssh access so that when Client access Server to establish the tunnel in the background, there is no password input is required.

Note, if you are using the private key to access the cloud Server, you do not need to take this step.

Client generate private key, public key (ssh-keygen -t rsa) and add its public key to Server ~/.ssh/authorized_key

$ cat ~/id_rsa.pub >> ~/.ssh/authorized_keys

You can use scp to transfer the file between them. DO NOT transfer private key and please keep it secret.

# -P option is specify port, if you are not yet set-up  reverse ssh,,
# you can ignore it
$ sudo scp -P 12345 ./id_rsa.pub <>@<domain>:/home/

Reference

• Git - 生成 SSH 公開金鑰
• SSH/autossh不输入密码连接远程Linux主机
• 使用 autossh 建立反向 SSH 连接 · 不立不破

Second Step: Set-up Server Environment

We can now test if reverse ssh can be establish by entering the following command

Try to make a ssh call to Server from Client

$ ssh -NfR 12345:localhost:22 <server username>@<domain>

Once the reverse tunnel is established, you can now make a ssh call in Server to connect Client

$ ssh <client username>@localhost -p 12345

Next, we need to install autossh , it helps to monitor the ssh connection and resume it to normal if there is any accident on it. Use this command:

$ autossh -M 20000 -f -N -R 12345:localhost:22 <username>@<domain>

Last Step: Work as daemon

The previous step is perfect to implement but we need it to run in the background and even the server is auto-restart. To make this happen, we have to create a background services for it.

1. Create a systemd file using nano or vim or appropriate editor of choice

$ sudo vim /etc/systemd/system/autossh-cloud.service

2. Add the following contents

[Unit]
Description=AutoSSH service for remote tunnel
After=network-online.target

[Service]
User=ataluser
Environment="AUTOSSH_GATETIME=0"
ExecStart=/usr/bin/autossh -M 20001 -N -o "StrictHostKeyChecking=false" -o "ServerAliveInterval 60" -o "ServerAliveCountMax 3" -R 12345:localhost:22 <username>@<domain>
Restart=always

[Install]
WantedBy=multi-user.target

3. Reload systemd

$ sudo systemctl daemon-reload

4. Start the autossh service

$ sudo systemctl start autossh-cloud.service

5. Enable at boot

$ sudo systemctl enable autossh-cloud.service

6. check status with

$ sudo systemctl status autossh-cloud

There is however an important thing to note about systemd and AutoSSH: -f (background usage) already implies AUTOSSH_GATETIME=0, however -f is not supported by systemd. So in the case of systemd you need to make use of AUTOSSH_GATETIME

Reference

• SSH TUNNELLING FOR FUN AND PROFIT: AUTOS
• systemd.service
• Reverse SSH Tunnel 反向打洞實錄
• My reverse-ssh tunnel is using keepalives but they’re not helping
• Automatically restart SSH sessions and tunnels
• linux - How to reliably keep an SSH tunnel open?

Troubleshoot

Show All Running Processes in Linux

$ ps -aux | grep ssh

• A : Select all processes

• u : Select all processes on a terminal, including those of other users

• x : Select processes without controlling ttys

Kill Process

$ sudo kill -9 PID

Check network status (Port is listening)

$ sudo netstat -ntlp

Issues

As mentioned before, do not apply -f in .services file, otherwise you will trigger this exception:

autossh-cloud.service - AutoSSH service for remote tunnel
   Loaded: loaded (/etc/systemd/system/autossh-cloud.service; enabled; vendor preset: enabled)
   Active: failed (Result: timeout) since Tue 2018-05-08 18:41:29 HKT; 6s ago
  Process: 2409 ExecStart=/usr/bin/autossh -M 20001 -f -N -o StrictHostKeyChecking=false -o ServerAliveInterval 60 -o ServerAliveCountMax 3 -R 12345:localhost:22 [e-mail] (code=exited, status=0/SUCCESS)
 Main PID: 2409 (code=exited, status=0/SUCCESS)

May 08 18:39:59 ATALCESBMSMobile systemd[1]: Started AutoSSH service for remote tunnel.
May 08 18:39:59 ATALCESBMSMobile autossh[2413]: starting ssh (count 1)
May 08 18:39:59 ATALCESBMSMobile autossh[2413]: ssh child pid is 2414
May 08 18:41:29 ATALCESBMSMobile systemd[1]: autossh-cloud.service: State 'stop-sigterm' timed out. Killing.
May 08 18:41:29 ATALCESBMSMobile systemd[1]: autossh-cloud.service: Unit entered failed state.
May 08 18:41:29 ATALCESBMSMobile systemd[1]: autossh-cloud.service: Failed with result 'timeout'.