Notice: Undefined offset: 1 in /usr/local/src/wordpress/wp-content/themes/montezuma/includes/parse_php.php on line 79

HowTo: Set Up Reverse SSH Tunnels to Forward Ports

~~   Forward   ~~

VideofunnelSometimes, we have a powerful machine on our LAN, where we would like to run -all- our services like Squid, CUPS, MythTV, TOR, and so on.   In my case this is my Home Theater PeeCee.   I have all the appropriate daemons running on that machine and their listening ports are only on 127.0.0.1, and not on any outside interfaces (which would be a security problem).

But I also want these services on the other machines of my LAN, like the laptop and so on.   With reverse SSH tunnels, on the laptop I instigate a tunnel to the HTPC, and the HTPC’s daemon port is then forwarded through the encrypted tunnel to the laptop.   That port now appears on the laptop at 127.0.0.1 as if it’s local.   When I use that service, the laptop reaches into its bellybutton, goes through the encrypted tunnel to the remote server, and accesses the service running on the remote HTPC.   All of this is done through SSH with military-grade encryption, so you can do this no matter where you are, securely.   No matter what daemon, only port 22 is ever open to the outside.   And, it’s fast.

There are several ways to do this, and (less informed) ppl opt to set up the tunnel either as root or as their regular user.   But either of these has root/sudo access and other vulns, which is not secure in the event of a MitM attack.   So I set up my tunnels in the name of a dead-nonprivileged user called ‘sleeper’.   It’s a good practice to set up daemons each with their own non-prived user.

~~   Construction of the Tunnels   ~~

This HowTo is Debian-centric.

Set up the SSH Server

ON THE SERVER MACHINE

First install SSH and lsof:
# apt-get install openssh-server lsof command-not-found

Now make sure it is running and listening:
# lsof -i -n -P |grep “22 (LISTEN)”
sshd 1010 root 3u IPv4 1966 0t0 TCP *:22 (LISTEN)
sshd 1010 root 4u IPv6 1968 0t0 TCP *:22 (LISTEN)

Whups, it’s running on ipv6, which is a security problem as you do not have a firewall running for ipv6.   So we are going to tweak the SSH daemon’s config file.
– Edit /etc/ssh/sshd_config
– Remove the # from ListenAddress 0.0.0.0 . This way it will listen to all interfaces, but IPV4 only.   If   ::1   is there, remove it with alacrity…

While we’re at it, let’s speed up SSH for establishing connections:
– Also in sshd_config look for UseDNS .
– Remove the # and set it to ‘no’.
– Save the config file, and
# systemctl restart sshd
# lsof -i -n -P |grep “22 (LISTEN)”

sshd 1010 root 3u IPv4 1966 0t0 TCP *:22 (LISTEN)

Create our new impoverished user called ‘sleeper’ to use for setting up the ssh tunnel from other hosts.   We will generate a key for this account so you can login with a keyfile instead of typing your password everytime.   The account will also be restricted to execute ‘sleep’ through this way;   other commands will simply fail.

# useradd –system -N –group sleeper sleeper
# su sleeper

sleeper$

Our new user has absolutely no privileges as group sleeper can’t do nothing.   Also notice that you can su to sleeper as root, but not as any other user since there is no password!

~~   Set up the SSH Client   ~~

SWITCH OVER TO THE CLIENT MACHINE

Install the SSH client and lsof.
# apt-get install openssh-client command-not-found

Create our impoverished user sleeper as above, and:
# su sleeper
sleeper$

We need an SSH keypair (private, public) for this user.   We’re going to use the most secure algorithm, with the largest possible key.   (Note: Key size won’t affect data velocity, as the key is only used locally) (Note: elliptic-curve may be compromised, so use RSA):
sleeper$ ssh-keygen -t rsa -b 16384
… use the defaults and NO passphrase!

Now our new private/public keypair is in /home/sleeper/.ssh, so copy your public keyfile (-never- your private one) to the Server.   Unfortunately ssh-copy-id doesn’t work these days, so we have to do it this way:
sleeper$ exit
# cp /home/sleeper/.ssh/id_rsa.pub /home/{youruser}/authorized_keys
# su {youruser}
{user}$ scp ~/authorized_keys   {remote-serverIP}/home/{youruser}

(Note: root can’t scp or ssh these days, and reasonably so)

Putting our public key on the remote Server, lets us ssh over to that server without need for a password, which is necessary when automatically setting up SSH tunnels.

THEN ON THE SERVER MACHINE

# cp /home/{youruser}/authorized_keys   /home/sleeper/.ssh
# chown sleeper:sleeper /home/sleeper/.ssh/authorized_keys
# chmod 700 /home/sleeper/.ssh/authorized_keys

So, we’ve just put sleeper’s public key from the client machine into the authorized_keys file on the server machine.   From now on, sleeper can ssh from client to server. (but not the other way)   The principle here is that you put a user’s public key on the remote machine, in that user’s .ssh/authorized_keys file.    You can just append different public keys here with # cp {keyfile} >> authorized_keys

SWITCH BACK TO THE CLIENT MACHINE

Let’s test it:
sleeper$ ssh {ServerIP}
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.

camelopardalis:sleeper~$ exit
logout
Connection to camelopardalis closed.

sleeper$ exit
exit
#

It twerks!

Now, sometimes SSH tunnels time out, giving ‘broken pipe’, so let’s prevent that:
– Edit /etc/ssh/ssh_config (not sshd_config), since we’re configuring the client this time, and at the bottom:

#************************************************
# Maintain SSH connexion if network goes down or sleep
KeepAlive yes
ServerAliveCountMax 10
ServerAliveInterval 20
#************************************************

~~   Authorizing Keys on the Server   ~~

BACK ON THE SERVER

sleeper$ cd ~/.ssh
sleeper$ ls -al

total 36
drwx—— 2 sleeper nobody 56 Oct 17 2012 ./
drwx—— 13 sleeper nobody 4096 Oct 17 2012 ../
-rwx—— 1 sleeper nobody 0 Oct 17 2012 authorized_keys*
-rwx—— 1 sleeper nobody 668 Aug 10 2011 id_rsa*
-rwx—— 1 sleeper nobody 603 Aug 10 2011 id_rsa.pub*
-rwx—— 1 sleeper nobody 2083 Jul 5 13:55 known_hosts*

– Edit /home/sleeper/.ssh/authorized_keys
– Add this to the beginning of the file, before anything else:

client="{CLIENT IP}.linux.lan",command="/bin/sleep 365d"

… of course substituting your actual client’s IP for {CLIENT IP}.

I recommend that you switch over to all static IPs for the machines on your LAN, rather than DHCP, so you always know where they are and can set variables like the above.   Choose something unusual for your LAN like 192.168.321.* .   For every client machine that will be tunneling to the server, repeat the client= line in authorized_keys .

We’re almost there!   Now it’s time to create a tunnel.

~~   Creating a Tunnel   ~~

BACK ON THE CLIENT

Create the SSH tunnel by reaching out to the server and bring that remote port local:
sleeper$ exit
# /usr/bin/ssh -v -l sleeper -i /home/sleeper/.ssh/id_rsa -2 -4 -c aes256-ctr,aes128-ctr -L 3128:localhost:3128 {SERVERNAME or IP} sleep 365d

… substituting your Server’s name or IP for {SERVERNAME or IP} of course, and the port on the remote server that you want to extend to the local client instead of 3128 if you want to extend a service other than Squid.

So this sets up the tunnel in sleeper’s name to the remote Server.   In this case we are forwarding Squid‘s port 3128 (a web object caching server) from the remote Server to exist on the local client, so 3128 should now show up on localhost.   Let’s check it:
# lsof -i -n -P |grep 3128
ssh 2181 root 4u IPv4 729718 0t0 TCP 127.0.0.1:3128 (LISTEN)

Success!   We now have Squid access local, but really running on the remote server.   So on the local machine we go into Firefox   Edit|Preferences|Advanced|ConnectionSettings, Manual Proxy and set it to 127.0.0.1 and 3128.   From now on we are Squidding everywhere, through the remote server and highly securely.

In the tunnel setup command we have specified -v for verbose.   This will show the stages it goes through to establish the tunnel.   Look carefully through the SSH setup lines (# journalctl -xe);   these will give you a clue to the cause of an error, if any, and if it works you can remove -v.   You can set up any number of tunnels to extend services from a remote server this way;   I have 21 remote ports brought local.

But we don’t want to enter this frickin’ command everytime we want a tunnel set up, do we.   So let’s make it automatic and self-repairing.

~~   Automating Tunnel Creation with systemd   ~~

My presumption is you’ve set up systemd on Debian, which will be the default soon.   systemd is the init system and it controls daemons (services), which means that it will start our tunnel automatically on boot, wake it on sleep, and restart it if it goes away.   BUT the catch is that in order for systemd to keep track of these tunnels, each one must have its own .service file.

You can put your systemd .service files anywhere you want, by specifying the path to the .service file when you enable.   But it’s best to stick with POSIX compliance, meaning that custom .service files belong in /usr/local/lib/systemd/system.   If this path doesn’t exist, create it, and any .service files you put there will be automatically seen by systemd.   So in that directory I create the .service file for the Squid tunnel.

– Edit tunnel-squid.service and:

[Unit]
Description=tunnel for Squid
BindsTo=tunnel.target

[Service]
Type=simple
User=sleeper
ExecStart=/usr/bin/ssh -l sleeper -i /home/sleeper/.ssh/id_rsa -2 -4 -L 3128:127.0.0.1:3128 {SERVERNAME or IP} sleep 365d
Restart=always
RestartSec=30
TimeoutSec=15

# Security
PrivateTmp=yes
InaccessibleDirectories=/bin /boot /.config /.config /home/{USER} /lib32 /media /mnt /opt /proc /root /sbin /srv /sys
ReadOnlyDirectories=/dev /etc /usr /var
CapabilityBoundingSet=~CAP_SYS_PTRACE
DeviceAllow=/dev/null rw
NoNewPrivileges=yes

[Install]
WantedBy=tunnel.target

… of course substituting your Server’s name or IP for {SERVERNAME or IP} and your username for {USER}.

Save it, and
# systemctl enable tunnel-squid.service
ln -s ‘/usr/local/lib/systemd/system/tunnel-squid.service’ ‘/etc/systemd/system/multi-user.target.wants/tunnel-squid.service’

So the service is enabled but not started yet.   Remember before, we’d set up the 3128 tunnel manually, so we must first take that down before we start the service.   Notice its PID is 2181 (above), so we:
# kill -15 2181
# lsof -i -n -P |grep 3128
# systemctl start tunnel-squid.service
# lsof -i -n -P |grep 3128

ssh 2195 root 4u IPv4 729718 0t0 TCP 127.0.0.1:3128 (LISTEN)

Perfect.   Now you don’t have to worry about it anymore.   For all other ports you’d like to forward, all you have to do now is create and enable a .service file for that port.

Now;   I like to keep track of what’s going on, so I have a special script to check all of my tunnels, called ‘tunnels’:

#!/usr/bin/bash

echo -n "Squid  >>>  " && lsof -i -n -P |grep "3128 (LISTEN)"
echo -n "Cups   >>>  " && lsof -i -n -P |grep "631 (LISTEN)"
echo -n "Mysql  >>>  " && lsof -i -n -P |grep "22306 (LISTEN)"
echo -n "Myth43 >>>  " && lsof -i -n -P |grep "6543 (LISTEN)"
echo -n "Myth44 >>>  " && lsof -i -n -P |grep "6544 (LISTEN)"
echo -n "Sane   >>>  " && lsof -i -n -P |grep "6566 (LISTEN)"
echo -n "Tor9151 >>>  " && lsof -i -n -P |grep "9151 (LISTEN)"
echo -n "Tor9152 >>>  " && lsof -i -n -P |grep "9152 (LISTEN)"
echo -n "Tor9153 >>>  " && lsof -i -n -P |grep "9153 (LISTEN)"
echo -n "Tor9154 >>>  " && lsof -i -n -P |grep "9154 (LISTEN)"
echo -n "Tor9155 >>>  " && lsof -i -n -P |grep "9155 (LISTEN)"
echo -n "Tor9156 >>>  " && lsof -i -n -P |grep "9156 (LISTEN)"
echo -n "Tor9157 >>>  " && lsof -i -n -P |grep "9157 (LISTEN)"
echo -n "Tor9158 >>>  " && lsof -i -n -P |grep "9158 (LISTEN)"
echo -n "Tor9159 >>>  " && lsof -i -n -P |grep "9159 (LISTEN)"
echo -n "Tor9160 >>>  " && lsof -i -n -P |grep "9160 (LISTEN)"
echo -n "Tor9161 >>>  " && lsof -i -n -P |grep "9161 (LISTEN)"
echo -n "Tor9162 >>>  " && lsof -i -n -P |grep "9162 (LISTEN)"
echo -n "Tor9163 >>>  " && lsof -i -n -P |grep "9163 (LISTEN)"
echo -n "Tor9164 >>>  " && lsof -i -n -P |grep "9164 (LISTEN)"
echo -n "Tor9165 >>>  " && lsof -i -n -P |grep "9165 (LISTEN)"
echo

Of course, adjust to the ports you’re tunneling.   I put this in /usr/local/bin (where custom scripts belong under POSIX standards) and set it executable.   Its proper output is:
# tunnels
Squid >>> ssh 2181 root 4u IPv4 729718 0t0 TCP 127.0.0.1:3128 (LISTEN)”
Cups >>> ssh 2179 root 4u IPv4 721705 0t0 TCP 127.0.0.1:631 (LISTEN)
Mysql >>> ssh 2172 root 4u IPv4 729707 0t0 TCP 127.0.0.1:22306 (LISTEN
Myth43 >>> ssh 2180 root 4u IPv4 729717 0t0 TCP 127.0.0.1:6543 (LISTEN)
Myth44 >>> ssh 2187 root 4u IPv4 729726 0t0 TCP 127.0.0.1:6544 (LISTEN)
Sane >>> ssh 4640 root 4u IPv4 752053 0t0 TCP 127.0.0.1:6566 (LISTEN)
Tor9151 >>> ssh 2195 root 4u IPv4 730727 0t0 TCP 127.0.0.1:9151 (LISTEN)
Tor9152 >>> ssh 2191 root 4u IPv4 730726 0t0 TCP 127.0.0.1:9152 (LISTEN)
Tor9153 >>> ssh 2176 root 4u IPv4 729711 0t0 TCP 127.0.0.1:9153 (LISTEN)
Tor9154 >>> ssh 2185 root 4u IPv4 729721 0t0 TCP 127.0.0.1:9154 (LISTEN)
Tor9155 >>> ssh 2192 root 4u IPv4 729731 0t0 TCP 127.0.0.1:9155 (LISTEN)
Tor9156 >>> ssh 2182 root 4u IPv4 729719 0t0 TCP 127.0.0.1:9156 (LISTEN)
Tor9157 >>> ssh 2194 root 4u IPv4 729732 0t0 TCP 127.0.0.1:9157 (LISTEN)
Tor9158 >>> ssh 2170 root 4u IPv4 729706 0t0 TCP 127.0.0.1:9158 (LISTEN)
Tor9159 >>> ssh 2184 root 4u IPv4 729720 0t0 TCP 127.0.0.1:9159 (LISTEN)
Tor9160 >>> ssh 2178 root 4u IPv4 729715 0t0 TCP 127.0.0.1:9160 (LISTEN)
Tor9161 >>> ssh 2174 root 4u IPv4 729709 0t0 TCP 127.0.0.1:9161 (LISTEN)
Tor9162 >>> ssh 2189 root 4u IPv4 729728 0t0 TCP 127.0.0.1:9162 (LISTEN)
Tor9163 >>> ssh 2173 root 4u IPv4 729708 0t0 TCP 127.0.0.1:9163 (LISTEN)
Tor9164 >>> ssh 2175 root 4u IPv4 729710 0t0 TCP 127.0.0.1:9164 (LISTEN)
Tor9165 >>> ssh 2171 root 4u IPv4 729705 0t0 TCP 127.0.0.1:9165 (LISTEN)

See?   Real easy to see if someone is misbehaving, although I should never need to use it.

~~   BONUS: ‘listen’ command   ~~

You’ll notice how nice the command # lsof -i -n -P is for checking open ports, so let’s make that into a command.
– Edit /etc/profile and put in:

#***********************************

alias listen='lsof -i -n -P |more'

#***********************************

… and save, so after you log out/in you can just type in the command ‘listen’ to see all of your open ports.   Handy.

You will find some ports open which are unnecessary, which is a security problem.   Track those to their service and either deinstall the unneeded service, or change the config file so it only listens on 127.0.0.1, not * or 0.0.0.0.

,'after' => '

') )