Using ngrok to work around Carrier Grade NAT (CGNAT)

I wrote a while back about my troubles with Carrier Grade Nat (CGNAT), and described a solution that involved tunneling out of CGNAT using a combination of SSH and an AWS server – the full article is here.

That worked ok, but it was pretty fragile and not ideal – connections could be dropped, sessions expired, hosts rebooted etc etc. Passing data through my EC2 host is also not ideal.

My “new and improved” solution to this is to use a local tool like ngrok to create the tunnel for me. This is proving to be far simpler to manage, more reliable, and ngrok also provides a load of handy additional features too.

Here’s a very quick run through of getting it up and running on my Ubuntu VM, which sits behind CGNAT and hosts a webserver I’d like to be able to access from the outside occasionally. This is the front end to my ZoneMinder CCTV interface, but it could be anything you want to host and on any port.

First off, don’t use the default Ubuntu install, that will give you version 1.x which is out of date and didn’t work for me at all – it’s better, quicker and easier to get the latest binary for your platform directly from the ngrok website, extract that on your host and run it directly or add it to you PATH.

wget http://<YourDownloadURL>/ngrok-stable-linux-amd64.zip

unzip ngrok-stable-linux-amd64.zip

once that’s downloaded and extracted, you can (optionally) add your auth token, which you get when you register on the ngrok site. This is optional, but you get some worthwhile features from doing so.

./ngrok authtoken <YourAuthTokenFromTheNgrokWebsite>

Then you simply run ngrok like so:

./ngrok http 80

which should give you a console something like this:

from here you can get the Forwarding URL (http://<uniqueid>.eu.ngrok.io in this example) and your local port 80 should be available on that from anywhere on the internet.

Note I’m using this command:

screen ./ngrok http -region eu 80

to start up ngrok using screen, so I can CTRL+A+D out of that and resume it when I want using screen -r,

Here’s a pic of the console running, showing requests, and Apache being served by the ngrok URL:

That’s it – quick and easy, more stable, and far less faffing too.

 

There are tons of other options worth exploring, like specifying basic HTTP auth, saving your config to a local file, running other ports etc, all of them are explained in the documentation.

There’s a handy review of ngrok and several very similar tools here: http://john-sheehan.com/blog/a-survey-of-the-localhost-proxying-landscape

And some good tips & tricks with ngrok here:
https://developer.atlassian.com/blog/2015/05/secure-localhost-tunnels-with-ngrok/
as noted in the comments on that page: you obviously need to be safe and sensible when opening up ports to the internet…

Cheers,

Don

PS: Update to add the script I use to update the ngrok URL when it changes.

I have this in a local Jenkins job that runs every 30 mins or so, and it has been happily doing the job for a couple of years now – it’s far from perfect and it’s a lot to set up if you’re not used to these tools, but I’m adding it here just in case it helps anyone else….

#!/bin/bash

# Backup of the Jenkins job/script I put together to automatically update my home ngrok tunnel.
# When the tunnel dies, this script will (via Jenkins) create a new one and update a PHP redirect file on my
# AWS Host that allows me to connect to my CGNET'd home server via my AWS website using a dynamic ngrok end point
# Uses:
# - Jenkins
# - bash
# - ngrok
# - jq
# - grep and awk
# - PHP
# - Apache
# - AWS


# check if ngrok is running/not
pidof  ngrok >/dev/null
if [[ $? -ne 0 ]] ; then
		# A (re)start and update is required
    echo "Starting ngrok on $(date)"
    # Start up a new instance of ngrok
    BUILD_ID=dontKillMe nohup /root/ngrok/ngrok http -region eu 80 &
		# Give it a moment before testing it...
		echo "Sleeping for 15 seconds..."
    sleep 15
    # Get the updated publish_url value from the ngrok api
		export NGROKURL=`curl -s http://127.0.0.1:4040/api/tunnels | jq '.' | grep public_url | grep https | awk -F\" '{print $4}'`
    echo "NGROKURL is $NGROKURL"
    # add that to a one-line PHP redirect page
		echo "<?php header('Location: $NGROKURL/zm'); exit;?>" > ZoneMinder.php
    # upload that to my AWS host
    echo "scp'ing zm.php to AWS host..."
		scp -i /MY_AWS_KEY_FILE.pem ZoneMinder.php MY_AWS_USER@MY_AWS_HOST.amazonaws.com:/MY_HTDOCS_DIR/ZoneMinder.php
		echo "Transfer complete."
    # Send an update message via email
		echo "New ngrok url is $NGROKURL/zm" | mailx -s "ngrok zm url updated" MY_EMAIL@gmail.com
else
		# Nothing needed, carry on
		echo "ngrok is currently running, nothing to do"
fi

13 thoughts on “Using ngrok to work around Carrier Grade NAT (CGNAT)”

  1. Hi there,
    I’ve just come across your blog as I’ve not long had my new 4G broadband installed. My guess is it’s the same ISP as yours!?
    A few issues……..my 2 kids had a shiny new Xbox one for Xmas, which is great until they got their fave game and tried to play it. You see, it’s an online only game which I spent the best part of 6 hours to get going, contact EA games and Xbox Live people. No joy….more research checking everything everywhere on the internet revealed it was probably my ISP! Their support is pretty much useless. I had my service installed by a 3rd party company using a grant, then got a new router as the new router gave me better speed. I’ve also discovered that my AudioPro wifi speakers don’t seem to work on wifi anymore???
    I signed up to get a L2TP but that hasn’t seemed to work. I’m not a huge expert in this stuff……would I need to set up my router (HUAWEI) to solve these problems? Any advice would be hugely appreciated…..bearing in mind, I’m no expert in this stuff!

    1. Hi Chris,

      Sorry to hear about the issues, not good 🙁 Do you get an internet connection through the Xbox at all, is it just certain games that don’t work or everything you try on it? Have you tried connect to your router via Ethernet too?

      I don’t have an Xbox but my old PS3 can connect and play games through a similar sounding setup no problem, and with no special settings.

      This page explains CGNAT and related issues pretty well: https://chrisgrundemann.com/index.php/2011/nat444-cgn-lsn-breaks/ and that first pic sums up the issue – lots of connections going out the way over a single address, meaning anything trying to connect inwards (from the internet to your home connection) are not routable. If your XBox is acting as a server in any way (accepting incoming connections from others on the internet) it’ll be listening on that single shared IP address and traffic wont be routed to you, without something like the reverse tunnel I described in this post.

      You’ve probably seen them already, but there are similar issues and a few suggestions here:
      https://community.ee.co.uk/t5/4GEE-WiFi/How-do-I-connect-an-Xbox-one-X-to-the-ee4g-WiFi-mini/td-p/708136
      https://community.ee.co.uk/t5/Gaming/Connecting-my-Xbox-to-EE-mobile-broadband/td-p/561692
      https://community.ee.co.uk/t5/Gaming/Connecting-my-Xbox-to-EE-mobile-broadband/td-p/561692

      I’ll also send you an email with another idea you could try, and if anyone else sees this post and has a similar issue hopefully they can help!

      Cheers,

      Don

  2. Hi Don

    I stumbled across your blog today as your article on CGNAT and the inability to access your CCTV footage remotely is pertinent to my situation. I used to use Dropbox, but your setup with ngrok appealed to me. Looking through the ngrok docs I noticed that the URL constantly changes on the free version. Isn’t this a problem for you? How do you get around this issue? Surely you need to know the current URL in order to access the server remotely, or maybe I’m missing something. If the connection is constant does the URL remain the same, but only changes when reconnecting, although I thought that’s what you were doing with screen.

    Any help would be much appreciated.

    Cheers
    Richard

    1. Hi Richard,

      Yes, I’ve just added a “PS” to this post with a copy of the script I wrote to do this.

      I didn’t initially include this as it’s very rough and takes a bit to set up Jenkins and all the tools the script uses, but hopefully it may give you some ideas.

      The general idea is to have a static publicly available URL that performs an HTTP redirect to the current ngrok location, and that location is updated automatically whenever it changes. As I mentioned, this looks very fragile but it’s worked reliably for quite a while now.

      You could perhaps do something similar with cron and dropbox – if you do, please let me know!

      Cheers,

      Don

      1. Don, thanks for the quick reply. I’ll have a closer look at your script tomorrow and see if I can come up with something. Thanks for the heads-up; I’ll let you know how I get on.

        Cheers
        Richard

  3. Don, sorry for the tardy response but, as promised, I’m reporting back. I think that your setup is different from mine. I have a permanent link set up between my remote server and a client computer; the server pushes the CCTV images to the client where they can be viewed. The client communicates via a 4G router over the Three network, and, as the remote server was about to be moved from a conventional broadband setup to a different 4G router, this required looking into your ngrok idea owing to the problems regarding CGNAT.

    Well, ngrok appears to be working perfectly for me. Having a fixed link between the two machines means that the ngrok URL doesn’t change; the tunnel always remains open. The ngrok documentation is a little vague about how long a tunnel can remain open, but it has been open for about one week so far. In any event, although the server is remote I am able to visit the premises regularly. In terms of viewing the .jpg images on the client it just required a simple wget one-liner after setting ngrok to expose the image directory. A security advantage is that ngrok allows password protection on accessing the directory.

    Anyway, I hope that’s of some interest to you. If you have any questions just let me know, and thanks for the original heads-up regarding ngrok.

    Cheers
    Richard

    1. OK, I’ve just noticed a small error: originally, the client was pulling the images rather than the server pushing them. That became impossible when the server was moved to a 4G router with a SIM.

    2. Thanks for the update Richard, and good to hear you’ve got things sorted.

      I’ve not seen any mention of how long ngrok tunnels can stay open either, but my process to update/reconnect emails me whenever it runs, and it’s very infrequent – and always the result of a power cut or reboot of something going wrong at my end.

      Cheers,

      Don

      1. Yes, a power cut or a reboot will kill the tunnel, but losing an internet connection doesn’t. When the internet connection is re-established then the tunnel re-establishes itself. I found that to be quite interesting, and useful. As far as tunnel longevity is concerned, it would appear to last until the user closes the tunnel, or it is forced shut by one of the issues outlined above. Otherwise, it appears to last indefinitely. Clearly, time will tell.

        Anyway, thanks again Don; it was good talking with you.

        Cheers
        Richard

  4. Hi Don, thanks for this blog post. I’m trying to get a Pyronix Alarm to talk to it’s servers and need port 25000 UDP/TCP exposed, I guess in both directions. I’m behind a CGNAT using the Three 5g broadband service in the UK. My question to you is: is it possible to have Ngrok running on a machine on the network and expose the ports of another machine on the network? I can have a machine running in my office, but I can’t add Ngrok to IOT devices that don’t have UI or CLI interaction. Forgive me if I’ve missed this in your description, I read the links you posted too but just couldnt see how.

    1. Hi Paul,

      Thanks – I haven’t done that myself, but guess you could possibly do it by combining multiple ngrok tunnel definitions on one machine like this:
      https://ngrok.com/docs/ngrok-agent/config#config-ngrok-tunnel-definitions

      combined with some iptables rules redirecting traffic to/from ports on your IoT devices, maybe along these lines:
      https://serverfault.com/questions/586486/how-to-do-the-port-forwarding-from-one-ip-to-another-ip-in-same-network

      Does that help at all? Please let me know how you get on!

      Cheers,

      Donald

      1. I really appreciate your response Don, that’s at least a glimmer of hope! Thank you. I’ll check out the links and let you know how I get on.

Leave a Reply to Chris Winter Cancel reply

Your email address will not be published.

Pin It on Pinterest

%d bloggers like this: