ngrok is a tunneling, reverse proxy that establishes secure tunnels from a public endpoint to a locally running network service while capturing all traffic for inspection and replay. It is an open-source project on GitHub.
-
This week, I released a new version of ngrok that includes a number of significant improvements. This effort involved a serious refactoring of the ngrok codebase and a redesign of the ngrok protocol. The result is a cleaner, more extensible design as well as an improved set of core functionality that includes the following features and improvements.
ngrok’s raison d’etre is to selectively expose services behind firewalls and NATs to the public internet while capturing the traffic for inspection and replay. While most users have no problems getting ngrok to work, there was a small segment of users who reported that ngrok failed to connect on some restricted networks. The root cause of this issue is the implementation of network policies which block traffic on all ports except those responsible for the most commonly used internet protocols (HTTP, DNS, SMTP, etc). In an effort to make ngrok work absolutely everywhere, ngrok.com now combats these draconian filters by running all of its traffic over port 443 by default. Since all ngrok connections are encrypted with TLS, ngrok traffic looks sufficiently similar to regular HTTPS traffic to defeat all port/traffic restricting networks that I’ve been able to test.
Unfortunately, the change to port 443 alone doesn’t allow ngrok to run on all restricted networks. Many corporate networks reject all traffic to the public internet and provide HTTP proxy servers as the only public gateway. Supporting ngrok tunnels in this environment was another common support request. The latest version of ngrok now supports establishing its tunnels through an HTTP proxy by honoring the traditional *nix environment variable “http_proxy” like other tools. You may also specify a value explicitly by setting the http_proxy parameter in the configuration file. ngrok uses the same CONNECT method used by browsers to run HTTPS traffic over an HTTP proxy.
One of the most major changes in this version of ngrok was adding the support to run multiple ngrok tunnels simultaneously. That means it’s now possible to run a single ngrok client which creates multiple public endpoints that route to multiple local services. Here’s a new ngrok client running:
ngrok
Tunnel Status online
Version 1.3/1.3
Forwarding http://blog.ngrok.com -> 127.0.0.1:4000
Forwarding https://blog.ngrok.com -> 127.0.0.1:4000
Forwarding https://api.ngrok.com -> 127.0.0.1:8080
Forwarding http://hacks.inconshreveable.com -> 127.0.0.1:9090
Forwarding tcp://ngrok.com:44764 -> 127.0.0.1:22
Web Interface 127.0.0.1:4040
...
To enable you to easily create multiple tunnels without complicating the command-line interface, ngrok now supports reading from a simple YAML configuration file for starting multiple tunnels. Here’s the configuration file:
authtoken: "..."
tunnels:
blog:
proto:
http: 4000
https: 4000
api:
auth: "user:password"
proto:
https: 8080
personal:
hostname: "hacks.inconshreveable.com"
proto:
http: 9090
ssh:
proto:
tcp: 22
Then you can start any number of your pre-configured tunnels from the command line, like so:
ngrok start blog api personal
Here’s another example to just start remote access to my machine:
ngrok start ssh
I expect this capability to be especially valuable to contractors and consultants working on multiple projects simultaneously as well as for developers who are working on distributed systems that involve running multiple public components. I already use it myself to switch between sharing different projects without rewriting command-line switches all of the time.
Some users of ngrok in countries outside of the United States reported that requests routed through ngrok were very slow to complete. To understand why this was the case, I will briefly explain how ngrok tunnels a new connection to a public ngrok endpoint through to your local service.
The old procedure ngrok used for tunneling a new connection was as follows:
Now that you understand the ngrok protocol for tunneling a connection, it’s easy to understand why it was so slow. The setup time for tunneling a new connection involved a significant amount of network latency, including: - Sending a message from the server to the client (.5 network round trips) - A new TCP connection (1 network round trip) - A TLS negotiation (2 network round trips)
In cases where the ngrok client was across the world or on slow connections, 3.5 network round trips could easily add over a full second to connection setup times!
The latest version of ngrok now pre-fetches connections so it can immediately service new requests. The ngrokd implementation now executes the following logic: 1. ngrokd server receives a new public connection to your-app.ngrok.com 1. ngrokd server retrieves a proxy connection from a pool and joins it with the public connection to service the request 1. ngrokd server sends a new Request Proxy Connection Message to the ngrok client to replace the connection removed from the pool
Now, in the common case when a connection is available in the pool, zero network round trips are required before the connection begins to tunnel. Avoiding the time to establish these connections makes tunneling connections feel much, much faster.
When older versions of ngrok crashed, they destroyed your terminal. In order to display the terminal interface, ngrok uses termbox-go which changes the mode of your terminal and needs to be cleanly shut down to return your terminal to a proper state. ngrok now wraps the execution of most of its goroutines with a function that will execute a controlled shutdown of the program if the goroutine panics. In the unfortunate but inevitable case that ngrok crashes, a user will no longer be confronted with a wrecked terminal. Instead, they will be presented with a clean stack trace and instructions for reporting the bug on Github.
All of the code for ngrok and ngrokd (the client and server) has always been open-source, but running your own ngrokd server wasn’t entirely trivial. I’ve made a great effort to make running your own ngrokd server easier than ever. I’ve posted a guide on how to self-host your own service that’s only six steps.
Self-hosting: How to run your own ngrokd server
Another common inquiry I received was whether ngrok ran on linux/arm devices and if so where the download was available. ngrok certainly runs on Linux/ARM (in fact, I developed this new version entirely on a Chromebook), and as of today, ngrok.com now has download links available for the builds. Now you can try it out on your Chromebook or RaspberryPi without compiling!
Finally, I’d like to mention one last change to ngrok that is unrelated to the new version. Previously, ngrok only allowed you to use HTTPS tunnels after you signed up on ngrok.com. In response to the news regarding the surveillance of internet traffic and possible weaknesses in internet encryption implementations, I’m making my own small effort to improve the state of internet security. I’ve made the following changes to the ngrok.com service:
The latest version of ngrok is available for download on ngrok.com.
If you love ngrok and would like to see more improvements, please show your support in any of the following ways:
You can file bugs or feature requests on the ngrok project on github.
ngrok is an open source project! Submit a pull request to fix a bug or add a new feature on the ngrok project on github.
ngrok is a pay-what-you-want service. You can pay for ngrok after you sign-up for an account on ngrok.com