Go has execellent support for cross compilation. There are scores of tutorials and helper tools on the internet with instructions on how to do it. This, in and of itself, is rather astounding because of how simple it is:
1 # set up cross compilation to windows_amd64
2 # you only need to do this once
3 cd $GOROOT/src
4 GOOS=windows GOARCH=amd64 ./make.bash --no-clean
5
6 # whenever you want to cross compile
7 cd $YOUR_APP
8 GOOS=windows GOARCH=amd64 go buildThere is a well-known limitation to this simplicty. If you want to use any C libraries, you’ll need to compile and link them with Cgo and from there you’re back to the mess that is cross-compiling C. Upon hearing this limitation you might say something totally reasonable like:
That’s not really a problem, there are tons of programs that don’t use Cgo.
And you’d be right, until I told you:
Some critical parts of the standard library use Cgo.
What exactly are those parts exactly? It turns out that if you cross-compile Go to Darwin or Linux your programs won’t use the system DNS resolver. They also can’t use the native host certificate store. They also can’t look up the user’s home directory, either. For some applications, that doesn’t matter so much. For ngrok, though, all three could matter, and two have already caused bugs.
When I spoke at GopherCon, I told everyone that if they want to distribute production applications they should be compiling natively because of this limitation. I was wrong. It turns out that there’s a clever workaround for this that has been mentioned occasionally on the mailing list and which minux just reminded me of when he posted:
If the only cgo-dependent packages are in the standard library, then you do have workaround without setting up a full cross compilation environment.
Download the binary distribution for the target platform, extract the pkg/$GOOS_$GOARCH directory into your $GOROOT/pkg …
Which is quite clever and I want the simplicity of that build and deployment story. But instead of doing this as a one off, I decided to automate the entire process since I want you all to use it as well and because I’m going to need to do it again at every new release of Go.
gonative is a simple tool which creates a build of Go that can cross compile to all platforms while still using the Cgo-enabled versions of the stdlib packages. It’s a simple Go program of a few hundred lines and is available on github! If you’re the author of a Go program doing multi-platform distribution, please try it out and contribute:
github.com/inconshreveable/gonative
It’s really easy to install and run:
1 go get github.com/inconshreveable/gonative
2 gonativeI’ve tested targeting Windows/Linux/Darwin and they all work correctly!
One of the oft-touted advantages of Go programs is that they’re “dependency-free”. They don’t require a runtime or link against dynamic libraries which your users might not have installed on their systems.
So when I compiled my native-stdlib linux_386 test code and tried to run it on an amd64 machine, I was rather confused when I was confronted with this error message:
1 alan@inconshreveable:~$ ./test
2 -bash: ./test: No such file or directoryminux’s help on the mailing list helped me understand what was going on. Let’s read the ELF header of this binary.
1 alan@inconshreveable:~$ readelf -l testElf file type is EXEC (Executable file)
Entry point 0x8066a50
There are 9 program headers, starting at offset 52
Program Headers:
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
PHDR 0x000034 0x08048034 0x08048034 0x00120 0x00120 R E 0x1000
INTERP 0x000bed 0x08048bed 0x08048bed 0x00013 0x00013 R 0x1
[Requesting program interpreter: /lib/ld-linux.so.2]
LOAD 0x000000 0x08048000 0x08048000 0x68e70 0x68e70 R E 0x1000
LOAD 0x069000 0x080b1000 0x080b1000 0xc9403 0xc9403 R 0x1000
LOAD 0x133000 0x0817b000 0x0817b000 0x10580 0x23554 RW 0x1000
DYNAMIC 0x133080 0x0817b080 0x0817b080 0x00098 0x00098 RW 0x4
TLS 0x000000 0x00000000 0x00000000 0x00000 0x00008 R 0x4
GNU_STACK 0x000000 0x00000000 0x00000000 0x00000 0x00000 RW 0x4
LOOS+5041580 0x000000 0x00000000 0x00000000 0x00000 0x00000 0x4
See that INTERP section? The one that says:
[Requesting program interpreter: /lib/ld-linux.so.2]
/lib/ld-linux.so.2 is the 32-bit ELF interpreter that is required for programs that need to dynamically link libraries. My linux box didn’t have it installed, so the program was failing.
But wait, you said that Go programs doesn’t require a runtime or dynamically linked libraries!
And that’s totally true when you cross compile without native libraries. Those executables don’t call anything via C APIs. But when you link against native libraries, there are a few libraries you need to load dynamically. Let’s take a look at them:
alan@inconshreveable:~$ ldd test
not a dynamic executable
What!? This hung me up for a while too, until I remembered seeing that DYNAMIC section in the ELF headers. It turns out that if you don’t have the 32-bit libraries on your system, ldd can’t analyze the binary properly. minux explains:
actually if you take a look at ldd’s source code you will find that it’s a script and just set some magic environment variable and then execute the program, essentially it’s asking the elf interpreter to dump needed shared library.
so if you don’t have the elf interpreter, the program will fail to execute and ldd mistake that as the program not being dynamically linked.
If we use readelf instead, we can see what’s going on:
alan@inconshreveable:~$ readelf -d test
Dynamic section at offset 0x133080 contains 19 entries:
Tag Type Name/Value
0x00000001 (NEEDED) Shared library: [libpthread.so.0]
0x00000001 (NEEDED) Shared library: [libc.so.6]
more …
So now we can finally see that our natively compiled program now depends on the 32-bit elf interpreter so that it can dynamically load libpthread and libc.
Understanding executable formats and the guts of how Linux sets up your program to support dynamic linking is not an area I need to wander into frequently, so it was rather enjoyable to push my knowledge deeper into this area. For those of you who don’t wade there often either, I hope this was useful in understanding more about exactly how your programs are loaded and run and what you need to be aware of when you compile with native vs. non-native Go libraries. And for those of you who need to distribute Go programs that use native libs, try out gonative! I hope it makes things a little less painful.
https://groups.google.com/forum/#!msg/golang-nuts/2XoGUvBalcw/ErSWiTlO17kJ