A Story of a Big Go Binary

Building an IoT Agent

Let’s breeze through a story that shows how to reason about and build a service — an IoT agent — that we’ll deploy to a modest hardware device somewhere across the globe. We will highlight the architecture of such an agent from an operations point of view.

$ git clone https://github.com/jondot/go-cli-starter fattyproject
Cloning into 'fattyproject'...
remote: Counting objects: 55, done.
remote: Total 55 (delta 0), reused 0 (delta 0), pack-reused 55
Unpacking objects: 100% (55/55), done.
for {
f := NewFarble(&Counter{})
f.Bumple()
time.Sleep(time.Second * 1)
}
import (
_ "expvar"
"net/http"

:
:

go func() {
http.ListenAndServe(":5160", nil)
}()
import(
:
"go.uber.org/zap"
:


logger, _ := zap.NewProduction()
logger.Info("OK", zap.Int("ip", *ip))
import(
:
"github.com/robertkrimen/otto"
:

for {
:

vm.Run(`
abc = 2 + 2;
console.log("\nThe value of abc is " + abc); // 4
`)

:
}

Understanding Go Binary Dependencies

Let’s look at what we’ve got so far.

$ ls -lha fattyproject
... 13M ... fattyproject*
$ ls -lha /usr/l/.../-0_2/lib/libMagickCore-6.Q16.2.dylib
... 1.7M ... /usr/.../libMagickCore-6.Q16.2.dylib
$ cat main.go
package main

func main() {
print("hello")
}

$ go build && otool -L main
main:
$ go tool
addr2line
api
asm
cgo
compile
cover
dist
doc
fix
link
nm
objdump
pack
pprof
trace
vet
yacc
$ go tool nm -sort size -size fattyproject | head -n 20
5ee8a0 1960408 R runtime.eitablink
5ee8a0 1960408 R runtime.symtab
5ee8a0 1960408 R runtime.pclntab
5ee8a0 1960408 R runtime.esymtab
4421e0 1011800 R type.*
4421e0 1011800 R runtime.types
4421e0 1011800 R runtime.rodata
551a80 543204 R go.func.*
551a80 543204 R go.string.hdr.*
12d160 246512 T github.com/robertkrimen/otto._newContext
539238 100424 R go.string.*
804760 65712 B runtime.trace
cd1e0 23072 T net/http.init
5e3b80 21766 R runtime.findfunctab
1ae1a0 18720 T go.uber.org/zap.Any
301510 18208 T unicode.init
5e9088 17924 R runtime.typelink
3b7fe0 16160 T crypto/sha512.block
8008a0 16064 B runtime.semtable
3f6d60 14640 T crypto/sha256.block

Gofat

There’s one last trick that will work. When you compile your Go binary, Go will generate interim binaries for each dependency, before statically linking these all up into the one binary you get in the end.

eval `go build -work -a 2>&1`
find $WORK -type f -name "*.a" | xargs -I{} du -hxs "{}" | gsort -rh
sed -e s:${WORK}/::g

Trimming Down the Fat

Let’s run gofat for the first time on our mock IoT agent project. Here’s the output:

1.8M    net/http.a
788K    gopkg.in/alecthomas/kingpin.v2.a
388K github.com/alecthomas/template.a
668K    github.com/newrelic/go-agent.a
624K github.com/newrelic/go-agent/internal.a
392K    go.uber.org/zap/zapcore.a
2.2M github.com/robertkrimen/otto.a 
312K github.com/robertkrimen/otto/parser.a
172K github.com/robertkrimen/otto/ast.a
128K    github.com/Sirupsen/logrus.a

Take Aways

We found a way to break down Go dependency sizes and we saved around 7MB by accepting that we don’t have to use certain dependencies and that we can take alternative from Go’s standard library in their stead.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Dotan Nahum

@jondot | Founder & CEO @ Spectral. Rust + FP + Hacking + Cracking + OSS. Previously CTO @ HiredScore, Como, Conduit.