Category Archives: golang

Benchmarking golang code

Let’s say that you want to know if EncodeToString is faster than fmt.Sprintf : you will need to compare the speed of this method

func Md5Encode(str string) string {
	md5HashInBytes := md5.Sum([]byte(str))
	md5HashInString := hex.EncodeToString(md5HashInBytes[:])
	return md5HashInString
}

with this other one

func Md5EncodeFmt(str string) string {
	md5HashInBytes := md5.Sum([]byte(str))
	md5HashInString := fmt.Sprintf("%x", md5HashInBytes)
	return md5HashInString
}

Go provides benchmarking features in the testing package which is pretty usefull :

func BenchmarkMd5EncodeFmt(b *testing.B) {
	// run the md5Encode function b.N times
	for n := 0; n < b.N; n++ {
		Md5EncodeFmt("aldfhasdl la fasdfeo8ekldjh asdkj fh lksdjfhwoieuxnroiAUN;laiDJ;ANIfub;OEIRBUF;OEfuN;ALFJ;AL")
	}
}

func BenchmarkMd5Encode(b *testing.B) {
	// run the md5Encode function b.N times
	for n := 0; n < b.N; n++ {
		Md5Encode("aldfhasdl la fasdfeo8ekldjh asdkj fh lksdjfhwoieuxnroiAUN;laiDJ;ANIfub;OEIRBUF;OEfuN;ALFJ;AL")
	}
}

Run

$ go test -bench=.
goos: linux
goarch: amd64
BenchmarkMd5EncodeFmt-8   	 1894791	       625 ns/op
BenchmarkMd5Encode-8      	 3068509	       363 ns/op
PASS
ok  	_/home/paul/LazyInit/bench	3.342s

Run 3 times the benchmarks :

$ go test -count 3 -bench=. 
goos: linux
goarch: amd64
BenchmarkMd5EncodeFmt-8   	 1882105	       627 ns/op
BenchmarkMd5EncodeFmt-8   	 1918942	       624 ns/op
BenchmarkMd5EncodeFmt-8   	 1902894	       625 ns/op
BenchmarkMd5Encode-8      	 3139585	       386 ns/op
BenchmarkMd5Encode-8      	 2937154	       397 ns/op
BenchmarkMd5Encode-8      	 3009801	       380 ns/op
PASS
ok  	_/home/paul/LazyInit/bench	10.217s

EncodeToString() makes your method almost twice faster !

Thanks year 2000 : less is immensely more (the 90s produced a lot of crap)

Thanks to god after year 2000 information technology has started moving towards more pragmatic, simple and effective tools and languages. Some examples that in my opinion make this evident : 

Languages and language tools

  • go, rust, swift are all born with the goal of simplifying their direct parents (c++, objectiveC) and removing their pitfalls.
  • UML abandoned : this is a relief for all coders which had to deal with it. I don’t know anyone using it nowadays.
  • git : finally some one (thanks Linux Torvalds) simplified svn/sourcesafe by putting features that are needed by developers in a clear, pretty intuitive command line interface
  • atom/sublime : reaction to the complexity of Visual Studio, IBM Rational, Eclipse ? I think yes

Databases

  • Key-value stores/noSQL are just taking ER/SQL model and making it simpler, providing only the features needed in 99% of the applications. Boyce-Codd normal form is pretty nice and interesting but in real world applications you’ll never use it. 
  • Object Databases completely disappeared and in some way also the idea that OO methodology/hierarchy could be applied everywhere (just because you are where using OO languages)

Virtualization

  • docker/rkt are slim alternatives to virtualization and virtual machines

Architectures

  • plain old REST API aren’t just a simple way for doing things without having to Corba/Soap ?
  • gRPC : provides corba like features while being 1 order of magnitude more efficent and portable on any platform.

What I’m saying is that the 90s produced a lot of unnecessarily complicated tools and technology which developer just did not need/like which is being progressively substituted with simpler stuff.

 

Profiling a golang REST API server

go tool profiling

Profiling :

is a form of dynamic program analysis that measures, for example, the space (memory) or time complexity of a program, the usage of particular instructions, or the frequency and duration of function calls. Most commonly, profiling information serves to aid program optimization.

How can you profile your golang REST API server in a super simple way :

First : add some lines to your server code

import _ "net/http/pprof"

And then add a listener (I normally use a command line flag to trigger this) :

go func() {
http.ListenAndServe("localhost:6000", nil)
}()

Start your server and generate some load. While your code is running under the load you generated extract the profiler data :

go tool pprof http://localhost:6000/debug/pprof/profile
Fetching profile over HTTP from http://localhost:6000/debug/pprof/profile
Saved profile in /home/paul/pprof/pprof.wm-server.samples.cpu.008.pb.gz
File: wm-server
Build ID: c806572b51954da99ceb779f6d7eee3600eae0fb
Type: cpu
Time: Dec 19, 2018 at 1:41pm (CET)
Duration: 30.13s, Total samples = 17.35s (57.58%)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof)

You have many commands at this point but what I prefer to do, having used kcachegrind for years, is to fire it up using the kcachegrind command :

(pprof) kcachegrind

This will generate a callgrind formatted file and run kcachegrind on it to let you do all the usual analysis that you’re probably already used to do (call graph, callers, callees ..)

 

gcc/clang -fsanitize is saving lifes

 There where times when you had to write C/C++ code and find all the bounds errors and memory leaks by hand or with ancient tools (who remembers Purify?). Valgrind introduced a lot of features but sanitize features added in gcc/clang are just awesome (thanks google guys for this). Just add -fsanitize=address both to compilation and link of your unit tests (we use google test framework), execute the test and get valuable info on memory leaks and out of bounds access to data. In detail :

Testing and inclusion of sanitize is under way in our lab : more info on this in a future post. Work from this team led to the inclusion in Golang of the Data Race detector.

 

Yes, I like go programming language

Yes, I like Go programming language. I’m liking it so much that I have to resist from being a fan boy. I’m trying to understand where all this enthusiasm is coming from (I’m a seasoned coder)  so here’s an attempt to find why :

  1. Code readability — and maintainability — first, language features second
  2. Integrated test environment : go test <package> executes all tests for the package. Unit testing features are builtin.
  3. Code Coverage is builtin (with some limitations, for example if you use cgo it will not work).
  4. Integrated tool chain : no need to have makefile at the cost of rigid hierarchy of data.
  5. Exhaustive standard library containing everything you need to do server-side/network programming
  6. Good Multi-threading features/model included in language (sync package, goroutines, channels), fast goroutines thanks to segmented stack implementation.
  7. Basic set of OOP features, centered on composition, not inheritance : you won’t be able to mess up your code at the cost of not being perceived as an OO language by OO fanboys. For more details on whether go is oo or not go here.
  8. Go is backed by some Famous Names in computing, and this inspires confidence.
  9. CamelCase 🙂 ? Naaah, I hate camel case but I like the choice of having standard style, comments, indent; all supported by the language via go fmt package so that all code will look coherent.

So basically I like the fact that Go is a very opinionated language. You might like the single decisions or not but what I like most is that someone took care of taking them for you (so you don’t have to enforce them team wide or company wide)
Interesting read also on how and why go was born : quoting from Rob Pike speech at go conference SF 2012 :

“To put it another way, oversimplifying of course:

Python and Ruby programmers come to Go because they don’t have to surrender much expressiveness, but gain performance and get to play with concurrency.

C++ programmers don’t come to Go because they have fought hard to gain exquisite control of their programming domain, and don’t want to surrender any of it. To them, software isn’t just about getting the job done, it’s about doing it a certain way.

The issue, then, is that Go’s success would contradict their world view.

And we should have realized that from the beginning. People who are excited about C++11’s new features are not going to care about a language that has so much less.  Even if, in the end, it offers so much more.”