Tag Archives: golang

2023 Reading list (updated as we go)

AI generated sort algorithms make it into LLVM/C++ https://www.nature.com/articles/s41586-023-06004-9

Superfast hash https://github.com/wangyi-fudan/wyhash

Google leaked internal document on AI : https://www.semianalysis.com/p/google-we-have-no-moat-and-neitherkkjhnmmmm

Compression algorithms benchmarks : https://github.com/inikep/lzbench

What’s inside training data for LLM https://arxiv.org/pdf/2101.00027.pdf

Cache Optimization Models and Algorithms : https://arxiv.org/pdf/1912.12339.pdf

ChatGPT Is a Blurry JPEG of the Web https://twitter.com/paulborile/status/1647640363555758083

Ordinals https://docs.ordinals.com/overview.html

NO Pause on AI development : https://medium.com/enrique-dans/a-pause-on-the-development-of-ai-its-not-going-to-happen-d4f894816e82

Cheatsheet for golang test, benchmarks, profiling https://github.com/samonzeweb/profilinggo

Yet another browser https://mullvad.net/it/browser promising to minimize tracking and fingerprinting.

everything cli https://www.commandlinefu.com/commands/browse

Amazon Is Making a New Web Browser ? https://gizmodo.com/amazon-prime-new-web-browser-survey-1850224922?utm_source=tldrnewsletter

Interview with OpenAI’s Greg Brockman: GPT-4 isn’t perfect, but neither are you https://techcrunch.com/2023/03/15/interview-with-openais-greg-brockman-gpt-4-isnt-perfect-but-neither-are-you/?utm_source=tldrnewsletter&guccounter=1

On jpeg-XL https://cloudfour.com/thinks/on-container-queries-responsive-images-and-jpeg-xl/#jpeg-xl-the-holy-grail-image-format
https://cloudinary.com/blog/the-case-for-jpeg-xl

On crypto and thieves :
“In February 2022, a hacker stole 120,000 wrapped Ethereum from Wormhole, a cross-blockchain bridge” https://newsletter.mollywhite.net/p/oasis-defi-centralization – subscribe to Molly White newsletter for unbiased crypto news.

B Corporation Certification : https://en.wikipedia.org/wiki/B_Corporation_(certification) “As of September 2022, there are 5,697 certified B Corporations across 158 industries in 85 countries.” https://www.bcorporation.net/en-us/
https://www.positive.news/economics/five-trends-sustainable-business/

https://lumalabs.ai/ : acquire 3d assets anywhere with a phone

https://roaringbitmap.org/ compressed bitmaps

Code RED in google : https://medium.com/@ignacio.de.gregorio.noblejas/can-chatgpt-kill-google-6d59742ee635

Node.js is aging like milk https://javascript.plainenglish.io/node-js-is-aging-like-milk-now-whats-next-a4e726ae668f :

Ryan Dahl, creator of Node.js moved to Go.

TJ Holowaychuk, creator of Express.js framework moved to Go.

Boston dynamics showing off https://youtu.be/-e1_QhJ1EhQ

Chat GPT4 talk https://www.theverge.com/23560328/openai-gpt-4-rumor-release-date-sam-altman-interview

Fyne IDE, currently work in progress https://github.com/fyne-io/defyne

1995 paper by the Niklaus Wirth “A Plea for Lean Software” https://cr.yp.to/bib/1995/wirth.pdf

Benchmarking golang code

Let’s say that you want to know if EncodeToString is faster than fmt.Sprintf : you will need to compare the speed of this method

func Md5Encode(str string) string {
	md5HashInBytes := md5.Sum([]byte(str))
	md5HashInString := hex.EncodeToString(md5HashInBytes[:])
	return md5HashInString
}

with this other one

func Md5EncodeFmt(str string) string {
	md5HashInBytes := md5.Sum([]byte(str))
	md5HashInString := fmt.Sprintf("%x", md5HashInBytes)
	return md5HashInString
}

Go provides benchmarking features in the testing package which is pretty usefull :

func BenchmarkMd5EncodeFmt(b *testing.B) {
	// run the md5Encode function b.N times
	for n := 0; n < b.N; n++ {
		Md5EncodeFmt("aldfhasdl la fasdfeo8ekldjh asdkj fh lksdjfhwoieuxnroiAUN;laiDJ;ANIfub;OEIRBUF;OEfuN;ALFJ;AL")
	}
}

func BenchmarkMd5Encode(b *testing.B) {
	// run the md5Encode function b.N times
	for n := 0; n < b.N; n++ {
		Md5Encode("aldfhasdl la fasdfeo8ekldjh asdkj fh lksdjfhwoieuxnroiAUN;laiDJ;ANIfub;OEIRBUF;OEfuN;ALFJ;AL")
	}
}

Run

$ go test -bench=.
goos: linux
goarch: amd64
BenchmarkMd5EncodeFmt-8   	 1894791	       625 ns/op
BenchmarkMd5Encode-8      	 3068509	       363 ns/op
PASS
ok  	_/home/paul/LazyInit/bench	3.342s

Run 3 times the benchmarks :

$ go test -count 3 -bench=. 
goos: linux
goarch: amd64
BenchmarkMd5EncodeFmt-8   	 1882105	       627 ns/op
BenchmarkMd5EncodeFmt-8   	 1918942	       624 ns/op
BenchmarkMd5EncodeFmt-8   	 1902894	       625 ns/op
BenchmarkMd5Encode-8      	 3139585	       386 ns/op
BenchmarkMd5Encode-8      	 2937154	       397 ns/op
BenchmarkMd5Encode-8      	 3009801	       380 ns/op
PASS
ok  	_/home/paul/LazyInit/bench	10.217s

EncodeToString() makes your method almost twice faster !

In the case you need to isolate the code that will be measured from some init/fixture code you can use ResetTimer() and StopTimer() to accurately isolate what you want to measure :

func BenchmarkMd5EncodeFmt(b *testing.B) {
	// Some init code
	initRandomBenchStrings()
	b.ResetTimer()
	for n := 0; n < b.N; n++ {
		Md5EncodeFmt(getRandomstrings())
	}
	b.StopTimer()
	// some final code
}

If the function you’re measuring is very slow you might want to increase to maximum time of excecution of the benchmark (default is 1 s) with -benchtime=20s

Recap

  • without any benchmarks : go test .
  • with benchmarks (time) : go test -bench .
  • with benchmarks (time and memory) : go test -bench . -benchmem

The argument following -bench is a regular expression. All benchmark functions whose names match are executed. The . in the previous examples isn’t the current directory but a pattern matching all tests. To run a specific benchmark, use the regexp : -bench Suite (means everything containing Suite).

(thanks to https://github.com/samonzeweb/profilinggo )

Thanks year 2000 : less is (immensely) more (the 90s produced a lot of crap)

Thanks to god after year 2000 information technology has started moving towards more pragmatic, simple and effective tools and languages. Some examples that in my opinion make this evident : 

Languages and language tools

  • go, rust, swift are all born with the goal of simplifying their direct parents (c++, objectiveC) and removing their pitfalls.
  • UML abandoned : this is a relief for all coders which had to deal with it. I don’t know anyone using it nowadays.
  • git : finally some one (thanks Linux Torvalds) simplified svn/sourcesafe by putting features that are needed by developers in a clear, pretty intuitive command line interface
  • atom/sublime : reaction to the complexity of Visual Studio, IBM Rational, Eclipse ? I think yes

Databases

  • Key-value stores/noSQL are just taking ER/SQL model and making it simpler, providing only the features needed in 99% of the applications. Boyce-Codd normal form is pretty nice and interesting but in real world applications you’ll never use it. 
  • Object Databases completely disappeared and in some way also the idea that OO methodology/hierarchy could be applied everywhere (just because you are where using OO languages)

Virtualization

  • docker/rkt are slim alternatives to virtualization and virtual machines

Architectures

  • plain old REST API aren’t just a simple way for doing things without having to Corba/Soap ?
  • gRPC : provides corba like features while being 1 order of magnitude more efficent and portable on any platform.

What I’m saying is that the 90s produced a lot of unnecessarily complicated tools and technology which developer just did not need/like which is being progressively substituted with simpler stuff.

Interesting to note that the phrase “Less is more” is originally attributed to Mies Van Der Rohe for his minimalism in architecture design . Looks at his buildings : nothing more than necessary and functional elements are present.

But before him Leonardo da Vinci : “Simplicity is the Ultimate Sophistication”

Profiling a golang REST API server

Profiling :

is a form of dynamic program analysis that measures, for example, the space (memory) or time complexity of a program, the usage of particular instructions, or the frequency and duration of function calls. Most commonly, profiling information serves to aid program optimization.

How can you profile your golang REST API server in a super simple way :

First : add some lines to your server code

import _ "net/http/pprof"

And then add a listener (I normally use a command line flag to trigger this) :

go func() {
http.ListenAndServe("localhost:6000", nil)
}()

Start your server and generate some load. While your code is running under the load you generated extract the profiler data :

go tool pprof http://localhost:6000/debug/pprof/profile
Fetching profile over HTTP from http://localhost:6000/debug/pprof/profile
Saved profile in /home/paul/pprof/pprof.wm-server.samples.cpu.008.pb.gz
File: wm-server
Build ID: c806572b51954da99ceb779f6d7eee3600eae0fb
Type: cpu
Time: Dec 19, 2018 at 1:41pm (CET)
Duration: 30.13s, Total samples = 17.35s (57.58%)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof)

You have many commands at this point but what I prefer to do, having used kcachegrind for years, is to fire it up using the kcachegrind command :

(pprof) kcachegrind

This will generate a callgrind formatted file and run kcachegrind on it to let you do all the usual analysis that you’re probably already used to do (call graph, callers, callees ..)

Yes, I like go programming language

Yes, I like Go programming language. I’m liking it so much that I have to resist from being a fan boy. I’m trying to understand where all this enthusiasm is coming from (I’m a seasoned coder)  so here’s an attempt to find why :

  1. Code readability — and maintainability — first, language features second
  2. Integrated test environment : go test <package> executes all tests for the package. Unit testing features are builtin.
  3. Code Coverage is builtin (with some limitations, for example if you use cgo it will not work).
  4. Integrated tool chain : no need to have makefile at the cost of rigid hierarchy of data.
  5. Exhaustive standard library containing everything you need to do server-side/network programming
  6. Good Multi-threading features/model included in language (sync package, goroutines, channels), fast goroutines thanks to segmented stack implementation.
  7. Basic set of OOP features, centered on composition, not inheritance : you won’t be able to mess up your code at the cost of not being perceived as an OO language by OO fanboys. For more details on whether go is oo or not go here.
  8. Go is backed by some Famous Names in computing, and this inspires confidence.
  9. CamelCase 🙂 ? Naaah, I hate camel case but I like the choice of having standard style, comments, indent; all supported by the language via go fmt package so that all code will look coherent.

So basically I like the fact that Go is a very opinionated language. You might like the single decisions or not but what I like most is that someone took care of taking them for you (so you don’t have to enforce them team wide or company wide)
Interesting read also on how and why go was born : quoting from Rob Pike speech at go conference SF 2012 :

“To put it another way, oversimplifying of course:

Python and Ruby programmers come to Go because they don’t have to surrender much expressiveness, but gain performance and get to play with concurrency.

C++ programmers don’t come to Go because they have fought hard to gain exquisite control of their programming domain, and don’t want to surrender any of it. To them, software isn’t just about getting the job done, it’s about doing it a certain way.

The issue, then, is that Go’s success would contradict their world view.

And we should have realized that from the beginning. People who are excited about C++11’s new features are not going to care about a language that has so much less.  Even if, in the end, it offers so much more.”