Author Archives: paulborile

About paulborile

I’m a multi-skilled IT professional with a good all-round supervisory and technical expertise. Extensive, 20+ years of professional experience in software development allowed me to investigate computer science and software engineering inside out. During these years I built up a solid base of design patterns, software architectures and programming languages such as C/C++, Golang, Java, Python, SQL, Assembly (and many others). I worked on mission-critical and multi-channel applications, applying distributed computing, messaging, image/data processing and computer graphics techniques. I faced both architecture design and systems rearchitecting, microservices introduction and technology migration as well as company wide adoption of new technologies/methodologies multiple times. As an entrepreneur I have built and grown teams and development organizations from the ground up (internal/out sourced/at customer site) focusing on software engineering methodologies as well as recruiting, budget/financial control and operations support. I am particularly interested in software testing methodologies, software quality metrics and tools to make software development faster and better. Currently leading the Italian development team for ScientiaMobile Inc, a Reston (US) based startup focused on image optimizing CDN and mobile detection technologies and services. Born in Dearborn Michigan and living in Italy since many years now I speak fluently both English and Italian, studied French and learned some Russian while working for some time for a Olivetti/Aeroflot project.

Feb 2020 reading list

Python package formats : it’s hard to keep track of them 🙂

https://www.bernat.tech/pep-517-and-python-packaging/

Always though that variables shadowing was a problem : https://doc.rust-lang.org/book/ch03-01-variables-and-mutability.html#shadowing
and I still thinks so : https://mail.mozilla.org/pipermail/rust-dev/2013-May/004306.html

Microservice design : brokerless vs. broker/message bus based, interesting read

Nice dependency injection in Go : https://medium.com/@fsufitch/dependency-injection-and-testability-in-a-go-webservice-a91d0e5469dd

API Gateways : https://konghq.com/ https://tyk.io/

Goroutines in go 1.13 how they work : https://medium.com/a-journey-with-go/go-what-does-a-goroutine-switch-actually-involve-394c202dddb7

Monitoring air quality : http://www.esa.int/Applications/Observing_the_Earth/Copernicus/Monitoring_air_quality_takes_next_step

Croatia islands

Jan 2020 reading list

https://medium.com/@alexkatrompas/the-fall-of-the-software-engineer-the-rise-of-the-programmer-technician-451a572d28b0

Spotify Engineering culture : https://medium.com/productmanagement101/spotify-squad-framework-part-i-8f74bcfcd761

QUIC : https://quicwg.org/base-drafts/draft-ietf-quic-http.html

Teamcity REST API : https://confluence.jetbrains.com/display/TCD10/REST+API

Object-Oriented Programming — The Trillion Dollar Disaster https://medium.com/better-programming/object-oriented-programming-the-trillion-dollar-disaster-92a4b666c7c7

Stop using classes in JS : https://medium.com/javascript-in-plain-english/please-stop-using-classes-in-javascript-and-become-a-better-developer-a185c9fbede1

Interesting read about latency and how 99% Quantile may not be enough : https://bravenewgeek.com/everything-you-know-about-latency-is-wrong/

tc

TeamCity : pin and tag a build using the REST API from inside the script

We use Teamcity for most of automation in building/testing/deploying golang microservices to prod and during test phase I wanted to be able to notice easily if an integration/benchmark test crashed a microservice. Pinning the build and tagging it with the word “panic” seemed to be a good idea, from inside the buildconf script :

# txt are are the microservice logfiles, substitute as neeeded
grep panic.go *.txt
retVal=$?
if [ $retVal -eq 0 ]; then
   # grep found panic in some file, tag with panic
   nohup sh -c 'sleep 10 ; curl --header "Origin: https://<your_tc_server>" --header "Authorization: Bearer <yourbearertoken>" --request POST "%teamcity.serverUrl%/app/rest/builds/id:%teamcity.build.id%/tags/" --data panic --header "Content-type: text/plain"' &
   nohup sh -c 'sleep 10 ; curl --header "Origin: https://<your_tc_server>" --header "Authorization: Bearer <yourbearertoken>" --request PUT "%teamcity.serverUrl%/app/rest/builds/id:%teamcity.build.id%/pin/"' &
fi

Took more than expected to figure out this : some examples would be helpful. Note that the build needs to be finished to be able to pin it/tag it so we have to put this ugly sleep to postpone operations on the rest api when the build is finished.

grigna

On the importance of design in software

Many time we face situation were productivity of a software team is impaired by initial flaws in system design. These initial flaws require too much time to be completely removed (the effort of complete rewrite is only marginally touched here and is normally and indication of other problems in your software project )

So, in order for a software team to be able to work at its maximum, good design is a must. Design is much more responsible for productivity than any single coder, scrum master, product manager or development methodology/language in the sense that bad design can take an all-star team using top tools/methodology to perform badly.

There were times when it was taken for granted that before building a system, it was necessary to design it. That time looks gone (even though Agile does not explicitly prohibit making good design). The general idea seems to be that there is “never enough time to do something right, but there’s always enough time to do it over” again.

But what is design in software ?

That kind of intellectual activity which creates a whole from its diverse parts may be called the design of a system

Melvin E. Conway, How Do Committees Invent?

So you compose parts to make the whole system. How you do this ?

It is basically decomposition to generate single parts that acting together will generate the goal. The 2 tools you will need as a designer are :

  • decomposition
  • composition

 But first of all you need :

  • understanding of the system boundaries : the borders of the nation your system lives in
  • understanding the global scenario of what your system is going to do. Designing subsystems without knowing the whole picture is not a good idea.

Why decomposition of a system into smaller components (services) is good for you :

  • information hiding : well identified interfaces create a contract that the component has to provide, hiding any network/system implementation details. This will make the component implementation completely free to be modified/enhanced as long as the new implementation is compliant with the contract.
  • less development time : separate components/services can be developed in parallel since they don’t require (or require little) external dependencies, so less contention between engineers. Every component is independent on integration and performance test that can be developed autonomously.
  • scalability : with little effort in design every component might be deployed in a way to be one of many instances that will make that service scalable when traffic increases.
  • clarity : the system could be studied a component (service) at a time with the result that the whole system could be better designed because it was better understood

We could go on here and analyze what are the criteria to be used in decomposing a system into services. I’ll leave this to another moment and you can find some interesting notes here :

https://blog.acolyer.org/2016/09/05/on-the-criteria-to-be-used-in-decomposing-systems-into-modules/

What I would like to stress is that design is a fundamental phase in software engineering. You can’t just skip it and pretend that since you have a good team you’ll get a good job done.

Thanks to the following for inspiring me on this post :

https://en.wikipedia.org/wiki/Melvin_Conway

https://blog.acolyer.org/2019/12/13/how-do-committees-invent/

https://blog.acolyer.org/2016/09/05/on-the-criteria-to-be-used-in-decomposing-systems-into-modules/

Benchmarking golang code

Let’s say that you want to know if EncodeToString is faster than fmt.Sprintf : you will need to compare the speed of this method

func Md5Encode(str string) string {
	md5HashInBytes := md5.Sum([]byte(str))
	md5HashInString := hex.EncodeToString(md5HashInBytes[:])
	return md5HashInString
}

with this other one

func Md5EncodeFmt(str string) string {
	md5HashInBytes := md5.Sum([]byte(str))
	md5HashInString := fmt.Sprintf("%x", md5HashInBytes)
	return md5HashInString
}

Go provides benchmarking features in the testing package which is pretty usefull :

func BenchmarkMd5EncodeFmt(b *testing.B) {
	// run the md5Encode function b.N times
	for n := 0; n < b.N; n++ {
		Md5EncodeFmt("aldfhasdl la fasdfeo8ekldjh asdkj fh lksdjfhwoieuxnroiAUN;laiDJ;ANIfub;OEIRBUF;OEfuN;ALFJ;AL")
	}
}

func BenchmarkMd5Encode(b *testing.B) {
	// run the md5Encode function b.N times
	for n := 0; n < b.N; n++ {
		Md5Encode("aldfhasdl la fasdfeo8ekldjh asdkj fh lksdjfhwoieuxnroiAUN;laiDJ;ANIfub;OEIRBUF;OEfuN;ALFJ;AL")
	}
}

Run

$ go test -bench=.
goos: linux
goarch: amd64
BenchmarkMd5EncodeFmt-8   	 1894791	       625 ns/op
BenchmarkMd5Encode-8      	 3068509	       363 ns/op
PASS
ok  	_/home/paul/LazyInit/bench	3.342s

Run 3 times the benchmarks :

$ go test -count 3 -bench=. 
goos: linux
goarch: amd64
BenchmarkMd5EncodeFmt-8   	 1882105	       627 ns/op
BenchmarkMd5EncodeFmt-8   	 1918942	       624 ns/op
BenchmarkMd5EncodeFmt-8   	 1902894	       625 ns/op
BenchmarkMd5Encode-8      	 3139585	       386 ns/op
BenchmarkMd5Encode-8      	 2937154	       397 ns/op
BenchmarkMd5Encode-8      	 3009801	       380 ns/op
PASS
ok  	_/home/paul/LazyInit/bench	10.217s

EncodeToString() makes your method almost twice faster !

In the case you need to isolate the code that will be measured from some init/fixture code you can use ResetTimer() and StopTimer() to accurately isolate what you want to measure :

func BenchmarkMd5EncodeFmt(b *testing.B) {
	// Some init code
	initRandomBenchStrings()
	b.ResetTimer()
	for n := 0; n < b.N; n++ {
		Md5EncodeFmt(getRandomstrings())
	}
	b.StopTimer()
	// some final code
}

If the function you’re measuring is very slow you might want to increase to maximum time of excecution of the benchmark (default is 1 s) with -benchtime=20s

Recap

  • without any benchmarks : go test .
  • with benchmarks (time) : go test -bench .
  • with benchmarks (time and memory) : go test -bench . -benchmem

The argument following -bench is a regular expression. All benchmark functions whose names match are executed. The . in the previous examples isn’t the current directory but a pattern matching all tests. To run a specific benchmark, use the regexp : -bench Suite (means everything containing Suite).

(thanks to https://github.com/samonzeweb/profilinggo )