Category Archives: coding

January Reading list

Found this interesting magazine Increment (thanks Swan !) and I really like it :
https://increment.com/software-architecture/case-studies-in-rearchitecting/

My oldest Email address : paul@cusun.sublink.org (I guess), 1990
Oldest reference to something written by me on the net : https://groups.google.com/g/comp.unix.questions/c/UwGUVEAFKz4/m/mzybPpn3mkwJ
there should be something even older by I guess google does not have newsgroups from 1989/90 …

Difference between Reward and Recognition : https://intenseminimalism.com/2020/motivating-motivation-from-reward-to-recognition/

Alec Ross

https://en.wikipedia.org/wiki/Code_Complete

https://khaninews.blogspot.com/2019/11/how-computers-are-reinventing-chess.html

“Simple things should be simple, complex things should be possible” Alan Kay https://slideplayer.com/slide/7456299/

AssemblyScript https://www.assemblyscript.org/

Leadership : https://intenseminimalism.com/2015/the-six-styles-of-leadership/

Again on leadership : https://intenseminimalism.com/2015/disassembling-leadership-team-and-vision-are-two-different-skills/

Interview with Ken Thompson

(this comes from a DrDobbs’s post May 18, 2011 – the article disappeared from the site so I’m just saving it here)

The creator of UNIX discusses writing UNIX, the Go language, and collaborating with Dennis Ritchie

The Japan Prize, one of the highest honors awarded for outstanding contribution to science and technology, was awarded jointly this year to Ken Thompson and Dennis Ritchie for the creation of UNIX. The prize is normally given to the recipients at a lavish banquet in Tokyo attended by the emperor. However, due to the April earthquake and tsunami, the prizes this year were distributed at the honorees’ place of work. I was able to attend the ceremony for Ken Thompson, held at Google headquarters, where he currently works. After the ceremony, he consented to this exclusive interview.


DDJ: Congratulations on winning this prize.

KT: Thanks.

Developing UNIX

DDJ: You’ve received a lot of awards over the years for UNIX. At what point in UNIX’s development did it become clear it was going to be something much bigger than you’d anticipated?

KT: The actual magnitute, that no one could have guessed. I gather it’s still growing now. I thought it would be useful to essentially anybody like me because it was not built for someone else or some third party. That was a perjorative term then. It was written for Dennis and me and our group to do its work. And I think it would have been useful to anybody who did the kind of work that we did. And therefore, I always thought it was something really good that was going to take off.

Especially the language [C]. The language grew up with one of the rewritings of the system and, as such, it became perfect for writing systems. We would change it daily as we ran into trouble building UNIX out of the language and we’d modify it for our needs.

DDJ: A symbiosis of sorts…

KT: Yeah. It became the perfect language for what it was designed to do. I always thought the language and the system were widely applicable.

DDJ: In the presentation today, it mentioned that UNIX was open source. Was UNIX open source from the beginning?

KT: Well there was no such term as “open source” then.

DDJ: I was under the impression that UNIX really became open source with the Berkeley distribution.

KT: No, we charged $100, which was essentially the reproduction cost of the tape, and then send it out. And we distributed, oh, probably close to 100 copies to universities and others.

Go Language

DDJ: Skipping several decades of work, let’s speak about Go. I was just at the Google I/O Conference, where it was announced that Go will be supported on the Google App Engine. Does that presage a wider adoption of Go within Google, or is it still experimental?

KT: It’s expanding every day and not being forced down anybody’s throat. It’s hard to adopt it to a project inside of Google because of the learning curve. It’s brand new and there aren’t good manuals for it, except what’s on the Web. And then, of course, its label of being experimental, so people are a little afraid. In spite of that, it’s growing very fast inside of Google.

DDJ: In the presentation before the awarding of the Japan Prize today, you were quoted on the distinction between reasearch and development. [The former, Thompson stated, was directionless, whereas development had a specific goal in mind.] So in that context, is Go experimental?

KT: Yes. When the three of us [Thompson, Rob Pike, and Robert Griesemer] got started, it was pure research. The three of us got together and decided that we hated C++. [laughter]

DDJ: I think there’d be a lot of people who are with you on that.

KT: It’s too complex. And going back, if we’d thought of it, we’d have done an object-oriented version of C back in the old days.

DDJ: You’re saying you would have?

KT: Yes, but we were not evangelists of object orientation. [Returning to Go,] we started off with the idea that all three of us had to be talked into every feature in the language, so there was no extraneous garbage put into the language for any reason.

DDJ: It’s a lean language, indeed.

Collaboration with Dennis Ritchie

DDJ: Returning to UNIX, for a moment, when you and Dennis worked together, how did that collaboration operate? Were you working side by side?

KT: I did the first of two or three versions of UNIX all alone. And Dennis became an evangelist. Then there was a rewrite in a higher-level language that would come to be called C. He worked mostly on the language and on the I/O system, and I worked on all the rest of the operating system. That was for the PDP-11, which was serendipitous, because that was the computer that took over the academic community.

DDJ: Right.

KT: We collaborated every day. There was a lunch that we went to. And we’d talk over lunch. Then, at night, we each worked from our separate homes but we were in constant communication. In those days, we had mail and writ (pronounced ‘write’), and writ would pop up on your screen and say there was a message from so-and-so.

DDJ: So, IM essentially.

KT: Yes, IM. There was no doubt about that! And we discussed things from home with writ. We worked very well together and didn’t collaborate a lot except to decide who was going to do what. Then we’d run and very independently do separate things. Rarely did we ever work on the same thing.

DDJ: Was there any concept of looking at each other’s code or doing code reviews?

KT: [Shaking head] We were all pretty good coders.

DDJ: I suspect you probably were! [Laughter]

SCM

DDJ: Did you use any kind of source code management product when working together?

KT: No, those products really came later; after UNIX. We had something like it, which we called “the code motel” because you could check your code in but you couldn’t check it out! So, really, no we didn’t.

DDJ: I bet you use SCM today in your work on Go.

KT: Oh, yes, Google makes us do that!

Golang ElasticSearch for beginners

(You may not be familiar with what elasticsearch is : for a good introduction checkout this post from Igor Kopanev)
Extracting data from ElasticSearch in golang is not a simple task. First of all you might not be the one who wrote data into it so these are some commands to find out how is data structured in your ES server :

List all your indexes

$ curl https://<your-server>/_aliases?pretty=true
{  
  "billingdata" : {
    "aliases" : { }
  },
  "checkpoints" : {
    "aliases" : { }
  }
}

Your indexes data structures can be queried with :

curl https://<your-server>/<index_name> 


Now that I had an idea of how data was structured I started looking at the ES official library ( https://github.com/elastic/go-elasticsearch/ ) to find out that it is probably not used that much directly. This a code example to execute a query :

  // Build the request body.
  var buf bytes.Buffer
  query := map[string]interface{}{
    "query": map[string]interface{}{
      "match": map[string]interface{}{
        "title": "test",
      },
    },
  }
  if err := json.NewEncoder(&buf).Encode(query); err != nil {
    log.Fatalf("Error encoding query: %s", err)
  }
  // Perform the search request.
  res, err = es.Search(
    es.Search.WithContext(context.Background()),
    es.Search.WithIndex("billingdata"),
    es.Search.WithBody(&buf),
    es.Search.WithTrackTotalHits(true),
    es.Search.WithPretty(),
  )

Then (God bless Oliver Eilhard) I ray of light came in through my window and I found this package : https://godoc.org/github.com/olivere/elastic that makes everything much more simple :

	searchResult, err := es6.Search().
		Index("billingdata").
		// Query(timerangeQuery). // if you need to filter
		Pretty(true).
		Do(context.Background()) // execute

Much better. If you need a time range query :

// sdate,edate are time.Time	
timerangeQuery := elastic.NewBoolQuery().
		Filter(elastic.NewRangeQuery("@timestamp").
		From(sdate).
		To(edate))

Or maybe you need to query by time range and field value :

companyQuery = elastic.NewTermQuery("company_id", CompanyId)
// combine queries :
CombinedQuery = elastic.NewBoolQuery()
CombinedQuery = CombinedQuery.Must(timerangeQuery).Must(companyQuery)

// now search using Query(CombinedQuery)

Setting up your ES Client is pretty easy :

	// Create a custom HTTP client to setup timeouts and TLS config
	// esHTTPClient := &http.Client{
	// 	Timeout: time.Second * 10,
	// 	Transport: &http.Transport{
	// 		Dial: (&net.Dialer{
	// 			Timeout: 5 * time.Second,
	// 		}).Dial,
	// 		TLSHandshakeTimeout: 5 * time.Second,
	// 		TLSClientConfig: &tls.Config{
	// 			InsecureSkipVerify: true,
	// 		},
	// 	},
	// }

	// Create a client
	client, err := elastic.NewClient(
		// elastic.SetHttpClient(esHTTPClient),
		elastic.SetSniff(false),
		elastic.SetURL("http://localhost:9200"))

You can also aggregate data easily, sum, avergage. I’ll come up with more examples as soon as I have some working code.

October reading list

“simplicity, or the art of maximizing the work not done” : I really like this sentence, from Marco Cecconi https://sklivvz.com/posts/consider-using-simple-models-instead. Kind of summarizes my thoughts around simplicity in software, removing accidental complexity, over engineering code, get rid of complex and slow learning curve tools and languages. Thanks to Orfware for reference to Marco.

CORS : https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS

Golang elasticsearch API : kudos to Oliver Eilhard for his package! https://godoc.org/github.com/olivere/elastic

What is Software Development ?

According to Steve McConnell “Code Complete” there are plenty of metaphors out there:

This last view seems to be shared by many ( https://signalvnoise.com/posts/591-brainstorm-the-software-garden , Jeff Atwood likes it too https://blog.codinghorror.com/tending-your-software-garden/ and I feel close to him on really many things) and I tend to see it more close to what software development is today : a continuously changing (growing) artifact, not totally manageable, that requires constant maintenance (here the building metaphor fails a bit), designed and redesigned over time like a garden, subject to seasons (“lots of new requirements” season, “robustness and reliability” season).

September reading list

Nice article on gRPC in a js client, Envoy and go gRPC server https://medium.com/swlh/building-a-realtime-dashboard-with-reactjs-go-grpc-and-envoy-7be155dfabfb

https://blog.codinghorror.com/tending-your-software-garden/

https://blog.codinghorror.com/bridges-software-engineering-and-god/

https://signalvnoise.com/posts/591-brainstorm-the-software-garden

https://blog.codinghorror.com/the-big-ball-of-mud-and-other-architectural-disasters/

Reaction/rejection to complexity and “worse is better”

Some points to further investigate : the main idea is that early 2000 years started a reaction to (unnecessary) complexity :

start here : https://medium.com/better-programming/how-unix-works-everything-you-were-too-afraid-to-ask-f8396aeb2763

simple is better (worse is better) : https://blog.codinghorror.com/worse-is-better/ the point here is all about what is considered to be better. Some furher reading “what’s wrong with word simple

Why use inheritance when a simple object will do : https://www.philweber.com/2002/11/is_inheritance_overrated

optimism in coding is dangerous : https://blog.codinghorror.com/defeating-optimism/

worse is better : https://www.dreamsongs.com/Files/WorseIsBetterPositionPaper.pdf

worse is better http://wiki.c2.com/?WorseIsBetter

again : https://www.jwz.org/doc/worse-is-better.html

Mar 2020 reading list

Image optimization and compression is one of main features of ImageEngine CDN (I work on this project). This Netflix blog article is great at going into details of why we probably need a new, modern image format : “AVIF for Next-Generation Image Coding” https://netflixtechblog.com/avif-for-next-generation-image-coding-b1d75675fe4

Faster Paint Metrics with Responsive Image Optimization CDNs
https://medium.com/@firt/faster-paint-metrics-with-responsive-image-optimization-cdns-d43340d4a48c

Speed up Woocommerce with an Image CDN https://hackernoon.com/speed-up-woocommerce-with-an-image-cdn-kt1q3yeg

Faster Image delivery with Chrome Light mode https://medium.com/imageengine/faster-image-delivery-with-chromes-lite-mode-93900eac126e

How to Reduce the Carbon Footprint of Your Website by 50% https://imageengine.io/blog/how-to-reduce-the-carbon-footprint-of-your-website-by-50/

Images are not static content https://css-tricks.com/images-are-not-static-content/

Prepare for the Ultimate Gaslighting https://forge.medium.com/prepare-for-the-ultimate-gaslighting-6a8ce3f0a0e0

TeamCity : pin and tag a build using the REST API from inside the script

We use Teamcity for most of automation in building/testing/deploying golang microservices to prod and during test phase I wanted to be able to notice easily if an integration/benchmark test crashed a microservice. Pinning the build and tagging it with the word “panic” seemed to be a good idea, from inside the buildconf script :

# txt are are the microservice logfiles, substitute as neeeded
grep panic.go *.txt
retVal=$?
if [ $retVal -eq 0 ]; then
   # grep found panic in some file, tag with panic
   nohup sh -c 'sleep 10 ; curl --header "Origin: https://<your_tc_server>" --header "Authorization: Bearer <yourbearertoken>" --request POST "%teamcity.serverUrl%/app/rest/builds/id:%teamcity.build.id%/tags/" --data panic --header "Content-type: text/plain"' &
   nohup sh -c 'sleep 10 ; curl --header "Origin: https://<your_tc_server>" --header "Authorization: Bearer <yourbearertoken>" --request PUT "%teamcity.serverUrl%/app/rest/builds/id:%teamcity.build.id%/pin/"' &
fi

Took more than expected to figure out this : some examples would be helpful. Note that the build needs to be finished to be able to pin it/tag it so we have to put this ugly sleep to postpone operations on the rest api when the build is finished.