Author Archives: paulborile

About paulborile

I’m a multi-skilled IT professional with a good all-round supervisory and technical expertise. Extensive, 20+ years of professional experience in software development allowed me to investigate computer science and software engineering inside out. During these years I built up a solid base of design patterns, software architectures and programming languages such as C/C++, Golang, Java, Python, SQL, Assembly (and many others). I worked on mission-critical and multi-channel applications, applying distributed computing, messaging, image/data processing and computer graphics techniques. I faced both architecture design and systems rearchitecting, microservices introduction and technology migration as well as company wide adoption of new technologies/methodologies multiple times. As an entrepreneur I have built and grown teams and development organizations from the ground up (internal/out sourced/at customer site) focusing on software engineering methodologies as well as recruiting, budget/financial control and operations support. I am particularly interested in software testing methodologies, software quality metrics and tools to make software development faster and better. Currently leading the Italian development team for ScientiaMobile Inc, a Reston (US) based startup focused on image optimizing CDN and mobile detection technologies and services. Born in Dearborn Michigan and living in Italy since many years now I speak fluently both English and Italian, studied French and learned some Russian while working for some time for a Olivetti/Aeroflot project.

Golang ElasticSearch for beginners

(You may not be familiar with what elasticsearch is : for a good introduction checkout this post from Igor Kopanev)
Extracting data from ElasticSearch in golang is not a simple task. First of all you might not be the one who wrote data into it so these are some commands to find out how is data structured in your ES server :

List all your indexes

$ curl https://<your-server>/_aliases?pretty=true
{  
  "billingdata" : {
    "aliases" : { }
  },
  "checkpoints" : {
    "aliases" : { }
  }
}

Your indexes data structures can be queried with :

curl https://<your-server>/<index_name> 


Now that I had an idea of how data was structured I started looking at the ES official library ( https://github.com/elastic/go-elasticsearch/ ) to find out that it is probably not used that much directly. This a code example to execute a query :

  // Build the request body.
  var buf bytes.Buffer
  query := map[string]interface{}{
    "query": map[string]interface{}{
      "match": map[string]interface{}{
        "title": "test",
      },
    },
  }
  if err := json.NewEncoder(&buf).Encode(query); err != nil {
    log.Fatalf("Error encoding query: %s", err)
  }
  // Perform the search request.
  res, err = es.Search(
    es.Search.WithContext(context.Background()),
    es.Search.WithIndex("billingdata"),
    es.Search.WithBody(&buf),
    es.Search.WithTrackTotalHits(true),
    es.Search.WithPretty(),
  )

Then (God bless Oliver Eilhard) I ray of light came in through my window and I found this package : https://godoc.org/github.com/olivere/elastic that makes everything much more simple :

	searchResult, err := es6.Search().
		Index("billingdata").
		// Query(timerangeQuery). // if you need to filter
		Pretty(true).
		Do(context.Background()) // execute

Much better. If you need a time range query :

// sdate,edate are time.Time	
timerangeQuery := elastic.NewBoolQuery().
		Filter(elastic.NewRangeQuery("@timestamp").
		From(sdate).
		To(edate))

Or maybe you need to query by time range and field value :

companyQuery = elastic.NewTermQuery("company_id", CompanyId)
// combine queries :
CombinedQuery = elastic.NewBoolQuery()
CombinedQuery = CombinedQuery.Must(timerangeQuery).Must(companyQuery)

// now search using Query(CombinedQuery)

Setting up your ES Client is pretty easy :

	// Create a custom HTTP client to setup timeouts and TLS config
	// esHTTPClient := &http.Client{
	// 	Timeout: time.Second * 10,
	// 	Transport: &http.Transport{
	// 		Dial: (&net.Dialer{
	// 			Timeout: 5 * time.Second,
	// 		}).Dial,
	// 		TLSHandshakeTimeout: 5 * time.Second,
	// 		TLSClientConfig: &tls.Config{
	// 			InsecureSkipVerify: true,
	// 		},
	// 	},
	// }

	// Create a client
	client, err := elastic.NewClient(
		// elastic.SetHttpClient(esHTTPClient),
		elastic.SetSniff(false),
		elastic.SetURL("http://localhost:9200"))

You can also aggregate data easily, sum, avergage. I’ll come up with more examples as soon as I have some working code.

Val Sassina

October reading list

“simplicity, or the art of maximizing the work not done” : I really like this sentence, from Marco Cecconi https://sklivvz.com/posts/consider-using-simple-models-instead. Kind of summarizes my thoughts around simplicity in software, removing accidental complexity, over engineering code, get rid of complex and slow learning curve tools and languages. Thanks to Orfware for reference to Marco.

CORS : https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS

Golang elasticsearch API : kudos to Oliver Eilhard for his package! https://godoc.org/github.com/olivere/elastic

What is Software Development ?

According to Steve McConnell “Code Complete” there are plenty of metaphors out there:

This last view seems to be shared by many ( https://signalvnoise.com/posts/591-brainstorm-the-software-garden , Jeff Atwood likes it too https://blog.codinghorror.com/tending-your-software-garden/ and I feel close to him on really many things) and I tend to see it more close to what software development is today : a continuously changing (growing) artifact, not totally manageable, that requires constant maintenance (here the building metaphor fails a bit), designed and redesigned over time like a garden, subject to seasons (“lots of new requirements” season, “robustness and reliability” season).

September reading list

Nice article on gRPC in a js client, Envoy and go gRPC server https://medium.com/swlh/building-a-realtime-dashboard-with-reactjs-go-grpc-and-envoy-7be155dfabfb

https://blog.codinghorror.com/tending-your-software-garden/

https://blog.codinghorror.com/bridges-software-engineering-and-god/

https://signalvnoise.com/posts/591-brainstorm-the-software-garden

https://blog.codinghorror.com/the-big-ball-of-mud-and-other-architectural-disasters/