Coordinating different goroutines to achieve a single goal

Image for post
Image for post
Photo by Hunter Harritt on Unsplash

In a previous article I talked about how we can use the context.Context struct to share data and group distinct goroutines. One thing that’s missing from that article, is how to synchronise that group of things. Synchronise concurrent tasks means either wait for all of them to finish before doing something else, or cancelling all of them in case things go wrong. That’s where the sync/errgroup comes to the scene. Here are the official docs for the package. The package name conveys two things: used to synchronise, and we are going to be dealing with errors. That pretty much matches what was mentioned above, right? The name errgroup may sound a bit weird as the mechanism to synchronise goroutines, but bear with me, it will make sense later. Essentially you will be registering goroutines to that group and wait for them to finish, one way or the other. …


Let’s write

There’s no one size fits all when it comes to testing apps

Image for post
Image for post
Photo by Scott Graham on Unsplash

Most people agree that testing software is one of the most important activities software engineers perform. Nothing matters if a piece of software is very fast or well written but it doesn’t accomplish what it is intended to. Testing is a very interesting discipline and definitely an important one to master if you want to become a better software engineer. In this article, we are going to review some very good patterns for improving the tests of a Golang application. We will go through various different scenarios that differ in nature, complexity, and objectives we want to achieve with tests.

Testing package

Golang has a testing package as part of its standard library. The official documentation is very thorough and contains a lot of examples. In short, it allows you to write automated tests that can be run with the go test command. When creating tests you have to follow the convention of naming the testing function as TestXxxxx where Xxxx does not start with a lowercase letter. The package provides all the infrastructure you need for running, failing, skipping, log, and so on. …


Let’s get some

Harnessing one of the most useful packages in go

Image for post
Image for post
Photo by Pietro Jeng on Unsplash

Applications in golang use Contexts for controlling and managing very important aspects of reliable applications, such as cancellation and data sharing in concurrent programming. This may sound trivial but in reality, it’s not that so. The entry point for the contexts in golang is the context package. It is extremely useful and probably one of the most versatile packages of the entire language. If you haven’t come across anything dealing with contexts yet, you probably will very soon (or maybe you just didn’t pay much attention to it). The usage of context is so widespread that multiple other packages rely on it and assume you will do the same. …


Tracing requests in distributed systems

Image for post
Image for post
Photo by Stephen Dawson on Unsplash

In this article we will be focusing on the implementation of the architecture presented in the previous article. Take your time to review it.

The entire code can be found on my github repo. I’ll be creating and pasting some gists here for reference, those will only contain the most relevant and important pieces of the code. So I really encourage you to check my github for a full reference. …


Understanding OpenTelemetry and distributed tracing

Image for post
Image for post
Photo by Luke Chesser on Unsplash

Running distributed systems in production is not an easy task. Access is limited, therefore issues are hard to track down and understand as well as to reproduce in other environments. No other environment is like production, no matter how similar it may seem. Usually you rely on different pieces of information to troubleshoot and understand what is happening to your applications in production. The most common ones are logs and custom metrics. This two things are really powerful when combined and used together. With logs you can understand and troubleshoot exactly what happened with a specific request or a scenario, while with metrics you can see the overall behaviour and performance of the system. It looks like a really good solution, and it really is, the problem is tuning it to reflect exactly what you need. …


Combining SonarQube and gosec to get more information out of your source code

Image for post
Image for post
Photo by Carlos Muza on Unsplash

Static analysis (or static code analysis) is a great technique to find issues related to security, performance, coverage, coding style, and some times even logic without running your application. Usually this type of analysis is run against the source code of the application only. No execution required.

This is a very easy and powerful type of analysis to run against new code being integrated. Most tools are designed to run as part of the continuous integration pipeline because of that. You can compare the results from the new code and create acceptance rules based on that. This will help the code reviewing process. For example, you may want to automatically reject a pull request if a new major security issue is introduced on that, or if the test coverage is not satisfactory. …


How to harness immutability to enhance readability and stability in your Golang app

Image for post
Image for post
Photo by Ferenc Almasi on Unsplash

The concept of immutability is very simple. After an object (or struct) is created, it can never be changed. It’s immutable. Although the concept seems simple, it’s not always easy to use it or benefit from it.

As most things in computer science (and life), there are multiple ways of achieving the same result, in regards to immutability it is not different. You have to consider it as another tool in your toolkit and use what works best for each issue you face. One very strong use case for immutability is when you are working with concurrent programming. Golang was designed with concurrency in mind, so it’s very common to use concurrency in go. …


Leveraging Golang’s concurrency model to create concurrent data pipelines for data intensive applications

Image for post
Image for post
Photo by Mika Baumeister on Unsplash

Data is a vital piece for all applications. Applications mainly receive data, process it and then create some sort of output out of it. The amount of data that is available nowadays is immense, which brings many challenges when trying to make sense out of it and transforming it into useful information. Besides the usual data system process, we also have to consider the data that is generated by the execution of such systems, like logs and metrics. Everything is data.

Therefore, there’s a growing need for data driven applications. It’s not unusual to have systems only dealing with moving and transforming data. From one place to another or from one structure to another. That is even more common in distributed systems, because each system generates data for their own purpose, structured the way it’s easier for them to process. However, those systems still have to communicate, so the data has to be transformed and moved. Moreover, sometimes the data has to be transferred to a different environment, or to a different platform. …


Creating pipelines for handling requests

Image for post
Image for post
Photo by Remy Ludo Gieling on Unsplash

When working with web apps, like web API’s, web sites, or any application that exposes a web server and handle requests, the most common behaviour is to handle each URL in a different way. Return different web pages, persist data in a storage, fetch and return data as JSON, among many other things.

However, some tasks need to be executed regardless of the request you receive in a generic way. Copying and pasting the code or a call to a separate function, although being an option, is not the best way of approaching this issue. One problem with that approach is code duplication, and another one is human error. Maybe some developer forgets to add the call to that function that gathers metrics for a given URL, or logs errors. That will cause some headache later. Big time. …


Simple yet powerful technique to implement dependency injection in golang

Image for post
Image for post
Photo by Markus Spiske on Unsplash

Dependency injection is great technique for decoupling code, making it more stable, testable and robust. The technique consists in designing your code to receive dependencies. In object oriented languages, we usually use the class constructor to do it. So when someone creates an instance of your object, all dependencies will be injected into it. That makes perfect sense as dependencies shouldn’t be part of interface signatures, otherwise all implementations would have to receive the dependencies as parameters for doing their job. But each implementation can have different dependencies based on what they do. Well you get the problem, right?

Awesome, so in an object oriented world, you just have to design your code to receive the dependencies on the constructor and you have a very decouple code ready for being tested using mocks and all that jazz. What’s the deal then? …

About

Ricardo Linck

Software Engineer. Distributed systems lover, golang and .net enthusiast. Curious by nature. https://github.com/ricardolinck

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store