Recently I came across a scenario where I needed to execute some long running functions and capture the results for later use. Of course to reduce the latency of this code I ran the functions concurrently with goroutines and a waitgroup. The kicker here was that to return the results from the goroutines I had to use channels and getting the complete data from the channel was more complex than I thought.
My first attempt at this problem looked like this (long running code replaced by a trivial string example, but you get the point):
Now what happens when you run this code? If you guessed deadlocks…you’d be right!
The reason that the deadlock occurs here is due to the nature of channels. If you checkout the Golang Tour you’ll see that unbuffered channels block on sends and receives. My code is blocking on sending to the channel hence the goroutines never complete and my receiving code never executes.
The solution here is initially a little confusing but once implemented makes a lot of sense:
Notice that the wg.Wait() and close(ch) have been moved into another goroutine. This allows the receiving code and the waitgroup/channel management to run on separate goroutines. This allows the receiving code to run which unblocks the channel sends and as such the goroutines are able to complete. Worth noting that the waitgroup is only concerned with the first 2 goroutines where the work is happening.
Taking Things Further
The solution that was applied here can be taken even further. We could extend the goroutines further to return data to additional channels. Say for example we wanted to return data but also capture any errors that occurred during the process. We could do the following:
Notice that we have added a second channel of type err. We have also made this a buffered channel of length 2, this is because we know that a maximum of 2 errors could ever be returned here. Also unbuffered channels only block when the buffer is full. We can use that to our advantage here; as we don’t know how many errors will be occur (0, 1 or 2) we use a buffer to prevent the errors channel from blocking us and causing more deadlocks.