Designing and writing multi-threaded code is hard. Threading adds a debilitating amount of complexity to the Von Neumann architecture and, despite 50+ years of theory and implementation, the threading support in most programming languages is low level, the equivalent of malloc rather than GC managed memory.
If you’re a developer and you approach threading with trepidation then here are some experience tested guidelines and techniques that I try to follow. The overall strategy is to reduce the surface area for bugs and to increase the maintainability of the code. These are not hard and fast rules. It’s important to be pragmatic and to adjust for the specific situation.
But first, please note that I’m no expert. I’m sharing what I’ve learned and I know I’m not done learning. If this post helps you, I want to know. If you think I’ve got something wrong, I want to know. Thank you.
Avoid sharing data.
Sharing data between threads is the most common source of threading bugs. By definition many of the nastiest threading bugs can only occur when data is shared. Safely sharing data requires synchronization either in the form of a lock or an atomic primitive. Getting the synchronization right is not always easy. Avoid sharing data and the threading bug potential goes way down. As a bonus, the code can potentially run faster because the possibility of synchronization contention is eliminated.
It may seem that without sharing data, threads are pretty useless. Not so. But this is a case where you need to be willing and able to trade space for speed and safety. Give a thread it’s own private copy of everything it may need, let the thread run to completion (‘wait’ or ‘join’ on the thread), and then collect the results of it’s computations.
Avoid sharing logging.
Logging is data. See above. Sometimes the messages generated on a thread need to be reported more or less immediately. But if a delay can be tolerated, a thread can have it’s own private log that gets collected after the thread has run to completion.
When you must share, prefer lighter weight synchronization mechanisms.
I didn’t understand lock-free threading until I understood that lock-free doesn’t mean no synchronization mechanisms at all.
Under Windows the Interlocked* APIs represent atomic operations implemented by specific processor instructions. Critical sections are implemented via the interlocked functions. The interlocked functions and the critical section on Windows and the equivalent on other platforms are generally the lightest weigh synchronization mechanisms.
Technically the interlocked functions are not locks, they are hardware implemented primitives. But colloquially developers will speak of ‘locks’ and mean the whole set of synchronization mechanisms, hence my confusion over lock-free threading.
Having said that I will now forgo rigor and refer to all synchronization mechanisms as ‘locks’ because it’s pithier.
Use the smallest number of locks possible.
Don’t treat locks like magic pixie dust and sprinkle them everywhere. Synchronization locks provide safety but at a cost in performance and a greater potential for bugs. Yes, it’s kind of paradoxical.
Hold locks for the shortest length of time possible.
A critical section, for an example, allows only one thread to enter a section of code at a time. The longer the execution time of the section, the longer the window for other threads to box car up waiting for entry.
If there are values that can be computed before entering a critical section, do so. Only the statements that absolutely must be protected should be within the section.
Sometimes a value needs to be retrieved from a synchronized source, used as part of a computation, and a product of the computation needs to be stored to a synchronized destination. If the whole operation does not need to be atomic then, despite the desire to minimize locks, two independent synchronization locks could be better than one. Why? Because the source and destination are decoupled and the combined lock hold time is reduced because only the source read and the destination write are covered.
Be DRY - Don’t Repeat Yourself.
Always a good policy, being DRY has special importance with threaded code. There should be only one expression or implementation of any given synchronized operation. Every thread should be executing the same code to ensure that the operation is performed consistently.
Design to ensure that every acquisition of a lock is balanced by a release of the lock.
Take advantage of the RAII pattern or the dispose pattern or whatever is appropriate to the language and platform. Don’t rely on the developer (even when the developer is yourself) to remember to explicitly add every release for every acquisition.
Finish the threads you start.
Don’t fire and forget. Wait or join and clean up your threads and their resources.
Don’t let threads you created outlive the primary thread in the process. Some platforms have the unfortunate design of performing runtime set up and initialization, calling the program code, and then tearing down and de-initializing on the primary thread. Other threads that may still be running after the primary thread exits may fail when the runtime is pulled out from underneath them.
Don’t kill a thread. That will leave data in an undeterminable state. If appropriate implement a way to signal your thread to finish.
Don’t get focused on locking the data when it’s the operation that needs to be synchronized.
It’s easy to get fixated on the shared data but often times the design is better served by paying more attention to the shared operations.
Avoid nested locks.
Acquiring lock A and then lock B in one part of the code and elsewhere acquiring lock B and then lock A is asking for a deadlock. No-one sets out to write a deadlock. But sometimes the code isn’t exactly what was intended or a change is made and the impact of the change isn’t fully understood. If locks are never nested then the potential for this kind of deadlock just doesn’t exist.
If locks must be nested then alway acquire the locks in the same order and always release the locks in the same opposite order. And be DRY about it.
Avoid method calls within a lock.
This may seem aggressively limiting but method implementations can change and introduce new issues like unwanted nested locks. Keeping method calls out of the scope of locks reduces coupling and the potential for deleterious side effects.
(I think of this one as following in the spirit of LoD.)
Encapsulate intermediate values.
Avoid creating inconsistent state in shared data. Operations that create intermediate values shouldn’t be performed in situ on shared objects or data structures. Use local temporaries for intermediate values. Only update the shared data with the end products of a operation.
Be careful of using the double check lock pattern.
It’s a wonderful idea that’s so clever that’s it’s broken in many languages and runtime platforms.
Where the DCLP is not just inherently broken and can be used, it still needs to be implemented correctly. Mark the variable under test as ‘volatile’ (or the closest equivalent) so that the language compiler doesn’t optimize away one of the two checks. Don’t operate directly on the variable under test. Use a temporary to hold intermediate values and wait to assign to the variable under test until the new state is fully computed.
The DCLP is often used for one time or rare initializations.
Don’t lock the ‘this’ pointer.
If there are member functions of an object that need to synchronize, create a private member variable that is used as the lock target rather than locking on ‘this’. In addition to encapsulating the locking, using a private member guards against potential issues where a framework, library, or runtime may be performing synchronization by using the ‘this’ pointer.
Explain your code.
Leave comments for yourself and the next person.
Strive to make the design grokkable.
The original design and implementation can be bug free but if the design is hard to understand and follow, bugs can be introduced in maintenance cycles. Even if you are the only developer who may ever touch the code, remember that the details of the design that are so crystal clear in your mind today will get supplanted by tomorrow’s events and miscellany like … Squirrel!
