Concurrency
Immutability and the data-centric approach – “whole” structured values “on the wire” instead of calling methods on objects – solves most of the problems.
This includes propagation of “errors”, and a monad is a good fit.
“Failures” or “Errors” must be modeled explicitly as a part of the domain model. They are just “backtracting”.
The authors of Erlang have everything solved in principle. We just take advantage of better-designed languages of the ML-family.
The multi-threaded imperative OO crap is fucked beyond repair. They piled up on the wrong (faulty) principles.
Immutability is not an academic fancy, it is a universal principle, which makes Life Itself possible (every molecule of the same kind is as good as every other, and they are being passing around freely).
Concurrency vs. Parallelism
Concurrency means some sort of time-sharing /scheduling or cooperative multitasking by some “supervisor” process.
Sharing (of the call stack, for example) means they cannot run simultaneously in principle.
Parallelism implies share-nothing processes (total closures), running on the same hardware or on different nodes.
Really smart people – the developers of Erlang – solved everything back then by postulating that share-nothing processes which can perform asynchronous message-passing is the only viable solution. It is even better when the code is pure functional.
Asynchronous vs. Synchronous
A normal function (or a chain of deeply nested functions) has to return.
A compiler usually inlines nested (or composed) function into one big block of code which has an entry point and en exit point.
If a “function” is annotated as asynchronous, it means that it is wrapped in some “execution context” or a light-wright process (which usually shares everything except code in memory) of its own.
This means that it is not a simple sequential (graph reduction) process anymore.
To such asynchronous processes (or “execution contexts”) some additional signaling of completion is always required via some “external” mechanism (runtime-support).
More than one
This is not merely pure function composition (via nesting). Having more than one “thread” (or a routine) of execution implies /coordination and communication of some kind by runtime.
Message passing
The right thing to do is what Erlang did - spawn a process, get its “address”, and send it a message. It will eventually send you “the result” back in a message (result will be delivered to your “mailbox”).
Callbacks
The most straightforward solution is to pass an additional parameter to the asynchronous “function” (an explicit callback) which will be callled instead of just returning.
Notice that the callback (or a whole chain of them) has to return eventually. Because it is the tail-call it could be optimized nicely using the standard TCO.
Concurrent futures
Futures do not signal completion but they can be called (polled) more than once.
More precisely, they can be created or dispatched (and some callable reference is returned) and then called one or more time.
When future is not ready it just blocks the caller.
Notice that every future is still a separate sub-process with sharing, so what appears as a normal function call is really a coordination and communication at runtime.
Futures are similar to what OS poll or select calls do.
Non-blocking vs. Blocking
The only way to achieve “non-blocking” is to have multiple “execution contexts”. There is no other way in principle.
Non-blocking code just spawns (starts) more “processes or routines” (or sends messages to do so) and continues. This is exactly what “delegation” or “dispatching” is.
This, again, requires some signalling of completion or use of concurrent futures.
A newly created future can me send (via some channel) or passed to another async function as a parameter, and then “forgotten”.
“Forgetting” is the part of non-blocking again. in principle.
Problems
In pure a functional setting most problems do not arise.
- Immutability solves the sharing and concurrent execution problems.
- Implicit passing along of a /context solves the state problem.
- Sending structured data (messages) through the “wire” (a queue) instead of doing
method calls and returns solves the communication problems.
Imperative OO crap, however, is fundamentally broken beyond repair.
- the shared call stack (between threads) is a problem (fundamental)
- multi-threading access for statefull objects is a problem (fundamental)
- internal state and invariants cannot be guaranteed in imperative “objects” with a concurrent execution
- there is also an I/O
Summary
To summarize, any form of concurrency requires multiple execution context (coroutines, fibers, threads, whatever) and coordination and communication between there by the language runtime.
This, in turn, results in some time-sharing scheduling or cooperative (interleaving) multitasking, which is hard and should be done by an OS with share-nothing guarantees. Erlang authors have researched this.
The only thing we can do is to rely on some “trusted” (LOL) runtime, which is experimentally shown that it is “stable”. - JVM, DotNET, Go, etc.
All these, however, being over-“engineered” imperative OO crap (except Go), are necessarily buggy and error-prone. They will fail eventually.
Erlang is an exception.
Imperative calls and returns
Again, asynchronity means starting another processes and delegating to them.
This cannot be a simple method/function call because calls are strictly local to the thread.
This implies use of some shared mutable memory location, and some “external” (hidden) event-loop.
This how the caller function “forgets” and moves on.
Now we need to solve notification issues and separate error signaling issues.
In the functional world we just use sum-types for error propagation and message-passing for signalling. Not even callbacks.