Why Functional Programming
This is sort of a tribute and a continuation for the pair of the classic papers.
- https://web.cecs.pdx.edu/~apt/cs457_2005/hudak-jones.pdf
- https://www.cse.chalmers.se/~rjmh/Papers/whyfp.pdf
Since these have been published lots of things happened, notably
- actual (non-theoretical) success of Erlang/OTP in the telecom industry
- the use of Ocaml by “Jane Street” (market-making and automated trading)
- Tezos, a blockchain in Ocaml
- the Twitter stack in Scala and the enterprise-grade stream processing tools
- evolution of Clojure from a hobby project to Datomic and deployed banking software.
- lots of in-house Haskell code by institutions like JP Morgan HSBC
For us, however, the most illustrative and illuminating story would be of Clojure.
This languages has been designed based on certain fundamental “theoretical or academic” principles (similar to Haskell, which, as a miracle, still retains its purity) and “succeeded” nevertheless (or, rather due to).
https://www.cse.msu.edu/~cse452/Readings/a12-hudak.pdf
The fundamental principles are:
- a mostly-functional dialect of Lisp, with
- immutability and persistence of all the data
Of course, almost everything has been “borrowed” from the Classic Languages (both LISP and ML “families”) of the Golden Age (70s, 80s, 90s) and based on almost 50 years or research (since the first LISP paper).
What we want to understand and realize (to have an “a-ha moment”) is that immutable and persistent data structures (as a principle – nothing is mutable within Clojure and its core libraries) are good-enough for everything (performance-wise) and absolutely “game-changing” from the semantics and data integrity points of view.
For the costs of a negligible overhead of implementations (everything is some kind of a persistent tree under the hood) we retain the referential transparency property for the data bindings (actual immutability and persistence).
https://www.youtube.com/watch?v=wASCH_gPnDw
Why not Erlang (this time)
Erlang is exceptionally well-researched and principle-guided language (made by the people with scientific and mathematical backgrounds) but it is has a very clever, truly innovative and very complex implementation.
It creates for the developers an /illusion of using a pure-functional language with structural pattern-matching on receive, while in reality only pure functional interfaces with a lot of callbacks has been exposed.
One writes the code in a “functional language” but only within a carefully designed environment according to a complex set rules (behaviors).
So yes, Erlang is absolutely innovative, clever and is a true miracle of software engineering, but it is too complex for a definitive example.
Small tribute to Joe
Erlang is arguably one of the well- (or deep-) researched languages, on par with LISP and ML.
Among other things, it clrarly formulated some fundamental priniples:
- values are immutable, so are all the bindings
- so for any operation (function application) new values are always returned
- all the actual parameter values (of a function) remains immutable (eternal)
- a new value requires a new (fresh) binding, (without even shadowing)
- a structured value never change its “form” (or shape), just like molecules
- a new strucured value requires a new (fresh) binding
- the parts it has been composed from remain immutable (eternal)
This is why always returing a new value (or even think of a new version) of a List or a Map is “natural”.
Imperative “coders” get utterly confused by this principle. They think in terms of illusory identities (imperative variables, which are names memory locations).
A new strucured value is a new cobination (arrangement) of pre-existent (ideally – eternal) immutale elements, so it “naturally” requires a new (fresh) binding.
So it is always a fresh new binding, which is a reference to a just appeared (new) value.
X1 = X + 1.
Functional Programmer’s view, not incedentally, are closer to what we call “reality”
The hows
Clojure is a simple dynamically typed language, which means the compiler does almost nothing, but basic type-checking, so all the “safety” are at the level of code (not the types or a restrictive compiler) and “at runtime”.
It also delegates to (or actually just passes onto) the most difficult part - the runtime – the JVM, which actually runs the code (does all the memory management and performs I/O, etc).
Clojure implements its pure-functional data structures (yes, as in the Okasaki book), using Java’s “primitives”, which is both smart and clever.
And it actually just works “in the cloud” and in the banks.
Again, the main lesson is that “it just works” because it was principle-guided and took immutability (and persistence) as a principle. Not a single value can be over-written, destructively updated, or changed, in principle.
The “best textbook-like” code
It seems that there is a particular style of writing code, which is so “textbook-like” as if it has been actually taken from a very principled and nuanced book (let’s say by Richard Bird).
Surprisingly, the Ocaml’s stdlib is exactly like this, as if it has been taken from a very good textbook. It has been refined and polished into this state of aesthetic “perfection” (the “aesthetic is never optional” mantra).
And yet, Coq runs on top of this stdlib and runs just fine.
Essentially, you just write everything like in a CS 101 class, trying to come up with “classic” types, like Lists, Sets, Maps, very idiomatic, even pedantic, without any fancy bullshit.
Who would do this? Well, Norvig with his Pytudes, the authors of Isabelle and Coq, and Simon Peyton Jones, of course.
The “hard way”
Sometimes it is worth doing things “the hard way”.
By this I mean to write everything down in a principle-guided way with careful attention to details, just like the standard libraries of the classic languages has been written, and use only these standard libraries (they have everything for compilers and proof-assistants and basic networking).
By doing so one will spare oneself for the dependency hell, (of other people), “tech dept” (having a truly crappy code at the “core” of the project) and just to stay sane and emotionally stable (by not being exposed to some real fucking bullshit - they actually write Haskell as they used to write some J2EE).
How will it “pay back”?
Doing thing “just right” (principle-guided with careful attention to details) requires a lot of education (to know the whys) and learning by doing (to know the hows) but the results (the code) has the properties similar to good mathematics – once it is “done” it stays forever (become eternal).
Yes, there are always some changes in wording and symbols and re-formulations and clarifications, but the “essence” will stay the same.
So it could be (more or less) with your code. Once the formalization of the problem domain (hate this cliche) is done, the resulting “formalism” (in a pure functional languages) will essentially stay the same, just as the Set or List modules in a standard library do not change much.
It is for an older you, who would try to read it later, and to other people who might want to contribute or fix some bugs. It pays greatly.
Layers of DSLs
The idea to partition the code into layers of embedded “functional” DSLs, each of which interracts only with the layer immediately below it, exports its own functions, and makes no assumptions about the layers above it, is still the most important one in all programming.
It has been re-discovered and it re-emerged innumerable number of times, because it reflects the universal pattern which the process of a biological evolution has been “discovered”. Complex systems are hierarchical and layered with “asynchronous message-passing” between laywrs.
Embedded DSLs are well-understood and emphasized with the LISP family of languages (due to “more permissive” dynamic typing). DSLs, however, could be easily defined with Algebraic Types and pattern-matching on data constructors. All the Classic Books have at least one DSL in them.
This “universal” methodology has much deeper connections to reality. A well-designed DSL taps into a domain-experts knowledge which is “included” into the slang and jargon the experts use among themselves. It matches perfectly with DDD.
There is also a deep connection with the notion of a Monoid and combinatorics. The best DSLs form a Monoid with respect to the composing operators, so everything composes to everything like within a Ring. There is a lot going on.
Last but not least, aminoacids compose into proteins. They are not Monoids, but Monads (ironically), since each resulting strucure is a “lifted” one (has its own emergent electro-chemical properties which the individual parts do no possess). Kleisli Arrows could be “seen” there.
Standard Libraries
- Ocaml stdlib
- “Core” by Jane Street.
- Haskell libs (the set of libraries bundled with GHC)
I could argue that