UP | HOME

Scala3

When the language specification and the compiler are done, we face a slow, painful but doable 2 -> 3 transition, similar to what Python went through.

It is not just matter of accessing old classes, it is an effort to rewrite parts of important codebases (Spark, Kafka, the Twitter stack) to use the new standard library and its better abstractions (due to better typing).

The main goal was, and still is to write high-level libraries as layers of functional DSLs.

However, over-“engineering”, over-abstraction and even over-DSLing are the actual problems. Over-complicated bullshit is not a solution to complexity, just like an idiotic pretentious, over-verbose language used by liberal-arts and humanties.

To choose just right abstractions and just right level of wrapping (proper types) is what constitutes the art part of the craft.

Things like scalatest, cats and zio2 are the real-world examples of such layered DSLs. This is the way to program, and the real understanding comes from the principles behind the syntactic sugar and the higher-kinded types.

Overview

It is much better to think in terms of nested Sets of type-tagged structured values. With proper universal nesting almost everything can be abstracted as such a value. Objects may have fields and methods, traits may have types, fields, methods and other traits, which may have types, methods and other traits… At a higher level, modules and whole packages are such “values” too.

Scala 3 import system hints at this universality, allowing one to import everything from everything almost uniformly. This is a consequence of a proper design, and proper modeling based on just Sets and (Second Order) Logic.

Scala 3 got a lot of principle-guided unification from being a cool collection of ad-hook design decisions. It is by far the best-designed language out there, especially compared to Java or C++.

Product types

Case-classes are used to define immutable values which have its own structure and “normal” classes can create mutable record types. Companion objects provide an additional name-space. /Extensions are also useful.

Sum types

Enums are Scala’s attempt on generalized sum-types, or even GADTs, and they seem OK due to the ability to define fields and methods, just like in a companion object. They can be parameterized, just like Traits.

Type-classes

Parameterized Traits are now the solid basis of proper type-classes, which are as fundamental as Liskov’s ADTs, and one should abstract out with traits.

Just like ADTs, type-classes (based on composable Traits) define abstract interfaces (to be implemented), thus introducing an abstraction barrier by abstracting away everything about implementations.

I really hope that they at least attempted to unify various in-house implementations of Streams and Futures and what not. There is no easy or quick way to evolve stable intermediate forms which is a principal requirement for an ongoing evolutionary process.

The mantra Principle-based is better than rule-based should be the motto of Scala 3.

The sad thing is refusal to just copy fundamental innovations from other languages

  • where clauses from CLU
  • modules of Ocaml (it has to be the Java way)
  • Some Haskell’s typing extensions which have proven to be useful.

Anyway, whatever.

Modern tools

  • cs - artifact manager
  • amm - advanced REPL

The Compiler, sbt and amm

cs setup

Language-server, linter, formater

cs install bloop metals scalafix scalafmt
cs update

Launch the REPL

cs launch ammonite --scala 3.2.1

import straight from Maven

import $ivy.`com.typesafe.akka:akka-stream_3:2.7.0`

Basic configuration

add this to the build.sbt file

val scala3Version = "3.2.1"
val AkkaVersion = "2.8.0-M1"

ThisBuild / scalaVersion := scala3Version
ThisBuild / scalacOptions ++= Seq("-language:_", "-deprecation")

lazy val root = project
  .in(file("."))
  .settings(

    scalaVersion := scala3Version,

    libraryDependencies += Seq("com.typesafe.akka" %% "akka-http" % AkkaVersion,
                               "org.scalatest"     %% "scalatest" % "3.2.14" % Test)
  )

and project/plugins.sbt is

addSbtPlugin("ch.epfl.scala" % "sbt-bloop" % "1.5.4")

addSbtPlugin("ch.epfl.scala" % "sbt-scalafix" % "0.10.4")
addSbtPlugin("org.scalameta" % "sbt-scalafmt" % "2.4.6")

Editor support

The Giga Chad Odersky and the team are using VSCode, so it almost always works. Vim and Emacs require some effort.

Be aware of which sbt binary is in your $PATH, which version is specified in project/build.properties and which version has been installed by cs. Usually there is a version mismatch.

Older Vim plugins check by calling scalac, so make sure there is no Scala2 scalac binary in the $PATH. By default Scala3 calls the compiler scala3-compiler.

Since cs is the modern tool of choice, make sure your $HOME/.local/share/coursier/bin/ is in the $PATH

scalafmt needs to specify a dialect in the .scalafmt.conf.

Metals

The only LSP-server available is metals but it is buggy as fuck. It relies on bloop, so make sure it is installed by cs.

It fails with idiotic errors (cannot start bsp, etc) when you have proxy variables set in your shell environment.

Degenerates cannot ignore proxy settings for localhost connections.

They are also trying to install big jars in background, not being aware that there are millions of people behind crappy ADSL lines in the third world.

The fucking crap failed with timed out exceptions, leaving everything in an inconsistent state.

It is trying to install scalafmt and other tools by itsef, while they are already installed by cs, which is the default, preferred way to do this in Scala.

Better abstractions

Refinement of implicits (of implicit contexts) by using a common mathematical jargon.

The general principle of a context is that one does not have to always mention some properties or attributes.

Conceptually similar to partial application, so one does not have to pass it all around.

An implicit value can be “inferred” by the compiler or selected explicitly among many.

Using (clauses)

Optional arguments with default values which can be omitted at a call site.

Values should be automatically provided by the context.

Given (instances)

This corresponds to some “standard property” that we always assume - given that…. The common synonym is provided that…

Define terms that can be used by the compiler to fill in the missing arguments.

Libraries

Most of them are over-“engineered” and bloated, without any effort of making the abstractions minimal and “just right” (optimal). Such “natural selection” (of gradual refinement and reduction to a local optimum) takes decades (like Common Lisp, Ocaml stdlib or Haskell’s prelude).

Most of corporate code are of low-effort, and mostly “copying” of imperative implementations from one another.

  • zio2 - functional style I/O
  • cats - some standard type-classes for common algebraic structures.
  • akka - the supposedly done right basic building blocks. Use it.
  • The Twitter stack (still Scala 2)
  • Kafka - steams
  • Spark - the killer-app (still Scala 2)

Coming home

package s3.payload

import akka.http.scaladsl.model.Uri

import java.net.URLEncoder
import java.io.UnsupportedEncodingException

import scala.sys.process.*

import s3.keys.{getSecretKey, privateKeyFile}

// We shall use their (Binance's) terminology consistently
// hoping that they use it consistently too

// They call this /Singed Payload/

// functions for constructing Signed Payloads

// local type aliases
type QueryString  = String
type DigestSHA256 = String

// This extension can be imported from this module
extension (uri: Uri)
  def withSignature: Uri =
    uri.rawQueryString match
      case Some(s) =>
        uri.withRawQueryString(addSignature(s), Uri.ParsingMode.Strict)
      case None => uri

  def addSignature(s: QueryString): QueryString =
    s ++ f"?signature=${digest(s)}"

def digest(q: QueryString): DigestSHA256 =
  val sk = getSecretKey
  val a =
    f"bash -c \"echo -n '${q}' | openssl dgst -sha256 -hmac '${sk}'\""
      .!!
      .split(' ')
      .map(_.trim)
  a(1)
import akka.http.scaladsl.model.*

import org.scalatest._
import flatspec._
import matchers._

class PayloadSpec extends AnyFlatSpec with should.Matchers {
  "A signed URI" should "be constructed" in {
    val p =
      "symbol=LTCBTC&side=BUY&type=LIMIT&timeInForce=GTC&quantity=1&price=0.1&recvWindow=5000&timestamp=1499827319559"
    val d =
      "c8db56825ae71d6d79447849e617115f4a920fa2acdcab2b053c4b2838bd6b71"

    import s3.binance.uri
    import s3.payload.withSignature

    val uri1 = uri
      .withRawQueryString(p, Uri.ParsingMode.Strict)
      .withSignature

    uri1.toString should endWith(d)
  }
}

Author: <schiptsov@gmail.com>

Email: lngnmn2@yahoo.com

Created: 2023-08-08 Tue 18:39

Emacs 29.1.50 (Org mode 9.7-pre)