On an Implementation for Anoma Level

[mariari]

The Current Issue

When designing and implementation a programming language for a new operating system, it becomes hard to separate where the language begins and where the OS begins. Even in an operating system like Unix with C this is the case. Although we can distinguish C, our interface to changing and interacting with Unix is primarily through C in the form of sys calls. General system administration is segmented into C and bash. The system is suboptimal but you can tell that Unix works as fundamentally a C machine. In operating systems like Xerox’s Smalltalk D or Symbolic’s Lisp machine these lines get even blurrier as the live nature of the system allows us to natively run/interpret any code given to it, making system extension and adminstration flow from the same user experience. So what parts of a lisp implementation are really the programming language and what parts really the operating system and how it operates?

We currently do not have this problem, instead we suffer the worse problem of our system being unergonomic as instead of having a singular language that blends into the OS at a fundamental way, we instead have to deal with 3-4 different systems to write anything:

1. The Elixir codebase which offers special end points via the local domain
2. Juvix to write RL’s and the Transaction Functions in
3. Nock the VM language. This rears it’s head current in terms of errors issues etc.
4. Shell or Elixir to orchestrate the glue between submitting Juvix to metamask and to local domain exectuion/controller submission.

We already have plans on reducing some of these, 1 can be brought into the system (v0.4 has plans for this), 3 can go back to being a normal target like assembly is in lisp when we have better VM tooling and integrations, and 4 can sorta blend into Juvix with the repl being tied directly to the local domain.

However even if we solve these issues perfectly and end up just working with Juvix and Anoma, the fundamental mismatch in models will lay bare making it clear where one system ends and other begins. This is because Juvix as it currently stands is a fairly standard purely functional ML designed ontop of current operating system.

Let us explore a list of reasons why:

1. The type checker and compiler are not invokable dynamically. Having a repl means we can statically invoke the compiler on data we know. For example we could scry for library functions in the repl, however we can not do this in Juvix code (dynamically)
2. Type information is dropped early, meaning if we have some library online that makes a type Foo, and we scry for a stored instance of Foo, we can not verify that the data is of the same type nor even run the type checker on it to enforce it (this is what scry enforces typically). The information is tossed at a very early stage.
3. The lack of IO functionality makes scripting interactions with how the actual system operates is hard.

These issues arise out of the mode in the kinds of applications that Juvix was designed. It was made for short lived programs that had stringent requirements, Anoma indeed has these (RL), but this is a small part of the overall system, where to even orchestrate writing these short lived programs is a process that requires munging through live data. At the time of design we did not consider the system as the whole and over the years we’ve tried targeting many different things. One key feature of our system is that data is live meaning most Anoma code is stored online as opposed to the files on a particular disk. We can grab these libraries in a static fashion but never properly orchestrate it from within Juvix as such functionality is locked away from the language. This is fundamental issue that motivates 1 and 2. There are of course more issues we run into grafting a language not designed for Anoma but they aren’t as fundamental (such as module names being tied to files on Windows/Unix).

Thus when it comes designing a language to truly blend into the system of Anoma and blur the lines of operating system and language, we must consider how the system fetches data, writes data, and operates before coming up with a convincing language and implementation. However if we were to do this well, languages like Juvix would greatly benefit, as rather than having to implement a large compiler to handle both compiling ontop of Unix/Windows and Anoma, we can instead do much of the heavy lifting from the language that blends into the system and have new languages appear as a coherent part of the whole. Languages such as: F#, Clojure, Scala, Elixir, and Kotlin come to mind. As these languages rely on the underlying Operating system and language (C#/Windows, JVM/Java, and Erlang/OTP) to simplify their own compilation procedure. For example imagine if we went to a declarative object system for this language, Juvix could very easily become declarative by hooking into the primitives of this system!

This rest of this post will concern itself with ideas and things we must do to have this integrated system. I dub this language, Anoma Level.

Considerations for an AL Implementation

To properly consider an AL implementation we must come up with a few things:

1. The kind of model that Anoma itself is suited for (Design for the language)
2. How the system mechanically works in terms of itself (Considerations for implementation strategy)

1 I believe a declarative OO approach is most appropriate, for how Anoma works. To keep this part brief, intents are best represented by constraints and the late binding of objects and messages fit with how our general model works. The actual system is a very weird object model due to how the RM works, please read the post below for more details about this model:

The Peculiarities of the Anoma Level Object System

The Peculiarities of the Anoma Level Object System The Object system that is in the works for Anoma Level (AL) is a peculiar to say the least let me first outline the features of our object system: It’s a true object system, meaning that Object Oriented in the William Cook sense 1. I.E. Objects only know themselves and they work only on message sends. However we do go further away from Cook in how methods themselves operate and specialize. The object system mostly takes a declarative form, mea…

2 is what I wish to focus on, let us list some of the things we wish to do with the system:

1. We wish to be able to dynamically compile code, load code, store and type check code.
2. We wish to be able to invoke the compiler from the local domain directly, and even on controllers.
3. We wish to store information online about the kinds of data (If we have an instance of Foo, then a marker that it ought to be read and checked as the kind of Foo when fetching from online).
4. We wish to be able to run long running isolated processes that can receive some kind of messages. Further these processes will receive events across the system (from it’s perspective) at arbitrary times.
5. We wish to be able to send objects to controllers whenever we wish.

1. implies that we can not write a standard interpreter, I.E. we can not write an interpreter in a language like CL that simply stores a definition like (def foo (x) (is-cool? x)) in a hashtable in said language and lookedup later for invocation. Instead we must store code inside the local domain or in a controller and then scry for it at a later time.

2. to achieve invoking the compiler on a controller or a local domain, this has severe implications on how we can implement the language. The most logical options are either pure nock or in Erlang/Elixir. nock is fairly obvious, as if we had nock code for the implementation then we can invoke it even on controller that run any nock backend, this has the most flexibility and grounds us on a fairly clean bootstrapping. However this has the downside is that if we wish to do an interpreter that the language itself should compile to nock. Our choices are both hoon and Juvix which are suboptimal because we can not invoke it from the elixir codebase itself meaning that we have the same external tooling bootstrap packaging issues we currently have. The other option is to write it directly in Erlang/Elixir, doing this would let us package the compiler with the anoma system and offer it’s interpreter and interpreter primitives as opcodes in our standard library we can ship. This could technically be done in any language like prolog or common lisp, however we would need to somehow be able to call it from the codebase and ship it without too many issues, due to this I believe doing it in pure erlang/elixir is probably the best. This implies that the compiler is just another function/method in the language, so an implementation ought to reify it’s own compiler.

3. means that we need to store some kind of meta data with the object that we ship online. I.E. if we have a class Foo and an instance of it, we need to store what it’s original class is to check it against it’s form. We can invoke the predicates that must be true on an object to verify it and ensure it’s valid for the kind it claims to be. This can be stored plainly in the code that we upload. I believe at some point we ought to associate meta data structures with objects somehow, however we can elide that implementation for now.

4. means that we ought to introduce a light weight process abstraction to the language. We can piggy bank off of what the local domain allows and either introduce it via nock or by introducing it via elixir. What is important for this is how the events across controllers work, I know we can intercept them by either elixir or nock so either are fair game. Events across the system can be viewed as messages in the Elixir process that can be serialized to an AL message send. This interopt needs to be seemless and not leak any Elixir or LD Nock details.

5. means that we wish to have a nock compiler and perhaps future compilers in the future. Meaning we want to be able to serialize arbitrary objects as nock when requested and send them off to a controller. This does not imply executing in the local domain has to be done in nock, we could for example compile AL to x86 machine code and only compile things to nock when we send it to the.

I believe if we can in the implementation of AL account for these requirements we can have a language that starts to blur the lines between the operating system and the language. We can kind of see why! The storage of terms is done on the operating systems own terms, not a separate language construct terms! The compiler looks apart of the system and is a non special component! This goes as far as even the object system blends into the system and better yet the concurrency system is precisely that of what our specs has been championing in the design of how the system works.