Anoma meta-app & the Babeloma fish

[cwgoes]

Preface

What is an application? This question is harder than it might seem, but for the purposes of this post we will take an application to be a collection of resource logics (and possibly resources) organized for some holistic purpose, along with read and write interfaces which make sense in the application context.

Applications are not part of the Anoma protocol itself – they must be designed and engineered. How does this happen? I contend that the application development loop as it exists at the moment looks something like the following:

We can describe the steps as follows:

1. First, a would-be application designer starts with a conceptual model: a (possibly very rough) idea of what they would like the application to do. For example, an application designer trying to design “Kudos” might start with a concept like:

   “Kudos are individual, fungible tokens. It should be possible for a specific party to issue new kudos from nothing, for any party who owns some kudos to transfer them to another party, and for any party who owns some kudos to burn them (into nothing). Each kudo is always owned by exactly one party, and the consent of that party is required to transfer or burn that kudo. Kudos are fungible, so one kudo of amount a and one of amount b can be combined into one of amount a+b, or one of amount a+b can be split into one of amount a and one of amount b.”

2. Second, the application designer takes this conceptual model and attempts to translate it into code.
3. Third, the application designer reads, tests, and reasons about a particular application definition.
4. Depending on the results of this testing and reasoning, the application designer may iterate on their conceptual model and/or iterate on their application definition and re-run the loop. Eventually (one hopes), they will be satisfied with the application definition and confident that it matches their conceptual model, and the loop terminates successfully with an application that they (or others) can now use.

The specifics here can take many different forms. Let’s consider a few example cases:

• Suppose that we have a hypothetical application designer Annie, where Annie perfectly understands the entire Anoma system and can hold all of it in her mind at once. Annie should be able to simply take the application concept (if it is clear enough) and imagine the simplest set of resource logics, transaction functions, etc. that correctly implement this concept. If the application concept is not yet fully defined, Annie should already be able to tell by imagining the implementation, and she could refine the concept until it is coherent, then type the exactly optimal sequence of keys on her keyboard in order to implement it.
• Suppose that we have another hypothetical application designer Bob, who doesn’t understand Anoma at all, and can only play with it as a black box (e.g. try inputs, read outputs, and gradually build a model of it over time). Luckily for our hypothetical purposes, Bob happens to have an infinite amount of time, so he would simply take the application concept and start entering random inputs into Anoma, testing the outputs, and gradually building a model of the system (properties of the black box) over time, refining his inputs in order to produce output behaviour that matches his desired application concept. Given infinite time, Bob would eventually succeed, although he would never know for sure that the application he has written would actually implement the desired concept in all future cases (there’s nothing he could do to test, for example, that there isn’t some specific number n for which kudos of amount n are treated differently).
• Suppose that we have yet another hypothetical application designer Charlie. Charlie has a perfectly formed application concept – an exact idea of what observable behavior he wants the application to have – but he doesn’t understand Anoma either. However, Charlie has access to a rather magical tool: the Babeloma fish. This Babeloma fish is capable of translating between conceptual models and Anoma application definitions: it can take a conceptual model and produce an Anoma application definition which implements that conceptual model correctly, or take an Anoma application definition, and describe conceptually what observable behaviour that application definition has (in terms that would make sense to Charlie). Even though he doesn’t understand Anoma at all, Charlie can achieve the same results as Annie – a well-formed, correct application definition immediately – by using this Babeloma fish, and he can also use the Babeloma fish to double-check that the application definition produces the behaviour he wants. Charlie does have to trust the Babeloma fish, though.

Right now, in practice, in order to design and write Anoma applications, one has to be pretty close to Annie: understand the resource machine, know Juvix, be able to reason about controllers (when we implement that), etc. These are steep requirements, and even for those who do understand such things, writing applications is a lot of work, much of it finicky and low-level. The low-level languages of Anoma – resources, the resource machine, engines, and the engine model – are powerful components, designed to give programmers exact control over nearly every aspect of how the system or a particular application behaves. Requiring that developers think about all of the possiblities of these low-level interfaces at once, and program manually in them, doesn’t just restrict the accessibility of the system to experts – it limits what even those experts can do. If we want the power of Anoma to be practically accessible, we must start building this Babeloma fish.

What might this Babeloma fish look like? Before we embark upon a project of piscine design, we must first examine another aspect of application design: time.

Temporality

So far, we’ve been assuming that the application designer and application definition exist in a vacuum: the designer operates alone, building an application which is entirely self-contained, without reference in the concept or definitions to existing state anywhere, application definitions elsewhere, or really anything outside this loop at all. This assumption is wildly unrealistic – in fact, it perhaps holds only for the very first Anoma application designer of the very first Anoma application – subsequent applications might want to interoperate with earlier ones, and so must keep them in mind. The real situation looks a little more like:

In particular, I think that there are (at least) a few aspects to keep in mind here:

1. Subsequent applications may want to interoperate with prior applications, and so must take into account at least some of the external interfaces of the applications with which they want to interoperate. Applications may even want to design their interfaces so that future applications can easily interoperate with them.
2. Technical translations of certain concepts can likely be universal. For example, we can come up with a standard for what it means to “own” something which is independent of any specific authorization mechanism (e.g. signature scheme), so it should work for any application which wants a (standard) notion of “ownership”. Later applications likely don’t want to create a separate-but-equivalent ownership standard – they want to use the one which is already there. Clever design of standards may even allow us to define them in a way which does not rely on any arbitrary names (e.g. field names) (see this discussion), which would help, but application designers will still want to be able to look up and reuse existing patterns.
3. The Anoma network itself can store arbitrary data and programs, including application design patterns, prior application definitions, educational material about how Anoma works, even parts of this hypothetical “Babeloma fish” function.
4. Applications can and should evolve over time, as their users’ needs change, conceptual models change, other applications change, etc. – the application design loop should not be conceptualized as prior to and exclusive from actual usage, but rather a part of it, which leads us into…

The Leaning Piscine Tower of Babeloma

Who should be able to design applications? Modern computing stacks have created a two-tier class structure: developers – possessed of enough arcane knowledge to bend the system to their will – and users – who lack such knowledge and find themselves at the mercy of the developer-priests. Without even touching upon the social problems which this stratification creates, we can demonstrate its indefensibility by appealing to the basic criterion of efficiency: it is the user who knows what they want, not some separate application designer sitting in an office halfway across the world. Why bother attempting to teach the developer about the user (or regulate the developer into acting in the best interests of the user) if we could simply give the user tools to build, evaluate, and modify applications themself?

What would this take? I think we can start with some boundary constraints:

• The user is not going to learn a new language or the Anoma system model (e.g. how the resource machine works). At most, the user might learn a high-level intuition for how Anoma works by interacting with it over time, and using simple conceptual / visual models which are well-known already (e.g. cells in a spreadsheet). The user must be able to describe their application in natural language – at most manipulate some data flow diagrams or rules in a GUI, but nothing complicated. Similarly, the behavior of applications must be described in natural language or visualized, the user cannot be expected to read any code. The user may be able to handle a higher-level abstract representation (such as “objects”) if this is conceptually easy to grasp and comes with graphical / natural language interaction tooling.
• The user is also not going to learn in detail how existing applications or state on Anoma work. The user may know of a few applications by name or function (having used them before), and want their application to interoperate with those applications in a way which they can describe. The user is not going to know about Anoma development patterns or standards.
• Natural language, preferred visual representations, application standards, existing applications, etc. all change over time – the Babeloma fish must be able to adapt to these changes.

So, what must our Babeloma fish anatomy consist of? Perhaps something like:

There’s one part in this diagram which we haven’t discussed yet: an application description language. What do I mean by this? The general idea is that we may want a higher level language / format to describe what an application should do at a higher level of abstraction than the Anoma “executable code” (e.g. transaction functions + resource logics) itself. The object model under development is one step in this direction, but we’ll probably also want to consider other aspects such as:

• Ways to reason about desired information flow control properties at a higher level.
• Ways to reason about desired trust (or lack-of-trust-required) properties at a higher level.
• Ways to reason about desired interoperability (with other apps) at a higher level.

In some sense, I think we could understand an “application description language” as a translation of a conceptual model (such as the example for kudos above) into a set of explicit constraints and preferences for how an application should work. With such an explicit high-level target, we could likely:

• Analyze whether the target application is possible to realize / underspecified / etc. (at that high level of abstraction) and give feedback to the user.
• Eventually, build a pipeline to construct / search for implementations that implement an application which satisfies those constraints and preferences as best as possible.

The head of the Babeloma fish will necessarily involved natural language, GUIs, LLMs, etc. – but I think we want to make the job of this translation-head as easy as possible by crafting as high-level + declarative as possible a representation of what applications should do. This seems like a natural next step in our development as well.

Enough for one post…

[vveiln]

I think it is a nice idea, but also quite ambitious, so I assume it is more of a long-term vision rather than a mainnet user starter pack kind of thing. If so, this is a slightly separate question from what I’m concerned with, but I’ll write my thoughts about this one first.

Application boundaries

I think one big problem that we need to address before (or rather not after) progressing towards this goal at the moment is very low understanding of applications. The way we define applications and how they interact with each other is very complicated and not intuitive. I’m not talking about the simple stuff like the example applications we’ve built and designed, but rather more complex concepts of “application network” where we expect applications to work together in somewhat natural but not intuitive ways. We talk about applications in simple concepts to each other, but this is not how we actually defined them. We are stripping away the aspects that are difficult to understand, and this will backfire. Application composition, dependency, application boundaries and privacy properties are all very vague concepts. We defined these things to be programmable, we don’t want applications to exist separately from each other, we want to facilitate cooperation, but we have a very low understanding of how it would work in practice. We haven’t designed any complex applications even in theory, and the practice of it is prevented by efficiency concerns that move us from the long-term design goals to short-term design limitations.

How would a kudos application interact with a rock-paper-scissors application? That is not the question we need to answer. We need to understand how, with the definitions we gave to anapplication, they could interact with each other. Does it include the ways which we would like to be included? Does it include the ways which we would like to be excluded?

Steps towards having a more accurate idea of applications:

• understand the actual application boundaries
  • what is actually allowed by our definitions?
  • identify the malicious practices allowed by our definitions
  • articulate the actual application boundaries
• design more complex (intertwined, with multiple resource kinds and sophisticated relationships) applications without optimising for low-level resources (to have a realistic idea what we want to be able achieve)
  • describe how application boundaries apply in complex cases
  • describe how application boundaries apply in existing cases
• repeat all of that for shielded cases
• repeat all of that for mixed cases

After that, moving towards designing higher-level languages makes more sense.

Babeloma larva and babeloma fry

I think it generally is a cool idea, but somehow it feels like at some point we will have to make a trade-off between expressibility and simplicity. Designing a tool that turns a user’s most intricate desires into applications sounds cool, but I imagine there will be a few important milestones:

1. making designing applications accessible to builders. This is something I’m really concerned with. Talking about Babeloma fish seems almost ironic now. We talk about applications for users, which is a very long-term goal, but right now we are trying to make applications writable by builders. Why don’t we talk about that first and think how we can make it more accessible for them? I think it is a reasonable first step towards making applications accessible for users. This would include, I imagine, all of the steps of understanding applications I described above, making Juvix friendlier, and developing a higher level OO language.
2. Making simple applications accessible by users. You can’t have the full functionality of the low level AND the simplicity of representation. Simple things are simple because they do not include all of the details. Otherwise it is just renaming. LLMs change a lot, but I think there are a lot of safety questions to answer there first. Perhaps, providing a simple but limited in functionality application framework accessible by users is a good next step. This way we can give the users some autonomy without delaying it until we answer all the difficult questions

I think these two steps are natural steps towards the babeloma fish bright future, and once we are happy with their definitions, I think we should really focus on the first one

[cwgoes]

Excellent points , and I think the order of operations which you propose makes sense, I did indeed start at the stratospheric level here (but I think it’s at least helpful to have that goal in mind, while we focus on the immediate next steps).

I want to drill down on a few of the questions you raise here and see if we can make some progress. Let’s start with the question of application boundaries, which is more or less equivalent to the question of identity: what is an application? I’m going to try to approach this from two angles: the perspective of the developer (and, by proxy, the user), and the perspective of the network – and then attempt to synthesize them.

The developer/user perspective

From the perspective of the developer (and, by proxy, the user), an application is simply something that they’re using Anoma for, such as sending messages to friends or colleagues (multichat), operating a city-level bartering network (multidimensional DEX), or representing a complex biome (TerraTwin). From this perspective, the only thing that matters is the application’s observable behavior: what inputs can different users enter, and what outputs can they read? In order to develop an application, a developer must have a concept of what they want to be possible, and what not. If we were to try to visualize this perspective, I think it might look like this:

From the perspective of the developer or user, the application simply is its interface: the space of inputs, outputs, and behavioral properties thereof. How state, resource logics, or other Anoma network operational details work “under the hood” doesn’t matter as long as the correct inputs and outputs are supported and behavioral properties are preserved. A user can observe nothing else.

How should applications interoperate with each other, from the perspective of the developer or user? I think we can generally say that other applications should be able to provide inputs and consume outputs in the same way a user would directly, but that the desired behavioral properties must be preserved no matter what other applications are involved. Let’s suppose that we have three applications, where Application 2 provides inputs to and consumes outputs from both applications 1 & 3. Visually, we could depict this like:

Now, the inputs and outputs of application 2, and behavioral properties thereof, may be dependent on applications 1 and 3, so the developer of application 2 must understand and incorporate them correctly. Applications 1 and 3, by contrast, should not even need to specifically know of the existence of Application 2 – there is no need to differentiate the inputs coming from and outputs going to Application 2 from those of a direct user (and no way to do so anyways).

To recap, from the perspective of the developer (and, by proxy, the user):

1. An application is nothing more than an interface of possible inputs and outputs, and behavioral properties thereof. Internal details of Anoma do not matter as long as they operate in a way which preserves the desired behavior of this interface.
2. Applications should be able to interoperate on the basis of inputs and outputs just as if they were users. Developers of applications which depend on other applications will need to reason about all of the inputs, outputs, and behavioral properties involved.

Now, let’s look at applications from the network perspective.

The network perspective

From the perspective of the Anoma network – the distributed network of computers which run the protocol, run controllers, solvers, etc. – there is no such thing as an application. The network only knows about state, and rules for how that state can change. Whether a particular piece of state is considered by a user to be part of application 1 or 2 is not something the network can or needs to know. We may have many kinds of state in the future (such as MRDTs), but for this post let’s just talk about state in terms of resources. Limited to that scope, the Anoma network state consists of a set of unspent resources, each of which is controlled by a single controller, and each of which is governed by a resource logic which determines under what conditions it can be consumed (and under what conditions new resources can be created). Visually, we can put each resource in a controller box:

Synthesis

Whatever inputs and outputs applications have, they must be ultimately expressed in terms of consuming or creating resources (and perhaps storing or retrieving resource-linked data). While each resource is controlled by exactly one controller, it may be a part of any number of applications (in the sense of possibly being consumed or created by application inputs, and possibly being read as part of application outputs). For example, suppose that applications 1 and 2 both may write or read R3:

---

This merits further exploration, but unfortunately I’m running out of time for now, so I think it’s most helpful to summarize the main takeaway here:

1. Previously, e.g. in the RM specs, we’ve defined an application as the combination of a set of resource logics, a set of transaction functions (write interface), and a set of projection functions (read interface).
2. I now think that this definition is slightly wrong. Specifically, I think that we should exclude the resource logics. Applications should be defined only by their read and write interfaces (these will not be exactly the same as transaction functions) and behavioral properties thereof. Rather than saying that a particular resource logic is part of a particular application, we will say whether or not a set of resources implements a particular application. This distinction may seem subtle, but I think that in conjunction with further investigation it may clear up a lot of the conceptual issues here. Application definitions, similarly, should be only in terms of inputs, outputs, and behavioral properties – we should be able to completely separate the definition of an application from potential implementations. I think considering applications in terms of desired inputs, outputs, and behavioral properties can also clarify the questions around shielding and privacy, since – fully articulated – these inputs, outputs, and properties should include reference to who is supposed to know (or not know) what.
3. I see several immediate next steps here, such as:
   a. Trying to figure out more precisely what the types are (of inputs, outputs, and behavioral properties), and figuring out how these would map to transaction functions and state queries.
   b. Figuring out how this relates to the object model in discussion.

[cwgoes]

Let’s experiment with stating the kudos application in this form. That could look something like:

Preliminaries

• A user is an out-of-band actor identified by an external identity.
• Each kudo has a denomination, which controls minting and burning logic.

Inputs (possible writes)

• A user must be able to transfer any kudos they own to another user (or any number of other users, atomically).
• A user must be able to mint new kudos, when the kudos denomination it
• A user must be able to burn kudos they own, when the kudos denomination allows it.
• A user must be able to swap X kudos for Y kudos, with specified amounts.

Outputs (possible reads)

• A user must be able to read their kudo balance.
• A user must be able to read a history of changes to their kudo balance, and associated data.

Behavioral properties

• At any point, a user must be able to send or swap up to their balance, and no more.
• After a send, the sender’s balance must decrease by the amount of the send, and the receiver’s balance must increase by it.
• After a mint, the minter’s balance must increase by the minted amount.
• After a burn, the burner’s balance must decrease by the burned amount.
• After a swap, the swapper’s balance must increase by the target amount (of the received kudo), and decrease by the swapped amount (of the swapped kudo).

We already see, for example, that by articulating properties at this level of abstraction certain details of internal behavior are implied, such as an equivalence relation between having two kudos of quantities a and b and one of quantity a+b.

[cwgoes]

Notes from discussion

• @vveiln: takes a long time to figure out how to write applications in the resource model, particularly information flow control. Shielded applications: how many resources should we include in the action, how many resources should the resource logic take, ensuring properties of receive logic.
• @Michael: high-dimensional space in which you have to design an application. Resource scoping, different actors creating different actions, hard to reason about this.
• Hardest part: IFC translation to RM level IFC properties.

Action items

• @cwgoes to work with @mariari on object model that incorporates IFC

[vveiln]

@cwgoes

To recap, from the perspective of the developer (and, by proxy, the user):

1. An application is nothing more than an interface of possible inputs and outputs, and behavioral properties thereof. Internal details of Anoma do not matter as long as they operate in a way which preserves the desired behavior of this interface.

I feel like this property is the difference between a definition of an application and a definition of a properly functioning application. Anything that satisfies an interface can be called an application, but the internals cannot be fully ignored for properly functioning applications.

I understand that even for properly functioning applications we want the developers have a higher level view than now, but we shouldn’t exclude how things are now from consideration (which we do by not explicitly giving it space in the hierarchy / set of definitions): the developers need to be aware of the internals. In fact, this property also depends on what you define as internals. I cannot think of a system for which the application developer didn’t need to be aware of the internals and therefore it is hard for me to imagine it.

@cwgoes

Specifically, I think that we should exclude the resource logics. Applications should be defined only by their read and write interfaces (these will not be exactly the same as transaction functions) and behavioral properties thereof. Rather than saying that a particular resource logic is part of a particular application, we will say whether or not a set of resources implements a particular application.

Just noting: that also implies we should remove the application definition from the RM specs and put it somewhere else, perhaps in Application specs, which I think we should create. Resource machine is network architecture level, so defining applications from the user’s perspective doesn’t fit in there.

Generally I think the hardest part is connecting the two (user and network) views. Right now the developer must be aware of the network-level things, especially to define information flow control properties, and developing higher level abstractions must provide a higher level IFC interface as well. Although when we talk about who can see what, we are still talking about the “what” part. I wonder how abstract can the “what” become.

[cwgoes]

Part 1: The Current Problem Space

I’m going to try to break down the current problem space as I see it (focusing on the next immediate step, not the long-term “Babeloma” vision). In the current problem space, I think that we have three major levels and two major conversions, one of which is automated (compilation) and the other one of which is manual (what the application developer does).

The three levels:

1. The application interface – the high-level specification of what observable behavior an application should have, in terms of inputs, outputs, and behavioral properties (as described above).
2. The application intermediate representation (“Anoma-level”) – an “intermediate-level” definition of what kinds of state, reads, and writes an application consists of, including who should be able to see what (information flow control - type properties).
3. The application low-level implementation – the actual “operating-system-level” logic that implements an application, including resource logics, transaction functions, query logics, etc. (e.g. custom engines, MRDTs, etc. in the future). The low-level implementation also includes details like which resource machine primitives/proofs are used when, and all specific cryptographic choices.

The two conversions:

1. The application developer must write the intermediate representation, given an application interface specification, and reason about whether their intermediate representation correctly implements the application specification (eventually, maybe we can incorporate some automated verification tooling here, but that’s a future project).
2. We must be able to compile the intermediate representation to appropriate low-level implementation code which correctly implements it. This compilation should be fully automated, and need not be altered or understood by the developer (although it can be exposed to experienced developers so that they can control the low-level representation if they want to).

Pictorially, we could visualize this as follows:

What exactly is this intermediate representation? This is a complex design problem, and I think we’d best start with some “boundary constraints”. Let’s first address an elephant in the room: information flow control.

Part 2: Information Flow Control

Application Interface Level IFC

Let’s start at the application interface level. In the application interface, at the most basic level, information flow control means that the interface must specify who can read what outputs under which conditions.

Let’s take the kudos example from above, which had two specified outputs:

1. A user must be able to read their kudo balance.
2. A user must be able to read a history of changes to their kudo balance, and associated data.

I wasn’t clear in the above example, but suppose that we take the default to be non-access, e.g. no outputs should be possible to read other than the ones which the application interface explicitly specifies. Given that default, we can infer from this list of outputs that:

• No user should be able to learn anything about any other user’s kudo balance.
• No user should be able to learn anything about any other user’s kudo balance changes/history.

etc. – nothing should be able to be learned other than these explicit outputs. Of course, users can always choose to disclose any data they would like to, but this is not the primary concern of the application developer. In order to implement this application interface, then, we need a shielded resource machine – and fully private solving, if swapping shouldn’t reveal anything either. Likely this requirement is too strict, and the application designer might choose to alter the definition roughly as follows:

Inputs

• A user must be able to swap X kudos for Y kudos, with specified amounts, where the user selects a set of solvers S to whom they are comfortable revealing X-for-Y.

Outputs

• A solver who a user selects with a swap should be able to see X-for-Y as in their swap and search for solutions.

Given our non-access default, this output implies that no other solvers (or, indeed, nodes) other than the ones selected should be able to see X-for-Y for a particular user input. Implementing this requires a trust assumption – namely, that the solvers which the user selects do not further disclose the “X-for-Y” information – which must be made explicit in the expected output property. To be fully explicit, we we will likely want to specify information flow control properties as relations between outputs and inputs, given assumptions – for this example:

Input	Assumption	Output
Swap(X, Y, S)	\forall s \in S. honest(s)	SwapRequest(X, Y), visible to only s \in S

We’ve now introduced solvers as an actor in this application definition – there is no implied semantics here other than “the node which the user chose in their swap input” – but note how which solver is supposed to be able to read the output depends on the field in the user’s input. Presto, information flow control!

To contrast this: a fully transparent kudos application interface would specify that anyone can read anyone’s kudo balance, transaction history, and swap intents. This application interface would then be fully implementable on a transparent RM.

Note that for now, we’re just focusing on information flow control of state, including in intents, but not including metadata like transaction timing. This is also important to think about, but I think we should figure out the state-level information flow control throughout the whole application programming stack first.

So – roughly – information flow control at the application interface level consists of specification in the inputs, outputs, and behavioral properties of who can read which outputs under what conditions. Other than the explicitly specified outputs, and assuming explicitly listed assumptions, no one should learn anything (default “non-access”). This are really information flow policies, in that they are fully declarative and specify who should learn what (and, by exclusion, who should not), but prescribe no specific cryptographic or operational mechanisms. This needs to be worked out in much greater detail and with more examples, but let’s first take this rough concept and see what it means for the intermediate representation.

Intermediate Representation Level IFC

I’m going to take some interpretive jumps here, attempting to synthesize past work here, here, and elsewhere, and make a specific proposal: let’s imagine an intermediate representation with two sub-levels, both with “objects”, which I will call the “high-level declarative object application IR” and “mid-level immutable object application IR”. Both of these levels are total, in the sense that an application can be represented fully at the respective level of abstraction, without any “abstraction leakage” from other levels (and both levels can be understood to have defined semantics, which correct compilation to a lower level must respect).

I think it’s best to start with a diagram:

Roughly:

In the immutable object-level application IR representation:

The state of an application at any given time consists simply of a set of immutable atomic (indivisible) objects. These immutable objects define five methods:

1. holds_always, a predicate which must always return true for this object (and is checked on both creation and consumption), with scope of only the object itself. In a type theoretic formulation, this predicate corresponds to a well-formedness rule, and can be used to implement a “type” of object, colloquially speaking, e.g. int64.
2. holds_on_creation, a predicate which must return true in order for the given immutable object to be created. In a type theoretic formulation, this predicate corresponds to an introduction rule.
3. holds_on_consumption, a predicate which must return true in order for the given immutable object to be consumed. In a type theoretic formulation, this predicate corresponds to an elimination rule.
4. holds_on_transaction, a predicate which must return true in order for the given immutable object to be created or consumed, with scope of the transaction (details TBD, but this would correspond e.g. to the balance check).
5. can_read, a function which (given the object) returns an external identity which should be able to read the object (implying, by exclusion, that any identity not a member of this external identity should not be able to read the object).

These immutable objects can be written by object-level actions – which can query, create, and destroy immutable objects – and read by object-level queries, which can simply read immutable objects (and compute/return some result).

This representation is compiled to a low-level application implementation, with object-level actions roughly corresponding to transaction functions (and other code run in the local domain to generate intents, create proofs, etc.), immutable atomic objects roughly corresponding to resources (although this mapping is not 1-1, it will probably be n-1, where many small objects can be represented by a single resource), and object-level queries roughly corresponding to resource-level queries. The can_read information determines which kinds of proofs are created (when generating proofs / in transaction functions) and who data is encrypted to (and entails that e.g. a shielded RM is used unless all data is totally public).

In the high-level declarative object-level application IR representation:

The high-level declarative object-level application IR representation is currently a grab bag of “nice abstractions that we should be able to compile to immutable objects”, including:

1. Mutable objects (compiled to unique sequences of immutable objects) (this correspondence is described roughly here).
2. Read-only methods on objects which can have different permissions for who is “allowed to call” them (compiled to sets of immutable objects with the appropriate can_read functions).
3. State-level constraints (e.g. kudos total supply is constant) (compiled to holds_on_transaction predicates in immutable objects).
4. Pseudo-objects / views e.g. MyKudosContext, which contains all of my kudos (compiled to object-level actions and object-level queries).
5. Pseudo-methods e.g. MyKudo.send (compiled to object-level actions).
6. Higher-level information flow control properties, e.g. who can call what methods, complex conditions on who should learn what when (compiled to immutable objects / actions / queries).

I imagine that we might want to add more in the future. Many of these transformations are independent and don’t necessarily need to belong in the same “level”, but they all sit between the application interface and the mid-level immutable object IR.

I think the next step here is to try to work out the immutable object-level application IR in detail, but I first want to pause and solicit feedback from folks – particularly @mariari @vveiln – to see if this general approach / structure makes sense.

[mariari]

Argumentation

I believe this proposal needs a lot of work shopping, namely I believe there is an over focus on levels without meaningful distinctions that make it hard to properly plan and will devolve how we discuss these things moreso than bringing meaningful structure. However this proposal did give me a better idea of what you mean specifically with IFC and from there I believe we can move onto potential solutions that will let us express what you want (albeit not in the way that is expressed here, but more on that later) via considering what a meta level means for us. Further I think you outlining the specifics of what you want is helpful for us coming to what we should be designing.

With that said I’m going to now go through the post in more detail. The responses will range from low level nitpicks that matter for communication but get in the way of my overall response and pointed argumentation that reinforces the thrust I listed above.

Quoted Response

@cwgoes

The three levels:

1. The application interface – the high-level specification of what observable behavior an application should have, in terms of inputs, outputs, and behavioral properties (as described above).
2. The application intermediate representation (“Anoma-level”) – an “intermediate-level” definition of what kinds of state, reads, and writes an application consists of, including who should be able to see what (information flow control - type properties).
3. The application low-level implementation – the actual “operating-system-level” logic that implements an application, including resource logics, transaction functions, query logics, etc. (e.g. custom engines, MRDTs, etc. in the future). The low-level implementation also includes details like which resource machine primitives/proofs are used when, and all specific cryptographic choices.

I take umbrage with this characterization of these levels. For this particular section I want to focus on the issues between 2. and 3., when we get to the levels in IFC I will bring stronger arguments for why I believe how the distinctions between 1. and 2. are poor framing devices.

Let us view the issue from two scenarios that are completely removed from our domain:

1. We are making a computer algebra system and we are doing symbolic processing
2. We are making a computer algebra system and we are doing heavy numeric processing

In scenario 1. and 2. we are concerned with accurately representing algebra systems and various constructs found in analysis, group theory, etc etc etc. From the perspective of 1. symbols are very important and actual numeric representation does not matter; from the perspective of 2. the symbolic representation is maybe used rarely but the main purpose is doing computation where the numeric representations are the most important aspect of the system.

Now from the perspective of 1. is numerics (integers, floats, complex numbers, etc) considered low-level/operating-system-level style operations? Maybe? It’s a lower form of reasoning, however from the perspective of 2. is symbolic (representing operations with abstract symbols and doing rewrites of an algebra system) low-level/operating-system-level. Absolutely not, it is something simply not used in the toolbox.

What this analogy is trying to get at is that unused parts or parts not considered are not somehow lower level. Your 3. implies precisely this brand of thinking as you mentioned

@cwgoes

the actual “operating-system-level” logic that implements an application, including resource logics, transaction functions, query logics, etc. (e.g. custom engines, MRDTs, etc. in the future).

bolding to make the issue apparent.

Now I believe if we changed what “layer 3” means it could be useful with a well defined semantics.

Namely I believe your layer 3 ought to mean a reification of an underlying execution machinery. Reification is simply expressing something at the language level and that would be of the underlying execution machinery (RM object types (transactions, resources, etc), MRDT, etc). Engines would not be apart of this as it’s not a base assumption of our execution model in any meaningful way (they are a useful modeling tool, that only requires isolated cheap processes in practice).

@cwgoes

The application developer must write the intermediate representation, given an application interface specification, and reason about whether their intermediate representation correctly implements the application specification (eventually, maybe we can incorporate some automated verification tooling here, but that’s a future project).

This quote with the previous section gives me great confusion as to why we are calling anything an intermediary representation. If programmers are programming in it, nothing about this is an IR. We do not call C an IR nor call Lisp or Haskell an IR. These are simply language or systems we program in.

I will give a stronger point when I fully dissect the 1->2 representation later.

@cwgoes

We must be able to compile the intermediate representation to appropriate low-level implementation code which correctly implements it. This compilation should be fully automated, and need not be altered or understood by the developer (although it can be exposed to experienced developers so that they can control the low-level representation if they want to).

This section is mostly fine without much argumentation from me.

@cwgoes

What exactly is this intermediate representation? This is a complex design problem, and I think we’d best start with some “boundary constraints”. Let’s first address an elephant in the room: information flow control.

Same issue as above, but yes we should be precise on what we have issues with and I can rally research once I understand the problem.

Part 2

@cwgoes

Part 2: Information Flow Control

Instead of quoting this paragraph by paragraph I will give a greater rebuttal about the framing you are doing here, what you really want and attack plans we can commence. I will quote things as I find relevant.

Level Framing

The first overall point I want to make is that the framing of the given levels here are not what you think they are.

@cwgoes

@cwgoes

In the immutable object-level application IR representation:

1. holds_always …
2. holds_on_creation …
3. holds_on_consumption …
4. holds_on_transaction…
5. can_read …

@cwgoes

Both of these levels are total, in the sense that an application can be represented fully at the respective level of abstraction, without any “abstraction leakage” from other levels

For example we have layer Higher Declarative (HD) and Immutable Object (IO). We define here that IO has these 5 methods, if we truly abstract these layers out then HD would not have these… except we do want to have these concepts even there. For example is not a state_constraint not be expressed through this or mutable objects may not fulfill properties that must always be true? It becomes apparent that HD must too have all these properties. Making the facade that these are levels isolated complete levels dissipate.

Now what are these? There are indeed levels, however they are not all clean separate layers, instead we can think of these more akin to protocols that we can determine. For example when we consider the design of the Meta Object Protocol (MOP) there are many protocols that make the Common Lisp Object System (CLOS) operate. Namely there is an initialization protocol, there is a finalization protocol. In the machinery these work together in orchestration at the CLOS level, namley creating an instance works on these to realize itself. However what is interesting is that initialization need not care about what layer it is. The implementers simply use it’s effects in a determine way to make CLOS work as they believe a good default is, an user can instead change how this works, utilizing code from the CLOS level or maybe even peeking further under to change other behaviour. Nothing about this process stops the lower level talking about higher level things as this is exposed to the user and works as a system together (higher components have effects ABC, low level system have have specific effect D).

What we must do is something like this:

Excalidraw — Collaborative whiteboarding made easy

Notice that instead of having monolithic layers that must work together as a layer, we instead have the true dependence of the items and how they sorta relate to the user. Let us take mutable objects this as an IR layer does not make sense. What we need to talk about is how to even do these as resources we have to find a way on how this updating schema actually works for when it maps to a resource. What does get exposed is that we must explicitly talk about updating schemas as a protocol and this would be akin to the initialization protocol mentioned in the CLOS section. Then what we determine is how this properly looks to the user (we are at the CLOS level now not the MOP level), can we make the default schemantics make it looks like normal java style mutation? If so great we can give that as a default point, however we may determine we may want some other default behaviour at the user (CLOS) level. We can talk about this for most of the things you want, where some things need to be moved around. For example we should consider our execution model explicitly with constraints first as per something like a WAM machine doing it as a layer ontop is fine to do now as a thought experiment to get ourselves started but we should be conscious of what we really want there.

As per the other items in your diagram please note that only Objects and message sends exist there are no psuedo objects or psuedo methods. Even the example of Guis are just certain kinds of messages the object responds to (a method) and tagged to let the system know it’s a view/gui, this is something you’d want at all levels always as soon as your system bootstraps it (meaning you get it even at a low VM level).

An issue on meta

Now let us move onto a bigger meta point

@cwgoes

Let’s start at the application interface level. In the application interface, at the most basic level, information flow control means that the interface must specify who can read what outputs under which conditions.

@cwgoes

Input	Assumption	Output
Swap(X, Y, S)	\forall s \in S. honest(s)	SwapRequest(X, Y), visible to only s \in S

We’ve now introduced solvers as an actor in this application definition – there is no implied semantics here other than “the node which the user chose in their swap input” – but note how which solver is supposed to be able to read the output depends on the field in the user’s input. Presto, information flow control!

Your first quote denotes an understanding that these are meta operations but your followup here betrays this making a grave sin as thinking that swap should at this level should discuss a solver. Nay this is not something that concerns a base level operation, why would writing simple logic that swaps owners care about who unifies the result? Why would we take logic so universal and make it so unfit for any general purpose? No this kind of information strictly exists at a meta level and we should treat it as such. Where do we really want this information to be flown from? Your post gives us some hints

@cwgoes

To contrast this: a fully transparent kudos application interface would specify that anyone can read anyone’s kudo balance, transaction history, and swap intents. This application interface would then be fully implementable on a transparent RM.

See here you tied with the kudos object, notice before you tied it with some nebulous operation but in fact this exists as some meta property of the kudos object itself. A better interface would be as follows:

1. Add a reflect message on all objects, this lets us work at the meta level
2. For any IFC policy you want, write it down, write them all down and send them to @cdetroye and @mariari to further look into them
3. Together we can come together and come up with a protocol that is flexible enough to give us the behaviour
4. This protocol will be our IFC protocol (potentially many protocols working in tandem).
5. This protocol exists only at the meta-level and we can think about how we wish to make this interoperate with particular objects (mixins exist at a class (meta object) level, however we care about meta-object attached to a particular object. I know pharo has works on traits for specific objects so we can apply that research cc @cdetroye )

But overall I have a much better idea on what you mean by IFC now, however I believe shoehorning it at the base object level is a poor move, but I believe that shifting it to the meta level is a good solution that gives us a principled approach on solving this.

For intuition, (class-of object), is a metalevel operation as you are asking an object it’s type. This is much like how in many type systems they disallow this for certain theorems. Thus we’d have to say (class-of (reflect object)), as reflect is our only entry point into the meta level. Anything that isn’t directly related to the object gets put here.This gives us many interesting properties such as:

1. We can know statically when we reflect into the meta level
2. We can express compiler backends as a meta level operation
3. we can express where objects live in terms of reflecting and asking about where they live
4. we can express IFC by doing this

Thus we should consider our object design and interface design with these principles in mind

[cwgoes]

Thanks for the response – I’ve addressed your individual concerns below.

@mariari

What this analogy is trying to get at is that unused parts or parts not considered are not somehow lower level. Your 3. implies precisely this brand of thinking as you mentioned

@mariari

Namely I believe your layer 3 ought to mean a reification of an underlying execution machinery. Reification is simply expressing something at the language level and that would be of the underlying execution machinery (RM object types (transactions, resources, etc), MRDT, etc). Engines would not be apart of this as it’s not a base assumption of our execution model in any meaningful way (they are a useful modeling tool, that only requires isolated cheap processes in practice).

By “low-level”, I mean "low-level from the perspective of a developer who is thinking about application inputs, outputs, and properties (the high-level interface), and objects (the intermediate representation). From the perspective of this developer cryptographic details are indeed low-level as they do not pertain directly to anything about the application itself – any primitive satisfying the correct interface will do – and if we can manage to compile the intermediate object representation to the resource machine, the resource machine execution model itself is also low-level from the developer’s perspective (they just care that the object level semantics are correctly implemented). I think that this is indeed a “reification of underlying execution machinery”, I don’t see a contradiction there.

Do you take issue with this basic characterization, and if so, why? I admit that I haven’t thought through custom engines very much, I was just imagining that we would also want to support those as part of applications in the future.

@mariari

This quote with the previous section gives me great confusion as to why we are calling anything an intermediary representation. If programmers are programming in it, nothing about this is an IR. We do not call C an IR nor call Lisp or Haskell an IR. These are simply language or systems we program in.

We can come up with a different name, no strong preferences there.

@mariari

Now what are these? There are indeed levels, however they are not all clean separate layers, instead we can think of these more akin to protocols that we can determine. For example when we consider the design of the Meta Object Protocol (MOP) there are many protocols that make the Common Lisp Object System (CLOS) operate. Namely there is an initialization protocol, there is a finalization protocol. In the machinery these work together in orchestration at the CLOS level, namley creating an instance works on these to realize itself. However what is interesting is that initialization need not care about what layer it is. The implementers simply use it’s effects in a determine way to make CLOS work as they believe a good default is, an user can instead change how this works, utilizing code from the CLOS level or maybe even peeking further under to change other behaviour. Nothing about this process stops the lower level talking about higher level things as this is exposed to the user and works as a system together (higher components have effects ABC, low level system have have specific effect D).

I think that I generally agree with this for everything above the “immutable object-level representation” – indeed, all of the various features and abstractions above that are not really a clean level, and more a set of options that could be combined in different ways.

However, I think that the immutable object-level representation should be a complete level, in that no details from the underlying resource machine execution machinery (or other future execution systems, e.g. MRDTs) “bleed in” to the object-level representation – we should be able to define a complete semantics for the object-level representation, and mechanically verify that the result of compiling and executing that on the actual execution machinery will respect this semantics. That doesn’t mean that we can’t expose underlying details and choices to developers who want to customize them – we can, and I’m very much in favor of doing this – but it should not be necessary for developers to understand them in order to program correct applications.

@mariari

Nay this is not something that concerns a base level operation, why would writing simple logic that swaps owners care about who unifies the result?

That’s not what the application designer cares about in this scenario. What the application designer (and the user actually making the choice) cares about is who learns the information.

@mariari

Why would we take logic so universal and make it so unfit for any general purpose? No this kind of information strictly exists at a meta level and we should treat it as such. Where do we really want this information to be flown from? Your post gives us some hints

I don’t really understand what you’re objecting to here. If we define the application interface in terms of inputs, outputs, and behavioral properties, and we want users to have some control over who data is disclosed to as a result of their inputs, users must specify this in their inputs – there is no other possible way to implement this! There is no “meta level” from this perspective, a “meta level” sounds to me like a programming language or library design choice (which I would not object to at all, but which also doesn’t contradict anything I’m proposing here).

[cwgoes]

Action items

• @cwgoes to work out detailed mathematical semantics of immutable object protocol.
• @mariari and co. to write up engineering’s concept of a “base / underlying object model”.
• @mariari and co. to write up requests for any and all high-level protocols (e.g. scry with object-paths).

[apriori]

Notes from chat 4/16

Discuss object model questions for clarification

Jeremy issue is the IR object level api really different from interface? I don’t think these are technically separate levels.

Common lisp object system is an example. def message, def class etc.

there is not an immutable object application IR, you have immutable objects and user can program with those and if you want to give them specific abstractions thats nice. It kind of blends in, immutable object application IR blends into application in a ways.

application interface = definition of the application. definition of the external behaviors of the application in terms of input outputs and behavioral properties. In this case the application is object is singular.

Jeremy - I have an object which is an application composed of many objects internally thats why i am saying no. Kudos for example is an object internally it may be composed of different object. I can use kudos as an object in a bigger application. Now we have tic tac toe that is an object has a different object in there called kudos.

Agree internally applications will be composed of more than one. An application interface merely specifies what an application must do in terms of its external behavior. We implement applications in terms of many objects.

Jeremy - app interface is a mixture of real code and maybe high level english. As we get better at describing this we can move more of it into code as time goes on. App dev must be able to translate english to mathematics.

Details that a user doesn’t care about but if you are a programmer you care about, anyone who wants to debug kudos. If it breaks you can peak into it and make a patch for example. these are different concerns about who cares about that.

Chris - app interface is minimal way to define what a particular application is. kudos must take certain inputs have certain outputs and some behavioral properties. We should equate application identity tot the external interface because thats relevant to users.

modeling things as objects solved things didn’t expect. Jeremy solved all my problems with modules (Juvix).

Lets bracket the question of engines for now. Engines are just an unrelated entity.

If we want to broaden what application mean beyond the RM then engines are kind of irrelevant.

We agree on the bottom and the top lets talk about the middle

The Immutable object application representation - Jeremy would not phrase it like this, but if you have objects you don’t want to talk about things that are not objects and dont want the semantics of the RM to bleed through in unnatural ways.

Have this layer where everything is an immutable objects. Lets say we have a way to make immutable objects, maybe we make a protocol around this. Common Lisp has slots, first class citizens on an object and you can reprogram it and turn off mutation making slots read only. Even at the level of dealing with multi-level immutable object IR we can still talk about mutable objects there. Even at this lower level you can still talk about higher level concepts and do analysis on it. what you really want is certain properties about your objects.

Chris - thing looking for to be minimal is semantics to make the behaviour of this layer clear. My proposal is at this layer in order to define what it means to define applications dont need to care if the higher level thing we are representing is mutable or not just need to respect the transition rules.

an object is recursive on itself and it can dispatch messages. Our object model is very weird we are not like small talk we are like lisp. in small talk methods live on objects themselves. If we do it in a common lisp model there exists methods they live somewhere but they dont live with the object itself. Object is just concerned with its owned truth and thats how we will probably define it. What I dont have a good feeling for is actions and mapping it down to the RM. Have a fairly good idea about the semantics of the object model. Higher level features and mapping it down and what interfaces we want to give?

these are on top of the object machinery. these particular methods have semantics and we can define them mathematically and thats more of what we can get at than object semantics. Agree that many things are weird.

At the immutable object level, objects are atomic pieces of state with methods. the state we are treating as not visible to anything other than themselves. However the methods still correspond to these action, transaction, contexts. These could be described as a base object. For the type of state we are trying to model these are essential. If you have objects without these methods we wont be able to compile it.

Smalltalk - When you send an object a message, the object looks it up in its hash table. The method is an object and invokes a value function with it and sends the arguments. this isn’t baked into small talk.

If you want to engines in this you can get engines for free by changing how a message send with an object works. We can do this if we allow this to be reified with other mechanics. the base object model technically doesn’t need this. What you want to describe is the important mechanics.

We could plug in always true to be a type system for example.

Thinking about Interfaces and specific mechanisms necessary to implement the stack or create a state model.

There is arguably in the RM, how our state system works, the input to these predicates mainly holds on creation and consume are messages. Currently they are modeled as specific methods and the messages are passed in the arguments. when you create an object its like sending a message and when you consume it like receiving message.

this is normal as in small talk. when you create an integer class for example. this goes down to primitives in the VM. usually in smalltalk you have create and initialize. Everything is done through message sends in an object system. This formulation is consistent and it should work. Just wondering weather its aesthetically optimal? What I find intuitively fuzzy is this embeds assumptions about how execution works that doesn’t work for us.

Jeremy - its just a framework of talking about methods. If you want to update state you just return a new object. no assumptions about sequential execution or parallel execution.

Jeremy uses logtalk built on prolog , you can send objects messages, it just puts constraints on the state. Very complimentary with the declarative model. You send a message you get back constraints, don’t think it implies imperative execution.

My question is more weather its a good idea to think of the immutable object level - to think of these 5 things as methods or something else. Its not clear. If I cant come up with anything I like better we’ll go with this.

What do you think is necessary for doing this? We can expand on this - creation for example. What you are trying to design here is a protocol. What matters is - is this the interface you are satisfied with.

Jeremy gave three methods he thought were useful. Next foray add two more methods and continue to iterate on this. If we have something comprehensive we can make it a lot more flexible when we are satisfied with the design. Its fine being arbitrary for now, shouldn’t be the biggest concern.

Maybe it makes sense to split this into two tracks. Logically formalize immutable object system. So we have defined semantics. Want specified mathematically as opposed to just operationally. The thing that is most conceptually difficult is how to do information flow control and what this means for intermediate levels. At a point in logical time there is a set of mutable objects. There is a can read object which the lower level has to enforce.

Jeremy main issue is leaking meta level into base level.

One track - formalize immutable object level IR
other track - flesh out features we’d like at a higher level, higher level IFC properties, state constraints.

Jeremy not sure constraints are at a higher level necessarily. We may want constraints at the immutable object level. State constraints are not at the immutable object level given the kudos application. Specific constraints but not constraints in general. True global constraints we can never check them.

Jeremy - the base execution model probably has constraints at the VM level. Yes, but the base level has constraints scoped to transactions and actions. What is an action a message send? It is related to the input scope, holds on created and consumed, constraints to the object.

This just says we have constraints on objects.

There are objects at the immutable object level are “real” in the sense the objects above it are not “real”. Jeremy - just because they are higher doesn’t mean they are not real. The way to think about it is you have protocols that allow you to talk about more things which are just as real as everything else. Some object constraints are directly executable others need to be compiled.

The units of execution time. There is a specific time, certain scopes for times we can actually execute. Mutable objects do not belong on that level.

Specify a protocol like this and how it relates to a distributed state architecture. Protocol (messages between objects).

If there are other high level protocols you want can you sketch those? will think about these and try to keep these in mind. What about key-spaces? Paths are objects. Jeremy will write-up.