Questions I want to see researched

Protocol Design
1

Preamble

There are many interesting questions in Anoma that have gone answered. Below is a small collection of them mainly focusing on long term burning quests I need answers to to better outline the long term system of Anoma

Terminology

  • Message Send :: To invoke a method on an object.
  • Method :: An operation on an object. Methods are themselves objects
  • Chambers :: Private controller domain
  • Court :: Shared controller domain between many parties
  • Domain :: Either the court or chambers
  • Have Control :: To be the dictator of a domain and can force changes
    at will
  • CLOS :: Common Lisp Object System
  • Erlang :: The Erlang programming language™
  • Upgrade :: To change code from one state to another
  • Engine :: An Actor
  • Overload :: The ability to change the behaviour of a method in some
    manner

Problems we need to think/specify

  1. What times of computation are there?
  2. What is the object structure of courts and chambers?
  3. How do structures get upgraded within a controlled domain? Do we
    steal with CLOS does or what erlang does? How do we initiate
    updates for domains we don’t control?
    1. If we have a way to initiating code updating on say other
      courts, does this work via message send?
    2. Can we initiate consensus upgrades this way?
  4. How does consensus upgrades work, when does the new code take
    affect? Is this an overloadable method on the court?
  5. What is the boundary between court and chambers?
  6. What is the interface for engines in system
    • What properties do we wish to gain out of them?
    • How do we synthesize event based programming with them? Do we
      steal urbit’s design a bit?
  7. Can we realize our long lived actors as objects? What about them do
    we wish to prioritize?
  8. How do we realize an OO protocol ontop of the RM model
    • My first stab at it was here:
      On the Will to Adapt: A Resource story
    • If everything as a resource computation, then =(+ 1 1)= is a
      transaction, where we get back the resource/object =2=. In this,
      do we just return all new committed objects!?
    • Do we make the model declarative? If so, then do we get solvers
      for free by unification!?!? Further, should we go full in on the
      declarative OO model?
  9. Can we realize Identities in system, what is the interface like?
  10. Do we do RM transaction processing in the chambers? How does it
    relate to our current work? How does it relate to our Object story?
  11. What is the boundary between Objects as resources, and the
    immutable data in storage?
    • I got a partial answer from Ray before, when we scry the data
      since we give the type we inherently wrap it in an
      object/resource interface.
    • Another method could be we interpret raw data as an object in a
      wrapper.
  12. How do we ensure the code that each court runs is the same!? How
    can we trust the other parties to run exactly the same code?
    • I’d imagine it’d be a pain to maintain forks if there are
      frequent upgrades but if there is money on a line I’d imagine
      this would be an issue.
  13. What are the major flows of the versions?
    • Is every engine encompassed in some kind of flow?
    • Namely is the user flow and all components thought of for a
      particular release.
  14. What is interop like within the various resource machines?
    • This is a very broad question
  15. How does indexing work for shielded
    • I believe xuyang did research good to get this written up
  16. Can we define what an intent is
    • I like the idea of coalgebras but it should probably be defined
      in the specs, I think this may have interesting synergy if we go
      in a declarative model
1 Like
2

Thanks for writing this up. I’m going to start with three clarifying questions:

  1. For the purpose of this topic, how would you define “an object”? I need a definition which does not simply refer to what Erlang or Common Lisp does - what are the essential features of “an object” which you want Anoma to be able to support?
  2. What is the best existing example of the “declarative OO model”?
  3. What do you mean by “Urbit’s event-based programming design”?
1 Like
3

Are these intentionally circular references?

Regarding 14: you might be interested in reading this RM interoperability questions and concerns
Regarding 15: you might be interested in reading this Shielded State Sync - Anoma | Research & Development Forum

4

Question 1

Glad you asked, I will take my definition mostly fromthe paper above along with minor clarification, the paper takes a denotational approach.

Here is a quote from the summary on the features of Objects:

“An object is a value exporting a procedural interface to data or behavior. Objects use procedural abstraction for information hiding, not type abstraction. Object and and their types are often recursive. Objects provide a simple and powerful form of data abstraction. They can be understood as closures, first-class modules, records of functions, or processes. Objects can also be used for procedural abstraction.” (3.10)

I can thus summarize it as such

  1. Objects only know of themselves (they are autognostic)
  2. Objects are higher order values (it is the same as passing around closures!). This is often referred to as late binding or dynamic binding.
  3. Objects use interface abstractions for their interface (not type abstraction, I.E. if we have a set, we can have a set object contain many different types of Sets. See the papers definition of Sets for an example, I can also post my prolog code if so desired). Thus it is to say object interfaces do not prescribe a specific representation for values.

I believe this should be the “core features” though this isn’t everything I wish to support or go for.

For what I want us to support I want us to support the full Meta Object Protocol (MOP) as the reflective features and control of allocation and creation are highly important. We will see use of it when we wish to query (Block Chain word: index) for certain details about data. We should not really pick and chose features here as that leaves with a more broken system that gives us a harder time to realize the entire system of Anoma.

Thus here are some extras features of this:

  1. We should support inheritance, but it ought to be customizable by the user
  2. We should be able to reflect on every part of the object
  3. Objects should redefinable within a domain (reflection will help with this)

Thankfully there are books and design documents on this subject and it can be studied.

I can go on for a while about nuances and what “message sends” are, but I’ll stop myself while I’m ahead.

Some choice quotes from the paper can be seen below, please do read the paper in earnest:

  1. “An essential observation is that object interfaces do not use type abstractions: there is no type whose name is known but representation is hidden” (3.1)
  2. “Object interfaces are essentially higher-order types, in the same sense that passing functions as values is higher-order.” (3.1)
  3. “Objects are almost always self-referntial values, so every object definition uses µ” (3.2)
  4. “However, inheritance will not be used in this section because it is neither necessary for, nor specific to, object-oriented programming” (3.2)
  5. “Just as the ADT version of integer sets had two levels (set implementations and set values), the object-oriented version has two levels as well: interfaces and classes. A class is a procedure that returns a value satisfying an interface.” (3.2)
  6. “This means that the union method in a set object cannot know the representation of the other set being unioned.” (3.3)
  7. Objects are autognostic: “An object can only access other objects through their public interfaces.” (3.3)
  8. “Object interfaces do not prescribe a specific representation for values, but instead accept any value that implements the required methods. As a result, objects are flexible and extensible with new representations” (3.4)
  9. “abstract data types have a private, protected representation type that prohibits tampering or extension. Objects have behavioral interfaces which allow definition of new implementations at any time.” (3.4)

Question 2

There are probably others, but I was experimenting with this last weekend, it is an OO prolog library that adds OO data abstractions to Prolog.

Question 3

https://docs.urbit.org/system/kernel/arvo

I am talking about Urbit’s Arvo system, they independently recreated genservers but based on a different set of primitives, @ray is the best person to explain it exactly but it seems directionality correct of what I do understand.

3 Likes
5

No, when I wrote the Chambers and Court definitions I was thinking in terms of the controllers report as researchers like to use the word domain IIRC. And when I wrote Domain I was using it as a short hand for the definition I gave above. All the writing in the paper outside of the Court and Chambers definition refer to the one I posted.

These are good threads thanks for bringing them to my attention

1 Like
6

Thank you for the paper, I found it a clear, helpful, and entertaining read. I have never seen that definition of objects before, but it makes a lot of sense.

Just as a note, the resource machine – as it stands – does not enforce this, resources are able to inspect the representation of other resources involved in the same action. They do not, however, have to – so an object system written on top of the resource machine could enforce this restriction.

I do not think that this will be a problem, but I will note that the resource machine does not allow us to escape the fundamental efficiency tradeoffs of late binding.

I think that this is the main part that we need to figure out (with respect to how to implement objects on the resource model). This question is related to the question of “standardized resource semantics” discussed here, which I think we could alternatively call “methods with properties”.

For example, we want to be able to define an interface (class?) with a view function:

class OwnedResource r where
  owner : r -> ExternalIdentity

and the property that the resource cannot be changed in any way that the owner has not explicitly authorized (which the resource logic has to “prove” in some sense, see the linked thread for further context). Having interfaces with provable properties also seems to me like it solves a major challenge for object-oriented programming which is mentioned in that paper:

Screenshot 2025-02-14 at 23.14.17
Screenshot 2025-02-14 at 23.14.17936Ă—634 38.6 KB

This is basically the same problem as the one I describe in that thread, and the ideal future solution which I propose there is no more and no less than a distributed-operating-system-compatible form of a behavioral specification: in our case, a specification of how the state of the system itself can evolve over time, which is a little different than the cases considered in the paper. One might say that our objects are persistent.

In terms of the question of mathematical representation discussed in the paper, the resource machine should be perfectly capable of representing structures by their characteristic functions, although it doesn’t change the tradeoffs of that representation (e.g. the impossibility of iterating over sets represented by characteristic functions). This will perhaps make the work of solvers more complex (as compared to an algebraic representation), but it will also perhaps allow different (and perhaps at times more efficient) formulations of CSPs, I’m not sure yet.

As for the MOP:

I think this is purely a “programming system choice” and is fully compatible with (but not implemented by) the resource machine.

This sounds like more of a question of practical engineering and tooling. Seems possible to me.

I do not understand this requirement, can you further define or link to what you mean here?

One further thought: I would consider objects as they are defined in this paper to be purely a form of data abstraction, not a computational model. Contrast this to our concept of engines (as defined in the specs), which is a computational model and is not opinionated about the form of data abstraction. It is indeed a computational model that features sending messages around, which may sound superficially similar to “sending messages to objects”, but to me the choice of computational model and the choice of a form of data abstraction are independent. It may be the case that particular forms of data abstraction are more or less efficient to implement on particular computational models, but I don’t think this will matter, because the principal constraint in our choice of computational model is correspondence to physical reality, and that constraint is sufficient to fully determine the contours of the engine system.

2 Likes
7

I’m going to start addressing some of these questions in individual responses. Many may merit their own topics, but we can split those out gradually.

This question can be asked from many different perspectives. From the perspective of an intent and subsequent transaction, I’d say:

  1. Chambers (client) computation to translate the user intent into an Anoma network intent.
  2. Many different times of potential computation on solvers trying to match the intent as it flows around the network.
  3. Assuming that the intent is eventually matched, execution of the resulting transaction function on the appropriate controller, and verification of the transaction.

So, roughly, for a transaction, we might class the times of computation as:

  1. “Chambers-time computation”
  2. “Solver-time computation” (composed of many sub-times)
  3. “Court-time computation”

and they always happen in that sequence. Is this the flavor of question you wanted to ask?

n.b. I think we should pick terms other than “court” and “chambers”, they strike me as too… semantically loaded, at least for our own communications. I would prefer something like “local domain” and “foreign domain”, which also conveys the relativity that I think we want here. Thoughts?

8

What structures are you referring to here? Are you talking about resources (or objects implemented using resources) which are controlled by a particular domain? Are you talking about the rules for state updates in that domain itself (e.g. the RM version, or something related)? The answers will be different for different structures.

In general, I’d say that we don’t initiate upgrades for domains we don’t control, unless I misunderstand the question somehow (and in that case, what did you have in mind)?

9

I believe we can satisfy this for a given type system. I want AL to support pluggable types in time. Something important is that the type checker must be a function in system, so if you have different type checkers with different properties that you can attach to methods, then you could in practice query for all implementers of the interface and it must satisfy the specific type check you care about:

Namely the images under section 5: Standardization of Application-related Data - #19 by mariari

I post some example queries from GT here, just imagine another filter for satisfying some property.

I mention these, as these are required to have a live system of anoma without having to define extra out of system constructs. Just a note on practical requirements that are important that we try to realize.

These protocols are specified to be exact, and we will want all the features and maybe a few extra protocols for our own specific needs. It’s good to get in the habit of understand how things like the MOP compose as we’ll want a similar design ethos to our own extra needs.

I mean, we should be able to add new methods to an existing object, we should be able to add new slots and have all objects in the domain be upgraded with the new values. Etc etc etc.

Literature which talks about upgrades:

  1. The CLOS/MOP spec
  2. Erlang upgrading mechanism (on_codechange)
  3. Urbit’s on_load
  4. I believe some smalltalk book (I don’t know which book of theirs off hand, maybe the blue book?)
1 Like
10

I agree with pluggable type(systems) and having the ability to call the typechecker in-system, that is part-and-parcel of the interface-adherence-proofs I described in the other topic. Many typesystems are undecidable, though, so I do not think that we could guarantee in general that an indexer will return all valid implementations of an interface – it could only return all known implementations of an interface for which we have a valid proof.

What do you mean by this? Is there a clean definition of the MOP in mathematically precise language (in Lisp itself is fine actually if there’s a simple one)?

Adding new methods in the sense of imperative computation doesn’t even require an explicit state change with objects-as-resources, since objects-as-resources need not fix a set of methods. Changing the state transition invariant associated with an object-as-resource does require a state change, but is certainly possible. We might want to think about how objects might want (or not want) to limit how their state transition invariant (resource logic) can be changed.

One thing to keep in mind here is that we are dealing with permissioning in a distributed, multi-user shared-state system where the users do not, in general, trust one another – it is very, very different from running a local REPL on your personal machine. Whatever kinds of updates we allow will always entail trust assumptions, and these must be explicit.

11

Well that depends on the query you sent to the databroker, you can always start oring and anding clauses for primitives you know are fine or what someone has decided is valid. Many queries are fine just knowing who responds to some kind of messages, stricter guarantees about behaviour would be nice if possible but in practice there are many ways around it.

Btw why do we keep using the word indexer? We aren’t indexing into a table does it offer any semantic advantage over the word “query” which I’d argue is much more accurate to what is actually happening here. What is the role you imagine for these people, what are the requirements? Is it compute power? Is it storage power?

Namely I think a service could be that some courts or some people’s chambers will offer a databrokerage service, where you can send in queries to them, they can tell you what queries they have already precomputed, but you can send in a lambda to search particular datasets you may not have locally for whatever reason. This would look the same as querying for information locally but just on a non local entity (much like how I search google.

A lot of the design that is important to copy over is how much consideration is had for designs that are flexible and allow work at multiple layers. In essence it is a system and is a good system at that.

I’m not sure of a small mathematical definition, as it’s a system it’s many components coming together to make a whole, however here might be some leads:

https://franz.com/support/documentation/mop/concepts.html

If you are wanting to define your own the book “The art of the metaobject protocol” shows you how to define it precisely in a bunch of code.

Well new methods, mean the dispatch function of the object as a resource needs to have an updated reference so it can accept the new message handed to it, I.E. it now accepts a new message, so yes the state transition invariant does change, as it needs to contend with a new kind of message called on it. This might be handled by a mutable pointer (I.E. we lookup methods to what the pointer tells us so the data there changes), so it doesn’t require the object to be changed but something must be changing there to have the object react to a message properly.

Me loading stuff on domains under my control is fine, if you want to update something outside then send the changes, there is no guarantee that the changes may be accepted, and different courts will decide how they wish to proceed. We should work on permissioning systems in general with many levels of control that are overridable. It’s why I mentioned the ethos of the MOP design as if we take the same style of design we can bake in many layers that can be tweaked by users for niche use cases to allow for ultimate control for how objects we define to update.

1 Like
12

Is solver time any different from chamber time?

Rather to put it, can we describe the time without making reference to a particular actor on the system (solver computation likely happens in a court or chamber, making the entity not unique in it’s time). Further I can argue that everyone in their own Chambers is a solver, let me post a solution I had my chambers solve.

I want an X and an N such that X > 5 and X and N are used in factorial(X,N), where N is the solution, give me the first 5 solutions!

X = 6,
N = 720 ;
X = 7,
N = 5040 ;
X = 8,
N = 40320 ;
X = 9,
N = 362880 ;
X = 10,
N = 3628800 .

Further I can make another counter example… Let us think in terms of big data. Let’s say we are getting so much data over the network that the user can’t process it all, so instead of sending the subscription to their local chambers, they send it to a local court they control. Now the data is being fed into 20 of their machines in a cluster using a consensus algorithm to pick the machine with the lowest load for the data processing and finally given to the user. Here we have unfinished computation happening across the network across many machines but no intent is used, in fact we used many machines to fill some data but it certainly isn’t a solver!

The problem is, is a cluster of 5 of my computers a foreign domain? I own it so it’d be a local domain to me! With the court and chambers it isolates the computing power and environment and lets us use adjectives to be more specific (This is a court owned by identity X, this is a court where identity X is not the judge!). I’m fine moving away for discussions here but I’d like some better terms first.

1 Like
13

I think we’re roughly aligned here. I’m happy to replace “indexer” with “distributed query system” or something. I’d note however that “indexer” is not an artifact of weird crypto terminological shenanigans, the term just comes from “indexing” in databases, which is indeed what we’re talking about here (the difference between “indexing into a table” and “indexing into resources / historical state” is not important, “a table” isn’t anything specific). Maybe “indexer” as a distinct role is slightly crypto-specific, but the concept itself is not.

Thanks, maybe we can add selections here to a reading group session if you think it’d be useful.

My point is that we can separate – not only in the resource model but also in our object system – what state transitions an object accepts (governed by the resource logic of the associated resource) and how to compute a particular state transition (which doesn’t need to be part of the object’s definition itself at all). We may want the ability to both change the state transition invariant and the ability to define new methods of computing new states, but these are distinct (or at least they can be in our model), which is not the case for an imperative object-system (which interleaves state transition invariants and computation of new states in “message-handlers” without a clear delineation).

I agree with this, I think we might want something like “speculative future histories” of logical state domains (courts, in your lingo), where you can run different transactions locally, then perhaps choose whether or not to actually “apply” them. Note that the results of applying them may not always be the results which you obtained locally, since this is a multi-user system.

14

From the perspective of a particular intent, which originates from a source chambers (source user/machine), I’d say that it is, yes. There’s no objective categorization of “times” as far as I can see, there are only different perspectives, and how to classify events depends on what perspective you are interested in.

We can describe the time this way but I’m not sure why we would, because that wouldn’t describe any actual interaction pattern that we care about. If this is the kind of time you want to talk about, can you clarify what you’re looking for this set of “times of the system” to do?

Yes, that’s fine, “solver” is just whoever solves, it’s not a specifically permissioned role.

What is this a counterexample to? As mentioned above, I’m not arguing that a transaction-centric perspective is the only way to talk about time in the system, I just used it as example to help elicit what you were looking for. From this response I garner that you’re looking for something … more general? That sounds potentially useful, but I’m still not sure what you have in mind exactly. Do you want to talk about an arbitrary causal DAG of causally related messages or something like that?

As far as I understood per our discussion before, “chambers” meant literally the domain of a physical machine, so (if we replace the word “chambers” with “local domain”) anything else would be a “foreign domain”, including 5 of my computers, yes. If we want terms to describe zones of trust or control that’s a different separation (these are related to but not the same as logical state domains).