Remote subscriptions

Protocol Design
1

image
image1929×868 179 KB

Here are some considerations on remote subscriptions. Within a trusted scope (i.e., within a particular node) the event broker works for subscriptions by filtering based on subscriber-provided filter functions.

One approach might be to extend this across the network. However, it doesn’t extend across trust domains because a filter function is an Elixir function, and it could just do arbitrary things besides filter, so this is to be avoided.

The better approach is a filter DSL, which I’ve mentioned before. The simplest filter DSL can merely be an Elixir pattern as a string, which is turned into a filter function on the remote side; patterns are powerful and most filtering can just be done in function heads (i.e., most filters look like def filter(pattern) do true end; def filter(_) do false end). In theory this could be extended to a more powerful language if it proves necessary. Patterns have no side-effects, they just match or don’t (and bind variables, but this part isn’t relevant here, it can just be an invocation of the predicate match?/2).

It’s worth considering the possibility of timeouts on subscriptions as well; some subscriptions may be one-shot (you expect to receive exactly one event, e.g. “the status of this specific transaction”) and not requiring the subscriber to clean up is useful ergonomics.

2 Likes
2

Minor note: one way to make remote-code-injection less-bad is gas metering (everyone’s favourite topic, I know, but it could come in handy here) – with some reputation measurement between nodes (even ad hoc), incentive-wise I bet we could get away with running arbitrary filter functions as long as they are properly sandboxed and gas-limited. I’m not sure if that’s more or less work than developing a small filter DSL, but I thought it might be worth considering.

Also cc @tg-x as we have some parallel questions in the P2P pub/sub design.

3

Trivial counterexample:

def filter(_) do
  :init.stop()
  true # why not?
end
4

Why is that a counterexample? If :init.stop() is exposed, that seems like improper sandboxing, which I specifically mentioned as a requirement.

5

You can’t will “sandboxing” into existence just by saying it, though. Whatever it means, it has to become some actual implementation, e.g., creating filter functions from a limited filter DSL.

1 Like