S has agreed (via some prior mechanism) to pursue a goal G (which we can understand to be defined in a way that is sufficiently clear to governance mechanism participants, e.g. written language).
Periodically, governance participants (P_0, P_1, etc.) vote on what actions they think should be taken by S, and their votes are combined by some distribution governance mechanism in order to compute what action S actually takes.
Periodically, governance participants review historical actions taken by S, and assess how effective (or not) they were in furthering G.
Define a governance efficacy feedback mechanismM as a mechanism which feeds-back the assessments collected in (4) into weighting the votes of governance participants in the future (depending, essentially, on how effective the actions they voted for in (3) were in furthering G).
The basic rationale for such a mechanism is that – assuming that the goal of S is actually G – it would allow for G to be more effectively pursued, by feeding-back information about which participants best select actions to further G into the selection of the participants for future voting rounds (assuming that there is some correlation between past and future voting efficacy).
For example, suppose that we use median stream voting as the distribution governance mechanism, with two periods: a voting period, and an assessment period, where:
Each voting period (e.g. a quarter), participants vote on stream amounts, and
After the assessment period (e.g. a year) has passed after a given vote, participants assess how effective each stream was at furthering the goal, per unit tokens.
In accordance with these assessments, voters receive bonuses or penalties based on how they voted relative to the assessment – e.g. a voter who voted in the ideal way receives the maximum bonus, and a voter who voted for streams which were later rated to be maximally counterproductive receives the maximum penalty.
We immediately run into some underdetermined parameters and potential problems:
Rating per unit tokens does not clearly allow us to distinguish between two voters who voted for the identical relative distribution, but in different absolute amounts (so, in essence, voting for more issuance but towards the same sub-goals). We could change the distribution governance mechanism to separate out voting for (a) relative and (b) absolute distribution amounts, which would potentially make this easier.
What unit scale is this assessment conducted in? A simple one is to say that assessments range over (-1, 1) and are conducted on a purely relative basis – e.g. whatever you rate 1 is taken to be the most productive method, whatever you rate -1 is taken to be the most counterproductive method, a rating of 0 means “no effect”, and all other ratings are expected to be chosen relative to these.
Would voters not just rate their own choices as correct, regardless of any actual assessment? This is possible, but the set of participants may be changing anyways, which would render such attacks more difficult. Potentially a commit-reveal mechanism could be used for the assessment itself (to reward agreement).
As a note, this mechanism also makes the assumption that the effects of funding streams are (a) continuous and (b) relatively independent, and potentially more assumptions that I haven’t teased out here yet.
Much to be figured out. Nevertheless, I think that this class of mechanisms is worth investigating.
Yes, we should take all available information as input if we want to use the voting distributions to evaluate the quality of predictions regarding outcome improvement.
This seems like a good start to me.
As long as we have sybil resistance, this should not be too big of a problem, but collusion might be. We probably want something incentive compatible in the long run (if possible).
You mean to weigh by how much the assessment of any given individual diverges from the aggregate assessment? I can see how this could stifle cooperative and adversarial dissent alike, which is something I don’t think we want.
Some other assumptions we should make explicit:
Agents can meaningfully evaluate the outcomes of actions.
Conflated influence of action on outcomes is known and accounted for.
Sybil resistance holds (this might even be derived for later rounds, but it is necessary during bootstrapping).
In general, we probably want more robust evaluation of experiments than just relying on the votes. While this would run the risk of gaming the evaluation criteria, there will probably be other mechanisms learning the criterea, so there should be some dynamism which might lead to some resistance against goodharting.
We should also be careful to not over-infer general prediction quality over all types of actions from good predictions over a subclass, so we might want to separate feedback by domains.
Yes. Another question might be whether we even want to evaluate counterfactuals. For example, a voter could vote for a funding stream which does not receive funding, but ex post governance may determine that it should have received funding. Should we reward that voter?
Good point, that’s a big downside there. Might be better to rely on majority-honest assumptions and potential “social slashing” (out-of-band penalties) for obvious self-voting.
Yes, we do assume that, and I think it is reasonable to (otherwise there is little point trying to govern action in any way at all).
I don’t quite follow this one. Are you talking about multiple actions?
Yes, we assume initial Sybil-resistance (via token distribution) and that the evolution function preserves it.
I agree in general. My idea here at least is that those robust evaluation mechanisms could mostly live out of band (from the perspective of this mechanism), and the voters can use those evaluation mechanisms as important inputs to decide how to vote (in conjunction with their own assessments of how well those evaluation mechanisms are working).
Yes, agreed. This is very compatible with voters sourcing information from different evaluation mechanisms to decide how to vote.
This is remarkable. It’s a form of meritocratic or skin-in-the-game governance where if I understand correctly, voters are rewarded / penalized with additional/reduced voting power based on how effective G turns out to be. You basically hold people accountable for making bad predictions about policy effectiveness, while rewarding folks who make good predictions about policy effectiveness.
In the long-run those who make poor predictions have less say in the governance process. Those who make good predictions have more say.
This also creates an incentive for folks who want to participate in governance to take the role seriously, potentially shifting the game away from vibes-based tribalism (as seen in most voting systems on-chain).
The mechanism essentially turns governance into a long-term prediction game about policy effectiveness. (Ethereum could use something like this ASAP).
One interesting addition here could be to do the initial voting via a prediction market AMM.
Just as a pedantic note, G must be fixed for this system to work, at least for an assessment period – but I think you mean that voters will be rewarded/penalized based on how effective their votes were in furthering G (which is indeed the idea).
Yes, that is a potential benefit. There’s an implicit parameter here of “how much” the rewards / punishments are, which could potentially be configured to make effective governance voting – especially when the majority is later judged to have been wrong – extremely rewarding for participants.
I think that this is complementary, but does not need to be baked into the mechanism itself – the voters could simply use a prediction market as part of the information with which they determine how to vote (and if it is a functional prediction market, it might be a useful information source!).
What is the difference between voting in a system with Governance efficacy feedback mechanism and a prediction market?
You should have a strong filter/curation of what gets submitted to a vote for users not to gamify this mechanism.
Moreover, to simplify the measurement of efficacy, you could either objectify the efficacy metrics or use a third party oracle to judge the efficacy.
Overall, maybe it would be cleaner to have a prediction market that is not embedded with the governance of the protocol, to keep risks separated. I could also imagine the prediction market not to judge each vote, but to judge voters (whales or those to whom votes are delegated), to keep them accountable over time, as a further abstraction of that model.
Yes, @degregat and I discussed this (sadly not yet documented here), and also concluded that this efficacy feedback mechanism (or a slightly different one) really didn’t need to be a part of the “core mechanism”, since one could always use an issuance stream to fund an external efficacy feedback mechanism. I do think that there’s an important difference between predictive evaluation (as performed by a prediction market) and retroactive evaluation (as performed long after relevant events have already happened), and in general the latter should be much easier (although as soon as we cross the territory into counterfactual worlds the lines get blurry).