For those minds greater than mine, how does one go about measuring mev boost relay performance? It is the amount of MEV? How quickly it is able to order the blocks? Network proximity?
Just want to start a dialogue here and encourage the programmers / experts to discuss / share their thoughts.
I would say a few primary things to start off with
Average MEV Extracted per block over th last N days
Uptime defined in something like 9s to measure the expected downtime per month – This should be downtime measured in missed blocks from a validator propsective
Percentage of payments recieved from relays for each block proposed
(1) Is fairly straight forward. Pick the relay the meets the requirements with the highest block MEV average return
(2) I think this should be 99.9% if measured in missed blocks, as in the relay didn’t get the bundle to the validator so the block was missed measured oer 30 days. Or 99.95% if measured in pure connectability (The relay is connectable and sending/recieving traffic). 3 nins for blocks because things happen, 3.5 nines for connectivity as that still represents ~21 min of downtime a month which is sufficient to allow for patching an upgrades
(3) This should be 99.999% over 30 days. things happen, bugs happen, sometimes something goes wrong but a relay should almost always return fees to the fee-address a validator set
Those are my rought takes. Network proximity I am not sure super matters, though something more abstract may be where the endpoints are hosted (region of the world, and backing infra (AWS, GCP, etc) much like the conerns around client/node provider diversity though this is less “Scorable” than the above items.
I am going to drop this link here as well – This is a design doc for a MEV-Boost monitori being written by the same person. There are lots of ideas in here in regards to monitoring relay/auction/builder performance
Also the github which if anyone knows go would likely LOVE from PRs help – relay monitor design doc - HackMD