I'm on board with the idea about standardization/unicorns, but I think there is something a little backward in here. Regarding "wonky" PFF game grades, you bring up possibilities like a QB's receivers being wide open all day, preventing the QB from accumulating high-scoring plays in the PFF system. That's... a strength of the PFF approach! If a game happens where a QB has few chances to demonstrate their value, a decent single-game metric would tend to assign them grades right around average. That's an appropriate response to what is effectively a smaller-than-usual sample of meaningful plays for the QB.
That's not a defense of any particulars of how PFF assigns grades, but - even an *ideal* game metric should have this quality, if that's what the underlying game-to-game noise is like. Mahomes might not always receive the best or even one of the best grades every week partly because he does not always play the best (not every Steph Curry game is evidence pointing toward him being the greatest three-point shooter ever) and partly because he might not have had the opportunity (an outfielder could get a league-average "PFF defensive rating" because, while they recorded 12 outs, all were easy pop flies) - and that wouldn't be a bug, or evidence of bias.
Not that anyone will respond to the email side, but this was a great article.
Thank you!! That’s very much appreciated.
Every player I like is a unicorn and every player I dislike is gaming the system
I'm on board with the idea about standardization/unicorns, but I think there is something a little backward in here. Regarding "wonky" PFF game grades, you bring up possibilities like a QB's receivers being wide open all day, preventing the QB from accumulating high-scoring plays in the PFF system. That's... a strength of the PFF approach! If a game happens where a QB has few chances to demonstrate their value, a decent single-game metric would tend to assign them grades right around average. That's an appropriate response to what is effectively a smaller-than-usual sample of meaningful plays for the QB.
That's not a defense of any particulars of how PFF assigns grades, but - even an *ideal* game metric should have this quality, if that's what the underlying game-to-game noise is like. Mahomes might not always receive the best or even one of the best grades every week partly because he does not always play the best (not every Steph Curry game is evidence pointing toward him being the greatest three-point shooter ever) and partly because he might not have had the opportunity (an outfielder could get a league-average "PFF defensive rating" because, while they recorded 12 outs, all were easy pop flies) - and that wouldn't be a bug, or evidence of bias.