Alex Hoffman points me to this interview by Dylan Matthews of education researcher Thomas Kane, who at one point says,
Once you corrected for measurement error, a teacher’s score on their chosen videos and on their unchosen videos were correlated at 1. They were perfectly correlated.
Hoffman asks, “What do you think? Do you think that just maybe, perhaps, it’s possible we aught to consider, I’m just throwing out the possibility that it might be that the procedure for correcting measurement error might, you now, be a little too strong?”
I don’t know exactly what’s happening here, but it might…