The State of our Stats

Friends:

Dave, Jamie and I have spent considerable time — especially Dave — the last two days examining our stats collection and computing system.  We have even corresponded with Baseball Reference, who gave us a helpful but incomplete response.  Here are our most important findings:

• MLB routinely makes errors in its initial daily stats reports used by Baseball Reference, our source for daily stats.  We always knew errors were possible, but we have discovered they are considerably more frequent than we had ever suspected.
• BR doesn’t catch those errors in the data they supply us.  Apparently they do not correct errors themselves.  Instead, MLB will catch and correct errors, and send those corrections along in separate reports to BR.  
• We do not necessarily get those corrections under our current system.  We just take each new daily report and lay it on top of previous days to get our running totals each month.   
• The errors are generally small, and definitely random — at least as to hitters. And there may be a tendency for GIDP to go under-reported.  Their effect on the standings is small, but possibly could come into play in very close races.
• Under our old Baseball Prospectus system these errors were occurring, but since we were using month-to-date stats, we would get the corrections, generally without noticing anything.  We were still vulnerable to any errors on the last day of the month, but I don’t recall ever noticing one. 
• Under our current system we are vulnerable to every day’s errors, and have been since we adopted it.                                                                                                                                                                                                  

The upshot is this:  We have always had errors in our data.    We do not recommend going back to the BP system because we had to do wonky, highly unrealistic things to be able to modify our allocations mid-month — and they often had retroactive effect. The current system delivers more realistic results, even though we are all vulnerable to random errors. 

We do not believe there is any way to avoid the daily errors. We are still working on whether we can, on the last day of the month, run an extra error-correction “day” to undo all the month’s accumulated  errors.  We want to avoid having to hand-check all 300+ EFL players for errors each month.  That would be several hours of work.  Even if we can avoid that problem, it won’t be perfect because the errors might have occurred under different player allocations than what prevails at the end of the month.  And any errors in the last day of the month’s stats would be uncorrected (which has always been the case, it turns out, since the beginning of the league). 

Dave did the most to discover this. Using the Pears as guinea pigs, he checked every player’s accumulated August stats, according to BR and BP, against our EFL totals.  The EFL totals represent the sum of each daily BR report, without error correction. The monthly accumulated stats include any error corrections from MLB.  Here is what he found for Peshastin’s hitters this month, as of yesterday: