Evaluating Chiefs Football: The Dilemma
The first quarter of the Kansas City Chiefs season is over and the reaction many of us fans have had in response to the team’s airy-fairy loss to the Steelers has lead to rampant rants and relentless overall evaluations of the team. While I will always enjoy the process of evaluation, it has its inherent problems and challenges. Enough so, that we find ourselves at odds with other Chiefs fans over what really ails the team. Enough so, that we may find ourselves at odds… with ourselves.
Consider what general manager John Dorsey has done with Eric Berry: when he decided to not give in to safety Eric Berry’s contract demands before the season, it was the first time he’d not given into one of his “star” player’s demands. John Dorsey is the phrase that pays when it comes to writing contracts for his stars. Dorsey has handed out what might be seen as overly-generous contracts to several other players in recent years… so why not Eric Berry?
On one hand, Dorsey has done an admirable job of building the team’s roster but on the other hand, he’s overpaid for some players and there are those who believe that the roster has too many older and overpaid, players.
There is a subjectivity that goes into running any organization, football or otherwise.
Here’s an example of one writer’s subjective account of a play. One I’m not sure I agree with.
You cannot bail out of this kind of protection. Also, there were TWO receivers open down the field :(. pic.twitter.com/m3qsfd9wCe
— Seth Keysor (@RealMNchiefsfan) October 6, 2016
I don’t know what the normally reliable Seth Keysor is seeing here but, it looks to me like LT Eric Fisher is being easily driven right up into Alex Smith’s sternum. This a good example of how we all see things differently. I can understand if someone else sees this play differently… I just don’t see it that way myself.
So, is there any evaluation tool that can be relied upon 100%… or even 90% or 80%… of the time? I’ve heard some fans tout their preference for, and high-degree of confidence in, Pro Football Focus (PPF) and their rating system. I’ve used their services and am currently a paid member of one of their less costly systems.
Personal Note: If I may take a second to share a personal note about myself before proceeding along this query. I flunked first grade and was repeatedly told I was stupid during elementary school and that I couldn’t read early in my so-called “educational” career. The fact that I had (and still have) dyslexia didn’t help my situation, especially when it comes to test taking. What I soon learned about myself was that I enjoyed writing. I wrote poems and songs by the hundreds. Educationally, that came in handy as I began to apply that creativity to writing papers for my classes and would always choose to write a paper vs. taking a test any day of the week. Now, to bring this little saga full circle, my Master’s Degree is in, none other than, Reading Instruction.
The reason for sharing all this, is to say: I learned many moons ago that I could write a paper about almost any topic,… that could support any point of view… then doing research and finding facts to support that point of view.
This is in no way to disparage the work that PPF and its researchers do, but to make an important point about when taking any of their “figures” out of context. In education, a principal (or an assistant) will come in to do an annual evaluation of a teacher, as I’m sure that many of your bosses do a yearly evaluation of your work. When they do this evaluation they are essentially in the classroom for 50 minutes maximum. They come in twice per year with other stop-by (mini-evaluations) during the year. The point is, they are watching — at the most — 2 hours out of 1800 to 2000 hours of teaching per year and we call that a “snapshot” or a “dipstick” of the total teaching process. And… there’s just no way to grasp the full spectrum of what is going on with an employee with that kind of minimalists’ view.
So, when one of us quotes a stat referring to one game that a LT may have played — good or bad — it is nearly worthless when attempting to “grasp the full spectrum of what is going” on with that player. This is also true to an extent when looking at a whole season. You can’t judge a team’s whole progression based on one season… or part thereof.
There are also other issues related to the validity (being factually sound) of the data that PFF gathers. “Determiners” that may affect the final data outcomes include:
- How good, or bad, was the player they were going up against?
- What did the evaluator have for breakfast?
- Was the evaluator looking for results?
- What were the total number of observations of the evaluator?
A thousand things can affect an evaluator. We’re only human. So, here are some legitimate reasons a game day evaluation may be tainted:
How good, or bad, was the player they were going up against?
If Eric Fisher has looked like Zeus this season (not saying he has or hasn’t), it “could” be that he’s faced players who were not in the top echelon at their respective positions (not saying they are either). The point is, you can’t create a numerical representation to sum up the complete performance of a player based on who they matched up against in one game… or for that matter, one season.
What did the evaluator have for breakfast?
If you’ve ever experienced having an off day at work, you understand completely how an evaluator can be affected by outside forces which have nothing to do with the game or player they are evaluating. That’s why articles and posts written during the 24 to 48 period following a big loss often sound like knee-jerk reactions instead of thoughtful appraisals and cogent opinions.
Was the evaluator looking for results?
If an evaluator is noticing a pattern during the process of evaluation, and is influenced by that and then continues to look for patterns which support his earlier assumptions… that evaluation is not valid. By example: this is Eric Fisher’s 4th season and his progress has never been on a straight line upwards. No player’s progression is. So, it’s important to understand that every single play is a “stand alone” play and an evaluator should never be looking for patterns “during” the evaluation.
What were the total number of observations of the evaluator?
There are basically two kinds of research, quantitative and qualitative. Some research is both, and those are the best kinds. About 20 years ago I visited the elementary school I grew up attending and they had just finished being part of a six-university, 8-public school qualitative/quantitative study. Over 128,000 people participated in the end and that made the results reliable and valid to a great degree. The assumptions that professors would draw from that information is what comes into question. While PFF has years of so-called research to back their figures, there’s no way of knowing about the dependability of those who are interpreting the data. Then there’s another thing to consider: is there is any consistency from evaluator to evaluator?
Not to bore you with what the research says but, the Oxford Scholarly Research Reviews points out the fallacies that can be found in quantitative methodology:
“The six major fallacies include Contextual Variable Fallacies, Measurement Error Fallacies, Missing Data Fallacies, Significance Testing Fallacies, Statistical Power Fallacies, and Factor Analysis Fallacies.”
What this means for us who love and cheer for the Kansas City Chiefs is that we can trust our eyeballs all we want and believe what we see… even though we may not like it right now. What I can say with “factual” certainty about the 2016 Chiefs is… that they have an increasing number of questions, many more than they did on opening day.
Now that, is not a good sign… (my assumption)…
… and yet I will concede that this assumption about that fact… is entirely open to question.
Now, I’ll Lateral Pass the ball to you!
