From skillsets to the majors?
Every professional team-sport organization faces at least three problems surrounding talent: Assessment, Development and Projection.
From the moment a player is being considered for the draft to the moment he reaches the majors or gives up the dream of doing so, to the moment he is traded, let go or retires, and, if he is in any way memorable, for years to come thereafter, fans and others will debate his skillset, his performance, whether or not he reached his potential or fallen short.
It is natural but nevertheless misleading to conflate talent with skillsets. Talent is compositional. Skillsets are among its inputs, but so too are other less easily measurable characteristics, e.g effort, commitment, focus, love for the game, heart, capacity to overcome inevitable failures, and more.
The gap between skillsets and talent is matched by what may be an even larger one between talent and performance. It's a commonplace, and not just among Mets fans, that many a talented prospect or player has failed to meet expectations at the major league level, especially in New York.
Every organization, the Mets included, has a goal and a plan to achieve it. Draft talented players, develop their talent, prepare them to perform at the major league level, and elevate them to majors when needed or otherwise appropriate. And keep the process going, thus creating a home grown core of players under team control for a significant period of time.
In the best case scenario, the result is a relatively low cost core of players cycling through the majors over time, overlapping sufficiently with one another to ensure continuity. This core is periodically supplemented by trades and free agency as needed. The team is poised to be opportunistic, rather than being dogged by bouts of desperation.
The better a team is developing a steady stream of major league ready players, the better is their farm system and its reputation. The better the reputation, the more confident trading partners are in what they will be receiving in trades involving prospects and the more reliable a potential trading partner they become. If successful, relying on the assessment/development roadmap reduces the extent to which a team is forced to rely on costly free agents. An investment in assessment, development and projection is a win/win/win.
All good, but..... Statistics indicate that the road from identifying skillsets to cheering for a home-grown prospect making his debut at Citi Field is far from straight or smooth: that it is more beset by potholes, delays and other assorted obstacles. If it were an easy path to navigate everyone would have a successful program in place -- and, quite frankly, we might not be facing a potential lockdown as the result of ownership's desire to impose a salary cap!
It's more an obstacle course than a road
To get a sense of the magnitude of the issues surrounding assessment and development consider the following:
1. Roughly 600 players are drafted every year by MLB teams.
2. Of these, 40% of first round picks never make it to the majors. The percentage of success declines linearly for two more rounds and geometrically thereafter.
3. According to Baseball America, there were 584 drafted players on major league rosters opening day, 2024, or just under 20 players/team.
4. Baseball America also reports that in 2024, the Dodgers accounted for 29 of the drafted players on OD major league rosters, which led the majors, while the Mets accounted for 11, which put them above only the lowly Nationals who managed just 10.
Of the 29 players on OD rosters in 2024 drafted by the Dodgers, the majority were playing for other teams, which supports the point that the more successful the farm system, the better able a team is to build its major league team both through the draft and trades.
In contrast, it is an understatement to say that something has been amiss with the Mets approach to player assessment and development.
Draft gurus love measurable qualities and skillsets
Anyone who has watched NFL or NBA draft night has heard draft gurus wax poetic over this or that player's countless 'measurables,' (height, weight, wingspan, strength, vertical leap, sprint speed, etc) and skillsets (power, speed, motor, intelligence-- saved mostly for QBs and point guards).
I don't doubt that there are meaningful correlations between skillsets and performance, but I am skeptical about their heavy use in projecting talent, for two reasons: one drawn from experience, and another from work in the philosophy of science and social science. The latter first.
Predictions based on correlations rely on what I call 'tracking' explanations. All tracking explanations, including especially those that rely on correlations, are at best incomplete in the absence of a mechanism that explains why the chosen attribute tracks or is correlated with performance outcomes. Actually, I have argued for the stronger conclusion that the 'why' of the correlation, and that, after all, is the only thing that really matters -- the thing that provides the explanation and grounds the prediction.
If you remain unconvinced by my concerns, I would ask you to take a guess as to what the most projectable talent is in basketball. Which skill translates best from the amateur ranks to the pros. (Provide your answer in the comment section below.)
Nuance to one side, let's agree that a team has to invest to get the initial assessment stage right, which doesn't mean they have to be perfect, but it does mean (as I show in the next post) that you have to look in the right places for talent and know what to look for. Poor talent assessment creates negative externalities that impact the efficiency of the investment a team makes in development.
Development
When it comes to player development, there is no question that advances in technology have been hugely influential and by and large helpful. Calling on my own experience as a golfer and coach, the use of technology (much of it, like Trackman and Force Plates is employed in both baseball and golf) has had mixed results.
In golf, new technologies have helped already excellent players refine parts of their game, especially finesse wedge play and driving. But the technology of the driver itself has had far more of an impact. To be honest, advances in technology has been otherwise largely unhelpful in golf generally as players with very serviceable swings (and play well enough to enjoy the game) find themselves chasing numbers, like ball speed, without finding any measurable improvement in their handicaps.
There are many difference between golf and baseball, not just in the size and makeup of the ball and the tool designed to hit it. Golf, like tennis, is a one-person sport. Improve a person's skills and you improve their performance. Baseball is a team sport. Improve a player's skills, and you likely improve their contribution to the team. But you can also improve the contribution each makes to the team by developing team-awareness aspects of their performance. The phrases 'team defense' and 'team offense' are usually reserved for basketball, football and hockey because success in team performance in those sports relies playing as a 'unit' -- in concert with one another, each committed to their own role and to helping others successfully perform in theirs.
It is easy to think of baseball as a game played by nine people, each individually, and in that way very different from other team sports. This is misleading, however, as there are clearly team dependent talents that need to be developed, and these are require cognitive as well as performance talents, e.g. situational hitting, all aspects of defense, and more. The focus on individual skill development should never come at the expense of more holistic and cognitive skill development.
OK, I've got good players, I've developed them well, what's left to do?.
Fair question. Here's the answer. I've got to be able to use all the information I have gathered about the players I have and project how they are likely to perform at the major league level. In a way, projection is a matter of turning assessment into credible predictions. It is by far the most consequential judgment an organization will make.
The fact that on average fewer than 20 of the 26 players on a major league roster have been drafted by any major league organization implies that a lot of talent has fallen to the wayside. A lot of money is spent on assessment and development. In fact, considerably more is spent on players that don't make it to AAA, let alone the majors, than has been spent on those that do.
What should we glean from the numbers? The easy answer is that an enormous number of players are being drafted every year who shouldn't have been. Player behavior confirms this inference. A large number of drafted palyers realize within two years that they are not going to make the 'show' and leave organized ball to pursue other potential opportunities. Obviously many players are drafted who have literally no chance to play at the major league level. The organizations are spending money to allow those players to live their dreams a bit longer. I'm not prepared to say that is unwise to do so, or even that the money is being wasted, though others may disagree.
These facts ground more questions and suggest more avenues of inquiry than answers.
(1) It's a reasonable question to ask whether improving talent projection, and doing so earlier in a player's career, would lead to better initial talent assessment and development.
(2) It's also reasonable to ask whether reducing the number of player's drafted would do so.
(3) Instead of asking whether improving talent projection would have positive upstream effects on assessment and development, we might ask what changes in assessment and development would have positive downstream effects on talent projection, and why.
(4) Before we answer any of these questions, it makes sense to identify the costs of unreliable talent projections. What are the costs of uncertainty?
(5) What are the Mets doing to reduce uncertainty and create a reliable evidence based approach to judging the likelihood of their players being able to perform well at the next level.
I want to close this post by identifying the costs of uncertainty when it comes to talent projection. In the next post, I focus on what the Mets have done and other steps they may want to consider taking.
The costs of uncertainty
Recall from the discussion of risk in my previous post that well functioning modern organizations should manage risk (and there is risk aplenty in assessment, development and projection) so as to minimize the sum of the expected costs of the risks one faces and the costs of reducing them.
So what are the risks when it comes to making talent projections, and what are the steps that teams invariably take to mitigate those risks?
The risks are of two types: (1) You can mistakenly judge a person as ready to contribute at the next level in which case your evidence base has led to a false positive. (2) Or you can mistakenly judge a person not ready or never ready (or somewhere along the spectrum of not-readiness) in which case your evidence has created false negative judgment.
Given the formula I have just outlined, you want to invest in strategies that will optimally reduce the risk of both false negatives and positives. Easy enough. The trouble is figuring out what those strategies are.
The problem is exacerbated because the degree of confidence most teams have (based on past performance) is low relative to the confidence that they have in other ways of filling out the roster, i.e.trading prospects for major league players or signing free agents. That is definitely one way of reducing the risks created by a lack of rational confidence in their ability to project talent.
But it is hardly an optimal one. In fact it is hard to imagine a more costly approach to reducing risks.
Let's not even count the costs you have already spent on assessing and developing talent. That's now sunk costs. No matter which risk reduction strategy you follow, these are costs you have already taken on.
The key point is that you can't escape the consequences of your lack of confidence in talent projection by trying to solve for that problem by trading your prospects for proven major league talent. After all, you still have to figure out which talent you should be willing to give up in a trade? Worse, because you have done nothing to address the problem directly, you solve for that problem by signing the proven player to a longer term contract than you should. So you don't solve the problem and you don't avoid it. You just switch the costs you incur to risking bad trades and even worse contracts.
But why stop there.
The situation is different but no better with free agents. Typically, the costs of uncertainty are more urgent when the player in question plays a key position on the team: shortstop, starting pitcher, catcher, centerfielder. It is no surprise that proven talent at these positions is relatively scarce and thus very expensive. The cost of not addressing the problem directly leads to costly free agent signings in terms both of salary and years. Hardly optimal.
The problem of uncertainty that makes talent projection more guesswork than science has deleterious consequences almost too numerous to mention: sunk costs in the farm system, bad trades, long contracts, and the extremely high costs of free agents at key positions.
Why wouldn't you choose instead to see if you can reduce your uncertainty and increase your ability to create a reliable evidence-system of projecting talent? I think that is precisely what the Mets are doing. This post hopefully explains why doing so is rational. The next explores how they are attempting to do so.
Maybe there is even more they can do?
9 comments:
I am on board with this. I did a long and exhaustive series a few years back on the Mets ongoing poor drafting strategies. Simply they were drafting lots of players in higher rounds that had modest tools, Gavin Cecchini being but one of many examples. You have to draft players withsuperior tools. Some will fail, but someone will Excel.
They got very little out of their drafts. The way I look at it is some teams rely entirely on their drafts and on the international signings because they don’t have the money to throw around on free agents. I think if the Mets for many years, right after that, the deGrom wave of pitchers came up, only had to rely on drafting and the international signings, and no big free agent signings, they would’ve ended up with a record like the Chicago White Sox. When you draft that poorly, in order to not be the Chicago White Sox, you then have to spend big on free agents. Where are you running to risk on the big free agents is buying years where they would almost undoubtedly be in decline, but having to pay premium prices to get them. Thankfully, the drafting has been demonstrably better, the last handful of years, because you can’t go indefinitely with poor drafting and have a viable team. Unless you’re willing to spend $1 billion on salary, and all of the luxury charges that go with it. At which point your business model is a sure money loser.
I am a strong believer that the key factor in projection of talent is the mental state. In a game of frequent failure, only those who can maintain confidence and competitiveness through the adversity will be able to realize their potential.
Even after excelling (or dominating) at lower levels in an organization, a player eventually meets competition that can defeat them. This is where the seed of doubt is planted, and those that push through it have their confidence rewarded.
My guess on MBA success factor is consistent work ethic - very skilled players who continuously hone their craft.
Can wait to see your assessment of quantitative talent projections. This is the secret sauce for sustainable competitive teams.
There are so many variables & the human element intertwined - any team that can make some progress on successfully solving for this will have a huge advantage.
Let me cherry pick here
DEVELOPMENT
the Mets changes in this area under the Cohen administration have absolutely upped the game here.
Analysts, scouts, cross-checkers, basement needs, International commitment and development, and labs
This is not your Wilpon Mets
*BASEMENT NERDS
A couple of things, so far:
In NBA the desired “talent” is height and the skillset that translates is long distance shooting.
After the Mets traded for Kelly Johnson and Juan Uribe, watching a game I remember Ron Darling asking his co-announcers to guess the team with the most drafted players in MLB. The answer was the Mets with 40. That included John Gant, Matt Bowman, and other fringe relievers. I look at the Dodgers and do not see what all the hub is about. They brought up Andy Pages that hasn’t done much. Will Smith took forever to adjust to MLB standards and most of their lineup and roster are signed free agents. I don’t see the impressiveness of their organization. Conversely, there is this other team that has a mostly homegrown roster and had the best record in baseball last year. Also, there is this Blue Jays last year were stacked with home grown talent.
Is there any chance we are moving into the area of analysis paralysis?
Well Jules, you correctly state that very few drafted players make it to the majors, and it would have been better to not even draft the players that won’t make it. Here is the problem: you just signed Wandy Asigen. He needs a team to play on. If you refuse to draft players that you don’t think have any chance, how will you field a team? What happens as these players move up the levels and there is even less of them? If you do an Arizona Fall League style if just having the top players play on a handful of teams, will they get enough playing time? As it is, MLB wants to eliminate another layer of the minor leagues to….. cut costs. No kidding.
However, while reading this breakdown of measurement of skillset and talent, there doesn’t appear to be any mention of the one thing that is most important to a player’s development.
When we listen to our tv announcers talk the game to us, Hernandez continuously talks about playing the game between the ears. In fact, one of the most decorated and successful players in terms of championships, Yogi Berra, proclaimed the game to be 90% mental.Let me give you an example from our own team:
In the mid-2010’s, the Mets had a player that while I rooted for endlessly because I felt he wasn’t given a fair shake by Bozo the Manager, Wilmer Flores wasn’t able to get playing time even though the player in front of him of the depth chart, Ruben Tejada, wasn’t very successful. Only after Sandy Alderson finally cut Tejada did Flores play consistently. We never knew at the time that Fred Wilpon was shielding Collins. But, I also had a bone to pick with Flores. Every year I lamented that Flores didn’t spend enough (or any?) time on his throwing. I used to see his horrible throwing and ask why he doesn’t put a tire on first base while he stands on third base and make 100 throws a day, or even fifty, from all angles. He never gave me an impression that he took his defense seriously and I believe that if he had put in the time, he would have been a player with a better career.
Over the years, I heard of players that worked endlessly on their defense during the winter. Alonso did it once… congrats. He got better that one year. But Mike Schmidt was known as a butcher at third base, as was George Brett, but these guys eventually received gold gloves. Too, Tony Gwynn was a terrible outfielder with a weak arm. He worked on both and I recall the day he threw out a batter at first base, and he also eventually won a gold glove. Greatness isn’t an accident, and there must be a variable that measures “how badly a player wants it”. I truly believe Juan Soto will become a good defender because I’m willing to believe in his heart.
Post a Comment