You'll often see Sabermetric player rankings based on how players perform versus how a player's likely replacement would perform.  Baseball Prospectus in particular uses Value over Replacement Player (VORP), Wins Above Replacement Player (WARP), and Batting Runs Above Replacement (BRAR).  The Hardball Times modified Bill James' Win Shares into Win Shares Above Bench (WSAB).  All of these metrics run under the premise that if a star player is injured, traded, or lost to free agency, that his replacement would not post league average numbers, as often this replacement player is simply a callup from Triple-A.

But a replacement level player can mean several different things.  Take Phillies first basemen as a perfect example.  In 2005, Ryan Howard replaced the injured Jim Thome by slugging .567 over 88 games.  The previous year, Howard hit 48 combined homers over three levels of play.  And you all know what Howard did last year.  So in 2004, when BP calculated Thome as having 7.1 Wins Above Replacement Player, shouldn't it be closer to 0?  Although Thome banged out 42 homers and slugged nearly .600 that year, it is reasonable to assume that the 24-year old Howard would have put up nearly comparable numbers.

In a cycle of George Lucas proportions, Ryan Howard went down with an injury this year.  But this time there was no Young Apprentice to cover for Master Howard.  He had been "replaced" by a platoon of Wes Helms and Greg Dobbs.  Worse yet, when Wes Helms made a start at first base, it usually meant that Abraham Nunez was manning the hot corner.   What kind of a Value Over Replacement Player does Ryan Howard have when compared to the .600 OPS of Wes Helms or the .650 OPS of Abraham Nunez?  More than a Jim Thome has when a Ryan Howard is waiting in the wings, that is for sure, even though the raw numbers of Thome in 2004 are comparable to those of Ryan Howard last year.

Payroll can also impact the way we think about the replacement level of players.  For a team like the Yankees, Roger Clemens is essentially a replacement for various injuries to their starting rotation.  For the Nationals, Jason Bergmann's trip to the DL means starts for the likes of Levale Speigner.  Thinking about replacement-level players in this fashion, you could easily argue that Matt Chico has a Value Over Replacement Player equal to Andy Pettitte.

Also, when rating players, most analysts would agree that players should be awarded extra consideration for playing in a highly competitive league.  Certainly, few would penalize a player for it.  But if you think about replacement-level players in these terms, that's exactly what happens.  A fill-in player today is generally someone with years of professional experience and prime physical conditioning.  The final man on your roster in 1910 was most likely either an illiterate 18-year old or a pot-bellied old man.  Were star players from 1910 really more valuable than  

Of course, there isn't a replacement-level metric out there that takes these sorts of factors into account.  They all assume a replacement level player to be a certain percentage worse than a league average player, and apply that arbitrary weight to every player in a given league.

As a result, you have very misleading statistics based on made up figures.  We know that Matt Chico really isn't as valuable as Andy Pettitte is, but we also know that the replacement level for Yankees players is generally quite high.  Either way you look at it - replacement level as concept or as pure math - you're not going to get the whole story.  It is generally much better, in my opinion, to compare a player to the league average when ranking ballplayers.

There are one or two exceptions, however.  It's no secret that I favor Batting Runs, which are essentially a player's Linear Weights value over a league average player, when evaluating players from different eras.  The biggest problem in my reliance on Batting Runs occurs in my evaluations of catchers and shortstops.  Players who man these positions often have negative values for batting runs while still contributing, since Batting Runs compares a player to the league average rather than the league average at a certain position.  In other words, a shortstop may hit below the league's average, but still be above average offensively for a shortstop. 

The best example for how this can muddle things is the case of Joe tinker versus Ozzie Smith.  I rate the two shortstops about the same defensively (an incendiary comment that will no doubt require a separate article soon, I know).  Offensively, Joe Tinker has -68 Batting Runs while the Wizard amassed -143.  A cursory appreciation of these statistics leads one to interpret that Ozzie was twice as detrimental to his team with the stick than was Tinker.

Examined more closely, we realize that both of these players were actually boons offensively.  Tinker was arguably the best offensive shortstop of the dead ball era not named Honus Wagner, and Ozzie Smith was probably in the upper tier of offensive shortstops for his era, at least after his first six disastrous seasons.  The important thing to note is that Smith had a longer career by nearly 30%, and therefore continued to rack up more negative Batting Runs as he continued to contribute on offense.  Where a player with a longer career usually receives a boost to his Batting Runs totals, the Ozeroo took a hit because of his longevity.

Baseball Prospectus' Batting Runs Above Replacement Player (which is actually derived from EQA and not Linear Weights) shows Smith to have 363 BRAR while Tinker comes in at 249.  This isn't a definitive answer, but it's a lot closer than what we saw with Batting Runs. Add Smith's 580 stolen bases swiped at a near-80% clip, and it becomes clearer why the Wizard was in fact more valuable on offense than the Sad Lexicologist was.

Hopefully, this essay elucidated some of my reasoning for favoring Batting Runs over Sabermetrical measures involving Value Over Replacement Player, while also exposing some of the weaknesses of Batting Runs as well.  Batting Runs - not a perfect measure of a player's hitting ability, but almost certainly the best that we've got.