Joe Nocera has a 10 page piece in NYTimes on a particular risk model popular in finance firms (don't forget to check out the accompanying images on every page!), devoting a significant chunk of it to criticisms of that model from a host of players, including Nassim Nicholas Taleb, the author of Black Swan.
While the entire article is both informative and interesting, I want to highlight here a paragraph that bears a striking resemblance to what Nature argued in the context of assessing research quality using citation metrics alone. First, the paragraph from Nocera:
Two years ago, VaR worked for Goldman Sachs the way it once worked for Dennis Weatherstone — it gave the firm a signal that allowed it to make a judgment about risk. It wasn’t the only signal, but it helped. It wasn’t just the math that helped Goldman sidestep the early decline of mortgage-backed instruments. But it wasn’t just judgment either. It was both. The problem on Wall Street at the end of the housing bubble is that all judgment was cast aside. The math alone was never going to be enough.
But taken alone, publication citations have repeatedly been shown to be a poor measure of research quality. [... long snip ...] Expert review is far from a problem-free method of assessment, but policy-makers have no option but to recognize its indispensable and central role.