Jeff Friedman, editor of Critical Review. . . noted this problem in his compelling essay several years ago entitled "What's Wrong With Libertarianism?" In a nutshell, he observed that libertarians make a moral case for their philosophy (i.e., it is wrong for government to push people around) which they are unwilling to push to the extreme, namely, to the point where they argue that their system of governance would be best even if one could prove that people would be materially better off in some system of stronger government. At that point they switch to what we call consequentialism, and argue that not only is the libertarian system more just by virtue of its minimal coercion, but that it is also produces more prosperity for its citizens.
The problem, Friedman rightly observed, is that we have shown no such thing. To be sure, economists have done a good job of demonstrating that heavy government management of the economy reduces economic growth by destroying property rights and incentives. Nobody has shown, however, that a libertarian system of nearly non-existent government would make people better off. We have anecdotes, we have some notion that we can extrapolate from partial analyses of more ostensibly libertarian times at the turn of the century, and we have the rational profit-maximizer of economics -- but we do not have a methodologically rigorous study that can even explain, for example, the inescapable correlation between sizable government (say, 20-40% of gross domestic product) and sustained economic growth.
Okay, just because the theory isn't perfect, doesn't mean that it's wrong. I will have to think about this some more, but I agree that libertarianism has a strong and compelling moral argument. Now, we just need to refine the details.