One of the objections I've heard to libertarianism is that it takes too much brain power to understand.
Only because it goes against government-controlled education that almost everyone has been subjected to since early childhood. There is a certain natural individualism in man that, if left unmolested, would lead a lot more people to strive to raise themselves above the mob, to take responsibility for their own lives, to put barriers on them on their own terms, to demand individual dignity and, by extension, individual freedom.
Smarts have nothing to do with it. The libertarian philosophy requires, more than anything else, a certain inner stubbornness, a sense that life is too precious, too important to waste on keeping up with the herd. Your ideal libertarian third-grader isn't the straight-A's student, it's the C student who never misses an opportunity to cut school and to seize the day on one's own terms, and you just know that thirty years from now the straight-A's student will be calling him "boss"!