I've always known I didn't fit in with the Demopulican crowd. I was raised in an area where just about everybody knew the word "libertarian" but it was a party, and the non-aggression principle wasn't ever mentioned. I recall thinking "Those people suck" when I read the LP's stance on "more responsibility and harsher penalties for crime" because the idea of "abolishing victimless crimes" wasn't presented first.
About a two years ago, a friend and I ended up discussing open source software being sold. The short version is that it eventually had me question "is democracy a good thing" which sort of lit the fire under me. I was already thinking that "people telling other people what to do sucks" but I didn't really grasp just how ingrained this idea was. I went from "liberal" to "Ron Paul Republican" pretty quickly but it felt wrong. Less wrong than "borderline communist" but still not right. There was no principle behind some things.
I read The Market For Liberty and it really got me to re-evaluate things. The NAP made practical sense as well as moral sense, and I cast off my belief that government was needed.
Incidentally, my friend also eventually became a Voluntaryist himself as a result of that discussion and the subsequent thinking.