It seems to me that the labels applied to libertarian or anarchist like left, right, capitalist, etc. have to do with what people predict a world without a state would be like. I heard a good explanation from another guy who refused to apply labels. He said he just wants the state gone and described himself as agnostic with respect to what a world without an aggressive state would be like.
It's sort of like that but the reasonings and motivations differ. The state is a symptom of the problem I want to solve, not the problem itself. I want to eliminate perpetual power classes (i.e. Rich vs everyone else, bosses vs employees, cops vs everyone else, politicians vs everyone else). Inequality of a power of 10 I can understand ($10,000 vs $100,000/yr incomes), though smaller is better, anything bigger than that should immediately invite questioning as to why and how, not explanations about how some people are just plain 10 times better as people than others as I get from ancaps sometimes. I don't just predict something different, I also predict the way it will happen is different, the values of the opposing sides will be different. A lot of the values of anarchocapitalists are destructive to the goals of other anarchists. Defending power disparity that is cause and consequence of the state doesn't further libertarianism at all, and sets it back drastically. Socialists at least acknowledge the non-state aspects of the problem, and while many use the state to try to solve these problems out of seeing no other means available to them and an opportunity available to take control of capitalism's biggest tool, some see the state as part of the problem as well.