...but maybe there is a certain connotation I can't get over with selfish-ness.
Yes indeed, which you expressed here:
I really don't think that libertarians are selfish in the sense that it implies a lack of concern for others.
Being selfish does not specifically say anything about concern for others (or lack thereof)... but this is probably the way most people think about it. As a result, being selfish is often mistaken for
egocentrism, which is what you truly have an objection to if I'm not mistaken.
I personally believe that few people are truly (consistently) egocentric.. and I certainly don't believe this to be more prevalent within libertarian circles by any stretch. And neither do you (wow, so we are yet again in agreement here
).
I think the easiest way to put it when explaining this to non libertarians is to explain that we are "
against coercive redistribution of wealth but favorable to voluntary charity". That pretty much sums it up and avoids the charge of being "
selfish" (which is really intended to mean "
egocentric").
This is probably easier than attempting to explain the difference between egocentrism, selfishness and rational self interest... and then proving that selfishness is not something they can escape (people don't want to hear that and it'll turn them off - point in case, even you are not comfortable with it despite understanding it).