I think Ziggy explained Geolibertarianism wrongly on the show. Without advocating it myself, here is a different explaination: To have a right to do anything, you have to have a right to exclusively control the physical location of that action. So all rights are property rights, or property rights are the only rights. So far, so Rothbard, OK? This means that if anybody has any rights, they must be self-owners, because any action we have a right to do must involve some use of at least ourselves, so we cannot have any rights without being self-owners.
However, just about any action we can perform also involves use of, for want of a better term, external resources - those not just of ourselves. Some external resources are the products of labour, of course. The traditional Lockean theory is that we can mix our labour with the land, and thereby get to own some of it. Now we can accept that if I mix my X with my Y, then I own the composite, XY, because I own the component parts. So, if I use my planks to make a bench, I mix my labour with the planks, and produce a bench. However, what if they are your planks, and I, without your permission, mix my labour with them to make a bench? Now, surely, I do not own the bench. The reason, some Geolibertarians have said, is because the planks were not mine: I own X, but not Y, so I don't get to own the composite XY. By analogy, then, mixing my labour with the land would not mean I get to own the resultant farm land or whatever, because whilst I owned the labour, nobody initially owned the land, so I didn't.
Meanwhile, because all rights we have imply rights to control ourselves and some parts of the external world, it is the case that if people are to have any rights at all, they must have a right to land. And rights being initially equal, this must be an initially equal right to land. Look at it this way: Suppose that all land was appropriated, but not by you. You could not possibly act without violating the rights of others, since you would have to be acting on their land. But if any action you perform violates the rights of others, that would mean any right you have would be a right to violate the rights of others. But there cannot possibly be a right to violate rights: This generates an incompossibility - your rights, and the rights of others whilst both being logically possible, but are not simultaneously possible.
So for everybody to have a compossible set of equal rights, everybody would have to have an equal right to land. But if people appropriate so much land themselves that there is not enough left for an equal share to others, then they violate the rights of those others, and those others are entitled to compensation, this compensation being an equal share of the value of the land.
In this way, the Land Value Tax is not aggression, it is defense: It reclaims for people compensation for the violation of their rights.