Joe is part of the society, to which you're "giving back". Hypothetically, it's impossible to separate hypothetical Joe from hypothetical Jane who was recently born with severe disabilities and also didn't contribute jack shit, or a WW2 veteran, who is now unable to work. So you "give back" to all three.

Not hypothetically, and noting that you explicitly stated you didn't want to discuss factual reality, nonetheless: such hypothetical Joes are rare to nonexistent in the US post Clinton welfare reform.