I'm assuming there's tracking on the invites. So a recursive kick on X and all who X invited would still do the trick. If an IP address appears more than 5 times in an invite tree, ban the /24 or ASN if not from a friendly country for 10 minutes or other reasonable timeframe.
Getting unique IPs in any country you want is trivial for anyone but people building toy bots.
How far up the tree do you kick? Going too far up makes it so malicious people can "sabotage" by botting to get huge swatch of legitimate users banned.
Going to shallow means I just need to create N+1 distance between myself and my bot accounts
That's fine. If it is understood that you might be permanently banned because someone you invite starts doing bad stuff, maybe you'll be careful about who you invite.
Inviting people who invited bots chould also hurt your "social credit" score in various ways.
Your tree could for instance be pruned - you can still invite people, but the people you invited can no longer invite people.
There are not a lot of sites which have tried this and failed. Those which have tried to be even a little bit clever about it, have succeeded pretty well (Advogato was a really early example).
What there have been, are sites which rejected such restrictions after a while, because they would rather have a big number to show to investors than real people. Many have even run the fake accounts themselves (e.g. Reddit).