I would probably not pay $10 to post on HN, but many spammers who expect some kind of tangible return would pay that, so the fee just makes the problem worse.
The spammers wouldn't pay it once though - the idea is that it's a good way to scale moderation. Each time an admin needs to ban a user there is a 10$ subsidy supporting that action - and if the bots come back then they get to pay 10$ to be banned again.
Assuming the money isn't wasted and is actually used to fund moderation 10$ is probably comfortably above the cost to detect and ban most malicious users.
There are large swaths of spammers that indeed would not pay it. There are on the other hand plenty of NGO's that would pay it without a second thought to promote specific topics and dogpile on others. Those are the movements I would expect AI to take over if not already. AI does not sleep, humans do. AI won't miss the comments that groups believe need to be amplified or squelched.
Yeah, I love HN, but I wouldn't pay and I know many if not the majority of other people wouldn't. It would increase quality for awhile for sure, but what happens a year or two down the road? It would kill the user count and reduce comments and become less valuable over time.
History also shows you can take a $10 fee and maintain quality on SomethingAwful for quite some time.