Milliseconds are a commonly-used unit. It doesn't really matter if 1 ms is too fine a granularity -- you'll just have to write "autocorrect = 500" in your config file instead of "autocorrect = 5", but who cares?
And then you have the rhythm gamers who can adjust their inputs by 5 or 10ms. Hell, I'm not even that good of a player, but in Fortnite Festival, which has a perfect indicator whenever you're within 50ms of the target note timestamp (and a debug display that shows you a running average input offset) and I can easily adjust my play to be slightly earlier or slightly later and watch my average fall or climb.
Several top players have multiple "perfect full combos" under their belt, where they hit every note in the song within 50ms of the target. I even have one myself on one of the easier songs in the game.