Spelling preserves a lot of meaning and etymology in words. Phonetic spelling would remove some of this richness from the written language. Logographic systems like Chinese take this to the extreme. I think English strikes a nice balance.
Spelling can also bridge the gap between phonetic differences between accents. Many variants of English don't even have the same number of sounds as others, but we all "come together" when we write.
In English, it would frustrate me if every time I read marry it would be spelled the same as merry, because I say those differently, even if most Americans I come in contact with daily do not.
Spanish spelling is much more phonetic than English, but one thing I admire about it is when things are correctly spelled, you can adjust the text for accent based on spelling. The only area I can think of where spelling does not denote contrast in any accent is between b and v. (Edit: and that h is silent.) There are some contrasts that are only kept by a minority of speakers (Ll is merged with Y for most people, but maybe not in Bolivia or parts of Spain; s and z are merged in Latin America), but it's still a meaningful contrast for somebody.