Baseband vuln seems likely. Additionally, I've always wondered if the NSA or the like could just decap and read out data (IE: the passcode hash) from the chips. Then again, I'm not a computer engineer and don't really know anything about the physics of all this!
EDIT: This paper [1] seems to document such an attack, decapping the chip and reading the memory cells using lasers (although the details are over my head).
I think most of us are fine with the government being able to decap our phones and steal data, or at least try. I'm fine with that because It cost money and time and would be difficult for them to do in a massive, dragnet style operation.
The idea is once they have the phone they can do whatever they want with it, but they can't go to a manufacturer and force them to weaken the security of _all_ phones because they don't want to do any work.
Yeah, I'm completely fine with that as well - I'm happy to see the NSA do computer-security stuff that isn't this dystopian "monitor everybody all the time" nonsense.
EDIT: This paper [1] seems to document such an attack, decapping the chip and reading the memory cells using lasers (although the details are over my head).
[1]: https://www.cl.cam.ac.uk/~rja14/Papers/SISW02.pdf