The problem is that the judiciary is a timid institution by nature, because the Constitution actually says very little about it. All the courts lower than the Supreme Court are established by Congress in federal law versus the Constitution, and most of the Supreme Court's powers are implied and not explicitly defined in the Constitution, like their power to do judicial review for constitutionality. Even what is said about the Supreme Court is very vague; what it actually looks like is also at the discretion of Congress.

Given that they derive most of their power from Congress's discretion, and on top of that are not elected officials, they use their power sparingly, because it is supposed to be Congress's job to legislate.