Not entirely. Carter was seen as a failure because he came into DC with all sorts of ideas about how the place should work, was confronted with how it actually worked, and accomplished basically nothing.
Since the metric here is “presidents who did nothing” - Carter ranks pretty well up there.
He did nothing, while the economy burned. That doesn’t really count.