A LOR these days is career ending, for anyone.
No...they're not. I've written an LOR and that individual has moved on, improved and been promoted twice. It was an LOR for an AWOL. There was some politics involved (they had 1st and 2nd level command permission to be on leave) and I was ordered to write it by the Wing Commander.
In most cases an LOR stays in your file for 6 months...doesn't go forward...and after 6 months...if you've been good...it's shredded. Different procedures govern different LOR's...depending on the service.
Now...and LOR for a general or a field grade officer or senior NCO? Yeah...usually career ending. An LOR for a company grade, NCO or airman/soldier/seaman/jarhead? No. You can recover from that.