"Fine linen" definitely refers to the saints:
7Let us rejoice and be glad and give the glory to Him, for the marriage of the Lamb has come and His bride has made herself ready. 8It was given to her to clothe herself in fine linen, bright and clean; for the fine linen is the righteous acts of the saints." (Revelation 19: 7,8)
I like your theory that those in Jerusalem will be the saints.
I know it's just one of those little technical questions, but it's fun to try and put the pieces together.