That's when to buy a little more. The whole idea is to sell high and buy low (now there's a truism). The fluctuation is the lever. But the bottom line is that the "game" the Fed has been playing has a shelf-life. At some point, the "jig" (forgive the insensitive terminology) will be up. At that point, you won't be able to give $dollars away at hardly any price. That's when metals will look incredibly sane.
I did sell high. It still looks high to me. Buying as you have over the past year and a half would have been a losing proposition.