If the FBI knew about Hillary’s private server, it would fit the FBI’s history from its very origins to allow it to run. The FBI loves holding a legal problem or damaging secrets over politicians’ heads — the easier to get fat budget increases, appointments, etc. Monitoring operations without busting them also allows them to reel in others who are in on the operation. Stop them and you no longer have someone to blackmail or bait to catch others. Trump perceived that Comey was not so subtly blackmailing him when he informed Trump of the contents of the dossier, etc. The message was “we can ruin you, so come to heel.” I don’t think Trump appreciated his “warning”.
The Clintons’ historic innovation was converting the FBI’s self-serving dirt/blackmail operations into Clinton-serving dirt/blackmail operations (beginning exactly as you say, with Craig Livingston in the White House using FBI raw intel files — how’d he get those?). The Trump “dossier” seems unusual to people because of the creative use of FISA warrants and foreign IC allies to spy on domestic political opponents via overseas calls and communications.
That’s my 100% unqualified take on the matter...