If you're in the Young Men's program and actively performing your duties as the quorom advisor and Scouting leader for that quorom, you find that you have very little personal or family time. One weekend will be priesthood activity of some kind, another will be Scouting, another may be Elder's quorom, etc.
That's not taking into account the time spent during the week preparing for those activities or the time spent as a Scout leader ensuring your actively engaged in trying to ensure the Scouting program is run correctly.
The amount of stress placed on the family is immeasurable, but don't worry, you'll be "blessed" for your sacrifices for the "church". /s
I can only imagine. I was in the RS presidency in a STUDENT ward (single - no kids) and it took up an insane amount of time to do it right.
Families that have the husband in the Bishopric and wife in Primary or RS hardly see each other.
And non-Mormons often fail to understand that YOU DO NOT turn down a ‘calling’. That is like turning down God Himself.