And most of the population within them usually echo the same voice. You'll find sympathisers on both sides, but usually support what its Jarls support.
Wrong, half of the Jarls didn't want Ulfric, not a free Skyrim. It is said that if Ulfric asked the High King to declare Skyrim free of the Empire he might have done so. That clearly states the displeasure with the Empire if the High King would consider it. The terrain isn't everything, the Elves have to cross treacherous seas or go through the mountains.
