It comes down to what aspects of the game you enjoy. I find positioning, thinking ahead and trying to predict my opponent and strategizing during my opponents turn to be fun.
For some the most fun is pushing plastic and rolling dice.
For the latter it will not be an enjoyable experience to watch your opponent move twice and get to pick who to fight with first two turns back to back. If that is the case, play without it. House rule it for your friends or LGS, but to remove an element that requires additional foresight and planning at a competitive level would be frustrating to me. Making sure I buff the right units and knowing those buffs will carry through a double turn, placing screens, forcing awkward pile ins, knowing what I am and am not willing to extend with and use, what spells to dispell and command abilities to activate, who do I fight with first, where should move an endless spell, if I put it in the right place it could even force my opponent to take second even if they win the roll off.
The game actually has a fair amount of player agency even on your opponents turn and if a player is not paying attention to it they are missing a large part of the strategy to the game.
I definitely understand that is not fun for everyone but I do see it as a core mechanic to the game's strategy.