The problem with gaming mode is how quickly it falls appart the moment you try to use it for something other than gaming. Something as simple as having more than one window is impossible under Gamescope. That's pretty problematic when a toolkit decides to implement something as a stealth window, like GTK context menus. Qt doesn't do this as much as GTK does so using Qt applications isn't as problematic, but it's still a pain. For instance, you're extracting a file with Dolphin and a pop up window shows up to report progress, making you completely unable to access the main Dolphin window until the operation has been completed.
The best part is that SteamOS displays a little "Switch Windows" section under the "Exit game" button when you have multiple windows opened, which literally just doesn't work and as far as I can tell never has. The only thing that menu does is show you the names of the opened windows and let you close them by pressing X. Switching windows, the thing the section is literally named after, doesn't work and never has since I got a Steam Deck last year. You select a window, it gets highlighted in the menu and that's it. Nothing else happens. It doesn't switch focus or switch the window displayed by Gamescope, it does nothing.
Another thing that's often problematic is that you can't maximize windows. Say your app decides to open itself windowed, Gamescope is just going to blow that 480x360 window up to full screen and makes zero attempt to actually resize the window to fit the screen, so you're stuck with a very blurry and zoomed-in window. The maximize button in apps with CSD does nothing, but other built-in means of resizing windows or achieving full screen do often work. But these built-in means often don't exist, because applications expect to be running on a window manager that's actually capable of managing windows.
And then there's just all kinds of bugs. Say you open a game with a certain aspect ratio/resolution while also having apps with a different aspect ration/resolution open, you'll often find that when going back to your app you can't move your mouse outside the boundaries of the window for the game you just opened. Another thing I've seen with many games is that the view often gets shrunk to a tiny square in the center of the screen. There's a lot more, but I'm sick of ranting about gaming mode.
My personal take is that SteamOS's Big Picture/Gaming Mode shell sucks balls. It's impossible to make most desktop apps work well in Gaming mode without bending over backwards to work around the myriad of issues it has (for the ones that can even be worked around) and since it's closed source there's nothing you can do about it. Thus, the best solution would be to develop a new Gamepad-centric open source shell to replace it. I also think rather than repurposing Plasma Mobile applications like Angelfish it would be better to design new ones that are truly designed for gamepads. Perhaps Plasma Big Picture could be used as a starting point. But it would be a really big undertaking and there probably aren't enough devs interested right now.