Add stuff to the scene that's not just damage. Stuff that splits their attention and requires some strategic thinking.
Also, playing Mage and Fate most recently it's been really refreshing not having any of these DND problems.
This, this, this.
Also I often find inspiration in mechanics from MMOs, RPGs, and boss fight games like Elden Ring.
Gotta ask which mage you playing. Ascension feels really good imo, the health being the same for basically all mortals means combat, when it has to happen, is fast and lethal, with whoever had the foresight and prep time coming out on top. Not to mention things are not meant to be fully balanced, things are meant to be as strong as they are not to be punching bags for players.
I was playing Mage: The Awakening 2nd edition. I really like it but it's not super popular, so finding players is hard.
It sounds like it has the same way of modeling health. Adults have 5 + stamina health boxes. So your average person is 7, the strongest mortal in the world is 10. You have a pretty good idea that if you do nail that guy with a rifle shot, he's not getting up.
DND especially was frustrating to me when it was like "ok a veteran soldier... Does he have 10 HP? 20? 50? One attack? Two?". There's like no way to reliably reason from the narrative to the rules.
Awakening is essentially a different version of Ascension, so you've got the basic Idea of how it plays, although i think it's a bit more lethal and straight combat is more discouraged.
Playing a point buy system with character creation that is the same between NPCs and PCs really does let you have a better connection to the world. You can reasonably guess or study it logic out things about other people in the world.
Use the maximum HP possible from the dice instead of the average given (eg. 6d12 = 72 instead of 39), or at least a higher portion of the maximum quantity
Increase AC
Give it extra damage of a different type
Give non-lair monsters lair actions, and give monsters with lair actions an even stronger lair action they can use when below half-health. Same with legendary actions
Look at older DnD editions and see if the monster or any similar monsters have extra abilities you can add
Edit: I should have specified that these are in ascending levels of difficulty for the DM, but are also more interesting
The risk in 1 & 2 is that monsters become a slog and characters are no longer heroes. If you need to hit a monster 30 times before it dies (because half misses due to AC, and you need a lot of hits due to HP), it's just slow and boring.
Lair actions and distractions/barricades to get to BBEG are where its at. There's a trap in the way. New support keeps popping in until something is destroyed. Something changes drastically halfway through the fight.
1, 2, and 3 are easy options, but lazy AF and deeply unsatisfying in most cases.
4 and 5 is really where it's at, but it's quite a bit more work. 4e's early maths was really bad, with enemies becoming enormous damage sponges, though they were otherwise probably better than 5e. Late 4e improved its monster design somewhat. Pathfinder 2e is great, so borrowing monsters from that could be good. Or Matt Colville's books for 5e.
Look at older DnD editions and see if the monster or any similar monsters have extra abilities you can add
This. The Tarrasque had so many anti-cheese abilities in older editions. Now its best anti-cheese is running away, and even then there's ways to keep up.
Pick up a copy of Flee, Mortals, or port some Pathfinder creatures over (use the Proficiency Without Level options on Archives of Nethys). Or dig up a 4e monster manual and port those over.
Tome of beasts also has a little more bite than standard 5e, I think they've called their design 5e with teeth before, one of those books is also now available on D&DBeyond too if that's to the person's liking.
All that content is under various OGL / CC licenses too so it's available on open5e.com
Buffing the BBEG is the natural response but weakening the PCs is another route. Debuffs can go a long way and can be a problem for the players to focus on and strategize around.
When in doubt, its a [Lich] and there's a [Phylactery] that involves a moral conundrum or sacrificing something or becoming cursed in some fashion: convert the hard rolls a mental one for the players. No natural 20 rolling you way out of willfully giving up your PC's left hand. Or a permanent aura of stench.
Unless they do XYZ of course. Or maybe. They should have listened to that NPC on day one more.
Not to be that guy, but the best advice really is to not play 5e. It's a bad game, there was literally never any thought to balancing it despite being a combat focused game. Things that are meant for your character level will be unbelievably underpowered, things that are slightly too high will be a slog.
Rules option for DnD 5E, specifically for legendary and mythical monsters that tend to be the ones with the most HP:
'Attacks from legendary or mythical monsters hit automatically if they require an attack roll and the effects of the attack would not give the target a condition. Legendary resistances of monsters can only be used against effects that can be assigned to the spell schools abjuration, illusion, transformation or enchantment. Otherwise, monsters can still use their Legendary Resistance to pass saving throws against any effect that would give the monster one or more conditions. In case of doubt, the DM decides. After a monster has used Legendary Resistance, its movement rate is halved until the end of its next turn. The damage caused by the monster's attacks that require attack rolls is also halved until the end of the monster's next turn after using Legendary Resistance.
(These rules can ensure that battles are both shorter and more exciting due to the higher damage dealt by both sides. The battles also become more dangerous, making PCs think twice about taking the risk. Creativity is encouraged when considering alternative strategies for conflict resolution. Both ensure that there will be fewer battles overall, which remain the most time-consuming scenes in the game. The increased risk also makes combat itself more creative when searching for ways to damage the enemy or use the environment for a tactical advantage. It also encourages the player characters to improve their supply of healing options, e.g. through the help of NPCs that can cast healing spells.)'
Scribbled together based on this post and its comments as well as some additional ideas I've had.
EDIT: Fixed something that got lost in automatic translation.
