Like most automations they'll never replace us fully. But we're already seeing productivity boosts that will mean not as many people are needed.
Kind of like how excel massively reduced how many accountants you need.
I think we've seen pretty much the limit of what LLMs can pretend to do. They can sort of spit out code snippets in a stack overflow kind of way, but are not capable of starting with a set of requirements and producing a complex program. Programming has never been about remembering all the syntax and having all the design patterns memorized, that's what documentation and search engines are for. Programming is engineering a complete, maintainable solution given a set of requirements.
I do think LLMs will eat Stack Overflow's lunch as a source of quick code snippets for programmers to copy/paste.
LLMs are general purpose, basically for demonstration purposes. This is basically generation 0.
Once they can optimize the training process more, which is something being heavily researched, you'll be able to create ones for specific languages or even frameworks.
Dedicated hardware will be another huge boost.
Then they'll start to be amazing.
Only time will tell I guess, but it seems to me like they've trained what they can train, it stopped getting better a while ago, and the core issues of reliability are unsolvable problems.
I think the current issue is if you train something to write code and poetry and recipes etc. it's going to reduce your accuracy at each task.
But as you say we'll see.