According to Alan Bell's book they were more than prepared to have him back but Bell says Wilde suddenly wanted perks in his contract that he hadn't previously demanded. So they decided to move on. I've always found this explanation to be lacking, however. There is evidence that at least up until the filming of "There Goes the Groom" there was some belief Wilde might come back and I've read that wasn't filmed until after all the other episodes were filmed that year. Who knows what the full story was, however.From reading the various books about the series, Brian did have Shingles which meant he had to dive out of filming at short notice. But the vibe I got was the production team were happy to let him go as he was so difficult to work with. It's speculation on my part but I reckon even if Brian had wanted to return after his illness, he wouldn't have been allowed to do so. For that reason I think Truly was always going to be a permanent replacement for him.
There was a poster here a few years ago who claimed there was some talk of somebody else taking over after one year with Frank Thornton but I've never seen any talk of that elsewhere.