I always ride outside and will never stop. It’s fun to explore and get some fresh air and it keeps your skills sharp. One year i rode only on the trainer and i was super strong but I couldn’t ride my way out of a paper bag.
I have also had good results trying to replicate sufferfest workouts outdoors; you might not get them exactly identical but if you are hitting the energy systems roughly the same amount, i think it’s fine.
The other guy raises the good point that you can overdo it. One guardrail like he said is to strictly limit the duration and intensity of your outdoor rides, another option is to adjust your subsequent workouts if you end up going real hard outside. (That’s what I do and it works just fine, although SUF could help us out by making it so you can drag and drop workouts to more easily adjsut on the fly).
Edited to add: Some people are loathe to workout outside because they have scheduled a complicated trainer workout with many changes in power and cadence. The yare forgetting that many indoor cycling workouts that are optimized for smart trainers have all that variability precisely because they are trying to mimic the variability of outdoor riding; in a race, nobody is going for 20 minutes and then resting, 20 minutes and then resting. So to me it is perverse to not ride outside because doing so you couldn’t hit your power targets on a workout that is designed that way in order to mimic . . . riding outside.