Southgate says taking charge of the likes of former club Middlesbrough no longer has an appeal.
"I've got no passion to just go and manage in the Premier League," he told The Football Boardroom podcast.
"I did that at 35, I finished 11th, 12th. Who is in those positions now, probably Bournemouth, Brighton? I've had one of the biggest jobs in world football so I've been spoiled. Huge nights, working with outstanding players. No owner interfering."
Southgate also accepts the likely backlash that would come if he had landed the Manchester United job.
"The bit that, externally, people are saying is, 'well, he didn't win'. So how do you prove that you can win? You've got to go to one of those big clubs," he explained.
"Now we know those big clubs… do I think I could have done the job that some people who have been in those clubs recently have done? Could I do better? I think I could.
"But we've talked about the baggage that comes with me as an appointment if you're an owner. And there's a reality around (that), I can understand what that noise is. There's that bit which makes me potentially a complicated appointment for a club."