Guardiola recently dodged questions about whether he could leave City at the end of the season amid speculation about his future.
Reports claim City won't find out what Guardiola's next move will be until the season finishes, as he takes him time in deciding whether he will end his spell with the Premier League side.
Guardiola moved to the Etihad side in 2016 and even though his contract does not expire until 2027, he seems destined to finally depart the side after a decade in charge.
Speaking on talkSPORT, Pearce believes that City must tie Guardiola down immediately or find a replacement as soon as possible.
"We saw (Sir) Alex Ferguson leave Man United (after winning title in 2023), they haven't won the league since. (Jurgen) Klopp leaving Liverpool as well, won the league, but there's been a downturn, potentially.
"Brian Clough at Forest. Exactly, (Arsene) Wenger. These are icons of the game. I'm quite saddened when any of these managers, even when they're not my team, I'm saddened when they actually walk out.
"If you're Man City now, the one thing you've got to do, is either secure him on a longer-term contract, that's first priority.
"If not, establish when he decides to leave, and he calls that shot, and make sure you've got a replacement.
"I just think you've got to do everything you can as a football club to secure him and his signature.”
Guardiola is the longest-serving manager in the Premier League by some distance and his loss would hurt both City and the league who have struggled in Europe this season.