West Ham won 3-1 on the day, with Xhaka seen arguing with home fans behind the dugout late on.
The confrontation was sparked after Nordi Mukiele was substituted and clearly upset when taking his seat on the Sunderland bench.
The fan told The Sun: "Everyone’s going ‘Whey! Whey!’ and giving it to him, ‘dry your eyes, mate’ – like we always do it to be fair, being so close.
“Then he sort of turned around, went back towards the pitch and had a pop at the manager.
“That got another ‘Whey!’ again, a bigger roar. He’s turning around all the time, we’ve obviously rattled him.
“Then Xhaka‘s turned around because he’s sitting two or three rows behind the dugout and between us.
“He’s turned around and started giving it to one of my mates who was sitting just in front, basically going ‘show some respect, show some respect’ – quite ironic considering (Mukiele) just had a pop at his own manager.
“Everyone then starts giving it to Xhaka, telling him to wind his neck in.”
"I'll see you outside"
However, instead things escalated as one fan tried to cool things down.
The source continued: “Another guy came (towards the dugout) to calm it down.
“Then Xhaka started picking on him – he wasn’t doing anything, he’s just saying ‘stop, stop’ to Xhaka.
“And then Xhaka started calling him out saying, ‘I’ll see you outside, I’ll see you outside.’
“Then as he was heading down the tunnel after the game finished he was still going to the guy, ‘I’ll see you outside’.
“It was all Xhaka, complete head loss. The geezer’s on 100 grand a week getting rattled by absolute nobodies, just giving it the big ‘un.
“It was all on him.”