Longstaff struck the winning penalty in the shootout at St Andrew's. Before then, leading 1-0 through Lukas Nmecha, Leeds were taken to extra-time after Patrick Roberts equalised just before the 90 minutes.
With the scores locked at 1-1 at the end of extra-time, the tie went to penalties, which saw Leeds goalkeeper Lucas Perri decisive as he denied Tommy Doyle before Roberts blazed his kick over the crossbar in the shootout.
Longstaff then stepped up to convert his spotkick for the win and he said afterwards: "I think it makes it easier knowing that if you miss there's more penalties to come. But I thought Lucas (Perri) was massive today, obviously in the shootout but in the game, the first half especially.
"We've ground out a result. It's never pretty in the FA Cup but we're happy to get through.
"I think any time you get drawn away to a team in a lower division to you a lot of people expect you to rock up and win but it's never the case. I thought they were excellent today.
"Ultimately, it comes down to the moments and we were better in the bigger moments today and I'm really happy for Lucas because it's been - not sort of a tough time for him - but he deserves it today. I'm really happy for him."