The Citizens were leading 2-1 and seconds away from a place in the last 16 only for full-back Roberto Nditi to send the third-round tie into an additional 30 minutes at a sun-kissed Station Park.
Hamish Thomson saw red for the hosts following a cynical challenge on Josh Campbell with eight minutes remaining of extra time but it was the Angus men who held their nerve from 12 yards, City skipper Craig Thomson seeing his spot-kick saved by Forfar No.1 Daniel Hoban.
“There’s no doubt we were the better side over the 90 minutes,” a disappointed Naysmith said afterwards. “Their goalkeeper has made two or three unbelievable saves and we’ve hit the post all within ten minutes of the restart so you think if it goes to 3-1 then we go on and win the game comfortably. If we’d come away 3-1, 4-1 winners I don’t think anyone could have argued. I’m not trying to be disrespectful but I think Forfar got out of jail.”
It was the League Two play-off hopefuls who pounced first, Ouzy See converting a pinpoint Callum Crane cross in the fifth minute from a couple of yards.
Hibs chief Ben Kensell vows to 'address' missile throwing at Hearts defender Alex Cochrane; as Kris Boyd demands SPFL 'do more'
Martin Boyle seals Hibs return as club confirms contract length ahead of derby with Hearts
'Not good enough.' Hearts hit out at Hibs fans as lighter hits Alex Cochrane at Easter Road
How the Hearts players rated in the draw against Hibs
How the Hibs players rated in the derby draw with Hearts
Jordan Allan restored parity seven minutes later when he capitalised on some hesitancy in the visitors’ defence to nick the ball into the net ahead of goalkeeper Calum Antell.
Naysmith was looking for his fifth successive win of his tenure and the Capital side again found themselves in front through defender Lee Hamilton’s spectacular overhead kick.
City spurned a flurry of chances after the restart, Campbell also seeing his half-volley come back off a post, and were made to pay in the fourth minute of stoppage time when Nditi’s sweet strike from the angle of the box found the far corner.
Thomson’s dismissal in extra time handed Naysmith’s side the numerical advantage but they couldn’t capitalise.
Liam Henderson, Liam Brown and Marc Laird all converted from the spot but when Hoban saved substitute Thomson’s penalty, Daniel Scally stepped up to fire the League One strugglers into the fourth round.
“We’ve maybe got one or two things to work on defensively but, in an attacking sense, that’s probably the best we’ve played since I came to the club,” Naysmith explained. “It’s very difficult to be critical of your players who have just had five games in two weeks, coming up to a side in the division above and then being the better team and putting in the performance that we did.
“But if you don’t stop the goals at one end and put the ball in the net with the chances created then it’s very difficult to win matches.
“It’s frustrating. It doesn’t spoil the run we’ve been on because over 120 minutes we haven't been beaten but we wanted that momentum to kick on. Some people might say ‘listen it’s a blessing in disguise you’re out the cup so you can focus on the league’ but I don’t buy it. I picked a team I thought could put in a good performance and that’s what they did.
“I don’t want the players to be down but reflect on their disappointment, learn from the pain of losing on penalties, and be ready to go again against Annan tomorrow night.”