A Joe Cardle brace had the hosts on course for another three points until Danny Handling's neat finish with 15 minutes remaining at New Central Park gave the Citizens a route back into the game.
And their efforts were rewarded in stoppage time when on-loan Hibs defender Jack Brydon powered home a header from a tight angle to spark wild jubilation among the travelling support.
The well-deserved point sees City leapfrog Stenhousemuir into fourth in the League Two standings ahead of Saturday's mouth-watering Scottish Cup fourth-round clash against Premiership outfit Aberdeen at Pittodrie.
“I thought our performance was outstanding,” Ferguson said afterwards.
“We knew it was important to keep the ball off Kelty and we managed to do that really well in the first half so we were really unlucky to be behind at the break.
“Then to lose a second was really disappointing. The first ten minutes of the second half we struggled to impose ourselves, but the character the players showed was terrific to come back and we thoroughly deserved a point.
"It was a great finish from Danny and it gave us that lift to carry that momentum going into the final ten minutes.
“To get a last-minute equaliser at any ground is such a pleasing thing to do so I'm delighted for everybody.
“It's only a point and we now need to build on that moving forward. It's good that we're finally getting players back from injury so the squad is looking a lot stronger."
Eighteen-year-old Brydon said: "It was great to score my first goal in professional football. I peeled off to the back post with no-one marking me so I was always going to head towards goal as I was in the right place at the right time.
"I had a funny feeling before the game that we'd get the win but we'll take the draw.
“I've played against Kelty before, they're a good side but I feel with the signings we have made we can challenge these teams."