The win, Rose's tenth of the campaign, maintains the Midlothian outfit's four-point lead at the summit.
The hosts edged in front just before the half-hour mark from the penalty following a foul on Keiran McGachie. Up stepped George Hunter and despite seeing his initial effort saved, the striker was quickest to react to slam the rebound home.
The visitors levelled right on the stroke of half-time through a penalty of their own, Neil McLaughlin making no mistake from 12 yards.
But Rose restored their lead four minutes into the second half through McGachie's fine volley and despite some pressure as Kilby went in search of a second equaliser, the home side sealed the win with a third goal in the fourth minute of stoppage time courtesy of Bradley Barrett's solo effort.
"It's a massive three points," a delighted Horn said afterwards. "To be honest it was nip and tuck in the first half. They put us under a lot of pressure in the second half so we had to dig in and defend and to a man we did that. I thought we were outstanding defensively, the players threw their bodies on line, and Michael (Andrews) had a good game in goals.
"I've said it before that Michael has had to wait quite a long time for his opportunity because Mark (Weir) has been outstanding in previous seasons. But the injury to Mark has given Michael his chance and he's grabbed it with both hands. He's been outstanding and he's a great professional. When's he not played his attitude has been fantastic."
Elsewhere, third-placed Civil Service Strollers are in hot pursuit after a 2-1 victory over Gretna 2008 at Christie Gilles Park, Alieau Faye and Bulgarian teenager Svetoslav Kosakov with the goals for Gary Jardine's men.
Meanwhile, Spartans are level on points with Strollers following a hard-fought 2-1 success over Berwick Rangers.
Following a goalless first half, Blair Henderson headed Spartans in front at Ainslie Park on 49 minutes, Cammy Russell made it two before Lewis Baker set up a nervy finish.