A controversial winning goal by Alfredo Morelos gave ten-man Rangers their first victory at Tynecastle Park in six and a half years.
The Colombian's first-half strike stood despite being offside and it proved the decisive act of a match in which Rangers' Scott Arfield was sent off.
Arfield's second-half lunge on the Hearts goalkeeper Zdenek Zlamal left his team a man down, however they saw the game out to take three points from Gorgie for the first time since April 2012.
An own goal by the Northern Ireland defender Gareth McAuley put Hearts ahead but the visitors overhauled that deficit before the break. Connor Goldson equalised before Morelos finished James Tavernier's free-kick despite being in an offside position.
The win takes Rangers top of the Ladbrokes Premiership and leaves Hearts on a six-game run without victory, although one positive for those maroon was the return of Christophe Berra from injury.
He was back in the starting line-up after four months out with a torn hamstring. Harry Cochrane, Oliver Bozanic, Marcus Godinho and Demetri Mitchell were also reinstated by manager Craig Levein. Rangers made four changes after their midweek European exertions but left former Hearts striker Kyle Lafferty on the substitutes' bench.
After a minute's applause in honour of the late former Tynecastle director Pilmar Smith, the game began at predictably breakneck speed. The visiting goalkeeper Allan McGregor made the first save of the afternoon to push Steven MacLean's head flick behind for a corner. The Scotland No.1 then made an even better stop to touch Arnaud Djoum's looping header from Michael Smith's cross over his crossbar.
Hearts got the better of McGregor on 26 minutes with some assistance from one of the Rangers players. Bozanic exchanged passes with MacLean and then stroked the ball out to Godinho on the right. The Canadian's teasing low cross hit the sliding foot of MacAuley and crept in at the far post to give the hosts the advantage.
Their lead lasted eight minutes. Djoum cleared Oviemuno Ejaria's header off his own goal line and behind for a corner. Tavernier's delivery was met by Morelos, whose header hit Goldson's leg and was parried at point-blank range by Zlamal. Goldson then bundled the rebound over the goal line at the far post before Zlamal could react.
Steven Gerrard's side were celebrating again three minutes before the interval when they moved ahead. Concerted pressure brought a free-kick on the right which Tavernier whipped in towards the near post area. Morelos, despite being offside, got in front of Bozanic to execute a fine one-touch finish across the balance of Zlamal and into the far corner of the net. The Colombian then set off to antagonise Hearts fans in the Gorgie Road Stand by way of a celebration.
Hearts had looked comfortable until the scored but the Glasgow club had come to life after falling behind. They started the second half seeking a third goal, with Morelos continuing to antagonise the home support and Berra in particular. Tempers were rising all over the field and in the stands.
A red card was therefore no great surprise. Zlamal made a double save to thwart Daniel Candeias and Morelos but, with the Czech grounded and his hands on the ball, Arfield decided to slide in. He plunged into Zlamal and was instantly confronted by several Hearts players. Referee Bobby Madden immediately dismissed him for serious foul play.
Hearts pressed for an equaliser during the closing stages but failed to open up the Rangers defence. That left them facing another defeat as Rangers rejoiced in their climb to the Premiership summit.
Hearts (4-3-3): Zlamal; Godinho (Dikamona 64), M Smith, Berra, Mitchell; Cochrane (Lee 70), Haring, Bozanic; Djoum, MacLean, Clare (Morrison 77). Unused subs: Doyle, Wighton, Hughes, Mulraney.
Rangers (4-3-3): McGregor; Tavernier, McAuley, Goldson, Halliday; Arfield, Coulibaly (McCrorie 81), Ejaria; Candeias (Lafferty 85), Morelos, Grezda (Jack 64). Unused subs: Foderingham, Worrall, Flanagan, Middleton.
Referee: Bobby Madden.