Esmael Goncalves scored the winner and was sent off as Hearts recorded their first Premiership with of the season.
Goncalves and Kilmarnock centre-back Kirk Broadfoot were both ordered off after clashing midway through the second half. However, the Portuguese's goal in the opening five minutes decided what was a tempestuous affair in Ayrshire.
Jon Daly, the Edinburgh club's interim head coach, would have been pleased by the resilience his team showed under some second-half pressure. They also created chances and striker Kyle Lafferty ought to have scored a second goal from close range.
For Kilmarnock, it is three defeats on the bounce and the Rugby Park support left at full-time with some choice words for manager Lee McCulloch.
Hearts arrived with more than 1500 fans and a record of only one league win here in the last six years. They reinstated Jamie Walker to the squad having omitted him for the previous two weeks, although he had to settle for a place on the substitutes' bench.
McCulloch was desperate for a response after successive losses in the Premiership and League Cup in the last week. However, his team faced another deficit after only five minutes of this encounter.
Michael Smith - operating at left-back for Hearts - slid a perfect pass into Goncalves' feet and he was allowed time to take a touch and steady himself. He used it wisely to dispatch a low finish beyond the despairing Jamie MacDonald in Kilmarnock's goal.
MacDonald prevented a certain second goal for the visitors on 20 minutes by emerging from his goal to tackle Goncalves. The Portuguese striker had linked well with Lafferty in a two-man attack which the hosts found difficult to cope with.
Walker emerged from the substitutes' bench after 25 minutes to pretty unanimous applause as he warmed up in front of the travelling support. There was even chanting of his name as the Hearts fans backed their man despite continued interest from Rangers.
The main attacking threat for Kilmarnock in the first half was left midfielder Dominic Thomas, who gave young Jamie Brandon a few problems in trying to supply chances for strikers Kris Boyd and Lee Erwin. In the main, though, it was those in maroon who enjoyed the better of a very competitive first 45 minutes.
McCulloch introduced left-winger Jordan Jones for the second half hoping to cause 19-year-old Brandon further difficulty. The ploy to target the full-back's inexperience was part of a more assertive home display after the interval. Indeed, two impressive saves in succession by the Hearts goalkeeper Jack Hamilton denied Rory McKenzie a certain equaliser on the hour mark following intelligent build-up play by Thomas.
Lafferty wasted a glorious chance to kill the game when he failed to connect with Don Cowie's low cross with the goal gaping. Whatever this affair lacked in quality, it made up for in drama as play swung from end to end.
The afternoon erupted in controversy on 68 minutes when Goncalves and Broadfoot were red carded. They clashed on the edge of the Kilmarnock penalty area, with Broadfoot at first glance appearing to pull Goncalves back by the hair before the forward retaliated. Referee Stephen Finnie consulted his far-side assistant before dismissing both players.
The Ayrshire side continued their quest for an equaliser but Hearts held out to secure their first league win since April at the end of a hard-fought 90 minutes.
Kilmarnock (4-4-2): MacDonald; Wilson, Broadfoot, Greer, Taylor; Burke, McKenzie, Frizzell, Thomas; K Boyd, Erwin.
46: Jones for Burke
72: O'Donnell for Taylor
87: Samizadeh for Wilson
Unused subs: Bell, Waters, Power, Higgins.
Hearts (4-4-2): Hamilton; Brandon, Souttar, Berra, M Smith; Cowie, Randall, Djoum, Grzelak; Goncalves, Lafferty.
81: Nowak for Brandon
85: Stockton for Lafferty
90: Martin for Cowie
Unused subs: Noring, Hughes, Walker, Henderson.
Referee: Stephen Finnie.