Tynecastle were dealt a blow when Jordan Caddow was dismissed for violent conduct in the 16th minute.
Strollers took the lead ten minutes later. Ahmed Wague showed great strength to shrug off two challenges before setting up Ryan Raeburn.
The visitors doubled their lead in the 38th minute. Wague produced a deep cross from the halfway line to find Raeburn who layed the ball off for captain Ross Aitchison to smash the ball home from 20 yards.
Tynecastle hit back seven minutes after the restart through Kevin Brown.
With a stiff wind at their backs, Tynie had the momentum to go on and find an equaliser, but in the 65th minute Nicky Rendall hoisted a long ball to find Wague, who finished well from just inside the box.
Rendall added a fourth for Strollers in the 82nd minute John Williams netted a 25-yard free kick in the 90th minute.
Tynecastle: Newman, Caddow, Dewar, Kane (Czop 59), Williams, Lister, Brown, Speed, Butler, See, Devlin. Subs: Weir, Aitchison, Wales, McDonald.
Civil Service Strollers: Callan, O’Brien, Rendall, Lister, Newall, Aitchison, R Raeburn (Edema 88), Thomson (Ramsay 83), Wukunbwa, A Raeburn. Subs: Scott.