Vale took the lead in the 21st minute when Andy Martin’s cross was blasted home by Scott Moffat.
Their advantage was doubled with four minutes of the first half remaining when Jordan Hall found the net from 15 yards. Cove pulled a goal back on the stroke of half-time when a Blair Yule cross fell to Ritchie Singer whose scuffed shot nevertheless rolled over the line.
Just as it looked like Vale would hold out for the result, Cove grabbed an equaliser in the 89th minute, Kevin Webster firing home. But in the fifth minute of injury time John Ferguson was put through on goal and he chipped the ball into the net to win it for Vale.
Vale of Leithen: Wilson, Beaton, McKenzie, Miller, Lee, Greenhill, Martin, Paterson (Devlin 82), Moffat (Ferguson 75), Radzynski, Hall (Lauder 75). Subs: Sproule, Hill.
Cove Rangers: McKenzie, Yule, Smith, McCulloch, Small, Jeffrey, Singer (Webster 61), Emslie, Milne (Leydon 68), Johnston (Park 68), Beagrie. Subs: Watson, Sim.