On the 15-minute mark, Callum Connolly cut in from the right and gave Sean Robertson in goal no chance with a left foot shot to put Whitehill 1-0 up. With eight minutes remaining, Duns equalised after a great cross from the left was headed in by Kieran Bullen to take the game into extra time.
But, with just seven minutes remaining, Greg Kerr, a recent loan signing from St Johnstone, drilled a left-footed volley into the bottom corner to make it 2-1 to Whitehill and send his side through to a third-round meeting with Ormiston.
Whitehill Welfare: McGinley, Martin, Keane, Murray, Williams, McGregor, Anderson (Manson 66), Connolly, Wringe (Kerr 71), Knight, Muir. Subs: Jardine.
Duns: Robertson, Weatherburn, Walker, McHoul (Bullen 80), Waddell, Brydon, Chisholm, Rutherford, Lauder (Brydon 69), Robinson, Strangeways.