Goals from former Hearts player Jordan McGhee, Danny Mullen and a penalty from substitute Jonathan Afolabi secured a 3-1 victory for the Tayside club despite Andy Irving reducing the deficit to 2-1.
Afterwards, Neilson rounded on the official who was also in charge when Hearts lost their only other league game this season at Dunfermline.
"It’s Don Robertson. We’ve lost two games this season and twice Don Robertson was refereeing. It’s a common theme. I’m hoping we don’t see much of him again this season,” said Neilson.
He added that Robertson got the penalty decision wrong with the score at 2-1. “I’ve no idea what he sees. He booked Christophe but Christophe said he wasn’t pulling [Lee Ashcroft].
Hibs debrief: Alarm bells in the stand (and defence); Tavares decision; ref's big calls
Lee Johnson laments Hibs mentality after hairdryer treatment during Livingston loss
Livingston troll Hibs with 'Choose Livi' Twitter post following 2-1 win
Hibs on the wrong end of astounding statistic after latest defeat at Livingston
Former Hearts striker steps out of 'comfort zone' with move to Turkish top flight
“I’ve looked at the video – I don’t see it. You’d need to ask him,” said Neilson before insisting that Byrne deserved to be dismissed for an early challenge on Steven Naismith.
"It was a red card, let’s be honest. The boy was out of control, he's smashed Naisy, but again it’s Don Robertson we’re talking about. It could have gone either way.
“We didn’t start the game anywhere near well enough. You know shooting up the hill with the way the pitch was it’s just going to be long balls, knockdowns and fighting for it. We didn’t get a foothold in the game at all until the second half.
“We started to control the game a bit more. On the balance of chances, we probably should have won the game even though we got beat."
Neilson now wants a reaction from Hearts, who remain five points clear at the top of the Championship. “The boys are hungry and annoyed. We had the chances to get a point if not win the game.
"We know teams will up their game by 30, 40 or 50 per cent when they play us. We have to match that and we didn’t match it in the first half. We matched it in the second half and when we do that, nine times out of ten we will win.”