Ben Wallace had criticised SNP MPs David Linden and Drew Hendry and vowed to complain to the party over their alleged behaviour on a flight to the Overseas British Territory on Tuesday.
Mr Wallace also criticised Labour’s Charlotte Nichols after claims she required a wheelchair to make her way to a military minibus.
The two SNP MPs have denied “drinking heavily” on an Armistice Day visit to Gibraltar following allegations made by the defence secretary.
And First Minister Nicola Sturgeon said allegations made about her party’s two MPs drinking too much on the trip were “false”.
"There have been allegations made against two of my MPs that they say are completely false and unfounded, and some may say is an attempt by Conservatives to distract attention from their own travails,” she said.
“I know Drew and David well. They say these allegations are unfounded and that is something I’ve got confidence in.”
Mr Wallace had earlier said: “This type of behaviour shows a lack of respect for the enduring work of our armed forces.
“The armed forces scheme is an opportunity for both Parliamentarians and the military to understand each other.
“This behaviour puts military personnel in a difficult position and risks undermining respect for Parliament.”
It was reported the MPs allegedly at fault were part of a 15-strong all-parliamentary delegation on the trip, and had been “drinking heavily” during the flight to Gibraltar.
There were also claims Mr Linden and Mr Henry had been involved in an argument with officials upon landing.
However, Mr Wallace has now been accused of “political game playing” after not mentioning Tory MPs Ben Everitt and Christian Wakeford who are alleged to have gone out until the early hours.
A source close to the AFPS trip said: "People are staggered by the hypocrisy of Ben Wallace and his ill-advised attempts to drag the armed forces into political game playing.
“It was a serious error of judgement on his behalf, which has spectacularly backfired, given two Conservative MPs were out on the lash in Gibraltar until the early hours.
“If anything is disrespectful, its Conservative MPs turning up hungover to official events with service personnel on the eve of Armistice Day."
Both MPs and Mr Wallace have been approached for comment.
Boris Johnson this week insisted that he did not believe UK politics was "corrupt" amid the ongoing row after attempting to change the way MPs are investigated due to Owen Paterson facing a 30-day suspension for lobbying.
Mr Linden said he was "incredibly disappointed by what appears to be a bizarre Tory smear campaign in the media tonight, but I’ll leave the politics to others”.
He added: "Instead, I’ll continue to focus on participating in briefings with service personnel, meetings with local officials, as well as continuing to undertake training exercises and hear first hand from those who serve in the armed services here in Gibraltar."
Mr Hendry tweeted: "It's deeply disappointing that Tory MPs have made false claims in a shameless attempt to divert attention from the Tory corruption scandal engulfing Westminster. That is a matter for their conscience.”