McSheffrey, who bagged 10 goals in two loan spells at Luton, scored his 17th of the season one minute after Stern John's 29th minute opener for the visitors.
The 'Little Sheff' was a constant menace throughout as Coventry looked in control for much of the game.
Town did get a hope of salvaging a draw when Ady Williams diverted Enoch Showunmi's cross into his own goal five minutes from time but the Sky Blues saw out the win.
With the Hatters suffering a defensive crisis, leading goalscorer Steve Howard started the game at centre-half alongside Markus Heikkinen.
Rowan Vine wasted an early opening when he saw his fourth minute shock blocked after Peter Holmes had robbed Don Hutchison in the middle of the park.
Howard was called into his first piece of defensive work four minutes later as Williams whipped in a Sky Blues free-kick that Town's stand-in centre-half could only half head clear and Marlon Beresford was quickly out to smother Hutchison's six-yard shot.
As both sides settled down, Hatters created the first flowing move in the 13th minute when Vine fed Warren Feeney on the left and the Irishman cut inside before finding Ahmet Brkovic, whose instant 15-yard shot was tipped around the post by Marton Fulop.
The on-loan Spurs keeper then produced an even better block five minutes later when Feeney raced clear on to Showunmi's perfectly weighted through ball only to see his low effort blocked by the goalkeeper's legs.
As has been the case for much of the season, Luton were made to pay for their slack finishing when Coventry showed Town how it should be done with two goals in the space of two minutes.
Firstly John got ahead of Heikkinen to superbly volley home Andrew Whing's dangerous right-wing cross in the 29th minute and McSheffrey – well used to netting at Kenilworth Road – made it two on the half-hour.
The 23-year-old ran on to Michael Doyle's slipped pass and saw his first vicious 16-yard shot saved by Beresford, but as the ball appeared to be running out for a corner, McSheffrey was alert enough to net the rebound from a tight angle.
Luton finally woke up as Howard slipped in Holmes only for the midfielder to see his goalbound effort turned away Fulop in the 62nd minute.
Coventry were still able to attack with ease though and Hutchison found Andy Morrell in acres of space four minutes later as the winger saw his first effort saved by Beresford before firing the rebound over.
Leon Barnett nodded over in the 77th minute before denying John a minute later as the Trinidad & Tobago international threatened to burst clear.
The Hatters were given a lifeline five minutes from time when Showunmi burst to the by-line and saw his cut-back diverted into the goal off visiting defender Williams but it was too little too late.