The 65-year-old took the reins at Kenilworth Road in February 2013 and led the club to the Conference title in his first full season in charge.
The Hatters finished eighth in their first campaign back in Sky Bet League Two but have struggled to replicate that form this term and sit 17th ahead of Saturday's trip to Exeter.
Academy and development boss Andy Awford has taken charge of the side on a caretaker basis and will be assisted by the existing backroom staff
"On behalf of the board I would like to thank John for everything he has brought to Luton Town," Hatters chief executive Gary Sweet told lutontown.co.uk
"We have shared some unbelievable moments in this club's history together, ones which will never be forgotten.
"The decision to make a change was a unanimous one by the board but one of the most difficult since we took over as custodians in 2008
We are genuinely saddened that our partnership with John has come to an end earlier than we expected.
"We had hoped John would be with us for the longer term but now attention turns to finding the next man for the job."
Still left League Two club Dagenham after nine years in charge to take over at Luton and his decision was vindicated as the Hatters broke the 100-point mark and scored a century of goals on their way back to the Football League
But he departs Luton on the back of three league losses, including two 4-3 home defeats, and an FA Cup exit at the hands of Peterborough
"Naturally I am hugely disappointed to be leaving Kenilworth Road but I would like to thank everyone involved at Luton Town for making my time at the club an enjoyable one," he said
"The board of directors have been fantastic every step of the way, all of the staff - on the football side and behind the scenes - have been unbelievably supportive and the fans have been nothing short of incredible.
"The Conference championship season is one which will live long in my memory and I was proud and honoured to have played a part in bringing Luton Town back in to the Football League.
"I wish the club all the very best for the rest of the season and beyond."
Source : PA
Source: PA