Only hours after the summit opened in Colombo, Cameron flew into the northern Jaffna region where some 100,000 people lost their lives in fighting between Tamil rebels and troops from the majority Sinhalese government.
He is the first foreign leader to visit Jaffna since the former British colony gained independence in 1948.
The three-day summit in Colombo was meant to be a chance for Rajapakse, a Sinhalese nationalist leader who oversaw the crushing of Tamil Tiger rebels in 2009, to showcase the development of his country.
But after refusing to bow to demands for an independent investigation into the end of the conflict, he has been confronted by a public relations disaster.
The leaders of Canada, India and Mauritius have all snubbed the meeting and Cameron has said he wants his visit to shine a spotlight on the plight of war victims.
Cameron landed around 2:00pm (0830 GMT) at the Palaly airbase, around 20 kilometres (12 miles) from Jaffna town, in a Sri Lankan military plane.
He was then due to meet victims and survivors of the 37-year war which was one of the bloodiest in Asia.
“I’m the first PM or President to go to the north of Sri Lanka since 1948. I want to shine a light on chilling events there first hand,” he said on Twitter.
Cameron flew out shortly after Rajapakse said in an opening speech that the Commonwealth must not be a “judgemental body” and warned his fellow leaders of trying to impose their own “bilateral agendas”.
“If the Commonwealth is to remain relevant to its member countries, the association must respond to the needs of its people and not turn into a punitive or judgemental body,” he said in a speech ahead of the formal opening of the summit by Britain’s Prince Charles.
Since the war, the economy has enjoyed growth rates of up to 8.2 percent and more than one million tourists visited Sri Lanka last year — a new record.
But what was meant to be a chance to champion a new-look Sri Lanka has been overshadowed by the legacy of the war.
The prime minister of Canada, Stephen Harper, was the first to announce a boycott after his government said the summit was akin to “accommodating evil” while his Mauritian counterpart Navin Chandra Ramgoolam — due to host the next one — is also refusing to attend.
Even India’s Prime Minister Manmohan Singh is staying away, preferring to antagonise a neighbour rather than offend Tamil voters ahead of next year’s elections.
The agenda for the three-day summit includes sessions on debt restructuring and climate change.
But Rajapakse spent the build-up fending off allegations that his troops were responsible for the death of some 40,000 Tamil civilians in the final weeks of the war.
During an impassioned speech, the 67-year-old leader said his regime deserved credit for ending the conflict.
“We asserted the greatest human right — the right to life,” he said.
“In the last four years there has not been one single terrorism-related incident in Sri Lanka.”
At least 100,000 people lost their lives in the conflict which began in 1972.
The northern Jaffna peninsula, home to around 800,000 Tamils, was the main battlefield.
Before the war, Jaffna had a flourishing economy — second only in terms of wealth to Colombo.
But its towns and villages are now littered with shelled-out buildings, interspersed with abandoned farmland. Some 30,000 people still live in refugee camps.
“Everyone is pretending that everything is okay, that Tamils have equal rights but it’s not true,” said the editor of a Jaffna-based newspaper who is to meet Cameron.
“This needs to be told to the international world,” said M.V Kaanamylnathan, editor of the Uthayan (“The Sun”) daily whose printing presses were recently torched and which lost several reporters during the war.
At the last summit in 2011 in Perth, Commonwealth leaders drew up a charter of common values which committed members to respecting human rights.
Australian Prime Minister Tony Abbott, who is handing over the chairmanship of the Commonwealth, acknowledged Sri Lanka was emerging from a troubled past and that its people could take heart from how South Africa and Ireland had come through darker times.
“Sri Lanka’s willingness to host this Commonwealth shows its commitment to democratic pluralism and freedom based on law and ought to reassure all its citizens that just as today is better than yesterday, tomorrow will be better than today,” he said.
“So we are here to praise as much as to judge.”
Prince Charles, who is representing his mother Queen Elizabeth II, said the Sri Lankan people had “confronted great adversity,” as he also recalled the devastating impact on the island of the 2004 tsunami.