"Rabbani has been martyred," Mohammed Zahir, head of the Criminal Investigation Department of the Kabul Police, told Reuters.
The attack occurred at his heavily guarded residence in Kabul diplomatic enclave.
It came only a week after a 20-hour siege at the edge of the area sometimes known as the "green zone".
The news was confirmed by Kabul police spokesman Hashmat Stanikzai.
"An explosion happened inside the house of Burhanuddin Rabbani as a result of which Rabbani was martyred and several others were wounded," said Stanikzai, Agence France Presse (AFP) reported.
Another police source told Reuters that Masoom Stanekzai, a senior advisor to President Hamid Karzai, was also badly injured in the attack.
"Masoom Stanekzai is alive but badly wounded," said the police source, who asked not to be named as he is not authorized to talk to the media.
Rabbani, a former warlord himself, served as president in the 1990s when mujahideen factions waged war for control of the country after the Soviet withdrawal.
Blow to Peace
Rabbani had been tasked with trying to negotiate a political end to the US war on Afghanistan.
"This is a big blow to peace process and huge loss for Afghanistan," Sadiqa Balkhi, a member of peace council, told Reuters.
His plan included offering amnesties and jobs to Taliban foot soldiers and asylum in third countries to leaders.
Rabbani was an influential and spiritual leader and was successful in
luring Taliban fighters into peace process," she added.
There are currently about 100,000 US troops fighting in Afghanistan, up from about 34,000 when President Barack Obama took office in 2009.
It aims to gradually hand over all security operations to Afghan security forces by 2014.
The Taliban, which ruled Afghanistan in the 1990s, were ousted by the United States, which invaded Afghanistan shortly after the 9/11 attacks in 2001.
Since then, Taliban fighters have engaged in protracted guerrilla warfare against the US-led foreign troops and the West-backed Hamid Karzai government.
In the past two years, violence became at its worst across Afghanistan, with civilian and military casualties at record levels despite the presence of 150,000 foreign troops.