The participants called on the Sudan People's Liberation Movement/Army (SPLM/A) to enter into immediate dialogue with the armed groups with a view to integrating them as part of the territorial army of southern Sudan.
"In order to avert the danger posed by this armed elements, the participants urged all parties with armed groups under their control to adhere to the demobilization, disarmament and reintegration of combatants," they said in a joint statement.
Convened by former Kenyan president Daniel arap Moi's charitable organization, Moi African Institute, the conference was attended by members of Sudanese civil society organizations, church groups and 11 political parties.
Participants said disagreements among southern Sudanese leaders had, in the past, led to the creation of many splinter groups, several of them supported by the Khartoum government while others by the ex-rebel group.
They stressed the need to have equitable and balanced distribution of the development projects and emphasized that wealth be shared equitably in southern Sudan.
The political leaders also called for forgiveness, continuous dialogue and general amnesty as a way of healing the past grievances.
The SPLM/A leaders, who negotiated the final peace deal with the government, acknowledged that divisions, tribalism and a breakdown of law and order had caused much suffering in the region.
The leader of the delegation from Khartoum, Sudan's Second Vice President Moses Machar admitted that in more than 20 years of war, successive governments had failed to solve the problem in the south.
He urged the SPLM/A and Khartoum to cater for the needs of armed groups so that they could be reintegrated.
Southern Sudan was embroiled in a civil war for over two decades, leaving about two million people dead and over four million others uprooted from their homes.
The main rebel force, the SPLM/A, and the Sudanese government on January 9 signed a Comprehensive Peace Agreement, formally ending the long civil war. Included in the peace agreement were provisions for pulling Sudanese government troops out of the south, self-determination, power and wealth sharing and religious freedom.