In a statement, AI said that security personnel had brought the conference to a halt late on Sunday afternoon, calling the meeting "unauthorised".
They demanded the names of all participants, took photographs and recorded video footage of the meeting. Some participants were pushed and threatened, AI said.
The security personnel also attempted to divide the participants into international and national groups and to separate women from men. AI said they refused to be separated, fearing for the safety of Sudanese nationals.
The participants were released after having been held for three hours.
"This meeting was a transparent meeting - previously known to the authorities - of those working towards peace and justice in the region," said Kolawole Olaniyan, director of AI's Africa programme, on Monday.
"Detaining and harassing human rights defenders is a clear violation of Sudan's obligations under international and regional standards, including the Constitutive Act of the AU [African Union]," he added.
The NGO forum brought together national and international NGOs and representatives of the United Nations and the European Commission. It was scheduled parallel to the African Union (AU) summit, which is taking place in Khartoum on Monday and Tuesday.
"The government's actions have undermined the credibility and authority of the AU at a time when it is meeting in the Sudanese capital," Olaniyan noted.
"Last night's events signal a worrying increase in the crackdown against human rights workers on the part of the Sudanese government, and we fear there is more to come," he warned.
The AU summit is scheduled to name a new president on Tuesday. Sudanese President Umar al-Bashir is the main candidate for the post.
However, human rights organisations warned that his election would damage the organisation's credibility as senior Sudanese officials, including the president, have been implicated in war crimes in Darfur.
The AU plays a critical role in the war-torn western Sudanese region both by mediating between the parties in the conflict and by deploying peacekeeping forces.