"The area they have entered is a national park in the DRC and I think here they will be able to access water and animals for food," he said in Kampala, the Ugandan capital.
He said the development posed a new "inconvenience" for Uganda.
Uganda and Sudan's defence chiefs have begun discussing joint operations against the LRA, Uganda’s State House, the official home of the president, announced on Sunday. It said Sudanese President Omar el-Bashir had made the proposal to Uganda's President Yoweri Museveni in New York, through Sudan's foreign minister, Mustafa Osman Ismail.
Bantariza said the LRA rebels now in the DRC were led by the movement's second in command, Vincent Otti.
"Otti and others including a senior commander called Odhiambo have taken flight to DRC from the pressure we had mounted on them east of the River Nile," Bantariza said.
Other dissident Ugandan forces - the Allied Democratic Forces and the National Army for the Liberation of Uganda - have been in the DRC for years. The Congolese government has said their presence "would no longer be tolerated", and has given all foreign armed groups till 30 September to disarm and leave the country or face "serious consequences".
Bantariza said that LRA leader Joseph Kony was also on the run but he may be heading west.
Kony's fighters are accused of forcibly recruiting children and using them as sex slaves. Up to 1.6 million Ugandans have been displaced by the fighting.