The quotes are from Peoples of Middle Earth at the time of T.A. 1409,
|"In Rhudaur an evil folk, workers of sorcery, subjects of Angmar, slay the remants of the Dúnedain and build dark forts in the hills."|
"Cardolan is forsaken. The deserted mounds of Cardolan become filled with deadly spirits; but in Rhudaur for long there dwelt an evil people out of the North much given to sorcery."
This appears in the MERP group forum, in a post about Angmar. IMO the "evil people out of the North" maybe former Eriadorians driven away by Númenoreans in the S.A. which still kept their hate against those who took their fertile lands. Since the lossoth where dúnedain-friendly (at least they were not hostile and they didn't have knowledge about sorcery because if so, they probably had killed king Arvedui in 1975 T.A. and bring the palantir to the Witch-King), maybe they lived near Angmar, to the north of Arnor and to the south of lossoth lands.