A long-neglected manor house in Hereford could find a new use as part of a neighbouring college.