Whether a century-old Mosman mansion is a heritage-listed site or just another apartment development opportunity has become a topic of controversy among residents.