The mages’ wing of the castle is actually a (mostly) separate building facing the palace proper. Probably the sensible way to have built it…
Sorry if I’ve been slow in replying to comments lately! I will try to remedy that as soon as I get a little time.