That stupid voice keeps on saying "You must gather your party before venturing forth" my party is gathered but I can't pass through the gate. What is this?

You have to talk to the warden in front of the door first. If you show him your invitations to the crowning ceremony you can enter the building.

So I do, thanks!