The host keeps the door.
Entry
The host opens one or more ways in. The member list keeps only a hash of each member's pass.
Invite links
The host makes single-use links that expire. Whoever opens one is in.
Approval
Anyone with the address can ask, with a note. The host approves or declines.
Your own site
Your site checks whatever you require — a login, a membership, a token — and registers the member's pass with the den.