When you are no longer in the position to manage the membership for an organization, you should transfer the ownership to somebody who will deal with that responsibility.
That person must be a member of the organization and should be aware of the action and consequences.
Go to the specific organization overview page and navigate to the Members tab:
Use the Transfer Ownership button
for the person who will become the owner:
Ready & certain? Click on Confirm Transfer
Your role is changed to Developer, and the other person now has the role of Owner.
Your view on the organization is updated and reflects your new role, you no longer have membership management options: