I understand that, apart from any discussion of or pursuit of objective morality...
Communities, organizations, and/or tribes often have certain values or practices they collectively agree to abide by. That's essentially a social contract. When someone violates that social contract, they imperil the rest of the community. Holding that violator accountable isn't so much about appealing to an objective, transcendent, foundational moral principle as it is holding them accountable to their word or agreement.
So, if that's what you're getting at, I agree.