You’ve either gotta pivot the event from a party (where attendance is considered optional) to a specific gathering/dinner event or something, where people will feel like attendance is more expected. Or, have a frank conversation with your guests about your worries and expectations and see what comes with it.