There are better explanations than the vanilla call card of "conspiracy theory". One of them is that it is easy to get disorientated around there due a combination of many similar small islands in some areas and remote open ocean in another section, so it would be difficult to navigate by sight.

There are also widespread reports of magnetic anomalies which would mess with compasses, and it is within the hurricane zone providing another possible cause.