The CPAN 'Alien' infrastructure is great for this, you have pseudo-modules that you can depend on that use vendor packages if available and build the damn thing for you if not.

It's considered ... rude ... in most cases to write a module that needs to build against a native library without also having an Alien dist to handle making sure said library is there by the time it's trying to build against it.

Opinions on perl as a *language* ... vary, let us say ... but I wish people who didn't like writing perl would at least look at how our infrastructure works and steal more of the good parts to make dealing with their preferred language less painful.