This behavior is fairly general to Unix! A cute thing on Linux is that there are also references to open files under /proc, including synthetic kernel-generated filesystem links to the files, even if they've been deleted.

So if you delete a file from the filesystem that's open by PID 12345, you can find a reference to that file still present in /proc/12345/fd, and you can actually make a new copy of the file with cp or something.

And also when deleting a file that's on NFS, but it's reference count is non-zero in the kernel (because at least one process has it open) you end up renaming not deleting the file.