We do sort of have that with the capabilities stuff (although I admit hardly anyone knows how to use it).

But the tricky part is that "reading files" is done all the time in ways you might not think of as "reading files". For example loading dynamic libraries involves reading files. Making network connections involves reading files (resolv.conf, hosts). Formatting text for a specific locale involves reading files. Working out the timezone involves reading files.

Even just echoing "hello" to the terminal involves reading files:

  $ strace echo hello 2>&1 | grep ^open
  openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
  openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 3
  openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = 3
  openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
  openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
  openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) = 3
  openat(AT_FDCWD, "/proc/filesystems", O_RDONLY|O_CLOEXEC) = 3
  openat(AT_FDCWD, "/proc/self/maps", O_RDONLY|O_CLOEXEC) = 3
  openat(AT_FDCWD, "/usr/lib/cargo/bin/coreutils/echo/en-US.ftl", O_RDONLY|O_CLOEXEC) = -1 ENOTDIR (Not a directory)

OP says "restricted access to files". Read access to your home directory is not required for loading dynamic libraries or printing the time.