Depends on the amount of focus you dedicate to your listening. But unlike reading, it's much easier to use it as background activity.

Also, I'm very much convinced that the brain is distracted away from the content by the voice acting and intonation; same way that most people physically can't concentrate when listening to music with vocals, evolution made us really sensitive to the human voice.