Aluminum is generally thought to be benign by most authorities.

However, it is worth noting that humans were not exposed to aluminum in the environment until relatively recently, when we started extracting it from bauxite and melting it.

Aluminum is a major component of all soils, and soluble aluminum salts have been used in medicine and dyeing since ancient Greece, probably since ancient Egypt. They have been used in food preparation for at least several centuries.

Pure aluminium maybe not, but it's a major component of clay.