No. Poverty is not relative to what others have. Poverty is relative to baseline needs. So poverty absolutely can exist without inequality.

Inequality has no moral characteristics. It is not "insidious." The fact that disparate effort and disparate circumstance lead to disparate outcomes is just that; a simple cold fact.

That some people use their power or wealth to take advantage of others is a moral issue. But again, this is orthogonal to disparity of outcome. Or do you claim poor people never steal from other poor people, or that rich people never steal from other rich people?

Envy is not the basis of poverty, need is.