Depends on the state.

AZ:

>C. A merchant, or a merchant's agent or employee, with reasonable cause, may detain on the premises in a reasonable manner and for a reasonable time any person who is suspected of shoplifting as prescribed in subsection A of this section for questioning or summoning a law enforcement officer.

https://www.azleg.gov/ars/13/01805.htm

i.e. all they need is reasonable cause to suspect you are shoplifting. When I was detained no one ever saw me steal anything, I openly grabbed the cat litter, scanned it at the machine, paid for it, grabbed the receipt, then refused to show it to the receipt-checker (not about to slow down for that bullshit since it is now my property) so they just sent some dudes out to grab the cart out of my hands.