Ok here is the crucial part of the paper:

It's a difference in differences design, using individual-level test scores and de-seasonalized data (p. 13). Their wording is:

> Y_igst is the outcome of interest for student i in grade g in school s in time period t, HighAct_s is an indicator for high pre-ban smartphone activity schools, D_t is a series of time period dummies (t = 0 indicates the first period after the ban took effect), δ_s is school fixed effects, and θ_g is grade fixed effects. In this setting, β_t are the parameters of interest, reflecting the difference in the outcome of interest between treatment and comparison schools for each period, with the period before the ban serving as the omitted category, holding grade level constant.

To me some modeling choices seem a bit heavy-handed, but I'm not an economist and could not do better.

what it means is that this paper shows probable causality and models a lot of interesting features. it is most definitely not flawed.

i think the tough thing is that 0.6 percentage points gain for the average student is quite small. it's actually less than you gain by studying for 1h for the SAT, which is probably about 0.9 percentage points, depending on how you interpret college board's research (it recommends 20h of studying). that is to say, if students studied one fucking hour for the FAST, they would probably get a bigger benefit on it than all the time they get back not looking at their phones throughout two years of school.

so whatever cell phone use (1) in school (2) causes, it causes a small effect on test scores.

you would have to pick some other objective criteria, for example mental health assessment, for maybe a larger effect, or seek a larger treatment, perhaps a complete ban of cell phones period, to observe a larger effect.