> This is exactly my point. They are not repeating experiments. Each one is a different one, and independent of the other, because the chemicals in each color are different. They are not testing what you think that they are testing.

They ran the same experiment 20 times with a meaningless variable changed until they got a positive result with a confidence of 95%. (i.e. 1/20 chance of a random positive result.)

The point of the comic is that this is bad science. Or at least it is ver bad science reporting. This kind of nonsense is why there is a multibillion dollar market for supplements that do not hold up to further testing.

> This is not how science works in the real life.

True. There is a lot of bad science in real life.

"Scientists bad because I don't understand statistics" is boring. Just because you decided that is a meaningless variable does not mean that it is.

P-hacking doesn't mean doing a lot of experiments. This is called working hard or going the extra mile. Working hard is not cheating. That the real scientists would do is frown upon the green jellybean result, think "hum, this is strange" and test further for effects of the green dye in other products. Science knowledge is in-cre-men-tal.

P-hacking is: changing the statistical model that you use to analyze the data, until you find one that gives you the desired (publishable) p-value.

P-hacking is manipulating the methodology in your favor and sometimes grooming the data also. All of that while ignoring other models available known to be more robust, precise and better for your particular case. This is why p-hacking is cheating.

-------------------------

Second. There are a market for fake supplements because, to express it crudely: people is idiot and the government does not care about them.

1) Some people didn't cared to learn even basic biology at the school, and will believe that the moon is made of cheese if it comes from somebody in their tribal team. Scientists take a lot of pain to assure that their knowledge is available to the public.

There are thousands of scientific articles about creatine. Never has been so easy in the history to find them. And with AI, you don't even need to read the whole horror anymore, to have a general idea of cons and pros before to jump in the water. Do your f*k homework before to gulp a mysterious pill. If not. Don't blame scientists later.

2) The government make laws. Is not scientists fault, if it allows to sell legally dubious supplements, or even more if politicians profit of it, (ehum, mandated testosterone tests for every soldier, ka-ching!).

Their job is to protect citizens from scammers. Most countries ban anything that is remotely medical and not proven effective, or at least advice strongly against taking it.