They affect different things. You want more powerful models -> more memory. You want the models that run to be faster -> more gpu cores.