Most of the problem is the "only been a week" part, likely. Though you're fighting an algorithm that's been patched in inconsistent places for all sorts of weights like "authority" and "quality".

Thousands of little weights driven by obscure attributes of the site that you're not really going to figure out by thrashing and changing stuff.