MIT is not copyleft. The copyright notice must be included for those incorporated elements, but other downstream code it remains part of can be licensed however it wants.

AGPL and GPL are, on the other hand, as you describe.

Modifications can be licensed differently but that takes extra work. If I release a project with the MIT license at the top of each file and you download my project and make a 1-line change which you then redistribute, you need to explicitly mark that line as having a different license from the rest of the file otherwise it could be interpreted as also being MIT licensed.

You also could not legally remove the MIT license from those files and distribute with all rights reserved. My original granting of permission to modify and redistribute continues downstream.