I would translate "aufpassen" more with "to attend" or "to watch over".

The literal analogue to "pass on" or "fit on" is "anpassen"; "auf" generally means "over", "on/to the top of".