Can you just replace parts as you go with a PC? Yes, but not indefinitely. There will become a point where you would likely have to replace almost all of the components in a PC in order to upgrade, like when a new RAM or CPU socket type is introduced.

I’m not sure if this is still true, but in the very recent past you could find better deals on DDR5 RAM when buying it in a bundle with a CPU. A lot of stores did this as a way to try to prevent scalping.