Yes, apt caches stuff. I'm not actually sure what benefit it has and why it does so. I don't think you can turn off the behavior - but you can clean up after it. You have a couple of options to do so easily. Well, you have a zillion options - but I only go through a couple.
linux-tips.us
Feedback just is alright by me.

How To: Clear The apt Cache • Linux Tips
In today's article, we're going to learn how to clear the apt cache. This is only useful if your distro uses the apt package manager.

Feedback just is alright by me.