I used to do clean installs exactly for that reason: to cleanup all of the packages that I had installed and didn't use anymore.

Nowadays, I tend to just update, so that I don't have to reconfigure everything, and only do cleanup of packages if they are big and I'm short on space.

OTOH, now that I'm starting to use vcsh and mr, getting my configuration and my files on a new machine is getting easier, so I might just start doing a fresh installation now and then (not every debian release, however, maybe every other one or so).