Contact list best practices

Experiencing some slowness issues wondering if deleting old contacts is a what we should be doing. Have hundreds of thousands of contacts that are super cold leads maybe they are slowing down the system? Wondering if others are archiving or purging the old contacts?

Thanks