There is a point where quality is already reached, and you buy based on what you like having in your listening room, and I find that anything above $2000 per component does start to have diminishing returns.
Due to different pieces sounding different, different rooms sounding different, and different pieces sounding different in different rooms, when you reach a certain price, you are best just listening to the gear in your listening room, and if you like it, well it's the system for you.
That said if you can afford a $20,000 system, you should probably look into room treatment, as you'll able to make massive changes, and still order that $2000 system and have it sound better than imaginable before the acoustic treatment.