I don’t think it’s that clear-cut.
If you have a graphically complex layout that depends on pixel-perfect alignment of images between different blocks, for example, it’s incredibly important that it looks exactly the same in every browser because it looks like a bug if it doesn’t.
But if IE renders your column 15 pixels wider than Firefox and Safari doesn’t support the background color on your drop-down select boxes, will anyone notice except you? Does anything else break? If not, who cares?
The best web designs aren’t huge, pixel-perfect monstrosities based on some insane PSD designed by someone who doesn’t know what implementation will require. They’re simple, flowing, and resilient. They won’t break if the content-length changes and the right column is longer than the left. They won’t break when someone over age 40 views the site and magnifies the text to 150% because they can’t read your trendy 11px Verdana. And they certainly won’t break if IE slips a few pixels into the margin somewhere.
Instead of wasting hours upon hours to hammer out every little browser difference in an overly complex design, just design it to accommodate browser differences in the first place.