With that criteria
King Crimson - moonchild is probably my number one pick even though I really really like the song component in the first 2-3 minutes and hence it's nowhere near the worst song I have ever heard. It is however one of the weakest song I hear often and the song I most want to skip.
Number 2 would be Bob Dylan - Highlands - If you minus that track Time out of mind may well be Dylan's greatest album
I guess other choices are Beatles Revolution no. 9, Radiohead fitter happier (1.5 minute of fluff in an otherwise impeccable album), the Pet Sounds's instrumentals, Can - Aumgm and Peking O (even though I do think they both have artistic merit in terms of atmospheric and being genuine scary to listen to)
None of these songs are absolutely atrocious but they are all tarnish great albums IMO.