I don't really have a problem with certain details left unexplained- IMO that's part of what made the 1st season so good. And in the case of something like Gilead's actual territory, I think it's safe to assume that it's in a constant state of flux. And there's a lot of the things the show does really well- it's genuinely moving at times and makes you empathize with characters who seem genuinely evil.
So yeah, it's not a perfect show and there's definitely a dip in quality once it's done with the events from the book but like you say, there's definitely more positive than negative things about the show. And though I agree that season 1 was the best, I enjoyed most of season 2 almost as much- the main turning point for me was when June goes to live with Commander Lawrence, partly because he's a miscast, overly cartoony character and it just basically marked the point in the show where the suspension of disbelief factor started being really noticeable.