Cosmetic issues like dirt and decor are easily remedied and shouldn't spoil you on an otherwise good home. However, a few things that could be potential deal breakers for buyers are location, floor plan, bone structure, and health and safety issues.
There are many things you can change about a home, but location is not one of them. A property in a location that requires too long a commute; is in an unsafe neighbourhood; lacks quality schools and other amenities; or has some other barrier to quality of life or resale value (such as noise or odour) is one you may regret purchasing.
A floor plan that doesn't work given your particular needs and lifestyle can make life more difficult on a day to day basis. A home that is harder to live in isn't one where you will want to stay for very long. For buyers who love to entertain, for example, that too tiny kitchen and closed concept floor plan just won't work.
Problems with a properties "bones" include its foundation, walls, floors, roof, plumbing system, or electrical system can be very costly to fix and have a significant impact on resale value. Being able to walk away from a property with compromised structural integrity is why a home inspection condition is so important.
Health & Safety
Just as home inspections can uncover structural problems, they can turn up health & safety issues too. Some, like the lack of ground fault circuit interrupters (GFCI's), are minor and easily addressed. Other issues such as black mold, which can be toxic, or faulty wiring, which can cause fires, are serious and should not be overlooked by buyers.