Gee this sounds exactly like what I'll be doing on Tuesday. As a mediator for NARI I get assigned a lot of cases when the contractor and their customer have an unresolved dispute. All volunteer work. The case I'm going to this week involves a window replacement job of about 30+ windows that the homeowner says are leaking but the contractor says are fine. This is what I'll be looking at (you guys wanna come along?):
1. Are the windows installed square & level
2. Can I see visible daylight between the sash and frame (homeowners claims this is the case with several of them)
3. Is the weatherstrip present and in the correct lengths without kinks or gaps
4. Are the tolerances and margins OK
5. On double hungs do the checkrails align properly
6. Does everything work properly - screens, operation, wash feature
7. Are the units properly sealed/caulked
There are other things I can check for to a limited degree. I'm not an employee of the companies that I mediate for, so I'm not going to pull off trim, etc. But I also try to discern if the windows are properly insulated and shimmed. I'll bring a digital infrared thermometer (when it's cold) to measure the temperature at various points of the window (glass, bottom of sash, side of sash, corners, etc.) - temperatures should be consistent from one window to another if all conditions are equal (air circulation on inside, sunlight/solar heat per elevation). If there's a leaky one when it's cold, those temperatures could show up as different than the others.
Fortunately I'm not stuck in a position of trying to protect the manufacturer or installer since I'm a neutral party and I'm not selling them anything (In fact, I CAN'T sell them anything, that would be a violation of the Ethics committee). If the contractor would take care of their customers it shouldn't usually have to go as far as getting me or NARI involved anyway. I just want to resolve the issue, the benefit being that the contractor can get final payment (if any), the homeowner can get what they paid for, and all parties can have closure and move on to other things in life.
Actually I don't do it totally for free - I DO get a free golf outing out of the deal every August . . .