I've seen it twice (first time by myself when it came out in the UK, and second with a group of mates this weekend). Not bad, but not great.
It would have been a fairly average film and less enjoyable than Thor, Iron Man (not IM2 because that was a bit poor) and Captain America if it wasn't for Hulk.
While the film had some signposted jokes, nothing made the audience as a whole laugh out loud until a scene with Loki and Hulk. Mark Ruffalo played Bruce Banner so well - the repartee between Banner and Stark was on a par with that between Loki and Hulk - that without such a character and performance the film may otherwise have felt flat.
Hawkeye and Black Widow could have been excluded from the film and it wouldn't have suffered in the slightest.