I normally ride with a helmet. The only time I don't wear one is if I am quickly jumping on bareback to walk up from the bottom paddock mounted, or if I take my helmet off quickly to scratch my head. That's it.
For kids, helmet 100% of the time no ifs or buts! Even if you are just putting little Johnny on the tiny pony's back for a three second photo. Even if you are holding the kid. It only takes one tiny shy for the kid to end up under the horse with a crushed skull. I don't care how old or quiet the horse is. I am quite, erm, passionate about it - I even wrote a post about it a while ago.
But adults - it's your head I guess. If you have kids at home that depend on you to take care of them, wear a fricking helmet. If you have no back up plan for your animals if anything happens to you, wear a fricking helmet. If you don't want to end up dead or crippled, wear a fricking helmet. But again, your life, your head, your choice. I won't rant about it to you! :)