I think you have to think Noir in tone and mood more, and not necessarily image wise.
That's true, although for me "Film Noir" is all about shadows and accentuated dark/light contrasts. Consequently, something like Casablanca makes it into my definition of noir even though for many it isn't.
With Film Noirs (although possibly excluding neo-noirs), I tend to pay more attention to who the cinematographer is than who the director is.