There is not really anything in particular you should consider taking with you. Non-Jews are not expected to cover their heads unless they are in synagogue, in which case they should do so. Be aware the rabbi will have no problem shaking hands with males, but will not wish to do so with females.
