you just don't give it to the boss in front of everyone else.
A good idea. I'd personally (and I'm after a mega favor from my boss right now) thank him/her by text message for picking me up (a thank you is always appreciated) give the present after the first day's work, and yes, definitely not in front of staff. It should be a (very) modest present from the OP's home country.