I'm planning on multiclassing my elven druid to a rogue but am worried about the XP penalty. I know with human & half-elves that you do not get a XP penalty but i'm elven. Will I loose out on a lot of XP? Any suggestions? I'm a lvl 3 or 4 druid now.

You will get an XP penalty of 20%. You should have taken Rogue when you were still Druid 2. But now, if you really want to take Rogue levels, you can minimize the XP loss by staying Druid 3 and taking Rogue levels. That way the only level at which you will lose XP is the one where you're Druid 3/Rogue 1. Once you're Druid 3/Rogue 2 it will be gone again as long as you keep the difference between your Druid and Rogue levels smaller then 2. (meaning 0 or 1)