Hey there, I've run into this a couple of times. The general default from a Mac that I use is ANIMATION codec (RGB + ALPHA if needed); this will give you a .mov file. Encoder should recognize it though. "Animation" codec should be installed with After Effects no matter what operating system.
If file size is an issue... Apple pro-res (another AE default codec) is an option (again RGB + ALPHA if needed). These are smaller file sizes than Animation codec; but Encoder should recognize these files too.
Encoder should recognize MPEG-4 but when it doesn't I use the above options.
Hope this helps.
#XPression