RandomPHPDev,
Unfortunately, it is not possible to use a different file for the preview. And yes, a 35-50 mb pdf file will most definitely take time to process. It isn't recommended to use files that are that large.
The best thing you can do is try to regenerate the image into a smaller-sized one. Save it to a .ps file, and if you have Distiller, try to regenerate it using a lower version of pdf. You can also adjust the resolution. The lower, the faster, but of course, you might lose quality in the process if you make it too low.
It's worth trying. I've seen some pretty large pdf files becoming about five times smaller by doing this, and without losing quality. Depending on the tools you use to build the pdf in the first place, the pdf can sometimes contain more stuff than it needs to(libraries, codes, etc...). Regenerating it may elimitates these, and makes the file smaller.
Regards,
Raphaël Lalonde Lefebvre