We have a process in PlanetPress Watch that creates 18,000 single page PDF Files. It typically takes about 10-15 minutes to run. The other day the process seemed to run extremely slow. We did not have time to troubleshoot, so we rebooted quickly and ran the file while other things were processing. It finished in about 10-15 minutes so we thought it was just a one-time issue.

This morning we ran the process and it finished as usual in about 10 minutes. About an hour later we ran a similar file and it after about an hour it had only generated 1,000 or so pages. We thought it was the file, so we grabbed the original file that took 10 minutes this morning and re-ran it and found it was doing the same thing. Looking in the Image log, the SAME EXACT FILE this morning took less than a half second per page but this afternoon is taking 57 seconds.

We have rebooted twice. Checked hard drive room. Checked memory. The server hardware seems absolutely fine. All other processes on the system are running as normal. We actually shut off all processes and tested again with no better results. There are not any backups running and there are not any virus protection running. Basically, at this point our process works fine but just takes a tremendous time longer to run.

The server has dual Xeon chips with 24 logical processors and 24 gb of memory. This morning we had Watch Messenger Preference "Let me set how many instances to run" unchecked and we could see that 10 Alambic Processes running. This afternoon we checked this preference ON and set the instances to 20. At this point we can see all 20 are running but the speed is the same at about 57-58 seconds per PDF.

Here is the log entry from this morning:

-----------------------------------------------------------------------
REQST: PPImage Request - 11:10:39

INFO: Processing file C:\ProgramData\Objectif Lune\PlanetPress Suite 7\PlanetPress Image\QueuedJobs\3A53FCD.ps
DEBUG: Reading job options from input file
DEBUG: Watch Process Name : XDP2b - Live - Convert to PDF_00ZRLBZAOAEWN14
INFO: Starting conversion...
INFO: Initializing PlanetPress Alambic...
INFO: Distilling file to 2561&^&1334&^&X019.PDF...
INFO: Conversion completed successfully.
DEBUG: Moving file to destination folder: C:\data\Xdp2b\watch\xoDocs\output
INFO: Process completed successfully.

REQST: End time: 11:10:40 (elapsed time: 00:00:00:497)


Here is the log entry from this afternoon:

-----------------------------------------------------------------------
REQST: PPImage Request - 6:06:27 PM

INFO: Processing file C:\ProgramData\Objectif Lune\PlanetPress Suite 7\PlanetPress Image\QueuedJobs\511A7E4.ps
DEBUG: Reading job options from input file
DEBUG: Watch Process Name : XDP2b - Live - Convert to PDF_00ZRLF109X88Q3A
INFO: Starting conversion...
INFO: Initializing PlanetPress Alambic...
INFO: Distilling file to 2575&^&1334&^&X019.PDF...
INFO: Conversion completed successfully.
DEBUG: Moving file to destination folder: C:\data\Xdp2b\watch\xoDocs\output
INFO: Process completed successfully.

REQST: End time: 6:07:25 PM (elapsed time: 00:00:57:548)
-----------------------------------------------------------------------

Is there anything in PlanetPress that would cause it to go into some sort of slow mode? Maybe fighting processes or something? We really have no clue would could be causing this.

Thanks,
Brandon

This was extremely hard to figure out but we found the issue. The process that runs has a dynamic image on the page. The image is a PDF file. The PDF is generated from another app and can be small (1 page) or large (15,000 pages).

When we ran the process the first time and it ran really fast, the image resource PDF was around 1,000 pages (by mistake). The second time we ran the process, it was 15,000 pages. It was suppose to be the same PDF but evidently the first time we ran it there was a problem.


So, here is what we have learned:

If you generate a PDF and you have an image resource on the page that references a another PDF (think for the background), the time it takes (per page) is directly affected by the number of pages in the referenced PDF. It is enormously faster to use 18,000 (1) page PDF's as background images (referencing page 1 obviously) than to use (1) 18,000 page PDF as a background image (referencing a different page each time).

BWW,

You're right that opening a very large PDF for each of your pages and seeking that large PDF for a single page is very inefficient, and your solution is better (and you can split the PDF using Workflow and the PDF tools, so there's that).

In these sorts of situations, for example if splitting the PDF isn't an option, we generally recommend using the Alambic API (aka PDF API) in order to "stamp" one PDF onto the other, one page after another. This sort of implementation is scripted, and since it loads the PDFs at the start and only once, it's very fast and efficient.