Output imposed sheets to multiple PDFs - last batch

[Mikeness]

Hi all,

 

We would like to begin imposing using FusionPro to simplify production but have run into an issue regarding stack size on the final batch.

 

The document is to be printed 2-up, duplex, and we have a total of 566 records. The shop has asked for stacks of 100 sheets per batch. We therefore would output 3 pdf files of 200, 200 and 166 records respectively. This works fine up until the last batch where the imposer continues to create a PDF of 100 sheets when only 83 are required.

 

I've tried activating the InfiniteStack option combined with "Output to multiple files" in the composition settings and constantly hit a wall due to chunk size.

 

What is the recommended way to go about setting this up? I would ideally like to avoid writing any script to do this, as it can make it difficult to pass onto production staff to manage.

 

Thanks!

[dreimer]

Hmmmm, that is interesting because FP does the exact thing you want normally for me? Are you stacking 1-100 in one stack and 101-200 in the other, or are you going across the sheet?

 

What does your imposition file look like? Can you post it?

[Mikeness]

We're stacking 1-100 then 101-200. I've posted our imposition file and a screenshot of our Output setting in Composition Settings. Perhaps I'm making a blatant error somewhere?

 

Appreciate the help!

TestImpose.fpi

[dreimer]

Hmm, for some reason the FPI file isn't coming across correctly. Can you do a screenshot of the three windows? (Document, Press Sheet, & Layout)

[Mikeness]

Here they are.

[dreimer]

Everything seems to look correct. Maybe try changing the Pages Per Record to 1. I never change that and not really sure what it does. Also, I would uncheck the Infinite stack button since you want only 100 per. Everything else looks correct to me.

[Mikeness]

If I uncheck the Infinite stack option, my last batch file will contain 100 sheets even though it only though I would only require 86 (batch size is 200, 200, 166 records per batch respectively @ 2-up duplex).

 

We basically need the software to adjust the last batch (each batch in a separate PDF file) to not waste any sheets.

[dreimer]

This is how Fusion Pro is designed, to use the least amount of paper.

 

Trying to reproduce your results and not getting it. My test has 250 records per and the first 4 files are 500 PDF pages and the last is 36.

[Mikeness]

Hmmm... something strange going on here. Can you post that imposition file? And perhaps a quick shot of your Composition setting too?

 

Big thanks.

[dreimer]

Hopefully the fpi file comes across for you unlike it did for me.

4Up_.125Gutter.fpi

2015-01-07 09_32_26-Greenshot.pdf

[dreimer]

Infinite stack checked for me makes Fusion Pro create one PDF and ignores my stack count. Do you have any chunking rules in your template?

[Mikeness]

No I do not have any chunking rules setup. We we're simply applying the "Record per file" option in composition.

 

I had a look at your imposition file and I can't seem to find a difference between your file and mine other than the infinite stack option.

 

When I remove the Infinite stack, the imposed final file, which contains 166 records, continues to output 100 sheets (or 200 pages duplex) rather than the 86 I need. 200 records is the max file output we've assigned. When I check the Infinite Stack option, I get an error when composing.

[dreimer]

That doesn't make much sense. So in the final file, how are the records stacked? Is there 100 in one stack and 66 in the other? Can you post?

[Mikeness]

I'll create something that I can post, but that seems to be the case. The file comes out printed 2-up for the first few sheets and then reverts to 1-up for the balance as there are not enough records to fill 2-up over the full stack.

[dreimer]

Do you have blank records in your file? Is your file 566 records or 600? Not sure I can be any more help. Something doesn't sound correct. What version of FP are you using? Maybe someone else has run into this and can chime in.

[miker]

This is familiar... Excel sometimes contains additional records where they appear to be blank. Copy/paste the data to a new workbook, or otherwise ensure that everything beyond the required records is deleted.

 

I've experienced this on multiple VDP platforms; it's not restricted to Fusion. It's entirely an Excel thing.

[dreimer]

This is kind of what I was thinking but if there are blanks the background artwork(if any) would still get imposed. That is why I was wondering if the OP could post the final output file.

[Mikeness]

I'm confident this is not the case, as the file is only processing the correct number of records. Also, no blanks are created when we Compose without using imposition. Perhaps the problem is not in the imposition file, but it's not caused by blank records (all data is actually exported from Access where blanks are easily seen).

 

I've created a template PDF for illustration purposes and attached it so you may see the output.

 

We're running FP 9.3.9 on Windows XP. Really want to get to the bottom of this as it is really prohibiting us from implementing FP in our production process.

ImposeTest-Output2.pdf

[dreimer]

Still trying to figure this one out! Kind of stumped here!

 

Try changing the vertical Step and repeat to none?

[dreimer]

I just ran a test similar to yours, used your same setting in the fpi file with the exception of the size and I took out the vertical step and repeat and this is my output files.

Test-Output1.pdf

Test-Output2.pdf

Test-Output3.pdf

[dreimer]

Tried the same leaving the vertical set to 1 in fpi file and same results. I think you need to call support.

[Mikeness]

I suppose I will have to. Can you please do me one last favor and post the PDF template and Imposition file?

 

Finally, perhaps you could also take a couple of shot of your composition settings tabs? This will help me to make sure that I am absolutely not missing anything.

 

Thanks a bunch!

[dreimer]

I collected the job.

Test_FP.zip

[Mikeness]

Thanks so much for collecting the job.

 

After nearly pulling all my hair out, I at least found the source of the issue, but it just creates another...

 

When imposing an FP project where a DSN file is my data source, the imposition file creates full stacks of 100, regardless of whether or not there is enough data to fill the stack. When the FP file is linked to a CSV (simply exporting the same table that is being read in the DSN - i.e. same data), the stacking works properly...

 

So, now that I know what's causing it, my obvious next question is why this would be a problem and if anyone knows of a workaround?

[Dan Korn]

When imposing an FP project where a DSN file is my data source, the imposition file creates full stacks of 100, regardless of whether or not there is enough data to fill the stack. When the FP file is linked to a CSV (simply exporting the same table that is being read in the DSN - i.e. same data), the stacking works properly...

 

So, now that I know what's causing it, my obvious next question is why this would be a problem and if anyone knows of a workaround?

Stacked imposition doesn't work properly with ODBC. This has been a known issue for some time. I will try to explain why.

 

For a regular, non-imposed composition (or even for a non-stacked imposition), FusionPro simply reads one record from the data source at a time and composes it, then moves on to the next record. It doesn't have to know ahead of time how many total records there are, nor remember anything from previous records (though it does keep one previous record's data in memory for the FieldChanged function).

 

But it's a completely different story for stacked imposition. Here, FusionPro composes sheets, and each sheet can have output pages from multiple, non-contiguous records. So FusionPro needs to have "random access" to the data records, so that it can basically compose them out of order. For instance, in a 4-up example with 100 records per stack, it has to first compose a sheet with data from records 1, 101, 201, and 301, then compose a sheet with data from records 2, 102, 202, and 302, etc. It also has to know how many total records there are to compose in order to know how many sheets and stacks there will be. It accomplishes this by first doing a "preprocessing" run through the data, which not only counts the records, but sets up a mapping of offsets for each record into the data file, so that when it gets to the main composition pass, it can quickly access any record of data by seeking to the offset for the record in the data file.

 

The problem with ODBC is that most* ODBC drivers support only a "one-way cursor" (aka a "forward-only cursor"), so once to run your SQL query and get back a data set, you can only start with the first record and keep getting the next record until the end. This is fine for a non-imposed job, since FusionPro only needs the records in "forward" order, but stacked imposition needs to access the records "out of order." Unfortunately, you can't just say to the ODBC system, "Hey, I know I just got record 1, but now I need record 101." ODBC responds, "Sorry Dave, but if you just got record 1, then you can only get record 2 next." There's no "random access" available. The only way you can get to an arbitrary record is to go all the way back to record 1 and keep asking for the next record until you get the one you want, and that's expensive in terms of both I/O resources and time.

 

(* Some ODBC drivers do support "random access," or "dynamic cursors," but it depends on the driver, and frankly, getting it to work even when the driver supports it is fairly difficult and convoluted.)

 

What FusionPro used to do for ODBC with stacked imposition was to read all of the data into memory, but that gets unwieldy with large data sets, and didn't really work anymore once we introduced the "dynamic stacking" feature in FusionPro 8, where you can call FusionPro.Composition.StartNewStack() at any arbitrary time in OnRecordStart.

 

So, this is somewhat of an intractable problem. ODBC simply doesn't give us the kind of access we need to the data in order to compose records out of order for stacked imposition.

 

Therefore, you will need to use a different type of data, such as a flat file. The good news is that, in FusionPro 9.2, you can use Excel and DBF data sources directly, without having to go through ODBC, on both Windows and Mac, and both of those data sources work just fine with stacking.