There really are two different kinds of variable data submission!

There are two completely different forms of variable data handling in  the Harlequin RIP®, and I’m sometimes asked why we’ve duplicated functionality like that. The simple answer is that it’s not duplication; they each address very different use cases.

But those use cases are not, as many people then expect, “white paper workflows” vs imprinting, i.e. whether the whole design including both re-used and single-use elements is printed together vs adding variable data on top of a pre-printed substrate. Both Harlequin VariData™ and the “Dynamic overlays” that we added in Harlequin version 12 can address both of those requirements.

Incidentally, I put “white paper workflows” in quotes because that’s what it’s called in the transactional and direct mail spaces … but very similar approaches are used for variable data printing in other sectors, which may not be printing on anything even vaguely resembling paper!

The two use cases revolve around who has the data, when they have it, whether a job should start printing before all the data is available, and whether there are any requirements to restrict access to the data.

When most people in the transactional, direct mail or graphic arts print sectors think about variable data it tends to be in the form of a fully resolved document representing all of the many variations of one of a collection of pages, combining one or more static ‘backgrounds’ with single-use variable data elements, and maybe some re-used elements from which one is selected for each recipient. In other words, each page in the PDF file is meant to be printed as-is, and will be suitable for a single copy. That whole, fully resolved file is then sent to the press. It may be sent from one division of the printing company to the press room, or even from some other company entirely. The same approach is used for some VDP jobs in labels, folding carton, corrugated, signage and some industrial sectors.

This is the model for which optimized PostScript, and then optimized PDF, PDF/VT (and AFP) were designed. It’s a robust workflow that allows for significant amounts of proofing and process control at multiple stages. And it also allows very rich graphical variability. It’s the workflow for which Harlequin VariData was designed, to maximize the throughput of variable data files through the Digital Front End (DFE) and onto the press.

But in some cases the variable data is not available when the job starts printing. Indeed, the print ‘job’ may run for months in situations such as packaging lines or ID card printing. That can be managed by simply sending a whole series of optimized PDF files, each one representing a few thousand or a couple of million instances of the job to be printed. But in some cases that’s simply not convenient or efficient enough.

In other workflows the data to be printed must be selected based on the item to be printed on, and that’s only known at the very last minute … or second … before the item is printed. A rather extreme example of this is in printing ID cards. In some workflows a chip or magnetic strip is programmed first. When the card is to be printed it’s obviously important that the printed information matches the data on the chip or magnetic strip, so the printing unit reads the data from one of those, uses that to select the data to be printed, and prints it … sometimes all in less than a second. In this case you could use a fully resolved optimized PDF file and select the appropriate page from it based on identifying the next product to be printed on; I know there are companies doing exactly that. But it gets cumbersome when the selection time is very short and the number of items to be printed is very large. And you also need to have all of the data available up-front, so a more dynamic solution is better.

Printing magnetic strip on ID cards
Printing magnetic strip on ID cards.

In other cases there is a need to ensure that the data to be printed is held completely securely, which usually leads to a demand that there is never a complete set of that data in a standard file format outside of the DFE for the printer itself. ID cards are an example of this use case as well.

Printing Example ID cards

Moving away from very quick or secure responses, we’ve been observing an interesting trend in the labels and packaging market as digital presses are used more widely. Printing the graphics of the design itself and adding the kind of data that’s historically been applied using coding and marking are converging. Information like serial numbers, batch numbers, competition QR Codes, even sell & use by dates are being printed at the same time as the main graphics. Add in the growing demands for traceability, for less of a need for warehousing and for more print on demand of a larger number of different versions, and there can be some real benefits in moving all of the print process quite close to the bottling/filling/labelling lines. But it doesn’t make sense to make a million page PDF file just so you can change the batch number every 42 cartons because that’s what fits on a pallet.

These use cases are why we added Dynamic overlays to Harlequin. Locations on the output where marks should be added are specified, along with the type of mark (text, barcodes and images are the most commonly used). For most marks a data source must be specified; by default we support reading from CSV files or automated counters, but an interface to a database can easily be added for specific integrations. And, of course, formatting information such as font, color, barcode symbology etc must be provided.

The ‘overlay’ in “Dynamic overlays” gives away one of the limitations of this approach, in that the variable data added using it must be on top of all the static data. But we normally recommend that you do that for fully resolved VDP submissions using something like optimized PDF anyway because it makes processing much more efficient; there aren’t that many situations where the desired visual appearance requires variable graphics behind static ones. It’s also much less of a constraint that you’d have with imprinting, where you can only knock objects like white text out of a colored fill in the static background if you are using a white ink!

For what it’s worth, Dynamic overlays also work well for imprinting or for cases where you need to print graphics of middling complexity at high quality but where there are no static graphics at all (existing coding & marking systems can handle simple graphics at low to medium quality very well). In other words, there’s no need to have a background to print the variable data as a foreground over.

So now you know why we’ve doubled up on variable data functionality!

Further reading:

  1. What’s the best effective photographic image resolution for your variable data print jobs?
  2. Why does optimization of VDP jobs matter?

To be the first to receive our blog posts, news updates and product news why not subscribe to our monthly newsletter? Subscribe here

Follow us on LinkedIn and Twitter

 

What’s the best effective photographic image resolution for your variable data print jobs?

It goes without saying that the final quality of your printed piece is paramount. But when speed and time constraints are also critical, what can you do to ensure your files fly through the press and still reward you with the quality you expect? Optimizing the images in the piece is a good place to start, but if you’re creating a job with variable data, where there are thousands of pages to print, each with a different image, how do you know what a sensible effective resolution is for those images that will ensure your PDF file doesn’t trip up the print production workflow?

In his latest guide, Full Speed Ahead, how to make variable data PDF files that won’t slow your digital press, Martin Bailey, CTO at Global Graphics Software, advises not to ask the print workflow to do more work than necessary if that doesn’t change the look of the printed result. Images are commonly re-used within a VDP job, so being able to process each image only once and then re-use the result many times can significantly increase the throughput of the digital front end. On the other hand, some images are personal to every recipient and must therefore be processed for every single recipient, slowing the workflow down.

Martin offers the following tips for setting appropriate effective photographic image resolutions:

  1. Aim for 300 ppi, however the most appropriate image resolution for digital presses varies, depending on printing heads, media and screening used.
  2. Bear in mind image content; soft and dreamy images can be sometimes placed at a lower resolution.
  3. Don’t use a higher effective image resolution for photographic images than the output resolution as this is often not productive. The example in Fig 1 below illustrates how easy it is to use an image at several times the required resolution:

The same 12-megapixel image at 3 different sizes

Fig 1: The same 12-megapixel (4000 x 3000 px) image placed on the page at three different sizes. Source: Full Speed Ahead, how to make variable data PDF files that won’t slow your digital press.

When an image is placed onto a page the original resolution of that image is largely irrelevant; what matters is how many pixels there are per inch on the final printed page. As an example, if you have a photograph from a 12 MP compact camera it’ll probably be approximately 3000 pixels by 4000 pixels. If that’s placed on the page as 3 inches by 4 inches (7.5 x 10cm) the effective resolution is about 1000ppi (4000/4). That would usually be about three times as much as you need in each dimension.

A variety of tools are available for optimizing image resolution, and some composition tools can also do this automatically. To find out more about the best effective resolution for your images, and to pick up more tips for optimizing your images for variable data printing, download the guide:

Full Speed Ahead: how to make variable data PDF files that won't slow your digital press edited by Global Graphics Software

Full Speed Ahead – how to make variable data PDF files that won’t slow your digital press.

Be the first to receive our blog posts, news updates and product news. Why not subscribe to our monthly newsletter? Subscribe here

Follow us on LinkedIn and Twitter

 

Why does optimization of VDP jobs matter?

Would you fill your brand-new Ferrari with cheap and inferior fuel? It’s a question posed by Martin Bailey in his new guide: ‘Full Speed Ahead – how to make variable data PDF files that won’t slow your digital press’. It’s an analogy he uses to explain the importance of putting well-constructed PDF files through your DFE so that they don’t disrupt the printing process and the DFE runs as efficiently as possible. 

Here are Martin’s recommendations to help you avoid making jobs that delay the printing process, so you can be assured that you’ll meet your print deadline reliably and achieve your printing goals effectively:

If you’re printing work that doesn’t make use of variable data on a digital press, you’re probably producing short runs. If you weren’t, you’d be more likely to choose an offset or flexo press instead. But “short runs” very rarely means a single copy.

Let’s assume that you’re printing, for example, 50 copies of a series of booklets, or of an imposed form of labels. In this case the DFE on your digital press only needs to RIP each PDF page once.

To continue the example, let’s assume that you’re printing on a press that can produce 100 pages per minute (or the equivalent area for labels etc.). If all your jobs are 50 copies long, you therefore need to RIP jobs at only two pages per minute (100ppm/50 copies). Once a job is fully RIPped and the copies are running on press you have plenty of time to get the next job prepared before the current one clears the press.

But VDP jobs place additional demands on the processing power available in a DFE because most pages are different to every other page and must therefore each be RIPped separately. If you’re printing at 100 pages per minute the DFE must RIP at 100 pages per minute; fifty times faster than it needed to process for fifty copies of a static job.

Each minor inefficiency in a VDP job will often only add between a few milliseconds and a second or two to the processing of each page, but those times need to be multiplied up by the number of pages in the job. An individual delay of half a second on every page of a 10,000-page job adds up to around an hour and a half for the whole job. For a really big job of a million pages it only takes an extra tenth of a second per page to add 24 hours to the total processing time.

If you’re printing at 120ppm the DFE must process each page in an average of half a second or less to keep up with the press. The fastest continuous feed inkjet presses at the time of writing are capable of printing an area equivalent to over 13,000 pages per minute, which means each page must be processed in just over 4ms. It doesn’t take much of a slow-down to start impacting throughput.

If you’re involved in this kind of calculation you may find the digital press data rate calculator at https://blog.globalgraphics.com/tag/data-rate/ useful:

Global Graphics Software’s digital press data rate calculator.
Global Graphics Software’s digital press data rate calculator.

This extra load has led DFE builders to develop a variety of optimizations. Most of these work by reducing the amount of data that must be RIPped. But even with those optimizations a complex VDP job typically requires significantly more processing power than a ‘static’ job where every copy is the same.

The amount of processing required to prepare a PDF file for print in a DFE can vary hugely without affecting the visual appearance of the printed result, depending on how it is constructed.

Poorly constructed PDF files can therefore impact a print service provider in one or both of two ways:

  • Output is not achieved at engine speed, reducing return on investment (ROI) because fewer jobs can be produced per shift. In extreme cases when printing on a continuous feed (web-fed) press a failure to deliver rasters for printing fast enough can also lead to media wastage and may confuse in-line or near-line finishing.
  • In order to compensate for jobs that take longer to process in the DFE, press vendors often provide more hardware to expand the processing capability, increasing the bill of materials, and therefore the capital cost of the DFE.

Once the press is installed and running the production manager will usually calculate and tune their understanding of how many jobs of what type can be printed in a shift. Customer services representatives work to ensure that customer expectations are set appropriately, and the company falls into a regular pattern. Most jobs are quoted on an acceptable turn-round time and delivered on schedule.

Depending on how many presses the print site has, and how they are connected to one or more DFEs this may lead to a press sitting idle, waiting for pages to print. It may also delay other jobs in the queue or mean that they must be moved to a different press. Moving jobs at the last minute may not be easy if the presses available are not identical. Different presses may require different print streams or imposition and there may be limitations on stock availability, etc.

Many jobs have tight deadlines on delivery schedules; they may need to be ready for a specific time, with penalties for late delivery, or the potential for reduced return for the marketing department behind a direct mail campaign. Brand owners may be ordering labels or cartons on a just in time (JIT) plan, and there may be consequences for late delivery ranging from an annoyed customer to penalty clauses being invoked.

Those problems for the print service provider percolate upstream to brand owners and other groups commissioning digital print. Producing an inefficiently constructed PDF file will increase the risk that your job will not be delivered by the expected time.

You shouldn’t take these recommendations as suggesting that the DFE on any press is inadequate. Think of it as the equivalent of a suggestion that you should not fill your brand-new Ferrari with cheap and inferior fuel!

 

Full Speed Ahead: how to make variable data PDF files that won't slow your digital press edited by Global Graphics Software

The above is an excerpt from Full Speed Ahead: how to make variable data PDF files that won’t slow your digital press. The guide is designed to help you avoid making jobs that disrupt and delay the printing process, increasing the probability of everyone involved in delivering the printed piece; hitting their deadlines reliably and achieving their goals effectively.

DOWNLOAD THE FREE FULL GUIDE HERE: https://bit.ly/fsa-pdf

To be the first to receive our blog posts, news updates and product news why not subscribe to our monthly newsletter? Subscribe here

About the author:

Martin Bailey, CTO, Global Graphics Software
Martin Bailey, CTO, Global Graphics Software

Martin Bailey first joined what has now become Global Graphics Software in the early nineties, and has worked in customer support, development and product management for the Harlequin RIP as well as becoming the company’s Chief Technology Officer. During that time he’s also been actively involved in a number of print-related standards activities, including chairing CIP4, CGATS and the ISO PDF/X committee. He’s currently the primary UK expert to the ISO committees maintaining and developing PDF and PDF/VT.

 

To be the first to receive our blog posts, news updates and product news why not subscribe to our monthly newsletter? Subscribe here

Follow us on LinkedIn and Twitter

Full Speed Ahead: How to make variable data PDF files that won’t slow your digital press

The use of variable data has increased exponentially over the past five years and is emerging in new applications such as industrial inkjet. Yet poorly designed variable data PDF files disrupt production and reduce ROI.

Watch the recent webinar with Global Graphics Software’s CTO Martin Bailey, the author of Full Speed Ahead, a new guide to offer advice to anyone with a stake in variable data printing, including graphic designers, print buyers, production managers, press operators, composition tool developers and users.

In the webinar Martin presents an overview of the guide and highlights some of the key tips and tricks for graphic designers, prepress and print service providers, showing how, when they all work together, VDP jobs can fly through digital presses.

Sponsored by Delphax Solutions, Digimarc, HP Indigo, HP PageWide Industrial, HYBRID Software, Kodak, Racami and WhatTheyThink, the guide is a practical format for easy reference and includes:

• Tips and tricks for making fast, efficient PDF files for variable data printing
• Helpful illustrations, photos and explanatory diagrams
• Real examples from industry

You can download your copy here: https://www.globalgraphics.com/full-speed-ahead

For further reading about PDF documents and standards:

  1. PDF Processing Steps – the next evolution in handling technical marks

To be the first to receive our blog posts, news updates and product news why not subscribe to our monthly newsletter? Subscribe here

Follow us on LinkedIn and Twitter

Future-proofing your digital press to cope with rising data rates

When we hear the phrase “big data”, we’re meant to think of extremely large data sets that are too complex to process in traditional ways. But, in the context of the next generation of digital presses, you’d be forgiven for thinking it refers to the ultra-high data rates required to drive them.

For example, consider a typical narrow-web label press: 13 inches (330mm) wide, 4 colors, 600x600dpi, running at 230 fpm (70m/min). This requires 0.9 GB/s of raster data to drive it at its rated speed.

Assuming next year’s press adds three more colors (Orange, Green and White) and is upgraded to 1200x1200dpi and expected to run a little faster at 330 fpm (100m/min), the required data rate will jump to 8.6 GB/s: almost a factor of ten increase!

Already this is a data rate far in excess of what the fastest solid-state drives can manage, so what hope is there for a traditional disk-based workflow when moving to 20 inches wide, duplex or 200m/min? Clearly, any part of the workflow involving a disk drive is going to become a bottleneck.

Ditch the disk with Direct
Ditch the disk. Rather than write intermediate raster files to disk between RIPping and screening, or between screening and the printhead electronics, everything takes place in memory.

This was one of the reasons behind the creation of Direct™, the integrated software pipeline we announced at the end of April. Rather than write intermediate raster files to disk between RIPping and screening, or between screening and the printhead electronics, everything takes place in memory.

There’s more to future-proofing your press than eliminating comparatively slow disk accesses, however. You’ll need a system that’s scalable and built from the fastest components, which is why Harlequin Direct™ is composed from a configurable number of Harlequin Host Renderer™ and ScreenPro™ instances working in parallel to make the best of the most powerful desktop PCs available.

When it comes to adding new colors or supporting duplex, the scalability extends to multiple Harlequin Directs across multiple PCs, one per print bar.

When it comes to adding new colors or supporting duplex, the scalability extends to multiple Harlequin Directs across multiple PCs, one per printbar.

An added advantage of this approach is that each printbar need not use the same resolution or drop-count etc. For example, you might wish to use a lower resolution and disable color management for white or varnish. Our Press Operator Controller user interface is supplied to manage your configuration, along with submitting and controlling your print jobs.

Our Press Operator Controller user interface is supplied to manage your configuration, along with submitting and controlling your print jobs.

The beauty of a software-only solution like Direct is that once you have built it into your workflow, you are free to upgrade your PCs over time for greater performance without any further software integration expense. A Direct-based system will evolve as your needs evolve, making it the ideal choice for future-proofing your next digital press.

For more information about Direct, please visit globalgraphics.com/direct.

To be the first to receive our blog posts, news updates and product news why not subscribe to our monthly newsletter? Subscribe here

Ian Bolton, Product Manager, Direct
Ian Bolton, Product Manager, Direct

About the author:
Ian has over 15 years’ experience in industry as a software engineer focusing on high performance. With a passion for problem-solving, Ian’s role as product manager for the Direct range gives him the opportunity to work with printer OEMs and break down any new technology barriers that may be preventing them from reaching their digital printer’s full potential.

Time for an update on VDP!

Over the last fifteen years variable data in digital printing has grown from “the next big thing” with vast, untapped potential to a commonly used process for delivering all manner of personalized information. VDP is used for everything from credit card bills and bank statements to direct mail postcards and personalized catalogues, from college enrolment packs to Christmas cards and photobooks, from labels to tickets, checks to ID cards.

This huge variety of jobs is created and managed by an equally huge variety of software, from specialist composition tools to general purpose design applications carefully configured for VDP. And they are consumed by workflows involving (or even completely within) the Digital Front End (DFE) for a digital production press, where jobs must be imposed, color managed.

Time, then, to update our popular “Do PDF/VT Right” guide which has had thousands of downloads since it was first published in 2014 not to mention the number of printed copies distributed at trade shows and industry events.

Do PDF/VT Right - How to make problem-free PDF files for variable data printing
Do PDF/VT Right – How to make problem-free PDF files for variable data printing

In addition to a general overhaul there is a new section on the new ISO 21812 standard that allows workflow controls to be added to PDF files, and notes on Harlequin-specific hints, to get even more speed out of your DFE if you are a Harlequin user.

The goal remains the same: to provide a set of actionable recommendations that help you ensure that your jobs don’t slow down the print production workflow … without affecting the visual appearance that you’re trying to achieve. As a side benefit, several of the recommendations set out below will also ensure that your PDF files can be delivered more efficiently on the web and to PDF readers on mobile devices in a cross-media publishing environment.

Some of the recommendations made in this guide are things that a graphic designer can apply quickly and easily, using their current tools. Others are intended more for the software companies building composition tools. If all of us work together we can greatly reduce the chance of that “heart-attack” job; the one that absolutely, positively must be in the post today … but that runs really slowly on the press.

Download your copy here .

Simple VDP support

VDP is a topic that has the potential to get people very excited. We are no exception. For instance we were delighted when Mark Andy told us that our technology reduces process and RIP times on the Digital Series HD by 50% even with full color, every-page-different.

Confident of the benefits print shops would experience if they could take on higher premium personalised jobs, we made sure from the early days, that our technology would be a) able to handle variable data in “regular” flavour PDFs by intelligent rendering b) be PDF/VT compliant (since IPEX 2010) and capable of high-speeds without sacrificing quality. And now there’s a new development that we’ve introduced this year with the launch of Version 12 of the Harlequin Host Renderer in April 2018.

What about when your VDP workflow doesn’t really benefit from PDF/VT but needs a lighter weight solution for adding text, graphics and barcodes?
Harlequin Host Renderer 12 now supports Dynamic Overlays for these use cases. Some applications such as packaging, labels and industrial print, require a simple form of VDP support. This might be where a single background page is combined with overlay graphics that are selected on the basis of a data file supplied in a format like CSV. Serial and batch codes can be added using dynamic counters without writing values to a CSV first. Support has been added to apply overlays on top of a single page PDF file to add simple serial numbers on labels or QR codes for personalized URLs, postal barcodes and addresses on envelopes.

Secure tickets

This secure ticket is generated with in-RIP bar-code support where data is read dynamically from a CSV file.

The example shows:

  •  A complex guilloche pattern in the background
  • Two lines of micro text identifying the recipient by name
  • A QR Code encoding a personalized URL (PURL)
  • The Global Graphics ‘g’ is painted in the centre of the QR Code
  • A six-character code in which each character is drawn with one of six different colours

Folding cartons

The background for this image comprises three folding cartons using nested imposition.

The overlay includes:

  • The first name of the recipient in large white text with a silver border
  • The full name of the recipient together with their city and state
  • A line of microtext showing their full name repeated to fill the space available
  • A QR Code recording a personalized URL (PURL), with a Global Graphics logo placed over the centre of it
  • The flag for the state of the recipient

 

 

 

 

 

 

Why online print is set to change

Martin Bailey, CTO, Global Graphics Software

Remember in olden times how you sent a file to print on a wing and a prayer? OK, it wasn’t that bad!  But it was unreliable. Figures showed that print-ready file delivery had failure rates of between 30 – 70% and this was a real problem for print service providers with high throughput like magazine houses.

Then PDF/X came along and greatly improved the situation. It was strengthened by additional standardisation efforts from several other bodies including Ghent PDF Workgroup and Altona.

PDF/X worked because it ensured delivery of files ready to high-quality print. And because it dealt with the headache so well, print service providers recommended it.

Fast forward for a moment to today and to the tidal wave that is variable data printing. Most buyers deliver the brief and the dataset to the print service provider (PSP). A full service PSP will offer data mining, graphic design, composition and print. Offering a full service promises higher margins. If you only provide a print service you can expect lower margins, but your model connects better to web-to-print services that are burgeoning. But if you try to “just print” VDP jobs, those that fail will eliminate profit.

I’ve been invited to speak at the Online Print Symposium in Munich (17th – 18th March) about why PDF/VT and Industry 4.0 are set to change online print forever. The truth is that VDP has been hanging around street corners looking for a PDF/X. Well, now it’s found one because that’s what PDF/VT is. It’s been created to deal with every page being different and to give PSPs more control over the workflow.

I would go so far to suggest that print-ready file delivery of graphically rich variable data from outside the print company is unlikely to succeed without it!  And on that bombshell…!

To receive a copy of my presentation email me. Meanwhile take a look at http://www.globalgraphics.com/doPDFVTright

PDF/VT – bringing all the advantages of PDF workflow to the world of variable data printing

Martin Bailey, CTO, Global Graphics Software

Standards for variable data printing (VDP) have come a long way since the first work by CGATS to develop a universal delivery format in the late 1990s. In 2010 the International Standards Organization published the PDF/VT standard, marking the first really effective specification for a reliable, vendor-neutral exchange of variable data jobs, both within and between companies.

A special type of the PDF file format, PDF/VT is specifically used for variable data and transactional printing in a variety of environments, from desktop printing to high volume digital production presses. Built on PDF/X, it therefore brings all the advantages of that standard in enforcing best practices for reproducible and predictable color and appearance to the variable data and transactional print worlds.

The industry is gradually realizing its value to improve quality, competitiveness and productivity, and I’ve been working with the PDF/VT Competence Center, especially with Christoph Oeters (Sofha), Paul Jones (Teclyn bv) and Tim Donahue (technical consultant) to produce a new set of Application Notes highlighting the benefits of using PDF/VT and the workflows that it enables.

The Application Notes explain how to make the highest quality and most efficient PDF/VT files to achieve the required visual appearance of a job, so if you develop software to read and write PDF/VT files, for example in composition tools, RIPs, digital front ends and imposition tools, or if you work on print workflow integration, you’ll find the notes really beneficial. They also show how document part metadata can be applied and leveraged for VDP specific production workflows.

Of course, there are wider benefits to using PDF/VT: The adoption of PDF/VT will allow the industry to finally move towards a reliable, vendor-neutral exchange of variable data jobs, simplifying the process of variable data printing significantly.

The application notes are free to download, pick up your copy here: http://www.pdfa.org/publication/pdfvt-application-notes/.

Let me know what you think of them – feedback is always welcome.

Additional reading:
Do PDF/VT right

Do PDF/VT right by Global Graphics Software

 

PDF/VT for personalized print by the PDF/A Competence Center

Read the press release from the PDF Association (http://www.pdfa.org/2016/01/pdf-association-publishes-pdfvt-application-notes-showcasing-the-benefits-for-variable-data-print-streams/)