Processing PDF without in-RIP color management

Twenty years ago it was common to find people RIPping jobs for production print with no color management. Indeed, many print service providers (PSPs), magazine publishers etc actively avoided it as being “too complicated” and “unpredictable”. You might read that as an indictment of their vendors for a lack of investment either in developing good product or in educating their users. Alternatively it might simply show that the printing companies were quite understandably risk-averse because it could be expensive if the client didn’t like the resulting color, especially in an environment like display advertising in a major magazine, or packaging for a major brand.

A decade after that more and more people (on both the buying and the printing sides) grasped the value of color management in print and were using it, but there was still a significant minority that had not managed to make the time to understand it. This is borne out by the uproar when Adobe ‘forced’ people to use color management by changing from using CMYK for the alternate color space for Pantone spots in Creative Cloud to using Lab1, and by the continuing demand for support for PDF/X‑1a, where everything has already been converted to press colorants before the PDF is made.

Now we’re in 2022, and the need for color management is accepted almost universally in print sectors that use an ink set based on CMYK. I phrased it that way because some of the industrial print space (textiles, ceramics, laminate flooring etc) have historically used many inks, but usually job-specific rather than CMYK. Some of those markets will continue to use job-specific ink sets as they transition to digital, while others would find a switch to digital extremely challenging without a concurrent switch to a color managed workflow2.

So, why am I writing this now?

It’s because I still talk to people who tell me that they don’t need to do any color management inside the RIP when processing PDF; they RIP it first and then apply color management.

I’m sorry, that just won’t work reliably and with maximum quality.

There was a time, back in the days when PDF 1.3 was the latest and greatest (which pretty much means last millennium) when a PSP could get away with this approach, because their customers were happy to define all their colors in CMYK and spots. As soon as they used anything else, including Lab or colors tagged with ICC profiles, they’d have to have some fallback to generate CMYK values from that data. It doesn’t need a full color management module (CMM), but they’d need something.

And then along came PDF 1.4, adding transparency. And transparency requires that you can convert colors between color spaces, potentially multiple times. That’s because PDF transparency includes the concept of transparency groups. Each group is one or more graphics that are blended with any graphics that are behind them in the design.

The blending depends on a number of parameters, the most obvious of which are the blend mode (Overlay, Multiply, Hard Light etc), and the blend color space. The result of rendering all graphics that are underneath the transparency group will be transformed from whatever space the RIP holds it in (often the CMYK for the output device) into the blending color space. The result of rendering all the graphics inside the transparency group itself is also transformed into the blending color space. Then the blend mode is applied, to do the actual transparency calculation, and the result is transformed back into whatever color space the RIP needs it to be in for further processing (again, often the CMYK of the output device). The blending color space is quite often sRGB, because that’s the default in a number of popular design applications.

So correct rendering of the transparency will often require color transforms between the color space in which graphics are specified (such as, maybe, an image tagged with an ECI RGB ICC profile), the blend color space (most commonly sRGB) and the output device color space (usually a specific CMYK). That’s just not possible without applying a pretty complete color management process during RIPping. And if you try to take short-cuts you’ll usually get a visually different result, sometimes very different.

Color transformation with transparency requires a full color management capability.

Even so, back in the early 2000s a PSP could avoid the need to upgrade software, process control and operator training by insisting that their customers supplied files in a format such as PDF/X-1a, which prohibited device-independent colors and transparency. But making a PDF/X-1a file from a rich design in a creative application requires a number of compromises affecting graphical elements that were originally specified in device independent colors, or which use transparency. Both risk degrading the quality of the final piece.

These days insisting on PDF/X-1a to avoid the need for color management in the RIP is no longer widely acceptable to customers3. And therefore neither is color managing after the RIPping is complete.

Your check-list is therefore:

  • Don’t use PDF/X-1a. In fact don’t use PDF/X-3 either. Both are two decades old. PDF/X-3 may allow device-independent colors, but both of them force the creation tool to flatten transparency, discard layers and a bunch of other potentially damaging procedures. It’s over ten years since PDF/X-4 was published, and that’s currently the best balance between capability and getting too far ahead of common usage in print workflows.
  • If you’re a print service provider, converter, industrial printing manufacturer or digital press vendor, don’t cut corners; use a workflow that applies the color management in or before the RIP4. It shouldn’t be hard; all the leading RIP vendors (and therefore leading press vendors, because they license technology from the RIP vendors) supply suitable systems.

About the author

Martin Bailey, consultant and former 0CTO, Global Graphics Software

Martin Bailey, consultant at Global Graphics Software, is a former CTO of the company and currently the primary UK expert to the ISO committees maintaining and developing PDF and PDF/VT. He is the author of Full Speed Ahead: how to make variable data PDF files that won’t slow your digital press, a guide offering advice to anyone with a stake in variable data printing including graphic designers, print buyers, composition developers and users.

To be the first to receive our blog posts, news updates and product news why not subscribe to our monthly newsletter? Subscribe here

Follow us on LinkedIn,  Twitter and YouTube

Notes
1 – If a spot color will be emulated using process inks on press, then using a CMYK alternate gives predictable color numbers in those inks, but is less good at producing a predictable color appearance. Using Lab for the alternate color space often leads to unpredictable color numbers on each separation, but a more predictable color appearance on the print. There is a benefit to both models, but when it comes to paying for printing the color appearance usually wins!

2 – if run-lengths on digital are long enough to justify warehousing a variety of inks, and changings inks on inkjet presses, it can be reasonable to stay with job-specific ink sets, especially if it’s difficult or expensive to make usable inks for all of CMY and K. As an example, the best Magenta ink for inkjet printing on ceramics is made with gold. Any move to using digital presses for short-run printing more or less requires a fixed ink set to allow for quick job changes without excessive waste, and that typically means CMYK+.

3 – and I say that as the chair of the committees that developed PDF/X for many years, first in CGATS and then in ISO.

4 – There are situations where applying color management in a color server before the RIP can be useful, especially when multiple presses will be used in parallel. This approach brings its own challenges around handling spot colors in the job that will be emulated on press, but can produce excellent results when used with care.

Working with spot colors in Harlequin Core

Whenever we start working with a company who’s interested in using Harlequin Core™ for their Digital Front End (DFE), there are always three technical topics under discussion: speed, quality and capabilities. Speed and quality are often very quick discussions; much of the time they’ve approached us because they’re already convinced that Harlequin can do what they need. In the remaining cases we tend to jointly agree that the best way for them to be convinced is for them to take a copy of Harlequin Core and to run their own tests. There’s nothing quite like trying something on your own systems to give yourself confidence in the results.

So that leaves capabilities.

If the company already sells a DFE using a different core RIP they will almost always want to at least match, and usually to extend, the functionality of their existing solution when they switch to Harlequin. And if they’re building their first DFE they usually have a clear idea of what their target market will need.

At that stage we start by ensuring that we all understand that Harlequin Core can deliver rasters in whatever format is required (color channels, interleaving, resolution, bit depth, halftoning) and then cover color management pretty quickly (yes, Harlequin uses ICC profiles, including v4 and DeviceLink; yes, you can chain multiple profiles in arbitrary sequences, etc).

Then we usually come on to a series of questions that boil down to handling spot colors:

  • Most spot separations in jobs will be emulated on my digital press; can I adjust that emulation?
  • Can I make sure that the emulation works well with ICC profiles for different substrates?
  • Can I include special device colorants, such as White and Silver inks in that emulation?
  • Can I alias one spot separation name to another?
  • Can I make technical separations, like cut and fold lines, completely disappear, without knocking out if somebody upstream didn’t set them to overprint?
  • Alternatively, can I extract technical separations as vector graphics to drive a cutter/plotter with?

Since the answer to all of those is ‘yes’ we can then move on to areas where the vendor is looking for a unique capability …

But I’ve always been slightly disappointed that we don’t get to talk more about some of the interesting corners of spot handling in Harlequin. So I created a video to walk through some examples. Take a look, and I’d welcome your comments and questions!

Further reading:

  1. Channelling how many spot colors?!!
  2. Shade and color variation in textile printing
  3. Harlequin Core – the heart of your digital press
  4. What is a raster image processor 

Be the first to receive our blog posts, news updates and product news. Why not subscribe to our monthly newsletter? Subscribe here

Follow us on LinkedInTwitter and YouTube

Looking to reduce errors with simple job management, keep control of color, and run at ultra-high speed for jobs with variable data?

With just a few days to Labelexpo Europe 2019, preparation is in full swing. Come along to booth 9A15 where we’ll be previewing a new version of Fundamentals™, our toolkit for building a digital front end.

Fundamentals is a collaboration between Global Graphics Software and HYBRID Software – and its beauty lies in its simplicity: Fundamentals 2.0 makes it easy for the press operator to keep control of the workflow. Easy step and repeat and nesting via STEPZ with award-winning VDP composition from HYBRID Software makes it possible to estimate and plan single or multi-gang jobs and see how the output will appear when printed, helping to reduce errors and wasted media.

Consistent and predictable color for a wide range of design and creation workflows using industry-standard tools is achieved with Harlequin ColorPro ™  and there’s support for ICC profiles, including device ink and N-channel profiles too.

ScreenPro™, the award-winning multi-level screening engine, streams data directly to the print electronics at press speed, unlocking maximum productivity on variable data jobs to process ultra-high data rates with the reliability required to maximize press up time.

Find out more about Fundamentals here: www.globalgraphics.com/fundamentals

We’ll be on stand 9A15 – please stop by and say hello. If you’d like to book a time to chat, simply contact us: sales@globalgraphics.com. We look forward to seeing you.

Join us on stand 9A15 at Labelexpo2019

What do you really need in a RIP to drive a digital press in labels & packaging?

In this latest post, Global Graphics CTO Martin Bailey goes back to basics and explores what you need in a RIP to drive a digital press for labels & packaging.

Martin highlights rendering your jobs correctly, color management with CMYK inks and spot colors, PDF layering and technical separations, and provides a high-level view of the features of the Harlequin RIP® for digital labels and packaging.

Watch Martin’s presentation here:

Follow us on Twitter: @Global_Graphics

Channelling how many spot colors?!!

Martin Bailey, CTO, Global Graphics Software
Martin Bailey, CTO, Global Graphics Software

Recently my wife came home from a local sewing shop proudly waving a large piece of material, which turned out to be a “swatch book” for quilting fabrics. She now has it pinned up on the wall of her hobby room.

It made me wonder how many separations or spot colors I’d ever seen in a single job myself … ignoring jobs specifically designed as swatches.

I think my personal experience probably tops out at around 18 colors, which was for a design guide for a fuel company’s forecourts after a major redesign of their branding. It was a bit like a US banknote: lots of colors, but most of them green!

But I do occasionally hear about cases where a print company or converter, especially in packaging, is looking to buy a new digital press. I’m told it’s common for them to impose together all of their most challenging jobs on the grounds that if the new press (or rather, the DFE on the new press) can handle that, then they can be confident that it’ll handle any of the jobs they receive individually. Of course, if you gang together multiple unrelated jobs, each of which uses multiple spot colors, then you can end up with quite a few different ones on the whole sheet.

“Why does this matter?” I hear you ask.

It would be easy to assume that a request for a spot color in the incoming PDF file for a job is very ephemeral; that it’s immediately converted into an appropriate set of process colors to emulate that spot on the press. Several years ago, in the time of PostScript, and for PDF prior to version 1.4, you could do that. But the advent of live transparency in PDF made things a bit harder. If you naïvely transform spots to process builds as soon as you see them, and if the spot colored object is involved in any transparency blending, then you’ll get a result that looks very different to the same job being printed on a press that actually has an ink for that spot color. In other words, prints from your digital press might not match a print from a flexo press, which is definitely not a good place to be!

So in practice, the RIP needs to retain the spot as a spot until all of the transparency blending and composition has been done, and can only merge it into the process separations afterwards. And that goes for all of the spots in the job, however many of them there are.

Although I was a bit dismissive of swatches above, those are also important. Who would want to buy a wide format printer, or a printer for textiles, or even for packaging or labels, if you can’t provide swatches to your customers and to their designers?

All of this really came into focus for me because, until recently, the Harlequin RIP could only manage 250 spots per page. That sounds a lot, but wasn’t enough for some of our customers. In response to their requests we’ve just delivered a new revision to our OEM partners that can handle a little over 8000 spots per page. I’m hoping that will be enough for a while!

If you decide to take that as a challenge, I’d love to see what you print with it!

Getting to know PDF 2.0: halftones

Are you ready for PDF 2.0? Register now for the PDF 2.0 interoperability workshops in the UK and USA.

Martin Bailey, CTO, Global Graphics Software
Martin Bailey, CTO, Global Graphics Software

In the middle of 2017 ISO 32000-2 will be published, defining PDF 2.0. It’s eight years since there’s been a revision to the standard. In his next blog post about the changes afoot, Martin Bailey, the primary UK expert to the ISO committee developing PDF 2.0, looks at halftones, an area where the new specification will offer significant benefits for flexo jobs.

Lists of spot functions in halftones
PDF allows a PDF file to specify the halftone to be used for screening output in a variety of ways. The simplest is to identify a spot function by name, but that method was constrained in versions of the PDF standard up to PDF 1.7 to use only names that were explicitly listed in the specification itself. This has been a significant limitation in print sectors where custom halftones are common, such as flexography, gravure … and pretty much everywhere apart from offset plate-making!

PDF 2.0 allows the PDF file to specify the halftone dot shape as a list of spot function names, and those names no longer need to be picked from the ones specified in the standard. The renderer should use the first named spot function in the list that it supports. This allows a single file to be created that can be used in a variety of RIPs that support different sets of proprietary halftones and to select the best one available in each RIP for that specific object.

This functionality is expected to be used mainly for high-quality flexo press work, where it’s a key part of the workflow to specify which halftone should be used for each graphical element.

A PDF 1.7 reader will probably either error or completely ignore the screening information embedded in the PDF if a file using the new list form is encountered. In the flexo space that could easily cause problems on-press, so take care that you’ve upgraded your RIPs before you start to try rendering PDF files using this new capability.

Halftone Origin (HTO)
Very old versions of PDF (up to PDF 1.3) included a partial definition of an entry named HTP, which was intended to allow the location of the origin or phase of a halftone to be specified. That entry was unfortunately useless because it did not specify the coordinate system to apply and it was removed many years ago.

PDF 2.0 adds a new entry called HTO to achieve the same goal, but this time fully specified. The use case is anywhere where precise specification of the halftone phase is valuable. Examples include pre-imposed sheets for VLF plate-setters, where specifying the halftone phase for each imposed page can reduce the misalignment of halftones that can occur over very long distances, or setting the halftone phase of each of a set of step-and-repeat labels to ensure that the halftone dots are placed in exactly the same position relative to the design in each instance.

A PDF 1.7 reader will simply ignore the new key, so there’s no danger of new files causing problems in an older workflow. On the other hand, those older RIPs will render as they always have, which would be a missed opportunity for the target use cases.

Halftone selection in transparent areas
Up to PDF 1.7 there has been a requirement to apply the “default halftone” in all areas where transparency compositing has been applied. This was problematic for those print technologies where different halftones must be used for different object types to achieve maximum quality, e.g. for flexo. Transparency is used in these jobs most commonly for drop shadows, so that’s where you’re most likely to have encountered problems.

PDF 2.0 effectively gives complete freedom to renderers to apply the supplied screening parameters in whatever way they see fit, but two example implementations are provided to encourage similarity between implementations. One of those matches the requirements from PDF 1.7, while the other applies the screen defined for the top-most graphical element in areas where transparency was applied. The second one means that the screening selected for the drop shadow will now be used, matching requirements for the flexo market.

The background
The last few years have been pretty stable for PDF; PDF 1.7 was published in 2006, and the first ISO PDF standard (ISO 32000-1), published in 2008, was very similar to PDF 1.7. In the same way, PDF/X‑4 and PDF/X‑5, the most recent PDF/X standards, were both published in 2010, six years ago.

In the middle of 2017 ISO 32000-2 will be published, defining PDF 2.0. Much of the new work in this version is related to tagging for content re-use and accessibility, but there are also several areas that affect print production. Among them are some changes to the rendering of PDF transparency, ways to include additional data about spot colors and about how color management should be applied.

Getting to know PDF 2.0 – update from Down Under

Are you ready for PDF 2.0? Register now for the PDF 2.0 interoperability workshops in the UK and USA.

Martin Bailey, CTO, Global Graphics Software
Martin Bailey, CTO, Global Graphics Software

I’ve been in the ISO PDF committee meeting in Sydney, Australia for a couple of days this week to review the comments submitted to the most recent ballot on PDF 2.0. Over 100 comments were received, including some complex issues around digital signatures, structure tagging (especially lists), optional content, document parts and soft masks. In all cases the committee was able to reach a consensus on what should be done for PDF 2.0.

The plan is now for one more ballot, the responses to which will be reviewed in Q2 next year, with an expectation that final text for PDF 2.0 will be delivered to ISO for publication shortly thereafter.

So we’re still on track for publication next year.

All of which means that it’s past time that a couple of PDF’s unsung heroes were acknowledged. The project leaders for PDF 2.0 have invested very substantial amounts of time and mental energy updating text in response to comments and ballots over the last several years. When somebody like me requests a change it’s the project leaders who help to double-check that every last implication of that change is explored to ensure that we don’t have any inconsistency.

So a big thank you to Duff Johnson of the PDF Association and Peter Wyatt of CISRA (Canon)!

It’s also worth noting that one of the significant improvements in PDF 2.0 that probably won’t get highlighted elsewhere is that the text now is much more consistent. When you’re writing a detailed technical document 1000 pages long it’s inevitable that some disconnections between different sections will creep in. PDF 2.0 illustrates the value of a broad group of people from many countries and many industries reviewing text in the ISO process: we’ve managed to stamp on many of those cases in this new version.

Getting to know PDF 2.0: rendering PDF transparency

Are you ready for PDF 2.0? Register now for the PDF 2.0 interoperability workshops in the UK and USA.

ads_spread

In the middle of 2017 ISO 32000-2 will be published, defining PDF 2.0.  It’s eight years since there’s been a revision to the standard. In the second of a series of blog posts Martin Bailey, the primary UK expert to the ISO committee developing PDF 2.0, looks at the changes to rendering PDF transparency for print.
These changes are all driven by what we’ve learned in the last few years about where the previous PDF standards could trip people up in real-world jobs.
Inheritance of transparency color spaces
Under certain circumstances a RIP will now automatically apply a color-managed (CIEBased) color space when a device color space (such as DeviceCMYK) is used in a transparent object. It will do that by inheriting it from a containing Form XObject or the current page.
That sounds very technical, but the bottom line is that it will now be much easier to get the correct color when imposing multiple PDF files from different sources together. That’s especially the case when you’re imposing PDF/X files that use different profiles in their output intents, even though they may all be intended for the same target printing condition. The obvious examples of this kind of use case is placing display advertising for publications, or gang-printing.
We’ve tried hard to minimize impact on existing workflows in making these improvements, but there will inevitably be some cases where a PDF 2.0 workflow will produce different results from at least some existing solutions, and this is one case where that could happen. But we believe that the kinds of construct where PDF 2.0 will produce different output are very uncommon in PDF files apart from in the cases where it will provide a benefit by allowing a much closer color match to the designer/advertiser’s goal than could be achieved easily before.
Clarifications on when object colors must be transformed to the blend color space
The ISO PDF 1.7 standard, and all previous PDF specifications were somewhat vague about exactly when the color space of a graphical object involved with PDF transparency needed to be transformed into the blending color space. The uncertainty meant that implementations from different vendors could (and sometimes did) produce very different results.
Those statements have been greatly clarified in PDF 2.0.
This is another area where an upgrade to a PDF 2.0 workflow may mean that your jobs render slightly differently … but the up-side is that if you run pre-press systems or digital presses from multiple vendors they should now all be more similar to each other.
As a note to Harlequin RIP users, the new rules are in line with the way that Harlequin has always behaved; in other words, you won’t see any changes in this area when you upgrade.
ColorDodge & Burn
It tends to be taken for granted that the older PDF specifications must match what Adobe® Acrobat® does, but that’s not always correct. As an example, the formulae for the ColorDodge and ColorBurn transparency blending modes in the PDF specification have never matched the implementation in Acrobat. In pursuit of compatibility Harlequin was changed to match Acrobat rather than the specification many years ago. In PDF 2.0 the standard is finally catching up with reality and now both Acrobat and Harlequin will be formally ‘correct’!
The background
The last few years have been pretty stable for PDF; PDF 1.7 was published in 2006, and the first ISO PDF standard (ISO 32000-1), published in 2008, was very similar to PDF 1.7. In the same way, PDF/X 4 and PDF/X 5, the most recent PDF/X standards, were both published in 2010, six years ago.
In the middle of 2017 ISO 32000-2 will be published, defining PDF 2.0. Much of the new work in this version is related to tagging for content re-use and accessibility, but there are also several areas that affect print production. Among them are some changes to the rendering of PDF transparency, ways to include additional data about spot colors and about how color management should be applied.

Sign up to the Global Graphics newsletter here for regular updates.

Getting to know PDF 2.0

Are you ready for PDF 2.0? Register now for the PDF 2.0 interoperability workshops in the UK and USA.

Just when you’ve all cozied down with PDF 1.7 what happens?  Yes, that’s right.  A new standard rears its head.

Around the middle of 2017 the ISO committee will publish PDF 2.0 (ISO 32000-2). So by the end of 2017 you’ll probably need to be considering how to ensure that your workflow can handle PDF 2.0 files correctly.

As the primary UK expert to this committee I thought I’d give you a heads up now on what to expect.  And over the coming months via this blog and our newsletter I’ll endeavor to keep you posted on what to look out for as far as print is concerned.  Because, of course, there are many aspects to the standard that do not concern print at all.  For instance there are lots of changes in areas such as structure tagging for accessibility and digital signatures that might be important for business and consumer applications.

As you probably already know, in 2008 Adobe handed over ownership and development of the PDF standard to the International Standards Organization.  Since that time I’ve been working alongside other experts to ensure that standards have real-world applicability.

And here’s one example relating to color.

The printing condition for which a job was created can be encapsulated in professional print production jobs by specifying an “output intent” in the PDF file. The output intent structure was invented for the PDF/X standards, at first in support of pre-flight, and later to enable color management at the print site to match that used in proofing at the design stage.

But the PDF/X standards only allow a single output intent to be specified for all pages in a job.

PDF 2.0 allows separate output intents to be included for every page individually. The goal is to support jobs where different media are used for various pages, e.g. for the first sheet for each recipient of a transactional print job, or for the cover of a saddle-stitched book. The output intents in PDF 2.0 are an extension of those described in PDF/X, and the support for multiple output intents will probably be adopted back into PDF/X-6 and into the next PDF/VT standard.

But of course, like many improvements, this one does demand a little bit of care. A PDF 1.7 or existing PDF/X reader will ignore the new page level output intents and could therefore produce the wrong colors for a job that contains them.
In my next post I’ll be covering changes around live transparency in PDF 2.0.  Bet you can’t wait!
You can sign up to the Global Graphics newsletter here.

The background
The last few years have been pretty stable for PDF; PDF 1.7 was published in 2006, and the first ISO PDF standard (ISO 32000-1), published in 2010, was very similar to PDF 1.7. In the same way, PDF/X 4 and PDF/X 5, the most recent PDF/X standards, were both published in 2010, six years ago.

In the middle of 2017 ISO 32000-2 will be published, defining PDF 2.0. Much of the new work in this version is related to tagging for content re-use and accessibility, but there are also several areas that affect print production. Among them are some changes to the rendering of PDF transparency, ways to include additional data about spot colors and about how color management should be applied.