PDF Processing Steps – the next evolution in handling technical marks

Best practice in handling jobs containing both real graphic content and ‘technical marks’ has evolved over the last couple of decades. Technical marks include things like cut/die lines, fold lines, dimensions, legends etc in a page description language file (usually PDF these days). Much of the time, especially for pouches, folding carton and corrugated work, they’ll come originally from a CAD file and will have been merged with the graphics.

People will want to interact with the technical marks differently at various stages in the workflow:

  • Your CAD specialists will want to see the technical marks and make sure that they’ve not been changed from the original CAD input.
  • Brand owner approval may not want to see the technical marks, but prepress and production manager approvers will definitely want to see both the technical marks and the graphics together on their monitors, with the ability to make layers visible or invisible at will.
  • In some workflows the technical marks from the PDF may be used to make a physical die, or to drive a laser cutter; in others an original CAD file will be used instead.
  • On a digital press you may wish to print a short run of just the technical marks, or a combination of technical marks and graphics to ensure that finishing is properly registered with the prints.
  • The main print run, whether on a conventional press (flexo, offset, etc) or digital, will obviously include the graphics, but won’t include most of the technical marks. You may want to include the legend on the print as fool-proof identification of that job, but you’ll obviously need to disable printing of any marks that overlap with the live area or bleed, such as cut and fold marks.
  • Occasionally you may wish to do another short run with technical marks after the main print run, to ensure that finishing has not drifted out of register.

So there are a lot of places in the entire process where both technical marks and graphics may need to be turned on or off. How do you do that in your RIP?

Historically, the first method used to include technical marks, originally in PostScript, but now also in PDF, was to specify each kind of technical mark in a ‘technical separation’, encoded as a spot color in the job. Most operators tried to use a name for that spot color that indicated its intent, but there weren’t any standards, so you could end up with ‘Cut’ (or ‘CUT’, ‘cut’ etc), ‘cut-line’, ‘cut line’, ‘cutline’, ‘die’ etc etc. And that’s just thinking about naming in English. The names chosen are usually fairly meaningful to a human operator, but couldn’t be used reliably for automated processing because of the amount of variation.

As a result, many jobs arriving at a converter, at least from outside of that company, must be reviewed, and the spot names replaced, or the prepress and RIP configured to use the names from that job. That manual processing takes time and introduces the potential for errors.

But let’s assume you’ve completed that stage; how do you configure your RIP to achieve what you need with those technical separations?

The most obvious mechanism to turn off some technical marks is to tell the RIP to render the relevant spot colors as their own separations, but then not to image them on the print. It’s a very simple model, which works well as long as the job was constructed correctly, with all of the technical marks set to overprint. When somebody upstream forgot and left a cut or fold line as knockout (which never happens, of course!) you’d get a white line through the real graphics if the technical mark was on top of them.

The next evolution of that would be to configure the RIP to say that the nominated spot separation should never knock out of any other separation. That’s a configuration option in Harlequin RIPs but may not be widely available elsewhere.

Or you could tell the RIP to completely ignore one or more nominated spot colors, so they have no effect at all on any other marks on the page. Again, that’s a configuration option in Harlequin RIPs, and is one of the best ways of managing technical marks that are saved into the PDF file as technical separations.

Alternatively, since technical marks (like many other parts of a label or packaging job) are usually captured in a PDF layer (or optional content group to use the technical term), you can turn those layers on and off. Again, there are rich controls for managing PDF layers in Harlequin RIPs.

But none of these techniques get away from the need to manually check each file and set up prepress and the RIP appropriately for the spot names or layers that have been used for technical marks.

And that’s where the new ISO standard, 19593-1:2018 comes in. It defines “PDF processing Steps”, a mechanism to uniquely identify technical marks in PDF files, along with their intended function, from cutting to folding and creasing, to bleed areas, white and varnish, braille, dimensions, legends etc. It does this by building on the common practice of saving the technical marks in PDF layers, but adds some identification metadata that is not dependent on the vendor, the language or the normal practice of the originator, prepress or pressroom.

So now you can look at a PDF file and see definitively that a layer called ‘cut’ contains cutting lines. The name ‘cut’ is now just a convenience; the real information is in metadata which is completely and reliably computer-readable. In other words, it doesn’t matter if that layer were named ‘Schnittlinie’ or anything else; the manual step of identifying names that someone, somewhere put in the file upstream and figuring out what each one means, is completely eliminated.

We implemented support for PDF Processing Steps into version 12.1r0 of the Harlequin RIP, and have worked with a number of vendors whose products create files with Processing Steps in them (including Hybrid Software, Esko and Callas) to ensure that everything works seamlessly. We also worked through a wide variety of current and probable use cases to ensure that our implementation can address real-world needs. As an example we added the ability to control all graphics on a PDF page that aren’t in Processing Step layers as if they were just another layer.

In practice this means that Harlequin can be configured to deliver pretty much whatever you need, such as:

  • Export all technical marks identified as Cutting, PartialCutting, CuttingCreasing etc to a vector format to drive a cutting machine.
  • Render and print all technical marks, but none of the real graphics, for checking registration.
  • Render the real graphics, plus dimensions and legend, for the main print run.

    PDF Processing Steps promises the ability to control technical marks without needing to analyze each file and create a different setup for each job.
    PDF Processing Steps promises the ability to control technical marks without needing to analyze each file and create a different setup for each job.

The most important thing that PDF Processing Steps gives us is that you can create a configuration for one of those use cases (or for many other variations) and know that it will work for all jobs that are sent to you using PDF Processing Steps; you won’t need to reconfigure for the next job, just because an operator used different spot names.

Of course, it’ll take a while for everyone to migrate from using spot names to PDF Processing Steps. But I think you’ll agree that the benefits of doing so, in increasing efficiency and reducing the potential for errors, are obvious and significant.

For more information read the press release here.

Choosing the class of your raster image processor (RIP) – Part II

Part II: Factors influencing your choice of integration

If you’re in the process of building a digital front end for your press, you’ll need to consider how much RIPing power you need for the capabilities of the press and the kinds of jobs that will be run on it. The RIP converts text and image data from many file formats including PDF, TIFF™ or JPEG into a format that a printing device such as an inkjet print head, toner marking engine or laser plate-setter can understand. But how do you know what RIP is best for you and what solution can best deliver maximum throughput on your output device? In this second post, Global Graphics Software’s CTO, Martin Bailey, discusses the factors to consider when choosing a RIP.

In my last post I gave a pointer to a spreadsheet that can be used to calculate the data rate required for a digital press. This single number can be used to make a first approximation of which class of RIP integration you should be considering.

For integrations based on the Harlequin RIP® reasonable guidelines are:

  • Up to 250MB/s: can be done with a single RIP using multi-threading in that RIP
  • Up to 1GB/s: use multiple RIPs on a single server using the Harlequin Scalable RIP
  • Over 1GB/s: use multiple RIPs spread over multiple servers using the Harlequin Scalable RIP

These numbers indicate the data rate that the RIP needs to provide when every copy of the output is different. The value may need to be adjusted for other scenarios:

  • If you’re printing the same raster many times, the RIP data rate may be reduced in proportion; the RIP has 100 times as long to process a PDF page if you’re going to be printing 100 copies of it, for instance.
  • If you’re printing variable data print jobs with significant re-use of graphical elements between copies, then Harlequin VariData™ can be used to accelerate processing. This effect is already factored into the recommendations above.

The complexity of the jobs you’re rendering will also have an impact.

Transactional or industrial labelling jobs, for example, tend to be very simple, with virtually no live PDF transparency and relatively low image coverage. They are therefore typically fast to render. If your data rate calculation puts you just above a threshold in the list above, you may be able to take one step down to a simpler system.

On the other hand, jobs such as complex marketing designs or photobooks are very image-heavy and tend to use a lot of live transparency. If your data rate is just below a threshold on the list above, you will probably need to step up to a higher level of system.

But be careful when making those adjustments, however. If you do so you may have to choose either to build and support multiple variations of your DFE, to support different classes of print site, or to design a single model of DFE that can cope with the needs of the great majority of your customers. Building a single model certainly reduces development, test and support costs, and may reduce your average bill of materials. But doing that also tends to mean that you will need to base your design on the raw, “every copy different”, data rate requirements, because somebody, somewhere will expect to be able to use your press to do just that.

Our experience has also been that the complexity of jobs in any particular sector is increasing over time, and the run lengths that people will want to print are shortening. Designing for current expectations may give you an under-powered solution in a few years’ time, maybe even by the time you ship your first digital press. Moore’s law, that computers will continue to deliver higher and higher performance at about the same price point, will cancel out some of that effect, but usually not all of it.

And if your next press will print with more inks, at a higher resolution, and at higher speed you may be surprised at how much impact that combination will have on the data rate requirements, and therefore possibly on the whole architecture of the Digital Front End to drive it.

And finally, the recommendations above implicitly assume that a suitable computer configuration is used. You won’t achieve 1GB/s output from multiple RIPs on a computer with a single, four-core CPU, for example. Key aspects of hardware affecting speed are: number of cores, CPU clock speed, disk space available, RAM available, disk read and write speed, band-width to memory, L2 and L3 cache sizes on the CPU and (especially for multi-server configurations) network speed and bandwidth.

Fortunately, the latest version of the Harlequin RIP offers a framework that can help you to meet all these requirements. It offers a complete scale of solutions from a single RIP through multiple RIPs on a single server, up to multiple RIPs across multiple servers.

 

The above is an excerpt from our latest white paper: Scalable performance with the Harlequin RIP. Download the white paper here.

Read Part I – Calculating data rates here.

What does a RIP do?

Ever wondered what a raster image processor or RIP does? And what does RIPing a page mean? Read on to learn more about the phases of a RIP, the engine at the heart of your Digital Front End.

The RIP converts text and image data from many file formats including PDF, TIFF™ or JPEG into a format that a printing device such as an inkjet print head, toner marking engine or laser platesetter can understand. The process of RIPing a page requires several steps to be performed in order, regardless of whether that page is submitted as PostScript, PDF or any other page description language.

Interpreting: the page description language to be RIPed is read and decoded into an internal database of graphical elements that must be placed on the page. Each may be an image, a character of text (including font, size, color etc), a fill or stroke etc. This database is referred to as a display list.

Compositing: The display list is pre-processed to apply any live transparency that may be in the job. This phase is only required for any pages in PDF and XPS jobs that use live transparency; it’s not required for PostScript language pages because those cannot include live transparency.

Rendering: The display list is processed to convert every graphical element into the appropriate pattern of pixels to form the output raster. The term ‘rendering’ is sometimes used specifically for this part of the overall processing, and sometimes to describe the whole of the RIPing process. It’s only used it in the first sense in this document.

Output: the raster produced by the rendering process is sent to the marking engine in the output device, whether it’s exposing a plate, a drum for marking with toner, an inkjet head or any other technology.

Sometimes this step is completely decoupled from the RIP, perhaps because plate images are stored as TIFF files and then sent to a CTP platesetter later, or because a near-line or off-line RIP is used for a digital press. In other environments the output stage is tightly coupled with rendering.

RIPing often includes a number of additional processes; in the Harlequin RIP® for example:

  • In-RIP imposition is performed during interpretation
  • Color management (Harlequin ColorPro®) and calibration are applied during interpretation or compositing, depending on configuration and job content
  • Screening is applied during rendering or after the Harlequin RIP has delivered unscreened raster data if screening is being applied post- RIP, when Global Graphics’ ScreenPro™ and PrintFlat™ technologies are being used, for example.

These are all important processes in many print workflows.

 

The Harlequin Host Renderer
The Harlequin RIP includes native interpretation of PostScript, EPS, DCS, XPS, JPEG, BMP and TIFF as well as PDF, PDF/X and PDF/VT, so whatever workflows your target market uses, it gives accurate and predictable image output time after time.

 

The above is an excerpt from our latest white paper: Scalability with the Harlequin RIP®. Download the white paper here

What do you really need in a RIP to drive a digital press in labels & packaging?

In this latest post, Global Graphics CTO Martin Bailey goes back to basics and explores what you need in a RIP to drive a digital press for labels & packaging.

Martin highlights rendering your jobs correctly, color management with CMYK inks and spot colors, PDF layering and technical separations, and provides a high-level view of the features of the Harlequin RIP® for digital labels and packaging.

Watch Martin’s presentation here:

Follow us on Twitter: @Global_Graphics

See you at Labelexpo Europe 2017!

Visit us at Labelexpo Europe 2017 - Stand 9B17

With Labelexpo Europe 2017 less than a week away, it won’t be long before we start to pack the car and head off on the Eurostar to Brussels.

It’s going to be an exciting show for us this year, with some key announcements from our major OEM customers – we’re looking forward to supporting them and demonstrating our technologies.

If you have to create a digital front end but don’t know where to start, come and talk to us. With Fundamentals, we’ll show you how you can bring your digital inkjet label press to market quickly and easily. Our BreakThrough Technical Services team will also be on hand to answer your questions from color management to screening.

We also have some new features for labels and packaging in the Harlequin RIP® to show you, including controls for deciding when to blend emulated spot colors with process colors for exceptionally accurate brand color matching, and extended controls over PDF layers so that optional content used in process control can be individually switched on or off.

We’ll be on Stand 9B17, so please stop by and say hello. If you’d like to book an appointment, simply contact us: sales@globalgraphics.com.  In the meantime, we’ve made this short video to whet your appetite. Enjoy!

 

 

The healthy buzz of conversation at PDF 2.0 interops

Last week was the first PDF 2.0 interop event, hosted by Global Graphics in Cambridge, UK on behalf of the PDF Association. The interop was an opportunity for developers from various companies working on their support for PDF 2.0 to get together and share sample files, and to process them in their own solutions. If a sample file from one vendor isn’t read correctly by a product from another vendor the developers can then figure out why, and fix either the creation tool or the consumer, or even both, depending on the exact reason for that failure.

When we make our own PDF sample files to test the Harlequin RIP there’s always a risk that the developer making the file and the developer writing the code to consume it will make the same assumptions or misread the specification in the same way. That makes testing files created by another vendor invaluable, because it validates all of those assumptions and possible misinterpretations as well.

It’s pretty early in the PDF 2.0 process (the standard itself will probably be published later this month), which means that some vendors are not yet far enough through their own development cycles to get involved yet. But that actually makes this kind of event even more valuable for those who participate because there are no currently shipping products out there that we could just buy and make sample files with. And the last thing that any of us want to do as vendors is to find out about incompatibilities after our products are shipped and in our customers’ hands.

I can tell you that our testing and discussions at the interop in Cambridge were extremely useful in finding a few issues that our internal testing had not identified. We’re busy correcting those, and will be taking updated software to the next interop, in Boston, MA on June 12th and 13th.

If you’re a Harlequin OEM or member of the Harlequin Partner Network you can also get access to our PDF 2.0 preview code to test against your own or other partners’ products; just drop me a line. If you’re using Harlequin in production I’m afraid you’ll have to wait until we release our next major version!

If you’re a software vendor with products that consume or create PDF and you’re already working on your PDF 2.0 support I’d heartily recommend registering for the June interop. I don’t know of any more efficient way to identify defects in your implementation so you can fix them before your customers even see them. Visit https://www.pdfa.org/event/pdf-interoperability-workshop-north-america/ to get started.

And if you’re a PDF software vendor and you’re not working on PDF 2.0 yet … time to start your planning!

Channelling how many spot colors?!!

Martin Bailey, CTO, Global Graphics Software
Martin Bailey, CTO, Global Graphics Software

Recently my wife came home from a local sewing shop proudly waving a large piece of material, which turned out to be a “swatch book” for quilting fabrics. She now has it pinned up on the wall of her hobby room.

It made me wonder how many separations or spot colors I’d ever seen in a single job myself … ignoring jobs specifically designed as swatches.

I think my personal experience probably tops out at around 18 colors, which was for a design guide for a fuel company’s forecourts after a major redesign of their branding. It was a bit like a US banknote: lots of colors, but most of them green!

But I do occasionally hear about cases where a print company or converter, especially in packaging, is looking to buy a new digital press. I’m told it’s common for them to impose together all of their most challenging jobs on the grounds that if the new press (or rather, the DFE on the new press) can handle that, then they can be confident that it’ll handle any of the jobs they receive individually. Of course, if you gang together multiple unrelated jobs, each of which uses multiple spot colors, then you can end up with quite a few different ones on the whole sheet.

“Why does this matter?” I hear you ask.

It would be easy to assume that a request for a spot color in the incoming PDF file for a job is very ephemeral; that it’s immediately converted into an appropriate set of process colors to emulate that spot on the press. Several years ago, in the time of PostScript, and for PDF prior to version 1.4, you could do that. But the advent of live transparency in PDF made things a bit harder. If you naïvely transform spots to process builds as soon as you see them, and if the spot colored object is involved in any transparency blending, then you’ll get a result that looks very different to the same job being printed on a press that actually has an ink for that spot color. In other words, prints from your digital press might not match a print from a flexo press, which is definitely not a good place to be!

So in practice, the RIP needs to retain the spot as a spot until all of the transparency blending and composition has been done, and can only merge it into the process separations afterwards. And that goes for all of the spots in the job, however many of them there are.

Although I was a bit dismissive of swatches above, those are also important. Who would want to buy a wide format printer, or a printer for textiles, or even for packaging or labels, if you can’t provide swatches to your customers and to their designers?

All of this really came into focus for me because, until recently, the Harlequin RIP could only manage 250 spots per page. That sounds a lot, but wasn’t enough for some of our customers. In response to their requests we’ve just delivered a new revision to our OEM partners that can handle a little over 8000 spots per page. I’m hoping that will be enough for a while!

If you decide to take that as a challenge, I’d love to see what you print with it!

Getting to know PDF 2.0: not only but also!

Are you ready for PDF 2.0? Register now for the PDF 2.0 interoperability workshops in the UK and USA.

In the middle of 2017 ISO 32000-2 will be published, defining PDF 2.0.  It’s eight years since there’s been a revision to the standard. We’ve already covered the main changes affecting print in previous blog posts and here Martin Bailey, the primary UK expert to the ISO committee developing PDF 2.0, gives a roundup of a few other changes to expect.

Security
The encryption algorithms included in previous versions of PDF have fallen behind current best practices in security, so PDF adds AES-256-bit and states that all passwords used for AES-256 encryption must be encoded in Unicode.
A PDF 1.7 reader will almost certainly error and refuse to process any PDF files using the new AES-256 encryption.
Note that Adobe’s ExtensionLevel 3 to ISO 32000-1 defines a different AES-256 encryption algorithm, as used in Acrobat 9 (R=5). That implementation is now regarded as dangerously insecure and Adobe has deprecated it completely, to the extent that use of it is forbidden in PDF 2.0.
Deprecation and what this means in PDF!
PDF 2.0 has deprecated a number of implementation details and features that were defined in previous versions. In this context ‘deprecation’ means that tools writing PDF 2.0 are recommended not to include those features in a file; and that tools reading PDF 2.0 files are recommended to ignore those features if they find them.
Global Graphics has taken the deliberate decision not to ignore relevant deprecated items in PDF files that are submitted and happen to be identified as PDF 2.0. This is because it is quite likely that some files will be created using an older version of PDF and using those features. If those files are then pre-processed in some way before submitting to Harlequin (e.g. to impose or trap the files) the pre-processor may well tag them as now being PDF 2.0. It would not be appropriate in such cases to ignore anything in the PDF file simply because it is now tagged as PDF 2.0.
We expect most other PDF readers to take the same course, at least for the next few years.
And the rest…
PDF 2.0 header: It’s only a small thing, but a PDF reader must be prepared to encounter a value of 2.0 in the file header and as the value of the Version key in the Catalog.
PDF 1.7 readers will probably vary significantly in their handling of files marked as PDF 2.0. Some may error, others may warn that a future version of that product is required, while others may simply ignore the version completely.
Harlequin 11 reports “PDF Warning: Unexpected PDF version – 2.0” and then continues to process the job. Obviously that warning will disappear when we ship a new version that fully supports PDF 2.0.
UFT-8 text strings: Previous versions of PDF allowed certain strings in the file to be encoded in PDFDocEncoding or in 16-bit Unicode. PDF 2.0 adds support for UTF-8. Many PDF 1.7 readers may not recognise the UTF-8 string as UTF-8 and will therefore treat it as using PDFDocEncoding, resulting in those strings being treated as what looks like a random sequence of mainly accented characters.
Print scaling: PDF 1.6 added a viewer preferences key that allowed a PDF file to specify the preferred scaling for use when printing it. This was primarily in support of engineering drawings. PDF 2.0 adds the ability to say that the nominated scaling should be enforced.
Document parts: The PDF/VT standard defines a structure of Document parts (common called DPart) that can be used to associate hierarchical metadata with ranges of pages within the document. In PDF/VT the purpose is to enable embedding of data to guide the application of different processing to each page range.
PDF 2.0 has added the Document parts structure into baseline PDF, although no associated semantics or required processing for that data have been defined.
It is anticipated that the new ISO standard on workflow control (ISO 21812, expected to be published around the end of 2017) will make use of the DPart structure, as will the next version of PDF/VT. The specification in PDF 2.0 is largely meaningless until such time as products are written to work with those new standards.

 

The background
The last few years have been pretty stable for PDF; PDF 1.7 was published in 2006, and the first ISO PDF standard (ISO 32000-1), published in 2008, was very similar to PDF 1.7. In the same way, PDF/X‑4 and PDF/X‑5, the most recent PDF/X standards, were both published in 2010, six years ago.
In the middle of 2017 ISO 32000-2 will be published, defining PDF 2.0. Much of the new work in this version is related to tagging for content re-use and accessibility, but there are also several areas that affect print production. Among them are some changes to the rendering of PDF transparency, ways to include additional data about spot colors and about how color management should be applied.

Fancy a test drive of the new Harlequin v11 at drupa 2016?

One of the many highlights of our drupa stand will be the new Harlequin RIP. We asked Martin Bailey, CTO at Global Graphics, to tell us more about it. He told us that there are a host of new features to improve inkjet output quality including richer, multi-level screening controls, more controls for variable data printing, and new features for labels and packaging applications. Hear his summary in this video below.

Fancy a test drive? Join us at drupa 2016, Stand 70 B21/C20 in the dip. Simply contact us to book a demonstration.

Stay tuned for more announcements over the next couple of weeks.