ISO – Global Graphics Software blog

May 31, 2022July 25, 2022

A nostalgic look back at the ISO PDF/X standard

In this blog post, Martin Bailey recalls his days as the first chair of the ISO PDF/X task force and how the standard has developed over the last 20 years.

Over the last few years there has been quite an outpouring of nostalgia around PDF. That was first for PDF itself, but at the end of 2021 we reached two decades since the first publication of an ISO PDF/X standard.

I’d been involved with PDF/X in its original home of CGATS (the Committee for Graphic Arts Technical Standards, the body accredited by ANSI to develop US national standards for printing) for several years before it moved to ISO. And then I became the first chair of the PDF/X task force in ISO. So I thought I’d add a few words to the pile, and those have now been published on the PDF Association’s web site at https://www.pdfa.org/the-route-to-pdf-x-and-where-we-are-now-a-personal-history/.

I realised while I was writing it that it really was a personal history for me. PDF/X was one of the first standards that I was involved in developing, back when the very idea of software standards was quite novel. Since then, supported and encouraged by Harlequin and Global Graphics Software, I’ve also worked on standards and chaired committees in CIP3, CIP4, Ecma, the Ghent Working Group, ISO and the PDF Association (I apologise if I’ve missed any off that list!).

It would be easy to assume that working on all of those standards meant that I knew a lot about what we were standardising from day one. But the reality is that I’ve learned a huge amount of what I know about print from being involved, and from talking to a lot of people.

Perhaps the most important lesson was that you can’t (or at least shouldn’t) only take into account your own use cases while writing a standard. Most of the time a standard that satisfies only a single company should just be proprietary working practice instead. It’s only valuable as a standard if it enables technologies, products and workflows in many different companies.

That sounds as if it should be obvious, but the second major lesson was something that has been very useful in environments outside of standards as well. An awful lot of people assume that everyone cares a lot about the things that they care about, and that everything else is unimportant. As an example, next time you’re at a trade show (assuming they ever come back in their historical form) take a look and see how many vendors claim to have product for “the whole workflow”. Trust me, for production printing, nobody has product for the whole workflow. Each one just means that they have product for the bits of the workflow that they think are important. The trouble is that you can’t actually print stuff effectively and profitably if all you have is those ‘important’ bits. To write a good standard you have to take off the blinkers and see beyond what your own products and workflows are doing. And in doing that I’ve found that it also teaches you more about what your own ‘important’ parts of the workflow need to do.

Along the way I’ve also met some wonderful people and made some good friends. Our conversations may have a tendency to dip in and out of print geek topics, but sometimes those are best covered over a beer or two!

About the author

Martin Bailey, CTO, Global Graphics Software

Martin Bailey is currently the primary UK expert to the ISO committees maintaining and developing PDF and PDF/VT and is the author of Full Speed Ahead: how to make variable data PDF files that won’t slow your digital press, a new guide offering advice to anyone with a stake in variable data printing including graphic designers, print buyers, composition developers and users.

Time for an update on VDP!

Over the last fifteen years variable data in digital printing has grown from “the next big thing” with vast, untapped potential to a commonly used process for delivering all manner of personalized information. VDP is used for everything from credit card bills and bank statements to direct mail postcards and personalized catalogues, from college enrolment packs to Christmas cards and photobooks, from labels to tickets, checks to ID cards.

This huge variety of jobs is created and managed by an equally huge variety of software, from specialist composition tools to general purpose design applications carefully configured for VDP. And they are consumed by workflows involving (or even completely within) the Digital Front End (DFE) for a digital production press, where jobs must be imposed, color managed.

Time, then, to update our popular “Do PDF/VT Right” guide which has had thousands of downloads since it was first published in 2014 not to mention the number of printed copies distributed at trade shows and industry events.

Do PDF/VT Right - How to make problem-free PDF files for variable data printing — Do PDF/VT Right – How to make problem-free PDF files for variable data printing

In addition to a general overhaul there is a new section on the new ISO 21812 standard that allows workflow controls to be added to PDF files, and notes on Harlequin-specific hints, to get even more speed out of your DFE if you are a Harlequin user.

The goal remains the same: to provide a set of actionable recommendations that help you ensure that your jobs don’t slow down the print production workflow … without affecting the visual appearance that you’re trying to achieve. As a side benefit, several of the recommendations set out below will also ensure that your PDF files can be delivered more efficiently on the web and to PDF readers on mobile devices in a cross-media publishing environment.

Some of the recommendations made in this guide are things that a graphic designer can apply quickly and easily, using their current tools. Others are intended more for the software companies building composition tools. If all of us work together we can greatly reduce the chance of that “heart-attack” job; the one that absolutely, positively must be in the post today … but that runs really slowly on the press.

Download your copy here .

December 7, 2017March 14, 2018

Martin Bailey appointed co-chair of PDF Technical Working Group

Congratulations to our very own Martin Bailey and to Peter Wyatt, the general manager of CiSRA, for being nominated co-chairs of the PDF Technical Working Group (TWG) within the PDF Competence Centre branch of the PDF Association. https://www.pdfa.org/working-group/pdf-competence-center/

Following the publication of the new ISO PDF 2.0 standard – ISO 32000-2 in July 2017, the PDF TWG will be producing PDF 2.0 Application Notes to support the implementation of the standard by developers whose PDF tools create and consume PDF.

ISO 32000-2 is the first PDF specification developed within the ISO working-group structure involving subject matter experts from many countries and is the first “post Adobe” standard since they handed over its development to the ISO.

Speaking on news of his appointment Martin said, “The value of a standard can be greatly increased by a wider involvement of the relevant communities in shared education and discussion. The PDF Association has become the obvious group to help foster and guide that wider involvement for PDF itself and for many of the PDF-based standards in use today.”

Duff Johnson, the PDF Association’s executive director said, “PDF 2.0 is designed to be largely backward compatible, but older processors won’t handle new features. The purpose of the new documents that will be developed by the PDF TWG is to help developers develop a common understanding of the new specification as well as best practices for implementation.” We’re very happy that Martin and Peter have agreed to lead this effort.

Martin Bailey is the primary UK expert to the ISO committees working on PDF, PDF/X and PDF/VT. In 2017 Global Graphics Software hosted two PDF 2.0 interoperability workshops on behalf of the PDF Association to provide a way for PDF tool developers to validate their work against the new ISO 32000-2 (PDF 2.0) standard by working with vendors of other tools.

May 12, 2017February 27, 2023

The healthy buzz of conversation at PDF 2.0 interops

Last week was the first PDF 2.0 interop event in Cambridge, UK, hosted by Global Graphics on behalf of the PDF Association. The interop was an opportunity for developers from various companies working on their support for PDF 2.0 to get together and share sample files, and to process them in their own solutions. If a sample file from one vendor isn’t read correctly by a product from another vendor the developers can then figure out why, and fix either the creation tool or the consumer, or even both, depending on the exact reason for that failure.

When we make our own PDF sample files to test the Harlequin RIP there’s always a risk that the developer making the file and the developer writing the code to consume it will make the same assumptions or misread the specification in the same way. That makes testing files created by another vendor invaluable, because it validates all of those assumptions and possible misinterpretations as well.

It’s pretty early in the PDF 2.0 process (the standard itself will probably be published later this month), which means that some vendors are not yet far enough through their own development cycles to get involved yet. But that actually makes this kind of event even more valuable for those who participate because there are no currently shipping products out there that we could just buy and make sample files with. And the last thing that any of us want to do as vendors is to find out about incompatibilities after our products are shipped and in our customers’ hands.

I can tell you that our testing and discussions at the interop in Cambridge were extremely useful in finding a few issues that our internal testing had not identified. We’re busy correcting those, and will be taking updated software to the next interop, in Boston, MA on June 12th and 13th.

If you’re a Harlequin OEM or member of the Harlequin Partner Network you can also get access to our PDF 2.0 preview code to test against your own or other partners’ products; just drop me a line. If you’re using Harlequin in production I’m afraid you’ll have to wait until we release our next major version!

If you’re a software vendor with products that consume or create PDF and you’re already working on your PDF 2.0 support I’d heartily recommend registering for the June interop. I don’t know of any more efficient way to identify defects in your implementation so you can fix them before your customers even see them. Visit https://www.pdfa.org/event/pdf-interoperability-workshop-north-america/ to get started.

And if you’re a PDF software vendor and you’re not working on PDF 2.0 yet … time to start your planning!

About the author

Martin Bailey, consultant and former 0CTO, Global Graphics Software

Martin Bailey, consultant at Global Graphics Software, is a former CTO of the company and currently the primary UK expert to the ISO committees maintaining and developing PDF and PDF/VT. He is the author of Full Speed Ahead: how to make variable data PDF files that won’t slow your digital press, a guide offering advice to anyone with a stake in variable data printing including graphic designers, print buyers, composition developers and users.

To be the first to receive our blog posts,

To be the first to receive our blog posts, news updates and product news why not subscribe to our monthly newsletter? Subscribe here

Follow us on LinkedIn, Twitter and YouTube

January 6, 2017November 21, 2019

Getting to know PDF 2.0: not only but also!

In the middle of 2017 ISO 32000-2 will be published, defining PDF 2.0. It’s eight years since there’s been a revision to the standard. We’ve already covered the main changes affecting print in previous blog posts and here Martin Bailey, the primary UK expert to the ISO committee developing PDF 2.0, gives a roundup of a few other changes to expect.

Security
The encryption algorithms included in previous versions of PDF have fallen behind current best practices in security, so PDF adds AES-256-bit and states that all passwords used for AES-256 encryption must be encoded in Unicode.
A PDF 1.7 reader will almost certainly error and refuse to process any PDF files using the new AES-256 encryption.
Note that Adobe’s ExtensionLevel 3 to ISO 32000-1 defines a different AES-256 encryption algorithm, as used in Acrobat 9 (R=5). That implementation is now regarded as dangerously insecure and Adobe has deprecated it completely, to the extent that use of it is forbidden in PDF 2.0.
Deprecation and what this means in PDF!
PDF 2.0 has deprecated a number of implementation details and features that were defined in previous versions. In this context ‘deprecation’ means that tools writing PDF 2.0 are recommended not to include those features in a file; and that tools reading PDF 2.0 files are recommended to ignore those features if they find them.
Global Graphics has taken the deliberate decision not to ignore relevant deprecated items in PDF files that are submitted and happen to be identified as PDF 2.0. This is because it is quite likely that some files will be created using an older version of PDF and using those features. If those files are then pre-processed in some way before submitting to Harlequin (e.g. to impose or trap the files) the pre-processor may well tag them as now being PDF 2.0. It would not be appropriate in such cases to ignore anything in the PDF file simply because it is now tagged as PDF 2.0.
We expect most other PDF readers to take the same course, at least for the next few years.
And the rest…
PDF 2.0 header: It’s only a small thing, but a PDF reader must be prepared to encounter a value of 2.0 in the file header and as the value of the Version key in the Catalog.
PDF 1.7 readers will probably vary significantly in their handling of files marked as PDF 2.0. Some may error, others may warn that a future version of that product is required, while others may simply ignore the version completely.
Harlequin 11 reports “PDF Warning: Unexpected PDF version – 2.0” and then continues to process the job. Obviously that warning will disappear when we ship a new version that fully supports PDF 2.0.
UFT-8 text strings: Previous versions of PDF allowed certain strings in the file to be encoded in PDFDocEncoding or in 16-bit Unicode. PDF 2.0 adds support for UTF-8. Many PDF 1.7 readers may not recognise the UTF-8 string as UTF-8 and will therefore treat it as using PDFDocEncoding, resulting in those strings being treated as what looks like a random sequence of mainly accented characters.
Print scaling: PDF 1.6 added a viewer preferences key that allowed a PDF file to specify the preferred scaling for use when printing it. This was primarily in support of engineering drawings. PDF 2.0 adds the ability to say that the nominated scaling should be enforced.
Document parts: The PDF/VT standard defines a structure of Document parts (common called DPart) that can be used to associate hierarchical metadata with ranges of pages within the document. In PDF/VT the purpose is to enable embedding of data to guide the application of different processing to each page range.
PDF 2.0 has added the Document parts structure into baseline PDF, although no associated semantics or required processing for that data have been defined.
It is anticipated that the new ISO standard on workflow control (ISO 21812, expected to be published around the end of 2017) will make use of the DPart structure, as will the next version of PDF/VT. The specification in PDF 2.0 is largely meaningless until such time as products are written to work with those new standards.

The background
The last few years have been pretty stable for PDF; PDF 1.7 was published in 2006, and the first ISO PDF standard (ISO 32000-1), published in 2008, was very similar to PDF 1.7. In the same way, PDF/X‑4 and PDF/X‑5, the most recent PDF/X standards, were both published in 2010, six years ago.
In the middle of 2017 ISO 32000-2 will be published, defining PDF 2.0. Much of the new work in this version is related to tagging for content re-use and accessibility, but there are also several areas that affect print production. Among them are some changes to the rendering of PDF transparency, ways to include additional data about spot colors and about how color management should be applied.

December 20, 2016November 21, 2019

Getting to know PDF 2.0: halftones

In the middle of 2017 ISO 32000-2 will be published, defining PDF 2.0. It’s eight years since there’s been a revision to the standard. In his next blog post about the changes afoot, Martin Bailey, the primary UK expert to the ISO committee developing PDF 2.0, looks at halftones, an area where the new specification will offer significant benefits for flexo jobs.

Lists of spot functions in halftones
PDF allows a PDF file to specify the halftone to be used for screening output in a variety of ways. The simplest is to identify a spot function by name, but that method was constrained in versions of the PDF standard up to PDF 1.7 to use only names that were explicitly listed in the specification itself. This has been a significant limitation in print sectors where custom halftones are common, such as flexography, gravure … and pretty much everywhere apart from offset plate-making!

PDF 2.0 allows the PDF file to specify the halftone dot shape as a list of spot function names, and those names no longer need to be picked from the ones specified in the standard. The renderer should use the first named spot function in the list that it supports. This allows a single file to be created that can be used in a variety of RIPs that support different sets of proprietary halftones and to select the best one available in each RIP for that specific object.

This functionality is expected to be used mainly for high-quality flexo press work, where it’s a key part of the workflow to specify which halftone should be used for each graphical element.

A PDF 1.7 reader will probably either error or completely ignore the screening information embedded in the PDF if a file using the new list form is encountered. In the flexo space that could easily cause problems on-press, so take care that you’ve upgraded your RIPs before you start to try rendering PDF files using this new capability.

Halftone Origin (HTO)
Very old versions of PDF (up to PDF 1.3) included a partial definition of an entry named HTP, which was intended to allow the location of the origin or phase of a halftone to be specified. That entry was unfortunately useless because it did not specify the coordinate system to apply and it was removed many years ago.

PDF 2.0 adds a new entry called HTO to achieve the same goal, but this time fully specified. The use case is anywhere where precise specification of the halftone phase is valuable. Examples include pre-imposed sheets for VLF plate-setters, where specifying the halftone phase for each imposed page can reduce the misalignment of halftones that can occur over very long distances, or setting the halftone phase of each of a set of step-and-repeat labels to ensure that the halftone dots are placed in exactly the same position relative to the design in each instance.

A PDF 1.7 reader will simply ignore the new key, so there’s no danger of new files causing problems in an older workflow. On the other hand, those older RIPs will render as they always have, which would be a missed opportunity for the target use cases.

Halftone selection in transparent areas
Up to PDF 1.7 there has been a requirement to apply the “default halftone” in all areas where transparency compositing has been applied. This was problematic for those print technologies where different halftones must be used for different object types to achieve maximum quality, e.g. for flexo. Transparency is used in these jobs most commonly for drop shadows, so that’s where you’re most likely to have encountered problems.

PDF 2.0 effectively gives complete freedom to renderers to apply the supplied screening parameters in whatever way they see fit, but two example implementations are provided to encourage similarity between implementations. One of those matches the requirements from PDF 1.7, while the other applies the screen defined for the top-most graphical element in areas where transparency was applied. The second one means that the screening selected for the drop shadow will now be used, matching requirements for the flexo market.

In the middle of 2017 ISO 32000-2 will be published, defining PDF 2.0. Much of the new work in this version is related to tagging for content re-use and accessibility, but there are also several areas that affect print production. Among them are some changes to the rendering of PDF transparency, ways to include additional data about spot colors and about how color management should be applied.

November 30, 2016November 21, 2019

Getting to know PDF 2.0 – update from Down Under

I’ve been in the ISO PDF committee meeting in Sydney, Australia for a couple of days this week to review the comments submitted to the most recent ballot on PDF 2.0. Over 100 comments were received, including some complex issues around digital signatures, structure tagging (especially lists), optional content, document parts and soft masks. In all cases the committee was able to reach a consensus on what should be done for PDF 2.0.

The plan is now for one more ballot, the responses to which will be reviewed in Q2 next year, with an expectation that final text for PDF 2.0 will be delivered to ISO for publication shortly thereafter.

So we’re still on track for publication next year.

All of which means that it’s past time that a couple of PDF’s unsung heroes were acknowledged. The project leaders for PDF 2.0 have invested very substantial amounts of time and mental energy updating text in response to comments and ballots over the last several years. When somebody like me requests a change it’s the project leaders who help to double-check that every last implication of that change is explored to ensure that we don’t have any inconsistency.

So a big thank you to Duff Johnson of the PDF Association and Peter Wyatt of CISRA (Canon)!

It’s also worth noting that one of the significant improvements in PDF 2.0 that probably won’t get highlighted elsewhere is that the text now is much more consistent. When you’re writing a detailed technical document 1000 pages long it’s inevitable that some disconnections between different sections will creep in. PDF 2.0 illustrates the value of a broad group of people from many countries and many industries reviewing text in the ISO process: we’ve managed to stamp on many of those cases in this new version.

November 1, 2016November 21, 2019

Getting to know PDF 2.0

Just when you’ve all cozied down with PDF 1.7 what happens? Yes, that’s right. A new standard rears its head.

Around the middle of 2017 the ISO committee will publish PDF 2.0 (ISO 32000-2). So by the end of 2017 you’ll probably need to be considering how to ensure that your workflow can handle PDF 2.0 files correctly.

As the primary UK expert to this committee I thought I’d give you a heads up now on what to expect. And over the coming months via this blog and our newsletter I’ll endeavor to keep you posted on what to look out for as far as print is concerned. Because, of course, there are many aspects to the standard that do not concern print at all. For instance there are lots of changes in areas such as structure tagging for accessibility and digital signatures that might be important for business and consumer applications.

As you probably already know, in 2008 Adobe handed over ownership and development of the PDF standard to the International Standards Organization. Since that time I’ve been working alongside other experts to ensure that standards have real-world applicability.

And here’s one example relating to color.

The printing condition for which a job was created can be encapsulated in professional print production jobs by specifying an “output intent” in the PDF file. The output intent structure was invented for the PDF/X standards, at first in support of pre-flight, and later to enable color management at the print site to match that used in proofing at the design stage.

But the PDF/X standards only allow a single output intent to be specified for all pages in a job.

PDF 2.0 allows separate output intents to be included for every page individually. The goal is to support jobs where different media are used for various pages, e.g. for the first sheet for each recipient of a transactional print job, or for the cover of a saddle-stitched book. The output intents in PDF 2.0 are an extension of those described in PDF/X, and the support for multiple output intents will probably be adopted back into PDF/X-6 and into the next PDF/VT standard.

But of course, like many improvements, this one does demand a little bit of care. A PDF 1.7 or existing PDF/X reader will ignore the new page level output intents and could therefore produce the wrong colors for a job that contains them.
In my next post I’ll be covering changes around live transparency in PDF 2.0. Bet you can’t wait!
You can sign up to the Global Graphics newsletter here.

The background
The last few years have been pretty stable for PDF; PDF 1.7 was published in 2006, and the first ISO PDF standard (ISO 32000-1), published in 2010, was very similar to PDF 1.7. In the same way, PDF/X 4 and PDF/X 5, the most recent PDF/X standards, were both published in 2010, six years ago.

September 21, 2016November 21, 2019

New task force concentrates on metadata framework

I’m proud to announce that I’m chairing a new task force that has just been created in TC130, the ISO committee focused on standardization for the printing industry. The task force is named “PDF Common Metadata”, and its focus is on constructing a metadata framework that can be embedded within a PDF file to guide production workflow decisions.

We created a precursor to this work in PDF/VT, in cooperation with CIP4. In that case a hierarchical structure of metadata in the PDF file was intended to be used with a templated JDF job ticket (or similar structure) to ensure that complex variable data jobs could be imposed, printed and finished correctly. Unfortunately the model we used set the bar too high and most composition vendors and press manufacturers felt that implementation was too difficult.

But there are a wide range of situations where a simpler model has real value. Indeed, the current work grew out of requests from the transactional print space to be able to include media selections and simplex/duplex controls in a PDF file. That request was initially reviewed by the PDF/VT Competence Center in the PDF Association who concluded that the benefits of a suitable solution would apply across the printing industry, not just in variable data.
The solution proposed is to build on the concept of ‘intents’ from JDF (although not directly on JDF itself). These describe what the final printed piece is supposed to look like, rather than specifying the details of the processes required to make it. The thought process is that the digital front end (DFE) on a digital press can map from that to the actual steps needed.

As a simple example, a request for a specific substrate should be fairly easy to map to an entry in the media library in a DFE and therefore to tray selections (on a sheet-fed press) and to installing the correct ICC color profile. In closed loop workflows such as web to print the first mapping shouldn’t be necessary at all, because the media selection will be pre-populated from the same data as the media library.

The committee met for the first time in San Jose last week, and we’re looking forward to some lively debate. Our first goal is a standard for graphic arts, but there has already been discussion of following on with equivalents targeted more specifically at packaging and at wide format.

If you’re interested in getting involved please contact your national standards body and tell them you want to work in ISO TC130/WG2/TF5. If you don’t know who to contact in your country, drop me a line and I’m happy to make introductions.

February 3, 2016November 21, 2019

PDF/VT – bringing all the advantages of PDF workflow to the world of variable data printing

Standards for variable data printing (VDP) have come a long way since the first work by CGATS to develop a universal delivery format in the late 1990s. In 2010 the International Standards Organization published the PDF/VT standard, marking the first really effective specification for a reliable, vendor-neutral exchange of variable data jobs, both within and between companies.

A special type of the PDF file format, PDF/VT is specifically used for variable data and transactional printing in a variety of environments, from desktop printing to high volume digital production presses. Built on PDF/X, it therefore brings all the advantages of that standard in enforcing best practices for reproducible and predictable color and appearance to the variable data and transactional print worlds.

The industry is gradually realizing its value to improve quality, competitiveness and productivity, and I’ve been working with the PDF/VT Competence Center, especially with Christoph Oeters (Sofha), Paul Jones (Teclyn bv) and Tim Donahue (technical consultant) to produce a new set of Application Notes highlighting the benefits of using PDF/VT and the workflows that it enables.

The Application Notes explain how to make the highest quality and most efficient PDF/VT files to achieve the required visual appearance of a job, so if you develop software to read and write PDF/VT files, for example in composition tools, RIPs, digital front ends and imposition tools, or if you work on print workflow integration, you’ll find the notes really beneficial. They also show how document part metadata can be applied and leveraged for VDP specific production workflows.

Of course, there are wider benefits to using PDF/VT: The adoption of PDF/VT will allow the industry to finally move towards a reliable, vendor-neutral exchange of variable data jobs, simplifying the process of variable data printing significantly.

The application notes are free to download, pick up your copy here: http://www.pdfa.org/publication/pdfvt-application-notes/.

Let me know what you think of them – feedback is always welcome.

Additional reading:

Do PDF/VT right by Global Graphics Software

PDF/VT for personalized print by the PDF/A Competence Center

Read the press release from the PDF Association (http://www.pdfa.org/2016/01/pdf-association-publishes-pdfvt-application-notes-showcasing-the-benefits-for-variable-data-print-streams/)