Compliance, compatibility, and why some tools are more forgiving of bad PDFs

Compliant and compatible PDF documents and the Harlequin RIP

We added support for native processing of PDF files to the Harlequin RIP® way back in 1997. When we started working on that support we somewhat naïvely assumed that we should implement the written specification and that all would be well. But it was obvious from the very first tests that we performed that we would need to do something a bit more intelligent because a large proportion of PDF files that had been supplied as suitable for production printing did not actually comply with the specification.

Launching a product that would reject many PDF files that could be accepted by other RIPs would be commercial suicide. The fact that, at the time, those other RIPs needed the PDF to be transformed into PostScript first didn’t change the business case.

Unfortunately a lot of PDF files are still being made that don’t comply with the standard, so over the almost a quarter of a century since first launching PDF support we’ve developed our own rules around what Harlequin should do with non-compliant files, and invested many decades of effort in test and development to accept non-compliant files from major applications.

The first rule that we put in place is that Harlequin is not a validation tool. A Harlequin RIP user will have PDF files to be printed, and Harlequin should render those files as long as we can have a high level of confidence that the pages will be rendered as expected.

In other words, we treat both compliance with the PDF standard and compatibility with major PDF creation tools as equally important … and supporting Harlequin RIP users in running profitable businesses as even more so!

The second rule is that silently rendering something incorrectly can be very bad, increasing costs if a reprint is required and causing a print buyer/brand to lose faith in a print service provider/converter. So Harlequin is written to require a reasonably high level of confidence that it can render the file as expected. If a developer opening up the internals of a PDF file couldn’t be sure how it was intended to be rendered then Harlequin should not be rendering it.

We’d expect most other vendors of PDF readers to apply similar logic in their products, and the evidence we’ve seen supports that expectation. The differences between how each product treats invalid PDF are the result of differences in the primary goal of each product, and therefore to the cost of output that is viewed as incorrect.

Consider a PDF viewer for general office or home use, running on a mobile device or PC. The business case for that viewer implies that the most important thing it has to do is to show as much of the information from a PDF file as possible, preferably without worrying the user with warnings or errors. It’s not usually going to be hugely important or costly if the formatting is slightly wrong. You could think of this as being at the opposite end of the scale from a RIP for production printing. In other words, the required level of confidence in accurately rendering the appearance of the page is much lower for the on-screen viewer.

You may have noticed that my description of a viewer could easily be applied to Adobe Reader or Acrobat Pro. Acrobat is also not written primarily as a validation tool, and it’s definitely not appropriate to assume that a PDF file complies with the standard just because it opens in Acrobat. Remember the Acrobat business case, and imagine what the average office user’s response would be if it would not open a significant proportion of PDF files because it flagged them as invalid!

For further reading about PDF documents and standards:

  1. Full Speed Ahead: How to make variable data PDF files that won’t slow your digital press
  2. PDF Processing Steps – the next evolution in handling technical marks

About the author

Martin Bailey, CTO, Global Graphics Software

 

 

 

Martin Bailey, Distinguished Technologist, Global Graphics Software, is currently the primary UK expert to the ISO committees maintaining and developing PDF and PDF/VT and is the author of Full Speed Ahead: how to make variable data PDF files that won’t slow your digital press, a new guide offering advice to anyone with  a stake in variable data printing including graphic designers, print buyers, composition developers and users.

Be the first to receive our news updates and product news. Why not subscribe to our monthly newsletter? Subscribe here

Follow us on LinkedInTwitter, and YouTube

The Print Report with Deborah Corn and Pat McGrew

Martin Bailey, distinguished technologist at Global Graphics Software, joins hosts Deborah Corn and Pat McGrew in this special episode of The Print Report. Together they discuss the innovative methods used at Global Graphics to solve complex and common printing problems using software.

Martin highlights the award-winning PrintFlat™ technology, which gives smooth, uniform tints and accurate tone reproduction via a simple ‘fingerprint’ calibration of the screening process, and the value of creating optmized PDF files so they don’t slow down your digital press and disrupt the production process.

Tune in here:

The Print Report with Deborah Corn and Pat McGrew

 

Follow us on LinkedInTwitter, and YouTube

Using the Mako Core SDK to modify documents in Microsoft’s Universal Print

Over the past year, Microsoft has been working hard to bring its new Cloud printing service, Universal Print, to general availability.

As a part of Universal Print, developers get access to a set of Graph APIs that allows analysis and modification of print job payload data. This feature enables a few different scenarios, including adding security (e.g. redactions or watermarks) to a Universal Print-based workflow.

As a curious engineer, I wanted to see how different it would be for an independent software vendor (ISV) to use our Mako™ Core SDK to modify a print job flowing through Universal Print, instead of using a more traditional route of using a virtual printer driver.

Thinking about the workflow a little more, I came up with the following design:

Using the Mako SDK to modify documents in Universal Print.
Using the Mako SDK to modify documents in Universal Print.

In the design above, we can see the end-user’s Word document gets printed to a virtual printer. This allows the ISV to be notified of the job, and modify it accordingly using Mako. Once modified, the ISV then redirects the job on to the physical printer for printing.

There’s a couple of nice things about this design:

Firstly, it uses the Graph API to access Universal Print, which is an easy-to-use and well documented REST API. Secondly, since the functionality is accessed via a REST API, it allows our ISV service to be written in whichever Mako supported language we like.

I chose C# to make best use of the C# Graph API SDK.

Developing the service

There are five main steps to developing the service:

  1. Handle print job notifications
  2. Download the print job payload
  3. Modify the payload
  4. Upload the payload
  5. Redirect to the target printer

Handle print job notifications

To be notified of print jobs in Universal print, you can use the Graph’s change notifications. These will allow you to sign up to a notification, which will call a provided webhook.

Download the print job payload

Once we have notification that a print job has been sent to our virtual printer, we can start downloading its payload.

Here we use the appropriate Graph APIs, along with standard Graph authentication to access the print job’s document. We then simply save it to disk.

Modify the payload

Once we have the document on disk (although Mako can also modify streams too!), we can open the document and modify it using Mako’s document object model (DOM).

Alternatively, Mako can also convert from one page description language (PDL) to another. This is useful in situations where your destination printer doesn’t support the input PDL.

Upload the payload

Uploading the modified document is straightforward. This time we use the Graph API to create an upload session, and use the WebClient class to put the document back into the original print job.

Redirect to the target printer

And finally, after the print job has been updated, we can redirect it onto another printer. This redirection also automatically completes the print job and task.

Alternatively, if we want to be a little more green, we could always send the document to OneDrive, Sharepoint, or another document management system. After doing so, you then complete the print job and its associated task.

See it in action

We actually coded this demo live in our last Mako webinar, showing an implementation where an ISV wants to automatically redact content.

Access the code directly at our GitHub repository or watch the webinar recording below:

Try it out

We’re keen to talk to you about your Universal Print project and see how we can help. Contact us here.

For more information about Mako, visit globalgraphics.com/mako.

About the author

Andy Cardy, Principal Engineer at Global Graphics Software
Andy Cardy, Principal Engineer at Global Graphics Software

Further reading:

  1. Carry out complex tasks for your print workflow easily with Mako
  2. Improving PDF accessibility with Structure Tagging

Be the first to receive our news updates and product news. Why not subscribe to our monthly newsletter? Subscribe here

Follow us on LinkedIn and Twitter

Compensating for blocked or deviated nozzles

As digital presses become wider and higher resolution, supporting additional inks beyond CMYK, the total number of nozzles required per press increases, which in turn raises the likelihood of at least one of those nozzles failing to jet or becoming deviated (and needing to be disabled). Each nozzle that is out will result in a lack of ink where it should have been in a straight line along the direction of print.

One obvious response would be to fix the problematic nozzles, when they have been detected, but this kind of physical intervention interrupts production and can be impractical. Replacing the affected printhead altogether is another possibility, but again this will have an unwelcome financial impact.

Our sister company, Meteor Inkjet, offers NozzleFix™, an embedded software/hardware solution to compensate for missing nozzles.  But for digital printing equipment which does not incorporate Meteor printhead electronics, software compensation may also be beneficial.  In this post, we will look at a software-based approach where good working nozzles can compensate for any misbehaving neighbors as required.

Nozzle-Out compensation in software

As the word “compensation” suggests, this solution works by telling certain nozzles to jet more ink than they otherwise would have to make up for a deficit caused by nozzles that were not jetting or had to be turned off because they had deviated.

For one-drop screening, this could mean that whenever a drop should have been output by a nozzle that’s out, a nozzle on either side could be asked to jet that drop instead. An alternative to this, which can accommodate any drop size, is to apply compensation before screening, working with intensities rather than drops.

Here’s how it works: when a nozzle is off, the intensity for each pixel associated with that nozzle is used to boost the intensity of neighboring pixels, which are then screened as normal.

This is best understood by way of an example. Consider an 8-bit continuous tone input TIFF™ which has the value 160 for every pixel. If we are using one neighbor on either side of a pixel to compensate, we will add 80 to their intensity before screening. If we are using two neighbors on each side, we would add 40.

The greatly zoomed-in images above illustrate the results after 3-drop screening for this example: the left image has one nozzle out and no compensation; the middle image shows compensation with one neighbor either side; the right image shows the use of two neighbors on each side.

Of course, the impact is best judged from the actual prints, where the combination of high resolution and ink results in an impressive optical illusion!

Image shows nozzles out and then a compensated version

 

The scans above were printed at 1200dpi. The image on the left simulates a number of missing nozzles, which can be seen by the presence of thin horizontal white lines (you may need to enlarge it for the full effect). The image on the right shows the impact of applying compensation to each nozzle neighboring a missing nozzle.

Rapid response and adjustment

A key benefit of this software-based approach is that a rapid response and adjustment can happen as soon as a problematic nozzle is detected. There is no need to wait for engineer and parts availability. Additionally, there is no trade-off to make between taking costly action each time a problematic nozzle presents itself or having a tolerance or threshold where nothing is done until a certain number of quality issues have accumulated.

Learn more about the technologies available to improve the quality of your inkjet output by visiting our website.

About the author

Ian Bolton, Product Manager, Direct
Ian Bolton, Product Manager – Direct

Ian has over 15 years’ experience in industry as a software engineer focusing on high performance. With a passion for problem-solving, Ian’s role as product manager for the Direct range gives him the opportunity to work with printer OEMs and break down any new technology barriers that may be preventing them from reaching their digital printer’s full potential.

Further reading:

  1. What causes banding in inkjet? (And the smart software solution to fix it.)
  2. Streaks and Banding: Measuring macro uniformity in the context of optimization processes for inkjet printing

To be the first to receive our blog posts, news updates and product news why not subscribe to our monthly newsletter? Subscribe here

Follow us on LinkedInTwitter, and YouTube

 

Live coding with Mako Core

Last week, my colleague David Stevenson and I ignored all common sense and ran a live coding demo using Mako™. What could have gone wrong!?

Mako is a software development kit that can be used to add a variety of functions into software products, which is why it’s often referred to as the software engineer’s Swiss Army knife.

During the demo we showed how to use Mako to modernize your print infrastructure in three simple ways:

Firstly, we looked at modernization through library consolidation and showed how you can operate on multiple PDLs including PDF, PCL, PostScript® and XPS, all using a single Mako SDK library. We then looked at adopting automated workflows with Mako and demonstrated how to analyze and redact text automatically in a PDF, using Mako’s layout analysis and text search capabilities. Finally, we showed how you can make the most of print infrastructure-as-a-service by integrating Mako with Microsoft’s Universal Print, including modifying and redirecting print jobs.

Thankfully, nothing did go wrong and if you missed it, don’t worry. We recorded everything and you can watch the recording on demand here:

Also, feel free to watch some of our other past webinars on our YouTube channel to find out more about the Mako SDK.

For more information about Mako, visit globalgraphics.com/mako

Further reading

  1. Carry out complex tasks for your print workflow easily with Mako
  2. Improving PDF accessibility with Structure Tagging

About the author

Andy Cardy, Principal Engineer at Global Graphics Software
Andy Cardy, Principal Engineer at Global Graphics Software

To be the first to receive our blog posts, news updates and product news why not subscribe to our monthly newsletter? Subscribe here

Follow us on LinkedInTwitter, and YouTube

The Digimarc interview: Fast, efficient print production with variable data printing

The impact of poorly constructed PDF files on production schedules has increased as press resolution, colorant count, speed, and width rise – greatly increasing the data rate required to drive them.

This increase in data places additional demands on the processing power of the DFE and risks slowing down the digital press: a delay of half a second on every page of a 10,000-page job adds 90 minutes to the whole job, while for a job of a million pages an extra tenth of a second per page adds 24 hours to the total processing time.

In his guide: Full Speed Ahead – How to make variable data PDF files that won’t slow your digital press, Martin Bailey, distinguished technologist at Global Graphics Software, gives some technical recommendations as to how you can make sure that you don’t make a PDF file for a variable data job that will bring a digital press to its knees. It provides objective information for graphic designers, print buyers, production managers, press operators, owners of PSPs, and developers of digital presses and composition tools.

Martin has just released a second edition of the guide and in this film he talks about the updates to Digimarc‘s marketing communications manager, Rob Fay. Digimarc provides additional functionality to Global Graphics’ software platforms and is a sponsor of the guide.

Topics in the interview include:

  • The guide’s purpose and target audiences
  • Background on updates related to the standards PDF/X-6 and PDF/VT-3
  • Differences in the various VDP applications: traceability; trackability; and personalization
  • Recent improvements in DFE (digital front end) technology that are enabling more advanced VDP

Martin Bailey, CTO, Global Graphics Software, and Rob Fay of Digimarc

WATCH THE INTERVIEW HERE

DOWNLOAD THE GUIDE HERE

Further reading:

  1. What’s the best effective photographic image resolution for your variable data print jobs?
  2. Why does optimization of VDP jobs matter?

To be the first to receive our blog posts, news updates and product news why not subscribe to our monthly newsletter? Subscribe here

Follow us on LinkedInTwitter, and YouTube

Is your printer software up to the job? The impact of rising data rates on software evolved from traditional print processes

Direct™ product manager Ian Bolton explores the impact of using software that has evolved from traditional print processes to drive digital inkjet presses as they advance to print faster, in higher resolution, a wider variety of colors and applications. In particular, Ian focuses on the impact that rising data rates have on the workflow:

Digital press software evolved from traditional print processes has already reached its limit. Digital presses are becoming higher resolution – most are moving from 600 dpi to 1200 dpi, quadrupling the data. They’re also becoming deeper, with up to 7 drop sizes – and these drops are being made from a wider variety of colors. Digital presses are also becoming wider, up to 4 meters wide, and faster,  up to 1,000 feet per minute!

And what if you need to print where every item is different? For example, fully personalized – like curtains, flooring, wall coverings, clothing etc. All of these require software that can deliver ultra-high data rates.

Let’s look at how those data rates scale up as digital presses advance:

The next generation presses demand ultra-high data rates
The next generation presses demand ultra-high data rates

 

If we start with 600 dpi, 20 inches wide, 3 drop sizes and 100 m per minute, then that’s 120 MBps per colorant, which is not too challenging. But once we move up to 1200 dpi, we’ve now quadrupled the data to 480 MBps, which is the read speed of all but the most bleeding-edge solid state drives today.

With printhead, nozzle and roller technology improving, the rated speeds also increase, so what happens when we go up to 300 m per min? It’s now 1.4 GBps and you will need one of those bleeding-edge solid state drives to keep up, bearing in mind you will now be writing as well as reading.

And if we go wider to print our wallcoverings at 40 inches wide, we’re now at 2.8 GBps … and we want our walls to look great close up, so we might be using 7 drop sizes, which takes us up to 5.7 GBps … and this is all just for one colorant!

Based on these numbers, it should be clear now that, for this generation of digital presses and beyond, a disk-based workflow just isn’t going to cut it: reading and writing this amount of data to disk would not actually be fast enough and would require ridiculous amounts of physical storage. This is where software evolved from traditional workflows hits a barrier: the data rate barrier.

To solve this we need to go back to the drawing board. It’s similar to the engineering challenge of moving from propeller-driven aircraft to jets that could break the sound barrier. Firstly, you need to develop a new engine and then you need to commercialize it.

So, if you’re looking for software to power your first or next digital press it’s going to need the right  kind of software engine that isn’t based on disk technology so that you can drive your digital press electronics directly and smash through the data rate barrier. In other words, you need to go Direct.

To learn more about the impact of rising data rates and how you can futureproof your next digital press, visit our website to find out more about going Direct.

If you’re interested in calculating data rates take a look at this blog post where you can download your own data rate calculator: Choosing the class of your raster image processor

Further reading:

  1. Harlequin Core – the heart of your digital press
  2. What is a raster image processor 
  3. Ditch the disk: a new generation of RIPs to drive your digital press
  4. Is your printer software up to the job?
  5. Where is screening performed in the workflow
  6. What is halftone screening?
  7. Unlocking document potential
  8. Future-proofing your digital press to cope with rising data rates

About the author

Ian Bolton, Product Manager, Direct
Ian Bolton, Product Manager – Direct
Ian has over 15 years’ experience in industry as a software engineer focusing on high performance. With a passion for problem-solving, Ian’s role as product manager for the Direct™ range gives him the opportunity to work with printer OEMs and break down any new technology barriers that may be preventing them from reaching their digital printer’s full potential.

To be the first to receive our blog posts, news updates and product news why not subscribe to our monthly newsletter? Subscribe here

Follow us on LinkedInTwitter and YouTube

Second edition now available: Full Speed Ahead: How to make variable data PDF files that won’t slow your digital press

At the beginning of 2020, in what we thought was the run-up to drupa, Global Graphics published a new guide called “Full Speed Ahead: How to make variable data PDF files that won’t slow your digital press”. It was designed to complement the recommendations available for how to maximize sales from direct mail campaigns, with technical recommendations as to how you can make sure that you don’t make a PDF file for a variable data job that will bring a digital press to its knees. It also carried those lessons into additional print sectors that are rapidly adopting variable data, such as labels, packaging, product decoration and industrial print, with hints around using variable data in unusual ways for premium jobs at premium margins.

Well, as they say, a lot has happened since then.

And some of that has been positive. At the end of 2020 several new International Standards were published, including a “dated revision” (a 2nd edition) of the PDF 2.0 standard, a new standard for submission of PDF files for production printing: PDF/X-6, and a new standard for submission of variable data PDF files for printing: PDF/VT-3.

We’ve therefore updated Full Speed Ahead to cover the new standards. And at the same time we’ve taken the opportunity to extend and clarify some of the rest of the text in response to feedback on the first edition.

So now you can keep up to date, just by downloading the new edition!

DOWNLOAD THE GUIDE

Further reading:

  1. What’s the best effective photographic image resolution for your variable data print jobs?
  2. Why does optimization of VDP jobs matter?

To be the first to receive our blog posts, news updates and product news why not subscribe to our monthly newsletter? Subscribe here

Follow us on LinkedIn and Twitter

MPI Tech joins the Global Graphics Software Partner Network

MPI Tech joins the Global Graphics Software Partner Network

Leading provider of document management and document output solutions MPI Tech has joined the Global Graphics Software Partner Network as a technology partner.

MPI Tech will enable Global Graphics Software’s Harlequin Direct™ and Fundamentals™ products to support AFP and IPDS  jobs. AFP (Advanced Function Presentation) is the most widely used format for high-speed transactional printing in many industries including finance, insurance, manufacturing, health care and education. IPDS (Intelligent Printer Data Stream) is the print description language (PDL) to print AFP documents online.

MPI Tech offers a range of solutions to process AFPDS and native AFP/IPDS at speeds over 6,000 ipm or convert them into the most popular PDL (PCL, PDF, PDF/A, PS etc) on almost every platform (Windows, AIX, Linux, Solaris, UNIX).

Justin Bailey, managing director of Global Graphics Software commented: “We’re pleased to welcome MPI Tech to our partner network. With a proven technology and know-how for processing AFP or IPDS print jobs, Global Graphics Software turns to MPI Tech as its ‘go-to partner’ when our customers require solutions for these transactional print data-streams.”

MPI Tech has been a licensee of Global Graphics technology for many years, using it for converting to, and processing, PostScript, PDF, and other PDLs.

If you’re interested in joining the Partner Network visit our website to find out more.

 

Image Access joins the Global Graphics Software Partner Network

Image Access offers book, flatbed, sheetfeed, duplex & art scanners for digitizing large format originals for archives, libraries, museums & industry.
Image Access offers book, flatbed, sheetfeed, duplex & art scanners for digitizing large format originals for archives, libraries, museums & industry.

Leading German scanner manufacturer Image Access GmbH has joined the Global Graphics Software Partner Network as a technology partner. The two companies began working together last year to develop a solution that would enable printer vendors to implement a new family of Image Access line scan bars called WideSCAN into their printers to support the creation of PrintFlat™ calibrations. The WideSCAN scan bars, together with PrintFlat, will dramatically increase print quality with minimal impact on overall production time.

PrintFlat technology maximizes the print quality of your printer by addressing common defects found in inkjet printing, including banding and stripes, as well as mottling, streaking and chaining. By adding PrintFlat to a print workflow it helps to boost printer sales and reduce support calls for OEMs by expanding the range of printable jobs and consistently achieving exceptional image quality.

For the PrintFlat process to work, printed target pages need to be scanned and the resultant image data fed into the PrintFlat solution. This is where the Image Access partnership comes into play: the WideSCAN scan bars will be available in various sizes in 12-inch increments, starting at 24 inches, and are the most compact and easy-to-use line scan bars in the industry.

Justin Bailey, managing director of Global Graphics Software commented: “Image Access are known for their experience and innovative use of emerging camera technologies and I’m really pleased to welcome them to our Partner Network. Introducing the new WideSCAN scan bar is a game changer for printer manufacturers. Now they will be able to implement the scan bar into the printer so that calibration and PrintFlat correction can all be done inline on the printer, making that device PrintFlat Ready.”

Image Access has manufactured wide format scanners, including brands Bookeye® and WideTEK®, for more than 25 years.

If you’re interested in joining the Partner Network visit our website to find out more.