Using the Mako Core SDK to modify documents in Microsoft’s Universal Print

Over the past year, Microsoft has been working hard to bring its new Cloud printing service, Universal Print, to general availability.

As a part of Universal Print, developers get access to a set of Graph APIs that allows analysis and modification of print job payload data. This feature enables a few different scenarios, including adding security (e.g. redactions or watermarks) to a Universal Print-based workflow.

As a curious engineer, I wanted to see how different it would be for an independent software vendor (ISV) to use our Mako™ Core SDK to modify a print job flowing through Universal Print, instead of using a more traditional route of using a virtual printer driver.

Thinking about the workflow a little more, I came up with the following design:

Using the Mako SDK to modify documents in Universal Print.
Using the Mako SDK to modify documents in Universal Print.

In the design above, we can see the end-user’s Word document gets printed to a virtual printer. This allows the ISV to be notified of the job, and modify it accordingly using Mako. Once modified, the ISV then redirects the job on to the physical printer for printing.

There’s a couple of nice things about this design:

Firstly, it uses the Graph API to access Universal Print, which is an easy-to-use and well documented REST API. Secondly, since the functionality is accessed via a REST API, it allows our ISV service to be written in whichever Mako supported language we like.

I chose C# to make best use of the C# Graph API SDK.

Developing the service

There are five main steps to developing the service:

  1. Handle print job notifications
  2. Download the print job payload
  3. Modify the payload
  4. Upload the payload
  5. Redirect to the target printer

Handle print job notifications

To be notified of print jobs in Universal print, you can use the Graph’s change notifications. These will allow you to sign up to a notification, which will call a provided webhook.

Download the print job payload

Once we have notification that a print job has been sent to our virtual printer, we can start downloading its payload.

Here we use the appropriate Graph APIs, along with standard Graph authentication to access the print job’s document. We then simply save it to disk.

Modify the payload

Once we have the document on disk (although Mako can also modify streams too!), we can open the document and modify it using Mako’s document object model (DOM).

Alternatively, Mako can also convert from one page description language (PDL) to another. This is useful in situations where your destination printer doesn’t support the input PDL.

Upload the payload

Uploading the modified document is straightforward. This time we use the Graph API to create an upload session, and use the WebClient class to put the document back into the original print job.

Redirect to the target printer

And finally, after the print job has been updated, we can redirect it onto another printer. This redirection also automatically completes the print job and task.

Alternatively, if we want to be a little more green, we could always send the document to OneDrive, Sharepoint, or another document management system. After doing so, you then complete the print job and its associated task.

See it in action

We actually coded this demo live in our last Mako webinar, showing an implementation where an ISV wants to automatically redact content.

Access the code directly at our GitHub repository or watch the webinar recording below:

Try it out

We’re keen to talk to you about your Universal Print project and see how we can help. Contact us here.

For more information about Mako, visit globalgraphics.com/mako.

About the author

Andy Cardy, Principal Engineer at Global Graphics Software
Andy Cardy, Principal Engineer at Global Graphics Software

Further reading:

  1. Carry out complex tasks for your print workflow easily with Mako
  2. Improving PDF accessibility with Structure Tagging

Be the first to receive our news updates and product news. Why not subscribe to our monthly newsletter? Subscribe here

Follow us on LinkedIn and Twitter

Second edition now available: Full Speed Ahead: How to make variable data PDF files that won’t slow your digital press

At the beginning of 2020, in what we thought was the run-up to drupa, Global Graphics published a new guide called “Full Speed Ahead: How to make variable data PDF files that won’t slow your digital press”. It was designed to complement the recommendations available for how to maximize sales from direct mail campaigns, with technical recommendations as to how you can make sure that you don’t make a PDF file for a variable data job that will bring a digital press to its knees. It also carried those lessons into additional print sectors that are rapidly adopting variable data, such as labels, packaging, product decoration and industrial print, with hints around using variable data in unusual ways for premium jobs at premium margins.

Well, as they say, a lot has happened since then.

And some of that has been positive. At the end of 2020 several new International Standards were published, including a “dated revision” (a 2nd edition) of the PDF 2.0 standard, a new standard for submission of PDF files for production printing: PDF/X-6, and a new standard for submission of variable data PDF files for printing: PDF/VT-3.

We’ve therefore updated Full Speed Ahead to cover the new standards. And at the same time we’ve taken the opportunity to extend and clarify some of the rest of the text in response to feedback on the first edition.

So now you can keep up to date, just by downloading the new edition!

DOWNLOAD THE GUIDE

Further reading:

  1. What’s the best effective photographic image resolution for your variable data print jobs?
  2. Why does optimization of VDP jobs matter?

To be the first to receive our blog posts, news updates and product news why not subscribe to our monthly newsletter? Subscribe here

Follow us on LinkedIn and Twitter

Shade and color variation in textile printing – why it’s important and what you can do about it

Printing soft furnishings

With fewer design limitations, a faster turnaround, no minimum run length and higher margins (not to mention reduced use of power and water, and of pollution), it’s not surprising that the digitally printed textile market is growing.1 Inkjet has certainly made textile design and printing much more flexible than screen printing – and that goes for everybody involved, from the designer through the printing company, to the buyer.

But printing textiles on inkjet doesn’t come without its challenges: as a software provider focusing on print quality issues, we often hear from print service providers who can only digitally print two thirds of the jobs they receive because they would not be paid for the quality they could achieve on the others.

Shade or color variation is a common problem. It’s not new in digital printing (it’s always been an issue for screen-printed and dyed textiles as well) and is usually managed by providing a shade band, which printer operators refer to, to check allowable color variations between pieces.

But, unlike screen-printing or dyeing, the color variation on an inkjet press can be visible over a small distance, just a few centimeters, and this results in visible bands across the output. Banding describes features that tend to be 1 – 10 cm across and they’re often caused by variation of inkjet pressure or voltage differences within the head, which typically results in a frown or smile shape. We also see a certain amount of manufacturing variation between heads so that one may print lighter or darker than the head next to it in a print bar. Some types of heads can also wear in use, which can result in less regular banding that can change over time. This means that large areas which should be flat color may not be.

When such a variation occurs it can greatly complicate a lot of post-print steps, especially if you need to put more than one piece of textile together, either in sewing or use (such as a pair of curtains). If that’s the case, then a significant difference may be unacceptable and your printing rejected by your buyer. Ultimately this leads to print service providers rejecting jobs, because they know their digital press can’t handle printing those tricky flat tints or smooth tones.

What can you do about it?

The first thing many companies do to try to overcome this banding is to adjust the voltage to the inkjet head, but this is often time-consuming and expensive because it requires an expert technician. A better alternative is to make the correction in software, which is a more cost-effective and faster solution. It means it can be automated and can act at a much finer granularity, so printing is more accurate. There’s no need to mess with controls that could damage the press, and printing companies themselves can make corrections without the vendor sending a technician on-site.

Our solution at Global Graphics Software for improving banding is PrintFlat™. It corrects tonality to hide banding based on measurements from the press. It adjusts every nozzle separately and doesn’t need a specialist engineer to make press adjustments. PrintFlat can be integrated into different digital front ends, using a variety of RIPs, including Caldera and Colorgate and, not to mention, our own Harlequin RIP®.

Over the years of working with many press manufacturers we’ve discovered that many technical issues and solutions are common across different sectors, including transactional, wide-format, commercial, labels and packaging, and industrial, including ceramics, wall coverings, flooring and of course textiles. That means that we already have years of experience in correcting for banding. Using PrintFlat in your press means print service providers can now take on those jobs they would normally reject.

To learn more about how to eliminate shade and color variation when printing on an inkjet press, listen to Global Graphics Software’s CTO Martin Bailey’s talk for FESPA 2020:

“New techniques to eliminate in-lot shade variation when printing textiles with inkjet.”

Or visit the PrintFlat website: https://www.printflat.com/

Further reading:

  1. What causes banding in inkjet? (And the smart software solution to fix it.)
  2. Streaks and Banding: Measuring macro uniformity in the context of optimization processes for inkjet printing

To be the first to receive our blog posts, news updates and product news why not subscribe to our monthly newsletter? Subscribe here

Follow us on LinkedIn and Twitter

_______________________________

  1. Digitally printed textiles are estimated to be between 2% – 5% of the total printed textiles market. Estimated at $146.5 billion in 2018 by Grand View Research

What is a Raster Image Processor (RIP)?

Ever wondered what a raster image processor or RIP does? And what does RIPping a file mean? Read on to learn more about the phases of a RIP, the engine at the heart of your Digital Front End (DFE).

The RIP converts text and image data from many file formats including PDF, TIFF™ or JPEG into a format that a printing device such as an inkjet printhead, toner marking engine or laser platesetter can understand. The process of RIPping a job requires several steps to be performed in order, regardless of the page description language (such as PDF) that it’s submitted in. Even image file formats such as TIFF, JPEG or PNG usually need to be RIPped, to convert them into the correct color space, at the right resolution and with the right halftone screening for the press.

Interpreting: The file to be RIPped is read and decoded into an internal database of graphical elements that must be placed on the output. Each may be an image, a character of text (including font, size, color etc), a fill or stroke etc. This database is referred to as a display list.

Compositing: The display list is pre-processed to apply any live transparency that may be in the job. This phase is only required for any graphics in formats that support live transparency, such as PDF; it’s not required for PostScript language jobs or for TIFF and JPEG images because those cannot include live transparency.

Rendering: The display list is processed to convert every graphical element into the appropriate pattern of pixels to form the output raster. The term ‘rendering’ is sometimes used specifically for this part of the overall processing, and sometimes to describe the whole of the RIPing process.

Output: The raster produced by the rendering process is sent to the marking engine in the output device, whether it’s exposing a plate, a drum for marking with toner, an inkjet head or any other technology.

Sometimes this step is completely decoupled from the RIP, perhaps because plate images are stored as TIFF files and then sent to a CTP platesetter later, or because a near-line or off-line RIP is used for a digital press. In other environments the output stage is tightly coupled with rendering, and the output raster is kept in memory instead of writing it to disk to increase speed.

RIPping often includes a number of additional processes; in the Harlequin RIP® for example:

  • In-RIP imposition is performed during interpretation
  • Color management (Harlequin ColorPro®) and calibration are applied during interpretation or compositing, depending on configuration and job content
  • Screening can be applied during rendering. Alternatively it can be done after the Harlequin RIP has delivered unscreened raster data; this is valuable if screening is being applied using Global Graphics’ ScreenPro™ and PrintFlat™ technologies, for example.

A DFE for a high-speed press will typically be using multiple RIPs running in parallel to ensure that they can deliver data fast enough. File formats that can hold multiple pages in a single file, such as PDF, are split so that some pages go to each RIP, load-balancing to ensure that all RIPs are kept busy. For very large presses huge single pages or images may also be split into multiple tiles and those tiles sent to different RIPs to maximize throughput.

The raster image processor pipeline. The Harlequin RIP includes native interpretation of PostScript, EPS, DCS, TIFF, JPEG, PNG and BMP as well as PDF, PDF/X and PDF/VT, so whatever workflows your target market uses, it gives accurate and predictable image output time after time.
The raster image processor pipeline. The Harlequin RIP includes native interpretation of PostScript, EPS, DCS, TIFF, JPEG, PNG and BMP as well as PDF, PDF/X and PDF/VT, so whatever workflows your target market uses, it gives accurate and predictable image output time after time.

Harlequin Host Renderer brochure

 

To find out more about the Harlequin RIP, download the latest brochure here.

 

This post was first published in June 2019.

Further reading:

1. Where is screening performed in the workflow

2. What is halftone screening?

3. Unlocking document potential


To be the first to receive our blog posts, news updates and product news why not subscribe to our monthly newsletter? Subscribe here

Follow us on LinkedIn and Twitter

 

What’s the difference between PDF/X-1a and PDF/X-4?

PDFX-1 PDFX-4

Which PDF/X should I use?

Somebody asked me recently what the difference is between PDF/X-1a (first published in 2001) and PDF/X-4 (published in 2010). I thought this might also be interesting to a wider audience.

Both are ISO standards that deliberately restrict some aspects of what you can put into a PDF file in order to make them more reliable for delivery of jobs for professional print. But the two standards address different needs/desires:

PDF/X-1a content must all have been transformed into CMYK (optionally plus spots) already, so it puts all of the responsibility for correct separation and transparency handling onto the creation side. When it hits Harlequin, all the RIP can do is to lock in the correct overprint settings and (optionally) pre-flight the intended print output condition, as encapsulated in the output intent.

On the other hand, PDF/X-4 supports quite a few things that PDF/X-1a does not, including:

  • Device-independent color spaces
  • Live PDF transparency
  • Optional content (layers)

That moves a lot more of the responsibility downstream into the RIP, because it can carry unseparated colors and transparency.

Back when the earlier PDF/X standards were designed transparency handling was a bit inconsistent between RIPs, and color management was an inaccessible black art to many print service providers, which is why PDF/X-1a was popular with many printers. That’s not been the case for a decade now, so PDF/X-4 will work just fine.

In other words, the choice is more down to where the participants in the exchange want the responsibility to sit than to anything technical any more.

In addition, PDF/X-4 is much more easily transitioned between different presses, and even between completely different print technologies, such as moving a job from offset or flexo to a digital press. And it can also be used much more easily for digital delivery alongside using it for print. For many people that’s enough to push the balance firmly in favour of PDF/X-4.

For further reading about PDF documents and standards:

  1. Full Speed Ahead: How to make variable data PDF files that won’t slow your digital press
  2. PDF Processing Steps – the next evolution in handling technical marks
  3. Compliance, compatibility, and why some tools are more forgiving of bad pdfs

About the author

Martin Bailey, CTO, Global Graphics Software
Martin Bailey, CTO, Global Graphics Software

Martin Bailey is Global Graphics’ Chief Technology Officer. He’s currently the primary UK expert to the ISO committees maintaining and developing PDF and PDF/VT and is the author of Full Speed Ahead: how to make variable data PDF files that won’t slow your digital press, a new guide offering advice to anyone with  a stake in variable data printing including graphic designers, print buyers, composition developers and users.

Be the first to receive our news updates and product news. Why not subscribe to our monthly newsletter? Subscribe here

Follow us on LinkedIn and Twitter

What’s the best effective photographic image resolution for your variable data print jobs?

It goes without saying that the final quality of your printed piece is paramount. But when speed and time constraints are also critical, what can you do to ensure your files fly through the press and still reward you with the quality you expect? Optimizing the images in the piece is a good place to start, but if you’re creating a job with variable data, where there are thousands of pages to print, each with a different image, how do you know what a sensible effective resolution is for those images that will ensure your PDF file doesn’t trip up the print production workflow?

In his latest guide, Full Speed Ahead, how to make variable data PDF files that won’t slow your digital press, Martin Bailey, CTO at Global Graphics Software, advises not to ask the print workflow to do more work than necessary if that doesn’t change the look of the printed result. Images are commonly re-used within a VDP job, so being able to process each image only once and then re-use the result many times can significantly increase the throughput of the digital front end. On the other hand, some images are personal to every recipient and must therefore be processed for every single recipient, slowing the workflow down.

Martin offers the following tips for setting appropriate effective photographic image resolutions:

  1. Aim for 300 ppi, however the most appropriate image resolution for digital presses varies, depending on printing heads, media and screening used.
  2. Bear in mind image content; soft and dreamy images can be sometimes placed at a lower resolution.
  3. Don’t use a higher effective image resolution for photographic images than the output resolution as this is often not productive. The example in Fig 1 below illustrates how easy it is to use an image at several times the required resolution:

The same 12-megapixel image at 3 different sizes

Fig 1: The same 12-megapixel (4000 x 3000 px) image placed on the page at three different sizes. Source: Full Speed Ahead, how to make variable data PDF files that won’t slow your digital press.

When an image is placed onto a page the original resolution of that image is largely irrelevant; what matters is how many pixels there are per inch on the final printed page. As an example, if you have a photograph from a 12 MP compact camera it’ll probably be approximately 3000 pixels by 4000 pixels. If that’s placed on the page as 3 inches by 4 inches (7.5 x 10cm) the effective resolution is about 1000ppi (4000/4). That would usually be about three times as much as you need in each dimension.

A variety of tools are available for optimizing image resolution, and some composition tools can also do this automatically. To find out more about the best effective resolution for your images, and to pick up more tips for optimizing your images for variable data printing, download the guide:

Full Speed Ahead: how to make variable data PDF files that won't slow your digital press edited by Global Graphics Software

Full Speed Ahead – how to make variable data PDF files that won’t slow your digital press.

Be the first to receive our blog posts, news updates and product news. Why not subscribe to our monthly newsletter? Subscribe here

Follow us on LinkedIn and Twitter

 

Adjusting rendering of outlined text in Harlequin

By Martin Bailey, CTO, Global Graphics Software

In several sectors of the print market it is common practice to convert text to outlines upstream of a RIP, on the grounds that it’s then impossible for the wrong glyph to be printed. This is normal, for instance, in much of the label and packaging industry, especially when there is very robust regulation in place, such as in pharmaceuticals.

Every page description language defines “scan conversion” rules that specify which pixels should be marked when a graphic is painted onto a page; these build on the concept of “pixel touching”, specifying exactly when a vector shape counts as touching a pixel and therefore marking it.

When you’re using PDF (or PostScript, before that) the scan conversion rules are different for text specified using live fonts and for vector shapes. If you started with live text and then converted it to outlines then you have switched from using the text scan conversion rules to using the vector graphic rules. That has always meant that text converted to outlines tends to render slightly heavier than text using live fonts. And the smaller the text is, the more the weight difference becomes apparent.

FIG 1
FIG 1 – 2pt text in Times Roman showing various scan conversion rules.

In Fig 1 you can see this difference very clearly for very small Western text rendered at 2pt and 600dpi, still a common resolution for digital printers and presses. The top line shows text using live fonts, and the second line shows the PDF scan conversion rule for a vector fill. Note that at 2pt the RIP only has about 12 pixels for the height of an upper-case glyph.

In early 2018 we added a new scan conversion rule for vector fills alongside our pre-existing rules in the Harlequin RIP. The intention was to make it possible to emulate the much lighter output that Esko’s FlexRIPs produce. Unfortunately, it also tended to emulate the ability for very fine structures, especially fine horizontal strokes in small text, to disappear. You can see this in the third row of text in Fig 1.

This is obviously not an optimal solution, so we continued our development, and have now extended the original solution with what is called “dropout control”. This prevents very fine sections of a vector fill “dropping out” when they manage to fall on the page in such a way that they don’t cross the locations in the pixels that would trigger anything being marked. You can see the effect of this in the bottom line in Fig 1.

Light rendering with dropout control was delivered to our OEM partners in late 2018 under the name RenderAccurate.

Even this optimized output won’t exactly match the output of live fonts, because the fonts themselves often include hints to the rendering engine, designed to ensure maximum legibility and conformance to the font designer’s vision. These hints can, for instance, ensure that vertical stems are the same width in all glyphs, or that the curved base of a glyph will extend slightly below the baseline to make it visually balance with glyphs with flat bases that sit on the baseline. Those hints were discarded when the text was converted to an outline, and so can’t be used any more. But the new scan conversion algorithm certainly strikes a good balance between matching the weight of live fonts and maintaining legibility.

The effect is visible in very small text in Latin fonts, as shown in Fig 1, but the impact is often masked by the physical effects of printing. And Latin glyphs tend to be relatively simple, so that the human eye and brain are pretty good at filling in the missing segments without too much impact on legibility or comprehension.

On the other hand, Chinese, Japanese and Korean (CJK) fonts are often more complex, with the result that the effect is visible at larger point sizes. And the meaning can be obscured or altered much more easily if strokes are missing. Fig 2 illustrates the same effects on Japanese text at 3pt, rendered at 600dpi. At this size and resolution, the RIP has about 22 pixels for the height of each glyph.

FIG 2
FIG 2 – 3pt text in MS Mincho, showing, from top to bottom: live fonts; default rendering for outlined text; the new, lighter, outlined text; and lighter text with dropout control.

The glyphs shown in FIG 2 are complex compared to Western scripts, but any solution that will be used with CJK scripts must obviously also be proven with the most complex character shapes, such as the Kanji in FIG 3. Some of these have so many horizontal strokes that they simply cannot be rendered with fewer than 22 device pixels vertically and require more than that for reliable rendering. The sample in this figure is rendered at with around 27 pixels for the height of each glyph.

FIG 3
FIG 3 – More complex Kanji in KozGoPro-Regular, showing, from top to bottom: live fonts; default rendering for outlined text; and the new, lighter text with dropout control.

This article has deliberately used very small text sizes as examples, simply because the effects are easier to see. But the same issues arise at larger sizes as well, albeit more rarely.

On the other hand, it is precisely because the issue appears more rarely, and because the effects are less immediately noticeable, that makes the risk of dropping strokes so dangerous. It’s perfectly possible that an occasional missing stroke, perhaps in an unusually light font, may go unnoticed in process control. And that might result in a print that disappoints a brand owner, or even that fails a regulatory check, after the label has been applied or the carton converted and filled, or even after the product being shipped.

So, when a brand demands lighter rendering of pre-outlined fonts, make sure you’re safe by also using dropout control in your RIP!