The Need For Speed

Color correction systems are so different that it can be quite difficult to compare them in an objective way. Consequently, many people settle for a simple test. For better or worse hardware color correctors are most often judged by how many windows they can do. It is usually a flawed comparison, but it is easy to make. Software color systems however, can have an infinite number of windows, so a new method of evaluation is needed. One current favorite is to compare speed. That seems simple enough, but actually it is quite complicated.

What is Real Time?

I have always maintained that color correction needs to be real-time so that the hand can correct faster than the eye adjusts. In video, real-time is easily defined as the correct playback speed. Admittedly, there are an increasing number of correct playback speeds; 24fps for film, 25fps for PAL, 29.97 fps for NTSC and more recently 23.98 fps but the speed of video is defined by the standard not the equipment.

Software systems are not limited to fixed standards and system speed is measured by playback, transfer speed, render speed, control interactivity and workflow. Playback and transfer speeds are directly related to the performance and number of processors, the amount of RAM, and the performance of the storage. The render speed and control interactivity are further affected by the efficiency of the software and the interface design. All of these are useful ways to evaluate a system, but equally important is how efficiently they integrate into the workflow. 

Real-time control is most important of all. Changes must take effect as a knob is turned because it is hard to fine-tune the image if there is any delay. When slow processing causes the delay, changes step and are applied in bursts. When a buffer causes the delay, changes are smooth but not immediate, which is not quite so bad. Real-time control needs to be just fast enough and can be as little as twelve updates a second on a static frame. 

Color Tools

Given real-time interactivity I, as a colorist, consider the color toolset the most significant feature of a color corrector. I have to write such a truism because color tools are often not given enough consideration, perhaps because they are so hard to compare. The justification is usually that we can assume the color tools are great, but in truth some are only just good enough, and others are awkward to use.

In 1868 Christopher Latham Sholes rearranged the letters on his invention, the typewriter, so that the most common letter combinations were separated and their type bars less likely to clash. Contrary to popular legend, the intention was not to slow down typists by making the layout less intuitive, but to make them faster by avoiding jams. Over 130 years later we are still using the QWERTY layout on our computers. 

What is the moral of the story? Good design does improve speed, but there is a huge resistance to change. It is essential to optimize a design from the beginning. Systems that use generic third party control surfaces cannot redesign the panel, but can nevertheless design an efficient and intuitive layout. However, since each system uses a different layout it can be confusing for colorists, just like re-arranging the letters on a computer keyboard. Several companies build their own panels, but fall into other traps. Panels based on a graphic user interface tend to group controls into logical groups, but a colorist might need to access several of these groups to complete a single task. Some panels are re-purposed from hardware color systems, and there is a natural temptation to map controls to familiar places. This again is likely to be inefficient because newer software tools are likely to be less accessible. An efficient control panel must be intuitive, and minimize key presses and menu changes. In short it needs to be based on the workflow of experienced colorists.

Comparing Tasks

Counting the number of layers that work real-time is a good speed test if the layers are identical, but in practice they rarely are. The problem is that one system might use a new layer for each tool, whereas another has multiple toolsets in every layer. Even the concept of layers is questionable, since it implies that a system applies grades are built up in stages. In fact some systems are built with tools in a fixed processing sequence so that the tools themselves define the order of operations. Setting a number of common tasks is a better performance test since it takes into account panel ergonomics, toolset flexibility and processing speed. 

Acceleration

Having established the speed of a system, it is also useful to figure how easily and cheaply it can be accelerated. In terms of hardware, proprietary systems are typically most expensive, multi-computer solutions are next expensive and single computer, off the shelf systems are likely to be the cheapest to upgrade. From a software point of view, simpler hardware configurations encourage the development of more efficient code. Some systems depend heavily on dedicating hardware to boost performance, while others rely on optimized algorithms.

Using   CPU clusters by tiling the image is an example of a hardware solution to software performance. Clusters work well on basic color correction, but suffer from increased over-heads, causing diminishing returns on many of the most intensive tasks. Pan and scan, zoom, defocus and noise reduction need most or all the image to work from, so the actual benefits are diminished. Clusters are of course more expensive to upgrade too. With the realistic expectation of a single PC unit with eight quad core processors and two graphics cards in the near future, investment in cluster based image enhancement is debatable.

Workflow

The workflow integration of hardware is straightforward to assess. Connect a video source, color correct it, and then record it. For the least demanding jobs, such as one-light dailies, the job time is little more than the running time.

Software systems that need to import material, color correct, render, and then record, would seem to need at least three times as long. So how can software possibly be as fast, or faster than hardware?

In fact the linear hardware workflow has a lot in common with the typewriter. Put in paper, type and take out the finished article. Using this analogy, software DI workflow is more like a word processor. Information can be input from a variety of sources, modified, re-arranged, reviewed and then finally sent to a printer.

In the future software might well be faster than video hardware because the real-time speed, which seems impressive now, will one day be a limitation. In the data world transfers can be faster than real time, so one-light dailies could theoretically be assessed, processed, rendered and transferred in less than the playback time. Sounds fantastic perhaps, but film laboratories have been printing at faster than running time for decades.

 However, a more useful workflow example is long-form broadcast. The traditional video route is best light from film, offline edit, online edit, color correction and then a final edit for titles. It is very time efficient, but can compromise quality because final grading is often from compressed videotape and is dependent on the initial best light grade. Visual effects and transitions also complicate the final grading. Software based digital intermediate (DI) workflow is very similar to this model, except that the initial transfer is a calibrated log scan that does not involve colorist decisions, the final grade is from uncompressed data with greater dynamic range than standard video, and the effects and transitions are graded as elements, not a flattened, rendered sub-master. So

DI has all the benefits and none of the disadvantages. Software has other workflow advantages that make it attractive. For example when working with random access of scenes arranged in context it is convenient to do a base grade on each shot, taking note of which shots need most attention. A colorist can easily review and refine a project many times. In a telecine or tape environment, reviewing the project is time consuming, especially if the material is on several rolls of film. As a result the traditional workflow involves spending a long time on each shot, minimizing the need to repeatedly shuttle or change rolls. This makes it difficult to know in advance which shots need the most work, and slow if a decision later in the session affects a scene that was graded earlier.

Another significant advantage of the software DI workflow is multi-tasking. In typical video workflows, each stage of the post-production process needs to be completed and approved before going on to the next. For example, selected takes must be cleaned and graded before online editing, whereas in the DI workflow grading, dust busting and editing can all happen simultaneously and the results easily updated by any of the operators. This not only reduces the overall project time, but it also eases the pressure on scheduling. There is a caveat though. Systems that do not work from open architecture shared storage cannot properly multi-task. As a guide, any system that has to load material on to its own local storage requires at least one extra data transfer, and cannot benefit from different operators working in parallel.

Although the usual workflow for short form productions is quite different, there are a growing number of facilities using software color enhancement for commercials and music videos. This could be explained by the lower cost of a software suite. However, these new color suites must be competitive to exist at all, and since they are competing with existing rooms that are already paid for, cost cannot explain the trend. Clearly the principal benefits of better tools, in context grading, random access, resolution independence, and workflow flexibility outweigh any concerns about speed. One opinion is that clients are so accustomed to working in non real-time editing and compositing suites that they are not averse to rendering for color enhancement. It is indeed the colorist who needs reassurance that software can replace hardware, but those that have already switched are setting new standards that are hard to match in the traditional suites. Some of the usual benefits of DI do not apply fully to a commercial workflow where selected scenes are graded with handles, transferred with original time code and then finished in an online workstation. However, software developers are now developing features that help commercial and video orientated users.  

Productivity

Measuring speed without considering workflow is futile. Imagine buying a car for daily use, based only on the criteria of speed. Without doubt, the quickest cars around are drag racers. They accelerate much faster, and they reach higher top speeds than the cars we usually drive. However, they are designed to race in a straight line, (dare I say they are linear?), over short distances and would be useless for shopping trips, vacations or rush hour traffic.

The most realistic way to compare color correction systems is to repeat the same tasks on the same material. The results can be surprising. Good workflow, like clever design, improves speed, but there is a reluctance to change. The new technologies that have made software DI grading realistic, offer a unique opportunity to re-evaluate workflow. In my mind there is no doubt that the DI workflow is the only way we will work in the future, so it’s worth getting it correct now. After all, how many typewriters are in use today? Most of us use a computer instead, and we usually choose our software based on functionality and compatibility.

I feel the need for speed, how fast can I go?

Happy Coloring!

Commissioned by Digital Vision March 2006

First published by Asia Image, April 200