Anti-alias or Moiré? (buy the Nikon D800 instead of the D800E)

Bottom line: the D800E misses a proper anti-aliasing filter. This probably does more harm than good, so buy the D800 instead, unless you absolutely need maximum per-pixel resolution and know how to avoid Moiré artefacts.
Disclaimer: The D800/D800E are great cameras that differ in subtle ways. Making a purchasing decision based on these differences will necessarily involve “splitting hairs”, but that is probably why you are reading this blog entry, so let’s do that.
I started writing this post before D800E reviews were available online. 
Now that it’s been tested by dpreview and DxO I have to admit that the D800E is less prone to Moiré than I expected it to be. Yet I stand by what I wrote in this post and I still advise you to get the D800 instead of the D800E unless you know exactly what you’re doing. Read on.

The D800 and the D800E

When Nikon announced their brand new D800 and D800E full frame (FX) DSLRs a few months ago, it was a breakthrough. These cameras push the limits in resolution beyond anything previously seen on 35mm and set new benchmarks in DSLR video capabilities. This is great news, because whether or not competitors come up with even better models, you (the consumer) still win. So, we know that they are excellent. All he reviews tell us so. The big question in photographers’ minds is probably whether to get the D800 or the D800E. The “E” sounds somehow more Exotic and Exclusive, and promises even sharper photos. Given this, the D800E’s 10% higher ($3300 vs $3000) price sounds justified. Sort of like a D800 “de luxe” edition, right? Not necessarily.

The D800E is actually missing something, namely something that every sanely designed DSLR has – an anti-alias (AA) filter. To be more exact, the D800E claims to have a special kind of filter that first performs anti-aliasing, and then undoes it again. The net effect is supposedly the same as having no anti-alias filter.

Anti-alias actually means “blurring”. So why do you need an AA filter? You need such a filter to avoid aliasing artefacts, also known as Moiré.

Severe Moiré (false colour bands) caused by the grid on the outside of a astronomical observatory building. Photo from a friend’s Pentax K100D. (click for 1:1 detail)

In a susceptible camera these patterns can appear whenever it tries to capture an image that contains frequencies that are higher that the sensor’s “Nyquist frequency”. This can happen when photographing thin lines or regular repeating pattern such as clothing fabric, bird feathers, or chicken wire from far away. This is especially common with man-made objects encountered in fashion and architectural photographs.

Aliasing: resolution’s ugly sister

Nerd alert: I'll now explain how aliasing/Moiré works. 
Skip to the next colour photograph if you want to be spared 
the details. The brave may keep reading. ;-)

In gray-scale

Let’s ignore colour for the moment (I’ll come back to that in a little while), and think only of a monochrome camera photographing a monochrome world. To make it even easier let us imagine (for the moment) that the sensor is a one-dimensional line of detectors. A one-dimensional monochrome digital camera, if you will. The pixels are regularly spaced, and each one captures a single image intensity (gray-scale) value.

An example image captured by a line of gray-scale DSLR sensor pixels

The maximum spatial frequency that the camera’s sensor can resolve is one that flips from bright and dark, or dark to bright, at every pixel transition. This means that the oscillating dark-bright-dark image completes a cycle every two pixels (a cycle ends back where it started, then repeats. In this case white to white, or black to black).

The same sensor pixels, now displaying the maximum frequency that they could possibly capture. This equals the “Nyquist frequency”.

This limitation is formally expressed in the Nyquist-Shannon sampling theorem, a fundamental theorem in signal processing which says that if you want to accurately describe an oscillating signal by sampling it at regular intervals you need at least two samples per cycle. Conversely, if you have a fixed sampling rate (i.e. pixel spacing) there is a limit to the maximum frequency that you can capture. This limit is the Nyquist frequency.

So, what happens if we feed the sensor a higher frequency than Nyquist? Well, weird things happen. From the Nyquist-Shannon theorem we now know that the sensor cannot capture this new higher frequency; instead it records an erroneous signal that by necessity consists of lower frequencies, and might even have a different “shape” than the original. In the following images I simulated what the above nine sensor pixels will “see” as the sinusoidal input frequency is progressively increased beyond Nyquist.

Up to the Nyquist frequency the captured signal (dotted blue) that can be reconstructed from the “pixel measurements” (blue circles) matches the original input signal (red). At higher frequencies all sorts of weird things happen. This is called aliasing.

You can see this phenomenon interactively in the video below.

Caveat: In my visual explanation above I show the signal being 
point-sampled. A camera sensor doesn't take point samples as in 
the illustrations above, but averages the signal over each pixel's 
surface area. This tends to suppress aliasing somewhat. Thanks to 
"peridotfaceted" and Anne Archibald for pointing this out.

In colour

To make things worse for digital cameras, not only do they sample at discrete positions, but almost all cameras sample each colour channel separately. This is done by overlaying the camera sensor with a “Bayer grid” – a chequerboard pattern of red, green and blue. Almost all digital cameras use this pattern — including the Nikon D800 and D800E.

Notably the green channel is sampled twice as densely as the red and blue channels. This means that aliasing will occur at lower frequencies for red and blue than for green, and will generally be non-overlapping. You would therefore be right in guessing that colour aliasing typically shows up as weird reddish and bluish patterns in your photos.

The Bayer grid used in almost all digital cameras. This colour grid makes aliasing occur at different positions and frequencies for different colours, causing colour bands.

The Kodak DCS-14n was one of the first DSLRs to do away with a low-pass filter, and the result was pretty ugly under the wrong conditions.

The Kodak DCS-14n (discontinued in 2005) had big problems with Moiré because it lacked the necessary low-pass filter. This is a black-and-white test chart, but just look at those colour bands! (photo: Sergio Lovisolo)

Note that even the Foveon X3 sensor (used in the Sigma SD1 and DP2) that samples R,G,B colour at every pixel location can suffer from luminance aliasing according to the “monochrome” example in the sequence above. This is better than having colour banding, but not problem-free.

The Foveon X3 chip used in the Sigma SD1 samples all three colours at every pixel position so won’t show colour banding, but can still suffer from aliasing in image luminance.

The Sigma SD9 uses a Foveon X3 sensor and doesn’t suffer from colour banding but can still produce gray-scale (luminance) Moiré. (photo: Imaging Resource)

Can you fix it in software?

Unfortunately, there is no reliable way to remove Moiré in software. The camera and post-processing software can only see the sensor’s output. The camera or software cannot detect where real-world frequencies exceeded the sensor’s capabilities – that information is forever lost.

However, with a bit of manual tweaking you can improve the situation. The newest version of Adobe Lightroom (version 4) has a Moiré reduction brush that can significantly reduce the effect.

On his blog, Nasim Mansurov shows how this brush can be used to good effect. Notice in the mouse-over “after” image that even though the discolouring disappeared, the Moiré patterns linger on as gray-scale bands. I also suspect that the brush works best on a desaturated image region like the man’s suit and less well in colourful regions.

Having designed the D800E to be aliasing-prone, Nikon thought ahead and also implemented a solution in their Nikon Capture NX2 software that seems to work quite well. The disadvantage is that it requires you to buy and use their software. Depending on your existing work-flow this may be undesirable (e.g. I personally prefer working in Lightroom). As with other software solutions, this one will also not always be able to completely fix the problem.

So how can we prevent Moiré / aliasing?

The only sure-fire way to prevent aliasing is by guaranteeing that the image that reaches the camera’s sensor pixels doesn’t contain frequencies exceeding the Nyquist frequency. We usually can’t control the spatial frequencies (detail) in the objects we photograph, so we have to manipulate the light that reaches the sensor.

If you’re out shooting and notice that you’re getting Moiré you could shift to a smaller aperture (bigger F number) where diffraction will limit your lens’ resolution. Or if that is impractical you could defocus slightly, to purposefully blur your image.

Of course it would be better to prevent Moiré by design. You can design lenses not to be too sharp, but this is actually the opposite of what lens designers try to do, and difficult to control. And since different cameras have different resolutions a lens that is limited in this way will not be suitable for other resolutions. For interchangeable lens cameras it therefore makes sense to limit the image resolution at the sensor, and not in the lens.

This is how we get to the the anti-aliasing filter, also called a low-pass filter. This is essentially a piece of blurry glass that sits above the sensor’s pixels and makes sure that any details that are too fine get blurred to “acceptable” levels. In practice this filter also has other useful features such as blocking unwanted infra-red light, and a dust-reduction system.

(image: photographytips.com)

An anti-aliasing filter is also called a “low pass” filter since it only lets low frequencies pass unhindered. (image from maxim-ic.com)

There are two major practical engineering reasons why a low-pass filter reduces image sharpness. Firstly, as we saw above the Nyquist frequency for the green colour channel is twice as high as for the red and blue channels. It is almost impossible to design a filter that blurs red, green and blue so differently. Secondly, the filter’s transition region cannot be made perfectly abrupt. Placing the whole transition region to the “right” of the Nyquist frequency allows some Moiré to survive. Anything else and you start losing useful detail.

And now, the Nikon D800 vs D800E

Because landscape photographers and lab-rat pixel-peepers are demanding folks, they want as much resolution as possible. Especially when they are paying lots of money for a 36 megapixel camera. So they might want any extra resolution they can get – even if it means ripping out the anti-alias filter. This was probably some of the philosophy behind the Nikon D800E.

Side-by-side comparisons like these show extra detail in the D800E image. (click to enlarge). Note the carefully chosen background : lacking any regular man-made patterns or strong colour transitions. This makes aliasing unlikely to rear its head.

But as we’ve seen, the flip-side of this laissez-faire approach to anti-aliasing is Moiré. When shooting subjects with fine repetitive details like fabric, Moiré can (and does) appear in the D800E’s images. You may argue that these artefacts are barely visible if you look at the whole image, but remember that any sharpness gains will be even less visible if you look at the whole image.

This time showing an example where the D800E’s produces Moiré colour banding where the D800 does not. Image courtesy of Nikon. (Click for detail)

As we saw with the Kodak DCS-14n, the Nikon D800E is by no means unique among DSLRs in its proclivity to produce Moiré. The original Canon 5D could also be pushed into severe Moiré territory. Since it had “only” 12.8 megapixels this would have been even easier since its lower resolution equates an accordingly lower Nyquist frequency, making it easier to exceed. But just as with the D800, a well-designed low-pass filter could have prevented this from happening.

My suspicion is that whenever a manufacturer wants to brag with a camera’s resolution they are willing to slash the low-pass filter in favour of absolute sharpness. Remember that the Canon 5D had the highest resolution in its class, when it was released in 2005. And Canon probably wanted sample images showing how much resolution one could squeeze out of it (under ideal conditions).

In this case I strongly believe that you’re better of with a “properly” damped 36MP Nikon D800 than with a D800E that flirts with artefacts at the edge of its sensor’s abilities.

Conclusion

For the following reasons I think that almost any photographer should choose the D800 instead of the D800E

  • Only the very sharpest of lenses at their optimal settings and with perfect focus will be able to show the differences between the D800 and D800E. In all other cases you will be hard-pressed to see any difference between the two.
  • If you actually do have the aforementioned sufficiently sharp lenses and focus, you can run into problems with Moiré. While subtle changes in pixel-level sharpness are rarely visible at normal magnification levels (in printed or published photos), Moiré artefacts can be.
  • The D800 needs its low-pass filter just as much as any other camera does. A low-pass filter is always tailored to the sensor, therefore the D800’s low-pass filter will be perfectly adapted to its 36MP super-high resolution. Since you already get the advantages of 36MP in the standard D800, why would you try and push it into the domain where you can reasonably expect aliasing problems?
  • One of the D800(E)’s headline features is their video ability. It is very inconvenient to deal with Moiré in video, should it occur.
  • If one looks at the tests conducted by DPReview, one sees that post-processing sharpening can significantly lessen the perceived sharpness difference between the D800 and D800E. This either implies that the majority of softening by the D800’s anti-alias filter can be compensated for by post-processing, and/or that the D800E uses stronger in-camera sharpening to start with. Both possibilities imply a smaller detail advantage than one might initially suspect.
  • And lastly, the D800E is $300 more expensive than the D800 but in all other respects (except aliasing filter) the same camera.

What do you think?

Plug: If you agree with what I wrote and want to order your brand new D800 I suggest using a reputable retailer like B&H photo video. They currently sell the D800 for $2999.95, and the D800E for $3299.95.
Despite what one might expect from an expensive camera, DSLRs also have “lemons” (bad copies) that slip through quality control. Some copies of the D800(E) reportedly suffer from AF calibration issues. Stores like B&H offer you the peace of mind of good after-sales service and a fair return policy.