PHYSICALLY BASED ILLUMINATION

In order to get a more realistic representation of lighting, we need to move away from the simplistic models that are found hard coded in most graphics pipelines and move to something that is based more in a physical representation of light as a wave with properties of its own that can interact with its environment. To do this, we'll need to understand how light passes through a medium and how hitting the boundary layer at the intersection of two media can affect light's properties. there's an incident light hitting a surface. At the boundary of the two media (in this case, air and glass), there are two resulting rays of light. The reflected ray is the one that we've already discussed to some extent, and the other ray is the refracted or transmitted ray.

In addition to examining the interaction of light with the surface boundary, we need a better description of real surface geometries. Until now, we've been treating our surfaces as perfectly smooth and uniform. Unfortunately, this prevents us from getting some interesting effects. We'll go over trying to model a real surface later, but first let's look at the physics of light interacting at a material boundary.

Reflection

Reflection of a light wave is the change in direction of the light ray when it bounces off the boundary between two media. The reflected light wave turns out to be a simple case since light is reflected at the same angle as the incident wave (when the surface is smooth and uniform, as we'll assume for now). Thus for a light wave reflecting off a perfectly smooth surface

Until now, we've treated all of our specular lighting calculations as essentially reflection off a perfect surface, a surface that doesn't interact with the light in any manner other than reflecting light in proportion to the color of the surface itself. Using a lighting model based upon the Blinn—Phong model means that we'll always get a uniform specular highlight based upon the color of the reflecting light and material, which means that all reflections based on this model will be reminiscent of plastic. In order to get a more interesting and realistic lighting model, we need to add in some nonlinear elements to our calculations. First, let's examine what occurs when light is reflected off a surface. For a perfect reflecting surface, the angle of the incoming light (the angle of incidence) is equal to that of the reflected light. Phong's equation just blurs out the highlight a bit in a symmetrical fashion. Until we start dealing with nonuniform smooth surfaces in a manner a bit more realistic than Phong's in the section on surface geometry, this will have to do.

Refraction

Refraction happens when a light wave goes from one medium into another. Because of the difference in the speed of light of the media, light bends when it crosses the boundary. Snell's law gives the change in angles.

where the n's are the material's index of refraction. Snell's law states that when light refracts through a surface, the refracted angle is shifted by a function of the ratio of the two material's indices of refraction. The index of refraction of vacuum is 1, and all other material's indices of refraction are greater than 1.

What this means is that in order to realistically model refraction, we need to know the indices of refraction of the two materials that the light is traveling through. Let's look at an example Let's take a simple case of a ray of light traveling through the air (n_air ≅ 1) and intersecting a glass surface (n_glass ≅ 1.5). If the light ray hits the glass surface at 45°, at what angle does the refracted ray leave the interface?

The angle of incidence is the angle between the incoming vector and the surface. Rearranging Snell's law, we can solve for the refracted angle.

which is a fairly significant change in the angle! If we change things around so that we are following a light ray emerging from water into the air, we can run into another phenomenon. Since the index of refraction is just a measure of the change in speed that light travels in a material, we can observe from Snell's law (and the fact that the index of refraction in a vacuum is 1) that light bends toward the normal when it slows down (i.e., when the material it's intersecting with has a higher index of refraction). Consequently, when we intersect a medium that has a lower index of refraction (e.g., going from glass to air), then the angle will increase. Ah, you must be thinking, we're approaching a singularity here since we can then easily generate numbers that we can't take the inverse sine of! If we use Snell's law for light going from water to air, and plug in 90° for the refracted angle, we get 41.8° for the incident angle. This is called the critical angle at which we observe the phenomenon of total internal reflection. At any angle greater than this, light will not pass though a boundary but will be reflected internally.

One place that you get interesting visual properties is in the diamond—air interface. The refractive index of a diamond is fairly high, 2.24, which means that it's got a very low critical angle, just 24.4°. This means that a good portion of the light entering a diamond will bounce around the inside of the diamond hitting a number of air—diamond boundaries, and as long as the angle is 24.4° or greater, it will keep reflecting internally. This is why diamonds are cut to be relatively flatish on the top but with many faceted sides, so that light entering in one spot will bounce around and exit at another, giving rise to the sparkle normally associated with diamonds.

Another place where a small change in the indices of refraction occurs is on a road heated by the sun when viewed from far away (hence a glancing incident angle). The hot air at the road's surface has a slightly smaller index of refraction than the denser, cooler air above it. This is why you get the effect of a road looking as though it were covered with water and reflecting the image above it—the light waves are actually reflected off the warm air—cold air interface.

What makes this really challenging to model is that the index of refraction for most materials is a function of the wavelength of the light. This means that not only is there a shift in the angle of refraction, but that the shift is different for differing wavelengths of light. You can see the general trend that shorter wavelength light (bluish) tends to bend more than the longer (reddish) wavelengths.

This is the phenomenon that's responsible for the spectrum that can be seen when white light is passed through a prism. It's refraction that will break apart a light source into its component colors, not reflection.

This is one area where our simplistic model of light breaks down since we're not computing an entire spectrum of light waves, but we're limited to three primary colors. For reference, the rgb values can be assigned to a range of wavelengths as follows:

There's a lot more to color science than just determining wavelengths, but that's beyond the scope of this book.

While the spectrum spreading effect of refraction is interesting in itself, the rgb nature of computer color representation precludes performing this spreading directly—we can't break up a color value into multiple color values. However, with some work, you can compute the shade of the color for a particular angle of refraction and then use that as the material color to influence the refracted color.

Temperature Correction for Refractive Index

Refractive index is a function of temperature, mostly due to density changes in materials with changes in temperature. A simple correction can be applied in most circumstances to allow you to use a value given at one temperature at another. For example, suppose the index of refraction value you have is given at 25°C: η²⁵. To convert the index to another temperature, η^t, you can use the following equation:

where the actual temperature you want is t, and the 25 is the temperature (both in °C) of the actual index you have, η²⁵.

The Fresnel Equations

The Fresnel (pronounced Freh-nel) equations are used to calculate the percentage of energy in the refracted and the reflected parts of the wave (ignoring adsorption). In order to understand what the equations calculate, we'll have to take a look at what happened when a light wave (as opposed to a photon) interacts with a surface. We have to do it this way since this is the only way to describe the subtle (and realistic) visual effects we are looking for. A wave of energy has both an electric field and a magnetic field that travel in perpendicular phase, as shown in.

In general, when a wave reaches a boundary between two different dielectric constants, part of the wave is reflected and part is transmitted, with the sum of the energies in these two waves equal to that of the original wave.

The Fresnel equations are a solution to Maxwell's equations for electromagnetic waves at the interface. If you are really interested in seeing how this works see [HECHT 1987], but I'll spare you the actual derivation and get to the important part. What Fresnel did (besides proving once and for all that light can behave like a wave) is figure out that for the two extrema of the light wave—the light wave with the electric field parallel to the surface and the light wave with the electric field perpendicular to the surface, the energy transmitted and reflected—are functions of the angle of incidence and the indices of refraction for the two media. This is for a nonconductive (dielectric) medium like plastic or glass.

For a conductive media, there's actually some interaction between the free electrons in the conductor (or else it wouldn't be a conductor) and the magnetic field of the light wave. Being a conductor, there are free electrons in the material. When the light wave interacts with the material, the electrons in the material oscillate with the magnetic field of the light wave, matching its frequency. These oscillations radiate (and, in effect, reflect) the light wave. In addition, a conductor has some resistance to electron movement, so the material absorbs some of the energy that would have been reradiated. Fresnel equations for conductive interfaces (since they absorb light they are also generally opaque) usually involve a dielectric (like air) and a conductor since you don't have a light interface between two opaque materials. The parallel and perpendicular components are sometimes referred to as the p-polarized and s-polarized components, respectively.

Fresnel Equations for Dielectrics

The simplest form of the Fresnel equations are for dielectrics.

where r and t are broken into parallel and perpendicular segments. n_t and n_i are the indices of refraction for the transmitting (reflecting) and incident materials respectively.

Now we can simplify these equations by assuming normalized vectors and multiplying out the dot products, noting that n • 1 = cos(φ_i) and n • t = −cos(φ_i), and using Snell's law to get rid of the indices of refraction. φ_i is the angle of incidence and φ_t the reflected angle. These simplified equations are

Using these equations, we can calculate the percent of energy transmitted or reflected. Since these equations represent the maxima and minima of the interaction between the media depending upon the orientation (polarization) of the light wave's fields, we'll just take the average to compute the amount transmitted and reflected. Thus

gives us the fraction of unpolarized light transmitted and reflected. Note that this average is for unpolarized light. If your light source was polarized, you could pick one equation instead of the average. Also note that due to the conservation of energy, we could also write

POISSON'S BRIGHT SPOT

In 1818, the French Academy of Science held a competition for best essay covering the wave theory of light. At this time, there was great debate about whether light was a particle or a wave. A 30–year-old civil engineer by the name of Augustin Fresnel submitted a monster 135–page paper on the theory of light waves and their diffraction for the competition, which completely went against the particle theory since particles don't bend. Though an excellent theorist, Fresnel couldn't solve some of his equations in his paper. Unfortunately for him, one of the judges of the competition was an excellent mathematician: Siméon Denis Poisson. Poisson also happened to be a very strong believer in Newton's particle theory of light and was able to solve Fresnel's equations. He then set out to "prove" that the wave theory was wrong using Fresnel's own equations. Poisson stated that Fresnel's equations gave the absurd prediction that a disk casting a shadow would show a bright spot at the center of the shadow due to the diffraction waves all being in phase. The waves would all contribute to make the spot just as bright as if the disk weren't there. A simple experiment would prove it so. Another judge of the competition, Fran¸ois Arago, arranged to have the experiment performed, and the bright spot was seen! Fresnel was awarded the prize in 1819, and Poisson greatly, though unintentionally, advanced the wave theory of light. The bright spot is usually referred to as Poisson's bright spot, though Poisson refused to believe in the wave theory up to his death 30 years later. You can perform this experiment yourself—it works best with laser light. The spot shows up a little over a disk's width behind the disk. You can also see it during a solar eclipse as a bright ring around the moon's disk.

Let's take a look at what this means in a practical application. Let's plot the values for the Fresnel equation for the air-glass interface , The first thing you should notice is that even when the incident angle is at 0°—that is, the light is shining along the surface normal—there is still some loss in transmittance.

The reflection and transmission curves for the parallel and perpendicular waves in the air-glass interface.If we do the math, we will discover that there is no distinction between the parallel and perpendicular components when the angle is 0°, and in this case, the Fresnel equations simplify to the following:

Thus at 0° for the air-glass interface, we can see that about 4% of the light is reflected. This means that if you have a glass window, you will get only 92% of the light transmitted through (you have the air-glass interface, 4% reflected, and the glass-air interface, another 4%, when the ray comes out of the glass). Glass used in optics is usually coated with a thin film of antireflective coating, which reduces the reflectivity to something around 1%. You might also note that at an angle of about 56°, the parallel reflectance drops to zero. This is called Brewster's angle or the polarization angle (φ_p) and is the effect on which polarized lenses work. You can use Snell's law and the observation that for polarized light φ_t = 90° − φ_p to derive the following equation for Brewster's angle:

Let's plot the average reflected and transmitted energy and take a look at the plot.

We can see that as the incident angle approaches 90°, the reflectivity approaches 100%. This is one of the more important aspects of the Fresnel equations—at a glancing angle, all surfaces become perfect reflectors, regardless of what the surface is made. In fact, this is one way in which x-rays are focused. The only fly in the ointment is that this is true only for perfectly flat surfaces, but this will be covered in the next section.To be thorough, we should also take a look at the other side of the interface, when looking though a medium of higher refractive index to that of a lower one. In this case, we will reach an incident angle where we reach total internal reflection, and no light will be transmitted through the interface. If we reverse the indices of refraction and take a look at the glass-air interface.

At the critical angle of 41.8°, we get total internal reflection, and no light is transmitted through the interface. Thus if you wanted to model something like a scene underwater, you'd need to treat the water surface as a mirror surface at any angles over the critical angle.

The Fresnel Term in Practice

For real materials, the Fresnel term depends upon the incoming angle of light and the reflected angle. The reflected angle is a function of the indices of refraction of both materials, which in turn, are dependant upon light wavelength and density of the materials. The density, in turn, typically is a function of temperature. The index of refraction usually increases with the density of the medium, and usually decreases with increasing temperature. This is all very fine if you happen to have data for the index of refraction for the wavelengths of interest over the temperatures you'll need. Since this kind of data is difficult to find, we'll do what innumerable computer graphics researchers have done before us—we'll fake it.

To get a reasonable estimate on the Fresnel term, Cook and Torrance [COOK 1982] note that the values of reflectance at normal angles of incidence are more commonly found. You can get the Fresnel value at this angle (called F₀) and then calculate the angular dependence by back-substituting for the index of refraction using F₀ and then plugging this value for the index of refraction back into the original Fresnel equation. This gives you the Fresnel value as a function of the angle of incidence. In order to perform this calculation, we need to reformat the Fresnel equation for reflectance into a form derived by Blinn [BLINN 1977]. This is easier if we do it in steps. The parallel reflectance part of Fresnel's equation is

If we get rid of the tangent using tan²(φ) + 1 = sec²(φ) = 1/cos²(φ), then we can rework the equation into terms that involve only cosines. The new equation is

And then we can use the cos(φ ± θ) = cos(φ) cos(θ) ± (−1) sin(φ) sin(θ) identity to break it down into terms involving θ_i and φ_t separately, and use Snell's law as η_λsin(φ_r) = sin(φ_i), where η_λ = η_r/η_i.

Blinn introduced the term g in his paper to simplify the equations a bit, and we'll do the same here.

First multiplying through by η_λ/η_λ and then replacing with g yields

Then we can use the sin²(φ) + cos²(φ) = 1 identity to remove the sine term and replace it with a cosine term, which will allow a further replacement with the g term to further simplify the equation to

Finally, we use the term g² = η²_λ + cos²(φ_i) − 1 and factor out common terms to get

Whew! What we've now got is an expression for a Fresnel term that is only a function of η_λ and φ_i. Let's do the same with the perpendicular reflection term.

Using the sin(φ ± θ) = sin(φ) cos(θ) ± cos(φ) sin(θ) identity breaks out the equation into terms involving φ_i and φ_t separately.

Then use Snell's law to get rid of the sine terms.

and finally replace the η_λ cos(φ_r) with g to get

Finally, add the two pieces of the Fresnel reflection terms and average, and we get

Since

we now have a Fresnel equation that's dependant upon only three values–the two indices of refraction and the incident angle.Well, that's great, but why have we gone through all that? It's to further simplify the equation. If we look at the equation at normal incidence, when θ = 0, then cos(θ) = 1 and g = η_λ, then most of the equation cancels out, and we're left with

Ok, what's the advantage of this? Well, this lets us assume a value for η_λ when the light is normal to the surface. In other words, when we shine a light directly at the surface and look from the same direction (normal incidence), then this equation lets us solve for η_λ at this angle. Thus we can rearrange the equation to read

You can then use the values of η_λ generated in this way to plug into the equation to generate F_rat other angles.

PHYSICALLY BASED SURFACE MODELS

The most widely used model of surfaces that are not perfectly smooth and uniform is the Cook-Torrance [COOK 1982] model. What they did was to assume that

• The geometry of the roughness of the surface is larger than that of the wavelength of the light.

• The geometry is considered to be made up of v-shaped facets.

• The facets are randomly oriented.

• The facets are mirrorlike.

Using such a model, there are three different ways that a light ray can interact with the entire internal reflecting surface of the v-shaped geometry (termed "microfacets") depending on the angle of the v.

• The ray can reflect with no interference.

• The ray could be partially shadowed by other geometry.

• The ray could get blocked by part of the geometry.

These three cases are shown in.

Roughness Distribution Function

When using this model, we need a way to specify the distribution of the slopes of the facets. This is termed the slope distribution function, D. Blinn [BLINN 1977] used a Gaussian distribution function to model the slope distribution.

where c is some arbitrary constant and cos(α) = n • h. The parameter m is the RMS slope parameter, for which smaller values (0.2) indicate a smooth surface, whereas larger values (0.8) indicate a rougher surface.

Cook and Torrance [COOK 1982] used a Beckmann distribution function, which they state can successfully model both rough and smooth dielectrics and conductors.

This model has the advantage of not requiring a constant but just relies on one parameter m to specify the surface roughness.

There are other models such as the Trowbridge-Reitz model [TROWBRIDGE 1975], which models the microfacets as ellipsoids. You can even consider the Phong specular term to be a distribution function with the roughness specified as the exponential power value.

Geometric Attenuation Function

In addition to describing how the geometry of a rough surface is laid out, the Cook-Torrance model can be used to calculate the amount of light actually hitting on the microfacets. Blinn [BLINN 1977] has a very nice derivation of the geometry involved. Basically, there are three different cases to consider.

1. There is no interference in any of the light.

2. Some of the incoming light is blocked (shadowed).

3. Some of the reflected light is blocked (masked).

Blinn calculated the amount of light that would get blocked in each case. These are basically functions of the light direction and the facet normal. You then calculate these three values and select the minimum as the geometric attenuation function, G.

THE BIDIRECTIONAL REFLECTANCE DISTRIBUTION FUNCTION (BRDF)

We're trying to accurately simulate reflectance from light traveling through a dielectric (something transparent, air, vacuum, etc.) and hitting the surface of a conductor (or something nontransparent) and reflecting off that surface.

The bidirectional reflectance distribution function (BRDF) takes into account the structure of the reflecting surface, the attenuation of the incident light by that structure, and the optical properties of the surface. It relates the incident light energy with the outgoing light energy. The incoming and outgoing light rays need to have not only their angles with the surface normal considered, but also the orientation of the rays with the surface orientation. This allows surfaces that reflect light differently, depending on their orientation around the surface normal (i.e., an isotropic surface), to be modeled.

the BRDF parameters relate to the surface normal and orientation of the surface. a BRDF depends upon a total of four angles. Of course, you can simplify these by making assumptions about which terms are important to get the effect you want.

BRDFs in Practice

The problem with BRDFs is that they are tough to implement in a practical manner. It's perfectly fine to have a BRDF that's expensive to calculate if you are doing some non-real-time imagery, but for performing BRDFs in shaders, you frequently have to simplify the model. The Lambertian model for diffuse reflection, for example, can be considered a BRDF that's just a constant. Phong's illumination model takes the (typical) approach of breaking a BRDF into diffuse and specular parts. It uses the Lambertian model for the diffuse and a cosθ term for the specular, treating the specular BRDF as a function of the incident light vertex normal angle only.

However, we've already tried these models and found them wanting, so we'll take a look at a model for specular reflection developed by Blinn [BLINN 1977] that is still popular. Blinn proposed that, using the Cook-Torrance model for surface geometry, the specular reflection is composed of four parts.

• The distribution function, D.

• The Fresnel reflection law, F.

• The geometric attenuation factor, G.

• The fraction of the microfacets that are visible to the light and the viewer by a (n • v)(n • l) term.

The BRDF specular function is then

Now we've already gone though the Fresnel term, the distribution function, and the geometric attenuation factor. You're free to make these as complicated as you like. For example, Blinn leaves the Fresnel term = 1, whereas Cook-Torrance doesn't.

It's possible to precompute the BRDF by judiciously choosing some of the parameters, and then generating one or more textures to account for the other terms. Typically, you might generate a texture where the u, v values of the texture are mapped to the n • v and n • l terms. For more information on BRDF factorization, you can refer to [ENGEL 2002] and [LENGYEL 2002].

Anisotropic Reflection

One interesting feature of the BRDF is that is supports anisotropic reflection, that is, reflection that varies in strength depending upon the orientation of the material's surface. Many surfaces exhibit this type of shading: hair, brushed metal, grooved surfaces (CDs, records), some fabrics, etc. Some of the more complex BRDFs can model the effects.

Poulin-Fournier [POULIN 1990] wrote one of the earliest papers on anisotropic reflection models. The Poulin-Fourier model replaced the randomly oriented v-shaped grooves of the Torrance-Sparrow model with aligned cylindrical shapes (either grooves or protrusions). He, Torrance, Sillion, and Greenberg [HE 1991] proposed a model where they broke the specular term into two parts, a diffuse specular and a directional specular. Their full model is quite complex and can be used with polarized light. Even the unpolarized simplified equations are quite complex and would be nearly impossible to fit inside a shader. In a latter paper [HE 1992], they address issues of computational speed, and in a time-honored tradition, compute a lookup table from which they can closely reproduce the values in their original paper. A different approach was taken by Ward [WARD 1992], who proposed finding the simplest empirical mode that would fit the data. This is still a very active area of research. Many papers in Siggraph Proceedings of recent years are worth looking into if you are interested in seeing further details and research. [BRDF] lists some online BRDF databases.

NONPHOTOREALISTIC RENDERING (NPR)

On the other end of the rendering spectrum is nonphotorealistic rendering (NPR). This style of rendering throws out most of the attempt to simulate real-world reflection models to achieve a different artistic goal. This can be the simplification of a scene to make it easier to understand, the modification of the scene to highlight an aspect of the scene, or the simulation of some other method of illustration such as watercolor or pen and ink drawing. Using shaders, it's possible to create your own method of illustration along any one of these styles. Unfortunately, much of the research cited in these examples was done before hardware shaders existed (though there are quite a few RenderMan shaders out there), so ready-made hardware shader examples are few and far between for some of these techniques, but they are starting to become available. The basic technique is to use the lighting equations for specular light (basically, some n • l term) to modulate the intensity of the effect produced. Look at any of the Siggraph Proceedings since 1995, and you'll typically find a couple of papers on these techniques. You can also find some good reviews of the latest research in [GOOCH 2001] and [STROTHOTTE 2002]. Craig Reynolds maintains an excellent Web page that's up to date, with a ton of links to various papers at http://www.red3d.com/cwr/npr/. There's now an annual Nonphotorealistic Animation and Rendering Conference (NPAR). You can get more information about it at http://www.npar.org.

NPR Styles in 3D Rendering

NPR in 3D rendering can be roughly broken into a few different styles.

Painterly Rendering

This style is intended to simulate the results from brush-applied media. It's characterized by having a virtual brush apply media to the object's surface. Brush attributes include stroke weight, media loading, stroke attack and media attenuation over time, etc. Application of the media is sometimes done by calculating the particle flow of the media. You can find further examples of this style in [HAEBERLI 1990] and [MEIER 1996].

Pen and Ink, Engraving, and Line Art

This style is a high-contrast style where you're limited to an unvarying color intensity and can only adjust the line width. It's a simulation of using only an instrument of constant color intensity such as a pen to apply color. a rendering of Frank Lloyd Wright's Robie House, uses the technique described in [WINKENBACH 1994].

A nice example of creating digital facial engraving from 2D images, which attempts to imitate traditional copperplate engraving using the techniques from [OSTROMOUKHOV 1999], This technique uses multiple layers to lay down the different areas of the face in order to provide a sharp demarcation of the different facial regions.

Sketching and Hatching

This style is similar to the preceding one but allows the use of tone as well. It imitates the look of charcoal and pencil with strokes that are hatched images scaled to approximate stroke density. Some nice examples of this work can be found in [WEBB 2002] and [PRAUN 2001]. The use of tonal art maps (TAMs) as textures that are used to control the degree of shading on an object gives some particularly nice results.

When these are applied in place of the traditional shading equations, you get some nice effects.

Halftoning, Dithering, and Artistic Screening

These techniques use digital halftoning to get range and material. Rather than simulate a brush or pen stroke, the simulation is of the halftoning technique typically used by printers to achieve density through the use of dots or lines of varying density [OSTROMOUKHOV 1999], [STREIT 1999].

‘Toon Shading, Cel Shading, Outlining, and Stylized Rendering

This is probably one of the best-known areas of NPR. These styles use a combination of simple gradient shading and edge outlining to get some visually stunning results. A particularly striking example found in [GOOCH 1998] shows how using changes in hue and saturation to show changes in the orientation of the model's surface clarify structure. They present an alternative lighting model to traditional Phong shading.

Another popular look is to use cel shading to get a cartoonlike feel. It's found a lot on computer-rendered scenes because it's fairly easy to generate automatically. A popular way of cel shading nontextured objects is to shade according to two areas (lit and unlit)-also called hard shading because of the hard delineation-or three areas (brightly lit, lit, and unlit). The vertex color is then set to one of these dark or bright colors according to the amount of light that's falling on the vertex.

Other Styles

Of course, there are many styles that don't quite fit into one of the previous types. These range from rendering fur and/or grass based upon a procedural texture placed near silhouette edges (graftals) by Kowalski [KOWALSKI 1999], which creates scenes rendered in a Dr. Seuss-like style.

The edges of objects are determined and then an algorithm is used to graft on textured geometry, the edges are outlined, and all else is rendered with flat shading. With some attention to graftal coherency, it's possible to actually move around the scene.