1. Physically Based Shading
A simplified artist interface
Niklas Hansson
Head Teacher Game programming The Game Assembly.

2. Physically Based Shading
What is it ?
Why should I use it ?
How do I integrate it into a project ?
Lightning base on the physical properties of light.
Replacement of Ad Hoc solutions
Instead of having
Spec Intensity,Spec Gloss,Reflectivity,Fresnel strength map and even a separate reflection gloss factor.
With a physically based solution we will be able to collapse this down to two simple maps.
Instead of Ad hoc solutions competing we have one stable system.

3. Boundary : When light goes from one material to another
Basic BRDF Theory
Boundary : When light goes from one material to another
Light splits up into two directions : reflection and
refraction.
The amount of light
reflected depends on the
materials reflective index.
Been asked to keep this not to technical
Basics are important.
Boundrary
L is split into ri and t
Split
Refractive index
BRDF
Works no matter the materials
We will only look at air to other material
Image from “Real-Time Rendering 3rd Edition”

4. Basic BRDF Theory Basic Material - Part is Reflected
- Part enters the material
Metal
Part is Reflected
The part entering the
Material is absorbed
Image from “Real-Time Rendering 3rd Edition”

5. Basic BRDF Theory Non metals
- Light that enters the material scatters internally
- Is absorbed
- and often exits the surface again.
Image from “Real-Time Rendering 3rd Edition”

6. Sub surface scattering
Basic BRDF Theory
Sub surface scattering
Distance between enter and exit positions are determined by the material.
If red pixel and green exit pos
needs SSS
BRDF cant cope.
If other way
Works
Image from “Real-Time Rendering 3rd Edition”

7. Basic BRDF Theory Point model
By ignoring distance between exit and enter points we can model lighting with an BRDF.
Holds holds true no matter the source of the light.
Works for perfectly flat boundaries
Most real world materials (except mirrors) doesn’t share this property.
Image from “Real-Time Rendering 3rd Edition”

8. Basic Microfacet BRDF Theory
Microfacets
A surface consists of many small perfectly flat surfaces.
How much they differ in normal determines how light interact
Microfactes
Normal differences encoded as a normal distribution
The difference from the Phong model to the Blinn Phong model
Blinn Phong instead used the microfacet
If parallel light hits small solid angle with all light, the rougher the larger solid angle the light will hit.
We use a normal distribution function to model this
The data it uses is the roughness of the surface (or flattnes would be a better name)
Image from “Real-Time Rendering 3rd Edition”

9. Basic Microfacet BRDF Theory
As you can see the rougher the surface the blurrier reflection and larger specular highlights.
Microfacets model sub normal map resolution shapes
Images from “Real-Time Rendering 3rd Edition”

10. Shadowing & Masking
Not all micro facets facing half vectors direction contribute to lightning
Shadowing
Masking
Strength of being physically based
Values have real value
Can update formulas without changing the value
Can be used to uppdate difuse to for exampel to Oren-Nayar
Rough material not gonna be as reflective due to ridges blocking light
Microfacet BRDF cant handle bounces on ridges
Images from “Real-Time Rendering 3rd Edition”

11. So why is specular so important ?
Images from “Everything is Shiny” by John Hable

12. So why is specular so important ?
Images from “Everything is Shiny” by John Hable

13. The Microfacet BRDF Microfacet BRDF LEGO parts
Most different BRDF models only contains changes to the D and G term.
Except denominator
//Is a correction factor that accounts for quantities being transformed between the local space of the micro facets and the overall macro surface
The ‘4’ is related to transforming from the space of half-angle directions to other spaces,
and the two clamped dot products are foreshortening factors that relate to the way that the visible area of microfacets change in relation to their orientation.

14. Returns a Value from 0-1 in RGB
Fresnel Reflectance
Returns a Value from 0-1 in RGB
Fraction of light reflected from optically flat surface given Light directon l and Half vector h
Frist part Fresnel reflectivity
Gives the reflctivity of the surface vs the viweing and light angle.
Returns spectral values
Reflected color doesn’t absorb light
However they reflect different amounts of lights at different wavelengths
We simplify this using RGB just as in the rest of CG

15. Fresnel Reflectance
Depends on refraction index (in our terms the substance of the object) and the light angle.
Fresnel curve
Light bounce amount varies with angle.
Try putting your head next to a door and se a clear reflection appear.
Starts on index of refraction which we will call substance
Objects with RGB spec has 3 index of refraction
Barely changes in beginning.
Turns quickly upwards at the end
Some materials dip
We wont model this
Image from “Real-Time Rendering 3rd Edition”

16. Fresnel Images from “Everything has Fresnel” by John Hable
Some real world examples
Images from “Everything has Fresnel” by John Hable

17. Schlick’s Approximation
Decently Accurate, uses F(0) the refractive index as input.
Which is our substance term.

18. Gives a scalar value between 0 and 1
Geometric term
Gives a scalar value between 0 and 1
Contains the fraction of non obscured microfacets
Various functions exists most using roughness as input
Using the geometric term to cancel out the denominator is extremely cheap as you can just remove booth from the calculation but will result in a to dark specular.
Look at lazarovs keynote for good solutions to this.

19. Distribution function
Calculates factor of microfacets reflecting light towards the eye
The function determines size and shape of the specular highlight
Distribution function calculates how many microfacets reflects light based on the roughnees
Decides specular size and strength
The blinn-phong distribution function is widely used in the games industry

20. Blinn Phong Distribution Functions
Modified Blinn Phong
Basic Blinn Phong
Doesn’t account for amount of light hitting the surface
Using a max cap is not enough
Calcualte amount of light hittign surface
And multiply with that
Cant reflect more light than hits it.

21. Beckman Distribution function
Conversion :
Cook torrence used beckaman
More accurate
More expensive.
Can convert between

22. Blinn Phong Energy Conversation
If the specular highlight is larger the specular intensity must be lower.
N = roughness
A surface cant emit more light than hits it
Therefore of light is reflected over a larger solid angle.
It needs to be weaker
We use a normalization factor to adapt light strength to scale.
This makes the equation energy conserving
The normalization factors differs depending on the distribution
N= roughness
phong has a very different normalization factor than blinn Phong.
This normalization was derived by Fabian Giesen

23. Blinn Phong Energy Conversation
Formula from physically based shading 2010
Seems vey off ?
Pi is lacking
Comes from hemisphere integration
Applies constantly over all light so can be removed in our case.
Just change light scale with pi

24. Blinn Phong Energy Conversation
Adding in the pi seems to give a correct curve so lets zoom in to make certain

25. Blinn Phong Energy Conversation
Zooming in it seems that this curve is off with quite a bit in the lower 100 range at least.

26. Blinn Phong Energy Conversation
Another common approximation as an 8 instead of a 2 at the top. This one fits much more well.
An important part is that it fits better at lower values which contains most of the real world materials. So we get some extra precision where it counts.
The factor of 8 is also used in the real time rendering book.

27. Blinn Phong Energy Conversation
This is another common approximation taken from tri Arcs papers but as you can see it plots vs the n+2/8 curve and not the real one

28. Blinn Phong Energy Conversation
This is another common approximation taken from tri Arcs papers but as you can see it plots vs the n+2/8 curve and not the real one

29. Blinn Phong Energy Conversation
Mathematically actually this formula is a better match but as you can see we have a higher margin of error on the lower values where most natural objects resides which is why the formula with an 8 is more often used.

30. Our Normalized equation for specular light
Energy Conversation
Our Normalized equation for specular light
Lambert
Cdiff*PI <=1
Full BRDF
Cdiff+Cspec <=1
Our equation
Lambert equation for diffuse
Using Cdiff for albedo
To be energy conserving Cdiff*pi<1
Putting Cdiff and Cspec together the rule is that no material can reflect more light than hits it,
This means albedo plus frefractive index need to stay below 1

31. Ambient Light is just Light
Has a specular component too
reflections
roughness controls reflections blurriness substance controls reflection strength.
Pre blurred cubemaps
Ambient is light
Has specular
Is handled as reflection
No need for extra data as we can already caluclate reflected light
If not using ansitropic look up you can preblur mip map levels and use roughness as a curve.
For spec we use a logaritmic curve to use as much a sthe color space as possible in the range the artist works
For Mip map select we use the default roughnees to keep more mip maps in the just slightely blured part.

32. ATI cubemap gen has nice functions for preblurring mipmaps.
Ambient Light
ATI cubemap gen has nice functions for preblurring mipmaps.
For correctnes the blurring needs to match your specular light
Prefiltered cubemap will start start looking as diffuse light
Cube map gen can prefilter mip maps.
Doesn’t do it correctly for this
Can roll your own or just download and modify

33. Blurred cubemap Use the angular extent and always sample from top mip.
Use the glossiness for that mipmap level and evaluate max size of highlight for size of angular extent.
Use the phong function to evaluate a contribution for all the pixels inside the extent.
Cubemap gen uses angular extents with different functions.
We want is filtered in the same way as phong (cant use blinn phong as it’s anisotropic) but phong is perfectly round
Calculate angular extent per mip using roughness
Use phong to evaluate contribution for sample.
Normalize
Phong simplifies to cosine filtering . Already in Cubemap gen raised to the roughness
Phing normalisation can be ignore as its the same for all samples with the same roughnees
Doing this makes specular highlight size and reflection bluriness match.

34. Why physically based ?
It’s more realistic thanks to being physically based?
Materials does not require as much retweaking under changing lighting conditions.
You can create more easily varied materials with lesser number of values
It saves artist time and simplifies workflow
The top one here isn’t one that I worry about. It might be to some people but I am a pragmatic.
Materials that doesn’t need retweaking as much when lighting changes is a good thing and saves time but by itself it is not a seller.
The more varied material interface and the easy to create different materials with a low number of values is a big seller especially combined with deferred rendering
The last one here however is the seller for me. Saving artists time is just pure financial sense and the reason we call this a simplified artist interface and that’s the issue I with start with.

35. Simplified Artist interface
Reduces number of maps from 4-5 to 2.
Simplifies Junior artist training
No longer take albedo and desaturate for intensity
And then lower color for gloss
We have all seen it
Reduces maps vs ad hocc
Reduces work amount
Simplifies artist mental model
A lot of erroneus tutorials around just using filters.
Some tutorials talk about taking the inverse of albedo instead.
Makes spec depend on albedo brightness not material
Brigher gets brighter or darker gets darker,
Gloss not really understood just lower color
Reflection just for special objects
Fresnel no clue
Will look like above.

36. Simplified Artist interface
PBL simplifies this by creating a material interface that makes sense.
Substance
Is the index of refraction for the material
No need to hand paint.
Mathematically matches old intensity
A interface you can understand.
Using existing physical values
Using a look up table and masks to select areas and add material
Makes substance quick to create
Doesn’t need hand painting
Matches old intensity

37. Color picker Easy visual look up Handles specular color properly
Linear and gamma depending on what the project uses
Non metals taken from physics books.
Metals index for different wavelengths taken from scientific data
Feed into a program by sebastian lagarde that integrates them for RGB color space.
Here is the swatch we use at The game assembly. We have booth linear and gamm
For average materials default values at the bottom
Last values to be tweaked.
Else you might tweak others values for an incorrect here and never get it right.

38. Simplified Artist interface
Roughness
The old gloss
But thanks to Normalization a lot more useful
Where hand painting happens
Controls reflection blurrinees and spec
Surface Roughness that normal maps can’t show
Touch
Observe
Just old gloss with normalization
Can make brighter specular thanks to normalisation
Makes ideal for hand painting
Controls reflection blurrines as well
Artists doesn’t need to worry
He only needs to think about the roughness of the surface on the level that won’t shop up in the normal maps
Using touch to get a feel
Looking for fresnel effects at glancing angles.

39. Example Material rusty Paint
Substance just a mask for material
Roughness contain details

40. Flexibillity Don’t forget albedo and normals importance
We made a test chart with some different materials to test it out and se how different looks we can achieve.

41. Sample Oskar stålberg
This is all rendered in our maya plugin for PBL and is a good example of how flexible material looks you can get with this system.
Look especially at the mush rooms and metal vs the main body to the left
And the fuse and plastic cables to the right

42. Substance almost no detail except for big material change
Roughness contains the work
Tried on junior and more senior artists
Senior directly grasped and improved speed.
Juniors took a while to make the shift then increased speed

43. Lower numbers of values to tweak Smaller tighter gbuffer
Why physically based ?
Lower numbers of values to tweak
Smaller tighter gbuffer
Low bandwitdh
Small ALU increase
Ideal for deferred rendering
While still allowing very
varied materials
And now the technical reasons
Having fewer variables reduces bandwidth
Especially when using deffered rendering
PBL works for almost all materials right of the bat so you need very few special cass-
We get a tighter gbuffer for just a small ALU increasel

44. How to use it What did we do ? Microfacet BRDF with
Normalized Blinn Phong
Fresnel reflectance
Void Geometry Function
G(h) = 4(n dot l)(n dot v)
Prefiltered glossy reflections from cubemap probes
Ambient from ambient probes

45. What we did Cref is the value from the reflection probe
Camb is the value from the ambient probe
Fschlick(v,h) is the fresnel term using the view direction instead of the light direction
Removed the pi division to just change the light intensity scale
Cref is the reflected light in the cubemap
Camb is the ambient light for the point.
fresnel term for the reflection uses v and h instead of l and h.
energy conversation between reflection and ambient.
while not 100% correct it does a good job of mimicking real life behavior
Other are using a special case reflection fresnel to reduce edge shine.
This worked well for us.

46. Ambient vs reflection Energy Conversation
Energy conversation created better metalic surfaces
While still allowing artists to use an albedo texture.
Effect is subtle
Right side is more metallic
Left side seems washed out.

47. Also solved issues where artist tried to tweak substance and roughness incorrectly
To remove to strong fresnel effect.

48. Substance and Roughness vs Intensity and Gloss
Lessons learned
Substance and Roughness vs Intensity and Gloss
Spend time training and communicating to your artists
Help them use the new interface
Tools is essential
Beware of bugs
You got to have artist buy in.
Need a mind shift.
New names helped.
We created a maya plugin to allow the artists to preview models.
And had some issues due to bugs in it 

49. Maya plugin Tool was double edges sword. Artist used new interface
Maya visuals differed from game visuals

50. Reflection and ambient is a big part of PBL
In game they are dependant on light conditions so that their strength match light strength. Using in engine probes.
In maya the artist could select maps and tweak light to values that could not exist together in game.
Which meeant what they saw in game and engine differed.
A solution would be to try and normalzie cubemap vs light strength but getting 100% is imposible.
We are looking at using premade light rigs to solve the problem
Using an in engine viewer is even better but artists doesn’t like the extra step of leaving maya.

51. Flexible Kind of projects Space shooter Realtime Strategy
What we did
Flexible
Kind of projects
Space shooter
Realtime Strategy
First Person shooter
Used the techinique in 12 projects
Over 4 teams
And 3 genres

52. Space shooter Space shooter was perfect case
Ambient and reflection from skybox.
Only thing we would change is use ambient vs reflection energy conversion

53. RTS games showed up the pipeline problems with maya vs ingame
And to strong fresnel due to lack of energy conversation caused a lot of issues.

54. Fps project Goals Techniques
Resolve all issues found during the last project
Get an easy to light solution that took little rebuild time and would work with just a few spot lights.
Techniques
All enviroment and ambient probes generated by the ingame engine to achieve consistency.
Box projected cubemaps was used for reflections and a variation was used for ambient
Simple Multi bounce solution for lightning

55. Box projectedion made cubemap lookup ansitropic
would require importance sampling and multipel filtering to look weel.
Solution was to interpolate between box proejcted look up and ordinary
In engine generated ambient and reflection resolved earlier issues

56. Specular and reflection doing more work
Occlusion
Specular and reflection doing more work
Will look weird in areas normally don’t reach by much light
Use Ambient occlusion
Multiply to spec
Multiply to reflection
If needed add special Specular occlusion
Since booth specular and reflection is doing a lot more work
We get problems
Dark crevices shines up due to reflection.
Made worse by the fact that reflection is added to normal color makes reflection in dark creavices overly shiny.
A solution is using ambeint occlusion
Holds the amount of direction that light can hit the surface from. The more hid the darker.
Will keep your crevice dark.
With everything as geometry and perfect shadows this would not be needed for specular.
You can run into problem that the specular for some materials is so bright that AO is not enough
Tri Arc proposed using a special specualr occlusion that has a much higher occlusion factor.

57. References
Sébastien Lagarde ”Adopting a physically based shading model “,”Feeding a physically based shading model”
Roy Driscol “Energy Conversation In Games”
Fabien Giesen “Blinn-Phong normalization factor”
Nathaniel Hoff­mann, “Crafting Physically Moti­vated Shading Models for Game Devel­op­ment”, SIGGRAPH 2010
Dim­i­tar Lazarov, “Physically-​based lighting in Call of Duty: Black Ops” SIGGRAPH 2011
Chris­t­ian Schüler, “An Efficient and Physically Plausible Real-​Time Shad­ing Model.” ,” The Blinn-​Phong Normalization Zoo”
Akienne-​Möller, Haines and Hoff­mann, “Real-​Time Rendering” book
Yoshiharu Gotanda, “Practical Implementation of Physically-​Based Shading Models at tri-​Ace”, SIGGRAPH 2010
Yoshiharu Gotanda, “Practical Physically ​Based Rendering in Real-time”, SIGGRAPH 2012

58. Testing on real projects Better Geometry term
Going forward
Testing on real projects
Better Geometry term
Using the proper mipmapping for cubemaps
Some kind of Diffuse- specular energy conversation
We are in the lucky situation that we can run tests of this on a lot of projects carefully tuning and twaking and see what makes the artists life easier and what does not.
Having a better geometry term is a no brainer for us. Even though it will mean having to handle normal map mipmapping differently with variance values as in lazarovs note on black ops lightning
Nothing in these demos are done with the correct cube blurring because in the beginning the used cubemap gen and later they wrote their own blurring code. We have tested proper blurring and it does give a bit of an extra visual edge.
Again tests has been made that are promising. This way artist can use albedo photo references more easily since we overide part of the issues in it by using proper valeus in the substance map) However we haven’t finished a project with this setting yet.

59. Physically based for non realistic rendering
Can we do it ?
Yes just a material interface
Rules can be bent and broken.
Specular not that important for cartoonish look
Diffuse light ,albedo,normal,AO more important
Non realistic rendering
Can we do it ?
Yes, just a material interface
Most of cartoony look comes from albedo,normal and AO.
And diffuse of course

60. RTS game Here is an example of a semi realistic style running in PBL.
The reason this is important is of course so that we can keep giving the artists one interface that they are as familiar with as possible.

61. More non photo And here is another different view.
So the material interfaces is still flexible enough for any view while still keeping the interfaces low in fact the realism factor comes more from diffuse/normal and the design of the cube maps etcs.