Paper Mario is probably one of the most influential games of all time in my life, even though I didn’t really “get” it till years after I first played it – I used to dodge enemies a lot and wondered why all the bosses were so hard, I was not used to the idea of an RPG.
Once I understood the leveling concept, my second try at a play through went much better- I finally made it through and grew even more appreciation for it.
even later, a couple years ago in fact, I decided to play through again. now with more experience with game development and the history of various graphics techniques, my jaw hit the floor when I noticed the most inconsequential detail – in certain environments, when a character was lit from behind, and a little above, the character had what appeared to be rim lighting, based on the angle to the light.
while nowadays it’s normal to have normal mapping on sprite sheets to allow for dynamic lighting on 2D assets, this would have been unheard of in the 90’s. I would sit for many minutes moving around to see how the light moved, trying to figure it out like a magician’s illusion. I was out of ideas until I saw Peach’s rim light when she was on her balcony.
the shadow of the crown was… oddly straight. with a couple more observations, I had it – there is no 3D effect going on whatsoever. instead, we’re just seeing a shifting shadow mask of the exact shape of the sprite.
In the following, we’ll create a shader with custom lighting calculations that emulates the Paper Mario lighting system, to create a quick and easy rim lighting solution for any 2.5D game
Step 0: The Setup
While I set up a little more to be able to move around and view the effect in animations, the only setup necessary is the following:
- No Directional light- just a point light or a spot light
- a floor
- a quad
We’ll create an unlit URP shader graph, a material, and apply it to the quad.
Next, make a MainTex variable on your shader, and set up whatever sprite you want on it- I chose to fix up a sprite sheet of Paper Mario himself from the Spriters Resource. Plug that into your color and alpha channels, and then we’ll start building the shader.
In mine, I’m doing two separate texture samples- this is redundant and not necessary, but will make things look cleaner later.
Shadow masking
Let’s discuss the way the shader will work at a high level. when a dynamic light (such as a point light, spot light, etc) is behind the sprite, we want to create a shadow on the sprite, offset by the direction toward the light. if it’s up and to the right, the shadow mask will be down and to the left, leaving some of the sprite still “within” the light.
Additionally, we want the shadow mask itself to have some dynamics when moving the sprite behind the light. the shadow should fade away to match the lit color on the way around the light, and when moving away from the light, the lit color should attenuate down into shadow.
To begin with, we’ll make all the hookups we need to ensure these blends and offsets work.
We’re going to make another sprite sampler. The shadow mask matches the sprite mask, so we’ll want to take the alpha mask out. we’ll reverse it be subtracting it from one, giving us black where the sprite is and white everywhere else. We can port that and the original sprite color into a blend node with a shadow color (with the shadow color in base and the sprite color in blend), set the blend type to overwrite, and set the blend color to black. We’ll use a UV node and add a Vector2 to it that we can define an offset with, and port that into the UV pin on the sampler.
We’ll also set a custom sample for the shadow mask- the shadow mask in Paper Mario is slightly blurry around the edges- also set the wrap to clamp, so we don’t get looped shadows if we move it very far.
Playing around with the Vector2 offset, we can see the mask shift around, giving us the emulation of a harsh backlight
Ambient fill
The Shadow is a little TOO harsh. instead of just porting in solid black, it would be nicer to have a base ambient fill. The easiest way to do this is to take some color, multiply it with our sprite color, and port that into our overwrite blend node.
This is much better, but it’s still not exactly like the game. There’s what looks to be a vertex color gradient, brighter toward Mario’s face and darker to the back.
It’s probably WAY easier to have a quad, give it vertex colors, and just sample those in the shader, but we’re doing this without external modeling tools- we’ll have to get creative.
I won’t go into too much detail on the nodes here, but the gist is that we take the position of our fragment in object space, and process it’s local horizontal position to create the gradient we want. by using a power function, we can tighten up the gradient, and offset it a bit to make sure that gradient lies within the visible part of the sprite where we want it. We then multiply that by our shadow color, and then multiply that by our sprite color like before.
To take down a lot of the clunk here, we can put a lot of that in a sub-shader graph with the sprite color and shadow color as the inputs with the final multiplied color as an output
Perfect! Now it’s starting to look like the real deal.
Mask Fading
To finish up how we handle point lights, we’re going to want to handle walking around the other side of the light, and walking away from the light. when moving around the light, the shadow mask should blend out. this can be achieved by messing with the shadow mask value, by making the black area of the mask white. we can achieve this by uniformly adding a value to the shadow mask, and re-clamping the result between 1 and 0 – this effectively fades out the black part of the mask, therefore blending out the shadow.
For fading the light out to the shadow color, we can scale the value of our shadow mask. combining both effects, we now have full control of our shadow mask
We Need one last thing for our shadow mask- we have the offset parameter as a Vector2, but how will we actually process that? The way it will work is that we’ll take dot products between the light direction and each orthogonal direction of the sprite object’s transform to create a 2D offset from the X and Y axis, and a value for the addition from the Z axis.
We’ll create a new Sub-shader with the following inputs and outputs:
- Direction – light direction in world space
- Dampener – divider to shrink initial offset from direction
- MinOffsetClamp – lowest values in 2D the shadow mask can be moved to
- MaxOffsetClamp – highest values in 2D the shadow mask can be moved to
Inside the sub shader, we’ll create two paths.
Path A will handle creating the UV offset. we’ll do this by putting the light direction input into object space, and getting dot products against the objects’ axes. We split the direction into it’s component parts to ensure we’re getting the angle (and scale!) along ONLY the desired axis.
we can then combine the dot products together into one offset, dampen how big that offset is, and clamp it
Path B will handle the additive blend to our shadow mask based on if the character is in front of or behind the light. Same as before,we’ll just get the dot product along the desired axis and clamp between 0 and 1. This means we have a full shadow anywhere in front of the light, but a blended shadow anywhere behind it. I add a multiply by 2 here to make sure that blend happens faster, so we’re fully lit at much shallower angles.
Then, we can replace the vector2 offset we had before in the shadow mask block, and pass the Out_Blend into the add node. by setting the direction vector, you can see the effect different lighting directions can have on the sprite.
Outdoor Coloring
The last thing we really need is something to handle when we have a full environmental light, like a directional light. The effects we’ve been working on replicate the indoor, dungeon style lighting, but there’s also the basic lighting that we have for the outdoor scenes.
For that, we’ll just lerp our combined light an shadow color with the sprite color itself
We’ve got our color sources, our offset nodes, and our blend values- now we just need to have all those offsets and blends controlled by our lights!
Custom Lighting
To begin with, we’ll start by following the creation of the HLSL file from this wonderful tutorial about setting up custom lighting in a shader graph. Just as in the tutorial, we’ll create a custom HLSL file and fill it initially with the Main Light function.
We’re going to differ from the tutorial however on how to handle things like point lights and the like, which will deal with the shadow mask and indoor lighting.
We’ll make another function, call it what you want- just make sure it has _float at the end!
We’ll want WorldPos as an input again, but we also want a WorldDirection- this will be used to determine what side of the sprite the light is on.
It might be odd to put our Weight output between the Direction and Color output, but this will help keep our graph untangled later on.
First, we just need to set up a few variables. we’ll get the count of additional lights, set up a running total of the colors, directions, and weights of our lights, and a counter for the lights that are in front of the sprite.
We’ll create a for loop to go through each light. We’ll grab a light at the given index, and check it’s direction against the input direction, in this case, our sprite object’s world normal.
The light’s direction is the vector FROM the light TO the object. In my setup, the forward of the sprite object reaches into the world away from the camera, so the light is in front of the object when the dot product is positive.
if we have a front light, we’ll increment the amount we’ve found. If we haven’t found one before this, we’ll reset our total light contributions- we don’t need to worry about shadows of lights behind us when we’re lit by a light in front of us.
likewise, if the light is behind the sprite instead, and we have found any front lights, we’ll skip adding it’s contributions.
whatever lights we end up with after this filter, we’ll add to our totals, weighted by their attenuation. I’m doubling the attenuation to make the contributions of lights a little more equal, so we don’t get harsh shadow mask movement when passing a series of lights.
after our loop, we’ll set up our outputs with the total color, the weighted average light direction, and the clamped total weight of the lights.
With this hlsl file saved, we can start implementing this custom lighting in our shader.
Implementing the lighting
back in the shader, add a Custom Function node.
Set it to have the same inputs and outputs of our function, in the same order and data types, set Type to File, set the name of the function to the Additional Lights function name we made (minus the _float), and set Source as our hlsl file.
Add an object node, and port it’s position pin into the WorldPos pin of our custom function. Make a Normal Vector node in object space, and put the resulting normal into the WorldDirection pin.
Send the output Direction into the Direction input of the shadow offset sub graph, and send the weight into the multiply node of the shadow mask adjustments.
With just this alone, we can already see that we can adjust lights around our sprite and the shadow mask and intensity moves around!
But we still need to use that color output. we can simply multiply that with the sprite color, and now the sprite’s color will fit with the light
NOTE: this color does not effect the ambient fill in this setup. I'll leave having that match your lighting as an exercise for the reader.The only thing missing now is the Main light. Setup another custom function node, this time matching the information needed to run the MainLight function of our custom lighting HLSL file.
Add another object node and send it’s position once again into the WorldPos input. in that upper main light track we created earlier, multiply the resulting color with the sprite color.
All that’s left is blending between our main light and the additional lights somehow. We already have a lerp function set up, and a floating alpha input we haven’t done anything with yet. to replace it, we’ll take the Color of the main light, convert it into HSV, and use the V value (or, B in the case of the split node) to blend the lights- the lower the value, the lower the luminosity of the main light, meaning we can blend in the other light effects as a result.
lastly, we’ll put all that in a sub-graph to tidy things up, using the object position and sprite color as inputs, while the final color and value blend come out as outputs
And with that, the final graph is done! You should be able to see similar results to those below
And that’s that! I hope this helped you gain a deeper appreciation not just for Paper Mario, but for the clever techniques game devs in the 90’s used to create more immersive effects and really make the worlds we played in feel like more than just a game.