With the testmap issues fixed and bunch of new heroes added to the importer tool I feel like it’s time to talk about texture files and masks . The requirements page tells us that:
- The “_color” texture can be 24 or 32 bit.
- Alpha channel of 32 bit “_color” texture can be used as opacity for your item if alpha is allowed for this hero.
- The “_normal” texture should be 24-bit and use the 3dsmax format(red channel is left; green is up; blue is top). If you aren’t creating a custom normal map, use a flat normal color (128,128,255).
- “_mask1” is a 32-bit texture and every channel is used.
Red is the detail map mask.
Green is the diffuse and fresnel warp mask.
Blue is the metalness mask which controls where the Color and Fresnel are suppressed. Works best in combination with mask 2 blue channel “tint specular by base color” which brings the color back through specular.
Alpha is the self-illumination mask, where the color is drawn from Color texture. Even if you are not using this, fill it with black and pixel of non-black in the corner of the material. This is done in order for texture compiler to keep this channel. If it is entirely black or white, it will be thrown out and the shader will think it’s pure white and your item will be full bright.
- “_mask2” is 32-bit and again every channel is used.
Red is specular intensity.
Green is rimlight intensity.
Blue is “tint specular by base color”, white color in this channel will color specular highlights by diffuse color from _color texture, black will keep specular highlights of whatever color light affecting the surface is.
Alpha is the scale factor of specular exponent from hero shader(vmt). For example, if the specular exponent is set to 16, then values for this channel of 0…255 will control exponent values from 0 to 16. Even if you are not using this, fill it with white and pixel of non-black in the corner of the texture. This is done in order for texture compiler to keep this channel. If it is entirely black or white, it will be thrown out resulting in a different texture format when compiled.
And Valve even rolled a nice guide about the shader masks. You can find it here. Let’s take another look at them.
Each texture map channel serves it’s own purpose and we’ll start with the “color” map.
Color map is your main texture, it contains your diffuse map and an alpha opacity channel.
This will probably contain all your baked maps, color works and tweaks that you did and it’ll look something like this:
The alpha channel of your model defines where your texture is transparent and where it’s not. With every hero getting alpha opacity support you can use this powerfull tool to emulate volume or save your triangles by painting holes. Good example of that will be original Ursa fur which is actually a bunch of planes with alpha opacity:
Keep in mind that alpha map that looks like this:
Will look like this ingame:
Meaning that it will be hard approximated to black or white so you can use alpha opacity only for 100% transparent or 100% opaque states.
The way Dota 2 compiles their models after loading prevents us from using multiple/different materials for different parts of the hero, but it keeps the hero all in one shader space, preventing cosmetics looking out of place (but there’s still bunch of ways to fuck it up!).
Next one is a normal map, fella that you should all befamiliar with. it doesn’t use it’s alpha channel for now so you better save them as 24 bit TGA to avoid any problems with those for now. There’s a rare bug that will crash your Dota if there’s alpha on your normal map.
Average normal map will look something like this:
And it will give your item that extra volume and combined with the spec map will make it look even prettier!
It’s better to use normals where possible and bake them from you highpoly model all the time. It’ll only add to your model, will allow you to bake maps and will combine greatly with the other masks.
While some heroes rarely use normals (like Juggernaut, hald of his economy items and even default ones have blank normals) While others heavy rely on them (Lion is a great example of that). Normals will allow you to add volume to the painted on details
Max and Zbrush can bake normals themselves, or you can use something like xNormal to bake your normals, it’ll take short amount of time, but will save you a lot!
Next one we’ll talk about is a mask1 map!
Whis one is easy, first of all Red channels contains Detail map mask. This one allows you to add animated texture effect for heroes that support it (Rubick, Outworld Destroyer, Razor, Chaos Knight, Chen and so on). OD for example. His detail map looks like this:
And on the model it looks like this
And the mask looks like this:
Note that mask1 and mask2 allow full grey spectrum so you can use gradients on you detail mask and every other mask.
Green channel is the diffuse and fresnel warp mask. This one israrely used, and most of the time it’s used to add custom shading to the light falling on the surface, like adding more green in shadows. Page 9 of the Shader guide explains this really well, but most of the time this one goes unused.
Next one is our favorite, Metalness mask on the Blue channel. It does exactly what you think it does, makes your material look metal. It darkens the diffuse map, and reduces the rimglight, so if you’re going to crank the metalness maks, you may end up with glossy, but pitch black model.
The black parts of the SK armor use white metalness mask, whilethe sword uses only 45% grey. Metalness mask works great with your normal map that we talked about earlier and can help you achieve shiny metal look that you can also tweak by tinkering with the mask2.
Alpha channel of the mask1 map is the self-illumination mask.It’s very important to never leave this one fully black, you should at least have one non-black pixel on this map to avoid issues with your model when importing it ingame. This mask makes masked parts of your texture to ignore lighting, which practically means glowing in the dark.
The green lenses are all self-illuminated which makes them glow in the dark and add glow (bloom) effect ingame. You can use this one to keep pitch blacks, add subtle glow or justto make sure your model is well-lit on the Dire side of the map.
Next up is mask2!
Red channel is specular intensity. This one will allow you to modify the amount of specular on your model, some parts of it maybe shiny, like leather, metal, others could be more matt, like fur or hair.
Great example of use will be Centaur Warruner:
Notice how shiny his armor is and how his skin and fur is almost completely opposite.
As you can see on his map, his skin is really close to blacks, while the armore bits, chains and fingernails are way brighter. You can use this mask to create different effect on your texture. For example, most of the engraving on the Nyx armor is made not only via diffuse map, but also through the clever use of using different spec intensity:
On the Green channel we have Rimlight intensity. Rimlight is that strong, contouring backlighting that’s used in both Left 4 Dead and Dota 2. Some heroes almost don’t use it, see Nyx above and others use A LOT of it, like the Centaur
You may want to use this one every chance you’ve got, with the rare exception of the parts of your model that are supposed to be black, like crevices or seams, so it might be a good idea to use your baked AO map for this. make sure you match the amout of rimlight on your model to the rest of the hero.
For example, DK helm is pretty plain Rinlight-wise, as you can see, gold trim has lower rimlight values, while the res of it is pretty bright and plain
Third channel, the Green one is “tint specular by base color”. What this means is that normally, your specs are white, this means that the surface is fully reflective. When we’re changing this mask , we can change the specular to be the color of the diffuse map. For example, if you take a look at the Ogre Magi, you can see that the specs on his bracers gold and on his skin are not white, but rather colored with the diffuse.
His masks looks like this:
Darker parts are more tinted by the diffuse color. You can use this effect for variety of results, mostly on skin, fur, colored metals, leather and lots more. Note that if you want your material to look like glass(highly reflective material), for example, you’ll want your tint to be fully white.
And last but not least is the specular exponent on the Alpha channel.
This mask allows you to modify the scale of your specular exponent. What this means for you is that you can make you specs sharper or more diffuse.
Keep in mind that this mask is highly dependent on the heroes material, so sometimes full white exponent will still be far from sharp specular. There’s a great diagramm showing how combining specular and exponent works:
Combining different maps yields different results and can help you achieve lots of looks in the confines of a singlt material. Maps can be really simple (Tusk weapon):
Or complex (Slark shiv):
But this in no way means that having flat mask will retract from the overall look of the model. if you’re not sure about how to go with the masks for your item, just reference the original items of the hero you’re creating for and it’ll come out great.
What different use of masks can you spot on the new Tusk set we did: