Displacement Surfaces 2: Alpha Masking
by ReNo (view all articles)

unrated
Texture variation within a terrain surface
by ReNo (view all articles)

unrated

EDIT: This tutorial is somewhat redundant now, as the paint alpha tool in hammer now works with the release of HL2. I'll update it soon to reflect the changes, and discuss how to use the paint alpha tool in detail. Feel free to read the tutorial to get a better idea of what exactly it is doing if you feel so inclined however!
So you have made your piece of terrain, but you can't help but feel that grass really shouldn't be growing on the steeper sections, or that the flat areas would look much better covered in sand rather than rock. Well the heavens have heard your cries and the ability to vary between two textures on a single piece of terrain is made possible using the "draw alpha" tool. Before you tackle this tutorial, it is suggested you have read through and understand the principles discussed in part one.
Below is a piece of terrain created using the displacement surface tool in hammer. It comprises both steep and flat sections, and while the structure of the terrain is just how we want it, we can't help but notice it looks pretty unnatural being made up of only one material.
Unfortunately you are limited in two ways as to what materials you can use on your terrain. The first limitation is that you can only use materials specifically made for this merging technique, those that are typically specified by the word "blend" in their name. The other limitation is that you cannot choose the second texture that will be used, as it is defined in the first material's definition file and cannot be changed from inside hammer. Obviously to get around this you can create your own materials to get the combination you desire, but that is outside the scope of this tutorial. For now, pick a blend texture you like and its variant can be viewed by selecting the displacement surface and using the "invert alpha" command.
Now comes the fun bit...or at least it would be if the feature worked. You see, CURRENTLY hammer seems to have a bug that prevents the "paint alpha" tool - the one used to determines how much to merge between the two materials - from doing anything at all. We can speculate about how it works, in that the interface is the same as that of the paint geometry tool, but for the time being we have to do our material merging the long and hard way! Below is a picture of the paint alpha tool, it all its non-functioning glory...
And so it is time for notepad! Thats right, until hammer has this problem fixed, our workaround come by manually entering alpha values for each point on our terrain grid. Because of this, I highly recommend you copy your terrain into an empty VMF file as this will make the face you wish to change the alpha values for much easier to find. After you've done your tinkering you can then just copy and paste the terrain back into place in your actual map. You may also want to rethink bothering with this if you have a very large piece of terrain - you are going to be looking at a grid of numbers alone and if you have hundreds of points that will be hard to visualise!
Open up your terrain VMF using notepad and you will be presented with a piece of, hopefully short (depending on how complex the map is - hence why you should copy your terrain into an empty map!), code. Your first task is to locate the correct section to do your tinkering. If you only have one displacement surface this should be a reasonably simple affair - scroll through it until you see "world", then look inside this group for "solid", and finally look for a "side" group that contains a "distances" subgroup with varying values. These values are the heights of each point on your displacement surface, so if you feel like fine tuning them manually you now know where to look!
The part we are interested in is the "alphas" section, which should be a few subgroups down from the "distances" part. The format of this section is a two dimensional array (or a grid to joe mapper), and it doesn't take a genius to see how this relates to our terrain grid in hammer. By default this will either be filled with 0's or 255's, depending on which material you had applied when you saved the file in hammer. Having a zero on a point means you will be using the default material, while having 255 means you will be using the alternative. You can also enter intermediate values to have a mix. Once compiled the map will merge the textures across each polygon based on the values at each point, giving a nice smooth result. Its worth using the "distances" part of the file as a reference for what values to enter here - if there are big differences between two values in that section then that polygon will be steep, while if they are quite similar then the polygon will be quite flat. Here are two pictures - one of the values entered in notepad, and the other the resulting terrain in hammer.
Now, you guessed it, compile and enjoy! God knows you deserved it for putting up with this annoying workaround
As always, comments, suggestions and questions should be left below.
So you have made your piece of terrain, but you can't help but feel that grass really shouldn't be growing on the steeper sections, or that the flat areas would look much better covered in sand rather than rock. Well the heavens have heard your cries and the ability to vary between two textures on a single piece of terrain is made possible using the "draw alpha" tool. Before you tackle this tutorial, it is suggested you have read through and understand the principles discussed in part one.
Below is a piece of terrain created using the displacement surface tool in hammer. It comprises both steep and flat sections, and while the structure of the terrain is just how we want it, we can't help but notice it looks pretty unnatural being made up of only one material.

Unfortunately you are limited in two ways as to what materials you can use on your terrain. The first limitation is that you can only use materials specifically made for this merging technique, those that are typically specified by the word "blend" in their name. The other limitation is that you cannot choose the second texture that will be used, as it is defined in the first material's definition file and cannot be changed from inside hammer. Obviously to get around this you can create your own materials to get the combination you desire, but that is outside the scope of this tutorial. For now, pick a blend texture you like and its variant can be viewed by selecting the displacement surface and using the "invert alpha" command.

Now comes the fun bit...or at least it would be if the feature worked. You see, CURRENTLY hammer seems to have a bug that prevents the "paint alpha" tool - the one used to determines how much to merge between the two materials - from doing anything at all. We can speculate about how it works, in that the interface is the same as that of the paint geometry tool, but for the time being we have to do our material merging the long and hard way! Below is a picture of the paint alpha tool, it all its non-functioning glory...

And so it is time for notepad! Thats right, until hammer has this problem fixed, our workaround come by manually entering alpha values for each point on our terrain grid. Because of this, I highly recommend you copy your terrain into an empty VMF file as this will make the face you wish to change the alpha values for much easier to find. After you've done your tinkering you can then just copy and paste the terrain back into place in your actual map. You may also want to rethink bothering with this if you have a very large piece of terrain - you are going to be looking at a grid of numbers alone and if you have hundreds of points that will be hard to visualise!
Open up your terrain VMF using notepad and you will be presented with a piece of, hopefully short (depending on how complex the map is - hence why you should copy your terrain into an empty map!), code. Your first task is to locate the correct section to do your tinkering. If you only have one displacement surface this should be a reasonably simple affair - scroll through it until you see "world", then look inside this group for "solid", and finally look for a "side" group that contains a "distances" subgroup with varying values. These values are the heights of each point on your displacement surface, so if you feel like fine tuning them manually you now know where to look!

The part we are interested in is the "alphas" section, which should be a few subgroups down from the "distances" part. The format of this section is a two dimensional array (or a grid to joe mapper), and it doesn't take a genius to see how this relates to our terrain grid in hammer. By default this will either be filled with 0's or 255's, depending on which material you had applied when you saved the file in hammer. Having a zero on a point means you will be using the default material, while having 255 means you will be using the alternative. You can also enter intermediate values to have a mix. Once compiled the map will merge the textures across each polygon based on the values at each point, giving a nice smooth result. Its worth using the "distances" part of the file as a reference for what values to enter here - if there are big differences between two values in that section then that polygon will be steep, while if they are quite similar then the polygon will be quite flat. Here are two pictures - one of the values entered in notepad, and the other the resulting terrain in hammer.


Now, you guessed it, compile and enjoy! God knows you deserved it for putting up with this annoying workaround
