Need help with a pixel shader
Posted: Thu Aug 24, 2006 4:58 pm
Background:
I am trying to work around color key transparency used in D2(X(-XL)). Background is that D2 has two kinds of transparency: A regular one you can handle with an alpha channel, and 'supertransparency', where transparency in an overlay texture goes through the base texture.
So what I am trying to do is to pass a mask texture to the pixel shader rendering the base and overlay textures, telling it where the super transparency has to be applied. The mask has a 0 for each super transparent and a 1 for each opaque pixel.
I am using a GL_ALPHA texture as I only need the transparency info (actually a single bit would be sufficient).
I am binding the mask to TMU 2.
Now my problem is that apparently I can only get vec4 values from the TMU (RGBA, all float), although I specified the mask texture as single byte per pixel alpha. So the shader doesn't even compile. Here it is:
I cannot find anything on the inet telling me how to achieve the desired functionality in my shader program. Can someone here help me with this?
I am trying to work around color key transparency used in D2(X(-XL)). Background is that D2 has two kinds of transparency: A regular one you can handle with an alpha channel, and 'supertransparency', where transparency in an overlay texture goes through the base texture.
So what I am trying to do is to pass a mask texture to the pixel shader rendering the base and overlay textures, telling it where the super transparency has to be applied. The mask has a 0 for each super transparent and a 1 for each opaque pixel.
I am using a GL_ALPHA texture as I only need the transparency info (actually a single bit would be sufficient).
I am binding the mask to TMU 2.
Now my problem is that apparently I can only get vec4 values from the TMU (RGBA, all float), although I specified the mask texture as single byte per pixel alpha. So the shader doesn't even compile. Here it is:
Code: Select all
uniform sampler2D btmTex, topTex, maskTex;
uniform float grAlpha;
vec4 btmColor, topColor;
bool bMask;
void main (void)
{
bMask = texture2D (maskTex, vec2 (gl_TexCoord [1]));
if (!bMask)
discard;
else
{
topColor = texture2D (topTex, vec2 (gl_TexCoord [1]));
btmColor = texture2D (btmTex, vec2 (gl_TexCoord [0]));
if(topColor.a == 0.0)
gl_FragColor = vec4 (vec3(btmColor), btmColor.a * grAlpha) * gl_Color;
else
gl_FragColor = vec4 (vec3 (mix (btmColor, topColor, topColor.a)), (btmColor.a + topColor.a) * grAlpha) * gl_Color;
}
}