I don't need your pyramid models, I have all the models I need. I also don't need your insight about face culling, because I am already three miles ahead of you in this area.
But I will explain to you what the problem is. To render the shadow volumes, you have to extrude the volume sides from the edges that form the silhouette of the model as seen from the light source.
Now face direction (front/back) is determined by OpenGL with some geometrical operation depending on the position of a polygons vertices (polygon winding). Lets take a quad for the sake of simplicity: You have four corners v[4] at (0,0), (1,0), (1,1) and (0,1). That is GL_CCW (counter-clockwise). You can determine this by determining the sign of (v[1].x - v[0].x) * (v[2].y - v [1].y) (you don't need to consider z for this). If it's < 0, you have GL_CCW, otherwise GL_CW.
Extruding a volume side quad goes by using its edge's two vertices (ev[2]) and computing two more vertices by projecting them to infinity as seen from the light source (lv). This goes like
ev[2] = (ev [1] - vl) * INFINITY
ev[3] = (ev [0] - vl) * INFINITY
where INFINITY is some sufficiently huge value. (You can do infinity projections by setting up a special transformation matrix, but it works good enough this way).
So far, so good, but when computing the silhouette edges, I am getting their vertices in some arbitrary order. That means that the volume sides extending from them have some random winding. That means that they can be considered a back face by OpenGL although they should be a front face and vice versa.
(A silhouette edge is an edge between two model faces where one of the faces is visible from the light source and the other is not).
Generally in D2 all faces are defined GL_CW (so you have to enable GL_CULL_FRONT if you want to have OpenGL not render the faces pointing away from the viewer - a small quirk in the engine). Now consider two adjacent model polygons (i.e. sharing an edge). Depending on which I treat first when determining silhouette edges, I will get the edge's vertices in the order (v[i],v[i+1]) or (v[i+1],v[i]) (i being a valid index of the polygon's vertex list).
I am having an example using a cubic model where this already causes problems.
I have no idea how to fix this. And this the key to making this stuff finally work.
Edit: Maybe I can determine the winding of the faces after having transformed their vertices ... a volume face should have the winding of that face of its edge that is invisible to the light source ...
Edit 2: Seems like I can't ...