Graphics

We are growing more involved with AR on Ross XPression systems. I am working on a template scene that I will hand off to my 3D animator to actually fill in with the "real" graphics.

Today's issue is shadows and reflections. I'm wondering if and when XPression will actually include shaders to do these - even the Ross people I've worked with that are doing AR are wanting such things. Until then, we need to figure out another way to do it. It seems to me you can find shaders online that will do shadows and reflections.

My technique for shadows is to duplicate the geometry, apply a "shadow" material to it, and scale that whole group down to Y=0. We can adjust alpha to make it more transparent instead of a hard black shadow. The problems are that it duplicates geometry, that it will only map to the floor, and we can't blur it (so the shadow always has a hard edge).

My technique for reflections is to use a "reflection" camera that mimics the perspective camera except: negative Y, negative tilt, and negative roll. A renderview from the perspective camera is then composited over a renderview from the reflection camera (with -Y scaling) in a third, orthographic camera that is far away from the actual scene. Far clipping planes prevent the quads the textures are mapped to from appearing in the perspective scene. I like this technique a lot, because not only does the animator not have to duplicate geometry, but since we're applying a material to a quad we can add a guassian blur effect to it.

So the scene contains two perspective cameras (one for the reflection), and an ortho camera that is showing the two quads the reflection and perspective views are mapped onto.

The problem is that when we actually try to use this with AR, only the active camera can be tracked. The active camera needs to be the ortho camera to get the composited views of the other two cameras, but it's the perspective camera that needs to be tracked. So now I'm stuck without a good solution for reflections. We can resort to duplicating geometry (again), but unless I'm missing something they will be the inverse geometry - but lit from above. And no blurring.

So I'm asking the community for help in solving the problem of getting good, realistic reflections and shadows. I'm attaching my simple template scene in hopes someone has some ideas.

Hello fhaab and thank you for sharing your work !

Following the same things you did for reflection , you can "cast" shadow with another perspective camera .
This camera has the same information of your light : position , direction, ....
You use far plane to hide object "under" the floor.
map this camera on a quad with renderview material (alpha mask blending mode seems to do the trick to remove colors)

You don't need anymore to duplicate objects, and if you configure properly your camera, you will have a good projection of the shadow .
see project attached .
In this example the ShadowCamera is wrongly configured for position, fov, etc.. but it's just to show you how you can make it work.

Hope it helps,

Antoine.

#XPression

Thanks - I actually tried (something like) that at first, it's very awkward to make it work right. I may try again with a lot of clipping planes added it, assuming that would work. Far clipping plane doesn't work if the camera is not pointed directly down at the floor, because the far plane will otherwise cut through the "floor" at an angle.... and the question of how to map it to the floor is not simple.... the benefit was that you could add a blur effect. I may look into it again.

Thanks for looking, though. My main problem right now is still that I cannot track the perspective camera unless it's active, and when it's active the compositing technique of putting all the parts together doesn't work.
#XPression

you're right, it will not work unless there is a specific configuration for the camera and floor plane (like in the example).

I see there is a "drop shadow" in the keying effects group, but I don't know how to use it.
#XPression

A basic shadow is possible using a camera directly above the group, pointed down, and it must follow the general position of the objects if they are moved (and they are, in our case, not just animated but have a number of positions depending on how we're using them).

The far clipping plane of the shadow camera must be set so as not to capture the shadow being mapped on the floor itself (otherwise it captures itself). It cannot angle like the light normally would. These are all tolerable drawbacks - that shadow is little different than the technique of duplicating geometry, and yet has added benefits.

I've also solved the reflection plane problem by putting a quad under the perspective camera group to map the reflection image onto. It follows the position and orientation of the camera automatically; it's set far back in Z, and is scaled in a way that when FOV changes, the size of the quad automatically fills the frame. it was more complicated than I thought - it's a non-linear scale through the FOV range. In a somewhat inaccurate simplification: scale = Z * tan(FOV) in visual logic. It actually works.

The only problem is my world was shattered as I was playing around with it and realized the perspective on the reflection is inaccurate (and I completely understand why, but can't think of a way to fix it and get the proper view of the objects). The only way to do it would be with a proper reflection shader. It might also be possible using multiple scenes, but that's not a satisfactory solution for a number of reasons. The way it is now is wrong, but under most circumstances should not cause a problem.... still, it should be possible to do what I'm asking without all these hoops and drawbacks.

I realize XPression was not designed with AR in mind, but they are selling it as an AR solution, so I think these things should get some consideration.

Drop shadow is an interesting effect, but not really applicable here.
#XPression

Lastest version of project, using a shadow camera and eliminating the ortho view composite.

#XPression

Aaah... fixed one problem; in the visual logic there's a value that's labeled "Horizontal Size/2" and it's actually a quarter of 1920.... it should actually be Vertical Size/2 and should have a value of 540.

EDIT: attached latest version of scene; the scale value is corrected, uses an overhead perspective camera to capture the shadow. The scene runs very quickly without any effects added; adding guassian blur really dogs it. Maybe on our production machines it would work great, but not on my desktop.

Still welcoming any comments or suggestions.
#XPression