Second Life Graphics Could Be AAA Quality if Linden Lab Added Custom Skeletons & Shaders (Comment of the Week)

Global illumination in Unity 3D

The passionate conversation over Second Life graphics quality continues, with master SL builder Adeon Writer (more on him soon) arguing that SL could have AAA game quality if Linden Lab addressed two gaping holes:

The problem is, SL content creators are not pros, and that's why it doesn't look as good, right?

No. If you hired an expert game artist to make SL content, it still wouldn't look good, for the following reasons:

1.) The avatar skeleton is pre-defined and you can only make animations for that specific skeleton. Modern games use a skeleton with many more bones, to make it look much more natural.

2.) SL's shaders are bad. You can take the best model in the world, but it will look very bad with SL's shaders. At one point, SL had top level shaders ("Global Illumination" is what the feature was called) but they pulled it out and never put it back. It would have looked wonderful today now that we have normalmaps, but sadly we never got to have both at the same time. So we're back to flat shading.

If Linden improved these two things -- 1.) Let us have custom skeleton models and 2.) Give us custom shaders -- then yes, if you hired an expert game art designer, you SL could look just like a AAA game. These really are the only two things holding SL back graphically.

And no, Adeon doesn't think Second Life's new Bento skeleton doesn't improve things enough:

"Nah, Bento doesn’t give us custom skeletons," Adeon told me in a follow-up e-mail. "To be fair it gives us a more detailed one, with finger bones, face bones for expressions, etc, but if you want to make a avatar for second life you still need to design for their specific skeleton rig, and that’s what causes serious modelers to skip on SL. You can’t make your own thing, make animations for it, and put it in SL, which is what any pro designer would be expecting.

"Now, Linden would probably say that they do that for a reason, of course: you can’t have an animation market if everyone isn’t using the same skeleton. Animations only work for the skeleton they were made for! However, games don’t work that way. Every characters is free to have their own rig for their own needs, and is not forced into a one-skeleton-fits-all system, and because of that, 3D artists are free to make exactly what they want, instead of designing around someone else’s (Linden’s, in this case) base."

As for global illumination for Second Life, more on that from Adeon:

"Basically think of an untextured sphere prim being lit by the sun in Second Life. It’s white on one side, grey on the other, and is a simple gradient. That is the most basic, simple shading possible, and no matter what crazy amazing WindLight settings you use, that’s the only shader you get. A gradient from the sun color to the ambient color. Games never look like that, they use various colors at various angles to give objects a much more varied shading." (See the Unity pic above.) "Better ones do it dynamically, taking what’s in the surrounding and making a shader based on that, so it looks as if light is bouncing around and lighting objects realistically.

"The saddest thing of all, is SL had this at one point, it was called a global illumination, and it was perfect. However, we did not have the toolset at the time to actually fully utilize it and make it sing (alpha masking and normalmaps), and had we had both features at the same time, they’d have never have removed it, because SL would have looked amazing."

Video of the feature above, from 2010. Thing is, if enough SL developers call for custom skeletons and illumination, Linden Lab might finally enable them.

Comments

[Adam]

I'm a bit of a biased source; but I do know 3D graphics engines, and in my personal opinion shaders aren't a magic bullet. (I say this as someone with a pretty soft spot for SL - I spent many years in there, and wish it the best!)

SL's renderer is ancient - it's a 2001-era engine that has been retrofitted a couple of times to try drag it a little bit further into the modern age; to do this kind of thing justice, you'd have to do a total rewrite of the renderer. The highly optimised renderer that preserves a lot of quirks to ensure rendering compatibility.

Your first problem will be backwards compatibility - upgrading the renderer means replicating all those quirks (e.g. 'invisiprims') - because breaking existing content is a strict 'no'. This is going to make life hard; for two reasons.

1) To light things properly, you need to live in 'linear' colorspace - everything in SL is designed for gamma, and taking gamma content and putting it in linear looks garbage - washed out gray with no vibrancy. Everyone will revolt.

2) Modern rendering now is a purely PBR ("Physically Based Rendering") workflow - to realise the benefits of that, all the content needs to be upgraded to it. Unfortunately, there's no magic "make this PBR" wand - you need to think in terms of materials, not textures; and often PBR materials need 3-5 textures per set, instead of where you only had one or two in the past.

Those need to be created for petabytes of legacy content; and again, if you don't - it looks a bit garbage. Either everything is plastic, clay or metal, or some godawful mix of the three.

This is worse because 'Diffuse' texture maps with lighting baked in, don't look good in PBR (where you need to separate lighting from color information).

Keeping content looking modern for the long term (10+yr horizon) - requires a healthy amount of forethought and planning explicitly for it. More input data than a single texture helps a lot too. (Look at film rendering for hints at what is to come - what they're doing today, games'll be doing in 5-10 years)

That however; is not the only problem, the next problem you're going to face is performance - GI (Global Illumination) can be tackled in a few ways, but due to SL's architecture ("potentially, nothing is static") about 90% of those options are off the table (baked GI is the best in terms of performance) --- you're left with a bunch of realtime voxel solutions which have nasty light leakage on fast moving objects or scenes, or horrendous framerate impacts.

This is compounded by the average system spec, which is now getting quite old; and 'modern' computers tend to be laptops with underpowered Intel chipsets -- your average user does not have a gaming PC.

Finally, the biggest problem is the lack of lighting - modern game scenes often have hundreds of lights within the scene, placed by professional lighting artists to achieve the nice mood lighting you get. SL's existing content doesn't have that because there's no real way to place those kinds of objects within a region easily. Ditto reflection probes and other modern 'tricks' for realistic rendering. (With mostly-static scenes, you can at least do a nice radiosity bake which makes up for this.)

I feel the images/video used actually don't really show the gulf between where modern rendering is at; this is Unity's current tech demo: https://www.youtube.com/watch?v=DDsRfbfnC_A

The screenshot listed is one of theirs from over 3 years ago... and I'm sure Unreal's current demo is even better (although Unreal has problems with virtual worlds due to content loading/unloading issues) --- that's today, by the time this is implemented (which is a 2+ year project), the technology will be even further ahead --- the brains in rendering these days is jumping all over the realtime raytracing tech Microsoft, AMD and nVidia just announced; which looks potentially phenomenal.

If you want to make this happen, you need to accept that content'll have to be re-authored, or at the very least, tweaked, and "old stuff will look quite bad" - which might be better suited for a '2.0' product where that kind of thing can happen.

[Hamlet Au]

Holy crap that demo tho

[Medhue]

No offense to Adeon Writer, but he is COMPLETELY full of shit about the Bento Skeleton. I've made animated animals and Human avatar for almost every game engine. Those very same things that I made for those other engines I also rigged to the Bento skeleton, and released in SL. The Bento Skeleton is not some obscure rig that is different from any other rig. Essentially, a rig is a fucking rig. The Bento rig, and SL system can do FAR more than any rig/system out there, including morph with the SL sliders. Find me another rig that can do that, ANYWHERE!

Custom rig support would be a complete nighmare in SL! It would require a whole new system from the ground up, and NOBODY would fricken buy a Custom rig avatar because the Bento Avatars would be 10 times better, with SL sliders. Not only SL slider, but full integration with the Look At feature, gestures, Lip syncing and all.

AND, you can then take all those BENTO AVATARS, and upload them to any fucking game engine in the world, because like I said, the bento rig is not so different it can't be used elsewhere. Maybe Adeon should stick to shit he actually knows.

[APW]

As far as rig goes, they should just make a componentized one that breaks major skeleton segments apart then use something like an IK rig with animation blending to stitch them back into a full body animation afterwards so people can selectively lego-brick their skeletons and animations together.

Not sure if Medhue really wants to stand by that claim of theirs, as the fundamental problem is the fact that rigging to a skeleton that has to be excessively deformed into a shape to fit in a desired body type that it was not originally made for is itself a horrible compromise that can have many problems down the line in actual run-time animation of whatever is rigged to that skeleton.

The bento skeleton poses many problems because of that. Yes, you can take a bipedal rig and force it onto all fours and make the limbs flex in the wrong direction while forcing some bones to collapse on themselves in order to mush the thing into a dog-like form for animating. And then you have the uncaught errors that result from that where someone loads into an area where that's been done to a rig and their avatar looks like an explosion of nonsense because those deformers were not loaded on their client to see what your client does.

Bento has more flexibility, but is a mechanical nightmare still. There are far better solutions to be offered through componentized rigging and skeleton stitching than Bento will ever be a practical solution for.