The Free Talk Live BBS

Free Talk Live => General => Topic started by: Level 20 Anklebiter on March 08, 2010, 10:28:18 PM

Title: [Soon-to-be graduate project] SquareOne: the SecondLife simulator done right.
Post by: Level 20 Anklebiter on March 08, 2010, 10:28:18 PM
Okay, I'm just posting it here until I feel up to putting together a wordpress blog to document the project properly.

First, what is SquareOne and what is SecondLife?

SecondLife is a product/service by Linden Lab which is more or less a 3d virtual world where content creation is relatively simple (note: relatively in terms of existing persistent environments or the lack thereof). It started out as a product that was suppose to be coupled with a 3d immersion rig until Philip Rosedale (founder of LL and former CTO of RealNetworks) realized that the rig wasn't practical or economical (to say the least). So they kept the core of what was then known as Linden World which became SecondLife. In SecondLife content comes in a few strict categories/types; animations, sounds, images (skins, clothes, and etc), primitives (aka prims), and scripts. Three of the five content types have to be uploaded (there's a special case for prims where sculpt maps are applied to them, thus the maps are images which must be uploaded too) and the other two can be created at no cost in game (prims and scripts). These are all uploaded to a server called an asset server. The actual world space is handled by a server called a simulator which contains a space at 256^3 in full (quite large until you realize most of the action happens below 180 meters on the Z axis, thus most space is wasted). Each simulator also handles the scripts which are attached to prims, thus these scripts are not compiled into machine code rather they're handled in bytecode form. Tying together the whole thing (aside from the login server frontend) is the grid, which is more or less a construct that allows simulators to talk to each other when a user moves from one to the other either those simulators adjacent or in a teleport across the entire grid space itself to a non-adjacent simulator. There is also a physics engine which is handling the activity of physical objects and actions on its side of the server/client relationship. And there's one other piece to this puzzle which handles microtransactions via their virtual currency called a Linden (L$), but its functionality bears no importance to the actual instant action of content and users in the simulator which is in the grid space.

Now, this all seems to work in general, but there are critical flaws in the design of SecondLife which even Linden Lab (corporation that owns this product/service). The first one is latency. Latency is often caused by many problems which tend to crop up together: too many users per simulator, too many scripts per simulator, and too many requests to an asset server from the same (or other) users. Pretty much this makes the architecture very brittle in its activity under heavy loads. At most, a single simulator can hold 40 users concurrently, but this is done with high latency and many failures of asset server requests. Even if there's not many users near that limit other things can trigger failures of simulators: physics. Physics is very hard on many computers, especially when there's many objects being acted upon at once like an explosion or an avalanche. And the asset server itself can simply fail to send requested content in a timely fashion too either due to too many user requests or due to its implementation in UDP (rather than TCP) for the transport layer activity. All this means is that the server side of SecondLife is quite a confusing mess of trouble due to latency alone. Other issues can arise such as lost/corrupt assets or scripts running amok due to a scripting language change by the developers, but these are oddly the rarest of the failures of the serverside part of SecondLife. I won't discuss the clientside issues of SecondLife as my project focuses specifically on the serverside issues and reimplementations.

Second, what is SquareOne? It's the name of the project in which I propose to reimplement the SecondLife server architecture in a manner that is more sane and practical for the existing technology as well as it will be future technology tolerant as opposed to how it is implemented today in its haphazard form. The term SquareOne came to mind when discussing this project with a friend and whether or not I should use the OpenSimulator code as my base which we both agreed that OpenSimulator was a buggier mess than SecondLife's own implementation. So you can see I am literally reimplementing SecondLife from 'square one.'

What are going to be the differences between SecondLife simulators and SquareOne simulators?

SquareOne servers will be space and content agnostic. That means you can design your public space in whatever dimensions you wish; however large or small. It also means whatever you create as content both as objects in the space or as behavior for those objects (as scripts) will not be handled directly by the server, but rather by the clients. This will be possible by the use of templates (implemented in XML) to tell clients what they are receiving and how they are suppose to behave or be displayed. Also in SquareOne no space will exist without a user. All spaces in SquareOne will be handled by users directly as private (client side only) or shared (cross client via client interactions) spaces or indirectly as a public space (persistent user space offloaded to a server in lieu of the user). This means that all content will have a user responsible to it (or even many users). All of this means that the server simply handles three basic tasks: authentication of users (and their content access), routing users templates for their clients to load (as to define how the space is to be presented), and to assign users to each other per the space they share (public space).

Another key difference between SecondLife simulators and SquareOne simulators will be the matter of distributed network computing via its implementation in Stackless Python. Which will mean that many physical servers can be assigned to any public space as needed rather than the current situation where classes of simulators are assigned either a single core of a rack or N number of simulators per core as Linden Lab does today. In this context, this means load can be balanced more fairly in a SquareOne implementation versus a SecondLife implementation (where even a class 7 simulator can have a core all by itself even if it doesn't utilize a significant portion of its purchased CPU time). What this meas for SquareOne is that it is designed in such a way that can utilize resources more properly and in a manner that can exploit maturing models of content distribution (web caching, cloud storage, and etc).

To be continued...
Title: Re: [Soon-to-be graduate project] SquareOne: the SecondLife simulator done right.
Post by: Level 20 Anklebiter on March 08, 2010, 10:28:37 PM
Space added for FAQs and links in the future.
Title: Re: [Soon-to-be graduate project] SquareOne: the SecondLife simulator done right.
Post by: Level 20 Anklebiter on March 24, 2010, 12:12:46 PM
As part of the design I'm looking at databases that are not SQL based like Cassandra, if anyone has their own favorite feel free to post here.