Sunday, July 18, 2010

Do No Harm!


I logged in tonight to find two of my scripted objects were returned to me by the land manager where I live. The manager tells me that the scripts were killing the sim, so she had to return them. Fair enough. I don't want to be the cause of pain. Now comes the rub. The scripts in the troublesome objects have been running on this sim for just over 2 weeks without an issue. Today, there is an issue. Today they have suddenly become sim killers. What went wrong??

I learned that a patch was put into place on the server running the sim I live on. That patch it seems, changed the environment in such a way that my scripts suddenly became hostile to the environment. The result, was that the scripts were now, well on the way to killing the sim before they were returned.

Do No Harm! Had I been selling this object in my shop, they could be spread all over the grid. If that were the case, as this new patch is distributed across the grid, several sims would be choked into submission by my now, rogue script.

Say what you want about papa grid but one thing I know they never do is break existing content. There is no excuse for breaking content. I understand why it may happen, but the lack of due diligence isn't an excuse, it is a problem. A problem, if not addressed, will cripple this growing grid faster than any other problem that comes to mind. So with this in mind, I would like to propose a rule for IW development:

Rule #1: Do Not Break Existing Content - EVER.

14 comments:

  1. This bug was patched an hour after it was reported. We rolled the update to land owners that were online at the time and noticed the problem. We were not able to roll to the entire grid but all affected regions should be patched today.

    This is a bug. Nothing is broken. I will never promise we won't have bugs, that would be silly.

    ReplyDelete
  2. While it is true your reaction to the issue, once reported by the user community, was quick and to the point, this bug should never have reached the grid. The bug wasn't some deep Trojan, revealed only during a little known use case, but was evident the moment it hit the grid. So either, you didn't test your patch before applying, or your test procedures are woefully inadequate.

    Bugs happen, this is true, but it is not "silly" to expect patches to receive the most basic of quality tests prior to release.

    ReplyDelete
  3. did they break it or did they render it dysfunction for a short amount of time until they caught it.

    Can't really say SL doesn't 'break things' cause they have..mono to LSL or whichever way that went had people scrambling. Sculpties..didn't break content but it sure did make some obsolete. New script limits if they ever get put in place...it may not break content the way we traditionally look at it but it sure will tick off a whole bunch of users when their content doesn't work because it goes over the script limit.

    I think it's really important (for me to remember too! I forget all the freaking time) IW is not SL and there is no real comparison between the two other then appearance.

    ReplyDelete
  4. LMFAO!! All they do is break content over there! I can't even count the number of times over the years I've had to rescript, do a thousand (literally) change-outs for customers, because the dork grid decided to change something.

    ReplyDelete
  5. This comment has been removed by the author.

    ReplyDelete
  6. Misty - My content caused the sim to crash after the environment changed under it. The development team did not notice the issue until it was pointed out by the user community, which took only a few short minutes. The negative impact was obvious and should have been discovered during a testing phase.

    ELQ - I may have been lucky with never having my content broken by an environmental change over on papa grid so I defer to the experiences of others. In that light, I retract the statement saying papa never breaks content. I was not around for the early years so I may have missed the fun.

    However, I do believe that the two grids can be compared to each other for they share more similarities than dissimilitudes. The vast majority of the source code is the same. I do share your hopes for a different culture though. I already believe the culture in IW is much more open, accommodating and generally friendlier. This is directly attributable to the founders and the many other early contributors such as the mentors.

    In short - I am trying to hold IW to a higher standard than the one typically used on papa grid. Over there, people are use to the rug being ripped out from under them but that is not a necessary evil we creators must endure. Solid development practices, established early, can contribute to IW standing head and shoulders above all other grids. While it may not be practical to achieve a stated goal 100% of the time, that does not mean you shouldn't set it as an objective. My proposed rule for IW development stands and I strongly hope it is adopted.

    ReplyDelete
  7. Hmm well my viewpoint is affected recently by a quote I saw on the OSGrid page "OpenSimulator is experimental software. If it breaks, you get to keep both pieces."

    This was in reference to making your own home region and connecting it to the OSGrid but fundamentally the system; be it run by others or used on a small scale on your personal hard drive, is far from complications and unpredictability.

    I also made my own home region on my hard drive and it was challenging. I'm not at all a programer and I have a new respect for those who do this all the time on a larger scale and actually comprehend the code fully.

    I do understand your frustration Tiger. Try to be compassionate though to the grand scale of what the programers deal with. Hopefully your situation is temporary and soon restored to functionality. One thing I can say about IW is they seem so very committed to doing their best to accomodate content creators and all of our numerous concerns.

    ReplyDelete
  8. Ooh now I understand what happened. I did follow that thread, knew it was fixed relatively quickly but didn't get what actually happened. Yes, definitely more testing is needed. I'm not sure how that works though, can everything really be tested..that's alot of testing. Anyway, glad they got it fixed.

    Which if we do compare, is very different from the papa grid...because they would have shrugged their shoulders and let the ticket sit without a second glance. That is definitely a higher standard.

    For the record...I think Tiger is probably one of the most compassionate for IW, from everything she has posted and by all appearances, she *really* wants this place to succeed. I hope you bring more of these issues to the blogs (lord knows I'm fluffy enough for us both at times) because it makes us THINK and that's always a good thing!

    ReplyDelete
  9. Let me clarify.

    This bug only affected scripts that had a script reset in the inventory changed event. That's it. It happened that a very popular script by one of our residents had this behavior in it which is why the bug reached so many simulators. It was a temporary condition. Nothing was lost or changed permanently because of our changes.

    We will never be able to produce a test suite that covers every possible thing a script could do, however, one thing that could be done is to always take a copy of a few live regions and test those on beta grid to make sure nothing jumps out. Most of the time though testing requires a lot more than this and needs to be better targeted. This kind of stuff doesnt show up in unit testing because that will only test that the inventory item was actually added to the collection properly, which it was, except it was firing a stray event.

    I will mention that the patch went out more quickly than normal because there was a serious issue that I will be reporting to the opensim people. It was serious enough that I think it needs to be fixed there as well.

    I will never promise bug free releases. We will do the best we can and that's the best I can do. I will promise to improve our process as our experiences change.

    Anyone that promises that all their releases will be bug free also has a bridge to sell you.

    Trying to compare us to the big grid saying they dont make buggy releases that break the existing world is highly inaccurate. Just recently the 1.40 release started destroying people's inventories and had to be rolled back. This is a company at least 100x our size. Rollbacks happen all the time there, just look at the grid release notes.

    http://status.secondlifegrid.net/2010/06/17/post1030/

    If you want to compare apples to apples with the facts, there you go. Im sure if you go back far enough you'll actually find objects permanently removed or somehow broken. We'll try our best to not let that happen. That's the best I can do.

    I understand your frustration, and we will never intentionally break in-world content.

    ReplyDelete
  10. Another comment.

    The secondlife server is written in C++. OpenSim is a completely reverse engineered product. It does not share code with the secondlife server and is not affiliated with linden labs.

    ReplyDelete
  11. Thanks for the clarification David. I am speaking as someone new to the whole idea so good to know.

    Tiger, I know you care deeply for IW;hence making this blog. I was simply responding and discussing. Keep up the good work:)

    ReplyDelete
  12. Thank you Dave for correcting me on several points (too many to actually list). While I try to avoid presenting false information, as with bugs, some leak out. My editor / tester is on holiday but I know that is not a valid excuse. :) With any criticism, be it constructive or destructive, you run the risk of alienating those around you. I am glad mine was taken to be the former, especially by those whom have a vested interest.

    I am gonna do a fluff post next after my research is done. I truly believe it is important fluff though some fluff so I guess that makes it a non-fluff blog entry. Goodie. :)

    ReplyDelete
  13. TIGERBEATLE DOING FLUFF?! No way...the world is ending as we know it

    *gets her camera ready for future fluff-bid wars then runs like hell in the opposite direction*

    ReplyDelete
  14. >>"With any criticism, be it constructive or destructive, you run the risk of alienating those around you. I am glad mine was taken to be the former, especially by those whom have a vested interest. "

    Over the years I've found that listening and trying to understand someone, even when you may not agree with everything they're saying is the most important step to learning and moving forward.

    In software if you don't learn this early you become bitter and burn out. I know I am not always right and I accept this not as a flaw, but as an opportunity.

    ReplyDelete