[IRCServices] Scalability

[&quot]

> >In my opinion the problem that is being ignored is the ability of
services
> >to act concurrently on data.
>
>      This isn't a "problem" if you had no intention of doing it in the
> first place.  I don't know what Andrew Kempe's plans are, of course, but
> I never intended Services to work on every network around; I figured from
> the start that Services would probably not work on a network the size of
> (e.g.) DALnet, and that was fine with me; that would take a different sort
> of program design, which wouldn't be as fit for the 20-30 user network I
> was designing Services for.  That said, I see no reason not to
> incorporate changes which improve performance on larger networks if they
> have no real disadvantages.

I agree with this. Currently IRC Services is mostly used by networks with
less than 500 users. There are a number that are less than 2000 and only a
few with more than that. My aim is to keep the majority happy while still
trying to allow the huge networks to function.

 > >Pick one and go with it :)  There's no use adding a bunch of bells and
> >whistles [...] if the end result is the same thing you started with --
> >only with code that is harder to understand and update.
>
>      Have you tried looking at the current database load/save code?
> That's one of the uglier pieces of code I've written, and that's even
> after the cleanup I put in around version 4.0.  Rewriting it would be
> an improvement in and of itself, no matter what method was used.

Currently, if one changes existing data structures, it is an absolute
mission to ensure that the existing load code, and pervious "upgrade" code
works correctly.

What I have also been considering is doing incremental saves where only
updated data is saved. This would mean much faster database saves. Each save
would take place to a new file. An external program could then merge those
files with the main database every hour or something. When Services loads
its database at startup, it would first ensure that all merges had taken
place - ensuring a single up-to-date database.

This sounds like a lot of additional work, but when you consider what
percentage of the data actually changes between saves you'll see that it's
actually very small.

*shrug*

Just an idea.

Andrew




---------------------------------------------------------------
To unsubscribe, send email to majordomo at ender.shadowfire.org
with "unsubscribe ircservices" in the body, without the quotes.