[IRCServices] cheers. ideas.

[Scott Seufert]

ok, ignorant question follows:


What makes DALnet services work well with it's database(s)? 60K users, 100k 
registered nicks easily.

DALnet has splits just like all other large nets, granted, but it is 
usually an IRCd that splits, and not services. Services of course still has 
it's downtime, but as a general rule they are up.

Scott

At 04:10 PM 10/11/2000 -0700, you wrote:
>      Okay, my two cents:
>
>      I'll be the first to admit that the current Services database format
>is less than ideal.  When I originally designed Services, I designed it
>the network I had started a bit earlier, which at the time had at most
>20-30 simultaneous users.  Services can write a database of a few thousand
>nicks quickly enough not to matter, but up that by an order of magnitude
>and you'll start seeing lag just from writing the database.  Moreover, a
>single bit error in the wrong place and you lose the entire database (and
>apologies to anyone who's experienced this).
>
>      There's also the issue of accessing, and particularly altering, the
>databases from outside of Services.  Although access alone isn't really a
>problem--if you want to access the Services databases now, you can do it
>by borrowing the database loading code; listchans and listnicks work like
>that--altering the databases from outside of Services' control brings in
>all the classic problems of multithreaded applications--races, deadlocks,
>that sort of stuff.  Which isn't to say it can't be done, of course, but
>as Andrew Kempe commented, it'll take a good deal of work.  My personal
>thought is that it would be better done as a multi-step process: first
>implement some manner of sending Services certain commands from outside
>(e.g. register/drop a nick), iron out the problems there, then look at
>going to complete external read/write support.
>
>      As far as database format goes, I personally don't like either the
>XML or SQL options which have been presented so far.  While I agree that
>XML would make external viewing/editing of the databases easier and reduce
>the possibility of single-bit crashes, it would place even more strain on
>Services when updating the databases--not only do you have to write all
>the data to disk each time, you have to convert it all to ASCII and you
>have to add all sorts of formatting stuff, meaning both more CPU time and
>more disk space needed (I could easily see the database size tripling or
>worse).  As far as SQL goes, relying on an external program to be running
>in order to write data is just ugly; besides which, the delay in sending
>commands to the SQL server to read/write data would almost certainly be
>untenable.  (A 30-millisecond delay on an update request, for example, is
>fine if the requests are spread out among lots of clients; it's not fine
>if you have one program trying to do 50 updates a second.)  Besides which,
>there are probably a lot of people, myself included, who wouldn't want to
>use Services if it required installing an SQL server as well.
>
>      I personally see no reason why a proprietary format wouldn't work
>fine, as long as a library or some such was provided to allow other
>programs to access it.  In fact, given that the most important thing for
>Services is to be able to respond quickly to user requests, I think this
>is the best solution.  There really aren't any similar systems you can
>compare Services to (at least, none come to mind); if you want to talk
>about SQL, I would almost expect Services to act as an SQL _server_, not
>a client.
>
>      Actually, something like that may end up being a good solution after
>all.  There's no reason why external programs have to access the data
>_files_ for Services, just like database clients typically don't directly
>access the database server's data files; you can just have Services
>respond to requests through some channel--e.g. a control socket--and
>allow data access through that channel, like a database server.  The only
>need to access the files directly would be if they got corrupted, and
>with a good file structure design such occurrences (and the damage they
>caused) could be reduced to almost nil.
>
>      Comments?
>
>   --Andrew Church
>     achurch at dragonfire.net
>     http://achurch.dragonfire.net/
>
>---------------------------------------------------------------
>To unsubscribe, send email to majordomo at ender.shadowfire.org
>with "unsubscribe ircservices" in the body, without the quotes.


---------------------------------------------------------------
To unsubscribe, send email to majordomo at ender.shadowfire.org
with "unsubscribe ircservices" in the body, without the quotes.