[IRCServices] cheers. ideas.

[Andrew Church]

     Okay, my two cents:

     I'll be the first to admit that the current Services database format
is less than ideal.  When I originally designed Services, I designed it
the network I had started a bit earlier, which at the time had at most
20-30 simultaneous users.  Services can write a database of a few thousand
nicks quickly enough not to matter, but up that by an order of magnitude
and you'll start seeing lag just from writing the database.  Moreover, a
single bit error in the wrong place and you lose the entire database (and
apologies to anyone who's experienced this).

     There's also the issue of accessing, and particularly altering, the
databases from outside of Services.  Although access alone isn't really a
problem--if you want to access the Services databases now, you can do it
by borrowing the database loading code; listchans and listnicks work like
that--altering the databases from outside of Services' control brings in
all the classic problems of multithreaded applications--races, deadlocks,
that sort of stuff.  Which isn't to say it can't be done, of course, but
as Andrew Kempe commented, it'll take a good deal of work.  My personal
thought is that it would be better done as a multi-step process: first
implement some manner of sending Services certain commands from outside
(e.g. register/drop a nick), iron out the problems there, then look at
going to complete external read/write support.

     As far as database format goes, I personally don't like either the
XML or SQL options which have been presented so far.  While I agree that
XML would make external viewing/editing of the databases easier and reduce
the possibility of single-bit crashes, it would place even more strain on
Services when updating the databases--not only do you have to write all
the data to disk each time, you have to convert it all to ASCII and you
have to add all sorts of formatting stuff, meaning both more CPU time and
more disk space needed (I could easily see the database size tripling or
worse).  As far as SQL goes, relying on an external program to be running
in order to write data is just ugly; besides which, the delay in sending
commands to the SQL server to read/write data would almost certainly be
untenable.  (A 30-millisecond delay on an update request, for example, is
fine if the requests are spread out among lots of clients; it's not fine
if you have one program trying to do 50 updates a second.)  Besides which,
there are probably a lot of people, myself included, who wouldn't want to
use Services if it required installing an SQL server as well.

     I personally see no reason why a proprietary format wouldn't work
fine, as long as a library or some such was provided to allow other
programs to access it.  In fact, given that the most important thing for
Services is to be able to respond quickly to user requests, I think this
is the best solution.  There really aren't any similar systems you can
compare Services to (at least, none come to mind); if you want to talk
about SQL, I would almost expect Services to act as an SQL _server_, not
a client.

     Actually, something like that may end up being a good solution after
all.  There's no reason why external programs have to access the data
_files_ for Services, just like database clients typically don't directly
access the database server's data files; you can just have Services
respond to requests through some channel--e.g. a control socket--and
allow data access through that channel, like a database server.  The only
need to access the files directly would be if they got corrupted, and
with a good file structure design such occurrences (and the damage they
caused) could be reduced to almost nil.

     Comments?

  --Andrew Church
    achurch at dragonfire.net
    http://achurch.dragonfire.net/

---------------------------------------------------------------
To unsubscribe, send email to majordomo at ender.shadowfire.org
with "unsubscribe ircservices" in the body, without the quotes.