IRC Services Manual

5. Importing and exporting databases

5-1. Exporting Services databases in XML format
5-2. Importing (merging) data from an XML file
5-3. Converting data from other Services-like programs
    5-3-1. Using the convert-db program
    5-3-2. Notes on converting databases from particular programs

Previous section: Services command reference | Table of Contents

5-1. Exporting Services databases in XML format

In order to facilitate processing of the Services databases by other programs, Services provides the ability to export the databases in XML format. This is accomplished through the misc/xml-export module and, optionally, the HTTP server built into Services.

The misc/xml-export module adds an extra command-line option, -export, to Services. Since the command line is parsed before Services connects to the network, the module must be specified in your ircservices.conf file to enable this option. The option has the format "-export" or "-export=filename", where filename is the name of the file to write the XML data to; if it does not begin with a slash, it is taken to be relative to the Services data directory (the directory containing the database and log files). If filename is not given or is a single hyphen ("-"), the data will be printed on standard output. When the export completes, Services will terminate without connecting to the IRC network. This action can be performed while another copy of Services is running, but the output will not include changes made online since the last time the databases were saved to disk. (It is also possible for the running copy of Services to report a "data directory locked" error if it tries to save the databases while the export operation is in progress. This is harmless; you can force the databases to be written after the export completes by using the OperServ UPDATE command, but do not use the FORCE option while the export is still running as this may result in corrupted database files.)

Additionally, if the misc/xml-export module is loaded into Services, the HTTP database access module (httpd/dbaccess) will display an "XML database download" link. Clicking on the link will display the XML data in your browser, which you can then save (for Firefox and Internet Explorer, select "Save as..." or "Save page as..." from the browser's File menu). Alternatively, you can download the link directly without displaying it in your browser; in Firefox, hold the Alt key while clicking on the link, and in Internet Explorer, right-click on the link and select "Save target as..." from the pop-up menu.

The format of the exported XML data is described in Appendix B.

Notes: Services will not respond to any network activity while you are downloading the database. If you have a large database, this can make Services appear to "freeze" with respect to the IRC network, and it may even get disconnected from the network in extreme cases. Additionally, be aware that the data may change after you have downloaded it; if you want to avoid this, for example when exporting data to move it to another machine or network, use the command-line option -export (after shutting Services down) rather than the HTTP server interface.

Note to Internet Explorer users: Some versions of Internet Explorer have a bug which can prevent the XML data from being displayed or downloaded properly. If you have this problem, use another browser, or add the following key to your registry using the Registry Editor (regedit), as described in Microsoft Knowledge Base article Q239750 [support.microsoft.com]:

Back to top

5-2. Importing (merging) data from an XML file

Just as it can export data in XML format, Services can also read in XML data and import (merge) it into its databases. Importing is handled by the misc/xml-import module. Unlike exporting, however, importing cannot be done via the HTTP server; it is always done through the command line. For this reason, the misc/xml-import module must be loaded at startup if it is to be used, rather than via an OperServ REHASH or a SIGHUP signal.

To import a set of data stored in XML format into Services, start Services with the command-line option "-import=filename", where filename is the name of the file containing the data; if the filename does not begin with a slash, it is taken to be relative to the Services data directory. Services will parse the data file, and if no errors are found, the data will be merged into Services' databases and the program will terminate. If problems are found, they will be printed on standard output and the databases will not be modified.

Normally, if nickname or channel "collisions" occur—that is, if a nickname or channel in the input data is already registered in Services' databases—Services will skip over that nickname group or channel, respectively. (For nicknames, the entire nickname group is skipped to prevent cases where, for example, a user's secondary nick is available but the primary nick is registered to someone else.) This behavior can be modified by the OnNicknameCollision and OnChannelCollision options in modules.conf. Additionally, Services can be made to output a list of all data imported by setting the VerboseImport option (which is set in the example modules.conf provided with Services).

Note that the maximum user count and OperServ SU password will not be imported; news items will not be imported unless the ImportNews option (in modules.conf) is set. Also, any autokills, autokill exclusions, session limit exceptions, or S-lines which collide with those in the database will be automatically skipped.

Back to top

5-3. Converting data from other Services-like programs

5-3-1. Using the convert-db program

For users of other Services-like programs who want to switch to IRC Services, a program called convert-db is supplied to convert databases used by such programs into XML data files, which can then be imported into Services as described in section 5-2 above. This program is installed into the Services data directory (/var/opt/ircservices for binary distributions, /usr/local/lib/ircservices by default for source distributions).

In its simplest form, convert-db takes the pathname of the directory containing the database files to convert on the command line, and writes the converted XML data to standard output. A typical invocation might look like this:

convert-db /usr/local/lib/someprogram > someprogram.xml

Note that not all features of all programs are supported by Services, and some settings may be lost when you import data. See section 5-3-2 below for notes on particular programs, and section 5-2 for information on importing the converted data into Services.

The full syntax for convert-db is as follows:

convert-db [-v] [+program-name] [options . . . ] pathname

• -v: Display progress information as the data is converted.
• +program-name: Explicitly indicates what program was used to create the data files. A list of possible program-name values is given in table 5-1 below. program-name values are not case sensitive.
• options: Additional options specific to each type of data file controlling details of the data conversion; see below.
• pathname: Specifies the pathname of the directory containing the database files.

Note that the following programs and versions are not supported, because they do not store data in standard disk files:

• Anope 1.7.11 and later, when using MySQL databases (a standard file format is still available as of version 1.7.19)
• PTlink Services 3.0.0 and later

Additional options are available for the following data file types:

anope

• -ircd=protocol: Selects the protocol to use for interpreting channel modes. By default, only basic channel modes (+i, +k, +l, +m, +n, +p, +s, and +t) will be imported for channel mode locks; to import extended modes provided by particular specify the corresponding protocol from the following list (for Anope 1.7.6 and later, use the same name as in the IRCDModule configuration directive):
  • bahamut for Bahamut
  • dreamforge for Dreamforge
  • hybrid for IRCD-Hybrid
  • ptlink for PTlink IRCd
  • ultimate2 for UltimateIRCD 2.x
  • ultimate3 for UltimateIRCD 3.x
  • unreal for UnrealIRCd
  • viagra for Viagra IRCd
• -encryption=type: Selects the encryption type for databases from Anope 1.7.18 and later. These versions of Anope do not record the encryption type used with passwords stored in the databases; in order to use such passwords, you must specify the proper encryption type with this option:
  • none: No encryption (enc_none module)
  • md5: MD5 encryption (enc_md5, enc_old modules)
  • sha1: SHA-1 encryption (enc_sha1 module)—not supported by Services (passwords will be converted correctly, but will not be usable without a third-party module)
  Note: If you use this option and your database contains a mix of encrypted and unencrypted passwords from Anope 1.7.17 or earlier, some of the passwords will not work.

cygnus

• -tzfile=filename: Specifies an alternative time zone file to use when loading nickname time zone data. convert-db defaults to the data from the cygnus.zone file distributed with Cygnus 0.2.0; if you have modified this file or created your own time zone list, you will need to use this option to get nickname time zones converted correctly.
• -no-timezones: Causes all nickname time zone settings to be reset to default (the time zone Services runs in).
• -reset-memo-limits: Causes all nickname memo limits to be reset to default; see the notes on Cygnus database conversion.

hybserv

• -crypt: Indicates that all passwords are encrypted. Without this option, all passwords are assumed to be in plain text.

Back to top

5-3-2. Notes on converting databases from particular programs

Note that this is not a comprehensive list of differences; only the major differences and types of data that cannot be converted are listed.

All programs

• The setter and reason for a forbidden nickname or channel will not be imported.
• Channel privilege level settings (the equivalent of the ChanServ LEVELS command), on programs that support them, will not be imported; all channels' privilege levels will be reset to the default values.
• Services does not record the setter and last used time for channel access entries; this data will be lost on conversion for programs that do record it.
• Services does not support "bots" or a "BotServ", and all such data will be lost on conversion.

Anope

• For version 1.7.11 and later, databases must be stored using the standard ("FFF") format; MySQL databases are not supported.
• Anope versions through 1.7.17 contain the MD5 encryption bug present in versions of IRC Services before 4.5; as a result, encrypted passwords will not work by default. As a workaround, you can enable the EnableAnopeWorkaround configuration option for the encryption/md5 module in modules.conf, but be aware that this may reduce the security of your passwords. There are also rare cases in which passwords may be truncated in the databases; if you encounter such a case, the password will have to be changed manually by a Services administrator.
• Services does not store the following information; it will be lost when the databases are converted:
  • ICQ number for nicknames
  • Last used time for channel access entries
• The NickServ SET AUTOOP OFF option present in Anope 1.7.15 and later is converted to SET NOOP ON; however, the behavior in of NOOP in Services is slightly different, as it only prevents the nickname from being added to channel access lists (the nickname will still be auto-opped if it already has that privilege in a channel).
• Services does not allow multiple users to have Services super-user privileges (except temporarily with the SU command); all users with "root" privileges in Anope will be converted to Services administrators.
• Services does not distinguish between using the ChanServ OP, VOICE, and related commands on oneself or on other users; both operations use the same privilege level.
• Services does not support the ChanServ SECUREFOUNDER, SIGNKICK, or XOP settings. (XOP-style commands can be enabled for all channels by loading the chanserv/access-xop module.)
• Nickname requests (nicknames which have been registered but not confirmed via E-mail) are discarded; users with pending nickname requests will need to re-register.

Auspice

• The NickServ NOOP option behaves slightly differently in Services: it prevents the nickname from being added to channel access lists, but the nickname will still be auto-opped if it already has that privilege in a channel.
• The ChanServ TIMEOP feature in version 2.6 and later is not supported. Nicknames on a channel's TIMEOP list will not be added to the channel's access list for Services, and thus will have no privileges on the channel.
• Services does not support "root" and "master" privilege levels as in Auspice. There can only be one root nickname, which is set in modules.conf; all root/master settings in the data files will be discarded. (Admin/oper settings will be retained.) Note that Services admins can perform any function (except modifying the admin list), including modifying Services settings and shutting down Services.
• NickServ AUTH, COMMENT, NOTE, LOCK, MARK, SET KILL HIGH, SETFLAG AUTOJOIN, and SETMLOCK, as well as ChanServ BADWORDS, FREEZE, and MARK, are not supported. Nickname notes will be converted to non-expiring memos with the user themselves as the sender; other data will be lost.
• Services uses a different news system than Auspice; network and channel news items will not be converted.
• Aside from the above, Services does not store the following information; it will be lost when the databases are converted:
  • UIN, age, and gender for nicknames
  • Last GETPASS user and HOLD setter and reason for channels
• E-mailing of log files is not supported.

Bolivia

• The NickServ NOOP option behaves slightly differently in Services: it prevents the nickname from being added to channel access lists, but the nickname will still be auto-opped if it already has that privilege in a channel.
• Limiting channels to privileged users (the LEVEL command) is not supported, and such restrictions will be discarded when converting the databases. However, some IRC servers (such as Unreal) have channel modes allowing channels to be limited to IRC operators or administrators only, and Services allows those modes to be locked on with the SET MLOCK command.
• ChanServ FREEZE is not supported.
• ChanServ HOLD setter will be lost when converting.
• E-mailing of log files is not supported.

Cygnus

• Services does not store the UIN, real name, age, sex, or location for nicknames; this information will be lost when the databases are converted.
• Services does not send replies using PRIVMSG; the NOTICE and PRIVMSG nickname settings will be ignored.
• The following options/features are not supported:
  • NickServ NEVEROP and NOSUCCESSOR
  • ChanServ LIMITED
  • MemoServ RECEIPTS
  • RootServ MARK
• Nicknames designated as CSOPs will be converted to Services operators. Note that the privileges granted by IRC Services to Services operators are different from those granted by Cygnus to CSOPs.
• Nicknames will retain their memo limit settings in the converted data; however, they will not be affected by changes to the MSMaxMemos setting. (Memo limits can be reset to the default, which follows the MSMaxMemos setting, with the -reset-memo-limits option.)
• If you have modified the time zone list used by Cygnus (cygnus.zone), nickname time zones will not be transferred correctly by default. Use the -tzfile= option to ensure that time zones are loaded from your time zone file, or use the -no-timezones option to reset all time zone settings to default.
• Authentication for channels is not supported; authentication codes and "verify" information will be discarded.
• Channel access list entries in IRC Services must always be registered nicknames; user@host entries, and any entries not matching a registered nickname, will be deleted.
• While the channel topic-lock setting will be retained, the lock level (founder/SOP/AOP/HOP/VOP) will be reset to the IRC Services default (AOP). If you use the LEVELS command (in the chanserv/access-levels submodule), you can change the level of the TOPIC command to set which users are allowed to change the topic.
• While not directly related to the databases, note that Services uses the AUTH command, not the REGISTER or SET EMAIL commands, to authenticate nicknames.

Daylight

• Services does not support the XMANAGEMENT channel setting.

Epona

• Epona contains the MD5 encryption bug present in versions of IRC Services before 4.5; as a result, encrypted passwords will not work by default. As a workaround, you can enable the EnableAnopeWorkaround configuration option for the encryption/md5 module in modules.conf, but be aware that this may reduce the security of your passwords. There are also rare cases in which passwords may be truncated in the databases; if you encounter such a case, the password will have to be changed manually by a Services administrator.
• Services does not store the following information; it will be lost when the databases are converted:
  • ICQ number for nicknames
  • Last used time for channel access entries
• Services does not distinguish between using the ChanServ OP, VOICE, and related commands on oneself or on other users.
• By default, only basic channel modes (+i, +k, +l, +m, +n, +p, +s, and +t) will be imported for channel mode locks. To import extended modes provided by particular IRC servers, use one of the following command-line options:
  • -ircd=bahamut for Bahamut
  • -ircd=ultimate2 for UltimateIRCD
  • -ircd=unreal for UnrealIRCd

HybServ

• Services does not store GSM, phone, or ICQ number for nicknames; this information will be lost when the databases are converted.
• Services does not have the concept of "permanent passwords", as in HybServ 1.6.1+UniBG, and such passwords will be discarded on import.
• Services does not support the NickServ AUTOMASK, HIDE URL, NOREGISTER, or NOCHANOPS settings, or the ChanServ GUARD or VERBOSE settings.
• The NickServ NOOP option behaves slightly differently in Services: it prevents the nickname from being added to channel access lists, but the nickname will still be auto-opped if it already has that privilege in a channel.
• The NickServ HIDE ALL setting is treated as a combination of HIDE EMAIL, HIDE USERMASK, and HIDE QUIT. The nickname URL and options cannot be hidden.
• Temporary forbids are imported as permanent (Services does not support the notion of a "temporary" or "timed" forbid).
• "Forgotten" channels are not supported, and will not be imported.
• Services does not have a system of "operator modes" like HybServ, and no operator mode information will be imported.
• Statistics information (maximum number of users/channels/etc.) will not be imported.
• The founder of a channel will be removed from the channel's access list (founders always have full channel privileges and do not need to be added to the access list).
• Channel privilege levels will be reset to the Services defaults on import. Channel access levels will also be modified to fit within the -999 . . . 999 range used by Services, as follows:

  HybServ level	Services level
  -10000	-999
  -9999 . . . -10	-999 . . . -1
  -9 . . . -1	-1
  0 . . . 5	0 . . . 30
  5 . . . 10	30 . . . 50
  10 . . . 15	50 . . . 100
  15 . . . 20	100 . . . 110
  20 . . . 200	110 . . . 200
  200 . . . 1000	200 . . . 300
  1000 . . . 9000	300 . . . 900
  9000 . . . 9999	900 . . . 999
  10000	999

  WARNING: As a result of differences in the default level settings between HybServ and Services, some users will have access to commands which they did not previously have access to. In particular, the ACCESS command is available to all users with access level 40 (HybServ level 8) or above, and the CLEAR command is available to all users with access level 100 (HybServ level 15) or above. Conversely, the INVITE command is available only to users with access level 50 (HybServ level 10) or above; auto-voice users cannot use it.

IRCS

• By default, IRCS supports channels with names longer than 63 characters (including the leading "#"). Services does not support these by default, and such channels will not be imported.
• Channel autoop/autovoice cannot be set for individual nicknames. All nicks with appropriate access will be opped even if ChanServ SET AUTOOP OFF was used for the nick.
• The numeric access level for SOP is different from that used in Services, so some access levels will be altered to match the default SOP level in Services. (Channel privileges will remain unchanged.)
• The "senior", "junior", and "helper" OperServ access levels do not exist in Services. Users on the "senior" list will be moved to the Services admin list; users on other lists will be moved to the Services operator list. WARNING: As a result of the above, users on the "senior" list will be given access to the JUPE and LISTIGNORE commands.

Magick I 1.4

• Services does not send replies using PRIVMSG; the NI_PRIVMSG nickname setting will be ignored.
• Services does not have a "channel news" system like Magick, and all channel news items will be discarded.

PTlink

• PTlink supports several encryption methods; of these, the "JP2" method is not supported by Services, and any passwords encrypted with this method will be reset to the respective nick or channel name.
• Services does not store the following information; it will be lost when the databases are converted:
  • ICQ number, location, birthdate, time of last identify, and authentication code (2.22.0 and later) for nicknames
  • maximum number of users and time of maximum for channels
• An apparent bug in PTlink causes the successor field of some channels to be corrupted; this will result in warnings when loading the channel database.
• What PTlink refers to as "SXlines" are known as "SGlines" in Services; SXline database records (in versions of PTlink that support it) will be converted to SGline records.

SirvNET

• The NickServ NOOP option behaves slightly differently in Services: it prevents the nickname from being added to channel access lists, but the nickname will still be auto-opped if it already has that privilege in a channel.
• Limiting channels to priviliged users (the LEVEL command) is not supported, and such restrictions will be discarded when converting the databases. However, some IRC servers (such as Unreal) have channel modes allowing channels to be limited to IRC operators or administrators only, and Services allows those modes to be locked on with the SET MLOCK command.
• NickServ AUTH and ChanServ FREEZE are not supported.
• ChanServ HOLD setter and reason will be lost when converting.
• E-mailing of log files is not supported.

trircd

• SET EXCEPTION, SET INVITES, HIDE OPTIONS, and HIDE DESC channel settings are not supported.

Wrecked 1.2.0

• Services does not send replies using PRIVMSG; the NI_PRIVMSG nickname setting will be ignored.
• The VOPALL channel setting is not supported.

Back to top

Previous section: Services command reference | Table of Contents