aboutsummaryrefslogtreecommitdiffstats
path: root/doc/HACKING
blob: 9d064d588d1dfe36b4ce5098ed96c6c0046f2b85 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
BitlBee post-1.x "architecture"

DISCLAIMER: The messy architecture is being cleaned up. Although lots of
progress was made already, this is still a work in progress, and possibly
parts of this document aren't entirely accurate anymore by the time you
read this.

It's been a while since BitlBee started, as a semi-fork of Gaim (version
0.58 at the time). Some people believe nothing changed, but fortunately,
many things have.

The API is gone for a while already - which wasn't incredibly intrusive,
just a few functions renamed for slightly better consistency, added some
calls and arguments where that seemed useful, etc.

However, up to late in the 1.2 series, the IRC core was still spread accross
several files, mostly irc.c + irc_commands.c and pieces and bits in
nogaim.c. If you're looking for a textbook example of layer violation, start
there.

This was all finally redone. Most of the IRC protocol code was rewritten,
as was most of the glue between that and the IM modules.

The core of BitlBee is now protocols/bee*. Some pieces are still left in
protocols/nogaim*. Most stuff in the "root" directory belongs to the IRC
UI, which should be considered "a" frontend (although currently, and
possibly forever, the only one). Every subdirectory of protocols/ is another
IM protocol backend (including purple/ which uses libpurple to define
many different protocols).


/

The IRC core has code to show an IRC interface to a user, with contacts,
channels, etc. To make channels and contacts do something, you add event
handlers (that translate a message sent to a nick into an instant message
to an IM contact, or translates joining a channel into joining an IM
chatroom).

To get events back from the BitlBee core, the bee_t object has a bunch of
functions (struct bee_ui_funcs) that catch them and convert them back to
IRC.

Short description of what all irc*.c files (and some related ones) do:

bitlbee.c: BitlBee bootstrap code, doing bits of I/O as well.
ipc.c: For inter-process communication - communication between BitlBee
    sessions. Also used in daemon mode (in which it's not so much inter-
    process).
irc.c: The main core, with bits of I/O handling, parsing, etc.
irc_channel.c: Most things related to standard channels (also defines some
    of the control channel behaviour).
irc_commands.c: Defines all IRC commands (JOIN, NICK, PRIVMSG, etc.).
irc_im.c: Most of the glue between IRC and the IM core live here. This is
    where instant messages are converted to IRC and vice versa, contacts
    coming online is translated to one or more joins and/or mode changes.
irc_send.c: Simple functions that send pieces of IRC output. Somewhat
    random, but if an IRC response is slightly more complicated than just a
    simple line, make it a function here.
irc_user.c: Defines all IRC user details. Mostly defines the user "object".
irc_util.c: Misc. stuff. Not much ATM.
nick.c: Handling of nicknames: compare, ucase/lcase, generating and storing
    nicks for IM contacts.
set.c: Settings management, used for user-changeable global/account/channel
    settings. Should really be considered part of the core.
storage*.c: Storing user accounts. (The stuff you normally find in
    /var/lib/bitlbee)


/protocols

The IM core lives in protocols/. Whenever you write code there, try to avoid
using any IRCisms there.

Most header files in there have some of their details explained in comments.
bee*.c and nogaim.c are the layer between the IM modules and the IRC
frontend. They keep track of IM accounts, contacts and their status,
groupchats, etc.

You can control them by calling functions in there if available, and
otherwise by just calling the functions exported via the prpl struct. Most
of these functions are briefly explained in the header files, otherwise the
best documentation is sadly in irc_im.c and root_commands.c.

Events from the IM module go back to the core + frontend via imcb_*
functions defined in bee*.c and nogaim.c. They're all described in the
header files.


/lib

BitlBee uses GLib, which is a pretty nifty library adding a bunch of things
that make life of a C coder better. Please try to not use features from
recent GLib versions as whenever this happens, some people get cranky. :>

There's also a whole bunch of nice stuff in lib/ that you can use:

arc.c: ARC4 encryption, mostly used for encrypting IM passwords in the XML
    storage module.
base64.c
events_*.c: Event handling, using either GLib (default) or libevent (may
    make non-forking daemon mode with many users a little bit more efficient).
ftutil.c: Some small utility functions currently just used for file transfers.
http_client.c: A simple (but asynchronous) HTTP(S) client, used by the MSN,
    Yahoo! and Twitter module by now.
ini.c: Simple INI file parser, used to parse bitlbee.conf.
md5.c
misc.c: What the name says, really.
oauth.c: What the name says. If you don't know what OAuth is, ask Google.
    Currently only used by the Twitter module. Implements just version 1a ATM.
proxy.c: Used together with events_*.c for asynchronous I/O directly or via
    proxies.
sha1.c
ssl_*.c: SSL client stuff, using GnuTLS (preferred) or OpenSSL. Other modules
    aren't working well ATM.
url.c: URL parser.
xmltree.c: Uses the GLib stream parser to build XML parse trees from a stream
    and convert the same structs back into XML. Good enough to do Jabber but
    not very aware of stuff like XML namespaces. Used for Jabber and Twitter.

This, together with the headerfile comments, is most of the documentation we
have right now. If you're trying to make some changes to BitlBee, do feel
free to join #bitlbee on irc.oftc.net to ask any question you have. 
Suggestions for specific parts to document a little bit more are also
welcome.
458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549
/***************************************************************************\
*                                                                           *
*  BitlBee - An IRC to IM gateway                                           *
*  Jabber module - I/O stuff (plain, SSL), queues, etc                      *
*                                                                           *
*  Copyright 2006 Wilmer van der Gaast <wilmer@gaast.net>                   *
*                                                                           *
*  This program is free software; you can redistribute it and/or modify     *
*  it under the terms of the GNU General Public License as published by     *
*  the Free Software Foundation; either version 2 of the License, or        *
*  (at your option) any later version.                                      *
*                                                                           *
*  This program is distributed in the hope that it will be useful,          *
*  but WITHOUT ANY WARRANTY; without even the implied warranty of           *
*  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the            *
*  GNU General Public License for more details.                             *
*                                                                           *
*  You should have received a copy of the GNU General Public License along  *
*  with this program; if not, write to the Free Software Foundation, Inc.,  *
*  51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.              *
*                                                                           *
\***************************************************************************/

#include "jabber.h"
#include "ssl_client.h"

static gboolean jabber_write_callback( gpointer data, gint fd, b_input_condition cond );
static gboolean jabber_write_queue( struct im_connection *ic );

int jabber_write_packet( struct im_connection *ic, struct xt_node *node )
{
	char *buf;
	int st;
	
	buf = xt_to_string( node );
	st = jabber_write( ic, buf, strlen( buf ) );
	g_free( buf );
	
	return st;
}

int jabber_write( struct im_connection *ic, char *buf, int len )
{
	struct jabber_data *jd = ic->proto_data;
	gboolean ret;
	
	if( jd->flags & JFLAG_XMLCONSOLE && !( ic->flags & OPT_LOGGING_OUT ) )
	{
		char *msg, *s;
		
		msg = g_strdup_printf( "TX: %s", buf );
		/* Don't include auth info in XML logs. */
		if( strncmp( msg, "TX: <auth ", 10 ) == 0 && ( s = strchr( msg, '>' ) ) )
		{
			s++;
			while( *s && *s != '<' )
				*(s++) = '*';
		}
		imcb_buddy_msg( ic, JABBER_XMLCONSOLE_HANDLE, msg, 0, 0 );
		g_free( msg );
	}
	
	if( jd->tx_len == 0 )
	{
		/* If the queue is empty, allocate a new buffer. */
		jd->tx_len = len;
		jd->txq = g_memdup( buf, len );
		
		/* Try if we can write it immediately so we don't have to do
		   it via the event handler. If not, add the handler. (In
		   most cases it probably won't be necessary.) */
		if( ( ret = jabber_write_queue( ic ) ) && jd->tx_len > 0 )
			jd->w_inpa = b_input_add( jd->fd, B_EV_IO_WRITE, jabber_write_callback, ic );
	}
	else
	{
		/* Just add it to the buffer if it's already filled. The
		   event handler is already set. */
		jd->txq = g_renew( char, jd->txq, jd->tx_len + len );
		memcpy( jd->txq + jd->tx_len, buf, len );
		jd->tx_len += len;
		
		/* The return value for write() doesn't necessarily mean
		   that everything got sent, it mainly means that the
		   connection (officially) still exists and can still
		   be accessed without hitting SIGSEGV. IOW: */
		ret = TRUE;
	}
	
	return ret;
}

/* Splitting up in two separate functions: One to use as a callback and one
   to use in the function above to escape from having to wait for the event
   handler to call us, if possible.
   
   Two different functions are necessary because of the return values: The
   callback should only return TRUE if the write was successful AND if the
   buffer is not empty yet (ie. if the handler has to be called again when
   the socket is ready for more data). */
static gboolean jabber_write_callback( gpointer data, gint fd, b_input_condition cond )
{
	struct jabber_data *jd = ((struct im_connection *)data)->proto_data;
	
	return jd->fd != -1 &&
	       jabber_write_queue( data ) &&
	       jd->tx_len > 0;
}

static gboolean jabber_write_queue( struct im_connection *ic )
{
	struct jabber_data *jd = ic->proto_data;
	int st;
	
	if( jd->ssl )
		st = ssl_write( jd->ssl, jd->txq, jd->tx_len );
	else
		st = write( jd->fd, jd->txq, jd->tx_len );
	
	if( st == jd->tx_len )
	{
		/* We wrote everything, clear the buffer. */
		g_free( jd->txq );
		jd->txq = NULL;
		jd->tx_len = 0;
		
		return TRUE;
	}
	else if( st == 0 || ( st < 0 && !ssl_sockerr_again( jd->ssl ) ) )
	{
		/* Set fd to -1 to make sure we won't write to it anymore. */
		closesocket( jd->fd );	/* Shouldn't be necessary after errors? */
		jd->fd = -1;
		
		imcb_error( ic, "Short write() to server" );
		imc_logout( ic, TRUE );
		return FALSE;
	}
	else if( st > 0 )
	{
		char *s;
		
		s = g_memdup( jd->txq + st, jd->tx_len - st );
		jd->tx_len -= st;
		g_free( jd->txq );
		jd->txq = s;
		
		return TRUE;
	}
	else
	{
		/* Just in case we had EINPROGRESS/EAGAIN: */
		
		return TRUE;
	}
}

static gboolean jabber_read_callback( gpointer data, gint fd, b_input_condition cond )
{
	struct im_connection *ic = data;
	struct jabber_data *jd = ic->proto_data;
	char buf[512];
	int st;
	
	if( jd->fd == -1 )
		return FALSE;
	
	if( jd->ssl )
		st = ssl_read( jd->ssl, buf, sizeof( buf ) );
	else
		st = read( jd->fd, buf, sizeof( buf ) );
	
	if( st > 0 )
	{
		/* Parse. */
		if( xt_feed( jd->xt, buf, st ) < 0 )
		{
			imcb_error( ic, "XML stream error" );
			imc_logout( ic, TRUE );
			return FALSE;
		}
		
		/* Execute all handlers. */
		if( !xt_handle( jd->xt, NULL, 1 ) )
		{
			/* Don't do anything, the handlers should have
			   aborted the connection already. */
			return FALSE;
		}
		
		if( jd->flags & JFLAG_STREAM_RESTART )
		{
			jd->flags &= ~JFLAG_STREAM_RESTART;
			jabber_start_stream( ic );
		}
		
		/* Garbage collection. */
		xt_cleanup( jd->xt, NULL, 1 );
		
		/* This is a bit hackish, unfortunately. Although xmltree
		   has nifty event handler stuff, it only calls handlers
		   when nodes are complete. Since the server should only
		   send an opening <stream:stream> tag, we have to check
		   this by hand. :-( */
		if( !( jd->flags & JFLAG_STREAM_STARTED ) && jd->xt && jd->xt->root )
		{
			if( g_strcasecmp( jd->xt->root->name, "stream:stream" ) == 0 )
			{
				jd->flags |= JFLAG_STREAM_STARTED;
				
				/* If there's no version attribute, assume
				   this is an old server that can't do SASL
				   authentication. */
				if( !set_getbool( &ic->acc->set, "sasl") || !sasl_supported( ic ) )
				{
					/* If there's no version= tag, we suppose
					   this server does NOT implement: XMPP 1.0,
					   SASL and TLS. */
					if( set_getbool( &ic->acc->set, "tls" ) )
					{
						imcb_error( ic, "TLS is turned on for this "
						          "account, but is not supported by this server" );
						imc_logout( ic, FALSE );
						return FALSE;
					}
					else
					{
						return jabber_init_iq_auth( ic );
					}
				}
			}
			else
			{
				imcb_error( ic, "XML stream error" );
				imc_logout( ic, TRUE );
				return FALSE;
			}
		}
	}
	else if( st == 0 || ( st < 0 && !ssl_sockerr_again( jd->ssl ) ) )
	{
		closesocket( jd->fd );
		jd->fd = -1;
		
		imcb_error( ic, "Error while reading from server" );
		imc_logout( ic, TRUE );
		return FALSE;
	}
	
	if( ssl_pending( jd->ssl ) )
		/* OpenSSL empties the TCP buffers completely but may keep some
		   data in its internap buffers. select() won't see that, but
		   ssl_pending() does. */
		return jabber_read_callback( data, fd, cond );
	else
		return TRUE;
}

gboolean jabber_connected_plain( gpointer data, gint source, b_input_condition cond )
{
	struct im_connection *ic = data;
	
	if( g_slist_find( jabber_connections, ic ) == NULL )
		return FALSE;
	
	if( source == -1 )
	{
		imcb_error( ic, "Could not connect to server" );
		imc_logout( ic, TRUE );
		return FALSE;
	}
	
	imcb_log( ic, "Connected to server, logging in" );
	
	return jabber_start_stream( ic );
}

gboolean jabber_connected_ssl( gpointer data, void *source, b_input_condition cond )
{
	struct im_connection *ic = data;
	struct jabber_data *jd;
	
	if( g_slist_find( jabber_connections, ic ) == NULL )
		return FALSE;
	
	jd = ic->proto_data;
	
	if( source == NULL )
	{
		/* The SSL connection will be cleaned up by the SSL lib
		   already, set it to NULL here to prevent a double cleanup: */
		jd->ssl = NULL;
		
		imcb_error( ic, "Could not connect to server" );
		imc_logout( ic, TRUE );
		return FALSE;
	}
	
	imcb_log( ic, "Connected to server, logging in" );
	
	return jabber_start_stream( ic );
}

static xt_status jabber_end_of_stream( struct xt_node *node, gpointer data )
{
	imc_logout( data, TRUE );
	return XT_ABORT;
}

static xt_status jabber_pkt_features( struct xt_node *node, gpointer data )
{
	struct im_connection *ic = data;
	struct jabber_data *jd = ic->proto_data;
	struct xt_node *c, *reply;
	int trytls;
	
	trytls = g_strcasecmp( set_getstr( &ic->acc->set, "tls" ), "try" ) == 0;
	c = xt_find_node( node->children, "starttls" );
	if( c && !jd->ssl )
	{
		/* If the server advertises the STARTTLS feature and if we're
		   not in a secure connection already: */
		
		c = xt_find_node( c->children, "required" );
		
		if( c && ( !trytls && !set_getbool( &ic->acc->set, "tls" ) ) )
		{
			imcb_error( ic, "Server requires TLS connections, but TLS is turned off for this account" );
			imc_logout( ic, FALSE );
			
			return XT_ABORT;
		}
		
		/* Only run this if the tls setting is set to true or try: */
		if( ( trytls || set_getbool( &ic->acc->set, "tls" ) ) )
		{
			reply = xt_new_node( "starttls", NULL, NULL );
			xt_add_attr( reply, "xmlns", XMLNS_TLS );
			if( !jabber_write_packet( ic, reply ) )
			{
				xt_free_node( reply );
				return XT_ABORT;
			}
			xt_free_node( reply );
			
			return XT_HANDLED;
		}
	}
	else if( !c && !jd->ssl )
	{
		/* If the server does not advertise the STARTTLS feature and
		   we're not in a secure connection already: (Servers have a
		   habit of not advertising <starttls/> anymore when already
		   using SSL/TLS. */
		
		if( !trytls && set_getbool( &ic->acc->set, "tls" ) )
		{
			imcb_error( ic, "TLS is turned on for this account, but is not supported by this server" );
			imc_logout( ic, FALSE );
			
			return XT_ABORT;
		}
	}
	
	/* This one used to be in jabber_handlers[], but it has to be done
	   from here to make sure the TLS session will be initialized
	   properly before we attempt SASL authentication. */
	if( ( c = xt_find_node( node->children, "mechanisms" ) ) )
	{
		if( sasl_pkt_mechanisms( c, data ) == XT_ABORT )
			return XT_ABORT;
	}
	/* If the server *SEEMS* to support SASL authentication but doesn't
	   support it after all, we should try to do authentication the
	   other way. jabber.com doesn't seem to do SASL while it pretends
	   to be XMPP 1.0 compliant! */
	else if( !( jd->flags & JFLAG_AUTHENTICATED ) && set_getbool( &ic->acc->set, "sasl") && sasl_supported( ic ) )
	{
		if( !jabber_init_iq_auth( ic ) )
			return XT_ABORT;
	}
	
	if( ( c = xt_find_node( node->children, "bind" ) ) )
		jd->flags |= JFLAG_WANT_BIND;
	
	if( ( c = xt_find_node( node->children, "session" ) ) )
		jd->flags |= JFLAG_WANT_SESSION;
	
	if( jd->flags & JFLAG_AUTHENTICATED )
		return jabber_pkt_bind_sess( ic, NULL, NULL );
	
	return XT_HANDLED;
}

static xt_status jabber_pkt_proceed_tls( struct xt_node *node, gpointer data )
{
	struct im_connection *ic = data;
	struct jabber_data *jd = ic->proto_data;
	char *xmlns;
	
	xmlns = xt_find_attr( node, "xmlns" );
	
	/* Just ignore it when it doesn't seem to be TLS-related (is that at
	   all possible??). */
	if( !xmlns || strcmp( xmlns, XMLNS_TLS ) != 0 )
		return XT_HANDLED;
	
	/* We don't want event handlers to touch our TLS session while it's
	   still initializing! */
	b_event_remove( jd->r_inpa );
	if( jd->tx_len > 0 )
	{
		/* Actually the write queue should be empty here, but just
		   to be sure... */
		b_event_remove( jd->w_inpa );
		g_free( jd->txq );
		jd->txq = NULL;
		jd->tx_len = 0;
	}
	jd->w_inpa = jd->r_inpa = 0;
	
	imcb_log( ic, "Converting stream to TLS" );
	
	jd->flags |= JFLAG_STARTTLS_DONE;
	jd->ssl = ssl_starttls( jd->fd, jabber_connected_ssl, ic );
	
	return XT_HANDLED;
}

static xt_status jabber_pkt_stream_error( struct xt_node *node, gpointer data )
{
	struct im_connection *ic = data;
	int allow_reconnect = TRUE;
	struct jabber_error *err;
	
	err = jabber_error_parse( node, XMLNS_STREAM_ERROR );
	
	/* Tssk... */
	if( err->code == NULL )
	{
		imcb_error( ic, "Unknown stream error reported by server" );
		imc_logout( ic, allow_reconnect );
		jabber_error_free( err );
		return XT_ABORT;
	}
	
	/* We know that this is a fatal error. If it's a "conflict" error, we
	   should turn off auto-reconnect to make sure we won't get some nasty
	   infinite loop! */
	if( strcmp( err->code, "conflict" ) == 0 )
	{
		imcb_error( ic, "Account and resource used from a different location" );
		allow_reconnect = FALSE;
	}
	else
	{
		imcb_error( ic, "Stream error: %s%s%s", err->code, err->text ? ": " : "",
		            err->text ? err->text : "" );
	}
	
	jabber_error_free( err );
	imc_logout( ic, allow_reconnect );
	
	return XT_ABORT;
}

static xt_status jabber_xmlconsole( struct xt_node *node, gpointer data )
{
	struct im_connection *ic = data;
	struct jabber_data *jd = ic->proto_data;
	
	if( jd->flags & JFLAG_XMLCONSOLE )
	{
		char *msg, *pkt;
		
		pkt = xt_to_string( node );
		msg = g_strdup_printf( "RX: %s", pkt );
		imcb_buddy_msg( ic, JABBER_XMLCONSOLE_HANDLE, msg, 0, 0 );
		g_free( msg );
		g_free( pkt );
	}
	
	return XT_NEXT;
}

static const struct xt_handler_entry jabber_handlers[] = {
	{ NULL,                 "stream:stream",        jabber_xmlconsole },
	{ "stream:stream",      "<root>",               jabber_end_of_stream },
	{ "message",            "stream:stream",        jabber_pkt_message },
	{ "presence",           "stream:stream",        jabber_pkt_presence },
	{ "iq",                 "stream:stream",        jabber_pkt_iq },
	{ "stream:features",    "stream:stream",        jabber_pkt_features },
	{ "stream:error",       "stream:stream",        jabber_pkt_stream_error },
	{ "proceed",            "stream:stream",        jabber_pkt_proceed_tls },
	{ "challenge",          "stream:stream",        sasl_pkt_challenge },
	{ "success",            "stream:stream",        sasl_pkt_result },
	{ "failure",            "stream:stream",        sasl_pkt_result },
	{ NULL,                 NULL,                   NULL }
};

gboolean jabber_start_stream( struct im_connection *ic )
{
	struct jabber_data *jd = ic->proto_data;
	int st;
	char *greet;
	
	/* We'll start our stream now, so prepare everything to receive one
	   from the server too. */
	xt_free( jd->xt );	/* In case we're RE-starting. */
	jd->xt = xt_new( jabber_handlers, ic );
	
	if( jd->r_inpa <= 0 )
		jd->r_inpa = b_input_add( jd->fd, B_EV_IO_READ, jabber_read_callback, ic );
	
	greet = g_strdup_printf( "%s<stream:stream to=\"%s\" xmlns=\"jabber:client\" "
	                          "xmlns:stream=\"http://etherx.jabber.org/streams\" version=\"1.0\">", 
	                          ( jd->flags & JFLAG_STARTTLS_DONE ) ? "" : "<?xml version='1.0' ?>",
	                          jd->server );
	
	st = jabber_write( ic, greet, strlen( greet ) );
	
	g_free( greet );
	
	return st;
}

void jabber_end_stream( struct im_connection *ic )
{
	struct jabber_data *jd = ic->proto_data;
	
	/* Let's only do this if the queue is currently empty, otherwise it'd
	   take too long anyway. */
	if( jd->tx_len == 0 )
	{
		char eos[] = "</stream:stream>";
		struct xt_node *node;
		int st = 1;
		
		if( ic->flags & OPT_LOGGED_IN )
		{
			node = jabber_make_packet( "presence", "unavailable", NULL, NULL );
			st = jabber_write_packet( ic, node );
			xt_free_node( node );
		}
		
		if( st )
			jabber_write( ic, eos, strlen( eos ) );
	}
}