Author Topic: OmnomIRC Dev Ideas (Read 7676 times)

Netham45 · « **on:** May 10, 2011, 01:21:41 am »

Posting some thoughts I've been having about updating Omnom to be less resource intensive. Just throwing some of this stuff out there if anyone wants to comment.

Don't refresh every 5 seconds or so, but use long polling loop and just return a result when something is said(Would need some sort of client-side parser, but that could be really easy to
do.)

Basically, request update.php, update.php waits with server-side sleep()'s. When something is updated, it finishes update.php and returns the event, something like this:
eventType:base64(message)
Join:base64("Netham45")

the client-side parser would then split it based off of the :, and run Join('Netham45') which would update the textbox, which would print something like 'Netham45 has joined.' or something. Here is what I was thinking for those:

names,messages,modes,targets would be base64_encode'd
line would be the current line, a unique ID for the last line its gotten, for keeping track of where it should check server-side

Code: [Select]

line:join:name
line:join:netham45

line:part:name:message
line:part:netham45:Z8015-190D

line:quit:name:message *Merge quit and part?
line:quit:netham45:Z8015-190D

line:kick:kicker:kicked:message
line:kick:netham45:netbot45:Kick!


line:message:name:message:isOmnom
line:message:netham45:Nom!:1

line:action:name:message
line:action:netham45:explodes


line:mode:modes:targets
line:mode:+ovib:Netham45 Netham45 *@*!*

I was also planning on storing however many lines I'm displaying in an array.

Code: [Select]

Lines = Array();
Lines[0] = "1:message:Netham45:NOM!";
Lines[1] = "2:kick:Netham45:Netbot45:(Netham45)";
...

addLine(message)
{
	var i = 1;
	for (i;i<Lines.count();i++)
	{
		Lines[i-1] = Lines[i];
	}
	Lines[i] = message;
	parseMessages();
}

and to parse, something similar:

Code: (parserish thingey javascript-ish code) [Select]


parseMessages()
{
	for (i = 0;i<Lines.count();i++)
	{
		parse(Lines[i]);
	}
}

parse(message)
{
	chatBox = "";
	parts = message.split(":")
	type = parts[1]
	curLine = lines;
	switch type
	{
		case "join":
			chatBox = chatBox + join(parts[2]);break;
		case "part":
			chatBox = chatBox + part(parts[2],parts[3]);break;
		case "quit":
			chatBox = chatBox + quit(parts[2],parts[3]);break;
		case "kick":
			chatBox = chatBox + kick(parts[2],parts[3],parts[4]);break;
		case "message":	
			chatBox = chatBox + message(parts[2],parts[3]);break;
		case "action":
			chatBox = chatBox + action(parts[2],parts[3]);break;
		case "mode":
			chatBox = chatBox + mode(parts[2],parts[3]);break;
	}
}

join(name)
	return name + " has joined";
part(name,reason)
	return name + " has left #Omnimaga (" + reason + ")";
quit(name,reason)
	return name + " has quit IRC (" + reason + ")";

etc...

Server-side, I plan on having an IRC bot that logs the last 1000 or so lines to a SQL table as they're said. Any more is worthless, half that would still be worthless.

irc_lines = (`line_number`k,ai,`name1`,`name2`,`message`)

Code: (pseudo) [Select]

curPos = sql_query("SELECT MAX(`line_number') FROM `irc_lines`");
sql_query("INSERT INTO `irc_lines` (`action`,`name1`,`name2`,`message`) VALUES ('%s','%s','%s','%s')",curPos,"Join","Netham45","0","0");
sql_query("INSERT INTO `irc_lines` (`action`,`name1`,`name2`,`message`) VALUES ('%s','%s','%s','%s')",curPos,"Kick","Netham45","Netbot45","Stop Lagging you stupid bot!");
sql_query("DELETE FROM `irc_lines` WHERE `line_number` < %s",curPos - 1000);

And to get it from the server, something like this:

Code: (more pseudo) [Select]

getLines()
{
	while (true)
	{
		stuffs = mysql_fetch_array(sql_query("SELECT * FROM `irc_lines` WHERE 'line_number' > %s",reqLines));
		
		for (i=0;i<count(stuffs);i++)
		{
			switch (stuffs['action'])
			{
			case "join:
				echo stuffs['line_number'] . ':' . stuffs['action'] . ':' . stuffs['name1];
			etc...
			}
		}
		if (count(stuffs)) break;
		sleep(50);
	}
}

Gimmeh ideas/suggestions.

ZippyDee · « **Reply #1 on:** May 10, 2011, 01:25:28 am »

I think your "line:kick:kicker:kicked:message" example is missing the "kicked" part...

But I think that would be a great way to go. That would surely lighten the load on omnimaga.

Also, I think part/quit shouldn't be merged.

Netham45 · « **Reply #2 on:** May 10, 2011, 01:27:49 am »

fix'd.

Another thing I'd like to add is that this wouldn't replace the logs, those are best logged to a file as they currently are. This would, though, fix the multi-byte char problems for #omnimaga-fr too.

It'd also be easy to implement a jsonp feed so Eeems would stop raging.

ZippyDee · « **Reply #3 on:** May 10, 2011, 01:29:12 am »

Of course, logs should definitely be stored normally. But fixing the char issue would be GREAT.

DJ Omnimaga · « **Reply #4 on:** May 10, 2011, 03:25:53 pm »

Hmm I don't understand that stuff much, but it's cool to see it will be updated.

As for logs does it means the old logs would no longer be available?

Netham45 · « **Reply #5 on:** May 10, 2011, 04:33:55 pm »

The logs will still be available, but I won't be relying on them for OmnomIRC.

I got most of the client-side parser done, I believe. Just need to add a few things to make it tell between Omnom and Non-Omnom users, cookies, and such.

Code: [Select]

messageList = Array();
userList = Array();
curLine = 0;
messageBox = document.getElementById("MessageBox");
Lines = Array();
//******************************
// Start Request Loop functions
//******************************
	function startLoop()
	{
		xmlhttp=getAjaxObject();
		if (xmlhttp==null) { 
			alert ("Your browser does not support AJAX! Please update for OmnomIRC compatibility."); 
			return;
		}
		xmlhttp.onreadystatechange=getIncomingLine;
		sendRequest();
	}
	
	function sendRequest()
	{
		url = "http://omnom.omnimaga.org/OmnomIRC_Dev/Update.php?lineNum=" + curLine;
		xmlhttp.open("GET",url,true);
		xmlhttp.send(null);
	}
	
	function getIncomingLine()
	{
		if (xmlhttp.readyState==4 || xmlhttp.readyState=="complete") { 
			addLine(xmlhttp.responseText);
			sendRequest();
		}
	}
	
	function getAjaxObject()
	{
		xmlhttp=new XMLHttpRequest(); //Decent Browsers
		if (!xmlhttp || xmlhttp == undefined || xmlhttp == null) xmlhttp=new ActiveXObject("Msxml2.XMLHTTP");  //IE7+
		if (!xmlhttp || xmlhttp == undefined || xmlhttp == null) xmlhttp=new ActiveXObject("Microsoft.XMLHTTP"); //IE6-
		return xmlhttp;
	}
	
//******************************
// End Request Loop functions
//******************************

//******************************
// Start Line adding functions
//******************************
	function addLineStart(message)
	{
		setCurLine(message);
		Lines[Lines.length] = message;
	}

	function addLine(message)
	{
		var i = 1;
		setCurLine(message);
		for (i;i<Lines.length;i++)
		{
			Lines[i-1] = Lines[i];
		}
		Lines[i-1] = message;
		parseMessages();
	}

	function setCurLine(message)
	{
		curLine = message.split(":")[0];
	}
//******************************
// End Line Adding Functions
//******************************

//******************************
// Parser Start
//******************************
	function parseMessages()
	{
		text = "";
		var i = 0;
		for (i = 0;i<Lines.length;i++)
		{
			text = text + parse(Lines[i]);
		}
		messageBox.innerHTML = text;
	}

	function parse(message)
	{
		a = message;
		parts = message.split(":");
		type = parts[1];
		
		for (i = 2;i < parts.length;i++)
			parts[i] = atob(parts[i]); //De-base64 it. ***** WILL NOT WORK IN IE *****
		switch (type)
		{
			case "join":
				return  join(parts[2]);break;
			case "part":
				return  part(parts[2],parts[3]);break;
			case "quit":
				return  quit(parts[2],parts[3]);break;
			case "kick":
				return  kick(parts[2],parts[3],parts[4]);break;
			case "message":	
				return  sendMessage(parts[2],parts[3],parts[4]);break;
			case "action":
				return  action(parts[2],parts[3],parts[4]);break;
			case "mode":
				return  mode(parts[2],parts[3]);break;
		}
		return "";
	}

	function join(name)
	{
		return name + " has joined #omnimaga" + "\n<br/>";
	}
	function part(name,reason)
	{
		return name + " has left #omnimaga (" + reason + ")" + "\n<br/>";
	}
	function quit(name,reason)
	{
		return name + " has quit IRC (" + reason + ")" + "\n<br/>";
	}
	function kick(kicker,kicked,reason)
	{
		return kicker + " has kicked " + kicked + " from #omnimaga (" + reason + ")" + "\n<br/>";
	}
	function sendMessage(name,message,isOmnom)
	{
		return "&lt;" + name + "&gt " + message + "\n<br/>";
	}
	function action(name,action)
	{
		return " * " + name + " " + action + "\n<br/>";
	}
	function mode(name,mode)
	{
		return name + " set #omnimaga mode " + mode + "\n<br/>";
	}
//******************************
// Parser End
//******************************
function load()
{
	var body= document.getElementsByTagName('body')[0];
	var script= document.createElement('script');
	script.type= 'text/javascript';
	script.src= 'Load.php?count=17';
	script.onload= function(){parseMessages();startLoop();};
	body.appendChild(script);
}
window.onLoad = load();

Edit: I have a preview of it available at http://omnom.omnimaga.org/OmnomIRC_Dev/OmnomIRCv2.html (Just spewing randomly generated data now)

DJ Omnimaga · « **Reply #6 on:** May 10, 2011, 04:37:46 pm »

Ah ok, sounds cool. Good luck with this.

Jim Bauwens · « **Reply #7 on:** May 10, 2011, 04:57:46 pm »

Hmm, looks quite interesting.
If I'm not mistaken, the loop keeps a xmlhttprequest open until it timeouts, and then starts a new request (endless loop here) ?

Netham45 · « **Reply #8 on:** May 10, 2011, 06:03:58 pm »

No, it keeps it open until my server responds. It shouldn't time out at all. I have a random-length sleep() on the script that's spewing data for testing.

Jim Bauwens · « **Reply #9 on:** May 10, 2011, 06:05:58 pm »

Oh, ok. Ajax isn't my best subject

DJ Omnimaga · « **Reply #10 on:** May 10, 2011, 06:18:52 pm »

By the way I noticed something with the links at the top: When clicking Francais or the logs, it does nothing, but the second time it works.

Somebody else reported that.

Netham45 · « **Reply #11 on:** May 10, 2011, 08:34:50 pm »

IRC Bot! (Not quite done, as with everything else. Still need to track userlists, and add sending.)

Code: (IRC_Bot.php) [Select]

<?PHP
function sql_query()
{
	global $sqlConnection;
	$params = func_get_args();
	$query = $params[0];
	$args = Array();
	for ($i=1;$i<count($params);$i++)
		$args[$i-1] = htmlspecialchars(mysql_real_escape_string(trim($params[$i]),$sqlConnection));
	$result = mysql_query(vsprintf($query,$args),$sqlConnection);
	if (!$result) 
		die(mysql_error() . "Query: " . vsprintf($query,$args));
	return $result;
}

function getMessage($parts,$start,$trim)
{
	if($trim)
		$message = substr($parts[$start++],1);
	for ($i = $start; $i < count($parts);$i++)
		$message = $message . " " . $parts[$i];
	
	$message = trim($message);
	return $message;
}

function parseMsg($allMessage)
{
	global $socket,$hasIdent,$ident;
	$lines = explode("\n",$allMessage);
	foreach ($lines as $Message)
	{
		$parts = explode(" ",$Message);
		preg_match("/:(.*)!(.*)@(.*)/",$parts[0],$info);
		$channel = strtolower($parts[2]);
		if (strtolower($parts[0]) == "ping")
			socket_write($socket,"PONG " . trim($parts[1]) . "\n");
		switch(strtolower($parts[1]))
		{
			case "privmsg":
				$message = getMessage($parts,3,true);
				if ($message == "!quit") socket_write($socket,"QUIT :Requested!\n");
				if (preg_match("/ACTION (.*)/",$message,$messageA))
					addLine($info[1],'','action',$messageA[1],$channel);
				else
					addLine($info[1],'','message',$message,$channel);
			break;
			case "join":
				addLine($info[1],'','join','',$channel);
			break;
			case "part":		
				$message = getMessage($parts,3,true);
				addLine($info[1],'','part',$message,$channel);
			break;
			case "mode":
				$message = getMessage($parts,3,false);
				addLine($info[1],'','mode',$message,$channel);
			break;
			case "kick":
				$message = getMessage($parts,4,true);
				addLine($info[1],$parts[3],"kick",$message,$channel);
			break;
			case "quit":
				$message = getMessage($parts,2,true);
				addLine($info[1],$parts[3],"quit",$message,'');
			break;
			case "376":
				socket_write($socket,"JOIN #omnimaga-test\n");
			break;
		}
	}
}
	
function addLine($name1,$name2,$type,$message,$channel)
{
	global $socket;
	$curPosArr = mysql_fetch_array(sql_query("SELECT MAX('line_number') FROM `irc_lines`"));
	$curPos =  $curPosArr[0]+ 1;
	sql_query("INSERT INTO `irc_lines` (`name1`,`name2`,`message`,`type`,`channel`) VALUES ('%s','%s','%s','%s','%s')",$name1,$name2,$message,$type,$channel);
	sql_query("DELETE FROM `irc_lines` WHERE `line_number` < %s",$curPos - 1000);
}
?>

<?PHP
	error_reporting(0);
	$socket = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
	if (!socket_connect($socket,"irc.mzima.net",6667))
		die("Could not connect. Error: " . socket_last_error());
		
	$sqlConnection = mysql_connect("localhost","OmnomIRC","-snip-");
	if (!$sqlConnection) 
		die("Could not connect to SQL DB.");
	if (!mysql_select_db("omnomirc",$sqlConnection)) die('Invalid query: ' . mysql_error());

	$ident = "PASS none\nUSER OmnomIRC OmnomIRC OmnomIRC :OmnomIRC\nNICK Omnom_Dev\n";

	sleep(1);
	socket_write($socket,$ident);
	while ($recBuf = socket_read($socket,1024))
	{
		echo $recBuf;
		parseMsg($recBuf);
	}
	socket_close($socket);
?>

Edit: Here's what I got of my server-side update scripts too(I just need to add chan support to these, which is going to be easy.):

Code: (Load.php) [Select]

<?PHP
	header('Content-type: text/javascript');
	header("Last-Modified: " . gmdate("D, d M Y H:i:s") . " GMT");
	header("Cache-Control: no-store, no-cache, must-revalidate");
	header("Cache-Control: post-check=0, pre-check=0", false);
	header("Pragma: no-cache"); 
	$curLine = $_GET['count'];
	$sqlConnection = mysql_connect("localhost","OmnomIRC","-snip-");
	if (!$sqlConnection) 
		die("Could not connect to SQL DB.");
	if (!mysql_select_db("omnomirc",$sqlConnection)) die('Invalid query: ' . mysql_error());
	function sql_query()
	{
		global $sqlConnection;
		$params = func_get_args();
		$query = $params[0];
		$args = Array();
		for ($i=1;$i<count($params);$i++)
			$args[$i-1] = mysql_real_escape_string($params[$i],$sqlConnection);
		$result = mysql_query(vsprintf($query,$args),$sqlConnection);
		if (!$result) 
			die(mysql_error() . "Query: " . vsprintf($query,$args));
		return $result;
	}
	$curLineArr = mysql_fetch_array(sql_query("SELECT MAX(`line_number`) FROM `irc_lines`"));
	$curLine = $curLineArr[0] - $curLine;
	$destLine = $curLineArr[0];
	for ($curLine;$curLine <= $destLine;$curLine++)
	{
		$result = mysql_fetch_array(sql_query("SELECT * FROM `irc_lines` WHERE `line_number` = %s",$curLine));
		echo "addLineStart('";
		switch (strtolower($result['type']))
		{
			case "message":
				echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result['message'])) . ':' . base64_encode("0");break;
			case "action":
				echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result['message'])) . ':' . base64_encode("0");break;
			case "join":
				echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1']));break;
			case "part":
				echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result["message"]));break;
			case "kick":
				echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result["name2"])) . ":" . base64_encode(htmlspecialchars($result["message"]));break;
			case "quit":
				echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result["message"]));break;
			case "mode":
				echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result["message"]));break;
		}
		echo "');\n";
	}
	mysql_close($sqlConnection);
?>

Code: (update.php) [Select]

<?PHP
	header("Last-Modified: " . gmdate("D, d M Y H:i:s") . " GMT");
	header("Cache-Control: no-store, no-cache, must-revalidate");
	header("Cache-Control: post-check=0, pre-check=0", false);
	header("Pragma: no-cache"); 
	$curLine = $_GET['lineNum'];
	$sqlConnection = mysql_connect("localhost","OmnomIRC","-snip-");
	if (!$sqlConnection) 
		die("Could not connect to SQL DB.");
	if (!mysql_select_db("omnomirc",$sqlConnection)) die('Invalid query: ' . mysql_error());
	function sql_query()
	{
		global $sqlConnection;
		$params = func_get_args();
		$query = $params[0];
		$args = Array();
		for ($i=1;$i<count($params);$i++)
			$args[$i-1] = mysql_real_escape_string($params[$i],$sqlConnection);
		$result = mysql_query(vsprintf($query,$args),$sqlConnection);
		if (!$result) 
			die(mysql_error() . "Query: " . vsprintf($query,$args));
		return $result;
	}
	while (true)
	{
		$result = mysql_fetch_array(sql_query("SELECT * FROM `irc_lines` WHERE `line_number` = %s",$curLine + 1));
		if (!isset($result[0])) {sleep(1); continue;}
		switch (strtolower($result['type']))
		{
			case "message":
				echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result['message'])) . ':' . base64_encode("0");break;
			case "action":
				echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result['message'])) . ':' . base64_encode("0");break;
			case "join":
				echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1']));break;
			case "part":
				echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result["message"]));break;
			case "kick":
				echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result["name2"])) . ":" . base64_encode(htmlspecialchars($result["message"]));break;
			case "quit":
				echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result["message"]));break;
			case "mode":
				echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result["message"]));break;
		}
		
		break;
	}
	mysql_close($sqlConnection);
?>

Quote from: jimbauwens on May 10, 2011, 04:57:46 pm

Hmm, looks quite interesting.
If I'm not mistaken, the loop keeps a xmlhttprequest open until it timeouts, and then starts a new request (endless loop here) ?

Quote from: Netham45 on May 10, 2011, 06:03:58 pm

No, it keeps it open until my server responds. It shouldn't time out at all. I have a random-length sleep() on the script that's spewing data for testing.

It does actually time out after 45 seconds, to prevent a large number of lingering PHP scripts running on my server.

DJ Omnimaga · « **Reply #12 on:** May 10, 2011, 11:10:29 pm »

What is the bot? Is it just OmnomIRC?

Netham45 · « **Reply #13 on:** May 10, 2011, 11:12:14 pm »

Yea, it's going to replace the OmnomIRC bot in the chan.

DJ Omnimaga · « **Reply #14 on:** May 10, 2011, 11:13:26 pm »

By the way, did you make it so when a netsplit occurs, it automatically reconnects to another server?

Author Topic: OmnomIRC Dev Ideas (Read 7676 times)

Netham45

OmnomIRC Dev Ideas

ZippyDee

Re: OmnomIRC Dev Ideas

Netham45

Re: OmnomIRC Dev Ideas

ZippyDee

Re: OmnomIRC Dev Ideas

DJ Omnimaga

Re: OmnomIRC Dev Ideas

Netham45

Re: OmnomIRC Dev Ideas

DJ Omnimaga

Re: OmnomIRC Dev Ideas

Jim Bauwens

Re: OmnomIRC Dev Ideas

Netham45

Re: OmnomIRC Dev Ideas

Jim Bauwens

Re: OmnomIRC Dev Ideas

DJ Omnimaga

Re: OmnomIRC Dev Ideas

Netham45

Re: OmnomIRC Dev Ideas

DJ Omnimaga

Re: OmnomIRC Dev Ideas

Netham45

Re: OmnomIRC Dev Ideas

DJ Omnimaga

Re: OmnomIRC Dev Ideas