OmnomIRC Dev Ideas

[Netham45]

Posting some thoughts I've been having about updating Omnom to be less resource intensive. Just throwing some of this stuff out there if anyone wants to comment.

Don't refresh every 5 seconds or so, but use long polling loop and just return a result when something is said(Would need some sort of client-side parser, but that could be really easy to
do.)

Basically, request update.php, update.php waits with server-side sleep()'s. When something is updated, it finishes update.php and returns the event, something like this:
eventType:base64(message)
Join:base64("Netham45")

the client-side parser would then split it based off of the :, and run Join('Netham45') which would update the textbox, which would print something like 'Netham45 has joined.' or something. Here is what I was thinking for those:

names,messages,modes,targets would be base64_encode'd
line would be the current line, a unique ID for the last line its gotten, for keeping track of where it should check server-side

Code: [Select]

line:join:name
line:join:netham45

line:part:name:message
line:part:netham45:Z8015-190D

line:quit:name:message *Merge quit and part?
line:quit:netham45:Z8015-190D

line:kick:kicker:kicked:message
line:kick:netham45:netbot45:Kick!


line:message:name:message:isOmnom
line:message:netham45:Nom!:1

line:action:name:message
line:action:netham45:explodes


line:mode:modes:targets
line:mode:+ovib:Netham45 Netham45 *@*!*

I was also planning on storing however many lines I'm displaying in an array.

Code: [Select]

Lines = Array();
Lines[0] = "1:message:Netham45:NOM!";
Lines[1] = "2:kick:Netham45:Netbot45:(Netham45)";
...

addLine(message)
{
var i = 1;
for (i;i<Lines.count();i++)
{
Lines[i-1] = Lines[i];
}
Lines[i] = message;
parseMessages();
}

and to parse, something similar:

Code: (parserish thingey javascript-ish code) [Select]


parseMessages()
{
for (i = 0;i<Lines.count();i++)
{
parse(Lines[i]);
}
}

parse(message)
{
chatBox = "";
parts = message.split(":")
type = parts[1]
curLine = lines;
switch type
{
case "join":
chatBox = chatBox + join(parts[2]);break;
case "part":
chatBox = chatBox + part(parts[2],parts[3]);break;
case "quit":
chatBox = chatBox + quit(parts[2],parts[3]);break;
case "kick":
chatBox = chatBox + kick(parts[2],parts[3],parts[4]);break;
case "message":
chatBox = chatBox + message(parts[2],parts[3]);break;
case "action":
chatBox = chatBox + action(parts[2],parts[3]);break;
case "mode":
chatBox = chatBox + mode(parts[2],parts[3]);break;
}
}

join(name)
return name + " has joined";
part(name,reason)
return name + " has left #Omnimaga (" + reason + ")";
quit(name,reason)
return name + " has quit IRC (" + reason + ")";
etc...

Server-side, I plan on having an IRC bot that logs the last 1000 or so lines to a SQL table as they're said. Any more is worthless, half that would still be worthless.
irc_lines = (`line_number`k,ai,`name1`,`name2`,`message`)

Code: (pseudo) [Select]

curPos = sql_query("SELECT MAX(`line_number') FROM `irc_lines`");
sql_query("INSERT INTO `irc_lines` (`action`,`name1`,`name2`,`message`) VALUES ('%s','%s','%s','%s')",curPos,"Join","Netham45","0","0");
sql_query("INSERT INTO `irc_lines` (`action`,`name1`,`name2`,`message`) VALUES ('%s','%s','%s','%s')",curPos,"Kick","Netham45","Netbot45","Stop Lagging you stupid bot!");
sql_query("DELETE FROM `irc_lines` WHERE `line_number` < %s",curPos - 1000);
And to get it from the server, something like this:

Code: (more pseudo) [Select]

getLines()
{
while (true)
{
stuffs = mysql_fetch_array(sql_query("SELECT * FROM `irc_lines` WHERE 'line_number' > %s",reqLines));

for (i=0;i<count(stuffs);i++)
{
switch (stuffs['action'])
{
case "join:
echo stuffs['line_number'] . ':' . stuffs['action'] . ':' . stuffs['name1];
etc...
}
}
if (count(stuffs)) break;
sleep(50);
}
}



Gimmeh ideas/suggestions.

[ZippyDee]

I think your "line:kick:kicker:kicked:message" example is missing the "kicked" part...

But I think that would be a great way to go. That would surely lighten the load on omnimaga.

Also, I think part/quit shouldn't be merged.

[Netham45]

fix'd.

Another thing I'd like to add is that this wouldn't replace the logs, those are best logged to a file as they currently are. This would, though, fix the multi-byte char problems for #omnimaga-fr too.

It'd also be easy to implement a jsonp feed so Eeems would stop raging.

[ZippyDee]

Of course, logs should definitely be stored normally. But fixing the char issue would be GREAT.

[DJ Omnimaga]

Hmm I don't understand that stuff much, but it's cool to see it will be updated.

As for logs does it means the old logs would no longer be available?

[Netham45]

The logs will still be available, but I won't be relying on them for OmnomIRC.


I got most of the client-side parser done, I believe. Just need to add a few things to make it tell between Omnom and Non-Omnom users, cookies, and such.

Code: [Select]

messageList = Array();
userList = Array();
curLine = 0;
messageBox = document.getElementById("MessageBox");
Lines = Array();
//******************************
// Start Request Loop functions
//******************************
function startLoop()
{
xmlhttp=getAjaxObject();
if (xmlhttp==null) {
alert ("Your browser does not support AJAX! Please update for OmnomIRC compatibility.");
return;
}
xmlhttp.onreadystatechange=getIncomingLine;
sendRequest();
}

function sendRequest()
{
url = "http://omnom.omnimaga.org/OmnomIRC_Dev/Update.php?lineNum=" + curLine;
xmlhttp.open("GET",url,true);
xmlhttp.send(null);
}

function getIncomingLine()
{
if (xmlhttp.readyState==4 || xmlhttp.readyState=="complete") {
addLine(xmlhttp.responseText);
sendRequest();
}
}

function getAjaxObject()
{
xmlhttp=new XMLHttpRequest(); //Decent Browsers
if (!xmlhttp || xmlhttp == undefined || xmlhttp == null) xmlhttp=new ActiveXObject("Msxml2.XMLHTTP");  //IE7+
if (!xmlhttp || xmlhttp == undefined || xmlhttp == null) xmlhttp=new ActiveXObject("Microsoft.XMLHTTP"); //IE6-
return xmlhttp;
}

//******************************
// End Request Loop functions
//******************************

//******************************
// Start Line adding functions
//******************************
function addLineStart(message)
{
setCurLine(message);
Lines[Lines.length] = message;
}

function addLine(message)
{
var i = 1;
setCurLine(message);
for (i;i<Lines.length;i++)
{
Lines[i-1] = Lines[i];
}
Lines[i-1] = message;
parseMessages();
}

function setCurLine(message)
{
curLine = message.split(":")[0];
}
//******************************
// End Line Adding Functions
//******************************

//******************************
// Parser Start
//******************************
function parseMessages()
{
text = "";
var i = 0;
for (i = 0;i<Lines.length;i++)
{
text = text + parse(Lines[i]);
}
messageBox.innerHTML = text;
}

function parse(message)
{
a = message;
parts = message.split(":");
type = parts[1];

for (i = 2;i < parts.length;i++)
parts[i] = atob(parts[i]); //De-base64 it. ***** WILL NOT WORK IN IE *****
switch (type)
{
case "join":
return  join(parts[2]);break;
case "part":
return  part(parts[2],parts[3]);break;
case "quit":
return  quit(parts[2],parts[3]);break;
case "kick":
return  kick(parts[2],parts[3],parts[4]);break;
case "message":
return  sendMessage(parts[2],parts[3],parts[4]);break;
case "action":
return  action(parts[2],parts[3],parts[4]);break;
case "mode":
return  mode(parts[2],parts[3]);break;
}
return "";
}

function join(name)
{
return name + " has joined #omnimaga" + "\n<br/>";
}
function part(name,reason)
{
return name + " has left #omnimaga (" + reason + ")" + "\n<br/>";
}
function quit(name,reason)
{
return name + " has quit IRC (" + reason + ")" + "\n<br/>";
}
function kick(kicker,kicked,reason)
{
return kicker + " has kicked " + kicked + " from #omnimaga (" + reason + ")" + "\n<br/>";
}
function sendMessage(name,message,isOmnom)
{
return "&lt;" + name + "&gt " + message + "\n<br/>";
}
function action(name,action)
{
return " * " + name + " " + action + "\n<br/>";
}
function mode(name,mode)
{
return name + " set #omnimaga mode " + mode + "\n<br/>";
}
//******************************
// Parser End
//******************************
function load()
{
var body= document.getElementsByTagName('body')[0];
var script= document.createElement('script');
script.type= 'text/javascript';
script.src= 'Load.php?count=17';
script.onload= function(){parseMessages();startLoop();};
body.appendChild(script);
}
window.onLoad = load();

Edit: I have a preview of it available at http://omnom.omnimaga.org/OmnomIRC_Dev/OmnomIRCv2.html (Just spewing randomly generated data now)

[DJ Omnimaga]

Ah ok, sounds cool. Good luck with this.

[Jim Bauwens]

Hmm, looks quite interesting.
If I'm not mistaken, the loop keeps a xmlhttprequest open until it timeouts, and then starts a new request (endless loop here) ?

[Netham45]

No, it keeps it open until my server responds. It shouldn't time out at all. I have a random-length sleep() on the script that's spewing data for testing.

[Jim Bauwens]

Oh, ok. Ajax isn't my best subject

[DJ Omnimaga]

By the way I noticed something with the links at the top: When clicking Francais or the logs, it does nothing, but the second time it works. Somebody else reported that.

[Netham45]

IRC Bot! (Not quite done, as with everything else. Still need to track userlists, and add sending.)

Code: (IRC_Bot.php) [Select]

<?PHP
function sql_query()
{
global $sqlConnection;
$params = func_get_args();
$query = $params[0];
$args = Array();
for ($i=1;$i<count($params);$i++)
$args[$i-1] = htmlspecialchars(mysql_real_escape_string(trim($params[$i]),$sqlConnection));
$result = mysql_query(vsprintf($query,$args),$sqlConnection);
if (!$result)
die(mysql_error() . "Query: " . vsprintf($query,$args));
return $result;
}

function getMessage($parts,$start,$trim)
{
if($trim)
$message = substr($parts[$start++],1);
for ($i = $start; $i < count($parts);$i++)
$message = $message . " " . $parts[$i];

$message = trim($message);
return $message;
}

function parseMsg($allMessage)
{
global $socket,$hasIdent,$ident;
$lines = explode("\n",$allMessage);
foreach ($lines as $Message)
{
$parts = explode(" ",$Message);
preg_match("/:(.*)!(.*)@(.*)/",$parts[0],$info);
$channel = strtolower($parts[2]);
if (strtolower($parts[0]) == "ping")
socket_write($socket,"PONG " . trim($parts[1]) . "\n");
switch(strtolower($parts[1]))
{
case "privmsg":
$message = getMessage($parts,3,true);
if ($message == "!quit") socket_write($socket,"QUIT :Requested!\n");
if (preg_match("/
ACTION (.*)
/",$message,$messageA))
addLine($info[1],'','action',$messageA[1],$channel);
else
addLine($info[1],'','message',$message,$channel);
break;
case "join":
addLine($info[1],'','join','',$channel);
break;
case "part":
$message = getMessage($parts,3,true);
addLine($info[1],'','part',$message,$channel);
break;
case "mode":
$message = getMessage($parts,3,false);
addLine($info[1],'','mode',$message,$channel);
break;
case "kick":
$message = getMessage($parts,4,true);
addLine($info[1],$parts[3],"kick",$message,$channel);
break;
case "quit":
$message = getMessage($parts,2,true);
addLine($info[1],$parts[3],"quit",$message,'');
break;
case "376":
socket_write($socket,"JOIN #omnimaga-test\n");
break;
}
}
}

function addLine($name1,$name2,$type,$message,$channel)
{
global $socket;
$curPosArr = mysql_fetch_array(sql_query("SELECT MAX('line_number') FROM `irc_lines`"));
$curPos =  $curPosArr[0]+ 1;
sql_query("INSERT INTO `irc_lines` (`name1`,`name2`,`message`,`type`,`channel`) VALUES ('%s','%s','%s','%s','%s')",$name1,$name2,$message,$type,$channel);
sql_query("DELETE FROM `irc_lines` WHERE `line_number` < %s",$curPos - 1000);
}
?>

<?PHP
error_reporting(0);
$socket = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
if (!socket_connect($socket,"irc.mzima.net",6667))
die("Could not connect. Error: " . socket_last_error());

$sqlConnection = mysql_connect("localhost","OmnomIRC","-snip-");
if (!$sqlConnection)
die("Could not connect to SQL DB.");
if (!mysql_select_db("omnomirc",$sqlConnection)) die('Invalid query: ' . mysql_error());

$ident = "PASS none\nUSER OmnomIRC OmnomIRC OmnomIRC :OmnomIRC\nNICK Omnom_Dev\n";

sleep(1);
socket_write($socket,$ident);
while ($recBuf = socket_read($socket,1024))
{
echo $recBuf;
parseMsg($recBuf);
}
socket_close($socket);
?>

Edit: Here's what I got of my server-side update scripts too(I just need to add chan support to these, which is going to be easy.):

Code: (Load.php) [Select]

<?PHP
header('Content-type: text/javascript');
header("Last-Modified: " . gmdate("D, d M Y H:i:s") . " GMT");
header("Cache-Control: no-store, no-cache, must-revalidate");
header("Cache-Control: post-check=0, pre-check=0", false);
header("Pragma: no-cache");
$curLine = $_GET['count'];
$sqlConnection = mysql_connect("localhost","OmnomIRC","-snip-");
if (!$sqlConnection)
die("Could not connect to SQL DB.");
if (!mysql_select_db("omnomirc",$sqlConnection)) die('Invalid query: ' . mysql_error());
function sql_query()
{
global $sqlConnection;
$params = func_get_args();
$query = $params[0];
$args = Array();
for ($i=1;$i<count($params);$i++)
$args[$i-1] = mysql_real_escape_string($params[$i],$sqlConnection);
$result = mysql_query(vsprintf($query,$args),$sqlConnection);
if (!$result)
die(mysql_error() . "Query: " . vsprintf($query,$args));
return $result;
}
$curLineArr = mysql_fetch_array(sql_query("SELECT MAX(`line_number`) FROM `irc_lines`"));
$curLine = $curLineArr[0] - $curLine;
$destLine = $curLineArr[0];
for ($curLine;$curLine <= $destLine;$curLine++)
{
$result = mysql_fetch_array(sql_query("SELECT * FROM `irc_lines` WHERE `line_number` = %s",$curLine));
echo "addLineStart('";
switch (strtolower($result['type']))
{
case "message":
echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result['message'])) . ':' . base64_encode("0");break;
case "action":
echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result['message'])) . ':' . base64_encode("0");break;
case "join":
echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1']));break;
case "part":
echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result["message"]));break;
case "kick":
echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result["name2"])) . ":" . base64_encode(htmlspecialchars($result["message"]));break;
case "quit":
echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result["message"]));break;
case "mode":
echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result["message"]));break;
}
echo "');\n";
}
mysql_close($sqlConnection);
?>


Code: (update.php) [Select]

<?PHP
header("Last-Modified: " . gmdate("D, d M Y H:i:s") . " GMT");
header("Cache-Control: no-store, no-cache, must-revalidate");
header("Cache-Control: post-check=0, pre-check=0", false);
header("Pragma: no-cache");
$curLine = $_GET['lineNum'];
$sqlConnection = mysql_connect("localhost","OmnomIRC","-snip-");
if (!$sqlConnection)
die("Could not connect to SQL DB.");
if (!mysql_select_db("omnomirc",$sqlConnection)) die('Invalid query: ' . mysql_error());
function sql_query()
{
global $sqlConnection;
$params = func_get_args();
$query = $params[0];
$args = Array();
for ($i=1;$i<count($params);$i++)
$args[$i-1] = mysql_real_escape_string($params[$i],$sqlConnection);
$result = mysql_query(vsprintf($query,$args),$sqlConnection);
if (!$result)
die(mysql_error() . "Query: " . vsprintf($query,$args));
return $result;
}
while (true)
{
$result = mysql_fetch_array(sql_query("SELECT * FROM `irc_lines` WHERE `line_number` = %s",$curLine + 1));
if (!isset($result[0])) {sleep(1); continue;}
switch (strtolower($result['type']))
{
case "message":
echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result['message'])) . ':' . base64_encode("0");break;
case "action":
echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result['message'])) . ':' . base64_encode("0");break;
case "join":
echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1']));break;
case "part":
echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result["message"]));break;
case "kick":
echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result["name2"])) . ":" . base64_encode(htmlspecialchars($result["message"]));break;
case "quit":
echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result["message"]));break;
case "mode":
echo $result['line_number'] . ":" . $result['type'] . ":" . base64_encode(htmlspecialchars($result['name1'])) . ":" . base64_encode(htmlspecialchars($result["message"]));break;
}

break;
}
mysql_close($sqlConnection);
?>


Hmm, looks quite interesting.
If I'm not mistaken, the loop keeps a xmlhttprequest open until it timeouts, and then starts a new request (endless loop here) ?

No, it keeps it open until my server responds. It shouldn't time out at all. I have a random-length sleep() on the script that's spewing data for testing.

It does actually time out after 45 seconds, to prevent a large number of lingering PHP scripts running on my server.

[DJ Omnimaga]

What is the bot? Is it just OmnomIRC?

[Netham45]

Yea, it's going to replace the OmnomIRC bot in the chan.

[DJ Omnimaga]

By the way, did you make it so when a netsplit occurs, it automatically reconnects to another server?