Tag Archives: development

Net_DNS2 Moved to GitHub

I’ve never been a bit fan of git- I’ve got used to using SVN over the years, and never saw a compelling reason to change- until now- that Google is shutting down the Google Code service- so I’m forced to move.

Luckily I can still keep using SVN with GitHub- I can put off actually using git for the foreseeable future!

The new Net_DNS2 repository is officially moved to GitHub:

https://github.com/mikepultz/netdns2

Net_DNS2 Version 1.3.2 Released

I’ve released version 1.3.2 of the PEAR Net_DNS2 library- you can install it now through the command line PEAR installer:

pear install Net_DNS2

Download it directly from the Google Code page here.

Or, you can also add it to your project using composer.

Version 1.3.2

  • added support for the EUI48 and EUI64 resource records (RFC7043).
  • fixed how we handle the return values from socket select() statements; this wasn’t causing a problem, but it wasn’t quite right.
  • added some error messaging when the socket times out).
  • before we cache the data, unset the rdata value; this was causing some JSON errors to be generated, and we don’t need the data anyway.

Mining Twitter API v1.1 Streams from PHP – with OAuth

This is a quick update to my post about a year ago, with details on how to mine Twitter streams in real-time using PHP. This new code includes updates for the v1.1 API, including authentication using OAuth.

The first thing you need to do is sign in to the Twitter developer portal with your Twitter account here: https://dev.twitter.com/user/login twitter_account

Once you’ve logged in, click on your profile icon in the top right hand corner, select
“My applications”, and create a new application if you don’t already have one.

Select the option to create the access token as well, as the requests need to be signed by a Twitter account.

The Code

ctwitter_stream.php

class ctwitter_stream
{
    private $m_oauth_consumer_key;
    private $m_oauth_consumer_secret;
    private $m_oauth_token;
    private $m_oauth_token_secret;

    private $m_oauth_nonce;
    private $m_oauth_signature;
    private $m_oauth_signature_method = 'HMAC-SHA1';
    private $m_oauth_timestamp;
    private $m_oauth_version = '1.0';

    public function __construct()
    {
        //
        // set a time limit to unlimited
        //
        set_time_limit(0);
    }

    //
    // set the login details
    //
    public function login($_consumer_key, $_consumer_secret, $_token, $_token_secret)
    {
        $this->m_oauth_consumer_key     = $_consumer_key;
        $this->m_oauth_consumer_secret  = $_consumer_secret;
        $this->m_oauth_token            = $_token;
        $this->m_oauth_token_secret     = $_token_secret;

        //
        // generate a nonce; we're just using a random md5() hash here.
        //
        $this->m_oauth_nonce = md5(mt_rand());

        return true;
    }

    //
    // process a tweet object from the stream
    //
    private function process_tweet(array $_data)
    {
        print_r($_data);

        return true;
    }

    //
    // the main stream manager
    //
    public function start(array $_keywords)
    {
        while(1)
        {
            $fp = fsockopen("ssl://stream.twitter.com", 443, $errno, $errstr, 30);
            if (!$fp)
            {
                echo "ERROR: Twitter Stream Error: failed to open socket";
            } else
            {
                //
                // build the data and store it so we can get a length
                //
                $data = 'track=' . rawurlencode(implode($_keywords, ','));

                //
                // store the current timestamp
                //
                $this->m_oauth_timestamp = time();

                //
                // generate the base string based on all the data
                //
                $base_string = 'POST&' . 
                    rawurlencode('https://stream.twitter.com/1.1/statuses/filter.json') . '&' .
                    rawurlencode('oauth_consumer_key=' . $this->m_oauth_consumer_key . '&' .
                        'oauth_nonce=' . $this->m_oauth_nonce . '&' .
                        'oauth_signature_method=' . $this->m_oauth_signature_method . '&' . 
                        'oauth_timestamp=' . $this->m_oauth_timestamp . '&' .
                        'oauth_token=' . $this->m_oauth_token . '&' .
                        'oauth_version=' . $this->m_oauth_version . '&' .
                        $data);

                //
                // generate the secret key to use to hash
                //
                $secret = rawurlencode($this->m_oauth_consumer_secret) . '&' . 
                    rawurlencode($this->m_oauth_token_secret);

                //
                // generate the signature using HMAC-SHA1
                //
                // hash_hmac() requires PHP >= 5.1.2 or PECL hash >= 1.1
                //
                $raw_hash = hash_hmac('sha1', $base_string, $secret, true);

                //
                // base64 then urlencode the raw hash
                //
                $this->m_oauth_signature = rawurlencode(base64_encode($raw_hash));

                //
                // build the OAuth Authorization header
                //
                $oauth = 'OAuth oauth_consumer_key="' . $this->m_oauth_consumer_key . '", ' .
                        'oauth_nonce="' . $this->m_oauth_nonce . '", ' .
                        'oauth_signature="' . $this->m_oauth_signature . '", ' .
                        'oauth_signature_method="' . $this->m_oauth_signature_method . '", ' .
                        'oauth_timestamp="' . $this->m_oauth_timestamp . '", ' .
                        'oauth_token="' . $this->m_oauth_token . '", ' .
                        'oauth_version="' . $this->m_oauth_version . '"';

                //
                // build the request
                //
                $request  = "POST /1.1/statuses/filter.json HTTP/1.1\r\n";
                $request .= "Host: stream.twitter.com\r\n";
                $request .= "Authorization: " . $oauth . "\r\n";
                $request .= "Content-Length: " . strlen($data) . "\r\n";
                $request .= "Content-Type: application/x-www-form-urlencoded\r\n\r\n";
                $request .= $data;

                //
                // write the request
                //
                fwrite($fp, $request);

                //
                // set it to non-blocking
                //
                stream_set_blocking($fp, 0);

                while(!feof($fp))
                {
                    $read   = array($fp);
                    $write  = null;
                    $except = null;

                    //
                    // select, waiting up to 10 minutes for a tweet; if we don't get one, then
                    // then reconnect, because it's possible something went wrong.
                    //
                    $res = stream_select($read, $write, $except, 600, 0);
                    if ( ($res == false) || ($res == 0) )
                    {
                        break;
                    }

                    //
                    // read the JSON object from the socket
                    //
                    $json = fgets($fp);

                    //
                    // look for a HTTP response code
                    //
                    if (strncmp($json, 'HTTP/1.1', 8) == 0)
                    {
                        $json = trim($json);
                        if ($json != 'HTTP/1.1 200 OK')
                        {
                            echo 'ERROR: ' . $json . "\n";
                            return false;
                        }
                    }

                    //
                    // if there is some data, then process it
                    //
                    if ( ($json !== false) && (strlen($json) > 0) )
                    {
                        //
                        // decode the socket to a PHP array
                        //
                        $data = json_decode($json, true);
                        if ($data)
                        {
                            //
                            // process it
                            //
                            $this->process_tweet($data);
                        }
                    }
                }
            }

            fclose($fp);
            sleep(10);
        }

        return;
    }
};

The “process_tweet()” method will be called for each matching tweet- just modify that method to process the tweet however you want (load it into a database, print it to screen, email it, etc). The keyword matching isn’t perfect- if you search for a string of words, it won’t necessarily match the words in that exact order, but you can check that yourself from the process_tweet() method.

Then create a simple PHP application to run the collector:

require 'ctwitter_stream.php';

$t = new ctwitter_stream();

$t->login('consumer_key', 'consumer secret', 'access token', 'access secret');

$t->start(array('facebook', 'fbook', 'fb'));

You’ll need to provide the Consumer Key, Consumer Secret, Access Token, and the Access Secret, all of which are available from the Details section of your Application.

This new class uses the PHP hash_hmac() function for OAuth, which is available only in PHP 5.2.1 and up, and in the PECL hash extension 1.1 and up.

You can also Download the file here: http://mikepultz.com/uploads/ctwitter_stream.php.zip

How to Make Images Look Good on the IPad 3

The new Apple IPad (IPad 3) came out on the 16th, and probably the first thing I noticed about it is how great the screen looks. It’s armed with same “retina” display that the IPhone 4 came out with a few years ago. It’s great! Except for one thing- a lot of images on the web look like crap now!

(This first image is the standard resolution, the second is optimized for IPhone 4 and IPad 3 screens)

    

Now- this isn’t new. The IPhone 4 has the same “quirk”, but I think it’s less noticeable given the the size of it’s screen. It really stands out on the IPad 3’s 9.7 inch screen.

The Problem with Pixels

With the advent of high pixel density displays, the pixel itself is now a relative unit.

According to the CSS 2.1 Spec:

Pixel units are relative to the resolution of the viewing device, i.e., most often a computer display. If the pixel density of the output device is very different from that of a typical computer display, the user agent should rescale pixel values.

So, a CSS “pixel” indicates one point on the “virtual” pixel grid to which your CSS design aligns. This either directly matches the actual device, or it is “somehow” scaled to suite.

Talking about the new IPad 3 specifically, the new retina display has a huge 2048 x 1536 pixel resolution- double what most sites are designed for. On a desktop machine, if you doubled your screen resolution, websites would just show up half as big. But on the IPad 3, it stretches the site so it “fills up” the screen. The problem with this is stretching raster images (gif, png, jpeg’s) can make them look really distorted and full of artifacts.

So- how do you fix this?

The easiest way to fix this is to make a second copy of all your images at double the resolution, and then use these versions when visitors are on an IPad or IPhone (or any device that has a higher pixel density).

Fixing it in CSS

The min-device-pixel-ratio media query can be used to target style for high pixel density displays. For the moment vendor prefixes are required, until there is a standard format. For example, Mozilla and Webkit prefixes work the same way, but Opera requires the pixel ratio as a fraction.

-moz-min-device-pixel-ratio: 2
-o-min-device-pixel-ratio: 2/1
-webkit-min-device-pixel-ratio: 2
min-device-pixel-ratio: 2

Right now, we only care about IPhone/IPad, so we’ll use the -webkit-min-device-pixel-ratio tag.

So let’s say you had a single class, loading the 300 x 300 px image:

.logo {
 background-image: url(cat_300.jpg);
 width: 300px;
 height: 300px;
}

You would then create a second copy of the image at 600 x 600 px resolution, and add this in your CSS:

@media only screen and (-webkit-min-device-pixel-ratio: 2) {
 .logo {
   background-image: url(cat_600.jpg);
   background-size: 300px 300px;
 }
}

This loads the 600 x 600 px image, but forces the background-size to 300 x 300 px when the device is a webkit device, and the pixel ratio is 2.

Forcing the 600 x 600 px image into a 300 x 300 px box, forces the image to a pixel density of 2.

Fixing Inline Images

So that’s CSS- what about plain old <img> tags?

You can use the window.devicePixelRatio property in JavaScript to determine if the pixel density of the screen is > 1 and if so, cycle through all the images on the page and change their image src to the higher resolution image.

An easy way to do this is to add a class to all the images you want to replace. In this case, I’ve added the “hd” class to the image tags.

<img src="cat_300.jpg" width="300" height="300" class="hd" />

Then add some simple JavaScript to update the tags. In my example, I’ve used jQuery just to make things easier, and simply did a text replace in the src image name, changing the “300” to a 600″. The width/height of the <img> tag needs to stay at 300 x 300 px, forcing the pixel density of 2.

$(document).ready(function()
{
    if ( (window.devicePixelRatio) && (window.devicePixelRatio >= 2) )
    {
        var images = $('img.hd');        

        for(var i=0; i<images.length; i++)
        {
            images.eq(i).attr('src', images.eq(i).attr('src').replace('300', '600'))
        }
    }
});

Now, there are all sorts of ways to do this- this is just one example.

The other way to do this is to just always load the higher resolution images. The only downside, is those higher resolution images are likely almost twice the size of their originals- so loading them only when required will save bandwidth.

What about SVG?

SVG (Scalable Vector Graphics) is also another great way to handle this. SVG files are actually XML files that have instructions on how to “draw” the image on a canvas, rather than using a static raster image. SVG files can scale to different sizes/pixel densities, without distorting.

The only downsides with SVG, is that sometimes the file size for complex images are actually a lot bigger than their raster counterparts, and browser support is still incomplete- so if you care about your site working in Internet Explorer, then you still need to have some raster images and do conditional loading.

Parting Thoughts

Remember- you don’t *actually* need to do any of this; a lot of images will still look “fine” being stretched.

But if you want your site to look it’s best, it’s worth spending the time to optimize it for higher pixel density devices- there’s going to be no shortage of them in the coming years!

How To Mine Twitter Streams from PHP in Real Time

UPDATE: I’ve wrote a new post with an example on how to connect to the v1.1 Twitter API, using OAuth – here.

Need to mine Twitter for tweets related to certain keywords?

No problem-

Twitter provides a pretty simple streaming interface to the onslaught of tweets it receives, letting you specify whatever keywords you want to search for, in a real-time “live” way.

To do this, I created a simple PHP class that can run in the background, collecting tweets for certain keywords:

ctwitter_stream.php

class ctwitter_stream
{
    private $m_username;
    private $m_password;

    public function __construct()
    {
        //
        // set a time limit to unlimited
        //
        set_time_limit(0);
    }

    //
    // set the login details
    //
    public function login($_username, $_password)
    {
        $this->m_username = $_username;
        $this->m_password = $_password;
    }

    //
    // process a tweet object from the stream
    //
    private function process_tweet(array $_data)
    {
        print_r($_data);

        return true;
    }

    //
    // the main stream manager
    //
    public function start(array $_keywords)
    {
        while(1)
        {
            $fp = fsockopen("ssl://stream.twitter.com", 443, $errno, $errstr, 30);
            if (!$fp)
            {
                echo "ERROR: Twitter Stream Error: failed to open socket";
            } else
            {
                //
                // build the request
                //
                $request  = "GET /1/statuses/filter.json?track=";
                $request .= urlencode(implode($_keywords, ',')) . " HTTP/1.1\r\n";
                $request .= "Host: stream.twitter.com\r\n";
                $request .= "Authorization: Basic ";
                $request .= base64_encode($this->m_username . ':' . $this->m_password);
                $request .= "\r\n\r\n";

                //
                // write the request
                //
                fwrite($fp, $request);

                //
                // set it to non-blocking
                //
                stream_set_blocking($fp, 0);

                while(!feof($fp))
                {
                    $read   = array($fp);
                    $write  = null;
                    $except = null;

                    //
                    // select, waiting up to 10 minutes for a tweet; if we don't get one, then
                    // then reconnect, because it's possible something went wrong.
                    //
                    $res = stream_select($read, $write, $except, 600, 0);
                    if ( ($res == false) || ($res == 0) )
                    {
                        break;
                    }

                    //
                    // read the JSON object from the socket
                    //
                    $json = fgets($fp);
                    if ( ($json !== false) && (strlen($json) > 0) )
                    {
                        //
                        // decode the socket to a PHP array
                        //
                        $data = json_decode($json, true);
                        if ($data)
                        {
                            //
                            // process it
                            //
                            $this->process_tweet($data);
                        }
                    }
                }
            }

            fclose($fp);
            sleep(10);
        }

        return;
    }
};

The “process_tweet()” method will be called for each matching tweet- just modify that method to process the tweet however you want (load it into a database, print it to screen, email it, etc). The keyword matching isn’t perfect- if you search for a string of words, it won’t necessarily match the words in that exact order, but you can check that yourself from the process_tweet() method.

Then create a simple PHP application to run the collector:

require 'ctwitter_stream.php';

$t = new ctwitter_stream();

$t->login('your twitter username', 'your twitter password');

$t->start(array('facebook', 'fbook', 'fb'));

Just provide your twitter account username/password, and then an array of keywords/strings to search for.

Since this application runs continuously in the background, it’s obviously not meant to be run via a web request, but meant to be run from the command line of your Unix or Windows box.

According to the Twitter documentation, the default access level allows up to 400 keywords, so you can track all sorts of things at the same time. If you need more details about the Twitter streaming API, it’s available here.

This class uses the HTTPS PHP stream– so you’ll need the OpenSSL extension enabled for it to work.