Memcache

Memcache & MySQL PHP Session Handler

Posted on April 8, 2009. Filed under: Memcache, mysql+, PHP | Tags: , , , , |

LAST UPDATED MAY 17, 2009

I have recently read Cal Henderson’s book, Building Scalable Web Sites, and was inspired 6 ways from Sunday on just about every approach I take to web development. I highly recommend you purchase this book, and read through it asap… it is a must read [even though it was published back in 2006]. My friend Erik Kastner recommended it to me, and now I’m recommending it to you :-)

Anyways, back on topic… One of the major concepts I picked up was that of Write Through Caches. Write Through Caches are explained in depth all throughout the internetS, so I leave you with this: tinyurl.com/cgeobs. This script also solves the scalability issue you will experience when you move your web site / application to more than one web server. If you put a Memcache and MySQL daemon on one box and have all your web servers connect to it, your sessions are in one centralized place [though, this doesn’t account for fail over].

I also built in a simple check to see if the session data had changed before writing it to the DB. Most of the time it doesn’t, so this should offer some more performance.

This was written for PHP5, if you’re still using PHP4, click here.

This script assumes you have already connected to your database.


<?php
    class SessionHandler {
        public $lifeTime;
        public $memcache;
        public $initSessionData;

        function __construct() {
            # Thanks, inf3rno
            register_shutdown_function("session_write_close");

            $this->memcache = new Memcache;
            $this->lifeTime = intval(ini_get("session.gc_maxlifetime"));
            $this->initSessionData = null;
            $this->memcache->connect("127.0.0.1",11211);

            return true;
        }

        function open($savePath,$sessionName) {
            $sessionID = session_id();
            if ($sessionID !== "") {
                $this->initSessionData = $this->read($sessionID);
            }

            return true;
        }

        function close() {
            $this->lifeTime = null;
            $this->memcache = null;
            $this->initSessionData = null;

            return true;
        }

        function read($sessionID) {
            $data = $this->memcache->get($sessionID);
            if ($data === false) {
                # Couldn't find it in MC, ask the DB for it

                $sessionIDEscaped = mysql_real_escape_string($sessionID);
                $r = mysql_query("SELECT `sessionData` FROM `tblsessions` WHERE `sessionID`='$sessionIDEscaped'");
                if (is_resource($r) && (mysql_num_rows($r) !== 0)) {
                    $data = mysql_result($r,0,"sessionData");
                }

                # Refresh MC key: [Thanks Cal :-)]
                $this->memcache->set($sessionID,$data,false,$this->lifeTime);
            }

            # The default miss for MC is (bool) false, so return it
            return $data;
        }

        function write($sessionID,$data) {
            # This is called upon script termination or when session_write_close() is called, which ever is first.
            $result = $this->memcache->set($sessionID,$data,false,$this->lifeTime);

            if ($this->initSessionData !== $data) {
                $sessionID = mysql_real_escape_string($sessionID);
                $sessionExpirationTS = ($this->lifeTime + time());
                $sessionData = mysql_real_escape_string($data);

                $r = mysql_query("REPLACE INTO `tblsessions` (`sessionID`,`sessionExpirationTS`,`sessionData`) VALUES('$sessionID',$sessionExpirationTS,'$sessionData')");
                $result = is_resource($r);
            }

            return $result;
        }

        function destroy($sessionID) {
            # Called when a user logs out...
            $this->memcache->delete($sessionID);
            $sessionID = mysql_real_escape_string($sessionID);
            mysql_query("DELETE FROM `tblsessions` WHERE `sessionID`='$sessionID'");

            return true;
        }

        function gc($maxlifetime) {
            # We need this atomic so it can clear MC keys as well...
            $r = mysql_query("SELECT `sessionID` FROM `tblsessions` WHERE `sessionExpirationTS`<" . (time() - $this->lifeTime));
            if (is_resource($r) && (($rows = mysql_num_rows($r)) !== 0)) {
                for ($i=0;$i<$rows;$i++) {
                    $this->destroy(mysql_result($r,$i,"sessionID"));
                }
            }

            return true;
        }
    }

    ini_set("session.gc_maxlifetime",60 * 30); # 30 minutes
    session_set_cookie_params(0,"/",".myapp.com",false,true);
    session_name("MYAPPSESSION");
    $sessionHandler = new SessionHandler();
    session_set_save_handler(array (&$sessionHandler,"open"),array (&$sessionHandler,"close"),array (&$sessionHandler,"read"),array (&$sessionHandler,"write"),array (&$sessionHandler,"destroy"),array (&$sessionHandler,"gc"));
    session_start();
?>

And here’s the SQL for the DB portion of the Session Handler:


CREATE TABLE `tblsessions` (
    `sessionID` VARCHAR(32) CHARACTER SET ascii COLLATE ascii_bin NOT NULL,
    `sessionExpirationTS` INT(10) UNSIGNED NOT NULL,
    `sessionData` TEXT COLLATE utf8_unicode_ci NOT NULL,
    PRIMARY KEY (`sessionID`),
    KEY `sessionExpirationTS` (`sessionExpirationTS`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;

As always, I welcome any and all CONSTRUCTIVE criticism. If you have something to add, or a bug fix or any suggestions I fully welcome them here. Trolls need not apply.

Advertisements
Read Full Post | Make a Comment ( 35 so far )

Using Memcache with MySQL and PHP

Posted on May 21, 2008. Filed under: Memcache, mysql+, PHP | Tags: , , , , , |

Memcache is a great caching tool available for nearly every scripting or programming environment. I use it with PHP to speed up some applications I have written by completely avoiding asking the database for information. I wanted a very clean way of implementing this in my various projects, with as little change to the existing code as possible. Since memcache is an OBJECT caching system, you can’t simply drop your mysql_query resource into memcache, ask for it at another time, and get your results back … Which is how I had originally thought it worked the first time I read about it. The good news is that it can also store Strings, Integers, Floats, Arrays etc … Below I have a few other things I’ve learned whilst using memcache as well as the caching script I use for database queries.

What I’ve learned:

  1. memcache is great for storing slow queries that return small data sets [1 – 50 results, depending on the average row weight]
  2. memcache is not so great for any query that returns large data sets [100 – ∞, depending on the average row weight]

… in fact, I’ve found that memcache can occasionally be slower than running high-yield queries again [that is, if you have your MySQL server caching queries as well]. It all really boils down to benchmarking the scripts yourself: test every situation with and without the cache! [the more realistic the DB load, the better]

Caching is faster? Yay! Cache it!
DB query is faster? Yay! DON’T cache it!

So, basically, this isn’t a plug-and-play solution for ALL slow page loads / queries, just some of them.

Here’s the code I use to cache MySQL queries:


<?php
    # Connect to memcache:
    global $memcache;
    $memcache = new Memcache;

    # Gets key / value pair into memcache ... called by mysql_query_cache()
    function getCache($key) {
        global $memcache;
        return ($memcache) ? $memcache->get($key) : false;
    }

    # Puts key / value pair into memcache ... called by mysql_query_cache()
    function setCache($key,$object,$timeout = 60) {
        global $memcache;
        return ($memcache) ? $memcache->set($key,$object,MEMCACHE_COMPRESSED,$timeout) : false;
    }

    # Caching version of mysql_query()
    function mysql_query_cache($sql,$linkIdentifier = false,$timeout = 60) {
        if (($cache = getCache(md5("mysql_query" . $sql))) !== false) {
            $cache = false;
            $r = ($linkIdentifier !== false) ? mysql_query($sql,$linkIdentifier) : mysql_query($sql);
            if (is_resource($r) && (($rows = mysql_num_rows($r)) !== 0)) {
                for ($i=0;$i<$rows;$i++) {
                    $fields = mysql_num_fields($r);
                    $row = mysql_fetch_array($r);
                    for ($j=0;$j<$fields;$j++) {
                        if ($i === 0) {
                            $columns[$j] = mysql_field_name($r,$j);
                        }
                        $cache[$i][$columns[$j]] = $row[$j];
                    }
                }
                if (!setCache(md5("mysql_query" . $sql),$cache,$timeout)) {
                    # If we get here, there isn't a memcache daemon running or responding
                }
            }
        }
        return $cache;
    }
?>

The function mysql_query_cache() will return an array filled with the results. Since I don’t use this for large result sets, I don’t free the MySQL resource … you may want to free the resource after it’s been used if you get larger data sets.

Like I had mentioned before, I wanted the code to be as easy-to-use as possible when using it. So, I’ve set up a before and after test scenario showing how to retrofit your code with the new caching code:


<?php
    $sql = "
        SELECT `dataID`, `dataTitle`
        FROM `tbldata`
        WHERE `dataTypeID` BETWEEN 2 AND 2093
        AND `dataStatusID` IN (1,2,3,4)
        AND `dataTitle` LIKE '%something%'
        ORDER BY `dataDate` DESC
        LIMIT 10
    ";

    # Before: [without memcache]
    $rSlowQuery = mysql_query($sql);
    # $rSlowQuery is a MySQL resource
    $rows = mysql_num_rows($rSlowQuery);
    for ($i=0;$i<$rows;$i++) {
        $dataID = intval(mysql_result($rSlowQuery,$i,"dataID"));
        $dataTitle = mysql_result($rSlowQuery,$i,"dataTitle");

        echo "<a href=\"/somewhere/$dataID\">$dataTitle</a><br />\n";
    }


    # After: [with memcache]
    $rSlowQuery = mysql_query_cache($sql);
    # $rSlowQuery is an array
    $rows = count($rSlowQuery);
    for ($i=0;$i<$rows;$i++) {
        $dataID = intval($rSlowQuery[$i]["dataID"]);
        $dataTitle = $rSlowQuery[$i]["dataTitle"];

        echo "<a href=\"/somewhere/$dataID\">$dataTitle</a><br />\n";
    }

?>

Easy, huh? Run print_r() on the returned array to get an idea of how the array is structured if need be.

As always, if you have a better, more efficient, objective, more adaptable solution than mine, please leave a comment! I am 100% open to constructive criticism :-)

Read Full Post | Make a Comment ( 63 so far )

Installing memcache on Windows for PHP

Posted on January 10, 2008. Filed under: Apache, Memcache, PHP | Tags: , , , , , |

LAST UPDATED SEPTEMBER 11, 2010

Installing memcache on Windows XP / Vista is kind of like voodoo for those of us who are not disciplined with compiling code from source. I initially attempted to install memcache a few months ago after reading a few articles about how much performance it can pump into your web application. The problem is that memcache was written with Linux in mind, not windows. So you can’t download any installers or exe files from memcache’s site for windows … which leaves people like me, who use WAMP stacks to develop applications, out in the cold.

So after a few hours of Googling I found a cocktail of methods and files to get memcache to work for win32.

A few things about memcache:
Memcache is a daemon, meaning it runs as a separate service on your machine. Just like MySQL runs as a separate service. In fact, to use memcache in PHP you have to connect to it, just like MySQL.

Think of memcache as the $_SESSION variable for PHP, but instead of it working on a per-user basis, it runs over the entire application — like MySQL. In fact, you can use memcache as you session handler for PHP.

 

This is how I got memcache to work on my windows machine:

  1. Download memcache from code.jellycan.com/memcached/ [grab the ‘win32 binary’ version]
  2. Install memcache as a service:
    • Unzip and copy the binaries to your desired directory (eg. c:\memcached) [you should see one file, memcached.exe] – thanks to Stephen for the heads up on the new version
    • If you’re running Vista, right click on memcached.exe and click Properties. Click the Compatibility tab. Near the bottom you’ll see Privilege Level, check “Run this program as an administrator”.
    • Install the service using the command: c:\memcached\memcached.exe -d install from the command line
    • Start the server from the Microsoft Management Console or by running one of the following commands: c:\memcached\memcached.exe -d start, or net start "memcached Server"

 

Now that you have memcache installed, you’ll have to tie it in with PHP in order to use it.

  1. Check your php extensions directory [should be something like: C:\php\ext] for php_memcache.dll
    If you don’t have any luck finding it, try looking at one of these sites:
    downloads.php.net/pierre/ [thanks to Henrik Gemal]
    pecl4win.php.net/ext.php/php_memcache.dll [currently down]
    www.pureformsolutions.com/pureform.wordpress.com/2008/06/17/php_memcache.dll for PHP 5.2.*
    kromann.info/download.php?strFolder=php5_1-Release_TS&strIndex=PHP5_1 for PHP 5.1.* [thanks, Rich]
  2. Now find your php.ini file [default location for XP Pro is C:\WINDOWS\php.ini] and add this line to the extensions list:
    
    extension=php_memcache.dll
    
    
  3. Restart apache
  4. Run this code to test the installation: [found on www.php.net/memcache]
    
    <?php
        $memcache = new Memcache;
        $memcache->connect("localhost",11211); # You might need to set "localhost" to "127.0.0.1"
    
        echo "Server's version: " . $memcache->getVersion() . "<br />\n";
    
        $tmp_object = new stdClass;
        $tmp_object->str_attr = "test";
        $tmp_object->int_attr = 123;
    
        $memcache->set("key",$tmp_object,false,10);
        echo "Store data in the cache (data will expire in 10 seconds)<br />\n";
    
        echo "Data from the cache:<br />\n";
        var_dump($memcache->get("key"));
    ?>
    
    

If you see anything but errors, you are now using memcache!

EDIT

Memcached, by default, loads with 64mb of memory for it’s use which is low for most applications. To change this to something else, navigate to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\memcached Server in your registry, find the ImagePath entry and change it to look something like this:

“C:\memcached\memcached.exe” -d runservice -m 512

Now when you start the service via net start “memcached Server”, it will run with 512mb of memory at it’s disposal.

EDIT

[Thanks to Travis]
If anyone is wondering what other options can be set (other than the memory limit), run “memcached -help” in a command prompt window. Then modify the ImagePath command line per this article with the desired switches and values.

Read Full Post | Make a Comment ( 179 so far )

Liked it here?
Why not try sites on the blogroll...