Simple Performance Testing with Apache Benchmark

VN:R_N [1.3.1_645]
Voting Closed. Rating: 5.0/5 (1 vote cast)


I’ve been knee deep in performance and scalability for some time now, and have used and learned of many useful tools and techniques to help out. One of my favorite command line tools for seeing how well a single Apache server is churning out pages in development comes stock on Ubuntu, and Mac OS X: Apache Benchmark.

A simple performance test against the homepage of one of my client site’s using AB at the command line:

ab -t5 -n100 http://www.teamgzfs.com/

The results:

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking www.teamgzfs.com (be patient)
Finished 664 requests

Server Software:        Apache/2.2.8
Server Hostname:        www.teamgzfs.com
Server Port:            80

Document Path:          /
Document Length:        306 bytes

Concurrency Level:      100
Time taken for tests:   5.054 seconds
Complete requests:      664
Failed requests:        0
Write errors:           0
Total transferred:      465003 bytes
HTML transferred:       205326 bytes
Requests per second:    131.38 [#/sec] (mean)
Time per request:       761.143 [ms] (mean)
Time per request:       7.611 [ms] (mean, across all concurrent requests)
Transfer rate:          89.85 [Kbytes/sec] received

Connection Times (ms)
min  mean[+/-sd] median   max
Connect:       38   78  48.7     64     994
Processing:   114  540 226.2    493    1690
Waiting:      114  531 205.6    493    1485
Total:        191  618 236.0    558    1808

Percentage of the requests served within a certain time (ms)
50%    558
66%    587
75%    598
80%    617
90%    845
95%   1163
98%   1402
99%   1746
100%   1808 (longest request)

There is quite a bit of useful information here that can help you tune your code and server. It’s important to note however, that when working on a larger site, that expects quite a bit more traffic, you might want to investigate some more thorough solutions outside of just a single machine and ab. It is, however, a nice starting point into useful information.

One rather funny pitfall you can run into however, is if the host you are sending requests to is smartly secured – these types of tests become a bit useless, as they may have security settings to limit or delay requests – providing you with timeouts and/or inaccurate information. Best to run these types of things in a semi-developmental mode with those types of security settings turned down, and rely on bigger guns or fleets of boxes and scripts to hit a production secure site.

In addition to hitting just a landing page, you can use AB to send COOKIE or POST data too! This is very useful if you want to see how pages perform but need credentials to get in first. This is a little trickier to do using the -c, -T, -p, and -v flags. I noticed there are under-useful resources online to figuring it out with AB, so it would seem worthwhile to write it – as it took me some trickery to figuring it out as well:

Sending POST data to a login form:

First we create a file that contains our URL encoded post data. Note, AB expects the values to be URL encoded, but not the equal (=) or ampersands (&).

post_data.txt

username=foo%40bar.com&password=foobar

Capturing a cookie:

Here, we use the verbosity (-v) flag so we can see the response headers that come back — many sites will send back a cookie once authenticated, we’ll want to capture that cookie here. Though, some sites will not require it, I demonstrate it for the sake of example:

ab -v4 -n1 -T 'application/x-www-form-urlencoded' -p post_data.txt http://www.foobar.com/login

The returned response header will fly by quick, you’re looking for something like the following:

Set-cookie somesession=somerandomsessiondata...;

The session data may come back encrypted, unencrypted, a serialization, or just a number. That varies by site. The point here is you have a key/value pair for the cookie. All you need is the part up to the semi-colon ( not including the semi-colon ). Copy that “key=val” string, and use it when hitting other pages on the site you are testing, ie:

Using a cookie to test a page that requires a cookie:

ab -t10 -n100 -c 'somessession=somerandomsessiondata' http://www.foobar.com/login_required_page

This can become a lot of fun once you get the hang of it. Now you have the know how, go enjoy creating an arsenal of these scripts and start performance tuning your sites – or script hacking your favorite social network ( or obnoxious Blizzard clan website hahaha… drum roll for D3 – 2010? Please???! ).

Gym Class Heroes in NYC

VN:R_N [1.3.1_645]
Voting Closed. Rating: 0.0/5 (0 votes cast)

As usual, it’s been awhile since the last post. As for pictures say a thousand words update:




Gym Class Heros at Lockerz Launch Party in NYC

Not pictured: “open bar, rock. the lockerz staff enjoying said open bar, rock. hundreds of lockerz fans enjoying the concert, rock. a great meal at the cafeteria in NYC, rock. oh, and Manhattan, ROCK!” Needless to say we had a blast! Thanks to all those supporting Lockerz, and all those who worked hard to throw an awesome launch party, as well as all those who continue to work hard every day to take the company to the next level!

In other news, we’ve been pouring our blood and sweat into work and home – doesn’t mean it hurts though ;-) My long time heterosexual life mate visited from Chicago over the weekend – we had a great time and already miss his company. In addition to his visit, my long time friend, college room mate, and colleague, Drew Kerrigan, stayed for 2 weeks as he joined Lockerz this month and is relocating from NoVA! A special congratulations to him!

So now we have at least 3 huge Diablo 3 fans ( Anthony, Myself, and Drew ) ready to pour vacation time down the drain :-)

One last note! Happy birthday Peter!

Life in Pittsburgh during the G20 Summit

VN:R_N [1.3.1_645]
Voting Closed. Rating: 0.0/5 (0 votes cast)
Troopers in Lawrenceville

Troopers in Lawrenceville

What a strange day! This may be a tad ignorant, but I really don’t know what the G20 is truly all about. I’ve only caught bits and pieces from other people – needless to say, it’s a big event. The office is just a few blocks from the convention center where the meetings are being held, and there are more cops, S.W.A.T. teams, and state troopers marching the streets than a small army.

Driving around, we’ve been behind vehicles tagged “diplomat” more than once, hit a few road blocks, and she had the unfortunate event of protesters banging on the car down Penn Avenue. I’ve caught rumors of tear gas being used, our office building was on lock down towards the end of the day, and on the way home the radio said President Obama was making an appearance in Oakland, just around the area we were staying for the last 2 months!

So, WTF is going on?! Ha… something good better come out of this ( aside of the awesome food Lockerz provided today to keep it’s employees safe during lunch ). We had to drive through Wilkensburg to get home, and that’s just no treat – cab driver once told us that Wilkensburg doesn’t get city funding, so it’s falling apart… crack whores calling out to their suga daddies – no lie. Though, given my ignorance to the world changing event downtown, perhaps I’m no better, ha.

Attached is a picture Drew Zhrodague’s mom took on her way for coffee.

PHP Microtime Tutorial

VN:R_N [1.3.1_645]
Voting Closed. Rating: 0.0/5 (0 votes cast)

Every once in awhile one has to write a script that is concerned with execution time. Consider the following snippet that utilizes epoch seconds to calculate execution time:

$start_time = microtime(true);
usleep(200000);  // sleep 2 seconds
$end_time = microtime(true);

echo 'Execution time:  ' . ($end_time - $start_time) . "\n";

Pretty straight forward and has many useful applications. By using built in php functions, `microtime` and `usleep`, it’s easy to benchmark operations, and even set time limits on operations. Consider the next snippet which will halt execution after 30 seconds:

$max_execution_time = 30;
$start_time = microtime(true);

while (1) {
    if ((microtime(true) - $start_time) > $max_execution_time)) {
        break;
    }

    //Do something
}

One more clever application, is to pace your scripts. Such may be useful when concerned with load:

while (1) {
     usleep(10000);  // sleep 1/10th of a second
    //Send a request... query a DB... do something
}

You can do quite a bit with these two functions. It never hurts to read up on microtime, and usleep. Enjoy!

Exciting Updates! Contrary to popular belief, Pittsburgh Rocks, not Cleveland – Period.

VN:R_N [1.3.1_645]
Voting Closed. Rating: 0.0/5 (0 votes cast)
Lil Wayne In Pittsburgh

Lil Wayne loves Pitt too!

Two weeks down, this blog is due for an update – time has been flying!

As far as home life in Pittsburgh goes – ROCK. I’m currently staying roughly between one of Carnegie Mellon and Pittsburgh University’s campus intersections – the architecture is gorgeous. I enjoyed a meal at the famous Primanti Brothers down the street, learned the public transit ( a work in progress… ), and discovered a few open mic hot spots I’ll be hitting up eventually – oh, and Lil Wayne at the Post Gazette Pavilion was also ROCK! I’ve had numerous other experiences and visited quite a few restaurants, but won’t detail all my eating habits here :-)

As far as Lockerz goes… ROCK ON! I’ve made some new friends, thoroughly enjoyed many philosophical discussions, and am getting knee deep into working on the next best thing. Today was a moment for Lockerz history – as we saw an incredible traffic increase this morning – ranking 4 on Google Trends around 10 a.m.

The boost seemed grass roots in nature, as people are catching wind and telling their friends, who are telling their friends. The most personable for me was Amber’s friend made a comment about needing a Lockerz invite to join the site, not even realizing I work there. Small world? Needless to say, he got the invite.

I won’t detail much else about the job – you’ll have to keep tabs on the site! Need an invite? Let me know.

Chris Page Joins New Venture – Lockerz LLC

VN:R_N [1.3.1_645]
Voting Closed. Rating: 5.0/5 (2 votes cast)
Night shot of Pittsburgh, PA - our new home city!

Night shot of Pittsburgh, PA - our new home city!

I’ve been quite busy lately!  Amber and I decided to leave Morgantown, WV to pursue better opportunities wherever we may fall.  We bunked up with family for a few weeks… a grueling thing with a 2 month old.  Thankfully, it didn’t take too long to stumble upon the next big thing!  Spending so much time increasing my online presence has paid off – as I received a call a few weeks ago from Lockerz CTO, Peter Meulbroek.

Peter and I exchanged a few phone calls – I could tell there was something “right” about the vibe I gained.  Three weeks later I drove into Pittsburgh for a lengthy yet fun and relaxing 5 stage interview – their idea is spectacular and the team was very talented and cool with a work hard and play harder mentality, perfect.  I realized “I’m going to make this happen”.  Thankfully the team at Lockerz shared that realization, and I’m very proud to announce that as of July 27th 2009, I will begin my first day as a ground floor Lockerz technical team member!

This proves it… I’m a startup junkie!

Things are moving very quickly, with only a week to relocate, and it reminds me of one of Amber’s favorite sayings:  “Good things take time, Great things happen all at once.” – this seems to hold true for me, a man of extremes to say the least!

Everybody involved in the process made it feel absolutely “right”.  I’m super psyched about the challenging workload ahead, and Amber and I both have never been so happy with the direction life is taking us!

Go check out Lockerz.com!

<a href=

Upgrading from PHP 5.2 to 5.3 in Ubuntu – Part 1

VN:R_N [1.3.1_645]
Voting Closed. Rating: 0.0/5 (0 votes cast)

( I’m publishing this partially done so that it acts as a reminder for me to FINISH it… bare with me! )

Overview of Important Changes

variable class naming

Previously in PHP, only method and function names could be variables.  Ie:

$func = "print";
$func("Hello World");
class Foo {
  public static function Bar()
  {
     echo "Hello World";
  }
}

$method = "Bar";
Foo::$method();

Now, in PHP 5.3+, variable class naming is also supported – making possible this syntax:

class Foo {
  public static function Bar()
  {
     echo "Hello World";
  }
}

$class = "Foo";
$method = "Bar";
$class::$method

This new class variable naming should provide a much desired level of ambiguity for PHP developers, I know for my MVC framework, it could really change the Typhoon PHP Typhoon->run() method in a positive way.

late static binding

This is a more advance PHP OOP topic – that almost came off as a bug in previous versions of PHP. I’ve personally ran into it myself on occasion, and am happy to see a solution available in PHP 5.3. In previous versions of PHP, static calls resolved to the inherited class. In cases where one needed a static call to resolve in it’s own scope, unexpected results were common. Ie:

class Foo {
    public static function who() {
        echo __CLASS__;
    }
    public static function test() {
        self::who();
    }
}

class Bar extends Foo {
    public static function who() {
         echo __CLASS__;
    }
}

Bar::test();

Output:

Foo

The solution, “Late Static Bindings”, solves this using the static keyword:

class Foo {
    public static function who() {
        echo __CLASS__;
    }
    public static function test() {
        static::who();
    }
}

class Bar extends Foo {
    public static function who() {
         echo __CLASS__;
    }
}

Bar::test();

The static keyword resolves to the calling class and produces the expected output:

Bar

Additions to Standard PHP Library (SPL)

Circular Garbage Collection

Lambda Functions (Anonymous Functions)

Closures

Overriding Internal Functions

New Reserved Words

Namespaces

Jump Labels

Changes to Functions and Methods

Extensions

Phar

php.ini Changes

Deprecated Methods

Install php 5.3 on Ubuntu

mkdir download && cd download
wget http://snaps.php.net/php5.3-200906131830.tar.gz

Facebook Username ( Vanity URL / Permalink )

VN:R_N [1.3.1_645]
Voting Closed. Rating: 4.5/5 (2 votes cast)

My wife and I found a baby sitter to watch our daughter, Gabriella, tonight, and we were able to enjoy a Saturday night out, finally!  Upon our return home, our sitter and good friend notified us that at midnight June 14th EST, Facebook opened up username based URL’s – or to us tech savvy people, vanity URLs or permalinks.  Facebook did already have permalinks to user profiles, but they were difficult to remember and non-arbitrary.

While many major companies were able to acquire their “facebook/mycompany” URL ahead of public release, I know many will be in competition to get theirs ASAP. Our sitter quickly rushed home to be ready to snatch up his – I was unfortunate to still be stuck using my middle initial:

facebook.com/chriswpage

With as many software people named Chris Page – I didn’t expect for it to be easy…but what really chaps me, is a dancer in Australia got to it first!  WTF?  Amber was equally disappointed in the commonness of our last name, when she too had to use her middle initial:

facebook.com/amberdpage

You can register your vanity URL at facebook.com/username.

Ubuntu + Compiz

VN:R_N [1.3.1_645]
Voting Closed. Rating: 5.0/5 (1 vote cast)

This is actually from awhile ago ( almost a year ), but I thought I’d share. It’s my desktop pc, running Ubuntu 8.04 and Compiz Fusion engine.

To be honest, it was a fun exploration for the novelty, but these days, the most I really use from the effects engine is the magnifier to help people standing over my shoulder focus on whatever I’m demonstrating.

Pretty cool none the less! Read up at Compiz Fusion and Ubuntu

Setup a webcam security system with Ubuntu Linux and Motion

VN:F [1.3.1_645]
Rating: 5.0/5 (5 votes cast)
Snap from Office Security Cam

Snap from Office Security Cam

So, now that I’m in Morgantown – my home is too small to comfortably work on side gigs and personal projects – especially now that my family is getting bigger with the baby!  I’ve been using the office space I leased out more and more.  While exploring video conferencing with Matt last week, I had the thought “wouldn’t it be cool to have a security camera in the office?”.  So I did just that, and it’s actually quite easy for Ubuntu linux users.

What you need:
  • Ubuntu Linux ( I was using 8.04.1 at the time of installation )
  • one or more USB web cameras
What you can do:
  • Motion detection – record video/and or frames if there is motion.
  • Snapshot intervals – take time interval snapshots regardless of motion detection.
  • Live video IP stream in mjpeg format.
  • Specify recorded video to be saved in your choice mpeg, avi, flv, swf format.
  • When motion exists, have frames and videos draw a box around the specific motion for more obvious recognition of subtle movements ( this actually shows the shadow of the janitor near the door around 6 a.m. every morning – I wouldn’t have noticed otherwise! )
  • Easily send all data to a backup server in a variety of ways – I keep it simple by saving data to my Dropbox directory, a wonderful cross-platform data syncronization and sharing utility.
Steps:

1.  Plugin your webcam.
For me, the Logitech QuickCam® Pro 9000 worked right out of the box, and was only 105$.

2.  Install Motion – software motion detector, and turn it on.

sudo apt-get install motion
sudo motion

3. Configure Motion

Everything really works out of the box with this – but isn’t quite organized to my liking, and probably not yours either. Global configuration is located inside /etc/motion.conf ( You’ll notice there are multiple threadN.conf files in this directory – which can be used for custom configured individual cameras if you are setting up more than one ).

Note: Be sure to restart the Motion server everytime you make a configuration change.

sudo /etc/init.d/motion restart

Take a look at the files, they are well documented. Below are a few helpful configurations to get your data organized quicker:

#/etc/motion/motion.conf

# Locate and draw a box around the moving object.
locate on

# Draws the timestamp using same options as C function strftime(3)
text_right %Y-%m-%dn%T-%q

# Text is placed in lower left corner
text_left SECURITY CAMERA %t - Office

Organize the filesytem to save data by date, instead of all in one directory.

# File path for snapshots (jpeg or ppm) relative to target_dir
snapshot_filename %Y%m%d/camera-%t/snapshots/hour-%H/camera-%t-%v-%Y%m%d%H%M%S-snapshot

# File path for motion triggered images (jpeg or ppm) relative to target_dir
jpeg_filename %Y%m%d/camera-%t/motions/hour-%H/camera-%t-%v-%Y%m%d%H%M%S-%q-motion

# File path for motion triggered ffmpeg films (mpeg) relative to target_dir
movie_filename %Y%m%d/camera-%t/movies/hour-%H/camera-%t-%v-%Y%m%d%H%M%S-movie

# File path for timelapse mpegs relative to target_dir
timelapse_filename %Y%m%d/camera-%t/timelapses/hour-%H/camera-%t-%Y%m%d-timelapse

4.  (Optional)  Setup a backup solution

a. Easy solution, get and install Dropbox — instructions on the Dropbox site.  Then update your motion.conf to save to your Dropbox directory:

#/etc/motion/motion.conf
...
target_dir /path/to/dropbox/security_camera
...

b. A more granular solution is to take advantage of hooks configurable in motion.conf. Using these, you can create bash scripts to do anything your heart desires ( like trigger a silent alarm on motion detection outside business hours ). Available hooks: on_event_start, on_event_end, on_picture_save, on_motion_detected, on_movie_start, on_movie_end.

If you have wput installed, you can easily upload files to a remote backup server with these hooks:

#motion.conf
...
on_picture_save wput ftp://user@pass@server %f
...

However, this solution is somewhat less secure, as it uses FTP. In a future post I will detail how to secure this up using encrypted transfer and phrase free keys. ( Stay tuned! )

5. Live feed

This comes working out of the box with Motion. Check out your live stream in your web browser by navigating to: http://localhost:8081

That’s it! Webcam security made easy :-)

Visit Other Sites!

Find me on other sites...

Archives

All entries, chronologically...

Pages List

General info about this site...