Load Testing with PHP and RedLine13

Load Testingwith Jason Lotito
“Guaranteed to be the best talk on load testing
with PHP @ MidwestPHP 2016 during that time
slot in that room with me as the speaker"

Jason Lotito
• DevOps Lead Architect
• MeetMe.com
• @jasonlotito
• jasonlotito.com
• jlotito@meetme.com

Lorem Ipsum Dolor
meetme.com/apps/careers

Load Testing
Not Unit Testing, Integration Testing, Acceptance
Testings, manual Testing, or Standardized Testing

Jason Lotito
Load test, or your users will do it for you.

Your Servers

And if they do, you won’t have any users.

You can use load testing to practice
dealing with lots of requests

Load Testing
• Putting demand on a system
• Determine behavior under
load
• Determine maximum operating
capacity
• Discover Bottlenecks
• Determining steps to increase
operating capacity
• Optimize for costs

Other Load Testing Tools
• ApacheBench (ab)
• Apache JMeter
• siege
• BlazeMeter
• and many more…

Why RedLine13?
• Simple, up and running in little time
• Freemium, free level you only pay AWS
• Scriptable (PHP, Node.js)
• AWS for Infinite Scalability™
• I know Rich, one of the founders

RedLine13 has a bunch of features you can read
about on the website. I’m really here to talk to
you about the cool stuff I do with RedLine13.

Load Testing with PHP and RedLine13

Let’s walk through a
Custom Load Test

CustomTest.php
The class that actually runs the test.

In our example, we’ll use sleep to
simulate performing actions such as
making HTTP requests to an API

<?php 
 
require_once( 'LoadTestingTest.class.php' ); 
 
class CustomTest extends LoadTestingTest 
{ 
public function startTest() 
{ 
for ( $x = 1; $x <= 100; $x++ ) 
{ 
$startTime = microtime(true); 
sleep( mt_rand( 2, 5 ) ); 
$diff = microtime( true ) - $startTime; 
recordPageTime( $startTime, $diff ); 
recordURLPageLoad( $x, $startTime, $diff ); 
} 
 
return true; 
} 
}
CustomTest.php
This is
where your
code goes

<?php 
 
 
{ 
{ 
for ( $x = 1; $x <= 100; $x++ ) 
{ 
} 
 
return true; 
} 
}
CustomTest.php

<?php 
 
 
{ 
{ 
for ( $x = 1; $x <= 100; $x++ ) 
{ 
} 
 
return true; 
} 
}
CustomTest.php
Start Time
Start/End Time
Difference

<?php 
 
 
{ 
{ 
for ( $x = 1; $x <= 100; $x++ ) 
{ 
} 
 
return true; 
} 
}
CustomTest.php
URL/String

These next two ﬁles
help us test locally

redline.php
Mock Redline functions to
test load test locally

<?php 
/** 
* @return bool 
*/ 
function isRedlineVerbose() 
{ 
return (int) getenv( 'REDLINE_VERBOSE' ) === 1; 
} 
 
/** 
* @param int $timestamp 
* @param int $elapsedTime 
*/ 
function recordPageTime( $timestamp, $elapsedTime ) 
{ 
isRedlineVerbose() && output( "Record Page Time: timestamp
$timestamp; elapsed time: $elapsedTime" ); 
} 
 
/** 
redline.php

<?php 
/** 
* @return bool 
*/ 
function isRedlineVerbose() 
{ 
return (int) getenv( 'REDLINE_VERBOSE' ) === 1; 
} 
 
/** 
*/ 
{ 
} 
 
/** 
* @param string $url 
*/ 
function recordURLPageLoad( $url, $timestamp, $elapsedTime ) 
{ 
output( "$url -> $elapsedTime seconds" ); 
} 
redline.php

*/ 
{ 
} 
 
/** 
*/ 
{ 
} 
 
/** 
* @param int $kilobytes 
*/ 
function recordDownloadSize( $kilobytes ) 
{ 
isRedlineVerbose() && output( sprintf( "Download Size: %dkb",
$kilobytes ) ); 
} 
 
redline.php

*/ 
{ 
} 
 
/** 
*/ 
{ 
$kilobytes ) ); 
} 
 
/** 
* @param string $error 
*/ 
function recordError( $error ) 
{ 
output( "Error: $error" ); 
} 
 
/** 
* @param string $msg 
redline.php

/** 
*/ 
{ 
$kilobytes ) ); 
} 
 
/** 
*/ 
{ 
} 
 
/** 
* @return bool 
*/ 
function output( $msg ) 
{ 
echo sprintf( "[%s] %s]n", getmypid(), $msg ); 
 
return true; 
}
redline.php

} 
 
/** 
*/ 
{ 
} 
 
/** 
* @return bool 
*/ 
function output( $msg ) 
{ 
echo sprintf( "[%s] %s]n", getmypid(), $msg ); 
 
return true; 
}
redline.php

run.php
The script we create to
test CustomTest.php locally

<?php 
require_once('redline.php'); 
require_once('CustomTest.php'); 
 
$customTest = new CustomTest(1, mt_rand(1000000,9999999)); 
$customTest->startTest();
run.php

<?php 
 
run.php
Instance
Number

<?php 
 
run.php
User
Number
Random
Number

You can use error reporting to
simulate real time logging
Errors show up in real time, logs show up after
the test is completed. Use error reports to let you
know if you should stop your load test.

Sample App: Chat Service
• 1-on-1 Chat
• Group Chat
• Browse Chat Rooms
• Search for Chat Room Topics

Why split 1-on-1 and group
chat into separate tests?

Multiple tests can be run at
the same time.

1-on-1
Group
30-40%
60-70%
30-40%
60-70% 60-70%
30-40%
Time
Start at the
same time
Increase
group chat %
Could mean just more group chat or
both more group and less 1-on-1
Revert to
normal ratio
20:00 40:000:00

1-on-1
Group
30-40%
60-70%
30-40%
60-70% 60-70%
30-40%
Time
Could be done in your
script or in RedLine13

How?
• Start 4 tests:
• For both group and 1-on-1:
• 350 users
• 300 users

1-on-1
Group
350
350+300
350
350+300 350
350
Time
300 test sleeps
for 20 minutes
After 20 minutes,
it stops.

Or, instead of timers,
just do it manually.
Usually the easiest on the day of.

The point is, you can adjust
your load test as needed.

Things You Need
• Production-level systems to test against
• Monitoring
• Automated systems to deploy changes
• Dedicated people
• A time to test (load testing day!)

Things You Need
• Goals, such as
• 500 requests per second
• 90th percentile @ 50ms

Planning the Load Test
• Start at what you expect normal traffic to be
• 500rps
• Start a second test that is just the burst traffic
• 900rps
• Ramps up over 20 mins
• Lasts 40 mins at peak
• Ramps down over 20 mins

Learning
• Is proper monitoring setup?
• Is a scaling plan in place? Can you scale?
• What happens to the app under load?
• What slows down first?
• Are things alerting as expected?

As Load Tester, you are responsible for…
• Ensuring those involved with the system under test are
aware and available
• Putting the system under test under load
• Completely destroying the system under load
• Providing a clear and concise write up of exactly what
occurred
• Take charge! It’s fun!

Reporting Results: Shortest Version
• 99% @ 59ms

Reporting Results: Short Version
• Short version: During an hour starting with 0rps to 1400 rps
in the first 10 minutes....
• ...when starting with 5 instances and scaling to 11
instances, the response times were: 50% 23ms, 75% 54ms,
95% 303ms, and 99% 1069ms.
• ...when starting with 11 instances, the response times
were: 50% 16ms, 75% 24ms, 95% 45ms, and 99% 59ms.

Reporting Results: Detailed Version
• Provide more information
• Results
• Story driven
• Have data supporting results prepared in case
• Account for changes such as auto-scaling
• With auto-scaling: 99% at 1069ms
• Without auto-scaling: 99% at 59ms

Testing elasticsearch on c3.4xlarge
Detailed Reporting
Also included in this report
was a link to the actual
GitHub repository. Make sure
you are keeping your load
tests in version control!

Tag your load test in git
Associate the tag with a Load Test,
so you can replay any load test easily

Things to Keep In Mind
• Understand expected usage
• X% of users using the app while
• Y% are chatting with one another
• Users are logging in
• Creating accounts
• Backend systems
• Determine what’s important

• User input
• Random filters
• Weighted filters
• Cached results are expected
• Client constraints

• User flow through service
• Try to understand how users use the app
• Script should try to mimic

• Be careful about testing a single system
• System will have logging
• System will have backend services
• You’d be surprised what can cause failure
• A load test helps you learn before it’s in production

• User interaction
• MeetMe is social, so we’ve load tested chatting
• 1 test per 2 users, both chatting with one another

• Have developers available
• Better, have developers with you when load testing

• Find a problem, Fix it, Reroll app, Rerun test
• FFRR, or F2R2
• I just made that up
• Don’t use it.

• Start testing from your laptop
• Seriously, my MacBook Air could bring down Erlang
services
• Database indexes are a thing
• While running a load test, you can run a single client form
your laptop

• Someone should be testing the app/service as well
• Response times are only a number
• What does 50ms vs 300ms response times feel to the user
• What impact does 2x/3x/4x load have
• When auto-scaling, how does the client handle

• Review how the client is using the API
• Review how the API developer expects the client to use the
API
• Model after what the client is doing
• Call out differences early

Thanks!
• @jasonlotito
• jasonlotito.com
• jlotito@meetme.com
• https://joind.in/talk/80910

Load Testing with PHP and RedLine13

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (12)

Similar to Load Testing with PHP and RedLine13

Similar to Load Testing with PHP and RedLine13 (20)

Recently uploaded

Recently uploaded (20)

Load Testing with PHP and RedLine13