Website and WebServices load testing using JMeter

JMeter can be used to load test web application and Web services (SOAP/REST). Following steps and screenshots will help you configure JMeter (assuming you have JRE installed and class path set)

      For Website with a typical scenario of opening a homepage and logging in using username/pwd,         the following steps needs to be done.

a)       Add the “Thread group” (virtual users). Ramp up time is – It will take that amount of time to load specified max. number of the threads (users). Example – 100 users in 5 seconds.

b)       Add “Http Request Default” to set the server name and port etc (information which is independent of the other steps)

c)        Add “Http Cookie Manager” if your site uses cookies.

d)       Add the step to open the homepage. (HTTP Request)

e)       For a site which uses anti-forgery token, we need to retrieve the token from the hidden field. This can be done using the Regular Expression Extractor on the response of the step-d.

f)        Add the step to login. (HTTP Request) with the form fields. The form fields could be found using fiddler, (if not already known).

g)       For results, add “View Result Tree” and “Aggregate graph”.

h)       Press Ctrl + E to clear the results.

For web services, similar setup needs to be done (no need of “HTTP cookie manager”). Instead of “HTTP Request”, we add the “SOAP/XML-RPC Request” where we specify the full request body (SOAP) .

Then we add the “HTTP Header Manager” (to pass soap action and content type) and  “Response Assertion” to check the response value and compare.

Below are the links to JMeter files (jmx files) for load testing web sites and webservices

JMeter file for Website loadtesting

JMeter file for Webservice loadtesting

--------------------------------------------------------------------------------------------------


Example of simple regular expression extractor – 

Case – when you need different input for each request.

You can use CSV data config element as shown below – 

 

Jmeter when asked to run more than 500 threads can crash because the default heap size can be exceeded. This can be fixed by starting jMeter (inside jMeter.bat) with a high heap size (upto 80 % of system’s RAM).

Below, I have commemted default settings and added new (for a 32GB RAM machine) –

rem set HEAP=-Xms512m -Xmx512m

set HEAP=-Xms22528m -Xmx22528m

jMeter when used in GUI mode can take lot of resources. You can use the following command line to run jMeter in non-GUI mode.

C:\Apache-Jmeter-2.13\bin>jmeter -n -t "C:\MyDir\MyJMX.jmx" –l "C:\MyDir\Results\resultsfile.jtl"

To jMeter running in non-GUI mode, run stoptest/shutdown.cmd in a separate command prompt.

The resulting jtl can be opened in jMeter by adding listeners like “View Results tree”, “Aggregate graph” and clicking on “browse” button to load the jtl file.

By default in the non-GUI mode, the request and response are not logged. To enable this, you have add a few options to the jMeter command line.

C:\Apache-Jmeter-2.13\bin>jmeter -Jjmeter.save.saveservice.samplerData=true -Jjmeter.save.saveservice.response_data=true -Jjmeter.save.saveservice.output_format=xml  -n -t " C:\MyDir\MyJMX.jmx " -l " C:\MyDir\Results\resultsfile.jtl "

Using jMeter for a flat load of (say 20 users) for 20 mins can be done using scheduler as shown below.

This is needed for soak test. Remember – A system can touch a peak load of 1000 users without any errors but when subjected to a low load (of 20 users) but for several minutes can take the system down.

Implementing long running processes using Service Bus

Following design pattern can be used to implement a long running process with multiple independent transactions.

Scenario – On the click of “Submit”, Call to the payment gateway (via PaymentService) needs to be done, Address is to be updated (AddressService) and MobileNumber is to be updated (UserProfileService).

Assumption – 1) Call to AddressService.UpdateAddress and UserProfileService.UpdateMobileNumber is idempotent.

2) Retry logic is not implemented.

Caution – Duplicate payment scenario needs to be taken care of.

Step1) On the click the “Submit”, Capture all the details like Customer Id, new address, new mobile number, accountnumber, Bank IFSC code, amount, and unique RequestId  etc, create a XML message and send it to ServiceBus topic/queue via ABCService. On receiving the success message from ABCService, Show message to the end user, that “Your request is successfully recorded and the changes will be reflected shortly.”

Step2) There will be another service ABCTransactionService, which is auto-started and listening to the ServiceBus topic/queue for any incoming message.  As soon as the message in Point 1 arrives, it is processed by calling the PaymentService, UserProfileService, AddressService and recording their status in a TrackingTable.

Details below -

The structure of the TrackingTable will be –

·         RequestId (Primary Key)

·         CustomerId

·         IsCallingPaymentService (Flag set before calling payment gateway)

·         IsPaymentSuccessful

·         IsAddressChangeSuccessful

·         IsMobileUpdateSuccessful

·         IsRequestComplete

Processing Detail –

a)      Read message from ServiceBus topic/queue.

b)      Check if IsCallingPaymentService is false or empty. (empty/false for a fresh case)

c)       If IsCallingPaymentService  is false or empty, Update IsCallingPaymentService to true and Call  PaymentService.

d)      Else Abandon the message to send it to the deadletter queue and process manually. This approach is needed to prevent duplicate call to PaymentService (deduct from Customer account twice).

e)      Capture the response of PaymentService and update the IsPaymentSuccessful to true/false depending on the response code. In case of exceptions (timeout,communication failure), Update IsPaymentSuccessful to false. If the IsPaymentSuccessful is false, abandon the message (manual check required) else proceed.

UpdateAddress and UpdateMobileNumber is not critical and are idempotent. Hence we are not capturing the flag before the call is happened. The “before operation” flag is needed in the following case – Call to PaymentService is successful but Call to TrackingTable for updating flag failed (because of DBConnection!!). In this case, we might think that Call to PaymentService never happened (as per the tracking table,  IsPaymentSuccessful is empty), and call it again during manual processing (unless someone checks the logs in detail).

f)       Call UpdateAddress and capture the response. Update the response in TrackingTable – IsAddressChangeSuccessful. In case of any failure/exception abandon the message to move it to the deadletter queue.

g)      Do the same for UpdateMobileNumber operation.

h)      Update the IsRequestComplete to true and de-queue the message. (receivecontext.complete)

FAILURE SCENARIOS –

1)      Go to Deadletter queue to see if there are issues.

2)      Check the RequestId and find the values against that requestid in the TrackingTable.

3)      Check the values of IsPaymentSuccessful=true and IsCallingPaymentService=true to see if the payment was successful. Check logs if payment failed.

4)      To reprocess the message, Open ServiceBusExplorer and go the Deadletter queue and resubmit the message to the required queue/topic.

5)      We will be safe to reprocess the message as Call to PaymentService is will not go (if already called) – Manual check of logs required in case of Payment failure. Other operation can be  called anyway because of idempotency.

Create a MVC site in Visual Studio with an empty website template (and not using MVC template)

Creating a MVC website in Visual Studio using the templates available makes it bloated with tons of JS, nuget package references and other stuff. Following are the steps to make a MVC  site in Visual Studio with an empty website template.


1) Create an empty Web Application with nothing in it. (no Global asax, no entry in web.config, no JS)
2) Add Microsoft.AspNet.Mvc nuget package to the project.
3) Add a Global.asax file and insert these lines
        protected void Application_Start(object sender, EventArgs e)
        {
            ViewEngines.Engines.Clear();
            ViewEngines.Engines.Add(new RazorViewEngine());
            RouteConfig.RegisterRoutes(RouteTable.Routes);
        }
4) Add a App_Start folder with RouteConfig.cs
public class RouteConfig
    {
        public static void RegisterRoutes(RouteCollection routes)
        {
            routes.IgnoreRoute("{resource}.axd/{*pathInfo}");

            routes.MapRoute(
                name: "Default",
                url: "{controller}/{action}/{id}",
                defaults: new { controller = "Home", action = "Index", id = UrlParameter.Optional }
            );
        }
    }
5) Add folders - "Controllers","Views","Views\Home".
6) Add a file - "HomeController.cs" under "Controllers" folder with the following content -
 public class HomeController : Controller
    {
        public ActionResult Index()
        {
            return View();
        }
    }
7) Add a file - "Index.cshtml" under "Views\Home" folder with the following content -
<h2>Home Index View</h2>
8) Add a web.config under "Views" folder and TAKE CARE OF THE VERSIONS -

<?xml version="1.0"?>

<configuration>
  <configSections>
    <sectionGroup name="system.web.webPages.razor" type="System.Web.WebPages.Razor.Configuration.RazorWebSectionGroup, System.Web.WebPages.Razor, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35">
      <section name="host" type="System.Web.WebPages.Razor.Configuration.HostSection, System.Web.WebPages.Razor, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" requirePermission="false" />
      <section name="pages" type="System.Web.WebPages.Razor.Configuration.RazorPagesSection, System.Web.WebPages.Razor, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" requirePermission="false" />
    </sectionGroup>
  </configSections>

  <system.web.webPages.razor>
    <host factoryType="System.Web.Mvc.MvcWebRazorHostFactory, System.Web.Mvc, Version=5.2.3.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />
    <pages pageBaseType="System.Web.Mvc.WebViewPage">
      <namespaces>
        <add namespace="System.Web.Mvc" />
        <add namespace="System.Web.Mvc.Ajax" />
        <add namespace="System.Web.Mvc.Html" />
        <add namespace="System.Web.Routing" />
      </namespaces>
    </pages>
  </system.web.webPages.razor>

  <appSettings>
    <add key="webpages:Enabled" value="false" />
  </appSettings>

  <system.web>
    <httpHandlers>
      <add path="*" verb="*" type="System.Web.HttpNotFoundHandler"/>
    </httpHandlers>

    <!--
        Enabling request validation in view pages would cause validation to occur
        after the input has already been processed by the controller. By default
        MVC performs request validation before a controller processes the input.
        To change this behavior apply the ValidateInputAttribute to a
        controller or action.
    -->
    <pages
        validateRequest="false"
        pageParserFilterType="System.Web.Mvc.ViewTypeParserFilter, System.Web.Mvc, Version=5.2.3.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35"
        pageBaseType="System.Web.Mvc.ViewPage, System.Web.Mvc, Version=5.2.3.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35"
        userControlBaseType="System.Web.Mvc.ViewUserControl, System.Web.Mvc, Version=5.2.3.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35">
      <controls>
        <add assembly="System.Web.Mvc, Version=5.2.3.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" namespace="System.Web.Mvc" tagPrefix="mvc" />
      </controls>
    </pages>
  </system.web>

  <system.webServer>
    <validation validateIntegratedModeConfiguration="false" />

    <handlers>
      <remove name="BlockViewHandler"/>
      <add name="BlockViewHandler" path="*" verb="*" preCondition="integratedMode" type="System.Web.HttpNotFoundHandler" />
    </handlers>
  </system.webServer>
</configuration>




9) Open the root of the website.

Service Bus authentication and permissions

1)      Clients can only connect to the Service Bus, if they have the SB certificates in their Certificate Store. The ServiceBus certificates, if autogenerated, can be found using the cmdlet: Get-SBAutoGeneratedCA.

2)      Second we either need to use Shared Access Key or Windows STS Uri. If ServiceBus is installed in a domain and the client (Windows 7) is not connected to the domain (example – Corpnet), then its better to use Shared Access Key, which can be retrieved by the cmdlet: Get-SBAuthorizationRule. There are mainly 3 rights –Listen, Send, Manage.

3)      Third – To create Topics, Queues and other admin related task, the user must be a part of ManageUsers collection of the ServiceBus Namespace, which can be retrieved by the cmdlet: Get-SBNamespace.