Increasing usage of web based application has raised a need to test the performance of web-based applications for speed i.e. responsiveness of the application, scalability i.e.  sustainabilty of an your web application at peak load or maximum users load and stability i.e stable state of the system under peak load. In order to  conduct these performance tests against web applications we need to bombard the system with humongous amount of load in terms of requests and monitor system differently depending on the goal of the test. One of the most widely used open source tool for performance testing is JMeter. I will be covering JMeter in detail right from recording to most of commonly used JMeter components like Transaction Controller, Authorization manager and also issues with recording https requests and so on.

What is JMeter?

Apache JMeter is an Open Source tool mainly used to test performance both on static and dynamic resources (files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more). It can simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types. It can also be used for graphical analysis of performance or to test your server/script/object behavior under heavy concurrent load.

Downloading JMeter

You need jdk 1.6 in order to use JMeter 2.8 and above. Make sure that you have made appropriate entry in PATH variable.

You can get latest JMeter build from: http://jmeter.apache.org/download_jmeter.cgi

Launching JMeter

JMeter UI can be launched in two ways:

1. Open JMeter folder -> Bin -> Double Click on JMeter.bat file
2. Open command prompt and go to Apache JMeter directory using cd command.

And type JMeter.bat

If you are working behind proxy, you are bound to get “socket connection” exception if you launch JMeter as mentioned above. You need to launch JMeter from command prompt as below if you are working behind proxy.

JMeter.bat  –H “proxyname” –P “PortNumber” –u “username” –a “Password”

This will resolve initial “socket connection” exception caused due to proxy.

You will see JMeter UI as below once you launch JMeter using any of the above mentioned ways.

When you launch JMeter it has TestPlan and WorkBench by default. We will look into these components one by one.

What is a TestPlan

Test Plan usually contains overall the setting for your JMeter script. You can add user defined variable which can be referred across all the threads coming under the test plan.

You can see three check-boxes when you click on TestPlan.

1. Run Thread Consecutively: By default all the threads coming under test plan run concurrently. If you check this option threads will run one after other.
2. Run tearDown Thread Groups after shutdown of main threads: if selected, the tearDown groups (if any) will be run after graceful shutdown of the main threads. The tearDown threads won’t be run if the test is forcibly stopped
3. Functional Test Mode: If you check this option JMeter will save additional sample information – Response Data and Sampler Data to all result files affecting the performance of system.

What is a WorkBench?

The WorkBench simply provides a place to temporarily store test elements while not in use, for copy/paste purposes, or any other purpose you desire. When you save your test plan, WorkBench items are not saved with it

Certain test elements are only available on the WorkBench:

1. HTTP Proxy Server
2. HTTP Mirror Server
3. Property Display

Now let’s begin with recording in JMeter:

You need to add a ThreadGroup to TestPlan and add HTTP Proxy Server to WorkBench as first step to begin with recording.

We will talk in detail about ThreadGroup and HTTP Proxy Server later. For now make the settings as shown below to begin with recording.

Configuring your web browser to record requests using JMeter

You need to modify your connection settings so that JMeter will capture all the requests which are being sent from your browser to target application. Modify your browser settings as shown below.

Make sure that the port number mentioned in JMeter HTTP Proxy Server and Port number mentioned in connection settings should be same.

Now go to JMeter UI, select HTTP Proxy Server and click on start button as shown below.

Now go to your already configured web browser and type url of the target application. All the requests will be captured in TestPlan->ThreadGroup as you record your workflow as shown below:

Follow above mentioned steps and am sure you shall be able to record HTTP requests using JMeter.

How to customize the Oracle VirtualBox to support screen resolution you want? Here are the steps to follow -

On windows

• Open command prompt (Windows key + R -> cmd -> hit enter)
• Navigate to folder where your VirtualBox is installed. Generally it is installed at -

  "C:\Program Files\Oracle\VirtualBox"

cd "C:\Program Files\Oracle\VirtualBox"

• Use following command to get exact name of virtual machine-

VBoxManage list vms

• The custom resolution can be set for your VM by running following command -

VBoxManage.exe setextradata {VM NAME Obtained in previous step} 

CustomVideoMode1 1366x768x32

• Power On the virtual machine and you will see the custom resolution on the resolution screen.

On Linux

On Linux machine, its relatively simple to setup custom resolution.

VBoxManage setextradata {VM NAME} CustomVideoMode1 1360x768x32

Enjoy!

Here are the notes from that session –

What it was about (Satyamev Jayate)?

• Link - http://www.satyamevjayate.in/
• Some of the topics on which people were not comfortable discussing in Indian society
• Format
  • 90 mins show
  • Telecasted  at 11 AM  to 12 PM in morning which was prime time 10 years ago but not these days
  • Contained very thorough statistics and research about the topics
  • Simultaneously telecasted in multiple languages and had unbelievable reach in India

Why analytics was necessary?

• Key was understand the pulse of audience
• Goal was not to have revolution in society but the main focus was to change the perception of individual
• To measure the impact of the show. Have audience been touched? How to measure the success?

Some Statistics about the show-

• Most talked about show in Indian TV shows history
• 13 episodes, Television reach – 500 million
• Over billion impressions (FB, SMS, and other social media)
• 64 million engagements
• 1.45 million Facebook fans for just 13 episodes. (IPL – 1.2 million fans after 5 seasons)
• Was top of twitter trends in India every week show aired and twice in global trends
• 843 cities in India and 5435 cities across world
• Approx. 1.2 billion connections, 15 million+ responses,  8 million+ community members
• Average response content had average 100 words per message. Rich and quality content!
• Read More - http://www.satyamevjayate.in/impact/impact.php

Data Analytics helped the show close the loop with audience?

• Understand how sentiments of people came off. Example: first episode – 99% of people responded positively. Data was sent to Chef Minister of Rajasthan and he took the action
• To measure impact of the show on the society
• Analyze the response with different demographics

How they did it? – The persistent story

• Stakeholders Expectations-
  • Amir khan productions –
    • Whether goals met or not?
    • Responses, sentiments
  • Star India –
    • Viewership audience/analysis
    • Impact of the show
    • Comments/criticism
  • Satyamev Jayate Field research team
    • 360 degree view of social issue
    • In-depth analysis of the topic
    • Accurate and instant statistics

• Weekly cycle
• Sunday – 90 mins show. Collect the information from various social media channels (Twitter, Facebook page etc.) after the show
• First deliverable of the team – on Tuesday or Wednesday on show happened last Sunday
• Publish the story on website
• Featured feedback
• On every Friday –
• Finish all analysis by Friday. Includes -
• Social graph of influencers
• Detailed analysis about show happened last week
• Show impact analysis by Amir Khan where he would discuss the impact he seen on society about the topic discussed of last week

Workflow

Taken from the webinar slides

Get the content from various mediums

• Aggregate
  • Social media/websites – > Bots to crawl and store data into database
  • What technology used? –
    • Separate connectors for every social platform. API to pull data from Twitter – Data partner, Facebook – Directly from page through APIs
    • SMS/IVR- tie-ups with providers. Directly pull data through APIs
• Processing Filtering the content, Ranking the content
  • While show was going on live content from twitter was used as a baseline and analyzed to create tag taxonomy
  • Almost 80 tags for every show. Can analyses these on various demographics
• Sentiment and emotional analysis
  • Every message was assigned the sentiment analytics score
  • Live insights were shared after every hour
• Visualization
• Challenges
  • Unstructured data – How to tag/rank them?
  • Every show was different topic – Tagging was extremely difficult
    • Topic was out on Sunday morning
    • Need to create tag taxonomy
    • Mix of the languages – Hinglish. Adopt the algorithms to mixed sentences
• How regional languages were dealt with?
• IVR channel – Manual transcribers for Indian language as no speech to text technology available for Indian languages
• Most of the words were Hinglish – > Create custom dictionary and custom platform to analyses
• Unique algorithms used?
  • Topic evolution – Identifying the keywords and identify where discussion was evolving
  • Taxonomy identification – automated approach
  • Ranking and sentiments – Mostly sentiment analysis algorithms used.
  • Influence analysis – Mix of visual + computational techniques
    • Basic scoring technique
    • Visually identify the influencers
• Data storage?
• Every box in diagram has their own storage of HDFS
• Charts were stored on MySQL databases
• Cloud sourced platforms has their own RDBMS and UI
• Visualization tools Key was to check if the goals of the show were met or not?
  • Persistent built Google clustered map maker to show audience response across the world
  • How every topic response in every part of India/gender. (Different dimensions of demographics)
  • Dashboard showing best content in terms of videos/stories (ensure that Content is not biased) (updated hourly)
  • Advanced analytics dashboard and visualizations –
    • Impact calculation of show
    • Detailed dashboard for Amir Khan Production house for analysis and impact calculations
    • Example: How young audience reacted to the show?
    • Animated 3D globe (available on website)
• Reporting tools
  • Quick view
  • Little series of JavaScript solution for UI on websites
• Amount of donations? And mechanism?
  • People sending SMS as nominal grade. Revenue generated from SMS went for that cause
  • Website – Enabled people to donate the money for that NGO
  • Reliance foundation – Every money collected, they will equal the same amount and donate it
  • Used Axis bank’s payment gateway

Update on 27th Sep 2012: The recording of webinar is available here

Yesterday, I wanted to browse through the links saved in my skype chat done from another machine. As skype stores all its history locally and does not push it to cloud, it was not possible to retrieve chat history from another machine. After few hours of brainstorming and googling I got this idea to push chat history to cloud.

Here is the trick if you are using ubuntu as a primary OS-

1. Install Ubuntu One if it’s not already installed

You can do this simply by searching the Ubuntu one application from dashboard. After system pops up with option to install Ubuntu One-

2. Start syncing Skype history

Skype stores all your chat history locally in ~/.Skype folder. Navigate to this folder and then enable sync for this folder through Ubuntu one menu.

3. Enable download and sync on machine 2

After sync is complete, login to another machine and start ubuntu one. The .Skype folder will appear in the folders list of Ubuntu one. Check “Sync locally” option for the .Skype folder.

After download is complete, you can now access the chat history from any ubuntu machine!

Like this trick? or have something to say? comment it down!

CloudStack manages two system VMs for it’s internal processing – Console proxy VM (CPVM) and secondary storage VM (SSVM). Console proxy VM is used by CloudStack to provide console access of the user instances whereas Secondary Storage VM (SSVM) downloads and maintains VM templates. For enabling advanced networking functions such as load balancing, port forwarding and VPN CloudStack maintains a virtual router (small capacity linux based instance).

These instances are not accessible from public Ips and can be accessible only through management server or through the host on which instances are spawned.

For XenServer based zone, console access to system VMs and virtual router are accessible by logging into the host and then executing following command in shell-

ssh -i /root/.ssh/id_rsa.cloud -p 3922 root@169.254.x.x

Where,

/root/.ssh/id_rsa.cloud – Public key for SSH access (Note that, system VMs/Routers are only accessible using SSH keys)
3922 – Port for SSH. Default port used by CloudStack for accessing system VMs/Router
169.254.x.x – Link local IP. Can be found by logging into management server on System VM details page

CloudStack System VM Information

For VMWare based zone, console access to system VMs and virtual router are accessible by logging into the management server and then executing following command in shell-

ssh -i /var/lib/cloud/management/.ssh/id_rsa.cloud -p 3922 root@PRIVATE_IP

Where,

/var/lib/cloud/management/.ssh/id_rsa.cloud – Location of the public key stored on management server
3922 – Port for SSH. Default port used by CloudStack for accessing system VMs/Router
PRIVATE_IP – CloudStack does not have a link local Ip for VMWare based instances. Instead, it uses private IP range specified while setting up the zone.

Update: As David Nalley pointed out in comments, we can login to SSVMs in KVM based zone using same steps mentioned for XenServer.

Like this post? Or have something to say? Comment it down!

Now days we all are addicted to Google let it be a developer, businessman, architect, engineer or any other professional person. Its use has been intensified in every walk of life. Also with its massive data chunk and billions of indexed web pages, one has to effectively provide a search query (keywords) to get the desired search results.

So the question arises what are the different techniques rather ways to specify a search query. Let’s dig into some points in this regards:

Using the Keyword ‘Site’:

This keyword restricts the search to particular site type like .com, .edu, etc. It can also be used to narrow down search on a particular site like google.com. Let’s have some examples:

• We need some important contact information of “AIIMS” educational institute. To narrow down the search we can employ the help of ‘site’ keyword as given below:
  Phone aims site:edu
• We need to find tuition fees of MIT university
  Tuition fees site:mit.edu

Searching for exact phrase:

To get an exact search result, one can specify a phrase (search keywords between quotes “”) instead of using separate words. These search keywords inside quote (“”) are presented as a single unit to Google search algorithm, enabling it to narrow down our search results. Here are some examples:

• Need Google API wiki page. Using phrases the search can be narrow down to results which contains words “Google API wiki”, eliminating results with separate words
  “Google API wiki” page – Returns 37,000 results
  Google API wiki page – Returns 54, 800, 000 results
• Need use case diagram of ATM machine
  “use case” diagram ATM

Excluding some terms from search results:

The Hyphen (-) is used to exclude a particular term from search keywords. Google by default uses ‘AND’ operation on all the keywords provided, that is in short it matches all keywords provided. But when we append hyphen operator with any keyword, Google simply excludes the term. Let’s have some examples to get a detailed picture:

• Suppose we need information on photography but excluding Wikipedia photography -site:wikipedia.org, Google presents us search results with excluded Wikipedia info

Search results with similar keywords:

When we need search results to be more generic and should cover words similar in meaning to the keywords, we can use negate (~) sign to indicate it. This can be demonstrated by the search term given below:

• ~college pune , the negate sign appended before the keyword ‘college’, enforces Google to include similar terms like ‘higher education’, ‘school’, ‘university’, etc in it.

Search results with particular document type:

Sometimes we need information in particular document format like pdf, excel, doc, etc. To scrutinize are search results for particular document type use ‘doc:’ keyword. Its use can be demonstrated by:

• “robotics controller” doc:pdf , Google find all pdf document which
  contains words robotics controller.

Using logical operators:

Google also allows us to use various logical operators like OR, AND, wildcard, parenthesis or ranges. Let’s check how it goes:

• “NASA discovery shuttle” 1999..2006, Google finds all info about discovery shuttle which were published or occurred from 1999 to 2006
• Nasa (*earth like planets OR discovery mission) 1999..2006, Google searches info about NASA’s discovery mission or NASA’s mission to find another earth.

Google as a calculator:

Google’s special query is so versatile that there is no need to open calculator for numeric expressions, just enter your expression (3892 / 89 * (90 + 23)) in Google search bar and your result will be ready.

Some other quick searches:

• Finding definition: Use keyword ‘define:’ Example: define: constipation
• For unit conversion: 1$ in INR or 400 gram in kilograms
• Search a book by author:  use keyword ‘author:’ Example: triton author:Randolph Lalonde

What’s your favourite Google search technique? Tell us about it in comments.

Ever wondered whether you can access Facebook via command line? Is there any command line interface (CLI) available for those people who like to stick with terminal window? The answer is fbcmd.

fbcmd is command line application available for Linux flavours as well as Windows. I admit, accessing Facebook through browser window or mobile devices is a lot easier. but, what’s the typical use-case of fbcmd is- you can now schedule your status updates to be published at particular time with help of cron jobs. Sounds cool..isn’t it?

Installation and use is very simple for those who love to fire commands in their terminals.

Prerequisite

You must have PHP5 installed on your system to install fbcmd. To check whether PHP is already installed on your computer type “php -v” in terminal. If installed, the output will be the version installed on your computer else use the command below to install PHP.

If Linux flavour is Ubuntu,

sudo apt-get install php5

If you are using centos,

yum install php5 php-cli

Installation

Obtain the fbcmd_udpate.php program by executing:

curl -O https://github.com/dtompkins/fbcmd/raw/master/fbcmd_update.php

Execute the following two commands:

curl -O https://raw.github.com/dtompkins/fbcmd/master/fbcmd_update.php
sudo php fbcmd_update.php
sudo php fbcmd_update.php install

Initial Test

1. Try the command:

fbcmd

The output should be-

Welcome to fbcmd! [version 1.0-beta5-dev1]

This application needs to be authorized to access your Facebook account.
You are almost done! This application needs to be authorized to access your Facebook account. This is a one time process.

2. Allow basic (initial) access to your account via this URL:

fbcmd go access

3. Generate an offline authorization code

fbcmd go auth

4. Obtain your authorization code (XXXXXX)

fbcmd auth XXXXXX

5. Grant additional permissions to fbcmd

fbcmd addperm

Commands:

To get syntax of any command in fbcmd use-

fbcmd help

Set your status update

fbcmd status “Accessing FB through console”

Get your profile ID

fbcmd whoami

Fetch (and optionally save) profile pics of all the friends

fbcmd ppics =all /tmp/fbcmd/ppics/

Export all your friend’s B’day date in csv format

fbcmd finfo birthday_date -csv

Get list any friends who are currently online

fbcmd fonline

There are many other commands available, all of them are listed on fbcmd wiki page.
Like this article or have something to say? Comment it down

When going from one language to another, some things have to be unlearned. What you know from other languages may not be always useful in Python. Over time, a lot of idioms and ideas arose in the Python community about what the right way to use Python is, and also what would be a wrong way. One of the major part of “pythonic” way of coding is making code more readable. In this article, we will discuss the various tools available online/offline for formatting python code according to international standards (See Pep8)

Eclipse (Pydev):

Eclipse has a very useful plugin – Pydev, for python development. The recent release of Pydev comes with code formatting and style guide. Pydev also includes Pep8 validations for your code. If properly configured, eclipse can be used to code in python with international standards (like pep8).

1. To configure Pep8 within Eclipse follow the steps below-
2. Open your Eclipse
3. Go to Help and select Install New Software
4. Click the Add button and a “Add Repository” Dialog box will appear
5. You can use any name you like for it.
6. For the location, enter “http://pydev.org/updates”
7. Click Ok.
8. You are now in the process of installation. Just wait for it to finish.
9. After the installation, close Eclipse and Open it again.
10. Now that PyDev is installed in your Eclipse, go to Window->Preferences
11. Choose PyDev->Editor->Code Analysis
12. Go to pep8.py tab
13. Choose the radio button for warning and click Ok.

The code style warnings will be shown, if any, each time you do the code analysis (Right click on project folder ->Pydev -> Code Analysis)

Pylint:

Pylint is a lint-like tool for Python code. It performs almost all the verifications that pychecker does, and additionally can perform some stylistic verification and coding standard enforcements. The checked code is assigned a mark based on the number and the severity of the encountered problems. The previous mark of a given piece of code is cached so that you can see if the code quality has improved since the last check.

PythonTidy:

A command line utility for beautification and formatting of Python code. It’s a simple python program which indents your code. If you need consistent indentation across all the python files in project this PythonTidy is best utility.

Pyindent:

Another command line utility for indenting python code. This file performs three basic operations on python programs. When called as “pindent -c”, it takes a valid Python program as input and outputs a version augmented with block-closing comments. When called as “pindent -d”, it assumes its input is a Python program with block-closing comments and outputs a comment-less version. When called as “pindent -r” it assumes its input is a Python program with block-closing comments but with its indentation messed up, and outputs a properly indented version.

Reindent.py:

Command line utility for indenting has a tokenizer allowing it to do smart reindenting based on the indentation level rather than on the number of spaces osed per level (which can vary in bad code)
It comes with following options-

reindent [-d][-r][-v] [ path ... ]

-d (–dryrun) Dry run. Analyze, but don’t make any changes to, files.
-r (–recurse) Recurse. Search for all .py files in subdirectories too.
-v (–verbose) Verbose. Print informative msgs; else no output.
-h (–help) Help. Print this usage information and exit.

Do you know any other better python code formatters? Feel free to share it with our readers in comments.

Have you noticed that blank status on friend’s profile? or someone in your friend’s list tagged with some different name?

Curious..? Well, here are those awesome tricks..

Tag someone with different name

It’s simple-

Copy and Paste the below line in your status or comment box

@@[0:[235959506635:0: Any text you like Here]]

Where,

235959506635 : Replace this with the profile ID or page you wish to Tag. But, now a days most of the profiles and pages have set their custom usernames, page/profile ID is not available that easily. You can use Facebook’s Open Graph API to find ID. Simply add ‘graph’ before www in your address bar and hit Enter. You will get a JSON response with parameters like “id”, “name”, “picture” etc.

Copy the number next to “id” field which is a profile/page ID.

Any text you like Here: Replace this text with any other text you would like to display for that link.

NOTE: After recent changes, Facebook removed support for this trick.

Blank Status on Facebook

Facebook users can update their status as blank by following trick.

• Copy paste @[0: ] to your status bar
• Click update.

Your Facebook status will be now updated with blank message.

Like this article? or want some more tricks like this? subscribe to our mailing list for more updates

Facebook recently launched a bunch of enhancements that impressed most of us. The series of changes like smart lists, chat, timeline just put Facebook ahead of it’s competitors. But, there are few bugs in new interface which Facebook team might have missed.

I have already reported these to Facebook and probably they might have started working on them. but, I thought you might be interested in what are those bugs.

Bug #1

Smart lists allows us to merge custom lists into those created by Facebook with the help of your profile information. For Ex. previously before smart list rolled out for me, I had created a custom list of those people who graduated with me from PICT. Now, after Facebook introduced Smart lists, I have an option to merge my existing custom list, i.e. “PICT”, into the one created by Facebook, say “Pune Institute of Computer Technology, Pune”.

Merging custom list will move all your friends to new list and will delete custom list. So, after merge my new list is “Pune Institute of Computer Technology, Pune” and the older one was scrapped by Facebook.

Problem here is, the new “Smart” lists are not yet integrated with Facebook chat, the result is you cannot chat with all of your friends in that new list.

Bug #2

Now, after loosing a way of contact with those friends you might be thinking of restoring your existing custom list. But, Facebook haven’t gave you copy or delete rights of these smart lists yet. So, the only way of restoring a custom list is to create one and one by one add your friends in it, a tedious task!

Bug #3

Even complete strangers can also view partial contents on someone’s wall; like comments by that person on friend’s status updates, photos in which he is tagged in, his posts on friend’s wall etc etc. This might evade privacy, though Facebook provides an option to hide all the comment activity from wall but, this will hide it even from your friends.

Bug #4

Old interface allowed us to filter by “Only Status Updates”, which was indeed very useful in reducing noise in updates stream. In new interface, that option just disappeared. This might not sound like a bug but, yeah I feel most of the people will be missing that option.

Hope someone from Facebook team take a note of these. The new features like timeline look equally promising, I wonder where they are going to integrate it though.

As an “addicted” Facebook user you might have noticed some bugs or some features that need enhancements. Feel free to share ‘em here.