Recent progress in the Software Automation Industry has shown a tremendous framework implementation in order to execute test cases for bringing out a better quality web application. As millions of people are spending many hours of time in Internet, due to which, the web applications are becoming more popular and increasingly more complex since the goal of it, is to impress and gain the potential market place. Thus the need for the code quality and maintainability has grown drastically and as a result, the need for testing of these applications has become more important. Testing the web application during the application development phase has become an integral part of the Software Development Life Cycle (S.D.L.C) and these tests are typically planned to check the functionality of the application.
As the web applications face a lot of enhancements, probability of conducting repetitive testing of the application functionality goes high and thus we go for test automation in place of regression testing. Test Automation is a strategic approach designed in a form of a framework to gain control over the execution of the tests.
The acceptance test of web applications often involves some manual tasks. Doing it manually, the operating personnel human error can occur easily. At the same, it wastes the time. The web-based applications were bearing the more and more complex business logic and more and more huge information platform formation, along with the characteristics of short release cycle and the quick regeneration. This requires the web application tests to be of comprehensiveness, expansibility and efficiency. The Browser carrying on web applications also was emerging in diversification and pluralism. It also increases complexity and workload for the web application testing. More attention should be paid to the web application testing efficiency, reusability, and comprehensiveness.
Common Web test automation tools, such as QTP, Win Runner Load Runner, Robot, SilkTest, Selenium are used widely in automation test. Selenium can be said to be the most comprehensive of open source Web test automation tools, especially well support to the Web application which employed the BackBase framework.
Implementation of the test automation framework shows that the test automation framework is convenient to carry on function test, performance testing, and loading test in web applications. It also shows that it has better test performance in reusability, extensibility, comprehensiveness, and feasibility.
As the web applications face a lot of enhancements, probability of conducting repetitive testing of the application functionality goes high and thus we go for test automation in place of regression testing. Test Automation is a strategic approach designed in a form of a framework to gain control over the execution of the tests.
There is a belief that proprietary tools make the automation process very effective, as it provides support for 3rd party add-ons but is very costly. Instead of investing millions of dollars in buying the proprietary tools, companies go for open source tools to make the automation testing process cost effective since there is a challenging task on the hands of the automation industry for automating the web applications with 0% cost.
1.1 Basic Concepts & Technical Overview
A. Selenium
Initially developed by Jason Huggins and released in 2004. Selenium is very famous as it is an open source suite of tools specifically developed for testing web applications. It supports a test domain specific language (DSL) to execute tests in many programming languages such as C#, Java, Ruby, Groovy, Python, PHP, and Perl.,
B. Selenium IDE
It is a Mozilla Firefox add-on which is capable of recording the user actions and also play-back. Since selenium IDE is an integrated development environment, Tester can also edit and debug the tests in above mentioned languages. Latest version of Selenium IDE is 1.0.10.
Fig. 1.1: Selenium IDE
C. Selenium RC
It is a server / solution for executing linear / regression / integration tests. RC obtains the commands from the selenium server and run the tests via the browser. Recorded user actions from the selenium IDE can be obtained in any supported programming language then imported to eclipse for test case/suite execution. It contains the client drivers such as dotnet, java, Perl, Php, python, ruby and the selenium server.
D. Eclipse Java EE IDE:
It is an extensible open source IDE. It provides a number of aids that makes programmers /testers write code much quicker and easier than using a text editor. This means that a person can spend more time learning Java, less time typing. Here eclipse IDE is user to write test cases and execute it.
Fig. 1.2: Selenium RC
Advantages of Selenium
Open source, free software
Easy Installation
Scripting Techniques :
Easy recording and playback of scripts
Modular scripts
Compatibility :
Multiple operating systems (Windows, Linux, Mac)
Allows cross browser testing (Record in FireFox, execute in IE)
No dedicated machine required for test execution (user can work in parallel).
Integration with third party tools.
Example : RTH Test Case Management Tool.
Drawbacks & Limitations
Lack of exhaustive formal guidance material
Only works with web based applications
Don't expect Selenium to work as you expect it to 100% of the time. For instance, sometimes it doesn't know when the page has finished loading (because of AJAX running in the background).
It can't use a proxy to connect to any website That's because Selenium, technically is already a proxy, and you can't really configure it to go through another proxy (at least not that I know of)
Buggy with IE, Opera, and even Google Chrome For the most part, most of the functionality interacting with the GUI will work in Firefox. Not so much with the other browsers.
You can't interact with flash elements and java applets
Selenium doesn't have an integrated spreadsheet for data driven testing (parameterization), always need to use external files for it.
Selenium IDE does not allow for conditional statements, loops, asserting HTML. Selenium RC has problems with modal dialogs, yet there are hacks, it's not completely flexible for it.
Just like any automated framework, there will be overhead and costs for maintenance, but overall I'm enjoying my experience with it.
QuickTest Pro (QTP)
QuickTest Professional provides an interactive, visual environment for test development. QuickTest Professional can automatically introduce checkpoints to verify application properties and functionality, for example to validate output or check link validity. For each step in the Keyword View, there is an ActiveScreen showing exactly how the application under test looked at that step. You can also add several types of checkpoints for any object to verify that components behave as expected, simply by clicking on that object in the ActiveScreen. You can then enter test data into the Data Table, an integrated spreadsheet with the full functionality of Excel, to manipulate data sets and create multiple test iterations, without programming, to expand test case coverage.
Data can be typed in or imported from databases, spreadsheets, or text files.
Advanced testers can view and edit their test scripts in the Expert View, which reveals the underlying industry-standard VBScript that QuickTest Professional automatically generates. Any changes made in the Expert View are automatically synchronized with the Keyword View. Once a tester has run a script, a TestFusion report displays all aspects of the test run: a high-level results overview, an expandable Tree View of the test script specifying exactly where application failures occurred, the test data used, application screen shots for every step that highlight any discrepancies, and detailed explanations of each checkpoint pass and failure. By combining TestFusion reports with Mercury Quality Management, you can share reports across an entire QA and development team. QuickTest Professional also facilitates the update process. As an application under test changes, such as when a "Login" button is renamed "Sign In," you can make one update to the Shared Object Repository, and the update will propagate to all scripts that reference this object. You can publish test scripts to Mercury Quality Management, enabling other QA team members to reuse your test scripts, eliminating duplicative work. QuickTest Professional supports functional testing of all popular environments, including Windows, Web, .Net, Visual Basic, ActiveX, Java, SAP, Siebel, Oracle, PeopleSoft, terminal emulators, and Web services.
QuickTest Pro has more options and they are easier to implement compared to WinRunner in our opinion. Due to the similarities in concept and features, an experienced WinRunner user can easily convert to QuickTest Pro and quickly become an efficient Test Automation Engineer.
Advantages
It is easy even for a non-programmer to understand QTP and start adding test cases.
Support for record and playback and ability to edit scripts after recording. Also different recording modes are provided in QTP viz. Normal, Analog & Low level.
Excellent Object Identification process / mechanism
Support for different addins like Java, Oracle, SAP, .NET, Web Forms, Siebel, PeopleSoft, Web services, Main frame (Terminal Emulator) etc.
Ability to let you enhance the existing tests even without the AUT (Application under test) through active screen.
Supports all popular Automation frameworks - Keyword driven testing approach, Data driven testing approach, Modular testing approach, Hybrid frameworks etc.
QTP comes with an inbuilt IDE, which is simple and easy to use.
QTP uses VBScript which is English-like and very easy to learn and gain expertise. We have plenty of resource available to learn VBScripts.
Microsoft Object model can be implanted in QTP easily (Example - Word document object, Excel Object, Outlook Object, ADO objects, File system objects, DOM etc)
QTP can be integrated with Test management tools like QC (Quality Center), Test director and also functional test tools like Winrunner. The test cases can be mapped to the automation scripts and be executed from QC (Quality Center) itself. Also, it can kick off Winrunner test execution from within.
Easy to maintain different types of suites viz. Smoke, Sanity, Regression etc.
It comes with loads of inbuilt properties and methods in QTP as well asinbuilt functions in VBScripts
Support for XML.
Use of Data tables/Excel files are easier and provides a variety of methods to play around with rows and columns.
Easy to maintain test iterations and data driving the tests through configurations.
Test reporting with all necessary details for analysis is provided.
Disadvantages
Cost is high - License and maintenance.
Cannot run multiple threads/instances - For example the Grid support available in Selenium, where we can run multiple instances of the application on different browsers at the same time.
Slow in execution when compared to even open source tools like Selenium.
You get support from HP only when you renew the license.
You need to buy different addins - Java, Oracle, SAP, .Net, Seibel, Peoplesoft etc.
1.2 Comparison: The Automation Process
Selenium
Automating the Selenium is complex. It requires multiple programs and components to work together for a complete and viable solution. Below are the minimum steps required to create and run data-driven test automation scripts in Selenium using Java. Note that the steps assume that all of the components have already been installed and configured to support test automation.
Launch Firefox.
Invoke Selenium IDE.
Record user actions.
Convert recorded script to Java via Selenium IDE.
Copy Java code from Selenium IDE to clipboard and paste it into Java class file in Eclipse IDE.
Resolve inconsistencies and compilation errors in Eclipse IDE.
Write custom logic to parameterize hard-coded elements.
Write custom logic to replace parameters with data from an outside source.
Write custom logic to iterate through the script.
Write custom logic to improve reporting capabilities.
Start up the Selenium RC server using the command prompt. Command: java -jar C:\seleniuim\selenium-server.jar
Run script from Eclipse IDE
Launch Junit test runner for Eclipse to view results. Without additional custom coding the errors will be displayed only in the form of Java exception stack traces.
QTP:
Automation with HP Quick Test Professional is straightforward. QTP is a complete and end-to-end software test automation solution that includes a built-in Object Repository (to capture and organize test objects), and a built-in data table interface (for test data management). Below are the minimum steps required to create and run useful data-driven test automation scripts in QTP.
Launch QTP.
Open a new test.
Specify starting URL and browser.
Record user actions.
Parameterize and data drive via built-in data-table and data-driving wizard.
Add rows in the data-table to create iterations; no looping required.
Run the script.
Upon completion, results are displayed showing pass and failure of each command. Screenshots are captured in the event of a failure.
LITERATURE REVIEW
Many software applications today are written as web-based applications to be run in an Internet browser. Selenium is a set of powerful different software tools working with many browsers, operating systems, programming languages, and testing frameworks each with a different approach to supporting automation test for testing web-based applications. JMeter is used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types. JMeter operates at the protocol-level, on the other hand, Selenium works at the user-level.In this paper, authors have designed an automatic software testing framework for web applications based on the Selenium and JMeter. With the use of the software framework, we efficiently improve the extensibility and reusability of automated test. The results show that the new software framework improves software products quality and develop efficiency. [01]
Selenium Remote Control (RC) is a test tool that simulates web browsers and supports six programming languages. In this framework, "selenium-java-client-driver-1.0.1" is used for driving the browsers using java. Selenium RC provides only basic functionalities using the client drivers. But there is an effective way of expanding the functionalities by implementing the thought process to capture user actions automatically, creates a customized logger file, generation of customized test summary report. On the later part, the paper will brief on SELENIUM IDE, ANT, ECLIPSE JEE IDE, JUNIT and JXL.[02]
Event driven software's take the sequence of events as an input changes their state and produces the output. There are various classes of the event driven software:
Graphical user interfaces (GUIs): They are used in washing machines, phones, cars etc. The software changes its state based on incoming events, produces the output and waits for the next input.
Web applications are used via the cell phones, televisions, computers etc. Users perform actions to interact with these applications that change their state, produces the output ant waits for the next user action.
Embedded software: It controls cars, modern buildings, elevators etc. The sensors send signals to the software which changes its state, sends output signals to control devices and waits for the signals
Software components: these components form the building blocks of the large software systems. The messages are sent from the one component to another component. Components react by changing their state, respond with the messages and waits for next message.
Network protocols, device drivers, database applications, robot software are also the examples of Event Driven Software.[10]
There are various testing techniques are exists because the different types of software require different types of testing. Many researchers have shown that the existing techniques do not apply directly to certain classes of Event Driven System such as GUI, network protocols, device drivers, embedded system, web applications etc. So testing them requires the development of new techniques. Many researchers have used state-machine models to test the specific classes of EDS. Researchers have developed the new representation of event driven system based on the event interactions.[11][12]
Jeff Offutt has developed the web application bypass testing technique. The common technique in web applications is to perform the input validation on client by using HTML attributes and scripting language like JavaScript. The most common activity of web applications is to validate the user's data. Bypass testing technique is used for the web applications. This technique skips the client side testing mechanism and automatically generates the input data to verify security, performance and other quality aspects of the web applications. Ricca and Tonella proposed an analysis model and corresponding test strategies for web page analysis. Kung et al. Have developed a model to represent the websites using graphs and provides the definitions for developing tests based on these graphs. Andrews et al. use hierarchical FSM to model the large web applications and test cases are generated based on FSM and using input constraints. [12] [16]
Kam and Dean have been concluded that there is no one single testing technique can be used to cover all bugs of web applications. So we required to integrate many testing techniques with each other to fully validate and verify the web applications. [13]
GUI testing is the process of testing software GUI to detect the application errors and faults which restricts the software required functionality. GUI has the different characteristics than the traditional software so the traditional testing techniques cannot be applied to the GUI. [17]
Characteristics of GUI are:
Object oriented software programming.
Graphical orientation.
Hierarchical structure.
Event driven software.
Many authors have developed a technique to perform automatic tests for GUI based applications. Genetic algorithm was used to automatically test GUI functionality. Reverse engineering technique was used to automatically test GUI functionality. A new testing technique ALT was developed to generate GUI test cases in batches. [14]
Researchers have concluded that there no any single testing technique which covers all the bugs of web applications. So we need to integrate many testing techniques. For GUI it was concluded that there is no any single testing tool which covers all testable items. [15]
PROBLEM STATEMENT
There are number of tools available which dynamically test the web based GUI. The different tools use different approaches to test the GUI. Mostly used tools are FEST, ABBOT, Selenium and QTP etc. Each tool gets help of other frameworks such as UISpec4j java library which provides the different inbuilt classes and an interface which helps to test the swing based GUI. This component works on static pages, means developer has to develop the test suite class and test cases depending upon the functional requirements. So it's not possible to test any swing based window. If you want to test the window then you have to create the test cases and have to develop the class according to test cases. In our testing tool we are not using any inbuilt API library or tools or even we are not generating classes for each window. So our tool works dynamically that is it can check any swing based application restricting some GUI components. So there is no need to generate separate classes for each window. The researchers have already developed different models for automated GUI testing and web application testing. Our specific contribution is to develop a single model for testing both GUI and web applications together.
Additionally existing systems demand core programming knowledge to create Project, package, method and class. Proposed system aims to automate this process relaxing the condition of having knowledge of programming languages. There are drawbacks and limitations of both QTP and Selenium as discussed above. Proposed system will try to build a framework on the top of the Selenium, which is open source- making the tool cost effective. Also it will provide most of the QTP features making Selenium based framework highly efficient and reliable.
PROPOSED SYSTEM
Researchers have developed the several testing tools for automated GUI testing and web application testing. Now it's a challenge to develop a single abstract tool for testing GUI and web applications. Many of the testers does not have complete java knowledge, so developing selenium framework or using selenium IDE, RC is very critical and time consuming job. So to reduce this criticality I am planning to develop selenium UI based tool which has below features:
This features for Base version 1.0:
This tool support selenium 1.0 and selenium 2.0 (web driver)
This tool is java based so we can deploy and use this tool on any of the platforms
Selenium support many browsers so in our tool we have given the facility to execute the script on different browsers
Script creation time can be reduce and tester can easily use this tool
Creating Project, package, Method and class using selenium Tool
Creating test flows using selenium tool
Data driven is the key features in this tool as you can pass n number of data through its features.
Importing data from excel and database
Test case flow wise report, HTML reports
4.1 Methodology
Design and Implementation
The information about the same type of web applications abstracts way from the web-based test automation framework and form into a single configuration file. By doing so, we ensure that test automation framework and the application to be tested should be avoided interfere with each other, relatively independent. The test automation framework divides diversity interface elements and polygonal interface elements into non-variance elements and variable collection of space. It ensures that this test automation framework can be compatible with the same type of many applications, does not need any modification.
I am proposing a test automation framework by employing selenium test tools with supporting a variety of Web browser for solving web applications compatibility across the browsers (Fig. 4.1). It provides testers with a simple interface. It lets testers to configure different browsers conveniently for web application test.
In order to provide testers with tool to simulate browser to send a variety of reasons of request functions to a web application server, Jmeter function was integrated into the test automation framework designed in this paper. It will do not cause any inconvenience to the performance test in which the browser was opened and PC resources were occupied. At the same time, a test automation framework can prepare a mass of data for performance test by directly configure data resources.
In order to ensure universality, the test automation framework designed in my work not only provides testers with unified test cases, but also leaves the extensibility space for introducing other test tools in the future.
Using the new test automation framework, testers can configure a test type easily just through modifying a test type and can switch among different tests without altering any elements related to test cases.
Fig. 4.1 : A Web-Based Test Automation Framework
Testing the Framework
The web-based test automation framework, like any another being developed, needs testing. As a new test automation framework, there are no tests to perform and all tests need. We must evaluate its performance of reusability, extensibility, compatibility, and universality in particular.
In order to test the performance of sharing test processes and test data between UI test and performance test. The test processes are designed as following:
Step1. To prepare all data for IU system and data for test tools configuration item, that is, to prepare the test cases and test type configuration file for both UI test and load test.
Step2. To launch the test automation framework, start UI test for IU system, modify default browser class of configuration items in test tools
Step3. To Change test type into load test first, and then to launch the test automation framework, final modify test parameters of load test for examples, each request spacing interval, How many requests launch at same time, How many threads need to implement these requests?
Hardware / Software Assumptions:
1. JDK 1.6
2. Mozila browser, IE
3. Windows XP/Windows 7 Operating System
4. RAM : Minimum 2GB
