Software Testing guide

Software Testing - Boundary-value analysis

2008-08-04T04:57:00.001-07:00

Boundary-value analysis is a variant and refinement of equivalence partitioning, with two major differences: First, rather than selecting any element in an equivalence class as being representative, elements are selected such that each edge of the EC is the subject of a test. Boundaries are always a good place to look for defects. Second, rather than focusing exclusively on input conditions, output conditions are also explored by defining output ECs. What can be output? What are the classes of output? What should I create as an input to force a useful set of classes that represent the outputs that ought to be produced? The guidelines for boundary-value analysis are: · If an input specifies a range of valid values, write test cases for the ends of the range and invalid-input test cases for conditions just beyond the ends. Example: If the input requires a real number in the range 0.0 to 90.0 degrees, then write test cases for 0.0, 90.0, -0.001, and 90.001. · If an input specifies a number of valid values, write test cases for the minimum and maximum number of values and one beneath and beyond these values. Example: If the input requires the titles of at least 3, but no more than 8, books, then write test cases for 2, 3, 8, and 9 books. · Use the above guidelines for each output condition. Boundary-value analysis is not as simple as it sounds, because boundary conditions may be subtle and difficult to identify. The method does not test combinations of input conditions.

What is Cookie testing & Security testing???

2008-08-04T04:55:00.001-07:00

We will first focus on what exactly cookies are and how they work.

It would be easy for you to understand the test cases for testing cookies when you have clear understanding of how cookies work? How cookies stored on hard drive? And how can we edit cookie settings? Explain Cookies? Cookie is small information stored in text file on user’s hard drive by web server. This information is later used by web browser to retrieve information from that machine. Generally cookie contains personalized user data or information that is used to communicate between different web pages.

Why Cookies are used?

Cookies are nothing but the user’s identity and used to track where the user navigated throughout the web site pages. The communication between web browser and web server is stateless. For example if you are accessing domain http://www.example.com/1.html then web browser will simply query to example.com web server for the page 1.html. Next time if you type page as http://www.example.com/2.html then new request is send to example.com web server for sending 2.html page and web server don’t know anything about to whom the previous page 1.html served. What if you want the previous history of this user communication with the web server? You need to maintain the user state and interaction between web browser and web server somewhere. This is where cookie comes into picture. Cookies serve the purpose of maintaining the user interactions with web server. How cookies work?] The HTTP protocol used to exchange information files on the web is used to maintain the cookies. There are two types of HTTP protocol. Stateless HTTP and Stateful HTTP protocol. Stateless HTTP protocol does not keep any record of previously accessed web page history. While Stateful HTTP protocol do keep some history of previous web browser and web server interactions and this protocol is used by cookies to maintain the user interactions. Whenever user visits the site or page that is using cookie, small code inside that HTML page (Generally a call to some language script to write the cookie like cookies in JAVAScript, PHP, Perl) writes a text file on users machine called cookie. Here is one example of the code that is used to write cookie and can be placed inside any HTML page: Set-Cookie:

NAME=VALUE; expires=DATE; path=PATH; domain=DOMAIN_NAME;

When user visits the same page or domain later time this cookie is read from disk and used to identify the second visit of the same user on that domain. Expiration time is set while writing the cookie. This time is decided by the application that is going to use the cookie. Generally two types of cookies are written on user machine. 1) Session cookies: This cookie is active till the browser that invoked the cookie is open. When we close the browser this session cookie gets deleted. Some time session of say 20 minutes can be set to expire the cookie. 2) Persistent cookies: The cookies that are written permanently on user machine and lasts for months or years. Where cookies are stored? When any web page application writes cookie it get saved in a text file on user hard disk drive. The path where the cookies get stored depends on the browser. Different browsers store cookie in different paths. E.g. Internet explorer store cookies on path “C:\Documents and Settings\Default User\Cookies”Here the “Default User” can be replaced by the current user you logged in as. Like “Administrator”, or user name like “Vijay” etc.The cookie path can be easily found by navigating through the browser options. In Mozilla Firefox browser you can even see the cookies in browser options itself. Open the Mozila browser, click on Tools->Options->Privacy and then “Show cookies” button. How cookies are stored? Lets take example of cookie written by rediff.com on Mozilla Firefox browser:On Mozilla Firefox browser when you open the page rediff.com or login to your rediffmail account, a cookie will get written on your Hard disk. To view this cookie simply click on “Show cookies” button mentioned on above path. Click on Rediff.com site under this cookie list. You can see different cookies written by rediff domain with different names. Site: Rediff.com Cookie name: RMID Name: RMID (Name of the cookie) Content: 1d11c8ec44bf49e0… (Encrypted content) Domain: .rediff.comPath: / (Any path after the domain name) Send For: Any type of connection Expires: Thursday, December 31, 2020 11:59:59 PM Applications where cookies can be used: 1) To implement shopping cart:Cookies are used for maintaining online ordering system. Cookies remember what user wants to buy. What if user adds some products in their shopping cart and if due to some reason user don’t want to buy those products this time and closes the browser window? When next time same user visits the purchase page he can see all the products he added in shopping cart in his last visit.
2) Personalized sites:When user visits certain pages they are asked which pages they don’t want to visit or display. User options are get stored in cookie and till the user is online, those pages are not shown to him.
3) User tracking: To track number of unique visitors online at particular time. 4) Marketing:Some companies use cookies to display advertisements on user machines. Cookies control these advertisements. When and which advertisement should be shown? What is the interest of the user? Which keywords he searches on the site? All these things can be maintained using cookies.
5) User sessions:Cookies can track user sessions to particular domain using user ID and password. Drawbacks of cookies:

1) Even writing Cookie is a great way to maintain user interaction, if user has set browser options to warn before writing any cookie or disabled the cookies completely then site containing cookie will be completely disabled and can not perform any operation resulting in loss of site traffic.
2) Too many Cookies:If you are writing too many cookies on every page navigation and if user has turned on option to warn before writing cookie, this could turn away user from your site.
3) Security issues:Some times users personal information is stored in cookies and if someone hack the cookie then hacker can get access to your personal information. Even corrupted cookies can be read by different domains and lead to security issues.
4) Sensitive information:Some sites may write and store your sensitive information in cookies, which should not be allowed due to privacy concerns. Some Major Test cases for web application cookie testing: The first obvious test case is to test if your application is writing cookies properly on disk.
You can use the Cookie Tester application also if you don’t have any web application to test but you want to understand the cookie concept for testing.

Test cases:
1) As a Cookie privacy policy make sure from your design documents that no personal or sensitive data is stored in the cookie.
2) If you have no option than saving sensitive data in cookie make sure data stored in cookie is stored in encrypted format. 3) Make sure that there is no overuse of cookies on your site under test. Overuse of cookies will annoy users if browser is prompting for cookies more often and this could result in loss of site traffic and eventually loss of business. 4) Disable the cookies from your browser settings: If you are using cookies on your site, your sites major functionality will not work by disabling the cookies. Then try to access the web site under test. Navigate through the site. See if appropriate messages are displayed to user like “For smooth functioning of this site make sure that cookies are enabled on your browser”. There should not be any page crash due to disabling the cookies. (Please make sure that you close all browsers, delete all previously written cookies before performing this test)
5) Accepts/Reject some cookies: The best way to check web site functionality is, not to accept all cookies. If you are writing 10 cookies in your web application then randomly accept some cookies say accept 5 and reject 5 cookies. For executing this test case you can set browser options to prompt whenever cookie is being written to disk. On this prompt window you can either accept or reject cookie. Try to access major functionality of web site. See if pages are getting crashed or data is getting corrupted. 6) Delete cookie: Allow site to write the cookies and then close all browsers and manually delete all cookies for web site under test. Access the web pages and check the behavior of the pages.
7) Corrupt the cookies: Corrupting cookie is easy. You know where cookies are stored. Manually edit the cookie in notepad and change the parameters to some vague values. Like alter the cookie content, Name of the cookie or expiry date of the cookie and see the site functionality. In some cases corrupted cookies allow to read the data inside it for any other domain. This should not happen in case of your web site cookies. Note that the cookies written by one domain say rediff.com can’t be accessed by other domain say yahoo.com unless and until the cookies are corrupted and someone trying to hack the cookie data.
8 ) Checking the deletion of cookies from your web application page: Some times cookie written by domain say rediff.com may be deleted by same domain but by different page under that domain. This is the general case if you are testing some ‘action tracking’ web portal. Action tracking or purchase tracking pixel is placed on the action web page and when any action or purchase occurs by user the cookie written on disk get deleted to avoid multiple action logging from same cookie. Check if reaching to your action or purchase page deletes the cookie properly and no more invalid actions or purchase get logged from same user.
9) Cookie Testing on Multiple browsers: This is the important case to check if your web application page is writing the cookies properly on different browsers as intended and site works properly using these cookies. You can test your web application on Major used browsers like Internet explorer (Various versions), Mozilla Firefox, Netscape, Opera etc.
10) If your web application is using cookies to maintain the logging state of any user then log in to your web application using some username and password. In many cases you can see the logged in user ID parameter directly in browser address bar. Change this parameter to different value say if previous user ID is 100 then make it 101 and press enter. The proper access message should be displayed to user and user should not be able to see other users account. These are some Major test cases to be considered while testing website cookies. You can write multiple test cases from these test cases by performing various combinations. If you have some different application scenario, you can mention your test cases in comments below.

What is Client server testing & Web testing?

2008-08-04T04:54:00.001-07:00

Projects are broadly divided into two types of: 2 tier applications 3 tier applications CLIENT / SERVER TESTING
This type of testing usually done for 2 tier applications (usually developed for LAN)Here we will be having front-end and backend. The application launched on front-end will be having forms and reports which will be monitoring and manipulating data E.g: applications developed in VB, VC++, Core Java, C, C++, D2K, PowerBuilder etc.,The backend for these applications would be MS Access, SQL Server, Oracle, Sybase, Mysql, Quadbase The tests performed on these types of applications would be- User interface testing- Manual support testing- Functionality testing- Compatibility testing & configuration testing- Intersystem testing WEB TESTING
This is done for 3 tier applications (developed for Internet / intranet / xtranet)Here we will be having Browser, web server and DB server. The applications accessible in browser would be developed in HTML, DHTML, XML, JavaScript etc. (We can monitor through these applications) Applications for the web server would be developed in Java, ASP, JSP, VBScript, JavaScript, Perl, Cold Fusion, PHP etc. (All the manipulations are done on the web server with the help of these programs developed) The DBserver would be having oracle, sql server, sybase, mysql etc. (All data is stored in the database available on the DB server) The tests performed on these types of applications would be- User interface testing- Functionality testing- Security testing- Browser compatibility testing- Load / stress testing- Interoperability testing/intersystem testing- Storage and data volume testing A web-application is a three-tier application.This has a browser (monitors data) [monitoring is done using html, dhtml, xml, javascript]-> webserver (manipulates data) [manipulations are done using programming languages or scripts like adv java, asp, jsp, vbscript, javascript, perl, coldfusion, php] -> database server (stores data) [data storage and retrieval is done using databases like oracle, sql server, sybase, mysql]. The types of tests, which can be applied on this type of applications, are:
1. User interface testing for validation & user friendliness
2. Functionality testing to validate behaviors, i/p, error handling, o/p, manipulations, services levels, order of functionality, links, content of web page & backend coverage’s
3. Security testing
4. Browser compatibility
5. Load / stress testing
6. Interoperability testing
7. Storage & data volume testing
A client-server application is a two tier application.This has forms & reporting at front-end (monitoring & manipulations are done) [using vb, vc++, core java, c, c++, d2k, power builder etc.,] -> database server at the backend [data storage & retrieval) [using ms access, sql server, oracle, sybase, mysql, quadbase etc.,] The tests performed on these applications would be
1. User interface testing
2. Manual support testing
3. Functionality testing
4. Compatibility testing
5. Intersystem testingSome more points to clear the difference between client server, web and desktop applications: Desktop application:
1. Application runs in single memory (Front end and Back end in one place)
2. Single user only Client/Server application:
1. Application runs in two or more machines
2. Application is a menu-driven
3. Connected mode (connection exists always until logout)
4. Limited number of users
5. Less number of network issues when compared to web app.
Web application:
1. Application runs in two or more machines
2. URL-driven
3. Disconnected mode (state less)
4. Unlimited number of users5. Many issues like hardware compatibility, browser compatibility, version compatibility, security issues, performance issues etc. As per difference in both the applications come where, how to access the resources. In client server once connection is made it will be in state on connected, whereas in case of web testing http protocol is stateless, then there comes logic of cookies, which is not in client server. For client server application users are well known, whereas for web application any user can login and access the content, he/she will use it as per his intentions. So, there are always issues of security and compatibility for web application.

Software Testing - Error Guessing

2008-08-04T04:52:00.002-07:00

Error guessing is an ad hoc approach, based on intuition and experience, to identify tests that are considered likely to expose errors. The basic idea is to make a list of possible errors or error-prone situations and then develop tests based on the list. What are the most common error-prone situations we have seen before? Defects’ histories are useful. There is a high probability that defects that have been there in the past are the kind that are going to be there in the future. Some items to try are: · empty or null lists/strings · zero instances/occurrences · blanks or null characters in strings · negative numbers One of the studies done by Myers (1979) states that the probability of errors remaining in the program is proportional to the number of errors that have been found so far. This alone provides a rich source of focus for productive error guessing.

What is Load and Performance testing

2008-08-04T04:52:00.001-07:00

Load Testing and Performance Testing are commonly said as positive testing where as Stress Testing is said to be as negative testing. Say for example there is a application which can handle 25 simultaneous user logins at a time. In load testing we will test the application for 25 users and check how application is working in this stage, in performance testing we will concentrate on the time taken to perform the operation. Where as in stress testing we will test with more users than 25 and the test will continue to any number and we will check where the application is cracking the Hardware resources.

Software Testing - Graph Matrix

2008-08-04T04:51:00.000-07:00

A graph matrix is a square matrix whose size is equivalent to the number of nodes in the flow graph. Each row and column correspond to a particular node and the matrix corresponds to the connections (edges) between nodes. By adding a link weight to each matrix entry, more information about the control flow can be captured. In its simplest form, the link weight is 1 if an edge exists and 0 if it does not. But other types of link weights can be represented:1. the probability that an edge will be executed,2. the processing time expended during link traversal,3. the memory required during link traversal, or 4. the resources required during link traversal.Graph theory algorithms can be applied to these graph matrices to help in the analysis necessary to produce the basis set.

Manual Testing - Cause Effect Graph

2008-08-04T04:49:00.000-07:00

A cause-effect graph is a directed graph that maps a set of causes to a set of effects. The causes may be thought of as the input to the program, and the effects may be thought of as the output. Usually the graph shows the nodes representing the causes on the left side and the nodes representing the effects on the right side. There may be intermediate nodes in between that combine inputs using logical operators such as AND and OR.
Constraints may be added to the causes and effects. These are represented as edges labelled with the constraint symbol using a dashed line. For causes, valid constraint symbols are E (exclusive?), O (one and only one?), and I (at least one?). For effects, valid constraint symbols are R (Require?) and M (Mask?). (This information needs to be verified.)

The graph's direction is as follows:
(XXX: a figure would represent this better)Causes --> intermediate nodes --> Effects It is a proven technique for effective requirements validation and test case design, cause-effect graphing is the process of transforming specifications into a graphic representation. This graphic representation depicts the functional relationships and conditions present in the requirements, illustrating how each input relates to every other input, as well as every output.

Constraints and observability of nodes also are established during this process, allowing the project team to identify potential problem areas. In developing the cause-effect graph, the test team evaluates the requirements for completeness, consistency, sufficient level of detail and lack of ambiguity, often finding defects that otherwise would not be found until integration testing.

What is Equivalence partitioning

2008-01-14T02:01:00.000-08:00

Equivalence partitioning is a systematic process that identifies, on the basis of whatever information is available, a set of interesting classes of input conditions to be tested, where each class is representative of (or covers) a large set of other possible tests. If partitioning is applied to the product under test, the product is going to behave in much the same way for all members of the class.

The aim is to minimize the number of test cases required to cover these input conditions.

There are two distinct steps. The first is to identify the equivalence classes (ECs) and the second is to identify the test cases.

(1) Identifying equivalence classes

For each external input:

(i) If the input specifies a range of valid values, define one valid EC (within the range) and two invalid Ecs (one outside each end of the range).

Example: If the input requires a month in the range of 1-12, define one valid EC for months 1 through 12 and two invalid ECs (month<1>12).

(ii) If the input specifies the number (N) of valid values, define one valid EC and two invalid ECs (none, and more than N).

Example: If the input requires the titles of at least three but no more than eight books, then define one valid EC and two invalid ECs (<3>8 books).

(iii) If the input specifies a set of valid values, define one valid EC (within the set) and one invalid EC (outside the set).

Example: If the input requires one of the names TOM, DICK, or HARRY, then define one valid EC (using one of the valid names) and one invalid EC (using the name JOE).

(iv) If there is reason to believe that the program handles each valid input differently, then define one valid EC per valid input.

(v) If the input specifies a “must be” situation, define one valid EC and one invalid EC.

Example: If the first character of the input must be numeric, then define one valid EC where the first character is a number and one invalid EC where the first character is not a number.

(vi) If there is reason to believe that elements in an EC are not handled in an identical manner by the program, subdivide the EC into smaller ECs.

(2) Identifying test cases

(i) Assign a unique number to each EC.
(ii) Until all valid ECs have been covered by test cases, write a new test case covering as many of the uncovered ECs as possible.
(iii) Until all invalid Ecs have been covered by test cases, write a test case that covers one, and only one, of the uncovered invalid ECs.
(iv) If multiple invalid ECs are tested in the same test case, some of those tests may never be executed because the first test may mask other tests or terminate execution of the test case.

Equivalence partitioning significantly reduces the number of input conditions to be tested by identifying classes of conditions that are equivalent to many other conditions. It does not test combinations of input conditions.

Explain Testing Methodologies

2008-01-14T01:59:00.000-08:00

Testing methodologies are set of rules or guidelines that are followed to minimize the number of test cases and also provide with maximum test coverage.

The following methods are commonly used:

· equivalence partioning
· boundary-value analysis
· error guessing

The following are lesser-used methods:

· cause-effect graphing
· syntax testing
· state transition testing
· graph matrix

What is ANSI?

2008-01-13T08:12:00.005-08:00

ANSI = 'American National Standards Institute', the primary industrial standards body in the U.S.; publishes some software-related standards in conjunction with the IEEE and ASQ (American Society for Quality).

What is IEEE?

2008-01-13T08:12:00.003-08:00

IEEE = 'Institute of Electrical and Electronics Engineers' - among other things, creates standards such as 'IEEE Standard for Software Test Documentation' (IEEE/ANSI Standard 829), 'IEEE Standard of Software Unit Testing (IEEE/ANSI Standard 1008), 'IEEE Standard for Software Quality Assurance Plans' (IEEE/ANSI Standard 730), and others.

What is ISO?

2008-01-13T08:12:00.001-08:00

ISO = 'International Organisation for Standardization' - The ISO 9001:2000 standard (which replaces the previous standard of 1994) concerns quality systems that are assessed by outside auditors, and it applies to many kinds of production and manufacturing organizations, not just software. It covers documentation, design, development, production, testing, installation, servicing, and other processes. The full set of standards consists of: (a)Q9001-2000 - Quality Management Systems: Requirements; (b)Q9000-2000 - Quality Management Systems: Fundamentals and Vocabulary; (c)Q9004-2000 - Quality Management Systems: Guidelines for Performance Improvements. To be ISO 9001 certified, a third-party auditor assesses an organization, and certification is typically good for about 3 years, after which a complete reassessment is required. Note that ISO certification does not necessarily indicate quality products - it indicates only that documented processes are followed. Also see http://www.iso.ch/ for the latest information. In the U.S. the standards can be purchased via the ASQ web site at http://e-standards.asq.org/

What is CMM?

2008-01-13T08:10:00.000-08:00

CMM = 'Capability Maturity Model', developed by the SEI. It's a model of 5 levels of organizational 'maturity' that determine effectiveness in delivering quality software. It is geared to large organizations such as large U.S. Defense Department contractors. However, many of the QA processes involved are appropriate to any organization, and if reasonably applied can be helpful. Organizations can receive CMM ratings by undergoing assessments by qualified auditors.

Level 1 - characterized by chaos, periodic panics, and heroic efforts required by individuals to successfully complete projects. Fw if any processes in place; successes may not be repeatable.

Level 2 - software project tracking, requirements management, realistic planning, and configuration management processes are in place; successful practices can be repeated.

Level 3 - standard software development and maintenance processes are integrated throughout an organization; a Software Engineering Process Group is is in place to oversee software processes, and training programs are used to ensure understanding and compliance.

Level 4 - metrics are used to track productivity, processes, and products. Project performance is predictable, and quality is consistently high.

Level 5 - the focus is on continouous process improvement. The impact of new processes and technologies can be predicted and effectively implemented when required.

Perspective on CMM ratings: During 1997-2001, 1018 organizations were assessed. Of those, 27% were rated at Level 1, 39% at 2, 23% at 3, 6% at 4, and 5% at 5. (For ratings during the period 1992-96, 62% were at Level 1, 23% at 2, 13% at 3, 2% at 4, and 0.4% at 5.) The median size of organizations was 100 software engineering/maintenance personnel; 32% of organizations were U.S. federal contractors or agencies. For those rated at Level 1, the most problematical key process area was in Software Quality Assurance.

What is SEI?

2008-01-13T08:09:00.003-08:00

SEI = 'Software Engineering Institute' at Carnegie-Mellon University; initiated by the U.S. Defense Department to help improve software development processes.

What is the 'software life cycle'?

2008-01-13T08:09:00.001-08:00

The life cycle begins when an application is first conceived and ends when it is no longer in use. It includes aspects such as initial concept, requirements analysis, functional design, internal design, documentation planning, test planning, coding, document preparation, integration, testing, maintenance, updates, retesting, phase-out, and other aspects.

What is Smoke Testing ?

2008-01-13T05:33:00.001-08:00

Smoke testing is a relatively simple check to see whether the product "smokes" when it runs. Smoke testing is also sometimes known as ad hoc testing, i.e. testing without a formal test plan.

With many projects, smoke testing is carried out in addition to formal testing. If smoke testing is carried out by a skilled tester, it can often find problems that are not caught during regular testing. Sometimes, if testing occurs very early or very late in the software development life cycle, this can be the only kind of testing that can be performed.

Smoke testing, by definition, is not exhaustive, but, over time, you can increase your coverage of smoke testing.

A common practice at Microsoft, and some other software companies, is the daily build and smoke test process. This means, every file is compiled, linked, and combined into an executable file every single day, and then the software is smoke tested.

Smoke testing minimizes integration risk, reduces the risk of low quality, supports easier defect diagnosis, and improves morale. Smoke testing does not have to be exhaustive, but should expose any major problems. Smoke testing should be thorough enough that, if it passes, the tester can assume the product is stable enough to be tested more thoroughly. Without smoke testing, the daily build is just a time wasting exercise. Smoke testing is the sentry that guards against any errors in development and future problems during integration. At first, smoke testing might be the testing of something that is easy to test. Then, as the system grows, smoke testing should expand and grow, from a few seconds to 30 minutes or more.

Checklist for conducting Unit Tests

2008-01-13T05:30:00.000-08:00

- Is the number of input parameters equal to number of arguments?
- Do parameter and argument attributes match?- Do parameter and argument units system match?
- Is the number of arguments transmitted to called modules equal to number of parameters?
- Are the attributes of arguments transmitted to called modules equal to attributes of parameters?
- Is the units system of arguments transmitted to called modules equal to units system of parameters?
- Are the number of attributes and the order of arguments to built-in functions correct?
- Are any references to parameters not associated with current point of entry?
- Have input only arguments altered?
- Are global variable definitions consistent across modules?- Are constraints passed as arguments?

When a module performs external I/O, additional interface tests must be conducted:

- File attributes correct?
- OPEN/CLOSE statements correct?- Format specification matches I/O statement?- Buffer size matches record size?
- Files opened before use?- End-of-file conditions handled?
- I/O errors handled?- Any textual errors in output information?
The local data structure for a module is a common source of errors.

Test cases should be designed to uncover errors in the following categories:

- improper or inconsistent typing- erroneous initialization or default values
- incorrect (misspelled or truncated) variable names- inconsistent data types
- underflow, overflow and addressing exceptions

From a strategic point of view, the following questions should be addressed:

- Has the component interface been fully tested?
- Have local data structured been exercised at their boundaries?
- Has the cyclomatic complexity of the module been determined?
- Have all independent basis paths been tested?
- Have all loops been tested appropriately?
- Have data flow paths been tested? Have all error handling paths been tested?

How do you create a test plan/design?

2008-01-13T05:29:00.000-08:00

Test scenarios and/or cases are prepared by reviewing functional requirements of the release and preparing logical groups of functions that can be further broken into test procedures. Test procedures define test conditions, data to be used for testing and expected results, including database updates, file outputs, report results.. Test cases and scenarios are designed to represent both typical and unusual situations that may occur in the application.. Test engineers define unit test requirements and unit test cases. Test engineers also execute unit test cases. . It is the test team that, with assistance of developers and clients, develops test cases and scenarios for integration and system testing.. Test scenarios are executed through the use of test procedures or scripts.. Test procedures or scripts define a series of steps necessary to perform one or more test scenarios.. Test procedures or scripts include the specific data that will be used for testing the process or transaction.. Test procedures or scripts may cover multiple test scenarios.. Test scripts are mapped back to the requirements and traceability matrices are used to ensure each test is within scope.. Test data is captured and base lined, prior to testing. This data serves as the foundation for unit and system testing and used to exercise system functionality in a controlled environment.. Some output data is also base-lined for future comparison. Base-lined data is used to support future application maintenance via regression testing.. A pretest meeting is held to assess the readiness of the application and the environment and data to be tested. A test readiness document is created to indicate the status of the entrance criteria of the release.

Inputs for this process:. Approved Test Strategy Document.. Test tools, or automated test tools, if applicable.. Previously developed scripts, if applicable.. Test documentation problems uncovered as a result of testing.. A good understanding of software complexity and module path coverage, derived from general and detailed design documents, e.g. software design document, source code and software complexity data.

Outputs for this process:. Approved documents of test scenarios, test cases, test conditions and test data.. Reports of software design issues, given to software developers for correction.

How do you create a Test Strategy?

2008-01-13T05:27:00.000-08:00

The test strategy is a formal description of how a software product will be tested. A test strategy is developed for all levels of testing, as required. The test team analyzes the requirements, writes the test strategy and reviews the plan with the project team. The test plan may include test cases, conditions, the test environment, a list of related tasks, pass/fail criteria and risk assessment.
Inputs for this process:· A description of the required hardware and software components, including test tools. This information comes from the test environment, including test tool data.· A description of roles and responsibilities of the resources required for the test and schedule constraints. This information comes from man-hours and schedules.· Testing methodology. This is based on known standards.· Functional and technical requirements of the application. This information comes from requirements, change request, technical and functional design documents.· Requirements that the system can not provide, e.g. system limitations.Contents of Test Strategy document:Sections in the test strategy document include:. A description of the required hardware and software components, including test tools: This information comes from the test environment, including test tool data.. A description of roles and responsibilities of the resources required for the test and schedule constraints: This information comes from man-hours and schedules.. Testing methodology: This is based on known standards.. Functional and technical requirements of the application: This information comes from requirements, change request, technical, and functional design documents.. Requirements that the system cannot provide, e.g. system limitations.
Outputs for this process:· An approved and signed off test strategy document, test plan, including test cases.· Testing issues requiring resolution. Usually this requires additional negotiation at the project management level.

Software Testing - Taxonomy

2008-01-06T23:28:00.000-08:00

There is a plethora of testing methods and testing techniques, serving multiple purposes in different life cycle phases. Classified by purpose, software testing can be divided into: correctness testing, performance testing, reliability testing and security testing. Classified by life-cycle phase, software testing can be classified into the following categories: requirements phase testing, design phase testing, program phase testing, evaluating test results, installation phase testing, acceptance testing and maintenance testing. By scope, software testing can be categorized as follows: unit testing, component testing, integration testing, and system testing.

Manual testing procedure

2008-01-06T23:24:00.000-08:00

Introduction

There are many potential pitfalls to manual software testing, including:
1. Manual testing is slow and costly.
2. Manual tests do not scale well.
3. Manual testing is not consistent or repeatable.
4. Lack of training.
5. Testing is difficult to manage.

This article will cover five "best practices" recommendations to help avoid the pitfalls associated with manual software testing.

Be Thorough in Test Design and Documentation
In designing the tests, there should be agreement among the business staff, product and project managers, developers, and testers on test coverage. This can be documented as test requirements in a test plan. With this documentation, management can have visibility of the test coverage and know that the right areas are being tested. This then becomes an important management tool in managing testing.

The goal is to find the easiest way to document as many test cases as possible without having the test effort turn into a documentation effort.

Have the test requirements and test cases peer-reviewed, just as you would have software design reviews. The software development staff and the test staff should jointly develop the test designs, as they have complementary skill sets and knowledge bases.

In manual testing, like in other processes, several factors influence the effectiveness of the tests, including the completeness of the test cases and the thoroughness of the documentation. The goal should be to maximize management's understanding of the testing by spending the appropriate resources in each area, within the overall resource constraints. If you do not document your tests, you will not understand the coverage or software quality as revealed by the tests, and you also will not be able to determine that the test team is testing features that are the most important to your development team and customers. However, if you document everything related to each test case, you will not have time to do as many tests as you should. Documenting test cases can get expensive.

Automate Turnkey Tests as Much as Possible
There are various tools available that support test automation. When it is cost-effective and time-efficient to do so, there is no excuse for not automating software tests.
Automation 5% Rule: No more than 5% of the tests should be executed manually.
The benefit of automation is that the testing becomes less burdensome, and less likely to be scrimped on when under pressure. This also makes the testing easier to manage.
Manage the Test Activities Well

Do this as closely, and by establishing as full a procedure, as the software development activities. Be sure that the project plan has sufficient resources and time allocated to testing so that it does not get short shrift.

Rank Test Cases in Order of Importance, by Impact on Quality, by Risk, by How Often the Feature Tested is Used, or Some Other Related Metric
A goal should be to run all important test cases, but if there are resource constraints that prevent all test cases from being run, then the ranking will enable the important test cases to be run. This provides the maximum impact of the testing with the available resources. In manual software testing, you are always short on time. There should be an agreement or signoff procedure on the ranking and the coverage of the tests.

Have a Separate Budget with Proper Funding Allocated for Testing
Just as there is a budget allocated for the software code development, there should be a budget allocated for testing. Be sure that this budget is well-funded. Watch how much is spent on testing, and what the return on investment is.

Even when you have a good test automation program in place, you will still need to do some manual testing. The usability testing, for example, requires human involvement. However, manual testing is not the solution for short-cycle, high-volume test challenges. For example, if you have a daily build process where you need to run smoke-tests to assess the changes and stability of the software from build to build, these tests can be high-volume and short-cycle. Manual testing cannot solve your testing problem in this case.

Conclusion
To summarize the preceding in a simple list, the five best practice suggestions for manual software testing success are:
1. Be thorough in test design and documentation
2. Automate turnkey tests as much as possible
3. Manage the test activities well
4. Rank test cases in order of importance, by impact on quality, by risk, by how often the feature tested is used, or some other related metric
5. Have a separate budget with proper funding allocated for testing

Explain Boundary value testing and Equivalence testing with some examples.

2008-01-06T23:20:00.004-08:00

Boundary value testing is a technique to find whether the application is accepting the expected range of values and rejecting the values which falls out of range.

Ex. A user ID text box has to accept alphabet characters ( a-z ) with length of 4 to 10 characters.
BVA is done like this, max value:10 pass; max-1: 9 pass;
max+1=11 fail ;min=4 pass;min+1=5 pass;min-1=3 fail;
Like wise we check the corner values and come out with a conclusion whether the application is accepting correct range of values.
Equivalence testing is normally used to check the type of the object.
Ex. A user ID text box has to accept alphabet characters ( a - z ) with length of 4 to 10 characters.
In +ve condition we have test the object by giving alphabets. i.e a-z char only, after that we need to check whether the object accepts the value, it will pass.
In -ve condition we have to test by giving other than alphabets (a-z) i.e A-Z,0-9,blank etc, it will fail.

What is Security testing?

2008-01-06T23:20:00.003-08:00

It is a process used to look out whether the security features of a system are implemented as designed and also whether they are adequate for a proposed application environment. This process involves functional testing, penetration testing and verification

What is Installation testing?

2008-01-06T23:20:00.001-08:00

Installation testing is done to verify whether the hardware and software are installed and configured properly. This will ensure that all the system components were used during the testing process. This Installation testing will look out the testing for a high volume data, error messages as well as security testing

What is AUT ?

2008-01-06T23:19:00.003-08:00

AUT is nothing but "Application Under Test". After the designing and coding phase in Software development life cycle, the application comes for testing then at that time the application is stated as Application Under Test.

What is Defect Leakage ?

2008-01-06T23:19:00.001-08:00

Defect leakage occurs at the Customer or the End user side after the application delivery. After the release of the application to the client, if the end user gets any type of defects by using that application then it is called as Defect leakage. This Defect Leakage is also called as Bug Leak.

What are the contents in an effective Bug report?

2008-01-06T23:17:00.004-08:00

Project, Subject,
Description,
Summary,
Detected By (Name of the Tester),
Assigned To (Name of the Developer who is supposed to the Bug),
Test Lead ( Name ),
Detected in Version,
Closed in Version,
Date Detected,
Expected Date of Closure,
Actual Date of Closure,
Priority (Medium, Low, High, Urgent),
Severity (Ranges from 1 to 5),
Status,
Bug ID,
Attachment,
Test Case Failed (Test case that is failed for the Bug)

What is Bug Life Cycle?

2008-01-06T23:17:00.003-08:00

Bug Life Cycle is nothing but the various phases a Bug undergoes after it is raised or reported.
New or Opened
Assigned
Fixed
Tested
Closed

What is Error guessing and Error seeding ?

2008-01-06T23:17:00.001-08:00

Error Guessing is a test case design technique where the tester has to guess what faults might occur and to design the tests to represent them.

Error Seeding is the process of adding known faults intentionally in a program for the reason of monitoring the rate of detection & removal and also to estimate the number of faults remaining in the program

What is the difference between Bug, Error and Defect?

2008-01-06T23:16:00.003-08:00

Error : It is the Deviation from actual and the expected value.

Bug : It is found in the development environment before the product is shipped to the respective customer.

Defect : It is found in the product itself after it is shipped to the respective customer.

What is Test bed and Test data ?

2008-01-06T23:16:00.001-08:00

Test Bed is an execution environment configured for software testing. It consists of specific hardware, network topology, Operating System, configuration of the product to be under test, system software and other applications. The Test Plan for a project should be developed from the test beds to be used.
Test Data is that run through a computer program to test the software. Test data can be used to test the compliance with effective controls in the software.

What is Negative testing?

2008-01-06T23:15:00.001-08:00

Negative testing - Testing the system using negative data is called negative testing, e.g. testing the password where it should be minimum of 8 characters so testing it using 6 characters is negative testing

What are SDLC and STLC ? Explain its different phases.

2008-01-06T23:14:00.000-08:00

SDLC - Software Developement life cycle

Requirement phase
Designing phase (HLD, DLD (Program spec))
Coding
Testing
Release
Maintenance

STLC

System Study
Test planning
Writing Test case or scripts
Review the test case
Executing test case
Bug tracking
Report the defect

What is Ad-hoc testing?

2008-01-06T23:11:00.002-08:00

Ad hoc testing is concern with the Application Testing without following any rules or test cases.
For Ad hoc testing one should have strong knowledge about the Application.

Describe bottom-up and top-down approaches in Regression Testing.

2008-01-06T23:11:00.001-08:00

Bottom-up approach : In this approach testing is conducted from sub module to main module, if the main module is not developed a temporary program called DRIVERS is used to simulate the main module.

Top-down approach : In this approach testing is conducted from main module to sub module. if the sub module is not developed a temporary program called STUB is used for simulate the submodule.

What is the difference between structural and functional testing?

2008-01-06T23:08:00.002-08:00

Structural testing is a "white box" testing and it is based on the algorithm or code.
Functional testing is a "black box" (behavioral) testing where the tester verifies the functional specification

What is Re- test ? What is Regression Testing ?

2008-01-06T23:08:00.001-08:00

Re- test - Retesting means we testing only the certain part of an application again and not considering how it will effect in the other part or in the whole application.
Regression Testing - Testing the application after a change in a module or part of the application for testing that is the code change will affect rest of the application.

What is UAT testing? When it is to be done?

2008-01-06T23:07:00.001-08:00

UAT testing - UAT stands for 'User acceptance Testing. This testing is carried out with the user perspective and it is usually done before the release.

What are the basic solutions for the software development problems?

2008-01-06T23:06:00.000-08:00

Basic requirements - clear, detailed, complete, achievable, testable requirements has to be developed. Use some prototypes to help pin down requirements. In nimble environments, continuous and close coordination with customers/end-users is needed.
Schedules should be realistic - enough time to plan, design, test, bug fix, re-test, change, and document in the given schedule.
Adequate testing – testing should be started early, it should be re-tested after the bug fixed or changed, enough time should be spend for testing and bug-fixing.
Proper study on initial requirements – be ready to look after more changes after the development has begun and be ready to explain the changes done to others. Work closely with the customers and end-users to manage expectations. This avoids excessive changes in the later stages.
Communication – conduct frequent inspections and walkthroughs in appropriate time period; ensure that the information and the documentation is available on up-to-date if possible electronic. More emphasize on promoting teamwork and cooperation inside the team; use prototypes and proper communication with the end-users to clarify their doubts and expectations.

What are the common problems in the software development process?

2008-01-06T22:59:00.002-08:00

Inadequate requirements from the Client - if the requirements given by the client is not clear, unfinished and not testable, then problems may come.
Unrealistic schedules – Sometimes too much of work is being given to the developer and ask him to complete in a Short duration, then the problems are unavoidable.
Insufficient testing – The problems can arise when the developed software is not tested properly.
Given another work under the existing process – request from the higher management to work on another project or task will bring some problems when the project is being tested as a team.
Miscommunication – in some cases, the developer was not informed about the Clients requirement and expectations, so there can be deviations.

Why does software have bugs?

2008-01-06T22:59:00.001-08:00

Miscommunication or no communication – about the details of what an application should or shouldn't do
Programming errors – in some cases the programmers can make mistakes.
Changing requirements – there are chances of the end-user not understanding the effects of changes, or may understand and request them anyway to redesign, rescheduling of engineers, effects of other projects, work already completed may have to be redone or thrown out.
Time force - preparation of software projects is difficult at best, often requiring a lot of guesswork. When deadlines are given and the crisis comes, mistakes will be made.

What software testing types can be considered?

2008-01-06T22:57:00.004-08:00

Black box testing – This type of testing doesn’t require any knowledge of the internal design or coding. These Tests are based on the requirements and functionality.
White box testing – This kind of testing is based on the knowledge of internal logic of a particular application code. The Testing is done based on the coverage of code statements, paths, conditions.
Unit testing – the 'micro' scale of testing; this is mostly used to test the particular functions or code modules. This is typically done by the programmer and not by testers; it requires detailed knowledge of the internal program design and code. It cannot be done easily unless the application has a well-designed architecture with tight code; this type may require developing test driver modules or test harnesses.
Sanity testing or Smoke testing – This type of testing is done initially to determine if a new software version is performing well enough to accept it for a major testing effort. For example, if the new software is crashing the systems in every 5 minutes or corrupting databases, the software may not be in a 'sound’ condition to proceed for further testing in its current state.
Functional testing – This a commonly used black-box testing geared to check the functional requirements of an application; this type of testing should be done by testers.
Integration testing – This testing is combining the ‘parts’ of an application to determine if they function together correctly. The 'parts' can be code modules, individual applications, client and server applications on a network, etc. This type of testing is especially relevant to the client/server and distributed systems.
Incremental Integration testing – This is continuous testing of an application when a new functionality is added the existing ones; it checks the application functionality by verifying whether it works separately before all parts of the program are completed, in this type it will be checked whether to introduce test drivers or not; this is done by programmers or by testers.
Regression testing – This is testing the whole application again after the fixes or the modifications are done on the software. This is mostly done at the end of the Software development life cycle. Mostly Automated testing tools are used for these type of testing.
System testing – This is a type of black-box type testing that is based on overall requirements specifications; covers all combined parts of a system.
End-to-end testing – This is similar to system testing; this involves testing of a complete application environment such as interacting with a database, using network communications, or interacting with other hardware, applications and so on.
UAT ( User Acceptance Testing ) – This type of testing comes on the final stage and mostly done on the specifications of the end-user or client.
Usability testing – This testing is done to check the 'user-friendliness' of the application. This depends on the targeted end-user or customer. User interviews, surveys, video recording of user sessions, and other techniques can be used. Programmers and testers are usually not appropriate as usability testers.
Compatibility testing – Testing how well the software performs in a particular hardware, software, operating system, network etc.
Comparison testing – This is nothing comparing the software strengths and weakness with another competing product.
Mutation testing – This is another method for determining if a set of test data or test cases is useful, by purposely introducing various code changes or bugs and retesting with the original test data or cases to determine whether the 'bugs' are detected.

How do you decide when you have 'tested enough’?

2008-01-06T22:57:00.003-08:00

Common factors in deciding when to stop are:
Deadlines (release deadlines, testing deadlines, etc.)
Test cases completed with certain percentage passed
Test budget depleted
Coverage of code/functionality/requirements reaches a specified point
Bug rate falls below a certain level
Beta or alpha testing period ends

Describe the Software Development Life Cycle

2008-01-06T22:57:00.001-08:00

It includes aspects such as initial concept, requirements analysis, functional design, internal design, documentation planning, test planning, coding, document preparation, integration, testing, maintenance, updates, retesting, phase-out, and other aspects.

Describe the difference between validation and verification

2008-01-06T22:56:00.004-08:00

Verification is done by frequent evaluation and meetings to appraise the documents, policy, code, requirements, and specifications. This is done with the checklists, walkthroughs, and inspection meetings.
Validation is done during actual testing and it takes place after all the verifications are being done.

What is the difference between QA and testing?

2008-01-06T22:56:00.003-08:00

Testing involves operation of a system or application under controlled conditions and evaluating the results. It is oriented to 'detection'.
Software QA involves the entire software development PROCESS - monitoring and improving the process, making sure that any agreed-upon standards and procedures are followed, and ensuring that problems are found and dealt with. It is oriented to 'prevention'.

What is quality assurance?

2008-01-06T22:56:00.001-08:00

Software QA involves the entire software development PROCESS - monitoring and improving the process, making sure that any agreed-upon standards and procedures are followed, and ensuring that problems are found and dealt with. It is oriented to 'prevention'.

What is the purpose of the testing?

2008-01-06T22:53:00.002-08:00

Software testing is the process used to help identify the Correctness, Completeness, Security and Quality of the developed Computer Software.
Software Testing is the process of executing a program or system with the intent of finding errors.

Explain Load, Performance, Stress Testing with an example

2008-01-06T22:53:00.001-08:00

Load Testing and Performance Testing are commonly said as positive testing where as Stress Testing is said to be as negative testing.
Say for example there is a application which can handle 25 simultaneous user logins at a time. In load testing we will test the application for 25 users and check how application is working in this stage, in performance testing we will concentrate on the time taken to perform the operation. Where as in stress testing we will test with more users than 25 and the test will continue to any number and we will check where the application is cracking the Hardware resources.

What is Traceability Matrix ?

2008-01-06T22:52:00.000-08:00

Traceability Matrix is a document used for tracking the requirement, Test cases and the defect. This document is prepared to make the clients satisfy that the coverage done is complete as end to end, This document consists of Requirement/Base line doc Ref No., Test case/Condition, Defects/Bug id. Using this document the person can track the Requirement based on the Defect id.

Explain Compatibility Testing with an example

2008-01-06T22:48:00.000-08:00

Compatibility testing is to evaluate the application compatibility with the computing environment like Operating System, Database, Browser compatibility, Backwards compatibility, Computing capacity of the Hardware Platform and compatibility of the Peripherals.
Ex : If Compatibility testing is done on a Game application, before installing a game on a computer, its compatibility is checked with the computer specification that whether it is compatible with the computer having that much of specification or not.

Explain Peer Review in Software Testing

2008-01-06T22:47:00.000-08:00

It is an alternative form of Testing, where some colleagues were invited to examine your work products for defects and improvement opportunities.
Some Peer review approaches are,
Inspection – It is a more systematic and rigorous type of peer review. Inspections are more effective at finding defects than are informal reviews.Ex : In Motorola’s Iridium project nearly 80% of the defects were detected through inspections where only 60% of the defects were detected through informal reviews.
Team Reviews – It is a planned and structured approach but less formal and less rigorous comparing to Inspections.
Walkthrough – It is an informal review because the work product’s author describes it to some colleagues and asks for suggestions. Walkthroughs are informal because they typically do not follow a defined procedure, do not specify exit criteria, require no management reporting, and generate no metrics.
Pair Programming – In Pair Programming, two developers work together on the same program at a single workstation and continuously reviewing their work.
Peer Deskcheck – In Peer Deskcheck only one person besides the author examines the work product. It is an informal review, where the reviewer can use defect checklists and some analysis methods to increase the effectiveness.
Passaround – It is a multiple, concurrent peer deskcheck where several people are invited to provide comments on the product.

Software testing guide - Testing tutorial

2008-01-06T22:09:00.000-08:00

Introduction
There are many potential pitfalls to manual software testing, including:
1. Manual testing is slow and costly.
2. Manual tests do not scale well.
3. Manual testing is not consistent or repeatable.
4. Lack of training.
5. Testing is difficult to manage.
This article will cover five "best practices" recommendations to help avoid the pitfalls associated with manual software testing.
Be Thorough in Test Design and Documentation
In designing the tests, there should be agreement among the business staff, product and project managers, developers, and testers on test coverage. This can be documented as test requirements in a test plan. With this documentation, management can have visibility of the test coverage and know that the right areas are being tested. This then becomes an important management tool in managing testing.
The goal is to find the easiest way to document as many test cases as possible without having the test effort turn into a documentation effort.
Have the test requirements and test cases peer-reviewed, just as you would have software design reviews. The software development staff and the test staff should jointly develop the test designs, as they have complementary skill sets and knowledge bases.
In manual testing, like in other processes, several factors influence the effectiveness of the tests, including the completeness of the test cases and the thoroughness of the documentation. The goal should be to maximize management's understanding of the testing by spending the appropriate resources in each area, within the overall resource constraints. If you do not document your tests, you will not understand the coverage or software quality as revealed by the tests, and you also will not be able to determine that the test team is testing features that are the most important to your development team and customers. However, if you document everything related to each test case, you will not have time to do as many tests as you should. Documenting test cases can get expensive.
Automate Turnkey Tests as Much as Possible
There are various tools available that support test automation. When it is cost-effective and time-efficient to do so, there is no excuse for not automating software tests.
Automation 5% Rule: No more than 5% of the tests should be executed manually.
The benefit of automation is that the testing becomes less burdensome, and less likely to be scrimped on when under pressure. This also makes the testing easier to manage.
Manage the Test Activities Well
Do this as closely, and by establishing as full a procedure, as the software development activities. Be sure that the project plan has sufficient resources and time allocated to testing so that it does not get short shrift.
Rank Test Cases in Order of Importance, by Impact on Quality, by Risk, by How Often the Feature Tested is Used, or Some Other Related Metric
A goal should be to run all important test cases, but if there are resource constraints that prevent all test cases from being run, then the ranking will enable the important test cases to be run. This provides the maximum impact of the testing with the available resources. In manual software testing, you are always short on time. There should be an agreement or signoff procedure on the ranking and the coverage of the tests.
Have a Separate Budget with Proper Funding Allocated for Testing
Just as there is a budget allocated for the software code development, there should be a budget allocated for testing. Be sure that this budget is well-funded. Watch how much is spent on testing, and what the return on investment is.
Even when you have a good test automation program in place, you will still need to do some manual testing. The usability testing, for example, requires human involvement. However, manual testing is not the solution for short-cycle, high-volume test challenges. For example, if you have a daily build process where you need to run smoke-tests to assess the changes and stability of the software from build to build, these tests can be high-volume and short-cycle. Manual testing cannot solve your testing problem in this case.
Conclusion
To summarize the preceding in a simple list, the five best practice suggestions for manual software testing success are:
1. Be thorough in test design and documentation
2. Automate turnkey tests as much as possible
3. Manage the test activities well
4. Rank test cases in order of importance, by impact on quality, by risk, by how often the feature tested is used, or some other related metric
5. Have a separate budget with proper funding allocated for testing