Peer Reviews

Peer Reviews

Welcome friends , new week has just started. Monday blue’s are on full swing. Let’s start this week’s blogging with a less talked topic “PEER REVIWES”

What is it ? In simple language this term is used to describe a review process of the artifacts /documents by the peers in the team wherein the author of the document presents it to peer members in the team with the purpose of getting feedback in order to improve the document before making its final version.

Process of peer review is not very tedious where the document author presents the document to the group of reviewers who are members of same team and explains the purpose of the document as to what is the document, objective and details about the document and ask for the feedback on the document. From the author’s point of view it’s good to have a checklist ready at his end before submitting the document just to reduce the reviewing time and avoid small mistakes.

Purpose behind this activity is can be –
- Validate the correctness of the document.
- Validate the coverage of document with respect to purpose for ex. Peer review of a test plan/strategy document should verify that the test plan/strategy is in accordance with the project/product for which we are doing testing and it is covering each and every aspect of testing which can be done/carried on.
- Validate the conformance of the document with respect to the process followed in organization, standards of the organization and client needs if any.

With respect to software testing , there are many documents for which we can conduct peer review such as test plan, test strategy , test cases and scripts and other such documents. Benefits of the PEER REVIEW process can be achieved only when the review comments are documented and in turn implemented properly. Also as a suggestion it should be done be senior members of the team who have good command over the product/system and have good domain knowledge

Ad-hoc testing and Exploratory testing

Today’s post is just a MICRO blog about Ad-hoc testing and Exploratory testing

These two terms are often used interchangeably by the development guys. I wonder senior dev guys in my team don’t even know difference between black box & white box testing. Anyways coming back to our main topic both ad-hoc and exploratory testing

- are system testing techniques where in we test the whole software system.
- are undocumented form of testing in the sense that the test we perform are no formally documented. That is the scenarios in this testing are apart from the documented test cases.
- are informal testing techniques performed at small scale generally after the completion of formal testing OR when we have less time.

The only difference between these two is that in order to perform Ad-hoc testing a tester should have domain /product knowledge while no such knowledge is required for exploratory testing, success of ad-hoc testing totally depends on the knowledge and experience level of the tester. We can understand exploratory testing going be name , it means to explore the system with the intent of learning and also to find bugs. Main aim of tester performing exploratory testing is to get knowledge about the system and report bugs after getting them validated from a person having sound knowledge of the product. While the sole purpose of ad-hoc testing is to find bugs in the system as the tester already have the necessary knowledge about the system/product under testing. One important implementation of ADHOC TESTING is when we want to test the application beyond the documented OR obvious tests and also we may want to focus on most critical areas of application in case we have less time for testing and therefore can’t prepare test cases. For exploratory testing one such implication is when we do not have enough documentation available and want to know more about the application so that we can prepare some formal document regarding the functionality of the application.

Positive, Negative testing and test execution

Ideal flow of test execution , some days before a friend of mine (a fresher is testing field) asked me about positive, negative testing and the flow of test execution and his understanding inspired me for this post. As per my understanding, positive testing is ‘Test to pass’ that is testing done to confirm that the system meets the specified requirements. In these types of tests we usually do not input invalid conditions and main concentration should be on finding out the defects/bugs by executing the system as per the normal functionality. Under this we concentrate on the written requirements and normal flows and validate the system against it. In other words we validate that the system is working as per the documented requirements.

Coming on to negative testing, it is ‘Test to break’. Now that we are done with the normal testing we try to break the system by trying negative scenarios on it such as invalid inputs, unusual load and other negative scenarios so as to know the behavior of the system with such conditions. Suppose that a website is expected to have load of 1000 hits at a time then in negative testing we may try to overload along with the abnormal communication conditions.

Coming on to the test execution part, I follow a thumb rule that as a tester my job is to first validate that the system is working as per the written requirements (TEST TO PASS) and then after this I should try the negative scenarios (TEST TO BREAK). Remember, our first job is to execute the documented test cases since this is what we have agreed to deliver to the client and these are the test for which client is expecting the test report. So the order of execution should be first the positive tests and then the negative tests that is after positive testing we can try our hands on some undocumented testing (Ad-hoc and exploratory testing)

Requirement traceability matrix (RTM)

Requirement traceability matrix (RTM) in simple words is a document containing the requirements mapped with the test cases/scripts covering that particular requirement. it is one of the most important document created during a project life cycle and is very important in software testing activities. Going further RTM can contain more details like name and brief detail about the requirement, details of test case/script document which contains the test case(s)/script(s) handling the requirement, priority & severity of the requirement and also the verification status of the requirement (whether the test cases have been executed OR not) along with the status of requirement (Pass/Fail – requirement is catered in the software system and in fully functional without any bug). These details are not mandatory but definitely adds value to the RTM document as it provides complete picture about a particular requirement.

Creating RTM – there are 100’s and 1000’s of template available on the internet. Just pick one of them OR create your own. In a very simple way , you just need to create a table containing requirement ID (As mentioned in the SRS/FRS/Requirement Specification document) and test case ID of the test case in which this requirement is being tested. In addition to this we can provide other fields as mentioned above. Also we can create a single RTM document for both unit and functional/system test cases

Benefits – RTM is very beneficial in software testing as it –
- Track all requirements and whether or not they are being met by the test.
- Help ensure that all system requirements have been met during the Verification process.
- Above all it’s a very good tool from management and testers perspective as well. For testers , it serves the benefit of knowing how many requirements are covered and from management’s perspective it’s a tool from which they can gather all the required information pertaining to a particular requirement.

Mobile Application Certifications

There days mobile applications are on real boom specially after the entry of mobile platforms like Iphone/Blackberry/Android. As far as testing of mobile applications is concerned , a mobile app has to be tested on functionality aspects and on other aspects as well. You might have gone through my previous post mentioning major points of consideration while testing a mobile application. In this post, I am concentrating on testing the mobile application on other aspects. One such aspect is vendor certification testing which is done in order to get your application available on different app stores so that end users can download and use it further. Due to the boom and increasing number of mobile applications everyday different mobile technology/platform vendors have provided centralized places from where user can download the available applications such as Apple app store for apple, Windows Marketplace for windows, Blackberry App World for windows, Mobile Shop for BREW. These technology vendors have provided generalized tests, In order to place/release their application to users software vendors has to get them passed through these tests.

So far, following vendors have defined their test process and provided the test cases

- BREW – The process is known as ‘True BREW Certification’ and a set of test cases provide for this is known as ‘‘True BREW Test’ OR TBT in popular.
- Windows marketplace submission criteria for placing the application on Windows market place.
- Java verified program for J2ME applications.
- Symbian signed criteria for getting the application signed by symbian

All these above concentrates on general behavior, stability and functionality of the application. These are not at all related to testing of specific functionality but concentrates on general attributes which every application should posses.

Build Installation Procedure for Brew Applications

When a BREW application build is released for testing, it basically comes under two different folders as follows –

- Arm
- Win


ARM - This folder contains files which are to be loaded on actual mobile device. It should contain following 4 types of files

- .mod
- .mif
- .bar
- .sig

All these files are loaded on actual device for testing. MOD file contains the compiled code of the application for BREW handsets. MIF (Module Information File) contains icon, thumbnail etc of the application. This file is used by the handset to identify the application. BAR file contains resource files like images, text etc and SIG file is digital signature file which is unique for every handset, we can generate it by providing ESN/MEID of the handset. This file and is independent of the application but dependent on the handset.

These are minimum number of files needed to load and run the application on the BREW device. While loading SIG file name should be same as MIF file and the name should be in small letters. Generally the arm folder is renamed the same as MIF file and the whole folder is drag and dropped onto handset once the handset is connected to the PC through BREW apploader. Remember to power cycle the handset after loading the build.


WIN – The content of this folder is basically used when the application is to be tested on emulator. This folder contains minimum three types of files

- .dll
- .mif
- .bar

DLL contains compiled code of the app for emulator/simulator. MIF and BAR are same as above. The process to load the application into the emulator / simulator is as follows –

1. Copy this folder to ‘Examples’ folder of the BREW emulator / simulator which we are using (SDK 2.0.1/SDK 3.1.5) and rename the folder to the ‘Application Name’.
2. Rename the .dll and .mif files to the name same as the parent folder. Make sure that the folder name, .dll file name and the .mif file name should be same.
3. Copy the .mif file to the main folder ‘Examples’
4. Copy the device files / folder (Emulator) to the devices folder of the BREW emulator
5. Launch the emulator and then load the device on which we have to test the application.
6. Launch the AUT for testing.


Happy testing !.......................

Real time projects - 1

Friends apologies for a week delay in blogging but I must admit that I am running short of ideas. Actually , I am not getting expected comments on my posts and as result not getting new ideas. Anyways, for today I am just sharing one of the assignment I worked on. It was a testing project for a US based mobile marketing company …. Sorry cant disclose the name as I have signed NON DISCLOSURE AGREEMENT in the company where I am employed  .

In today’s blog, I am just sharing my experience working a project, different types of testing carried out and overall experience.

Now the project was testing of compatibility of a third party device (I will name this CON ) with various mobile devices. CON delivers contents to mobile devices through Bluetooth. This is a new form of marketing “Bluetooth Marketing”. The manufacturing company has deployed this device at various public places in US. These CON’s detects different mobile device (whose Bluetooth is ON) falling in its range (which is approx. 30 meters) and as soon as a mobile device comes into the range some flashy message is transferred to the device for example – DO YOU WANT TO GET SOME NEW WALLPAPERS FROM PEPSI or something like that. The mobile user has an option to accept/reject the request. Now there are two different cases –

Case # 1 – if the user accepts the request then he/she is provided with some free content which can be a wallpaper, animated GIF file, ring tone, video, flash, HTML/URL OR a simple J2ME application and also some paid contents.

Case # 2 – if the user rejects the request then the information of the user (BDA/COD details of the device) is passed over the network to other CON’s so that if the mobile device comes into range of some other CON then no request should be send.

Coming to the testing part , testing activities includes –
- Interaction between the CON and the mobile device whether CON is able to detect the mobile device , able to communicate with the mobile device, able to transfer the contents
- Compatibility testing of the mobile with respect to the various contents delivered.
- Usability of the various contents delivered on the mobile device.
- Functional testing of the J2ME applications.

Addition to this there was a web interface provided by the marketing team which contains pictorial representation of the CON’s and the devices surrounding them (Off course blue tooth of the mobile is kept ON otherwise CON would not have been detected it). This web based tool was programmed to display the details of the surrounding devices and also to transfer contents to these devices in case a device rejects a request by mistake.

The concept ‘Bluetooth Marketing’ is still very new and it was exiting working on this project as I tested almost 300 + mobile devices of various carriers in US/UK, moreover all the testing part was done on DeviceAnywhere (DAW) platform. Overall this project was a good learning experience in terms of knowledge of new mobile devices (BREW/PALM/Blackberry/WINDOWS platform). Above all the client was very cool and does not pressurize for work which was the most good part 

Writing effective test cases

Tips for creating effective test cases –

Today’s post it about writing effective test cases. Many times people use to jump on to test case creation just after reading the requirement which is not the right way. In this way your test cases will be very limited although covering the requirements. Here are points we should follow while writing test cases –

# 1 – Review the requirement document fully . Study all the requirements of the module you are working on thoroughly and make notes. If possible try to get detail understanding of the requirements by discussing with the BA team members. It’s better to make a query log and document all the queries and corresponding resolutions and share it with team. This will help in developing team confidence on you and your confidence on the system to be build. This will also help in validating your understanding on the system.

# 2 – Do brainstorming, after reading the requirements discuss it with peer members. Make notes of this discussion. Such discussion will help in deriving more test cases which one might miss. Remember everyone has different perspective and such discussions can help an individual to find out more cases for testing the application.

# 3 – After doing the above exercises do not jump onto the test case creation part rather first prepare high level test scenarios in relation to the requirements. You may/may not discuss these scenarios with the peer members.

# 4 – it is advisable to have a “Test Case Checklist” maintained at your end containing the general rules to be followed while creating the test cases like naming conventions should be in accordance with what has been decided, spelling mistakes should not be there, some specific words that should/should not be used in test cases and so on. Many a times organization have such checklist ready in case it is not then you can maintain your own. Such checklist helps a lot while submitting the test cases for peer / lead review and saves a lot of review time.

# 5 – Once you are ready with the high level test scenarios then move on the test case creation wherein you can create test cases in the organization specific template mentioning the detailed tests steps, test data, expected results and other related details.

# 6 – Have a “Traceability Matrix” . In case your organization does not maintains it, maintain one at your end. This is for the obvious reason for mapping the requirements with the test cases. It will also help in finding out the left requirements and the redundant test cases if any.

Thumb Rule – Do enough , fast and effective paper work/brainstorming before moving on to actual test case creation.

Bug found in iGoogle

Friends, today’s post is in continuation of my previous posts “Bugs found in routine internet surfing – 1 “ . today I am sharing some bugs observed in igoogle

Bug # 1 – Count of the emails does not increments/decrements (REFRESHES) properly on deleting/receiving of emails

Preconditions – User should have added ‘GMAIL PREVIEW’ feature on its igoogle homepage.
Browser – Mozilla Firefox V 3.0.19

Steps –
1. Open the igoogle webpage and log into your account with a valid user name password. (URL - http://www.google.com/ig)
2. On the home page locate the GMAIL PREVIEW section and click on INBOX link
3. Note the current count of the emails (Is should be In format 1 - 50 of about 500)
4. Select an email to delete (by checking the checkbox against it) and click on DELETE button.
5. Observe the count now

Actual Result – the count of emails does not get decrements and it still shows the old count number. Pressing the refresh link / browser refresh button also shows the same count.

But # 2 – In some cases it has been observed that the email count increments / decrements by 10 upon receiving /deleting an email

Regression Testing and Retesting

These two terms are often used interchangeably by inexperienced testers OR experienced developers/PM’s who are not known to testing. Anyways, I am not pointing to them in this post but just telling the difference between the two terms.

First Retesting , it is the term which is used to describe the testing effort of verifying the bugs OR defects logged in for the previous build and taking appropriate action on them. By appropriate action , I mean to close/defer/reopen the defect. To be specific “Retesting” is a subset of “Regression Testing”. Coming on the super set “Regression Testing” , it is testing of the module(s) in which bugs where logged for previous build and also the interrelated modules just to make sure that the bug fixes has not introduced new bugs into the system and the modules are working fine overall. Regression testing has wider scope as compared to Retesting because in the former we are covering the whole module and also the other interrelated modules and not just concentrating on the bug fixes only.

Thanks & Regards,
Amit

Bug in Facebook

One of my close friend shared a bug in facbook. Have a look on the image and notice the contents inside the red rectangular box on the left side and botom right side

Usability bug on blogger website

Usability bug on blogger website

Description – Website is not consistent, there is no SIGNOUT option available on ‘VIEW PROFILE’ page

Priority – Low. (I am not sure how many customers reported it)
Severity – Medium

Steps –
1. Login to your blog with valid user name and password.
2. On the dashboard , click on the ‘VIEW BLO’ link to view any blog.
3. On the respective blog , click on ‘View my complete profile’ link to land on your profile page.

Actual Result – On the profile page there is no link to get back to the blog/dashboard page. User has to explicitly click the BACK button on the browser.

As per usability point of view this is an issue related to ease of navigation. Also there is no way by which a user can submit the issues which he find during his blogging / normal browsing of the /blogs & website

Software Testing is not an easy job

Testing is not an easy job as people think about it. Everybody from the project manger to developers feels that software testing is a easy job in which you have to just test OR validate the software against the written set of requirements and for this ask they write the test cases for it and execute them on the product. This is totally true but this is just one side of the whole picture. Every software testers primary job is to test the software and make sure that it working as per the requirements. But here is the twist for performing this task he has to –

- Have thorough knowledge of the system under development which can be as vast as knowledge of the different modules of the system how these modules interact with each other, data flow between these modules and many more things. In contrast to this developers are confined to their individual modules, they are more concerned with the output of their module and many times it happens that they shed of their responsibilities by saying “I am sending data properly and in desired format from my screen/module (Say Module A) but if other module (Say Module B) is not fetching it then it’s not my job” . At this note, testers job is to find out this integration defect and let them know what is expected so a testers should know the functionality of module A , module B and the purpose of integrating these modules.
- Testers need to have ‘End User’ mindset to test the software. They need to think in the same way as the end users uses the software. At the same time they need to follow the documented requirements only then they will be able to deliver quality product. So they have a dual responsibility of their shoulders.
- Moreover they need to have domain knowledge of the industry which the software is catering and the knowledge of the current process practiced by the client. As for developers this is not as important.
- They need to test the software on wide range of data as against small set of data tested by developers in unit testing phase.

So please do not consider testing as easy job !.................

Very very minor bug on blog site

A very minor bug on blogger.com

I am new to this site and have started active blogging just a month back. There is one minor defect which I observed on this website. When you post a new article/post on the website, on the bottom it says ‘0 Comments’ which is OK and can be accepted. But as you post a comment on the post the messages says ‘1 Comments’. Now are there numerous comments. There is only one comment so it should be ‘1 Comment’ OR even ‘1 Comment(s)’ is fine.

I discussed the same thing with my colleague a fellow tester he also agrees that ideally it should ‘ 1 COMMENT’ but then he also told me about his experience where he reported the same defect in one of the project but it got rejected with the comment “It is as implemented”

What’s your thought..

Thanks & Regards,

Amit

Bugs found in routine internet surfing – 1

Bugs found in routine internet surfing – 1

Just starting off a new series of post where I thought of posting the bugs which I found in the routine internet surfing. As I am new to blogging off course this idea came to me while surfing different blogs and thought of implementing it as I want to make a routine practice of writing blogs and making my blog popular. Anyways, just sharing two interesting bugs with you –

Bug # 1 > High priority and medium severity (As per my perception)

Short description – News Paper Website missing with the value of one city in the dropdown on the home page

I use to visit website on local newspaper “Danik Bhaskar” (URL - www.bhaskar.com) daily to look on some local news of the city I was brought up – GWALIOR (Madhya Pradesh). Now there is dropdown to select the respective city edition on the home page. But there is no entry of GWALIOR in this drop down. However when I select the state (Madhya Pradesh) hyperlink and lands on the state home page and look on for the available cities in the dropdown on this page , I am able to locate GWALIOR and on selecting it I navigates to the respective page (Home Page of Gwalior News)

Now as per my perception it’s a high priority bug for obvious reasons as the drop down on home page is missing a value and there is no way to navigate to the respective city’s home page form the home page. However all other values are present and user can navigate to other cities pages. It is medium severity bug because user has some other way round to navigate to the respective city home page.


Bug # 2 > High priority and High severity (As per my perception)

Short description – Wrong address info shown on the ‘Contact Us’ section of a company’s website.

Recently , I visited website of a company and found that the address shown in the ‘Contact Us’ section is confusing for customers. Actually, the address written was different and the corresponding Google map was pointing to a different address , though in the same city.

As per my perception, this is high priority and high severity defect as the user gets confused as to which address he/she should make correspondence to.

Just FYI, I sent an email to the concerned company and was amazed to see the reply of the CEO/Founder accepting its a bug.

End of this part, will keep posting if something new found.. till then … happy browsing and as always comments/suggestions are welcome............ .


Thanks & Regards,
Amit

We the TESTERS

Today’s post is about the pains of software testing industry. In my 4.5 years of experience I have not seen a company who treats testers as a respected team members. When It comes to testing, most of the company’s pay less to the tester as compared to developers. Testers are not the respected members of project team. Developers think that it is because of them testers are getting jobs. Yes its true because these people make mistakes very often. I have faced this numerous times that developers have given days from testing schedules. Developers take their own time , eats up testers time and then mangers expect that testing should be perfect and no bugs should be missed. I don’t know how is this possible. Further when it comes to any issue reported in UAT / by client these mangers come to testers asking how this happened and the developers just shed off the responsibilities by saying ask testers. Many a times developers spend whole day on some small issues and release the builds to the testing team at EOD expecting the results by tomorrow morning.

Come on give me a break we the TESTERS are equally important to developers, we too have families to support and spend time with …………..

Basics of Software Testing - 3

Friends, starting the first post of may about a basic document in testing “TEST CASE”.

Test case – A test case is one of the major document prepared while testing any software. Generally defined test case is document containing some precondition , steps to be followed, test data to be given (Optional as some tests may not contain any test data) , Expected results after performing the steps to be executed for testing a particular requirement. Thus a test case must contain following essential components –
- Test case ID / #
- Precondition.
- Objective of the test.
- Steps to be followed.
- Test data.
- Expected Result.

Apart from the above, some organizations include other heads in test case document such as –
- Defect ID
- Priority
- Type
- Requirement ID/#

Now coming on to details of above heads -

Test case ID is the unique identifier for identifying a particular test case. It can be number/alphanumeric as agreed upon.
Precondition is a state/condition which is essential to execute the given test case. A very-2 common Example if I am executing test case(s) for testing a login web page then the preconditions can be as follows –
- Browser should be installed and running on the PC.
- URL for login should be present.
Objective of the test case is the scenario what we are testing. Continuing the above example my objective is to test the functionality of the login screen. There can be another test case with the objective of testing the user interface of the login screen.
Steps to be followed are the detail sequence of steps to be followed while executing a particular test case
Test data is the data to be keyed in while executing the test case. Continuing the login screen example, test data can be User Name, Password.
Expected Result is the output/results expected after performing the steps as mentioned in “Steps to be followed” and with the test data keyed in.
Actual Result is the actual output observed after executing steps given in the test case. It may/may not differ from the Expected Result and it determines the status of the test case.
Status of the test case can be PASS/FAIL based on the actual result. It can also be NOT RUN/TESTED based on the execution status.
Defect ID is the unique identifier of the defect logged against the failed test case.
Priority of the test case is how important the test case is . It can be HIGH/MEDIUM/LOW or P1/P2/P3 depending on how critical that test case is OR how important the test case is. It is determined by the functionality/scenarios that the test case is covering.
Type of the test case can be functional, UI, database depending on the requirement the test case is covering.
Requirement ID is the unique identifier of the requirement which the test case is covering. The purpose of including the requirement is in the test case document is just to make out the each and every requirement is covered and also to determine that which test case is covering which requirement.

Based on the above, we can now design a generic template to write test case(s) which can be as follows –

Precondition :

Test Case ID
Priority
Type
Requirement ID
Objective
Steps to be followed
Test Data
Expected Result
Actual Result
Status
Defect ID

Some basic points while preparing/reviewing/executing test cases –
- Read the requirement document fully and get the full understanding of the requirements.
- First prepare high level test cases OR test scenarios describing the what we are going to test and possible different states of what is to be tested.
- After this move on to detail test case writing with the steps, test data, expected results.
- Have a checklist ready before preparing test cases this will help at the time of reviewing the test cases. This checklist should not contain details but should point out generic points applicable to all the test cases. While submitting the test cases for further review validate the test cases against this checklist and submit it along with the test case document.
- While reviewing the test cases, reviewer should also validate the test cases against the checklist.
- All the review points should be documented in a proper format. Avoid sending review points in the email instead document in a proper document and send it. A separate copy can be kept at a central repository. Apart from this, another advantage is that document can be amended further. As with email, the trail gets long and its very tedious to keep a track of it.
- While executing the test cases try to indentify the important test cases and prioritize them. This will help to identify test cases which are to be executed in ‘Sanity level’ testing on the forthcoming releases.
- Also try to execute tests with combinations of test data other than the documented test data.


END OF POST. As always, suggestions/comments are most welcome.

Thanks & Regards,
Amit