Testing Challenge - Puzzle #5

So here we go again with a puzzle that will require you to send me questions in order to solve this one. I'll start these with an easy one so you might get this even with the first question.

Please remember, I don't want you to ruin the puzzle for anyone in the comments section so if you want to solve this, send me an e-mail or ping over Twitter so we can sort out the details. Thanks!

There was a road construction and a lot of people didn’t like it. After a while of constructions, people on pension started calling an old lady and complain to her about the construction, nevertheless she wasn’t part of the firm or had anything to do with them. Can you explain why this happened?

Testing Challenge - Puzzle #4

This time we will talk about trains. Some of you are more familiar with them than others which might give a helping edge, but anyone with good questioning skills will solve this.

Please remember, I don't want you to ruin the puzzle for anyone in the comments section so if you want to solve this, send me an e-mail or ping over Twitter so we can sort out the details. Thanks!

Two trains are heading each others and will crash in a matter of seconds. There are no secondary tracks and the brakes don't work. How can the accident be avoided?

Testing Challenge - Puzzle #3

This is the second lateral puzzle. I got huge help from Ilari Henrik Aegerter (www.ilari.com/blog/), James Bach (www.satisfice.com/blog/), Pekka Marjamäki (www.how-do-i-test.blogspot.com/) and Michael Bolton (www.developsense.com/blog/). I'd like to thank them for helping with the setup, clarifying a lot of questions, bringing insights and of course a lot of good time!

Please remember, I don't want you to ruin the puzzle for anyone in the comments section so if you want to solve this, send me an e-mail or ping over Twitter so we can sort out the details. Thanks!

There is a 15 year old boy studying in a high school. He loves ice hockey and is the best of the team from his year. The team has been excellent in the high school championships. Recently, the dean and the teacher’s council had a meeting where they decided he is so good they must dismiss him from the team. Explain why.

Testing Challenge - Puzzle #2

This is the first (they might get a bit harder after the easy start) of the "yes/no/not relevant" kind of lateral puzzle I am publishing. More will follow. Please remember, I don't want you to ruin the puzzle for anyone in the comments section so if you want to solve this, send me an e-mail or ping over Twitter so we can sort out the details. Thanks!

In the world championships of  relay running in 2654, the Chinese team will be the last to cross over the finish line (as in, the slowest team). However, they still won. Explain why this happened.

Testing Challenge - Puzzle #1

After thinking about this for a long time, I decided I will start publishing puzzles I have made. Because I keep coming up with new ones also, most likely I will add them here every now and then.

I have not yet fully decided, but my initial idea was to have problem solving/mathematical/logical puzzles in the blog so that everyone can try to solve them here and lateral/creative puzzles only presented with the setup. If a reader would be interested to solve a puzzle of the latter kind, we could do it for example over Skype or Twitter. I am also planning to add these to the TdT Cluj-Napoca (if you don't know what that is, check out http://tabaradetestare.ro/) workshops, but maybe more about that later.

Please remember, I don't want you to ruin the puzzle for anyone in the comments section so if you want to solve this, send me an e-mail or ping over Twitter so we can sort out the details. Thanks!

So here is the first logical one!

Continue the series (as in, replace X's with correct letters) and present your logic:
gra avar rvtXX

My Answers to 18 Testing Challenges from Santhosh Tuppad

My friend and a great tester Santhosh Tuppad (https://twitter.com/#!/santhoshst) got an idea of making a testing competition. He put the questions on his blog (http://www.tuppad.com/blog/) and mentioned everyone can participate. I thought it would be a cool thing for practicing my thinking and seeing things from another perspective. My answers were written without too much effort on the visual side, more like a collection of thoughts. They seem to be rather lengthy too. I’d love to hear your comments on my answers so please take the time to read and reply. Here are the questions and answers:

1. What if you click on something (A hyperlink) and to process or navigate to that webpage you need to be signed in? Currently, you are not signed in. Should you be taken to Sign up form or Sign in form? What is the better solution that you can provide?

Firstly, this answer should be included in all answers below: using passwords is an outdated way to handle authorization. Has been for years already. I would have passwords, if someone sees value in using them, for example with TV Guides and online magazines, but in no system that includes sensitive information. Now to the questions in hand!

Making a few assumptions here to get started… Let’s say the “something” would be: you choose an item to shopping cart and click “pay”. The site would require user to be logged in to continue. The first-come-to-mind option would be to have “Login with credentials here” view with an additional option “Don’t have an account yet? Click here!” for registering a new account.

If I would be somewhere in a completely different place, would be redirected to another site and asked to login, I would prefer for example to see what I am about to enter/access. In the shopping cart example I already knew that, but it’s not the same for all hyperlinks.

The question comes down to “will the webpage know if I have an account or not”. If the webpage doesn’t know whether you have an account or not, both options should be visible. If the webpage knows you have an account, login would be visible. If the webpage knows you don’t have an account, registering would be visible. Considering, the webpage has little knowledge who is actually using the computer, knowing if you have an account or not is tricky. A cookie might be present, but that could actually let a “wrong” user to login.

2. Using “Close” naming convention to go back to the homepage is good or it should be named as “Cancel” or it is not really required because there is a “Home” link which is accessible. What are your thoughts?

I assume this in some web page because you mention “homepage”. What would you close in this case to return to homepage? Specifically, if you close something, do you need to return to homepage or would it be open in the background and you would close a popup window? If this is the case, but homepage would not be on the background, what would be there?

Commonly, I would say it’s good to have a few different options to return to homepage because people are used to navigate in different manners.

Returning back to the Close vs. Cancel. Close could be usable when there is actually something to be closed, such as a popup window. Cancel could be usable when user is for example filling up a registration form and decides he doesn’t want to complete it. A context where both could be used would be for example a Flash app appearing on the page. (A concrete example: open a car manufacturer web page, choose a car model, click “customize” which opens a Flash app over the page where you can adjust the configuration of the car. In this case, we could have buttons Close and Cancel – maybe even Back and Back to Home Page.)

3. Logout should be placed on top right hand side? What if it is on the top left hand side or in the left hand sidebar which is menu widget like “My Profile”, “Change Password” etc. – Is it a problem or what is your thought process?

I am used to have logout on the top right, this is how most web pages work. I have asked around from people to put their finger on paper where they think certain functionality exists and “logout finger” goes on top right 100% of the times. This makes sense because many people think “logout” is a way to close the application they are using and most (GUI-based) operating systems provide closing functionality from the top right corner.

Same goes for profile etc. They tend to be on top right. I think this is good for example because people tend to look a bit on the up left (not top left, but a bit higher than center). So when focus is on the left side, it’s better to put insignificant information on the right so there is nothing extra on the concentration area. Another reason is that we look up right when we access so called “visually remembered images”, so when we want to remember something, we tend to move the eyes on top right. This, with the addition of “logout is on top right” to be almost an industry standard, speaks on behalf of keeping logout on top right. When looking right in general, we are trying to remember something instead of using our imagination to figure it out.

4. Current design of forgot password asks for username and security answer and then sends a link to e-mail inbox to set new password. How does “security answer” increase the cost of operations? Also, what questions do you frame for security questions?

I am not sure what “increase the cost of operations” means. Nevertheless, I will think about this situation from usability and security point of view. Maybe you will later explain what the original question meant. :-) (Note: later on I understood this, but as the answer was already written, I thought it would be fun to leave it as it is.)

Let’s consider I own an account in Amazon.com and my username is JariLaakso. I use Amazon rarely so sometimes I need to reset my password because I want it to be unique compared to any other password I use, I want it to be long etc. I go to “forgot password” in Amazon.com, enter “JariLaakso” as my ID and get a prompt about the security question. As this is my account, I want to be able to remember the answer always. I would most likely choose something from my life, such as names or places.

Now let’s consider a malicious user (for example ex-girlfriend) would want to break into my Amazon account. The user would need to know how to reply to that question in order to get my password reset. This is not such a big deal in case she doesn’t have access to my e-mail. Basically, I would be screwed (in the bad sense) if she would be able to access my e-mail already, so the risk doesn’t increase here too much. There would be a problem if the site would directly allow changing the password, but when e-mail is needed in between the risks are lower.

How about if the “forgot password” is for the e-mail? Where the link would be sent when answering correctly to the question? We have found a gold vein! Ultimately, when using this “security question” pipe, we would find out the account what we need to break in order to gain access to pretty much everything else. This rises up alarming concerns. Now when we add “Internet knows everything about you” spice to the soup, we have made a dinner with 5 courses. (From here on, everything is depending how “the last line of defense” is protected. A bad design is to allow a user to change the password when answering correctly to a security question. This is because there is so much information about a general user online already. A better option would be for example via SMS, but phone numbers change etc. so it’s not without risk either. A completely new method will be needed in the near future.)

5. If you had to design “Forgot Password” working, how would you do it and why? You are free to give different many functional designs.

I’ll start again by describing a sort of starting point. I want to do this to explain what kind of context I am thinking. In a different context, the feature could lean more towards security instead of usability. The feature exists on a web page (non-webmail). The page doesn’t store too much personal and/or sensitive information, however, identity thieves are not welcomed warmheartedly. Every user has a unique username and registration happens via a form on the page. I want to login to the site, but I have forgotten my password, so I click “Forgot Password” link. Layout and graphical part is not considered as I focus on how things would work.

There would be a text fields (Note: the information input on the text fields should not be remembered by the browser) where to write your e-mail address and username to get a “reset password” link in your e-mail. The sent e-mail would only have a link to reset the password if needed; it would not be done before clicking that link. Why? Because I want to prevent others from resetting my password. I would also restrict the amount one can reset the password consecutively. There would not be any sense to send multiple “you can reset your password from this link” e-mails to a user.

I would not add captcha because those can be circumvented and they annoy users. They might work for some registrations, but mostly irritate in this function.

In case the user doesn’t remember the e-mail address assigned to the ID or username for that site… most sites would have “security question” feature. I am not too fond of them, as described on previous answer, but I could still have a similar feature if the webpage would be for example something rather meaningless like “online TV Guide”. Even in this case, I would like answering correctly to this question to send an e-mail, but not reset the password etc. just like above. I still don’t want other people to reset my passwords or allow them to spam me from a service I am registered into.

… And just for the sake of argument, for a system which contains sensitive information, such as online bank, the abovementioned is not adequate enough. Basically, all current security systems can be cheated, but I think it’s satisfactory to demand a customer to visit an office in certain situations. This rises up so many branches of discussion, I better continue to the next question. :-)

6. There is neither account lockout policy nor captcha for the login or security answer forms; what kind of problems do you see with the current implementation and what do you propose?

This reminds me of the online banking issues I blogged (http://jarilaakso.blogspot.com/2012/02/internet-banking-experiences-from.html) earlier about. :-)

Not that captcha actually increases security, but let’s say not all jerks in the neighborhood can attack your service automatically if you add a captcha check. However, like said before, captcha can be circumvented (by machine and human force) and there are examples of both online.

The biggest problem comes from so called brute-force attack. The brute-force doesn’t have to be a “stupid one”, but can be firstly based on common password lists etc. It doesn’t even matter, because if you allow a user to guess all the way, they will figure out all usernames and passwords from the database(s).

What the question doesn’t mention is if there is a waiting algorithm between login attempts and if there is one, how will it function. For example, between 1^st and 2^nd login would be a mandatory 5 sec pause. Between 2^nd and 3^rd login would be a 25 sec pause. I guess you got the algorithm. This could be usable for a system where it is not of interest to let someone find out the usernames and passwords. However, it would also cause some usability issues, so I would not recommend it for “everyday systems”.

Not having a security answer form is not a problem itself. It becomes a problem if there is no other way to obtain username or password reset.

My suggestion would depend heavily on how secure the system should be. For example, if has to be really secure, I could also consider limiting access to certain IP ranges, having a certificate on the accessing machine, etc.

7. Well, it is about context and there are no best practices in general. What are your thoughts on usage of captcha? Where should they be used and why?

Oh so I wrote 3 pages of text to arrive here and see I have replied to some of the questions already above. Great!

Captcha is good if you want to limit the amount of potential hackers. It’s not good if you think it prevents anything else. In most cases, I hate captcha. I have actually seen some interesting research articles where computers have gotten better results than humans with captcha. :-)

If it would be up to me, I would not use captcha pretty much anywhere. The benefit is smaller than the loss of usability. If nothing else, I would come up with a completely new way to “verify” the user is not a machine.

8. If you are the solution architect for a retail website which has to be developed; what kind of questions would you ask with respect to “Scalability” purpose with respect to “Technology” being used for the website?

Do you mean hardware with technology? If we include also programming languages, what else do we add? What do other retail websites use? Why? Why not something else? (For example, Facebook goes with LAMP and that seems to work for them. Why? How about Amazon and eBay or maybe the local shop in my town?)

Things to consider about: user amount growth, user amount growth in different countries/continents, what kind of requests are sent from different actions, how long their processing takes, what loads the servers most, how to handle load balancing, what kind of user profiles we will have, what are most common functions and pages (for caching and optimization), etc.

Nevertheless (especially early) users will abandon the system if the response times are not magnificent, context will matter a lot. If your purpose is to sell locally and you expect 99% of traffic from a certain city, you might want to scale for that. However, remember also to read this http://www.zdnet.com/blog/foremski/report-51-of-web-site-traffic-is-non-human-and-mostly-malicious/2201 carefully.

More questions? Ok here are a few: Are we talking about administrative scalability, geographical scalability, load scalability or functional scalability? Maybe all? Maybe a combination? How will we scale out? Do we need to scale up? How will the database affect on this? How about system design?

9. How do you think “Deactivate Account” should work functionally keeping in mind about “Usability” & “Security” quality criteria?

Hopefully by deactivating the desired account from a user. :-)

There are 2 common good ways to handle deactivation:

1) Allow it for a user who has an active session

a. Benefit: Nobody would deactivate your account if they can’t use it.

b. Detriment: It’s more than easy to forget an active session for a computer which can be accessed by other people.

c. Solution: Either the confirmation as I describe next or canceling the deactivation if trying to login again within a certain amount of time.

2) Allow it for a user via a confirmation (e-mail for example)

a. Benefit: It’s not possible to deactivate someone’s account without confirming it.

b. Detriment: People tend to dislike “extra” confirmations.

c. Solution: (Considering this would not be a common use case someone anyway does often, it’s not a big problem, imo.) Explain the user clearly why the confirmation is needed. Ease up the confirmation process for example with a simple clickable link in an e-mail. Include still a “remorse time” during which the user would reactivate the account when signing in again (and maybe receiving a “do you want to keep this account active” question).

All in all, in my opinion, deactivating an account should not remove the data of that person from a system. The information might be needed/usable in the future.

10. For every registration, there is an e-mail sent with activation link. Once this activation link is used account is activated and a “Welcome E-mail” is sent to the end-users e-mail inbox. Now, list down the test ideas which could result in spamming if specific tests are not done.

I assume you are asking “which could result in spamming if specific code is not done” or something like that as testing won’t prevent anything. :-) So let’s start this from design point of view.

Prevent spamming a single user: The system should allow only 1 Activation Link e-mail to be sent to a specific e-mail address. The system should allow only 1 Welcome E-mail to be sent to a specific e-mail address. Exception: If a user deactivates/deletes the account, he should be allowed to register again with the same e-mail.

Prevent spamming multiple users: The system should have a limitation for incoming requests so a malicious user can’t register lots of accounts automatically.

Next step is how to test these. For single user point of view, you would test registering same account a few times and checking if your e-mail received more than 1 e-mail (ideally we would not print “this e-mail address already exists in the system” because it allows malicious users to gain information what e-mail addresses are used in the system). You would also test if it’s possible to register again after deactivation/deleting the account.

Spamming multiple users would require tests such as using many computers from different IP ranges, multiple computers from a small IP range (even from a single IP, like behind a NAT), single computer and to see if there is a delay in consecutive registrations.

Slightly out of the provided context, but closely related: When we are talking about registrations and such, we would also need to consider for example XSS, CSRF and SQL Injection tests. Not as a direct consequence from the Welcome E-mail and the link, but when requesting and storing user data, it would be good if the system would prevent giving those to malicious users.

11. In what different ways can you use “Tamper Data” add-on from “Mozilla Firefox” web browser? If you have not used it till date then how about exploring it and using it; then you can share your experience here.

Phew, finally a shorter answer! I am assuming the question is more about what different tests I do with Tamper Data.

I use it mostly for editing POST parameters, but it’s also usable to tracing HTTP requests/responses. There is a time provided and it can be used for example to see if some actions are causing more load on a server when a single user accesses the service. An example of a check that *might* be useful: measure response times when logging in with different usernames, longer wait could imply the username was found if the code firstly checks if the user exists and then compares the password.

I use Tamper Data also to view headers and sometimes to modify them. Cookie manipulation can be done, however, there are other tools as well for this purpose.

12. Application is being launched in a month from now and management has decided not to test for “Usability” or there are no testers in the team who can perform it and it is a web application. What is your take on this?

Firstly, I would say testers are not responsible for managing a project/product or making release decisions. Testing is done for obtaining information for decision-making. I could advocate what consequences this could have, but the decision to launch and bear the risks is not mine to carry.

That being said, I would question the question: What means to decide not to test for usability? Why the testers can’t perform usability testing? Are there any testers in the team? Who else understands usability? Was it considered in the design? Why it matters if the application is a web app? Why it matters if the application will be launched in the future if the decision is made no usability testing will be done?

In a real-life situation, in my context, working with a customer I am currently working in a project I am currently working, I would of course do something different. If the advocating would not result in convincing the management how much needed the usability testing is, I would:

1) Try to understand the decision and either agree or find a new way to persuade them (if still rejected, one could see it’s time to back off)

2) Use a short while of my own time to collect usability issues, thus either gaining confidence it’s in a good shape or to show what are the major problems I see

3) Talk with the devs or their lead(s) to see what actually could be used, whose responsibility the decision to fix would be, etc.

If the team could not handle usability testing, but the management would like it to be done, I would either involve myself on it, find other people from the company, get a third party involved (if allowed)… In this case, when the team can’t handle the testing, I see it so that I am not part of the team. This would greatly limit my options, however, as I already wrote, a lot of things can be done. The challenge would be to see what usability issues could be fixed before the release considering there might be still features to be implemented and other bugs to be fixed.

13. Share your experience wherein; the developer did not accept security vulnerability and you did great bug advocacy to prove that it is a bug and finally it was fixed. Even if it was not fixed then please let me know about what was the bug and how did you do bug advocacy without revealing the application / company details.

I can’t recall a case where my security bugs would have been marked “invalid” or something else suggesting they are “not accepted” by a developer. I remember cases where I have defended other testers’ bugs, but I can’t recall this happening for myself. I tend to write the risk and other relevant information on security bugs because it saves time on a long run. (So I have been told, lol.)

As a guideline, if you don’t know the dev and/or have history to know how he might understand your bugs, it’s a good idea to include your deductions and claims already in the initial bug report. Describing what problems a bug might cause, how they can be abused and what other kind of risks there are usually helps the dev to make the correct decision.

I also would like to note that in 99% of my context the devs don’t mark bugs “invalid” or “won’t fix” before a meeting is held with more people.

14. What do you have in your tester’s toolkit? Name at least 10 such tools or utilities. Please do not list like QTP, LoadRunner, SilkTest and such things. Something which you have discovered (Example: Process Explorer from SysInternals) on your own or from your colleague. If you can also share how you use it then it would be fantastic.

Considering this is a blog post that has been in my queue for a long time, I will try to just summarize a few things here. The way how the question is put makes me believe the tools you are referring to should not be test tools, but other tools which can be used in testing. If I misunderstood, please correct me. List is in order how they came to my mind.

#1 tool coming into my mind is FreeMind http://freemind.sourceforge.net/wiki/index.php/Download. It is a great free tool for making mindmaps.

#2 tool has to be Excel. Excel is fantastic for keeping notes, making reports, collecting data, comparing data etc.

#3 could be Firebug & Web Developer together because I use them so much. I use then for example to manipulate hidden elements, modify JS from a page, change input validation and enter all kind of values to forms. I put them in the same category for the fun of it, no particular logic.

#4 is Twitter as I use it sometimes to find out what people say about the companies/products we are testing. Twitter is not the only tool for this, but a really good one.

#5 shall be Paint as it’s a very lightweight application for simple picture modification. I could maybe write a blog post later about using pictures in web testing.

#6 must be something for test/check automation purposes: SVN. It just makes your life so much easier. SVN combined with a CI system is a really good combination for many projects.

#7 place goes for … paper and pencil! I love drawing pictures, writing fast notes, storing words/ideas etc. with paper and pencil. I have been thinking to buy a tablet of some kind for this, but not yet decided on a product.

#8 seems to be Total Commander. This is a fantastic lightweight application for Windows users who want to compare/synchronize folders, copy/delete files etc.

#9 I am not pointing to any single application, but applications that capture “video” of what you do with your computer as sometimes really awesome to track down ways to reproduce a bug, show what really happened etc.

#10 is YSlow which I haven’t used in a while. It’s good for measuring performance of different functionalities of a web page easily while you do other testing. (By the way, I was really tempted to put “my brain” as the last one. J)

15. Let us say there is a commenting feature for the blog post; there are 100 comments currently. How would you load / render every comment. Is it one by one or all 100 at once? Justify.

A few things to consider: how many people read your blog, what devices people use to read the blog, what kind of internet connection they have, do you want to have a compromise solution for everyone or optimize for a certain group, how long the comments are and do they contain other things than plain text too. There are more variables, but these seem to be the ones directing this kind of decision the most.

Now to the loading itself. If you choose to load one-by-one, you might face a situation where the server is getting a lot of requests just for the sake of loading text (if that is the case). That could lead to performance issues with many concurrent users. If you would like to be really clever, you could let the user decide this by clicking “how many comments you want to load at once” selection where you could have a few different options. Default being, for example, 10.

Basically, any decent server should be able to handle loading 100 messages (depending on size of course) really easy, but as we don’t know any details of the environment etc., I need to abstain from giving a clear answer on this one.

16. Have you ever done check automation using open-source tools? How did you identify the checks and what value did you add by automating them? Explain.

I’ve done GUI automation tests/checks for a few reasons. The reasons and analysis are too much to write here at the moment. Maybe I’ll write a blog post about it. Internet already has great writings from this subject and I’d like to recommend “Test automation snake oil” for a starter. Main idea is that I don’t have anything against or pro test automation without an analysis. The answer is as multifold as if the term “automation” would be replaced with “manual”. (Note: I’ve used for example Selenium for check automation. One can get pretty rapid feedback for smoke tests with it when combined with a continuous integration server.)

17. What kind of information do you gather before starting to test a software? (Example: Purpose of this application)

Depends a lot of the application, platform, test “phase” (security, performance etc.) and many other factors, like the customer. Let’s say we would have a web site to test and our job is to see how the functionalities work etc. I would most likely start with CIDTESTD mnemonic. Not because it’s the best one, but it’s a good starting point if you don’t have anything else to compare. CIDTESTD includes information about who customers are, manuals, documents, history, developers, test team, equipment and tools, schedule, test items and deliverables. That is a pretty comprehensive list to start with, but not everything needs to be specified. However, it is usually better the more you know.

In a more general manner, I feel it’s important to understand who uses the software, why they use it, what is my mission (what is expected from me), how much time I have, what kind of reporting is needed etc. I could also want to know if there are legal requirements for using/testing the software, restrictions on what systems it works/should work with, severe impacts on society due a bug (for example a nuclear weapon launch system) and for example if the software is working together with other systems such as banking software.

18. How do you achieve data coverage (Inputs coverage) for a specific form with text fields like mobile number, date of birth etc? There are so many character sets and how do you achieve the coverage? You could share your past experience. If not any then you can talk about how it could be done.

Firstly, I would note the coverage includes also outputs, not only inputs. Secondly, I would like to note I have used a lot of “checklists” for this and I review them with colleagues to see if someone comes up with new test ideas. That is great fun always! Thirdly, I must stress that this is somewhat case-by-case basis for example because with web services one can do so many different things with inputs.

One common way for me is to use automation for storing + giving variance for inputs. Second is that I tend to categorize (XSS, SQL injection, empty, too small, too big, way too big etc.) the tests and use sort of “equivalence classes” in the tests, as in I make assumptions “if X and Y passes, the class they represent is less-likely to be risky”.

I always add some sort of random tests in those equivalence classes if making the tests is cheap. For example, with a web service, you could leave your test computer send different kinds of inputs overnight and check fast in the morning if any input caused strange behavior/errors.

Overcoming Illusions on Testing – Part III

This entry (first part http://jarilaakso.blogspot.com/2012/02/illusions-on-testing-part-i-chimera.html and second part http://jarilaakso.blogspot.com/2012/02/reasons-of-illusions-on-testing-part-ii.html) has been on hold for quite a while; mostly because I have been focusing on other things. (I actually changed even the topic because I thought to write about the future later, in the future!) As we are building the Romanian testing community, I have actual work to do and I’m now a father, I didn’t feel motivated enough to keep on writing. The motivation is back and I’m hoping to start adding more posts in the near future. (If you wonder why I didn’t say “I didn’t have enough time”, it’s simply because it’s a lie. Time is a matter if prioritization. We have it, but not for everything.)

I this post, I will be referring to RST quite many times. This is because I like to use examples in my writing and there were fantastic examples in the course. I won’t, however, tell those examples so you don’t lose anything from the experience.

Without further ado, let’s dig in to the topic at hands. So we have seen detailed pre-scripting doesn’t work too well because a) it kills creativity, b) it doesn’t tolerate change too well, and c) people are just not that good in writing detailed instructions. Actually it does work. Just that it doesn’t work for excellent testing, but it works great for invoicing customers and setting up a fake quality assessment done by “even a stranger from the streets”. The latter means the turnover for testers doesn’t matter for the company.

RST has a few exercises around this topic. One was from the tester’s point of view and one from who was trying to write the script for the tester. The exercises are designed to fail with simple answers, which is fantastic from my point of view. So what to do in this case? Firstly, don’t start writing very detailed test scripts for testers. Secondly, to help improve your (data) coverage, you can always ask help from others. Great testers love to help other testers; and in most organizations I have seen, the developers are keen on working with this.

I mentioned in the previous posts that terminology seems challenging for testers. RST has a lot of keywords you should understand. Otherwise it’s hard to follow the discussions. At least James explained all the terms he was using and based on what I have seen for example here http://www.developsense.com/blog/2012/04/all-oracles-are-heuristic/, I am sure Michael is doing the same. The thing with terminology is that you don’t need a word listing for this. What you need to do is to start reading and talking with others and find a common language.

There was an observation earlier that women did better because they had more sense of context in their replies. I understand this easily from a Finnish point of view where we are told that women analyze things while men drink beer, eat sausages and find The Best Way. I could easily see myself fitting in that form still some years ago. How I got out from it? I think it was just a phase actually, but at the end it’s about what kind of people you gather around yourself and what do you do on your free time. I am advocating on reading and writing, but remember that you can get ideas from pretty much anywhere if you keep your eyes/ears open.

Yes, I covered only 3 of the 5 things I mentioned on earlier posts. Why? Because I want you to tell me what you think about them! 

Getting a Career in Software Testing

I was recently approached by Alfred (name changed to protect his identity as he removed the comment from my blog) who asked if I have any advice on what steps to take in order to train for and start a career in Software Testing. Well, Alfred, thank you for asking! You have already taken important steps; reading blogs and asking questions.

Your concern about ISTQB is a valid one. They do claim they are the industry standard, but that’s only based on the amount of people they have certified, instead of what actually matters. Don’t get fooled by their sales pitch, just focus on learning. Especially a beginner tester might seriously harm his learning when taking their certification. Essentially, they will teach a bad way to use their vocabulary, advocate there are best practices and require that a question can be answered correctly with only one choice.

If you are interested to learn software testing at home, you could do some of the following (and add your own ideas of course):

1)      Start writing a blog about your learning (please let me also know about it so I can read and comment)

2)      Read online material (as the Internet is full of itself, I recommend to stay with only a few sources in the beginning, such as BBST)

3)      Read and comment blogs on software testing (to get you started, here are some http://www.developsense.com/blog/, http://visible-quality.blogspot.com/, http://www.satisfice.com/blog/ and http://scott-barber.blogspot.com/)

4)      Join Twitter and start following testers (you can take a look for example on my list https://twitter.com/#!/jarilaakso/following as a starting point)

5)      Test software, write a report and either leave it on your blog publicly for everyone to comment or ask other testers, like me, to comment via e-mail or whatever you prefer
(Addition after posting already: forgot completely that you could search for conference videos)

If you want to work in software testing, you want to know companies, possibly join recruitment sites etc. Twitter works as a great channel to get to know about new job openings, to find out who is recruiting and for example to get yourself known. But no matter how good Twitter is, you will need to work a lot to be successful. It’s true there are a lot of people who don’t thrive for excellence and they do pretty well in their life. It’s up to you to decide which path you will take.

You mentioned you are based in England. There is a huge software testing market (go ahead for example to Monster.com and verify my claim) and you will have a lot of competition. Think how you want to separate from the crowd and use it in your advantage. For example, if I’d be recruiting at the moment, I would love to receive for example test reports and/or Youtube videos instead of the typical CV’s I tend to see. But hey, that’s just me, I can’t say what you should do, besides that follow your heart in your decisions.

I noticed you mentioned the “test is dead” part in your comment. Don’t take it literally. There is a continuously growing demand for good testing in the world. Think of it as an opportunity for people who want to do a good job, or even yet, an excellent job. The more there is software around us, there more there is request for testing.

If you have open questions, please leave a comment below or for example e-mail me. I will be glad to help everyone I can.

(This post was written after a suggestion in Twitter from "Mike Talks" https://twitter.com/#!/TestSheepNZ/status/199099553587273729)

What I learned in 3 days of RST Training

I'll start be noting this post doesn't include a complete list of things I learned in the training. The reality is that it's been 4 days since the training and I keep learning and understanding things differently. This describes the RST training really well. This course is not just about information and tricks. It will lift you up on a whole new level.

Testing is often understood as asking questions of/from the product. The course will give you hands-on exercises around questions. You will learn what to ask, how to ask, how to find information etc. You will also learn how to deal with situations when the replies are vague or non-existing.

Testers often need information and help, but feel alone with their challenges. You will learn to use resources in a creative manner. As an example, in one testing exercise, I realized I can gather information faster by asking from others what they found out instead of trying to solve the puzzle by myself. When time is limited, you need to be able to be creative with information gathering and this you will be learning each day on various exercises.

Software testing without tools can be a fun thing, but you will limit your testing if you do everything manually. This course will show you diverse tools, including for example randomizers and hacking applications, that help you along the way. When this happens, people will start telling about the tools they use in their testing. If you listen carefully, you will learn about many tools that help you in your testing. As an example, I taught a course colleague to use 3 different tools in one evening while we discussed about philosophy and ethics around web service testing.

Many times we see testers having fights around ethics in testing. RST includes a lot of philosophical discussion and examples about ethics. Not only you will learn what things to avoid but you will also learn how to deal with those situations and what things you can offer instead. You will learn to say no and advice what else to do.

Testers often find themselves in situations where they feel their words were taken out of context or misinterpreted. In RST you will learn to use safety language more effectively and you will learn why it's important to be used. You will not say anymore silly things like "we can ship this product", but you will advice the manager in charge to understand the results of your testing. You will not say "the product doesn't have bugs" because that could result in losing your credibility 5 seconds after a public release.

Commonly testers are talking about boundary testing, BVA and ECP. You will learn what boundary testing actually is. No, not the ISTQB version where you have a formula that fits in all situations. You will learn what it actually means, what is required from a tester and why it's simply wrong to say "most bugs are found around boundaries". You will also learn that the reason to this claim is because you are mostly testing around the boundaries! By clever examples and exercises, you will learn how this goes in real life.

A big part of tester's work is reporting. RST will teach you the different levels of reports, how you should do the reporting and what will give the most value to the stakeholders. After the course, you will see all reports with different eyes. Reporting used to take a big part of your time, but you will now be ready with tools and techniques that help you to minimize "waste work".

Often testers say their work is repetitive and doesn't need creativity. Yep, you guessed correctly, RST will teach you to think and work differently. You will learn to do testing in the hard way, which is also the most rewarding way. You won't anymore see the fake simplicity in software testing problems. Your eyes will open and soon you will find yourself doing the kind of testing that the big guys were talking about.

Oh yeah, and this is pretty much the first day!

RST = Required for all Software Testers

I’ll start by saying I am sorry the trilogy (http://jarilaakso.blogspot.com/2012/02/reasons-of-illusions-on-testing-part-ii.html) is missing the last part. The last part has been coming up a long time already, but due to some excuses, I have been delaying it. I will focus on it again, but currently I have something more urgent in my mind: RST (held by James Bach) and this whole week in Bucharest.

I was able to convince my boss to send me to Bucharest for the whole week to participate to RST and a peer-conference. I can’t actually take the whole credit for it; my boss is fantastic, really supportive and saw this as a great investment from many points of view. After the week, I’ll say to all the bosses around there who care about testing/quality: make the investment!

Before the RST begun, I made a list of questions I want to ask from James and a list of things I want to learn. Little did I know… but here are a few things I listed: how RST works in my context with projects that last sometimes years, how will this make me a better tester, and how will my reporting skills increase with 3 days of training. If RST would be anything like a traditional training, those would be really good questions, and maybe they still are, but what you get from RST is far more than that.

The whole idea of RST training rolls around doing exercises, showing the weak points of poor testing and challenging everyone mentally. RST gives tools to do excellent testing, including what are important questions and for example why testing nearby some (imaginary) boundaries is just not enough.  Now I’ve read earlier that RST should have more hands-on exercises. I counted we did at least 25 exercises which most comprised of multiple stages, a lot of verbal reporting, discussions with various stakeholders and applying RST in practice. Please bear in mind this is only a part of the training. It’s unbelievable how much of information you receive in these 3 intense days!

James is the toughest teacher I have ever seen and the better you do, the more he will push you. This is because he wants you to learn, not because he is mean. He has incredible skills to read people and activate their minds. Getting an answer from him is sometimes difficult, but if you are persistent enough, he will make you answer your own question! Like I noted earlier, this training gives you much more than you will ever imagine. You will understand even reporting on a whole new level after the course.

Quite commonly I use a lot of safety language and after a training I could say something in the lines of “I think I did pretty well because I had some good ideas”. Now there is nothing wrong in this, but this time I can say “I did really well because I had sharp questions, I talked about philosophy of testing and I showed I care strongly about testers’ ethics.” Why am I so confident, some might say overconfident, to state something like that? Because James is also a rewarding teacher and he Tweeted about it! In my world, this means the people who care about testing just all got to know a bit about me. What other training could possibly have anything even close to that?

If you want to participate in the course, but have hard time convincing your manager to get the funds approved, please send me an e-mail. I will be happy to help. It’s not about if your company can afford this, but more if they can afford not sending you to the training. It is maybe the most important training of your whole professional career. We need to make it happen!

Note: I need to do the training with Michael Bolton, too. Hopefully this year, but latest on next year.