Anyone can be a (bad) tester

Jonathan Roe recently wrote an article called “anyone can be a tester” (http://www.morethanfunctional.co.uk/1/post/2014/04/anyone-can-be-a-tester.html). After reading it, I exchanged a few comments with him and thought to leave a longer comment on his blog. Soon after starting to write that comment, I realized it fits better in my blog. Mostly because it reflects how I currently think about the subject.

In this blog entry, I will quote text from him in order to comment on the points and sentences he wrote. It’s entirely possible I have misunderstood some of his points or my quote doesn’t cover enough text, but by no means will I do that intentionally. I know he talks a lot about usability and user experience, but because he repeats “anyone can test” theme, I’ll pick up on that. Having that said, let’s begin.

“The problem with saying "anyone can test software" to someone who, you know, tests software for a living is that it seemingly devalues the career they've probably put a lot of time and effort into progressing. So I say this to anyone who thinks anyone can do software testing:

You're right. Have a biscuit.”

I prefer to add the qualifier “bad” in front of “tester” in the claim. Yes, it’s true that anyone can test software (the title of the article said “can be a tester”, not “can test”), so in that sense anyone can be a tester. But this is shallow. Anyone can be a programmer and a manager and a director and a CEO and whatnot, too. Not anyone can be a dentist because there are training requirements (http://en.wikipedia.org/wiki/Dentist#Training), at least in some countries. But we could say “anyone can pull out a tooth from someone”.

Yes, I am using loosely the word “anyone”. I don’t really mean anyone, but that’s not the point to me here today. My point is that pretty much anyone can be a bad tester and it takes effort to become good. Just like all testing is exploratory (up to some degree), good exploratory testing is on a completely different level than someone clicking here and there while chatting and browsing funny cat pictures.

When I pointed out in Twitter “anyone can test software badly” is a better expression from my point of view, Jonathan Roe replied:

“'badly' is irrelevant. If in testing they find a bug previously unfound, the testing was a success.”

I think we have a set of problems here: 

1) How would we know the bug was previously unfound? Maybe it was discussed with a programmer, but decided to be ignored. Maybe the PO ignored it. Maybe it’s not yet prioritized to be fixed. Maybe it’s not important. Maybe it only bugs someone who doesn’t matter that much. 

2) A bug is not an attribute of the product you are testing, but a relationship between the product and a person (or persons) at some time.  In other words, it’s not a bug if it doesn’t bug anyone; and this can change over time. 

3) Is testing a success if we spend a huge amount of money to find a trivial bug? It might be that a test was successful up to some degree, but even this is questionable. If the bug is not properly reported, will it be fixed? What if the team doesn’t understand the report at all? What if the report doesn’t have any means to reproduce the bug and a very complex set of actions is needed for that? How do we determine testing was a success? 4) By telling this publicly to people, effectively you promote the idea how easy software testing is and by doing that you are cutting the branch you are sitting on. You don’t say it directly, but when you say anyone can be a tester and all their bug findings make their testing a success, you are conveying the impression how ridiculously simple profession we have.

“Will they be able to find as many issues as someone who's been doing it for several years? Probably not. Will they be able to communicate the issue in such a clear and efficient way? I doubt it.

However, even if it means they can't test software well, that does not mean that they can't test software.

In fact, I recommend you get as many non-testers to test your software as possible.”

Ok, here we soften the punch a bit. If the point of the article is to explain why it’s beneficial to include more people testing a product, how about saying that from the start? When saying “anyone”, maybe the author meant stakeholders from the team. From my point of view, if one is not testing well, they are testing bad or mediocre at best. Most non-testers I’ve worked with did not reach the level what I’d call mediocre because I’m talking about a mediocre level software testing professional.

“You can use as many testing techniques and tools as you want but if it isn't very usable, the software's crap.”

I’d remind for starters that testing doesn’t make software more usable. I’m also not familiar with Jonathan Roe’s definition of “very usable”, but I know I’ve used a whole lot of software in my life that was utter crap from usability point of view and still they got the job done. Most of these were professional tools for which you don’t necessarily have any installation guide and configuration file needs to be built first. Four (4) of them have been tools for creating invoices based on worked hours. You know, the kind of software which costs a lot.

“If a user can't easily use the functionality or find the content, you might as well take your test sessions, your mindmaps and your automation scripts then insert them promptly and firmly up your arse.”

I need to check a few arses next time when I meet my friends who have been testing banking software. Especially online banking software. Until now, I haven’t seen good online banking software. I’ve seen extremely poor attempts and good-enough solutions, but they all have disappointed me in a way or another. Like the Finnish online bank where I can’t change my address to a foreign address without visiting their office in Finland. However, I’ve never changed my bank because of these problems. I’m not saying that’s an excuse for bad software, but I’m saying it’s good to remember not all software is what you test.

“They seemed very content just slagging off the 'old way' to test software, preaching about the importance of automated tests and using pretentious terms like 'test charter' (that means a test objective or mission for anyone who doesn't attend the church of James Bach).”

Sounds like they were slagging off the old way how they used to do software testing. (I don’t like the expression “to do software testing” because testing is much more than the visible actions we produce, but more about that some other time.) It could be they have learned something along the years and they are preaching about it now. It could be many things and I hope I can attend there at some point so I can talk with people to understand better.

What do you mean “pretentious term like ‘test charter’”? How is “charter” attempting to impress more than “mission”? A charter is a mission for a test session in Session-Based Testing Management. It’s not a mission for testing, but for a session of testing. I can recommend the church, by the way. Awesome hats, cloaks, and you should definitely see the altar.

“However, software testing recently seems to be all about using the fashionable terms and techniques instead of doing what a tester is supposed to do; provide information to the people in control to assist them in making decisions that will improve the product.”

I haven’t been there so I don’t know what is going on, but I find it rather important that we talk about what we do. I also find it important that we have terms for specific things, like “user experience”. When you name a technique, it’s easier to explain to others and apply in practice. How would you describe “user experience” without using those words? Seems handy to use those two words, in my opinion.

“Anyone can test software and that's what we should be telling people. We should be asking as many people as possible to test the software we're working on and listening to their feedback.”

That’s definitely not what we should be telling people. We have better ways to ask for their participation without pushing down our craft. For example, we can tell them “Hey we would like to hear different opinions about the usability of this product. Would you like to provide feedback?” Before a testers pairs up with a programmer, he won’t be invited there by the programmer saying “anyone can do this”.

“Welcome to my blog.”

Thanks! I don’t need you to agree with all my points, if any, but I’d like you to comment back. Welcome to my blog, too!