Jonathan Roe recently wrote an article called “anyone can be
a tester” (http://www.morethanfunctional.co.uk/1/post/2014/04/anyone-can-be-a-tester.html).
After reading it, I exchanged a few comments with him and thought to leave a
longer comment on his blog. Soon after starting to write that comment, I
realized it fits better in my blog. Mostly because it reflects how I currently
think about the subject.
In this blog entry, I will quote text from him in order to
comment on the points and sentences he wrote. It’s entirely possible I have
misunderstood some of his points or my quote doesn’t cover enough text, but by
no means will I do that intentionally. I know he talks a lot about usability
and user experience, but because he repeats “anyone can test” theme, I’ll pick
up on that. Having that said, let’s begin.
“The problem with saying
"anyone can test software" to someone who, you know, tests software
for a living is that it seemingly devalues the career they've probably put a
lot of time and effort into progressing. So I say this to anyone who thinks
anyone can do software testing:
You're right. Have a biscuit.”
I prefer to add the qualifier “bad” in front of “tester” in
the claim. Yes, it’s true that anyone can test software (the title of the
article said “can be a tester”, not “can test”), so in that sense anyone can be
a tester. But this is shallow. Anyone can be a programmer and a manager and a director
and a CEO and whatnot, too. Not anyone can be a dentist because there are
training requirements (http://en.wikipedia.org/wiki/Dentist#Training),
at least in some countries. But we could say “anyone can pull out a tooth from
someone”.
Yes, I am using loosely the word “anyone”. I don’t really
mean anyone, but that’s not the point
to me here today. My point is that pretty much anyone can be a bad tester and it
takes effort to become good. Just like all testing is exploratory (up to some
degree), good exploratory testing is on a completely different level than
someone clicking here and there while chatting and browsing funny cat pictures.
When I pointed out in Twitter “anyone can test software
badly” is a better expression from my point of view, Jonathan Roe replied:
“'badly' is irrelevant. If in
testing they find a bug previously unfound, the testing was a success.”
I think we have a set of problems here:
1) How would we know
the bug was previously unfound? Maybe it was discussed with a programmer, but
decided to be ignored. Maybe the PO ignored it. Maybe it’s not yet prioritized
to be fixed. Maybe it’s not important. Maybe it only bugs someone who doesn’t
matter that much.
2) A bug is not an attribute of the product you are testing,
but a relationship between the product and a person (or persons) at some time. In other words, it’s not a bug if it doesn’t
bug anyone; and this can change over time.
3) Is testing a success if we spend
a huge amount of money to find a trivial bug? It might be that a test was successful
up to some degree, but even this is questionable. If the bug is not properly
reported, will it be fixed? What if the team doesn’t understand the report at
all? What if the report doesn’t have any means to reproduce the bug and a very
complex set of actions is needed for that? How do we determine testing was a
success? 4) By telling this publicly to people, effectively you promote the
idea how easy software testing is and by doing that you are cutting the branch
you are sitting on. You don’t say it directly, but when you say anyone can be a
tester and all their bug findings make their testing a success, you are
conveying the impression how ridiculously simple profession we have.
“Will they be able to find as
many issues as someone who's been doing it for several years? Probably not.
Will they be able to communicate the issue in such a clear and efficient way? I
doubt it.
However, even if it means they
can't test software well, that does not mean that they can't test software.
In fact, I recommend you get as
many non-testers to test your software as possible.”
Ok, here we soften the punch a bit. If the point of the
article is to explain why it’s beneficial to include more people testing a
product, how about saying that from the start? When saying “anyone”, maybe the
author meant stakeholders from the team. From my point of view, if one is not
testing well, they are testing bad or mediocre at best. Most non-testers I’ve
worked with did not reach the level what I’d call mediocre because I’m talking
about a mediocre level software testing professional.
“You can use as many testing
techniques and tools as you want but if it isn't very usable, the software's
crap.”
I’d remind for starters that testing doesn’t make software
more usable. I’m also not familiar with Jonathan Roe’s definition of “very
usable”, but I know I’ve used a whole lot of software in my life that was utter
crap from usability point of view and still they got the job done. Most of these
were professional tools for which you don’t necessarily have any installation
guide and configuration file needs to be built first. Four (4) of them have
been tools for creating invoices based on worked hours. You know, the kind of
software which costs a lot.
“If a user can't easily use the
functionality or find the content, you might as well take your test sessions,
your mindmaps and your automation scripts then insert them promptly and firmly
up your arse.”
I need to check a few arses next time when I meet my friends
who have been testing banking software. Especially online banking software.
Until now, I haven’t seen good online banking software. I’ve seen extremely
poor attempts and good-enough solutions, but they all have disappointed me in a
way or another. Like the Finnish online bank where I can’t change my address to
a foreign address without visiting their office in Finland. However, I’ve never
changed my bank because of these problems. I’m not saying that’s an excuse for
bad software, but I’m saying it’s good to remember not all software is what you
test.
“They seemed very content just slagging
off the 'old way' to test software, preaching about the importance of automated
tests and using pretentious terms like 'test charter' (that means a test
objective or mission for anyone who doesn't attend the church of James Bach).”
Sounds like they were slagging off the old way how they used
to do software testing. (I don’t like the expression “to do software testing”
because testing is much more than the visible actions we produce, but more
about that some other time.) It could be they have learned something along the
years and they are preaching about it now. It could be many things and I hope I
can attend there at some point so I can talk with people to understand better.
What do you mean “pretentious term like ‘test charter’”? How
is “charter” attempting to impress more than “mission”? A charter is a mission
for a test session in Session-Based Testing Management. It’s not a mission for
testing, but for a session of testing. I can recommend the church, by the way.
Awesome hats, cloaks, and you should definitely see the altar.
“However, software testing
recently seems to be all about using the fashionable terms and techniques
instead of doing what a tester is supposed to do; provide information to the
people in control to assist them in making decisions that will improve the
product.”
I haven’t been there so I don’t know what is going on, but I
find it rather important that we talk about what we do. I also find it
important that we have terms for specific things, like “user experience”. When
you name a technique, it’s easier to explain to others and apply in practice. How
would you describe “user experience” without using those words? Seems handy to
use those two words, in my opinion.
“Anyone can test software and
that's what we should be telling people. We should be asking as many people as
possible to test the software we're working on and listening to their feedback.”
That’s definitely not what we should be telling people. We
have better ways to ask for their participation without pushing down our craft.
For example, we can tell them “Hey we would like to hear different opinions
about the usability of this product. Would you like to provide feedback?” Before
a testers pairs up with a programmer, he won’t be invited there by the
programmer saying “anyone can do this”.
“Welcome to my blog.”
Thanks! I don’t need you to agree with all my points, if
any, but I’d like you to comment back. Welcome to my blog, too!
