Saturday, April 5, 2014

Anyone can be a (bad) tester

Jonathan Roe recently wrote an article called “anyone can be a tester” (http://www.morethanfunctional.co.uk/1/post/2014/04/anyone-can-be-a-tester.html). After reading it, I exchanged a few comments with him and thought to leave a longer comment on his blog. Soon after starting to write that comment, I realized it fits better in my blog. Mostly because it reflects how I currently think about the subject.

In this blog entry, I will quote text from him in order to comment on the points and sentences he wrote. It’s entirely possible I have misunderstood some of his points or my quote doesn’t cover enough text, but by no means will I do that intentionally. I know he talks a lot about usability and user experience, but because he repeats “anyone can test” theme, I’ll pick up on that. Having that said, let’s begin.

“The problem with saying "anyone can test software" to someone who, you know, tests software for a living is that it seemingly devalues the career they've probably put a lot of time and effort into progressing. So I say this to anyone who thinks anyone can do software testing:
You're right. Have a biscuit.”

I prefer to add the qualifier “bad” in front of “tester” in the claim. Yes, it’s true that anyone can test software (the title of the article said “can be a tester”, not “can test”), so in that sense anyone can be a tester. But this is shallow. Anyone can be a programmer and a manager and a director and a CEO and whatnot, too. Not anyone can be a dentist because there are training requirements (http://en.wikipedia.org/wiki/Dentist#Training), at least in some countries. But we could say “anyone can pull out a tooth from someone”.

Yes, I am using loosely the word “anyone”. I don’t really mean anyone, but that’s not the point to me here today. My point is that pretty much anyone can be a bad tester and it takes effort to become good. Just like all testing is exploratory (up to some degree), good exploratory testing is on a completely different level than someone clicking here and there while chatting and browsing funny cat pictures.

When I pointed out in Twitter “anyone can test software badly” is a better expression from my point of view, Jonathan Roe replied:
“'badly' is irrelevant. If in testing they find a bug previously unfound, the testing was a success.”

I think we have a set of problems here: 
1) How would we know the bug was previously unfound? Maybe it was discussed with a programmer, but decided to be ignored. Maybe the PO ignored it. Maybe it’s not yet prioritized to be fixed. Maybe it’s not important. Maybe it only bugs someone who doesn’t matter that much. 
2) A bug is not an attribute of the product you are testing, but a relationship between the product and a person (or persons) at some time.  In other words, it’s not a bug if it doesn’t bug anyone; and this can change over time. 
3) Is testing a success if we spend a huge amount of money to find a trivial bug? It might be that a test was successful up to some degree, but even this is questionable. If the bug is not properly reported, will it be fixed? What if the team doesn’t understand the report at all? What if the report doesn’t have any means to reproduce the bug and a very complex set of actions is needed for that? How do we determine testing was a success? 4) By telling this publicly to people, effectively you promote the idea how easy software testing is and by doing that you are cutting the branch you are sitting on. You don’t say it directly, but when you say anyone can be a tester and all their bug findings make their testing a success, you are conveying the impression how ridiculously simple profession we have.

“Will they be able to find as many issues as someone who's been doing it for several years? Probably not. Will they be able to communicate the issue in such a clear and efficient way? I doubt it.
However, even if it means they can't test software well, that does not mean that they can't test software.
In fact, I recommend you get as many non-testers to test your software as possible.”

Ok, here we soften the punch a bit. If the point of the article is to explain why it’s beneficial to include more people testing a product, how about saying that from the start? When saying “anyone”, maybe the author meant stakeholders from the team. From my point of view, if one is not testing well, they are testing bad or mediocre at best. Most non-testers I’ve worked with did not reach the level what I’d call mediocre because I’m talking about a mediocre level software testing professional.

“You can use as many testing techniques and tools as you want but if it isn't very usable, the software's crap.”

I’d remind for starters that testing doesn’t make software more usable. I’m also not familiar with Jonathan Roe’s definition of “very usable”, but I know I’ve used a whole lot of software in my life that was utter crap from usability point of view and still they got the job done. Most of these were professional tools for which you don’t necessarily have any installation guide and configuration file needs to be built first. Four (4) of them have been tools for creating invoices based on worked hours. You know, the kind of software which costs a lot.

“If a user can't easily use the functionality or find the content, you might as well take your test sessions, your mindmaps and your automation scripts then insert them promptly and firmly up your arse.”

I need to check a few arses next time when I meet my friends who have been testing banking software. Especially online banking software. Until now, I haven’t seen good online banking software. I’ve seen extremely poor attempts and good-enough solutions, but they all have disappointed me in a way or another. Like the Finnish online bank where I can’t change my address to a foreign address without visiting their office in Finland. However, I’ve never changed my bank because of these problems. I’m not saying that’s an excuse for bad software, but I’m saying it’s good to remember not all software is what you test.

“They seemed very content just slagging off the 'old way' to test software, preaching about the importance of automated tests and using pretentious terms like 'test charter' (that means a test objective or mission for anyone who doesn't attend the church of James Bach).”

Sounds like they were slagging off the old way how they used to do software testing. (I don’t like the expression “to do software testing” because testing is much more than the visible actions we produce, but more about that some other time.) It could be they have learned something along the years and they are preaching about it now. It could be many things and I hope I can attend there at some point so I can talk with people to understand better.

What do you mean “pretentious term like ‘test charter’”? How is “charter” attempting to impress more than “mission”? A charter is a mission for a test session in Session-Based Testing Management. It’s not a mission for testing, but for a session of testing. I can recommend the church, by the way. Awesome hats, cloaks, and you should definitely see the altar.

“However, software testing recently seems to be all about using the fashionable terms and techniques instead of doing what a tester is supposed to do; provide information to the people in control to assist them in making decisions that will improve the product.”

I haven’t been there so I don’t know what is going on, but I find it rather important that we talk about what we do. I also find it important that we have terms for specific things, like “user experience”. When you name a technique, it’s easier to explain to others and apply in practice. How would you describe “user experience” without using those words? Seems handy to use those two words, in my opinion.

“Anyone can test software and that's what we should be telling people. We should be asking as many people as possible to test the software we're working on and listening to their feedback.”

That’s definitely not what we should be telling people. We have better ways to ask for their participation without pushing down our craft. For example, we can tell them “Hey we would like to hear different opinions about the usability of this product. Would you like to provide feedback?” Before a testers pairs up with a programmer, he won’t be invited there by the programmer saying “anyone can do this”.

“Welcome to my blog.”


Thanks! I don’t need you to agree with all my points, if any, but I’d like you to comment back. Welcome to my blog, too!

7 comments:

  1. When I wrote my first entry, I was hoping for a response but I never expected such an in-depth response as this! I'm very grateful for such extensive feedback.

    I must admit right now that the entry was deliberately baiting.

    I left TestBash disappointed and wound up at some of the superior attitudes displayed along with the very limited number of methodology and approaches covered.

    It also wound me up how people seemed to make a point of saying that not everyone can be a tester. It's a professional footballer saying not everyone can play football.

    If they did that, some people will take it as not to try.

    If we go around saying not everyone can test, people may dismiss their opinions and feedback.

    Of course you're right, without the training or experience that most of us had, someone is likely to be a bad tester.

    That said, I'd rather someone give us feedback because they decided to give it a test than not because we said not everyone can.

    Our jobs is all around compiling and supplying information. Having more information only helps us.

    It doesn't diminish our profession by encouraging people to test. It may even provide us with more original ideas or questions. That's a good thing.

    I called 'test charter' a pretentious term because unless you're in the know about SBT, it isn't a clear term. But if you say test mission or test objective, it is. It just feels like charter was used because of the in-the-know nature of it.

    I'd much rather use language that is self-explanatory. That way it's inclusive not exclusive.

    Again, thanks for your response to the entry. My next entry won't be as deliberately provocative. Maybe it'll even get onto your blog list. Who knows...

    JR

    ReplyDelete
  2. Thanks for the comment, Jonathan Roe!

    I can see from your reply we have things to talk about. I also notice a lot of the things you dislike are related to communication. Are you familiar with the work of Dale H. Emery? He is not the first to talk about how communication works, but he wrote an interesting blog post about untangling communication http://dhemery.com/articles/untangling_communication/.

    I'm not sure what you mean with "If we go around saying not everyone can test, people may dismiss their opinions and feedback." What are the problems you see here? Do you think the solution is to say "everyone can test"?

    I'll get back tomorrow on your topics. Maybe we can Skype so the feedvback is fast,

    Thanks again and enjoy the weekend!


    Best regards,
    Jari

    ReplyDelete
  3. Hi again JR,

    I suspected your entry was written to cause discussion. I've done that too in the past. I've done that multiple times with my bosses and "a few times" with clients. Usually when I do it, I do it because I feel I'm running out of options for people to hear me screaming, for people to give feedback. If you have something to say and you feel you are not heard enough, let me know and I'll help you with that.

    If you feel TestBash was limited in content and attitude, would it be possible you apply for a speaking slot? If you want to talk from your passion, from your experience, from your heart, chances are other people want to listen to you.

    I don't think people will take it as not to try if we say not everyone can be a good tester. (Most people can, but they don't work enough on their skills and capabilities.) I don't mind welcoming people in the craft, but I want to be clear doing good testing requires a lot more than most people are willing to do, IME. Have you ever heard about someone not trying testing as a career because the expectations of some people were too high?

    Yes, encouraging people to test doesn't diminish our profession. But claims like "everyone can do it" do. You don't hear that too much from other specialists and that's possibly because they have more spine and self-respect than software testers in general. (Yes, I'm being speculative here.) I'm fully aware about biases, but when I try to remember hearing specialists talk about their professions, they all say it requires a lot of work.

    I don't think "test mission" is any clearer than "test charter". If you tell me "test mission", I will understand a whole different thing about it than "test charter", which is defined for a short period. Ask around from people what they mean with "test case" and you'll see we don't even agree on that.

    I've added your blog on my reading list because I didn't hear (or I don't remember it) about you before and clearly you have things to say. Let the community amplify your words,

    Have a wonderful Sunday!



    Best regards,
    Jari

    ReplyDelete
    Replies
    1. Jari, I need to emphasize in my work continuously that while I might be good / better than the developers in testing, we're even better together. Leaving me alone with testing isn't what we need. And there's one way to get better - practice

      Delete
    2. Hi Maaret,

      Thanks for the comment! Somehow it was stuck on Blogger and I got the notification about it only today.

      I find those good points and I completely agree with them. I might add: programmers and other stakeholders can make me a better tester. So not only they help make testing better, but I learn from them and improve myself. Thus, definitely I'm not saying we should keep testing only for testers. If you can point me to what makes you think I said so, I'd like to know about it and see if a post-edit is needed.

      Thanks and have a great week!


      Best regards,
      Jari

      Delete
  4. anyone who is a curious and thinking individual can get more information about how to grow as a proffesional software tester.

    and most people grow up as very curious industrious little kids, with a biological imperative to explore the world and test its limits and norms.

    except society.

    we train them not to paint the walls (without understanding why they did it, only thinking about having to clean it up).

    we train them not to ask so many questions, instead of training them to possibly ask them at a better time.

    we train them to be still and watch tv, because we then get some rest, without thinking how that might impact their development (have you seen the shit on tv? i can feel my iq droping to negative whenever i switch it on...)

    then, we put them in schools where we tell them that there is a right answer and a wrong answer, and the right answer shall be given to you by a figure of authority.

    and after that, only SOME are left to be mediocre, or good software testers.

    ReplyDelete
  5. Hi Andrei and thanks for the comment!

    I think you meant "to become", but I'm not sure. Anyway, as much as I've talked with you, I know your definition of a good software tester has much higher standard than what I often see in the Romanian market. We should meet and figure out what else we can do in order to affect on the things you mentioned.


    Best regards,
    Jari

    ReplyDelete