Thursday, November 29, 2012

Software Testers Don’t Break Code – or Do They


(Update: As the blog text came at 6 AM, it seems only appropriate to wake up at 4:30 AM to update the post. I fixed some inconsistencies, rewrote phrases and clarified my own point of view in the subject.)

***
Short description around the discussion before I go to rephrase what I wrote earlier. This all started from a semi-joke from my side to Erik Davis. I thought it's funny to say I am a tester who sometimes break code. I had the intention behind that I also write code sometimes. As the discussion didn't stop there, I thought first to write how to defend "testers break code" but while writing it all collapsed in my mind. This is why the original post was really confusing and my replies to the comments about it. I tried to keep the body of the original post here so there still can be apparent mistakes. I'd be glad to hear about those.
***

I have discussed about this topic a few times, but I never gave it much of a thought. I did think different aspects on a slightly superficial level, however, as I didn't write about those sides, I decided finally today at 6 AM it's time to update my blog. This won't be an exhaustive research on the subject, but it's intended to show how I think about the topic (and how I see discussion around it) currently. My thinking will change the more I dig into this.

When I talk about testers in this case, I talk about people who mainly do software testing. I don't include for example programmers and managers in the definition of a software tester, even if they would do testing. I can include them in the definition if testing is what they get paid for and they are continuously trying to improve their skills as a software tester. But I think in this case their title should be a tester or so.

Next logical step would be to think what is breaking. For looking into different meanings of that word, I used The Free Dictionary by Farlex. I won't go copy/pasting the text here, please just take a glance and see how many ways there are to use that word. I take a few snippets that I think apply nicely for software world:
  1. To force or make a way through; puncture or penetrate => break into software /  crack it
  2. To find an opening or flaw in => bug
  3. To render useless or inoperative => pretty major bug
  4. Go down, crash => software breaks during a test / over a period of time

In this respect, it does indeed /seem/ like testers would break software. This sounds counter-intuitive? Yes, definitely I have the same thought. (As I didn't find any better translations, I leave it here from the dictionary point of view. In my opinion, breaking would imply making the code unusable/working wrong.) But on the other hand, as these are used as arguments by some people, let's look into what they say: "If we have software running and by our actions it becomes useless, who actually broke it? Sure, the code was not resistant to *all* possible use cases, but please, show me an application that is."

There are many levels of breaking, one could argue to us. "There is the misuse, happy flow and how ever people want to call ways to use an application." This doesn't really add content to the topic. The definition of breaking is not subject to different ways to use an application. In my opinion, we can’t have in a long time, if ever, applications that handle every possible scenario without “bugs”. However, when we find those, we don't break the code, we find ways to expose threats to the value of the application.

There are two other points of view I still want to write about. I used some definitions of breaking above, but what if we say "testers receive the code broken"? Now this changes the discussion closer to existentialism, where many thorough deductions seem to end. The argument we would hear could be: "Let's say we have a light bulb, drop it on the floor and it shatters to pieces. Did we receive the light already broken or who/what broke it? One could say we broke it, one blames the manufacturing company, someone the engineers or faulty machinery. And there are always those who said the ground brake it with a devilish co-operation with gravity. Damn you Newton - why you just didn't eat your apples!"

In reality, software is not like a physical object that breaks by dropping it on the ground. Software breaks because there are problems with the code - and sometimes it seems to break because of problems in the infrastructure. If you hear analogies with breaking physical objects, changes are the discussion needs to change focus from "what is breaking" into "what is software" before the first question can be answered.

The second point is about how we actually use these words. I hear this really often and sometimes fall into it myself. Language is not static; it's constantly evolving to many directions. It's different if we write or say aloud the words. It's a lot different if we shout or whisper. For these reasons I don't consider it always that important what words people use if we know what they mean. (And obviously, we need to ask them about this if we are not "sure"; and even then in some cases.) But this is not the case with people such as scientists and engineers; they need to be accurate with their wordings. Beware of arguments that back up on interpretations!

I want to note here I prefer saying “I like to find ways how the application/system seems to reveal a threat to the value of the product” (shorter version: I like to see stuff break) or something in those lines. In other words, I prefer telling I want to /find/ something that I /believe/ is a threat to /some/ value. (As value is a relationship, I understand my “important” findings are not valuable to all stakeholders.) I don't like talking about breaking code or code being broken. However, I like breaking *things*. 

15 comments:

  1. This is a really interesting (and new) look at this problem. I'm in the "I receive it broken" camp of the polar argument, but I agree that a "break" in the software doesn't really make sense, linguistically. It's a decision based on reality versus desire - the gaps between the subjective vision of the stakeholders and the actual reality of the product under test; and the relationship between the observer and the observed through a subjective mental model of both. Is that gap a "break"? In one sense, I suppose, but not the sense in which it is so often used. Thanks for a great post :)

    ReplyDelete
    Replies
    1. Thanks for the comment, Chris!

      I am not yet sure in which camp I am, but I am moving slightly to the direction what my post indicates. It could be really interesting to move deeper in this subject. Writing simple posts helps in the process!

      Have a good day!


      Best regards,
      Jari

      Delete
  2. I, like Chris S, stand on the side of "I receive it broken", at least in most cases. As far as I see it, there is no one right way to define "broken", and because of that, discussing this should lead to some mind-expanding conversations, especially for anyone new to testing. Thank you Jari for writing this.

    ReplyDelete
    Replies
    1. Thanks for the comment, Erik! (And support with writing this post.)

      I myself try to avoid using "broken" in these discussions. But there is one thing with breaking things: many people get enjoyment from "breaking" things and what they actually mean is to learn something new, internal, about the thing they broke. So they like learning how things work and what actions change that.

      So from my point of view, people are free to use break/broken, but I expect/hope them to be able to explain to me what they mean with it.

      Take care thinking testers!


      Best regards,
      Jari

      Delete
    2. Jari, I try to avoid using the term broken as well, at least I do when I am talking to a developer. Broken, crashing, words like that don't provide any real useful information regarding the issue at hand. They are divisive terms (intentionally used or not) that cause friction between those participating in the conversation.

      Are they fun to use colloquially? Sure. Are they interesting discussions starters? Absolutely.

      I agree that people can use terms like that, but they need to be defined, in context, to be meaningful.

      Delete
    3. Very my like and appreciate your comment, Erik!

      It's a very good point to note that testers need to pay attention on the words they use when talking with different stakeholders. I was given a similar advice by an HR specialist; not to use the term "interview" in a job interview, but to call it a "discussion". Apparently, that works for many people.

      Enjoy your Friday!


      Best regards,
      Jari

      Delete
  3. Interesting post Jari. It made me think.
    I believe I am still of the mindset that just because I found a bug (a condition or series of inputs that cause behaviour that threatens quality) does not mean I broke the product.
    I think a better analogy with the lightbulb would be this: Someone hands you a lightbulb and you try it in a few sockets and it appears to work fine. You then carefully examine the bulb and notice a small crack in the metal. You determine that this may impact the life or perhaps the brightness of the bulb so you report it. The design team finds out that they need to change the design to prevent the cracks from occurring. The bulb was not broken when you received it but the design was already broken before you found the flaw.
    Your lightbulb analogy is more like the tester going into the code base and scrambling some of the files.

    So, although you made me think about the definitions, it wasn't enough for me to change my original tweet:
    Testers don't break things. We point out how others have (unintentionally) broken the product during specification, design, and coding

    ReplyDelete
    Replies
    1. Thanks for the comment, Paul!

      I believe you stay in your state of mind because you are thinking of an interpretation for the word "break". That is a good thing, as long as you recognize this - and I know you do. When I input a malicious SQL Injection on a web site and it crashes, I think I broke the product. (Even if there could be other interpretations about relationships etc.) I don't think the product was broken when I got it, even if it had problems.

      I think the analogies with physical objects don't work too well, unfortunately. There are interesting philosophical questions in relation to a light bulb being broken, because for example after some time of usage, the illumination power decreases. So when is it low enough for the light bulb to be broken?

      I think this is a discussion where I would like to include more thinking testers. If not before, I will bring it up at Let's Test.

      Have a fantastic day!


      Best regards,
      Jari

      Delete
  4. I realize no one is (exactly) saying it here, but there is absolutely nothing wrong with saying that testers "break stuff" and in fact can be a really healthy attitude to explain our value. "Breaking" software can be analogous to certain types of testing - inserting malicious code, stress testing, etc. or just basically using products not as designed in order to learn about them. Too often, testers undermine their own value to organizations by trying to philosophize away any confrontational aspects of our jobs in order to redefine (and hopefully increase) their prospects and position for either their mission or themselves. I believe (experientially) that it is a VERY slippery slope to irrelevance getting into a "well, that depends what your definition of is, is" speak with the business or your sponsors and besides that, lots of testers got into this job because the LIKE breaking stuff!!

    ReplyDelete
    Replies
    1. Thanks for the comment, Keith!

      I tried to point that ("nothing wrong in saying it") out above in the reply to Erik's first comment. But my idea was that those people should be able to explain what they mean with that phrase. If they open it up and can defend it, it's all good. As said, breaking has so many interpretations only in a dictionary (and rare people are dictionaries), the discussion "is it a correct wording or not" is not so interesting to me as the discussion of what they mean with it.

      For the time being, I am in the "I like breaking" side, but I am trying to avoid (generally, not always) that wording in my professional life. The reasons for that could be worth of a book. :-)

      Have a good day!


      Best regards,
      Jari

      Delete
  5. This comment has been removed by a blog administrator.

    ReplyDelete
  6. This comment has been removed by a blog administrator.

    ReplyDelete

  7. Thank you for the info. It sounds pretty user friendly. I guess I’ll pick one up for fun. thank u








    Software Development

    ReplyDelete
  8. If some one wishes expert view about blogging after that i propose him/her to pay a quick visit this weblog, Keep up the nice work....
    software testing in indore

    ReplyDelete
  9. It seems that you are a true thinker and great content of this article.Thanks for sharing.
    Physical Therapy Software

    ReplyDelete