Tuesday, September 27, 2016

Feedback Lesson From Baby Toys

I thought to write for TesterMeetup, a new community for testers, started in Pakistan. You can find the blog post from here.

Feedback Lesson From Baby Toys

I thought to write for TesterMeetup, a new community for testers, started in Pakistan. You can find the blog post from here.

Tuesday, March 1, 2016

Saturday, April 5, 2014

Anyone can be a (bad) tester

Jonathan Roe recently wrote an article called “anyone can be a tester” (http://www.morethanfunctional.co.uk/1/post/2014/04/anyone-can-be-a-tester.html). After reading it, I exchanged a few comments with him and thought to leave a longer comment on his blog. Soon after starting to write that comment, I realized it fits better in my blog. Mostly because it reflects how I currently think about the subject.

In this blog entry, I will quote text from him in order to comment on the points and sentences he wrote. It’s entirely possible I have misunderstood some of his points or my quote doesn’t cover enough text, but by no means will I do that intentionally. I know he talks a lot about usability and user experience, but because he repeats “anyone can test” theme, I’ll pick up on that. Having that said, let’s begin.

“The problem with saying "anyone can test software" to someone who, you know, tests software for a living is that it seemingly devalues the career they've probably put a lot of time and effort into progressing. So I say this to anyone who thinks anyone can do software testing:
You're right. Have a biscuit.”

I prefer to add the qualifier “bad” in front of “tester” in the claim. Yes, it’s true that anyone can test software (the title of the article said “can be a tester”, not “can test”), so in that sense anyone can be a tester. But this is shallow. Anyone can be a programmer and a manager and a director and a CEO and whatnot, too. Not anyone can be a dentist because there are training requirements (http://en.wikipedia.org/wiki/Dentist#Training), at least in some countries. But we could say “anyone can pull out a tooth from someone”.

Yes, I am using loosely the word “anyone”. I don’t really mean anyone, but that’s not the point to me here today. My point is that pretty much anyone can be a bad tester and it takes effort to become good. Just like all testing is exploratory (up to some degree), good exploratory testing is on a completely different level than someone clicking here and there while chatting and browsing funny cat pictures.

When I pointed out in Twitter “anyone can test software badly” is a better expression from my point of view, Jonathan Roe replied:
“'badly' is irrelevant. If in testing they find a bug previously unfound, the testing was a success.”

I think we have a set of problems here: 
1) How would we know the bug was previously unfound? Maybe it was discussed with a programmer, but decided to be ignored. Maybe the PO ignored it. Maybe it’s not yet prioritized to be fixed. Maybe it’s not important. Maybe it only bugs someone who doesn’t matter that much. 
2) A bug is not an attribute of the product you are testing, but a relationship between the product and a person (or persons) at some time.  In other words, it’s not a bug if it doesn’t bug anyone; and this can change over time. 
3) Is testing a success if we spend a huge amount of money to find a trivial bug? It might be that a test was successful up to some degree, but even this is questionable. If the bug is not properly reported, will it be fixed? What if the team doesn’t understand the report at all? What if the report doesn’t have any means to reproduce the bug and a very complex set of actions is needed for that? How do we determine testing was a success? 4) By telling this publicly to people, effectively you promote the idea how easy software testing is and by doing that you are cutting the branch you are sitting on. You don’t say it directly, but when you say anyone can be a tester and all their bug findings make their testing a success, you are conveying the impression how ridiculously simple profession we have.

“Will they be able to find as many issues as someone who's been doing it for several years? Probably not. Will they be able to communicate the issue in such a clear and efficient way? I doubt it.
However, even if it means they can't test software well, that does not mean that they can't test software.
In fact, I recommend you get as many non-testers to test your software as possible.”

Ok, here we soften the punch a bit. If the point of the article is to explain why it’s beneficial to include more people testing a product, how about saying that from the start? When saying “anyone”, maybe the author meant stakeholders from the team. From my point of view, if one is not testing well, they are testing bad or mediocre at best. Most non-testers I’ve worked with did not reach the level what I’d call mediocre because I’m talking about a mediocre level software testing professional.

“You can use as many testing techniques and tools as you want but if it isn't very usable, the software's crap.”

I’d remind for starters that testing doesn’t make software more usable. I’m also not familiar with Jonathan Roe’s definition of “very usable”, but I know I’ve used a whole lot of software in my life that was utter crap from usability point of view and still they got the job done. Most of these were professional tools for which you don’t necessarily have any installation guide and configuration file needs to be built first. Four (4) of them have been tools for creating invoices based on worked hours. You know, the kind of software which costs a lot.

“If a user can't easily use the functionality or find the content, you might as well take your test sessions, your mindmaps and your automation scripts then insert them promptly and firmly up your arse.”

I need to check a few arses next time when I meet my friends who have been testing banking software. Especially online banking software. Until now, I haven’t seen good online banking software. I’ve seen extremely poor attempts and good-enough solutions, but they all have disappointed me in a way or another. Like the Finnish online bank where I can’t change my address to a foreign address without visiting their office in Finland. However, I’ve never changed my bank because of these problems. I’m not saying that’s an excuse for bad software, but I’m saying it’s good to remember not all software is what you test.

“They seemed very content just slagging off the 'old way' to test software, preaching about the importance of automated tests and using pretentious terms like 'test charter' (that means a test objective or mission for anyone who doesn't attend the church of James Bach).”

Sounds like they were slagging off the old way how they used to do software testing. (I don’t like the expression “to do software testing” because testing is much more than the visible actions we produce, but more about that some other time.) It could be they have learned something along the years and they are preaching about it now. It could be many things and I hope I can attend there at some point so I can talk with people to understand better.

What do you mean “pretentious term like ‘test charter’”? How is “charter” attempting to impress more than “mission”? A charter is a mission for a test session in Session-Based Testing Management. It’s not a mission for testing, but for a session of testing. I can recommend the church, by the way. Awesome hats, cloaks, and you should definitely see the altar.

“However, software testing recently seems to be all about using the fashionable terms and techniques instead of doing what a tester is supposed to do; provide information to the people in control to assist them in making decisions that will improve the product.”

I haven’t been there so I don’t know what is going on, but I find it rather important that we talk about what we do. I also find it important that we have terms for specific things, like “user experience”. When you name a technique, it’s easier to explain to others and apply in practice. How would you describe “user experience” without using those words? Seems handy to use those two words, in my opinion.

“Anyone can test software and that's what we should be telling people. We should be asking as many people as possible to test the software we're working on and listening to their feedback.”

That’s definitely not what we should be telling people. We have better ways to ask for their participation without pushing down our craft. For example, we can tell them “Hey we would like to hear different opinions about the usability of this product. Would you like to provide feedback?” Before a testers pairs up with a programmer, he won’t be invited there by the programmer saying “anyone can do this”.

“Welcome to my blog.”


Thanks! I don’t need you to agree with all my points, if any, but I’d like you to comment back. Welcome to my blog, too!

Saturday, March 23, 2013

Answers to test management survey from Mike Lyles


Answers to survey from Mike Lyles

I recently answered to a survey that was done by Mike Lyles. In many cases, I felt like I want to explain my answer, thus I thought to put the questions in here and use a bit of time to answer with more words. Rock on

Demographics
1.          What is your current role?
a.          Test Manager?
b.          Tester / Test Engineer?
c.          Development team
d.          Program Management Office (PMO)
e.          Other – please enter description [_______]
My title is “Head of QA”, but my responsibilities go from head to toe. We are in building a testing department and we aim to do it really well. I am lucky to have the possibility to do hands-on testing sometimes still at this point even if the departmental matters need my time. This is mostly because my manager is really clever and let's me figure things out myself; even if I  make mistakes.

2.          What is your current employment status?
a.          Full time employee with the company?
b.          Contractor with the company?
c.          Unemployed at the moment?
This is slightly complicated, but let's say I am a full time employee with the company for the sake of simplicity.

3.          Are you Male or Female (M/F)?
I am a male who reads Keith Klain's blog when I need more balls.
4.          How did you get into QA / Testing?
a.          By Choice (gradual progression into the role – possibly from other roles)?
b.          By Need (“pushed” into the role and/or asked to volunteer)?
c.          My career has been in QA since the very start?
Well, this is a really long story and it's almost the same I tell when people ask how I got married. I was offered a position in software testing after I explained the manager why I don't want to write code for a profession.

5.          How long have you been in IT?
a.          1-5 years?
b.          6-10 years?
c.          10-15 years?
d.          15-20 years?
e.          More than 20 years?    If more than 20, enter your number of years [____]
If this implies the time I have been paid working with software, it's pretty much since 2007. I used to test hardware earlier. If it implies something else, then I could say I've been into information systems and computers since around 1990 when my dad bought an Olivetti 80286 with a CGA monitor.
6.          How long have you been in a Test Manager / QA Role?
a.          1-5 years?
b.          6-10 years?
c.          10-15 years?
d.          15-20 years?
e.          More than 20 years?    If more than 20, enter your number of years [____]
I've been a test and project manager since 2009 until almost end of last year, but currently my role is quite different and I say I am a tester who has also the responsibility of the department. I'm not a fan of titles, especially when they are used in a confusing manner; like “CEO” for a company of 3 people.
7.          What is the number of direct reports you have?
a.          0 people?
b.          1-5 people?
c.          6-10 people?
d.          10-25 people?
e.          More than 25?    If more than 25, enter your team size [____]
I have test reports from all the testers in the organization, but I don't ask them from everyone personally. I have also reports from project managers, Scrum Masters and for example Team Leaders. Instead of written reports, I talk a lot with people and figure out how I can help them and if I could do something in their projects.

How You Spend Your Time
8.          How much time do you spend during your week in  PLANNING?
a.          0-25%?
b.          26-50%?
c.          51-75%?
d.          76-100%?
Now these percent questions are tricky, but let's reformulate the questions a bit. I don't do that much of preplanning once a plan has been made for a project. For the department, when I have the time, I am usually planning something. My table is full of post-it notes where I have plans for many things.

9.          How much time do you spend during your week in  PEOPLE MANAGEMENT?
a.          0-25%?
b.          26-50%?
c.          51-75%?
d.          76-100%?
I don't think people are managed. I talk with people, I guide them, I help them etc. but I don't think people can be managed in the sense I understand the word. I use a lot of time to talk with our HR specialist (she doesn't like to be called a manager, so I usually call her a director or VP :-) ) about people and my department. I do this because I think I have a million things to learn from her and I want to focus most of my energy on the people.

10.    How much time do you spend during your week in  COMMUNIICATING?
a.          0-25%?
b.          26-50%?
c.          51-75%?
d.          76-100%?
Only a really small part of my work is not about communicating with other people. If I am not talking (F2F, e-mail, chat, bug report etc.) with someone, most likely I am preparing for it.

11.    How much time do you spend during your week in  MANAGING  EXECUTION?
a.          0-25%?
b.          26-50%?
c.          51-75%?
d.          76-100%?
I don't quite understand this question. Most likely it's because I tell people what I want, but not how I want it. I am a firm believer that people can be awesome and that they can have a lot better ideas than I have.
12.    As a Test Manager, what is your role in  EXECUTION of TESTING?
a.          My team executes all of the testing always – I do not execute?
b.          I execute testing with my team?
c.          My team executes the testing – but I participate if needed?
Tricky question. I am not a test manager, though, I manage testing. I execute a lot of testing myself, but I don't do that with others. I let people get their stuff done and learn from their mistakes. I even tell them about my own mistakes so they know I am not just saying it.
13.    How much time do you spend during your week in  MEETINGS (stakeholders, team, reporting)?
a.          0-25%?
b.          26-50%?
c.          51-75%?
d.          76-100%?
I try to be efficient with meetings. Usually I don't call them meetings, I just show up to someone and have a list of questions or answers. Recently, I have been changing that with a few people, but with most, I prefer to have more informal discussions.

14.    How much time do you spend during your week in  PROCESS IMPROVEMENT OF THE QA ORGANIZATION?
a.          0% (the team uses current defined processes & never changes)?
b.          1-25%
c.          26-50%?
d.          51-75%?
e.          76-100%?
Process definition and documentation has been one of the biggest tasks for me while building the department. While doing that, the processes have been improving and we have used a lot of discussions to talk about this. I am the kind of person that I see improvement opportunities everywhere so I spend a lot of time to figure out improvements for all kinds of processes in the company; especially if they are directly about people.

Your Thoughts on Test Managers
15.    Do you think that people make better Test Managers if they have a background of software development?
a.          YES?
b.          NO?
I assume this means they have experience in programming. Yes, I think it can be a benefit to know programming. I don't, however, think it's a general kind of thing that makes people better. There are so many different kind of roles for test managers and definitions for having background in software development... But in general, I think people should learn their craft more. If this means learning programming for a software tester, I don't see what is the reason not to do it.

16.    Which skill do you think is the most important for a Test Managers?
a.          Project Management Skills
b.          Technical Management
c.          Test Management Skills
d.          Communication Skills
e.          People Management - HR & Personnel
f.            Defining Processes & Methodologies
g.          Handling Exceptions, Risks, Issues
This seems like another trick question. If you work as a test manager, it would sound obvious your most important skill is test management. Interestingly, we are not provided any skills that the options actually contain. I think Mike did this on purpose. I would say that critical thinking, questioning, systems thinking and leadership skills are really important in many cases for a test manager.

17.    Is the role of the Test Manager clearly defined in your organization?
a.          YES
b.          NO
We actually don't have any test managers, if I am not counted as one.

18.    In your organization, is the Testing/QA function a global centralized function or are the teams integrated separately with each specific area of the organization?
a.          Centralized
b.          Integrated / Spread out as multiple teams
We have testers in various teams, like Scrum, Kanban and we have even sold testing as a service. We do gather up and talk together on a weekly basis. We don't really make “teams” from testers, but we have testers working as parts of development teams. The testers support each other and they work solutions together, which is really cool.
 
How Do You Manage
19.    What do you feel is the best way to communicate to your stakeholders?
a.          Team Meetings?
b.          Emails?
c.          One on One?
I have a lot of different stakeholders. I use e-mail in many cases when I want something “documented”. I prefer face-to-face discussions when possible, but not always. I think I am way better in face-to-face discussions and I need to work a lot on my skills when it comes to written communication.

20.    How often do you speak to your customers?
a.          Daily?
b.          Once a Week?
c.          Every other Week?
d.          Once A Month / Quarter?
We have clients, McDonald's has customers. :-) There are some clients I talk with more often, some I don't talk almost at all. It depends a lot on my role for those projects. I love talking with different customer support people because they give me great information of systems that are already live. I would like to talk with the clients more often, but currently I focus on something else.

21.    How often do you manage / assess / review RISKS?
a.          Daily?
b.          Once a Week?
c.          Every other Week?
d.          Once A Month / Quarter?
Basically, this happens constantly. If it was meant for example to have a formal meeting about it, then maybe daily. I am involved in many projects so it's not really even possible currently to avoid this. Everything we do causes a risk, so managing risks is really important.

22.    What is your most probable/preferred method for mitigation of RISK?
a.          Stand up meetings with the testing team that
b.          Escalation to management?
c.          Meeting with the Stakeholders?
I talk a lot with people and sometimes I send e-mails. I don't think talking with “testing team” is a good way to mitigate a risk, but talking with the scrum master / project manager / whatnot and a team is better.

23.    How do you handle conflict with other teams (stakeholders, developers, PMO)?
a.          Heated debates?
b.          Escalation to upper management?
c.          Structured negotiation to reach a mutual agreement?
d.          Depends on the situation?
This depends a lot on the situation. I have one manager over me; who happens to be really cool. He lets me fail and learn. Sometimes I think he is even pushing me to make mistakes and go to conflict situations. I handle most conflicts (caused by me) with face-to-face discussions and I forget to follow-up on them frequently.

Testing Group Training / Growth / Improvements
24.    What stages of a measurement / metrics program are you currently in?
a.          No program exists
b.          Program exists but no improvement to the overall process
c.          Program exists and assists with showing the value of the organization & quality of deliverables
We collect some numbers and we make some reports. If a client wants something, we talk with them, let them know our opinion and find an agreement. We try to keep the number information small scale and only as triggers for questions.

25.    Which areas of test management do you feel needs more training in your organization?
a.          Test Strategy?
b.          Test Planning?
c.          Defect Management?
d.          Automation?
e.          Performance?
f.            Measurement & Metrics?
If performance here means personnel performance management, I think it's where I have been focusing a lot lately. I am not a huge fan of training; I prefer for example coaching and learning from experience. This is a question I should ask myself again on Monday. :-)

26.    What methods do you use the most to ensure you are doing “the right things the right way”?
a.          An internal governance process to review, modify, and publish the processes?
b.          Trainings / whitepapers / consulting to align with testing industry best practices?
c.          Feedback (either with surveys, lessons learned, project closure meetings) from the stakeholders and other project team members to review effectiveness of testing?
Something like the a) option, but not really. I put a lot of trust to people. I give them direction and guidance (and a lot of difficult puzzles), but I think I don't need to do much more. If we make mistakes, we learn from those and share them with everyone.

27.    How many conferences does your team attend per year?
a.          0?
b.          1-5?
c.          6-10?
d.          More than 10?
“Not as many as I would like to” is a common answer to this one, I presume. :-) I would really like to do that a lot more, but there are some resource problems currently with this and it's not really possible. I think next year will be more active on this side. We are also planning to have a few conferences in Romania where we have a fantastic testing community!

28.    Is keeping relevant and up to date with QA technology/processes critical to the success of a testing organization?
a.          YES
b.          NO
For me, yes. Dunno about others. I think in general we need a software testing to step up in Europe.
 
Internal Issues / Showing Value of QA / etc
29.    Complete this sentence…..”As a Test Manager, I feel “
a.          Respected by the other teams (stakeholders, developers, PMO)?
b.          Disliked / distrusted by the other teams?
c.          Respected at times, disliked at others – a constant struggle?
I am really respected and valued by my colleagues, clients and my manager. I am very lucky, it seems. I value our testers and my colleagues a lot too, so I give them gifts sometimes just because I feel like so. :-)

30.    Testing is often seen as a cost center instead of a profit center.    As a Test Manager, what methods do you use most office to show the value of testing?
a.          Leverage measurement/metrics results to show the value?
b.          Focus on the reduction in rework by finding and resolving defects early in the SDLC?
c.          Showcase improvements to testing which simply processes, reduce overall project costs, and improve overall product quality?
We find things that are important for release decisions. We find important bugs. We provide quality-related information of the products we test and this is what others want from us.

31.    What percent of your week do you feel you  have to “sell” the value of the testing organization?
a.          0% (organization completely supports and backs the QA group)?
b.          1-25%
c.          26-50%?
d.          51-75%?
e.          76-100%?
I don't have to sell testing internally. We have a support from all departments and managers.

Your Opinion
32.    What role do you think the Quality organization has in the decision to move to production?
a.          Quality Organization has the GO / NO GO decision – ultimately?
b.          Quality Organization provides inputs to those accountable for releasing – and may suggest strongly to not go, but does not own the decision to move to production?
We inform; we don't make release decisions. We don't fake we do other people's work, we focus on doing our work better.

33.    What Percent of Releases to Production have Defects or Code Issues in your organization?
a.          0-25%?
b.          26-50%?
c.          51-75%?
d.          76-100%?
All. At least I haven't  seen a software product without a defect. It's also important to note here that a defect to someone at some time might not be a defect to someone else or for example on some other time.

34.    In my organization, QA is involved early enough in the project?
a.          YES
b.          NO
I don't call testing as QA, even though it's in my title. :-) We include testing in the project as soon as we see possible. I think we do a really good job in that.

35.    In my organization, the senior decision makers & stakeholders listen to the QA team?
a.          YES
b.          NO
This is a definite yes. We have a lot of trust and listening.

36.    In my organization, the senior decision makers & stakeholders understand QA / QA methodologies?
a.          YES
b.          NO
They don't need to understand what testers do. They might, but I don't want to even go to these details with them. Just like I don't ask them about code.

37.    In my organization, QA is provided adequate testing resources for the projects?
a.          YES
b.          NO
This question sounds a bit confusing to me. Mike is again playing with my mind here. I am convinced he used terms like “QA” and “resources” in contexts where he knows people like me get derailed. :-) Anyways, we have the tools and time we need for doing good testing. Sometimes we might ask for example for more time, but usually we don't even need to ask for anything. Testing is really valued in our company.

38.    In my organization, QA is almost always blamed or held accountable for defects or issues in production?
a.          YES
b.          NO
Nobody would be so silly in our organization because they know testers are not making the code. We could, in theory, face a question “why didn't we spot this”, but we would have that as a learning point.

39.    In my organization, deadlines will take precedence over the quality of the deliverables?
a.          YES
b.          NO
We don't deliver if we don't test. It's more important for us to deliver good than to deliver. In everything we do, we focus on having great products for the clients.

40.    In believe that my organization truly WANTS quality deliverables?
a.          YES
b.          NO
As stated above, we focus a lot on this stuff.

41.    Do you believe it is possible to test everything?
a.          YES
b.          NO
This is not a question about belief. It's not possible. For this, we would need infinite resources and it's not quite something we could have.

42.    In my organization, I am given clear direction and support from my management?
a.          YES
b.          NO
Yes, my manager gives me direction and support. He also lets me figure things out so sometimes I just don't hear anything from him and then all the sudden he says “we have a problem with project X, fix it”. It's pretty neat to have that much of trust from the manager of the company. I also get support if I need, but mostly all support I need is to know I am trusted.

43.    In my organization, I provide clear direction and support to my subordinates?
a.          YES
b.          NO
Depends a lot on the case. Lately I've been focusing on other things mostly because they are doing a good job in the projects. If I see the need to be more clear on something, I do it. Otherwise I ask if it's needed.
 
Area 2: Survey Questions for Everyone (Discussion Questions)
 
1.          Enter your name (OPTIONAL)
Jari Laakso

2.          Why did you want to be a Test Manager?
I never really wanted to be a test manager. I didn't even want to be a manager of a department. But I like both of those really much! I like testing and I like working with people. The role is not so important for me. I think I am still a better tester than a manager, thus I have a huge deal to learn on a continuous basis.

3.          Name 3 to 5 things you would do differently if you had the authority to choose how your day as a Test Manager would look?    (Include suggestions on how that could be achieved without disrupting the testing functions or degrading quality of the deliverables)
I don't really know what to say here. I have the authority to pick my days and actions. I give that for the testers also. If they need something, I take it as a personal challenge to figure out how to get it.

4.          Please list topics within your testing organization where you feel there is a lack of alignment/agreement.    Think about topics that seem to always start a debate or where the team is not in agreement.    Describe the various viewpoints and why the alignment is lacking or non-existent.
I am always picking up on words. That's pretty much it. But I do that a lot. I don't do it anymore that often with non-testers because it doesn't matter that much to me what they say. If people are really into learning testing and improving skills in it, then I also go “tough” on them so they have the chance to learn.

Thursday, November 29, 2012

Software Testers Don’t Break Code – or Do They


(Update: As the blog text came at 6 AM, it seems only appropriate to wake up at 4:30 AM to update the post. I fixed some inconsistencies, rewrote phrases and clarified my own point of view in the subject.)

***
Short description around the discussion before I go to rephrase what I wrote earlier. This all started from a semi-joke from my side to Erik Davis. I thought it's funny to say I am a tester who sometimes break code. I had the intention behind that I also write code sometimes. As the discussion didn't stop there, I thought first to write how to defend "testers break code" but while writing it all collapsed in my mind. This is why the original post was really confusing and my replies to the comments about it. I tried to keep the body of the original post here so there still can be apparent mistakes. I'd be glad to hear about those.
***

I have discussed about this topic a few times, but I never gave it much of a thought. I did think different aspects on a slightly superficial level, however, as I didn't write about those sides, I decided finally today at 6 AM it's time to update my blog. This won't be an exhaustive research on the subject, but it's intended to show how I think about the topic (and how I see discussion around it) currently. My thinking will change the more I dig into this.

When I talk about testers in this case, I talk about people who mainly do software testing. I don't include for example programmers and managers in the definition of a software tester, even if they would do testing. I can include them in the definition if testing is what they get paid for and they are continuously trying to improve their skills as a software tester. But I think in this case their title should be a tester or so.

Next logical step would be to think what is breaking. For looking into different meanings of that word, I used The Free Dictionary by Farlex. I won't go copy/pasting the text here, please just take a glance and see how many ways there are to use that word. I take a few snippets that I think apply nicely for software world:
  1. To force or make a way through; puncture or penetrate => break into software /  crack it
  2. To find an opening or flaw in => bug
  3. To render useless or inoperative => pretty major bug
  4. Go down, crash => software breaks during a test / over a period of time

In this respect, it does indeed /seem/ like testers would break software. This sounds counter-intuitive? Yes, definitely I have the same thought. (As I didn't find any better translations, I leave it here from the dictionary point of view. In my opinion, breaking would imply making the code unusable/working wrong.) But on the other hand, as these are used as arguments by some people, let's look into what they say: "If we have software running and by our actions it becomes useless, who actually broke it? Sure, the code was not resistant to *all* possible use cases, but please, show me an application that is."

There are many levels of breaking, one could argue to us. "There is the misuse, happy flow and how ever people want to call ways to use an application." This doesn't really add content to the topic. The definition of breaking is not subject to different ways to use an application. In my opinion, we can’t have in a long time, if ever, applications that handle every possible scenario without “bugs”. However, when we find those, we don't break the code, we find ways to expose threats to the value of the application.

There are two other points of view I still want to write about. I used some definitions of breaking above, but what if we say "testers receive the code broken"? Now this changes the discussion closer to existentialism, where many thorough deductions seem to end. The argument we would hear could be: "Let's say we have a light bulb, drop it on the floor and it shatters to pieces. Did we receive the light already broken or who/what broke it? One could say we broke it, one blames the manufacturing company, someone the engineers or faulty machinery. And there are always those who said the ground brake it with a devilish co-operation with gravity. Damn you Newton - why you just didn't eat your apples!"

In reality, software is not like a physical object that breaks by dropping it on the ground. Software breaks because there are problems with the code - and sometimes it seems to break because of problems in the infrastructure. If you hear analogies with breaking physical objects, changes are the discussion needs to change focus from "what is breaking" into "what is software" before the first question can be answered.

The second point is about how we actually use these words. I hear this really often and sometimes fall into it myself. Language is not static; it's constantly evolving to many directions. It's different if we write or say aloud the words. It's a lot different if we shout or whisper. For these reasons I don't consider it always that important what words people use if we know what they mean. (And obviously, we need to ask them about this if we are not "sure"; and even then in some cases.) But this is not the case with people such as scientists and engineers; they need to be accurate with their wordings. Beware of arguments that back up on interpretations!

I want to note here I prefer saying “I like to find ways how the application/system seems to reveal a threat to the value of the product” (shorter version: I like to see stuff break) or something in those lines. In other words, I prefer telling I want to /find/ something that I /believe/ is a threat to /some/ value. (As value is a relationship, I understand my “important” findings are not valuable to all stakeholders.) I don't like talking about breaking code or code being broken. However, I like breaking *things*. 

Friday, June 29, 2012

(Dark) Secret of a Great Tester


I recently read a Finnish article (http://www.taloussanomat.fi/ihmiset/2012/06/26/hyvan-tyontekijan-synkka-salaisuus-ilmainen-ylityo/201232164/137) from a news paper which said the “dark secret of great employees is free overtime work”. This started a chain of thoughts in my mind that I would like to open up in this blog post. In the same time, I am hoping I will clarify my thoughts while writing.

The article is referring to another article where is estimated the workers of a Finnish labor union Pro are making about 2 million free hours yearly. Specialists and managerial roles are told to be with most unpaid extra hours. Some questions that popped up my mind:
-          Are they working at their fullest through the whole day or do they take additional “breaks” (like check their Facebook messages)?
-          Is this a problem because of bad management or for example employee’s own time management skills?
-          Do these people record their hours or this is based on their gut feeling? If they record the hours, how accurately it’s done?
-          What are the reasons they work extra hours? Why they don’t get paid for them?
-          Is anyone looking after what gets done instead of how many hours are done?

When I look back at my work history, I see myself being one of those doing long days, being enthusiastic about the product we have been building/testing, having very late and early meetings/e-mails/discussions with customers, taking responsibility for doing a great job and working for the team. So, did I work extra hours? Yes, always when I saw it was needed. So, did I get appreciation from it? Yes, always when it was seen by the team/management/customer. So, did I get paid for those hours? Yes, always when I reported the hours in the ERP. Obviously, I could not be paid for hours I didn’t mark.

Do I study “on my own time”? Of course! Do I think my employer should pay for this? No, because it’s something I am doing mostly for myself. I don’t except them to compensate for what they didn’t ask me to do, but I will greatly appreciate if they do so. I don’t want to be someone who is just hanging around there and executing test cases someone else has written. I don’t want mediocrity. I want to carry kittens from burning houses. I want to throw myself over an exploding grenade. I want to become great in whatever I am going to do.

So, what’s the problem here? According to the article, this should not be the case and even a person who is not going to do all that should be considered a great member of staff.

The article does rise up also the concern of people not getting any kind of compensation for their work. I don’t see this as relevant to the “dark secret” because clearly it’s not a secret of someone who is successful. That is a person who is being abused. That is about bad management instead of greatness. Maybe this is what the journalist even meant, but in this case she should re-write the article.


I am not saying everyone has to do it (= work hard; really hard). I am saying you have to do it if you want to be great. It’s not a dark secret all great people have worked huge amounts. It’s easy to say “oh but Einstein was so intelligent” and do nothing to achieve even a bit of what he did. If you don’t want to do it, don’t do it. But don’t come and ask for the same recognition those get who do it.

Thursday, June 28, 2012

Testing Challenge - Puzzle #1, part 2


This is the second part of the first (http://jarilaakso.blogspot.com/2012/06/testing-challenge-puzzle-1.html) puzzle I wrote. 


The reason for the second part is that James Marcus Bach (http://www.satisfice.com/blog) found a solution to the original one. His solution was much different than what I had in mind and he adviced me how to change the wording. Nicely solved and thanks for the tip!


Please remember, I don't want you to ruin the puzzle for anyone in the comments section so if you want to solve this, send me an e-mail or ping over Twitter so we can sort out the details. Thanks! 


So here goes!


What comes next in the series (you need to replace X's with correct letters to continue the series) and present your logic:
gra, avar, rvtXX

Wednesday, June 27, 2012

15 CAPTCHA Testing Ideas







This is a continuation to my friend Santhosh Tuppad's blog entry (http://tuppad.com/blog/2012/06/26/captcha-testing-dedicated-to-andy-glover/). He decided to make a fantastic list what could be used when testing a CAPTCHA. I read it, liked it, and decided I want to add my few cents to the topic! Before I go to the test ideas, I want to note CAPTCHA will not increase security and it should not be used for such purpose.

Now I am sure some of these things will overlap with either each other and/or with Santhosh's list. This is fine by me because I wrote the list as things were coming to my mind:

1. Not to pass the information to the server in plain text.
2. Not to have the same CAPTCHA repeat (in a reasonable interval).
3. Try sending a decoded old CAPTCHA value with an old CAPTCHA/Session ID.
4. Catch the HTTP(S) request and check all parameters (for example possible ecnrypted CAPTCHA ID).
5. Server-side validation for inputs (for ex. injections like here http://osvdb.org/show/osvdb/82267).
6. Using dynamic noise in the CAPTCHA is harder to break automatically than static noise (however, obscurity is not security).
7. Avoid possibility to "random success" like selecting an answer from a list.
8. If you are using a visual CAPTCHA, check out from here if it's of any good http://www2.cs.sfu.ca/~mori/research/gimpy/#results.
9. Test the CAPTCHA with some CAPTCHA breaking tools to see if it's any good.
10. If you have access to the code (or someone can tell you technical details), verify from Open Source Vulnerability DataBase if the CAPTCHA has known issues http://osvdb.org/search/advsearch.
11. Test if the session is destroyed after a correct phrase is entered. Reusing session ID of a known image could make it possible to automate requests to the page.
12. Should try to avoid using I, l, 1, 0, o, O etc. because users have problems with them.
13. Can your granny register? Should she be able to?
14. Will the CAPTCHA make people disappear from the service?
15. Do you actually need a CAPTCHA or could you use another means for the purpose you had on your mind?