What I learned in 3 days of RST Training

I'll start be noting this post doesn't include a complete list of things I learned in the training. The reality is that it's been 4 days since the training and I keep learning and understanding things differently. This describes the RST training really well. This course is not just about information and tricks. It will lift you up on a whole new level.

Testing is often understood as asking questions of/from the product. The course will give you hands-on exercises around questions. You will learn what to ask, how to ask, how to find information etc. You will also learn how to deal with situations when the replies are vague or non-existing.

Testers often need information and help, but feel alone with their challenges. You will learn to use resources in a creative manner. As an example, in one testing exercise, I realized I can gather information faster by asking from others what they found out instead of trying to solve the puzzle by myself. When time is limited, you need to be able to be creative with information gathering and this you will be learning each day on various exercises.

Software testing without tools can be a fun thing, but you will limit your testing if you do everything manually. This course will show you diverse tools, including for example randomizers and hacking applications, that help you along the way. When this happens, people will start telling about the tools they use in their testing. If you listen carefully, you will learn about many tools that help you in your testing. As an example, I taught a course colleague to use 3 different tools in one evening while we discussed about philosophy and ethics around web service testing.

Many times we see testers having fights around ethics in testing. RST includes a lot of philosophical discussion and examples about ethics. Not only you will learn what things to avoid but you will also learn how to deal with those situations and what things you can offer instead. You will learn to say no and advice what else to do.

Testers often find themselves in situations where they feel their words were taken out of context or misinterpreted. In RST you will learn to use safety language more effectively and you will learn why it's important to be used. You will not say anymore silly things like "we can ship this product", but you will advice the manager in charge to understand the results of your testing. You will not say "the product doesn't have bugs" because that could result in losing your credibility 5 seconds after a public release.

Commonly testers are talking about boundary testing, BVA and ECP. You will learn what boundary testing actually is. No, not the ISTQB version where you have a formula that fits in all situations. You will learn what it actually means, what is required from a tester and why it's simply wrong to say "most bugs are found around boundaries". You will also learn that the reason to this claim is because you are mostly testing around the boundaries! By clever examples and exercises, you will learn how this goes in real life.

A big part of tester's work is reporting. RST will teach you the different levels of reports, how you should do the reporting and what will give the most value to the stakeholders. After the course, you will see all reports with different eyes. Reporting used to take a big part of your time, but you will now be ready with tools and techniques that help you to minimize "waste work".

Often testers say their work is repetitive and doesn't need creativity. Yep, you guessed correctly, RST will teach you to think and work differently. You will learn to do testing in the hard way, which is also the most rewarding way. You won't anymore see the fake simplicity in software testing problems. Your eyes will open and soon you will find yourself doing the kind of testing that the big guys were talking about.

Oh yeah, and this is pretty much the first day!

RST = Required for all Software Testers

I’ll start by saying I am sorry the trilogy (http://jarilaakso.blogspot.com/2012/02/reasons-of-illusions-on-testing-part-ii.html) is missing the last part. The last part has been coming up a long time already, but due to some excuses, I have been delaying it. I will focus on it again, but currently I have something more urgent in my mind: RST (held by James Bach) and this whole week in Bucharest.

I was able to convince my boss to send me to Bucharest for the whole week to participate to RST and a peer-conference. I can’t actually take the whole credit for it; my boss is fantastic, really supportive and saw this as a great investment from many points of view. After the week, I’ll say to all the bosses around there who care about testing/quality: make the investment!

Before the RST begun, I made a list of questions I want to ask from James and a list of things I want to learn. Little did I know… but here are a few things I listed: how RST works in my context with projects that last sometimes years, how will this make me a better tester, and how will my reporting skills increase with 3 days of training. If RST would be anything like a traditional training, those would be really good questions, and maybe they still are, but what you get from RST is far more than that.

The whole idea of RST training rolls around doing exercises, showing the weak points of poor testing and challenging everyone mentally. RST gives tools to do excellent testing, including what are important questions and for example why testing nearby some (imaginary) boundaries is just not enough.  Now I’ve read earlier that RST should have more hands-on exercises. I counted we did at least 25 exercises which most comprised of multiple stages, a lot of verbal reporting, discussions with various stakeholders and applying RST in practice. Please bear in mind this is only a part of the training. It’s unbelievable how much of information you receive in these 3 intense days!

James is the toughest teacher I have ever seen and the better you do, the more he will push you. This is because he wants you to learn, not because he is mean. He has incredible skills to read people and activate their minds. Getting an answer from him is sometimes difficult, but if you are persistent enough, he will make you answer your own question! Like I noted earlier, this training gives you much more than you will ever imagine. You will understand even reporting on a whole new level after the course.

Quite commonly I use a lot of safety language and after a training I could say something in the lines of “I think I did pretty well because I had some good ideas”. Now there is nothing wrong in this, but this time I can say “I did really well because I had sharp questions, I talked about philosophy of testing and I showed I care strongly about testers’ ethics.” Why am I so confident, some might say overconfident, to state something like that? Because James is also a rewarding teacher and he Tweeted about it! In my world, this means the people who care about testing just all got to know a bit about me. What other training could possibly have anything even close to that?

If you want to participate in the course, but have hard time convincing your manager to get the funds approved, please send me an e-mail. I will be happy to help. It’s not about if your company can afford this, but more if they can afford not sending you to the training. It is maybe the most important training of your whole professional career. We need to make it happen!

Note: I need to do the training with Michael Bolton, too. Hopefully this year, but latest on next year.

Reasons of Illusions on Testing - Part II (Reasons)

In the previous (http://jarilaakso.blogspot.com/2012/02/illusions-on-testing-part-i-chimera.html) post I noted five different chimeras of testing. They were:

1.       Pre-scripting testing work

2.       Using terminology (test vs. check)

3.       Women are better on an average

4.       Comprehending exploratory testing

5.       Writing claims without arguments

As some of you already speculated, I am going to put some blame on ISTQB etc. for all these problems. I am not saying it’s all because of them. I actually blame the testers for all this, they are responsible of their own answers, but it’s quite clear how “certification” is visible in their answers. I don’t want to turn my blog as a rampage against any kind of “authority”; I am only pointing out for example what kind of problems certifications can cause.

When we think about the problems with pre-scripting testing, as in creating test cases, we can rapidly see the testers were able only to cover very shallow cases. There were indeed really good ideas too, but the problem I see is that testers weren’t able to write down some really high risk cases, things I would call “basic tests”. On the other hand, I am pretty sure many of the testers would perform these tests. Maybe not on the first “round”, but evidently when given time. Possible reasons why this problem occurred: too hasty work, poor documentation skills and lack of imagination.

The second problem I noted was lack of differentiating testing and checking. I’ve talked lately with quite many people about this and pretty much everyone seemed to think it’s not a necessary differentiation. I know this might sound like a troll, however, I claim everyone serious about testing understands the difference and uses it accordingly. I don’t really care if non-testers use “test” in any way as long as testers use it properly. All professions have their own terminology (jargon) for a reason. An additional rationale for differentiating the words is for example to avoid anyone thinking that testing can be automated. A few reasons for the problem: not being serious about testing, using ISTQB terminology and working with customers who use a certain kind of terminology.

On the third point I illustrated how women were doing better than men in the test. It’s true that some of the best answers came from men, but in overall women had better results. (As a side note, I remember reading about quite many surveys on various domains where the same patters repeats.) There are many possible reasons for this result and I think one big part is that generally women analyze situations more than men. Men seemed to lack confidence (online searches for answers and even misquoted) and have short answers, which could imply they either were lazy or want to use more time with hands-on activities.

The fourth chimera was about exploratory testing. Everyone appears to write it in their CV, but few understand what it is. It’s seen as something that will be done after all test cases have been executed and there is time to look for other issues, it’s widely described as experience-based technique and commonly understood it doesn’t have any means for providing information what has been done. My blame-list for reasons: ISTQB, lack of professional aspiration and the culture in the company they are working in.

I saved the best part for the last. There seems to be a strong culture of giving claims without justification. If the claims would make sense (i.e. I would agree with the claim without hesitation and I would know how the tester came to the conclusion), I could accept some of them. But in this case there were a lot of claims I just can’t accept. Most likely not even with strong argumentation, which would be the minimum requirement for me to consider the statements as valid ones. (Before sending the problems to testers, I mentioned there are no correct answers, but they need to be able to convince me their answer is a good one.) The main reason behind claims without explanations: lack of critical thinking.

What do you think are the reasons for the mentioned issues? Maybe you want to share another general problem you have seen in testers thinking?

Illusions on Testing - Part I (Chimera)

I don’t know yet how many parts this post will have. I only know it will be more than one so I already started numbering the titles. My initial idea is to have 3 different parts (Chimeras, Reasons and Future), but I won’t promise anything as I write like I live my life; which is pretty exploratory and not based on too much preplanning.

Recently I made a small-scale research for example on different concepts of testing and how testers approach testing problems. I call them testing problems because I presented them as problems, questions. I’ll present and analyze the results in this series of posts.

The first observation is related to pre-scripting testers’ work. There were a few problems where the testers needed to create test cases, test ideas, minimalistic test plans and so forth. Some of the people had really nice ideas with interesting inputs they might use in testing. The biggest shortcoming s came from forgetting testing outputs and “the big picture” (as in, you have a dialog box, but how does actions on it affect on OS, browser, file system …). Basic things like user rights, file system connections and “simultaneous” manipulation was forgotten. As said earlier, I’ll go to the reasons in the next post.

A really interesting observation is most people thought a “check” is a “test”. I could understand how the words get confused with “common” people, but it was a surprise coming from software testers. I see this as something that should be addressed as soon as possible. Checking is something I can do with a computer program.  Testing needs skills, imagination and intelligence – something way over the league of current computers and software. Testing is about learning and questioning.

Recently I had a wonderful discussion with Tobias Karlsson (https://twitter.com/#!/tobias_karlsson) about the difference between testing and checking.  He is against the distinction and feels it’s useless. The discussion was wonderful for two reasons. Firstly, because he admitted he understands the difference between checks and tests. Secondly, because his logic was wrong, which gives confidence my own logic was working well.

He mentioned calling UAT not testing is like saying thrash/speed metal is not metal. This is clearly a play with words instead of a real comparison between the distinctions. Checking has a one-dimensional result; there needs to be a presumption/belief before a check can be done. A test, however, finds new information and it answers to questions like “what will happen if”.

A third remark I made was not surprising at all. Women were doing much better than men. When I say “much better”, I mean *much* better. Simply put, women had way more context in their answers, they analyzed the problems from many points of view, they challenged me by leaving questions in their answers and they used online search engines only to confirm some facts they already thought of. There is an enormous difference between most replies from men in this respect.

The fourth point was something I was prepared for. Except that I wasn’t ready on the scale of confusion with “exploratory testing”. I found it alarming when testers don’t see exploratory testing in the way it should be respected. For some reason (I blame mostly ISTQB Foundation certificate for this, but more about that later) ET is seen as something testers do when they don’t have anything better to do. The love many have for test cases is slightly distressing, too.

Fifth, and last, observation I am going to list was the colossal amount of claims without explanations/arguments.  A few examples: “test cases show how much testing is covered”, “testing efficiency depends on (code) design plans” and what I really enjoy “test cases can be run by inexperienced testers”. The last was repeated by many testers. I don’t mention here any of the “best ways” to do things, because I used that as a trap in a few problems.

With all the saddening replies on some of the testing problems, I still believe there is a lot of potential in each tester in question. They only need guidance, motivation and someone to walk along them. What could be a bigger challenge than to improve others to a level they were not even aware of?

Internet Banking Experiences From Tester’s Point of View

Obs: I am not going to reveal the names of the banks in question for the time being. However, I might change my mind later and add them here. My intentions are not to show how bad these banks are but to show how bad experiences I’ve had. I’d appreciate if readers would respect this and not even try to guess the names of the banks in the comment section. However, I do recommend readers to share their experiences and I am not imposing any rules on what you should or should not tell.

I have experiences with Internet banking from S (Finnish), R (Austrian) and T (Romanian). The experiences differentiate widely from UX point of view. In this post I will put in plain words some of the obscurities I have seen with the Internet services of these banks. Please note already at this point that in my opinion an Internet bank should be top-notch from quality, usability and user experience point of view. Not only because my examples are hugely profitable and wealthy organizations, but also for example because they have so diverse end users in their systems. Respecting end users is a must for them.

S

There is a person who can access my account details. I asked from the bank about this and was told they don’t know how it’s possible. I updated my service agreement with them and was told the problem is not going to remain. The problem remains.

There was a big integration project to D (Danish) system which was a disaster. (They lost in Finland almost 30 000 personal and over 2000 corporate customers in about half a year.) Even ATM’s stopped working and gave Danish messages in Finland. The online banking integration resulted in people not being able to sign in the system, getting duplicate payments, receiving error messages in Danish… After this, S has had their fare share of problems with an average of over 1 major/critical issue each year.

R

I started using them around the beginning of 2008. At this point, the system didn’t need a token or anything that changes; it was enough to know a username and password.

I forgot my password at some point as I didn’t use their services in a long time. I noticed quickly two things:

1) The system was clever enough to give a different error message for a wrong username and a wrong password.

2) The login page didn’t have a restriction for unsuccessful login attempts.

Later on in 2010 or so I thought to return as their Internet banking customer. I went to the bank and realized how nicely they had adopted using tokens. I felt security was finally taken care of. Unfortunately, this time the bureaucracy to actually get it working was too much, so I left after 30 – 60 min of trying to get an account.

T

I wanted to leave the ultimate example as the last. I have seriously considered many times to change my bank because of the problems I have had. I still am. I just haven’t found a (local) bank with much better ensemble.

First was to take Internet banking in usage. This happened either with Token/SMS or installing a certificate on the PC and using that for credentials. I wasn’t given a token, but thought to try to SMS option. Surprisingly the SMS option wasn’t implemented. All I saw was “login with password/token”.

Then I thought to try the certificate authentication, but that page returns “SSL peer was unable to negotiate an acceptable set of security parameters.” when trying to access it. I tried to request some certificate via an online service, but never received anything.

Finally I got a token from the bank and I was ready to give another try. I changed the language to English and noticed the bank cared only to translate part of the essentials. I thought to keep a consistent language all over so I moved back to Romanian. This logs you off their service and asks to login again.

An interesting note is that the login screen has Romanian, English, Hungarian and French as the language options. Other parts of their web page had only Romania, English and Greek (!).

Then some issues from inside the banking system. When you are on the Messages view, the oldest message is on top of messages by default. Always. Change the order, read a message, return to the Messages and notice the default order is again applied.

I paid two bills with great success. The first bill was accepted (I was even prompted “Operation successful!” and presented a green circle to represent “OK”) and I was happy how easy it all was. Until today, 8 days later, when I found I had a new message in the Messages section (there was no alert for it, nothing in main page to get my attention etc.). The Message was that the operation was not successful because a mandatory input was missing.

The second bill was a success story, too. I had to pay a gas bill of 133 lei and 77 bani (1 ban is a cent of 1 leu), so I logged again to the online bank. I used the numpad for inputting numbers, because it’s more fluent for me like that, so I input 133,77 lei. I authorized the payment and noticed something odd on the screen: I just signed a payment of 13,377.00 lei. Happens that comma is forcing the amount to kilo-amounts. What a wonderful feature!

Now I needed to cancel the transaction before it’s done. (I forgot to mention that I did this at 8:30 AM and their Internet banking is performing transactions only 09:00 – 17:00 Monday to Friday. Yes, I am not lying.) I found “Order Tracking”, but there one is allowed only to view transactions, not to do anything on them. Finally from “Operations” -> “Standing orders” I found the payment. I was happy to see there is a Cancel checkbox; until I noticed it’s grayed out. I didn’t have user rights to cancel a payment I did.

Regarding user rights, when I got my Internet banking working, in a way, I had access only to other of my accounts. It took 3 personal visits to the bank to have rights to see details of my other account.

Tips for CV's and Job Applications

The following is slightly idealistic view on applying for software testing jobs. Reader is recommended to reflect his own values and think what kind of compromises he is ready to make.

What to considering avoiding on your CV/cover letter

1)      Lying

2)      Using (for ex. ISTQB) certificates as a proof of testing skills

3)      Explaining experience without details (“6 years in web testing” doesn’t tell really anything)

4)      Explaining experience with too many/waste details

5)      Exaggerating your experience (don’t put “MySQL” in your skills if you didn’t work directly with it)

6)      Typing mistakes

7)      “Empty holes” in the timeline

8)      Abbreviations and jargon

9)      Confidential information (for example customer name)

What to consider including

1)      Explaining your skills and how you have improved on them

2)      Explaining what you bring to the company

3)      Explaining your way of testing

4)      Giving arguments to your approach to testing

5)      Showing a capability to critical thinking

6)      Showing contextual talent

7)      Humor (this is quite a bit so-and-so case)

8)      Extra curriculum activities such as blogging, writing, public speaking, visiting conferences

9)      A sample test report as an attachment

What jobs to consider passing up

1)      If they have “certification” written in them without a negate

2)      If the job includes creating/aiming at one-dimensional metrics (bug counts, test case amounts, …)

3)      If the application has a lot of buzzwords or nonsense in it

4)      If the job/company doesn’t meet your values

5)      If you are not even close to what they are asking for

How would you continue/edit the lists? Let me know about your thoughts!

Next post most likely about Internet banking experiences from a tester’s point of view. Stay tuned.