Answers to survey from Mike Lyles
I recently answered to a survey that was done by Mike Lyles. In many cases, I felt like I want to explain my answer, thus I thought to put the questions in here and use a bit of time to answer with more words. Rock on
Demographics
1. What is your current role?
a. Test Manager?
b. Tester / Test Engineer?
c. Development team
d. Program Management Office (PMO)
e. Other – please enter description [_______]
My title is “Head of QA”, but my responsibilities go from head to toe. We are in building a testing department and we aim to do it really well. I am lucky to have the possibility to do hands-on testing sometimes still at this point even if the departmental matters need my time. This is mostly because my manager is really clever and let's me figure things out myself; even if I make mistakes.
2. What is your current employment status?
a. Full time employee with the company?
b. Contractor with the company?
c. Unemployed at the moment?
This is slightly complicated, but let's say I am a full time employee with the company for the sake of simplicity.
3. Are you Male or Female (M/F)?
I am a male who reads Keith Klain's blog when I need more balls.
4. How did you get into QA / Testing?
a. By Choice (gradual progression into the role – possibly from other roles)?
b. By Need (“pushed” into the role and/or asked to volunteer)?
c. My career has been in QA since the very start?
Well, this is a really long story and it's almost the same I tell when people ask how I got married. I was offered a position in software testing after I explained the manager why I don't want to write code for a profession.
5. How long have you been in IT?
a. 1-5 years?
b. 6-10 years?
c. 10-15 years?
d. 15-20 years?
e. More than 20 years? If more than 20, enter your number of years [____]
If this implies the time I have been paid working with software, it's pretty much since 2007. I used to test hardware earlier. If it implies something else, then I could say I've been into information systems and computers since around 1990 when my dad bought an Olivetti 80286 with a CGA monitor.
6. How long have you been in a Test Manager / QA Role?
a. 1-5 years?
b. 6-10 years?
c. 10-15 years?
d. 15-20 years?
e. More than 20 years? If more than 20, enter your number of years [____]
I've been a test and project manager since 2009 until almost end of last year, but currently my role is quite different and I say I am a tester who has also the responsibility of the department. I'm not a fan of titles, especially when they are used in a confusing manner; like “CEO” for a company of 3 people.
7. What is the number of direct reports you have?
a. 0 people?
b. 1-5 people?
c. 6-10 people?
d. 10-25 people?
e. More than 25? If more than 25, enter your team size [____]
I have test reports from all the testers in the organization, but I don't ask them from everyone personally. I have also reports from project managers, Scrum Masters and for example Team Leaders. Instead of written reports, I talk a lot with people and figure out how I can help them and if I could do something in their projects.
How You Spend Your Time
8. How much time do you spend during your week in PLANNING?
a. 0-25%?
b. 26-50%?
c. 51-75%?
d. 76-100%?
Now these percent questions are tricky, but let's reformulate the questions a bit. I don't do that much of preplanning once a plan has been made for a project. For the department, when I have the time, I am usually planning something. My table is full of post-it notes where I have plans for many things.
9. How much time do you spend during your week in PEOPLE MANAGEMENT?
a. 0-25%?
b. 26-50%?
c. 51-75%?
d. 76-100%?
I don't think people are managed. I talk with people, I guide them, I help them etc. but I don't think people can be managed in the sense I understand the word. I use a lot of time to talk with our HR specialist (she doesn't like to be called a manager, so I usually call her a director or VP :-) ) about people and my department. I do this because I think I have a million things to learn from her and I want to focus most of my energy on the people.
10. How much time do you spend during your week in COMMUNIICATING?
a. 0-25%?
b. 26-50%?
c. 51-75%?
d. 76-100%?
Only a really small part of my work is not about communicating with other people. If I am not talking (F2F, e-mail, chat, bug report etc.) with someone, most likely I am preparing for it.
11. How much time do you spend during your week in MANAGING EXECUTION?
a. 0-25%?
b. 26-50%?
c. 51-75%?
d. 76-100%?
I don't quite understand this question. Most likely it's because I tell people what I want, but not how I want it. I am a firm believer that people can be awesome and that they can have a lot better ideas than I have.
12. As a Test Manager, what is your role in EXECUTION of TESTING?
a. My team executes all of the testing always – I do not execute?
b. I execute testing with my team?
c. My team executes the testing – but I participate if needed?
Tricky question. I am not a test manager, though, I manage testing. I execute a lot of testing myself, but I don't do that with others. I let people get their stuff done and learn from their mistakes. I even tell them about my own mistakes so they know I am not just saying it.
13. How much time do you spend during your week in MEETINGS (stakeholders, team, reporting)?
a. 0-25%?
b. 26-50%?
c. 51-75%?
d. 76-100%?
I try to be efficient with meetings. Usually I don't call them meetings, I just show up to someone and have a list of questions or answers. Recently, I have been changing that with a few people, but with most, I prefer to have more informal discussions.
14. How much time do you spend during your week in PROCESS IMPROVEMENT OF THE QA ORGANIZATION?
a. 0% (the team uses current defined processes & never changes)?
b. 1-25%
c. 26-50%?
d. 51-75%?
e. 76-100%?
Process definition and documentation has been one of the biggest tasks for me while building the department. While doing that, the processes have been improving and we have used a lot of discussions to talk about this. I am the kind of person that I see improvement opportunities everywhere so I spend a lot of time to figure out improvements for all kinds of processes in the company; especially if they are directly about people.
Your Thoughts on Test Managers
15. Do you think that people make better Test Managers if they have a background of software development?
a. YES?
b. NO?
I assume this means they have experience in programming. Yes, I think it can be a benefit to know programming. I don't, however, think it's a general kind of thing that makes people better. There are so many different kind of roles for test managers and definitions for having background in software development... But in general, I think people should learn their craft more. If this means learning programming for a software tester, I don't see what is the reason not to do it.
16. Which skill do you think is the most important for a Test Managers?
a. Project Management Skills
b. Technical Management
c. Test Management Skills
d. Communication Skills
e. People Management - HR & Personnel
f. Defining Processes & Methodologies
g. Handling Exceptions, Risks, Issues
This seems like another trick question. If you work as a test manager, it would sound obvious your most important skill is test management. Interestingly, we are not provided any skills that the options actually contain. I think Mike did this on purpose. I would say that critical thinking, questioning, systems thinking and leadership skills are really important in many cases for a test manager.
17. Is the role of the Test Manager clearly defined in your organization?
a. YES
b. NO
We actually don't have any test managers, if I am not counted as one.
18. In your organization, is the Testing/QA function a global centralized function or are the teams integrated separately with each specific area of the organization?
a. Centralized
b. Integrated / Spread out as multiple teams
We have testers in various teams, like Scrum, Kanban and we have even sold testing as a service. We do gather up and talk together on a weekly basis. We don't really make “teams” from testers, but we have testers working as parts of development teams. The testers support each other and they work solutions together, which is really cool.
How Do You Manage
19. What do you feel is the best way to communicate to your stakeholders?
a. Team Meetings?
b. Emails?
c. One on One?
I have a lot of different stakeholders. I use e-mail in many cases when I want something “documented”. I prefer face-to-face discussions when possible, but not always. I think I am way better in face-to-face discussions and I need to work a lot on my skills when it comes to written communication.
20. How often do you speak to your customers?
a. Daily?
b. Once a Week?
c. Every other Week?
d. Once A Month / Quarter?
We have clients, McDonald's has customers. :-) There are some clients I talk with more often, some I don't talk almost at all. It depends a lot on my role for those projects. I love talking with different customer support people because they give me great information of systems that are already live. I would like to talk with the clients more often, but currently I focus on something else.
21. How often do you manage / assess / review RISKS?
a. Daily?
b. Once a Week?
c. Every other Week?
d. Once A Month / Quarter?
Basically, this happens constantly. If it was meant for example to have a formal meeting about it, then maybe daily. I am involved in many projects so it's not really even possible currently to avoid this. Everything we do causes a risk, so managing risks is really important.
22. What is your most probable/preferred method for mitigation of RISK?
a. Stand up meetings with the testing team that
b. Escalation to management?
c. Meeting with the Stakeholders?
I talk a lot with people and sometimes I send e-mails. I don't think talking with “testing team” is a good way to mitigate a risk, but talking with the scrum master / project manager / whatnot and a team is better.
23. How do you handle conflict with other teams (stakeholders, developers, PMO)?
a. Heated debates?
b. Escalation to upper management?
c. Structured negotiation to reach a mutual agreement?
d. Depends on the situation?
This depends a lot on the situation. I have one manager over me; who happens to be really cool. He lets me fail and learn. Sometimes I think he is even pushing me to make mistakes and go to conflict situations. I handle most conflicts (caused by me) with face-to-face discussions and I forget to follow-up on them frequently.
Testing Group Training / Growth / Improvements
24. What stages of a measurement / metrics program are you currently in?
a. No program exists
b. Program exists but no improvement to the overall process
c. Program exists and assists with showing the value of the organization & quality of deliverables
We collect some numbers and we make some reports. If a client wants something, we talk with them, let them know our opinion and find an agreement. We try to keep the number information small scale and only as triggers for questions.
25. Which areas of test management do you feel needs more training in your organization?
a. Test Strategy?
b. Test Planning?
c. Defect Management?
d. Automation?
e. Performance?
f. Measurement & Metrics?
If performance here means personnel performance management, I think it's where I have been focusing a lot lately. I am not a huge fan of training; I prefer for example coaching and learning from experience. This is a question I should ask myself again on Monday. :-)
26. What methods do you use the most to ensure you are doing “the right things the right way”?
a. An internal governance process to review, modify, and publish the processes?
b. Trainings / whitepapers / consulting to align with testing industry best practices?
c. Feedback (either with surveys, lessons learned, project closure meetings) from the stakeholders and other project team members to review effectiveness of testing?
Something like the a) option, but not really. I put a lot of trust to people. I give them direction and guidance (and a lot of difficult puzzles), but I think I don't need to do much more. If we make mistakes, we learn from those and share them with everyone.
27. How many conferences does your team attend per year?
a. 0?
b. 1-5?
c. 6-10?
d. More than 10?
“Not as many as I would like to” is a common answer to this one, I presume. :-) I would really like to do that a lot more, but there are some resource problems currently with this and it's not really possible. I think next year will be more active on this side. We are also planning to have a few conferences in Romania where we have a fantastic testing community!
28. Is keeping relevant and up to date with QA technology/processes critical to the success of a testing organization?
a. YES
b. NO
For me, yes. Dunno about others. I think in general we need a software testing to step up in Europe.
Internal Issues / Showing Value of QA / etc
29. Complete this sentence…..”As a Test Manager, I feel “
a. Respected by the other teams (stakeholders, developers, PMO)?
b. Disliked / distrusted by the other teams?
c. Respected at times, disliked at others – a constant struggle?
I am really respected and valued by my colleagues, clients and my manager. I am very lucky, it seems. I value our testers and my colleagues a lot too, so I give them gifts sometimes just because I feel like so. :-)
30. Testing is often seen as a cost center instead of a profit center. As a Test Manager, what methods do you use most office to show the value of testing?
a. Leverage measurement/metrics results to show the value?
b. Focus on the reduction in rework by finding and resolving defects early in the SDLC?
c. Showcase improvements to testing which simply processes, reduce overall project costs, and improve overall product quality?
We find things that are important for release decisions. We find important bugs. We provide quality-related information of the products we test and this is what others want from us.
31. What percent of your week do you feel you have to “sell” the value of the testing organization?
a. 0% (organization completely supports and backs the QA group)?
b. 1-25%
c. 26-50%?
d. 51-75%?
e. 76-100%?
I don't have to sell testing internally. We have a support from all departments and managers.
Your Opinion
32. What role do you think the Quality organization has in the decision to move to production?
a. Quality Organization has the GO / NO GO decision – ultimately?
b. Quality Organization provides inputs to those accountable for releasing – and may suggest strongly to not go, but does not own the decision to move to production?
We inform; we don't make release decisions. We don't fake we do other people's work, we focus on doing our work better.
33. What Percent of Releases to Production have Defects or Code Issues in your organization?
a. 0-25%?
b. 26-50%?
c. 51-75%?
d. 76-100%?
All. At least I haven't seen a software product without a defect. It's also important to note here that a defect to someone at some time might not be a defect to someone else or for example on some other time.
34. In my organization, QA is involved early enough in the project?
a. YES
b. NO
I don't call testing as QA, even though it's in my title. :-) We include testing in the project as soon as we see possible. I think we do a really good job in that.
35. In my organization, the senior decision makers & stakeholders listen to the QA team?
a. YES
b. NO
This is a definite yes. We have a lot of trust and listening.
36. In my organization, the senior decision makers & stakeholders understand QA / QA methodologies?
a. YES
b. NO
They don't need to understand what testers do. They might, but I don't want to even go to these details with them. Just like I don't ask them about code.
37. In my organization, QA is provided adequate testing resources for the projects?
a. YES
b. NO
This question sounds a bit confusing to me. Mike is again playing with my mind here. I am convinced he used terms like “QA” and “resources” in contexts where he knows people like me get derailed. :-) Anyways, we have the tools and time we need for doing good testing. Sometimes we might ask for example for more time, but usually we don't even need to ask for anything. Testing is really valued in our company.
38. In my organization, QA is almost always blamed or held accountable for defects or issues in production?
a. YES
b. NO
Nobody would be so silly in our organization because they know testers are not making the code. We could, in theory, face a question “why didn't we spot this”, but we would have that as a learning point.
39. In my organization, deadlines will take precedence over the quality of the deliverables?
a. YES
b. NO
We don't deliver if we don't test. It's more important for us to deliver good than to deliver. In everything we do, we focus on having great products for the clients.
40. In believe that my organization truly WANTS quality deliverables?
a. YES
b. NO
As stated above, we focus a lot on this stuff.
41. Do you believe it is possible to test everything?
a. YES
b. NO
This is not a question about belief. It's not possible. For this, we would need infinite resources and it's not quite something we could have.
42. In my organization, I am given clear direction and support from my management?
a. YES
b. NO
Yes, my manager gives me direction and support. He also lets me figure things out so sometimes I just don't hear anything from him and then all the sudden he says “we have a problem with project X, fix it”. It's pretty neat to have that much of trust from the manager of the company. I also get support if I need, but mostly all support I need is to know I am trusted.
43. In my organization, I provide clear direction and support to my subordinates?
a. YES
b. NO
Depends a lot on the case. Lately I've been focusing on other things mostly because they are doing a good job in the projects. If I see the need to be more clear on something, I do it. Otherwise I ask if it's needed.
Area 2: Survey Questions for Everyone (Discussion Questions)
1. Enter your name (OPTIONAL)
Jari Laakso
2. Why did you want to be a Test Manager?
I never really wanted to be a test manager. I didn't even want to be a manager of a department. But I like both of those really much! I like testing and I like working with people. The role is not so important for me. I think I am still a better tester than a manager, thus I have a huge deal to learn on a continuous basis.
3. Name 3 to 5 things you would do differently if you had the authority to choose how your day as a Test Manager would look? (Include suggestions on how that could be achieved without disrupting the testing functions or degrading quality of the deliverables)
I don't really know what to say here. I have the authority to pick my days and actions. I give that for the testers also. If they need something, I take it as a personal challenge to figure out how to get it.
4. Please list topics within your testing organization where you feel there is a lack of alignment/agreement. Think about topics that seem to always start a debate or where the team is not in agreement. Describe the various viewpoints and why the alignment is lacking or non-existent.
I am always picking up on words. That's pretty much it. But I do that a lot. I don't do it anymore that often with non-testers because it doesn't matter that much to me what they say. If people are really into learning testing and improving skills in it, then I also go “tough” on them so they have the chance to learn.
