Time to get back at it

Hello there everyone,

Been a while, hasn’t it? All I can say the summer of 2017 has been a little crazy and although my contributions to SPaMCast have been going strong for the past 3 months (hint: prerecorded 🙂 ) I have not kept this site in sync with them. For that I apologize  and look to get things back on track.

So for this blog it will be a condensed version of what was discussed in the podcasts, I am sure you have all be listening to them. If not here is the link, http://spamcast.libsyn.com/, outside of my contributions they are worth listening to.

So in the end I only missed 2 episodes: Leading in QA and QA leads.

Leading in QA

When this topic came up we were initially going  to talk about how leadership in a QA environment was different and how difficult it is. Don’t get me wrong leading in QA is a tough thing to do. I have the grey hairs to prove it. A lot of re planning, modifying goals on the fly, changing directions on a dime and dealing with multiple stakeholders with different priorities. As the conversation went on we started to talk about leadership as a whole, not just with QA.

With all the stuff I mentioned above, how is that different from what any other leader has to deal with? Not just in software delivery, anywhere? In the end it is all about doing the right juggling act to get things done. The one thing I have seen where leaders fail is that they try and make everyone happy. That, to me, is a pipe dream. Even in some of my MBA course discussions we all talked about how it can never happen, someone is not going to like any decisions that people make.

Now here is the key to my success in being a leader, I make sure everyone is not completely happy with what I decide. Now before you say “WHAT THE …..” hear me out. The way I go about it is I get people to talk, not to me, to others that are impacted with what I would decide. I point those individuals in a direction and find a compromise. Are some people going to be more upset than others? Possibly. The outcome of what I do is hear them all out, understand their side of things and find as close to middle ground as possible. In the end it gives the sense of ownership to everyone on what I decide. It is not a dictatorship where I rule with an iron fist.

Now you are probably going: Jeremy, now you are not doing anything or making actual decisions. True, to a point. Usually by the time I get to this point where I bring everyone together I have made my decision based on the data provided and have planned accordingly. Here is the problem with that, and I am sure you have all experienced this, sometimes you don’t have all the data. Normally I find meetings like this, or a couple emails things come out of the woodwork that could throw a big wrench into the plans. So it is important to get people involved. I would keep going and will gladly answer comments, it is I want to talk about the other topic that was missed over the summer.

QA Leads

Right off the bat during my career I will say that being a QA Lead for projects was probably the toughest job I have had to this point. Here is the main reason: I had no real authority and all the responsibility. That combination is not a fun position to be in. The analogy I like is you are stuck waist deep in mud with a rope above you just out of reach, you can’t move, there are people on the outside of the mud pit with sticks to reach out to you, yet they have their backs turned to you and they are wearing headphones. Basically you are stuck.

Now it is not all doom and gloom, it is a challenge. How do you get people that have do not report to you to do things for your, or how do you get people that you report to do things you want them to do (This is one of my discussion proposals to QUEST 2018)?

Not only are you leading without authority, you are also planning and understanding risks to what you are trying to get done. It’s tough yet doable, regardless whether you are an extrovert or introvert. In the end it is how you treat others and get them to side with you.

The other aspects is what do I expect out of QA Lead. In the end I need to see someone that will tell me what is really going on, take initiative to make decisions that will get stuff done (letting me know what they are), and plan appropriately. Yes there will be times the schedule will seem unreasonable; welcome to QA. I have never seen a project schedule that was perfect or provided sufficient time. This is the opportunity of a QA lead to show resilience and innovation. When I say innovation I do not mean create some new tool to get things done. Look at the definition of innovation there is more to it than tools and such. Thinking of doing things outside of what is normally done to make it better.

The one thing I do not like to hear is “we have always done it this way”. That tells me people are not willing to find improvements. I have never seen a process or system that could not be improved upon. Even small tweaks can have big impacts or open doors to better opportunities.

Although I said the QA lead was the toughest role I have had, it was also the best. It gave me the opportunity to learn and lead without authority. I took those skills that I acquired and use them with my current role. They are easily transferable and  makes the team better.

I look forward to reading comments and keeping the discussion going.

If you want to hear my views on other topics feel free to let me know.

Value of QA

Back to my very first blog, Root Cause Analysis thoughts, I brought up my value diagram for QA.

QA Value

This Porter competitive advantage graph is how I truly see how QA should, not only use to promote the discipline to stakeholders, it should also be used within our very discipline.

Since I achieved my MBA years ago I have viewed our discipline in a different set of eyes. Speaking about the value QA will provide and remove the stereotypes that people have regarding it. I have lost count on how many times I need to correct people when discussing my career.

“What do you do?”

“I am a Director of QA”

“QA?”

“Quality Assurance”

“Oh, you test things?”

Then I would go into a long conversation of what it is I actually do. Now don’t get me wrong I do talk about how I am responsible for “Testing”, it is just I go further down the rabbit hole in what that all involves.

Now when this conversation started about early in my career I would have just agreed that I “test things” and left it at that. To say that now, with what I have learned and practiced over the years,  lowers the value of what it is that is actually done.

As a discipline and community I feel that are some areas that don’t help the cause to better legitimize what it is that we do.  Just as there are great books, speakers, articles and classes on the subject matter there are more that lowers peoples views of what it really means to be QA.

Just recently I did a search for courses on QA and Test planning for some peers that were looking for something to do outside of going to a physical class. A few were based on certification standards,  my views on that Thoughts on certification, while other were just sites that offered a course or a series of courses. Viewing some of the overviews I provided some recommendations and sent them off.

Do say I was disappointed with the feedback coming back on what they went through is an understatement. The courses did not even go into QA, they were focused on Quality Control. Physical testing of software not trying to avoid issues early. I get it, it is something that we do within the discipline. What gets me upset is they talked about learning everything they need to know about QA. In the overview it looked like they were going to go over some of that, unfortunately it was wrong.

I have asked this for years is “why can’t there be a diploma program on software QA?”. There are ones for software engineering and development. To help with that I am trying to get into post secondary institutions to see what can be done to get that going or to enhance what they already have. I feel getting something a little more solid than a printed piece of paper with some third party logo in the corner for attending a 3 or 5 day course would help legitimize the QA discipline more.

We as a community have to not only show the value of QA to those on the outside, we need to also keep up the value realization within ourselves. Help each other build and grow.

Conferences

Last month I had the privilege to share my ideas at the QAI QUEST conference. It is something that you all know I enjoy doing and even had the opportunity to fill in for another presenter, who unfortunately fell ill and was not able to present.

I love attending conferences it is a great situation to not only meet others within your own community, it is also where everybody has the ability to take ideas and lessons learned by others into their organizations. The networking that goes on is unbelievable and understanding how others see and approach QA.  Every year I have been at QUEST I catch up with people that remember something I spoke about years ago and mention that some of my ideas have been introduced in their teams successfully. Now I did have some not so successful stories come in, and it was great to discuss it and see what could have been done differently.

Not all ideas will work for all organizations and that is one of the key things that anyone that goes to them should understand. When attending courses, workshops or track sessions it must be viewed with an open mind while keeping in mind how their own organizations work. There are limitations that could stop anything from happening and having the right way to frame ideas to work is key. To go in there to take someone’s idea and directly try to apply it right away will have a high probability of failure.

In my talk at QUEST 2017 about Root Cause Analysis; one of the things I brought up was that when initially trying what I was discussing was going to be hard at first and that people would need to be persistent in getting the right information. I didn’t want to sugar coat it or give anyone false hope that what I was presenting was going to be the silver bullet that everyone wants. In the end it was a well received discussion where people came up to me afterwards to discuss what they were going through and asked for ideas on how to implement points to help them out.

I also had the opportunity to have one of the senior development leaders from my place of employment come with me. It was a good learning experience for him to see the different aspects of QA and how what I am doing with the organization’s QA team fits into what the discipline is doing. It provided a better appreciation for what is done and how closely Development and QA are tied and how close the two are tied to efficiency in development of a product.

I think this is something that should be done with different discipline’s conferences. The opportunity to see how the “other side” works would be such a great benefit It is a great opportunity for leaders in QA Development, BA and Project Managers to fully step into the others shoes for few days and gain heightened awareness of impacts they have with each other. It will also provide opportunities to find areas of improvement and effectively work together.

I look forward to future conferences, as a speaker or attendant, because each time the amount of knowledge sharing is at a different level because instead of just reading about something I will have the opportunity to foster discussion.

UAT in Agile

In my previous discussion I talked about Product owners being involved in the testing process. In some way shape or form they are, whether they realize it or not. So with the current topic it is a perfect segue: UAT in Agile testing. Depending who you talk to or read there are different opinions, the yes and no side.

 

Yes side:

With the user involved within the team it makes sense for them to play with what is being changed so they can provide the feedback sooner.

No side:

Takes time and the feedback loop occurs during the demo at the end of the sprint for future planning.

There are a lot more detailed arguments for both sides and I don’t want to delve too deep into each before I discuss my thoughts on the topic. If you are at the QUEST 2017 conference in Chicago maybe, we can meet up for a discussion over some beverages.

So here it is, what I think: I am on the yes and no side. Makes sense? Probably not, let me walk you through my logic and hopefully it makes sense in the end.

If we look at it from a pure Static Testing view it is most definitely a yes. Reviews and assurances that the stories have sufficient detail or will have enough detail as the sprint moves along is critical to have what you want in the end. At the beginning it is a vague idea of what is wanted:

“As a (someone), I want (something), so that (there is a benefit)”

That at the beginning is pretty vague, although what happens is ideas start to flow and things get into motion. When that happens the vagueness lifts and more details are needed to ensure the endpoint. The key to all this is the fact that although there is no physical program to use in the early stages the fact that the feedback loop early is a form of testing that the user has a lot of involvement.

So the code is done and being test within the team, what does the user do now? Aside from planning what is needed in the next few sprints why can’t they start to look at what is being developed early? The caveat with this is that they need to be aware that it is early and it may not exactly work as planned due to multiple reasons like environment, data availability, or general issues. From there they can get that sneak peek that may help clear that picture up early.

In the end is all about the planning and timing of what can be done and by whom. Regardless of what methodology is followed planning is needed, formal or informal, and as long as that plan allows for flexibility great things can be easily achieved.

Now on to the No side. Sometimes the timing just doesn’t allow it and the demo is the only time users would see what was developed during the sprint and provide the feedback needed. Or there is the preparation for future sprints that take up the time available to look at something early. These scenarios fall within scheduling and planning. Depending on how the team or organization operates it might not be suitable for a true form of UAT to occur. Now they could still do the Static form of testing and that is usually what will occur to help improve the quality, there just won’t be any Dynamic testing.

The other reason it may not occur during a sprint could be that the changes are strictly back end or architecture in nature and there would be really nothing for the user to see or do. They may play with the front end to see nothing has changed yet that would be a quick set of activities.

Something that could occur is taking a sprint for full on UAT when everything is completed, very counter intuitive to what Agile is. That one clean cycle that shows that everything that was requested is there and working as expected. Usually it would be a Beta test in the end.

It is all about planning, comfort level and maturity of a development group that would determine in the end if or when UAT occurs.

 

Surprise

A lot of people like surprises: parties, finding that 20 dollar bill in your jacket you forgot about, or finding out some really good news. Yet there are some that don’t like getting scared at a haunted house at the amusement park or getting a flat tire. We live in a world where there is always something that will give you that quick initial reaction of joy or fear. In testing, at any level there are surprises on a daily basis, and for the most part they don’t end up being any fun.

No matter how much planning is done from the start there will always be that one or two things that could cause things to go off the rails. Risk analysis and mitigation is something everyone should be doing when going through any initiative, regardless of the methodology use. The old adage “hope for the best, plan for the worst” is something that I use at anytime I am thinking about how to go about things. Now there are limits to how much someone should plan for the worst. Like if I plan a family trip I don’t plan for earthquakes because they are very rare. Just like in a software delivery project, you don’t plan for a potential flood or power outage the probability for those to happen are low, one would hope.

So the best thing anyone can do is plan for the most probable things to happen with a few not so probable. That is the best thing one can do and the use of a risk analysis process is something that would help identify what issues could come up and how they will be mitigated or handled if they occur.

The best two stories that I have for surprises where at two different stages in my career and helped me understand the best way to handle them. One was from a previous post about taking ownership of your actions. A simple keystroke can do a lot of damage :).

The other story is when I was working on a project with a third party where information needed to me passed back and forth. During a two month execution process of sending files we were about to consider the project closed and ready for deployment. I think you can see where this is going to to. Well about a week before we were going to deploy the third party decided to tell us that the information that was being sent was incorrect and was always incorrect. This not only put the project in jeopardy it potentially put an entire release, with about 4 other projects, at risk of not going in. After some discussions and investigation it was found that a requirement was not detailed enough, yet both sides signed off on the expectations. Luckily it was a quick fix and a full day of validation and risk assessments to ensure the release was ok to go.

Now that surprise wasn’t one of the fun ones and could have been avoided. Asking the right questions, pushing back on requirements, or getting consensus on every detail.  Regardless in the end it is how someone deals with a surprise that will determine success. A person can be one of the ones that can be seen on Youtube completely freaking out over everything of it could be a person that is calm and works through the issue to get things right. It may not have to be a perfect plan on the spot, as long as something is thought through and collaborated with others within the team to get things done.

 

 

Product owner in the testing process

Product owners are great to have within a team. They spend time understanding the needs of the product from a client side and disseminate that information to the development teams. With that knowledge, should they be involved in the testing process? I would say yes. In fact I find that they have to be ingrained in the process from the start to have good visibility of what is going on.

Static testing is one of the key things they would need to be cognizant of. They need to have the assurance of the information that they are passing of is detailed enough to get the work done. Now in an Agile environment that level of detail may not be there from the start to get the exact product change they are looking for, yet they should have enough information to allow the team to see the path that they must go down.

Additional understandings of the process, such as issues found, changes to stories and subsequent impacts help the overall process. Clear paths of bidirectional communication allows for effective risk analysis and decision making to ensure the product is what is needed by the clients.

Now onto dynamic testing. Why shouldn’t the Product Owner be involved in doing some testing of the product? In the end they should have some idea of what the product looks like and how it processes the information according to client needs. Now does that mean they test each sprint or during the beginning stages of the test cycle? I don’t think so. There are potentials to find too many issues that could raise unwarranted alarms. Let the people running the tests get those out of the way and come back to the product owners for clarification or advice on what was found. In the end they should be a part of some form of UAT type of cycle prior to completion. Let them get their hands a little dirty using all the information they gathered throughout the process to see if it is what they want.

In the end it is their product and they should have an idea of what it does and how the clients will feel when they start using it after it goes into production. They may have an unusual workflow that they found out a client does and will need to see if it is run through. Whether it is the test team themselves or they try it to find potential issues. This level of understanding along with strong communications will help with up front quality improvement along with closing any gaps with testing.

 

 

Thoughts on certification

Doing a Google search on QA certification you will come up with over a million results with the first page having a fair amount of individual certification programs. To me I think that is a bit much. There are the big ones from the Quality Assurance Institute and International Software Testing Qualifications Board to smaller College certificates of private training facilities. Going through each one has the same general premise yet what is being asked for to get the certification seems to be consistent across.

Now don’t get me wrong, I appreciate it when someone takes the time and energy to learn the craft and prove they have the general knowledge of the discipline. What I have issues with is what I find most people do after the certification is achieved. From QA to PM I have seen people come into my career with different certifications yet do not follow the general premises that they learned while getting it. To me what I see is that they are willing to spend the hours to cram for the exam, pass it and collect the prize at the end thinking this is what will get them that sweet job they are looking for. Yet once they get it they do what they want and not help with any improvements for the organization.

Yes this view is a general and does not apply to all those that use what they learned in practice. In discussions with others, not only in the QA discipline, the general feel is that the “certification” is watered down. There is a term in the martial arts world called McDojo where belts are handed out almost at will as long as the person paid membership dues and testing fees. That is not to say all martial arts schools are like that, much like not all certifications are like that as well, it is the fact that there are easy paths to achieve somethings that shouldn’t be so simple.

I have interviewed over a 100 QA analysts, coordinators, and managers over the years and I do look to see if they have a certification, yet it is not a deciding factor on whether they will move forward in the process. I formulate my questions to see what type of understanding the individual has with QA and software development. I also look for the passion of what value QA has for an organization aside from executing testing. To me that is a big part of what I look for, second to experience.

To help I think the QA thought leaders within the discipline should look to post secondary establishments to enhance or start a potential diploma program. No different then electricians or construction, there is a specific skills that QA analysts need to know, understand and know how to apply it.  Most of what is being done is “Learn as you go” , which is great yet there are gaps that can happen.

I think it is time to get more focus on help the discipline show the true value we can provide.

 

QA Value

 

Exploratory testing

Time to get the hiking boots, walking sticks, pen and paper it is time to go where nobody thought of going in a testing cycle, the weird and undocumented paths that could happen in the product under question: exploratory testing.

OK so there is no hiking boots or walking sticks needed, I was adding dramatic effect, although you will need a pen and paper to document what you do. Well it is the digital age so some form of documentation software will do.

Here I will give my views and insight on this tool in the vast toolbox of a QA professional that can use and help improve overall quality of the product. Now I am sure as you are reading this a flow of questions, comments or just plain disagreement to what I have said. Perfect, that is what I am looking for. As some of you know I am a regular on SPaMCAST with Tom Cagley in a segment of the same name of this blog. I wanted another avenue for those that listen in addition to those that will just read here to foster conversation that will generally help this great discipline call Quality Assurance.

So on to the topic at hand, exploratory testing. When I hear exploratory testing I have this vision of going off the beaten path, basically trying something that was not thought of to see what happens. There is plenty of documentation out there that states the same thing. As I said it is a tool, and as with construction there isn’t one sole tool that will do everything you need to build something, so we should not be reliant on exploring different paths as the only method for QA work.

Exploratory testing, in my opinion, is putting on the hat of a user for a short period of time. We have all come across some form of ticket from users where they have done something odd like open a bunch of windows in a workflow then went back to a previous window in the flow and made changes, then the system would freeze up when trying to save because it didn’t know what to do with the transaction. This example sticks with me because it was one of my first tests I ever did as a UAT tester at the start of my career. My thought process was simple, what happens when I do this?

With my teams now I ask that there be some time set aside as: User time. Basically they go in and do whatever they want, documenting what they do of course. Just to see what happens. This is usually done at the end of a cycle, if time allows, so that planned work on the changes are not impacted. We have also had testing parties, or Bug Blitz as we called them, and we get not only the QA team involved we get all the stakeholders involved in running through the applications. Our last session we had a few executives come in and play around. We found a few things, nothing serious although the biggest intention of the outcome is to give the understanding to everyone what the QA team goes through in a short period of time.

In Explore IT! by Ellisabeth Hendrickson she uses the analogy of a net and testing when trying to find issues. The more testing that is done the tighter the weave of the net becomes. There is still a risk of issues popping up in production, as we all know a system is not fully tested it is the elimination of risk. Exploratory testing is adding a few more strings to the weave to catch what we can.

The additional benefits of Exploratory testing is improvements in requirements. Think about it, why was it decided to go off and try something on a system that was not documented? To see what happens. The end result is driving out a new way of thinking for those that are documenting change. Now with the knowledge they have from this out of the box thinking they will now have to keep in mind that when developing the WHAT  in a change they will add some additional detail so things could be avoided.

Use a tool right and the quality will be there. Use it wrong and something will get built, unfortunately it will not look like the way it was intended.

 

 

Owning up to mistakes

Watching the Super Bowl this past weekend was an emotional roller coaster leading up to one of the most emotional ending for any fan. A high if you are a Patriots fan or low if you’re not. I was in the not category, the reasoning behind that is another story all together.

For those of you that do not know what happened, the Seahawks were less then a few feet away from winning, instead of handing the ball to their running back to blast through the defense they decided to throw the ball with a lot of defensive players around. Unfortunately that pass got intercepted and the game was over. the announcers, twitter, facebook and all other social media went insane with the amount of traffic asking “what were you thinking?”.

Looking back on that it got me to thinking about  some of the missteps I did during my long career and how I handled them. Each one I owned up to, there was no real reason not to. Non of my missteps were illegal, some did cost time and money to fix but nothing that was an exorbitant amount. Probably the biggest issue I caused was deleting about 75% of clean test data in a database by accident. It was a department initiative to clean up the data and automated test scripts to improve execution efficiency. Myself and another QA analyst worked on the project for over six months. It was tedious and involved using a mainframe that was very particular on the instructions that would be coded in. During one of my data moves to the new “clean” database I did not notice that I accidentally hit the space bar at one point in keying my move instructions. Because of that the system moved all data from the “sandbox” data base that was used to sanitize the data before moving to the clean database. That move included all the garbage data and other non-functional data that need specific one off scripts to work.

Needless to say there was a very large knot in my stomach as I noticed the move was taking longer that it should and there was no way for me to stop it. Months of work looked to be thrown out the window and the project would be at square two, some of the stuff that was moved was clean in both databases. I didn’t try to hide it or fix it before anyone noticed. I went to my boss and broke the news. He was a little upset but was happy when I also told him about how I planned on fixing it. We were lucky to find out that the system did weekly back-ups, so we only lost about a weeks worth of work after it was restored.

There are plenty of stories out there where one small mistake can have huge consequences. Yesterday I heard about Toy Story 2 was accidentally deleted (http://thenextweb.com/media/2012/05/21/how-pixars-toy-story-2-was-deleted-twice-once-by-technology-and-again-for-its-own-good/). I am sure others have experienced something like that, especially in IT where a simple keystroke can do so much.

Out of all of this is I feel that owning up to the mistake and working with others on how to fix it shows a lot of leadership. Even if you may not want to lead a group or team, it shows that you are taking responsibility for your actions.  It is better to do that then let it fester, but that is another blog altogether.

Root Cause Analysis thoughts

Within an organization’s Quality Assurance (QA) program there is a need to continuously improve on processes throughout the Software Delivery Life Cycle (SDLC). One key process improvement initiative is the use of Root Cause Analysis (RCA) however  a common concern is achieving buy-in from the Business end to pay for these initiatives. It is essential that the organization recognize that metrics from this process is very valuable to the future success of an organization. Previous research supports the importance of a communicative relationship between QA and Business groups (Berriault, 2012). During QUEST 2014 discussion occurred  regarding recognition of both disciplines’ needs and consensus of high level points to encourage a communication focus shift  for far reaching improvements within any organization. The QA Value Chain is one of the key processes within the Root Cause Analysis process to assist with justification.

QA Value

RCA is a critical component within any SDLC whether it is Waterfall, Iterative or Agile delivery. There must be a form of tracking and identifying  consistent issues that will allow for the development of solutions to improve delivery quality and speed.  A good Root Cause Analysis will impact each of the support activities of Innovation, Technology Efficiency, output and efficiency. The RCA process should not be taken lightly, viewed as an afterthought or considered a vehicle that is quick and easy or a reason to place blame. It is a slow and methodical process that uses statistics to drive out solutions within a collaborative team environment.

Some groups see the end product of this process as a set of final defect metrics as follows:

 

 

 

Root Cause Defect count
Requirements 3
Code 15
Environment 4
Design 2
Testing 1

This negative outcome is one of the issues some organizations see with Root Cause Analysis like QA. This is sometimes viewed as a good metric; however it is counterintuitive to RCA. Organizations usually expand resources to deal with defects faster.

The three alternatives that come out of this type of RCA is:

  • Expand capacity – more staff
  • Work harder – overtime
  • Keep fixing – postpone for future releases

The first two are considered as the easiest to accomplish. Here is the kicker, they are not. Each will increase the overall costs and decrease the organization’s Return on Investment (ROI), while not resolving the repetitive problematic issue. This a keep fixing solution which just adds more costs and gives the organization’s customers a faulty product. The ROI is impacted through continued associated costs and tucking it away in a scheduled release is only fooling the organization that they are doing the right thing. To achieve  quality output a clear definition of needed data is required before the RCA process can be considered.

Defects, variances or bugs are terms used to identify process issues. Ann Hunngate (Quest 2014) coined the term “saves” a more positive and forward term.

A good definition of a save should be: Any situation that adversely affects the project’s ability to deliver a product within the project’s expectations. To drill down on the definition consider: Any situation that requires unplanned investigation causing delays. This means anything from a coding issue to printer jams will get a save documented against it.  Consideration of this definition could possibly increase the number of saves, creating anxiety due to elevated statistics. A high number of saves might be viewed as a heavily flawed product, which would be further from the truth. There needs to be an organization change in mindset, where saves only happen during test case execution The outcome provides valuable data on the inner workings of the SDLC. Think of it as the computer centre within a car. It is a central location that is watching all aspects of the car and providing warnings when something occurs.

As stated earlier the RCA process is more than collecting and displaying final save numbers. The actual Analysis is the critical aspect providing the organization’s decision makers with the needed metrics to make positive assessments for future improvements.

RCA requires a focus on processes and process tools. This eliminates the human aspect, where emotions could add an unwanted hurdle, and zeroes in on what is truly causing the issues.

“If you don’t ask the right questions, you don’t get the right answers. Asking questions is the ABC of diagnosis. Only the inquiring mind solves problems” – Edward Hodnett, American poet.

Utilizing a change management process is the best way to ensure stakeholders are comfortable with change. The Why, How and What are components recognised in most models. Communication is the link between the disciplines. Begin with why the change is needed. Once that is clear the how and what of the process are the next explanations to be shared. This will allow for easier utilization and acceptance (Sinek, 2009). A side effect of this route of action is the improved overall perception of the QA team; highlighting the team as more than test executioners. There are plenty of change management techniques that will provide additional suggestions to introduce change.

Appropriate use of this process prevents laying blame or placing the failure of some or all of the components on any individual or group creating conflict. Conflict decreases employee morale and creates a barrier for communication between groups and a breakdown of trust.

Quite simply, RCA has two parts:

  • Collecting Data
  • Asking question

The data collection is a little more difficult than identifying a save, such as a code, requirements or environment issue.  It begins early within the SDLC and not in just the execution phases. This is why it is important when explaining the why process to the stakeholders and the rest of the organization.  the following are some points that can be used when having these discussions.

Process focused

RCA will help determine what processes need adjusting or possible required revamping. Much like an assembly line the movement of data and work products follow a similar design where information from one area feeds into another. On such lines the flow and or quality throughout could be improved upon to increase efficiency and the bottom line for the organization. RCA will provide that data to make informed decisions on improvement initiatives (Slack, Chambers, & Johnston, 2010).

One of the key elements to find a process issue where stakeholders will take notice is to have a dollar figure attached to it.  Here is where some additional work is needed when documenting saves. Depending on how the organization is set up from a time tracking perspective the data could already be present and formulated to provide the information needed. If not there is a simple work around that can be used with any tool set that is used to calculate the financial impact. There are three components that need to be documented: Investigation, Fixing and Retest. As stated earlier this process occurs throughout the entire SDLC.

Using these terms are not solely for the use of test execution. Investigation and Fixing are universal statements for any work that is done throughout while Retest would be new in some areas. The easiest way to think about retest is confirmation the Fix is what is expected. For example, ensuring a reference in a requirement that was not there before is there.

For each component there must be a financial component. With most large organizations there would be a time tracking system where data can be pulled off, such as hourly rates.  Defect tracking tools can be customized to allow for each group working on the save to key in the amount of time spent on it. With these two sets of data you now have a cost associated with each issue.

Here is an example using a base day cost of an issue:

How many hours each person worked on the issue

E.g. Lead 1hr, Tester 2 hrs, Developer 3 hours (all rough estimates and keyed into the comment section of the save)

Then determine the number of days – 7.5 hours = 1 day

Then multiply by average “person day” cost.(for this example  use $450)

Using the example above 6 hours would be 0.8 Person day X 450.00 =360.00.

Therefore the cost associated with the Fix is $360.00. Next a process or non-employee issue must be associated with it as well. This component can become difficult due to the amount of analysis that is needed. Frequently it is much simpler to put a quick label on it and forget about it.

Here are some guidelines to determine if time is needed to spend on more detailed analysis:

  • Do I have an idea how this save could have been prevented in the first place?
  • Is the save significant? (i.e. Was there a lot of rework time spent on the defect?)
  • Could small saves turn into monsters?
    • Some saves could be seen as bricks. On its own in one project it could be nothing.
    • If it is repeated in other projects it will turn into a wall.
    • Unresolved saves would have to be monitored throughout the year.

One of the simplest RCA tools that would help determine the root of the issue is the why-why analysis (Slack, Chambers, & Johnston, 2010). At the end of this type of analysis the human component will not be present.

Here is an example:

A coding issue caused all the reports to be deleted.

1 – Why did the reports get deleted?

Developer incorrectly coded the module

2 – Why did the developer incorrectly code the module?

The Developer did not follow the coding standards

3 – Why did the developer not follow the coding standards?

The developer is new to the organization

4 – Why was the developer not trained on the new standards?

The developer was given access to the share-point where the standards reside

5 – Why didn’t anyone confirm with the developer that the standards were understood?

There was no time

As per the above example the broken process here could be charged to poor training procedure. With this information now it can be tracked to see if it will continuously occur. Also now that there is a cost associated with it and using cost benefit analysis suggests a regular similar process break can be applied and solution costs can be calculated.

Taking the above example the cost benefit analysis could be as follows:

Each occurrence costs $360.00

It occurred 8 times in the past 12 months

Total costs over the year $4320.00

Potential solution a two hour job shadow session for new developers with senior developers with a daily average per person cost of $450 for 8 hours the total solution costs is $225.00.

If the department hires 4 developers a year the total yearly costs would be approximately $900. That would be a costs savings of $3420.00 a year. Senior management would find this information purposeful due to the ROI.

RCA is a very powerful tool to help organizations improve SDLC and it can be adapted to use in all forms of development from Agile to Waterfall. These metrics will assist the QA department to provide true quality assurance.

 

Works Cited

Berriault, J. (2012, September 16). Breaking down the root cause for the knowledge gap between testing and business. Retrieved February 5, 2013, from Athabasca Library: http://dtpr.lib.athabascau.ca/action/download.php?filename=mba-12/open/BerriaultJeremy.pdf

TED (Producer). (2009). How great leaders inspire action [Motion Picture].

Slack, N., Chambers, S., & Johnston, R. (2010). Operations Management (6th edition ed.). Edinborough Gate: Prentice Hall.