Blog Archives
How Does Your Company Deal with a Failure?
Having spent forty years in R&D and production I’ve experienced my share of unexpected and undesired outcomes of projects, products, processes, and tasks. I’ve also been in a number of different environments where these undesired outcomes were treated very differently. I even had one boss early in my career chide me for being complacent about a successful test of a new design. He got on my case about design margins on a couple of the critical components and told me to “go break it.” Turns out he was right. Subsequent testing proved that there was not enough margin in the design. In contrast, and counter to that attitude, I had one corporate VP at another company admonish my team for a rocket test firing in which we ejected the nozzle before completion of the firing. He couldn’t accept that this was a test to examine the margins on a new design. We were attempting to determine how much material we could shave off to reduce weight, and we were operating in a regime beyond the resolution of our computer models. With customer concurrence, we had decided to conduct a test. The customer fully understood that the result was not a failure. “This company will not accept failures!” this VP proclaimed proudly at a monthly program review meeting. He was only concerned about his perception of the reputation of the company even though the customer had signed off on the test. As I indicated, our customer was perfectly satisfied with the result. The VP wasn’t. Or maybe his ego wasn’t. This was R&D and this was a test designed to push the margin. If this had been a production motor I would have been on his side about declaring it a failure, but this was R&D. Of course, this was the same VP who said he didn’t believe in his corporation investing in R&D. Instead, he believed if the company needed a technology he could just buy it. Let other companies deal with R&D failures. Well, that’s a subject for another blog.
When you get into the production arena, attitudes toward failures change. Production failures can have significant long term effects on the bottom line, customer relationships, and company reputation, depending on the industry and circumstances. Acceptance tests and inspections are conducted to ensure the quality of the unit going out the door to a customer. How does your company treat an acceptance test failure? Or an out of tolerance dimension?
I’ve worked at companies that operate on the opposite ends of the spectrum when it comes to quality and treatment of failed units. I worked for an automobile parts manufacturer delivering a $.25 spray nozzle assembled from two pressed-fit injection-molded plastic parts. The units were 100% tested. Nozzles exhibiting spray considered out of tolerance were discarded. Statistics were kept of the number of failures. No paperwork was generated for a failure. The part was just discarded. The company understood the variability in the manufacturing and assembly processes and had calculated what percentage it needed to make its required margins. Only when the rejection percentage begin to creep up (they counted the discarded units), endangering their margins would they begin an investigation. They also understood that the design was simple enough that it was highly unlikely there would be any long term latent defects hidden in the nozzle.
Contrast this to the rocket launch industry. I’ve worked for both a components supplier and for a launch integrator. The industry’s slogan concerning failures can be summed up as “one failure is a trend.” Failure during acceptance testing was seen as an indication of a potential latent failure in units, even those that passed, that could have an impact during a launch or satellite/payload operation. When you consider that the value of the payload, rocket and launch cost is typically in the hundreds of millions of dollars or more, you understand this philosophy. You only get one chance with a launch and there are no repair facilities in orbit or deep space. So the launch and satellite industry has, in general, accepted this manifesto of “one failure is a trend.” Other industries, such as healthcare, either have or are adopting similar attitudes toward failure because of the potential cost and damage impact of a systemic failure. On the other hand, I also recognize new entrepreneur-led space launch companies like Space-X are trying lower the costs of launch; how that will change the launch industry remains to be seen..
So what are the implications of the “one failure is a trend rule?” Essentially a root cause investigation must be conducted for each failure. There are many methods of performing root cause analysis (RCA) including the Five Whys, Fault Trees, “Fishbone”, and Kepner-Tregoe. It really doesn’t matter which process you use, as long as you work through the layers of design, manufacturing, human influence, etc., like peeling back an onion. The one thing you don’t want to do is take shortcuts, or jump to conclusions. I’ve been on too many of these exercises where part-way through everyone “knew the answer,” only to find, once every box was checked. that something else, often a seemingly innocuous something that no one suspected, was the cause. This is why root-cause analysis is not cheap, because it has to be comprehensive and complete.
Am I advocating using RCA in every case of a production or process failure? No, of course not. The method used on that automobile component worked for them. For the rocket launch industry, they’ve decided it’s a case of “pay something now, or pay much more later.” A company has to weigh the costs vs. the consequences and then decide how it will treat failures. If you determine an RCA is necessary, whatever method you choose, finger pointing should not be part of the process. It doesn’t mean that the consequences of a failure shouldn’t be addressed if it involves personnel. It means that everyone participating in the process understands that this is being undertaken to find and correct the cause of a failure, not to blame someone. The culture of a company will determine how this plays out.
If you have a failure and decide RCA is required, and that you need some help, Rocket Science Technologies can provide assistance. We offer a free hour of consultation with initial inquiries that may help you decide whether you need to proceed with RCA and what method suits you the best. There are no one size fits all answers but there are no shortcuts either. Rocket science involves the science of getting the details right and that is our goal in helping you.
Secret of Rocket Science: Getting the Details Right
“This is not rocket science…” How many times have you heard that expression? In general, that statement is used to indicate that whatever you’re doing is not overly complex. It’s a tribute to the perceived complexity of rocket science. But just what is rocket science? Is it some arcane form of engineering that doesn’t relate to the things done in the commercial world? Or is there more to it? And, more importantly, can rocket science be relevant in today’s fast-paced market?
Dictionary.com’s first definition of rocket science is “rocketry” (English teachers used to scream at me for using different forms of the word in the definition but dictionaries seem to get away with it). Rocketry, in turn, is defined as “the science of rocket design, development, and flight.” The website’s second definition of rocket science is “something requiring great intelligence, especially mathematical ability.” So, on the surface, it appears rocket science is just that, the science of building and launching rockets with a nod toward things being complicated. Neither of those definitions satisfies me. Based on my experience in the industry, I believe they are incomplete. Only when you get into the nitty-gritty of a space launch does the essence of rocket science become clear. Rocket science is all about getting the details right.
With a space launch there are no second chances. There are no do-overs. If the launch fails, that’s it. A billion dollars may end up in the ocean, or scattered in pieces around the launch pad, or in a useless orbit around the Earth. No second chances. Once in space you can’t bring your malfunctioning satellite or probe into a local garage for repairs. You build in redundancies when you can and work to reduce the risk as much as you can.
The launch vehicle and its payload combined have hundreds of thousands of parts, subassemblies, and assemblies that must all work and function together for success to occur. A system with a million parts and 99.99% reliability can still exhibit a hundred malfunctions, of which any one may lead to a catastrophic failure. So the emphasis in the commercial space launch world is on the details. You have to get them all right. So when it comes down to it, rocket science is really the science of managing millions of details, while also working to bring operational risk down.
How does that relate to you? Your project of replacing a machine on your assembly line, or developing a new drug, or testing your electronics package certainly doesn’t involve millions of details. True, but it still may entail hundreds, or even thousands, of interrelated tasks, components, and tests or inspections when you add up everything that has to be done. These are the details you must account for in what you do. Furthermore, there may be one detail that is ignored because your team members believe someone else must be paying attention to it. You find this out after it rises up and bites you in the butt. Or there may be a detail you just didn’t think of. There’s a reason why project management is one of the core competencies in rocket science. But it’s far from the only one.
Every component, subassembly, and assembly used in a rocket launch is either analyzed or tested to determine its suitability for use during the launch or on the payload. For many projects and products this idea of analyzing and testing everything may seem like overkill and too expensive and time consuming. In the commercial world it probably is. Until you have a problem. Let me give you a real life past example. An automobile OEM supplier couldn’t seem to get its electronics to pass its shock or vibration testing. Since the electronics were packaged the way it was done previously they were confident in the design. However, to be on the safe side, because this new version was slightly different in size and shape, they decided to run some tests. They used the same fixture they always used that never had a problem. They made what seemed like a very slight modification to that fixture to accommodate new mounting holes. Yet the parts failed.
In discussions with them, the question arose whether it was the actual electronic hardware that failed or whether it was something in the test set-up causing the failure. They didn’t have the capability to run the analysis to know whether the fixture, the environment, or the part design was the cause. My company had the capability. Our engineer on the project had done this sort of analysis countless times. We proved the fixture was resonating, adding higher loads than the electronic components would see in real life. We helped them redesign the fixture and their parts passed.
Rocket Science Technologies, Inc. has the knowledge and experience to help you in situations like this. We have the experience to guide you through this kind of failure recovery in an efficient manner to find a solution. We can also help you plan your next new product design and development to help avoid these kinds of issues. We can help you corral those details so the risk of failure or issues is significantly decreased. We can’t guarantee success, but we can improve the chances dramatically. And, in the case of something going wrong, we can help you get back on track and recover.
Rocket Science Technologies, Inc., has gathered engineers, physicists, mathematicians, and project managers as associates, available on an as-needed basis, to add their know-how to help you get past even the most challenging technical obstacles. All of RST’s personnel have shown through their careers a propensity for taking on tough problems and solving them. We relish solving technical challenges. We also understand the needs of the commercial market for reduced costs and higher velocity to production and market. Check us out at http://rocketscitech.com.
Project Management as a Cornerstone of Success for Small Business
Introduction
In my years as IPT (integrated Product Team) lead and program manager, I don’t know how many times I’ve heard from a team member: “You worry about the budget and schedule and I’ll worry about the technical. That’s my job as an…” Fill in the blank: engineer, software developer, analyst, designer, technician, assembler, etc. This attitude is a recipe for failure, a formula for schedule and cost overruns. It’s not that these are bad employees. Most of them were top-notch people. They just didn’t understand the impact their actions could have on the project and ultimately on the company’s bottom line. As engineers, they were trained to strive to achieve technical perfection. As software developers they have an innate drive to make their software better and to include more functionality. However, in the real world of business, you live under constraints and expectations that impose limitations. Resources are not unlimited. Risks are present. Customers have a delivery time expectation. Project management provides the framework for performing under the constraints imposed on the project or task. More importantly, project management serves as a discipline that relates to almost all aspects of company activities. The disciplines learned in project management can help increase productivity and the bottom line because they provide a sound methodology for examining the impact of making a change or on how you approach a problem.
Project Space
As human beings we live in a physical world framed by four dimensions: length, height, width, and time. Everything we do physically can be described by those four dimensions. For instance, if you plan to meet someone at the mall, you have to state a location, which represents a place in the three physical dimensions, and a time. Thus, four dimensions. Within those four dimensions there are constraints from the physical world that limit things we can do. An example would be that gravity here on Earth is always present and makes thing fall, limiting our ability in height direction. Similarly in the world of software, upload and download throughputs are limited by the fiber optic pipes of the ISP. You get the point. We live in a physical realm whose physical laws place constraints on what is possible. We learn to navigate within these constraints.
A project is built upon a structure based on its scope, i.e., what you’re going to do; schedule, i.e., when you’re going to do it; and budget, i.e., how much it’s going to cost. You can see that there’s an inherent time element involved. For example, the budget must be spread out over time. I call the combination of these three elements of scope, schedule, and budget project space, analogous to space-time universe we live in (the three dimensions plus time) we live in. This is useful in understanding that they form a framework for projects in a manner similar to the way the three spatial dimensions plus time form the framework of the universe we live in. We must learn to navigate project space just as we lewn to navigate the physical world.
In industry, these three elements have been commonly known as the triple constraint, implying that every project event or detail must be evaluated in light of these three constraints. The Project Management Institute added, through its Guide to the Project Management Body of Knowledge (PMBOK®), \ three more constraints: quality, risk, and resources. The PMBOK® refers to these as the six constraints. I’ve given them the fun nickname, the sexi constraints, based on the Latin prefix, sex, for six (like sextuplets).
Some people in industryargue there are two other constraints: requirements and customer satisfaction. I’ll discuss that topic of how many constraints there are in a couple of other white papers. For now let’s stick to the six and understand their implications. Even if there are seven or eight it doesn’t change how you consider and respond to these constraints (except I lose the snazzy sexi nickname).
The point is, these constraints interface and effect each other, whenever you do something or make a change on a project. In performing a task on a project, you’re operating within these six constraints. You were assigned the task and it was approved by your manager (resources). Your labor must be paid for (budget). You’re expected to do it within a specified time (schedule). Your work may be reviewed (quality). The difficulty of the task must be taken into account to determine the best way to navigate any problems you may face (risk). Note that they all interact. For example, the review/quality element has a cost (labor of the reviewer(s), resource considerations (their availability and if it involves an inspection, the availability of the inspection equipment/facility), and risk (what if changes are required).
In another example to show the constrain interactions, let’s suppose you’re forced to reduce a project budget by 10%. What do you have to change to meet the goal of a 10% budget cut? Do you decrease scope by taking out some tasks? What does that do to quality and risk? Do you need to modify requirements to reflect the change, which may mean changing the project charter for an internal endeavor and the statement of work for an external contract? Or to reduce costs do you reduce quality, maybe use sampling instead of 100% inspections? Does that increase risk, such as missing a bad component? Is that another requirement modification? All of these constraints are tied together and must be considered when you make a change, or, in fact, when you perform the original project planning. They must be evaluated and balanced to reach an optimum combination that allows you to reach your goal. So, as you can see from the sexi constraints, there is more to project management than just Gantt charts and budgets. Learning to live and navigate in project space provides your employees with a new outlook on conducting business that extends to all aspects of your company operations.
Implications to the Bottom Line and to the Company Culture
Project management is a discipline that enables a project leader and his/her team members to navigate through project space and have a decent chance of arriving on time and within budget. If something untoward happens, the team is prepared to respond and recover. Furthermore, management is in a position to understand the project’s (and the team’s) progress and is less likely to be surprised if something unfortunate does occur.
What do I mean by project management being a discipline? Basically, it represents a way of thinking about and approaching tasks and problems. Even routine tasks that are not part of a project. It comes with a new awareness of accounting for all the factors impacting a task. If your company’s personnel are trained at least in the basics of project management, and now consider the six constraints when they perform a task, might they end up with a more efficient way of accomplishing that task with less negative impact on other aspects of your company? For example, if I require five signatures on this new form, what will that do to the schedule of accomplishing this task? If I reduce the number to four what are the risks? Does that impact the quality because someone isn’t in the loop? Or, in requiring five signatures, am I creating a roadblock (schedule and cost) because of the potential delays to obtain approvals? It’s a different way of looking at things. However, care must be taken that this discipline is not applied by rote, where conforming to the process is more important than the results.
Project management involves a methodology that instills a systematic way of thinking about tasks and the implications surrounding the actions taken to conduct and support those tasks. Your employees learn to appreciate the consequences of their actions. Having your people imbued with this philosophy expands the possibilities for more critical reasoning and the resulting improved efficiency, even on things that are not part of a project.
The Rocket Lair 