November 05, 2017

I was once pitching an idea to a high level engineering manager at a fortune 50 company. He was an engineer and the head of the product validation group. As part of the strategy, I wanted him to own the expansion of testing to include over-stressing the products until failure. It seemed to me to be a minor point since the majority of money was already spent, and the information gained would be huge compared to the additional cost. Before I even spoke of the value of such testing, his reaction was "why would I want to purposely fail the products?" He only saw the cost of testing and failure, and any response of mine would now put him in a defensive posture rather than an accepting and accommodating posture. His ego was now an obstacle to his understanding. 

After the audible clunk of my jaw hitting table, I realized I had assumed his title (Engineering Manager) and area of responsibility (Product Validation) meant he would understand the basic principle of learning from failure. He did not. He wanted all of his products to pass the tests and many of his tests were biased to accommodate the desire. Intuitively I knew it, but I had not openly acknowledged it, and taken preparatory actions to counter it. I had blundered and not prepared this leader with the ability to take the high ground based on a better understanding. 

The lessons. Important meetings with other decision makers must always have a written communication plan that gathers information about their work constraints, and reward system before setting the meeting. Always start the meeting with information, trends and data that sets the stage rather than going directly to the request. Don't assume knowledge or understanding based on a position or title.

I knew this but thought I could short cut this based on my own reputation and influence. It would be months before I would get this opportunity again.

August 07, 2017

ABS problems. Looking through warranty data it became apparent there were a lot of problems occurring with ABS braking systems. I opened a project to look deeper into the problems. The individual warranty reports were difficult to make sense of and the problems were attributed to various components. When I asked dealers for replaced parts they would say they would send them but never did. So I decided to not pay warranty until I received the offending part. That started an avalanche of parts being shipped directly to my desk. With all of those parts there in front of me the problems became obvious. This allowed me to move quickly through the analysis phase of the project and into the improvement phase.

The major driver of this project was that the dealers didn't get their warranty money and that affected them personally, and the engineer was driven to quickly analyze the parts for failure, because his personal space was invaded and consumed by greasy brake parts needing to be processed. This created a personal crisis for the dealer and the engineer to fix the customer's problem and make it go away permanently.

Painful? Yes!

Effective? Yes!

June 05, 2017

We had circuit boards moving less than 100 yards between two buildings. The boards would pass at the originating building, but there was a 40% fail rate of boards at the second building. Great effort and expense went into ensuring all test equipment and procedures were the same and everything was properly calibrated. This problem existed for 6 months with no resolution. Finally a quality engineer hid out of sight where he could watch the tests in the originating building. He observed the test operator test the boards. When the board failed he would place his thumb on a certain chip on the board and hold it there while he ran the test again. It would then pass and he would print the passed report and ready the board to be shipped. The quality engineer watched the first and second shift (16 hrs) do exactly the same thing to get the boards to pass, and it was obvious there was coordination to get boards to pass the test on all shifts.

The test cell operators were ordered by the facility director to perform this "thumb test" and keep quiet in order to keep production numbers up. This was written up in a counter email that was criticizing the quality engineer who found the problem and stated "they would continue to pass boards with the "thumb test",  because they could get the board to pass the test and it was therefore legitimate. The director was promoted out of that position within 3 months of this discovery.

The driving force that lead to resolution of the problem was the knowledge that the measurement systems in both facilities were exactly equivalent and that there must be another factor at play. The practice of go see was deployed and the answer discovered. The culprit was not evil intent by a single individual but the reward system that was in place to drive his behavior. The lesson is be careful what you measure and what you reward, because it will drive behavior.

April 03, 2017

A hydraulic system on a large machine included a hydraulic pump attached to a PTO, and various hydraulic components located on the frame of the machine. The design team needed more room on the frame, so they decided to replace hydraulic lines with a ported valve body, and thread all of the various shutoff and one way valves into the machined ports on the valve body. This would free up needed room on the frame for their other design work. They then bolted the valve body directly to the hydraulic pump. Essentially eliminating all of the hydraulic lines between the pump and its flow and pressure controlling valves.

When a quality engineer helped facilitate an FMEA on the system changes, it discovered the lines which were eliminated acted as accumulators which dissipate pressure spikes. Without an accumulator, the pump would experience large pressure spikes when the pump would turn on or off, and when certain valves would open or close. These pressure spikes could deteriorate pump performance and life. Knowing this the engineers could account for the pressure spikes in their design and in their test plans.

The driving force behind this failure avoidance was following a disciplined approach of analyzing change, and understanding how to think of the system and not just the components that make up that system.