Qualification of Materials & Processes (M&P): Best Practices from Space Engineering

In the development of physical products and systems, from consumer goods to satellites, the choice and implementation of Materials and Processes (“M&P”) and in particular their qualification, is a critical field - and yet too often overlooked and underestimated.?

The same applies to Commercial-off-the-shelf (COTS) items, which is a term often used in software but equally applies in physical systems.?

Here are lessons drawn from Space engineering and to a lesser extent from Medical Devices development, both areas with mature regulations and compliance practices.

Though it’s a very specific area, it beautifully illustrates the many challenges and options in the more general Verification and Validation (V&V) area. This article is primarily intended for those developing instruments and physical products but the lessons are also relevant for software and people systems (e.g. services industry).?

Interesting but no time to read right now? Just hit Like or bookmark it and come back later!

Human Generated Content - 100% written by the author, graphics by AI

Let me walk you through Qualification of Materials and Processes: what it is, its difficulties and best practices in both technical and management.

?and a particular template logic that I found extremely useful for managing it.?

What is Materials and Processes (“M&P”) Design and Engineering??

The engineering process of developing a product or system includes many choices on Materials and Processes. For example, the chemicals used in the outer paint and the finishing process of the same surface. The adhesive for putting two pieces together. A relatively simple system such as a bicycle can entail 20 choices or more. Something more complex, can imply 100+ (e.g. a telescope or a pacemaker) or 1000+ choices (e.g. a satellite or a car).?

Each material choice is defined by?

• specific function(s) in the system?
• the loads to which it is subject in the system, in particular from its environment, such as temperature or radiation or mechanical stress, which may affect its function(s) or reliability, integrity, etc.?
• the rationale for the choice (technical, commercial, etc.) incl. but not limited to the satisfaction of any requirements flown down to that part of the system, e.g. norms like REACH or RoHS (forbidden materials)
• the evidence-based qualification that supports the technical rationale (beyond its declared specifications), easily accounting for 80% of the effort in M&P engineering?

The same applies to many processes associated with the specific materials or sometimes with production and integration (assembling) the system.?

Even though the rationale may be influenced by requirements, the M&P choices are at the lowest or deepest design level, thus being part of the solution and design output, and as such, they are not the extension of the requirements flowdown. Instead, the design specification must declare these choices and provide specs for procurement and production.? And because of the sheer quantity of choices, it is impractical to cover them by individual requirements specifications and conduct a formal verification control. If you ever mused how to resolve this dilemma, keep on reading..?

What is is M&P Qualification??

Qualification makes up the bulk of M&P effort. To validate that a particular M&P choice in a design is safe, reliable and fit-for-purpose, we need a systematic approach akin to verification yet practical enough to cover all of those choices. M&P Qualification is an engineering field full of detail, deep in material science, with much content generated by testing, data and reports, specifications, and reviews.?

The Problem

Why not just assemble a prototype and put it to the test, see which materials fail, replace them and repeat until no failures? In some cases this more agile approach might be possible. In most cases, however, the cost and time to make such high level prototypes and conduct system tests is too high for a trial and error approach. Conceptually, it’s also complicated: we would need all choices to be fixed before being able to make a representative prototype. Once things are assembled, we cannot access and observe all materials. If there’s a failure, can we really pinpoint what failed and due to which effect? And while we can observe materials after testing, the processes associated are implicit, so it is difficult to identify an unfit process like that. If the paint came off, is it the choice of paint or the painting process? We easily get more questions than answers.?

With software, it is evidently easier to take an agile approach, i.e. just build, test, diagnose and debug, rebuild. But with physical systems, the materials must be checked individually, to a reasonable extent.?

The Nature of the task

Inherent to M&P verification, generally it is not possible to test a large statistical sample for the actual lifetime of the product and in all possible variations of the environment. So typically, it is necessary to use an equivalent accelerated testing with a reasonable quantity of samples and surpassing the operating conditions with some margin, adding a rationale for the equivalency of that verification. Taking the example of the adhesive, to qualify a new material for a satellite with a 10 yr lifetime, we do not test it in orbit with say 30+ samples for 10yrs. Instead, we test enough samples on Earth, in vacuum if relevant, in a wider range of temperature, vibrations, etc. and for enough cycles and duration to generate sufficient confidence that it will withstand the conditions during the lifetime.?

M&P Heritage

Must we test all M&P choices with each new product development? Not necessarily.??

Another way to minimize the qualification effort down to a reasonable task is to use the prior Art.?

M&P heritage is the data and results already gathered in previous projects, typically through documented testing, which can be re-used to justify and validate a particular choice of material or process.? It is extremely important in highly regulated environments. Also the supplier of a material or process can provide their heritage to build up the heritage at a particular level. A well managed and documented heritage, useful for future projects, can save a lot of costs and make all the difference between a new project being commercially feasible or not. A solid heritage at our level (and the suppliers’) contributes to the confidence of a prospective customer or partner and can be a deciding factor for the proposal to be chosen over a competitor’s.??

For a single system development, the overall approach is managed in a M&P Qualification Plan and/or Status document(s) or database, which can be done at component, unit or system level.?

I personally find that it is best to use a single Plan and Status at the system level for each deliverable system of a project or even a single plan for the entire project. This allows to synergize between material choices and optimize the qualification effort. For example, you can identify the opportunity to use the same adhesive in multiple components and cover those qualifications in a single initiative, thus reducing costs. Also, a single plan & status provides an overview on the qualification and KPI’s such as percent complete.?

What makes M&P Qualification challenging??

Remember Challenger, the space shuttle that tragically exploded shortly after launch? A subsequent investigation identified the technical root cause to be an o-ring (a large rubber piece in the fuselage) which was unfit for its purpose under the loads before and during launch. Most failures at the M&P level are of course much less dramatic but it is the same kind of failure. That should be enough to motivate us to take it seriously.?

There are many difficulties in M&P Qualification when planning and executing a project:?

Technical challenges

• Agreement on justified equivalence: as described above, generally M&P qualification inherently relies on a rationale for correspondence between the testing that is technically and cost feasible and the theoretically ideal test. If there is no agreed or imposed norm, this can be very difficult to agree.?
• Norms agreement: Even if given a norm, stakeholders and even colleagues internally may disagree on which norms and standards are applicable or how they translate into specific tests or verifications. There can also be disagreement about methods and results.?
• Coverage: the applicability of the heritage is often underestimated or uncertain. For example, the environment slightly changed wrt the previous product and the material no longer withstands the temperature or pressure;? the supplier or their process changes; or a unit test showed a deviation which is not acceptable anymore.
• Acceptance: the credibility of the heritage is often underestimated or uncertain. For example, if the customer had a bad experience with that particular choice they are bound to require extra reassurances and testing; or an M&P officer at a stakeholder (regulators, customers, etc. ) can reject the heritage either by being overzealous or simply less than competent to understand the data and its implications.?
• Time evolution: things change. For example, regulations evolve and the Heritage that was valid a few years ago may not be valid anymore; the supplier for a critical material changes; a process is moved to a different lab or factory or simply newer equipment or different operators and that can lead to dramatic changes in the quality of the outcome.?

Project challenges

• Cost and Risk responsibility: even though everyone wants a reliable system with no failures, there is a clear conflict of interest between a customer and the supplier on the point of verification, which is exacerbated in M&P testing. The customer interest is to minimize the risk of failure which is generally achieved by more extensive and generous testing, implying more costs. On the other hand the supplier's interest is to minimize and keep the testing costs within the budget, thus striving for essential testing as a default scope and to externalize the residual risk and associated responsibility.?
• Time and budget planning: the effort and the difficulty associated with this process is often underestimated or uncertain . So when the team finally realizes how much work is required, they are faced with too little budget and time for the task and many discussions and disagreement - internal and external vis e.g. the customer - on how much qualification is needed.?
• Time and budget pressure: besides the possible tension in customer-supplier relations throughout the supply chain, at every level there is the pressure to reach definite and positive results quickly and within budget. This can lead to tensions between managers, QA and engineers and even encourage overlooking deficiencies in results in order to “move forward” to freeze the design and test at the higher unit and system levels. This implies risks. It’s almost an art , to establish the good balance here.?
• Roles: it often is not clear who or which departments are responsible for which tasks in the M&P process. The challenges listed here mean that it is an activity often with high effort, high risk and low reward or recognition for those engaged in it. As such, naturally many engineers and departments avoid getting involved in it as much as possible..?
• Impact of technical challenges: these often lead to delays, scope creep and re-appraisal of the task effort with clear impact to the project budget and timeline.?

Good Practices in M&P?

The text above already points in evident directions for best practices. Here is a list capturing all those points and expanding on some more.?

Technical Best Practices?

Design or Solution specification: keep a section dedicated to Materials and Processes (“M&P”), with a comprehensive list of all the choices and explicit information, e.g.:?

• Specification, function(s) in the system, the loads to which it is subject
• The rationale (technical, commercial, etc.) incl. a mention to the qualification and a pointer to its detailed status

M&P Heritage: Manage it systematically and periodically, this is a key to your competitive edge. Each material and process needs to be identified very specifically and will be associated with substantial information, e.g.?

• Supplier, specs, procedure, functions and effects for which it was qualified, norms used at the time
• Quality and availability of the data (procedures and results, is it well documented, can it be shared with other partners?)

Systematic Qualification Plan and Status: keep an updated database on all the M&P choices for a product or system, using parameters derived from the above two points, i.e. go to detail on the functions and the effects for each choice.?

• Ideally, it would be managed in the same tool as the heritage , so that any changes are updated in both the product and heritage, as relevant.?
• Producing a plan and status document is then easy as an extract out of the database.?
• (in a future article I’ll share a particular template logic that I found extremely useful for such database and documents)

Specified technical gap: as part of the above status, be explicit on what choices are still open as well as qualifications, including for the choices already taken.?

• The more explicit the challenge is, the better you can manage it and plan for it. Additionally, you reduce the risk of surprises later on, from “unknown unknowns”.?

Critical M&P choices: Identify these and prioritize them in the planning and execution. They can come from e.g. Risk and FMECA registers or Design rationales.??

• A good starting point in case of limited time and resources for qualification, in particular for difficult choices between risk and further testing.

Qualification by experts: assign M&P tasks to those with experience in the particular technical field related to the primary function of the material, from the engineering and/or the verification teams.?

• E.g. for an adhesive or metal (function: mechanical) subject to radiation and thermal loads (effects), it works best for a mechanical engineer to plan and report the qualification, supported by radiation and thermal engineers to devise (and conduct) the tests.?
• If the testing is outsourced, still the above team should be involved to plan and review.

Management Best Practices?

M&P Visibility: give this field an explicit place in the product development, with well defined responsibilities and budgets.?

• Celebrate the achievements there and reward those working and delivering on it.?
• Avoid and discourage that M&P work be seen as secondary or obscure.

M&P effort estimation: evaluate responsibly and honestly the effort both to select and qualify the materials and process choices including suppliers.?

• Include a breakdown of the tasks with specific materials or engineering disciplines in the plan or backlog, not just a generic “Plan - Select - Qualify - Document M&P” set.?
• Take into account the gaps from a technical perspective (see above) as well as other factors: are the other stakeholders and reviewers easy or difficult about this topic? Is the team ready for the challenge? How much uncertainty -? and task buffer - is there??

Realistic proposal: both for customer and internal projects, incorporate the M&P estimate in the proposal.?

• Take a parametric estimate from actual costs in similar past projects if available and compare with your estimate. If there is a gap, do you have a solid rationale for it?
• Is there any way you could be grossly underestimating the effort? Could you be exposing your organization to a liability?

Regular status and plan updates: like any other project activity with potential critical surprises, worth tracking closely.

• Quantify e.g. # total of materials and processes decisions, % frozen choices, % submitted qualification, % accepted and closed. As possible, have an effort estimation per M&P design choice and respective qualification.
• Use the comprehensive Qualification Plan and Status database, makes it easy to extract KPIs for completion, identify the major challenges and steps ahead, on a regular basis.?
• If overwhelmed by the M&P choices, the critical label will help to filter and prioritize.?

M&P Qualification Reviews (QR): include them in the project plan to support the Design Reviews, making the milestone precedence explicit and allowing to identify their impact to the project critical path.?

• It pays to prepare: a failed QR can kill the momentum of a project or open an endless discussion on risk.
• Be ready and prepare a way forward in case the qualification is not yet accepted in the QR.

Roles, Process, Tools and Templates for qualification: define them early in the Project Plan.?

• You can always adapt and update as you go forward but you need to start with a realistic setup which is agreed by the team and eventually other stakeholders.

Training & Learning curve: Educate all your team members on M&P engineering and qualification, including non-technical ones.?

• Everyone will be involved in it to some extent and should take into account the impact of their decisions on the M&P effort.

Stakeholder/ Customer alignment: agree early in the process in as much detail as reasonable.

• The risk and development philosophy (e.g. what is reasonable to accept as rationales for qualification equivalence, test approaches, acceptable risks)?
• The sharing of the effort and the responsibility for the risk between the stakeholders. M&P qualification can quickly get costly and lead to a push back-and-forth, degrading the relationships between partners.?

Lessons Learned: gather the experiences during and at the end of a project, what worked and not worked

• Especially if there were any unplanned challenges, how they were solved and the time and effort it actually took.??
• Think of how realistic the plan turned out as well as on the Roles, Process, Tools and Templates.?

Further reading:

https://www.dhirubhai.net/advice/0/how-do-you-manage-cost-time-materials-testing

https://en.wikipedia.org/wiki/Process_qualification

https://ecss.nl/standard/ecss-q-st-70c-rev-2-materials-mechanical-parts-and-processes-15-october-2019/

https://www.dhirubhai.net/pulse/uncertainty-risk-accuracy-business-philosophy-insight-jose-barros-yutvf

For a different but equally useful view on the topic, here’s a prompt suggestion for OpenAI GPT?

“how would you describe materials and processes qualification, in the context of space engineering?”

My article is Human Generated Content - 100% written by the author, graphics by AI