ISO 9001:2015 Clause 9.1 Monitoring, measurement, analysis and evaluation

9.1.1 General

The organization shall determine:
a) what needs to be monitored and measured;
b) the methods for monitoring, measurement, analysis and evaluation needed to ensure valid results;
c) when the monitoring and measuring shall be performed;
d) when the results from monitoring and measurement shall be analysed and evaluated.
The organization shall evaluate the performance and the effectiveness of the quality management system.
The organization shall retain appropriate documented information as evidence of the results.

1) The organization shall determine what needs to be monitored and measured.

The organization must evaluate of the results of monitoring and measurement, not just their analysis and consider what, how and when to measure and that the outcomes from decisions result are ensuring appropriate process control. The organization must monitor the performance and effectiveness of the its Quality management system. Monitoring and measuring QMS operations and activities will establish a mechanism to ensure that organization is meeting its policies, objectives and targets. In order to meet this requirement, organization must perform six steps:

1. Identify the activities that can have a significant impacts and risks;
2. Determine key characteristics of the activity to be monitored;
3. Select the best way to measure the key characteristics;
4. Record data on performance, controls and conformance with objectives and targets;
5. Determine the frequency with which to measure the key characteristics;
6. Establish management review and reporting.

In the context of quality management, organizations are responsible for determining what aspects of their processes, products, or services need to be monitored and measured to ensure they meet established quality objectives and requirements. Organizations must identify key processes, products, or services that are critical to their quality objectives and customer satisfaction. This involves considering factors like customer requirements, regulatory standards, internal policies, and business goals. Once the critical areas are identified, organizations need to define specific performance metrics or indicators. These metrics should be measurable and quantifiable, enabling organizations to assess whether they are meeting their objectives and requirements. Determine how you will collect data and measure performance against the established metrics. This may involve various methods, including inspections, testing, surveys, data analysis, and more. Decide how often you will conduct monitoring and measurement activities. Some metrics may require continuous monitoring, while others may be assessed periodically. Collect data according to the chosen methods and analyze it to assess performance. This data-driven approach allows organizations to make informed decisions and take corrective actions when necessary. It’s crucial to document the entire monitoring and measurement process, including what is monitored, how it is monitored, the results obtained, and any actions taken in response to those results. Documentation is often required to demonstrate compliance with quality management standards. Use the data collected from monitoring and measurement activities to drive continuous improvement efforts within the organization. If performance falls short of objectives, take corrective and preventive actions to address any issues. Determining what needs to be monitored and measured is a fundamental step in ensuring the effectiveness of a quality management system. It helps organizations maintain consistency, meet customer expectations, and continuously improve their processes and products/services.

2) The organization shall determine the methods for monitoring, measurement, analysis and evaluation needed to ensure valid results.

The organization must develope a process (method, techniques, format, etc.) to identify, collect and analyze various data and information from both internal and external sources, including:

1. Monitoring and measuring results;
2. Process performance results;
3. Meeting objectives;
4. Internal audit findings;
5. Customer surveys and feedback;
6. 2nd or 3rd party audit results;
7. Competitor and bench-marking information;
8. Product test results;
9. Supplier performance information.

Determining the methods for monitoring, measurement, analysis, and evaluation to ensure valid results in an organization involves a systematic approach. Here’s a step-by-step process:

1. Identify What Needs to Be Measured: Begin by identifying the specific aspects of your organization’s operations that require monitoring, measurement, analysis, or evaluation. This can include product quality, process performance, customer satisfaction, regulatory compliance, and more.
2. Understand Requirements: Understand the requirements of relevant standards, regulations, and customer expectations that apply to your organization. These requirements often dictate what needs to be measured and how.
3. Establish Objectives: Define clear objectives for what you want to achieve through monitoring and measurement. For example, if you’re measuring customer satisfaction, your objective might be to improve it by a certain percentage over a defined period.
4. Select Appropriate Metrics and Indicators: Choose specific metrics, key performance indicators (KPIs), or parameters that align with your objectives. These should be relevant, measurable, and aligned with your organization’s goals.
5. Consult Experts and Stakeholders: Seek input from subject matter experts within your organization and engage with relevant stakeholders. They can provide valuable insights into which methods and metrics are most appropriate.
6. Research Best Practices: Research industry best practices and benchmark against similar organizations. This can help you identify commonly used methods and metrics that have proven effective.
7. Consider Data Availability and Resources: Evaluate the availability of data and resources required for the selected methods. Ensure that you have access to the necessary tools, equipment, and expertise.
8. Document Methods and Procedures: Clearly document the methods, procedures, and protocols for monitoring, measurement, analysis, and evaluation. This documentation should include step-by-step instructions, data collection forms, and guidelines for data interpretation.
9. Calibration and Validation: If measurement instruments or equipment are involved, establish a calibration and validation process to ensure the accuracy and reliability of measurements. Regularly calibrate instruments to maintain their precision.
10. Training and Competency: Provide training to personnel responsible for carrying out monitoring and measurement activities. Ensure that they are competent in using the selected methods and equipment.
11. Pilot Testing: Before full-scale implementation, conduct pilot tests to validate the chosen methods. This helps identify any issues or adjustments needed before widespread deployment.
12. Data Management: Establish a data management system to handle the collected data, including data storage, security, and integrity.
13. Continuous Review and Improvement: Periodically review the effectiveness of the chosen methods. If they are not producing valid results or need improvement, be prepared to make adjustments and improvements.
14. Feedback and Learning: Encourage feedback from employees involved in the process and be open to learning from both successes and failures. Continuous improvement is a key principle of quality management.
15. Compliance and Auditing: Ensure that your monitoring and measurement methods align with the requirements of relevant standards and regulations. Regularly audit your processes to confirm compliance.

By following this systematic approach, organizations can determine and implement effective methods for monitoring, measurement, analysis, and evaluation that will yield valid and reliable results, helping them make informed decisions and drive improvement initiatives.

3) The organization shall determine when the monitoring and measuring shall be performed

Determining when monitoring and measuring activities shall be performed in an organization is crucial to ensure that data is collected at the right time and frequency to support decision-making and quality management. Here are steps to help organizations determine when monitoring and measuring should take place:

1. Identify Critical Points and Processes: Begin by identifying the critical points and processes within your organization. These are the areas where monitoring and measurement are essential to ensure quality, compliance, and effectiveness.
2. Define Objectives and Requirements: Clearly define the objectives of your monitoring and measurement activities. Consider both internal objectives (e.g., process improvement) and external requirements (e.g., customer expectations or regulatory standards).
3. Consult Relevant Standards and Regulations: Review applicable industry standards, regulations, and customer agreements to identify specific requirements related to monitoring and measurement frequencies. These standards often provide guidance on when and how often certain activities should occur.
4. Risk Assessment: Conduct a risk assessment to determine which aspects of your operations carry higher risks if not monitored or measured frequently. High-risk areas may require more frequent monitoring.
5. Process Cycle Times: Consider the cycle times of your processes. Some processes may naturally dictate the frequency of monitoring and measurement. For example, in a manufacturing setting, you may measure product quality at the end of each production run.
6. Customer Feedback and Expectations: Review customer feedback and expectations. Customers may have specific preferences for when they expect you to measure and monitor certain aspects of your products or services.
7. Statistical Process Control (SPC): Implement Statistical Process Control (SPC) techniques where applicable. SPC involves continuous monitoring and measurement of processes in real-time, allowing organizations to identify variations and take corrective actions promptly.
8. Change Management: Assess the impact of any changes within your organization. When processes, materials, equipment, or personnel change, it may affect the timing of monitoring and measurement activities. Adjust your schedules accordingly.
9. Data for Decision-Making: Determine when data is needed for decision-making. If you need data to make daily, weekly, or monthly decisions, set monitoring and measurement frequencies accordingly.
10. Resource Availability: Consider the availability of resources, including personnel and equipment. Ensure that you have the necessary resources to conduct monitoring and measurement activities at the chosen frequencies.
11. Documentation and Scheduling: Document the monitoring and measurement frequencies in your quality management system documentation. Create schedules or checklists to ensure that activities are carried out as planned.
12. Continuous Review and Adjustment: Periodically review the effectiveness of your monitoring and measurement frequencies. If you find that certain activities are not providing timely or relevant data, be prepared to adjust the schedule accordingly.
13. External Factors: Take into account external factors that may influence when monitoring and measurement should occur, such as seasonality, market demand, or supplier lead times.

By following these steps and considering a combination of regulatory requirements, risk assessments, customer expectations, and process characteristics, organizations can establish a well-defined schedule for monitoring and measuring activities that aligns with their quality management objectives and helps ensure the timely availability of relevant data.

4) The organization shall determine when the results from monitoring and measurement shall be analysed and evaluated.

This ‘input’ (information and data) should reflect upon the adequacy, suitability and effectiveness of the quality management system and its processes. The ‘output’ (result of the analysis) must provide information (understanding, insight, awareness, confidence, knowledge of, etc.). The analysis output must provide insight to:

1. Customer satisfaction and perception;
2. Product conformance;
3. Process performance;
4. Product and process characteristics;
5. Trends in products and processes;
6. Opportunities for preventive action;
7. Suppliers and subcontractors.

Other potential or useful options might include:

1. Need for corrective action;
2. Opportunity for improvement;
3. Competition.

Determining when the results from monitoring and measurement should be analyzed and evaluated is a critical aspect of effective quality management. Here’s a systematic approach to help organizations make this determination:

1. Define Objectives and Requirements: Begin by clearly defining the objectives of your monitoring and measurement activities. Consider both internal objectives (e.g., process improvement) and external requirements (e.g., regulatory standards, customer expectations) that dictate when analysis and evaluation are necessary.
2. Identify Key Performance Indicators (KPIs): Identify the specific key performance indicators (KPIs) or metrics that are essential for achieving your objectives. These are the data points that should be regularly analyzed and evaluated.
3. Consult Relevant Standards and Regulations: Review applicable industry standards, regulations, and customer agreements to identify specific requirements related to the timing of result analysis and evaluation. These standards often provide guidance on when and how often analysis should occur.
4. Determine the Frequency: Consider the frequency at which the data changes or becomes relevant for decision-making. For example:
   • For real-time processes, analysis may need to occur continuously or at very short intervals.
   • For routine processes, daily, weekly, or monthly analysis may suffice.
   • For longer-term trends, quarterly or annual analysis may be appropriate.
5. Critical Control Points: Identify critical control points in your processes where deviations can have significant consequences. Analysis at these points should be more frequent to catch issues early.
6. Review Triggers: Establish triggers or thresholds that indicate when analysis and evaluation should occur. For example, you might set a threshold for product defects, and when that threshold is exceeded, it triggers an immediate analysis.
7. Customer Feedback and Expectations: Consider customer feedback and expectations regarding the timing of result analysis. Some customers may require more frequent reporting or immediate notifications of issues.
8. Continuous Improvement Initiatives: If your organization is actively involved in continuous improvement initiatives (e.g., Lean Six Sigma), analysis and evaluation should be an integral part of these efforts and may occur on a regular basis.
9. Data for Decision-Making: Determine when data is needed for decision-making. If data is required for daily, weekly, or monthly decisions, ensure that the analysis and evaluation align with these decision points.
10. Resource Availability: Ensure that you have the necessary resources, including personnel and tools, to conduct timely analysis and evaluation. Resource constraints can affect the frequency and timing of these activities.
11. Documentation and Reporting: Document the frequency and timing of result analysis and evaluation in your quality management system documentation. Create reporting schedules or checklists to ensure that activities are carried out as planned.
12. Continuous Review and Adjustment: Periodically review the effectiveness of your result analysis and evaluation timing. If you find that certain activities are not providing timely or relevant insights, be prepared to adjust the schedule accordingly.
13. Escalation Procedures: Develop escalation procedures for situations where critical issues are identified. Define who should be informed and what actions should be taken in response to specific findings.
14. External Factors: Take into account external factors that may influence when analysis and evaluation should occur, such as seasonality, market fluctuations, or supplier performance fluctuations.

By following these steps and considering a combination of regulatory requirements, risk assessments, customer expectations, process characteristics, and the need for timely decision-making, organizations can establish a well-defined schedule for result analysis and evaluation that aligns with their quality management objectives and ensures that insights are available when needed.

5) The organization shall evaluate the performance and the effectiveness of the quality management system.

This evaluation process is essential for ensuring that the QMS is meeting its intended objectives and for driving continual improvement. Here’s how organizations can effectively evaluate the performance and effectiveness of their QMS:

1. Establish Clear Objectives and Metrics: Define clear and measurable objectives for the QMS. These objectives should align with the organization’s overall goals and may include targets related to product/service quality, customer satisfaction, process efficiency, compliance, and more.
2. Select Key Performance Indicators (KPIs): Identify relevant key performance indicators (KPIs) that will be used to assess the QMS’s performance. KPIs should reflect the critical aspects of the QMS and provide actionable data for decision-making.
3. Collect and Analyze Data: Regularly collect data related to the selected KPIs and other relevant performance indicators. This data may come from various sources, such as process monitoring, customer feedback, internal audits, and non-conformance reports.
4. Conduct Regular Reviews: Schedule and conduct regular management reviews of the QMS. During these reviews, top management should assess the collected data, review performance against objectives and targets, and consider the effectiveness of the QMS in achieving the organization’s goals.
5. Identify Strengths and Weaknesses: Evaluate the strengths and weaknesses of the QMS based on the data and analysis. Identify areas where the QMS is performing well and areas that require improvement.
6. Root Cause Analysis: When issues or non-conformities are identified, perform root cause analysis to understand why these issues occurred. Address the root causes to prevent recurrence.
7. Implement Corrective and Preventive Actions: Take corrective actions to address non-conformities and issues that affect the QMS’s performance. Additionally, implement preventive actions to address potential issues before they occur.
8. Continual Improvement: Continually seek opportunities for improvement within the QMS. Encourage employees at all levels to contribute ideas for enhancements, and use data-driven insights to drive improvement initiatives.
9. Document Findings and Actions: Document the findings from performance evaluations, including identified strengths, weaknesses, opportunities, and threats. Also, document the actions taken to address these findings and improve the QMS.
10. Communicate and Share Results: Share the results of QMS evaluations and improvement efforts with relevant stakeholders, including employees, suppliers, and customers. Effective communication helps build confidence in the QMS.
11. Monitor Progress: Regularly monitor progress toward achieving QMS objectives and targets. Adjust strategies and actions as needed to stay on track.
12. Compliance and Conformance: Ensure that the QMS remains in compliance with ISO 9001 requirements and relevant regulations throughout the evaluation process.
13. Periodic External Audits: Engage in periodic external audits by certification bodies or regulatory authorities to verify the effectiveness and compliance of the QMS.

By following these steps and conducting regular, data-driven evaluations of the QMS, organizations can not only meet the requirements of ISO 9001 but also drive continuous improvement, enhance customer satisfaction, and achieve their quality and business objectives.

6) The organization shall retain appropriate documented information as evidence of the results.

Here are some common types of records that organizations typically maintain to fulfill the requirements .

1. Measurement and Monitoring Records:
   • Records of measurements taken during various processes, such as product inspections, equipment calibration, process parameter readings, and testing results.
   • Data related to product or service quality, including non-conformities and corrective actions.
   • Records of internal audits and management reviews, including findings, actions taken, and improvements made.
2. Performance Metrics and KPI Records:
   • Data related to key performance indicators (KPIs) or metrics that are used to assess process and organizational performance.
   • Records showing trends and performance against objectives and targets, including historical data for analysis.
3. Risk Assessment and Mitigation Records:
   • Documentation of risk assessments, including identification of risks, evaluation, and the actions taken to mitigate or manage them.
   • Records of actions taken to address identified risks, including the effectiveness of risk mitigation measures.
4. Customer Feedback and Satisfaction Records:
   • Customer feedback records, including complaints, compliments, and suggestions.
   • Surveys and feedback analysis related to customer satisfaction.
5. Supplier Performance Records:
   • Records related to supplier evaluation and performance monitoring.
   • Documentation of supplier audits, corrective actions, and improvements.
6. Management Review Records:
   • Documentation of management review meetings, including agendas, minutes, and actions taken.
   • Records of decisions made during management reviews, including strategic planning and process improvement initiatives.
7. Corrective and Preventive Action Records:
   • Records of non-conformities, incidents, or issues identified, including root cause analysis.
   • Documentation of corrective and preventive actions taken to address non-conformities and prevent recurrence.
8. Documented Information on Conformity and Effectiveness:
   • Any documented information that demonstrates conformity to requirements and the effectiveness of the quality management system.
   • Records of the organization’s performance against quality objectives and targets.

It’s important to note that the specific records and documentation required can vary based on the organization’s industry, size, and complexity. Therefore, organizations should establish a document control system and record-keeping procedures that align with their specific needs and compliance obligations. These records should be maintained, updated, and retained for the specified periods as determined by the organization’s policies and relevant regulatory requirements.