ISO 19011:2018 Clause 7.4 Selecting appropriate auditor evaluation method

The evaluation should be conducted using two or more of the methods given in Table below. In using Table below, the following should be noted:
a) the methods outlined represent a range of options and may not apply in all situations;
b) the various methods outlined may differ in their reliability;
c) a combination of methods should be used to ensure an outcome that is objective, consistent, fair and reliable.

Evaluation method	Objectives	Examples
Review of records	To verify the background of the auditor	Analysis of records of education, training, employment, professional credentials and auditing experience
Feedback	To provide information about how the performance of the auditor is perceived	Surveys, questionnaires, personal references, testimonials, complaints, performance evaluation, peer review
Interview	To evaluate desired professional behaviour and communication skills, to verify information and test knowledge and to acquire additional information	Personal interviews
Observation	To evaluate desired professional behaviour and the ability to apply knowledge and skills	Role playing, witnessed audits, on-the-job performance
Testing	To evaluate desired behaviour and knowledge and skills and their application	Oral and written exams, psychometric testing
Post-audit review	To provide information on the auditor performance during the audit activities, identify strengths and opportunities for improvement	Review of the audit report, interviews with the audit team leader, the audit team and, if appropriate, feedback from the auditee

One of the method for evaluation of Auditor is Review of records. This can be done by Analysis of records of education, training, employment, professional credentials and auditing experience. The objective is to verify the background of the auditor. Conducting a review of records is a valuable method for evaluating auditors, especially when verifying their background, education, training, employment history, professional credentials, and auditing experience. This process helps ensure that auditors possess the necessary qualifications and experience to effectively perform their roles. Here’s a breakdown of how the review of records can be implemented:

Steps in the Review of Records for Auditor Evaluation:

1. Education Verification:
   • Objective: Verify the academic qualifications claimed by the auditor.
   • Process: Confirm degrees, certifications, and other educational credentials through direct communication with educational institutions or verification services.
2. Training Records Analysis:
   • Objective: Assess the auditor’s engagement in relevant training activities.
   • Process: Review records of training courses attended, workshops, seminars, and certifications to ensure alignment with the organization’s training requirements.
3. Employment History Check:
   • Objective: Verify the auditor’s employment history and relevant work experience.
   • Process: Confirm details such as job titles, responsibilities, and duration of employment through reference checks, employment verification letters, or direct communication with previous employers.
4. Professional Credentials Verification:
   • Objective: Confirm the validity of professional certifications and memberships.
   • Process: Validate certifications and memberships by checking with issuing bodies, professional organizations, or relevant certification boards.
5. Auditing Experience Assessment:
   • Objective: Evaluate the auditor’s practical experience in auditing.
   • Process: Review the auditor’s record of audits conducted, types of audits, industries covered, and any specialized experience. Verify the accuracy of claims through audit reports or other relevant documentation.
6. Continuous Professional Development (CPD) Records:
   • Objective: Assess the auditor’s commitment to ongoing learning.
   • Process: Review records of continuing education, participation in conferences, workshops, and any other activities contributing to professional development.
7. Documentation of Achievements:
   • Objective: Confirm any awards, recognitions, or notable achievements.
   • Process: Validate claims of achievements by reviewing certificates, letters of recognition, or any documented evidence.
8. Background Check:
   • Objective: Ensure that the auditor’s background aligns with the organization’s values and ethical standards.
   • Process: Conduct a background check to identify any potential red flags, such as legal issues or ethical concerns.

Considerations for Review of Records:

1. Confidentiality and Privacy: Ensure that the review process adheres to privacy and confidentiality standards. Obtain consent from the auditor for accessing and verifying their records.
2. Comprehensive Recordkeeping: Maintain a comprehensive record of the review process, including the documents reviewed, verification outcomes, and any actions taken.
3. Verification Protocols: Establish clear protocols for verifying records, including communication channels with educational institutions, certification bodies, and employers.
4. Feedback and Communication: Provide feedback to the auditor regarding the results of the record review. Address any discrepancies or concerns transparently and collaboratively.
5. Regular Review Cycles: Implement periodic reviews of auditor records to ensure that qualifications and credentials remain current. This is particularly important in dynamic fields with evolving standards.
6. Integration with Overall Evaluation: Integrate the findings from the record review into the overall auditor evaluation process. Consider both qualitative and quantitative criteria for a comprehensive assessment.

The review of records is a foundational component of due diligence in auditor evaluation, contributing to the establishment of a competent and trustworthy audit team. It helps organizations maintain the integrity of their audit processes and ensures that auditors are well-equipped to fulfill their responsibilities.

One of the method for evaluation of Auditor is Feedback . This can be done by Surveys, questionnaires, personal references, testimonials, complaints, performance evaluation, peer review . The objective is to provide information about how the performance of the auditor is perceived .Utilizing feedback as a method for evaluating auditors is a valuable approach to gaining insights into their performance from various perspectives. Feedback can come from a range of sources, including surveys, questionnaires, personal references, testimonials, complaints, performance evaluations, and peer reviews. The objective is to gather information on how the auditor’s performance is perceived by different stakeholders. Here’s how each of these feedback mechanisms can contribute to the evaluation process:

1. Surveys and Questionnaires:

• Objective: Collect structured feedback from clients, colleagues, and other stakeholders.
• Process: Develop surveys or questionnaires with targeted questions about the auditor’s communication skills, professionalism, effectiveness, and adherence to audit standards.

2. Personal References and Testimonials:

• Objective: Obtain qualitative feedback on the auditor’s character and work ethic.
• Process: Request personal references or testimonials from individuals who have directly worked with the auditor. This could include clients, team members, or supervisors.

3. Complaints and Concerns:

• Objective: Address any concerns or issues raised by stakeholders.
• Process: Monitor and address any complaints or concerns related to the auditor’s performance. Investigate the nature of the complaints and take corrective actions as needed.

4. Performance Evaluation:

• Objective: Assess the auditor’s overall performance against established criteria.
• Process: Conduct a formal performance evaluation, considering both quantitative metrics (such as audit outcomes and efficiency) and qualitative factors (such as communication and teamwork).

5. Peer Review:

• Objective: Gather insights from colleagues within the audit team.
• Process: Implement a peer review process where fellow auditors provide feedback on the individual’s collaboration, contribution to team objectives, and overall effectiveness.

Considerations for Feedback-based Evaluation:

1. Anonymous Feedback: Consider offering the option for stakeholders to provide feedback anonymously, encouraging more honest and constructive responses.
2. Diverse Perspectives: Seek feedback from a diverse range of stakeholders, including clients, team members, supervisors, and peers, to obtain a well-rounded view of the auditor’s performance.
3. Structured and Open-Ended Questions: Combine structured questions with open-ended ones to gather both quantitative ratings and qualitative comments on specific aspects of the auditor’s performance.
4. Continuous Feedback Loop: Encourage continuous feedback rather than relying solely on periodic evaluations. Establish a culture of ongoing communication about performance.
5. Feedback Incorporation: Integrate the feedback received into the auditor’s professional development plan. Use constructive feedback as a basis for improvement.
6. Recognition of Strengths: Acknowledge and recognize the auditor’s strengths and positive contributions, fostering a positive and supportive work environment.
7. Timely Feedback Delivery: Provide feedback in a timely manner to ensure that the information is relevant and can be applied for continuous improvement.
8. Feedback Mechanism Accessibility: Ensure that the feedback mechanisms are accessible and user-friendly. This includes easy-to-complete surveys and clear processes for submitting testimonials or references.
9. Professional Development Opportunities: Identify areas for professional development based on feedback and provide opportunities for training or mentoring.
10. Communication of Findings: Communicate the findings of the feedback to the auditor in a constructive and supportive manner, focusing on both areas of strength and improvement.

The use of feedback as an evaluation method fosters a culture of continuous improvement and helps auditors understand how their performance is perceived by those they interact with. It also provides valuable insights for organizational leadership in making decisions related to training, professional development, and performance recognition.

One of the method for evaluation of Auditor is Interview . This can be done by Personal interviews . The objective is to evaluate desired professional behaviour and communication skills, to verify information and test knowledge and to acquire additional information. Personal interviews are a valuable method for evaluating auditors as they provide an opportunity to assess various aspects of an auditor’s performance, including professional behavior, communication skills, knowledge, and the ability to handle different situations. Here’s how personal interviews can be effectively utilized for auditor evaluation:

Objectives of Personal Interviews:

1. Evaluate Professional Behavior:
   • Approach: Observe the auditor’s demeanor, attitude, and overall professionalism during the interview.
   • Questions: Pose questions related to ethical decision-making, integrity, and how the auditor handles challenging situations.
2. Assess Communication Skills:
   • Approach: Evaluate the auditor’s ability to convey information clearly and interact with others.
   • Questions: Include scenario-based questions to assess communication in various contexts, such as client interactions or team collaborations.
3. Verify Information:
   • Approach: Confirm details provided in resumes or records through direct questioning.
   • Questions: Ask about specific experiences, projects, or accomplishments to verify the accuracy of information provided.
4. Test Knowledge:
   • Approach: Assess the depth and breadth of the auditor’s knowledge in relevant areas.
   • Questions: Include technical questions, case studies, or hypothetical scenarios to gauge the auditor’s understanding of audit principles and industry-specific knowledge.
5. Acquire Additional Information:
   • Approach: Use the interview as an opportunity to gather more insights into the auditor’s background, experiences, and motivations.
   • Questions: Ask open-ended questions that encourage the auditor to share additional information about their professional journey, achievements, and areas of interest.

Considerations for Personal Interviews:

1. Structured Interview Format: Use a structured interview format with a set of predetermined questions to ensure consistency across candidates and fair evaluation.
2. Behavioral Interviewing Techniques: Incorporate behavioral interview questions that prompt candidates to provide examples of past experiences and actions, giving insight into how they approach challenges.
3. Role-Specific Scenarios: Tailor interview questions to scenarios relevant to the auditor’s role, including client interactions, audit planning, and reporting.
4. Active Listening: Practice active listening during the interview to understand the nuances of the auditor’s responses and to identify areas for further exploration.
5. Open Communication: Create an open and welcoming environment that encourages candidates to express themselves freely.
6. Consistency Across Interviewers: Ensure consistency in the evaluation process by providing interviewers with clear guidelines and training on the desired evaluation criteria.
7. Multiple Interview Rounds: Conduct multiple interview rounds, if necessary, involving different stakeholders such as team members, managers, or clients to gather diverse perspectives.
8. Feedback Loop: Establish a feedback loop to gather input from multiple interviewers and consolidate insights for a comprehensive evaluation.
9. Professional Development Discussion: Include a discussion on the auditor’s professional development goals and aspirations to assess their commitment to ongoing learning and growth.
10. Cultural Fit Assessment: Assess cultural fit by exploring the auditor’s alignment with the organization’s values, work culture, and expectations.
11. Adaptability and Problem-Solving: Pose questions that assess the auditor’s adaptability to change and problem-solving skills, particularly in dynamic audit environments.
12. Follow-up Actions: Clearly communicate any follow-up actions or next steps after the interview, providing candidates with a transparent understanding of the evaluation process.

Personal interviews, when conducted thoughtfully and in alignment with the organization’s goals, play a crucial role in evaluating auditors and making informed decisions about their fit for specific roles within the audit team. They provide a dynamic platform for assessing both technical competence and interpersonal skills essential for effective auditing.

One of the method for evaluation of Auditor is Observation . This can be done by Role playing, witnessed audits, on-the-job performance . The objective is to evaluate desired professional behaviour and the ability to apply knowledge and skills .Observation is a powerful method for evaluating auditors, allowing for a firsthand assessment of their professional behavior, application of knowledge, and skills in real-world scenarios. Here are different ways observation can be implemented for auditor evaluation:

1. Role Playing:

• Objective: Assess the auditor’s ability to handle specific situations, simulate client interactions, and demonstrate communication skills.
• Process: Create scenarios relevant to the auditor’s role and observe how they navigate through challenges or address issues.

2. Witnessed Audits:

• Objective: Evaluate the auditor’s performance during an actual audit.
• Process: Have another experienced auditor or supervisor observe the auditor conducting an audit. This provides insights into their approach, decision-making, and adherence to audit protocols.

3. On-the-Job Performance:

• Objective: Evaluate the auditor’s day-to-day performance in the workplace.
• Process: Observe the auditor in their regular duties, including interactions with clients, colleagues, and how they manage their responsibilities. This provides a holistic view of their professional behavior.

Considerations for Observation-Based Evaluation:

1. Clear Performance Criteria: Establish clear criteria for what constitutes effective performance in the observed activities. This could include adherence to audit standards, effective communication, and problem-solving skills.
2. Scenarios Reflective of Real Work: Design role-playing scenarios or witnessed audits that closely mirror the challenges auditors may encounter in their actual work environment.
3. Feedback Mechanism: Implement a feedback mechanism where observers provide constructive feedback to the auditor. This can be used for continuous improvement.
4. Consistent Observation Practices: Ensure consistency in the observation process by providing guidelines to observers and aligning their assessments with predetermined evaluation criteria.
5. Documentation of Observations: Maintain thorough documentation of observed activities, including strengths and areas for improvement. This documentation can be used in performance reviews and professional development discussions.
6. Integration with Other Evaluation Methods: Integrate observation-based evaluation with other methods such as personal interviews, feedback, and performance metrics for a comprehensive assessment.
7. Respect for Auditor’s Autonomy: Balance observation with respect for the auditor’s autonomy. While providing guidance, allow them the space to demonstrate their individual approach and problem-solving skills.
8. Continuous Learning Opportunities: Use observation as an opportunity for continuous learning. Encourage auditors to reflect on their performance, identify areas for improvement, and actively engage in professional development.
9. Alignment with Organizational Goals: Ensure that observation activities are aligned with the broader goals and objectives of the organization, emphasizing behaviors and skills that contribute to the organization’s success.
10. Adaptability Assessment: Assess the auditor’s ability to adapt to unexpected situations and demonstrate flexibility in their approach to auditing.
11. Ethical Considerations: Incorporate ethical considerations into observed scenarios, allowing evaluators to assess the auditor’s ethical decision-making in practical situations.

Observation-based evaluation provides a dynamic and authentic way to assess auditors in action, offering valuable insights into their competencies and behaviors. It allows organizations to tailor evaluations to specific roles and responsibilities, contributing to a more accurate and nuanced understanding of an auditor’s performance.

One of the method for evaluation of Auditor is Testing. This can be done by Oral and written exams, psychometric testing. The objective is to evaluate desired behaviour and knowledge and skills and their application .Testing is an effective method for evaluating auditors, allowing organizations to assess their knowledge, skills, and the application of competencies in a controlled and standardized manner. There are various forms of testing that can be employed for auditor evaluation:

1. Oral and Written Exams:

• Objective: Assess the auditor’s theoretical knowledge, understanding of audit principles, and ability to communicate effectively.
• Process: Administer written exams covering relevant audit topics or conduct oral exams to evaluate the auditor’s verbal communication skills and depth of understanding.

2. Psychometric Testing:

• Objective: Evaluate cognitive abilities, personality traits, and behavioral tendencies relevant to the audit role.
• Process: Use psychometric tests to assess critical thinking, decision-making, and interpersonal skills. This type of testing can provide insights into how auditors approach problem-solving and decision-making.

Considerations for Testing-Based Evaluation:

1. Clear Test Objectives: Clearly define the objectives of the tests, specifying whether they are focused on knowledge assessment, skills application, or behavioral traits.
2. Test Design: Design tests that align with the specific competencies required for the auditor role. This could include scenarios reflective of real-world audit challenges.
3. Standardization: Ensure that tests are standardized to maintain consistency in evaluation across different auditors. This is crucial for fair and accurate assessments.
4. Applicability to the Job Role: Tailor tests to be directly applicable to the responsibilities and tasks associated with the auditor’s job role. This ensures relevance to their day-to-day work.
5. Combination of Formats: Use a combination of oral and written exams, as well as psychometric testing, to provide a comprehensive evaluation covering various aspects of the auditor’s abilities.
6. Feedback Provision: Provide constructive feedback to auditors based on their performance in tests. This can be valuable for their professional development.
7. Ethical Considerations: Ensure that testing practices adhere to ethical standards, respecting the privacy and dignity of the individuals being evaluated.
8. Continuous Improvement: Periodically review and update testing methods to align with evolving audit standards, industry best practices, and organizational needs.
9. Integration with Other Evaluation Methods: Integrate testing results with feedback from other evaluation methods, such as observation, interviews, and performance metrics, for a well-rounded assessment.
10. Accessible Testing Conditions: Create testing conditions that are fair, accessible, and free from unnecessary stressors that may impact performance.
11. Professional Development Opportunities: Use testing outcomes to identify areas for professional development. Provide resources and training opportunities to support auditors in improving their skills and knowledge.
12. Validity and Reliability: Ensure that tests are valid and reliable measures of the competencies being assessed. Regularly review and update test content to maintain relevance.

Testing provides a structured and objective means of evaluating auditors, contributing to the overall understanding of their competencies and suitability for their roles. It can be particularly valuable when combined with other evaluation methods to create a comprehensive and nuanced assessment.

One of the method for evaluation of Auditor is Post-audit review. This can be done by Review of the audit report, interviews with the audit team leader, the audit team and, if appropriate, feedback from the auditee . The objective is to provide information on the auditor performance during the audit activities, identify strengths and opportunities for improvement .

Post-audit reviews are crucial for evaluating auditor performance and identifying areas for improvement. This method involves a comprehensive examination of the audit process, including the audit report, interviews with the audit team leader and team members, and obtaining feedback from the auditee. Here’s how post-audit reviews can be conducted to achieve the objective of assessing auditor performance:

1. Review of the Audit Report:

• Objective: Assess the quality, accuracy, and completeness of the audit report.
• Process: Examine the audit report for clarity, adherence to audit standards, effectiveness in communicating findings, and overall professionalism.

2. Interviews with the Audit Team Leader:

• Objective: Gain insights into the leadership and coordination provided by the audit team leader.
• Process: Conduct interviews with the audit team leader to discuss their approach to planning, organizing, and leading the audit. Explore decision-making processes and any challenges encountered.

3. Interviews with the Audit Team:

• Objective: Evaluate individual and collective performance of the audit team members.
• Process: Conduct interviews with audit team members to discuss their roles, contributions, and experiences during the audit. Assess communication, collaboration, and the application of audit principles.

4. Feedback from the Auditee:

• Objective: Obtain the auditee’s perspective on the audit process and the auditors’ performance.
• Process: Seek feedback from the auditee regarding their satisfaction with the audit, the professionalism of the auditors, and any suggestions for improvement.

Considerations for Post-Audit Reviews:

1. Constructive Feedback: Provide constructive feedback that highlights both strengths and areas for improvement. This fosters a positive environment for professional development.
2. Open Communication: Encourage open and honest communication during interviews to gather candid insights into the audit process.
3. Objective Evaluation Criteria: Establish clear criteria for evaluating auditor performance, ensuring that the assessment is objective and aligned with organizational goals.
4. Learning Opportunities: View post-audit reviews as learning opportunities for auditors. Identify lessons learned and apply them to future audits.
5. Continuous Improvement Focus: Emphasize the goal of continuous improvement. Use the post-audit review process to enhance audit methodologies, teamwork, and overall effectiveness.
6. Root Cause Analysis: If issues or challenges are identified, conduct a root cause analysis to understand the underlying factors. This helps in implementing targeted improvements.
7. Documentation of Findings: Document findings from the post-audit review comprehensively. This documentation serves as a reference for future assessments and improvements.
8. Timely Review: Conduct post-audit reviews promptly after the completion of the audit to ensure that feedback is relevant and can be applied in a timely manner.
9. Integration with Professional Development: Integrate post-audit review outcomes with professional development plans for auditors. Tailor training and learning opportunities based on identified needs.
10. Communication of Outcomes: Communicate the outcomes of the post-audit review to the audit team in a transparent and collaborative manner. Include recommendations for improvement.
11. Feedback Loop Closure: Close the feedback loop by implementing changes based on the findings of the post-audit review. This demonstrates a commitment to continuous improvement.

Post-audit reviews play a pivotal role in the growth and enhancement of auditor performance. By systematically assessing the audit process and obtaining feedback from various stakeholders, organizations can refine their audit practices and contribute to the ongoing development of their audit teams.

The evaluation should be conducted using two or more of the methods. The methods outlined represent a range of options and may not apply in all situations. The various methods outlined may differ in their reliability. A combination of methods should be used to ensure an outcome that is objective, consistent, fair and reliable.

Effective evaluation of auditors typically involves the use of multiple methods to ensure a comprehensive and well-rounded assessment. The choice of evaluation methods can vary based on the organization’s goals, the nature of the audit activities, and the specific competencies being assessed. Combining different methods helps in mitigating the limitations or biases inherent in any single evaluation approach. Here are key considerations:

1. Diversity of Methods: Use a mix of methods, such as personal interviews, observation, testing, post-audit reviews, and feedback. Each method provides unique insights into different aspects of auditor performance.
2. Comprehensive Assessment: Combining methods allows for a more comprehensive assessment of auditors, covering various dimensions, including knowledge, skills, behavior, and application in real-world scenarios.
3. Objective and Reliable Outcomes: The use of multiple methods contributes to more objective, consistent, and reliable outcomes. Each method can validate or complement the findings of others, leading to a more robust evaluation.
4. Fairness and Consistency: By incorporating different evaluation methods, organizations can enhance fairness and consistency. This is particularly important when assessing a diverse range of auditors with varying roles and responsibilities.
5. Adaptability to Context: The suitability of evaluation methods may vary based on the context, organizational culture, and the specific goals of the audit. Adapt the mix of methods to align with the unique aspects of the audit activities.
6. Continuous Improvement: Regularly review and refine the mix of evaluation methods based on feedback and evolving organizational needs. This ensures that the evaluation process remains dynamic and responsive to changing requirements.
7. Balancing Qualitative and Quantitative Data: Incorporate both qualitative and quantitative data in the evaluation process. While methods like interviews and observation provide qualitative insights, testing and metrics contribute quantitative data.
8. Stakeholder Involvement: Involve various stakeholders in the evaluation process. Feedback from auditees, peers, team leaders, and supervisors can provide a more holistic view of auditor performance.
9. Transparency in Process: Clearly communicate the evaluation process to auditors, ensuring transparency in how different methods will be used and how the outcomes will be interpreted.
10. Regular Review of Evaluation Methods: Periodically review the effectiveness of chosen evaluation methods. Assess whether they align with organizational goals and if any adjustments are needed for continuous improvement.
11. Legal and Ethical Considerations: Ensure that the chosen methods adhere to legal and ethical standards, respecting the rights and privacy of auditors. This is particularly relevant when using testing or psychometric assessments.
12. Flexibility in Approach: Be flexible in the approach, recognizing that there is no one-size-fits-all solution. Tailor the mix of evaluation methods based on the specific needs and characteristics of the auditing context.

By adopting a multifaceted approach to auditor evaluation, organizations can derive richer insights into the competencies and performance of auditors, contributing to the ongoing improvement of audit processes and team effectiveness.