 |
 |
|
 |
Ten Essential Audit Questions
By Craig Cochran
All experienced auditors accumulate favorite audit
questions, and I'm no exception. I have a short, punchy list of queries
I invariably ask while evaluating a management system. Favorites aside,
though, what are truly the most important audit questions? What questions
will reveal a system's effectiveness and an organization's overall performance?
I compiled a list of the top 10.
ISO 9001:2000 requires that organizations establish measurable
objectives at relevant functions and levels. Perhaps an even more significant
requirement is that personnel understand how they contribute to these
objectives. This requirement doesn't apply only to some employees; it
applies to everyone. All personnel must be able to communicate, in their
own words, how they help move objectives in the right direction. It's
conceivable that not all objectives apply to everyone in the company,
and in those cases auditors would only expect that personnel understand
the objectives that apply to them.
This question directly reflects on an organization's ability to communicate
what matters most to its success. Truly comprehending objectives means
that people understand specifically what they can do to improve the organization.
They appreciate the significance of their roles and are prepared to carry
them out. This knowledge creates strategic focus throughout the organization.
Instead of having a limited view of activities and tasks, personnel begin
to understand how their jobs link to the organization's larger mission.
Closely related questions include:
* How are objectives determined?
* How are employees trained on objectives?
* How is progress against objectives communicated to the organization?
* What processes and/or tools are in place to help achieve objectives?
* Is there evidence of progress?
This question reflects on the organization's ability to deal with
product problems in a systematic way. Controlling nonconforming products
is a basic discipline and one that smart auditors always probe. The answer
to this question can be compared to the documented procedure and, more
important, to the auditor's observations.
Few other processes require as rigid adherence to procedures as controlling
nonconforming products. Simply put, there's no room for deviation. Problems
relating to controlling nonconforming products almost always pose significant
risks to the organization-in added costs, wasted time, aggravated employees,
angry customers and loss of competitive position.
During an audit, find some examples of nonconforming products--if any
exist--and follow-up with these questions:
* How are nonconforming products identified?
* Where are they located?
* What are the responsibilities and authorities related to dealing with
nonconforming products?
* How do dispositions get determined and implemented?
* What are the records of nonconforming products and actions taken on
them?
* What are the trends in nonconforming products?
* How is the procedure linked to the corrective action process?
It's worth mentioning that controlling nonconforming products applies
to services just as much as it does to tangible goods. Reports, data,
test results and intellectual property, to name just a few service outputs,
can all be potentially nonconforming, in which case all the disciplines
of this process apply.
Everybody has a product of some sort. It might go to an external
customer or simply to the next process inside the organization. In all
cases, though, personnel must understand the product requirements. ISO
9001:2000 specifically requires that organizations identify product requirements
in four ways:
* As stated by the customer
* Unstated by the customer but necessary for intended use
* As statutory and legal regulations related to the product
* As any additional requirements determined by the organization
The standard additionally requires that information describing the product
be available (i.e., documented). Asking how personnel access product requirements
is an important audit question because when requirements aren't accessible,
big problems often result. Employees don't need to know product requirements
by heart, but they should certainly be able to find the current versions
of requirements and describe how they carry them out.
Specific points of inquiry related to product requirements include:
* Are product requirements complete?
* How does the organization ensure that correct versions are available?
* How are requirements reviewed prior to acceptance?
* How do you ensure that product meets the stated requirements?
* What happens when changes are made to product requirements?
Problem correction is relatively simple: Define the problem, identify
the cause and take action to remove it. Problem prevention, on the other
hand, is more complex. Many people would also argue that it's more important.
Preventive action is specifically required by ISO 9001:2000, and it provides
one of the most valuable links to continual improvement.
The most obvious way to generate preventive action is by analyzing data.
Data analysis is a primary job of top management, but it can happen at
other levels of the organization as well. When an organization openly
shares data and encourages its analysis on a broad scale, then preventive
action becomes easy. Employee creativity and innovation can also be a
valuable starting point for preventive action. Savvy organizations look
for ways to solicit improvement ideas from their employees and provide
feedback on the viability of the ideas. Another source of preventive action
is feedback from customers. Often, customers will provide ideas for improving
the product in subtle yet significant ways.
Additional points of inquiry related to preventive action include:
* How do data trends get analyzed?
* How do employees communicate their improvement ideas?
* How do preventive actions get recorded?
* Are statistical techniques used?
* How are customer perceptions captured on a proactive basis?
This question probably won't apply to all personnel. It's especially
relevant to top management and employees responsible for gauging customer
perceptions. The question is significant because most organizations manage
fairly well to capture perceptions but usually fall short of actually
doing something with the information.
ISO 9001:2000 specifically requires that organizations define methods
for obtaining and using customer satisfaction data. This is another reason
for relying on simple methods for capturing customer perceptions. The
more complex and resource-intensive your customer satisfaction methods
are, the less likely you'll take action on what you learn. It's a curious
paradox. Many organizations run out of gas before they get to the action
phase, and the valuable opportunities afforded by customer feedback are
ignored as other problems arise.
Here are some related audit questions:
* How are data on customer satisfaction analyzed?
* How are opportunities identified and prioritized?
* What's the connection to the corrective and preventive action systems?
* What are the organization's long-term trends in customer satisfaction?
* How are resources for customer satisfaction identified and provided?
* What connections exist between customer satisfaction and the organization's
objectives?
Despite everyone's best efforts, customers will occasionally complain.
Customer complaints represent both a huge risk and valuable opportunity
to the organization-it all depends on how they're handled. This question
is especially relevant to salespeople, customer service representatives,
technical personnel and top management. The auditor is looking for proof
of a systematic approach to dealing with complaints. This will typically
include defined responsibilities for logging and tracking complaints,
clear problem statements with all relevant facts included, determination
of problem causes and actions that address the causes. Specific examples
of complaints must be sampled, of course. The link between the complaint
process and corrective action also requires special scrutiny.
Here are some related questions:
* What's the largest complaint category?
* What's being done about it?
* Has the amount of complaints changed over time?
* How are personnel trained in their roles in preventing complaints?
* How are customers made aware of actions on their complaints?
* What tools are used to identify the causes of complaints?
One of top management's most important responsibilities is reviewing
the organization's performance. I'm not talking about employee performances
but the organization's success as a whole. Is your organization becoming
more efficient, more competitive, better at serving customers? Or is it
moving in the opposite direction? Top management should regularly analyze
data and trends that provide the answers to these questions. ISO 9001:2000
specifically requires management review with defined inputs and outputs.
And there's no sense in conducting an ISO 9001 management review, then
conducting a separate review of the organization's performance--they should
be the same review. The more timely and action-oriented the review, the
better.
Some of the best approaches to reviewing organizational performance are
the most creative. Many organizations design their reviews across a number
of different forums and time frames, which is a practical and realistic
way to approach the process. Regardless of how the review is configured,
the three imperatives include data analysis, identifying opportunities
and taking action on them. Smart organizations treat these three activities
as inseparable.
Here are some related questions:
* Who's involved in reviewing the organization's performance?
* What actions have resulted from these reviews?
* How are records of the reviews generated?
* Are all required inputs and outputs addressed by records?
* How does the rest of the organization learn of actions and decisions
that are determined during reviews?
This question can be asked of everybody in the organization. In
organizations that have developed improvement tools and provided opportunities
for their application, this is an easy question. In organizations where
improvement efforts are very narrowly applied, it becomes a much harder
question. There should certainly be some evidence of continual improvement
within the scope of the audit. Strategic improvements are impressive,
of course, but all improvements have value. This question actually summarizes
many of the earlier questions into a single point of inquiry. The ultimate
purpose of a management system is to provide a means for improvement.
Just because one or two people aren't able to provide evidence of improvement
isn't necessarily a problem. It could indicate weak improvement efforts,
though, and further investigation would certainly be warranted. In very
mature organizations, all personnel are involved in making improvements,
and proof of this happening is abundant.
These are some related lines of inquiry:
* Who's involved in improvement efforts?
* What tools are used to pursue continual improvement?
* How are personnel trained to use improvement tools?
* How are improvement ideas prioritized?
* How are employees made aware of improvement efforts and successes?
This might seem like a strange question on a list of most important
audit questions, but it's very significant. Developing human resources
is one of the keys to organizational success. This audit question attempts
to probe the degree of planning that goes into developing these resources.
Is training performed as a knee-jerk activity without any underlying objectives?
Or is it geared toward empowering each employee with the skills and knowledge
needed to propel the organization forward? During the audit, make sure
to probe the training needs that have been determined for all levels of
personnel: hourly, salaried, temporary, highly skilled technicians and
top management. Training is an activity that applies to all personnel,
not just a narrow slice of the organization.
Here are some related questions:
* What kind of orientation training is provided when employees are hired?
* How are personnel made aware of the organization's mission, values and
measurable objectives?
* How is the effectiveness of training evaluated?
* What happens when training is determined to have been ineffective?
* What records of training are maintained?
This is an exploratory question aimed at assessing the degree of
planning that went into developing the management system. The answer can
be compared to the formal controls in place (e.g., documentation, training,
verifications, data analysis, etc.) to determine how deliberately the
management system was designed and implemented. If you learn that the
most important thing about the job is receiving timely and complete feedback
from the downstream department, then it will be revealing to explore if
the feedback exists and what's done with it.
This question requires a great deal of skill on the auditor's part because
the information may or may not lead to any logical conclusions. The auditor
must have the experience and maturity to know when an issue is worth exploring
in detail. In other words, don't allow this question to become an endless
fishing expedition. Explore the important elements of a job, compare what
you learn against the controls in place and cross-check the facts with
other personnel doing similar jobs. It can produce powerful insights.
Consider these related questions:
* What's the hardest thing about your job?
* What are some things you'd like to change about your job?
* What resource would help you be more effective?
* What should your manager know that he or she currently doesn't know?
* If you were the manager here, what would you do differently?
The 10 questions presented here represent only a slice of what might matter
to a typical organization. You will want to refine this list based on
special concerns and risks faced by your company. Figure out what matters
most to your organization and focus your audit process on those things.
There isn't enough time and energy to focus on everything. An audit process
that monitors the organization's key success factors will always be relevant
and always produce powerful results.
Craig Cochran |
About the Author:
Craig Cochran is a project manager with the Center for International
Standards & Quality, part of Georgia Tech's Economic Development
Institute. He's an RAB-certified QMS lead auditor and the author of
Customer Satisfaction: Tools, Techniques and Formulas for Success
and The Continual Improvement Process: From Strategy to the Bottom
Line, both available from Paton
Press. CISQ can be reached at (800) 859-0968 or on the Web at
www.cisq.gatech.edu. |
|
The Continual Improvement Process: From Strategy to the
Bottom Line
Continual improvement is not optional. It is a condition of survival.
Every organization must have systematic methods for making smart decisions,
attacking problems, improving its products and services, and repelling
competitors. Anything less than a systematic, disciplined approach
is leaving your future in the hands of chance. This book presents
a range of practical methods for driving continual improvement throughout
the organization. The starting point is leadership, with a clear definition
of mission, strategy, and key measures. These themes are then carried
throughout the enterprise, informing everyone on the issues that matter
most to survival and success. Strategic approaches for the deployment
of metrics, review of organizational performance, effective problem
solving, internal auditing, process orientation, and cultural development
are also described in detail. Practical tools and examples are provided
at every step of the way, enabling immediate implementation of the
concepts. This book is more than a guide to continual improvement;
it is a guide to leading and managing any organization. |
| 
Buy
Amazon
|
Customer Satisfaction: Tools, Techniques and Formulas
for Success
Customer satisfaction is the single most important issue affecting
organizational survival. Despite this fact, most companies have
no clue what their customers really think. They operate in a state
of ignorant bliss, believing that if their customers were anything
less than 100-percent satisfied they'd hear about it. Then they
are shocked when their customer base erodes and their existence
is threatened. The key to competitive advantage is proactively gauging
customer perceptions and aggressively acting on the findings. The
techniques for doing this don't have to be difficult, they just
have to be timely and effective. This book explores a range of practical
techniques for probing your customers' true level of satisfaction.
Tools and specific instructions for use are described in detail,
enabling the organization to get started immediately. The tools
range from very basic to highly sophisticated, providing a path
for organizations to follow as they progressively become more familiar
with the unique drivers of customer satisfaction. This is the perfect
reference for organizations that want to continually improve and
outpace their competition.
|
top of page |
|
