6. Reliability
Effective SEA reports contain information that is verifiable, objective, comprehensive in coverage and faithfully representative. In doing so, the following attributes of the reported information need to be considered:
- Is the SEA Performance information being reported verifiable to provide assurance that the information in an SEA report would be replicated by independent evaluators using the same measurement method?
Reliable SEA performance information communicates the level of assurance received, if any, on the reported information. Assurance of a performance measure may be achieved by verification of the measure itself or by selected testing to verify the procedures used to obtain the information reported by the SEA performance measure. While independent verification is ideal, there are other ways of verifying performance information that may provide a reasonable degree of assurance.
- Is the SEA performance information being reported derived from systems producing verifiable data?
Reliable SEA performance information is derived from systems from producing verifiable data. This means that it is important to have system control reviews conducted internally and on a regular basis. This may be done on a recurrent cycle over a period of years and may consider the size, importance, and risk associated with different performance information.
- Is the SEA performance information being reported objective?
Reliable SEA performance information includes both positive and negative results of key measure of SEA performance. SEA performance information is considered objective if it has not been modified by the government either to overstate or understate positive or negative results.
- Is the SEA performance information being reported comprehensive in coverage?
The SEA performance information is thorough in its presentation of the measures. Comprehensive in coverage means that significant data, that would make a difference in a user’s assessment or decision, is not omitted from the information that is necessary to faithfully represent major results (within the bounds of what is significant and feasible considering cost). Nothing should be included that would cause the information to be misleading.
- Does the SEA performance information includes estimates for some performance measures?
Reliable SEA performance information may include estimates for some performance measures because it may not, in all cases, be feasible to obtain precise information about some of the more relevant performance measures. In cases where estimates are used, it needs to be clearly explained that the measure being reported is based on an estimate and not actual results.
- Does the SEA performance information communicate to users that the reliability of some or all of the performance measures being reported may not be assured, if applicable?
Reliable SEA performance information communicates when the reliability of some performance measures has not been assured. If there are concerns about the reliability of the reported SEA performance information, because of the results of the verification procedures performed, these concern also need to be communicated to users.