Question NW2613 to the Minister of Transport

Reply:

South African Civil Aviation Authority (SACAA)

With reference to the SA Civil Aviation Authority, (a)(i)(ii)(aa),(bb),(cc)

• The number of examinations written for the 2012-2013 financial year, including Air Law and Procedures was 23686.
• The number of examinations written for the April 2013 to December 2014, including Air Law and Procedures was 21773.
• The number of examinations written for the period January 2014 to 31 March 2015, which included Air Law and Procedures was 24186.
• The SACAA introduced a new examination system in January 2014, and as such monthly statistics for the specified period could not be automatically generated. The legacy system did not have the capabilities for such reporting.

(b) It is not possible to generate the average percentage per assessment or paper by using the current reporting template. However, in all cases, a print-out is provided to the examinee on completion of the examination, which indicates in what aspect of the syllabus a deficiency was encountered.

(c) The examination fees are published in the Civil Aviation Regulations 2011, Part 187, Regulation 187.01.9 (e)(ii) which is R260 per online examination and Regulation 187.01.9 (e)(iii) which is R370 per hand written examination.

(d) The SACAA makes use of reputable examination system, known as “Question Mark Perception” (QMP). The QMP provides for a learning syllabus (which is published and known to the examinee through the Appendices contained in the South African Civil Aviation Technical Standards of 2011, Document SA-CATS 61) that forms the foundation of a database of questions to be populated. Four detractors (answers) are provided in the database for each question and the student is then provided with four answers, from which the correct one must be chosen.

The QMP, through its pre-set assessment criteria, generates an examination paper for each examinee when they enter into the examination system. No two examination papers are the same. Each examination paper uses the same pre-set assessment criteria in the system to examine competency across the entire set syllabus.

Examinees are required to select the correct answer from the four answers provided for each question. On completion of the exam, the examinee requests the system to assess and the system then immediately provides the assessment mark.

(e) (i), (ii) and (iii) No investigation was conducted during the 2012-2013 financial year. No investigation was conducted during the 2013-2014 financial year. Only one (1) investigation was conducted during the 2014-2015 financial year. The investigation was initiated as the result of a complaint received from a member of the public and a holder of an expired Airline Transport Pilot’s Licence, who alleged that the questions used in the Air Law and Procedures examination were inappropriate, irrelevant and designed in such a way that no person can get the right answers. The complainant was of the view that such amounted to “fraud and that the SACAA was fleecing the public by continuously collecting examination fees whilst they knew that no person could pass such examinations”.

(f)(g) and (h) The investigation was conducted by the SACAA’s Internal Forensic Unit, with the assistance of an independent aviation expert. Among others, the investigation probed the examination admission process and procedures, question formulation, suitability, appropriateness and relevance.  The investigation also focused on determining the suitability, appropriateness and relevance of examination questions. The investigation revealed and concluded that whilst there was slight room for future improvement on the processes and procedures of admitting candidates into the examination room; the Air Law and Procedures examination questions were suitable, appropriate and relevant to the establishment of the candidate’s theoretical knowledge and competency. The investigation, therefore, found no evidence to support the allegations levelled against the SACAA.