Outstanding National Certificate Vocational NC(V) certificates: Department update; UMALUSI on quality assurance of 2014 examination results for NC(V), GETC, ABET, IEB

Higher Education, Science and Innovation

03 June 2015
Chairperson: Ms Y Phosa (ANC)
Share this page:

Meeting Summary

The Committee received a briefing from the Department of Higher Education and Training (DHET) on the situation regarding outstanding National Certificate Vocational (NC (V)) certificates. There was also a briefing by the Council of Quality Assurance in General and Further Education and Training (Umalusi) on the quality assurance of the 2014 examination results.

The DHET said it relied on information technology (IT) services from the State Information Technology Agency (SITA) in order to operationalise the national examinations of the Technical and Vocational Education and Training (TVET) colleges. This process extended from the registration of candidates for examination, to the issuance of results and the certification of candidates. There were still a number of challenges that had been experienced by the Department in issuing outstanding NC (V) certificates. These included the instability of data sets submitted by SITA to Umalusi, which had then been rejected by the quality assurance and certification body, and inappropriate IT systems at SITA for the NC (V) qualification. An additional challenge was the allocation of insufficient human resources by SITA for the TVET college examination system, resulting in delays in rectifying identified errors. This had led to the backlog of NC (V) certification from 2007 to 2011.

Several engagements had taken place between the Department, Umalusi and SITA, and a management plan had been developed to resolve the backlog of certificates. The backlog status at the moment showed that all Report 190/1 candidates for business and engineering studies were issued with their qualification within three months. 496 544 NC (V) certificates had been issued since the backlog project began in 2012. From March to April 2015, 30 545 NC (V) certificates had been issued, and as at that date, 103 195 NC (V) certificates were still outstanding. From November 2007 to Supplementary 2012, 15 099 certificates had been issued and from November 2012 to Supplementary 2014, about 88 096 certificates had been issued.

Members suggested that SITA should be called before the Committee to provide a time-line for fixing the accumulated backlog. What could be done to fast-track this process, as it was impossible to follow the normal process while a new backlog was accumulating? They wanted to know whether the new service provider would come under the control of the Department or SITA. There should be an assurance that a proper project plan would be followed when transferring from the old to the new system in order to avoid another collapse when the new system came on board. It was an extremely complex system and a lot of modification would have to be made.

Most Members asked about the precise challenges in the issuing of NC (V) certificates, as countries like China and India had been able to issue their certificates on time, despite their sizeable populations. One Member wanted to know whether the Department had a master plan to address this major challenge, where there would be stipulated time-frames on action to be taken. The delay in the issuing of certificates affected the future of young people, as it was impossible to eliminate youth unemployment when it was made difficult for them to apply to tertiary institutions or for job opportunities.

The approval of question papers for the General Education and Training Certificate (GETC) was still at 54% from the first moderation and 46% from the second and third moderation, and for the Benchmarking (BM): GETC it was 75% from the first moderation and 25% from the second moderation. The areas of good practice in the moderation of the question papers included no question paper leakages, and questions that were not predictable, with creativity and innovation. The setting of an extra set of papers, the first stage towards an eighteen month cycle, was a step in the right direction. To ensure equivalence of standards, the November, March and back-up examination question papers for DHET:NC(V) and the National Technical Education (NATED) programmes were moderated simultaneously. Areas of concern in the moderation of question papers mostly pointed to the need for a improvement in internal moderation for the DHET, Independent Examination Board (IEB) and BM. The training of examiners and internal moderators was important to enhance understanding of subject and assessment guidelines.

It was noted from the findings of the quality assurance processes that many good practices were evident but on the other hand, there were still many challenges that needed to be addressed. Significant interventions were required to improve the curriculum, teaching and learning, and exposure of learners to a compulsory practical component and assessment for the NATED programmes to make it a meaningful qualification. There was concern regarding the outdated and under-specified curriculum and a lack of subject assessment guidelines for the NATED programmes, as this often created problems for appropriate quality assurance to be conducted.

Members expressed dismay about the state of colleges in the country, as there were so many irregularities and leakages that still needed to be addressed. They wanted to know whether there was a specific reason for the reinstatement of the NATED curriculum, as it had already been pointed out that it was outdated and lacked subject assessment guidelines. What had been done about the poor practices that had been noted, as Umalusi seemed like a toothless regulator? Was it possible to withdraw accreditation from a particular college that had been continuing with bad practices? Was there a benchmark on the required standard for colleges? They wanted to know whether there was a way of ensuring that the markers in colleges had been adequately trained and capacitated, and whether colleges were also externally examined like universities to ensure that the training that had been offered matched international best practice. What was the Department doing to ensure that all the accredited colleges were able to pay on time, to avoid the accumulation of debt?      

Meeting report

Department of Higher Education and Training (DHET): Briefing

Ms Nadine Pote, Chief Director: Examinations, said that following the introduction of Report (191) programmes 1998, the Council of Education Ministers had resolved that the entire Technical and Vocational Education and Training (TVET) College examination function should become a national competency. The then Department of Education had introduced a new qualification in the form of the National Certificate Vocational (NC (V)) in 2007, which would also be resulted and certified at the national level. Examinations for TVET Colleges were conducted seven times a year and twice for Adult Education and Training, with candidates being resulted and certified in each of the nine examinations cycle. In order to fulfil the responsibilities inherent in being the examining body, the Department of Higher Education and Training (DHET) relied on Information Technology (IT) services from the State Information Technology Agency (SITA) to operationalise the national examination of TVET colleges. This included the registration of candidates for examination, to the issuance of results and certification of candidates. On completion of an examination cycle, all candidates had a legitimate expectation to receive their complete and correct statement of results, and a certificate if they had met the full requirement of the qualification.

From 2007 to 2010, the assessment body had processed, resulted and issued certificates for the National Certificate Vocational qualification. In 2010, Umalusi had been requested by the Minister to fulfil its certification as mandated in the General and Further Education and Training Quality Assurance Act (GENFETQA), which required Umalusi to certificate candidates. Negotiations had been undertaken with Umalusi on the process to be followed. Enhancements were made to the examinations information system, managed and maintained by SITA, to incorporate Umalusi’s certification requirements.

Ms Pote took the Committee through various challenges still experienced by the Department on the issuing of outstanding of NC (V) certificates and these included:

- Instability of datasets submitted by SITA to Umalusi, which was then rejected by the quality assurance and certification body;

- Inappropriate IT systems at SITA for the NC (V) qualification;

- Allocation of insufficient human resource by SITA for the TVET college examination system, resulting in delays in rectifying identified errors;

- The backlog of NC (V) certification from 2007-2011.

Several engagements had taken place between the Department, Umalusi and SITA. A management plan had been developed, to resolve the backlog of certificates. SITA had made numerous commitments, but they had been unable to honour these commitments.

The backlog status at the moment showed that all Report 190/1 candidates for Business and Engineering Studies were issued with their qualifications within three months. 496 544 NC (V) certificates had been issued since the backlog project began in 2012. In the main, NC (V) candidates who had completed their qualification in a single examination cycle had received certification to date. The outstanding certificates were owed to candidates who had completed their qualification across more than one examination cycle due to the examinations’ information system being unable to combine the subject results from different examination cycles when a candidate qualified for certification. From March 2015 to April, 30 545 NC (V) certificates had been issued and as at 30 April 2015, 103 195 NC (V) certificates were still outstanding. From November 2007 to Supplementary 2012, 15 099 certificates had been issued and from November 2012 to Supplementary 2014, about 88 096 certificates had been issued. The datasets for the 103 195 NC (V) certificates were currently being verified and based on the feedback from Umalusi, more definitive timelines would be set to finally address this backlog.

In conclusion, Ms Pote stated that it was important to highlight that about 496 544 certificates had been issued, with 103 195 certificates still being processed. The students that were affected by the problem did have statements of results and the Department, on request, provided students with letters confirming that they had complied with the requirements for the issuance of a certificate. The potential employers and institutions of higher education were provided with the necessary verification and supporting letters on request so that students were not disadvantaged. The Department continued working with SITA and Umalusi to resolve this backlog. Through a procurement process, the appointment of a service provider was being finalised.

Discussion

Ms J Kilian (ANC) commented that it was of the concern that despite the interventions by the Minister, the Director General (DG) and SITA, the problem of the backlog in the issuing of certificates could not be resolved. The problem was clearly systemic -- mainly an Information Technology (IT) problem -- and it was important for the Department to elaborate on where the problem was exactly, whether it was on SITA or because of out-dated IT equipment. She wanted to know the time-frame for the completion of the process of appointing a service provider, as it was evident that when the Department was trying to eradicate the old backlog. Had a new backlog developed? It was  commendable that as from November 2007 to Supplementary 2012, 15 099 certificates had been issued and from November 2012 to Supplementary 2014 about 88 096 certificates had been issued. However, it was of a major concern that about 103 195 NC (V) certificates were still currently being verified and with exams approaching, it was likely that a new backlog would soon develop. What could be done to fast-track this process, as it was impossible to follow the ordinary process while a new backlog was accumulating?  

Dr B Bozzoli (DA) asked whether there was any way that Umalusi could assist SITA in order to solve the problem of backlogs in the issuing of certificates. The Committee should call SITA before the Committee to demand a time-line for fixing this accumulation of backlog. It was important to know whether students who were affecting by this problem were aware that they could get a letter confirming that they complied with the requirements for the issuance of a certificate, as this often put the responsibility on students and many students may not know this. She wanted to know if the new service provider would come under the control of the Department or SITA. There should be an assurance that a proper project plan would be followed when transferring from the old to the new system so as to avoid another collapse when the new system came on board. It was an extremely complex system and a lot of modification would have to be made.       

Mr C Kekana (ANC) commented that he did not understand why there was a problem that was making it difficult to issue the certificates to students on time, as certificates had been delivered on time even when the system was manual. The population might have been less than it was at the current moment, but the modern technologies should have been able to make the process more efficient. He highlighted that big countries like India, China and Germany were able to deliver their certificates on time despite their populations being many times bigger than that of South Arica. The Department needed to check the international best practices that could be implemented in the country to ensure that certificates were delivered on time, especially when considering that students had worked hard to get their results.     

Ms S Mchunu (ANC) wanted to commend the progress the Department had been made thus far, and although the Members were not happy with the current backlog, most certificates had been issued to date. What had been the main challenges in the issuing of NC (V) certificates? Were the progression requirements not clear enough? She wanted to know whether the Department had a master plan to address this major challenge, where there should be stipulated time-frames on action to be taken. The delay in the issuing of certificates affected the future of young people as it was impossible to eliminate youth unemployment when making it difficult for them to apply for tertiary institution or job opportunities.    

Mr E Siwela (ANC) emphasised that everyone who sat for an exam had a legitimate expectation to get the certificate and it was absurd that the Department could offer supporting letters confirming that the students complied with the requirements for the issuance of a certificate, but made it difficult to issue the actual certificate. The issuing of letters could serve some purpose for a while and the tertiary institutions would eventually demand the original copy of the certificate. What was the recourse for the student when the tertiary institution eventually reached this stage? What was the time-frame of the Department to solve the problem of the backlog in the issuing of certificates?

Mr M Mbatha (EFF) described it as disgraceful and unacceptable that young people in the country found many ways to be displaced by systems that were unexplainable. The interest of the government should be to drive the prosperity of young people and the issuing of any certificate after achievement was the greatest thing that everyone always relished. The Department had not offered any concrete reasons for the failure to issue the NC (V) certificates on time. There was a college called Mbumbulu College of Education that used to produce more than 11 000 teachers without a single computer, and this was up until the early 1990s. If SITA was unable to deliver on its mandate for the interest of young people and everyone in the country, then the Committee would need to recommend an alternative service provider. He expressed his frustration that there were many young people who had lost their jobs after waiting for more than four months for their certificates, while there were officials that were getting paid for failure to do their work.      

Mr Gwebinkundla Qonde, Director-General (DG), DHET, responded that most of the questions and comments that had been made by the Members were helpful in going forward to address the problem of the backlog in the issuing of NC (V) certificates. There were nine cycles of examinations per annum in the TVET system and with NC (V) alone, with 300 courses and 700 subjects. There was only one examination cycle for Basic Education, which was at the end of the year, with 58 subjects across the system and this showed the nature and magnitude of what the system was carrying at the moment. It was important to highlight that a new qualification had been introduced in 2007 in the form of the NC (V), which would also be resulted and certified at national level. However, this system had been introduced without having its own dedicated IT system in order to avoid this major backlog and the problem was that the current system and programme was out-dated. There was a lot that was being done at present to upgrade the system to provide assurance that the problem would be sustainably eradicated.

The Department had engaged with SITA in trying to get the system ratified and fixed, and there had been engagements with the Minister which had culminated in a Cabinet memo in 2012. This had allowed the Department to procure an IT system to address this systemic problem in an efficient and effective manner. Unfortunately, the legislation did not allow the Department to procure an IT system by itself, as it had to be done through SITA, and there had been engagement on processes and timelines in order to procure the IT system that would be suitable to address the backlog. The country with a similar system to South Africa was India in respect of the manner in which the certificates were delivered. The Department had been interacting with various tertiary institutions and potential employers in order to address this problem, hence the provision of supporting letters and necessary verifications so as to ensure that young people were not being denied access to opportunities. The Department was still working on the appointment of a service provider, and the process would be finalised soon. There was a concerted effort to ensure that the problems in the current system would not be replicated in the new system. The new service provider would be under the control of the Department, as ordered by the Cabinet. He wanted to assure the Members that the Department was doing whatever was humanly possible to address the problem.

Ms Pote said that regarding the issue of capacity at SITA, both the Department and the Department of Basic Education (DBE) were serviced by SITA and it was the same team that was doing the programming and IT systems for examinations and issuing of certificates for both Departments. Capacity was still identified as one of the major challenges, including the length of time it took to fix the consolidated problem. The Department had issued information in various colleges for students with outstanding original certificates, in order to get temporary supporting letters and necessary verifications. The letter made it clear that “the lack of certificate was not the doing of the candidate but it was a systemic problem in the Department”. Colleges had cooperated with the Department to ensure that all the affected students were able to receive these letters. The Department had received feedback in the last 48 hours from the officials who worked on the certification data indicating that they had tested the functionality of combining the results across cycles and found it to be working, and Umalusi was testing one of those datasets. The Committee would be provided with the plan and timeframe for addressing the problem.

The issuing of the letter was a manual process and the Department was not using the system to get the consolidated datasets in that situation. That was the reason it was  much easier to issue a letter than doing the actual certification in the system. The Department usually intervened in cases where the tertiary institution would eventually demand the original copy while the Department was still battling with the major backlog. There had been similar interventions at the University of South Africa (UNISA) and other universities.  

The Chairperson indicated that the issuing of supporting letters to students should be regarded as a temporary and interim solution, as some people had been dismissed from work after the failure to produce the original copy of certificate.

Dr Bozzoli again suggested that SITA should be urgently invited to the Committee in order to provide a timeline for addressing the problem of the backlog in the issuing of certificates. Was there funding already for the new system to be put in place? She wanted to know whether there was any specific reason for having so many subjects, considering that Umalusi had already indicated that quality control in many of the courses was not satisfactory anyway. The Department should look at the streamlining of the problem so as to address the backlog in the certification process and quality problem. 

Mr Siwela wanted to know the number of programmes that were offered in the NC (V) and the subjects in each programme, as 700 subjects sounded like too much.

The Chairperson wanted to know about progress in the procurement process, as this problem needed to be solved once and for all. There was still no concrete strategy that had been put in place to address the problem, as it had been indicated that the problem had been accumulating since 2007 and this was really too long. It was very concerning that about 70% of young people were unemployed and this problem was likely to be compounded by this delay in the issuing of certificates. One should avoid a situation where students would resort to protests in order for the problem to be resolved. There should be a concerted effort to reduce the scourge of youth unemployment in the country and the government had made it clear that it was only education which could unlock opportunities for young people. 

Ms Kilian supported the suggestion to invite SITA to the Committee to provide a timeline for addressing the problem. She wanted to know whether there was a way that the system that had been used by the Department could be made simpler and flexible, as it was currently complicated.

Mr Qonde responded that the Minster had sent a task team to review the NC (V) and that task team had gone through the exercise and made recommendations which the Minister had adopted. The recommendations had been sent to Umalusi for the purposes of pronouncing on a policy regime, and the target date was 2016. There was a task team which was currently working at reviewing the NC (V) examinations regime and the Department would get the feedback in trying to look at the matter so that the system became more user-friendly, effective and efficient to deal with all the complexities. The Department was trying to push with high speed to ensure that the problem was rectified as soon as possible. There was funding for the new system and it was adequately budgeted for at a cost of R54 million. The bid adjudication committee of the Department was still finalising the process of procuring a new service provider, and this should be wrapped up very soon as there were no any unforeseen inconveniences.   

Ms Pote said that in the NC (V), there were currently 21 programmes on offer across the three levels, which translated into just under 300 subjects, and the other 300 to 400 subjects were N1 to N6 programmes (engineering studies) and N4 to N6 (business studies).

The Chairperson asked whether there was any possibility of separating the functions of SITA in dealing with the examinations of Basic Education and Higher Education and Training, so as to offload the system and fast-track the process.

Mr Qonde responded that in the current processes, the Department had demanded SITA to ensure that there was capacity in place to deal with the separation in the processing and issuing of certificates for Basic Education and Higher Education and Training. However, this was not within the competencies of the Department to ensure that those things were being done as it was still SITA’s responsibility to make the final decision.

Ms Mchunu wanted to know how the results of the students were loaded into the IT system of SITA and whether the challenges that were currently being experienced are the result of human, technical or programme errors.

Dr Bheki Mahlobo, DDG: VCET;  responded that computers should be programmed by capable programmers and without having capable programmers, it became even more difficult to have a stable and efficient system in place. The challenge at SITA was firstly limited to human capacity and the simple lack of programming capabilities. What had happened in the past was that the fixing of one technical error had created another problem. Therefore, in essence the problem was both one of human capacity and a lack of technical expertise.

The Chairperson said that from her personal experience, what may further delay the procurement process was personal interest, and she urged the Department to ensure that this did not stall the whole process.

Mr Qonde assured the Committee that due diligence would be carried out thoroughly to ensure that the procurement process met the requirements. 

Mr Kunene recommended that the perhaps the Department needed to invite countries like India and China, which had been able to deliver the certificates to students on time, despite their sizeable population. It was disappointing that the Department was producing computer engineers, but there was still mention of a lack of capacity at SITA. 

Mr Mbatha suggested that Mr Siwela needed to read an interesting inquiry that had just been established which showed the calculations on both the subject and the programme and the content of the subjects as this could raise some interesting ideas.

Ms Kilian said that Members were giving the Department a grilling on the matter merely because they were very passionate about the future of young people. SITA was responsible for the procurement of IT systems, and they were reporting to the Department of Telecommunications and Postal Services (DTPS). To illustrate the deep-seated problem at SITA, the entity had been sent back by the Portfolio Committee on Telecommunications and Postal Services, as they had presented a “copy and paste” of last year’s strategic plan and annual performance plan (APP). The Committee needed to invite the whole executive of SITA and relevant Ministers, including the DG of the DTPS, so as to be given a timeline for resolving the issue of the backlog in the issuing of certificates.

The Chairperson noted the suggestion that had been made by Ms Kilian, and then requested the next presentation by Umalusi.                                         
Council for Quality Assurance in General Further Education and Training (Umalusi): Briefing

Dr Mafu Rakometsi, Chief Executive Officer, Umalusi, explained how the Council was related to the Department. In 2009, after the split of the Department of Education into two departments, the President had issued a proclamation that Umalusi must report to the DHET on the National Qualification Statement (NQS) with the South African Qualification Authority (SAQA). However, when it came to the strategic plan, budget allocation and annual reports, the Council then reported to the Department of Basic Education (DBE).  However, it served both Departments when it came to quality assurance. The responsibility borne by Umalusi on the DHET was huge, which was why reporting to the Committee was critically important. There were about 825 479 learners for the National Technical Education (NATED) programmes, about 1.47 million students for NC (V) and about 398 000 students for the General Education and Training Certificate (GETC). It was important to amplify the importance of the relationship of the Council with the Committee and the Department -- the Council interacted often with the DG of the DHET, and this relationship was important to ensure that there was coordination over ways to deal with irregularities and leakages in various colleges around the country.      

Ms Faith Ramotlhale, Senior Manager: Quality Assurance of Assessment, Umalusi, said that the moderation of question papers was to ensure that the question papers were of the required standard and were relatively, fair, reliable and representative of an adequate sample of the curriculum. The criteria for the moderation of the question papers were to look at cognitive demands, marking memorandums/guidelines, language and predictability. The approval of GETC question papers had been 54% from the first moderation and 46% from the second and third moderation. With the Independent Examination Board (IEB): GETC, approval was 75% from the first moderation and 25% from the second moderation. The areas of good practice in the moderation of the question papers included no question paper leakages, questions were not predictable, and showed creativity and innovation. The setting of an extra set of papers, the first stage towards an eighteen month cycle, was a step in the right direction (DHET-NC(V)), and this was to ensure equivalence of standards. The November, March and back-up examination question papers were moderated simultaneously.

Areas of concern in the moderation of question papers highlighted the need for an improvement of internal moderation for the DHET, IEB and Benchmark (BM) tests. Examiners and internal moderators must be trained to enhance their understanding of subject and assessment guidelines. There had been poor quality memoranda, and an over-reliance on the memorandum discussion process to improve the quality of marking guidelines. It was also noted that the question papers for A4TECH and A4HSSC did not meet the cognitive demands at first moderation, and correct versions of question papers, marking guidelines, internal moderator reports and assessment frameworks had to be made available for external moderation. The turnaround time for the resubmission of rejected question papers to Umalusi had to be improved, while out-dated or under-specified curricula still required urgent revision.

Ms Ramotlhale stated that internal assessment referred to any assessment conducted by the provider, the outcome of which counted towards the achievement of the qualifications. For the GETC, it was 50%; for NC (V) fundamental subjects, 25%; for NC (V) vocational subjects 50%; and for NATED subjects, 40%. The internal assessment was to ascertain the degree to which the assessment body was attempting to ensure standardisation across providers and the standard and quality of the tasks. The internal assessment was also important as it assisted in establishing the extent and quality of internal moderation and feedback, and verified the reliability and validity of the assessment outcomes. There were a number of good practices in the task of internal assessment, and these included a year-on-year improvement regarding the administration compliance requirements and the School-Based Assessment (SBA) user guides developed by the IEB were helpful in structuring the content of Common Assessment Tasks (CATs).

There were areas of concern that still needed to be addressed. These included the fact that there was  no system in place to monitor the implementation of the CATs, and the CATs had not adequately covered the specific assessment criteria as specified in the unit standards at first moderation of BM. The Council also expressed concern that the weightings of the CATs also did not comply with the Subject Assessment Guidelines (SAG) requirements, the centres had not submitted educator portfolios with the sample requested, there had been no evidence of assessment plans (IEB, Benchmark, DHET) and the assessors (educators) did not understand the use of rubrics, and marking guidelines (Benchmark, DHET). The directives for improvement of NC (V) focused on the following:

* Improving and deepening the understanding of the latest ICASS Guidelines (NC (V));

* Development of assignments and tasks;

* Implementation of practical assessments was still problematic;

* Implementation of analysis grids and Bloom’s taxonomy;

* Development of marking tools and rubrics;

* Assessment and moderation practices needed improvement.

She said there were key areas of concern in the monitoring of examinations, as there were often challenges because of a lack of storage rooms and inadequate security for question papers, and the transportation of exam material using public transport, such as in the Eastern Cape. There were also cases where candidates were allowed to write without proper identification and some centres kept answer scripts overnight, pointing to the issue of poor security. The areas of concern in the monitoring of examinations for the General Education and Training Certificate (GETC) pointed to a low number of candidates who wrote the exam, poor compliance with exam regulations and inadequate storage facilities for question papers, with poor security arrangements. There were also challenges in poor administration regarding exam regulations. Chief invigilators often lacked the expertise to manage exams and private exam centres often applied to the IEB to conduct the exam on alternative dates.

The area of good practice for TVET colleges was the general management of the examinations, as it was good in terms of security of papers, and the appointment and training of invigilators. However, there were still many challenges that needed to be dealt with, like irregularities during the writing phase of the examination, where some candidates were found with cell phones. Others entered the examination without identity documents or examination numbers, and at some of the colleges monitored, papers were not sent to the marking centre within 24 hours, nor securely sealed, raising the possibility of wrongdoing.

There were areas of good practices in the marking phase for GETC. These included the fact that marking centres were generally well organised and suitable for the task, and adequate and experienced security personnel were at the gates and entrances to the venues. The Council was pleased that the training of marking personnel was generally thorough and effective. However, there were also areas of bad practice, as the Council had found that there was a shortage of markers in Northern Cape and KwaZulu-Natal (KZN) due to appointing markers for more than one learning area. In some cases, the internal moderation model was different and inconsistent across marking centres, and markers in Limpopo had disturbed the training process, apparently because of outstanding remuneration issues. The IEB marking centre management plan was not well documented – it was  based on experience and practice, and there had been no security in the building where the actual marking was done to control the flow of Benchmarking scripts. The moderation of marking determined the standard and quality of marking, and ensured that marking was conducted in accordance with agreed practices. Umalusi engaged with centralised memorandum discussions and verification of marking (centralised and on-site) processes during the moderation of marking.

Ms Ramotlhale said there were areas of good practice that had been observed. It was good procedure when there were panel discussions followed by marking of exemplar scripts, and marking commenced after markers had demonstrated an acceptable level of competence. An analysis of evidence found educator portfolios were meticulous and contained all relevant documents as per policy requirements. There were also areas of concern, however, and the Council noted the trend of poor practice of pre-marking in 15/26 learning areas (LAs), discussions had been very difficult and time-consuming (DHET), and there had been a lack of pre-marking or preparing alternative responses in advance at the IEB.

The standardisation and resulting showed that according to the provisions of GENFETQA, the Council may adjust raw marks. International practice was to use large scale assessment systems. Standardisation was the process used to mitigate the effect of factors other than the learner’s knowledge and aptitude on the learner’s performance. The Council did take into consideration the sources of variability, including the difficulty of the question paper, undetected errors, and the learner’s interpretation of questions. The areas of good practice in statistical moderation pointed to the fact that the IEB and BM booklets and data sets were submitted according to the prescribed timelines, and standardisation and approval meetings went smoothly. However, there were still concerns over the late submission and inaccurate standardisation, data delaying the approval of results (DHET), late submission of qualitative input reports (DHET) and poor performance of learners, where 30-40% of candidates did not write the exam, (DHET).

In conclusion, Dr Rakometsi highlighted that the quality assurance processes were based on Umalusi criteria that were subjected to constant review and refinement, to ensure that they were in line with current trends in assessment and examinations. From the findings of the quality assurance processes, many good practices were evident but on the other hand, there were still many challenges that needed to be addressed. Significant interventions were required to improve the curriculum, teaching and learning, with exposure of learners to a compulsory practical component and assessment for the NATED programmes to make it a meaningful qualification. The out-dated and under-specified curriculum and lack of subject assessment guidelines for the NATED programmes created problems for appropriate quality assurance to be conducted.

Discussion

Dr Bozzoli labelled the Umalusi presentation as depressing and worrying, not because the Council had been doing a bad job, but because of the general state of colleges in the country. The Council seemed to have found the areas of good practice, but she wondered whether the people who had been getting the certificates in various colleges in the country had been properly trained. She expressed concern that some of the exams were not properly monitored. Some students did not even bring their identity documents (IDs) during examinations and this seemed to just encapsulate the absolutely disastrous state of colleges in the country. What had been done about the poor practices that had been noted, as Umalusi seemed like a toothless regulator? Was it possible to withdraw accreditation for a particular college that had been continuing with bad practices? Was there a benchmark for the required standard for colleges? She suggested that various colleges needed to submit to Umalusi some mitigation plans on what to do to eradicate these bad practices.

She said there had been reports in the media that the South African Democratic Teachers Union (SADTU) was opposed to the proposal that markers should be measured on whether they were qualified for that particular work. Was there a similar problem in the colleges? Was there a way of assuring the Committee that the markers in colleges had been adequately trained and capacitated? She wanted to know whether the colleges were also externally examined like universities to ensure that the training that was offered was of international best practice. It would be interesting to hear whether Germany could help the country in terms of externally examining the work and training that was offered in various colleges, in order to drastically improve the standard and quality of colleges. 

Ms Kilian also expressed concern about the number of bad practices in various colleges in the country and highlighted that there would be a need to get the whole vocational and technical sector up and running in order to be an attractive career path. There should be a response from the Department, as the presentation had touched on quality, relevance and the credibility of the certificates that were being issued. The Department would need to respond on how it planned to address the serious matters that had been highlighted in the presentation. It was obviously difficult to address some of the problems that had been identified, as the country had 50 TVET colleges and more than 200 campuses nationwide, but Umalusi would need to tell the Committee about how to assess the extent of the problem. Why was the Council still using the NATED programmes, as it had been indicated that the programme was outdated? There was a  need to benchmark the qualification against the international best practices so as to improve the standard of colleges.

Mr Siwela welcomed the balanced presentation by the Council, as it had presented both the good and the bad side of the story.  What was the Department doing to ensure that all the accredited colleges were able to pay on time to avoid the accumulation of debt? Was there a strategy in place to train and capacitate the examiners, to prevent the repetition of the same question papers each and every year? It had been indicated in the presentation that 75% of the IEB’s GETC papers got approved on the first instance of moderation, while 54% of the DHET’s GETC papers got approved. What was causing this discrepancy between the two? 

Ms Mchunu requested more information on the statement that the question papers for A4TECH and A4HSSC (Human and Social Sciences) did not meet the cognitive demands at first moderation. What were the cognitive demands that were expected? Were examiners made aware of those expectations? What was the Department doing about the fact that Umalusi said the NATED programmes had been outdated? What was the knowledge of the Department about the challenges that had been enumerated by Umalusi on SITA? What had been done by the Department to address those challenges?

Mr Mbatha said he appreciated that the Committee had invited Umalusi and wished that such interactions could happen annually. Most of the colleges had been complaining that the Council was responsible for the major delays in the issuing of certificates. What could be done to address this problem? He requested the Council to provide a breakdown of the TVET colleges that had not been able to pay their fees for certificates. The Committee should support the move by the Department to have more collaboration with the TVET colleges and provide additional funding to secure the examination environment. It was quite clear that without a secured examination environment, the problems that had been identified regarding the areas of concern in the monitoring of examinations would continue. The fact that some official staff in the TVET colleges still ran around with examination papers in private and public vehicles was a serious concern, as examination papers needed to be kept in a secured location. The priority should not be on whether young people had managed to pass the enrolled courses and programmes, but their preparedness to enter the labour market.  It was also concerning that Umalusi had identified a problem of outdated or under-specified curricula. This required urgent revision, as students needed to get genuine and relevant qualifications that could be measured against international best practices. There were university skills that could be used to provide guidelines to the TVET colleges, and the senior personnel should be deployed to oversee the operation of these colleges.

Mr Kekana said that the Council for Higher Education (CHE) had assured the Members that the quality of TVET colleges was up to the standard, but the presentation by Umalusi was telling a different story. What could be the reason for such a discrepancy?  He asked if Umalusi could assist with the capacity to deal with the backlog in the issuing of NC (V) certificates. 

Prof John Volmink, Umalusi Chairperson, responded that it was perhaps helpful to admit that there were deep problems in the sector. There were many reasons for this, and it was partly because the quality assurance process required all the stakeholders to participate in holding accountable a particular sector, from the media to the parents. The TVET sector had been flying under the radar in that regard, and there had been a lack of discussion in the media -- particularly the electronic media -- on ways to improve the status quo in TVET colleges in the country. In the interests of not creating a national crisis, Umalusi had held back on taking drastic decisions at some of the colleges that had been poorly administered. However, the Council would eventually have to take drastic action and block some of the colleges where there had been irregularities and leakages. The Council had been upfront about the key challenges that were being experienced in the sector, but the assessment bodies had been unresponsive in addressing those key challenges.

He said that the issue of the competence of markers should be looked at in the context of the competence of lecturers, as the markers were the subset of lecturers. The competence of lecturers should be called into question, as there were lecturers who were qualified to teach at the lower level but were also allowed to teach in many TVET colleges, and this was totally unacceptable.

Dr Rakometsi responded that the NATED curriculum had been withdrawn from the system and replaced by the NC (V), but there had been an outcry from industry that the NATED curriculum was needed and the Minister had decided to bring it back. The issue of outdated and unspecified curricula was very serious, especially when one considered the rapid developments in technology. For example, the mechanic who used to know the electronics of vehicles in the 1970s could not touch the current vehicle models because of the rapid changes in technology. It was important to ensure that the students were trained for modern technology and the Council needed to revise the curriculum and highlight the areas of the curriculum so that the educators would be familiar with what to teach and assess. Industry and the universities were all involved in the approval of a curriculum, and Umalusi had inherited the majority of the qualifications in the sector. However, all new qualifications were quality assured in terms of their curriculum before they were put on the qualification framework. The NATED curriculum was a qualification that had been recalled and reinstated and at the point of reinstatement, it had not gone through the same rigorous quality assurance process. He found it very strange and paradoxical that this NATED curriculum was quite popular amongst the students and employers.

The Council would indeed be forced to block or prevent the results and operation of some colleges where there were serious irregularities and this was a protocol that needed to be followed. There was a committee of Council which reported to Umalusi on areas of concern in particular colleges. He rejected the suggestion that the Council was toothless, as there had been interventions in some colleges in various provinces which had resulted in a general improvement in their operations. There had been meetings with the DG of Higher Education and Training on ways to address the areas of concern and the interventions to be made at the colleges where there had been serious irregularities.

On the issue of the competence of educators, most of the educators in the sector were not educators but people who were qualified in their various technical fields, like engineers, artisans and electricians, and they often were not familiar with the principles of teaching. The markers were appointed by the Department according to the Personnel Administrative Measures (PAM), and Umalusi monitored the Department by conducting oversight to check whether the appointment of markers was according to policy or not.

Dr Rakometsi agreed with Ms Kilian that there was a need to make the sector attractive in order to turn the economy around and integrate young people into the economy. The assessment on the extent of problem in the sector was done by the Council through the Assessment Standard Committee (ASC), with the advice of the staff who had been on the ground and monitoring the problems in the sector on behalf of the Council in order to identify the “red flashing lights” and pronounce on areas of concern. The Department had been helpful to the Council by sending the list of colleges that still owed Umalusi, and had intervened in a number of areas through the help of the DG. The Committee would be provided with a breakdown of the colleges that still owed Umalusi, with the amount that was still owed.

It was the responsibility of all assessment bodies to train the examiners and markers, and the discrepancy in the approval of the GETC papers was all due to the quality of the examiners, as they were competent on how to assess the syllabus. Each and every subject had a syllabus and subject assessment guidelines so that the examiners and internal moderators were able to use the same criteria in assessing a particular subject, and it also specified the competency levels.

Prof Volmink added that the NATED courses were completely out of date, as they were not curriculum and exam specific, and there were no subject assessment guidelines. It was now back by popular demand. There was an assumption that the NATED courses would disappear with the advent of NC (V), but there were also problems with the NC (V) itself as it did not work very well with courses such as electrical engineering.

Dr Rakometsi said that the Council appreciated the opportunity to come and present annually to the Committee, but suggested that presenting in February after the end of each year’s processes could make a lot of sense as it would be soon after the reports had been finalised. Most of the people often blamed Umalusi for the delays in the issuing of certificates, and this was caused by a lack of information on how the whole process operated. It was a great pity that even the educators were not familiar with the whole process of issuing of certificates, as the Council required reliable data in order to issue the certificates. The issue of the preparedness of the learners emphasised the point of practical tasks and the college-based assessment to determine whether students were passing the practical tasks for vocational purposes, or not. The students needed to be exposed to not just the theory, but practical tasks as well, to be prepared when entering the labour market. 

It was not for the first time that Umalusi had offered to assist to both SITA and the Department, but the Council had been ignored and unappreciated by SITA. He wanted to put it on record that the Council was again ready to offer assistance to SITA and he would be most disappointed if this offer were turned down again.

The Chairperson wanted more information on the cause for the late submission and inaccurate standardisation data, as this delayed the approval of results. What was the turnaround strategy in addressing this problem? What had been causing the late submission of qualitative input records? What was the plan in address this area of concern? The presentation by Umalusi was still not clear on who would be responsible for significant innovations that were required to improve the curriculum, teaching and learning, and the exposure of learners to a practical component of assessment.

Ms Pote said that the Department relied on SITA for all the data, whether it was certification, standardisation or registration data. All data was produced in the SITA system. The preparation of lecturers and the quality of teaching and performance was part of the turnaround strategy and the Department was still waiting for feedback on the process and curriculum for teaching and learning. Lecture development was also part of that strategy.

Mr Kekana asked whether there was any particular reason why the curriculum was still outdated and irrelevant, despite the involvement of industry and all stakeholders in determining the curriculum. It was impossible to compete anywhere in the world while still using an outdated curriculum and the Department needed to ensure that the curriculum was updated all the time. 

The Chairperson commented that it was risky to travel with examination papers on public transport, and the Department needed to come up with a strategy to address this problem. How were the Chief Invigilators appointed to manage the exams? Was there any form of training that was offered to the chief invigilators? What was the reason for some of the papers not being sent to the marking centres within 24 hours, as this increased the risk of wrongdoing?

Ms Pote responded that in terms of the curriculum framework, the NATED curriculum was very scant in the sense that it did not offer the breadth and depth of the content of the subject. The Department was initiating a process of compiling content frameworks so that the lecturers and the candidates could know exactly what was to be taught and what had to be learned and assessed, in order to add some substance. There was a unit that had been responsible for the training of chief invigilators and administrators of exams, and this had resulted in some improvement in the conduct management and administration of examinations on the whole, although there were still some challenges. There had also been some improvements in invigilation and the Department would address the issue of incorrect data that was often provided by various colleges and the invigilation processes. The Department did give the colleges a sample on their state of readiness for examinations, and this was often taken into consideration and followed by site visits.

She said that the provinces were responsible for managing and conducting the examinations for the  GETC, and not the national Department, as the Department would be taking over that competence only from 1 April 2017. The Department would ensure that public transport was not being used for the delivery of examination papers. In terms of the retrieval of papers after the writing of an examination session, the Department had changed the delivery model, as the provinces collected the GETC scripts every day, and depended on the model that was used in that particular province. However, there had been changes in the delivery and retrieval model in the college sector, as all scripts had to be returned to the delivery point on the same day that the examination was being written so as to eliminate the irregularities and leakages that had been identified.

Prof Volmink concluded by appreciating the opportunity to share the mandate and the work that had been done by Umalusi and the kind of challenges that still needed to be addressed. There was a need for a change of curriculum for TVET colleges, and the NC (V) was the modern curriculum although it might not be perfect, as it been subjected to criticism from various stakeholders. There were still many others who had vested interests in retaining the old outdated system, and this might be due to areas of competency or just general comfort with the status quo. There were three words that described quality assurance as far as Umalusi was concerned -- moderation, monitoring and standardisation -- and the issue of certification came after a very long chain of quality assurance processes.

The Chairperson thanked the Department and Umalusi for their presentation and appreciated the input from the Members and the responses that had been provided. The issue of SITA needed to be taken into consideration, as the Committee needed to agree on a date to invite SITA and the DG of the DTPS.

Adoption of minutes    

Dr Bozzoli moved the adoption of the minutes of 20 May 2015, and was seconded by Mr Siwela.

The minutes were adopted as is.

Ms M Nkadimeng (ANC) moved the adoption of the minutes of 27 May 2015, and Dr Bozzoli seconded.

The minutes were adopted as is.

The meeting was adjourned.                                                       

Share this page: