2013 National Senior Certificate Results: briefing by Department and Umalusi, with Minister

Basic Education

17 February 2014
Chairperson: Ms H Malgas (ANC)
Share this page:

Meeting Summary

The Committee was briefed by Umalusi and the Department of Basic Education (DBE) on the Quality Assurance of the National Senior Certificate (NSC).

The Minister said that a great number of children drop out in Grades 11 and 12. It saddened her that 30% of Grade 12 students dropped out, given that they are survivors. This can improve, with the Department placing all its energy into the final implementation of the Curriculum Assessment Policy Statements (CAPS) this year. 

The report presented by Umalusi to the Committee highlighted areas of commendation and areas of concern pertaining to the results of the 2013 National Senior Certificate.   Moderation of internal assessments indicated that areas of good practice stemmed from good systems in place for school-based assessment (SBA) implementation and support of their educators, as well as the good quality of tasks set out by Independent Examination Board (IEB) educators. Areas of concern were related to how rubrics were developed, as there was an element of vagueness and subjectivity in criteria leading to differences in interpretation by educators, culminating in differences in the allocation of marks. Furthermore, moderation at the school level could also be improved.

Specific concerns regarding the quality assurance of the NSC examination and assessment relate to the delays in submission of question papers. While there had been some improvements in the administration of the SBA and presentation of learner performance, some areas of concern were highlighted. They include the unavailability of internal moderation reports, lack of constructive feedback given to learners after moderation, and teachers still being challenged in the development of tasks at the appropriate cognitive levels.  Assessment of practical investigations, research projects, assignments and simulations also remain a major problem. Furthermore, the use and development of rubrics remain vague, while assessment of Physical Education Task (PET) in Life Orientation continues to be a problem, through the inflation of marks. Lastly, a number of challenges were reported regarding the standard and quality of marking.  Umalusi urged the DBE to look closely into the appointment of markers. 

Umalusi were in general, pleased with the manner in which the 2013 NSC examination was administered. Umalusi further acknowledged that a number of irregularities were reported and that these were addressed appropriately, and thanked the national and provincial government for their concerted efforts in ensuring credible examinations.

The DBE told the Committee that the annual national assessment (ANA) had become a corner stone of the action plan to improve the sector.  The ANA had become one of the flagship programmes in terms of reflecting what had been achieved, lessons to be learnt and challenges that confronted them as a Department. ANA is uniquely South African -- home bred and home grown -- and serves the needs and stage at which South Africa is, in terms of development.  ANA serves as a diagnostic tool but at the same time, this diagnostic tool is used to comment on the performance of the whole system.  ANA is in a process of development and as the process unfolds, the DBE will be able to address the two critical purposes of ANA accurately. In order to ensure that the Department is not reporting on itself, it has ensured that there is an independent verification of the ANA.

If one compared the national senior certificate results of 2012 and 2013, there had been an improvement across all the grades, except for Grade 1. The Department was very concerned about Grade 9, which is the problematic grade.  This was an area which they were focusing on to make sure that the challenges were addressed decisively.

During discussion, Umalusi was asked what had been done to rectify marking, as this compromised the results. Had they sat down with the Department, indicated what had been found and asked what the Department was prepared to do to rectify this?  What had Umalusi recommended to the Department?  The DBE was asked if there was some programme to help those Grade 12 learners that had failed.  A Member suggested that there was a need to debate the teacher to student ratio.  It was also commented that  skills were needed that would change the country, not just Bachelor degree passes.
 

Meeting report

The Chairperson welcomed all Members and invited Ms Angie Motshekga, Minister of Basic Education to address the meeting.

Remarks by Minister
Ms Moshekga thanked the Portfolio Committee and set out to explain the reinstatement of the Director General at the Department of Basic Education (DBE), Mr Bobby Soobrayan, following certain allegations against him. The judge on the case, Justice Willem van der Merwe, had investigated these allegations and the report was placed on the website. It cleared all the allegations against the Director General (DG) and a letter was sent to him requesting him to return to his position.

With regard to the senior certificate results, she said that a great number of children drop out in Grades 11 and 12. It saddened her that 30% of Grade 12 students dropped out, given that they are survivors. This can improve, with the Department placing all its energy into the final implementation of the Curriculum Assessment Policy Statements (CAPS) this year.  The high schools will be piloting CAPS through a very able team.

Ms Moshega then opened the floor to the Mr Bobby Soobrayan, Director General of the Department of Basic Education.

Remarks by DG
Mr Soobrayan thanked the Committee for his reinstatement, and said that he was up to date with developments of the Department and was ready to begin work. The floor was then opened to the representatives of Umalusi.

Umalusi Report on the Quality Assurance of the NSC
Professor Sizwe Mabizela, Chair of Council of Umalusi, thanked to the Committee for the invitation to report on the NSC results, and expressed his appreciation to the Department for the strategy meeting held in December 2013.

Dr Mafu Rakometsi, CEO of Umalusi, continued with the presentation. He explained that Umalusi received its mandate from the National Qualifications Framework (NQF) Act, Section 27(h). The Act guides the quality assurance of learners’ achievements through a regulatory framework.

The approval for the release of the results was based on certain requirements. The first was that examinations were conducted in compliance with applicable policies regulating the conduct and administration of the examinations. The second requirement was that at the time of approval, there were no serious irregularities that could undermine the credibility of the examinations.

Mr Emmanuel Sibanda, Acting Senior Manager of Umalusi, said that papers were moderated to ensure that they were at the required standard and to ensure that all papers were relatively fair, reliable, representative of an adequate sample of the curriculum, and representative of relevant conceptual domains. Key areas of concern with respect to the moderation of papers were problems in adhering to timeframes, and the impact of this on the quality of setting and moderation, while some question papers required more than four moderations. 

With respect to internal moderation, he said that Umalusi appoints moderators and subject specialists to carry out internal assessment. The purpose of this internal assessment by Umalusi was to ensure rigour and appropriateness of the DBE moderation process, while ascertaining the extent to which bodies and provinces were ensuring standardisation across the board.  Furthermore, internal moderation seeks to ascertain the standard and quality of the tasks, to determine the extent and quality of internal moderation and feedback and lastly, to determine the reliability and validity of the assessment outcomes.

Key areas of concern regarding moderation were that much of the moderation process was on compliance, rather than quality of assessment. Teachers were unable to develop tasks pitched at the appropriate cognitive levels – the focus was more on lower cognitive levels. Assessments of practical investigation, research projects, assignments and simulations were still a major problem. The use and development of rubrics remained problematic.

Monitoring of examinations was assessed on the “state of readiness” by the DBE, the manner in which examinations were conducted, and marking. To ascertain the “state of readiness,” the findings by Umalusi indicated that all provinces had working examination systems in place, but areas of concern related to certain provincial Departments of Education (PDEs) that needed to closely monitor the printing of question papers.  Furthermore, there was an inefficient, or lack of, proper coordination with districts regarding exam-related processes.

Verification of marking determines the standards and quality of marking, in compliance with the agreed practices. Umalusi engaged in pre-marking and memorandum discussion, and moderation of marking. Areas of concern were related to the inability of some provinces to conduct pre-marking, while the time between the examination dates and the memo discussions were generally too short to enable pre-marking to occur. There are a total of 6 699 exam centres, and 159 were monitored by Umalusi with a total number of 36 monitors. 

Moderation of internal assessments indicated that areas of good practice stemmed from good systems in place for school-based assessment (SBA) implementation and support of their educators, as well as the good quality of tasks set out by Independent Examination Board (IEB) educators. Areas of concern were related to how rubrics were developed, as there was an element of vagueness and subjectivity in criteria leading to differences in interpretation by educators, culminating in differences in the allocation of marks. Furthermore, moderation at the school level could also be improved. During the monitoring of the writing phase, there were 187 exam centres and 22 were monitored by Umalusi, with 9 Umalusi staff deployed for IEB monitoring.
 
The findings of Umalusi state that the IEB conducts memo discussions in the presence of sub-examiners/senior markers, who are responsible for groups of markers during marking. The internal moderator and chief examiner ratify any additions made to the final memo. The marking was found to be fair, valid and reliable and can be attributed to the continuous feedback sessions between the chief examiner, senior sub-examiners and sub-examiners.

The extent to which standardization occurred was as follows: 62 subjects were standardised, “raw” marks for 47 subjects were accepted, two subjects were moderated upwards and 13 subjects were moderated downwards.

Specific concerns regarding the quality assurance of the NSC examination and assessment relate to the delays in submission of question papers. While there had been some improvements in the administration of the SBA and presentation of learner performance, some areas of concern were highlighted. They include the unavailability of internal moderation reports, lack of constructive feedback given to learners after moderation, and teachers still being challenged in the development of tasks at the appropriate cognitive levels.  Assessment of practical investigations, research projects, assignments and simulations also remain a major problem. Furthermore, the use and development of rubrics remain vague, while assessment of Physical Education Task (PET) in Life Orientation continues to be a problem, through the inflation of marks. Lastly, a number of challenges were reported regarding the standard and quality of marking.  Umalusi urged the DBE to look closely into the appointment of markers.  

Umalusi were in general, pleased with the manner in which the 2013 NSC examination was administered. Umalusi further acknowledged that a number of irregularities were reported and that these were addressed appropriately, and thanked the national and provincial government for their concerted efforts in ensuring credible examinations.

Department of Basic Education Report on 2013 National Senior Certificate
Mr Soobrayan said that the assessment of the National Senior Certificate served as a significant diagnostic tool. When the NSC results were very low in 2009, the Minister announced in her speech that as a system, they should not focus only Grade 12 as an exit exam. The problems manifested in the Grade 12 results happened early on in the year and because of that, the annual national assessment (ANA) became a corner stone of the action plan to improve the sector. It is with that background in mind that the results in the presentation must be assessed, in conjunction with the National Senior Certificate. Furthermore, the presentation pointed to significant challenges -- as well as the strategies that were put in place to mitigate those challenges.     

Dr Rufus Poliah, Chief Director: National Examinations and Assesments, DBE, said that the presentation would focus on the key trends and findings in terms of the ANA results and NSC. As a Department, they were acutely aware that assessments of examinations are the mirror to performance. In terms of ANA, this had become one of the flagship programmes in terms of reflecting what had been achieved, lessons to be learnt and challenges that confronted them as a Department. ANA is uniquely South African -- home bred and home grown -- and serves the needs and stage at which South Africa is, in terms of development.

ANA is complicated because it serves two purposes. Firstly, it serves as a diagnostic tool but at the same time, this diagnostic tool is used to comment on the performance of the whole system.  ANA is in a process of development and as the process unfolds, the DBE will be able to address the two critical purposes of ANA accurately. In order to ensure that the Department is not reporting on itself, it has ensured that there is an independent verification of the ANA. There were 125 schools per province that were monitored by an independent agent, who monitored testing administration, marked the test independently, captured and processed the marks, and the results were reported on independently. The final report -- data coming from the independent verification and the data coming from the ANA administered by the Department -- were very congruent. Of the various credibility measures put in place, this test is the critical one. In order to get as close as possible to what tests should be, they had international institutions help them with the test development process.  There were still limitations, but there concerted efforts were made to address each of these limitations.

If one compared the results of 2012 to 2013, except for Grade 1, there had been an improvement across all the grades. The Department was very concerned about Grade 9 which is the problematic grade.  That is an area which they were focusing on in terms of making sure that the problems were addressed decisively.

Mr Poliah emphasised that one must focus not only on average performance, because average performance reflected the performance of the group, but also on the number of learners that were performing above 50%. In terms of learners in Grade 3 that performed above 20%, there had been an improvement from just over 35% in 2012 to close to 60% in 2013. In terms of Grade 6, there was an improvement from just over 10% in 2012 to over 25% in 2013 for mathematics.  However, with respect to Grade 9, it could be observed that there had been no improvement from 2012 to 2013 in the number of learners obtaining over 50% for mathematics. For the first additional language, an improvement from 2012 to 2013 could be observed for Grades 4, 5 and 6, with Grade 9 indicating a decrease in performance.

Performance of the provinces for Grade 3 mathematics indicated that there has been an improvement across all provinces. The performance of provinces with Grade 3 language indicated that the national average had dropped, with only one province, KwaZulu-Natal doing marginally better in 2013. However, the national average had dropped from 52% to 50.8% in 2013 for the Grade 3 average language percentage mark.  Grade 6 average performance in mathematics has increased across all the provinces, with the national percentage increasing from 26.7% to 39%. The national acceptable level of achievement with regard to Grade 6 mathematics had increased from 10.6% to 26.5%. Grade 6 first additional language marks also showed an improvement across all provinces, with the national average increasing from 35.6% in 2012, to 45.7% in 2013.  An improvement could also be observed in the Grade 6 first additional language acceptable achievement. In terms of Grade 9 mathematics, a marginal increase in performance could be seen in only one province.  In terms of the acceptable achievement of Grade 9 mathematics, there had been a marginal increase in the national average.  The achievement levels of Grade 9 in mathematics indicated disappointing results across all provinces.  In terms of gender, girls in Grades 3,6 and 9 had all outperformed the boys.

Mr Suren Govender, Chief Director: Curriculum, DBE, looked at the way forward from the current results. The DBE was mindful as a sector to the initiatives that were successful in 2013.  In preparation for 2014 plans, a province by province analysis was presented to the Department to indicate what worked and what needed to be adjusted for 2014 to identify best practice.  Improvement plans had worked well in 2013 and were crafted at various levels, but had been adjusted to incorporate a management plan with objectives and time frames, so that the improvement plans could be finalised and the quality assured so that they could be implemented.

Another best practice action was the Minister’s personal engagement with all District Directors within the country. During the course of these engagements, progress reports were presented directly to the Minister, with areas of concern and areas of success highlighted.  Challenges were identified and remedies were implemented. The Department found that the direct level of accountability and engagement between the Minister and District Directors, who are at the forefront of service delivery, and senior officials played an important role in the manner in which results improved in 2013. Partnerships had been enhanced in 2013, and it is the DBE’s intention that these are strengthened further.

As a sector, the DBE sustains CAPS as the main mode of delivering teaching and learning in the country. The specific focus areas of teaching and learning begin with the National Strategy for Learner Attainment, which is the overriding framework for learner improvement in the country. It is a document that integrates the various sectors which require intervention and improvement.  Three core issues are LMST (Language, Maths, Science and Technology), CAPS and ECD (Early Childhood Development). 

For the 2014 interventions, two documents have been highlighted as important. The first is the Annual National Assessment diagnostic report, and the second is the ANA framework for improvement. The ANA framework for improvement identifies key areas and key weaknesses arising from the execution of exams in 2013. It takes each area of weakness and suggests the corrective measures that can be implemented. The ANA framework would be customised to serve the specific individual needs at each level. National time frames have been set out for each of the deliverables, as well as monitoring and support to ensure the framework is implemented.

The DBE has focused on key areas for ECD. With regard to the senior phase, there is a specific focus on mathematics, as well as career choices. In the Further Education and Training (FET) band, there are five key focus areas.  In FET, the DBE wants to make sure that English is given prominence in 2014.  Reasons for retention were presented, as well as actions that can be taken to further retention. Key issues for 2014 are to ascertain what can be done to turn around the sector, and to ensure implementation and accountability.

Discussion
Mr D Smiles (DA) asked Umalusi what had been done to rectify marking, as this compromised the results. He asked if they had sat down with the Department, indicated what had been found and asked what the Department was prepared to do to rectify this.  What had Umalusi recommended to the Department?  His second question, directed to the DBE, asked whether there is a directive for the Department to tell the provinces what should be done regarding Grade 9.

Ms J Ngubeni-Maluleka (ANC) wanted to know from the DBE if there was some programme to help those Grade 12 learners that had failed.

Mr A Mpontshane (IFP) asked who would be implementing these initiatives, and what would change in their implementation.

Ms F Mushwana (ANC) said that there was a need to debate the teacher to student ratio. With respect to skills, skills were needed that would change the country, not just Bachelor degree passes.

Ms N Gina (ANC) wanted to know what needed to be done to improve the schools, and what lessons were learnt from the adjustments.

Mr Rakometsi responded to Mr Smiles and said that the weaknesses found by Umalusi had been discussed with the Department and the DG -- all the issues of concern were raised with them.  Umalusi also indicated that the weaknesses had been identified and the DBE had been duly notified.  They had therefore been requested to come up with plans to illustrate how these weaknesses would be addressed. The deadline for these plans had been set for 30 March 2014.  Umalusi had indicated that markers must be competent and demonstrate subject content.

He then answered the question posed by Ms Gina. He said that examination memos were discussed thoroughly and there ought to be proper consultation with markers, to ensure they knew when answers were correct. They should further consult with moderators, and issues ought to be dealt with at the memo discussion meetings. Furthermore, if the markers were competent and had subject content and knowledge, then these questions should not arise. The issue of adjustments were based on the qualitative report and the trends.

Professor Mabizela said that the issue of markers was an important one, but a blame game should not be played. Teachers can teach only what they know, and society as a whole has to accept responsibility for teacher quality. He requested that the DBE work with teachers to improve their subject content knowledge.

Mr Soobrayan said that assessment is not an exact science. It is subjective and every issue and concern raised by Umalusi could not be addressed in one year. Monitoring has to be strengthened, as well as teacher assessment. The issue around Grade 9 illustrates that there is a problem in the intermediate and senior phase. It is the quality of the assessment task designed by the teacher which is problematic. The issue now is to provide good assessment tasks. The Department has also provided teachers with examples of learner evidence of what constitutes outstanding performance, mediocre performance and what constitutes a failure. Teachers are then able to recognise what constitutes good performance, average performance and poor performance. The DBE are, in essence, helping teachers with the marking that is required to go with the assessment. In terms of moderation, the problem is to get moderation systems in place in schools.

Mr Govender responded that subject advisors are going to prioritise school-based assessments, as per CAPS regulations. The teaching development component targets SBA as an area of priority. As a sector, there is a national strategy for the improvement of mathematics and science.  The strategy in itself is unable to overcome the challenges, therefore an implementation plan has been developed. The implementation plan also takes into consideration the recommendations that were made by the ministerial task team, some of which speak specifically to the challenges of Grade 9. There was another ministerial meeting in which recommendations were made and considered and filtered into the implementation plan. The Department’s strategies and plans are insufficient to gauge the outcomes they want to achieve, so there is a need for a maths and science round table to be convened.  A date has been set for 18 March, and all stakeholders will be present.

Meeting adjourned
 

Share this page: