Matric results 2011: briefing by Department of Basic Education and Umalusi

Meeting report

Opening Remarks
The Chairperson commented that although the overall National Senior Certificate results reached the benchmark of 70%, with Gauteng and the Western Cape provinces achieving the highest marks, there were a number of provinces that did not reach the 70% benchmark. However, the Committee was pleased with the announcement by Dr Sizwe Mabuzela, Chair of Umalusi Council, that the NSC results were valid, fair and credible, with no compromises.

Department of Basic Education on the National Senior Certificate Examinations 2011
Mr SG (Paddy) Padayachee, DBE Acting Deputy Director-General, said that he would like to place on record that the success of the 2011 NSC examinations and the pass rate, as well as numerous other NSC achievements, could not have been achieved without the effort of the Grade 12 learners, their parents, school teachers and provincial colleagues. DBE was only one part of the process in place to ensure improved quality of learning.

Dr Rufus Poliah, DBE Chief Director: National Examinations and Assessment, said that due to previous interaction with the Committee, only the important findings of the National Senior Certificate (NSC) would be outlined. The release of the 2011 NSC results had attracted immense public interest and this was very encouraging for South Africa. NSC results remained the most important indicator of performance of the schooling system.

The reason for the drop in the number of Grade 12 enrolments was of concern was being investigated. The possible reasons for the decrease in the numbers of learners qualifying for admissions to Grade 12 appeared to be that a standard for Grade 10, 11 and 12 levels had been introduced with implementation of the NSC in 2008. Grade 10 needed to be in sync with Grade 11 and 12. It was important to revisit the requirements for the NSC compared to the SC and try to determine the causes for the drop in Grade 12 enrolments.

The perception was that the standard had dropped due to the NSC, but in fact, standards were comparable, or slightly higher with the NSC. The NSC required a seven subject offering, with Life Orientation and Maths/Maths Lit being compulsory subjects, while SC had required a six subject offering. In order to pass the SC, students had to achieve 40% on higher grade, 33.3% on standard grade and 25% on lower grade (converted pass) – provided that an aggregate of 720 was obtained. This aggregate was made redundant with introduction of the NSC, which required that 40% had to be achieved for Home Language and a further two subjects and 30% had to be achieved for three other subjects. In terms of bachelors in the SC, candidates had to pass four subjects at 40% and 2 subjects at 33.3%. For the bachelors in the NSC, candidates had to pass four subjects at 40% and the remaining subjects at 30 % (home language at 40% or above). All subjects offered for the NSC were at one level, equivalent to higher grade.

Although the number of admissions to the NSC had decreased, the number of distinctions in accounting, business studies and English (1^st additional language) increased significantly and there was a significant drop in the distinction rate of Maths, Maths Literacy and Life Sciences.

140 584 learners achieved diploma admission (28%) and 120 767 learners achieved bachelor admission (24%). Thus 52% of candidates qualified for admission to higher education. In the short term, while the Free State and Mpumalanga provinces exceeded their bachelor target for 2011, the national figure was underachieved by 16 000. DBE took serious note of this deficit and focus would be on each individual province, with the resultant expectation that the national figure would then take care of itself. The number of bachelor admissions in the Eastern Cape dropped even though the number of learners who wrote in that province increased. Factors such as enrolment and previous targets not being met needed to be considered when setting targets for 2012 to 2014.

A total of 881 learners with Special Needs (five subjects for matric) wrote the NSC and 663 learners passed the NSC. DBE aimed to increase uptake in numbers in that regard.

The focus for DBE in 2012 would be on the 15 districts which achieved results below 60%. Six of these districts were located in the Eastern Cape.

DBE had introduced the Diagnostic Subject Report as a diagnostic tool to indicate specific areas of weakness and provide understanding of the nature of responses associated with each subject. From a range of districts, 100 scripts were randomly selected from 123 marking centres to cover low, medium and high scores and the report was compiled based on markers’ comments. General findings were that although there were improvements in the quality of responses of candidates in most subjects, there was general inability to answer questions assessing higher order thinking skills; inadequacies relating to foundational competency and basic concepts; and inadequate literacy and numeracy skills. Improvement in foundation level skills would improve the learner outcome drastically.

In conclusion, DBE would endeavour to set a new benchmark for attainment of beyond the 70% benchmark. An improvement in five gateway subjects meant that all eleven subjects could be improved. Focus would be on the under-performing districts with specific attention on mathematics, physical science, life sciences, economics and accounting.

The Umalusi report was provided to the Minister before release of the NSC results.

Discussion
Mr K Dikobo (Azapo) asked for clarification on what was meant by one higher grade level for the NSC as compared with higher and standard grade in the SC.

Dr Poliah replied that the comparison was complicated as it was not about comparing apples with apples. In the past, there were two streams of learners. It was DBE’s intent to develop one common standard to meet the standard of the learners and for the question paper to be internationally comparable. DBE was encouraged that the Scottish Qualifying Authority as well as Cambridge and New South Wales Examining Bodies had authorized the DBE Maths paper as internationally comparable.

Mr D Smiles (ANC) asked what was being done to ensure that there was no doubt at all on the quality of the NSC. In the general findings, with the inability to read and comprehend higher learning questions, it could not be assumed that the NSC was a better certificate than the SC.

Dr Poliah replied that both the NSC and SC were comparable in quality, with NSC possibly yielding a higher standard. The real test of the quality of the NSC was in its implementation – that it produced the desired quality and standard in terms of the curriculum document.

Mr Dikobo asked what the reason was for DBE wanting to reduce the number of marking centres.

Ms N Gina (ANC) said that she was also concerned as to why the number of marking centres would be reduced.

Dr Poliah replied that the issue was not about the large number of marking centres, but about the aim for marking of subjects in South Africa to be controlled on one screen, irrespective of where the marker was situated. Currently there were nine provincial centres with strong controls and various options for marking management.

Ms Gina said that while there was an effort to improve the quality of marking through competency tests, she felt that all Grade 12 teachers - not only the NSC markers- should be trained adequately and required to pass the competency test. In turn, it could identify shortcomings and be an instrument to establish where skills were lacking to produce the results required.

Mr Dikobo asked who would mark the markers’ competency test and how would DBE ensure competency of those appointed to marked the competency of other markers.

Dr Poliah replied that the more experienced senior markers would be tested and appointed to mark the competency test of the other markers. DBE was considering having a common competency test across the country to set a common standard for appointment of teachers to mark. Markers could be from Universities, but they charged exorbitant rates. He agreed that the competency test could be a tool to determine the capacity of teachers and the impact on learners.

Ms F Mushwana (ANC) asked if the unattained targets were realistic and if DBE had a plan to ensure that future targets were met.

Ms A Lovemore (DA) asked if DBE incorporated the number of learners who actually wrote Grade 12 when it calculated the target for pass rates.

Dr Poliah replied that when targets were set, DBE had not envisaged that enrolment figures would drop. DBE needed to renew the targets in light of the drop in enrolment figures.

Ms A Mashishi (ANC) asked for clarification on the number of Learners with Special Needs who enrolled to write the NSC. The number varied between two reports.

Ms Gina asked why some provinces had zero Learners with Special Needs writing the NSC.

Dr Poliah replied that 881 Learners with Special Needs had written the endorsed Grade 12 NSC, that is, the five subjects to matric. Many Learners with Special Needs had written and passed the full seven subjects and were not included in the 881 learners. These numbers needed to be tracked.

The Chairperson asked for correction on the number of Learners with Special Needs who wrote and passed the NSC. There had been a mistake in the report.
 
Dr Poliah said that this error had been caused by a print shift and had been corrected.

Mr Smiles asked for clarification on why the number of admissions to NSC had dropped. Standardization of Grade 10, 11 and 12 had been given as one reason.

Ms Lovemore asked if the drop-out rate in the number of scholars writing Grade 12 was calculated according to the number of scholars entering Grade R up until Grade 12 and if an improvement in Grade 12 results – which may be due to a decreased number of scholars writing the NSC - may also indicate that there was a lack of improvement in other areas of education.

Ms Gina said that she was concerned that learners were being withheld from enrolment for Grade 12 to improve the overall school matric results. This trend had become apparent in the Committee oversight visits and indicated that something was wrong in the other grades.

Dr Poliah agreed that there was gate-keeping by Principals to ensure ‘good Grade 12 crop’. Going forward, DBE would be auditing Grade 11 pass rates to prevent this from continuing.

Ms Lovemore asked if teacher strike action had affected the pass rate for Maths and Physical Science.

Mr Mathanzima Mweli, Acting Deputy Director-General: Curriculum, replied that although SADTU had threatened to take strike action, no strike action actually took place.

Ms Lovemore commented that the statement that ‘DBE thought that the standard in Grade 10 should be in sync with Grade 12’ was of concern. Grade 10 should be in sync with Grade 12.

Dr Poliah agreed that Grade 10 standard should be in sync with Grade 12. However, in the past, only Grade 11 level was set at the standard of Grade 12 and teachers of Grade 10 were setting their own tests. The target was to set a common standard from Grade 10 to Grade 12.

The following questions related to the under-performing districts and literacy and numeracy concerns:

Mr Dikobo asked what type of special attention would be given to the Eastern Cape.

Mr J Skosana (ANC) asked what was currently being done to address the Eastern Cape and other under-performing districts with regard to improving the learner outcome.

Ms Mashishi asked what the action plan was to address the literacy problem in learners
and how and when the under-performing districts would be assisted.

 Ms Lovemore asked when the literacy and numeracy problem would be dealt with by DBE, as it would continue to impact on future NSC results until it was rectified.

Mr Smiles commented that results would not improve and targets would not be met in the under-performing districts unless there was intervention.

Mr Z Makhubele (ANC) asked what was being done to ensure that the standard at feeder schools in all provinces was improved, as the quality of education at some feeder schools disadvantaged learners at higher education levels.

Mr Mweli replied that on the 16 to 18 January 2012, DBE converged with provincial agencies, SAQA and all stakeholders to finalise implementation of the plan for the National Strategy for Learner Attainment, to be used as a tool by curriculum managers to uniformly implement monitoring of curriculum coverage on a monthly basis. DBE had also implemented the National Strategy for Learner Attainment Framework to ensure, as core, delivery of curriculum support materials and support for psycho-social services at schools, including inclusive education. The framework included 124 activities which stipulated what was applicable and expected on a monthly basis, from Grade R to Grade 12. The School-Based Learner Attainment Strategy was mandatory for poor performing schools and optional to all schools. A Mentorship Programme had been implemented at under-performing schools and had been adopted in 2012 by the Premier of Mpumalanga. North West Province had been successfully implementing mentorship since 2006.

System-wide intervention included an Early Childhood Development (ECD) Plan to ensure ECD foundation education for numeracy and literacy from Grade R to Grade 9. Maths and Science targets were derived from the Action Plan (2014 to 2025) and was broken down according to the needs of each province. Intervention included a strategy for: Inclusive Learning - Learners with Special Needs – assisted by NGOs and other role players; Barriers To Learning – included at ordinary schools as 80% of learners were found to have barriers to learning; E- Education to offer exciting electronic gadgets to enhance interest in learning; and Workbook and Textbook Utilization, to optimize utilization by teachers and learners. In 2012, 54 million Maths and Science textbooks would be distributed to school learners for Grade 10, 11 and 12. Textbook coverage rate was 45% in South Africa and it was hoped that South Africa could join Swaziland with 85% or more textbook coverage in the near future.

National Government would monitor intervention for improvement in all under-performing districts. A report had been generated based on an oversight visit to the Eastern Cape in December 2011 and this would be shared with the Committee. The Deputy Director-General had been designated the job of co-ordinating intervention in the Eastern Cape and this had already commenced in Mount Frere District.

The following questions were not fully answered by DBE:

Mr C Moni (ANC) asked if the finding that there was an improvement in quality of the learners’ responses, as described in the Diagnostic Subject Report, was unrelated to the finding that there was inadequacy in numeracy and literacy skills.

Mr Smiles asked if the number of part-time versus full-time learners who wrote the NSC had been compared.

Mr Dikobo asked if the teachers were failing to do experiments in physical science or if there were no laboratories in which to do the experiments.

Ms Mushwana asked what plan was in place to ensure that there were laboratory facilities at every school.

Ms Gina asked what programme was in place to address the problem of SBA’s, teacher training on moderation of tests and quality of question papers to prepare the learner for the NSC. It appeared that teachers were not familiar with what was expected of them.

Mr Smiles commented that there appeared to be loopholes in the School Based Assessments (SBA’s) for both Life Orientation and Maths Literacy.

The Chairperson said the Members could engage further with DBE at the meeting scheduled for the following day. She asked DBE to respond on the workbooks and also to clarify the norms and standards for infrastructure, including laboratories at schools. It would also be important to know how the general findings of Umalusi would be added into the Annual Performance Plan and Action Plan. It was also important that the Committee had access to DBE’s plan regarding reducing the number of marking centres. The Chairperson further commented that there appeared to be a correlation between financial management disclaimers at schools and learner results and asked if DBE considered the relationship between finance management and quality of education at schools. She requested that DBE, at a later stage, should include a discussion on finances at schools as this would need to be presented in the Strategic Plan, Operational Plan and Annual Performance Plan. The situation in the Eastern Cape warranted a meeting on its own. Finally, she asked DBE to brief the Committee on SADTU’s ‘go slow’, as the ultimate victims of such action were the learners.

Umalusi Report on the Quality Assurance of the 2011 NSC
Professor Dan Kgxwadi, Chairperson: Umalusi Executive Council, said that Umalusi’s willingness to share the standardization decisions with the public remained unique in the world. In quality assuring the NSC, Umalusi had engaged in a number of short and long term research projects, and had conducted post exam analysis of the ten gateway subjects to measure consistency over time. The reports emanating from these investigations were used for standardising results. After looking at the 2011 NSC results, Umalusi was satisfied that the NSC exams were conducted in terms of policy and taking note that there were no serious irregularities that compromised the credibility of the examinations, it approved the release of the NSC results on the 2 December 2011.

Dr Mafu Rakometsi, Chief Executive Officer: Umalusi outlined the Regulatory Framework for Quality Assurance of Assessment and Learner Achievement, as well as the requirements for the approval of release of the results.

Mr Vijayen Naidoo, Senior Manager: Quality Assurance of Assessments, Umalusi, explained the process for moderation of the question papers to ensure that they were of the required standard. Of the 135 question papers for the 38 subjects in November 2011 (excluding unofficial language and Life Orientation), 41 were approved /conditionally approved at 1^st moderation; 73 at 2^nd moderation; 18 at 3^rd moderation; and 3 at 4^th moderation and beyond. The March 2012 supplementary question papers showed a similar trend.

Regarding the 2011 English (1^st additional language) question paper, each question paper was set anew by a panel of expert DBE examiners according to the standard was captured in the subject assessment guidelines. The paper was internally moderated by DBE, externally moderated by Umalusi and then subsequent moderation before approval. The true test of where those question papers were pitched could only be ascertained once the paper had been written. Umalusi did not use pre-tested items, in terms of cognitive levels etc, where according to a learner performance, decisions could be made about the learner. Umalusi could only pick up post standardisation, at the end of the process, by virtue of learner performance, that the English paper may have been on the easy side. The report would need investigating to ascertain whether the paper was of low standard and would be rectified in the 2012.

Areas of concern indicated by DBE, were adherence to time frames; simultaneous moderation of final and supplementary question papers; and predictability in the Afrikaans 2^nd Additional Language – limited prescribed short stories.

Umalusi had piloted a new SBA model in 2010 which focused on tasks and involved three moderation visits per year across the nine provinces. Despite the challenges which continued with SBA’s, the model had enhanced feedback to educators and improved implementation of SBA and DBE had implemented a similar model nationally in 2011. Areas of concern were that not all tasks were of the appropriate level and poor assessment of tasks impacted on the NSC exam standard. There was a dire need for training of educators on rubrics as marking tools for assessing practicals and a need for assessment around life orientation and how physical education tasks were assessed. There was also concern over the level of internal moderation on the quality of the tasks and non-compliance of some provincial departments with submission of samples as required. Umalusi engaged a process of statistical moderation on SBA marks to bring the marks in line with exam performance.

Umalusi monitored the state of readiness, conduct and marking of examinations. Vacant posts and use of contract staff were of major concern. In the 6593 exam centres, Umalusi found that writing of examinations was conducted in line with policy with isolated incidences of non-compliance. Shortage of question papers and answer books was a problem directly related to printing and packaging processes.

The major finding of the marking phase was concern with the quality and standard of marking. DBE had been addressing this issue over the past two years. Markers were not able to deal with marking particular types of questions.

The issue around reducing the number of marking centres, as raised by Members in the previous discussion, was about risk and security and some marking centres were not conducive to marking. Provincial capacity to cover large distances to monitor the centres situated far and wide in the province also posed challenges. The more centralised, the better Umalusi could maintain standards. 

Umalusi engaged centralised pre-marking/memorandum discussion and off-site moderation of marking to ensure that marking was conducted in accordance with agreed practices. Concerns were that there was non-attendance and poor pre-memo discussion preparation from some provinces and incidences of unauthorised changes to marking guidelines.

Of the marks received from DBE, 56 subjects were standardised; raw marks were accepted for 45 subjects; three subjects were moderated upward and eight subjects were moderated downward. All standardisation decisions were listed in the document.

Dr Rakometsi then presented a summary of Umalusi areas of concern:
▪ that DBE should submit question papers for moderation within the 18-month cycle for submission
▪ that SBA tasks developed by educators focused mainly on assessing the lower cognitive skills
▪ educators battled with the development and use of rubrics for scoring learner performance
▪ implementation and assessment of Life Orientation required serious attention as it was 100% internally assessed and standards varied between provinces and within provinces
▪ incidents of improper registration of candidates
▪ appointment of suitable Chief Invigilators was encouraged to address problems relating to the flouting of basic examination regulations
▪ criteria for the appointment of markers should be strictly adhered to
▪ additions were made to the final memoranda without consent of DBE and Umalusi
▪ some markers did not have ability and experience to handle higher-order cognitive level questions
▪ inaccurate totalling of marks and incorrect transferring of marks to cover page during marking
▪ large number of marking centres.

Umalusi recommended that moderation of the quality and standard of the SBA should be implemented at the school and cluster levels and that as part of the moderation process, compliance should be checked to ensure adherence to policy. It also recommended that all the assessment bodies should ensure that all educators were familiar with the policy requirements to ensure the meeting of national standards.

Regarding verification of marking, Umalusi recommended that criteria for the appointment of markers should be strictly adhered to and that all stakeholders should be consulted before changes were made relating to the appointment of markers. Regarding standardisation of marks, Umalusi recommended that a Common Framework should be introduced for the assessment of Languages and that teaching and learning of Maths and Science should be strengthened.

In general, Umalusi was satisfied with the manner in which the 2011 NSC examination was administered. The technical irregularities were addressed in a fitting manner.

Discussion
Mr Dikobo asked for clarification on whether the appropriately-appointed Chief Invigilator was the School Principal.

Dr Rakometsi replied that he believed that the Principal was usually the Chief Invigilator.

Mr Naidoo added that the words appropriately-trained Chief Invigilator should be added in solving the issue of invigilators managing the conduct of examinations appropriately.

Mr Dikobo asked what DBE was doing about teacher development to ensure that Grade 12 teachers were competent to teach and mark papers at Grade 12 level.

Dr Rakometsi replied that the teacher’s pass rate was an indicator of teaching competency but that some teachers, in their appropriation, could not mark papers. His feeling was that the teacher marking competency test should be rolled out to all the teachers across the country, as they should all be able to mark papers. The matter needed attention.

Ms Mushwana asked if there was consistency in adjustment of marks of the various languages.

Mr Makhubela commented that Common Framework for assessment of languages was urgent.

Dr Rakometsi replied that this was a high priority issue for Umalusi and it was working together with colleagues to find a way to standardise languages across the board.

Mr Naidoo added that if he understood Ms Mushwana’s question correctly, his response was that English home language and English 1^st Additional language in the NSC structure were subjects on their own and would not linked in terms of how the previous standard and higher grade were adjusted. If English home language marks were adjusted, it would not mean that English additional language marks would be adjusted.

Ms Mushwana said that she would expect that if one home language was adjusted, then the other home languages should be adjusted.

Mr Naidoo replied that while they were all home languages, the standard varied and this had been a recurrent problem over the years. The Common Framework had not been taken to the extent desired by Umalusi. The following weekend the instrument would be fine-tuned and a number of critical areas would be addressed.

Mr Naidoo also added that predictability was a problem, especially in languages (such as Venda) when the same setwork book was repeatedly used for Grade 12. Predictability was a problem at University level when the faculty complained that students that successfully passed the NSC were unable to apply themselves in other contexts.

Mr Smiles asked why the question paper could not be approved at 1^st moderation and whether this indicated that selection of the examiner was not suitable.

Dr Rakometsi replied that the goal was for question papers to be approved at 1^st moderation, but as long as examiners required subject assessment guidelines and internal moderators did not pick up on issues, and as long as there were no pre-standardised tests, papers would need a 2^nd moderation. The extent to which the problem presented itself was what Umalusi was interested in, for now.

Mr Smiles noted that the shortage of question papers, especially in KZN in 2011, tended to occur at independent schools.

Ms Gina asked if the Committee could be assisted by being informed about where exactly the shortages of question papers had occurred and which Chief Invigilators had not attended the memo discussions.

Dr Rakometsi replied that the shortage of papers was unacceptable and Umalusi would be checking on the statistical data relating to the tendency for shortages of papers to occur at independent schools. Centralised marking at one centre would make it logistically easy to manage a call for an adjustment in the memo of the question paper. Currently, irregularities existed when an adjustment existed in one province and not in another province.

Ms Gina asked for more information on the importance of Life Orientation.

Ms Lovemore asked how effective Life Orientation was when it was practically giving away a pass to learners.

Dr Rakometsi replied that Life Orientation was a ‘bonanza’. DBE indicated that it would provide a time-table in terms of national standardisation of Life Orientation tasks. It was important to have career guidance at school to inform learners on employment opportunities.

Ms Lovemore asked who decided if the question paper or learner mark required standardisation in the post analysis process.

Dr Rakometsi replied that during the standardisation process Umalusi did not correct for poor teaching or mistakes that had occurred in the classroom. Standardisation kicked in and Umalusi adjusted marks only if the paper was too difficult or easy - where learners were unfairly advantaged or disadvantaged, and the adjustment was based on qualitative specialist reports from external and internal moderators of DBE and from university experts and on trends observed in a particular subject over the years since 2008.

The Chairperson thanked Umalusi and acknowledged that Umalusi would submit the relevant information regarding the shortage of papers and non-attendance at memo discussions to the Committee Secretary. She added that it was important that Umalusi and DBE retained their independence.

A report the following week from the South African Council of Educators (SACE) on the training and development of teachers would assist Members with understanding how SACE was enhancing the standard of teaching.

The meeting was adjourned.