Certification backlog: progress report by State Information Technology Agency

Meeting report

SITA

Mr Sefuno Mohapi, Chief Executive Officer (CEO), SITA, explained that the backlog had been identified and prioritised for the presentation to the board. The executive team was currently making progress in addressing the issue, with the use of the necessary systems being implemented.

Resources had been deployed for SITA to achieve its goal within the expected timeframe. Inadequate observation of business rules had resulted in this issue. SITA had requested the assistance of Software AG as well as additional technical support. The collaboration between SITA, DHET and Umalusi had been increased substantially resulting in a small reduction in the NC (V) Certificate Backlog.

The original equipment manufacturer (OEM) of the technology used to build the current DHET Examinations System software application had been appointed and given the mandate to coordinate “getting the backlog to zero” and “implementing measures to keep it at zero”.

The NC (V) Backlog Elimination Project “kicked off” on 27 July 2015 and would achieve its “zero backlog” objectives by end of January 2016. The core business application for DHET, from the beginning had not been properly synchronized with DHET & Umalusi business process changes and SOP changes. The Examinations System was copied from the NATED System and that level 2 and level 3 certificates were not initially issued by Umalusi which caused differences in processing and standard operating procedures (SOPs).

The SOPs were not properly documented from the beginning and that as a result of the “out of sync” state of the DHET Examination and Assessment application, a backlog of NC (V) certificate requests had developed between the Examination Centres, DHET and Umalusi.

The SITA, DHET & Umalusi management teams had received a mandate from the DHET Portfolio Committee to work together to resolve the NC (V) certificate backlog. In response to the DHET Portfolio Committee mandate, SITA had contracted Software AG, the OEM of Natural/Adabas technology, to provide project management and technical resources to work with DHET, Umalusi and SITA stakeholders to jointly eliminate the backlog.

The primary objective of the NC (V) Certificate Backlog Elimination project was to “get the NC (V) Certificate backlog to zero, and keep it at zero” and that the targeted timeframe for elimination of the NC (V) Certificate backlog was six months. The project was commissioned as a result of unresolved backlog of certification processing. As of 6 August 129,932 candidates were affected across all exam cycles, across all levels.

Not all candidates were waiting for full certificates, many were waiting for subject certificates where candidates still had to complete certain subjects.

Due to the consolidation issue in the Examinations system, the number of full certificates decreased depending on exam results and subject certificates could increase or decrease depending on the consolidation process. However, some candidates ultimately would not receive any certifications if all subjects were not passed.

Project Background

The backlog scope consisted of the original backlog project with Umalusi from 2007/11 to 2012/03 and the period post this to date, through 2015/03. The project aimed to stop backlog from growing by implementing corrective actions to the existing examinations and assessment application, introducing new tools and techniques to ensure consistent data validation, building capacity to ensure data was corrected prior to updating mainframe database, and enhance visibility of the enrolment and certification process  by means of an online data dashboard.          

Some NC (V) Certification requests had not yet been submitted to Umalusi by DHET. Some examples of these were candidates who had taken supplementary and subsequent exams but could not be certificated due to defects in the consolidation function of the examination system. There were problems with the quality and completeness of the data submitted by the examination centers; and business logic gaps in the data validation functions within the examination system.

Some NC (V) Certification requests were submitted by DHET but rejected by Umalusi. These included duplicated records that were blocked by the examination system validity checks; requests where the condonation function was defective in the examination system; a high level of absenteeism in main and supplementary exams; and changes of marks after standardisation and statistical moderation.

There were further NC (V) Certification requests not printed due to non-payment of certification fees to Umalusi by examination centers.

Challenges

Several challenges related to the examination centers were identified. The NC (V) Data Validation was too far “downstream” in the candidate enrolment and certification business processes and examination centers did not have the capacity to did data integrity checks before submitting NC (V) data to DHET.

The Umalusi-related challenges included that Umalusi and DHET were not fully aligned in their execution of the NC (V) Certification Business Process. The marks and document records were not synchronised. There was no “Pre-Testing” functionality prior to submission to Umalusi. The change control was not fully operational when improving systems, directives, and certification rules.

The following DHET examination and assessment challenges were identified: the system was out of sync with current directives and SOPs; the error resolution was not implemented holistically, and the patches of implementation created conflicts; there was no DHET “Pre-Testing” functionality prior to submission to Umalusi; and capacity requirements were not met in order to satisfy work demands and workforce changes.

The following SITA maintenance and support team challenges were identified: SITA technical capacity was not sufficient to keep up with DHET demands; changes were treated as maintenance work rather than full project work; SITA Support and Technical Teams had become a part of DHET NC (V) operations in order to remediate examination system functional shortcomings; all DHET NC (V) functional operations were not committed to the Examinations Software System, which consumed SITA technical resource capacity, reducing responsiveness to DHET requirements; exam Centers and third party vendors of college management systems were limited to “batch” based interaction with the Examination system, instead of real time; exam Centers could not “stage” data and did data integrity checks before submitting data to DHET; finally, DHET, Exam Centers and Umalusi did not use a common rule base for certificate processing.

The Way Forward

SITA planned to empower examinations centres to ensure data integrity for enrolment and examination data. A staging portal would be created to provide capacity for data integrity and visibility of performance, and to move all manual operational activities for NC (V) data and processing into the examination system. Umalusi planned to work towards straight-through processing of certificates and a common business rule base. The DHET intended to clearly define SOPs, and work towards straight-through processing.

Project Approach: Target Alignment of NC (V) Certificate Processing Activity

The project approach involved aligning subject matter expertise with business processes and governance and examination system software.

Project Implementation

The project was implemented in three work streams. The first sought to reduce the backlog to zero. In order to do this, the NC (V) functionality in the DHET Natural/Adabas application required remediation. This outcome would be achieved through Adabas data cleansing for NC (V) candidate data, a natural/Adabas health check for NC (V) certificate processing, and a natural engineer code report for NC (V) functions

The second work stream required an alignment of the Natural/Adabas to the DHET NC (V) certificate process. This would be achieved through NC (V) certificate SOP process modelling, technical modelling of the Natural/Adabas NC (V) functionality, and gap analysis and remediation.

The third work stream required examination centre data integrity enablement, which would be achieved through up-front data and rules validation, straight-through processing and NC (V) process visibility.

The methodologies of the agency required change and SITA would be making advances in line with this.

Umalusi

Mr Gerhard Booyse, Information Technology (IT) Manager, Umalusi, explained that it was advantageous to have the assistance of Software AG. Mr Booyse indicated that the unpaid fees owed to Umalusi comprised of 5 percent from public institutions and 95 percent from private institutions, to the value of R2 million.

Department of Higher Education and Training

Ms Nadine Pote, Chief Director: Exams (DHET), explained that the DHET had met with Software AG for several sessions. Ms Pote explained that two administrative staff members had been appointed to address the outstanding fees and debt collection.

Discussion

Ms J Killian (DA) commented that the project was reassuring and that it was assuring that the team had identified the weaknesses which resulted in the certification backlog.

She asked for statistics on the candidates who were still affected by the certification backlog and requested a breakdown of the figures for the period 2007 to 2012 as stipulated in the presentation, and to the period thereafter. How many people were affected by subject certificates and how many were affected by full certificates?

Mr Booyse explained that for a candidate undertaking a qualification over a number of years, a subject certificate was required. The consolidation was on the system, and this system had not consolidated across examination cycles and that once the issues were rectified, that the certificates would be distinguishable.

Ms Killian expressed concern regarding the payment of certification fees and explained that at the end of March, the majority of colleges which owed Umalusi were private colleges. Was this historic debt or did it come from before 2013?

How would the issue of capacity for data integrity be resolved?

Mr Mohapi explained that the data breakdown dashboard with the live data was not ready and that the data would be made available once the dashboard was ready. He explained that the data in the management report would be simplified so that certain areas would be emphasised.             

After the registration data and examination data was validated, that it was sent back after the validation process.  The pre-testing of data before sending the data to Umalusi guaranteed the success of the validation process.

Patrick Shields, Chief Technical Officer (CTO), Software AG, explained that being successful in reducing the backlog to zero required the use of subject management expertise, the business process and governance and examination system software. The challenge to the improvements of the backlog was the necessity to constantly remain up-to-date

The company ensured that the SITA team experienced knowledge transfer on the new technologies, which included workshops to address similar challenges for other Departments and agencies they served.

Mr Booyse said that the number of certificates which have not been printed was not numerous and explained that there had been a ‘top-slicing’ of funding.

Ms M Nkadimeng (ANC) applauded the collaboration of DHET, SITA and Umalusi. Ms Nkadimeng asked if the DHET had not considered placing due dates for fee repayment.                                                                                                                                                             

Mr M Mbatha (EFF) asked how many colleges required certificates.

Mr Y Cassim (DA) asked if students at private colleges who did not receive their certificates due to the non-payment of fees, received recourse from the Department within the legislative environment. It was noted by the Department that private TVET colleges would be exonerated from paying these fees.

Mr Booyse said that Umalusi would approach its legal team and provide an updated list of outstanding fees for the month of August, to the Committee.

Mr Cassim enquired as to whether the certificates may still be printed because of debt and what the recourse for private colleges was.

Would the commitment for the backlog to be eradicated in December/January mean that students would have their certificates by this time?

He asked about the historical debt of public institutions to the value of R95 000 and remarked that the amount could be consolidated by the Department and Treasury.

Ms Killian referred to the historical debt in the institutions and indicated that she was significantly concerned as parents were enrolling their children in private institutions which were not paying Umalusi.

The debt in public institutions, exploited parents and students as they were still unable to receive their qualifications. How could the problem be prevented from escalating?

Dr Bheki Mahlobo, Acting Deputy Director-General: Further Education, DHET, explained that the Department did not have a line item for this debt. The approach to be taken was for the individuals presiding over colleges. There was a need to articulate to the colleges that their budget would be top-slicing for payment to Umalusi. Dr Mahlobo agreed with Umalusi that there was litigation route to recover debt from private colleges. Closing the colleges did not solve the problem as the students were unable to receive their qualification. It would be possible though to issue an intention to withdraw the qualification of the college. 

However, it was not in the best interest of the country to deregister private institutions as these institutions provided qualifications for students which could not be absorbed by public institutions.    

Prof B Bozzoli (DA) said that there was no detail regarding the new examination centres and asked for the Department to provide a similarly-focused plan for the December/January deadline.

Ms Pote said that the Department would provide a plan to the committee, including milestones and dates for 2016. 

Dr Mahlobo said that it was vital to indicate how the capacity would be established in private institutions.     

Mr Mbatha said that Umalusi should declare that the outstanding debt was historical debt.             

The Chairperson asked if there was a strong risk and compliance committee and if not, how Umalusi intended to implement the strategy without detractions. Was there a risk management report and plan in place?

Mr Jerry Vilakazi, Chairperson, SITA, responded that an audit and risk committee existed and was comprised of non-executive members. The committee provided oversight and monitoring.

Mr Mohapi explained that the project required a risk plan which could be made available. Meetings included a presentation on these issues.

Mr Vilakazi said that the institution was more comfortable as they understood the nature of the challenges which the institution was handling.

The Chairperson said that the emphasis should be on the eradication of the backlog and the provision of certificates by the deadline. She commended the reduction of the 12 month service delivery to 6 months as it indicated responsiveness to the needs of the country. She noted the significant process in implementing systems to address challenges. Historical debts were unacceptable as students were not receiving their certification to enable them to contribute to the economy. The Department should ensure that private colleges were paying their fees.

Members requested a breakdown of the certificates in terms of the number of full certificates, subject certificates and ready certificates which have not been released owing to outstanding debt.

The Committee requested that SITA submit reports on figures on a monthly basis for the Committee to monitor progress.

The Chairperson said that the Department needed to provide the plan of action based on milestones and the relevant dates.          

The meeting was adjourned.