COVID Impact and ROI analysis of skilling models in India; assessing the “agility of skill training eco-system” to respond

COVID Impact and ROI analysis of skilling models in India; assessing the “agility of skill training eco-system” to respond to the impact of COVID-19

DEFINITIONS OF KEY CONCEPTS AND BOUNDARY CONDITIONS

Study background

Impact of COVID-19

In a statement on 16th March 2020, the ministry said that in the wake of the Covid-19 outbreak, “all skill development training centres, Industrial Training Institutes (ITIs), Jan Shikshan Sansthans (JSS) under the aegis of Ministry of Skill Development and Entrepreneurship, will operate in adherence to the advisory issued by respective states and remain closed as per directions from the local government”.

Criticality of skill development post Covid-19 pandemic
The pandemic has left millions of workers unemployed at the moment. It is expected to have new hiring of semi-skilled and skilled workforce from scratch. New hiring will certainly require fresh skilling, upskilling and/or reskilling of the workers. This will be an opportunity for the training institutes especially the ones undertaking short term training.

Moreover, as a blessing in disguise, this pandemic situation has opened a window of opportunity for giving focus to remote trainings, e-learning (blended), technological interventions in skill development. This pandemic situation has also brought to light the need of special focus on training under healthcare, disaster management, e-commerce, robotics and others. Though the long-term outlook for the Skill Development sector remains stable, prolonged lockdown could impact the operations and the cash flows of the vocational training institutes in the short term as the income may get delayed for the time being. However, entities with liquidity backup and resourceful promoters are not likely to get affected much. Post pandemic situation, one may see a focus in special trainings including e-learning, healthcare, disaster management, e-commerce and robotics.1

ROI benchmarking
Realizing the potential of being the capital of global skilled workforce with a growing young population in the working age bracket2, India has been investing in upskilling the country’s workforce to meet the future demand. A report by MSDE mentions that some of the factors contributing towards low placement rates are the accreditation of inadequate technical institutes, standard assessments to measure the quality of institutes, quality of training and placements.3

NASSCOM foundation’s and Sattva’s extensive work in the skilling ecosystem shows that the effort so far is still not addressing issues around Return on Investment pertaining to:
- Cost per candidate varies from INR 7,000 to INR 45,000 (US$100-US$650) across organizations
- Duration of the programmes vary from 15 days to 3 years
- Training of the Trainers (ToTs) duration ranges between 5 hours to 60 days residential model
- process of placement varies from personal reference to highly technology driven matchmaking of competencies and job roles.

While all models claim to have certain percentage of placements, there is no dotted lines that are drawn to see how the input cost is impacting the salary and retention in the job.
Comparative analysis of cost and benefits of various models run in the same sector is unavailable or not publicized to aid the decision-makers from both funding and implementing sides.
Realizing the need of solving the puzzle, this research study aims to bring together leading organizations such as JP Morgan, Accenture and CISCO that support various skilling efforts in India to gain from their experiences and in turn helping them to make informed decisions for funding.

Objectives of the study
The overall study objective remains as study on adaptive skilling models:

To study the COVID impact on skill training eco-system
- Impact on job market post COVID
- Impact on aspirations of candidates post COVID
- Impact on the relevance of training models in the post COVID time
To study and analysis the key factors that influence the ROI of similar skilling models from a pre-COVID scenario o Analyze input cost for conducting training programme – mobilization, product development, ToT, classroom delivery, placement and programme management
- Analyze the outcome form a placement and salary point of view
- Assess the correlation between input cost and outcome

To build decision making scenarios for key stakeholders for adapting the skilling models in the post-COVID time
- ROI metric for decision making in the post COVID scenario

- Build “Adaptability Framework” for skill training organizations to analyze its capacity and recommend necessary actions for being relevant in post-COIVID time

KEY CONCEPTS

Benchmarking
Underpinning the concept of benchmarking is an understanding of how organisations achieve a certain level of performance and the subsequent application of these approaches to another organisation. The term ‘benchmarking’ is used to describe a large variety of different measurement and evaluation technologies which have been collected with one single aim: the improvement of skilling organisational performance. The study will include process mapping as well as outcome mapping to compare performance across skilling organisations. Benchmarking generally involves the measurement of key performance criteria, identification of entities which may have similar performance data, comparison of the performance of the organisations, and analysis of the reasons for the differences in performance. Benchmarking as a tool for organisational improvement offers a range of benefits for organisations. These include:

- changing the culture of organisations from being inward-looking to being outward-looking
- improving the quantity and quality of performance information within an organisation
- making monitoring of agency performance by executive, government and other stakeholders easier, thus improving accountability.

Benchmarking involves far more than the ad hoc collection of statistics. Benchmarking is a purposeful activity in which the processes used by an agency to deliver its products and services are compared with similar processes elsewhere. It is an ongoing and systematic process to search for and then introduce best practice into an organisation. It is by understanding the reasons for differences or gaps in performance that many organisations improve or grow.

In study context – Process mapping and outcome mapping and key areas/indicators to be studied The benchmarking exercise will help us understand the delivery of services of a skilling organization as compared with similar services in another organization.

The key areas/indicators to be studied:

- Best practices in the organization
- Per candidate cost of training
- Per trainer cost

- Employment opportunities made available to the candidates
- Success rate of placements for the candidates
- Average income per candidate

In COVID context – agility mapping of mobilization, delivery, and placement practices

This exercise will also help us understand the agility of a skilling organization compared with that of other organizations to understand their readiness in the post-COVID world. The key areas/indicators to be studied:

- Usage of technology in program delivery
- Agility in mobilizing and placement practices
- Change in placement strategies and models

Inputs and input costs
In this stage it is important that the financial value of the various changes that resulted from the training is estimated and the total costs incurred in implementing the training program is identified. In simpler terms, data is converted to monetary values to compare to program costs. Various approaches are available to convert data to monetary values.

Specific technique selected usually depends on the type of data and the situation (Phillips, 1996b): Output data converted to profit contribution or cost savings, direct costs saved, increased volumes of output produced and timeliness of output. The values of the performance improvements gained can be estimated by internal or external experts. Soft measures are linked, mathematically, to other measures that are easier to measure and value.

This approach is particularly helpful when establishing values for measures like intangible attributes -customer satisfaction, employee satisfaction, grievances, and employee complaints that are very difficult to convert to monetary values.

The next step in the process is tabulating the costs of the program. Tabulating the costs involves monitoring or developing all of the related costs of the program targeted for the ROI calculation. Among the cost components that should be included are:

- Cost towards external training services purchased
- the cost to design and develop the program, possibly prorated over the expected life of the program • the cost of all program materials provided to each participant
- the cost of the instructor/facilitator, including preparation times as well as delivery time
- the cost of the facilities for the program/internal training staff involvement, travel, lodging, and meal cost for the participants, if applicable
- salaries plus employee benefits of the participants for the time they attend the program
- administrative and overhead costs of the education and training function allocated in some convenient way to the training program
- the costs of carrying out the ROI on the training program.

After calculation of the direct financial value of the performance enhancements, it is also necessary, wherever possible, to estimate the value of the more “intangible benefits”, such as: Increased job satisfaction, benefits of increased staff retention and reduced recruitment costs, increased organizational commitment, improved teamwork, improved customer service, reduced problems and complaints, reduced conflict.

Outcomes
Outcomes indicators are based around the results generated from the system. In the case of vocational education and training, these can be considered as the ability to gain employment or further training, the ability to generate an income, or the ability to accumulate further qualifications which lead to a higher-level credential.

APPROACH and DATA SOURCES
The study comprises of two tracks –

Understanding the effectiveness of multiple models in the skilling value chain
Understanding the agility of organizations in Post COVID scenario and change in aspirations of the candidates

Data Sources

Candidate level secondary data from historic perspective (1500 candidates trained prior to Dec 2019) • Input costs of program component mobilization, training of trainers, training delivery and placement • Practices undertaken by skilling model though conversations with program team
Aspirational and Agility oriented practices through conversation with candidates, program team and leadership team

KEY CONSIDERATIONS

Key variables to be considered to analyze the data and draw conclusions

Variables will be categorized as depended and independent

Independent Dependent

Implementation partner
Location
Sector
Job role
Gender
Age
Education of candidates
Family income

Information sharing and confidentiality

Duration of the course • Selection criteria
On/Off campus models • Classroom vs blended models • Digital mediums
Infrastructure

The participant and organization level data shared by the skilling organization will not be utilized for any purpose other than those defined for the study. The data will be used to draw insights across organization, sector and skill development ecosystem. Financial and aggregated individual data published in the form of reports will not include identifying information such as Name, Location and so on.

Further, data sharing principles will be aligned with individual parties prior to the data sharing process

Status of the study

Participating organizations

JPM as funding and knowledge partner
NASSCOM Foundation as implementation partner
Accenture as knowledge partner
CISCO as knowledge partner
MSDF as knowledge partner
LTI as knowledge partner

Briefing sheet for interviews with Employers

The briefing sheet consists of the key discussion points for the interviews scheduled with the Employers engaging with skill training partners. This shall be helpful in understanding the key areas of research for the ‘COVID Impact and ROI analysis of skilling models in India’ study and setting the context for the interviews.

Note: Please note that the document is indicative of the theme of the interview and aims to give an overview of the key areas of research for the study for both the tracks – ROI and Agility & Scenario Building. Further, all the points would pertain to the sectors or employability skills as applicable for your organization and the operational model. All the points may not apply to your organization. Sattva will contextualize the questionnaire for each of the organizations.

Employer Interview (Time: 30 mins approx.)

Key points of discussion for interview:

ROI: (Outcomes of training of candidates)

Hiring engagement with skilling partner

- Types of job roles hired for
- Skill expectation for job roles hired

Nature of employment provided

- Average number of candidates hired per job role
- Average Salary of hired candidates
- Candidates requirement of organisation and % of requirement addressed by skilling organisation

Scenario Building: (Change in job roles and job requirements in the present and future)

Change in hiring requirement due to COVID at organisation level

- Job roles that have increased demand for entry and mid-level professionals
- Job roles that have decreased demand for entry and mid-level professionals

Change in hiring requirement due to COVID at sector level (BFSI/ITeS/Retail/Construction)

- Job roles that have increased demand for entry and mid-level professionals
- Job roles that have decreased demand for entry and mid-level professionals

Change in job type i.e. WFH/ Gig based/ Contract based for various roles
Hiring or rehiring pattern of organisation

- Current hiring requirement for entry and mid-level job roles
- Reduction in demand for entry and mid-level job roles

Evolving job roles and new skill requirements

- Types of job roles that have emerged in sector
- Technical, Soft skills, Remote working etiquette
- Reduction in demand for entry and mid-level job roles