Unlocking Insights into Pakistani Education: Machine Learning Projects with ASER Pakistan 2023 Data

This repository showcases a collection of Machine Learning projects focused on analyzing the ASER Pakistan 2023 dataset. Our goal is to uncover meaningful patterns and insights within Pakistan's educational landscape, utilizing both supervised and unsupervised learning techniques for prediction, classification, and exploration.

Project Overview & Motivation

The Annual Status of Education Report (ASER) Pakistan is a vital citizen-led, household-based initiative that provides crucial data on the state of education in Pakistan. The ASER Pakistan 2023 dataset offers a rich snapshot of learning outcomes, school facilities, and household characteristics, making it an invaluable resource for understanding the challenges and opportunities in the education sector.

The overarching goal of these Machine Learning projects is to:

• Gain deeper insights into educational trends and disparities across various regions and demographics.
• Identify key factors influencing student learning outcomes and school performance.
• Develop predictive models to forecast educational indicators and potential challenges.
• Classify student performance and school characteristics to better target interventions.
• Explore complex relationships within the data through unsupervised learning.

By applying diverse ML methodologies, we aim to provide actionable insights for researchers, policymakers, and educators, ultimately contributing to evidence-based decision-making and improvements in Pakistan's education system.

Key Features & Models

This repository explores a range of Machine Learning tasks, leveraging the depth of the ASER Pakistan 2023 dataset. Key areas of focus include:

• Predictive Modeling:
  • Forecasting student literacy and numeracy rates based on various influencing factors.
  • Predicting school dropout rates or identifying students at risk.
• Classification Tasks:
  • Classifying children as in-school or out-of-school based on household and regional data.
  • Categorizing schools based on their performance metrics or available resources.
• Unsupervised Learning:
  • Clustering schools or districts to identify common characteristics and challenges.
  • Employing dimensionality reduction techniques (like PCA) to uncover underlying patterns in the dataset.

We utilize a variety of Machine Learning models, including but not limited to:

• Supervised Learning:
  • Linear & Logistic Regression
  • Decision Trees & Random Forests
  • Support Vector Machines (SVM)
  • Gradient Boosting models (e.g., XGBoost, LightGBM)
• Unsupervised Learning:
  • K-Means Clustering
  • Hierarchical Clustering
  • Principal Component Analysis (PCA)

The projects within this repository demonstrate the application of these techniques to different facets of the ASER dataset.

ASER Pakistan 2023 Dataset

The ASER Pakistan 2023 dataset is a comprehensive collection of data gathered from households and schools across various districts in Pakistan. It typically includes information on:

• Child Data: Learning assessments (reading, arithmetic), school enrollment status, age, gender, etc.
• Household Data: Socio-economic status, parental education, household assets, etc.
• School Data: Teacher attendance, student attendance, availability of basic facilities (toilets, water, electricity), school type (government, private), etc.

This rich, multi-level dataset provides a unique opportunity to analyze the educational landscape from various perspectives.

Accessing the Dataset:

The original ASER Pakistan 2023 dataset and related resources can typically be found on the official ASER Pakistan website:

• ASER Pakistan Official Website: https://aserpakistan.org/

Data Preprocessing & Feature Engineering:

The raw data from ASER Pakistan often requires preprocessing to be suitable for Machine Learning. Common steps undertaken in these projects may include:

• Data cleaning (handling missing values, correcting inconsistencies).
• Feature scaling and normalization.
• Encoding categorical variables.
• Feature selection and creation of new relevant features from existing ones.

Specific preprocessing steps for each project can be found within their respective directories and documentation.

Repository Structure

This repository is organized into several project directories, each focusing on a specific analysis or ML task:

• 00_raw_data/: Contains the original, unprocessed ASER Pakistan 2023 dataset files (e.g., ITAASER2023Child.csv, ITAASER2023Household.csv, ITAASER2023School.csv).
• cluster_analysis_project/: This project (cluster_analysis_project.Rmd) applies K-Means clustering to classify rural Pakistani households based on socioeconomic status, climate change impact, and academic achievement factors from the ASER 2023 child and household datasets. It identifies three distinct household profiles and analyzes their characteristics, including internet access and school type enrollment.
• data_visualization_project/: This project (ali_abid_data_visualization_project_rmd.Rmd) focuses on generating a variety of visualizations to explore relationships between household factors, school characteristics, and children's educational outcomes using merged ASER 2023 child, household, and school data. It includes analyses of school type distribution, enrollment vs. attendance, climate change impacts, and arithmetic levels by medium of instruction.
• machine_learning_project/: This directory contains projects implementing supervised learning models:
  • One project (machine_learning_project.Rmd) uses logistic regression to predict a child's general knowledge (binary outcome) based on binarized household, climate, and academic variables.
  • Another project (linear_regression_project.Rmd) employs linear regression to predict a child's arithmetic level (treated as continuous) using household and climate change factors. Both projects utilize features from the ASER 2023 child and household datasets and explore model building with base R and the caret package.
• pca_project/: This project (pca_project_ali_abid.Rmd) first uses Principal Component Analysis (PCA) to reduce dimensions of household and climate change variables into meaningful components (Climate Change Factors, Household Vehicles, Displacement Factors). It then uses these components in a logistic regression model to predict a student's general knowledge. The file data_reduction_pca.Rmd provides general instructional material on PCA.

Each project directory contains its own code (primarily RMarkdown files), potentially modified datasets, and detailed reports or outputs (like HTML or PDF versions of the Rmd files).