Learning Analytics Data Analysis

A comprehensive data preprocessing and exploratory analysis pipeline for the Open University Learning Analytics (OULAD) dataset.

📋 Overview

This project provides a complete preprocessing pipeline for analyzing student learning behavior, engagement patterns, and performance outcomes using the OULAD dataset. The pipeline includes data loading, cleaning, feature engineering, and statistical analysis.

🗂️ Project Structure

anonymisedData/
├── cells/                      # Preprocessing modules
│   ├── load_data.py           # Load all CSV files
│   ├── handle_missing.py      # Handle missing values
│   ├── fix_datatypes.py       # Fix data types
│   ├── sample_vle.py          # Sample large VLE data
│   ├── summary_stats.py       # Generate statistics
│   ├── merge_datasets.py      # Merge datasets
│   ├── feature_engineering.py # Create derived features
│   └── save_data.py           # Save processed data
├── notebook.ipynb             # Main analysis notebook
├── documentation.md           # Dataset schema documentation
├── requirements.txt           # Python dependencies
└── README.md                  # This file

🚀 Getting Started

Prerequisites

• Python 3.8 or higher
• Jupyter Notebook or JupyterLab

1. Clone the Repository

git clone https://github.com/chucknuris/LTaLA-Exercises.git
cd LTaLA-Exercises

2. Download the Dataset

Download the Open University Learning Analytics Dataset and place the following CSV files in the project root directory:

Required files:

• courses.csv
• assessments.csv
• vle.csv
• studentInfo.csv
• studentRegistration.csv
• studentAssessment.csv
• studentVle.csv

Note: The dataset is not included in this repository due to its size. You can obtain it from the Open University Learning Analytics Dataset page.

3. Install Dependencies

pip install -r requirements.txt

4. Run the Analysis

Open the Jupyter notebook:

jupyter notebook notebook.ipynb

Execute cells step by step:

The notebook is organized into sequential steps:

1. Load Data - Import all CSV files
2. Handle Missing Values - Clean and analyze missing data
3. Fix Data Types - Convert to appropriate types
4. Sample VLE Data - Create stratified sample of large VLE file
5. Summary Statistics - Generate descriptive statistics
6. Merge Datasets - Create combined datasets
7. Feature Engineering - Create derived features
8. Save Results - Export processed data

📊 Features Created

The pipeline automatically creates 60+ features including:

Engagement Metrics

• Click intensity and engagement levels
• VLE interaction patterns
• Assessment activity

Performance Indicators

• Performance levels and risk scores
• Score consistency and improvement
• Submission behavior

Temporal Features

• Registration timing
• Activity patterns
• Engagement duration

Demographic Features

• Experience levels
• Study load categories
• Education groupings

Success Indicators

• Success predictions
• Completion status
• Risk assessment

📈 Output

The pipeline generates:

• Enriched Dataset: processed_data/student_profile_enriched.csv
  • Contains all original data plus engineered features
• Data Dictionary: processed_data/data_dictionary.csv
  • Descriptions of all columns
• Summary Report: processed_data/data_summary_report.txt
  • Statistical overview
• Merged Datasets: Various combined datasets for analysis

📖 Documentation

• Dataset Schema: See documentation.md for detailed description of all data files
• Data Dictionary: Generated automatically with feature descriptions
• Code Documentation: All functions include docstrings

🔧 Preprocessing Modules

Each module in the cells/ folder can be imported independently:

from cells import load_data, handle_missing, fix_datatypes
from cells import sample_vle, merge_datasets, feature_engineering

# Load and preprocess data
datasets, _ = load_data.run()
datasets, _, _ = handle_missing.run(datasets)
datasets = fix_datatypes.run(datasets)

📝 Key Notes

• The studentVle.csv file is very large (>200MB). The pipeline automatically creates a 10% stratified sample
• All preprocessing steps preserve data quality and relationships
• Missing values are handled intelligently based on data semantics
• Feature engineering is based on educational research best practices

🤝 Contributing

This project was created for learning analytics research and educational data mining exercises.

📄 License

This project is for educational purposes. Please refer to the OULAD dataset license for data usage terms.

🙏 Acknowledgments

• Open University for providing the Learning Analytics Dataset
• Built for Learning Through Analytics exercises (LTaLA)

📧 Contact

For questions or issues, please open an issue on GitHub.

---

Happy Analyzing! 📊🎓