Feedback

README.md

@@ -1,3 +1,4 @@
+[![Review Assignment Due Date](https://classroom.github.com/assets/deadline-readme-button-22041afd0340ce965d47ae6ef1cefeee28c7c493a6346c4f15d667ab976d596c.svg)](https://classroom.github.com/a/6B-rL6oS)
 # Collaborative Git Workflows
 
 | Key              | Value                                                                                                                                                                                                                                                        |
@@ -15,3 +16,23 @@ Refer to the files in the order specified below for more details:
 
 - [lab_instructions_part1.md](lab_instructions_part1.md)
 - [lab_instructions_part2.md](lab_instructions_part2.md)
+
+# Team Roles:
+Member 1: Maureen Waweru
+Role: Update README.md file with team roles.
+
+Member 2: Emma Karanja
+Role: Research on data sources and create a data_source.md file reviewing sources of data in a business.
+
+Member 3: Austine Namayi
+Role: Research on Star Schema and create a warehouse_schema.md describing a Star Schema.
+
+Member 4: Daisy Gitonga
+Role: Research on ETL, ELT, EtLT and create a data_pipeline.md file differentiating between ETL, ELT, and EtLT in the context of compliance with the legal requirements in an industry.
+
+Member 5: Austin Namayi
+Role: Research on data governance and add a governance.md file reviewing data governance and access to PII.
+
+Project lead: Member 5 - responible for governance and audit.
+Member 5: Kimberly Ondari
+Role: Research on data governance and add a governance.md file reviewing data governance and access to PII.

data_governance.md

@@ -0,0 +1,30 @@
+# Data Governance in Business Intelligence
+
+##  Data Governance
+
+Data Governance is a collection of processes, roles, policies, standards, and metrics that ensure the effective and efficient use of information in enabling an organization to achieve its goals.
+
+## The Pillars of Data Governance
+
+A solid governance framework ensures that data is high-quality and secure. It typically focuses on:
+
+* **Data Integrity:** Ensuring that data remains accurate and consistent throughout its entire lifecycle.
+* **Data Availability:** Ensuring that the right people can access the data when they need it for decision-making.
+* **Data Security:** Protecting sensitive information from unauthorized access or corruption.
+
+## PII (Personally Identifiable Information)
+
+In Business Intelligence, handling PII is a critical responsibility. PII includes any data that could potentially identify a specific individual.
+
+### How PII is Managed:
+
+* **Identification:** We categorize data like Names, Email addresses, and ID numbers as high-risk PII.
+* **Anonymization:** Using techniques like masking to ensure that individual identities are protected during analysis.
+* **Restricted Access:** Only authorized team members with specific roles are granted access to raw PII datasets.
+
+## How it is Used in this Project
+
+Data Governance is essential for our lab because it:
+1. Maintains trust in our reports and dashboards.
+2. Ensures compliance with data protection regulations.
+3. Provides a clear roadmap for how team members should handle sensitive assets.

data_pipeline.md

@@ -0,0 +1,22 @@
+# Differences between ETL, ELT, and EtLT 
+This is a document explaining the differences between ETL, ELT and EtLT in the context of compliance with the legal requirements in an industry.
+## ETL (Extract, Transform, Load)
+In this approach of data pipelining, the data is extracted, transformed and cleaned separately before it is loaded into  the data warehouse and it is best suitable for industries needing strict pre-data sanitization such as healthcare and finance which are governed by Health Insurance Portability and Accountability Act (HIPAA) and General Data Protection Regulation (GDPR) respectively. 
+### Advantage
+It ensures highest data security because sensitive data (Personal Identifiable Information) is masked before long-term storage.
+### Disadvantage
+It is rigid because it is difficult to re-process data if business rules change.
+
+## ELT (Extract, Load, Transform)
+Here, the data is loaded directly into the cloud data warehouse and transformation occurs within the warehouse. Since the data is loaded raw, there is need for strict security in the warehouse such as encryption and strict access controls to prevent unauthorized exposure.
+### Advantage 
+It preserves raw data which is valuable for audits.
+### Disadvantage
+There is risk exposure of the data since it is loaded in the warehouse in its raw form.
+
+## EtLT (Extract, Transform-light, Load, Transform)
+This is a hybrid approach whereby the data undergoes light transformation before it is loaded in the warehouse then goes through heavy transformation inside the data warehouse hence ensuring compliance and also providing an opportunity for cloud scalability for processing.
+### Advantage
+It prevents data swamps by ensuring the data warehouse does not have unmasked sensitive data.
+### Disadvantage
+There is risk of inconsistency due to two different transformation stages.

data_source.md

@@ -0,0 +1,38 @@
+Businesses rely on different data sources to make informed decisions, improve operations, and understand customers.
+
+Types of Data Sources
+
+1. Internal Data
+Data generated within the organization.
+
+Sales records
+Customer databases
+Financial reports
+Employee data
+
+2. External Data
+Data collected from outside the organization.
+
+Market research reports
+Government publications
+Industry reports
+Competitor analysis
+
+3. Primary Data
+Data collected firsthand for a specific purpose.
+
+Surveys
+Interviews
+Observations
+
+4. Secondary Data
+Data that already exists and is reused.
+
+Books and journals
+Online databases
+Reports from other organizations
+Importance
+Supports decision-making
+Identifies trends and opportunities
+Improves efficiency
+Enhances customer understanding

lab_instructions_part0.md

@@ -0,0 +1,182 @@
+# Collaborative Git Workflows: SSH Key Generation for GitHub
+
+An SSH key pair consists of a private key and a public key. The private key
+should be kept secure in your local machine, while the public key will be
+added to your GitHub account to allow for secure authentication.
+
+Summary of key points:
+
+* SSH replaces passwords
+* GitHub trusts your machine, not your authentication session
+* You can have multiple SSH keys for different purposes (e.g.,
+one for authentication and another for signing commits). In this setup, we
+use one SSH key pair for both authentication and signing commits to keep
+it simple.
+* You can have multiple SSH key pairs for different machines, e.g., one
+for your personal laptop and another for your work laptop. This way, if
+one of your machines gets compromised, you can easily revoke the
+corresponding SSH key from your GitHub account without affecting the other
+machine that has not been compromised.
+
+***Note:** Ensure you are using the terminal for all Git operations in this
+lab, not a graphical Git client like the built-in Visual Studio Code Git
+support or GitHub Desktop.*
+
+## Install OpenSSH Client (if not already installed)
+
+### Check if already installed
+
+`ssh -V`
+
+If `ssh -V` returns a version number, you already have OpenSSH installed. If it returns an error, you will need to install it.
+
+<img src="https://cdn.jsdelivr.net/gh/devicons/devicon@latest/icons/windows11/windows11-original.svg" width="40" />
+
+Installing OpenSSH in Windows:
+
+### Install the client (not the server)
+
+```bash
+Add-WindowsCapability -Online -Name OpenSSH.Client~~~~0.0.1.0
+```
+
+<img src="https://cdn.jsdelivr.net/gh/devicons/devicon@latest/icons/linux/linux-original.svg" width="40" /> <img src="https://cdn.jsdelivr.net/gh/devicons/devicon@latest/icons/apple/apple-original.svg" width="40"/>
+
+Installing OpenSSH Client in Linux/Mac is usually unnecessary as it is typically pre-installed. However, if you need to install it, you can use the following commands:
+
+* Execute:
+
+```bash
+sudo apt update
+sudo apt install openssh-client
+```
+
+Verify installation:
+
+```bash
+ssh -V
+```
+
+## Step 1: Open Terminal (Git Bash on Windows or Default Terminal on Linux/Mac)
+
+Execute the following command to generate a new SSH key pair.
+
+```bash
+ssh-keygen -t ed25519 -C "<place your comment here>" -f ~/.ssh/id_ed25519_auth_and_sign
+````
+
+* `-t ed25519` specifies the type of key to create, which is  [https://en.wikipedia.org/wiki/EdDSA](https://en.wikipedia.org/wiki/EdDSA). This is a modern and secure choice for SSH keys.
+
+* The comment can contain the name of the machine. That will help you identify
+which key was used and where it was used from, e.g., a key for your personal
+laptop and another key for your work laptop in future.
+
+* `-f` specifies the file name of the private key as well as the folder where it
+will be saved. The public key will be saved with the same name but with a
+`.pub` extension. For example, in this case, the private key will be saved as `~/.ssh/id_ed25519_auth_and_sign` and the public key will be saved as `~/.ssh/id_ed25519_auth_and_sign.pub`.
+
+* `~` represents the home directory of the current user. On Windows, this translates to `C:\Users\YourUsername`.
+
+After running the command, you will be prompted to enter a passphrase. You can
+leave it empty for no passphrase or enter a secure passphrase for added
+security. If you enter a passphrase, you will need to remember it to use the
+key. In this case, we leave it empty for simplicity, however,
+in a professional setting, it is recommended to use a passphrase for added
+security.
+
+## Step 2: Add the PUBLIC (.pub) Key to Your GitHub Account
+
+Log in to your GitHub account, navigate to "Settings" > "SSH and GPG keys" >
+"New SSH key". Paste **all the contents** of your public key file
+(`~/.ssh/id_ed25519_auth_and_sign.pub`) into the "Key" field and select
+"Authentication Key" as the type.
+
+Make sure that you are copy-pasting your public key, and **NOT** the private key.
+Your private key (`~/.ssh/id_ed25519_auth_and_sign`) should never be shared with anyone.
+The public key is safe to share and is used to authenticate your identity when
+connecting to GitHub.
+
+Give it a descriptive title, e.g., `Git authentication for GitHub from
+<your_laptop_name>`. That way, if your laptop gets compromised, you can
+easily identify which private key was being used for authentication and
+revoke it from your GitHub account.
+
+Click "Add SSH key" to save it.
+
+Register the same public key again, but classify it as a signing key. This
+ way, you can use the same SSH key pair for both authentication and
+ signing commits. You can also choose to use different SSH key pairs for
+ authentication and signing if you prefer, but using the same key pair
+ simplifies the setup.
+
+## Step 3: Add the SSH Key to the Local SSH Agent
+
+First confirm that the SSH agent is running:
+
+```bash
+eval "$(ssh-agent -s)"
+```
+
+On Windows, if this command fails, ensure that the OpenSSH Authentication Agent service is running via Services.
+
+```bash
+git config --global gpg.format ssh
+git config --global user.signingkey ~/.ssh/id_ed25519_auth_and_sign.pub
+git config --global commit.gpgsign true
+```
+
+Then add the private key to the SSH agent:
+
+```bash
+ssh-add ~/.ssh/id_ed25519_auth_and_sign
+```
+
+## Step 4: Test the SSH Connection to GitHub
+
+To verify that your SSH key is correctly set up and can authenticate with GitHub, run the following command:
+
+```bash
+ssh -T git@github.com
+```
+
+You should see a message like this:
+
+```text
+Hi <your_github_username>! You've successfully authenticated, but GitHub does not provide shell access.
+```
+
+This indicates that your SSH key is correctly configured and you can now use it for Git operations with GitHub.
+
+---
+
+However, if you see a message like this:
+
+```text
+Permission denied (publickey).
+```
+
+Or any other error message, you can troubleshoot the issue by running:
+
+```bash
+ssh -vT git@github.com
+```
+
+---
+
+You should now be using SSH for both Git authentication when cloning repositories and for Git signing when committing changes.
+
+![Use SSH not HTTPS](./assets/images/UseSSH_notHTTPS.png)
+
+The cloning command in this case would then be as follows to clone the repository using SSH and store the code in a folder named `Lab-1-Git`:
+
+```bash
+git clone git@github.com:course-files/Git.git Lab-1-Git
+```
+
+The syntax is:
+
+```bash
+git@github.com:<username>/<repository>.git <name-of-repository-in-your-local-machine>
+```
+
+This setup enhances the security of your interactions with GitHub while also providing a convenient way to manage your Git operations.

warehouse_schema.md

@@ -0,0 +1,21 @@
+# Star Schema in Data Warehousing
+
+## What is a Star Schema?
+A Star Schema is a type of database schema used in data warehousing 
+that organizes data into fact and dimension tables, resembling a star shape.
+
+## Fact Tables
+A fact table stores quantitative data (measures) for analysis, such as 
+sales amounts, quantities, or revenue. It sits at the center of the schema 
+and contains foreign keys linking to dimension tables.
+
+## Dimension Tables
+Dimension tables store descriptive attributes related to the facts, 
+such as time, location, product, or customer information. They surround 
+the fact table like points of a star.
+
+## How it is Used in Data Warehousing
+Star Schemas are widely used in Business Intelligence because they:
+- Simplify complex queries
+- Improve query performance
+- Make data easier to understand for analysts