Data Engineer Tech Challenge

Getting Started

1. Fork the repository.
2. Make your fork Private so other candidates cannot view your submission.
   • Settings > General > Leave fork network > wait for refresh > Change visibility > Change to private
3. Grant access to the assessors (GitHub handles are in your assessment instructions/email).
4. Fill in your details in submission/README.md.
5. Read the challenge details in instructions/README.md. Please use submission/ as your submission folder.

AI Usage Policy

If you intend to use AI Coding Assistants for the submission, please ensure the following:

• AI usage is declared in submission/README.md
• Prompts should be submitted inline together with code changes in the same commits. The prompt logging setup guide for common coding assistants is in prompt-logger/README.md. Please read it before you start.

Note:

• Capturing your prompts is your responsibility. The prompt logger setup covers common tools and setups, but it can't cover every scenario. Please set up your own scripts/workflows to ensure prompts are logged correctly.
• Do not share anything sensitive. Sessions are copied verbatim. Avoid pasting API keys, tokens, or passwords into your AI assistant, and rotate any credentials that may have been exposed before you commit.
• Do not edit or curate submitted prompts. They must be left exactly as captured.

Submission Guidelines

Best Practices (Dos)

• Commit frequently. Make regular, incremental commits to showcase your development process.
• Write descriptive commit messages. Your commit messages should clearly and concisely describe the changes made in each commit.
• Provide clear documentation. Include a main README.md in the root directory. For larger projects, consider adding separate README.md files in relevant sub-directories to explain specific parts of your work and include any instructions needed to run your code.
• Comment your code. Use comments where necessary to clarify complex logic or important implementation decisions.

What to Avoid (Don'ts)

• Single commits. Do not submit your entire project in one large commit.
• Zip files. Submissions in .zip or other archive formats will not be accepted.
• Poor documentation. Avoid sparse, incomplete, or non-existent documentation.
• Unreadable code. Ensure your code is well-structured, clean, and easy to follow.