Functionality to email users who don't have metadata added to files

[abuzarmahmood]

Use dataset_frame.csv generated by blech_data_sentry.py to see which datasets and which users don't have metadata added
Allow functionality to check and email periodically (e.g. every week)

[abuzarmahmood]

Summary of Suggestions and Changes

1. Initial Analysis and Suggestions:

   • The issue involves adding functionality to email users who have datasets without metadata. The dataset_frame.csv generated by blech_data_sentry.py should be used to identify such datasets. Additionally, the system should periodically check for missing metadata and send emails accordingly.
   • Suggested files for modification:
     • File: src/blech_data_sentry.py
       • Description: This script scans the server directory for datasets and checks if they have accompanying metadata. It generates a dataset_frame.csv file that can be used to identify datasets without metadata.
       • Suggested Modifications:
         • Add functionality to read the dataset_frame.csv and identify datasets without metadata.
         • Implement a mechanism to periodically check this CSV (e.g., using a scheduler like cron or a Python scheduling library).
         • Integrate an email sending feature to notify users about missing metadata.
     • File: tests/test_blech_data_sentry.py
       • Description: This file likely contains tests for the blech_data_sentry.py script.
       • Suggested Modifications:
         • Add tests to verify the new functionality for checking datasets without metadata and sending emails.
         • Ensure that the tests cover scenarios such as successful email sending, handling of missing metadata, and periodic checks.

2. Detailed Plan and Code Changes:

   • Add Functionality to Identify Missing Metadata and Send Emails

     • File: src/blech_data_sentry.py
     • Lines: After line 222
     • Description: Add a function to read dataset_frame.csv, identify datasets without metadata, and send emails to users.
     • Code Snippet:

       import smtplib
       from email.mime.text import MIMEText
       from email.mime.multipart import MIMEMultipart
       
       def send_email(user_email, dataset_info):
           sender_email = "your_email@example.com"
           receiver_email = user_email
           password = "your_password"
       
           message = MIMEMultipart("alternative")
           message["Subject"] = "Missing Metadata Notification"
           message["From"] = sender_email
           message["To"] = receiver_email
       
           text = f"Dear User,\\n\\nThe following datasets are missing metadata:\\n{dataset_info}\\n\\nPlease update them at your earliest convenience."
           part = MIMEText(text, "plain")
           message.attach(part)
       
           with smtplib.SMTP_SSL("smtp.example.com", 465) as server:
               server.login(sender_email, password)
               server.sendmail(sender_email, receiver_email, message.as_string())
       
       def check_and_notify():
           dataset_frame = pd.read_csv('dataset_frame.csv')
           missing_metadata = dataset_frame[~dataset_frame['metadata_present']]
           for index, row in missing_metadata.iterrows():
               user_email = get_user_email(row['root_dir'])  # Implement this function to map root_dir to user email
               send_email(user_email, row['data_dir'])

   • Implement Periodic Check

     • File: src/blech_data_sentry.py
     • Lines: After line 225
     • Description: Use a scheduling library to periodically run the check and notify function.
     • Code Snippet:

       from apscheduler.schedulers.blocking import BlockingScheduler
       
       def schedule_checks():
           scheduler = BlockingScheduler()
           scheduler.add_job(check_and_notify, 'interval', weeks=1)
           scheduler.start()
       
       if __name__ == "__main__":
           main()
           schedule_checks()

   • Add Tests for New Functionality

     • File: tests/test_blech_data_sentry.py
     • Lines: After line 335
     • Description: Add tests for email sending and periodic checks.
     • Code Snippet:

       @patch('src.blech_data_sentry.send_email')
       def test_check_and_notify(mock_send_email, mock_server_structure):
           (_, server_path), _ = mock_server_structure
           dataset_frame = pd.DataFrame({
               'root_dir': ['user1', 'user2'],
               'data_dir': ['user1/experiment1', 'user2/experiment3'],
               'metadata_present': [True, False]
           })
           dataset_frame.to_csv('dataset_frame.csv', index=False)
       
           with patch('src.blech_data_sentry.get_user_email', return_value='user@example.com'):
               check_and_notify()
               mock_send_email.assert_called_once_with('user@example.com', 'user2/experiment3')

By implementing these changes, the repository will be able to notify users about missing metadata in their datasets periodically.

---

This response was automatically generated by blech_bot using model gpt-4o