GitHub use handling sensitive data

[MarikaBerntien]

Hey everyone,
I have a question and would really appreciate hearing your oppinon.

In research, we do often work with unpublished data, and especially in this course, we are learning how to analyze such data using R.
How do you view the use of GitHub in this context? When committing local files and pushing them to GitHub, potentially sensitive data may be stored on external servers. How do you usually handle this situation or what do you think about that?

Thank you so much!

[DavidRach]

Hello @MarikaBerntienBER,

Obviously, this will depend partly on what kinds of data you are working (more basic research vs. clinical trial data) and what your group (collaborators, supervisors, etc.) think is appropriate.

On the lower end of the scale, you can ensure that your GitHub repository that will contain the code or any data summary outputs, etc. is set to private, and only share access to others in your group as collaborators. For our more academic research projects this is often the route we end up taking. While this prevents non-designated people from easily accessing it, it is still on an external server (albeit Microsoft's). So if GitHub/Microsoft are ever badly hacked, it could be at risk (assuming the hackers care about our field of study). Similarly, there is also debate whether GitHub/Microsoft is training their next Copilot AI model on private repositories.

Stepping further, one thing that you can do is treat GitHub as just your code repository only, and ensure no data inputs or outputs are being committed to version control. Using the .gitignore file (i.e. the do-not-track-with-version-control-list), you can add your data folder equivalent to it.

So within .gitignore

# Ignore data and outputs folders
data/
outputs/

Any files that you add are never passed to GitHub, and therefore never end up here. Likewise, if you don't want any reports, summaries, etc. also ending up here, as long as these are all getting saved to a designated folder (output or something similarly named), this could also be added to the .gitignore. A similar thing can be similarly accomplished by instead of keeping "data" or "output" folder in your working directory, you set within a code the file.path to retrieve/store the data from elsewhere in in your computer directory.

The above option ensures that you have access to your code via GitHub, that it is version controlled and backed up, and that you can access it quickly from multiple workstations, while ensuring that nothing important is being stored there.

Another step further (more relevant for clinical/industry) is there are GitHub alternatives that allow you to self-host. Git is still installed on your local computer, but the push/pull goes to your designated external. I am less familiar with this setup since haven't been in a setting where I needed to use this one yet.

You can also go a step further and not use GitHub at all, and just use Git locally for the version control. But you are then responsible for making sure to periodically back up your code elsewhere in case your local computer fails.

So all in all, it's a gradient. In my case, for general academic research, I keep the data folders local (since full .fcs files easily exceed the file size limit), but have my code and most outputs backed up. This has been more helpful to the group as a whole, since whenever we need code or a report, I have ready access to it. For the more clinical projects, the code is the only thing that resides on GitHub (and is written in such a way to not reveal additional details).

Hope this helps provide additional ideas to think about!

Best-
David

[MarikaBerntien]

Thank you very much for this detailed answer! I really appreciate it, and will try what works best for me and my data!