Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
4b7cb69
Update Session1_Sequencing.md
seokho92 Sep 12, 2025
4cccace
Update Session1_Sequencing.md
seokho92 Sep 15, 2025
b54cd98
Update Session1_Sequencing.md
seokho92 Sep 16, 2025
1b24884
Update Session1_Sequencing.md
seokho92 Sep 16, 2025
0e65001
Update Session1_Sequencing.md
seokho92 Sep 16, 2025
f6aafa9
Update Session1_Sequencing.md
seokho92 Sep 16, 2025
0bdbfd6
Update Session1_Sequencing.md
seokho92 Sep 16, 2025
4b34f2d
Update README.md
seokho92 Oct 2, 2025
0277fcf
Update Session3_PRS.md
nayeonkim93 Oct 2, 2025
83e3940
Update GWAS session details and commands
namks Oct 3, 2025
43b8b67
Update Session4_Mendelian_Randomization.Md
exagorazo Oct 23, 2025
f8c3760
Update README.md
exagorazo Oct 23, 2025
c0dcefa
Update README.md
exagorazo Oct 23, 2025
3bc0199
Update session dates and titles in README.md
chkang96 Oct 23, 2025
f4825c2
Update Session4_Mendelian_Randomization.Md
exagorazo Oct 24, 2025
b20dba1
Update Session4_Mendelian_Randomization.Md
exagorazo Oct 27, 2025
3cc4ecf
Update Session4_Mendelian_Randomization.Md
exagorazo Oct 27, 2025
77fc29e
Update Session4_Mendelian_Randomization.Md
exagorazo Oct 28, 2025
a64d7d4
Update Session4_Mendelian_Randomization.Md
exagorazo Oct 28, 2025
84d32e0
Update Session6_Medical_Note_LLM.md
sungmoonie Nov 24, 2025
aa1e164
Update session in README.md
sungmoonie Nov 24, 2025
aa8c6f5
Update session in README.md
sungmoonie Nov 24, 2025
0008440
Add Session6 Folder
sungmoonie Nov 24, 2025
11f3221
Update Session6_Medical_Note_LLM.md
sungmoonie Nov 24, 2025
7b81334
Update Session6_Medical_Note_LLM.md
sungmoonie Nov 24, 2025
6959d8e
Update Session6_Medical_Note_LLM.md
sungmoonie Nov 24, 2025
6e9533f
Update Session6_Medical_Note_LLM.md
sungmoonie Nov 24, 2025
dcb9a35
Add files via upload
Stomper10 Nov 28, 2025
7c427c0
Rename config_maisi_vae_train_brats_stage2 to config_maisi_vae_train_…
Stomper10 Nov 28, 2025
7135565
Update maisi_train_VAE_brats.py
Stomper10 Dec 1, 2025
721f428
Update maisi_train_UNET_brats.py
Stomper10 Dec 1, 2025
1a9efa6
Update README.md
Stomper10 Dec 1, 2025
7c540b3
Update README.md
Stomper10 Dec 1, 2025
2bae29d
Add files via upload
Stomper10 Dec 1, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 12 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# GCDA_Tutorial_2025

This repository is for the Genomic & Clinical Data Analysis course for the Spring 2025 semester.
This repository is for the Genomic & Clinical Data Analysis course for the Fall 2025 semester.

Processes and codes for various practice sessions will be shared here. Please note, the schedule is subject to change.

Expand All @@ -10,23 +10,27 @@ Processes and codes for various practice sessions will be shared here. Please no
**TA:** Seokho Jeong ([email](mailto:seokho92@snu.ac.kr))

### [Session 2 - GWAS](Session2_GWAS.md) (HW)
**Date:** TBA
**Date:** October 7, 2025
**TA:** Kisung Nam ([email](mailto:kisung.nam@snu.ac.kr))

### [Session 3 - PRS](Session3_PRS.md) (HW)
**Date:** TBA
**Date:** October 17, 2025
**TA:** Na Yeon Kim ([email](mailto:nayeonkim1@snu.ac.kr))

### [Session 4 - Mendelian Randomization](Session4_Mendelian_Randomization.Md)
**Date:** TBA
**Date:** October 28, 2025
**TA:** Bo-Eun Ahn ([email](mailto:ahnbo@snu.ac.kr))

### Session 5 - EHR
**Date:** TBA
### [Session 5 - Prediction using EHR (RNN)] (HW)
**Date:** November 18, 2025
**TA:** Junhyeong Lee ([email](mailto:lrainsoul@snu.ac.kr))

### Session 6 - Medical Image Analysis (HW)
**Date:** TBA
### [Session 6 - Medical Note and LLM](Session6_Medical_Note_LLM.md)
**Date:** November 25, 2025
**TA:** Soungmun Kim ([email](mailto:soungmun@snu.ac.kr))

### [Session 7 - Medical Image Data Analysis (CNN)]
**Date:** December 2, 2025
**TA:** Wonyoung Jang ([email](mailto:jwy4888@snu.ac.kr))

*For more details on each session, please refer to the specific folder within this repository.*
116 changes: 68 additions & 48 deletions Session1_Sequencing.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# Practice Session #1: Sequencing (Mar. 20, 2025)
# Practice Session #1: Sequencing (Sep. 16, 2025)

In this session, we will learn how to convert raw unmapped read files (`FASTQ`) to analysis-ready files (`VCF`). \
The overall process in this session is mainly based on the [GATK Best Practice](https://gatk.broadinstitute.org/hc/en-us/categories/360002302312-Getting-Started). \
This document was created on Mar. 19, 2025 and the following contents were tested on local WSL (Ubuntu 22.04.1 LTS) + GSDS Cluster.
### 0. Installing Linux and Anaconda in Windows
This document was created on Sep. 16, 2025 and the following contents were tested on local WSL (Ubuntu 22.04.1 LTS) + GSDS Cluster.
### 0. Installing Linux and Mamba(Anaconda) in Windows
Using Linux has become easy in Windows with WSL. \
To start, launch windows powershell in administration mode and run following.
```
Expand All @@ -24,34 +24,45 @@ ID: gcda_<xxx>
PW (default): gcda_<xxx>

```
### Use designated node for computation: leelabsg15
After login, use ssh to log on to designated node for GCDA 2025 Spring
#### Installing mamba for faster conda usage
Follow the instruction from https://github.com/conda-forge/miniforge

```
bash Miniforge3-Linux-x86_64.sh -b -p "${HOME}/conda"
conda/bin/conda init
```

### Use designated node for computation: leelabsg11
After login, use ssh to log on to designated node for GCDA 2025 Fall
```
ssh leelabsg15
ssh leelabsg11
```

### 1. Setting up the environment

We will use the Anaconda environment on the GSDS cluster. \
It is already created on the GSDS cluster, but you can create the environment on your local machine with the following command \
In this session, OpenJDK, samtools, GATK and BWA are installed in creation of conda environment and Picard is downloaded as java package \
All files and tools are included in '~/GCDA/1_sequencing/' folder
All files and tools are included in '~/1_sequencing/' folder
```
# Create conda environment and install softwares
conda create -n SEQ samtools bwa -c anaconda -c bioconda
# Create conda/mamba environment and install softwares
# conda create -n SEQ samtools bwa -c anaconda -c bioconda
mamba create -n SEQ samtools bwa bcftools -c anaconda -c bioconda

conda activate SEQ

# Install jdk 17 version (Required after picard 3.0.0)
## wget https://download.java.net/java/GA/jdk17.0.2/dfd4a8d0985749f896bed50d7138ee7f/8/GPL/openjdk-17.0.2_linux-x64_bin.tar.gz
tar xvf ~/GCDA/1_sequencing/utils/openjdk-17.0.2_linux-x64_bin.tar.gz
tar xvf ~/1_sequencing/utils/openjdk-17.0.2_linux-x64_bin.tar.gz
export JAVA_HOME=$JAVA_HOME/:~/jdk-17.0.2/
export PATH=$JAVA_HOME/bin:$PATH
alias java17="~/jdk-17.0.2/bin/java"

# install gatk4
conda install gatk4 -c bioconda
mamba install gatk4 -c bioconda
# Download Picard (Find Latest Release: https://github.com/broadinstitute/picard/releases/latest)
## cd ~/GCDA/1_sequencing/utils
## cd ~/1_sequencing/utils
## wget https://github.com/broadinstitute/picard/releases/download/3.0.0/picard.jar

```
Expand All @@ -62,9 +73,10 @@ To map our raw unmapped reads, we need the reference panel and the information f
Here, we will use the `FASTA` file of 1000 Genome Phase 3 (GRCh37 build) and the `VCF` file for known variants. \
You can browse [FTP server](ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/) of 1000 Genome Project.

#### Already done in your user account
```
mkdir -p ~/GCDA/1_sequencing/reference
cd ~/GCDA/1_sequencing/reference
mkdir -p ~/1_sequencing/reference
cd ~/1_sequencing/reference
# Download 1000 Genome reference panel
## wget http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/technical/reference/human_g1k_v37.fasta.gz
## gzip -d human_g1k_v37.fasta.gz
Expand Down Expand Up @@ -97,7 +109,7 @@ gatk CreateSequenceDictionary -R human_g1k_v37.fasta
And we need a sequence read file (`FASTQ`) for the sample individual (HG00096).
This can be downloaded from ftp server wih project description : [1000 Genome Project Phase 3](https://www.internationalgenome.org/1000-genomes-summary)
```
cd ~/GCDA/1_sequencing/raw_reads
cd ~/1_sequencing/raw_reads
# Download sequence read file from 1000 Genome
# sample HG00096
## wget http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/phase3/data/HG00096/sequence_read/SRR062634.filt.fastq.gz
Expand Down Expand Up @@ -175,21 +187,21 @@ This practice session consists of 4 steps.
Using `FastqToSam` function of Picard, we can convert the `FASTQ` file to an unmapped `BAM` file.

```
mkdir -p ~/GCDA/1_sequencing/data/
mkdir -p ~/1_sequencing/data/
SID=HG00096
java17 -jar ~/GCDA/1_sequencing/utils/picard.jar FastqToSam \
F1=~/GCDA/1_sequencing/raw_reads/SRR062634.filt.fastq \
O=~/GCDA/1_sequencing/data/fastq_to_bam_${SID}.bam \
java17 -jar ~/1_sequencing/utils/picard.jar FastqToSam \
F1=~/1_sequencing/raw_reads/SRR062634.filt.fastq \
O=~/1_sequencing/data/fastq_to_bam_${SID}.bam \
SM=${SID}
```

#### Add read group information in `BAM` file

read group is a set of sequences (in one or more fastq files) having a common set of metadata. This metadata generally includes patient/sample ID, library ID (the library is the preparation of the patient/sample DNA.
We can add read groups in `BAM` file by the following command:

```
cd ~/GCDA/1_sequencing/data/
java17 -jar ~/GCDA/1_sequencing/utils/picard.jar AddOrReplaceReadGroups \
cd ~/1_sequencing/data/
java17 -jar ~/1_sequencing/utils/picard.jar AddOrReplaceReadGroups \
I=fastq_to_bam_${SID}.bam \
O=add_read_groups_${SID}.bam \
RGID=4 \
Expand All @@ -203,8 +215,10 @@ RGSM=20

Using `MarkIlluminaAdapters`, we can mark adapter sequences.

Adapter sequences are short, synthetic DNA segments attached to the ends of fragmented DNA or RNA libraries for next-generation sequencing (NGS)

```
java17 -Xmx8G -jar ~/GCDA/1_sequencing/utils/picard.jar MarkIlluminaAdapters \
java17 -Xmx8G -jar ~/1_sequencing/utils/picard.jar MarkIlluminaAdapters \
I=add_read_groups_${SID}.bam \
O=mark_adapter_${SID}.bam \
M=mark_adapter_${SID}.metrics.txt
Expand All @@ -213,7 +227,7 @@ M=mark_adapter_${SID}.metrics.txt
#### Convert the preprocessed `BAM` file to a `FASTQ` file

```
java17 -Xmx8G -jar ~/GCDA/1_sequencing/utils/picard.jar SamToFastq \
java17 -Xmx8G -jar ~/1_sequencing/utils/picard.jar SamToFastq \
I=mark_adapter_${SID}.bam \
FASTQ=fastq_input_${SID}.fq \
CLIPPING_ATTRIBUTE=XT \
Expand All @@ -229,7 +243,7 @@ NON_PF=true
GATK's variant discovery workflow recommends Burrows-Wheeler Aligner's maximal exact matches (BWA-MEM) algorithm.

```
bwa mem -M -t 7 -p ~/GCDA/1_sequencing/reference/human_g1k_v37.fasta ~/GCDA/1_sequencing/data/fastq_input_${SID}.fq > aligned_${SID}.sam
bwa mem -M -t 7 -p /data/GCDA_2025_2/1_sequencing/reference/human_g1k_v37.fasta ~/1_sequencing/data/fastq_input_${SID}.fq > aligned_${SID}.sam
```

You can check the contents of the `SAM` file:
Expand Down Expand Up @@ -261,10 +275,10 @@ The information of some columns are as follows:
* Column 11: ASCII representation of phred-scale base quality

#### Add information to `BAM` file using `MergeBamAlignment`

the alignment information from the mapped file (bwa) + the metadata → one standardized, analysis-ready BAM.
```
java17 -Xmx10G -jar ~/GCDA/1_sequencing/utils/picard.jar MergeBamAlignment \
R=~/GCDA/1_sequencing/reference/human_g1k_v37.fasta \
java17 -Xmx10G -jar ~/1_sequencing/utils/picard.jar MergeBamAlignment \
R=/data/GCDA_2025_2/1_sequencing/reference//human_g1k_v37.fasta \
UNMAPPED=add_read_groups_${SID}.bam \
ALIGNED=aligned_${SID}.sam \
O=preprocessed_${SID}.bam \
Expand All @@ -281,7 +295,7 @@ ATTRIBUTES_TO_RETAIN=XS
#### Mark Duplicates

```
java17 -jar ~/GCDA/1_sequencing/utils/picard.jar MarkDuplicates \
java17 -jar ~/1_sequencing/utils/picard.jar MarkDuplicates \
I=preprocessed_${SID}.bam \
O=mark_dup_${SID}.bam \
M=mark_dup_${SID}.metrics.txt
Expand All @@ -290,19 +304,19 @@ M=mark_dup_${SID}.metrics.txt
#### Sort, index and convert alignment to a BAM using SortSam

```
java17 -jar ~/GCDA/1_sequencing/utils/picard.jar SortSam \
java17 -jar ~/1_sequencing/utils/picard.jar SortSam \
I=mark_dup_${SID}.bam \
O=sorted_${SID}.bam \
SO=coordinate
```

#### Create Recalibration Table using `BaseRecalibrator`

[BQSR Explaination](https://gatk.broadinstitute.org/hc/en-us/articles/360035890531-Base-Quality-Score-Recalibration-BQSR)
```
gatk --java-options '-Xmx10g' BaseRecalibrator \
-I sorted_${SID}.bam \
-R ~/GCDA/1_sequencing/reference/human_g1k_v37.fasta \
--known-sites ~/GCDA/1_sequencing/reference/ALL.wgs.mergedSV.v8.20130502.svs.genotypes.vcf.gz \
-R /data/GCDA_2025_2/1_sequencing/reference/human_g1k_v37.fasta \
--known-sites /data/GCDA_2025_2/1_sequencing/reference/ALL.wgs.mergedSV.v8.20130502.svs.genotypes.vcf.gz \
-O recal_data_${SID}.table
```

Expand All @@ -311,15 +325,15 @@ gatk --java-options '-Xmx10g' BaseRecalibrator \
```
gatk --java-options '-Xmx10g' ApplyBQSR \
-I sorted_${SID}.bam \
-R ~/GCDA/1_sequencing/reference/human_g1k_v37.fasta \
-R /data/GCDA_2025_2/1_sequencing/reference/human_g1k_v37.fasta \
--bqsr-recal-file recal_data_${SID}.table \
-O bqsr_${SID}.bam
```
#### Convert other samples into processed bam
```
for SID in HG00096 HG00097 HG00098
do
bash ~/GCDA/1_sequencing/bam_proc.sh ${SID}
bash ~/1_sequencing/bam_proc.sh ${SID}
done
```

Expand All @@ -330,7 +344,7 @@ We can visualize the aligned `BAM` file with the [IGV viewer](https://software.b

For example, we can observe high coverage around SUMO1P1 gene. (`HG00096.chrom20.ILLUMINA.bwa.GBR.exome.20120522.bam`)
```
cd ~/GCDA/1_sequencing/raw_reads
cd ~/1_sequencing/raw_reads
## wget http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/phase3/data/HG00096/exome_alignment/HG00096.chrom20.ILLUMINA.bwa.GBR.exome.20120522.bam
## wget http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/phase3/data/HG00096/exome_alignment/HG00096.chrom20.ILLUMINA.bwa.GBR.exome.20120522.bam.bai
## wget http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/phase3/data/HG00097/exome_alignment/HG00097.chrom20.ILLUMINA.bwa.GBR.exome.20130415.bam
Expand All @@ -347,11 +361,11 @@ In this section, we will use the real data of 3 individuals in 1000 Genome Proje
We can convert these `BAM` files to `GVCF` files.

```
cd ~/GCDA/1_sequencing/data
cd ~/1_sequencing/data
# Convert BAM for the first individual
gatk --java-options "-Xms4g" HaplotypeCaller \
-R ~/GCDA/1_sequencing/reference/human_g1k_v37.fasta \
-I ~/GCDA/1_sequencing/raw_reads/HG00096.chrom20.ILLUMINA.bwa.GBR.exome.20120522.bam \
-R /data/GCDA_2025_2/1_sequencing/reference/human_g1k_v37.fasta \
-I ~/1_sequencing/raw_reads/HG00096.chrom20.ILLUMINA.bwa.GBR.exome.20120522.bam \
-L 20 \
-ERC GVCF \
-O sample01_20.g.vcf
Expand All @@ -360,8 +374,8 @@ gatk --java-options "-Xms4g" HaplotypeCaller \
```
# Convert BAM for the second individual
gatk --java-options "-Xms4g" HaplotypeCaller \
-R ~/GCDA/1_sequencing/reference/human_g1k_v37.fasta \
-I ~/GCDA/1_sequencing/raw_reads/HG00097.chrom20.ILLUMINA.bwa.GBR.exome.20130415.bam \
-R /data/GCDA_2025_2/1_sequencing/reference/human_g1k_v37.fasta \
-I ~/1_sequencing/raw_reads/HG00097.chrom20.ILLUMINA.bwa.GBR.exome.20130415.bam \
-L 20 \
-ERC GVCF \
-O sample02_20.g.vcf
Expand All @@ -370,29 +384,38 @@ gatk --java-options "-Xms4g" HaplotypeCaller \
```
# Convert BAM for the third individual
gatk --java-options "-Xms4g" HaplotypeCaller \
-R ~/GCDA/1_sequencing/reference/human_g1k_v37.fasta \
-I ~/GCDA/1_sequencing/raw_reads/HG00099.chrom20.ILLUMINA.bwa.GBR.exome.20130415.bam \
-R /data/GCDA_2025_2/1_sequencing/reference/human_g1k_v37.fasta \
-I ~/1_sequencing/raw_reads/HG00099.chrom20.ILLUMINA.bwa.GBR.exome.20130415.bam \
-L 20 \
-ERC GVCF \
-O sample03_20.g.vcf
```
#### What GVCF provides us : SNV
```
bcftools view -v snps sample03_20.g.vcf | grep -v '##' | head
```
#### What GVCF provides us : Insertion/Deletion
```
bcftools view -v indels sample03_20.g.vcf | grep -v '##' | head
```
<img width="850" height="578" alt="image" src="https://github.com/user-attachments/assets/89a3c5c8-39e2-44fa-af8f-ac40344431ab" />

#### Combine individual `GVCF` files

```
gatk CombineGVCFs \
-R ~/GCDA/1_sequencing/reference/human_g1k_v37.fasta \
-R /data/GCDA_2025_2/1_sequencing/reference/human_g1k_v37.fasta \
--variant sample01_20.g.vcf \
--variant sample02_20.g.vcf \
--variant sample03_20.g.vcf \
-O sample_all.g.vcf.gz
```

### 7. Converting `GVCF` to `VCF`
### 7. Converting `GVCF` to `VCF` (Genotyping)

```
gatk --java-options "-Xmx4g" GenotypeGVCFs \
-R ~/GCDA/1_sequencing/reference/human_g1k_v37.fasta \
-R /data/GCDA_2025_2/1_sequencing/reference//human_g1k_v37.fasta \
-V sample_all.g.vcf.gz \
-O sample_all.vcf
```
Expand All @@ -417,6 +440,3 @@ DeepVariant is a deep learning-based variant caller that takes aligned reads (in

The simple concept of Deepvariant is explained and visualized in following post:
[Looking Through DeepVariant's Eyes](https://google.github.io/deepvariant/posts/2020-02-20-looking-through-deepvariants-eyes/)

### 9. [Geneformer](https://github.com/jkobject/geneformer) & [Enformer](https://github.com/lucidrains/enformer-pytorch)
Gene Expression prediction models or single-cell resolution foundation model via transformer.
Loading