Big-data

This is a repo for Big Data Project.

Whats included

within the download you will find the following directories and files.

├── code
│   ├── dataclean
│   │   ├── multiple
│   │   │   ├── ninthAndTenth
│   │   │   └── seventhAndEighth
│   │   └── single
│   └── statistics
│       ├── crimeByBoroughMonthComparison
│       ├── crimeByDay
│       ├── crimeByLoc
│       ├── crimeByMonth
│       ├── crimeByOffenceClassification
│       ├── crimeByPark
│       ├── crimeByPremises
│       ├── crimeByPremisesTop20
│       ├── crimeByRange
│       ├── crimeBySeverity
│       ├── crimeByStatus
│       ├── crimeByTypeGeo
│       └── totalCrime
├── data
│   └── dataclean
├── plot
│   ├── crimeByBorough
│   ├── crimeByDay
│   ├── crimeByLoc
│   ├── crimeByMonth
│   ├── crimeByOccur-hour
│   ├── crimeByOffenceClassification
│   ├── crimeByPark
│   ├── crimeByPrecinct
│   ├── crimeByPremises
│   ├── crimeBySeverity
│   ├── crimeByStatus
│   ├── crimeByTypeGeo
│   │   ├── Drugs
│   │   ├── Larceny
│   │   ├── robbery
│   │   └── weapons
│   ├── crimeNotGeocoded
│   ├── script
│   └── util
└── script
    └── dataclean
        ├── multiple
        └── single

---

Setup

1. download origina data from the internet and upload to dumbo, and execute run.sh
2. run scripts in code/dataclean to clean data
3. run script in code/statistics to extract features
4. data will generate in data/ folder
5. run html files in plot/ to draw figures.

---