README

Sinbad - (Automated Structure Inference and Binding of Data)
============================================================

Sinbad is a software library that makes it very easy to
uniformly access online data sources provided in standard
formats (CSV, JSON, XML). For more info, please visit:

http://berry-cs.github.io/sinbad



(Each of the examples below is intended to be run in 
Intermediate Student language level. In #lang racket
define the structures #:transparent to inspect results.)

Example 1
---------

Consider the following program, which connects to the
FAA live feed of airport status information (XML format):

    (require sinbad)

    (define faa
      (sail-to "http://services.faa.gov/airport/status/ATL?format=application/xml"
	       (cache-timeout 300)  ; refresh every 5 minutes
	       (manifest)           ; display schema for available data upon load
	       (load)))

    (define-struct port (name loc condition delay?))

    (fetch faa (make-port "Name" "State" "Weather/Weather" "Delay"))


Running this program should produce a value like:

    (make-port "Hartsfield-Jackson Atlanta International" "Georgia" "Fair" #false)


Example 2
---------

The following program connects to the USGS live feed of earthquake
events (JSON format):

    (require sinbad)
    
    (define Q
      (sail-to "http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.geojson"
               (cache-timeout 180)))

    (define-struct quake (place time mag))

    (load Q)   ; can (re)load on demand anytime
    (fetch Q (make-quake "place" "time" "mag") (base-path "features/properties"))    
    (fetch-first Q "features/properties")

Note that the first fetch produces a list of instances of a *user-defined* quake structure. The second form produces an association list.


Example 3
---------

An example of a very large data set (CSV + zip format):

    (require sinbad)

    (define vehicle-data-source
      (sail-to "http://www.fueleconomy.gov/feg/epadata/vehicles.csv.zip"
       (load) (manifest)))

    (define-struct auto (make model year trans city-mpg hwy-mpg))

    ;; pull a random data record...
    (fetch-random vehicle-data-source
          (make-auto "make" "model" "year" "trany" "city08" "highway08"))

    ;; get all ~40,000 (!) of them...
    (define all-vehicles
      (fetch vehicle-data-source
    	 (make-auto "make" "model" "year" "trany" "city08" "highway08")))

    (length all-vehicles) 


Have fun!