This is a sample Python application to pull metrics data from Google Analytics API and push those metrics in JSON form to a Kafka Topic.
You will need to setup several environment variables to get this to work
KAFKA_BROKERS - A comma separted list of your Kafka brokers including port
KEY_FILE - Location of your keyfile. See Hello Analytics Reporting API v4; Python quickstart for service accounts. Follow the instructions under Enable the API -> Create Credentials.
VIEW_ID - You can use the Account Explorer to find a View ID.
TOPIC - The name of the Kafka Topic that you want to publish to.
DELAY - The delay between pulls from Google Analytics. Defaults to 3600s (1 Hour)
Running the app is pretty easy.
$ virtualenv ../venvs/analytics-intake
$ source ../venvs/analytics-intake/bin/activate
$ pip install -r requirements.txt
$ python AnalyticsIntake.py
For more info about this app, checkout my Fast Data Architecture Series on my blog: