Skip to content

Latest commit

 

History

History
40 lines (26 loc) · 1.42 KB

README.md

File metadata and controls

40 lines (26 loc) · 1.42 KB
Raw

Minecart

utility to embellish transport gingerbread CSV forum data into Elasticsearch. Uses the Google Cloud Natural Language API to add analysis data to each forum post.

Building the utility:

  1. Make sure you have stack (install guides here and here).
  2. Build the project with stack build
  3. Create a postgres DB called minecart

Running

to run the utility:

> stack exec minecart

This will show a list of the actions minecart can perform

To run an action:

> stack exec minecart -- --action

Running Options

There are a number of actions minecart can perform

  • --pgsetup adds the tables to the minecart pogstres database and adds forum posts from gb-forum.csv
  • --entities runs google cloud natural language request for each post body and collects the entities in the db
  • --sentences runs google cloud natural language request for each post body and collects sentences sentiments in the db
  • --elastic sets up an elasticsearch index and adds the complete post data to it

Running Order

  • --pgsetup must be run first
  • --entities & --sentences are optional (need an environment var of GOOGLE_CLOUD_API_KEY)
  • finally run --elastic to set up an elasticsearch index with the post data