Check

Verify breaking news online.

Installation

Non-Docker-based

• Configure config/config.yml, config/database.yml, config/initializers/errbit.rb and config/initializers/secret_token.rb (check the example files)
• Run bundle install
• Run bundle exec rake db:migrate
• Create an API key: bundle exec rake lapis:api_keys:create
• Start the server: rails s
• Go to http://localhost:3000/api and use the API key you created

You can optionally use Puma, which allows you to restart the Rails server by doing: touch tmp/restart.txt. In order to do that, instead of rails s, start the server with bundle exec pumactl start.

Docker-based

• You can also start the application on Docker by running rake lapis:docker:run (it will run on port 3000 and your local hostname) - you first need to create an API key after entering the container (lapis:docker:shell) before using the web interface

Running the tests

• bundle install --without nothing
• RAILS_ENV=test bundle exec rake db:migrate
• RAILS_ENV=test bundle exec rake test:coverage

Integration

Other applications can communicate with this service (and test this communication) using the client library, which can be automatically generated. Other applications can also use functions exposed by this application in the client library. In order to do this, just add a tag @expose before the method definition, like this:

  # Other comments come here
  # @expose
  def this_function_will_be_exposed(params)
    # It's important that the exposed function can work standalonely
  end

Rake tasks

There are rake tasks for a few tasks (besides Rails' default ones). Run them this way: bundle exec rake <task name>

• test:coverage: Run all tests and calculate test coverage
• application=<application name> lapis:api_keys:create: Create a new API key for an application
• lapis:api_keys:delete_expired: Delete all expired keys
• lapis:error_codes: List all error codes that this application can return
• lapis:licenses: List the licenses of all libraries used by this project
• lapis:client:ruby: Generate a client Ruby gem, that allows other applications to communicate and test this service
• lapis:client:php: Generate a client PHP library, that allows other applications to communicate and test this service
• lapis:docs: Generate the documentation for this API, including models and controllers diagrams, Swagger, API endpoints, licenses, etc.
• lapis:docker:run: Run the application in Docker
• lapis:docker:shell: Enter the Docker container
• lapis:graphql:schema: Update the GraphQL schema JSON
• swagger:docs:markdown: Generate the documentation in markdown format
• transifex:localize: Localize the application using Transifex and I18n

GraphQL

There is a GraphQL interface that exposes the data model as a GraphQL schema. The GraphQL files should be under app/graph.

You can update the schema file by running rake lapis:graphql:schema.

Background processing

Some tasks run in background, for example: Slack notifications. They are processed using Sidekiq. Start Sidekiq with bundle exec sidekiq and monitor through the web interface at /sidekiq. We suggest that you protect that path with HTTP authentication.

Virus validation for uploaded files

In order to look for viruses on the files uploaded by users, you need to setup the configuration option clamav_service_path, which should be something like: host:port. A ClamAV service should be running at that address. If that configuration option is not set, uploaded files will skip the safety validation.

You can also test your instance of ClamAV REST this way:

• Set the test configuration clamav_service_path to point to your instance
• Run this: bundle exec ruby test/models/uploaded_file_test.rb -n /real/
• Two tests should pass

The test uses a EICAR file (a test file which is recognized as a virus by scanners even though it's not really a virus).

Localization

Localization is powered by Transifex + I18n. In order to localize the application, you need to set the transifex_user and transifex_password configuration options on config/config.yml. Then, when you run rake transifex:localize, the following will happen automatically:

• The supported languages on Transifex will be set as the available languages for I18n on config/application.rb
• New translations will be downloaded from Transifex and saved under config/locales
• New localizable strings will be parsed from code, saved on config/locales/en.yml and sent to Transifex

We call "localizable strings" any call to the I18n.t function like this: I18n.t(:string_unique_id, default: 'English string').

Clients should send the Accept-Language header in order to get localized content. If you want to serve everything in English, just add locale: 'en' to your config/config.yml.

Update CLDR data

CLDR data lives in data/ (symlinked as cldr-data too) and contains data from CLDR download by ruby-cldr. In order to update its contents (as explained here), run this in Rails console:

require 'cldr/thor'
thor = Cldr::Thor.new
thor.download
thor.export

Admin UI

Add new settings fields

• Create a method on model to receive the data and set it to the setting field. For example, in app/models/team.rb:

  def media_verification_statuses=(statuses)
    self.send(:set_media_verification_statuses, statuses)
  end

• Configure the fields on Admin UI (config/initializers/rails_admin.rb)

show block: on config for the model (as example, Team) configure the type of field (as example, json) and the label to be displayed. Fields that are supposed to be Array or Hash could be configured as json be easier to read.

show do
  configure :get_media_verification_statuses, :json do
    label 'Media verification statuses'
  end
  (...)
end

edit block: on config for the model (as example, Team) configure the type of field (as example, yaml), the label to be displayed and the help with a description. Fields that are supposed to be Array or Hash could be configured as yaml, that is more flexible than JSON format.

For yaml fields the content should be displayed on a textarea and have an example for the field. Just include render_settings('text') and add an example in a partial file, like app/views/rails_admin/main/_media_verification_statuses.html.erb

edit do
  field :media_verification_statuses, :yaml do
    label 'Media verification statuses'
    render_settings('text')
    help "A list of custom verification statuses for reports that match your team's journalistic guidelines."
  end
end

For string fields the content should be displayed on a text_field and have an example for the field. Just include render_settings('field') and add an example in a partial file, like app/views/rails_admin/main/_suggested_tags.html.erb. Also, the formatted_value should be included like formatted value { bindings[:object].get_suggested_tags }.

edit do
  field :suggested_tags do
    label 'Suggested tags'
    formatted_value { bindings[:object].get_suggested_tags }
    help "A list of common tags to be used with reports and sources in your team."
    render_settings('field')
  end
end

Checkdesk migration

Checkdesk side

• Run drush vset "check_story_ids" --format=json [id1,id2,id3] to migrate specific stories
• Run drush eval "_checkdesk_core_export_data_csv();" : This command will output a directory inside [files directory]/checkdesk_migration/[[instance-name]].
• [instance-name] directory contain the following CSV files
  • 00_teams.csv : id, team_name, slug, logo
  • 01_users.csv : name, email, password, profile_image, skip_confirmation_mail, uuid, provider, created_at, login
  • 02_team_users.csv : team_id, user_id, role, status, created_at, updated_at
  • 03_projects.csv : id, title, team_id, user_id, description, lead_image, created_at, updated_at
  • 04_project_medias.csv : id, project_id, url, user_id, created_at, updated_at
  • 05_tags.csv : annotator_id, annotator_type, annotated_id, annotated_type, tag, created_at, updated_at
  • 06_comments.csv : annotator_id, annotator_type, annotated_id, annotated_type, comment, created_at, updated_at
  • 07_statuses.csv : annotator_id, annotator_type, annotated_id, annotated_type, status, created_at, updated_at
  • 08_flags.csv : annotator_id, annotator_type, annotated_id, annotated_type, flag, created_at, updated_at
• Copy the output from the above step [instance-name] into Check db/data.

Check side

• Run rake db:migrate:checkdesk.
• Rake command will generate mapping_ids.yml to log Checkdesk => Check mapping and mark migrated model.

Credits

Meedan (hello@meedan.com)