gem – redpanthers.co

Readability of text using odyssey

rajana — Wed, 28 Jun 2017 12:07:19 +0000

Flesch-Kincaid readability test

Flesch Kincaid Grade Level

Gunning Fog Score

SMOG

Coleman Liau Index

Automated Readability Index (ARI)

Recently in a project that we worked on we were asked to find the readability of various pages of a website. We decided to start with Flesch-Kincaid test, as we found this to be a popular one in our research. Flesch-Kincaid readability test is designed to indicate how difficult a passage in English is to understand. In this test higher score indicates how easier to read and a lower score indicates how difficult it is to read.The formula to find Flesch-Kincaid reading-ease score is 206.835 – 1.015 * (total words / total sentences) – 84.6 * (total syllables / total words) The scores can be interrupted as

Score	School Level	Notes
100.00-90.00	5th grade	Very easy to read.
90.00-80.00	6th grade	Easy to read.
80.00-70.00	7th grade	Fairly easy to read.
70.00-60.00	8th & 9th grade	Plain English.
60.00-50.00	10th to 12th grade	Fairly difficult to read.
50.00-30.00	College	Difficult to read.
30.00-0.00	College Graduate	Very difficult to read.

Since we were not experts we wanted the ability to tweak and play around with it. We found an already build gem called Odyssey which had all these various tests and also provided the ability to extend this feature as well. So here in this article, we will discuss how to use Odyssey gem to find readability of an article and a web page.

Install Odyssey

Add in your Gemfile.

gem 'odyssey'

Usage

require 'odyssey'
Odyssey.formula_name(text, all_stats)

So if we want to use the Flesch-Kincaid test, we write the code as below.

require 'odyssey'
Odyssey.flesch_kincaid_re(text, all_stats)

To find the readability of a website we use Nokogiri and Odyssey together. Nokogiri to fetch the contents of the page and Odyssey to get the readability. Example of finding readability of our own website (https://redpanthers.co)/

url = "https://redpanthers.co/"
doc = Nokogiri::HTML(open(url))
# Get all the contents
paragraph = doc.css('p', 'article', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'a').map(&:text)
score = Odyssey.flesch_kincaid_re(para.join('. '), true)
p score

If all_stats is set to false, it returns score only. If it is true returns a hash like below

{
 "string_length"=>3024,
 "letter_count"=>2270,
 "syllable_count"=>808,
 "word_count"=>505,
 "sentence_count"=>75,
 "average_words_per_sentence"=>6.733333333333333,
 "average_syllables_per_word"=>1.6,
 "name"=>"Flesch-Kincaid Reading Ease",
 "formula"=>#,
 "score"=>64.6
}

We can perform multiple text analyses on the same text as shown below

url = "https://redpanthers.co/"
doc = Nokogiri::HTML(open(url))
para = doc.css('p', 'article', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'a').map(&:text)
score = Odyssey.analyze_multi(para.join('. ').gsub('\n', ' '),
          ['FleschKincaidRe', 'FleschKincaidGl', 'GunningFog', 'Smog','Ari','ColemanLiau'],
          true)

if all_stats is set to true it will return a hash like this

{
"string_length"=>19892,
 "letter_count"=>14932,
 "syllable_count"=>5079,
 "word_count"=>3325,
 "sentence_count"=>435,
 "average_words_per_sentence"=>7.64367816091954,
 "average_syllables_per_word"=>1.5275187969924813,
 "scores"=>
  {
   "FleschKincaidRe"=>69.8,
   "FleschKincaidGl"=>5.4,
   "GunningFog"=>3.1,
   "Smog"=>8.7,
   "Ari"=>3.5,
   "ColemanLiau"=>10.6
  }
}

Extending odyssey

To extending odyssey, you can create a class that inherit from formula

class NewFormula < Formula
  def score(passage, stats)
    p passage
    p stats
  end
  def sentence
    "Red Panthers is a Ruby on Rails development studio,
     based in the beautiful city of Cochin."
  end
end

To call your formula you just use

obj = NewFormula.new
Odyssey.new_formula(obj.sentence, false)

Resultant passage will be a Hash like this

{
 "raw"=>"Red Panthers is a Ruby on Rails development studio,
        based in the beautiful city of Cochin.",
 "words"=>["Red", "Panthers", "is", "a", "Ruby", "on", "Rails",
           "development", "studio", "based", "in", "the",
           "beautiful", "city", "of", "Cochin"],
 "sentences"=>["Red Panthers is a Ruby on Rails development studio,
               based in the beautiful city of Cochin."],
 "syllables"=>[1, 2, 1, 1, 2, 1, 1, 4, 2, 1, 1, 1, 4, 2, 1, 2]
}

and resultant status will be a Hash like this

{
 "string_length"=>90,
 "letter_count"=>73,
 "word_count"=>16,
 "syllable_count"=>27,
 "sentence_count"=>1,
 "average_words_per_sentence"=>16.0,
 "average_syllables_per_word"=>1.6875
}

Because we have access to formula’s class that is ‘status’ flag set to true then we have access to other methods or class formula. Thanks to Odyssey we were able to implement the feature quite easily and right now the algorithm we are using have evolved to new forms. But that’s another article. But if you want to build a simple readability checker then it’s quite easy and simple in Rails.

References

]]>

Taking screenshots of webpages using Ruby

nimmy — Mon, 01 May 2017 10:48:00 +0000

PhantomJS

Screencap gem

Webshot gem

Screencap and webshot are gems in Ruby that depends on a tool called PhantomJS. It is basically a web browser without a user interface, primarily used for testing websites. Such type of browsers are generally referred to as headless browsers,

Screenshots with PhantomJS

PhantomJS is quite easy to install and is multiplatform, available on major operating systems.

gem install phantomjs

To start, our script must require a reference to the webpage module then use it to create an instance:

var instance = require('webpage').create();

Use the instance.open() method and pass it the arguments, the url of the webpage that we want to capture screenshot.

instance.open('http://redpanthers.co/', function() {
instance.render('screenshot-phantom.png');
phantom.exit();
});

instance.render() method captures the screenshot and saves it to the file specified in the argument. Run the script as,

phantomjs filename.js

Screenshot is saved in the directory where we run the script. Now what we have above is all JavaScript, but to use the phantom JS in our rails application we have gems that provide us with an easy interface to acheive the same.

Screencap gem

The gem ‘screencap’ is introduced in Ruby to make screenshot capture easy. However, the gem depends on ‘PhantomJS’, so we need to have PhantomJS installed on machine to capture screenshots with screencap gem. But screencap does not work with PhantomJS 2.1.1, which is the latest version. So we need to install version 1.9.8

gem install phantomjs -v 1.9.8

check version

phantomjs --version

To install screencap gem in your application, add this line to gemfile

gem 'screencap'

Or install it as

gem install screencap

Usage

To capture screenshot with screencap

require 'screencap'
screenshot = Screencap::Fetcher.new()
image = screenshot.fetch

Fetcher class accepts url of the website to take screenshot.
Screenshot is taken when fetch method is called.
Also, fetch method supports various optional parameters to specify the area to capture the screenshot.
We can specify where to store the screenshot captured, as shown below.

image = obj.fetch(
 output: '~/folder/file-name.png',
 # dimensions for taking screenshot of specific area
  width: 1024,
 height: 768
 )

Example

require 'screencap'
screenshot = Screencap::Fetcher.new('http://redpanthers.co/')
image = screenshot.fetch(output: '~/screencap/screencap.png',
                                   width: 300,
                                  height: 500)

Our screenshot is captured in screencap.png

Webshot gem

Webshot is another gem introduced in ruby to capture screenshots, also depends on PhantomJS. For better experience, use latest version of PhantomJS. Webshot works with latest version of PhantomJS while screencap gem does not. Install PhantomJS latest version. To install webshot in your application,

gem 'webshot'

Or install as

gem install webshot

Usage

require 'webshot'
screenshot = Webshot::Screenshot.instance
screenshot.capture "https://github.com/", "image.png", width: 1024, height: 800

Provide url of the web page and file name where screenshot to be saved. Height and width can be given as optional parameters for taking screenshot with specified dimensions. Above code captures github’s home page in file image.png In all the above methods, we capture screenshots of a particular web page by passing its url. Also we can infer that screencap gem, even though efficiently takes screenshots, does not work for latest version of PhantomJS. This was an issue encountered while working with gem screencap as installing PhantomJS without specifying the version would install the latest version by default. So, choose gem webshot if you want to go ahead with the latest version of PhantomJS.

References

]]>

Chartkick: data visualization made easy with Ruby

govind — Wed, 18 Jan 2017 13:11:11 +0000

gem “chartkick” Then you have to choose your charting library. In case, if more than one charting library is loaded, choose between them with the following code:

<%= line_chart data, adapter: "google" %>

P.S:- highcharts is the default adapter is nothing else is defined. Using Chartkick Once it’s installed, you can start playing around with chartkick. For example:

<%= line_chart User.group_by_day(:created_at).count %>

In the above example, I have just shown you how we can make a line chart.Similarly, you can create Pie chart, Column chart, Bar chart, Area chart, Scatter chart, Geo charts, Timeline(Google Charts)etc. All you need is that one line of Ruby code. Cool, isn’t it!!

Chartkick gives you a variety of options to make your charts interesting. You can also set a Global option for each chart by using initializers and even customize the HTML. Data can be passed as a Hash or as an Array. You can find more details about using Chartkick options, sending data in Chartkick documentation. Now, let’s have some real fun by using Chartkick to create some interactive Graphs, using Dynamic data. To create a graph which drives data from the ajax request, you just need to define a method in the controller and pass the JSON data. For example app/controllers/views_controller.rb:

class ViewsController < ApplicationController
  def show
    @view = View.all
  end
  def views_by_day
    render json: View.group_by_day(:viewed_at, format: "%B %d, %Y").count
  end
end

Next, add in the code listed below to your views/show view. app/views/views/show.html.erb:


  Views By Day
  <%= line_chart views_by_day_view_path %>

We can also create graphs with Multiple Series with the below codes

<%= line_chart @.map { |view|
 {name: view.name, data: view.feats.group_by_week(:created_at).count}
} %>

<%= line_chart Feat.view(:view_id).group_by_week(:created_at).count %>

Voila! there you have it how you want. Here is an example charts I got for you So, next time if you want to create charts and graphs easily in your web application, don’t forget to use Chartkick and let me know in the comment section if you know any such tool that made your coding a bliss! Happy Coding!

References

]]>