Lazy enumerator to handle huge files

coderhs — Fri, 12 Aug 2016 08:49:12 +0000

Lazy evaluation, or call-by-need is an evaluation strategy which delays the evaluation of an expression until its value is needed. It’s frequently seen in functional languages, ruby introduced the lazy method in Ruby 2.0. For those who don’t know what are enumerators: enumerators are something that can be counted. So a collection of elements, files (file is an collection of lines of string), etc can be treated as an enumerator. In ruby we need to make something countable into an enumerator object, which is done by applying .each and .map on it.

[].each
[1, 3, 4].each
file.each
{}.each

Ruby has a wide range of operations we can do over a collection, it’s one of those features that makes Ruby such a powerful dynamic language. An enumerator can be used to generate series like the Fibonacci series.

fib = Enumerator.new do |y|
  a = b = 1
  loop do
    y << a
    a, b = b, a + b
  end
end

But when we do a .map / .each with a code block, then it would try to realize the enumerator fully and then apply the block over it. That would be fine when we are working on something small like:

[1,2,3].map { |i| i*i }

But when we take the above fib enumerator, which will grow into an infinite series, adding a .map would lead the code to an infinite loop. If you are crazy enough to write an infinite loop, feel free to run the below code.

foo.map { |i| i * * }

When we try to use these operations on a huge 10 GB file,there it makes your program realize the entire file is close to impossible. So this is the case where the lazy enumerator comes to place. It would ask ruby to defer the realization of the enumerator until it is required, and not before you run your operation. So with respect to the above Fibonacci enumerator, you should change the code to as below.

foo.lazy.map { |i| i * i }

So now, the program is ready, all good it would run fast. But it won’t execute them until you call or access those elements. So only when we do foo.lazy.map { |i| i* i }. First the first element is calculated and given to us. Like wise if you want the first 10 elements of the fibonacci series then you have to do foo.lazy.map { |i| i* i }.take(1o).to_a To do the same with the files, as a file is a collection of lines, follow the below code snippets:

fille = File.open('OurHugeFile.log')

get the first three lines, lazily:

file.each_line.lazy.take(3).to_a

get the firs two lines with word “INFO”

 file.each_line.lazy.select { |line| line.match(/INFO/) }.first(2)

So if you want to print something with line numbers, just do a map from the lazy.

file.each_line.lazy.each_with_index.map { |l, i| "#{i}: #{l}" }.select { |i| match(/INFO/) }.first(3)

If you have anything more to add to this, kindly comment below.]]>

Working with timezones in rails

coderhs — Thu, 11 Aug 2016 07:32:15 +0000

Ruby on Rails being an amazing framework, helps us manage the timezone of our rails application. It gives us access to a lot of helpers, to make our life easier. For example, if you want to change all the date and time of your application to the logged in users time zone, we just have to place the following code in the application_controller.

# app/controllers/application_controller.rb
around_action :set_time_zone, if: :current_user
private
def set_time_zone(&block)
  Time.use_zone(current_user.time_zone)
end

We assume that you have stored the user’s time_zone in your database in the time_zone column. The application to show timezone can be set in your application.rb, if we don’t set a particular timezone then the application will just show the systems timezone.

config.time_zone = 'Central Time (US & Canada)'

If you want to know all the timezone options available in rails, run the rake -D time command in your terminal. Even though rails would take care of the timezone, when we are using certain ruby commands, it gives us our systems timezone and not the one set by rails. So to avoid surprises, we should be aware of the timezones we are exposed to. A rails app, would always be exposed to three timezones:

System timezone
Database timezone
Rails applications own timezone

All three could be different, for example your system timezone could be in IST, database in UTC, and rails app running in PDT. To avoid accidental assessment to a different timezone, always keep in mind the commands you should and should not use. DO NOT USE – as they all give you time with respect to the system time

  * Time.now
  * Date.today
  * Date.today.to_time
  * Time.parse("2015-07-04 17:05:37")
  * Time.strptime(string, "%Y-%m-%dT%H:%M:%S%z")

USE – these give you access to the rails application time (which we can change with respect to the user)

* Time.current
* 2.hours.ago
* Time.zone.today
* Date.current
* 1.day.from_now
* Time.zone.parse("2015-07-04 17:05:37")
* Time.strptime(string, "%Y-%m-%dT%H:%M:%S%z").in_time_zone

]]>

programming – redpanthers.co

Lazy enumerator to handle huge files

Working with timezones in rails