• Parallel execution of sequential scans, joins and aggregates.
• Avoid scanning pages unnecessarily during vacuum freeze operations.
• Synchronous replication now allows multiple standby servers for increased reliability.
• Full-text search can now search for phrases (multiple adjacent words).
• postgres_fdw now supports remote joins, sorts, UPDATEs, and DELETEs.
• Substantial performance improvements, especially in the area of scalability on multi-CPU-socket servers.

Parallel execution of sequential scans, joins and aggregates

Avoid scanning pages unnecessarily during vacuum freeze operations

Synchronous replication now allows multiple standby servers for increased reliability

Full-text search can now search for phrases (multiple adjacent words)

postgres_fdw now supports remote joins, sorts, UPDATEs, and DELETEs

1. Install the postgres_fdw extension using CREATE EXTENSION.
2. Create a foreign server object, using CREATE SERVER, to represent each remote database you want to connect to. Specify connection information, except user and password, as options of the server object.
3. Create a user mapping, using CREATE USER MAPPING, for each database user you want to allow to access each foreign server. Specify the remote username and password to use as user and password options of the user mapping.
4. Create a foreign table, using CREATE FOREIGN TABLE, for each remote table you want to access. The columns of the foreign table must match the referenced remote table. You can, however, use table and/or column names different from the remote table’s, if you specify the correct remote names as options of the foreign table object.

 psql

psql mydb

Backups

References

• https://www.postgresql.org/docs/9.6/static/release-9-6.html
• http://www.craigkerstiens.com/2015/12/29/my-postgres-top-10-for-2016/
• https://wiki.postgresql.org/wiki/NewIn96
• https://momjian.us/main/writings/pgsql/features.pdf

• Has date attributes only (year, month, day)
• Based on integer whole-day intervals from an arbitrary “day zero”
• Can handle date arithmetic in units of whole days
• Date object is created with ::new, ::jd, ::ordinal, ::commercial, ::parse, ::strptime, ::today, Time#to_date etc.
• Takes 4 bytes to store.

$ require 'date'
$ Date.new(2001,2,3)
  Date: 2001-02-03
$ Date.jd(2451944)
  Date: 2001-02-03 ...
$ Date.ordinal(2001,34)
  Date: 2001-02-03 ...
$ Date.commercial(2001,5,6)
  Date: 2001-02-03 ...
$ Date.parse('2001-02-03')
  Date: 2001-02-03 ...
$ Date.strptime('03-02-2001', '%d-%m-%Y')
  Date: 2001-02-03 ...
$ Time.new(2001,2,3).to_date
  Date: 2001-02-03 ...
$ Date.today
  "Mon, 02 Jan 2017"

Time

• Has date and time attributes (year, month, day, hour, min, sec, subsec)
• Can handle negative times before unix time
• Can handle time arithmetic in units of seconds

$ require 'time'
$ Time.now
   2015-12-08 10:26:40 -0200
$ time = Time.new
  Components of a Time
$ time.year    # => Year of the date
$ time.month   # => Month of the date (1 to 12)
$ time.day     # => Day of the date (1 to 31 )
$ time.wday    # => 0: Day of week: 0 is Sunday
$ time.yday    # => 365: Day of year
$ time.hour    # => 23: 24-hour clock
$ time.min     # => 59
$ time.sec     # => 59
$ time.usec    # => 999999: microseconds
$ time.zone    # => "UTC": timezone name

DateTime

• Has date and time attributes (year, month, day, hour, min, sec)
• It  is formatted as YYYY-MM-DD HH:MM:SS
• Based on fractions of whole-day intervals from an arbitrary “day zero” (-4712-01-01)
• Can handle date arithmetic in units of whole days or fractions
• Takes 8 bytes to store, and has a precision of .001 seconds.
  • A four-byte integer packed as YYYY×10000 + MM×100 + DD
  • A four-byte integer packed as HH×10000 + MM×100 + SS
• Valid ranges go from the year 1000 to the year 9999
• It is created with ::new, ::jd, ::ordinal, ::commercial, ::parse, ::strptime, ::now, Time#to_datetime etc.

$ require 'date'
$ DateTime.new(2001,2,3,4,5,6)
  DateTime: 2001-02-03T04:05:06+00:00 ...

Let’s see the Differences among all of them which makes them unique.

• Date use rational and a “day zero” for storage. But Time doesn’t. So Time is faster.
• Date field is populated with a literal date and does not concern itself with time zones so this can cause trouble if it is not expressed in the user’s local time. A DateTime can always be converted to a user’s local time if required.
• Time used to track changes to records and update every time when the record is changed. DateTime used to store a specific and static value which is not affected by any changes in records.
• Time internally converted current time zone to UTC for storage, and during retrieval converted back to the current time zone. DateTime can not do this.
• Time affected by different TIME ZONE related setting. Datetime is constant.
• Ruby’s Time class implements a proleptic Gregorian calendar and has no concept of calendar reform. This problem can be overcome using DateTime class.

References

• https://ruby-doc.org/stdlib-2.4.0/libdoc/date/rdoc/Date.html
• http://stackoverflow.com/questions/3928275/in-ruby-on-rails-whats-the-difference-between-datetime-timestamp-time-and-da
• https://www.tutorialspoint.com/ruby/ruby_date_time.html
• http://stevenyue.com/blogs/date-time-datetime-in-ruby-and-rails/

What is a database view?

Making liberal use of views is a key aspect of good SQL database design. Views allow you to encapsulate the details of the structure of your tables, which might change as your application evolves, behind consistent interfaces.  

CREATE VIEW company_manager AS
SELECT id, name, email
FROM  companies
WHERE role='manager';

SELECT * FROM company_managers;

So what is a Materialized view?

1. It’s read-only (pseudo-table) so you can’t update it.
2. You need to refresh the table to get the latest data.
3. While refreshing, it would block other connections to access the existing data from the material view, so you need to make the refresh run concurrently

So why use Materialized views in Rails?

• Capture commonly used joins & filters.
• Push data intensive processing from Ruby to Database.
• Allow fast and live filtering of complex associations or calculation fields.

How do you use it in Rails?

bundle exec rails g migration create_all_time_sales_mat_view

class CreateAllTimesSalesMatView < ActiveRecord::Migration
  def up
    execute <<-SQL
      CREATE MATERIALIZED VIEW all_time_sales_mat_view AS
        SELECT sum(amount) as total_sale,
        DATE_TRUNC('day', invoice_adte) as date_of_sale
      FROM sales
      GROUP BY DATE_TRUNC('day', invoice_adte)
    SQL
  end
  def down
    execute("DROP MATERIALIZED VIEW IF EXISTS all_time_sales_view")
  end
end

class AllTimeSalesMatView < ActiveRecord::Base
  self.table_name = 'all_time_sales_mat_view'
  def readonly?
    true
  end
  def self.refresh
    ActiveRecord::Base.connection.execute('REFRESH MATERIALIZED VIEW CONCURRENTLY all_time_sales_mat_view')
  end
end

AllTimeSalesMatView.select(:date_of_sale)
AllTimeSalesMatView.sum(:total_sale)

Regular
       user     system      total        real
     (976.4ms)  0.020000   0.000000   0.020000 (  0.990258)
MatiView
     (2.3ms)    0.000000   0.010000   0.010000 (  0.012010)

Summarize

Good Points

• Faster to fetch data.
• Capture commonly used joins & filters.
• Push data intensive processing from Ruby to Database.
• Allow fast and live filtering of complex associations or calculation .fields.

Pain Points

• To alter table we need to write SQL
• We will be using more RAM and Storage
• Requires Postgres 9.3 for MatView
• Requires Postgres 9.4 to refresh concurrently
• Can’t have Live data
  • You can fix this by creating your own MatViewTable and updating it with the latest information

References

• https://www.postgresql.org/docs/9.3/static/rules-materializedviews.html
• http://en.wikipedia.org/wiki/Materialized_view
• http://dev.mysql.com/doc/refman/5.7/en/create-view.html
• https://blog.pivotal.io/labs/labs/database-views-performance-rails
• https://www.sitepoint.com/speed-up-with-materialized-views-on-postgresql-and-rails/

• Storing data in JSON and JSONB format
• Generating JSON results from the query itself

Getting Started

How to generate JSON

select row_to_json(users) from users where id = 1;

{"id":1,"email":"hsps@redpanthers.co","encrypted_password":"iwillbecrazytodisplaythat",
"reset_password_token":null,"reset_password_sent_at":null,
"remember_created_at":"2016-11-06T08:39:47.983222",
"sign_in_count":11,"current_sign_in_at":"2016-11-18T11:47:01.946542",
"last_sign_in_at":"2016-11-16T20:46:31.110257",
"current_sign_in_ip":"::1","last_sign_in_ip":"::1",
"created_at":"2016-11-06T08:38:46.193417",
"updated_at":"2016-11-18T11:47:01.956152",
"first_name":"Super","last_name":"Admin","role":3}

select row_to_json(results)
from (
  select id, email from users
) as results

{"id":1,"email":"hsps@redpanthers.co"}

select row_to_json(result)
from (
  select id, email,
    (
      select array_to_json(array_agg(row_to_json(user_projects)))
      from (
        select id, name
        from projects
        where user_id=users.id
        order by created_at asc
      ) user_projects
    ) as projects
  from users
  where id = 1
) result

{"id":1,"email":"hsps@redpanthers.co", "project":["id": 3, "name": "CSnipp"]}

array_to_json(array_agg(row_to_json(user_projects)))

json_agg(user_projects)

json_build_object('foo',1,'bar',2)

{"foo": 1, "bar": 2}

References

CREATE INDEX index_name
ON prices (user_id);

add_index :prices, :user_id

CREATE INDEX index_name
ON user_views (user_id, article_id);

add_index :user_views, [:user_id, :article_id]

scope :articles, where(:spam => 'true')

CREATE INDEX index_name
on articles (spam is true);

add_index :articles, :spam, name: "index_articles_on_spam", where: "(spam IS true)"

DROP INDEX index_name;

remove_index :books, :created_at

• Index Primary key
• Index Foreign key
• Index all columns you would be passing into where clause
• Index the keys used to Join tables
• Index the date column (if you are going to call it frequent, like rankings of a particular date)
• Index the type column in an STI or polymorphism.
• Add partial index to scopes
• Do not index tables with a lot of read, write
• Do not index tables you know that will remain small, all through out its life time
• Do not index columns where you will be manipulating lot of its values.