Wednesday, November 30, 2016

How To Backup PostgreSQL Databases

Backups are an essential component in any kind of data storage plan. Fortunately, PostgreSQL gives you the utilities necessary to effectively backup your important information.

PostgreSQL includes a utility called "pg_dump" that can be used to dump a database into a file for backup purposes.

pg_dump 'database' > 'file_backup'

You can include user, host and port in the previous command if you need it:

pg_dump -U 'user_name' -h 'host' -p 'port' 'database' > 'file_backup'

To restore the backup created by pg_dump, you can redirect the file into psql standard input:

psql 'empty_database' < 'file_backup'

Monday, September 05, 2016

Rails: Custom Authentication Strategies With Devise and Warden

You're probably familiar with Devise. Devise is a flexible authentication solution for Rails based on Warden

Warden uses strategies to try to authenticate sessions. However, it provides no strategy, you have to implement them yourself. Devise implements a few that you may already know.

  • Devise Model Module
Models are more commonly known as Database Authenticatable, Rememberable, Trackable, Confirmable, etc.
When you install and bootstrap Devise in your project you should see something like this in the model User:

class User < ActiveRecord::Base
    devise :database_authenticatable, :registerable, :confirmable,
         :recoverable, :rememberable, :trackable, :validatable

Since we want to write our own Authentication Implementation, we must follow suit and write a Model.

require 'devise/strategies/custom_authenticatable'

module Devise
  def self.bcrypt(klass, password)
    ActiveSupport::Deprecation.warn "Devise.bcrypt is deprecated; use Devise::Encryptor.digest instead"
    Devise::Encryptor.digest(klass, password)

  module Models
    # Authenticatable Module, responsible for hashing the password and
    # validating the authenticity of a user while signing in.
    # == Options
    # DatabaseAuthenticatable adds the following options to devise_for:
    #   * +pepper+: a random string used to provide a more secure hash. Use
    #     `rake secret` to generate new keys.
    #   * +stretches+: the cost given to bcrypt.
    # == Examples
    #    User.find(1).valid_password?('password123')         # returns true/false
    module CustomAuthenticatable
      extend ActiveSupport::Concern

      included do
        after_update :send_password_change_notification, if: :send_password_change_notification?

        attr_reader :password, :current_password
        attr_accessor :password_confirmation

      def self.required_fields(klass)
        [:encrypted_password] + klass.authentication_keys

      # Generates a hashed password based on the given value.
      # For legacy reasons, we use `encrypted_password` to store
      # the hashed password.
      def password=(new_password)
        @password = new_password
        self.encrypted_password = password_digest(@password) if @password.present?

      # Verifies whether a password (ie from sign in) is the user password.
      def valid_password?(password), encrypted_password, password)

      # Set password and password confirmation to nil
      def clean_up_passwords
        self.password = self.password_confirmation = nil

      # Update record attributes when :current_password matches, otherwise
      # returns error on :current_password.
      # This method also rejects the password field if it is blank (allowing
      # users to change relevant information like the e-mail without changing
      # their password). In case the password field is rejected, the confirmation
      # is also rejected as long as it is also blank.
      def update_with_password(params, *options)
        current_password = params.delete(:current_password)

        if params[:password].blank?
          params.delete(:password_confirmation) if params[:password_confirmation].blank?

        result = if valid_password?(current_password)
                   update_attributes(params, *options)
                   self.assign_attributes(params, *options)
                   self.errors.add(:current_password, current_password.blank? ? :blank : :invalid)


      # Updates record attributes without asking for the current password.
      # Never allows a change to the current password. If you are using this
      # method, you should probably override this method to protect other
      # attributes you would not like to be updated without a password.
      # Example:
      #   def update_without_password(params, *options)
      #     params.delete(:email)
      #     super(params)
      #   end
      def update_without_password(params, *options)

        result = update_attributes(params, *options)

      # Destroy record when :current_password matches, otherwise returns
      # error on :current_password. It also automatically rejects
      # :current_password if it is blank.
      def destroy_with_password(current_password)
        result = if valid_password?(current_password)
                   self.errors.add(:current_password, current_password.blank? ? :blank : :invalid)


      # A callback initiated after successfully authenticating. This can be
      # used to insert your own logic that is only run after the user successfully
      # authenticates.
      # Example:
      #   def after_database_authentication
      #     self.update_attribute(:invite_code, nil)
      #   end
      def after_database_authentication

      # A reliable way to expose the salt regardless of the implementation.
      def authenticatable_salt
        encrypted_password[0,29] if encrypted_password

      def send_password_change_notification


      # Hashes the password using bcrypt. Custom hash functions should override
      # this method to apply their own algorithm.
      # See for examples
      # of other hashing engines.
      def password_digest(password)
        Devise::Encryptor.digest(self.class, password)

      def send_password_change_notification?
        self.class.send_password_change_notification && encrypted_password_changed?

      module ClassMethods
        Devise::Models.config(self, :pepper, :stretches, :send_password_change_notification)

        # We assume this method already gets the sanitized values from the
        # DatabaseAuthenticatable strategy. If you are using this method on
        # your own, be sure to sanitize the conditions hash to only include
        # the proper fields.
        def find_for_database_authentication(conditions)

Next we need to register our Model with Devise. We use Devise#add_module to do this. We can do this inside the file devise.rb that you can see in the folder config/initializers. Just add it to the top of the file.

Devise.add_module(:custom_authenticatable, {
   strategy: true,
   controller: :sessions,
   model: 'devise/models/custom_authenticatable',
   route: :session,

Without all of those options, Devise and Rails’ Router won’t know to delegate Requests to SessionController into your Module.

  • Warden Strategy
A strategy is a place where you can put logic related to authentication. Any strategy inherits from Warden::Strategies::Base.

The Warden::Strategies.add method is a simple way to provide custom strategies. You must declare an authenticate! method. You may provide a valid? method. The valid method should return true or false depending on if the strategy is a valid one for the request.

require 'devise/strategies/authenticatable'

module Devise
  module Strategies
    class CustomAuthenticatable < Authenticatable

      def authenticate!
        resource  = password.present? &&
        hashed = false
        if validate(resource){ hashed = true; resource.valid_password?(password) && resource.role.is_active? }
        end = password if !hashed && Devise.paranoid
        fail(:not_found_in_database) unless resource


Warden::Strategies.add(:custom_authenticatable, Devise::Strategies::CustomAuthenticatable)

In this scenario, I need to verify that user is active or not. So I created my own Strategy and I added the necessary logic for this project.  Inactive users can not do log in.

One important thing. In order to use my new authentication strategy, you must modify your User model and add something like this:

class User < ActiveRecord::Base
    devise :custom_authenticatable, :registerable, :confirmable,
         :recoverable, :rememberable, :trackable, :validatable

Thursday, August 25, 2016

Cache Strategies for Rails: Low level caching

Sometimes we take a look at our log entries and we can see that some queries are taking a lot of time, maybe because the query is too heavy, or maybe because we are calling several queries in order to paint all the necessary information.

Low-level caching entails using the Rails.cache object directly to cache any information. For instance, I am envisioning the thypical scenario where we need to show a dropdown with all cities of a country. Maybe this query it is not heavy, but several queries at the same time like this, may affect to the performance. So in this case could be interesting save this query in the cache:

The  application has a Country model with an instance method returning all cities by country. Obviously we hace another model called City. The data returned by this method would be perfect for low-level caching. It will read a value from the cache if it available; otherwise it will execute a block passed to it and return the result:

# country.rb

def all_cities_by_country
  Rails.cache.fetch("cities_by_country_#{}") do
    City.all.where("cities.country_id = ?",

Now you could be thinking what happen if the admin creates a new city in the database? No problem, we have two possibilities, we could add to the key the column updated_at, so in this way each time admin updates the table the cache will be updated the next time user calls this method.
The other option could be create a callback in the model, so any time the table is updated we can call a private method that will update the cache.

Friday, February 26, 2016

Hive: Auto Increment column

In this post I will explain what I did in order to generate a unique and incremental ID in Hive.

Hive lets you  create custom UDFs to solve problems of this type.  As Hive is written in Java, UDFs will need to be written in JAVA.

Researching about Hive, I realized that exists a jar called hive-contrib-1.1.0.jar that lets you create row_sequences. For this you need to add the specific jar in Hive.

In my case, I copied this jar into HDFS and then I required it in Hive in this way:

add jar hdfs:///user/jars/hive-contrib-1.1.0.jar;

And now you can use the UDF to define row_sequence() function to process for auto increase ID:

CREATE TEMPORARY FUNCTION row_sequence as 'org.apache.hadoop.hive.contrib.udf.UDFRowSequence';

And you could use the query for getting the unique identifier:

ID int,
name  varchar(60)
) row format delimited fields terminated by '\073' stored as textfile;

INSERT OVERWRITE TABLE users SELECT row_sequence(), name FROM users;

If this post was useful for you, maybe you could be interested in this other topic: Analyse Tweets using Flume, Hadoop and Hive


Wednesday, February 17, 2016

Rails: How to use the comma as decimal separator

As you know, Rails use a dot as the decimal separator when displaying and processing text fields for float values.  In this post (at this moment I am using Rails 4.2) I will explain what I did in order to accept "," or "." as decimal separator since countries like Spain using a dot for the thousands delimiter and comma for the units.

At the beginning I was thinking that maybe I could use "alias_method_chain" for getting my purpose, modify the method type_cast that ActiveRecord uses for casting the values. After researching a little bit I found an interesting link about this and finally I decided to use the Module#prepend.
Here is the final code that I am using in my projects:
I created a file inside the folder config/initializers with the next code:

require 'active_record'

module CommaAsDecimalSeparator 

    def type_cast(value)
        separator = ","
        if type == :decimal && value.is_a?(String)
            value = value.gsub(separator, '.').to_d


ActiveRecord::Type::Value.send(:prepend, CommaAsDecimalSeparator)

Now that your model can deal with commas, you would like your text fields to show decimal values with commas. My solution for this is to define an additional form helper decimal_field which can be used in place of text_field to show an input box for decimal values. Copy the following as decimal_field.rb into the config/initializers folder of your Rails application:

ActionView::Helpers::FormBuilder.class_eval do
     def decimal_field(field, options = {})
        value = object.send(field).to_s
        separator = I18n::t('number.format.separator')
        precision = I18n::t('number.format.precision')
        precision = options[:precision].present? ? options[:precision] : precision
        value = ActiveSupport::NumberHelper.number_to_rounded(value, {separator: separator,     delimiter: "", precision:precision}) unless value.blank?
        options[:value] = value
        options[:class] = "#{options[:class]} number_field"
        text_field(field, options)

If you need to show the numbers in a view without a form you can use the helper number_with_delimiter(value, locale: I18n.locale). 
If you take a look to this code, I am taking into account the number.format from the locales.  In this way, you should see the decimal separator in your forms and views depending on the locale.

You can see the code in my Github repository.