Friday, February 26, 2016

Hive: Auto Increment column

In this post I will explain what I did in order to generate a unique and incremental ID in Hive.

Hive lets you  create custom UDFs to solve problems of this type.  As Hive is written in Java, UDFs will need to be written in JAVA.

Researching about Hive, I realized that exists a jar called hive-contrib-1.1.0.jar that lets you create row_sequences. For this you need to add the specific jar in Hive.

In my case, I copied this jar into HDFS and then I required it in Hive in this way:

add jar hdfs:///user/jars/hive-contrib-1.1.0.jar;

And now you can use the UDF to define row_sequence() function to process for auto increase ID:

CREATE TEMPORARY FUNCTION row_sequence as 'org.apache.hadoop.hive.contrib.udf.UDFRowSequence';

And you could use the query for getting the unique identifier:

ID int,
name  varchar(60)
) row format delimited fields terminated by '\073' stored as textfile;

INSERT OVERWRITE TABLE users SELECT row_sequence(), name FROM users;

If this post was useful for you, maybe you could be interested in this other topic: Analyse Tweets using Flume, Hadoop and Hive


Wednesday, February 17, 2016

Rails: How to use the comma as decimal separator

As you know, Rails use a dot as the decimal separator when displaying and processing text fields for float values.  In this post (at this moment I am using Rails 4.2) I will explain what I did in order to accept "," or "." as decimal separator since countries like Spain using a dot for the thousands delimiter and comma for the units.

At the beginning I was thinking that maybe I could use "alias_method_chain" for getting my purpose, modify the method type_cast that ActiveRecord uses for casting the values. After researching a little bit I found an interesting link about this and finally I decided to use the Module#prepend.
Here is the final code that I am using in my projects:
I created a file inside the folder config/initializers with the next code:

require 'active_record'

module CommaAsDecimalSeparator 

    def type_cast(value)
        separator = ","
        if type == :decimal && value.is_a?(String)
            value = value.gsub(separator, '.').to_d


ActiveRecord::Type::Value.send(:prepend, CommaAsDecimalSeparator)

Now that your model can deal with commas, you would like your text fields to show decimal values with commas. My solution for this is to define an additional form helper decimal_field which can be used in place of text_field to show an input box for decimal values. Copy the following as decimal_field.rb into the config/initializers folder of your Rails application:

ActionView::Helpers::FormBuilder.class_eval do
     def decimal_field(field, options = {})
        value = object.send(field).to_s
        separator = I18n::t('number.format.separator')
        precision = I18n::t('number.format.precision')
        precision = options[:precision].present? ? options[:precision] : precision
        value = ActiveSupport::NumberHelper.number_to_rounded(value, {separator: separator,     delimiter: "", precision:precision}) unless value.blank?
        options[:value] = value
        options[:class] = "#{options[:class]} number_field"
        text_field(field, options)

If you need to show the numbers in a view without a form you can use the helper number_with_delimiter(value, locale: I18n.locale). 
If you take a look to this code, I am taking into account the number.format from the locales.  In this way, you should see the decimal separator in your forms and views depending on the locale.

You can see the code in my Github repository.