Refresh your elasticsearch index with zero downtime

I’m using a basic elasticsearch-rails setup on one of the applications I’m working on. The thing is, we sometimes need to make changes to the ES index, add new fields, or redo the existing ones. And when you have a database that needs at least 10 minutes(it was closer to 1 hour before we reworked the implementation), you can’t really afford that downtime. As ES-rails sets things up for you in a certain way, it was needed to use some of the underlying ES features to get the job done correctly.

So, it was needed to completely remove the downtime while doing the full reindex(deleting and creating the elasticsearch index). If you are already using elasticsearch-rails then you are familiar with the nomenclature, and probably with the methods described. For doing a complete reindex we require a rake task to do the job for us like this:

namespace :elasticsearch do
  task :reindex => :environment do
    index_name = Person.index_name
    Person.__elasticsearch__.create_index! force: true
    Person.all.find_in_batches(batch_size: 1000) do |group|
      group_for_bulk = group.map do |a|
        { index: { _id: a.id, data: a.as_indexed_json } }
      end
      Person.__elasticsearch__.client.bulk(
        index: index_name,
        type: "person",
        body: group_for_bulk
      )
    end
  end
end

This will effectively remove the index if it exists and reload the new data in the index, making it practically unusable until the task is done. Because this wasn’t an option, we looked into ES aliases and found them to be helpful. Basically, what we needed to do was to create an index with a unique name, and assign an alias to it, so that we could create and fill the index while the current one was still operational. So no downtime needed.

#...
index_name = "#{Person.index_name}_#{SecureRandom.hex}"
client = Person.__elasticsearch__.client
Person.__elasticsearch__.create_index! index: index_name, force: true
Person.all.find_in_batches(batch_size: 1000) do |group|
  #...
end
# to be sure there is no index named Person.index_name
client.indices.delete(index: Person.index_name) rescue nil
  # collecting old indices
  old_indices = client.indices.get_alias(name: Person.index_name).map do |key, val|
    { index: key, name: val['aliases'].keys.first }
  end
  # creating new alias
  client.indices.put_alias(index: index_name, name: Person.index_name)
  # removing old indices
  old_indices.each do |index|
    client.indices.delete_alias(index)
    client.indices.delete(index: index[:index])
end

So that is about it, you just call bin/rake elasticsearch:reindex and you have refreshed your elasticsearch index with zero downtime. Of course, you will have to implement some system to track the changed records while you were reindexing(remember we are working on a live system, so data is changing all the time). We used redis for that, to mark when reindexing has started $redis.set('elasticsearch:reindex_running', true) collected all changed record ids in a redis array, processed them via the regular indexing worker, and deleted the key after the alias was linked with $redis.del('elasticsearch:reindex_running').

What I have learned from a hackathon

I’ve attended a hackathon yesterday, that was the first “real” one in my life, and i’m still a bit under the influence. It was organised by the great people from Osijek Software City, and i thank them so much for the experience. I’ve learned so much, and almost nothing technical, because hackathons aren’t about it, as you will find out.

The topic: Crowd funding application

The topic was to make a simple crowd funding application. Nothing really complex, just a basic feature list with points for each complete feature.

The implementation: Rails

We chose Ruby on Rails as our implementation platform. That was a great choice, as we know it so well, and it would show later, that rails was the best framework for the rapid prototyping needed to win the hackathon.
The only thing lacking, where we lost a lot of points, was the front-end design, but you can’t expect a team of 3 backend developers and one software tester to create something really pretty. Although our UX was great, as we were using bootstrap, and not really in a great way.

The mistakes…

Overthinking it

We had a feature list to complete, which is fairly easy to do with rails, but we managed not to complete every functionality, just because we were loosing time hacking on the front-end to make it look acceptable.

Making it too complex

In a hackathon, time is working against you, you can say the same in product work, as unlimited time time is a thing no one has.
You have to make your features concise, and make them do only the stuff that needs to be done.

The learning…

KISS

Keep it simple, stupid. The best thing anyone can tell you, for any kind of product. Make it do only a couple of required features, and make that features well. Everything else doesn’t need to be done, at least at the moment. Don’t overthink the implementation, there are many currents in the rails development world which pull to this side or the other, but in this case, i must lean to @dhh and the pure rails way. You can optimise and improve later, after you have a product that is selling, and only the things that have to be optimised at the moment to keep the product running smooth.

Time is scarce, so make use of it

You don’t have time, as you are reading this, you have less and less of it. If you are still wasting it on nonsense, you won’t accomplish anything tangible. We had about 9 hours to make the application, and looking back, it was a couple of hours more than we needed, but we wasted time on initial configuration, setting up stuff, design. Lesson learned from this, Do the thing you are best at, and delegate everything else, if you can’t design to save your life, hire a great designer. If you can’t code, hire a great programmer. Just go outside, mingle with the community and you will surely find someone to team up with, if that isn’t the case, ask some of your acquaintances or friends, they will probably know someone who can help you build your dream.

Have a great team

The crucial thing any team needs is great communication, you don’t need ninjas, rockstars and whatnot. Anyone can build a web app, including high school kids who did a great job yesterday, i’m really happy that kids are going this path, as that will make their lives better.

Have fun

The main thing you are building a software product shouldn’t be money, or investors, but to have fun while doing it, if you aren’t having fun in your life, what is it worth to you then. I really like this job, and the people in the community are nice, if you are nice to them.

Final notes:

We had fun, we worked hard, i was swamped after spitting ruby code for 10 hours straight, and the experience was great. I met new people, had some interesting conversations, and saw other technologies people used to solve the same problem. We finished second, with a point difference of 15 points(the total possible was 1000), but it doesn’t really matter. The real reward is participating, and staying until the end, giving your best to solve the problem in hand, and supporting your team, as much as you possibly can.

Override https errors for regular http calls

Let’s say you are building a web app that has some secure user data involved
in the picture. You have to secure it with https, that is a necessity. So what
happens when you have to include some outside service into your https only
application, and the service sadly doesn’t have an https site, or maybe it has
a self signed certificate? The error pops up, something in the context of This page is only
partialy secure
.
There is a way to override this, but you are doing it at you own risk. You
really should use only verified https resources in your public facing sites.

There can be many reasons why you would go with this solution, and i’ll leave
them to you to find that out by yourself, i’m just giving you the hammer.

So, you want to override https calls in your app, and it’s a rails app(this
approach can be done using sinatra or anything else ruby based, if you don’t
know how to implement it, contact me and i’ll try to help)

Basically you want to override the outside http call with a call to your own
application, yes this can hurt your performance, but not having a green icon in
the address bar can hurt you even more. I’ll make an example with a simple
JSON call to an external service, but it can be done with any call.

First you should set up a controller to handle these requests, lets say
ExternalResourcesController make an external_json method and add it to the
routes.rb file, of course you need net/http and uri for making the
remote calls.


# app/controllers/external_resources_controller.rb
require ‘net/http’
require ‘uri’
class ExternalResourcesController < ApplicationController def external_json options = { # fill your options with the params you are sending from the application } uri = URI('some external json service url') uri.query = URI.encode_www_form(options) resp = Net::HTTP.get(uri) render json: resp end end [/ruby] [ruby] # config/routes.rb get 'external_resources/external_json', to: 'external_resources#external_json' [/ruby] This approach also gives you the benefit of not having to send and expose all parameters you are sending in client facing JavaScript, and can only send the dynamic ones. That can pay off if you are using this call more than once. And you can call this from your CoffeeScript very easily: [javascript] # app/assets/javascripts/ajax.js.coffee json = $.ajax url: "/external_resources/external_json" dataType: "jsonp" data: first_param_name: "I'm a parameter" second_param_name: "I'm a parameter too" [/javascript] Once again, you should only use this approach if you really, really know what you are doing, i’m not liable for the problems you can cause.

Rails Integration testing with MiniTest::Spec and Capybara

So you want to do integration testing for your application, good for you. If you have chosen MiniTest and it’s specing DSL MiniTest::Spec  as your test framework here is an example how to setup the integration tests to run fairly fast. Integration tests are similar to Cucubmer features, if you have used that before. I just like this style better because Cucumber, IMO adds too much noise, but it serves you as application documentation. I will also user Capybara for the webpage testing.
Because we are using minitest-spec-rails gem, we don’t need to tinker with the config/application.rb and test settings, because ActiveSupport::TestCase is a subclass of MiniTest::Spec

First we will set up the gemfile, with the needed gems( I also use guard for my automated tests, but it’s worth writing another blog post, soon hopefully)

group :development, :test do
  gem 'minitest-spec-rails'
  gem 'capybara_minitest_spec' # for capybara integration and spec matchers
  gem 'capybara-webkit' # for headless javascript tests
  gem 'turn' # for prettier test output
end

After that we need to set up out test/test_helper.rb to use Turn and capybara

require 'rubygems'
require 'rails/test_help'
require 'minitest/autorun'
require 'capybara/rails'
require 'turn'

class ActionDispatch::IntegrationTest
  include Rails.application.routes.url_helpers # for x_path and x_url helpers
  include Capybara::DSL
  include PossibleJSDriver # switching Capybara driver for javascript tests, look below
end

# switching to the javascript driver in integration tests
module PossibleJSDriver
  def require_js
    Capybara.current_driver = :webkit
  end

  def teardown
    super
    Capybara.current_driver = nil
  end
end

After you have everything set up you can use all of this in your integration tests, here is an example of one integration test, without and with javascript browser turned on(require_js turns on the webkit driver)


require ‘test_helper’
class LoginTest < ActionDispatch::IntegrationTest before { visit login_path } describe "signup text" do it "must have a signup text" do page.must_have_content("Don't have an account?") end it "must have a signup link" do page.find_link('Sign up').wont_be_nil end it "signup link must go to the signup path" do require_js # this one turns on the webkit javascript driver click_link('Sign up') current_path.must_equal(signup_path) end end end [/ruby] I have tested the integration test speed with :webkit as a default capybara driver and it runs at the same speed as the normal :rack_test driver, so you can go that way too. Resources used: https://github.com/metaskills/minitest-spec-rails
https://github.com/blowmage/minitest-rails
https://github.com/blowmage/minitest-rails-capybara
http://stackoverflow.com/questions/5655154/how-do-you-perform-javascript-tests-with-minitest-capybara-selenium/13296544#13296544
https://github.com/thoughtbot/capybara-webkit

Integrating turbolinks in your Rails application

We are all excited in a new addition to rails plugins family that is turbolinks.

If you have tried to include it in your application, you may have noticed that turbolinks doesn’t trigger your $.ready, because there is no full page reload. As a consequence your javascripts that are bound to $.ready won’t trigger. I will only hack the finished event that is “page:change” which occurs when the page is refreshed. You have other events and their descriptions on the turbolinks github page. Here is a fast hack that i came with. Some of the code is taken from turbolinks library.

First have your $.ready trigger the “page:change” event. This is done, so we can bind everything that is now bound to document.ready to “page:change”

$.ready ->
  triggerEvent "page:change"

triggerEvent = (name) ->
  event = document.createEvent 'Events'
  event.initEvent name, true, true
  document.dispatchEvent event

Now when we have $.ready calling “page:change” we can use it in our application like this

document.addEventListener "page:change", ->
  alert "I am called from $.ready and page:change"

Will Paginate in Rails using remote request

Have you ever wondered why is there no remote option for will_paginate gem? It’s too complex and data dependent to handle all possible situations. I have made a workaround that can help you implement simple ajax pagination for your rails application. I will be replacing the whole yield part here, but you can customize it whatever way you like.

Step 1. Extract your required view into a partial so you have
index.html.erb

<!-- app/views/posts/index.html.erb -->
< %= render "index" %>

_index.html.erb

<!-- app/views/posts/_index.html.erb -->
<h1>Listing posts</h1>
<table id="posts">
  <tr>
    <th>Title</th>
    <th>Body</th>
    <th></th>
    <th></th>
    <th></th>
  </tr>
 < %= render partial: 'post', collection: @posts %>
</table>
< %= will_paginate @posts, remote: true %>
<br />
< %= link_to 'New Post', new_post_path, remote: true %>

Note the remote: true part of the will_paginate call, we will bind the javascript to it in an instance.

2. Add a div surrounding your yield tag in your layout

<!-- app/views/layouts/application.html.erb -->
<div id="content>
  < %= yield %>
</div>

3. Create index.js.erb which will replace the contents of the div with paged table data.

// app/views/posts/index.js.erb
$("div#content").html('< %= escape_javascript(render "index") %>');

4. Bind the will_paginate link click to the rails remote call using coffeescript in

# app/assets/javascripts/posts.js.coffee
$('.pagination[remote=true] a').live 'click', ->
  window.history.pushState(null, 'hi', $(this).attr("href"))
  $.rails.handleRemote($(this))
  return false

We also change the address in the navigation bar with PushState, because it can happen someone will press F5 or something and reload with a different params[:page].

Update: As my coworker Oliver mentioned, i forgot to include responding to js in our controller for the index action

def index
  @posts = Post.paginate(per_page: 8, page: params[:page]
  respond_to do |format|
    format.html
    format.js
  end
end

Installing Rails Server on Ubuntu 12.04 with RVM, Nginx, Passenger and Oracle support

We have a new product ready and it is a Ruby on Rails webshop bundled with an ERP software written on Oracle Forms 6i. I will not go into the details of this work, maybe later i will make some posts of the process but as it is very linked to current software(only authentication is done with Devise, every other data and processing is done internally on Oracle database) there is no sense for me to explain it deeper. Off to the installation.

Server installation and configuration

First we need a clean install of Ubuntu server 12.04, x86 or x64 version will work all the same, but be careful later on when you download the Oracle instant client. On the installation, fill all the relevant data needed, and for installed services, choose only the ssh server, as you won’t be needing anything more. Another reminder: this will only be the webserver, assuming that you have the Oracle database installed somewhere in network reach.

After you login to your server for the first time, take your time to set up the ip address because it defaults to dhcp.
Edit the /etc/network/interfaces file and fill your address, gateway and dns servers(this is a new thing, i think starting in 12.04 /etc/resolv.conf gets updated whenever you restart networking, so everything you write there will get overwritten)


# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
    address 192.168.0.100
    netmask 255.255.255.0
    gateway 192.168.0.1
    dns-nameservers 8.8.8.8 8.8.4.4.

After the config is saved, restart the networking and do a full update/upgrade of the server to get the latest packages installed:


sudo /etc/init.d/networking restart
sudo apt-get update
sudo apt-get -y dist-upgrade

After all is done, reboot your server.

Installing RVM and latest Ruby version

After your server is updated you can proceed with installing RVM and the latest stable Ruby(as i’m writing this, the latest version is 1.9.3p194 so i will be using this version)

First we install RVM prerequisites:

# rvm requires curl and git to install, and you will probably need them on your server so go ahead and install them
apt-get -y install git-core curl

# Installing RVM as sudo, to get RVM ruby system wide
curl -L get.rvm.io | sudo bash -s stable

# sourcing rvm environment so you can use it untill you login again
source /etc/profile.d/rvm.sh

And that will install RVM and make it available for all users that belong to the rvm group(hint: edit /etc/groups and add rvm after the desired username). After installing RVM and adding our user to the  rvm group we are off to installing latest Ruby version

# install all requirements needed for YARV/MRI Ruby (you can easily get these by running 'rvm requirements' in the terminal)
sudo apt-get install build-essential openssl libreadline6 libreadline6-dev curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt-dev autoconf libc6-dev ncurses-dev automake libtool bison subversion

# install requirements needed for Oracle Instant client
sudo apt-get install libaio1

# install latest stable 1.9.3 and set it as default ruby version
rvmsudo rvm install 1.9.3 --default

# disable installation of RDoc and RI when installing gems
echo 'gem: --no-ri --no-rdoc'  >> ~/.gemrc

# install bundler gem, as you will later need for deployment
rvmsudo gem install bundler

 Installing Passenger and Nginx web server

After you have ruby and everything needed installer go and install passenger gem and nginx web server with it

# install  passenger gem
rvmsudo gem install passenger

# install passenger nginx module (and nginx if you haven't installed it yet), choose first option(Yes: download, compile and install Nginx for me) when it asks you what to do
# if there are more requirements the script will tell you and after you install them rerun the command
rvmsudo passenger-install-nginx-module

The installation will set up the nginx.conf file for you and show you how you can enable a rails site on your server. But i will include a sample config later on.
Just for convenience, symlink nginx conf folder that is in /opt/nginx/conf to /etc/nginx and download linode nginx startup/shutdown script

# symlink nginx conf folder to /etc/nginx
sudo ln -s /opt/nginx/conf /etc/nginx

# download linode nginx init script and create the service
wget -O init-deb.sh http://library.linode.com/assets/660-init-deb.sh
sudo mv init-deb.sh /etc/init.d/nginx
sudo chmod +x /etc/init.d/nginx
sudo /usr/sbin/update-rc.d -f nginx defaults

Installing Oracle instant client

To connect your Rails application to Oracle database, you will have to install Oracle client, either full or instant one. As installing full client requires alien and converting .rpm packages to .deb it’s a bit of a hassle. Also, it will consume more space on your drive, and all that is unnecessary, we are trying to keep it small here.

First you should download zip files from Oracle site depending what architecture you are running:
Oracle Instant Client 11.2 for linux:  X86X64

Download only instantclient-basic, instantclient-sqlplus and instantclient-sdk zip archives and unzip them all into the same instantclient_11_2 folder. Now as sudo move that folder so the path is /opt/oracle/instantclient_11_2

After that you only have to add two environment variables to be able to install ruby-oci8 gem, which is prerequisite for running Ruby applications on Oracle database. Add the following lines to /etc/enviroment file

LD_LIBRARY_PATH=/opt/oracle/instantclient_11_2

# as i am from Croatia, and we use our national date and money preferences, my NLS_LANG is this one, yours will maybe be different
NLS_LANG=CROATIAN_CROATIA.AL32UTF8

After you reboot you have to symlink the oracle library so it will function properly(i don’t know why they don’t do this themselves).

ln -s /opt/oracle/instantclient_11_2/libclntsh.so.11.1 /opt/oracle/instantclient_11_2/libclntsh.so

After that try and install ruby-oci8 gem.

rvmsudo gem install ruby-oci8

If there are no errors, and there should not be if you followed the guide, you are set to deploy your application on the server you just installed. In the next post, i plan to write something about tips, tricks and caveats of developing Rails applications on Oracle database.

Useful links to get and learn about all the stuff i have mentioned in the post:

Ruby Version Manager(RVM): https://rvm.io
Ruby: http://ruby-lang.org
Oracle Instant Client 11.2 for linux:  X86, X64
Passenger server: http://www.modrails.com/

Deploying Rails applications on Windows

Due to some legacy issues, and HP workstations being shit that you can’t turn the hardware virtualization on, i had an issue with one client regarding speed and loading time of an application. My common deployment server is running Ubuntu on VMWare or Oracle VBox, passenger + nginx, or apache even. But, this “server” is just a regular machine(people here don’t want to invest in real hardware, or just don’t have the money).

So i had to deploy the app on windows 7, and i will show you how.

First, there are few steps you have to take care of:

1. Install Ruby with the ruby installer http://rubyinstaller.org/downloads/

2. Install Ruby DevKit from the same source(this will be needed for you to compile some native binaries)

3. Install Thin server(gem install thin)

4. Set up dropbox deployment for your application(i know this is monkeypatching as hell, but if it works…) as described on Rob Conery’s blog

5. Test the app and make sure everything is working ok, (run rails server), then thin start -e production, just to make sure everything is ok, and all of your gems work on windows(fix or replace the ones that don’t)

6. Set up the task scheduler(this is a fine piece of software on Win 7 and up) as described  here, to run rails application almost as a windows service( you start with ‘thin start -e production -p 80 -c “path_to_your_app” ‘)

7. Write a nifty batch script that will restart the server, run migrations, compile the assets and whatnot.


cd "path_to_your_app"

bundle exec rake db:migrate RAILS_ENV=production

bundle exec rake assets:precompile

schtasks /End /TN "name_of_your_task_in_scheduler"

schtasks /Run /TN "name_of_your_task_in_scheduler"

8. After you pull to your dropbox synced repository, login to the server and execute the batch file from there to almost automate the deployment process.

I know this is not capistrano, and cap deploy, and it is hacked as it can be, and also, it works, it’s a small client so i don’t expect much trouble with this.

Lesson here: Try as much as you can to deploy your rails server on a linux server, best in a virtual machine. But if HP messes with you and you have some shitty hardware running windows, this is the best way i have found. If anyone has anything better, put it into comments, i’m happy to make my life easier.

References:

1. http://wekeroad.com/danger-danger/git-and-dropbox-sitting-in-a-tree

2. http://unicornless.com/systems-administration/run-thin-as-windows-service

Speaking at a conference with no time to prepare

I was surprised to know that i am presenting a product this week at the oracle user conference HROUG.hr

A colleague and me were supposed to hold a talk about creating our latest web app, a nursery management app created using Ruby on Rails and Oracle XE. As we thought, the talk was canceled and we were free of all obligation. But no, this saturday, 5 days before the talk, i was reviewing the agenda and found out the talk is on. So what can i do?

I will walk you through the steps, though they shouldn’t be used unless you are in a crunch, and it is not longer than an hour, luckily my talk is only half an hour.

1. Gather as much irrelevant info about the company and the client(our somewhat of a manager sorted that out)

2. If you are the developer, try to talk as much as you can about the technology and its integration( I love Rails so that is no problem )

3. Scrape out the basic workflow of the application you are presenting, user experience and why is something done in that way.

4. Take as much screenshots as it takes

5. Talk about details in the application, and finally leave at least 5 minutes for the audience to ask questions.

I managed to cramp all that into 30 minutes, i will update this post after i’m done thursday morning, to write how the talk went.

Lesson for everyone: Know your talking schedule at least one month before you are doing a talk, this way you can prepare and practice.

Update: The talk went surprisingly well, the hall was almost full, with 24 people listening. I haven’t got my reviews yet, but i feel they will be great for my first conference talk.

Rails 3.1 JQuery-UI

If you are considering the upgrade to Rails 3.1.x and you wish to include the JQuery-UI in your web application follow these simple steps:

Load jquery-ui in your app/assets/javascripts/application.js

//= require jquery
//= require jquery_ujs
//= require jquery-ui
//= require_tree .

Pick and download a nice theme from: JQuery-UI themes site

Copy jquery-ui-1.8.16.custom.css(version number could differ) to your vendor/assets/stylesheets folder as jquery-ui.css (This will save you time later when you are updating or changing the theme) and include it in your app/assets/stylesheets/application.css


*= require_self
*= require jquery-ui
*= require_tree .

Copy images from the images folder to vendor/assets/images

Modify vendor/assets/stylesheets/jquery-ui.css and replace “images/” to “/assets/”

Now that you have everything set up and ready, go ahead and use the fancy JQuery-UI plugins