Crash or… Monit!

As an application grows it starts demanding more and more work. At some point of the ongoing project, processing a task in the background becomes a necessity. One of the most popular tools to accomplish this in Ruby is Sidekiq.

Now, did it ever happen to you that you get it all up and running on the server and the next thing you know is the server is down and you get a colorful variety of error reports yelling that stuff is not working?

Meet Monit, a tool that’ll let you get ahead of trouble. It automatically restarts programs when they crash and can send email reports on failures!

In case you want to try it out –and make yourself a favour– here’s some aid on installing, configuring and get it up and running.

Install Monit

Depending on your package manager this may differ:

 

Configure monit

  1. Edit /etc/monit.conf  or  /etc/monitrc
  1. Uncomment (or add) these lines:

 

Add Sidekiq entry

Create the file /etc/monit.d/sidekiq.monitrc  with:

 

Start Monit

Let’s not forget to start Monit:  /etc/init.d/monit start

Now a sudo monit status should show the sidekiq_application_production0  entry.

If you use capistrano you will notice the current and shared directories. If you don’t use capistrano just make sure the PID is stored in some place where the path doesn’t change to let Monit find it.

This configuration will also be useful if you use capistrano-sidekiq, just make sure the check process name in the config matches what is used during deploy.

This configuration includes a memory protection that will restart Sidekiq before it eats up all the memory in your server. Yes, I’ve got a server down more than once due to this issue :(

 

Configure email server

To receive Monit’s status via email just add the following to /etc/monit.conf  or  /etc/monitrc

Add the email server, e.g.: for gmail’s SMTP add in the following line to the config file

set mailserver smtp.gmail.com port 587 username "notifier@gmail.com" password "********" using tlsv1 with timeout 30 seconds

 

Subscribe

To subscribe to all emails, including non-error emails, add the following line to the config file:

set alert my_app_notifier@gmail.com

 

If everything is set up correctly, executing sudo monit reload  should trigger an email.

If you want to get emails for specific events or have more questions regarding Monit’s config check out http://mmonit.com/monit/documentation/monit.html.

A word of warning, the doc is huge. If you are looking for something specific like email, alert or whatever you need, just search it.

 

That’s pretty much it. Of course Monit doesn’t take care of starting the processes if the server restarts. You’ll have to look for another tool to do that.

Reposted from http://andisxp.blogspot.com/2015/03/keep-sidekiq-running-using-monit.html


Just Enough Linux to be Dangerous

This post is inspired on a talk about Ruby & Linux at the RubyConf Uruguay 2014.

The Linux Process in Ruby

Creating a new process

In linux we use fork to create a new child process. In Ruby we have access to this by using  Process::fork

It accepts an optional block. We can use either Process.fork or simply fork. When a block is given the child process runs the block and terminates with a status of zero. When no block is given the fork call returns twice, once in the parent process returning the child pid and once in the child process returning nil. When using fork only the thread calling fork will be running in the child process, no other thread will be copied. Since ruby 2 there’s been an improvement to copy on write with relation to the GC. Instead of duplicating all the data it just copies the data when the shared memory is modified by one of the processes.

Continue reading…


What’s new on RSpec

On June 2, 2014, RSpec 3 was released after months of announced features that were becoming available through new versions of the RSpec 2 branch.

Here at Moove-IT, we decided to take a look at the new stuff, as well as say goodbye to some of the old things that have been removed. Just in case some of us who use RSpec haven’t paid attention to them lately, we put a little bit of Better Specs on top of all that.

Continue reading…


When and why Clojure

Introduction

When you have to solve a problem, in order to make an efficient and effective solution you must choose the right tools to help you in the process of building it. As programmers, the languages we choose determine a big part of the solution and they will influence the way we reason about it, so it’s really helpful to understand what are the kind of problems that a language aims to solve before starting to use it. Following that line, this post tries to summarize the key features of Clojure that make it preferable over other languages under some conditions, based on what Clojure designers and its community predicate about the language.

Continue reading…


Token based Authentication – Json Web Token

What is it for?

Token based authentication is a new security technique for authenticating a user who attempts to log in to a secure system (e.g. server), using a signed token provided by the server.

The authentication is successful if the system can prove that the tokens belong to a valid user.

It is important to note that Json Web Token (JWT) provides signed tokens but not encrypted ones, so passwords or any critical information must not be included in the token unless you encrypt the data (e.g. using JWE).

Continue reading…


How we do Asynchronous Planning Poker Estimations

Have you ever worked in teams, where several and simultaneous projects take place and stories need to be estimated quickly and accurately so as to commit to a delivery date?
If you ever encountered this challenge, you would probably find this article very useful.

A different scenario from Sprint Planning

Traditional Scrum Sprint planning definitely works for some of our clients. But, in this particular scenario we are talking about a team working on several projects for a single client. Therefore, we are constantly receiving tickets which need to be estimated. Since it does not make sense to gather the whole team too often in order to estimate, we came up with this idea of an asynchronous planning. We are positive you would find it extremely helpful.

How it works?

Diagram

  • Once a new task arrives, the team members working for that particular project know they must post their personal estimates on a special board (asynchronously and not anonymously). This board would then contain several post-its with the right estimations facing the wall, so they are not visible for the rest of the team.
  • After a minimum of estimates have been cast, one team member turns around the pieces of paper and gets an average.
  • In cases where the estimates presented are too different (which is rare), and then the average could be just not that useful, the team members gather to discuss the estimates more in details and reach an agreement.

Every two weeks the estimations are analyzed and compared with the actual execution time. Along with this, a retrospective meeting takes place to improve the overall functioning of the team.

Pros

  • Super fast.
  • Average improves accuracy.
  • Visibility (every team member sees all estimates).
  • Time saving.
  • Team player philosophy.
  • Improves performance.

Cons

  • Fewer group discussions.
  • Only works with highly organized teams.

Taking it to the next level

We started this process using only a whiteboard, a few markers and some post-its. As we are geeks, and love creating software to solve pretty much everything, we decided to take this to the next level and create a specific tool for it. We called it “Zenkai” – playing with the letters of “Kaizen”.

Zenkai

It works by simply automating the above described process.

Pending Estimates
Dashboard with pending estimates.
Dashboard
Dashboard.
Report
Tickets per project report.
reports_2
Hours and Team Velocity over Time.

 

Zenkai on GitHub – Fork it and tweak it, or submit your Pull Request!

We’ve been following this process (and using this tool) for almost a year now and are constantly trying to improve it. Naturally, this facilitates the estimation process. Beyond that, it truly helps our teams to stay aligned, deliver in time, and keep track of their improvement.

If you want to implement this with your team, please contact us and we’ll be happy to discuss further!


Lean Startup Machine Montevideo

Lean Startup Machine is taking place in Montevideo this year, on August 1st – 3rd, 2014.

This three-day workshop focuses on teaching innovators and entrepreneurs how to start a business using Lean Startup.

Ariel Ludueña (CEO) and Martin Cabrera (CTO) both at Moove-IT have been invited as mentors to be part of this great Lean Startup educational experience.

For more information: https://www.leanstartupmachine.com/cities/montevideo/

 

Lean-Startup-Machine

Is Scala worthwhile?

If you’re a Java developer and start digging into some Java blogs, forums, books, etc. you’ll  surely find some people arguing in favor of Scala, and read a lot of people talking about its functional approach and how cool it is.

Here I won’t attempt to cover Scala’s features in depth, though I will point out why I think it’s worth giving it a try.

Some history

Scala was designed in 2001 and its first version was implemented in 2003 on the Java platform, by Ph D. Martin Odersky. He is a  professor of programming methods at EPFL, implemented Java generics and also the current generation of javac, so I am pretty sure this guy has a lot of Java Knowledge :)

Later, on January 2011, the Scala team won a five year research grant of over €2.3 million and with this funds, in May, Odersky and his collaborators founded Typesafe Inc. In 2012 Typesafe received a $3 million investment from Greylock Partners.

Though you may wonder, is this just a theoretical research from some university? or,  Is it a platform to build enterprise software? Well, after looking at who’s using it, it turns out that many companies are building new services based on Scala, including Siemens, Novell, Sony, LinkedIn, Foursquare and Twitter.

Let’s get into some code

Syntactic sugar

As a newer language it provides some syntactic constructions aiming at the developer efficiency. Some quick examples:

Scala aims at not disrupting object oriented principles, that’s why it has an object construction which is exactly what it says: one instance of a class. Provided that, and bearing in mind that you can name an object the same name as a class, there’s no more static keyword, just call an object by its name.

Functions

As you can read everywhere, Scala is an hybrid, it’s both an imperative object oriented language as well as a turing complete functional one. There’s a lot of interesting theory you can read about functional programming written by excellent authors, so here we’ll just see some examples again:

A function taking another function as a parameter. predicate is a function that takes one element of the same type A of the list and returns whether it should be filtered or not.

Here we’re creating another function utilizing the previous one but fixing one of its parameters, this returns a new function, which I called filterList. Note that it could have been declared as a val since in functional languages, functions are so called first-class citizens.

Finally,  we can call the filterList function with a predicate, which given an element of type String (inferred by the List of Strings) computes the standard java.lang.String.startsWith(String).

Pattern matching

Some people introduce pattern matching as a switch statement with some enhancements. In my opinion this is  like an entire new feature since it’s more powerful and less restrictive.

Let’s see an example:

Here we can see a lot of things going on. The first one is that we’re pattern matching on list value, and Scala will compare its value with the left side of each case expression, from top to bottom. In the first case we compare against a literal list, there we ask if it’s a list and if it contains those exact two strings. Then we simply compare to a string literal, or we can ask for the value’s type like the Int case. We can also add a conditional guard like the string with the if s startsWith “A” condition.

Everything is an expression

Another great thing to add is that the whole match we just saw is an expression and it’s returning the value on the right side of the matched case,  meaning that we could write:

And the compiler will infer the type of myListString, in this case being String,  because all right sides are Strings. Though they could be of other types,  and Scala would  take the base class that all of them extend.

In fact, all sentences in Scala are expressions, meaning we can get the result, for example of an if expression:

Traits

You can think of traits either as Java interfaces where you can implement methods,  or as abstract classes with the advantage that you can extend as many as you want.

Whatever way you see it, traits are a pretty way of encapsulating behavior and reutilizing code.

Imagine you have a Bird class like this:

Then you could use it like this:

That was easy. Now suppose you need a bird like the Penguin, who can swim but which can’t fly. Then you could make an abstraction like FlyingBird so Pigeon and Hawk would extend it while Penguin could directly extend Bird.

Ok, now there comes the Frigatebird, a bird that can fly but cannot swim. This bird will surely drive you crazy because he can’t swim and, in a language like Java, you can’t create a class hierarchy modeling all these birds, without duplicating code. The problem we are facing here is the well known problem of multiple inheritance and how Java handles it, you can create two abstract classes one for flying and another one for swimming, but you can’t extend both.

Here’s how you would likely solve it if you’re using Scala:

This is how birds would look like:

And now you could execute things like this:

But something like this won’t compile:

In this last case, Scala would complain that fly is not a member of Bird. That’s because it infers that Bird is the only class that Frigatebird and Penguin have in common, assigning Bird as the type parameter of the List.

So, why Scala?

Besides these features we’ve quickly looked at, Scala introduces implicits, structural types, extractors, and many more, taking the best from both worlds (Object Oriented and Functional Programming). Having that said, it could be stated that Scala is concise, flexible and Scalable.

I haven’t mentioned performance, but you could be worried about some functional constructions that create new instances instead of writing values in-place. Well, we know that the garbage collector is usually pretty good with short-lived objects but also, thanks to the concise language, you can spend your time optimizing critical parts instead of dealing with tedious non-performance critical parts. There are a few benchmarks that compare Scala to Java and they confirm that it performs fast.

As I mentioned earlier, Scala is accepted by the industry, so you can be sure it will last long and its community will be supported.

Finally it’s good to know there are plenty of tools to empower Scala, from plugins for almost every IDE and a very cool building tool, to web frameworks and database access libraries.

In the worst case scenario you could end up writing Java-style code in Scala and it would be compiled to almost the same binary code and it’d be run in the same JVM. The question we should ask ourselves is: why don’t we just give Scala a try?

References