Monthly Archives: June 2011

Startup, corporations? Why new graduated students should pick startups

After reading these blog posts on why new graduates should or shouldn’t join a startup, I feel I should give my insights based on my past experiences as well as on my experience at Miso. First of all, let’s put that statement in context, as I have only just started my career and I’m not yet graduated. However, I’ve already worked for more than 1 year (full time and part time) at Société Générale as Tech Op, which is one of the biggest banks in Europe, and all of my past internships have been in startups…

After coming across why we shouldn’t join a startup blog posts, there are 2 points I would agree with him on:

  • Startups are not for everyone, especially tech startups, everyone isn’t a “geek”, neither excited by new features on Twitter, that’s a fact.
  • Both posts quoted Steve Blank: “It’s your curiosity and enthusiasm that will get you noticed and make your life interesting.” It’s kind of obvious, you will enjoy your work if you’re passionate about it.

However, I disagree with most of his other ideas…

1. Passion is the ultimate competitive advantage.

Yes, in both startups and corporations you can work with passion, the main difference between these two is that in a startup, you are sure to work with people who enjoy what they’re doing, and that’s not always the case in my past experiences in corporations.

His main argument is that in large corporations, employees “can afford to have highly specialized jobs filled by people who only focus on one particular component”. Ok I can understand that, but to me being close in his skill area, it’s to me like communitarianism because you can’t learn and you can’t expand in your field. At Miso even if my official position is Rails Developer, sometimes I had to sketch out some design for new features. Basically my point is yes, in startups you have to assume many roles, but it’s necessary, it teaches you to think globally and not be focused on only your code.

2. Leverage

Have a big companies name on your resume always gives you some credibility, that’s true, but I would rather have an unknown startup on my resume which had failed and be able to explain why it had failed, and on the other hand if you have worked for a successful startup, being part of the small team looks significantly more impressive.

3. “Startup years are like dog years”

It’s been already 2 months I’ve been working at Miso and I feel like it was a year. It’s an intense internship, I learn so much things not only in ruby, in product side, and in community management… I’m not claiming I’ll be a guru of these domains at the end, but’s it’s definitely improving my culture and above all helping me to understand why I’m coding these new features.

4. More Money

If your first motivation is making money, don’t work for a startup… I turned down a better offer in terms of salary, to accept the Miso one because I feel it was and is the right place to learn. I don’t know if Miso will be the next big hit in the valley but anyway, I don’t care if we fail, again the most important thing it’s to understand why you failed.

“If you make meaning, you’ll probably make money”- Guy Kawasaki. I really believe this quote can be applied as employee not only in a company context, I really need to know why I’m doing this feature, for which purpose. At Miso, the purpose is simple; we want to “Make TV more fun, change the way how people are watching TV”. During my time at Société Générale, I didn’t even know what the company actually did. I worked in a subsidiary called Société Générale Securities Services and to this day, I still don’t know what it’s doing. The worst part was not one of my colleagues was able to explain it to me either. The only thing that mattered was to keep the servers running, beyond that I never was encouraged to understand the larger picture.

5. Relationships

It seems in a corporation you can build a network of “hundreds or thousands of coworkers”. If it’s the case, it’s a bit curious to build this network only within the company. I believe in openness and at Miso, the engineering team tries to be close to San Francisco Ruby Community. We attend meetups, hosting ruby workshop and generally try to be involved. There are 2 goals behind this, promoting what we’re doing and also promoting ruby. Most of all it’s about sharing and improving our culture. You could learn for example what’s the best practices in hot new startups just by making a connection. This gives you some insight on what you’re doing. It’s a win-win situation to be involved outside of the narrow focus of your company.

Relationships in startups with your colleagues are not the same as in a corporation. The day-to-day interactions drive more than a simple colleague relationship. I’m not saying your colleague will become your best friends, but I definitely know I will stay in touch with my Miso colleagues after my adventure here. And I still hang out with my lovely pariSoma friends as well.

To conclude, for these reasons I definitely know that I don’t want to work for corporations. On the other hand some people needs a “safe” work because their situation requires it, I can understand that too. But that doesn’t mean startups can’t work with “corporations”, we need them and they need us. We work with many larger organizations here such as Fox, Logo, Showtime, etc. The most important thing I believe is that working for a fast paced startup is worth trying because you will rapidly grow your experience and learn about things that you never expected.

Building a Platform API on Rails

Introduction

In my last post, I discussed the failings of to_json in Rails and how we approached writing our public APIs using RABL and JSON Templates in the View. Since that post, we have continued expanding our Developer API and adding new applications to our Application Gallery.

The approach of using views to intelligently craft our JSON APIs has continued to prove useful again and again as we strive to create clean and simple APIs for developers to consume. We have used the as_json approach on other projects for years in the past and I can say that I have not missed that strategy even once since we moved to RABL. To be clear, if your APIs and data models are dead simple and only for internal or private use and a one-to-one mapping suits your needs, then RABL may be overkill. RABL really excels once you find yourself needing to craft flexible and more complex APIs rich with data and associated information.

The last post was all about explaining the reasons for abandoning to_json and the driving forces behind our creation of RABL. This post is all about how to actually use Rails 3 and RABL in a psuedo real-world way to create a public API for your application. In other words, how to empower an “application” to become a developer’s platform.

Authentication

The first thing to determine when creating a platform is the authentication mechanism that developers will use when interacting with your API. If your API is read-only and/or easily cached on a global basis, then perhaps no authentication is necessary. In most cases though, your API will require authentication of some kind. These days OAuth has become a standard and using anything else is probably a bad idea.

The good news is that we are well past the point where you must implement an OAuth Provider strategy from scratch in your application. Nevertheless, I would strongly recommend you become familiar with the OAuth Authentication Protocol before trying to create a platform. Starting with this in mind, your next task is to pick an OAuth gem for Ruby. Searching Github, you can see there are many libraries to choose from.

The best one to start with is Rails OAuth Plugin which will provide a drop-in solution both to consuming and providing APIs in your Rails 3 applications. The documentation is fairly decent and coupled with the introduction blog post, I think reiterating the steps to set this up would be unnecessary. Follow the steps and generate an OAuth Provider system into your Rails 3 application. This is what will enable the “OAuth Dance” where developers can generate a request token and sign with that token for a user to retrieve and access token. Once the developer gets an access token, they can use that to interact with your APIs. The easiest way for developers to consume your API is to use a “consumer” OAuth library such as Ruby OAuth or equivalent in the client language.

Building your API

Now that we have authentication taken care of in our application, the next step is to create APIs for developers to interact with on our platform. For this, we can re-use our existing controllers or create new ones. You already likely have html representations of your application for users to interact with. These views are declared in a “index.html.erb” file or similar and describe the display for a user in the browser. When building APIs, the simplest way to think about them is to think of the JSON API as “just another view representation” that lives alongside your HTML templates.

To demonstrate how easy building an API is, let’s consider a sample application. This application is a simple blogging engine that has users and posts. Users can create posts and then their friends can read their streams. For now, we just want this to be a blogging platform and we will leave the clients up to developers to build using our APIs. First, let’s use RABL by declaring the gem in our Gemfile:

# Gemfile
gem 'rabl'

Next, we need to install the new gem into our gemset:

$ bundle install

Next, let’s generate the User and Post tables that are going to be used in this example application:

$ rails g model User first_name:string last_name:string age:integer
$ rails g model Post title:string body:text user_id:integer
$ rake db:migrate db:test:prepare

Nothing too crazy, just defining some models for user and posts with the minimum attributes needed. Let’s also setup model associations:

# app/models/user.rb
class User < ActiveRecord::Base
  has_many :posts
end

# app/models/post.rb
class Post < ActiveRecord::Base
  belongs_to :user
end

Great! So we now have a user model and a post model. The user can create many posts and they can be retrieved easily using the excellent rails has_many association. Now, let’s expose the user information for a particular user in our API. First, let’s generate the controller for Users:

$ rails g controller Users show

and of course setup the standard routes for the controller in routes.rb:

# config/routes.rb
SampleApiDemo::Application.routes.draw do
  resources :users
end

Next, let’s protect our controller so that the data can only be accessed when a user has authenticated, and setup our simple “show” method for accessing a user’s data:

# app/controllers/users_controller.rb
class UsersController < ApplicationController
  before_filter :oauth_required

  respond_to :json, :xml
  def show
    @user = User.find_by_id(params[:id])
  end
end

As you can see the controller is very lean here. We setup a before_filter from the oauth-plugin to require oauth authentication and then we define a “show” action which retrieves the user record given the associated user id. We also declare that the action can respond to both json and xml formats. At this point, you might wonder a few things.

First, “Why include XML up there if we are building a JSON API?” The short answer is because RABL gives this to us for free. The same RABL declarations actually power both our XML and JSON apis by default with no extra effort! The next question might be, “How does the action know what to render?”. This is where the power of the view template really shines. There is no need to declare any response handling logic in the action because Rails will automatically detect and render the associated view once the template is defined. But what does the template that powers our XML and JSON APIs look like? Let’s see below:

# app/views/users/show.rabl
object @user

# Declare the properties to include
attributes :first_name, :last_name

# Alias 'age' to 'years_old'
attributes :age => :years_old

# Include a custom node with full_name for user
node :full_name do |user|
  [user.first_name, user.last_name].join(" ")
end

# Include a custom node related to if the user can drink
node :can_drink do |user|
  user.age >= 21
end

The RABL template is all annotated above hopefully explaining fairly clearly each type of declaration. RABL is an all-ruby DSL for defining your APIs. Once the template is setup, testing out our new API endpoint is fairly simple. Probably the easiest way is to temporarily comment out the “oauth_required” line or disable in development. Then you can simply visit the endpoint in a browser or using curl:

// rails c
// User.create(...)
// rails s -p 3001
// http://localhost:3001/users/1.json
{
  user: {
    first_name: "Bob",
    last_name: "Hope",
    years_old: "92",
    full_name: "Bob Hope",
    can_drink: true
  }
}

With that we have a fully functional “show” action for a user. RABL also let’s us easily reuse templates as well. Let’s say we want to create an “index” action that lists all our users:

# app/views/users/index.rabl
object @users

# Reuse the show template definition
extends "users/show"

# Let's add an "id" resource for the index action
attributes :id

That would allow us to have an index action by re-using the show template and applying it to the entire collection:

// rails s -p 3001
// http://localhost:3001/users.json
[
  {
    user: {
      id : 1,
      first_name: "Bob",
      last_name: "Hope",
      years_old: "92",
      full_name: "Bob Hope",
      can_drink: true
    }
  },
  {
    user: {
      id: 2,
      first_name: "Alex",
      last_name: "Trebec",
      years_old: "102",
      full_name: "Alex Trebec",
      can_drink: true
    }
  }
]

Now that we have an “index” and “show” for users, let’s move onto the Post endpoints. First, let’s generate a PostsController:

$ rails g controller Posts index

and append the posts resource routes:

# config/routes.rb
SampleApiDemo::Application.routes.draw do
  resources :users, :only => [:show, :index]
  resources :posts, :only => [:index]
end

Now, we can fill in the index action by retrieving all posts into a posts collection:

# app/controllers/posts_controller.rb
class PostsController < ApplicationController
  before_filter :login_or_oauth_required     
  respond_to :html, :json, :xml      

  def index          
    @posts = Post.all      
  end  
end

and again we define the RABL template that serves as the definition for the XML and JSON output:

# app/views/posts/index.rabl  

# Declare the data source  
collection @posts  

# Declare attributes to display  
attributes :title, :body  

# Add custom node to declare if the post is recent 
node :is_recent do |post|      
  post.created_at > 1.week.ago
end

# Include user as child node, reusing the User 'show' template
child :user do
  extends "users/show"
end

Here we introduce a few new concepts. In RABL, there is support for “custom” nodes as well as for “child” association nodes which can also be defined in RABL or reuse existing templates as shown above. Let’s check out the result:

[
  {
    post: {
      title: "Being really old",
      body: "Let me tell you a story",
      is_recent: false,
      user: {
        id : 52,
        first_name: "Bob",
        last_name: "Hope",
        years_old: 92,
        full_name: "Bob Hope",
        can_drink: true
      }
    }
  },
  { ...more posts... }
]

Here you can see we have a full posts index with nested user association information and a custom node defined as well. This is where the power of JSON Templates lies. The incredible flexibility and simplicity that this approach affords you. Renaming an object or attribute is as simple as:

# app/views/posts/index.rabl

# Every post is now contained in an "article" root
collection @posts => :articles

# The attribute "first_name" is now "firstName" instead
attribute :first_name => :firstName

Another benefit is the ability to to use template inheritance which affords seamless reusability. In addition, the partial system allows you to embed a RABL template within another to reduce duplication.

Wrapping Up

For a full rundown of the functionality and DSL of RABL, I encourage you to check out the README and the Wiki. In addition, I would like to point out excellent resources by other users of RABL and encourage you to read teohm’s post as well as Nick Rowe’s overview of RABL as well. Please post any other resources in the comments!

My intention with this post was to provide a solid introductory guide for creating an API and a data platform easily in Rails. If there are topics that you would be interested in reading about more in-depth related to API design or creation or building out a platform, please let us know and we will consider it for a future post.

Hybrid (Native + Web) Mobile App Development • Part 1: The Motivation.

In the Beginning

Miso’s most popular platform, the iPhone, was initially conceived through the use of iOS’s native framework. Aside from occasional REST API calls to the web server for data, the entire user experience was delivered through native UI elements provided to us by the iOS SDK. Everything was great! Native apps are fast, performant, and did its job. As the iPhone app gained popularity and traction with the community, the natural next move was to bring the Miso experience to other mobile platforms. Thus, the android, iPad versions of Miso were born driving even more users to our service.

And then?

While native apps certainly have their advantages, launching features across multiple platforms was terribly time consuming. Eventually, we found that unless the size of our engineering team increased significantly, the native app approach just isn’t going to scale well in the long run.

At the time of writing this article, there are 3 popular approaches to mobile app development. Native, web, and hybrid. In this article, we will briefly discuss the pros and cons of each method, and what Miso eventually chose and the motivation behind it. (We chose hybrid!)

Native

Despite the complaints over the maintenance costs and scalability issues mentioned earlier, there are definitely use cases where you should strongly consider using the native SDK. Angry Birds would not have worked if they didn’t leverage hardware accelerated graphics. So any app that needs mobile device hardware, performance centric, and a very rich UI with bouncing and flashing buttons and labels should stick with the native approach.

For Miso, those strengths weren’t high in our list of priorities. We wanted multi-platform feature releases to be iterative, fast and scalable.

Web

The web approach touts strengths that seem to be what Miso needed. No need to submit to an app store, just deploy it on the web at any time and any mobile device with a web browser would be able to get the latest and greatest! Not to mention, you get all the Javascript/CSS goodies all for free!

We liked that. It was the opposite of the native approach. We can get away from painstakingly getting our pretty designs to work across multiple platforms, and release features and bug fixes with a push of a button almost instantly.

However, with web apps that means going anywhere in the app requires firing off a HTTP request and getting a response before you can render anything. It’s slow, clunky, and simply unsuitable for the user experience we wanted to deliver. We couldn’t leverage local caching of pages or data. Zero access to the iOS framework for nifty push notifications, gestures, or even the occasional use of native UI elements.

Hybrid

Miso engineers are a greedy bunch. We wanted the best of both worlds; The fast, snappy feel of a native apps coupled with the ease of styling and quick releases of web apps. So we embarked on a long journey of slowly porting native views of Miso’s iPhone app to web views.

It took us months to build a framework we are satisfied with, and we learned a ton on the way to achieving that goal. The Miso app you see now (3.0.x) has web views supporting a majority of our layouts by combining JSON responses from REST API requests with EJS templates. In this latest release (3.0.3), we’ve also introduced the leveraging of local disk caching of javascript/css and web templates to optimize performance; and added support for AJAX requests to make web views even prettier and user friendly.

At this time, we are happy with our solution as it gives us the ability to tweak the look and feel of our app without submitting a release to the app store. It also opens up the path to eventually integrating these web views into Miso apps across other mobile platforms. Ultimately allowing us to release the latest Miso features to all of our users in a timely fashion.

Next Time

While many other articles talk about the hybrid approach, few have attempted to educate in building a solid framework to support this idea.

In the subsequent articles, I will cover the concepts and designs of our web view framework and things we’ve learned along the way that may help should you choose this hybrid approach. Feel free to leave any comments and suggestions on topics you’d like me to cover!

Part 2

Low hanging fruit for Ruby performance optimization in Rails

Our goal: we currently spend about 150-180 ms in Ruby for an average web request, and we think it’s reasonable to improve that to be around 100 ms.

Two of the most expensive operations in Ruby are: object allocation and garbage collection. Unfortunately, in Ruby (unlike Java), garbage collection is synchronous and can have a major impact on the performance of requests. If you’ve ever noticed that a partial template rendering occasionally randomly takes a couple of seconds, you’re probably observing a request triggering garbage collection.

The good news: it’s easy to quickly (< 10 min) see how much your app is impacted from garbage collection. You’re likely to improve your performance by 20-30% just by tuning your garbage collection parameters.

If you have a production Rails app, and you’re even remotely interested in performance, I’m going to assume you’re using the excellent New Relic service.  If you’re using REE or a version of Ruby with the Railsbench GC patches, it’s easy to turn on garbage collection stats that will be visualized by New Relic.

You’ll get pretty charts like this:

by adding:

GC.enable_stats

somewhere in your initialization.  After enabling garbage collection stats in our application, we can see that approximately 20% of our Ruby time is spent in garbage collection, which implies that there’s also a not-so-insignificant portion of time spent in object allocation.

What’s next

We last tuned our Ruby garbage collection parameters about 5 months ago and, after an initial performance boost, we’ve seen the application time spent in Ruby creep back up.  To try to bring the response time back down, our next steps are to:

  • Consider taking another pass at garbage collection parameter tuning.  Since we’ve already taken one pass at this, I’m not sure if we’ll be as impactful the second time around, but we’ll see.
  • Identify the slowest controller actions via New Relic and profile them using ruby-prof or perftools.

Performance tuning using ruby-prof is likely going to vary a lot depending on the code, but if we find techniques that might apply more broadly, we’ll be sure to blog about it here.

Links