Why YAML Record?

At Miso, we occasionally ran into situations where we wanted to persist simple data to a text (or yaml) file. Examples include landing page email forms, contact forms, feedback forms, about us “team” pages, etc. In these situations, we wanted a way to persist data to YAML and easily view the results in a text file but also manage the data through forms and controllers as any other data. To achieve this we have created a YAML-backed persistence engine called YAML Record that allows access to YAML based data using a familiar ActiveRecord API.

If you have a small amount of data to persist which changes infrequently but that you do need to update from time to time, you should investigate using YAML Record. In other cases where you have frequently updated information, several query requirements or a medium/large set of data, a SQL or NoSQL solution would probably be a better fit. Using YAML for persistence doesn’t work in every case so use this approach wisely.

YamlRecord is a standalone, simple and lightweight way to map your data from a yaml file, and can easily be integrated along with ActiveRecord and we try to provide the same APIs. On the other hand if you’re using DataMapper, I would recommend checking out dm-yaml-adapter which allows for similar functionality.

How do we use this at Miso?

We have several use cases for YamlRecord. The first one is to persist our team members profiles on our about page. Using YAML Record, we can easily add, update or remove any of this information with a RESTful architecture and standard controllers. We even implemented full DragonFly support with YAML Record so we can link up to each team member with a picture. I’ll probably cover this functionality on my next blog post.

In another case, we use YAML Record for our FAQ. From an relational data perspective, storing this questions requires only a single table without any relationships to other tables or any indexes. For cases like this, a SQL solution and the overhead of creating a migration and a table doesn’t always make sense. We definitely could use a SQL table, but having one with 10 rows and only 3 columns seems almost silly.

------------------------------------
|  id  | question | answer         |
------------------------------------
|  1   | foo?     | blablabla      |
------------------------------------
|  2   | bar?     | blablaaaaa     |
------------------------------------
|  3   | L33t?    | Y3s            |
------------------------------------
|
...
|
------------------------------------
|  10  | n00b?    | n0!            |
------------------------------------

I would rather have a YAML file which looks like the example below. The main upside of using a YAML file is Rails is going to load entire file in memory once, that way your application doesn’t need to go back and forth to read your file each time.

---
- :id: 08cc5f757b26903b9e7b6fcc3a3fbe
  :question: foo?
  :answer: blablabla
- :id: 4f2e73f896d0443c2b66f57cbdb2bb
  :question: bar?
  :answer: blablaaaaa
- :id: 5197b037524a1629d27c69f7aa9891
  :question: L33t?
  :answer: Y3s
- :id: f77c91577af4bf9b1e994e1b5b9ecd
  :question: n00b?
  :answer: n0!

Using the same APIs as ActiveRecord

Here’s a quick sneak peek on what you can achieve with YamlRecord. Given a class HotGirl:

class HotGirl < YamlRecord::Base
  # Declare your properties
  properties :name, :sex_appeal_rate, :age

  # Declare source file path (config/hot_girls.yml)
  source Rails.root.join("config/hot_girls")
end

Here’s some of actions you can do:

# Create your instance as on active record
@hg = HotGirl.new(:name => "Jennifer Anniston", :sex_appeal_rate => 75, :age => 42) # @hg
@hg.save # => true

# Alternatively
@hg2 = HotGirl.create(:name => "Scarlett Johansson", :sex_appeal_rate => 92, :age => 26)

# You can get your items easily
HotGirl.all # => [@megan_fox, @jessica_alba ...]

# Update the record attributes
@megan_fox.update_attributes(:sex_appeal_rate => 100) # => true

# Destroy it
@miss_usa_2011.destroy => true

We tried to provide all essential APIs ActiveRecord provides and more are coming. If you want to know about the available APIs, you can find it on the README or Rdoc

Feedback?

We would love to hear your feedback on this concept to persist data to a YAML file in certain simple cases and about how we can improve this library in the future. The source code is available on Github and we welcome all patches and pull requests. Feel free to share your thoughts about it.

Conclusion

We really believe this concept of persisting data in YAML file makes a lot of sense for us in particular scenarios. Building this was actually a challenge for me as it was my first gem. I want to thank Nathan for his support building the gem and I really learned a lot in the process. After all, I guess we did it for fun and as an experiment to see how this would work.

We also found these 2 gems in same vein as YAML Record, which are worthwhile to check out:

• Datamapper GoogleSheet adapter
• Gitmodel

Welcome to part 2 of this multi-part series of blog posts where we venture into the world of hybrid frameworks. This is where we get into the juicy stuff, if you are new to this, I suggest reading part 1 to understand the motivation behind this approach.

Big Picture Stuff

In a nutshell, what we’re trying to do here is mash a JSON response from a RESTful API call with a ejs (similar to rail’s erb) template to form the html to be rendered by a UIWebview. Simple, until you start asking questions such as how to an element to transition to a different state? How do you AJAX style interactions? How is template/asset syncing managed? These are all critical questions to ask of a hybrid framework. So before we dive into it, let’s make a short list of what a hybrid framework should be capable of before diving deeper into each feature separately.

A hybrid framework should:

* Maintain template syncing from a third party resource

* Handle bridging interactions between the web and native views

* Manage a RESTful API SDK

Maintain Template Syncing

At Miso, our mobile web templates are served off an Amazon S3 server. The basic strategy we employed is to download the web templates and cache it on disk, and when you open up the app in the future it would update it’s local templates with the latest and greatest by poking S3.

To accomplish this we used ASIHTTPRequest, a popular web request wrapper, to maintain web requests we send to the S3 server. This means asynchronous requests, cache responses, and a ASINetworkQueue that lets you handle multiple web requests.

There were a few nuggets we threw in worth mentioning here that made this strategy more efficient and scalable. We maintain a mainfest file on the server that has the list of assets the mobile device needs to download. The file would look something like this:

homepage.ejs?12398789
javascript/application.js?9437392
images/logo.png?1283740

We keep a versioning number by appending ?<some numbers> after each file. This is generated automatically every time we make updates to the web templates. Having this file allows us to scale well when we add more files, because all we have to download is the manifest file on startup.We then leverage ASIHTTPRequest’s cachedResponse flag and iterate through each file and check with s3 to ensure that this file hasn’t changed since we last fetched it.

To give a more concrete view of how exactly we maintain template syncing from a web request stand point, here’s a code snippet of an implementation of the delegate method where a ASIHTTPRequest completes showing the logic explained earlier:

- (void)requestFinished:(ASIHTTPRequest *)request {
    NSString *responseString = [[[NSString alloc] initWithData:[request responseData] encoding:NSUTF8StringEncoding] autorelease];

    // If the returned file is the manifest file, iterate through it and send web requests for each asset
    if ([[[request url] description] hasSuffix:@"manifest.mf"]) {
        // Separate assets and shove them into an array
        NSArray *assets = [responseString componentsSeparatedByCharactersInSet:[NSCharacterSet newlineCharacterSet]];

        for (NSString *asset in assets) {
            // Get rid of trailing white spaces in assets list
            if ([[asset stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceCharacterSet]] isEqualToString:@""])
                continue;

            // Add each asset in ASINetworkQueue
            [self addUrl:asset queue:_queue];
        }
    } else {
        // If it's asset file, templates on all views need to reload unless the response is from cache
        _needsReload = _needsReload || ![request didUseCachedResponse];
    }

    NSString *file = [[request url] relativePath];
    NSString *fqFilePath = [_localBaseUrl stringByAppendingPathComponent:file];

    // If the file doesn't exist on disk, or the response is not from cache, then it's a updated file and we should save it
    if (![request didUseCachedResponse] || ![[NSFileManager defaultManager] fileExistsAtPath:fqFilePath]) {
        [self saveLayoutFile:[[request url] relativePath] data:[request responseData]];
        _needsReload = YES;
    }
}

I want to mention one last thing before I wrap up this section, and that is how we handle updating templates across different version of our app. Let’s consider the situation where you intend to release a new version of your app complete with the latest web templates. You realize that there are backwards compatibility issues of the new templates with an older version of the app. If you were to update the current web templates it’d break everyone on an older version of the app. Ooops! How we work around that is to maintain separate buckets of web templates for different versions of our app. It’s not always necessary, but something good to keep in mind.

Bridging Interactions between Web and Native views

Let’s elaborate on what this means via examples:

* User clicks on a link in a webview, and it pushes another UIViewController onto the navigation stack

* User posts a comment via a native controller, and we want that to fire off an AJAX request and render on to the web view

How Miso achieves this is by defining certain protocols to conform to when generating links within our EJS templates. That way, when a load request is captured by the UIWebView we can then route these requests to different parts of the app.

- (BOOL)webView:(UIWebView *)webView shouldStartLoadWithRequest:(NSURLRequest *)request
 navigationType:(UIWebViewNavigationType)navigationType {
    NSString *url = [[request URL] description];

    // Check for protocol type and determine routing
    if (navigationType == UIWebViewNavigationTypeOther) {
        return YES;
    } else if ([url hasPrefix:@"miso://ajax"]) {
        [_ajaxController fireAjaxRequest:url];
        return NO;
    } else {
        // RoutesController delegates native app actions given a miso://<controller>/<action>?<params>
        [[RoutesController instance] processRoute:url viewController:_vc webview:self];
        return NO;
    }
}

The code snippet above is a stripped down version of what we use currently, but the basic concept is the same. We delegate routing by implementing the shouldStartLoadWithRequest delegate method for UIWebViews. So in the case where we capture a miso:// protocol we pass it along to other classes to process.

In the next part, we will start venturing further down the rabbit hole and look into the concepts behind RoutesController and AjaxController. Stay tuned!

Miso (and I) had our first Hackathon last week, and both of us survived! Our goal: conceive of an idea and build it within 3 days. We built Listify – an iPhone app that let’s you create, browse, and upvote lists of TV show and movies. Also with accompanying web landing pages.

A Little Bit of Context

At Miso, we’ve been working on making fun things for people watching TV, starting with check-ins, but experimenting with synchronized trivia, exclusive content, live Q & A. However, over the last three months, we’ve been focusing on ideas that connect friends around what they watch.

Why a Hackathon?

Entrepreneurs seem to be drinking heavily from the “Lean Startup” and “customer development” kool-aid.  One basic tenet is that product ideas/assumptions can be proven in days, not weeks or months.  Common tactics include applying MacGyver-like strategies to challenge product assumptions.  For example, create a splash page (this was our first attempt – http://playredzone.herokuapp.com) for a product before writing any code.

Generally speaking, it can be challenging to apply their methodologies consistently, but Hackathons certainly seem to intrinsically embody the spirit of Lean Startup.  Also, as most engineers will understand, creating something new from end to end is a very rewarding experience.

On to the Main Event

As we were deciding what to build for the Hackathon, we set the following constraints:

• Should be consistent with the “connect friends around what they watch” theme.
• Should be built on the Miso API (i.e., not as a feature in our main product).
• We have to want to use it.
• Must be finished in 3 days.

After some happy hour brainstorming, we came up with the basic motivation and mechanics behind Listify.

The need: People need help discovering what movies and TV shows to watch.

The underlying psychologies:

• People want to express themselves. Lists are a structured and easy way to do that.
• People take pride in their sense of taste. The band they “discovered”, the new (or old) indie film, are all worthy Facebook posts.
• People enjoy communities of like-minded people. They’re hard to create, but when it works, it’s unbeatable.

Implementation

Over the course of a few hours, we defined a minimum (and a reach) feature set, then designed and sketched an iPhone application, with accompanying web landing pages for the lists. We also spec’d the RESTful API that would support the iPhone app.

Over the next two (long) days, Josh and I built the iPhone app, while Nathan and Nico implemented the API (which utilized the Miso API, RABL, and Padrino) and the web splash pages. Nico served double duty as our art director.  Aside from some idle discussion that a little thought and investment in library development could make building iPhone apps that are fairly straightforward views of a RESTful API much faster, the implementation went without much incident.

While we’ve been happily using the app in beta for a week or so, we hope to release the app to the iPhone App Store after about a day of cleanup work, as soon as we find the time.

Conclusion

Overall, putting aside all the business rationalizations for doing a company hackathon, building something that we were only imagining a few days before was a ton of fun and a testament to why we’re engineers in the first place.  I’ll be sure to update this blog post when we actually submit to the App Store.