as time goes by : blabbing ’bout sinatra

Posted on | May 15, 2009 | No Comments

Justin Smestad invited me up to factory labs to talk about sinatra recently for one of their brown bag lunches. He recorded the talk from his laptop and I’m amazed you can see or hear me.

My slides aren’t sync’d up but they’re here.

Sinatra 0.9 – Corey Donohoe

hoptoad for rack

Posted on | May 15, 2009 | No Comments

So Tim and I were talking about how we needed hoptoad for our rack apps the other day. This rack hoptoad notifier should get you going for rack and sinatra. Raise issues on github if you run into problems.

If all you want to do is install the gem you can install it like this:
% sudo gem install rack_hoptoad

Hancock-Client-Rails : Using Sinatra/Rack Middleware in Rails

Posted on | March 23, 2009 | 1 Comment

Recently I’ve been writing a bunch about sinatra as middleware and one of the things you always hear people glorifying is how rack middleware can be dropped into any framework that’s built on rack. Since I’m trying to get people to try hancock out, or atleast learn something from it, I figured I should investigate how you actually use the hancock-client sinatra app inside off rails 2.3.2.

One of the examples I’ve seen in quite a few presentations on using middleware in rack has something along the lines of:

Unfortunately, I’ve yet to get a sample like this working with sinatra middleware. For some reason sinatra always throws 404 errors instead of allowing those requests to pass through to rails. Even if I set “disable :raise_errors” in my sinatra app, it’d still throw 404s. So after a little googling I discovered a wonderful write up over at the rails on rack page. I definitely learned a lot about how rails middleware works but I still couldn’t get my sinatra app functioning properly inside of rails.

I was kind of surprised to find out that there was very little documentation on using sinatra as middleware. Sinatra is awesome for this, how could I be the first person trying to do this? It turns out that I wasn’t. Raggi had explained how to do this to someone in #rack on freenode a few days earlier. I looked through his gist on how he did it. I didn’t understand it all but I hadn’t tried using the metal generator that rails introduced recently. So I gave that a try and ended up with a piece of rails metal that looks like this.

This actually worked perfectly for me. All I needed to do was make a subclass of Hancock::Client::Default and set the appropriate configuration options. Next I gave myself a little helper in application.rb to check whether I was logged in or not, you’ve probably done something like this in every other app you’ve ever used.

Then there’s a simple controller that should be protected by the SSO middleware.

That’s pretty much it, hopefully this will save you some time. One thing that I’ve noticed in trying to make this work with the larger frameworks is that I really want some sort of implicit before filter. I’d really like to have the redirect to ‘/sso/login’ happen anytime an unauthenticated request comes in, I’ll try to get that working in the next release.

I’ve created an example application that I’m hoping to keep up to date as hancock grows. There are git tags that match up to a hancock release number as well. If you want to see a rails client in action you can run “rake features” inside of the hancock-client-rails application. You’ll need to be on a mac due to the safariwatir constraint but it should drive your browser and complete a full handshake against the hancock sso sandbox.

Hancock-Client : Sinatra Middleware

Posted on | March 22, 2009 | 1 Comment

With the release of sinatra version 0.9.1, programmers have the option to write micro-apps that double as middleware in any other rack enabled application. The beauty in this is that you can write rack middleware without getting bogged down in the details of writing rack directly. Perhaps it’s your first time leveraging rack and you want to test the waters or perhaps you’re just wanting to slap a little bit of functionality onto someone else’s code; sinatra is emerging as a great way for folks to start really sinking their teeth into how rack functions.

I recently found myself waist-deep in rack. A lot of my co-workers at EY have been telling me for some time where rack made sense and where it didn’t. I didn’t really grasp it all. I understood it as a concept but there was nothing in my day to day work that made me say “Hey! I’m gonna use rack here!” At least until the other day. Jon Crosby rocked out with a great talk at MWRC and one of my friends at work had written a little sinatra app that we needed to merge into our Single Sign On(SSO) infrastructure. Tim took advantage of an existing rack openid library and made the necessary modifications to hook it in cleanly, he then went on vacation and left the gem abstraction to me. I’m really grateful for this because it really made me sit down and acquaint myself with sinatra. What I found was something pleasurable, elegant, and useful.

The hancock-client gem is an abstraction of our rack based SSO middleware in use at Engine Yard but modified to communicate with the hancock sso server. The gem provides a sinatra application that can be run as a standalone application or used as middleware in rails or merb. The application itself encompasses all of the logic required to negotiate the SSO protocol with a provider and populate session variables. You can pretty much expect that the sinatra app provided by the hancock-client gem will integrate well with a hancock provider of the same version.

So when I started trying to get the middleware going I decided on three things that were necessary for it:

• you need to be able to login
• you need to be able to logout
• after you login you should be greeted

This makes sense for the simplest possible consumer that does something useful. The login and logout actions are mapped to /sso/login and /sso/logout respectively. Does the “after you login you should be greeted” step really make sense for middleware though? I feel like the answer is no in this situation and it’s where the beauty of rack really comes to light. Since my code shares the same rack session that my framework code will share, maybe I should leave that greeting page up to the app that’s using the middleware.

The SSO middleware handles the authentication earlier in the stack than your framework and all you need to do is rely on a set of conventions that the middleware provides to the framework layer. In the case of hancock-client it sets the :user_id session variable. Depending on your middleware ordering this happens way before your framework is hit. The basic approach for the initial release of hancock-client went something like this, “only implement login and logout but provide examples of how you might use it in your framework of choice.”

What we created was a class that inherited from Sinatra::Default called Hancock::Client::Default. This class was created with the idea that an application developer would inherit from it and implement the greeting page at “/“. So in its simplest form you can implement a hancock-client app in the following fashion:

Notice how the middleware can be extended to support requests to / in sinatra with ease and you can simply let those requests pass through in frameworks that implement them at a higher level.

Notice the usage of the configuration option sso_url. You need to set this to the url of your SSO server. If you ever need to reference this in the markup it’s available as options.sso_url.

If you’re using this as middleware in other frameworks you’ll have to make sure that you set disable :raise_errors on your Hancock::Client::Default subclass. Otherwise you’ll get 404s and that’s kind of annoying. Checkout raggi’s gist too.

Believe it or not, the standalone sinatra apps are hella useful. I love when we want to try out some new 3rd party application that offers remote authentication and we can hook the two apps up with a simple sinatra app in no time. Seriously, do not disregard sinatra as a possible solution because you think you might need big feature X in framework Y. Maybe you shouldn’t extend that monolith app that’s getting more complex by the day. Maybe some micro-apps will be just what the doctor ordered. I might be crazy or living some sheltered developer lifestyle, but it is a pleasure to write sinatra code.

If you investigate the tests in hancock-client you’ll find that there aren’t many unit tests. I think there’s 2 and one is currently pending. If you checkout the cucumber tests though you’ll find safariwatir tests that drive your browser and test against the hancock sso sandbox. All you need to do is investigate the examples/dragon directory in hancock-client and run the following command.

% rackup -p 4567 config.ru

In another terminal you should be able to run the following:

% rake features

A browser should appear, you should see that you successfully signed up for an SSO account, and that your consumer application has access to a few session variables. I chose safariwatir in this case because it allowed me to blackbox test the appserver and I chose cucumber because I was pretty sure I could copy this functionality over to merb and rails as I created examples for those frameworks. If you’re on a mac, please try to run these specs and let me know if they work for you.

One thing that I’m really hoping to handle in the next release is the ability to make the SSO before filters in rails/merb a thing of the past. I actually need to get the merb example working… If any of this interests you, you’re welcome to fork my projects on github or hook me up with patches.

Hancock : A Single Sign On Server in Sinatra

Posted on | March 20, 2009 | 1 Comment

A while back I blogged about the flatirons openid provider and how it was inspired by something we were doing at Engine Yard. We needed a single sign on provider and chose to use OpenID as the sso protocol. Since this is heavily influenced by openid I will often use the word “provider” to describe an SSO server. I also use the word “consumer” to describe an SSO enabled application.

So we pretty much drew up a diagram of how openid works and removed the steps we felt were unnecessary(decision/acceptance steps). We then extended it by adding auto discovery of the openid url. Here’s a somewhat up to date diagram that shows off how things work. You’ll notice that the user-agent(browser) never specifies its identity url, it’s automatically provided from the SSO server.

I’m pleased to announce a new piece of software that might interest you if you need a single sign on solution for your projects. It’s called hancock and it’s available(like all good things) on github. Here’s a quick run down of what it offers you:

• a rubygem that provides all the functionality you need for an sso server
• an sso server as an extendable sinatra app, aka rack middleware
• a required whitelist for consumer application access
• configurable sreg response parameters to customize what information is exposed to clients
• can hook into anything that datamapper supports as a user/consumer store.
• trivial deployment as a rack application under passenger
• simple user signup with email verification
• application sessions don’t overlap by using .domain.com style cookies

This is all implemented in about 400 lines of ruby code using the sinatra framework. With the release of Sinatra 0.9.1.1 programmers can write sinatra applications in a more modular fashion. The killer feature is that these modular applications are actually rack applications. This allows for services to be written in sinatra that can be deployed as either a standalone rack application or used as middleware in frameworks like merb or rails. Hancock is an example standalone rack application. I’ll be blogging about sinatra as middleware in the days to come.

Hancock requires that you provide atleast three things to get it going. This is normally provided in your rackup file.

• database connection info, specifically the datamapper connect string
• mailer info
• a custom layout file

Hancock assumes that you’re going to provide the layout that gives your site its customized look and feel. The hancock gem itself provides all of the forms for authentication. If you’re going to be serving static assets like images, stylesheets, or javascript files you’ll also need to set the public attribute. Here’s what my rackup file looks like on my server.

That’s your whole app.

The Hancock gem provides a sinatra application as a class named Hancock::App. You should inherit from this class and implement a landing page at “/“. The mailer configuration functions a lot like the merb-mailer gem, if you’re having issues google around for merb-mailer help.

The Hancock gem also provides a class named Hancock::Consumer that represents a consumer application that is to be accepted by your hancock application. The :url attribute MUST match up to the return_to parameter in the openid negotiation. If you’re using the hancock-client gem then it should take care of this for you. The main thing to remember about this class is that if you ever receive the Forbidden message on your SSO provider you should double check that the host is allowed to access the provider.

Your user model is exposed as Hancock::User. Right now there is simple sign up through web forms. If you want to test the provider without setting up the full email configuration then you should run your application in development mode. When run under development mode the registration url that would normally be sent via email is embedded as a comment. I found that it made it really easy to test.

The Hancock gem doesn’t have perfect tests, but I’ll be damned if we aren’t shooting for it. You’ll find both unit and acceptance tests done in rspec, rack-test, webrat, and cucumber. Even if you’re not interested in an SSO server the tests might help clarify a few things if you’re a testing junkie.

As I mentioned earlier, the killer feature of sinatra 0.9.1.1 is that sinatra apps are modular rack applications. While hancock is a pretty cool example of the power and simplicity of sinatra; its use as middleware in other frameworks is where I expect to see sinatra shine for the foreseeable future. Even if you’re stuck on a never ending rails project with no chance of innovation in sight, go learn about rack. I think you’ll be surprised where you can start leveraging it. Besides, writing sinatra is fun.

So what good is a simple SSO server if you don’t have a super simple client library? I bet if we did it as a simple sinatra app then we could turn it into middleware. Then we could hook our rails and merb apps up. Yeah, that’d be sweet.

I’m gonna follow this up in the next few days with something covering the hancock-client gem. This gem provides a sinatra application that can be run standalone or as middleware in other apps. I’ll prolly follow that up with two short articles on “how you get your middleware running in (rails|merb).”

Considerations for Creating a Test Driven Ruby Project

Posted on | February 1, 2009 | 6 Comments

I’d originally wanted to do something like “how our group at EY specs things” but I feel like a lot of these ideas extend beyond the merb/rspec/datamapper bubble we currently develop in.  When I went through the Integrity source code a couple weeks ago I realized they were doing the same things, but with sinatra, context, and matchy. I found it really easy to understand the integrity test suite because it was setup in a wonderfully sane manner. I figured I’d share a few thoughts on it since I’d like to see more projects testing things like this. There’s nothing earth shattering here, In reality though, few projects I come across embrace developers to collaborate and enhance. If you consider these things while creating your project, I promise, you’ll be in a much better state to collaborate.

Here’s a few things that stick out in my mind:

• You should have a README that tells another developer how to get up and running
• You should really be testing, even if it’s after the fact
• Name Your Tests/Specs after what it does
• Be able to toggle mocks and integration tests easily
• Use GitHub
• How will this commit affect Continuous Integration?

You should have a README that tells another developer how to get up and running

If some random person needs to add a feature or fix a bug in your software, they’re prolly going to have to do some setup.  Maintain this README.  The easier it is to get a system bootstrapped to start developing and enhancing the software the better. The old timers can send you a patch via email, the new schoolers send you pull requests. Let other people do some of the work for you. Even Dr Nic thinks it’s a good idea!

You should really be testing, even if it’s after the fact

I don’t care how fucking smart you are, software maintenance isn’t easy. I’ve seen folks argue that unit testing is fail, I’m starting to agree when it comes to the web. I’m finding that if you’re actually writing tests as you’re completing user stories you naturally exercise your models. With a code coverage tool like rcov you can really tell which portions of your models are being exercised fully. If you know your stuff then you know that rcov coverage doesn’t really mean ALL that much, but it helps. You can then use a mutator like heckle or boo hiss to cover your ass for the rest of your cases. We normally employ the mutators after we’re happy with the state of the software, it helps us prepare for handling weird errors in a more graceful manor. You don’t have to go so far as to running mutators on your code, but you should really really be testing. If you don’t have time to test code, perhaps it’s time for you to TAKE THE TIME to learn how to do it. Your co-workers and friends will thank you for it, I promise. Even if you develop software by changing code and refreshing your browser or running your script over and over, you owe it to your collaborators to let them know what portions of the code are important and what will cause major breakage if it changes. By not testing you’re saying “I really don’t care whether or not this project lives a long, happy life.”

Name Your Tests/Specs after what it does

At the first railsconf I remember srbaker talking about rspec and how much he hated the way rails generators generated test files. He told me something along the lines of “when I need to create a new spec I make it based on what it’s for.” I didn’t really understand him at the time, I was still learning how to test via the rails framework.

In the images above you can easily identify what functionality has tests. It annoys me to death when I open up spec/controllers/ and find specs named after classes. They might be empty or they might be a giant file testing all of the functionality of the class in isolation, both cases suck. I’m becoming more and more fond of just creating a spec file for a user story as I implement it. In the web space I think you should investigate webrat, you can use it with merb, sinatra, or rails. You’re basically writing acceptance tests that’s like a fake browser request with persistent sessions. Name your acceptance tests according to what they validate and you’ll be in business.
 

Be able to toggle mocks and integration tests easily

We recently did a bunch of stuff with braintree for some payment processing. They provide a pretty awesome test environment for you to test against. I wrote a whole bunch of specs that faked browser requests to the API server and I found their docs to be incorrect. Their examples worked but once I got down to requiring CVVs and address verification, the response hashes came back inconsistently. By really testing the requests/responses I was able to dm this really awesome ruby developer on twitter and resolve the issue in a day or two, long before we ever went live. If we’d generated mocks based on the documentation alone we would’ve gone live with a broken system. You could argue that it’s braintree’s fault and they should have correct documentation, but I’m not a big fan of finger pointing. Finger pointing doesn’t accomplish a damn thing when you just launched a site and it’s not completely functional. We’ve started shooting for something along the lines of “only mock things out if we can toggle a real integration test.” Some of our apps could really use some mocking help, but the speed hit isn’t really that big of a deal when you know it’s really running against the remote service. Of course my friend Tim has some ideas about mocks too.

Use GitHub

I use git, my homies at GitHub get it. It’s all I’ve been using since February of 2008. I really can’t say enough good things about github, it really simplifies things when collaborating on software.

How will this commit affect Continuous Integration?

You do use continuous integration, right? Continuous Integration is just a remote machine that runs your test suite every time someone pushes code. I use Integrity, some of my friends use cc.rb. Integrity works really frackin’ well with github, all of my code lives there so it makes it that much more attractive. If I push some code, how will my CI task handle installing any new dependencies I introduced? Will my co-workers want to punch me in the face because half of their day was wasted tracking down some shit I committed? How well do these changes work outside of my machine? This is stuff that you really want to consider when you collaborate.

Collaboration is King

Collaboration keeps bubbling to the surface here. Perhaps I should change the title. These suggestions have little merit if you’re not interested in collaborating on software. If you’re not interested in collaborating on software though, you’re going the way of the dodo. Think of all the awesome techniques you picked up from working with other people. Think of all the projects you tried to use or hack on but couldn’t because you couldn’t get it working. It’s up to you to make your projects easy to collaborate on.

So, what’d I forget to mention?

Merb and Rails Sitting in a Tree

Posted on | December 23, 2008 | 2 Comments

• Ruby on Rails: Merb
• Rails and Merb Merge

Who’d have thought? In case you didn’t know the two powerhouses in Ruby Frameworks are going to start working together to make the world a better place. This brings up a lot of emotions for me and gives me hope for the community.

4 years ago I spent my christmas holidays rewriting my php based weblog in rails, this new and crazy framework where you wrote less and did more. At the time I was a Solaris administrator in Georgia and the programming job landscape was primarily Java, C#, and PHP. I didn’t really want to write in those languages so I stuck to something I did enjoy, UNIX. Over that holiday season though, I really fell for the Ruby language. Not just the language but the community that surrounded rails at this time was nothing short of magical. This was my first introduction to entrepreneurs in the web space and it seemed like every new person I met in this community was just awesome at life(and hacking).

So rails took off and a lot of us got busy. A lot of us wrote software that never saw the light of day, others wrote stuff that made the world take rails seriously. I was lucky enough to be the first technical hire at Engine Yard and the growth that we’ve seen is inspirational. Not only did we target the “premium rails hosting” market and excel at meeting and exceeding our customer’s expectations but we made it clear that we love the Ruby community. We hired Evan Phoenix and Yehuda Katz to push the ruby interpreter and the merb framework respectively despite the fact that there was no immediate financial benefit.

So I’m sitting here avoiding work by writing this blog post, but I’m just so freaking excited by this merge. I’m delighted to know that our community can still surprise me, that folks can work together, and the future of ruby in the websphere is looking good.

Much respect to the merb and rails teams for making this happen. Thanks, a lot.

Gem Bundling – Where Merb Needs HELP

Posted on | December 10, 2008 | 4 Comments

UPDATE:

This has been more or less resolved in merb 1.0.6. It’s not too painful to upgrade your apps to it. Here’s the flatirons upgrade commit on github. The new bundling can also be used with older versions of merb. Check the release notes on Yehuda’s Blog

Have you ever seen this error?

This happens all the time if you follow merb development for a month or two.  The current state of ’supported’ merb gem bundling is weak sauce.

Here’s three areas I consistently waste time on because I chose merb+dm

• Ease of collaboration.  I stopped counting the number of hours I’ve lost trying to get a merb app running on a co-workers machine or vice versa.  Once you spend 2-3 hours swearing at version mismatch errors it wears on your drive to believe the software choice is a good decision.
• Ease of deployment. I bet 9 out of 10 merb developers develop code on some sort of Mac and deploy on a Linux box.  ”I stick it in vendor”™ fails miserably when it comes to binary gems.  Think mongrel, hpricot, nokogiri, libxml-ruby, thin, etc; all of these need to be compiled on atleast the same OS.  As a developer I should not care about this issue.
• Ease of Maintenance.  If I need to install a new gem or update an existing one it shouldn’t be a pain in the ass.  Period.

Just to be clear, when I talk about merb bundling from here on out, I’m talking specifically about thor merb:gem:redeploy.  Our group doesn’t use this approach in any of the apps we are in charge of managing at EY currently. Here’s why.

Quick Run Down

This is how you might use merb bundling.

 

You’re basically given a gems/ folder with a cache and specifications folder inside of it. Thor uses the files in these folders to install the gems on fresh checkouts, or your co-workers machine. Everything you need, and I mean EVERYTHING, is in that gems/cache folder. “Rake? Rspec? All 19 or so merb gems?” You bet, they’re all in there. “What if you need to do a minor version bump because merb did a point release?” Upgrade em all!

EASE OF COLLABORATION FAIL

Every time I collaborate with someone on a project where the gems are bundled via merb I feel like we waste 1-2 hours. If we need to bump a gem version number or add a new dependency, it should be trivial and routinely isn’t.  One of my coworkers recently told me “oh it’s so easy, thor merb:gem:redeploy or thor merb:gem:rebuild. Something like that.”  If you can’t remember the command,  it’s not easy.  Every rails user remembers script/server and script/generate.  You might also hear “Oh! Maybe your merb.thor is outta date!  They changed some namespace stuff, it’ll work if you just run this!”

…..  You’ve gotta be kidding me, bonus points if you know what the -L is for.  At least it’s pretty well covered on the thor wiki page.  If your app is pre 1.0, you might as well generate a new project. It’s like you’re being punished for being an early adopter..  By the way, does anyone know why the seattle.rb projects always throw errors?

EASE OF DEPLOYMENT FAIL

When you go to deploy your application, you need to make sure those gems are up to date! So you run thor merb:gem:redeploy on each deployment. Why does everything reinstall everytime though? In our production environments we have a gems directory that persists between deployments and only installs gems as needed, we symlink this directory into our Merb.root and our deployments are quite fast. We also don’t have to check our gems into our SCM at all, but that’s a different story.

If you’re into continuous integration you should know that without proper .gitignore files it’s really easy to get your CI system into a state where it stops pulling changes.  If your rebuilt gems in gems/ have minor changes, git will exit with a non-zero status and most CI systems(integrity,cc.rb) abort; then they run your OLD CODE. Since the specifications directory must be checked into gems/ for thor bundling to work, you can’t even run CI against your app without hacking your CI software…

EASE OF MAINTENANCE FAIL

Today I had to help update a co-worker upgrade to a new version of an internal gem. Here’s how we did it, how awesome is this?

Updating a gem should not require that much SCM specific interaction. Checking gems into my SCM makes me want to cry. I think it’d be really cool to just change a config file and make this happen. That’s not too much to ask, is it? I definitely see the win involved with “everything you need to get this specific commit up and running is contained in the app” but there has gotta be a better way. Is merb 1.0.1 gonna go away from the gem servers sometime soon? Does rubyforge go down so often that checking EVERY gem in is a huge win? Do you really work offline from a fresh repo frequently?

If you have some CRAZY AWESOME CODE that you can’t open source, setup a gem server. This might be something that you spin up for the lifetime of your deployment process and gem install against, or something behind basic auth that enables multiple developers to collaborate against the exact same versions of private gems. I like reproducibility as much as the next guy, but checking every gem my app depends on into my repo frustrates me.

A Temporary Alternative

Earlier this year, around the merb 0.9.3-0.9.9 days, we used to actually do a git pull and repackaged everything into the system gems on each deployment.  As we started pushing code more often we realized how silly this approach was so we decided to stick with one working set of system gems. Despite our best efforts we were still plagued by “already initialized” errors whenever a new version of dm or merb came out. This is how rubundler was born. Rubundler is:

• Not a long term replacement for thor bundling, it’s a band-aid.
• Does not require checking gems into your SCM at all
• Can go from no system gems to a working merb app if you have a net connection
• Requires people to familiarize themselves with the gems they actually use
• Works with continuous integration servers
• Makes for swift deployments
• Gracefully handles version bumps in gems you depend on
• Makes collaboration simple
• Works with more than just merb, we test some of our internal gems with it

As I told Yehuda yesterday when I was asking for thor bundling help in #merb. I’m not trying to peddle rubundler. Rubundler isn’t the right solution, but it sucks less than the current ’supported’ solution. I think people should be aware of an alternative and I think the community will be healthier if folks stand up and say “Hey! This is rubbish!” Stop taking every assumption your framework authors consider ‘correct’ and see if it’s ‘correct’ for you.

I did get confirmation on IRC last night that Yehuda is working on rewriting this stuff. Hopefully it’ll be more ‘correct’ for the general community in the coming weeks. In the meantime, if you waste an hour or so messing with gems and merb/dm, rubundler might be a good temp fix for you too.

Merb 1.0 Controller Testing

Posted on | November 29, 2008 | 15 Comments

This was originally going to be part of “testing flatirons” but it got kinda long and I felt it’d be most useful to community if I made it a standalone article. I’ve had the pleasure of experimenting with everything I cover in this write-up, and the state of merb’s testing environment is getting better. Since merb is at 1.0 these days this rundown should be valid for at least a few more months. I’m gonna run through what I know about how you test in merb, and in the next few days I’ll dissect flatirons. Here’s what I’ll be covering in this write up:

Request
• dispatch_to
• get/post/put/delete
• request

Requests with Authentication
• dispatch_to / http verb tests
• requests w/ given blocks

RSpec Matchers FTW
• have_selector
• have_xpath
• Roll Your Own

1.1 dispatch_to

This method allows you to unit test controllers. It also makes your co-workers cry, swear to kill you, and rewrite your code.

It seems harmless enough. dispatch_to takes 4 parameters and an optional block. The 3rd parameter is the actual params hash that the action will see. The 4th is your http environment and how you modify things like HTTP_ACCEPT to change content type. When you call this method it instantiates the Class and calls the Action with your http params, you bypass routing and have the option to stub/mock to your heart’s content. The response from dispatch_to returns an instance of the class, in this case it’s the Sessions controller. The response also has a few instance methods that might be useful to check in your testing:

• #status – the http response code
• #body – the document returned from the action
• #headers – the http headers returned
• #session – the session for the request

In my experience dispatch_to leads to really brittle tests when things start to get a little more complex. While I’m a HUGE fan of rr, I’m finding that mocks are mostly complicating my day to day work. I wanna see my code work from beginning to end, even if it means my test suite takes a little longer. The merb-core team advises against using dispatch_to. If you’re going to be using merb for the foreseeable future, do not get in the habit of using this. Seriously.

1.2 get/post/put/delete

The standard HTTP verbs are available as request helpers in your test environment too. It’s kinda like dispatch_to but it actually goes through the router. Insted of giving a Class and an Action to call, you give the same path you would request in a browser. The optional 2nd and 3rd parameters to this function are the request parameters and http environment respectively. This also has a block syntax that you’ll see later in this tutorial. Here’s what the above request looks like using the HTTP verb helpers.

The response from get / post / put / delete has the same instance methods available that dispatch_to has since they both return the actual controller that was executed.

This is cooler than dispatch_to because it goes through the router and we can make sure that our pretty “/login” url is properly configured in the router.

1.3 request

request is short and sweet. It’s a programmatic browser experience of sorts. At first glance it doesn’t seem much better than get but it actually preserves states between a series of requests. We’ll see why this kicks ass later in the tutorial. Here’s how you display the login form with the request helper.

The response from request is a struct that provides the following instance methods. In general I try not to call these instance methods, it tends to be more clean when you write matchers against the response object itself.

• #status – the http response code
• #body – the document returned from the action
• #headers – the http headers returned
• #url – the url that was requested
• #original_env – i think this is how stuff persists between requests. I’ve never actually called this for anything.

2.0 Testing with Authentication

2.1 merb-auth and all that jazz

So you’re supposed to be using merb-auth if you need authentication in your app. So how do you test it?

2.2 dispatch_to and HTTP verbs with Authentication

Both dispatch_to and get can take an optional block that allows you to spec and mock things on the controller. If you need to stub out authentication both helpers are easily spec’d like this.

While adding the mocks and stubs might yield some immediate results, it tends to be a maintenance nightmare. You either end up with a bunch of duplicated stuff in each spec or you end up wrapping these methods with things like employee_get/employee_put etc.

Even with this bit of abstraction it becomes a royal PITA to test real use cases. Since each spec tests the controller action in isolation you open yourself up to the possibility of having bugs in your tests that are directly related to keeping your mocks/specs synced up with your real code. “Does the signup process really work from start to finish?” “I don’t really know what that chunk of code does, but I know it returns true or false.” It becomes very easy to bypass chunks of code in certain situations without really understanding how things integrate. The request helper shines bright in this situation.

2.3 requests with given blocks

The request helper allows you to preserve the normal browser experience by making successive calls to request and preserving the user session. So here’s how you authenticate against merb-auth:

It sucks that you had to explicitly call that request to “/login” but it’s kinda nice to be able to comprehend the user experience that led up to the point where the code in question was tested. Someone logged in and then they could view their account settings.

Either Merb or RSpec provide you with this given block to setup pre conditions that the spec relies on. They’re kinda like shared_examples but if you’re using request you can build entire use cases up and attach them to describe blocks. Your session persists between requests, instant WIN! I added this to the bottom of my spec/spec_helper.rb to allow for authenticated requests.

After that your test code starts to look like this

If you’re super awesome you can pass “cookie jars” along with your requests to restore certain states. I think they’ll be really useful in situations where specs start taking forever because of the amount of setup required to get to a certain state. Thus far our specs aren’t taking a long time so I guess I’ll cross this bridge when our spec suite starts taking forever.

One other thing to keep in mind is that your HTTP_HOST environmental variable is http://example.org. So don’t be surprised if you see this popping up in your tests.

3. RSpec Matchers

Remember how both the dispatch_to and request responses have a few instance methods that are useful in testing? Notice that we didn’t explicitly call them in any of the spec examples above? It’s because we’re using the rspec matchers that merb provides. Not only should you be using the ones that merb provides, you should be writing custom ones for your app. Here’s the matchers that I use from merb on a daily basis.

3.1 HTTP Response Checks

• be_successful This basically checks that response.status was 200. It was a successful request.
• redirect This checks that response.status was 302. It’s your normal HTTP redirect code.
• redirect_to This not only checks that response.status was 302 but it takes a string for the path portion of the redirection. If you need to check query parameters you’re going to have to write your own matcher. We have one in one of our apps now, redirect_to_with_params that does exactly this, we’re hoping to give it back to the merb community shortly.

Markup Validation

have_selector is what you should be using to validate HTML markup. It allows you to specify to identify the presence of certain markup. If you’re into that thing, it’s great practice for selectors in js too. Let’s say we want to check for the presence of the ‘Hello World’ string in the following example.

In your spec you’d have something like the following:

It’s pretty straightforward, find the div with an h2 that has the dom id of ‘hello’ and contains the string ‘Hello World’. Now is a good time to review the example specs above, hopefully you can visualize the markup that those specs require now.

have_xpath is the slightly older cousin of have_selector. Instead of CSS 3 Selectors you specify an xpath that should be present in the response document. I’ve gotten in the habit of only using this for XML documents. Let’s see the Hello World example done with xpath.

3.3 Roll Your Own

Rolling your own RSpec matchers is the super pimp stuff. You can cut down on duplication in your tests by coming up with an expressive name for certain situations, their responses, and the markup that’s returned. Here’s a little matcher we wrote in flatirons that handles unauthenticated requests returned by merb-auth.

We put this matcher into our spec/spec_helper.rb file and included it into our rspec environment with the following code.

So that’s about it for the merb controller testing intro. Now that we’ve discussed the basics of testing we can get back into flatirons and talk about how things are tested in that app.

I’m sure I made some errors in here, please offer up corrections in the comments and I’ll try to work the fixes into the article soonish.

Flatirons : A Merb OpenID Provider Part 1

Posted on | November 24, 2008 | No Comments

What are the flatirons you say? They’re a rock formation near my house in Colorado, they look like this in the winter.

It’s also the name of the open source merb 1.0 based OpenID Provider that’s available on github. We recently started rolling out a customized OpenID Provider for all of our internal apps. Flatirons was how I familiarized myself with the OpenID spec.

The Low Down

I’m going to break this into two parts, one on the OpenID Provider and one on the way we’re testing in merb 1.0. There appears to be a lack of examples on writing specs in the merb 1.0 world and I assure you it rocks way more than it used to. This part will give you a brief overview of what’s really involved in the OpenID protocol. Part 2 will cover testing the spec that’s described here.

Paint a Perfect Picture

It’s not perfect, but it should give you an idea about the request/response life cycle for an OpenID Authentication. Keep in mind that User-Agent in these diagrams is normally a User’s browser. A Consumer is your new micro-app that doesn’t even have a concept of user passwords. The Provider is where flatirons comes into play, it’s the OpenID to Merb layer that allows you to take advantage of merb-auth’s strategies. The Auth Backend is trivial to customize, consult the merb-auth docs for more info. Check it:

Identity Discovery

Identity Discovery is the first part of the User-Agent authentication. The Consumer can request one of three options(the spec) but normally the Provider sends back a Yadis document that lets the Consumer know where to go to start talking that crypto shit with the Provider. This is normally in the form of an xrds+xml response.

Associate

This is where the magic starts. “An association between the Relying Party and the OpenID Provider establishes a shared secret between them, this is used to verify subsequent protocol messages and reduce round trips.”(the spec) The Consumer and the Provider negotiate some Diffie-Hellman keys and use these for communicating for the rest of the auth life-cycle.

Check ID Setup

Check ID Setup covers a few things, there’s a few modes that you should really investigate in the documentation if you’re curious. Suffice it to say there’s two interesting modes that are a part of the id setup, checkid_setup and immediate. Immediate is used less frequently and it’s designed for authenticating in a manner that isn’t interactive(think ajax logins). The checkid_setup mode is way more common, it involves normal http requests, redirects, and pages where you enter your username and password.

Acceptance – “Do you trust me?”

Acceptance is a form on the Provider displaying the specific information it’s going to send to the Consumer. The User-Agent specifies whether or not this information is OK to send to the Consumer. Normally users just click OK here but this is where a full fledged Provider allows you to pick one of many identities you have stored there.

Identity Result

Once the User-Agent accepts the Provider as a trusted source, an identity result is sent back to the Consumer from the Provider. The Consumer can then take this response and do whatever local bootstrapping they need to for the user. OpenID can send back more than just the identity URL, but flatirons doesn’t take advantage of these spec extensions yet.

WTF, Why do I even care?

Since this is based on merb-auth, you have all of the strategies it provides at your disposal. Flatirons defaults to the salted password setup, but you could easily hook this into whatever authentication backend that you need. If you’re considering Ruby applications and need single sign on then this is a good starting point regardless of your user store. In a trusted environment you can make this even more simple, whitelisting can simplify the request/response life-cycle by removing the Acceptance step. Depending on your user store you can mask the interaction with the actual identity URL. We’re able to mask the concept of the identity url a User-Agent uses and we just ask them for their email address. Ours users don’t even know what OpenID is, it rocks.

Where’s the Code

It’s on github of course.. In the next couple of days I’ll dissect the specs that validate the expectations defined above.

keep looking »

About

I'm Corey Donohoe. I write software at Engine Yard.

Photos

View more photos >

On Twitter

• @bbrinck @drawohara not even precipitating up north near sanitas. :) 8 hours ago
• @dastels strange, it's working fine here w/ my bluetooth keyboard. It lost the setting in the upgrade but worked after i reset it. 8 hours ago
• thinking today is a good day to play with the mpd and ableton live 14 hours ago
• @chacon congrats! 14 hours ago
• @igniteboulder definitely impressed. hopefully we can bring more of a ruckus next iteration! 2009/07/02
• Heading to @igniteboulder. 2009/07/02

Search

Pages

• About

Elsewhere

• Github
• Last.fm
• Flickr
• Twitter
• Tumblr
• Facebook
• LinkedIn

Inspiration

• Juxtapoz
• Alonetone