photo credit: mikecogh

This year started off with a foray into Rails, an experience I won’t be rushing to complete. Most of my time was spent building a small application during the Christmas break in 2010, but in 2011 I moved that site into production and wrote a little bit about separating production and development values (a feat I repeated for the Play! framework, which I actually like, later in the year). I think the only thing I really like from Rails, and this is a bit of a stretch, is the database migrations.

Having moved entirely over to a LAMP platform professionally, and getting good at the security nuances plus everything else, I reminisced a little about some of the creature comforts I missed in C#, like operators for default null variables. But when I discovered Time Machine on my Mac, there was no going back – until the company switched directions and I got thrown back into .NET development.

That’s right – back to .NET, and deep into the Windows Azure cloud. I dealt with things like figuring out which is better – table storage or SQL Azure, and figuring out the nuances of their multiple SLAs, and how to ensure we actually have Azure Compute instances on-line when it hits the fan. At this point I have a pretty good handle on Azure’s strengths and weaknesses and my overall impression of the platform is very positive. If I continue building sites on the Microsoft stack, I would definitely continue to use Azure – it seems more expensive than other options at first glance, but it has some serious computing power behind it and takes the majority of administration headaches off of my plate. It really has enabled me to, for the most part, just focus on development whereas I was spending an increasing amount of my work day on system administration issues when supporting the LAMP platform.

Scalability and high availability have been on my mind a lot, and I’ve been looking into some more ‘off-beat’ database solutions like Drizzle for my transactional needs, as well as speeding up existing deployments by moving as much as possible into RAM. There’s always a battle between changing the way we work to take advantage of the new paradigm or changing our existing configuration to get some more life out of it.

The whole cloud computing buzz feels tired but has enabled a whole new class of online business. If you have a lot of commodity hardware you can achieve, very cheaply, feats that were only possible with an expensive dedicated network just a few years ago. Sure, it adds a lot of new choke points you will need good people to help sort through, which is giving rise to a whole new sub-category of programmer specialization to make hiring in 2012 even more challenging.

There is a downside to all the cloud computing, though, as we learned during the high profile Amazon failures – backups are important. This includes geographically-redundant systems that most organizations don’t have the experience to deal effectively with just yet. Even so, the biggest lesson I learned was never let your server run into swap space or your performance will nose-dive. The growth of this site even prompted me to move more of the site into memory which prevented me from needing to spend a lot of money upgrading my infrastructure.

Social media continues to grow, with companies realizing they can’t control its effect on their business in traditional ways and less-than-useless cons ruining it for everyone by selling CEOs on cheap gimmicks.

Since my third child was born in February, I’ve definitely taken some time to sotp and reflect on what I want to work on, why I want to keep working, and what the next steps are career-wise and life-wise. I want to provide the best that I can for my family and 2012 is going to see a radical course change as I start to shift gears and begin building something that will really last, even outlast me. When will my website start paying my bills? I don’t expect it will.

I learned a lot by running dozens (over a hundred?) job interviews in the past two years. Ignoring the old adage that you shouldn’t judge a book by it’s cover, I learned that you can tell with pretty good accuracy whether or not someone will be a good match for your company within the first five minutes of an interview. I’m less interested in hiring people with domain knowledge than I am in surrounding myself with the most intelligent developers I can find – one is a skill that can be taught, the other is an aptitude candidates need to bring to the table. Really, when it comes down to it, what I really want is for people I hire to stand up for themselves (since they are adults) and make me look good by being awesome at what they do.

I also learned a lot by being responsible for some very large projects; things like the importance of continuous integration.

What’s next in 2012? Look for mobile device use to continue growth – every developer who plans to stay employed needs to know something about mobile development, because it’s going to be ubiquitous with regular desktop programming very soon. Now that version 0.6 has been released with Windows support is Node.js ready for prime-time? I had the opportunity to play with it a lot over the past few month – look for a book early in the new year co-authored by yours truly.

No related posts.

This is the old architecture I had deployed. Apache – the Internet’s workhorse – to perform all of the PHP processing, with Nginx as a reverse proxy, passing dynamic (PHP) requests to Apache but serving static files directly to give Apache a break and cut down its footprint.

I optimized MySQL with a large buffer, so it serves the vast majority of queries directly from memory.

The two places that hit the filesystem are Nginx and PHP – Nginx for static files (as mentioned, to take the load of Apache which would spin up a new instance for each file it serves) and PHP for session data (this is PHP’s default setting).

This is the new setup, and is very similar to the old with two key differences: Apache is gone, and Memcached is now in the mix.

As site traffic increased I noticed that Apache was using up bigger and bigger chunks of the system RAM compared to all of the other processes. I could pare this down by putting further restrictions on the number of child processes, decrease the number of connections before recycling, and limiting the maximum memory for each process, but that seemed like a lot of work when I already had one foot into a more scalable solution.

Taking advantage of the new(ish) since PHP 5.3.3 FastCGI Process Manager (FPM), I updated Nginx to send PHP traffic directly to PHP without using Apache as a middleman. The default settings were too generous for my fairly weak server, and the memory usage shot up. But by tweaking it down to 3 processes recycling after 500 requests, I’m now using half the physical memory as I was with Apache.

Previously I wrote about using Memcached as a PHP Session Handler and that’s exactly what I did here. Now the Filesystem is only hit for static files and the first run of PHP scripts – everything else is served from memory.

It may seem a little counter-intuitive that I cut memory consumption by moving more services into memory, but the trade-off improvements realized by hitting the disk less means that responses are sent out 30% faster, meaning I can fit more traffic onto the same machine and expect the same responsiveness on the web site.

One thing I could do to improve this even more would be to put Varnish in front of Nginx and serve all static content – including rendered PHP – from memory, which would give some seriously (<100ms) fast performance on read-only WordPress pages when users are not logged in. I may do that if traffic continues to rise, but for the moment the combination of Nginx's static file speed with offloading most of my site’s static files to a content delivery network (CDN) is giving me the performance I want to see.

Related posts:

1. Setting up WordPress with nginx and FastCGI
2. Performance Tuning Apache

Although it is still an extremely popular database, Oracle’s presence in the MySQL world has put a chill over business users considering using the product as the backbone of their data solutions. Other databases with similar purposes exists but none have the deep community boasted by PostgreSQL.

Related posts:

1. Using Memcache with MySQL
2. Drizzle – MySQL for the Cloud
3. Memcache as MySQL’s Hero

I think we have a hard time letting go of our memories. So when a friend, a boss, a client asks us for an unreasonable favour, we are inclined to bend over backwards to deliver. Maybe we’re afraid that if we say no, our value will somehow be dimished.

I see “no” as an invitation. Now we know where the limit is, so we can work to get to a comfortable “yes”.

I can’t help feeling that I once read an article with a similar message to this; if anyone knows what that is, do send me a link!

No related posts.

Migrations
This can be done manually or automatically. As database changes are made by developers, scripts are generated which can be run against a master database to bring it in line with the developer’s version. The most basic way to accomplish this is by writing a script manually, but frameworks like Django and Rails have built-in migration tools which manage this process. Rails in particular allows developers to move back and forth between snapshots of database schemas.

Evolutions
Evolutionary systems detect database schema changes against program code definitions. As of April 2011, Play Framework supports Evolutions.

Schema Versions
Microsoft SQL Server supports schema versions; wherein the underlying data remains the same, but multiple versions of the database schema rest on top and can be accessed simultaneously. This keeps older versions of the application or supporting clients working with the existing data set.

Keep Tracking…
Managing database changes can be a challenge for organizations of any size. The correct tool depends on a wide range of factors including your project size, number of team members, release schedule.

What kind of tools and processes do you use to manage database changes?

No related posts.

Read vs Write
Traditional relational databases place the burden of computation on read operations. In mainframe environments with powerful servers and relatively few users, this made sense. Database normalization prevents redundancy, and data can be joined together when needed to produce the desired results.

In a web application with 1,000,000 users, the normalized transactional model does not perform. Generally speaking it is way faster to make two queries to a small subset of data rather than attempt expensive joins in a client-facing web site.

Enter memcache: by storing the result of our SQL queries in memory, we improve the speed of subsequent requests by pulling the data from memory as well as avoiding a hit to the database entirely, freeing it to process urgent or real-time requests.

Anatomy of an SQL Query
When we run an SQL query, we are actually asking the server to perform a lot of work:

1. Break down the query into object references: The DBMS needs to understand which tables, columns, and filters you are using by tokenizing your SQL (by breaking out names from the keywords like SELECT, FROM, WHERE).
2. Identify which indexes (if any) are most relevant to the data: This is harder with more complex queries which must be broken down or which depend on outer tables for their values.
3. Read the source tables from the hard drive: Most DBMS implementations include some kind of memory caching which partially avoids this expensive read step, but some disk IO is a normal part of operation
4. Join Columns: If we specify a join, especially a LEFT or RIGHT join, the DBMS has to create a pseudo table from the joined sources in memory before it can do any additional processing.
5. Sub-selects: Any sub-select statements need to be processed. Depending on how the statement was written, this needs to be done for every row returned in the result set.
6. Filter and sort: Anything in the ‘WHERE’ clause needs to be filtered out of the result set. This is where we are going to start seeing performance improvements by narrowing our result set.
7. Aggregation: Once the database has its final result set it can do all of the aggregation we ask of it, both calculations and grouping

As we can imagine, this can be a time-consuming process. If it is repeated thousands of times in a short period, we will see significant performance loss.

Anatomy of a Cache Request
By contrast, when we perform a request for data from a cache like memcache, we do this:

1. Check the index for the presence of the supplied key
2. If a result exists, return it

In the case of memcache, this happens entirely in memory with no hit to the database whatsoever, resulting in a blindingly fast result set.

Speed Over Persistance
The reason “>memcache and MySQL work well as a pair is because they provide the tools needed to have reliable, persistent transactional storage (through MySQL) along with lightning fast data retrieval (through Memcache) especially for rarely-changing results.

Related posts:

1. Memcache as MySQL’s Hero
2. Drizzle – MySQL for the Cloud
3. MySQL Pulled From Mac OS X Lion

Why do I like this framework so much? Put simply, it is an elegant solution for nearly every problem I’ve ever run into in developing websites, both single-server and multi-server applications. Don’t take my word for it, see for yourself.

The Goodness of Java
To my mind, Java has the edge over more common scripting languages (like PHP) because it is compiled (fast) and statically typed (reliable). In the past, using compiled languages on the web was only possible if you were using ASP.NET or willing to put up with the hassles of existing Java frameworks and servers.

Play’s first innovation comes from wrapping the Java runtime inside a Python web server; using a Play application is as easy as running a command line script and connecting with your web browser. Play’s second innovation is its just-in-time compilation and display of error messages; if you make a mistake you will know in the amount of time it takes to hit refresh on your web browser.

Since it IS java, programmers can use libraries they have built for other applications or sourced from other vendors and plug directly into their code. This is one of the advantages Microsoft has had going for it and it is good to see it implemented so nicely in the open source world.

The Ease of Rails
Love it or hate it, Ruby on Rails has had an affect on the entire web world and its reach is definitely felt in the Play framework. Everything from the routing to JPA integration has that minimal-configuration design that is so prevalent in the Ruby world. Play has the edge though, due to Java annotations and the extra control you get as a developer.

Baked-in Unit Testing
Admittedly this is the first thing that drew me to the Play framework. Unit testing has to be one of the most important aspects of good programming; in fact, if your code is not covered by unit tests, I argue it is incomplete. Play has terrific support for unit testing, functional testing and selenium-based web testing. In version 1.1, Play added a headless web testing mode, paving the way to run framework applications in the context of an automated build environment – smart move!

Although awkward at first, using YAML files for database fixtures makes a lot of sense. Managing database access in unit tests has always been a challenge but thanks to the in-memory database server and fixture files Play offers us database integration testing – giving us the fresh-start benefits of mock frameworks along with the soundness of mind that comes from knowing you are testing the real database.

Share Nothing Architecture
Call it laziness, call it human error. At some point in the development cycle, the session always seems to end up carrying user data around. Even with data-sharing applications like memcached, that style of development does not scale well. With Play, sessions are stored in user cookies and consist of an encrypted key. The idea is the application takes care of loading any additional information it needs from this seed information, so the web cluster can be expanded to hundreds of nodes or reduced to a single server with no performance penalties on the other servers. Each Play instance operates as if it is the only one in existence, making it far easier to support complex site architectures.

Related posts:

1. Accessing Configuration Parameters using Play Framework’s Template Engine
2. Rely on Continuous Integration

photo credit: nimishgogri

Drizzle – a lightweight fork of MySQL – has reached general availability. Drizzle’s design goals are to create a highly performant and module database engine tailored for cloud computing.

Some of the interesting features this database has to offer are:

• No Views, Triggers, or Stored Procedures – I consider this a huge advantage. Stored Procedures were once a good thing when query optimizers were all but non-existant, but modern database systems perform this task extremely efficiently. Using stored procedures adds a second level of APIs to your application for what I would consider to be a rather dubious potential security benefit.
• Sharding – The client protocol is designed to decide which database server to target based upon a hashing key. This is similar to how memcached handles its linear horizontal scaling – leaving this calculation to the client is a huge advantage to systems hosting a high number of concurrent visitors
• Gearman Support – This is a terrific tool for spreading workload across machines. Use Gearman to handle logging in Drizzle, keeping the actual DB server available for database work

How Does Drizzle Compare to Percona?
Percona is a high-performance build of MySQL which promises to offer better performance than “stock” MySQL. It is built from a branch of publicly-available MySQL source code and enhanced by the folks at Percona. Percona maintains the same functionality as “real” MySQL, but attempts to achieve faster speeds.

How Does Drizzle Compare to MariaDB?
MariaDB is a different database engine started by the original creator of MySQL intended to replace MySQL using non-Oracle licensing. Anyone concerned with Oracle’s development practices or plans for the future of MySQL should consider switching to MariaDB. MariaDB implements all of the existing MySQL functionality plus more and provides a drop-in replacement for MySQL.

Related posts:

1. MySQL Pulled From Mac OS X Lion
2. Using Memcache with MySQL
3. Memcache as MySQL’s Hero

No related posts.

Installation:
1. Download the generators into your application’s lib folder:
cd myapp
git clone git://github.com/indirect/rails3-generators.git lib/generators

2. Install the rails3-generators gem
gem install rails3-generators

3. Add this to your project’s Gemfile
gem 'rails3-generators'

Usage:
rails generate model ModelName --orm=mongo_mapper

Related posts:

1. Displaying Production-Only Markup in Rails