Always Get Better

Never stop looking for ways to improve

September 29th, 2013

This is a common situation: I needed to compare two strings with unknown capitalization – “country” versus “Country”. Since these words should be considered equal even though the second “Country” has a capital “C”, we can’t do a straight comparison on the two – we need to do a case-insensitive comparison.

Option 1: strcasecmp
Whenever possible developers should try to use built-in functions which are compiled code and (in general) run much faster than anything you could write. PHP has strcasecmp to case-insentively compare two strings. Sounds like a perfect match!


if ( strcasecmp('country','Country') != 0 ) {
// We have a match!
}

Option 2: strtolower
Always read the documentation, but draw your own conclusions. One commentator in the PHP documentation suggested developers never use strcasecmp, and use strtolower with regular equality like this:

if ( strtolower('country') === strtolower('Country') ) {
// We have a match
}

Test the Speed
Both methods accomplish the same thing, but do we really want to skip using strcasecmp()? Which is the better option?
I wrote this short script to run each option for 10 seconds and see which is faster:

And the results:

strtolower: Done 18440869 cycles in 10 seconds
strcasecmp: Done 22187773 cycles in 10 seconds

So strcasecmp has the edge speed-wise, but not so huge that I would care to favour one over the other.

Apparantly strcasecmp does not support multi-byte (e.g. Unicode) characters, but I haven’t tested this. Presumably that would give strtolower an advantage over projects dealing with non-English input, however that is not the case at all in my particular use case so I did not explore this avenue any further. I also didn’t try against non-ascii characters, such as latin accents; including those would be an improvement on this test.

May 27th, 2013

Glen Canyon Bridge & Dam, Page, ArizonaI continue to have an on and off relationship with Twitter. It’s been fun to talk with other developers and reach people directly, but a huge part of the network is sorting through the signal-to-noise echo chamber. It doesn’t make sense to sit on Twitter all day trying to respond to everything; work needs to be done too!

Then there’s my reading. I read a lot. And I run into all kinds of cool stuff I want to share, and Twitter is the most natural place to share it, but of course that always ends up with Saturdays where I dump four dozen links in the span of a few hours… I hate it when other people do that, so rather than spamming everyone who follows me I’ve pretty much stopped sharing.

Until now.

Buffer to Spread Around the Outbursts
I found an app called Buffer (bufferapp.com) that sits in front of your twitter account and collects your tweets into a “buffer”, then sends them out on a schedule. So you can have a backlog of messages filter out slowly over a day instead of shoving them all out at once.

So my workflow with Twitter now is to monitor it (using Growl, of course) and have conversations where I can. I’ve met some incredible people using Twitter and made more than a few fun connections, and hope to keep building that over time. Whenever I read something interesting I’ll drop it into Buffer, and anyone who is interested can see those links without getting spammed all at once. I think it’s win-win.

Present in More Time Zones
At first night times were lonely when I came out west, since 9pm for me is midnight for friends back home, it got pretty quiet fast. I’ve since made more friends on the west coast, but I came away with a fresh appreciation of how easy it is to get disconnected from our core tribes because of time zones.

Since I started using Buffer I’ve noticed more activity from my contacts in Europe and Australia. Of course I’m asleep when Buffer sends out one of my stored tweets at 3am, but sometimes it’s sparked conversations I’m able to pick up when I wake up in the morning. Although there is a high latency in those communications, I feel more connected than ever to some old friends who I might not have otherwise interacted with so frequently.

In the End, Connections Matter Most
The strongest takeaway theme that seems to be cropping up again and again lately has been the difference between technology and communication. It’s very easy, especially coming from a technical background, to fall in love with a design, a language, a piece of software. The magic comes from the conversations that get enabled by these advances. There’s no reason to put up a web site or build an application if it doesn’t solve some problem – if we build something for the sake of doing it, are we building something that will last?

January 1st, 2013

This has been one of my busiest and most productive years, but you wouldn’t know it from the infrequency of my blog updates.

The majority of my writing time was spent creating two books on Node.js which were released this year – Node: Up and Running and Building Node Applications with MongoDB and Backbone. I’ll use that as my excuse for not updating here – I’m not planning to take on any major projects like this in the near future because I want to get some of my “smaller” ideas accrued over the year out in article form.

Apart from writing, I pulled up roots and moved to the West Coast; it was an epic move since we have a larger than normal family, but the new environment and business opportunities have been well worth it. Now that we live on a mountain next to the ocean, I’m going to feel very reluctant to go anywhere else.

At the end of 2011 I predicted that mobile development would become a necessary tool for serious developers. It’s certainly been my bread and butter this year, and will likely be even more so in 2013. Desktop development will remain important, but I can see future world where the majority of “normal” customers will be consuming content on a tablet or handheld device. There is going to be a greater dichotomy between content creators (think – developers on laptops) versus content consumers – designing for mobile first will position you for that economy by keeping your focus on ease of use.

The top three posts this year were:

  1. Using DateTime in the Play! Framework
  2. Setting up WordPress with nginx and FastCGI
  3. log4php Performance
September 10th, 2012

With just a single mobile release under my belt now, I’m hardly what you might call an expert on the subject. What I can say for certain is the past year has been an eye opener in terms of understanding the capabilities and limitations of mobile platforms in general.

So having gone from “reading about” to “doing” mobile development, these are some of the “aha” moments I’ve had:

Design for Mobile First
The biggest revelation: Even if you’re not setting out to develop a mobile application, your design absolutely must start from the point of view of a handheld.

Think about it for a second – how much information can you fit on a 3.5″ screen? Not much! So when you design for that form factor you have to go through a savage trimming exercise – everything from type, to layout, to navigation must communicate their intent in as tiny a space as possible. In other words, there’s no avoiding the need to effectively communicate your message.

When you build all of your applications this way, mobile or not, it’s hard not to come up with a user experience that has laser focus and goes straight for the mark. Once you have a bare minimum experience, now you can augment it with additional navigation and information (including advertising) for larger form factors like desktop and tablets.

Don’t Fret the Framework
Just as in web development, frameworks are powerful tools that come with sometimes painful trade-offs. Especially when you’re getting started, don’t worry about which framework you choose – just pick the one that caters most to the style of development you’re already familiar with and jump in. If that means Adobe Air for ActionScript, cool. If it means PhoneGap for JavaScript, great.

Most of the new devices coming onto the market have more than enough memory and processing horsepower to handle the massive extra overhead incurred through cross-platform development tools. If you’re starting a new project or iterating a prototype, don’t hesitate to jump on board a tool that will get you to a product faster. This is one of those areas where the short term gain is worth the longer term pain…

Native Wins
We’ve known since the 80s, when developers had to release to a boatload of PC platforms – IBM, Commodore, Amiga, Tandy, etc – that software written directly for a particular platform outperforms and outsells similar software written for a generic platform and ported across to others. The same idea is definitely the case now, even though our cross-platform tools are far more advanced, and our globally-usable app much higher in quality that what we could produce 30 years ago.

Some of the compelling reasons why you would want to take on the expense of building, testing and maintaining you app natively:

  • The UI of your application will integrate seamlessly into the underlying operating system – iOS widgets look like they belong, Android layouts are laid out consistently compared to other applications
  • Raw speed – you don’t need to go through someone else’s API shim to get at the underlying operating system features, you don’t have to develop custom native code since all the code is native; all CPU cycles are devoted to your application, resulting in much higher performance, particularly for graphic-intensive applications
  • Operating system features – each mobile operating system has its own paradigm and set of best practices which cross-platform tools gloss over to give you as a developer a consistent response. So your application misses the subtleties of the user’s hardware experience – for example Android uses Activities as its interaction model, but the Adobe Air framework squashes that instead of forcing developers to program in an Activity-centric way

In other words, cross-platform tools exist in order to give developers the best experience, not to give the user the best experience. Your customer doesn’t care if your app is available on Android, Windows, iPhone, Playbook and WebOS if all they have is an iPhone.

I believe cross-platform tools are the best way to get your project off the ground and usable fast, but right from the beginning you need to be thinking about converting your application to native code in order to optimize the experience for your customers.

Market Fragmentation
I bought an Android phone and have been enjoying developing for it. But I don’t think I would enjoy developing for Android at large because of the massive amount of devices and form factors I would need to support. This is where Apple has an edge – although the learning curve for Objective-C is higher, once I have an iOS application, I know it will run on an iPod Touch, and iPhone or and iPad. Not only that, but my guess is people are more likely to want to spend small amounts of money on app purchases, since they’ve been trained to do so from years of iTunes.

Backward Compatibility
Moving to the web was a huge advantage in terms of software support because if you ever found a bug in your program you could patch it and release it without affecting any of your users. No one has to upgrade a web page.

This isn’t true of mobile applications – ignoring mobile web applications – once someone downloads your app they may be slow to upgrade to newer verions, or they may never upgrade at all. This means any bugs that get released with your bundle are going to remain in the wild forever. If you have any kind of server-based presence, your server code needs to handle requests from all of those old app versions – so you need to make sure you get it right, and have good filtering and upgrade mechanisms in place.

Choosing a Platform
One thing that held me back from diving into mobile development was my hesitation to start. This is totally my fault – instead of just programming for WebOS when I had the Palm Pre, I thought it would be better/more accessible to use a more open JavaScript toolset so I could deploy my app to other phones. But really, what would have been the point? I only had a Palm Pre to run my software on and I definitely wasn’t going to buy new hardware to test other versions. Instead of getting locked in analysis paralysis I should have just started programming for the Pre right away and transferred those skills to a more mainstream platform later.

So if you don’t have a smartphone, go get one – it will change your life, or at least the way you interact with your phone. Then start building apps for it. That’s all it takes to get into the game. Don’t wait another second.

photo by: osde8info
May 20th, 2012

Which data type should you use for time information on your Java models? The Date class is mostly deprecated, pushing us into the direction of the heavyweight Calendar class. Storing the milliseconds since Epoch in a Long is both database-friendly and easy to perform math on and convert at runtime. But if you really want a good experience, you are using Joda Time.

Joda Time is built into the Play! Framework, and there really is no excuse to use anything else. But when it comes to saving dates in your JPA Models, there is a big “gotcha” in that there is no default type converter to move a DateTime object into and out of the database. Oops.

But that can be fixed with an annotation in your model, like this:

@Type(type="org.joda.time.contrib.hibernate.PersistentDateTime")
public DateTime created;

Unfortunately, Play does not ship with Hibernate support for the DateTime object. So to make this work you need to include the Joda-Time-Hibernate library in your dependencies.yml file:


require:
- play
- joda-time -> joda-time-hibernate 1.3

After updating the dependencies file, run the play deps –sync command to pull in the libraries from maven. Your models will now save their date and time into MySQL, and your programming experience will be smooth as silk – at least as far as time-based functionality is concerned.

April 30th, 2012

In just two weeks, Node: Up and Running will be released by O’Reilly Media. Writing a book has been a lot of hard work but also a terrific learning experience that I would love to repeat.

The biggest takeaway for me was how often I make stupid mistakes in my writing. As a developer and manager, I rely on my speaking and writing abilities every day – so I take my ability to express myself for granted because I have to do it every day.

When a professional editor takes a piece of writing, they aren’t looking at it in the same way a co-worker would. A co-worker knows me, understands some of the subtleties of the context I’m writing about, and can subconsciously apply meaning to ambiguities in the text or conversation. A casual reader doesn’t have the same context, and the copy editor is able to filter that out and make adjustments to the text that leave my meaning intact but change the delivery.

In other words, the text that came out of the editing process makes me look really smart (I wish!). I’ve learned the secret to clear communication is in keeping the message brief. Especially in a technical book, the audience can’t be expected to deconstruct prose – it’s up to the writer to make their point and get out of the way.

I’ve also learned that I use the same turns of phrases over and over again. Reading 50 pages of my own writing in a row with the same sentence transitions is boring as heck, and I’m able to see this strikingly clear when it’s annotated by a totally impartial writer.

February 29th, 2012

We can take for granted that whenever we introduce a library or framework to our application, we incur an overhead cost. The cost varies depending on what we’re trying to do, but we generally accept that the lost performance is worth it for the increased maintainability, functionality or ease of use.

For many teams, logging is something that gets thrown in the mix at the last minute rather than through of all the way through. That’s a shame because a well-implemented logging solution can make the difference between understanding what is going on in your system and having to guess by looking at the code. It needs to be lightweight enough that the overall performance is not affected, but feature-rich enough that important issues are surfaced properly.

Java programmers have had log4j for a long time, and log4net is a similarly mature solution in the .NET world. I’ve been watching log4php for awhile and now that it has escaped the Apache Incubator it is impressively full-featured and fast. But how much do all its features cost?

Benchmarks
I’ll be looking into different options as I go, but let’s consider a very basic case – you append all of your events to a text file. I’ve created a configuration that ignores all ‘trace’ and ‘debug’ events so only events with a severity of ‘INFO’ or above are covered.

In 5 seconds, this is what I saw:

Test Iterations
BASIC (direct PHP) 45,421
INFO STATIC 45,383
INFO DYNAMIC 41,847
INFO STATIC (no check) 51,801
INFO DYNAMIC (no check) 47,756
TRACE STATIC 310,255
TRACE DYNAMIC 213,554
TRACE STATIC (no check) 271,043
TRACE DYNAMIC (no check) 196,653

Tests
What is all that? There are two ways to initialize the logger class – statically, meaning declared once and used again and again; and dynamically, meaning declared each time. With log4X, we typically perform a log level check first, for example isTraceEnabled() to determine whether to proceed with the actual logging work.

Results
I was surprised by how little log4php actually lost in terms of speed versus raw PHP. The authors have clearly done a thorough job of optimizing their library because it runs at 90% of the speed of a direct access.

I’ve always intuitively used loggers as static variables – initialize once and use over and over. This seems to be the right way by a huge margin.

Checking for the log level before appending to the log was a big win for the INFO messages, which are always logged to the file due to the configuration settings. The intended use is to allow programmers to sprinkle their code with debug statements which don’t get processed – and therefore slow down – the production code. I would be very happen with this in my project. In the INFO metrics, the check slowed things down a bit – explained because the actual logging function performs the same check – so we are taking a double hit. But wait, there is a good reason…

The TRACE metric is interesting – these are events which are NOT appended to the log. In that case, when the check is not performed, we pass through the code more times. When the check is performed, the code has to execute deeper on the call stack before it figures out we aren’t doing any actual logging, taking more time.

Conclusion
If you know you will be logging every single event, don’t do a check. Otherwise do the check – it will save a lot of wasted cycles.