Always Get Better

Archive for the ‘General Programming’ Category

Malware in the iTunes Store – 500M Users Affected

Thursday, November 12th, 2015

Malware found in Xcode (xcode-ghost)People rely on their app stores to provide safe and high quality software for their phones and tablets. With iTunes in particular, we trust Apple’s strict review standards will raise the bar for quality and protect us from the shady apps and trojans we might find on file sharing services and third party app stores. What happens when that trust is misplaced?

A bunch of popular apps on Apple’s App store shipped with malware in September. “Popular”, of course, refers to app used by more than 500 million iPhone users primarily in China where the malware originated. Some apps, like WeChat, are also used in the west but by-and-large the affected programs are names written in hanzi most of my readers would not be able to pronounce let alone install to their devices.

What does the malware do?

First off, what can you expect to happen if your phone gets infected?

This post from MacRumours summarizes things pretty well, but the highlights are:

  • Sends device information, phone number and ID to C2 (command-and-control) servers
  • Fake system alerts phishing for passwords
  • Access clipboard (including stealing secrets from your password manager)
  • Hijack URL opening

We’re not talking about phone-takeover threat levels, which is a testament to the sandboxed environment apps run in. These functions are available to pretty much any developer — you’re not supposed to use the device ID but you can get it if you want it.

How Did Malware Get On the App Store?

The first thing that most people are asking is: how did this happen? With Apple’s notoriously stringent app review process, how did apps with this vulnerability get through?

As it turns out, developers chose to download updated versions of XCode from file sharing networks rather than directly from Apple. XCode is a huge download (larger than 3Gb) so downloading directly from Apple can take a long time. In order to save time these developers thought it would be a good idea to trust an unsigned version rather than the one from Apple.

How Did Malware Get Through the Walled Garden’s Gates?

But what about the review process? The UUID-gathering portion alone should have been enough to trigger warning flags, right?

Not so, according to people who know.

Although the App Store performs a cursory check of submitted apps for private API usage, the XCodeGhost files were added as a dynamically loaded bundle to the compiled app bundle — in other words, not subject to the code scanning… pretty sneaky.

How to Protect Yourself

How can you protect yourself when the store you rely on for your apps is poisoned?

Not much. Other than inspecting every single network connection there are not a lot of things you can do to detect this kind of activity on your device.

Installing app updates as they become available is the best thing you can do for your phone. Although an attack on this scale was made possible by the single-provider model of the App Store, it’s also true that having the App Store as the centre of the ecosystem is the best way to quickly push out fixes for problematic apps as they are discovered.

Always Get AppleCare

Sunday, April 5th, 2015

I never buy warranties for my purchases – if someone is expensive enough it should go on your home insurance policy. If it’s cheap enough, just replace it – we live in a society of garbage anyway. Getting serviced when things break is more often than not a huge waste of time. AppleCare is different.

AppleCare is worth the cost
AppleCare is the exception to my “policy”. Like the rest of Apple’s products it comes with a premium price tag, but unlike many company warranty products, Apple actually seems to stand behind their plans – except, apparently, if your device is damaged by water, but we can talk about that another time.

Fixing a Broken Screen

A few months ago (and not long after I first got it) my phone took a fall onto the pavement and the screen shattered. Not sure why I left it so long but on a whim I loaded up the Store app and booked an appointment to be seen by a “genius” at the nearest Apple store.

An hour later I was sitting at their genius bar with my phone being seen by a blue-shirted worker. No questions asked about how it got damaged, no trying to invalidate the AppleCare plan I had on the phone. The dent in the corner was too deep to replace the screen so they replaced my phone and I was out of there in under 15 minutes.

Without the AppleCare I would have still gotten the refurbished phone, but it would have cost hundreds of dollars. It was disappointing to break my phone when it was brand new, but I’m definitely glad I sprung for the protection plan.

[Edit: February 2016]

Servicing the Battery – AppleCare again

The battery on my Macbook Pro gave out just before a business trip. AppleCare to the rescue again! Without it, I would have been out $600, since the battery is connected to the case and the whole top part over the keyboard needed replacement.

Maybe it’s a bit dirty to have everything so proprietary and tightly packed that your customers are basically forced to buy insurance instead of pay hefty repair things, but based on the bill amounts I saw, the warranty has paid for itself.

Standing Desk Experiment

Thursday, March 19th, 2015

Over the past year I experimented with working at a standing desk. My setup was a converted “regular” sitting desk with the computer monitor lifted on a set of rubbermaid bins. It wasn’t the most beautiful rig but the ergonomics were right and the screens were at an appropriate level for my height. Unfortunately I didn’t take any pictures of the contraption.

Motivation: Not Wanting to Die

It should come as no surprise that sitting for long periods of time is killer bad for you. When you live a generally sedentary lifestyle, any excuse to get out of your chair is a good one; so my expectation going in was that I would probably be uncomfortable from being on my feet and subtly flexing my knees all day versus sitting.

In reality it wasn’t bad at all. The first two or three days my legs were pretty sore at the end of the day and then my body got accustomed to standing. Many standing desk users suggest putting an anti-fatigue mat down; I never did nor did I find I needed it. The first week I used a few towels to pad my feet from the hard floor but after the initial adjustment I was fine on my own.

Results: More Energy

After sitting at a desk all day I noticed I was always exhausted both physically and mentally after work. I would want to eat dinner and relax on the couch with the kids, maybe watch a movie. I expected to be ready to crash and put my legs up after standing all day, but the opposite happened.

Unlike days spent sitting, when you stand you never stand perfectly still; you’re always shifting your weight from one leg to the other even if you’re not noticing it. At the end of the day I felt wired – I didn’t want to sit at all. I was more likely to go for a walk than to vegetate and zone out after working.

Results: Sharper Focus

After all this I think every company should raise their conference tables and throw away the chairs, or ditch the conference room and go for walking meetings instead. When you’re standing you naturally approach your tasks with more urgency, and have quick brilliant insights. Everything gets done faster because there is less mind-wandering.

For a programmer, this works really well if you are doing a good job at breaking your work down to manageable tasks. A single work unit can be done quickly when there are no other distractions, and standing really helped cut through all those other distractions.

Results: Shorter Attention Span

The major downside I experienced was the flip side of the sharp focus that standing brought. Long monotonous jobs, like data-entry, research, or marathon programming sessions were a lot more difficult to focus on while standing. I would start tasks with a lot of focus but lose interest and need to switch to something else or walk around. This worked well for quick multi-tasking kinds of work, but when something too longer it felt like a struggle.

Mix It Up, Do Both

The standing desk streak was broken when I did some business travel last November. I had to force myself to sit at a desk all day where no standing spaces were available. Turns out, I didn’t hate it as much as I expected.

Nowadays I have a lot of surfaces to work on and I move around throughout the day, switching between sitting and standing positions depending on what needs to be done. The next purchase might be an adjustable desk capable of moving from a sitting to standing layout throughout the day.

I would recommend everyone try working in a standing position at least once. Like me you can try it out quickly without spending a fortune on a dedicated standing desk.

Native Languages are Best for Mobile Development

Monday, July 14th, 2014

Like it or not your clients want mobile and if you are a developer without experience in this realm you are behind the curve.

Having been mobile-centric for the past three years of my professional life, I have heard all of the different lines of thought around toolsets and approaches for creating software targeting these devices. Moreover, I’ve lived through a lot of those toolsets and approaches, and have spent time thinking about which worked well and which ones, well… sucked.

It all boils down to this:

Build native apps using the tools and languages provided by your platform.

Yes, this is the same position I held when I wrote about this almost two years ago, but I stand by it. I’ve had a chance to play and build with the likes of Adobe Air, PhoneGap/Cordova, SenchaTouch and Xamarin, but I always end up going back to Java (Android) or Objective-C (iOS) because…

Cross-Platform Isn’t Good for Anybody

Your app is iOS only… are your installed customers complaining about it not being on Android? No! Why would they care?

When talking about a new project one of the first things to come up is the idea of using X tool so we can build to a lot of platforms after writing the code once. I guess the thinking behind this is somewhere along the lines of “if we make our product available to more people, then it follows that more people will obtain it and we will make more money”. I don’t buy this.

Look at it this way – you’re up against a million apps. What sets you apart? How do you communicate that to all of your potential customers? Are downloads from the Windows Store equivalent to downloads from Google Play? Do they use their phones in the same way and monetize the same way? Are the demographics the same?

If everyone was the same, it would still be a bad idea to build your app around a cross-platform framework. Invariably these work in a lowest-common-denominator fashion meaning you get universal support by trading off native features. Yes, your app runs the same on all the platforms, but Android users are accustomed to navigating using their back button, which iPads don’t have. Instead of providing beautiful context-sensitive navigation using activities for Android and NavigationController hierarchies for iOS, you get uniform views.

“Universal” is a Lie

Okay, so most development kits come with a way to plug in your own native code for the device-specific goodness (better GPS, camera gestures, whatever). Now you’re writing native code anyway, and you’re on the hook to support all those platforms. So why would you want to go through the pain of hooking all that into a cross-platform tool instead of going directly to your platform in the first place.

Worse, your universal framework has bugs. I’m sure it’s great software but all software has bugs. When you hit a bug that stops you from moving forward, what are you going to do? When the underlying operating system software changes and your framework hasn’t been updated yet, are you not going to deploy until the author gets around to supporting the new requirements? When you choose a cross platform tool you are choosing someone else’s opinions and priorities and ceding control of your own product.

My recommendation is to pick a release target and excel at it. Will it be in the Apple ecosystem? If so, don’t be afraid to go all in. Learn how iPhone users are going to interact with your software and do everything you can to speak to them, please them, and turn them into paying customers. They don’t want a great Android experience, they want software they don’t even think about. They won’t patiently deal with your buggy software because you were so afraid of missing out on customers that you greedily deployed a boring app just to support some other platform. The “smaller” number of users is still a lot of users!

I get the fear factor, but the cross-platform tools are designed to make life easier for developers, not customers. The developer isn’t buying my product – I don’t care so much if he’s worried about synchronizing features in a future Android port of the app.

Speed is King

I want my app to load fast and I don’t want to wait around for it. You can’t get more performance than a well-written app build from native languages and tools. Don’t be afraid of XCode, Visual Studio and IntelliJ – embrace them and enjoy the barebones software you can build with them.

Thinking in New Paradigms

Suppose you do want to put your app on two different kinds of devices. Now you need to maintain your program logic in two programming languages, adding new features and squashing bugs in both places. That’s not really such a bad thing.

What if you have an ENORMOUS code base with hundreds of thousands of lines of code – how will you ever keep that synchronized between two development tracks? Well, you might a good candidate to actually use one of those cross-platform tools. Or you might be asking yourself if your work load is really appropriate for a mobile experience (something they won’t tell you — not every app belongs on a mobile device).

Apps don’t need to behave the same on different device formats. Every modern operating system has a different focus and enormous capability list that becomes adopted by its users – so evaluate what it is really like to use your app on each device. Especially for UI stuff – do you need backward navigation on Android when there is a hardware button for that? What buttons can you eliminate on iOS in favour of multi-touch gestures?

In the past 5 years I’ve added Java, Scala, Ruby, JavaScript (specifically, Node.js-style callbacks), Objective-C and lately Swift to the list of languages I’ve used extensively and thought deeply about. Each one has challenged me to look at programming differently and apply new practices to older thought patterns. Working withing multiple native environments is not a hindrance, it’s a huge boost to the quality of software you can create.

Streamlined Learning

When you use a cross-platform tool, you have an extra learning curve – that of the tool itself. I don’t buy into the thought that it improves your productivity, but I definitely see where it decreases your productivity. You need to be aware of bugs in your tool (as we mentioned earlier, your tool is great but it’s still software and software has bugs), you need to be aware of the capabilities of your tool including all the new features they add (which, by the way, just expose native features you should have been working with directly all along).

If you can avoid it, skip the cross-platform stuff and go right to your actual platform.

Putting it all together

As a programmer your job is not about writing code, it’s about connecting businesses with their objectives. A good technologist doesn’t get caught up in ideology over which approach they ought to take to build software – they decide which implementation detail will get them to their goal. So if a cross platform tool makes sense for your project, use it. Just be sure that you’re picking the right tool, and not the comfortable (for now) one

2012 in Review

Tuesday, January 1st, 2013

This has been one of my busiest and most productive years, but you wouldn’t know it from the infrequency of my blog updates.

The majority of my writing time was spent creating two books on Node.js which were released this year – Node: Up and Running and Building Node Applications with MongoDB and Backbone. I’ll use that as my excuse for not updating here – I’m not planning to take on any major projects like this in the near future because I want to get some of my “smaller” ideas accrued over the year out in article form.

Apart from writing, I pulled up roots and moved to the West Coast; it was an epic move since we have a larger than normal family, but the new environment and business opportunities have been well worth it. Now that we live on a mountain next to the ocean, I’m going to feel very reluctant to go anywhere else.

At the end of 2011 I predicted that mobile development would become a necessary tool for serious developers. It’s certainly been my bread and butter this year, and will likely be even more so in 2013. Desktop development will remain important, but I can see future world where the majority of “normal” customers will be consuming content on a tablet or handheld device. There is going to be a greater dichotomy between content creators (think – developers on laptops) versus content consumers – designing for mobile first will position you for that economy by keeping your focus on ease of use.

The top three posts this year were:

  1. Using DateTime in the Play! Framework
  2. Setting up WordPress with nginx and FastCGI
  3. log4php Performance

Observations From Mobile Development

Monday, September 10th, 2012

With just a single mobile release under my belt now, I’m hardly what you might call an expert on the subject. What I can say for certain is the past year has been an eye opener in terms of understanding the capabilities and limitations of mobile platforms in general.

So having gone from “reading about” to “doing” mobile development, these are some of the “aha” moments I’ve had:

Design for Mobile First
The biggest revelation: Even if you’re not setting out to develop a mobile application, your design absolutely must start from the point of view of a handheld.

Think about it for a second – how much information can you fit on a 3.5″ screen? Not much! So when you design for that form factor you have to go through a savage trimming exercise – everything from type, to layout, to navigation must communicate their intent in as tiny a space as possible. In other words, there’s no avoiding the need to effectively communicate your message.

When you build all of your applications this way, mobile or not, it’s hard not to come up with a user experience that has laser focus and goes straight for the mark. Once you have a bare minimum experience, now you can augment it with additional navigation and information (including advertising) for larger form factors like desktop and tablets.

Don’t Fret the Framework
Just as in web development, frameworks are powerful tools that come with sometimes painful trade-offs. Especially when you’re getting started, don’t worry about which framework you choose – just pick the one that caters most to the style of development you’re already familiar with and jump in. If that means Adobe Air for ActionScript, cool. If it means PhoneGap for JavaScript, great.

Most of the new devices coming onto the market have more than enough memory and processing horsepower to handle the massive extra overhead incurred through cross-platform development tools. If you’re starting a new project or iterating a prototype, don’t hesitate to jump on board a tool that will get you to a product faster. This is one of those areas where the short term gain is worth the longer term pain…

Native Wins
We’ve known since the 80s, when developers had to release to a boatload of PC platforms – IBM, Commodore, Amiga, Tandy, etc – that software written directly for a particular platform outperforms and outsells similar software written for a generic platform and ported across to others. The same idea is definitely the case now, even though our cross-platform tools are far more advanced, and our globally-usable app much higher in quality that what we could produce 30 years ago.

Some of the compelling reasons why you would want to take on the expense of building, testing and maintaining you app natively:

  • The UI of your application will integrate seamlessly into the underlying operating system – iOS widgets look like they belong, Android layouts are laid out consistently compared to other applications
  • Raw speed – you don’t need to go through someone else’s API shim to get at the underlying operating system features, you don’t have to develop custom native code since all the code is native; all CPU cycles are devoted to your application, resulting in much higher performance, particularly for graphic-intensive applications
  • Operating system features – each mobile operating system has its own paradigm and set of best practices which cross-platform tools gloss over to give you as a developer a consistent response. So your application misses the subtleties of the user’s hardware experience – for example Android uses Activities as its interaction model, but the Adobe Air framework squashes that instead of forcing developers to program in an Activity-centric way

In other words, cross-platform tools exist in order to give developers the best experience, not to give the user the best experience. Your customer doesn’t care if your app is available on Android, Windows, iPhone, Playbook and WebOS if all they have is an iPhone.

I believe cross-platform tools are the best way to get your project off the ground and usable fast, but right from the beginning you need to be thinking about converting your application to native code in order to optimize the experience for your customers.

Market Fragmentation
I bought an Android phone and have been enjoying developing for it. But I don’t think I would enjoy developing for Android at large because of the massive amount of devices and form factors I would need to support. This is where Apple has an edge – although the learning curve for Objective-C is higher, once I have an iOS application, I know it will run on an iPod Touch, and iPhone or and iPad. Not only that, but my guess is people are more likely to want to spend small amounts of money on app purchases, since they’ve been trained to do so from years of iTunes.

Backward Compatibility
Moving to the web was a huge advantage in terms of software support because if you ever found a bug in your program you could patch it and release it without affecting any of your users. No one has to upgrade a web page.

This isn’t true of mobile applications – ignoring mobile web applications – once someone downloads your app they may be slow to upgrade to newer verions, or they may never upgrade at all. This means any bugs that get released with your bundle are going to remain in the wild forever. If you have any kind of server-based presence, your server code needs to handle requests from all of those old app versions – so you need to make sure you get it right, and have good filtering and upgrade mechanisms in place.

Choosing a Platform
One thing that held me back from diving into mobile development was my hesitation to start. This is totally my fault – instead of just programming for WebOS when I had the Palm Pre, I thought it would be better/more accessible to use a more open JavaScript toolset so I could deploy my app to other phones. But really, what would have been the point? I only had a Palm Pre to run my software on and I definitely wasn’t going to buy new hardware to test other versions. Instead of getting locked in analysis paralysis I should have just started programming for the Pre right away and transferred those skills to a more mainstream platform later.

So if you don’t have a smartphone, go get one – it will change your life, or at least the way you interact with your phone. Then start building apps for it. That’s all it takes to get into the game. Don’t wait another second.

Using DateTime in the Play! Framework

Sunday, May 20th, 2012

Which data type should you use for time information on your Java models? The Date class is mostly deprecated, pushing us into the direction of the heavyweight Calendar class. Storing the milliseconds since Epoch in a Long is both database-friendly and easy to perform math on and convert at runtime. But if you really want a good experience, you are using Joda Time.

Joda Time is built into the Play! Framework, and there really is no excuse to use anything else. But when it comes to saving dates in your JPA Models, there is a big “gotcha” in that there is no default type converter to move a DateTime object into and out of the database. Oops.

But that can be fixed with an annotation in your model, like this:

@Type(type="org.joda.time.contrib.hibernate.PersistentDateTime")
public DateTime created;

Unfortunately, Play does not ship with Hibernate support for the DateTime object. So to make this work you need to include the Joda-Time-Hibernate library in your dependencies.yml file:


require:
- play
- joda-time -> joda-time-hibernate 1.3

After updating the dependencies file, run the play deps –sync command to pull in the libraries from maven. Your models will now save their date and time into MySQL, and your programming experience will be smooth as silk – at least as far as time-based functionality is concerned.