Eachan's Space

I am moving house - Blogger has been home to many of my contemporaries for a long time and, like today's teenagers, I am caving in to their peer pressure.

All future posts shall be here!

I never thought you were that bad Windows Live...

I am a really big fan of timeboxing at the moment.  That basic principle of looking at what you've got to do, deciding on a sensible block of time to allocate to it and when you're at the end of that time asking "if we spend twice as long on this will the results be twice as good?"  If yes then repeat, if no then next task...

I define a 'sensible' period of time as long enough to make adequate progress on the task at hand but not so long that your pace, focus and concentration drops off.  Typically you'll be dealing in a handful of hours at a time.

As the title of this post suggests it obviously works really well in estimating sessions (close of those long, drawn out revisions of man days) and keeping energy and pace in stand-ups (stopping those meandering speakers) but I've also found it works well as a personal time planning discipline.  Writing presentations, holding roadmap discussions, making a dent in the email backlog and even editing these posts...

It's easy to get tied up in building the perfect application every time but is that always the right thing to do?  Well if you're definition of 'perfect' is 'exactly fit for purpose' then I guess so; if you're definition of perfect is 'maximum quality in all areas' then perhaps not.  You're in a minority if you run a number of projects that are all equally beneficial to your organization; chances are your activities deliver various commercial and operational benefits over various periods of time.  That being so, should you build it all equally?

Let's think about some examples - 2 different projects and some random software attributes (such as quality, performance, total cost of ownership).  How much should we invest in each attribute for each project?  Well what's it worth to us...

Project 1 is building an embedded specialist site within our main sports trading site which is customized to take advantage of a very specific upcoming event.  We're expecting it to be popular and bring in a lot of extra revenue in the lead-up to and during this event but after the event it is of no further use.  Where would we spend our time?  Perhaps we'd divide our focus a little like the radar chart below.  We have a narrow window of opportunity to squeeze value from it (the event and it's lead-up) so we need it to be online all the time - so good quality.  We've said we expect it to be popular so lets make sure we build in enough capacity.  We know it's a unique event and there is minimal chance that we'll reuse it in the future so we can afford to skimp on the TCO and future flexibility.  It is temporary but it needs to be good while it's here.

Project 2 is adding a content management system to another existing product giving marketing droids the ability to control the displayed banners, articles and links in real time.  It's being launched in our product that needs the most cross-sell but if it works well we intend to roll it out to the rest of the suite.  What might that radar chart look like?  As below - we know we'll be keeping this for a while so let's spend more time on TCO (the longer you know you'll keep something the more cost effectively you should try and make it run), we know we'll extend it to the rest of the product suite if it's successful so let's keep some future flexibility in there.  We're not processing cash through this thing so we can afford to take a bit more quality risk and given it's an internal tool we know it has a limited pool of familiar users so ditto with performance and capacity.

It's about working smarter - concentrating on the areas that make the most difference to the product and avoiding over-engineering where you wont get ROI.  Excellent software makes perfect technical sense but successful software makes perfect commercial sense.  Developing product from principles rather than hard rules or templates allows you to do the right thing in the right proportions.

An interesting point I recently came up against in our hiring; we have a reasonably healthy balance in the team at the moment but, if anything, it's a little light at the top end.  We thought we should be looking for more senior engineers but actually it turns out we're really looking for more experienced engineers.

The difference?  A senior engineer is able to build very large, highly complex applications but an experienced one realises that this is a bad idea...

We recently passed our ISO 27001 audit (and in record time I might add) so hooray!  It nearly went all wrong though because I was slightly too fascinated by the assessors notebook although I obviously still managed to get through all the questioning.

I think I like tablet PCs - or maybe I like the idea better than the implementation - I just cant understand the sheer determination people seem to have about turning their scribbles into text characters.  Leave it alone I say; what's wrong with storing your notes in your own handwriting?  You know we were doing that for years and it seemed OK.  handwriting recognition software rarely gets it all correct and you usually have to "train" it - but do you or does it "train" you?  Taking it a bit far but if you change your natural stylus strokes so the machines can understand your curly script are you changing yourself?  I'd rather keep my doodles and sketches intact, it's much more meaningful to me but then I've been told I'm a bit classic.

You gotta make data portable, searchable, electronically transmittable, compressible yeah yeah yeah there just feels like there ought to be another way...

Before we tuck into the last post in this series let's have a quick recap of the 6 principles:

1. Scalability

2. Maintainability

3. Quality

4. Security

5. Availability

6. User Experience

I believe these things are the basics of building highly scalable online systems and as I've said before the basics are what every team succeeds or fails on.

User experience is a little different to the rest of our principles because it's largely subjective - the first 5 are about hard facts and actions, things you can use and measure but user experience is about how your customers actually feel about your application when it's all said and done.  Given our customers are who we're actually building these things for in the first place you could argue this is the most important principle of them all.

Now let's look at how we'd express this as a basic user story:

As the CTO, I want to represent products that are intuitive to use and feel fast and fresh to our customers, So that no matter where our users are and what their connectivity is they enjoy a high performance site, As proven by appropriate and justifiable answers to the user experience questions.

Like yesterday with availability notice our "so that" clause talks again about the considerations necessary for a global product.  Repetition is the building block of human learning so if it's important to you, you can never mention it too often.  Appropriate and justifiable are interesting in this context, if your product isn't usable people just wont use it.  How about betas and prototypes?  You've got to think carefully about what appropriate means here too, you can probably afford to spend less time on snappy performance and super quick load times but users are a fickle bunch so you'd better not skimp on the UI's look and feel - you wouldn't want a perfectly good product chucked out at prototype just because the human condition is to judge the kitchen by the dining room...

When you're aiming for the best possible user experience ask yourself:

• Are user-facing error messages helpful and friendly?
• How snappy does my UI feel?
• How long does my page take to load?
• How does my feature feel on lower spec machines?
• How does my feature feel on low bandwidth connections?
• Is my UI intuitive and uncluttered?
• What do my end users think of my interface?

Get to know your customers and get to know what they like.

Speaking about availability, Live was down for a couple of hours on Friday when I went to post this - ace timing...

The next principle of building megaprise software is availability.  That one in particular is really important to us; some recent market research we commissioned concluded that a 1% increase in customer confidence in our site resulted in a 16% increase in ARPU (average revenue per user).  Simplified - people spend more when they're surer you'll be up and a 1% improvement isn't out of anyone's reach, especially when it pays back by a factor of 16.

So how would a user story for availability look?  Let's start with:

As the CTO, I want to look out onto a resilient product suite that never stops even under catastrophic events, So that we never need to close our business and are always able to meet our users needs no matter what time zone they live in, As proven by appropriate and justifiable answers to the availability questions.

The time zone bit is an import part for us - we started off as a UK business, crept determinedly across Europe and with the recent inclusion of southern hemisphere markets I feel quite justified using the term global.  This has meant our traditional maintenance slots (the unsociable GMT hours) have evaporated and we're left with the challenge of keeping trading going 24x7 releases and all.  Appropriate and justifiable?  Well put it this way, I want that 16%...

What can we ask ourselves to make sure we've thought about availability enough?  A good start would be:

• How will I make this resilient?
• How will this recover from failure?
• How will it behave when it loses connection to it’s data?
• How will it behave when its dependencies are missing?
• How will the whole system behave when my feature fails?
• How will this features status be monitored?
• What are the major events in the system and how do I make these visible?
• Are the error messages the events produce meaningful in diagnosis?
• How will the system survive the loss of a feature?
• How will the system survive the loss of a node?
• How will the system survive the loss of a data centre?
• Can the system recover from common failure scenarios automatically?
• What tolerance to network and system maintenance have I built in?
• Does my feature meet any SLAs imposed?

Plan for failure, after all it's guaranteed.

Continuing this week's series about the six principles of webscale software we come to security.  This is a hot one for us as vulnerabilities can cost us serious cash, degrade our customers confidence in our business and irritate [understatement of the year] our regulators.  Simplified user story... familiar format... business value...

So what does our basic user story for security look like?  The sharper amongst us should be spotting a pattern by now:

As the CTO, I want to the peace of mind a highly secure suite of applications delivers, So that we continue to earn and keep our customers trust, As proven by appropriate and justifiable answers to the security questions.

And as the pattern dictates let's think about what appropriate and justifiable means in the context of security.  Security is one of those things that's easy to get carried away with as security professionals tend to be particularly good at raising awareness and we're never short of threats to cringe in fear from (code flaws, viruses, internal fraud) and standards (ISO27001, SOX) to measure up to.  The one thing I'd add to this is to remember security is an investment just like anything else; you wouldn't spend £20K insuring a £10K car and your products are no different.  Know the value of the asset, the risks it faces and the likelihood of eventuality - then you can make appropriate decisions.  Thinking about things like the industry you're in (do you have regulatory requirements or other standards to live up to?), the data you keep (how confidential is it?), where the money is (do you keep any?) and how important reputation and those other intrinsic things are will keep you able to justify time spent on testing and securing software.

Some worthwhile things to consider as you code are:

• How could this component be misused and how have I prevented this?
• How have I limited this feature’s access to the rest of the system?
• Does the feature use the smallest possible data range?
• How might people use this feature other than intended?
• Is all private customer data securely transmitted?
• Where is this features use logged and is its activity easy to deduce?
• How will we be alerted to abuse of this feature?

Thinking about these things will help keep security commensurate with risk as you deliver features - tomorrow is availability day.

Yesterday was maintainability day and today I'm continuing through the basics of building software by tackling quality.  Quality is a massive, massive topic encompassing practices, unit tests, automation, measurement, environments etc etc and clearly we cant talk about all of that in any single post.  Lucky for us we're talking about how you shape your thinking to make sure you're always dealing with quality in the most appropriate way rather than any of these quality solutions.  Oh and let's insert my usual comment about user stories and establishing the habit of thinking about the business value of an idea rather than just raw requirement...

Let's cook up a user story for quality replete with "as proven by" clause:

As the CTO, I want to represent high quality software following a distributed and decoupled architecture, So that we can leave behind reactive work and our engineering activities can be solely focused on adding value, As proven by appropriate and justifiable answers to the quality questions.

So what does appropriate and justifiable mean to quality?  You need to consider a feature's usage and the risk profile associated with it.  What's the business impact of bug?  You will probably want to treat a feature that calculates the money a user is able to withdraw very stringently.  You may quite possibly be less worried if your content carousel stops cycling fresh promo material for a while - you can fix that when you're good and ready.  Other things to think about are how many other people's hands this code is likely to go through and what the feature's planned shelf life is.

To help make sure you've covered your quality bases ask yourself:

• What is the anti-spec and how have I coded for it?
• Does my feature have tested and checked-in unit tests?
• How does my feature behave through a different user journey?
• Have I produced clear, readable documentation?
• Have I complied with any applicable design patterns?
• Do my comments and syntax make sense and match my documentation?
• Can I remove any lines of code?
• Will I be proud of this code if someone else reviews it?

If you can satisfy yourself with your answers to these questions you'll be well on your way to reusable, supportable and easy to understand software.

Yesterday was scalability day and today I'm continuing through the basics of building software by tackling maintainability.  As we said yesterday organizations like ours find a really simplified agile user story a particularly effective way to communicate these principles; it's a format people are already intimate with, practiced at delivering against and I personally like the fact that the basic clauses (as a... I want to... so that... as proven by...) force a habit of thinking about the business value of an idea rather than just raw requirement.

So how would a user story for maintainability look?  Building on how we decided to tackle the "as proven by" clause yesterday let's throw it in right away:

As the CTO, I want to govern a set of products benefiting from continuously improving feature time to market and able to be fully maintained online, So that our delivery capacity becomes more consistent and efficient, As proven by appropriate and justifiable answers to the maintainability questions.

Again keeping in mind the keywords appropriate and justifiable; you'll have to keep in mind the expected lifetime your application will have (yesterday we talked about seasonal features) and what it's future roadmap is; how many more features are on the horizon?  How is it likely to be extended?  Are the windows for maintenance that do not affect revenue?  Not everything needs to be hot maintainable but you need to be able to modify it quickly and easily when you do.

These are the sorts of things do you need to ask yourself ask you work:

• How will I upgrade this hot?
• How will I add features hot?
• How will changes be rolled back without tattooing?
• How can I run multiple versions of my feature concurrently?
• How have I isolated the feature to limit the scope of future changes?
• How have I limited the manual effort required to deploy?
• How will I add or reduce capacity online?
• Is all my functionality exposed via an API?
• How have I prepared my feature for quicker future time to market?

Having sensible answers to these questions will help make sure you've prepared your product the best way possible for future development.