Pareto Principle in Technology

I first heard of the Pareto Principle when I was at university studying economics. It was used in reference to wealth distribution (which was what Pareto was studying) where it had been observed that 20% of the people controlled 80% of the wealth. Over time more studies showed the same sort of distribution.

With respect to technology it is a handy rule of thumb that while not exact gives you something to go from when thinking about issues. A good recent example are analytical reports such as from AdMob of the applications on the Apple online software store (where Apple boasts of the tens of thousands of applications) wherein it was found that a small number of the applications account for most of the traffic and most of the profit. Various limitations (such as a set top 10 list) feed into some of this behavior but largely it boils down to most of the applications having very little value, leading to few people recommending or using them.

In software development the Pareto Principle also rears its head. In iterative approaches like Agile you work on a feature list from a priority list, with the highest value items first, in small batches. Rather than trying to deliver all 100 pieces of functionality that you would like to see you deliver them in batches of five or so and because the first ones being delivered have the highest value you might get 20% of them done (20 features) and have most of the benefits that you want, rather than taking a lot more time and maybe missing the mark with a lot of the work. Delivering sooner rather than later, even if it is a small amount of functionality, also plays into this approach.

Studies by large companies like Microsoft find results like fixing the top 20% of the bugs (by frequency of complaint) results in 80% percent of the errors and crashes going away.

Many examples can be found all around us of the Pareto Principle at work. As far as technology goes it is a good idea to keep it in the front of your mind when you are deciding how you should approach your projects. What is the smallest amount of features / product that you can deliver that provide the biggest bang? Grab this low hanging fruit and get something out rather than trying to design the ultimate application that does everything. You may find that after a few iterations that what you have is good enough and you can then move onto another project and find more success rather than continuing to try and reach a shifting and unattainable goal.

Smart Phones

Ah - a couple weeks of downtime always refreshes.

Lots happened in the last couple of weeks(ish). One was the new iPhone came out. A friend of mine is agonizing over getting one, versus a Blackberry or something else. He believes that government and corporations are all sewn up for Blackberry.

For government - agreed (for now).

Corporate - it's shifting, and shifting rapidly.

With the iPhone OS 3.0 release there is now tighter integration with Exchange servers (really the thing that made corporate folks ga-ga for Blackberry in the first place) plus security items like the ability to remotely wipe a handset.

In many companies that I directly know of the upgrade cycle (starting at the top) is resulting in iPhones instead of Blackberries (execs first, of course). Governments will take longer to make such a change, if they do, and again it will start at the top, with the peons getting the goodies last (naturally).

Me, personally, I do not talk on the phone very much, and if I do it is from a land line. I use a pre-pay phone for the occasional voice call and use a PDA for other functions. When my current PDA (a Palm) dies I'll get an iPod touch (seeing as Palm got out of the PDA business).

If you're going to pay the monthly bill anyways go with the iPhone. Blackberry will probably be what you're provided with by your company.

That said, the market keeps shifting, with the Palm Pre now released and decent UI having been grafted onto Android by HTC.

As always, make a decision based on your own needs and not what the market blares at you.

What kind of a database do I need?

So, we covered the need for one. Chances are you need one. But which one? First we have to think a bit.

There are some questions to consider the answers to:

• What kind of data am I storing? Is it just text and numbers, or do I have to store files or picture? Do I have to store documents?
  
• How much data am I storing? Do I have a few dozen customers and a couple of suppliers, or do I have hundreds of both? Do I have historical information I need to put in, and how much of it do I have?
  
• How many people need to get at it? Is this just for me, or do other people need to get at it also?
  
• What other requirements do I have? Speed, costs, etc. Does the system have to lightning quick or can you wait a second or two? Do you have a budget to spend on this, both for software and somebody to handle it?
  

There are several broad categories of alternatives.

• If the amount of data is small, simple and textual you could get by with a text file. An XML file can easily hold a fair amount of structured data and does not require very powerful software.
• If the amount of data is larger and maybe a bit harder to manage you might need a spreadsheet like Open Office Calc, Google Doc or Microsoft Excel.
• If your data needs are larger you start to get into actual "database" software, of which there are many different options from free to very expensive. The choices stretch out a great distance, but include smaller ones like Microsoft Access or FileMaker or ones that will scale out as needed like MySQL, PostreSQL or Microsoft SQL Server (various editions).
  

In each case the number of people need to access the data will drive what you wind up with. Think about your needs before buying what people are selling and come back to determine what kind you need. Answering these questions will help you be techgnostic in your choice.

Reduce, reuse, recycle.

Being green is seen as increasingly important for companies. Sometimes it's a stretch to find something to trumpet about but you can always find something like using recycled paper in your printers or such. If the new measure actually saves the company some money (not often), all the better.

One place that companies could definitely get more bang for their green buck is with computer hardware. The hardware itself caused a lot of pollution during its manufacture and when it winds up in a landfill it leaches a lot of nasty chemicals. If you can avoid having to buy new computer hardware you can feel better for being green as well as save some money.

Most often the things that will fail on a computer are the parts that move: the hard drive, any fans in the case and the power supply fan. Otherwise, in the absence of a power surge, the rest of the solid state electronics will keep going for years (at least five). It used to be that software always outpaced hardware and you always had to buy new hardware to keep up. Hardware is now ahead of software in most cases (unless you're talking about niche needs like high-demand video games or such). With some more RAM (inexpensive) and maybe some new parts that you interact with (monitor, keyboard and mouse) you will feel like you have a new computer.

If your needs are not directly tied to software available on Macintosh or Windows you can load one of the freely available Linux distributions and get your computing tasks done very handily, while also saving some money and making your computer more secure. If you are not comfortable with such a move you could just back up your data and do a clean re-install of your operating system and again feel the joy of a new space. If you are 100% uncomfortable with either of these suggestions you can go and buy yourself a new computer but keep in mind that the prices are much lower than they used to be and you don't need a lot of power for most things you're going to do.

As always, you need to think about what it is you're trying to do when you are considering a hardware purchase. Do you want what you're getting just because you're "due" for some new hardware, or do you have a specific need for it? Are you tied to a particular software platform? If not, when you get new hardware is an ideal time to take look at alternatives. The computer market is in flux right now and there are lots of new approaches being tried.

Obviously everyone's situation is different, but what I am advocating is not necessarily just going on to the next revision of what you've always done, instead taking a look at alternatives, including keeping your existing hardware and just refurbishing it a bit. You'll save some money and help the earth by being green.

The hard stuff is easy, the soft stuff is hard.

Technology, of whatever type, is a tool and that is all. Outside of enjoying a tool purely for its own sake (tool-o-phile?) there is no value to a tool except what you do with it. You don't buy a drill, you buy holes. A shovel might represent a crystallization of effort with a forged head and carefully turned handle, but it is just a shovel. Until someone picks it up and digs a hole with it there is nothing to see, move along.

"High" technology needs to be viewed the same way. It can become VERY complex and challenging to understand, but ultimately it is just a tool. A database without data in it is an empty shell. Even when full of data it means nothing until you DO something with it. Transform the pieces of data into information that you can use to make decisions. Again, the technology by itself is nothing. It takes people to use it to get value from it.

Technology is for most people the "hard" stuff. People skills like communication and collaboration are considered "soft" skills. Depending upon your personality type you might gravitate towards one or the other. In the end, though, the effectiveness of whatever your enterprise is will rest upon the behavior of the people using the technology. In most cases the hard stuff will come with a default setup that works in most cases, and if it doesn't you can find someone who can make it work for your situation. Really, in the grand scheme of things, that was the easy part.

The "soft stuff" is where the real challenge lies. Many technology people consider the soft stuff relatively worthless as it does not improve the speed of their hardware or software. It does not help directly with their up-time. It does not give them more disk space to store data. Very often they will gloss over the people-oriented items or leave them to others.

This is a mistake.

If you don't take into account who is going to be using your system you will fail. The end users of the system are most often not technologists. They are people who want to use a tool to get their job done. They want to get in the car, push the gas pedal and drive somewhere. They don't want to have to know any detail whatsoever about how the engine works, how the linkages between the control systems ensure that the vehicle goes in the correct direction, etc.

Gas, go, done, next.

You can build the most technically elegant system in the world but if you don't take into account the people who are going to use it you are dooming your efforts. This is true for systems that are internal to companies, or external to be sold to others. You can mandate usage internally (you WILL use the new system) but it will never be accepted and unless you have a monopoly you have no hope externally.

If you find yourself looking at a technology decision be sure to take into account the soft stuff. Who is going to use it? What are they going to do with it? What is the LEAST that you can do to make the system work? Systems do not exist for their own sake. They exist to serve people in some way. Keep that foremost in your mind and the decisions about what to do should be clearer.

Process

Process is kind of a loose word, where different people mean different things by it. When a process is broken, though, just stating that it is broken is useless. You have to describe WHAT about the process is broken, and why. Usually it is not very difficult to discover, but the piece that is broken is often there in order to make something easier for someone. It was not seen as a breakage, but just a short cut. Sometimes you can get away with one of these in a process, particularly when the process is long-running.

This is deceptive as people think "Well, that one little deviation did not hurt, one more won't hurt either." Soon the original deviation is one of many and the process breaks. Often there is a finger pointing exercise wherein everybody trots out their favorite theory of what went wrong, but the process remains broken. The deviations often have become the practice in the organization and people are invested in it.

Often an outsider is brought in to help "fix" the process.

Being an outsider, this person usually immediately sees where the problem is and tells everyone exactly what it is. Either the person is thanked and process is put back to a working state, or it is decided that the outsider does not "get it" and cannot understand the intricacies of how the organization works. Regardless of which result comes out of it, we can rest assured that the process will drift away from the ideal at some point. So, what can one do about it? Usually you have to have someone dedicated to a concept like "quality" where the process that is followed is periodically reviewed, and someone is held accountable for the worthiness of the process.

Many times this sort of function is seen as pure busywork of the worst kind, and a waste of effort. The time wasted every time the process breaks and profits or customers are lost more than make up for any investment in quality, but people will still make the short term decisions to save a few dollars.

In technology ventures, this whole "process" dance is even more vital as the processes are often automated and can occur thousands or millions of times before it becomes apparent that it is broken. Ironically, because of the fluid nature of software, process is given even shorter shrift than in real physical processes because you can often kludge something together to get to a short term goal. Inevitably you will pay for it later, with interest.

If you are involved in a technology project, regardless of vendor, platform, etc. be sure that you can describe your process for solving your problem in language that anybody could understand, and then be sure that you stick to it. I'm not saying never change the process, as they will evolve to meet changes, but if you cannot explain what you do, or how you accomplish your goals you are doomed to waste a lot of time and effort that could be used to help create more value from your technology investment.

So where does "techgnostic" come into it?

From my title block:

• gno·sis (no-sis) n. Intuitive apprehension of truths, an esoteric form of knowledge.


• ag·nos·tic (ag-nos-tik) n. One who is doubtful or noncommittal about something. A skeptic.


• tech-gno·stic (tek-nos-tik) n. Intuitive apprehension of technology truths, a skeptical take on technology knowledge.

Why append my name with such a title?

I have worked over the years on many flavors of Unix (Sun, HP, SG, BSD and Linux), different incarnations of the Mac OS (from when the whole OS ran off a single floppy in my Mac 512e) and DOS/Windows since the XT. There are zealots in every camp, and strong reasons for their biases. Untold billions of characters have been typed into computers during flame wars about the "best" OS, programming language, editor, database, you-name-it. There is only one thing that can be said for all of this.

All computers suck, equally. I'm not the only person who thinks so.

A bit glib, I admit, but nice and pithy and with enough truth in it to allow it to stand up. Each "platform" of hardware / software / applications was created to solve a particular kind of problem and usually accomplished its mission. When an attempt is made to repurpose it to another task it often fails. From a certain perspective, any other platform equates to "sucks".

So, beyond appearing to be a technological dilettante, so what?

My own technological belief is to use whatever tools, platform, etc. make sense for your own situation. Again this seems like just giving in, but it is not. There are still fundamentals to keep in mind in terms of getting support, safeguarding your data, etc. but they can be enacted on virtually any platform. For all the flak that Microsoft Windows takes it can be locked down quite well. For all the raves that the Mac gets for style there is a closely controlled platform that stifles some kinds of development. For all the intellectual purity of most Linux versions there is still often a steep curve for the computing novice. For all the vaunted stability, etc. of the mainframe there is a prohibitive cost and lack of common support in the workforce.

If you or your company already employ a bunch of Windows admins and everybody is used to Windows there is not much point in having a Mac or Linux jihad because of some perceived superiority. It just does not make sense in that situation. Similarly, if you have a Unix setup and all is well, why upset the system to introduce a single Microsoft application when there are ways to integrate it into your existing setup?

All that said, strong arguments can be made for using standardized PCs, running open source software in "clean sheet" situations where low cost is an overriding concern, such as with the One Laptop Per Child project, or indeed for any other organization who only needs certain kinds of generic functionality, not tied to a particular software package.

The techgnostic part of all this is that in an ideal situation where the platform has not been dictated there are many options to solve your technology problems and we should not shut ourselves off into any one camp. The personalities and business practices of the players should not play a major role in our decisions - what works for us in our situations should be front and center.

That is being techgnostic.

Here we go again!

Apropos of nothing, I will once more put my thoughts to virtual paper, if only to give some place for my thoughts so that I can look back. Probably I will direct some people here also if I think something I've written is worth their time.

I started a blog in 2005 and made a couple hundred entries into it before petering out. The posts were a mix of random rambles and meticulous notes following the Hacker's Diet and my attempts to implement Getting Things Done. That blog fizzled for me, or at least my interest in it did, and had enough personal items in it that I decided to abandon it. It was also written with a pseudonym, and I think that encouraged me to be a bit more open with things I'd rather have kept to myself.

This time around I am going to be more "professional" in that I will not regale anyone with tales of my personal doings, and will stick to observations on technology, management, society, philosophy, etc. - bracing subjects, all.

As much as I'd like to be like Rands and have a cool hacker handle I am sadly lacking in that department. OK, I thought, I will use my real name. Alas, my name is far more common than I thought, so I went back to one of my first user account names and recycled it.

Despite not having a cool blog name, I am, however, going to take up some of his practices, including:

• The opinions expressed on this website are entirely my own. They do not represent the strategy/plans/thoughts of my employer, my family, my friends, my cat, etc.
• In some of my posts, I will refer to specific people as a means of telling the story... making a point. These people do not exist - they are fabrications that are specifically constructed to make a point. There are traits or quirks of former co-workers and friends that I borrow to construct my story persona, but these small slices of personality do not a person make.
• Lastly, I go out of my way to never borrow traits, ideas, or personalities from my current set of co-workers and managers. That would be bad form.

There, some ground rules in place. Some ideas are already churning.

So, in case anybody reads this, welcome to the blog!