What do I do with my data?

At this point in time there are two broad categories of approaches you can use to manage your data:

• Relational
• Non-relational

Since one is defined in terms of the other I'll start with relational.

Like the name suggests, relational data are items that can be related to one another through the use of a key value. For example, your government issued an identifier number to you that is used to identify you. This number is then linked to your name, where you were, born, etc. It is a key to reaching that information.

If you think of a table of information, with rows running horizontally across the page and columns intersecting the rows vertically, running down the page, the key value identifies the row. So, for information about person you might have something like this:

Key	FirstName	LastName
100	John	Smith
101	Fred	Brown
...	...	...
174	Zachary	Miller

Each key identifies a row, and each row has columns of information. Then, in another table where you store, say, order information, you do not have to enter John Smith's personal information every time you create an order for him. The data exists in one place. Then, if you find out that he spells his name "Jon" and not "John" you can change it in once place.

There are other advantages to the model, but this is the crux of it. Information only has to be entered once, and then related to other information.

A relational database management system (RDBMS) is software that allows you manipulate this type of information easily.

Non-relational is a grab bag of different approaches that do not include necessarily the idea of a key value (or at least not in a separate tables sort of way). Generally a non-relational data store will store all of the information for any one thing all together, all at once. For example, an order would be stored all on one line, with the customer information repeated over and over for each order they are linked to. You wind up storing the information more than once for certain items and you have a lot of work to do if you want to update that information or use it to link to information that is held elsewhere.

Storing information in a spreadsheet often winds up being non-relational but in more modern systems there are ways to make the data relate to other data. The lack of management tools, however, will generally drive you towards a management system.

If the data you are storing is VERY simple and small in size you can save time and money by just storing it into a text file that you update by hand. For anything beyond that you should consider more sophisticated software to help you with the process of storing and updating the information.

Weighing the pros and cons of the various different packages is a very involved process. Think through what your needs will be and then invest the time in some research. Make sure you base your decision on your needs and not on what some marketing tells you your needs are.

Make a techgnostic choice for your database needs and you will achieve your goals effectively and efficiently.

What kind of a database do I need?

So, we covered the need for one. Chances are you need one. But which one? First we have to think a bit.

There are some questions to consider the answers to:

• What kind of data am I storing? Is it just text and numbers, or do I have to store files or picture? Do I have to store documents?
  
• How much data am I storing? Do I have a few dozen customers and a couple of suppliers, or do I have hundreds of both? Do I have historical information I need to put in, and how much of it do I have?
  
• How many people need to get at it? Is this just for me, or do other people need to get at it also?
  
• What other requirements do I have? Speed, costs, etc. Does the system have to lightning quick or can you wait a second or two? Do you have a budget to spend on this, both for software and somebody to handle it?
  

There are several broad categories of alternatives.

• If the amount of data is small, simple and textual you could get by with a text file. An XML file can easily hold a fair amount of structured data and does not require very powerful software.
• If the amount of data is larger and maybe a bit harder to manage you might need a spreadsheet like Open Office Calc, Google Doc or Microsoft Excel.
• If your data needs are larger you start to get into actual "database" software, of which there are many different options from free to very expensive. The choices stretch out a great distance, but include smaller ones like Microsoft Access or FileMaker or ones that will scale out as needed like MySQL, PostreSQL or Microsoft SQL Server (various editions).
  

In each case the number of people need to access the data will drive what you wind up with. Think about your needs before buying what people are selling and come back to determine what kind you need. Answering these questions will help you be techgnostic in your choice.

Do I need a database?

"Oh, obviously you need a database."

What?

What is a database, you ask? All you know is that it lives on the computer somewhere and keeps track of things. Why can't you just keep your paper and files? Or are you already using a computer? "Look, see - I have it all in a spreadsheet!".

The person who told you of your need for a database rolls his/her eyes and groans and says "THAT is not a database!"

Confusion reigns.

So, what is it?

Strictly speaking, ANY collection of data is a database. Your paper and files? That's a database. A bunch of index cards in a box? Database. That spreadsheet of yours. Also a database. Any one of many different software products with the word "database" in their name? Yeah, database. Really, a database is an organized body of related information. Period. The exact mechanics of it are where people get hung up.

Honestly, the person who recommended a database to you most likely meant that you would benefit from a "database management system" (DBMS) such as has been around for many years, specifically designed to manage your information. Even though your stack of paper is technically a database how long does it take you to find a very specific piece of data out of it, say, an invoice from a particular vendor? When the pile is small it is trivial. As the pile grows it becomes non-trivial.

You've moved from a small pile / number of cards / number of spreadsheets. What happens when someone else needs to get at the same information? Any more than a couple of people cannot effectively search through your pile. And when the information must be updated what do you do? Mark up the original? Attach sticky notes? Put the file on a shared drive and hope for the best? And if you want to know some kind of aggregate information like "how much money do I owe all people with outstanding invoices?" you now have a much larger task on your hands. That is where a DBMS really comes into its own.

A DBMS has been designed, from the beginning, to efficiently store your information, with methods to capture, store, analyze and create outputs based on the information stored within. They are at the core of much of the efficiency that is attributed to computer technology.

For anybody who is doing anything in their lives that requires capturing information, storing it for future use, sifting and analyzing it and creating output such as reports or simply answers to questions about the data the need for a computerized database is clear.

However, just like the question of "do I need a computer" (most often "yes") is followed by "what kind of computer do I need?" there are some questions you have to ask yourself before you start down the path of using a DBMS. As the blog title implies we will cover this in a techgnostic fashion in the next post.

For a sneak peek, the questions will be along the lines of:

• What kind of data am I storing?
• How much data am I storing?
• How many people need to get at it?
• What other requirements do I have?

That will be followed by another post wherein some of the more popular alternatives are considered, as well as some less popular choices. Come to think of it, this might become a series of posts. Stay tuned.

The rise of the Netbook

Making computers smaller was a goal from the very earliest incarnations of the technology. Down from the size of a room to the size of a few fridges. Down from that to a largish box that could sit on the top of a desk. Soon someone asked "How about one I can take with me?" Portable (in the sense that they COULD be moved, not necessarily EASY to move) computers themselves iterated through various forms. Of course someone said "What if it ran on batteries?" and computers became truly portable, freed from the tether of the electrical grid (or a generator). The same evolution has continued over the years with portables becoming smaller and more powerful with each cycle.

Until very recently these smaller and more powerful machines commanded premium prices. You could expect to pay much more than you'd pay for a desktop computer with similar power. The intent was that the portable computer should have the same power (or close to it) as a desktop. This meant all kinds of trade-offs had to be made and components had to be engineered to be even smaller. All this engineering did not come cheap and having a laptop computer was a badge of honor and the province of the technological (or just wealthy) elite.

Eventually a kind of tipping point was found, not unlike with cell phones. In the same way that the huge cell phones of the 1980's become smaller and smaller into the 1990's so that design decisions started to rotate around making the keys LARGE enough for human fingers to operate as the phones had become so small laptops started to change also. Two kinds of consumers were looking for portable machines, with two sets of criteria: as SMALL as possible (most often frequent travelers and/or gadget freaks) versus as POWERFUL as possible (the so called "desktop replacement" crowd). Each of these consumer types was provided with what they wanted, but in all cases you continued to pay dearly for the machines.

As time and Moore's Law marched on an intersection occurred. This was the intersection between relatively powerful laptops and the ubiquity of wireless internet. Again something that was once the province of the technical elite became very common, with wireless access available virtually anywhere for a very small fee, if at all. Instead of existing as an alternative to a desktop computer the laptop became basically a terminal to connect into the content of the internet. The need for the terminal hardware (i.e., the laptop) to be powerful was lessened. Beyond a certain amount of power was a waste. A simple machine could be used to read e-mail, surf the web, watch videos, etc. The hardware of a new type of laptop was ready. The other part necessary was the software.

Open source software, which languished for some years in obscurity from the public, leapt to the forefront of these new machines, dubbed "netbooks" as they are basically used to access networks and not much else. Rather than having to pay the "Microsoft tax" for every machine netbook manufacturers started to load open source operating systems like Linux onto their machines, erasing the extra payment. This dropped the price a little bit more, along with the smaller screen, small hard drive and lack of optical drive to produce a netbook for a few hundred dollars instead of almost a thousand dollars.

The effect on the market was slow to build but once it did it ran away, catching the large manufacturers unawares. Coinciding with some economic hard times netbooks flew off the shelves and accounted for most of the growth in the sales of portable computers. Now all the major manufacturers have their own netbooks out and Microsoft is scrambling to try and find a way to ensure that they are a presence in the market, albeit at a lower per unit cost as in the past.

For many people a netbook is all the computer that they need. They can use a web based provider of e-mail, store their pictures on a photo sharing site, use online applications for word processing and generally do what they need to do without very much regard to the computer that they do it on. If their inexpensive netbook bites the dust the only thing you might need to do is rediscover your bookmarks (although even those can be cached away by one of many online utilities).

There will always be a market for the more powerful and expensive laptops for people who need the power of local processing for tasks like video processing or engineering that burns up a lot of CPU cycles, in the same way that there will still be people who will spend thousands of dollars on desktop machines to do very specific tasks. In both cases, though, there are now inexpensive alternatives, often using open source software, that give people just what they need for a small amount of money.

Keeping in mind what you're trying to get done before you choose your technology is a techgnostic thing to do. Think it through the next time you advise somebody on what kind of computer to buy or examine your own needs the next time you put your money down.

Reduce, reuse, recycle.

Being green is seen as increasingly important for companies. Sometimes it's a stretch to find something to trumpet about but you can always find something like using recycled paper in your printers or such. If the new measure actually saves the company some money (not often), all the better.

One place that companies could definitely get more bang for their green buck is with computer hardware. The hardware itself caused a lot of pollution during its manufacture and when it winds up in a landfill it leaches a lot of nasty chemicals. If you can avoid having to buy new computer hardware you can feel better for being green as well as save some money.

Most often the things that will fail on a computer are the parts that move: the hard drive, any fans in the case and the power supply fan. Otherwise, in the absence of a power surge, the rest of the solid state electronics will keep going for years (at least five). It used to be that software always outpaced hardware and you always had to buy new hardware to keep up. Hardware is now ahead of software in most cases (unless you're talking about niche needs like high-demand video games or such). With some more RAM (inexpensive) and maybe some new parts that you interact with (monitor, keyboard and mouse) you will feel like you have a new computer.

If your needs are not directly tied to software available on Macintosh or Windows you can load one of the freely available Linux distributions and get your computing tasks done very handily, while also saving some money and making your computer more secure. If you are not comfortable with such a move you could just back up your data and do a clean re-install of your operating system and again feel the joy of a new space. If you are 100% uncomfortable with either of these suggestions you can go and buy yourself a new computer but keep in mind that the prices are much lower than they used to be and you don't need a lot of power for most things you're going to do.

As always, you need to think about what it is you're trying to do when you are considering a hardware purchase. Do you want what you're getting just because you're "due" for some new hardware, or do you have a specific need for it? Are you tied to a particular software platform? If not, when you get new hardware is an ideal time to take look at alternatives. The computer market is in flux right now and there are lots of new approaches being tried.

Obviously everyone's situation is different, but what I am advocating is not necessarily just going on to the next revision of what you've always done, instead taking a look at alternatives, including keeping your existing hardware and just refurbishing it a bit. You'll save some money and help the earth by being green.