Before you read on and jump quickly to scream foul, please make sure you realize that I write this with some humor, to try and get at the point that “In certain situations, de-normalization is good!” I am not suggesting that we throw away normalization!  Are you kidding me?  Might as well go play Russian Roulette!  However, I think this is an interesting trend among large Web 2.0 companies that are heavy on read, and low on writing.  With that said as my preface, read on and joy.

I’m just going through the process of designing a database architecture for a Web 2.0 site we will be launching in a few months, and the first thing the database architect talked to me about at a review was “Don’t worry, this design your about to see is going to rock, we have really normalized the data, its hot!”.  I was happy he said this, but at the same time, it got me thinking about a lot of things.  Synapses started flipping through my brain as they do at night when I can’t sleep (probably due to the Starbucks I was sipping on).

See, I have never seen eye to eye with database designers on their “Normalization” Zen.  When I first started getting into databases and teaching database design and architecture, I always thought to myself “Why all this normalization?”.  I know, I’m treading into dangerous territory here, but I feel it has to be said.  What is dangerous about this topic?  I guess the fact that I think we should NOT normalize data for the most part in a Web 2.0 world.   Inexperienced DBA’s I guess will find this completely incorrect and question my brain capacity, but I would bet experienced DBA’s after thinking about what I’m saying, would agree with me.  I’ll tell you why…

The Argument Pro Normalization

I won’t go into too much detail, because all of you know that a proper database is normalized.  My guess is at least 80% of web sites out there are running on normalized relational databases.  In the current world of relational databases, we have all been taught that you must normalize your data to ensure data integrity by minimizing duplicate data.  By doing this, you are ensuring that no application can write data in one location, and not updating another location, and creating two different records, for the same data!  This is supposed to also simplify software development, by putting all the data integrity logic at the database layer.  You can user stored procedures, views, joins, and have fairly applications that rely on the database to keep the data integrity.

The Argument Against Normalization

How many freaking joins do you have to do to get all the data you want?  When I first started my career, I could write ASP pages amazingly fast, with simple databases in the back end.  As I started learning more and reading on “Normalizing” your data, I started feeling crappy about some of my designs.  I didn’t use any joins, didn’t really have any transactions, and had very few tables.  But, I wrote applications very fast, got results, and never really had any speed issues.  I never really had issues bad data either, but whatever.

I Learned to Normalize

I started writing more “enterprise” applications, and had to do more joins, left, outer joins, had to worry about transactions, and had layers upon layers of data access to go through in order to add a new customer.  I was told that “It’s the databases job to maintain data integrity”.  Ok it all made sense, I did this.  My code became way more complex, my database designs became way more complex, and you couldn’t put bad data into the database.  Hot!  But, it took FOREVER to write the applications to put the data into the database.  Not only that, but now, if I needed to scale the application, I couldn’t simply add more web servers!  I now had to worry about database replication, transactions, and indexes.  I soon realized that doing joins took an INSANE amount of processing power.

Normalization Is Slow To Design

Are you still with me?  Splitting up your data into small chunks and insuring that no data was stored twice, made writing even the simplest web forms super complicated.  Sure I could call a stored proc, but if it wasn’t written, I had to get the DBA to write it, or write it myself.  If I had to get new data in a certain way, I had to create views, which I was told was fast!  Not (Ok, getting better… but still).  For one simple form, or simple display page, you could have an INSANE amount of joins!

Normalized Applications Are Slow On A Huge Scale

Why couldn’t I just have a table, with all the data I needed for that page?  Then for other pages, I could have the same data duplicated!  This would actually take stress away from the JOIN, and just let me get data from either location.  When an update was done, say the user updates his address, it would be updated in two locations.  Is that all that bad?  I don’t think it is.  Just add some constraints, and make sure the application layer follows the constraints.

If Web 2.0 is all about sorting data, indexing our knowledge, sharing our knowledge, speed in development, speed in viewing Web 2.0 applications, and scaling your sites quickly should you’re site become the next YouTube, I challenge you to do this successfully with a normalized database full of joins, stored procs, views, and transactions.  You might do it, but you will need an army of rocket scientists to do it.

Final Thoughts

Do whatever works for your environment.  In an enterprise environment where you are working with financials and maybe have 1,000 people entering data and reading data, by all means, normalize your data.  If you are writing an application in the web world, where one day you might have 100 users, and a month later, 10,000,000 users, you better have a quick way of scaling!  If you have kept your database de-normalized, and just have tables, and your database is basically a storage area, and has no business logic whatsoever, you will be able to quickly scale your web application by adding servers, replicating code across your web farm, and even putting in cool appliances like BIG-IP in place to cache data and load balance your web farm.

Good luck doing this in a normalized database.  You will be able to, you will just be crying the whole way.

So, What Should We Do?

If you are doing more writing than reading, stick to normalized databases.  If you are doing more reading than writing, still normalize… but then denormalize your data!  Look at the web forms and pages you will have, and see what data will always be displayed, and if you have 18 tables needed to display a users profile, denormalize the data!  Imagine what would happen if you have 10,000,000 users reading profiles and running joins on 18 tables.

One of the Reasons for Web 1.0 Failure

I would argue that a big reason Web 1.0 companies failed was because you had an army of people getting money from VC’s, getting expensive DBA’s, armies worth, to write complex normalized databases, then getting web guys to create the application layers for these complex databases.  Why is this the reason for failure?  You had slow performing sites that didn’t scale.  You had tons of bugs, bad performance, and sites and ideas that never even launched because they were “in development” forever!  In Web 2.0 you have things like Ruby on Rails, AJAX, and companies like eBay that rely on denormalized databases to scale.  You have good looking, fast performing, scalable web sites that are built fast, and not on overly normalized databases.

If you have read this far, I should share some sites with you that give me some legs to stand on for my arguments:

• Normalization is for Sissies
  Pat Helland – Database Guru from Microsoft, also had a stint at Amazon.
• Data normalization, is it really that good?
  Arnon Rotem-Gal-Oz

Let the comment bashing on denormalization begin!

I’ve conducted hundreds of software development interviews over the last few years, and been interview more times than I can remember as a consultant.  My wife is now an HR Manager and a recruitment consultant, so I’ve been spending the last few days contemplating what goes well in an interview, and what goes terribly bad. 

This posting is an attempt at helping software developers land that awesome job they deserve!  In an upcoming post which I have been working on and editing, I will focus more on the actual technical side to the interviews. However, to be honest, at least 80% of the reason you will get the job, or lose the job, is covered in the tips below.  The other 20% is your technical Ability.

The Secret

There is one thing that you have to know, that will improve your interviews instantly!  It’s something that some of you know and think everyone knows, but sadly, 90% of you have no idea!  What is the secret?  It’s simple.  The secret is that when you come in to see me for an interview, you already have the Job!  I want you to do well!  The job is yours to lose!  So be confident and strut your stuff.

1. Arrive Early

This is an easy one, but for some reason, some of you insist on letting me weed you out before you even let me start the interview.  No amount of excuses is going to save you if you arrive late for an interview.  Arrive early, that’s all I’m going to say about that one.

2. Dress Professionally

I’m expecting some flack on this one, but honestly, dress with at least dress pants, dress shirt, and if you want, a tie, or suit jacket if you have one.  Now on the flip side, one thing you could do is do your research on the company you want to work for, and see how they normally dress.  If you are applying at a Web 2.0 company that has everyone dressing in blue jeans and Abercrombie t-shirts, by all means, dress the part. 

All I’m saying is dress it professionally and clean.  Clean shaven, nicely cut hair, etc.  All these things help.  For example, even if it is a casual work environment, for the interview, wear some nice jeans (no rips), a nice casual dress shirt, nice belt, put on a blazer, and wear some dress shoes. Ok, now that I said my peace on the dressing professionally, let the flaming begin lol.

3. Show Your Passion for Software Development

If you read my blog regularly, you know passion is something I’m pretty much obsessed with.  If you are one of those developers that stay up all night long thinking about the solution to that insane software development problem at work, I want you to work for me!  Do you try and read every blog you can about software development, and stay up to date with everything you possibly can on technology, I want you to work for me!  Do you genuinely love software development?  Well?  I’ll tell you what, if you do love it, it’s going to show through!  It’s hard to hide! 

Every time you answer an interview question, your face will light up because you know the answer.  Every time you talk about a past experience, you voice will speed up, and you will forget you are in an interview for a few minutes.  Don’t be afraid of this!  Let is shine through!  It separates you from the average developer that is in it for the money or the job.  And you better believe that YOU are the one I want on my team.

4. Show You Like Them and The Company

During the interview, you should try and say nice things about the interviewer and the company.  I’m not asking or suggesting that you "kiss up" to the interviewer and the company, but what I am suggesting is:  If you want to work for the company, think it’s an awesome place to work, love their offices, love the personality of the interviewer, and love what you have heard from other people about the company, say so!  Honesty will get you far!

5. Avoid Discussing Salary and Compensation

Whatever you do, do not bring up compensation.  If it is brought up by the interviewer, ask it quickly, and move on.  Do not stumble on this question, or be the one to bring it up.  Once you have been offered the job, or just before you are, you will have a chance to talk about the compensation.  It should be avoided at all costs in the first interview.

6. Answer Questions From Experience

This goes for the personal and the technical side of the interview.  A lot of times I will specifically say "Thinking back on your experiences…", however if the candidate can bring up examples on his own, I usually start getting excited and start leaning towards giving the candidate some points.  If I ask a question like "What is Multi-Threading?" or "What is the difference between Finalize() and Dispose()?", don’t just answer the question like you are the human programming dictionary.  Instead, impress the interviewer by giving examples!  Start your answer with "It’s funny you brought that up, last week I was working on a project where we has to use Multi-Threading!  We had a situation where…" and go from there.

Not only are you answering the question, but you are also showing me that you have real world experience in working with the concept or technology!  Also (yes, there is more!), you are starting a more human like conversation with me, and we are talking and chatting!  You will become more comfortable, and I will also relax and get excited!  I will start seeing you on my team!

7. Ask Good Questions

When it comes time for you to ask questions of the company and interviewer, make sure you have some great questions!  Although it may seem like a great time to get to know the company, which it is, this is one of those rare golden opportunities that you have to put yourself over the top of probably 95% of every other person the interviewer has interviewed for your position.  Ask them about their quality assurance process, what kind of source control they use, if they use continuous integration, what they use for unit testing, etc.  In a future posting I will try and focus on just great questions to ask in an interview.

8. Do Not Be Negative

You would think nobody would have a negative attitude when going into an interview.  You would be shocked!  While you’re in the interview, take extra caution in not being negative.  There are a lot of people that fall into this trap.  One of the most common questions that gets a lot of potential software developers is "Why are you looking to leave your current employer?".  I don’t really want to hear things like "My boss was an *** ****", or "I was never given the promotion I wanted, it really pissed me off."  Even though those things could all be true, do not answer this question this way!  Be careful.  The better way to answer this question is "I’m looking for new challenges, learn new things, and I have learned so much where I am now, I’m ready to push myself further".  Saying negative things puts negativity into the air, and it will be there when they think about you later!

9. Constant Eye Contact

So many people have lost me here because they keep looking at their shoes, and not at me.  Even worse, sometimes there are a few people in the room and you need to look at everyone!  What to do?  Look at everyone!  Alternate!  But whatever you do, do not look at one person, and nobody else.  Or even worse, do not ignore one of the people, because you think they are less important.  A few times I’ve had interviews where I have a few of my senior developers in the room, and the candidate doesn’t even bother to look at them, just constantly looking at me!  While it can be flattering (I joke), you are demonstrating to me that you play sides, and view certain people higher than others. 

10. Do Not Appear Overconfident and "Know It All"

I’ve had so many interviews where the candidate was amazing, and I wanted to hire them so badly!  But they blew it!  How?  They were over confident. Every question I asked they answered with some attitude like they were saying "please don’t bother me with these silly questions".  Answering the question of "Name a time when you made a mistake or a misjudgement in timeline, and were going to be late on a milestone.  How did you handle it?" with "My estimates are always perfect, I have never been late, and I have always met all my milestones." does not impress me at all.