Several years ago, I rented a small house on the San Francisco Bay peninsula. The landlord was a very nice person, but she didn't want to spend a penny more than she had to for anything. In the mid-90's, she built a large house and installed the cheapest double-pane windows she could find. Typically, good windows that are properly installed last at least 25 years, but most of the internal vapor seals on the windows that she installed had broken in less than 10 years. The result was moisture and fogging between the panes that was impossible to remove. She was thinking about selling the house and knew that she could never get a good price with those windows, so she had to replace all of them.
There are times when you should spend money now in order to avoid having to spend more money later; the windows in that house are a good example. By installing good-quality windows when the house was built, the landlord could have avoided the hassle and expense of replacing them a few years later. However, there are also times when it's appropriate to spend the least amount possible building a "disposable" solution.
Software and Internet services startups have to make this same choice all the time: Spend a lot of money up front to get high-quality code that can be used for a long time, or go the cheap and dirty route and get something out that works but will have to be replaced quickly? The answer depends on what stage your startup is in. If you're just getting started and you're still trying to determine if the opportunity you've targeted is real and your technical solution will work, cheap and dirty is best. You're almost certainly going to throw out your code once, if not multiple times, before you're got the right product/market fit. Once you've got your product/market fit right, you can begin replacing "temporary" code with higher-quality, more maintainable code (I hesitate to say "permanent". because no code should be permanent.)
There are always situations where getting it right the first time is essential, especially in applications used for mission-critical or life-and-death situations. However, for most early-stage startups, cheap and dirty is the way to go, at least at the beginning.