Successful technology startups are usually those that hit the market in a sweet spot — where market conditions create significant demand just as the startup is introducing its product. From the look of the rapidly-consolidating hard drive business, it might appear that I’ve missed the sweet spot with the metal foil disk technology some readers may remember I’ve been working on for several years. Hopefully not. But in any case it is probably time for an update.

When I started down this path toward a drive that uses very thin metal foil instead of polished glass there were several potential customers. Then Western Digital and Seagate started buying their competitors until now there will shortly be just two hard disk companies that actually make their own products. Toshiba may continue on its own course, but Toshiba contracts its manufacturing to others, while Seagate and WD are vertically integrated, making almost everything in-house.

Some of this consolidation can be attributed to the growing success of solid state drives based on flash memory technology. That’s certainly the case with Samsung, which is selling its hard drive operation to Seagate and further expanding its flash business in the process. But hard drives are far from dead. The best technology roadmaps I can find suggest hard disks will remain dominant in PCs and data centers alike until 2025. So I keep telling myself there is plenty of time yet to deploy my technology.

Just because a platform is likely to survive doesn’t mean that it will do so without change, and change is one of the things that has hurt my foil drive. We focused all our work on the 48mm (1.8-inch) form factor drives common in media players and netbooks just in time to see those drives scheduled for retirement as the bottom end of the market gives way to flash. Uh-oh. Back to the drawing board.

What’s interesting about this impending demise of 48mm drives and how I made the wrong call is that by conventional thinking they shouldn’t be going away at all. We always need more storage space in our devices and flash is still too expensive to compete with 48mm on a bit-to-bit basis. What I failed to anticipate (and what killed me) was cloud storage.

For the first time, new media players, netbooks and notebooks generally have less internal storage than did the previous hardware generation. That saves money for the manufacturers but, more importantly, those same manufacturers are further leveraging this deliberate storage deficit to push their own cloud storage services. Who needs a lot of storage on your iPod if you can keep all your movies in MobileMe?

And that cloud storage will all be on hard drives, of course, so we’re back in business, though at a larger form factor.

All hard drives will shortly be either 2.5-inch or 3.5-inch and my sense is that in the long term (that is between now and 2025) the 2.5-inch form factor will prevail. Downward cost pressures will make it logical to standardize on a single physical size and 2.5-inch drives will fit in smaller devices like set-top boxes where 3.5-inch drives are already too big.

So I am aiming at the new sweet spot, the 65mm (2.5-inch) foil drive.

What’s taken so long is that making this stuff turns out to be very hard to do. You have to find the right alloy (more difficult than you’d guess), get someone to make it for you at a purity better, frankly, than the world has ever demanded before, then figure out a way to roll, stretch, punch, deburr, polish, clean, and package it, with each of those stages worth a startup in its own right.

Just the purity part can be difficult for many folks to get their minds around. In this case we’re talking about not just the precise alloy, but also about occlusions — particles within the alloy. Occlusions are chunks of metal that deserve to be in there but ideally would be more completely mixed in the alloy matrix. Typical occlusions might be tiny grains of chromium or cobalt. The trick is that occlusions have to be smaller in diameter than the thickness of the foil so they don’t mess-up the magnetic environment on both sides of the platter at once. If your disk is 30-microns (0.03mm or 0.00118110236 inches) thick, then occlusions have to be guaranteed less than 30-microns.

Try getting that from a company best known for making cannons.

This is not only a pain in the ass to accomplish, nobody had even requested it before. So when you show up at the door of a giant steel company asking them to do just a few kilograms of some obscure alloy at a purity level beyond anything they’d ever even imagined, don’t be surprised if they laugh in your face.

If you can get the right alloy at the right purity, then you have to figure a way to make it super-flat. This is traditionally accomplished through progressive rolling then stretch-leveling then polishing, again to a standard never before required for a material that is also very thin. It takes years and kissing lots of frogs to find vendors who can do such work.

Even then you still have to clean the disks and package them for sputtering. This involves not just special cleaning machines that have to be built, but also designing packaging to protect the disks in transit yet can be opened at the disk drive factory without damaging the very fragile disks.

And that’s all it takes to change the world.

At the bleeding edge of technology there is always a question of whether it is worth the effort. At each platform transition is it better to throw money and energy at being the best of the last generation or the first of the next generation? This question is embodied in the whole disk drive versus flash drive competition. I had decided already to continue with hard drives. But that doesn’t necessarily mean everything has to be made using refinements of techniques that were in some cases developed centuries ago.

We proved by spending a lot of time and money that the roll-stretch-punch-deburr-polish-clean-package process could be done at a level comparable to current glass disks. But what happens next year when our customers inevitably demand even tighter tolerances?

We’d be screwed.

While what we wanted to do was possible it wasn’t practical. Operating on that bleeding edge it was becoming clear that most of the blood would shortly be ours. It was time to either give up the dream or make a quantum leap in manufacturing what many already viewed as a technology that was itself obsolete.

So I of course opted to build my Model T using nanotechnology.

Our new process builds foil disks one atom at a time, completely eliminating every step of the previous process. The new process comes down to: Step One — remove finished foil disk from machine. There is no Step Two.

With the new process there are no occlusions and the disks are absolutely, perfectly flat. The disks can be made of any alloy ever imagined or some new alloy we invent. And here’s the really cool part: while we’re making foil disks why not make the magnetic recording layer, too? No more sputtering.

Every stage of a manufacturing process has some units that can’t pass inspection. Our new process collapses dozens of stages and their associated wastage into a single stage.

It’s all or nothing.

Now to prove we can do it 800 million times per year.