Databases are all about I/O speed. If you believe that 20% of the data is 80% of the queries, then any database over five times RAM is too large to be fast.

In most businesses data over a year old shouldn’t matter. It shouldn’t be part of the 20%.

For OLTP, that’s almost true except MySQL is a really crappy database. So the ratio is probably more like 2x RAM. Real databases allow you to organize the table by any column, so you can put data for the same foreign key together, or even just sort by creation date. InnoDB organizes by primary key, period.

For reporting, it’s table scans anyways with a good data mart, but the organization part is even more important there so the scan is sequential reads. Even in the cloud, bulk sequential reads are a sweet spot.

Luckily, there are some real databases that have skins so they can look like MySQL, so hopefully I can wean work off using it.

At every company I’ve ever worked at we needed more automated tests.

The problem is that you can only do so much with Unit Tests in Cloud/Enterprise development. You need “integration tests”, but those become a huge pain in the ass. Some people point to “Mocks” as the answer. Except Mocks don’t seem to work.

Recently I read a blog post from someone about what he called the “NoMock movement”. I’d like to make my own contribution, because I believe in Cupcake testing instead of Mocks. Let me explain what I mean by that.

So most cloud/enterprise applications look like this:

Key ideas:

That’s what the Cupcake testing model looks like. The key bit is writing the to/from bit via SnakeYAML, but that’s not too bad because SnakeYAML can already follow collections and dump arbitrary JavaBeans. The only catch is that you need to prevent SnakeYAML from dumping out the entire database given one object. You can follow a customer to an order, and from there to a list of lineitems, because the order owns the line items. You can follow the lineitem to the product, because the relationship is to-one and safe, and from product to vendor because that’s a to-one, but you shouldn’t follow vendor back to products because that will end up pulling in the entire database! To write the dump code so it pulls in all relevant pieces of data but not the entire database you just need to have the following rules:

• By Default, follow all to-one relationships. It’s almost always safe to add one more objects to the pile of objects.
• By Default, do not follow all to-many relationships. It’s the to-many relationships that are dangerous.
• By Default, strip out any primary keys. That way your test cases are floating free as detached objects to start with and won’t conflict with other data. This is about a zillion times easier if your primary key columns all have the same name, i.e. “Id” or “uuid”, because you can just special case those in your SnakeYAML dumper.
• Have a method  of overriding the defaults  on relationships. That’s easy to do by merely loading a config file via YAML in your SnakeYAML dumper class. All that config file needs is a class name and the properties you do and don’t want to follow. i.e. com.chegg.Order.lineitems should be follow from my example above.
• In general, the follow rule is: follow relationships that imply ownership. Orders own lineitems, so its ok to follow them. This can end up finding problems in your data model because what you may find is that the ownership isn’t what you think. It’s natural to think of Vendors as owning their products, but actually, vendor is an attribute of a product; product owns vendor, vendors don’t own products. That seems counter intuitive, but that’s a topic for another day.

Well, that’s the Cupcake model. Let me know how it works for you.

So it was time to replace my much loved Honda CR-V. I’d owned it almost 12 years, I’d put 141,000 miles on it, but it had reached that age where there was going to be a succession of little hassles with it. It was a great car for me when I lived in Flagstaff, AZ in the snow, but now I’d given up on the “I’m so awesome I don’t have to live in Silicon Valley” delusion, and embraced the reality of “I’m awesome, but if I’m actually in the same room as my employees I can make them awesome too”. So having a snow capable vehicle wasn’t a priority.

So I started looking around. The first thing I did was ask my friend Steve, “What kind of car should I get?”. He’s the one that turned me on to the CR-V in the first place, and he’s a car/tech buff. I’m a computer person, but I’m not really a car person. I told him:

I have an 8 mile commute that takes me 30-40 minutes, so I figure I’m spending 22-32 minutes sitting at lights. So what I really want is a car that’s comfortable for sitting at traffic lights.

He suggested a Leaf, a Volt, or some kind of electric car or hybrid. I wasn’t totally surprised by this, because I knew that Steve already owned a Leaf. He actually pushed me more towards a Volt than a Leaf though.

So I started looking into it. The first thing I found was sticker shock. MSRP for a Volt is $40,000. I think that’s where most people stop. “I could get a BMW for that!” they think, and move on. But like many things about a Volt, its more complicated than you might think. I ended up figuring out that the Volt seems expensive but its cheaper in the long run.

The first thing to consider is the state and federal tax credits. In California, a Volt costs about $9,000 less, because the Federal Government will kick in $7,500 and the state will kick in $1,500. It’s in the form of a tax credit so you have to wait until tax time, but that drops the price down to $31,000. Before you bleed for California and their budget problems, remember that the state still collects Sales Tax of about 8.5% on the vehicle.

So at $31,000 we’re now out of BMW territory. In fact, we’re pretty close to the $27,000 it would cost me to replace my CR-V with a CR-V-EX.

To make it even more interesting, GM was offering 0% financing for 72 months on the Volt.

So now I’m looking at a car that would:

• Save me money every month on gas.
• Wouldn’t cost me anything to finance.

Suddenly, the Volt went from being expensive to being competitive, on price. It got better from there.

So then I started looking into how much I would save on gas each month. This is where it really started to get interesting. Your Mileage May Vary, is very much true with the Volt. The Volt is one of the few cars on the road that actually typically gets better numbers than the EPA or GM estimates. For everything about the Volt, the answer is “it depends”. It depends on how you drive, it depends on the temperature outside, it depends on the terrain you’re driving over. Neither the EPA or GM have any idea of how to estimate anything about the car, so all the numbers they report are just guesses.

A digression on how the Volt works: In essence, its like a Prius or other type of hybrid with a larger battery. It’s battery is large enough that most of your mileage, your commute and local errands, will use the battery. When I get home, I plug it into the wall (literally, the charger that comes with it is a standard plug), and while I’m sleeping, the battery recharges. Where it gets interesting, and what makes the Volt different from the Leaf or other all-electric cars, is that after the battery is empty, there’s a gas engine that kicks in giving you similar mileage to a hybrid. So your mileage will vary because it depends on the mix of pure-electric to electric+gas miles. If that weren’t enough, because batteries are based on chemistry they work best in moderate temperatures. The key takeaway is that the Volt is a Hybrid Plus. You will always get at least as good of mileage as a hybrid, with any pure electric miles being gravy.

So if the EPA and GM can’t estimate how well the Volt will actually do, who can? That led me to two sites: gm-volt.com and voltstats.net. The first site is the unofficial site for Volt owners, and I was able to find tons of information about the Volt, both good and bad. The second site is a collection of statistics about Volt mileage and performance from real Volt owners. Chevy has an official site, myvolt.com that does the same thing but its kind of lame, websites aren’t exactly GM’s thing.

Via Voltstats.net my car comes with its own website. Here it is. As I write this, Volt Stats is reporting that my MPG is 209. In the last week, my MPG has been 1000+, because I haven’t used any gas. When I am using gas I’m getting about 40 MPG, i.e. typical hybrid mileage. The only reason I’m not above 209 MPG lifetime with the Volt is because I haven’t owned it long enough! The dealer had not been able to charge my Volt before I picked it up, so I had to use .6 Gallons to get home so every time I drive somewhere I’m making my average better.

Like I said, Your Mileage May Vary, but from the above sites I found out that I should be able to easily exceed the numbers provided by GM and the EPA. That made the Volt seem even more attractive.

Meanwhile, at current gas prices, I was putting $50 in my CRV once a week, or $200/month. That’s $2,400 a year. Even though I’d be replacing that with $7.50 worth of electricity, an electric vehicle was suddenly seeming like the cheaper option long term. With the right rate plan from PG&E, I might be replacing that with $1.50 worth of electricity. $200/month in gas savings would make a substantial part of the car payment.

That’s when I started looking at the 5-year-cost-of-ownership numbers from kbb.com (Kelley Blue Book) and Edmunds.com. Their numbers were off for the Volt because of the 0% financing, their gasoline costs were just a wild ass guess because of the EPA/GM problem. Gas in CA is running me $4.25, not $3.80. After correcting for those, it came out that the Volt was going to essentially cost me the same as the MSRP over 5 years.

Meanwhile, the costs for every other car I looked at were about $10,000 more because of gas ($200/month for 60 months adds up), and $5,000 more because of something I hadn’t considered. Maintenance. Maintenance on the Volt is much cheaper because it works via electricity, as opposed to a gasoline engine which works by exploding things. In the first 30,000 miles/2 years, the only maintenance on the Volt is to change the oil. You’re not changing the oil because its worn out, you’re changing the oil because its stale. Oil gets stale! Most of the maintenance later on in the schedule relates to the gasoline engine/generator that is part of the volt, but that’s pretty light because unlike in a regular car, the gasoline generator runs at a constant speed and its simpler than in a car.

The $15,000 extra that a regular car would cost me hit like a brick. We’ve reached a tipping point, where the cost of gas and maintenance for a car over 5 years can exceed the cost of the car. Suddenly, the Volt went from being cost competitive with a CR-V to being cost competitive with cars that were about $10,000 less and not nearly as nice in the interior. Going through kbb.com and edmunds.com, I had to line up cars like the Ford Fusion, the Hyundai Sonata, the Ford Focus, and the Mazda 3 to beat the Volt on cost, and of those, few of them had a very nice interior.

The Volt was now winning on several fronts:

• I would save significantly on gas, even over a different kind of hybrid. I was currently spending $2,400/year in gas, based on 4 fill-ups/month at $50 each.
• It wouldn’t cost me anything to finance.
• Maintenance was another savings.
• The $9,000 tax credit meant the government was going to pay me to drive it the first year, because the first year of payments were less than the tax credits.

Remember, I started all this looking for a car that was “nice to sit in at traffic lights”.

So at this point, I was leaning pretty seriously towards the Volt. It would save me money in the long run, I could stick it OPEC, and the interior looked nice. My friend Steve told me about a Hybrids and Hot Rods show that was sponsored by the Mercury News, so he and I went to the show and I got to actually sit in a Volt. Though I couldn’t test drive one, I was also able to look at various other hybrids, and I eliminated everything but the Hyundai Sonata based on crappy interior quality. Most of them were like rental cars.

From the show I also found out that while Volts weren’t selling well elsewhere in the country in California and specifically in Silicon Valley they were selling really well. Especially because the newest California Volts qualified for the HOV lane. For my commute, driving in the HOV lane would save me 10-15 minutes each way. That’s an hour/week of extra time in my life. So dealers were selling them before they arrived on the lot. So I was going to end up paying MSRP.

Since I could get MSRP anywhere, in a way, that made shopping for the Volt easier. But at MSRP, I was going to be super picky about the options I wanted. Basically, that boiled down to my deciding that I didn’t want the Nav system at an incremental cost of $2,000. Between my iPhone, and the turn by turn directions provided by OnStar (3 years free with the Volt), it wasn’t going to be that useful. After that, I wanted the rear-view camera for backing up, and I talked myself into the speaker upgrade. Via Gm-volt.com I found out that the reader seat divider thing was also worthwhile.

Via GM’s inventory website, I found a dealer who had a Volt with the set of options I wanted on the way, if not on the lot, in the color I wanted. That was a challenge because most of the dealers were order Volts either loaded or with nothing. I wanted something squarely in the middle, what I called the “iPhone mix”. It was time for a test drive.

From asking around, and the above websites, I’d learned a couple of things in preparation for the test drive.

First, the Volt has 5 driving modes. There’s “D” and “L”, which are “drive like a gas car” and “drive like an electric car”, and then you can mix in “Normal”, “Sport”, and “Mountain” on top of that. Generally, I drive in Sport and L.

Normal drives like a Cadillac. It accelerates gradually, its mellow and relaxing, but slightly boring. Most of my co-workers who had driven a Volt had driven it Normal, and found it kind of poky. Which is strange, because unlike a gas car, an electric car has all of its torque instantly available. Touch the gas, well, ok, I’ll speed up. IN the Volt if you stomp the gas in normal, zoom, but you have to know it does that.

Sport mode drives like a Camaro. I owned a Camaro, driving in Sport reminds me of driving the Camaro. Touch the gas, zoom. Electric Car zoom, which in many ways is zoomier than my Camaro. My Camaro had 280 HP to the Volts 143, but the Volt has 143 HP at all speeds. The Camaro only had 280 HP in its sweet spot, when you stomped the gas, there was always a fraction of a second for it to adjust to the new gas.

The Volt is a lot more fun in Sport mode, though if you have a heavy foot, you won’t notice much difference. It doesn’t top out at 140 MPH like my Camaro, more like 100, but I only could ever get the Camaro up to 140 about twice in the whole time I owned it.

D/L is a little more complicated. The drivers manual that comes with the Volt will tell you that L extends the electric vehicle range by causing it to regenerate electricity instead of coasting, so its better in the city. While that’s true, I’m going to describe it differently. I’m going to describe how it feels to drive in L.

In a regular gas car, the “accelerator” accelerates. If you want to go faster, you push down from where you are. If you want to slow, you let up on the gas, and the air friction will slow you down, at which point you have to adjust the gas position again. If you’re going up a hill, more gas. If you’re going down a hill, less gas. Driving a car is a series of constant adjustments to the gas pedal even when maintaining the same speed. When driving the Volt in D, the Volt behaves the same way.

In L, the accelerator pedal acts more like a velocity pedal. Want to be going a certain speed? Push your foot down that far, and the Volt will accelerate or decelerate to match. It feels like cruise control tied to a pedal. It takes a moment to get used to, but barely a moment, then it feels like the most natural thing in the world. You can drive around the city using mostly just one pedal; I use the brake only for stopping that last 5 feet. After you’ve experienced it, L actually feels more natural than D, and its much easier to drive in.

So armed with all that, it was off to the dealer for a test drive.

Driving a Volt, or any other electric car is very different, in a good way. You know how driving the little golf carts is the best part of golfing? Now imagine a golf cart that can go 100 miles an hour. They’re that much fun to drive. Having 100% of the torque available at any speed makes it very fun to drive. Zoom.

Then you notice something that isn’t there: the engine noise. Electric cars make so little noise when moving and zero noise when stopped. It’s a unique experience. It’s very easy to get going very fast without realizing it because you’re used to the engine noise telling you how fast you’re going. During the freeway part of my test drive, the Volt was up above 80+ before I noticed that the outside had gotten a bit blurry. Kind of like when they kick in the hyperdrive in Star Wars. No noise, quick acceleration, Zoom.

Meanwhile, the Volt was also comfortable.

I was sold. It was comfortable, fun to drive in sport, comfortable to drive in normal, and quiet which I hadn’t realized would be such a big deal. So I pulled out the VIN for the Volt with the color and options I wanted which was literally on a train somewhere between Ohio and California, and the Dealer sold it to me. He told me it would take about 20 days to arrive, it took about 14. So on May 24th, I drove it home from the dealer.

At this point, I’ve had my Volt about 2 weeks, but I’ve only been able to drive it 1 week, because I was out of town on vacation. I love it. I don’t have my thoughts organized about why, but here are some points:

Remember what I said above about how quiet the Volt is? What I’ve discovered is that a large part of the stress of my commute is the noise. Much like traveling in an airplane, you don’t realize it, but the engine makes noise, so you turn up the radio slightly louder, or you talk louder, or you just live with the noise. But all of that noise is stress. My commute feels shorter. It’s just me, and my thoughts for 30-40 minutes. It’s almost relaxing to commute now. It’s me time.

The sound system is better than what I have in my house and better than any other car I’ve ever been in. Bose did an incredible job of equalizing the sound in the Volt. A car is a weird environment for sound reproduction with its mix of glass, steel, leather, and plastic, but you wouldn’t know in the Volt. The music is clear and crisp. Add that to how quiet the car is when moving around, and its amazing.

No gas, no gas station. Every day, I come home, plug in my car on the way into the house, and say “Take that, OPEC, Hah!” I’ve only used .9 gallons of gas, .6 of which was driving home from the dealer, .3 of which was a long trip that wasn’t quite enough to do completely electric after we diverted to another town for a drink. Because I had a Volt I didn’t have to worry about it, and the only reason I knew it had switched from pure electric to hybrid was because the gauge switched from being an “electric gauge” to being a “gas gauge”. The only reason my wife knew was because I told her “Hey, we just switched to gas”.

The Turn by Turn navigation from OnStar works great, even in the model without the map navigation. It knows where you are, it prompts you at appropriate intervals, it shows you giant arrows indicating what you have to do, the only thing it doesn’t have is a map. Oh, and I can send it the destination from my iPhone and it downloads into the Volt.

The iPod integration works great. I didn’t think I would use the Sirius radio that much, but I just discovered they have a Comedy channel I think I’ll use, especially if its been a hard day at work.

Everything seems like its automatic. As I walk away from the car, it locks itself. When I want to get in, I touch the door handle, and it unlocks. Air conditioning/heat and so on is fully automatic. In my CR-V it seemed like I was constantly adjusting the Fan, setting the blowers, switching between heat and cold, turning on the air conditioning, turning off the air conditioning. None of that in the Volt.

Driving seems more like gliding. It seems like the Volt glides from place to place, partially because of the quiet, partially because electric motors just drive like that. My wife and I call the Volt the Blue Ninja, because its blue, and its so quiet.

It’s not perfect, but its one of the best cars I’ve owned. Driving a Volt is like driving the future.

The Volt isn’t for everyone. It’s a little confusing, because you’re driving the future, and all the modes and options can get a bit complicated. It’s still a little “engineery”, kind of like most cell phones were prior to the iPhone. That wasn’t a problem for me, because I’m an engineer, but it might bother you. They’ll smooth that out in the years to come, and GM assigns everyone a “Volt Advisor” when they buy a Volt to help them through it. That’s the main reason I think they aren’t jumping off the dealer lots yet, there’s too many questions people have where the answer is “it depends”. GM also needs to pick one simple thing about the Volt, and push that in each ad. It’s basic marketing. GM’s current slogan “It’s more Car than Electric” is a head scratcher, and I own a Volt!

At the Hot Rods and Hybrids show a lady came up to the Volt salesman and said wanted a hybrid but she didn’t want a Volt because she “didn’t want to have to plug it in”. Based on that, I would sell the Volt via this slogan: “Going to the Gas Station Should Be Optional”.

As far as the complexity goes though, I would say that about half of the “it depends” are my fault though. I think most people would be perfectly happy driving the Volt around. Set everything to Auto, pick a radio station, call OnStar and have them set your destination in the Nav, and you’re ready to go. You’ll use less gas, you’ll be comfortable, and you’ll have less stress as you drive around. Its my tendency to optimize everything that both led me to the Volt, and made me dive into the 5 different driving modes. But that’s the stuff that’s fun for me. It’s the stuff that’s not fun, like going to the gas station once/week, getting the car maintained every 3-6 months, that’s the stuff that’s not fun for me, and that’s the stuff I that made me buy a Volt so I don’t have to do it!

If a Volt makes sense for your commute, you should seriously consider buying one.

Apple needs to become a public cloud provider. This will be an ambitious undertaking because frankly, they’re very far behind Amazon.

But if they did so, it would be hugely successful. Because these days, every app you see ends up being backed by a server somewhere and that server is in a cloud somewhere. People want and expect their data to be everywhere, which means you need servers in the cloud.

Which means that every app you see is really two apps, the iOS app and the cloud app.

Right now, what everyone does is write both apps on MacOSX and then port the cloud apps to some flavor of Linux. So the immediate win for Apple developers is that the cloud apps would run on the same flavor of system they were developed on.

The immediate win for Apple is that all of the energy those developers spend on their server apps would be spent on MacOSX instead.

That’s just the win on day one. If at the same time Apple rolls out a suite of services that their cloud provides along with OS support for talking to those services that would mean that all of the energy their developers currently spend writing server code could instead be spent on their apps.

The resulting combined ecosystem of devices, software, services and cloud would be nearly unbeatable.

Apple has already started down this path with iCloud. But they need to go farther. Because right now, every app I design in my head has a server component that requires the cloud.

So there were rumors this week that Microsoft was going to ship a version of Office for iPad.

I have no opinion one way or another whether they will or not. But I admit to being intensely curious about it because Microsoft has always used Office as a weapon. If Microsoft makes an iPad version of Office it will say a lot about what they think about the state of the tablet market.

Microsoft is really about two products. Windows and Office. That’s where they make all of their money. Several times now they have pretended to make of Office for some Windows tablet system only to fuck it up abysmally. The product manager of PenWindows complained about this publicly back in the day that he couldn’t get the Excel or Word team to make their products usable on a tablet.

Meanwhile, Apple is so dominant in the tablet space that essentially all the other tablets are essentially single purpose like the Kindle Fire or a “so what?”.

Microsoft’s original play book with Apple went something like this:

1. Develop software for the Macintosh to learn about how to write for a GUI.
2. Port that knowledge and software to Windows.
3. Use Office as a club against Apple, while making Windows the dominant GUI.

Subsequently, I am 100% sure that Microsoft  has Office for iPad written. Microsoft has a whole division called Microsoft Bay whose job is just to spy on Apple and the rest of Silicon Valley. So of course they wrote one if only to scope out the tech. The real questions are:

• Is it any good? In Microsoft’s my-dick-is-bigger-than-your-dick culture, if I was the product manager for Office unless Steve Ballmer was coming by once a week to tell me it was important, I can guarantee you that I would completely ignore any requests by the Office-for-iPad team. (Duh, this is exactly what happened to Office-for-PenWindows the last time.) Meanwhile, Touch UI’s (TUI) are different then GUIs. I’ve used Pages on the iPad, and even Apple hasn’t quite gotten this right yet. (Though Bento is cool, and a better fit for the iPad). So why would Microsoft get it right?
• Do they choose to release it? Microsoft doesn’t need the money, and they wouldn’t be developing Office for iPad for the money. It may make more sense for them to sit on Office for iPad than it does to release it. If its good, have it be good on Windows 8. If its bad, have it be bad in private.
• Are the Windows 8 tablets any good? Since Microsoft doesn’t make hardware, if they’re looking at what the hardware vendors are going to ship, and the price points they’re going to ship them at and they expect none of them to succeed, then they might as well suck it up and ship Office for iPad.

The only way I see Office-for-iPad coming is if Windows8 on tablets suck.

Releasing Office for iPad if Windows 8 tablets are good only benefits Apple, not Microsoft so much. It benefits Apple because then they can say “Look, it runs Office”, which is important to Apple’s perception of the market, because when they launched iPad, they said “Look, you can view Office documents”. 2 years later, it is less important because everyone has gotten used to a platform that doesn’t have “Office”, but I can still see Apple talking it up.

On the other hand, if Windows 8 tablets suck, and are going to keep sucking until Windows 9, then they might as well release Office for iPad. They’ll get money and TUI knowledge they wouldn’t get otherwise. They may not want to endorse Apple by releasing Office for iPad though, so I wouldn’t hold your breath.

So I guess I do have an opinion. My prediction: No Office for iPad even if they’ve developed it. Because releasing Office-for-iPad means admitting that their tablet is crap.

So professionally, this was an interesting week for me.

Wednesday, Chegg shipped their new eReader which is the project I’ve been working on since August. It’s HTML5, and its a faithful representation of the books (more on why that’s important in a bit).

Thursday, Apple shipped an upgraded iBooks, announced a new textbook initiative aimed at K-12, and shipped a new “eBook authoring tool” called iBook Author.

That same day, I had an interesting discussion with a friend of mine about eBooks and their effects on libraries. He had the typical Silicon Valley arrogance for anything non-digital: “those publisher people are dinosaurs and we should legislate them out of existence” . Having lived outside the Valley for most of my professional life, I had a more measured response that the publishers do add some value, and that much more interesting things would happen if we made them allies instead of enemies.

Since iBook Author was free and I’m lucky enough to use a Macintosh for work I immediately had to go and check out iBook Author.

I’m underwhelmed. I know its a 1.0, but there are several missteps here. No print publisher is going to use this tool to create books. Even amateurs are going to find it a bit challenging to produce really high quality books. Even if an amateur makes it through the process, they will then be shocked to find out it will also cost them $125/book to publish their book even if they want to give it away free because you need an ISBN number to publish a book.

Still, its a really well done product, its just not solving the problem in the right way. Hopefully if someone at the relevant team at Apple reads this, they will take this as constructive criticism.

For you to understand my point of view, I first need to discuss some things I’ve learned in the last 21 years of being in the technology industry; the last 5 months of being in the digital textbook industry; and the last 45 years of being the son of an elementary school teacher.

iTunes did not kill the music Industry

Contrary to conventional wisdom, iTunes did not kill the music industry. If anything, iTunes and iPod saved the music industry. iTunes directly saved the music industry from online piracy. iPod helped as well, because they made it easier for people to listen to music.

Yet the music industry’s revenues are down. The music industry’s revenues are down because people spend slightly more of their income on entertainment but across many more categories. That is, the music industry has a smaller percentage of the entertainment dollar. Hour per hour, people have a certain amount of leisure time. They divide that time between music, books, exercise, movies, television, food, renaissance fairs, etc. All entertainment categories have seen a decline, but music has declined less than any other category precisely because iTunes and iPod have made it easier to consume music than any other category.

iTunes saved the music industry! But there’s a reason that all of the music companies are now owned by movie studios, or as we call them now, “entertainment conglomerates” that produce movies, video games, music, toys, etc. Even when people are spending more on entertainment than ever before, there’s more completion than ever before for that same dollar.

Publishers are not the enemy

Unfortunately, the publishers don’t know anything about the entertainment industry. All they know is that like everyone else, their revenues are down. Down by a two thirds! It’s making them understandably nervous and scared. They don’t know what to do about it, because they see what happened to the music industry and they expect it to happen to them with eBooks because they’re blaming iTunes instead of the Internet. Their old business models aren’t going to work any more, and change sucks.

There’s this classic New Yorker cover called the New Yorkers view of the World. In that cover, anything outside of New York is a vast desert; New Jersey is represented by a little mud bank on the side of the Hudson river and so on. It uniquely captures the arrogance of a New York denizen about their town. When I was a kid, I remember meeting people from New York who came out to California; they were always surprised it wasn’t like a John Wayne movie, all horses, Wild Indians and sagebrush.

Silicon Valley people have their own version of this. It’s the assumption that only they know how to do anything digital, and that digital technology is so superior to the “old way” of doing things that anyone doing things the “old way” should just roll over and die. It would be amusing if they didn’t also start arguing that the legislature should make sure that they die.

Here’s the thing. Its not necessary in life to make enemies of everyone who disagrees with you. In my job, I have to negotiate with the publishers all the time, and I never, ever approach them as the enemy. Do they ask for things that are unreasonable? Yes! But as I’m constantly telling my co-workers, they’re reacting from fear. So it doesn’t matter if their request seems irrational. You can’t argue them out of it with logic, you have to address the fear. It’s basic human relations that when people are reacting from emotion, you have to deal with the emotion first. When we do that, we get a much better response from the publisher.

Publishers are not in the dead tree business

The standard business school myth is that the railroad companies, once the largest and most profitable businesses in the United States, didn’t realize they weren’t in the railroad business, they were in the transportation business. As a consequence, they didn’t seize the opportunities presented to them by the rise of the trucking industry. Gradually that led to them become irrelevant because they couldn’t adapt. The thing that always bothers me about this story is that we could have had container shipping 100 years ago. Computerization helped, but we had people with clipboards 100 years ago.

The thing you have to realize is that textbook publishers are not in the dead tree business. They are in the education business. The publishers realize this. The holdback for electronic education has always been more of a problem of platform (Windows was too hard to use), expense (computers were too expensive), and the system (you have to get all the bureaucrats to agree). Plus let’s face it, no one really knows what e-learning looks like yet. Even eBooks are just a Old thing in a new package, and it’s not clear the new package is really better because dead trees don’t crash and need to reboot.

Currently, we’re in a transition period, which means the publishers are “living in interesting times”. Cut them some slack! The face of education is changing, and the publishers are desperately trying to figure out what e-education looks like. Before you mock them as dinosaurs, realize that no one else knows what e-education is going to look like either.

Textbooks are 50% design

Someone commented to me that Calculus hasn’t changed in 100 years, so why do we need a new Calculus textbook?

One of the most popular text books on Chegg’s site to rent is a textbook about how to use Microsoft Office 2010. Think about that. The most popular book, is about a subject that didn’t exist until 2 years ago. It’s also about a piece of computer software. Textbook publishers definitely do crank out edition after edition. Some of that isn’t warranted. But so does the computer industry! Leave the Silicon Snobbery at home. The Publishers are creating value in the world. I’m not sure that’s true of many Silicon Valley companies.

I’ve spent the last few months looking at a lot of textbooks, and I can tell you, these aren’t the textbooks I had in school. Calculus may not have changed, but the way we teach calculus has changed. The textbook publishers have taken every trick in the book made possible by desktop publishing and brought that level of design into the textbook. Textbooks are now 50% design, 50% writing. It would be a very cool thing if we started giving textbook designers credits in the same way we give the authors, or for that manner, film directors.

Yes, you can read Harry Potter on your iPad. But trying to shoehorn a modern textbook into ePub loses half of the design sensibilities present in the textbook design.

Neither ePub 3.0 or HTML5 can capture the fullness of a modern textbook. If you don’t believe me, try to to implement a sidebar, notes in a margin, and 2 column reflowing text, with an attached picture in HTML.

It can’t be done. CSS teases you into thinking its almost possible but it’s not. HTML cannot reflow text into columns! it can’t move a picture to float into the margin while the text reflows. For that you need a layout engine.

The ideal textbook writing application

If I was going to sit down and write an application to write textbooks I would start by thinking about the components of a textbook. Because what I would want would be something that I could use to describe how the information in a text book relates to each other and to the page. For instance picture the following 6 interrelated items:

• A block of text describing a concept. It should be nicely formatted on the page, justified, with two columns in landscape mode.
• A picture that goes with this block of text as an illustration.
• A caption that goes with that picture.
• A margin note goes with the block of text near about the center or rather next to the 3rd sentence in the 4th paragraph
• After this text, present this sidebar.
• A footnote is associated with this portion of the text.

While that seems like a lot, what I’m describing isnt any more complex than the layout of a “Dummies” book. That doesn’t seem like a particularly high bar to aim for! The textbooks I’ve spent the last 5 months looking at are considerably higher quality than this. In those books, every page is hand crafted.

Thinking about the components of a textbook would help me to build a data model to describe how the pieces of information in a textbook relate to each other and what those pieces are.

The next step for me in building this tool would be to reimagine how those elements should work in an e book. Not every print convention makes sense on a tablet. Do footnotes really need to be at the bottom of a page? Why not have them function as popups? What about footnotes that are extended asides? Should margin notes stay in the margin or should they be popups as well? What about sidebars?

So now I have a data model and some presentation ideas. The next thing I would need to build would be a layout engine. Because each reader has a different aspect ratio, and even the iPad is both a 4:3 and a 3:4 device, my ebook tool needs to be able to algorithmically layout a book the same way a human would layout a book.

That’s how I would build an ebook tool if I really wanted to revolutionize the text book industry.

Which brings us to iBook Author

iBook Author is not built the way I would build an eBook authoring tool. That’s because iBook Author makes a couple of simplifying assumptions.

• It only supports the iPad, that is 4:3 or 3:4 book orientations. iPhones, Kindles and 16:9 devices are not usable. Nor are laptops!
• The tool will be used to layout the book once, not write the book.

These assumptions are both iBook Authors greatest strengths and simultaneously it’s greatest weaknesses. iBook Author has no real provision for automatic layout of a book beyond basic text reflowing. It has no understanding of basic textbook idioms like sidebars, callouts, figures, tables, margin notes, etc.

What iBook Author instead understands is that on page X there are some boxes with certain specific pieces of content to display, in certain locations. If the iPad is in the other orientation, these boxes will be on Page Y instead, and they will be in these locations instead. Boxes can sort of float with specific pieces of text, or they can be on a specific page.

In other words, IBook Author is mostly a page layout tool. To get the same level of quality as a existing textbook, you would start by taking your existing textbook layout and throwing it away. Now you can rebuild your textbook as an iPad book from the original content.

Then, when you’re finished with that, you have to start over and layout the book all over again for the other orientation of the book.

So is it possible to use iBook Publisher to layout a modern textbook? Yes! People used to layout textbooks with paper scissors and glue. But it would also be incredibly tedious. Will the textbook publishers revamp some textbooks into this tool? Yes, especially the textbooks that are mostly text. Technical books such as the O’Reilly books that start out as a Word documents and essentially remain that way even in printed form will be easy to convert because layout isn’t really a factor for those books.

In fact, those sorts of books don’t need iBook Author. You can get any of the Pragmatic Programmer line of books in EPub or mobi already. In fact, Pages will already export to EPub for you without any weird licensing agreements required.

IBook Author is only interesting for books that require conplex layouts. But complex layouts mean complex work. Work that has to be done by someone manually. Is McGrawHill going to layout their entire 1,000 college textbook Catalog in IBook Author, then lay it out again for the other orientation? Not this week! not unless they hire 1,000 graphic designers to get it done.

No Programming required means it’s lame

The other big feature of iBooks are the widgets. Again, color me unimpressed. They’re pretty basic. There are 6 predefined widgets:

1. Gallery.
2. Media (movies and such).
3. Review.
4. Keynote.
5. Interactive image.
6. 3D.

There’s no real wow here. If your really want interactivity, you’re going to have to program it yourself, but you’ll only be able to go so far as a widget. Embedding Keynote is going to be kind of weird, because you open a book, then there’s a presentation inside the book? That seems like a strange user experience.

So at the end of the day, while iBooks Author is cool in many ways, I don’t see it replacing high end textbooks any time soon. Low and medium end textbooks will do just fine in the existing ePub format, but it will be too tedious for textbook publishers to layout their complex textbooks twice until the market has proven itself. While the widgets are kind of neat, none of them are especially innovative.

The “Fuck You” dialog

So lets say you do all this work to layout your book (twice!). You’re ready to export the book. What do you get? This dialog.

As others have commented, using iBook Author locks you into somehow involving Apple with selling your book. The license agreement goes on to say:

IMPORTANT NOTE:
If you charge a fee for any book or other work you generate using this software (a “Work”), you may only sell or distribute such Work through Apple (e.g., through the iBookstore) and such distribution will be subject to a separate agreement with Apple.

Apple will not be responsible for any costs, expenses, damages, losses (including without limitation lost business opportunities or lost profits) or other liabilities you may incur as a result of your use of this Apple Software, including without limitation the fact that your Work may not be selected for distribution by Apple.

A format that doesn’t work with a publisher’s workflow, differs in important but subtle ways from standard formats, isn’t supported by any of the industry’s tools, is, long-term, going to be a torture to support.

So Chegg launched our new eReader today so I thought I would take a moment to talk about writing a Project Charter per Johanna Rothman’s excellent book on project management.

Johanna recommends that for every project you sit down and write out a project charter. The project charter states the purpose of the project, additional goals, the single driving factor, and any constraints. Everyone on the project should be able to state why they’re building the project. Meanwhile, the project manager should know what the real driving factor for the project really is because that’s going to be the key to managing the project.

When I got to Chegg my boss gave me a set of desired attributes for a content management system based on his experiences at Netflix. He also gave me a deadline.

When I sat down to write the project charter I discovered an interesting thing. The project driver wasn’t what anyone thought. The fundamental driver for the project was that we needed to be able to convert 40,000 books in 2 weeks.

Figuring that out actually simplified the problem quite a bit. What we needed was not a content management system like we thought, but rather a cloud-friendly job coordination engine. The deadline, while a constraint, wasn’t the driving factor in the project. Doing 100 books by the deadline wasn’t interesting, it was doing 40K books for the launch that was interesting.

Writing a project charter made sure I built the right thing in the right way.

You can get Johanna’s book at Amazon, I highly recommend it.

A friend of mine who is job hunting (he’s looking for a job in Software QA if you know of any positions) emailed me this morning for help with his job hunt.

Here’s my tough-love answer:

Your job right now is looking for work. If you’re not spending 35-40 hours/week on looking for work, you’re not doing it right. That does not mean 35-40 hours/week sitting at your computer. That’s the least productive way to get a job. You get a job by meeting people.

Have you gone to any of these meetings?:

http://www.ssqa-sv.org/speakers.htm

It took me 10 seconds to find that group. That’s the kind of thing you should be doing. Sending a resume somewhere and having that actually work for getting you an interview is like winning the lottery. Meeting someone at a group like that is easy, and its like getting 5 interviews for no effort.

I couldn’t do it when I was in Flagstaff, but even though I’m not looking for a job any more, next week I’m going to my college alumni luncheon so that the next time I go looking for a job I’ll have 15 people I can call. Additionally, when my new company is looking for people, I’ll know 15 people I can recommend.

Besides hard core networking, in addition to the 35-40/hours week you need to spend looking for work, you need to be spending 10-20 hours/week Sharpening the Saw. That means exercising, learning a new skill, etc.

Finally, looking for work can be demoralizing because you do a lot of work before you see any success. When you see success, you’re done. You need to reframe the way you approach this so that you can refill your success well.

That means you need to start logging everything you do while looking for work. That’s what I had to do, I had to reframe things so that just doing the work was “success”. That also helped me budget my time so that I wasn’t doing too much of the things I liked to do (spend hours on the computer browsing websites) and more of the things I didn’t like to do (networking). Pick the part of the job hunt you hate the most (for me it was cold calling) and any day you do two of those in a day is a “win” for you.

There’s a scene in the movie Good Will Hunting where Matt Damon tells this stuck-up guy in a  bar:

” You blew $150,000 on an education you could have gotten for a $1.50 in late charges at the public library.”

Bar Scene from Good Will Hunting

Something I’ve done for years is prowl around college bookstores about the time the term starts and see what textbooks were selected for what classes. If the textbooks are good, I often buy them and read them on my own. Right now I have textbooks on my bookshelf on:

• Molecular Biology
• Protein Structure
• Biomechanics
• Behavioral Economics
• Astronomy
• Architecture

I’m always learning. I call this the Good Will Hunting method, because why should I pay some university a bunch of money for a piece of paper?

I was talking with a recruiter about a position at their company the other day and after reading the job requisition I commented to her that it was a “Bigfoot” requisition; the kind of req that reads “Must be a Lawyer and a Pastry Chef”.

There’s no way that the recruiter will be able to find a person to fill that req. All of the lawyer’s I know who burned out and became pastry chef’s are quite happy in their new bakery business. So what will happen instead is that the hiring manager will hire someone he knows and grumble about how HR can’t find anyone good.

So it goes.

Personally I’ve always hired for brains+attitude, not necessarily some specific laundry list of tech skills. My best engineer in my last position ported himself from Windows and C++ to MacOSX and Java in about a month; as I knew he would. I needed someone who knew object oriented programming and could design servers, Java was ultimately optional.

The silliest for me is when I see people listing source code control systems like svn, git and SourceSafe in a req. Are you really not going to hire someone who lists cvs on their resume? For that matter, who lists source code control applications on their resume?

The recruiter came back to me with the following:

I love the way you think… It’s always smartest to hire the best athlete and watch ‘em perform in the role. All the minutia is ridiculous.

Now, identifying a “best athlete” who’s not a referral… that’s a whole ‘nother challenge for those of us doing the hiring, isn’t it?

Ok, Stacy, challenge accepted. Here’s how to identify an athlete that’s not a referral, that is, from a resume. So, first rule:

• Athletes break the resume rules for specialization

Picture yourself as a software engineering manager with a star athlete on his team. Over and over you are going to find yourself saying to the athlete: “Can you help Fred with this? Can you look into this new technology for me? Can you work with Barney on that problem he’s having? Accounting is whining about Billing, and that involves money so we better fix it. Can you figure out what we need to do?” The athletes on a development team are going to have the broadest experience in terms of problems you’ve asked them to solve.

Translating that to a resume, software athletes are people that look like generalists, because their resume will show lots of broad experience. Broad experience is much more interesting in the long run than “depth” in the particular skill you’re looking for; tech skills rarely last longer than 5 years.

My resume for reference. I have been a software athlete, and am now a coach (Director/VP) of athletes.

Specifically, that means that if you’re looking for a Java engineer, a resume that lists:

• C++, Python, Perl, Ruby

Is quite probably a much more athletic engineer, and a better hire than one that lists:

• Java

That’s because the 4 languages above have their own strengths and weaknesses. Someone who gets object-oriented programming in C++ is going to pick up Java pretty quickly, and they’ll have learned things from doing programming in Ruby and Python that will serve them well with Java. Someone who has done Java and only Java for years will be much more narrow and less versatile in the long term. After 2 months, the C++/Python/Ruby/Perl guy could easily know more about Java than the original Java guy, because he’s more curious in fact, that’s been my experience.

Similarly, you want to look for the same sort of breadth in achievements. Someone who has the same 6 months of experience 10 times is not going to be a good hire. I call those people IT mushrooms because its common to see someone from the IT department at a big company who did the same thing for 5 years. Just like a regular athlete can do 20x what an average joe can do, so can a software athlete be 20x more productive than a regular software engineer. Software managers give their star athletes giant tasks to accomplish, tasks that are often hard to distill down onto a resume.

So if I was looking for someone with Business Intelligence experience, given a choice between:

• Developed 8 reports for Accounting using Oracle Business Intelligence system

and

• Reworked billing.

When asked to find a person with high-end Business Intelligence experience I would most likely hire the second guy. Because what I know happened is that billing was broken somehow and Accounting was always whining (it was taking too long, etc.). So I asked my software athlete to “Fix billing”. He fixed it, which involved meeting with Accounting to figure out what their needs actually were, setting up an ETL process from the OLTP database into a data warehouse, setting up the BI system, and developing a couple of reports. It probably took him 2-3 months.

After that, I would have assigned the first guy, the mushroom, to follow in the athletes footsteps and do any additional reports Accounting needed. It probably took him about a month to do each report, sometimes more. So even though nominally the first guy looks like a closer fit for the position, the athlete is the one that really did all the work.

Of course, I have an advantage as a technologist, that I can skim a resume and tell if a person is an athlete or not to a decent approximation. What are recruiters supposed to do? My suggestion is to get the hiring manager to follow the process I use when working with recruiters:

1. Talk to the hiring manager about whether they want an athlete for this position. “Do you want an exact match, or a smart guy?”
2. Select 20 resumes at random, after throwing out the obviously bad ones. (Grammar problems, misspellings, web designer for a programming position, etc.)
3. Have the hiring manager sit down with you and sort the resumes from best to worst, with the specific idea that you’re looking for athletes. They should immediately know which technologies overlap which other technologies and what tradeoffs they’re willing to make. Have them explain the top 5 resumes they selected to you, possibly against the bottom 5. You’ll get a lot of good information out of this about what they really need for the position, because as a hiring manager, I have to say that my job requisitions are rarely perfectly written…
4. Repeat steps 2-3 occasionally if needed if you’re having trouble finding a match.

This information gathering exercise will tell you a lot about what the hiring manager really wants, not what the job requisition says!

From your meeting with the hiring manager, you should now be able to glean a few “possible athlete” resumes from your queue for the position. The next step is in the phone screen. When phone screening for athletes, your goal is to separate the jack-of-all-trades-master-of-none candidates from the master-of-all-trades-throw-me-at-a-problem-and-I’ll-kill-it athletes.

My two biggest tools for doing this are the questions “How did you do that?” and “Why did you do it that way?”. It’s nice but not necessarily important for me to understand the details of their answer. What I’m looking for are that there are those details, that they really dug into the problem, solved it, and moved on. I’m looking for something that for any other candidate would be 2 years of work and required training classes but the athlete did in 3 months. I’m also looking for people who had a hammer in their hand, but had a problem that was a screw, so they put down the hammer and went and found a screwdriver. Finally, I’m looking for brains.

I can’t explain it any better than that, but it’s obvious to me when it happens during a phone screen.

So after scanning resumes, phone screening the results, and hopefully finding one or two possible athletes, that’s the point where I would make a note on the resulting candidate “May be athlete”. Because the manager is going to interview a potential athlete differently than they’ll interview someone who is merely a close fit.

I hope this helps, let me know!