With a title like “Why Google and Yahoo! can’t be better open source citizens” one might think that our companies were squeezing as much as possible out of the open source world and giving little back.
But after reading Matt Assay’s post a few times, I’ve begun to wonder how much open source code he’s been publishing.
Putting aside the many contributions that Yahoo and Google have already made to various open source projects (Linux, FreeBSD, Perl, MySQL, PHP, etc.), I’d like to debunk his conclusion:
All of which means, as Tim pointed out, that these companies have failed to write code according to a cardinal open source principle: modularity. Yahoo! and Google can’t open source more code because their code is too tightly bound together – layer upon layer upon layer requiring layer upon layer upon layer. This doesn’t mean that Yahoo! and Google are bad, but it is disappointing that they are such heavy users of open source, and have architected themselves into a corner that makes giving back impossible or problematic.
Well, I’ve got news for you, Matt. Some of our code is tightly bound together. You know why? Because abstractions often cost performance, and performance costs money. When you’re serving billions of pages per day, even the small stuff adds up quickly. But there really is a rhyme and reason behind the systems architecture and various bits of code.
But the layering or tight coupling isn’t the only problem. Heck, it’s not even the biggest problem.
So let’s suppose that we decided to release “what we can” into the open source world. Of course, there’d be a lot of legal vetting first. Code licensing is a real mine field, but let’s suppose that we cleared that hurdle. It would look as if Yahoo was doing exactly what businesses looking to get into open source are told NOT to do: throwing some half-baked code “over the wall” and slapping a license on it.
You noted this in your article too:
Jeremy eventually owned up to a reason that I found much more compelling – disappointing, but compelling. Jeremy said that Yahoo’s applications are tightly bound together, making it difficult to open one piece without giving away information about how the remainder is written, or making it useless because knowing 1/10th of the application wouldn’t be helpful (because of all the unknown code).
Right. There’d be places in the code where magic voodoo functions are called but we couldn’t really talk about what they do or how they might work. That’s called our secret sauce or “business logic” if you prefer. A good deal of that is kept under wraps for very legitimate reasons. It’d be like the FBI and CIA documents you get under a Freedom of Information request. The really juicy stuff is always hidden behind the thick black lines.
Open Source is supposed to be “open”, right?
How would that look? Would it encourage outsiders to use, improve, and hack on our code? Or would it make us look like we don’t really “get” open source? Like we’re trying to get free labor without anything in return?
Putting aside the fact that there’d be some “we can’t talk about what happens here” holes in the code, getting contributions back for the code would be… interesting. Are you willing to give us the necessary rights to use it in all the ways we’d like? Bug fixes might come in and be useful, but what about features? “Here’s a patch that finally adds ‘foo’ to Yahoo! Mail.” Then we’d be beat up for not integrating it fast enough. It’s tricky to introduce new features to a product that tens of millions of people are using.
But I’m jumping the gun a bit. How would a developer test a bug fix or new feature? Aside from the most trivial examples, you’d need to replicate a sufficient chunk of our infrastructure for that to work. Does that mean we have to release the code and documentation regarding our package management and distribution tools? What about our naming conventions, network ACLs, and so on? Where does it stop?
So Matt, I guess what I’m trying to say is that this is way more complicated than “Yahoo and Google don’t write modular code.” And I suspect that you know that but decided to opt for the cheap headline.
I don’t want to end this on a completely negative note, so here’s one example of Yahoo! code that’s been released on SourceForge under an open source license (BSD) with good documentation and support: YUI, our User Interface library. As I wrote on the YDN blog, a fair number of folks were interested in it. I’m even told that Valueclick is using it to revamp their site. Why not? We’re using it all over Yahoo! (Google has examples too, I’m sure. Perhaps ExplorerCanvas is one?)
How about that?
Claiming that we can’t be better open source citizens because of our inability to write good software is, pardon my French, just crap. Being good open source citizens means contributing where it makes sense, allowing our employees to be a part of the open source world, and helping to evangelize the benefits of open source software.
I think that if you’ve spent any time looking at Yahoo or Google’s involvement, you’d notice that we’ve been increasingly doing more and more to help the open source world.
Bookmark Murdok: 
Jeremy Zawodny is the author of the popular Jeremy Zawodny’s blog. Jeremy is part of the Yahoo search team and frequently posts in the Yahoo! Search blog as well.
Visit Jeremy’s blog: Jeremy Zawodny’s blog.
