Utah Open Source Planet

One of the reasons I love reading Jon Udell's blog is that he shares the results of his curiousity. Not only is Jon curious in general, but he's especially curious about data and how your can mundge it to produce information.

The latest example is Jon's look at where Oil comes from--not from where you think probably. The answer, if you live in the US is Canada and Africa. 33% of US oil comes from North America (with Canada being the largest "foreign" supplier) and 20% comes from Africa.

How did Jon find this out? By importing the data into DabbleDB, manipulating it, linking locations to maps, and then publishing it. It took him about 45 minutes. These kind of tools are freely available, but not widely used. Amazing information at our fingertips.

Tags: energy data politics

While listening to NPR this morning, I was thinking of the basic principle of development.  Whether it’s new technologies, new processes, or new products, it all seems to be developed through the same principle:  The shotgun effect.  

What is the shotgun effect?  Well, let’s say you go hunting for water fowl (duck or goose), and want to be sure you get one.  Say you and your buddy each take a gun, you have a rifle, your buddy has a shotgun.  You split up and start shooting.  Who gets more fowl?  The shotgun.  Why?  Because the shotgun, instead of firing one projectile in a (relatively) straight line, fires a collection of projectiles (shot) in a close group.  As such, it increases the likelihood a hit will come from a shot.  

So what does this have to do with development?  Let’s say your company is new to the market.  You have one killer product that is great, and you pour all your investment into this one project.  Then the Economy changes, and your product is no longer a necessary product.  Your company will most likely fail, and the product will go the way of the Turnip-flavored ice cream.  

But let’s say your company has that one killer product, but your development team has a number of other ideas to which they would like to devote time.  You give in, allowing your development team to have a small portion of their time to work on their own projects (sound familiar, like Google?).  Because they are developing on Company time their projects provide value to the company depending on whether or not it pans out.  

Now the economy changes again, and your killer product is no longer relevant, and as such goes the way of the beet-flavored popcicle.  All of a sudden you have a collection of products that you can fall back on, because you allowed your development team freedom to work on their own ideas and projects.  Given now the full investment capital from your company, one or several of these little projects become all killer products.  

This same principle is followed by Biology.  Genes have set codes that allow some variation without significant changes.  Yet they also mutate while being copied, and eventually significant variations start to show.  As one set of codes become too limiting in a given environment, another set may thrive, and those genes become dominant.  Whether you believe in Evolution or not, this concept of adaptation and genetic variation has been proven, within species.  Genetic codes follow the shotgun effect in their development, and only those viable characteristics that are the result dictate which codes work.  

So why was I thinking about this?  Because Energy policy and the Economy has been very prevalent in the political spectrum for this shining moment (who knows how the political landscape will change in a few weeks).  We can either put all our eggs in one basket, or allow the shotgun effect to work and invest in all technologies out there that can grow our energy production.  Only those that are the most viable in the given environment will succeed, while others may be waiting in the wings should the environment change.  

Perhaps it’s my bias showing through, but I see this happening again in Air travel.  Soon airlines will realize that airships will become less expensive to maintain for passengers than current airplanes, though it will take longer for someone to travel from one place to another.  It will probably start in the small communities that are currently losing airlines because of the low rate of return.  But soon I do see it becoming a more common method of travel and shipping.  After all, it’s just one more bit of shot from the gun.

Luiz Barroso from Google is speaking about Energy Efficient Operations. Computing has a great track record of having a positive impact on society. The world needs more computing. But more computing means more energy (usually).

World energy use of servers is around 1% of total electricity consumption. Making efficient computers is harder than making efficient refrigerators. Efficiency is computing speed divided by power usage. But that's too simple. For a server, you have to take into account the efficiency of the compute efficiency, server efficiency, and data center efficiency. These get multiplied together. Ugh.

Data centers are underutilized which accounts for a wasted power provisioning investment and less efficient power and air distribution. Typical serve power supplies dissipate 25% of total energy as heat. Computers are the least efficient in their most common operating points.

The operating cost of a data center is about $9/watt over 10 years. But the cost of building the data center is $10-22/watt. Facility costs are more important than operating costs in energy terms. Maximizing usage is a great way to save energy.

Here's some things to do:

1. Consolidate workloads into the minimum number of machines needed for peak usage requirements
2. Measure actual power usage of devices. Nameplates lie and overstate usage.
3. Study activity trends and investigate oversubscription potential. You don't want to go over (bad for machines and bad from a contractural standpoint).

This let's you pack the most servers in your data center that you can. A study at Google showed that you have to be able to spread computing over a larger number of machines in order to really take advantage of oversubscription. At the facility level, you might be able to host 20% more servers through oversubscription.

If you have a search cluster, a map-reduce cluster, and a web-mail cluster, the oversubscription potential is fairly low. But combined, they have substantially more because mixed workloads balance out demand better. Monitor and "victimize" a defined "best-effort" workload when problems arise.

Switching to energy-proportional computing. Consider the data center as a single computer. Call it a "land-held computer." :-) Most of the time aren't idle or at peak (unlike laptops). This is a result of the fact that high-performance and high-availability requires load balancing and wide data distribution. We design them to work this way. The result is there are no useful idle intervals in which to shut a processor down. There are lots of low activity intervals.

An idle server uses about 50% of the peak power requirement. But if you plot efficiency, the server becomes much less efficient below about 30% usage. 100% isn't realistic, but getting over 30% is.

So, energy-proportional computing is the idea of making the efficiency more linear. This would greatly reduce the need for complicated power management. CPUs are actually better at energy proportionality than other components (like RAM, disk, network, fans, etc.) An idle CPU, for example, consumes less than 30% of it's peak power where as DRAM is about 50%, disks are over 75%, and networking is over 85%!

Moreover, CPUs have active low power modes. A CPU at a slower clock rate still executes instructions, but DRAM and disks in low power mode need to bump up to full power to operate.

If there's any question whether this is a good idea, consider that the human body has a factor of 20 from it's resting power consumption to peak (at least for elite athletes).

The most basic thing you can do is to write fast code. This is the software engineer's biggest contribution to energy efficiency.

Throughout the talk Luiz referenced a paper from ISCA07. I believe this is it: Power provisioning for a warehouse-sized computer by Xiaobo Fan, Wolf-Dietrich Weber, and Luiz Andre Barroso

Tags: it+operations energy velocity08

Saul Griffith (click to enlarge)

This morning's opening keynote at ETech was Saul Griffith who ran down the steps he used to calculate his own carbon footprint and then what he had to do to put himself on a "carbon diet." It's not pretty. Doing the calculation is relatively straightforward in terms of the math, but gathering the data isn't easy. I'm hoping that we can get his slides when we put the audio up on IT Conversations because there's some great data there.

Speaking of IT Conversations, a recent IEEE show has a section on home co-generation. You can buy a furnace for your home right now that generates electricity to create the heat. You get power and heat from the same plant, making it much more efficient than buying power separately. You're still burning a hydrocarbon, but you're essentially getting the electricity for (close to) free. Retrofitting an existing home isn't a problem.

On a similar topic, today I put up the latest Technometria show on green computing. The guest is Jeremy Faludi, an expert in green computing. We talk about the carbon footprint of various parts of the computing industry and also mention where computers can help by reducing carbon use.

Tags: itconversations etech etech08 energy environment

Social Innovations is a sister channel to IT Conversations. They have a 10 part series of lectures by Amory Lovins, the Cofounder, Chairman, and Chief Scientist of the Rocky Mountain Institute. The series is a from set of five lectures he delivered at Stanford earlier this year.

I'm cross posting the Lovins lectures on ITC. The first lecture (see part I and part II) is on energy efficiency for buildings. This lecture has been highly rated by SIC listeners and I think ITC listeners will enjoy it too.

The second lecture, on energy efficiency in buildings (see part I, part II, and part III), has also been well received.

Many people I talk to are put off my environmentalism because it seems to always be telling them that they should "suffer" for the earth's sake. People don't like that message. One of the refreshing things about Armory's message is that you can do the right thing by the earth and be better off--more from less. That's a refreshing message and accounts for a great deal of the appeal. If you're interested in how good, smart design can result in more comfort with significantly reduced energy footprints, then I think you'll like these lectures as well.

A few notes about listening:

• I'm publishing all the parts of any given lecture at the same time, so if you haven't configured iTunes to download more than one show per day, you'll end up with the last part of the lecture and not the earlier parts. Take a minute now to configure iTunes.
• Also, you're MP3 player or podcatcher will probably order the shows according to the publish date in the RSS feed, so they will show up according to when they were published on SIC, not the order they are on the IT Conversations homepage. For example, the first lecture appeared on SIC in middle of September, so it's quite a ways back. Search down the list in your player and you'll see them.

Tags: itconversations energy conservation environment

This story about using nanotechnology to create wearable batteries puts last weeks story about a man's iPod battery catching his pants on fire in a whole new light.

Tags: technology energy batteries